creating a highly engaging interactive experience
For EY, partnering with the team at Ogilvy UK, we helped to envision, build, and develop an interactive kiosk experience that reacted to the human foot traffic in Frankfurt Airport.
EY contributes towards building vibrant economies by helping their clients achieve their objectives, advising on the business growth agenda, and supporting entrepreneurship.
As part of this mandate, we made the airport visitors think about the impact machines and AI will have on their futures by using real-time people detection tracking to turn people into machines. It's designed to get people to go to EY and start talking about the future of business transformation and show them as a thought leader in their industry.
As passengers walk past the Frankfurt Brand wall, they see that their reflections have changed into a robot that mimic their actions - ie: lift your left arm and the robot will lift its left arm. It begs the question, “ When machines become workers, what is the human role?”
Our team spent weeks in the London, UK offices brainstorming, modelling and prototyping CG robots and interactive movements with the client. Our Vancouver CG team supported the illusion of the mirrored environment by modelling and carefully lighting/crafting a complete replica of the interior of the airport lobby from photographs and existing footage. Building with Unreal Engine and C++ to develop this software was a unique challenge and we were able to create a fun interactive experience that brought the robots to life. We used Unreal Studio to transfer data from 3DS Max to Unreal Engine. The environment was heavily optimized in order to achieve a steady 60 fps at 4k resolution. The Kinect sensor tracked skeletal data to allow the robot avatar to pick up the joint movements and even size of the original person. As such, we displayed families of parent and children robots to really get the impact across that they user had been transformed. Each robot was subtly different in texture and colour variation to underscore uniqueness. Background and foreground robots were tracked to execute animated walk cycles, that best matched their speed, direction, and apparent intent. As the user approached the kiosk, there was a natural transition between environment animated robots and tracked interactions. The kiosk was a hit with children and adults alike and even prompted a soccer team and a music group to perform some antics in the virtual mirror.
We worked from a prototype wall of about 8 ft tall by about 20 ft wide to perfect the interactions. The existing wall for digital signage in Frankfurt was utilized for the final placement. A Kinect camera above the kiosk screen catches the motion of the people walking past to track their movements and turn them into robots.
This experience has resonated very well with everybody who has played with it and they are already talking about expanding it beyond Frankfurt and Amsterdam and bringing it over to the North American market. We are pleased to successfully launch this high-profile work with the folks from Ogilvy again.