IoT x MR is a proof of concept application developed at Holohacks in San Francisco conducted by Microsoft Hololens Team. The primary goal of the app is to provide a 3D spatial UI for cross-platform devices (Android Music Player app and Arduino controlled Fan and Light) and to interact with them using gaze and gesture control. The app has a unique use case which portrays devices not having a direct correlation in real-world can have one in the Mixed Reality. The application also incorporates a Zen mode where the user can have an engaging mixed reality experience, interacting with virtual objects fused in a real world environment.
Technical Details
+ Application was built in Unity and deployed on to Hololens using Visual Studio.
+ Application uses the Universal Windows Platform (UWP) networking packages.
+ Application uses the inbuilt gaze and gesture control on Hololens.
+ Hololens interactivity scripts programmed in C#.
+ Android Music Player app and Arduino controlled fan & light implement a TCP server in Java and Arduino sketch respectively along with desired functionalities.
+ All devices were connected to a single access point.
+ The app makes use of Hololens coordinate system to place 3D spatial UI and the Zen portal.
+ On any UI interaction, the app sends an appropriate TCP request to the concerned server.
+ The Zen mode’s state is toggled based on the user’s position relative to the portal.
Swaroop Pal 