Head Tracking

    Roles and Responsibilites:
  • • Complete Application Development

This is an experiment on head tracking to provide a sense of depth to a flat display. This is inspired by Johnny Lee Chung's Wii remote hack head tracking example. A major drawback of Johhny Lee's setup was that the Wii sensor bar with IR LEDs should always face the remote. Any major tilt of user's head will be mistaken as movement in Z depth.

As my final year college project in college, I improved Johnny Lee's project by using a stereo IR webcam setup and only a single IR LED for user tracking. Using Zitnick-Kanade algorithm for stereo tracking, I could get the position of the IR LED. This also allowed me to use the IR LED mounted to a pen's tip to work as 3D pointer which can be used in CAD modelling by drawing in air.

As soon as I worked with Kinect at Digital Jalebi, I instantly saw the possibility of improving my college project. I used Kinect camera to track a person's head position using skeleton data so the user does not have to wear anything. It changes the 3D projection matrix according to user's position to create the view from their point of view, hence will work for only one person at a time, but that problem was present in Johnny Lee's project as well.

Since this application tracks head position, to take this video, the camera had to be placed very close to my eyes while moving around. The effect is not as good as seen in video due to stereoscopic nature of our view. Hence for an upcoming museum installation, I'm trying to use active shutter 3D viewing glasses to provide stereoscopic head tracking.