In the Visual Code Recognition for Camera-Equipped Mobile Phones project, Michael Rohs and Beat Gfeller at the Federal Institute of Technology in Zurich explores the possibility to use the cameras built into mobile phones to act as sensors for 2-D visual codes. The codes can be attached to physical objects and act as a key to access object-related information and functionality.
Some interaction possibilities of the visual code system can be obtained by using the position of the camera focus, the rotation of the code in the image, and the amount of tilting of the code plane relative to the image plane as additional input parameters.
Possible applications include:
With visual codes, areas in printed documents can be linked to online content. For example, a cinema magazine, in which physical hyperlinks connect words and images via camera-phones to movie trailers, information about the actors, and the local cinema’s booking system.
In newspapers, the codes could store or rate newspaper articles, provide background information related to advertisements, or updates on information like stock quotes and weather forecasts. If physical hyperlinks are associated with points and regions on the page, the camera-phone can follow the hyperlink, fetch the additional information and display them. In the video (high or low-quality), the camera-phone retrieves the current snow conditions for various regions of Switzerland by moving the camera focus across the corresponding table entries on a newspaper page. Since the rotation of the code in the image is provided as an additional parameter, multiple aspects can be associated with a single point or region. For example, when the camera is rotated by a certain amount, the display changes to indicate the current temperature instead of the snow depths.
A PongSymbian version of the classic Pong game can be camera-controlled. The green ball bounces back from the walls. The red slider is controlled by the orientation of the phone relative to the background: moving the phone to the right moves the slider to the right – and vice versa. Video shows how this might be used for interaction with large-scale displays.
In another example, the team used the camera-phone to control a tram map of Zurich. Moving the phone updates the part of the map which is displayed.