Jiwon Kim, Steven M. Seitz (University of Washington) and Maneesh Agrawala (Microsoft Research) are working on the Video-Based Document Tracking project to integrate the paper world with the world of electronic data.

The prototypes use a computer and overhead camera to track physical documents on a desk and automatically link them to appropriate electronic documents. The prototypes built can track paper documents and sort photos without the use of special tags, paper or marks.

With the system, users can pinpoint the location of a given document within a stack of papers on their desktop, searching by keywords, document appearance, or by how recently a paper was moved.

The researchers’ system infers the structure of a stack of papers from video images. A user moves paper X from stack A to stack B, then moves paper Y from stack A to stack C, for example. The system parses the video into a pair of individual movements and then interprets each event to determine how the documents were reorganized.

Even better, users can also browse desktops in remote locations by clicking and dragging on an image of the remote desk.

Very bad news: the system could be ready for practical use in three to four years.

Video.

Via Technology Review.