Augmenting Reality

by Michael Mullaney on August 3, 2011

Picture this:

In one room, you have 20 exceptionally smart people from all different backgrounds. In the mix you have a structural engineer, a mechanical engineer, and a geologist. Also amongst the crowd are a biologist, a computer scientist, along with a psychologist and an architect.

Let’s say the group is planning something big. Maybe they’re in the final stages of designing a bridge or roadway. Or perhaps they’re planning a huge event like the Olympic Games. In a worst-case scenario, they’re in an emergency command center reacting in real time to a disaster like Hurricane Katrina or the Tohoku Tsunami.

Regardless, you have these 20 people all working from shared information. They’re all using the same data set, but interpreting and processing it from the unique perspectives of their different backgrounds. This can be accomplished a few different ways. Each person could use his or her computer to sift through and explore the data. Or it could be a board room setting, in which the participants take turns controlling the computer that is projected onto the wall. Another option is multiple screens displaying different computers at the same time.

Barb Cutler, a computer science professor here at RPI, is convinced there are better options.One of her research programs is dreaming up and creating new ways for humans to interact with data. This stuff is really fascinating.

In the above Reuters news story about Cutler’s research, you can see a few examples of this: multiple users in a room interacting with the same set of data, at the same time. In the video, look for clips of people using a laser pointer. They’re actually using a standard, off-the-shelf laser pointer to manipulate images on a huge screen.

Cutler asks 12 people armed with laser pointers to assemble a jumbled puzzle.  They do this by focusing the laser pointer on a “piece” of the puzzle, and then dragging it around the screen. It’s not unlike using a mouse to drag an icon around your PC desktop—except there are 12 people doing this at the same time on a huge screen. These demos were conducted in EMPAC, and the system uses off-the-shelf cameras and highly advanced tracking software developed by Cutler and her students.

A puzzle is a simple way to demonstrate the point. Picture more advanced possibilities: like engineers working on the design of an aircraft; or (as mentioned in the video) multiple architects designing a room; or a disaster response team seeking to save lives in an emergency situation. It’s a whole new paradigm for groups of people to interact with digital data.

Look back to The Approach soon for more posts on Cutler’s research. In the meantime, check out this post for more examples of research leveraging the unique capabilities of EMPAC.