Mouse 2.0: Multi-touch Meets the Mouse

Summary:

Touchscreen technology has been advancing rapidly as of late. However, it seems as though this technological advancement is only being taken advantage of in mobile and tabletop devices. Because a desktop environment is still preferred for most computing tasks, there needs to be some way to incorporate multi-touch and the desktop. Naturally, the mouse was chosen - enter the multi-touch (MT) mouse. This research covers five different prototypes of MT mice, each of which "presents a different implementation and sensing strategy that leads to varying device affordances and form-factors, and hence very unique interaction experiences":


Each mouse has a very intricate design. Some of them use a system of cameras and infrared illumination, while others use optical sensing grids. Though many attempts have been made to append to the standard mouse, the only successful idea thus far has been the scroll wheel. Since the use of a standard mouse is embedded in most users, the idea for MT mice is to "support multi-touch gestures alongside regular mousing operations." To do this, they conjured the multi-touch cloud. In the picture below, the cloud is shown on the computer screen and the user can click on any portion of the cloud. This is an example of seamlessly integrating multi-touch into standard mouse use.

The group conducting this research did a user study in which six different users were asked to perform Rotate-Scale-Translate (RST) actions with all five MT mice. Of the five, users preferred Arty mouse the most (pictured third above). The only problem with Arty mouse is that it has two points of touch - compared to the Orb mouse (pictured fourth above) which has five potential points of touch. The Orb mouse was also a hit amongst the users, but the feedback suggested that Arty was more natural.

The research concluded that their contribution was a technical one, opening the door to integrating multi-touch into the desktop. They plan to continue work in the area by making better prototypes of their models and testing them again. They will also study the interaction techniques of these new devices.


Comments:

I think that it goes without saying that this is very significant work. It is interesting too! I believe that the technology for multi-touch environments is going to make huge strides in this decade, and this research is pointing us in the right direction. If we can get users more accustomed to integrating multi-touch into everyday computing than we are essentially making the UI layer transparent. After all, we use our fingers and thumbs in an intricate way in everyday activities - why not take advantage of our hand mastery?

I thought that the design of each MT mouse was interesting, but I was more interested in the observations of the user study - I wanted to see how the users interacted with the mice. The paper seemed to be lacking here because it focused more on the concepts and design of each mouse than it did on the observations and results.

It is obvious that there is plenty of room to improve for desktop MT mice. However, I think that an even more interesting area to do continuing work is trying to make multi-touch the norm. How long is it going to take until users are generally able to perform MT tasks with ease? How long is it going to take for widely used applications to incorporate MT interfaces?