Gabriel Robles-De-La-Torre, PhD   Meet Dr. Robles-De-La-Torre in New York next April 14th, 2007. He will speak about this work at the New York Academy of Sciences   This ongoing research has been featured at: The Economist MIT Technology Review  (read here the German version) Nature The Washington Post Scientific American France (Pour la Science) Introduction Touch and related capabilities are commonly underrated. Yet, these capabilities are essential for normal human functioning, as demonstrated by the catastrophic consequences that follow after losing them (Robles-De-La-Torre 2006). Also, we perhaps tend to think that touch is somehow more veridical than the other senses. Perhaps this is because we  know that there are many visual illusions, but we rarely hear about touch illusions.  As we will see, there are compelling touch illusions of shape. In such illusions, persons touch objects that have a certain shape, but they perceive touching a  radically different shape (Figure 1).  This document will  help you understand how such illusions are created. In particular, a simple demo is provided in Section 4 below to give you an idea of how some illusory shapes feel like. I would recommend reading the document first and then watch the demo.  Alternatively, you can watch the demo after reading about impossible, paradoxical haptic objects in section 3.2 below. Keep in mind, however, that the demo is only a very rough approximation.

Figure 1. A person explores  a paradoxical object that combines the geometry of a real hole (gray bar) with an illusory bump (red dotted line) created through computer-controlled forces alone. These forces are generated by a haptic interface (blue machine). Surprisingly, when exploring this paradoxical object, persons perceive that they touch a bumpy object, and not an object with a hole.  See the text for details

Your feedback is always welcome at:

Table of Contents

1. How do we haptically perceive the shape of objects?

2. Haptic perception of shape: force or geometry?

3. Using virtual objects to understand haptic perception of shape

3.1 The normal, real-world scenario: forces and geometrical information vary together in a natural way
3.2 The paradoxical object scenario: forces and geometry vary in an "impossible" manner.
3.3 The key finding: force information can overcome geometrical information to determine haptic perception of shape

4. Experience how illusory objects feel like.

5. Future work

6. Applying haptic perception research:  illusory haptic objects for perception-based rendering of sharp objects
 

1. How do we haptically perceive the shape of objects?

I investigate how humans perceive the shape of objects when actively exploring them through touch (which is commonly called active or haptic touch). I use computer haptics to create virtual haptic objects for this research (see a demo of computer haptics and virtual objects here).  When a person haptically explores an object, he/she has access to several sources of information ("cues") about its shape.  A major shape cue is the geometry of the object. Consider the following case. When we slide a fingertip along the surface of an object, the geometry of the surface determines the way our finger will move. For example, if the object has a bump on it, our finger will ascend into the bump and then descend from it. 

Until very recently, it was assumed that we used such geometrical information to perceive the shape of an object. However, there are other sources of shape information that are also experienced when touching an object. These are the forces that we experience when exploring the object.  For example, when sliding the fingertip along a surface with a small bump, the bump will resist the movement of the fingertip.  Such resistance forces are largely determined by the local geometry of an object and by how much force we apply when exploring the object. 

2. Haptic perception of shape: force or geometry?

Note how force and geometrical information are experienced together  when exploring objects. Because of this, we cannot be sure that object geometry is the information that is used for haptic shape perception.  In other words, what is the relative contribution of geometry and force to haptic shape perception? When considering two extreme cases, is it enough to experience geometry, or is enough to experience force? How do these related sources of haptic shape interact with each other? (Robles-De-La-Torre & Hayward, 2000) . In the experiments discussed here, only touch information about objects was provided. Visual and auditory information were not provided.
 

3. Using virtual objects to understand haptic perception of shape
 

In collaboration with Vincent Hayward, we used haptic virtual shapes to tackle these questions (Robles-De-La-Torre & Hayward, 2000, 2001). Our virtual shapes are force fields that are generated by using a haptic interface (the blue machine in Figure 1, the figure is shown here again for your convenience). The interface we used (PenCAT/Pro®) is a robot that generates computer-controlled forces to create virtual objects. The interface has a small plate that can be held by a person's fingertip (Fig. 1).  The plate is attached to a wheeled tool that rests on top of a hard plastic object (the gray bar in Fig. 1, just ignore the red dotted line for the time being).  The tool is mechanically constrained, so it is always in a vertical position relative to the surface of the object.  The person can freely explore the plastic object by rolling the wheeled tool on top of it, as shown in Fig. 1. Note that the person's fingertip moves downwards when entering the plastic object's hole, and upwards when exiting it.  3.1 The normal, real-world scenario: forces and geometrical information vary together in a natural way  When the haptic interface is turned off, the person only experiences the forces that arise naturally from the tool-surface interaction. For example, when exiting the hole (Fig. 1), the person experiences some resistance forces. These forces depend on the of the hole's slope, and also on how hard the person pushes down to hold the tool. So far, there is nothing extraordinary going on here: the person experiences forces and geometrical information that vary together in a natural way.

Figure 1. This figure and its legend are shown again here for your convenience. A person explores  a paradoxical object that combines the geometry of a real hole (gray bar) with an illusory bump (red dotted line) created through computer-controlled forces alone. These forces are generated by a haptic interface (blue machine). Surprisingly, when exploring this paradoxical object, persons perceive that they touch a bumpy object, and not an object with a hole.

3.2 The paradoxical object scenario: forces and geometry vary in an "impossible" manner.

When the haptic interface is powered on, it generates computer-controlled forces. Such forces are added to the forces that arise naturally from the normal tool-surface interaction. In this manner, it is feasible to create normally impossible, paradoxical objects in which geometrical information conflicts with force information. Figure 1 presents one of these impossible objects. Here, a person is exploring and object with a real, physical hole object (Figure 1, gray bar). Normally, the person would simultaneously experience forces that are related to the geometry of the hole, as in the normal case described before. However,  in this impossible object, the haptic interface modifies such forces, so that the person experiences forces that are normally associated with an object that has a bump on it (Figure 1, red dotted line).  Here the bump is purely virtual: it is created with forces, and has no geometrical information of its own. That is, when exploring this object, the person's fingertip still follows the hole trajectory given by the plastic surface (gray bar in Figure 1). The end result is that the person experiences the geometrical information of a hole, together with the force information of a bump.

3.3 The key finding: force information can overcome geometrical information to determine haptic perception of shape

Which information is used by a person in such a situation to perceive the shape of the object?  We found (Robles-De-La-Torre & Hayward, 2001) that, surprisingly, people's perception depended on the forces they experienced, and not on the geometrical information they simultaneously received. That is, in the case shown in Figure 1, subjects typically perceived a surface with a haptic bump (created through forces alone), and not a surface with a haptic hole, even though there was a real physical hole present. This can be considered as a touch illusion, in which a conflict between sources of information is resolved in favor of force cues, perhaps by weighing each source of information differently (Robles-De-La-Torre & Hayward, 2001). It can be also considered that  the virtual object masked or  perceptually hid the real object.

4. Experience how illusory objects feel like.

We tested other paradoxical objects, too (Robles-De-La-Torre & Hayward, 2001). For example, if you replace the plastic object (Fig. 1, gray bar) with a totally flat object, and the haptic interface produces the forces of a hole (or bump), you will still experience exploring an object with a hole (or bump, depending on the case), even though your fingertip will move along a totally flat surface. Here you can find a simple demo to approximately experience how these virtual, illusory objects feel like.

5. Future work 

These results indicate that force can overcome geometrical information in haptic shape perception. Again, how do geometric and force information interact with each other? What is the relative  contribution of each one to haptic shape perception? I am currently exploring these and other, related questions.

6. Applying haptic perception research:  illusory haptic objects for perception-based rendering of sharp objects

The important role of  force in haptic shape perception provides the scientific basis for simple haptic rendering algorithms to generate illusory, force-based haptic shapes (Robles-De-La-Torre & Hayward, 2000). 

In this regard, as part of an ongoing collaboration with Carlo Alberto Avizzano , Otniel Portillo-Rodríguez  and Massimo Bergamasco (PERCRO, Italy), we used the GRAB haptic interface to apply illusory objects in the solution of a difficult problem in haptic rendering: how to produce realistic virtual objects with features such as sharp edges (Portillo-Rodríguez, Avizzano, Bergamasco, Robles-De-La-Torre 2006; Graham-Rowe. The Cutting Edge of Haptics, MIT Technology Review 2006). This is a difficult problem for several reasons. In particular, sudden force changes are common when using other rendering approaches to generate virtual objects with sharp edges. This is due to the spatial discontinuity at a sharp edge: forces vary greatly in their direction from one side of the edge to the other. As a result, an user's hand tends to be unnaturally pushed away from the virtual object when touching the vicinity of an edge. This decreases the realism of the virtual object.

Figure 2. Using haptic illusions to render challenging virtual objects with sharp edges. An user wears a thimble-like tool  (in blue, both panels) to explore a virtual object (gray surface, both panels). The tool is connected to a haptic interface (not shown). Although the user's hand actually travels along a smooth trajectory  (a), the user perceives touching a surface with a sharp edge, which is illusory (b). The illusory edge is created through lateral forces. Unlike the case depicted in Fig. 1, here the virtual object does not involve a real object such as the gray plastic bar in Figure 1. I thank Lorena Robles-De-La-Torre for designing this figure.

 Such undesirable artifacts can be avoided by rendering illusory, sharp edges instead (Portillo-Rodríguez, Avizzano, Bergamasco, Robles-De-La-Torre 2006; Graham-Rowe. The Cutting Edge of Haptics, MIT Technology Review 2006).  In this approach, when exploring  a virtual object, an user's hand travels along a smooth trajectory without geometrical sharp edges (Fig. 2a), and lateral forces are used to render illusory sharp edges (Fig. 2b). Our experiments indicate that this approach allows for a more natural rendering of such challenging objects. This is an example of perception-based haptic rendering, in which the characteristics of human haptic perception allow to overcome limitations in the engineering of haptic technology.

More generally, this research illustrates the many opportunities for combining basic and applied research in this exciting, rapidly developing field.