Table 1 Types of interactions when gesture is used with speech: Gesture-Speech Cooperative Interaction
User | Type of interaction | Example of gesture | Example of speech |
|---|---|---|---|
Both | Direction guidance | Showing with arm | “This way” |
Both | Direct partner’s attention | Pointing with finger | “Look at that” |
Both | Establishing visual annotation | Counting fingers | “One, two, three…” |
Indoor | Instructions related to an object | Pointing with finger (Pointing Cue used) | “Pick up this cup” |
Indoor | Instructions related to adjusting the objects | Showing the palm and turning it over | “Turn to the other side” |
Indoor | Instructions related to the size | Showing approximate size with two hands | “It’s this wide” |
Indoor | Confirmation to walking user | Thumb up/ring gesture | “Yes, it’s good” |
Walking | Query to indoor user | Pick up the object | “Is this correct?” |
Walking | Instructions related to an object | Holding a object and turn it to different sides | “Look at this side” |