What is the relationship between gesture and dialogue? Paul Boussac and Jack Sidnell of the Toronto Semiotic Society organized a two day symposium titled, Semiotics And Pragmatics of Gesture, Conversation and Dialogue. I gave a talk with the pretentious (but actually relevant) title, Dialogues of the Dead: Reanimated Interaction in Computer Games in which I tried to show that the same concerns found in dialogue theory from Plato on around the way we animate disreputable characters are feeding the public anxieties around computer games. It is Dr. Frankenstein’s problem all over, if we animate the dead we need to take responsibility for them. I was trying to find a way to engage in an ethical discussion of interaction which combines gesture and conversation.
Charles Goodwin of Applied Linguistics at UCLA gave us a very interesting tour through research he is doing on the interplay of gesture and conversation. He showed video of subjects (including his own father) using gestures to modify the utterance of another speaker (looking away in mid utterance) or supplementing their utterance. I left completely convinced that a simple Shannon messaging model doesn’t work. While the message is being encoded and delivered the speaker is getting feedback that changes it to the point that there is no single speaker, but negotiated meaning.
Deb Roy for the MIT Media Lab showed how he has been training robots to interact with people in simple exchanges that involve gestures. Some of this is based on hypotheses of how children learn language. He has a very interesting project called The Human Speechome Project where they recording video and audio in every room of his house for the first three years of his new child’s life in order to get a fairly complete archive of the language (and gestures) that a child experiences in the early years of language formation. The project has all sorts of interesting ethical and media ramifications.