Saturday, August 15, 2009

Reading facial expression

A colleague in the Intercultural Insights group has just drawn our collective attention to an article on the BBC website entitled, Facial expressions 'not global'.

For this kind of scientific reporting, I always feel the need to look for missing significant variables, whose absence could have an impact on the general conclusions put forward by the researchers. It's important to remember as well that these behavioural/psychological studies themselves conform to an array of cultural patterns and rules related to how research is funded, conducted and its results communicated in the West, including the role of the media in selecting and publicising the "conclusions".

It isn't just to be captious that we need to look for missing parameters. If our aim is truly scientific, we must assume that the failure to take any vital parameter into account can seriously influence the interpretation of the results.

So concerning this study, as it is reported in the BBC article, I propose two major observations:
  1. Real human emotions are never expressed as static poses... except in certain conventional iconographic and theatrical traditions, which vary from culture to culture! Emotion always derives from a context implying the presence of a number of dynamic elements in the expression of emotion, as well as a certain synaesthesia, or association of simultaneously processed sense perceptions (sound, movement, even smell as well as awareness of pbysical tension and what I would call "dramatic structure" or transitional logic in moving from one affective state to another, to say nothing about the phenomena related to unconscious synchronisation*).
  2. Every culture has developed, through its artistic and representational traditions (including advertising), an iconography of the static expression of human emotions. These traditions, which appeal to formal narrative including poetry, drama and religious and moral allegory - have a powerful influence on our perception of new images but - I would maintain - far less on our reactions to real communication situations.
In short I think it is an error to draw any cultural conclusions from this type of experiment other than to observe that, in this type of artificial interpretative exercise, East Asians are more likely to look for clues in the eyes and Westerners in the mouth. There may even be a link to the phonetic characteristics of the languages as well as to the strategies related to saving or losing face and respecting a principle of harmony by refraining from openly expressing one's emotion (meaning that the eyes may be the only reliable, though still ambiguous, guideline to interpreting emotion, as experts in lie detecting tell us!).

For all these reasons I think it would be an error to use this experimentation to draw conclusions about how people of different cultures read emotions in real situations. A simpler, more coherent and probably more honest conclusion would be to point out how this experiment seems to validate at a more trivial level Richard Nesbitt's research that led him to conclude that Asians are more dependent on context than Westerners to identify "meaning". Take away context - as photographs do - and the results are bound to be different.
An example of the type of photographs used in the study

The exercise used in the experiments is closer to the act - privately individual - of reading a book than engaging interactively in dialogue with another person, and yet the researchers are suggesting it tells us something about how people react to human dialogue. The proposed conclusions about eyes and mouth may be no more meaningful than to say that Arabs have a tendency to focus on the right side of the page when reading a book, whereas Europeans tend to focus on the left side... and then to conclude by creative extrapolation that one or the other culture privileges the right of left hemisphere of the brain!

As for the emoticons, which were initially a form of wit as practised by geeks on the Internet in the previous millennium before the advent of the multimedia Web, the principal variable I should expect to find should be sought for in the contrast between populations that use alphabets and those that use characters, which are already drawings. I find it curious that the researchers didn't seem to consider that influence. After all, if you turn a word composed of letters of the alphabet on its side, people still recognize it, but if you turn an ideogram on its side (given the indeterminate number of ideograms as compared to a strict limit of 24 to 26 letters), people are likely to seek a different meaning or simply fail to recognize the ideogram (or logogram). Another factor worth considering might be that script itself can be presented in vertical columns (the traditional way) or horizontally.

I would therefore propose changing the tagline of the article from "A new study suggests that people from different cultures read facial expressions differently" to "A new study suggests that people from different cultures use different strategies to classify emotions purportedly represented in photographs of facial expressions". Not very exciting, but certainly true.

I should add that I've personally done a lot of work on capturing and representing emotions through still images taken from video in the context of my work on multimedia resources for language learning. Because my concern was to use such resources to sensitise learners to the semantic component of affect and to help them discover and appreciate phonetic variations (rhythm, intonation, intensity, tension, etc.) in their relation to the expression affect, I can witness to a simple fact: that the exercise of capturing and representing unambiguously any emotion in a photograph is a perilous enterprise! Facial expression alone is always ambiguous, even in so-called "direct" cultures where the norm is to signify verbally and non-verbally what you think, "harmony be damned". In the course of my multimedia work I have organised and exploited photography shoots with professional actors to get them to express specific emotions and attitudes. The result on the page is never wholly satisfying in terms of representation... unless it is specifically iconographic (e.g. imitating the poses of sorrow derived ultimately from Renaissance depictions of the Crucifixion). But so long as one accepts ambiguity as a structural principle (and that is a key but much neglected point in traditional language teaching), the pedagogical result can be satisfying... precisely because the objective is to augment the sensitivity of the learner to the effect of context and synaesthesia.

After reading articles like this one, I'm invariably left with the impression that a lot of popular science just doesn't do nuance... or if it does, the media won't bother with it!

* In “The Dance of Life: The Other Dimension of Time” Edward T. Hall maintians that “people are tied together and yet isolated by hidden threads of rhythm and walls of time.” This is not only true of dialogue, which seems fairly straightforward, but also of much more complex interactions, as the following passage illustrates:

Rhythm is basic to synchrony. This principle is illustrated by a film of children on a playground. Who would think that widely scattered groups of children in a school playground could be in sync. Yet this is precisely the case. One of my students selected as a project an exercise in what can be learned from film. Hiding in an abandoned automobile, which he used as a blind, he filmed children in an adjacent school yard during recess. As he viewed the film, his first impression was the obvious one: a film of children playing in different parts of the school playground. Then — watching the film several times at different speeds, he began to notice one very active little girl who seemed to stand out from the rest. She was all over the place. Concentrating on the girl, my student noticed that whenever she was near a cluster of children the members of that group were in sync not only with each other but with her. Many viewings later, he realized that this girl, with her skipping and dancing and twirling, was actually orchestrating movements of the entire playground! There was something about the pattern of movement which translated into a beat — like a silent movie of people dancing. Furthermore, the beat of this playground was familiar! There was a rhythm he had encountered before. He went to a friend who was a rock music aficionado, and the two of them began to search for the beat. It wasn’t long until the friend reached out to a nearby shelf, took down a cassette and slipped it into a tape deck. That was it! It took a while to synchronize the beginning of the film with the recording — a piece of contemporary rock music — but once started, the entire three and a half minutes of the film clip stayed in sync with the taped music! Not a beat or a frame of the film was out of sync!

...When he showed his film to our seminar, however, even though his explanation of what he had done was perfectly lucid, the members of the seminar had difficulty understanding what had actually happened. One school superintendent spoke of the children as “dancing to the music”; another wanted to know if the children were “humming the tune.” They were voicing the commonly held belief that music is something that is “made up” by a composer, who then passes on “his creation” to others, who, in turn, diffuse it to the larger society. The children were moving, but as with the symphony orchestra, some participants’ parts were at times silent. Eventually all participated and all stayed in sync, but the music was in them. They brought it with them to the playground as a part of shared culture. They had been doing that sort of thing all their lives, beginning with the time they synchronized their movements to their mother’s voice even before they were born.

No comments: