Good day, our dear readers!
One of my good friends, Davide C. Cocco, is a very talented composer who recently participated in the Westworld Scoring Competition 2020. For the competition, he created a soundtrack composition for a Westworld scene. No winner announcements yet, but in any case, I wish and hope that Davide will win. Regarding the quality of his work, it is clear that not being a musical expert, I cannot evaluate his work professionally. As an ordinary cinema fan, on the other hand, I find this soundtrack to fit the action scene very well. It does not distract from the events on the screen, and, on the contrary, emphasizes the drama of the movie.
Actually, thanks to the soundtrack, the idea of this post was born. The task of a soundtrack, as far as I know and independently of how brilliant it may be, is just to enhance the effect of the events on the screen. But maybe I’m wrong? Maybe these musical compositions “hide” something else, that the viewers who are normally so focused on the plot do not pay attention to? For example, could it be that soundtracks affect the psychological perception of the movie scenes? In other words, does the musical accompaniment has the auditory Kuleshov effect?
Almost a hundred years ago, the Russian filmmaker Lev Kuleshov (1899–1970) conducted a series of experiments. He filmed a famous (back then) actor Ivan Mosjoukine while he looked on the camera with a neutral expression. Then, he edited four scenes that to include the actor’s face and a bowl of soup, a female in a coffin, a girl playing, or a beautiful woman lying on a couch. Kuleshov showed these short films to his colleagues and asked their opinion about the actor’s mood in every situation. The audience interpreted the facial expression of Mosjoukine differently, depending on the other object presented in the scene. The face was interpreted as hungry when the meal was shown, and as sad in the presence of the coffin. I just want to remind you that the facial expression of Mozhukin was neutral (!) in all these movies.
Only recently researchers began to study the Kuleshov effect in depth. Thus, Barratt et al (2016) performed the original experiment with five emotional stimuli (happiness, sadness, hunger, fear, and desire). The stimuli were presented by first showing the still image of a neutral face, followed by a short video clip ending with the same neutral face. Participants were asked to judge the emotional expression of the faces. The results mostly confirmed original Kuleshov’s outcomes; specifically, participants tended to choose the appropriate category more frequently than the alternative options. Mobbs et al (2006) also confirmed that the emotional contexts “push” an evaluation of the neutral face in accordance. Moreover, they also examined (using fMRI) the brain regions which are activated by each specific emotional impulse.
Now let’s ask the central question of this post: can a background audio-signal, or, more precisely, soundtrack, change the visual perception? The answer is yes!
Bouhuys et al (1995) and Tan et al (2007) demonstrated independently that music can be used as a primer to influence a viewer’s evaluations about a character’s emotions and expected intentions. Eldar et al (2007) presented the neutral film clips containing emotional music and found that the emotional evaluation of the clips had changed in accordance with the music. The further study of Hoeckner et al (2011) showed that empathy-related judgments of film characters may be modulated by music. The protagonist was rated as more likable when accompanied by melodramatic music as compared with tense music.
Finally, the most recent work of Baranowski and Hecht (2017) tried to examine this scientific question in an original pattern: happy, sad, and neutral facial expressions of the actors appeared in a cross-combination with funny, sad or no music. The results are curious: music has influenced the emotional judgments of the experiment’s participants about the expression on the face of the actor even stronger than the visual perception.
Briefly, happy music made neutral faces seem significantly happier, and sad music made neutral faces significant sadder. In the cross-combination examination, they found that happy faces were rated least sad when accompanied by sad music and (surprise!) sadder when accompanied by happy music or no music at all.
Interesting, isn’t it? However, it is still not clear how exactly the Kuleshov effect works. There are two general explanatory models for the Kuleshov effect. First, the visual (or in our case auditory) stimuli of the object shot may induce emotions in the participants, who project their own emotional state onto the actor. Second, a more popular explanation suggests that we set the observed face in the context of the object shot and cognitively adapt our perception. In any case, our emotional perception is the “victim” of our own imagination.
So, a summary for “dummies”: a soundtrack, of course, is a good, relatively quiet and inconspicuous thing, despite that, it can turn our perceptions and evaluation of film’s characters and plots on 180 degrees.
P.S. Can you imagine what will happen to our emotional perception in cinemas when one day, some genius will invent a way to transmit smells and tastes together with visual and auditory signals?
References:
- Prince, S., Hensley, W. E. (1992) The Kuleshov effect: Recreating the classic experiment. Cinema Journal 31: 59–75.
- Mobbs, D., Weiskopf, N., Lau, H., Featherstone, E., Dolan, R., Frith, C. (2006) The Kuleshov Effect: The influence of contextual framing on emotional attributions. Social Cognitive and Affective Neuroscience 1: 95–106.
- Barratt, D., Rédei, A. C., Innes-Ker, A., van de Weijer, J. (2016) Does the Kuleshov effect really exist? Revisiting a classic film experiment on facial expressions and emotional contexts. Perception 45: 847–874.
- Carroll, N. (1996). Toward a theory of point-of-view editing: Communication, emotion, and the movies. In N. Carroll (Ed.), Theorizing the moving image (pp. 125–138). Cambridge, England: Cambridge University Press.
- Bouhuys, A. L., Bloem, G. M., Groothuis, T. G. (1995) Induction of depressed and elated mood by music influences the perception of facial emotional expressions in healthy subjects. Journal of Affective Disorders 33: 215–226.
- Tan, S. L., Spackman, M. P., Bezdek, M. A. (2007) Viewer’s interpretations of film characters’ emotions: Effects of film music before or after a character is shown. Music Perception 25: 135–152.
- Eldar, E., Ganor, O., Admon, R., Bleich, A., Hendler, T. (2007) Feeling the real world: Limbic response to music depends on related content. Cerebral Cortex 17: 2828–2840.
- Hoeckner, B., Wyatt, E. W., Decety, J., Nusbaum, H. (2011) Film music influences how viewers relate to movie characters. Psychology of Aesthetics, Creativity and the Arts 5: 146–153.
- Baranowski A.M., Hecht H. The Auditory Kuleshov Effect: Multisensory Integration in Movie Editing Perception. 2017 May;46(5):624-631.
Featured image by Alfred Derks from Pixabay