Using facial expression analysis during a musical experiment
The influence of music on our emotions intrigues me, and raises many questions. For example, how is it possible that my emotion changes as music speeds up or slows down? Why does the song ‘Happy’ by Pharrell Williams make me smile, and why do I react in the opposite manner to the song ‘Tears in Heaven’ by Eric Clapton?
Is it really the music itself that triggers these emotions, or is it the memory of a particular event that fits with the music or any other association? Why do I prefer to listen to fast songs at one time and to slow songs at another time? Can anyone else around me read my face and figure out what emotion the music that I’m listening to evokes? Does everyone react the same way? And if they don’t, why not?
The link between music and emotion
Many processes that take place in our brains determine how we react to music. Scientist have figured out that music affects people, but to what extent varies per person. For example, the volume of the music has the most influence on emotions. After that, the pace of the music follows. The melody and the instruments play a role as well.
Time to do a small experiment
I decided to do a small experiment using FaceReader Online to measure facial expressions while my participants watched a music video on their own computer or laptop. I chose three different music fragments:
- Drummer boy Gordo performing his act on the street, using plastic baskets
- A rendition of Beethoven’s “Ode to Joy” by Beaker, Dr. Bunsen Honeydew's loyal assistant from the Muppets
- A sad and emotional piano song, purely instrumental
FaceReader Online has an advantage in that you can invite people from all over the world to join your study. Sending a link via an email and starting the experiment was fast and simple. Several people responded, and the recordings simply appeared one by one on my dashboard.
Did the Muppets make people smile?
After easily grouping my participants, the next step was to look at the results. Within FaceReader Online, it is possible to make a quick comparison of your results. For example, you can compare the media files, the participant groups (male/female or based on age or country e.g.), or certain episodes within the media files that you have selected beforehand.
Expression comparison of all participants
Expression Intensity of the males
The results mostly confirmed my expectations. The Muppets fragment indeed resulted in the happiest faces. The drummer boy and the piano man mainly triggered neutral facial expressions.
What surprised me was that the piano song showed little sad emotions overall. Maybe in this case, the distinction between a neutral feeling and a sad feeling was not large enough. When comparing males and females, two fragments showed some differences. Males responded with a much higher level of excitement (arousal) to the piano song, and were happier when listening to the drummer boy.
The ease of use of FaceReader Online
FaceReader Online offers an interactive web page allowing you to choose how you want to visualize your results. It is quick, easy, and straightforward to use. For a more detailed analysis, all results can be exported to the 'offline' FaceReader environment. This allows you to review all participant recordings and gives you access to a larger number of analysis tools and visualization options.
As a result of this experiment, I discovered that music and emotions are complex. But, on the other hand, FaceReader Online definitely is not. In fact, it is just the opposite; it is easy to work with, and the online manual is very useful. In my opinion, the convenience with which you can let people from all over the world participate in your research is its greatest advantage.
References
Get the latest blog posts delivered to your inbox - every 15th of the month
more
Emotional responses to infant crying
Hearing an infant cry can cause negative emotions, which can impact the way we respond. Researchers Riem and Karreman instructed parents to apply specific emotion regulation strategies in response to infant crying.
Predicting Advertising Effectiveness: Facial Coding of 120.000 Video Frames
The advertising and marketing companies have just received a new addition to their repertoire of the neuromarketing tools – automated coding of facial expressions of basic emotions.