While I am hoping to soon start blogging about responses to each stimulus used in the solo response project, it will take a bit longer to get all the signals tidied and a format of analysis set.  In the interim, here is a simple version of the mini talk I presented at the most recent NEMCOG meeting with links to audio for each example.

******

Variability of Emotional Valence: Inconsistencies in self-report continuous emotion ratings – Finn Upham, New York University

Listeners often report feeling emotions in response to music, whether happy or sad. Empirical work on continuous reports of felt emotion are however often challenged by substantial variation in emotional dynamics reported by different participants. Variability in responses is supposed to be caused in part by differences in individual listeners’ musical expertise, sensitivity and cultural background. Working with multiple responses from a single human subject should then make it easier to explore which other factors contribute to the variation in felt emotions reported during music listening.

This summer, I collect continuous responses from myself (see the solo response project post). The analysis to follow uses the two dimensional felt emotion ratings (Arousal X Valence) as well as information from notes collected during the listening sessions.

Rating consistency: Valence vs Arousal

Arousal X Valence ratings of emotion from different listeners to the same music tend to show greater consistentcy in the Arousal dimension than in that of emotional Valence. For these ratings felt emotion collected from a single participant, the same pattern is seen. Valence is generally more variable as measured in terms of both rating change coordination (Coordination scores (link to technique on wiki) A V , p < 0.05) and in absolute rating values reported relative mean response dispersion (link) A: V: , p<0.001).

Why is valence more variable? The following are a few examples of factors which resulted in my felt emotional response to these pieces of music changing from one listening to the next.

Multiple contradictory cues

Amongst the twenty five stimuli, there were multiple pieces which were very complex, with multiple lines in the music moving independently. Each listening was different as my attention was caught by different aspects of the music/performance. But different attention patterns need not lead to different empressions of the emotional experession (and resulting empathy response) unless the distinct lines expressed cues that lead in different directions.

Consider the Sigur Ros piece, Varud (also known as Varuo). above is a plot summarising the continuous felt emotion ratings I reported during the many listening to the piece. First, let me explain the graph. The two upper graphs summarise the Arousal ratings from my 24 listenings, the two below the Valence dimension of felt emotion.  In each pair, upper graph reports the activity profile for changes of at least 0.02 (yellow) and at least 0.05 (green) in the increasing (above zero) and decreasing (below zero) directions, while the lower reports the individual responses with the average response time series in black overlayed. The coordination scores and relative mean dispersion in range units are in the titles of each pair. The x axis in all graphs is time in seconds; the response recordings began 10 seconds before the stimulus began to play.

In the ratings of valence ratings shown in the bottom graph, there are a number of moments where some valence ratings bubble above the middle into positive valence while other ratings bubble below to negative valence (85s, 175s, 335s). This kind of agreement in degree of valence and valence change, with disagreement in orientation is not uncommon, even in collections of responses from multiple listeners.

The song can be heard with video. Add six seconds to timing cited above to sync with the video, as  it begins with a short intro sequence. A good example of valence splitting can be heard at 335s. The music is directly follows a climax long in growth and duration of peak (as can be seen in the arousal ratings). The fading of the heavier layers, the emergence of the children’s choir, and the never-resolving quality of the musical lines can each be interpreted differently in terms of emotional valence, but instead of the results being neutral (like the climax for most of my listenings, this moment moved towards the poles. The multiplicity of experiences made this piece of music both challenging and rewarding as a listener.

Empathy vs Aesthetics

Felt emotions in response to music are not as simple as a reflection of what is perceived to be expressed. In watching a dramatic performance, or even observing other humans in every day life, how we feel about what happens before us depends on plenty more factors, like our relationship to the actors, whether or not we can easily identify with them, the implication of the expression or interaction for our own interests, and so on. In the context of listening to recorded music, a big component of this other stuff is aesthetic judgement: how we enjoy what is presented in terms of its technical composition, the virtuosity of the performance, the creativity of the creator, etc.  These positive or negative feelings about the music in the moment then interact with whatever feels are experienced in reflect of the music. In listening to the same pieces every day, I found that the amount the aesthetics mattered changed between listenings, and this sometimes (but not always!) resulted in music which could be perceived as neutral or negatively valenced as giving me positive feelings.

Figure 2 shows my rating responses to the IDM/Glitch-hop piece 1685/Bach by Nosaj Thing . For those unfamiliar with this kind of electronica, it is strongly rhythmic, danceable, with a heaviness like dub, but is also enjoyed with more cerebral interest. Around 56s to after 60s is an interval when my responses were torn by different factors of valence. The expression of the music sometimes felt menacing, sometimes neutral, but my response was also influenced by the fact that I really liked the way the metallic shuffle sound is manipulated, yeilding more than a few of those “oh, yeah” smiling and head nodding moments.

When does the empathic response win out over the aesthetic liking or vis versa? The balance seemed to shift around as I learned the stimuli under such focused attention, but external factors likely also contributed.

Changing perspective

One of the pieces which provoked regularly strong emotional responses was the song Basket by Dan Mangan. This is a song I knew and liked before starting the experiment, despite the fact that it regularly makes me cry. The song tells of losing oneself to old age, moving from melancholy to anger at frailty to an urgent need for action to confusion and sorrow.

Though my responses were very consistent over much of the piece, there is notable disagreement in my responses around the climax of the songs, and this has to do with how I related to the story it tells. The lyrics are in the first person, sung by a male voice, and initially I listened to the story like a younger relative. The valence of my experience started as neutral and then moved lower as the song progressed, hitting the bottom edge around 160s and staying there to the end. But after a few sessions I began to relate more directly with the narrator/singer and felt a relief from sadness with the determination to act in lines like “but if i can make it to the street, i’ll steal a car, or a bike, or whatever there is to steal” (220s). By sharing the view of the singer, these moments felt like high arousal without any strong direction for valence. Feeling inspired to act is a state which does not fit neatly on the Arousal x Valence grid.

Factors of Valence

From the examples above, and many more not included, it appears that my continuous ratings of felt emotion were a composite of Empathy and Aesthetic judgement. Some of the variability in valence ratings reported can be explained with changes in interpretation of emotional expression, distinct sequences of attention to different voices/cues, and shifts in the balance between empathy and aesthetics. Which factors belong to the music and which to the listener?