ICMPC15 – Reliable Psychophysiological Changes to Music listening

Posted on by finnupham

A talk and a poster discussed a number psychophysiological responses to music, specifically events measurable over repeated listenings in skin conductance, respiration rate, heart rate, and zygomaticus and corrugator contractions. The talk explains how Activity Analysis can be applied to these responses in order to identify reliable reactions to musical stimuli, and the poster shares some of the consistencies and inconsistencies we can find in different listeners’ responses through these signals.

Demo: Activity Analysis on Psychophysiological Measures of Responses to Music

Download the slides (pdf)

Abstract

Continuous measurements of responses are particularly useful for music cognition as our experience develops during the presentation of this powerful stimulus. And yet, responses can vary substantially, and psychophysiological measurements are particularly noisy. Multiple studies have identified changes in psychophysiological states with the presentation of music, but identifying when changes are triggered is not a simple task, even with rapid changing signals like skin conductance, heart rate, and respiration.

Activity Analysis is a new analysis paradigm developed specifically for music research that focuses on response events and their co-occurrence across multiple listenings to the same stimulus, whether by different listeners to a live performance or repeated listenings by a single participant to recorded music. This approach accommodates the extraneous information in continuous measurements of response and leads to new results from these complex signals, including statistical assessment of coherence between responses at specific moments in music. It supports response-led exploration of the stimuli as well as addressing questions of whether and how individual pieces are coordinating the experiences of listeners.

Aims

To demonstrate the application of Activity Analysis with the MatLab toolbox on collections of psychophysiological responses from repeated response experiments.

Main Content

This demo will introduce Activity Analysis, demonstrate the visualisation capabilities of this approach to continuous responses, and apply tests of coordination to skin conductance, heart rate, and respiration belt measurements from audience response collections and repeated response collections. Particular attention will be paid to the local coordination test, which identifies when responses are in significant alignment with the music. All to be discussed is the process of determining appropriate parameters for coordination testing of response events such as orienting responses in skin conductance, along with the implications of a lack of measurable coordination in response activity.

Implications for practice

Activity Analysis may be very useful for the study of responses to music, allowing researchers to pin point when changes in responses occur and whether the timing of changes might be expected to replicate. With the MatLab toolbox, these techniques can be readily applied to existing data sets as well as future experiments.

Value for this conference

Activity Analysis can be applied to a number experiments reported at ICMPC and conducted in labs associated with many related organizations. The relationship between music and listeners bodily responses is a long-standing but still growing area of research and tools for the exploration of experimental data are needed along side methods for testing specific hypotheses.

References

Upham, F., & McAdams, S. (2018). Activity analysis and coordination in continuous responses to music.Music Perception: An Interdisciplinary Journal, 35(3), 253-294.

Upham, F. (2016) ActivityAnalysisToolbox_2.0. GitHub repository, https://github.com/finn42/ActivityAnalysisToolbox_2.0

Felt It My Way: Idiosyncratic Psychophysiological Responses to Recorded Music

Download the poster (pdf)

Abstract

Psychophysiological responses to music have been studied from decades, but the focus has typically been on common response patterns across listeners. The Repeated Response paradigm, recording a participant’s responses to a set playlist of stimuli over multiple listenings, allows for more in depth considerations of responses typical of individual listeners. Repeated exposures to the same stimuli have been associated with desensitization (Grewe, Nagel, Kopiez, & Altenmüller, 2007) as well as sensitization in increased coordination (Sato, Ohsuga, & Moriya, 2012). Either way, there is an opportunity to capture consistencies tied to individual participants musical histories and inclinations that can easily be lost when looking for agreement across a population or audience.

Aims

Identify consistency and coordination in individual participants psychophysiological responses to music and consider the contrasts between participants’ responses to pieces of music.

Method

Five participants heard the same six pieces of music 12 times over several weeks. During these listenings, skin conductance, finger temperature, heart rate, respiration, and facial muscle sEMG (Zygomaticus, Corrugator) were recorded continuously. Using activity analysis, we evaluated first which responses showed significantly coordinated in music relevant response events per participant. When participants showed coordination, their moments of consistency were compared to see whether they aligned or contrasted.

Results

Preliminary results show that participants vary how well their responses are coordinated between listenings and which response measures show the most coordination. For example, two participants showed very high respiratory coordination but different patterns of consistency in finger temperature decreases. Besides the overall pattern of disagreement, specific results will be shared on responses to specific works, including a late Beethoven String Quartet excerpt and a Dubstep track.

Conclusions

Listeners can show some shared patterns of behaviour to music, but they also develop idiosyncratic response sequences to pieces they come to know. This is not only measurable in post-stimulus ratings and preference but also in the sensitivity, reliability, and timing of changes in their psychophysiological responses.

References

Grewe, O., Nagel, F., Kopiez, R., Altenmüller, E. (2007). Listening to music as a re-creative process: Physiological, psychological, and psychoacoustical correlates of chills and strong emotions. Music Perception, 24(3), 297-314.

Sato, T. G., Ohsuga, M., and Moriya, T. (2012). Increase in the timing coincidence of a respiration event induced by listening repeatedly to the same music track. Acoustical Science and Technology, 33(4):255–261.

ICMPC15 – The Audience’s Breath

Posted on by finnupham

This year’s ICMPC recorded all of the talks in support of virtual attendance. Here is my long talk (20 min + Q&A) on coordination in respiration between audience members.

The Audience’s Breath: Collective Respiratory Coordination in Response to Music

F. Upham, H. Egermann, and S. McAdams

Abstract

Performers have used respiratory metaphors to describe the reactions of the audience’s engagement with a performance. We refer to an audience holding their collective breath, or sighing with a release of tension. Significant regularities in respiratory phase have been measured in participants’ responses over multiple listenings to some recorded music (Sato, Ohsuga, & Moriya, 2012), but this fleeting alignment has not yet been measured in audiences at live concerts.

Aims

With recordings of respiration from audience members at live performances, we aim to evaluate whether there is measurable respiratory alignment between them to some or all pieces. If there is coordination, we consider which phase of the respiratory cycle shows the highest degree of alignment and how this could relate to audience members’ experience of the music performed.

Method

Respiration data from two audiences were evaluated using new techniques in respiratory phase detection and measurement of coordination. From the first audience, 40 participants sat amongst a larger group in an experiment-led concert of chamber music including three pieces of contrasting genres. The second audience was composed of 48 participants who were presented solo flute music, some recorded and some played live. Half of this group continuously reported the unexpectedness of the music while the remaining half reported their felt emotional responses through handheld devices.

Five components of the respiratory phase were evaluated for coordination using activity analysis with parameters tuned to each: Inspiration Onset, High Inspiration Flow interval, Expiration Onset, High Expiration Flow interval, and Post-Expiration Pause. These phases relate to the mechanics of respiration and the sensory consequences of air exchange.

Results

Significant coordination in respiratory phase components were observed between audience members to most stimuli, but the most coordinated phases varied from piece to piece. High Inspiratory and Expiration Flow intervals were most often significantly coordinated, compared to onsets. Post Expiratory Pauses, which would count instances of breath holding, were only coordinated in one piece. Less than half of participants engage in phase alignment concurrently, however numerous instances relate well to developing theories of respiration/cognition interactions, including differences in the alignment patterns of participants per rating task.

Conclusions

Audiences engage in measurable collective respiratory coordination with live performance and recorded music through simultaneous inspirations and expirations. However, these behaviours are performed by only a subset of participants at a time. This inter-participant difference is consistent with the results from repeated response experiments, in which only some participants have shown respiratory coordination with their own previous listenings. The fact that different phases of respiration showed coordination underlines the possibility that multiple mechanisms like embodied listening, attention, and hearing facilitation may be encouraging adjustments in audience members’ respiratory sequences for alignment.

References

Sato, T. G., Ohsuga, M., and Moriya, T. (2012). Increase in the timing coincidence of a respiration event induced by listening repeatedly to the same music track. Acoustical Science and Technology, 33(4):255–261

Materials

The slides are also available for download.

The So Strangely Podcast on New Research in Music Science

Posted on by finnupham

Finally, the podcast has begun. The So Strangely podcast explore new work in Music Science by having academics recommend recent publications. The recommender and I interview the first author on how the project came about and what it means for music, science, and research. This is a podcast for people interested in music science, students and academics, but we also try to make the conversation friendly to people in the many different disciplines contributing to the area.

Listen to the first few episodes now on The So Strangely Podcast website and subscribe to catch future works. So far we’ve discussed papers in Music Perception, Music Information Retrieval, and Neuroscience, and we are always on the look out for more.

PhD Defended

Posted on by finnupham

On June 21st, 2018, I successfully defended by doctoral dissertation, Detection of Respiratory Phase Adaptation to Heard Music.  Without a doubt, listeners do subtly and subconsciously adjust when they breathe to fit with music, lining up specific respiratory phases to specific moments, but this happens under limited conditions. Only some moments of music draw respiratory phase alignment, and some people show stronger susceptibility to music’s coordinating influence.

With the extra three months granted by my committee, my quantitative analysis of listener respiration was extended with qualitative analysis of alignment patterns in repeated response studies and audience experiments. Activity analysis identified moments of exceptional phase alignment and music theory enriched my interpretation of the corresponding stimulus. Out of 36 pieces of music, 21 provoked identifiable moments of alignment and out of these arose four theories of how listeners’ breathing could be drawn or cued by what they heard:

  • Embodied perception/motor imagery: Some listeners toke inspirations when they might have, were they performing the music. This happens to vocal music, whether or not the performers’ breaths could be heard in music recordings. Examples from one case study participant can be seen in the attached figure, with inspirations (blue stars on chest expansion measurements) coinciding with performer inspirations during this a cappella folk song (highlighted in red on sound wave).
  • Inspiration suppression for attentive listening: The noise of inspiration and expiration can get in the way of auditory attention and there are (rare) moments in music when listeners seem to delay breathing in or out so as to hear better. A moment like this is also in the attached figure, with post-expiration pauses extended from 97.4 s.
  • Respiratory marking of salient moments: Listeners would sometimes breath in our out with recurring elements of musical motives, as if acting with something important or familiar. This was more common in structurally complex music and moments of strong affect, such as powerful lyrics, increasing tension, or exceptional aesthetics.
  • Post-event respiratory reset: In a few cases, well timed respiration cycles occurred after events, like after the last line of a song. This is reminiscent of relaxing sighs and similar actions through to help the respiratory system reset back to normal relaxed quiet breathing.

Causal mechanisms for these four theories are suggested by current respiration and music cognition research, however they each require further exploration on experimental data beyond what was studied here. And it is also possible they might arise more frequently than could be captured by these statistics, limited as they are to behaviour that co-occurs with the music at least 20-40% of the time. Between a theorize mechanism and well designed experiments, it may yet be possible to detect these deviation in action, giving us further clues into how listeners are engaging with the music they hear.

More details to come in the shape of my final dissertation document. To be completed in the next month or so.

Activity Analysis published in Music Perception

Posted on by finnupham

The Activity Analysis paper has been published in Music Perception!

Titled “Activity Analysis and Coordination in Continuous Responses to Music”, this paper explains what we can learn about the consistency of activity in continuous responses to music using the example of Continuous Ratings and (with the appendicies) all the technical details behind the results.

Abstract: Music affects us physically and emotionally. Determining when changes in these reactions tend to manifest themselves can help us understand how and why. Activity Analysis quantifies alignment of response events across listeners and listenings through continuous responses to musical works. Its coordination tests allow us to determine if there is enough inter-response coherence to merit linking their summary time series to the musical event structure and to identify moments of exceptional alignment in response events. In this paper, we apply Activity Analysis to continuous ratings from several music experiments, using this wealth of data to compare its performance with that of statistics used in previous studies. We compare the Coordination Scores and nonparametric measures of local activity coordination to other coherence measures, including those derived from correlations and Cronbach’s α. Activity Analysis reveals the variation in coordination of participants’ responses for different musical works, picks out moments of coordination in response to different interpretations of the same music, and demonstrates that responses along the two dimensions in continuous 2D rating tasks can be independent.

Download the PDF (Upham_McAdams_2018_ActivityAnalysis) and get the MatLab toolbox to use this technique on more continuous response data.

Million thanks to my co-author and mentor, Prof. Stephen McAdams, whose steadfast support made this work possible, and the patience of our editor at Music Perception, Prof. David Temperley.