Activity Analysis Paper

With great relief, I can say that the Activity Analysis paper for Music Perception has been accepted for publication! I don’t know exactly when it will come out but here are the essential components:

Activity Analysis and Coordination in Continuous Responses to Music

by Finn Upham and Stephen McAdams


Music affects us physically and emotionally. Determining when changes in these reactions tend to manifest themselves can help us understand how and why. Activity Analysis quantifies alignment of response events across listeners and listenings through continuous responses to musical works. Its coordination tests allow us to determine if there is enough inter-response coherence to merit linking their summary time series to the musical event structure and to identify moments of exceptional alignment in response events. In this paper, we apply Activity Analysis to continuous ratings from several music experiments, using this wealth of data to compare its performance with that of statistics used in previous studies. We compare the Coordination Scores and nonparametric measures of local activity coordination to other coherence measures, including those derived from correlations and Cronbach’s α. Activity Analysis reveals the variation in coordination of participants’ responses for different musical works, picks out moments of coordination in response to different interpretations of the same music, and demonstrates that responses along the two dimensions in continuous 2D rating tasks can be independent.


Download the Author’s Version (pdf, 4MB) and check out the latest version of the Activity Analysis Matlab Toolbox (Github, download files directly)

Besides vast improvements in terms of writing style and the like, this version also includes a quick comparison of ratings to two performances of the same piece. Here is the figure related to that analysis.

FIGURE 8. Continuous ratings of felt emotional arousal ratings to two different interpretations of the Renaissance madrigal Il blanco e dolce cigno by J. Arcadelt. A) 30 ratings (Rsp (Rec)) and their average times series (Avg) to a recording by the King’s Singers, plotted in metrical time and C) 17 ratings (Rsp (Rec)) and their average (Avg) to a live performance by the semi-professional choir, the Orpheus Singers, plotted in metrical time. B) The activity levels of rating increases (minimum 2.5% in 2-s time frames) on overlapping time frames, aligned in metrical time, of ratings to the recording above (Rec (inc)) (max NPC Score, 3.3), and to the live performance below (Live (inc)) (max NPC score, 3.3), with time frames of locally extreme high and low activity levels (X-Act) marked in grey circles and black diamonds.

The Motivating Power of Fannish Love

A close version of this article was published in Powers of Expression, vol 1, in 2016.

Fandom inspires us to create. You don’t need to construct replica sets or write fanfic to be a fan, but many of us take up new projects and learn new skills because of these favourite things. So how does enthusiasm for a show, team, or celebrity push us into action? Although cognitive science hasn’t turned its lens to fandom, related phenomena suggest causes for the behaviours we exhibit. Passionate love and devotional (religious)  communities are known to inspire amateur creativity, and perhaps the same mechanisms of feeling drive our outpourings of transformative works.

We love what we fan over but what kind of love is this? The neurological mechanisms behind parental love of young children are somewhat different from those supporting companionable love for friends and life partners, and they are also expressed in different ways. The rush of falling in love, from wanting to having or missing that special someone, is what some psychologists call “passionate love”. It’s said to include “emotional responses such as euphoria, intense focused attention on a preferred individual, obsessive thinking about [them], emotional dependency on and craving for emotional union with this beloved, and increased energy.” (Aron, Fisher, Mashek, Strong, Li, & Brown, 2005) This kind of love is a famous catalyst for action, with early love showing even stronger need to do things for the object of affection (Ibid.). If fans are passionately in love with the thing they fan over, this would bring all the urgency we’ve come to expect from new lovers. But can we really be in love with Sherlock, the show, or a Youtube star, or our One True Pairing (OTP)?

Some fans use the language of passionate love to describe their experience, “falling” for their focus of attention, with squeeful breathlessness when they connect with their obsession and despairing frustration when access is denied. I asked a friend to complete the Hatfield and Sprecher’s Passionate Love Scale (2011), a list of statements about feelings for another person to which the respondent rates their agreement in order to calculate the degree to which they feel passionate love. In order to capture her feelings about her OTP, a few questions to be modified, for example to describe interactions between characters rather than with herself, and a couple had to be dropped, but the vast majority were directly applicable.  When her feelings had been at peak intensity, she scored right at the top of the scale, as “Wildly, even recklessly, in love.” This included reporting complete agreement to many statements of impractical behaviours like “I sometimes find it difficult to concentrate on work because thoughts of [the OTP] occupy my mind” as well as positive and negative emotional experiences like “[The OTP] can make me feel effervescent and bubbly.” From what I’ve seen and experienced, I would not be surprised if many others in our fandom circles scored similarly.

Falling in love is not convenient; it’s overwhelming, irrational, and socially disruptive. This emotional intensity is easily pathologized outside of specific acceptable circumstances, for example a rare folly between unattached young adults. The analogy between fannish love and passionate love has been used to trivialize the experience of fans, particularly fangirls: Without the possibility of a person reciprocating the affection, we must be wasting our time and effort.  But while it’s true that fannish love can be harmful in excess, it can also lead to beneficial experiences and transformation.

DSC06483 copy
A popular object of fannish devotion. We can discuss why in another column. But honestly, what is there not to like about a funny looking, generous, curious, highly regarded, and emotionally dynamic actor? [Photo by Dixiebell, 2015]
One could say that I am writing this now because of Benedict Cumberbatch. My fannish love of Cumberbatch inspired me to face career-debilitating fears of writing out my thoughts and putting them where others can see them. I can read and talk about ideas all day long, but academics must write, and I needed a safe space to practice. My desire to impress this infinitely distant being finally made it worthwhile to scribble down and edit a few thousand words on cognition-related theories. Over the course of two years, I wrote a few dozen letters to Cumberbatch, each an essay tuned to this audience of a single professional performer. I still hate writing but the exercise of this uni-directional correspondence did a great deal of good for my prose and my intellectual process, arguably more good than the last ten years of university. Without an external deadline, I learned my own pace for developing ideas and laying them out, along with criteria for dropping theories when I would have once submitted the mess at hand. Whereas class assignments and academic publications might resist mixing disparate disciplines, I enjoyed the freedom to read and draw insights from anything I could find, be it criminology, literary criticism, comparative neuroscience, and beyond.

There are some important differences between the circumstances of fannish love and what is generally thought of as passionate love, and these differences make it possible to build our supportive fandom communities. As mentioned above, passionate love’s goals of physical proximity and reciprocation are simply not feasible. In the usual context of romantic infatuation, this would result in feelings of great distress and yet fans are overwhelmingly happy to be fans. Without the need for physical proximity and attention from a real physical person, we are not in competition with others who love the same thing. Instead fans can celebrate their love together, as humans have done for ages in the form of devotional communities. Some who study the psychology of religion have compared religious conversion to falling in love (Argyle, 2010, p. 21), particularly the experience of joining the communities with a charismatic leader (Jacobs, 1987). Fandoms as we know them are certainly not the first examples of individuals developing these kinds of intense feelings as a group.

Putting aside the question of whether the word fan is derived from fanatic and morality-tinted accusations of idolatry, there are important parallels between the behaviours of religious groups and fandom communities. Fandoms do not set moral codes nor do they answer existential questions, however they define how we share the experience of fannish love with dozens, hundreds, even thousands of others. And this crowd can encourage individual members to create in many ways.

Contact with other enthusiasts can amplify our own feelings through emotional contagion, but some fandoms take this interdependence to the next degree because our primary means of “union” with our shared object of affection are materials created by our fellow devotees. Large fandoms can create exponentially more hours of content than the official media canon and many fans speak of getting “sucked into” a fandom when they start consuming fanworks. Fed by a rich diet of meta, fan art, and fan fiction, our passion is stoked and sustained. Extending the squee through time is very important, particularly for those engaged in large scale projects like oil paintings and novel length fics.

Identifying with a group also changes how we act as fans. Fandom communities are not (necessarily) formal organisations, but even knowing of other fans can change how we judge ourselves. In online fandom communities, fan creators (from live bloggers to visual artists) actively define our idea of what fans can do and how they should be. In addition to making it okay to use our excitement for creative endeavours, these examples offer helpful constraints on the types of works to be made. In some communities, fanart is hugely popular, in others, it’s the analysis of player statistics, but whatever the standard, having something to create turns an impulse into work. And making works in the style of the fandom increases the chances that they will be appreciated. Whether our motives are altruistic (support the community) or self-interested (get those kudos), playing within common forms is a practical use of our creative energy.

Screen Shot 2016-03-13 at 15.25.47
Online fanwork archives are important conduits for fan-fan interactions. We encourage each others fannish with the works shared and reward creators with feedback.

More than a few religious communities run on volunteer effort. Be it guilt, or social ties, or the satisfaction of working for the common good, individual participants put great effort into tasks fantastic and mundane (Argyle, 2000, ch 10). After a few months of finding great comfort and joy in consuming works by the BBC Sherlock fandom, I was itching to give back to this creative community. I can’t draw and fiction writing is not my area, so my motivation floundered until I found podfics. Audiobooks of fanfiction, podfics are not the most popular form of fanworks, but they are easily made by anyone with a microphone and the patience to edit audio. It had been ages since I’d done any dramatic reading, but that didn’t matter in this unapologetically amateur environment. The opportunity to pay homage to a show and to fanworks that I respected, along with the potential of an audience (however small), was enough to get me started. From those early recordings grew a great love of audio technology, a resurgence of excitement about acting, and my current involvement in a fandom podcast. Some of that is work-work, boring effort for a community I want to prosper, but the openness of fandom spaces grants me leave to try new exciting projects I’d hate to miss.

If the cognitive and social processes behind our motivations to act follow the patterns of passionate love and fervent religious conversion, then these intense urges to create have a limited life span. We should lose interest within one or two years (Jacobs, 1987; Aron et al, 2005), unless other kinds of love and satisfaction develop in their place. Thankfully, for myself and many important friends, that is plenty of time to build strong social ties to the community, to find other fandoms or aspects to thrill over, and to build our skills at whatever creative practices we’ve taken up. And if, from time to time, I smile fondly at the thought of Benedict Cumberbatch, it is as much for what has come out of that love as it is for the man himself.


Aron, Arthur, Helen Fisher, Debra J. Mashek, Greg Strong, Haifang Li, and Lucy L. Brown. “Reward, motivation, and emotion systems associated with early-stage intense romantic love.” Journal of neurophysiology 94, no. 1 (2005): 327-337.

Argyle, M. (2000). Psychology and religion: An introduction. Routledge.

Hatfield, E., & Sprecher, S. (2011). The passionate love scale. Handbook of sexuality-related measures, 469-472.

Jacobs, J. (1987). Deconversion from religious movements: An analysis of charismatic bonding and spiritual commitment. Journal for the Scientific Study of Religion, 294-308.

Deviations in Quiet Breathing during Music Listening

(This is post is derived from a poster presentation at the 14th International Conference of Music Perception and Cognition, hosted in San Francisco, CA, USA, July 4th-9th, 2016)

Music listeners often fall into quiet breathing and yet music has been shown to influence when individual listeners inhale. Here is an explanation of how deviations in quiet breathing can be measured in the respiratory sequence, and tests of how these deviations can depend on the musical work.

Defining Quiet Breath

When we are at rest and not preparing to act or thinking about acting, our bodies generally fall into the state of quiet breathing:

  • Moderate depth
  • Short inspiralation, ~1 s
  • Short elastic expiration ~ 2.2 s
  • Stable periodic cycle

Quiet breathing is efficient and discrete, a respiratory sequence that does not require attention or conscious control. Compared breathing behaviour during physical actions, the regularity of quiet breathing suggests that it should be relatively easy to model.

Continue reading “Deviations in Quiet Breathing during Music Listening”

Breathing in Music: Measuring and Marking Time

(This is post is derived from a poster presentation at the Making Time in Music conference, hosted by the Faculty of Music of Oxford University, Sept 14-16th, 2016)


Our breath marks time for the entirety of our lives. Whether a period of 2 seconds or 20, we know roughly how it will continue or be adjusted to new demands, and this need for fresh air imposes an inescapable rhythm just beyond what is readily heard as metrical. We use breath to communicate with speech and affective displays, but we also monitor each others’ breathing and use this information to coordinate interactions: breathing in anti-phase when in dialogue, or together when synchronising actions. Obviously, musical activities such as singing and playing wind instruments involve exhalations and the particular physical constraints of our respiratory system. Other components of breath are used to prepare and set the timing of actions. For example, the inhalation at the beginning of a piece defines tempo and intensity for many solo performers and small ensembles, and some types of musicians are extremely practiced at picking up all that is needed to play in synch from one careful gasp. We might consider breath to be auxiliary to the actions of music making, just a means to the sound, but this biological system may be play a fundamental role in our understanding of music and musical time. There is growing evidence that listening to music can engage our respiratory system, drawing us into a specific physical division of time. This coordination is not so strict as breathing with the heard performers, but rather a subtle alignment of phase at specific moments in a particular piece. For this to occur, even intermittently, our respiratory system must be engaged in the work of understanding what we hear. Voluntarily or unconsciously, breathing informs synchrony on the scale of milliseconds, seconds, and minutes, and this phasic and adaptive system promises to be powerful in defining musical time both physically and metaphorically.

Continue reading “Breathing in Music: Measuring and Marking Time”

Activity Analysis toolbox

Activity Analysis is descriptive and statistical method for interpreting the temporal coordination of measured events across synchronously recorded time series, developed to investigate collections of continuous responses to music like ratings of tension.

I’ve been developing Activity Analysis since 2007, when I was presented a bunch of continuous felt emotional intensity ratings from audience members attending a concert of orchestral Mozart music and asked to do what I could with them. Looking at the music cognition literature, I wasn’t all that pleased with the analytic options already in practice. Some were numerical dubious, others more statistically justified but none did a very good job of telling me WHEN interesting or important stuff was happening in the responses. There are lots of questions one can put to time series data, some of them good, many of them less useful, and my master’s thesis and the continuous response analysis wiki came out of a long process to understand what kinds of questions we have and could ask of temporal traces of experience.

The advantage of continuous responses for music is that time makes a nice firm causal anchor on what the listener knows and can respond to. While someone can have expectations (and memories) reaching into the future, and thoughts and feelings tying to the notes that have passed, the chronology of presentation  and the present are essential to the reception of music (and many other parts of the conscious experience). We have been recording continuous traces of listener experience for decades but the temporal power of these data has mostly lain dormant. Continue reading “Activity Analysis toolbox”