The Stilling Response

Most of the time, when we are looking for a connection between music and how people move, the focus in on the movement: when the beat drops, when the feet start tapping, when people start to sway. In the Music Lab Copenhagen concert experiment, we found some of that (paper forthcoming) but more strikingly, we also found that audience members seem to move LESS at specific moments of the music. During the Danish String Quartet’s world class performances of music by Beethoven, Schnittke, and Bach, the audience didn’t move with any kind of regular pulse but they instead seem to still for the rests, the drops in texture, and the quiet parts.

After noticing a few of these short dips in average audience motion, we chose to test a hypothetical relationship between the music and the audience’s lack of motion by starting from the performance. Watching it back, could we identifying when the music might encourage stillness? I spent many hours with the music scores, audio, and video to try to get a feeling for what could be a cue for stilling. After a few rounds, criteria for “stilling points” were defined with easily tracked musical surface features. Focusing just on local qualities, just the last few seconds of music, Stilling points were marked with decreases in the number voices (down to full rests), decreases in loudness, decreases in tempo, and silences not indicated in the score. These kinds of moments are common in concert music, but how they show up varies a lot. Rests can be part of a musical theme, returning over and over, while points of repose are subverted in forms like the fugues. Working through the more classical concert repertoire and the concerts closing set of nordic folk song arrangements, I identified from the music 257 Stilling points.

These Stilling points were tracked back to the audience motion measurements (chest mounted accelerometers, collected mostly through a phone app) and tested for the ratio of participants showing less quantity of motion than three seconds before. This very simple metric of collective stilling should normally be around 0.5, with half this mostly-still audience moving marginally more than 3 seconds before and half moving marginally less. But at most of the stilling points, we found that more audience participants were more still than should happen by chance.

The full description of how this assessment and some subsequent evaluations of what cues mattered and how still this audience really got can be found in the Music & Science paper:

Upham, F., Høffding, S., & Rosas, F. E. (2024). The Stilling Response: From Musical Silence to Audience Stillness. Music & Science, 7. https://doi.org/10.1177/20592043241233422

Surrogate Synchrony (SUSY) Limits

I’ve received requests for my opinion on SUSY, or Surrogate Synchrony, as an analysis strategy for synchrony between music-concurrent physiology measurements. In the subsequent discussion with colleagues, I have learned that many folks in this research area are unfamiliar with the math defining SUSY’s limitations. Rather than sketch the issues repeatedly over email, I’ve produced a demonstration notebook to show what kind of information is caught and what kind on information is missed by this method dependent on the average cross-correlation over lags of many seconds. What is demonstrated through these toy and real data examples may be obvious to researchers with any training in signal processing, however that background is a privilege in the interdisciplinary world of music psychology.

Surrogate Synchrony (SUSY) is a strategy for identifying some shared information in parallel recordings of the same type of signal from people in some interactive context, originally in motion trajectories between people in dialogue (specifically therapy sessions), but it has been applied to many types of measurements taken for people in musical interaction conditions as well. Most recently in used in a Scientic Reports paper (Tschacher, et al., 2023) on physiological and motion measurements of an audience during a classical music concert, it has been applied in similar contexts a few times over without a reconning of how synchronised the shared information must be to be counted (Seibert, et al., 2019; Tschacher, et al., 2019). From my experience with similar measurements and other approaches to shared information in time series, I am very concern that this method is missing much of what readers (and maybe some authors) assume it is capturing in these musical contexts.

The current notebook is only on dyadic SUSY with average non-absolute-valued Fisher’s Z transformed cross-correlations, though many of the consequences generalise to the multivariate derivative (Meier & Tschacher, 2021). What I am trying to show is not an opinion about where this technique could be useful, these are the mathematical facts of what kinds of “synchrony” it is capable of assessing. As it stands, there are surely some published false negatives from using this method without sensitivity to what it cannot see.

If this analysis gives anyone some helpful context for the method, I’m glad the effort did some good. Please don’t just take my python implementation of SUSY and ignore these caveats on its application.

Toy Signals at a rate that SUSY can capture well: slow identical signal components offset by 4 s (less than the lag range of 8 s)

Toy Signals that SUSY can’t capture well: identical signal components with no offset oscillating faster than the crosscorrelation range.
Real signals that SUSY can’t capture unless deliberately tuned: Respiratory waves with intermittent phase alignment.

If this analysis gives anyone some helpful context for the method, I’m glad the effort did some good. Please don’t just take my python implementation of SUSY and ignore these caveats on its application. Full Github repo: https://github.com/finn42/susy_limits

References:

Meier, D., Tschacher, W. Beyond Dyadic Coupling: The Method of Multivariate Surrogate Synchrony (mv-SUSY). Entropy 2021, 23,1385. https://doi.org/10.3390/e23111385

Seibert, C., Greb, F., & Tschacher, W. (2019). Nonverbale synchronie und Musik-Erleben im klassischen Konzert. Jahrbuch Musikpsychologie. Musikpsychologie–Musik und Bewegung, 28, 53-85.

Tschacher, W., Greenwood, S., Egermann, H., Wald-Fuhrmann, M., Czepiel, A., Tröndle, M., & Meier, D. (2021, September 23). Physiological Synchrony in Audiences of Live Concerts. Psychology of Aesthetics, Creativity, and the Arts. Advance online publication. http://dx.doi.org/10.1037/aca0000431

Tschacher, W., Greenwood, S., Ramakrishnan, S., Tröndle, M., Wald-Fuhrmann, M., Seibert, C., … & Meier, D. (2023). Audience synchronies in live concerts illustrate the embodiment of music experience. Scientific Reports, 13(1), 14843. https://doi.org/10.1038/s41598-023-41960-2