This last year, I served virtual technology chair for the 21st meeting of the International Society for Music Information Retrieval ISMIR 2020, a position deemed necessary when COVID-19 lockdowns began to spread. The transformation of a week-long in person conference to a week-long 24 hr virtual meeting required a whirlwind of effort from all the organisers, and to sum up the lessons learned, we have compiled an exit report (pdf). The document lays out what technologies were used, how events were scheduled, instructions shared with participants, and details from many individual chairs on how they adapted their responsibilities. Also included is an analysis of how attendees actually used the platforms, looking at attendance numbers per event time and type, and a report on participants experiences shared via a post-conference survey.
The 40+ page report (pdf) may be of interest to anyone coordinating their first virtual academic conference and to researchers looking at how research communities are adapting to the current circumstances. To share a taste of what we learned, here are some highlights from the analysis of registration statistics, virtual platform usage, and the post-conference survey responses.
The virtual format allowed many more people to attend ISMIR than usual. In a conferences that normally sees 400 participants, we had over 800 sign up, with half of the registrants attending for the first time. In the report, we break down the distribution of registrants across a number of categories, below are the stats by country of residence, gender, and career position. The inset green wedges shows the proportion of first time attendees per category. Some of these ratios are to be expected, say the high proportion of grad students attending for the first time, others are more informative like the higher ratio of new women registrants than of new men.
With so many people registering, it was hard to know how many to expect in attendence at specific events. Attrition is very high for virtual events, particularly if registration is relatively low cost. Participation at any given conference event was split by our 24 hr doubled schedule, but careful review of the platform statistics found the vast majority of registrants visited the conference slack daily while ~50% commented and attended zoom events.
The 24 hr schedule was designed to ensure participants could have the full conference experience from any time zone represented in our registrants. The schedule was organised in two sets of shifts, spaced 11.5 hrs apart, with the Alpha-Gamma and Beta-Delta pairs offering the same poster sessions and main conference presentations such as keynotes. From the activity in slack channels, we found a significant difference in attendance levels for the poster sessions, with the first of both double sessions consistently more busy than the second. This difference in demand seems to be a mix of time zone concentrations and a kind of premier effect, and would be worth planning around in the future.
The post conference survey was answered by about 20% of attendees and they shared many useful comments about the experience. Top of mind was how this style of virtual conference compared to what the community was used to. On many points, the loss of in-person contact was keenly felt, but some aspects of this Slack-supported virtual experience were preferred by a solid minority.
Besides noting the limitations of the conference design and platforms, it’s worth noting that some of the differences in experiences reported are a consequence the conditions from which people were participating. Most were at least somewhat more constrained by the practicalities for attending without leaving their work and home. But despite all that was new and challenging about this way of conferencing, we were very happy to see that most survey respondents were still at least somewhat satisfied by the experience provided.
Please see the report for more survey results and analysis of participation, details of how the conference was designed, what we might suggest doing differently, and full credit to the many people who made ISMIR 2020 a success.