Report on RoSE Conference Debriefing

Motivation and context

The RoSE network can be thought of as a grassroots organization with most members being volunteers (who are not necessarily tenured). The RoSE Conference is one of the key events meant to establish name recognition of the RoSE Network. Ultimately, the conference is meant to:

• Increase the diversity in membership within the network
• Learn more about statistics education practices and research around the globe
• Reduce barriers to participation and access to public platforms for presenting one’s work and learning about other people’s work

My primary objective for conducting debriefings with past committee members and looking into materials from the RoSE Conference 2023 is to fine-tune the deployment of the conference for future iterations, while incorporating past experiences of organizers and participants.

Since we have roughly 1.5 to 2 years in between conferences, I thought about other objectives related to the primary objective:

• Ensuring continuity, reproducible deployment, and minimal organizational overhead
• Starting the possibility of generating passive funding
• Providing practice for event management and planning
• Creating a model for accessibility and EDI
• Making debriefing an important part of conference organization

At the conference committee level, we have to be mindful of the following:

• How to split up the jobs between the permanent and transitory committee members
• How to compile data and document experiences for future conference organizers
• How to define and strike a balance between research and teaching topics

RoSE Conference 2023

The members of the conference committee for RoSE 2023 are: Jenny Terry (chair), Megan Barnard, Danielle Evans, Elyssa Twedt, Abi Jones, Vimal Rao.

In different nutshells

• 2 keynote speakers, SIG-based invited sessions, workshops, speed networking session

• 745-2145 UTC, free, with shifts for tech support: 745-1500 UTC, 1400-2145 UTC

• 2 parallel streams for a one-day conference

• 38 sessions in total with the following themes:

  • Teaching statistics
  • Concerns about statistics education, transitions, attitudes, anxiety (stereotypes)
  • Statistical literacy
  • Support systems and software skills for statistical analysis
  • Analysis of data from LMS
  • Cognitive psychology and statistics education
  • Tools used in statistics education research and in classrooms

Rough task list

• Organizing events

  • Pre-launch event
  • Speed networking session
  • Zoom practice sessions for presenters

• Technology, data, and communications

  • Creating apps for the program and the links
  • Registration forms
  • Submission forms for potential paper presentations
  • Post-conference feedback survey

• Peer review

  • Volunteering or looking for volunteer reviewers
  • Creating rubric for peer review
  • Making decisions based on reviewer responses

• Collaboration

  • Drafting the conference program
  • Making suggestions
  • Sharing with social media

• Session control

  • Chairing sessions
  • Time management
  • Drafting and asking questions during sessions (to fill the awkward silences, of which there seems to be none)

Feedback from participants incorporating debriefing and commentary elements

• Not clear how many actually attended, not clear how many registered to attend: More of data and planning needs, may affect how we think about source of funds

• Have more breaks (for everyone including session chairs and tech support), prevent FOMO: There seems to be a strong preference for seeing sessions live rather than revisiting videos. Perhaps use the same 2 parallel stream format, but second stream is actually a delayed broadcast of the conference (in effect, similar to a radio station or a marathon channel like Adult Swim for US residents)

• Improve and communicate clearly format of poster sessions: When does a poster session actually work, especially in an online context?

• More opportunities and longer time for networking: must obtain direct feedback from speed networking sessions, unclear what kind of opportunities would be fruitful

• Aim for a wider variety of topics: Perhaps invite speakers after submissions are in so we know where there are gaps

• Clarify and make peer review process public

• Must be able to use conference/survey management tools which send confirmation emails or emails documenting choices made

• Internet stability is a problem: likely affect conference participants, usually participants from Africa and Carribean, perhaps watch parties supported by donations?

Tech infrastructure

• Institutional Zoom accounts: Not sure whether institutional Microsoft Teams accounts work well

• Google Sheets for the program with adjustments for most time zone differences

• Shiny app for a more interactive program interface: Baseline R code has been shared with me, but somehow I could not find the link to the app anymore.

• Map of presenter locations: R code shared with me as well.

RoSE Conference 2025

The members of the conference committee for RoSE 2025 are Andrew Pua (chair), Anna McGaughey, Magdalene Ho Hui Yi, Jessica Jaynes, Letetia Addison, Torsa Chattoraj, Dariyem Naandi Kruslat, Immad Shah, and Susan Cooper.

The conference is set on July 31, 2025.

Potential speakers

There is a plan to have 3 keynote speakers, but this is not set in stone. Currently, Andy Field from U Sussex has more or less confirmed to be one of the keynote speakers. Iddo Gal was invited last year, but had to decline.

We must try to extend the pool of potential speakers (not just as keynote) and I list some suggestions: Katie Makar (informal inference), Anna Reid (qualitative research in statistics education research), Joan Garfield (statistical literacy and reasoning), Enriqueta Reston (from the Philippines, curricular reform), Emi Tanaka (grammar of experimental designs), Nicolas Greliche (comics), Eric Daza (statistically discernible vs statistically significant).

It may also be instructive to look into the Center of Open Science Education Unconference and Teaching and Learning Mathematics Online to look for potential speakers or themes.

Formats to consider

There are different ways to organize the virtual conference. RoSE 2023 was based on a traditional approach with keynote sessions, invited sessions, contributed sessions, poster sessions, workshops, and special sessions (like a speed networking session). But there are other formats to also consider.

Some alternative formats are directly from what The Society for the Improvement of Psychological Science and the Center of Open Science are doing with respect to their hybrid and virtual conferences. Here are the usual formats:¹

• Unconference sessions: group discussions designed to reach consensus on a topic or generate a plan of action

• Workshops: hands on lessons that leave attendees with resources to use and apply

• Lightning talks: Presentations oon a topic in 5 minutes or less

• Hackathons: typically lasting for 24-48 hours, in which a group of people with diverse backgrounds come together to solve a problem by building a first working prototype of a solution (usually a web app, program or a utility)

Potential themes and topics

Consider the following tagline for the conference: Inspire, Provoke, Inform

Make sure to emphasize the network’s emphasis on research rather than purely pedagogy.

I asked about potential underrated topics and included some which arose during the debriefing. Here is a list (in no particular order):

• Research on the effects of EDI + accessibility + inclusion in statistics classrooms

• An evaluation of the statement: “Research is improved by having diversity.”

• Research methods for statistical education research: Best practices

• Evidence-based stats pedagogy: Interventions with program evaluation

• Resources for statistics software comparisons: usefulness for education and teaching

• Summarizing actual research on statistics software useful for administrators: one page editions

• “Share a practice”: minimal effort interventions which are shown to be effective and can be easily be adapted by almost anyone

• How is AI influencing research efforts?

• Are students more generally anxious with respect to learning in a more digital world?

• Are there questionable research practices in statistics education research?

• How often do collaborations occur in statistics education research?

Pre-conference discussion points

On what to include in the program

1. Must be on the same page with regard to the definition of statistics education research:

• connected to teaching and learning statistics
• empirical evidence
• wide application and scope

2. RoSE Network community invited presenters? Ask who they would want to see…
3. Submission of short versus long abstracts? Or request full paper?
4. What should peer review look like? Looking for volunteers for peer review
5. Organizing sessions with more involvement of (perhaps more independent?) early career researchers: PhD researchers, undergrad/masters, talk about dissertation research
6. Organizing sessions which feature more work in progress, leading to post-conference catch up and then turning into more stable research projects?
7. Organizing sessions with more qualitative research: for examples, student experiences, applying research methods, pandemic-related stuff

On logistics, participation, and administration

1. Construct survey questionnaire for the network: solicit accessibility issues, barriers to participation, what they want to see in the conference and in events
2. Internet access and stability are big issues. Should we provide support for watch parties or for some stable (but may be expensive) mobile data?
3. Should we continue using Zoom? Or should we consider Teams or other alternatives?
4. Should we continue or modify existing existing tech infrastructure? Avoid reinventing the wheel? Perhaps consider free options (having different features and focuses) by Oxford Abstracts or Conference Maker?
5. The minutes from a core committee meeting around October 2023 mentioned have some discussions about event planning and whether or not the conference committee may be involved. Should we consider members of committee organizing smaller-scale events to gain some practical hands-on experience?
6. Should we consider a pay-as-you-want model for the conference? Note that pay-as-you-want may not work because reimbursement may not work. If pay-as-you-want could be capped at a nominal amount, then perhaps reimbursement is no longer an issue?
7. How should the reporting through social media be organized and who would be responsible? Three streams – one for general news, one for writeups for more exposure of the network and the featured, one for live-reporting?
8. Should we prepare a proceedings of RoSE 2025 volume? Or should we consider a “recommender system” type approach where we recommend papers in the conference everyone should pay attention to?
9. Should we unify the conference background (for custom backgrounds when cameras are turned on), conference and network logos, other paraphernalia?
10. Who is responsible for handling code of conduct violations (if there are any)?
11. Blanket consent for recording: people’s faces being recorded, or give reminders?
12. Should we consider having sponsors?

In-conference discussion points

1. Which topics should go asynchronously? synchronously?
2. Should there even be a poster session? How should it be organized properly?

• Organized as JSM-style speed session: put all videos together for viewing, then set up schedules for Q&A
• Integrate contributed paper with a poster for broadest reach?
• Adult Swim loop
• Did posters from 2023 turn into papers?

3. Slack plus Zoom approach to dropping off links, allowing for communication and networking, and troubleshooting?
4. Points related to social media reporting, code of conduct, blanket consent for recording apply here too.
5. Volunteering for session chair and session tech support: assigning credit, freebies, software trials from sponsors (if there would be sponsors)?

Post-conference discussion points

1. Handling video editing work: perhaps this could be the “proceedings” in a different format?
2. Facilitate post-conference catch up / reaching out
3. Being more active in matching participants and to facilitate further networking
4. Tracking networking and participation outcomes
5. Post-conference resource bank development? What did we learn in the conference? Perhaps this is related to what medium or form the proceedings will take?
6. Designing post-conference surveys and feedback forms for RoSE 2027

Footnotes

1. These first three descriptions were directly copied from a Center of Open Science unconference webpage and the last is from a blog post from the Journal of European Psychology Students Bulletin.