Community & CodeLessons from NESCent hackathons

Arlin Stoltzfus, Michael S. Rosenberg, Hilmar Lapp, Aidan Budd, Karen

Cranston, Enrico Pontelli, Shann Oliver, and Rutger A. Vos

https://nescent.github.io/community-and-code/

A lot of questionsWhat practices make hackathons effective or ineffective?• What makes a good scope or theme? • How best to advertise and recruit? • How to engage before the event? • What logistics are preferable? • What supportive technology to use?• What modes of team formation? • What methods of target selection?

What do people expect hackathons to deliver? Why do• sponsors underwrite hackathons?• organizers plan and execute hackathons?• participants participate?

What do hackathons do?

Why are hackathons a thing?

Approach

• Retrospective analysis of a series of 9 events• “Open Science” practices left a record of – 9 events with 17 event outcomes– 54 projects with 133 project outcomes– 207 participants

• Guiding questions– What were the outcomes and impacts? – What practices or conditions favored positive outcomes?

NESCent hackathons are distinctive• Open and community-oriented– Sponsors have broad community-oriented goals– Everything is open-source– Diversity is a priority

• Domain: evolutionary informatics– a dispersed community of thousands worldwide

• Participants (25 to 30)– post-docs, faculty, grads (mostly PhDs)– offered full travel support– most apply in response to open call

ScheduleDay 1 Days 2 to 4 Day 5

Work

LunchWrap-up

Work

Lunch

WorkStand-ups

Why the events happened

Why did NESCent sponsor hackathons?• to address interop issues involving collaboration or

building community resources• to foster and energize a community of practice

– Spread awareness of best practices– Grow a professional network– Catalyze awareness of domain challenges & opportunities

Outcomes for

• 9 events – 17 event-level products

• 54 projects (teams) – 133 project products (mostly repos and team reports)

The most common products• new code repositories – most remain inactive

• additions to existing codebases– E.g., open-source toolboxes (BioJava, BioPerl, etc)

atypical: group keeps working, writes paper

spike of 238 commits peaking at hackathon

Less common products• documentation

– CRAN task view– Report on file format and parser compliance

• designs, standards and schemata– Phylorefs– Skelesim

• data products – Machine-readable tree annotations

• community infrastructure– r-sig-phylo mailing list

Details: see speaker notes

Follow-on products

• demonstrations and production code– DendroPy

• Blogs, conference presentations• manuscripts for publication

– 2 event publications – rotl– skelesim

• proposals for funding– GSOC proposals– phyloreferences– Phylotastic

Link is not via actual code, but schemes, community buy-in and working relationships

Project builds on code

Note

• End of stuff we counted objectively• Everything from here is subjective

Intangible outcomes

• Technology learning• Exposure to best practices• Awareness of challenges and

opportunities• Team programming experience

Credit: Randall Munroe https://xkcd.com/1425/

Lessons: scope a theme "must have the capacity to inspire participation by being specific enough to indicate the direction, while possessing sufficient openness to allow for the imagination of the group to take over” (OpenSpace philosophy)

Lessons: assistive technology

• At least– source code repository– event-wide communication channel

• provide training before or on day 1

Lessons: diversity

• open call response rate: ~ 1/100• personal appeal response rate: ~ 1/2

earlier events

later

Details: see speaker notes

2- to 3-fold increased diversity

Lessons: being welcoming

• reach out to non-networked participants– “Thanks again for applying to the hackathon. <sentence that shows I’m a human who read your application>. I look forward to meeting you.”

• do pre-event engagement • model positive communication– e.g., not "Isn’t that idea out of scope?" but

"What are some ways that could align with our goals?"

Lesson: be wary of remote participation

Works better when• Remote participant – is in similar time zone– Is already networked, experienced– commits 100 % of attention to event

• On-site participants– stick to schedule– Establish 2-way channel (buddy system)

Lesson: manage team formation

• Things work better when– Facilitators model asking questions to ensure plan is

• Relevant (in scope)• Feasible given skills of participants• Aimed at getting tangible outcomes by the end of the event

– A team is 3 to 7 people – A project requires collaboration for success

Explore & sift

Inform

pitch Team up

WorkCarefully managed, facilitated steps

Lessons: pre-event engagement

• introduction in online space or telecon• chance to ask questions and discuss ideas• chance for organizers to gauge– how well theme stimulates ideas– where participants need training or info

• don’t expect full participation

preparation for "invested participation" (Briscoe & Mulligan)

Concluding thoughts

Why participate? • learning new skills• networking• awareness• experience valued for its own sake

Details: see speaker notes

Resourceshttps://nescent.github.io/community-and-code/

Acknowledgements

One of 2 working groups

Some of the 207 NESCent hackathon participants