Hi everyone, having had the privilege to serve as both as a track chair and as a speaker at previous Summits, I'd like to offer my views here. [Claire]
Hi everyone,
For the past few Summits, we've received mixed feedback about having the community vote on proposed sessions as part of the call for speakers process. Historically, after the call for speakers closes, we publish all submitted sessions for community voting before the track chairs review them. The track chairs then choose how much weight to put on the voting resuls, if any, because they make the ultimate decision about which sessions are selected. More info on Track Chairs can be found at the bottom of this email.
With the growing number of speaking submissions (we had 1,300 for Austin), some community members have expressed concerns about social media channels and email getting spammed during the week of voting. We also think many community members are unclear as to how much the votes weigh on the final decision. For example, some think that if someone campaigns for votes or asks their colleagues to vote, the session will likely be accepted (which may not be the case).
I would agree that there was some concern on - ballot stuffing, - people voting along "party lines" (i.e. supporting their colleague's talks based more on shared company allegiance rather than talk merit), - companies with a strong social media reach doing their part to drive their own votes up, and such promoted talks being over-voted. [Claire]
We would like to propose removing voting from the selection process for the October 2016 Barcelona Summit, but want to get your input before making a final decision.
I have previously suggested a different approach: improve the *quality* of voting drastically, rather than abolish it altogether. Now what follows isn't a voting process of my own design. We owe the idea to a couple of professors of astronomy, who explain their approach in a paper and a couple of interviews, all of which are linked from here: https://plus.google.com/+FlorianHaas/posts/MsswaBHramG (Please note: I have deliberately disabled comments on that post, in an attempt to keep the discussion here on this list, where it belongs.) If you care about the process, please take a few minutes to review at least both videos linked from that post. If you're *really* interested, read the article as well -- but for the purposes of this discussion, the videos should suffice. When I first floated this idea on the track chair mailing list a few months ago, Duncan Thomas made this point: [Duncan]
I think limiting votes only to people who submit talks would lead to people/companies submitting poor talks just to get a vote (gaming the system).
To which I then replied: [Florian]
That's a fair point. However, reviewers could separately flag proposals that don't meet certain quality criteria. (*Some* formal criteria could even be checked by computers, not humans.) And there could be a rule that if, say, the majority of a talk's (anonymous) reviewers flag foul play, all the proposer's proposals *and* all and the proposer's votes would be invalidated. I think that would be a fairly strong deterrent. And in order to deter abuse of *that* system, the event of a proposer being thus sin-binned should probably be reviewed by a panel of some description.
(Just quoting this here to paint a clearer picture.) [Claire]
Our thinking is that by removing voting from the process, we will: - Save valuable time during the overall Summit programming process, which should allow us to publish the final agenda and notify speakers sooner - Allow our development teams more time to focus on improving the mobile app and web schedule developed during the last Summit cycle - Reduce the spam and noise around voting, so we don't cause Twitter fatigue before we're promoting the final agenda and key themes
- Level the playing field for speakers from startups, new community members, etc. who may not have an established network in the community for voting
While all the above intentions are noble and I agree with them, I do believe that the alternate approach which I'm humbly proposing has all those merits as well. And I do understand that this also means additional work on the voting toolchain. So even in case there is agreement on the approach, I am not suggesting for it to be implemented for Barcelona. Instead, my suggestion would be for the voting process to be unchanged for the upcoming Summit in spite of its imperfections, and for the new approach to be considered for Boston. [Claire]
We initially started the voting process for good reasons and we do think there's value, but we're reaching a point where the costs are starting to outweigh the benefits. We'd like to get your input before we open the call for speakers in early June for the Barcelona Summit.
May I offer a couple of additional thoughts on why I believe killing voting altogether is bad: - Firstly, we've seen some track chair lobbying in the run-up to the Austin cycle already, and in our track tried to nip it in the bud, but I think this would be far worse if track chairs were the only people influencing the talk selection. They already have the final say, but it's a different quality if they have the *only* say. - Secondly and more importantly, I do think of OpenStack as a community, and I much appreciate the fact that the Foundation has chosen a model involving broad individual membership. We as a community ought to have a stake in the Summit selection process; rather than weakening or eliminating that stake, strengthening it would be prudent. What do others think? Cheers, Florian