Open Stack

Hi everyone,

having had the privilege to serve as both as a track chair and as a
speaker at previous Summits, I'd like to offer my views here.

[Claire]
> Hi everyone,
> 
> For the past few Summits, we've received mixed feedback about having
> the community vote on proposed sessions as part of the call for speakers
> process. Historically, after the call for speakers closes, we publish
> all submitted sessions for community voting before the track chairs
> review them. The track chairs then choose how much weight to put on the
> voting resuls, if any, because they make the ultimate decision about
> which sessions are selected. More info on Track Chairs can be found at
> the bottom of this email.
> 
> With the growing number of speaking submissions (we had 1,300 for
> Austin), some community members have expressed concerns about social
> media channels and email getting spammed during the week of voting.
> We also think many community members are unclear as to how much the
> votes weigh on the final decision. For example, some think that if
> someone campaigns for votes or asks their colleagues to vote, the
> session will likely be accepted (which may not be the case).

I would agree that there was some concern on
- ballot stuffing,
- people voting along "party lines" (i.e. supporting their colleague's
talks based more on shared company allegiance rather than talk merit), -
companies with a strong social media reach doing their part to drive
their own votes up, and such promoted talks being over-voted.

[Claire]
> We would like to propose removing voting from the selection process
> for the October 2016 Barcelona Summit, but want to get your input
> before making a final decision.

I have previously suggested a different approach: improve the *quality*
of voting drastically, rather than abolish it altogether.

Now what follows isn't a voting process of my own design. We owe the
idea to a couple of professors of astronomy, who explain their approach
in a paper and a couple of interviews, all of which are linked from here:

https://plus.google.com/+FlorianHaas/posts/MsswaBHramG

(Please note: I have deliberately disabled comments on that post, in an
attempt to keep the discussion here on this list, where it belongs.)

If you care about the process, please take a few minutes to review at
least both videos linked from that post. If you're *really* interested,
read the article as well -- but for the purposes of this discussion, the
videos should suffice.

When I first floated this idea on the track chair mailing list a few
months ago, Duncan Thomas made this point:

[Duncan]
> I think limiting votes only to people who submit talks would lead to
> people/companies submitting poor talks just to get a vote (gaming the
> system).

To which I then replied:

[Florian]
> That's a fair point. However, reviewers could separately flag
> proposals that don't meet certain quality criteria. (*Some* formal
> criteria could even be checked by computers, not humans.) And there
> could be a rule that if, say, the majority of a talk's (anonymous)
> reviewers flag foul play, all the proposer's proposals *and* all and
> the proposer's votes would be invalidated. I think that would be a
> fairly strong deterrent. And in order to deter abuse of *that* system,
> the event of a proposer being thus sin-binned should probably be
> reviewed by a panel of some description.

(Just quoting this here to paint a clearer picture.)

[Claire]
> Our thinking is that by removing voting from the process, we will:
> - Save valuable time during the overall Summit programming process,
> which should allow us to publish the final agenda and notify speakers
> sooner
> - Allow our development teams more time to focus on improving the
> mobile app and web schedule developed during the last Summit cycle
> - Reduce the spam and noise around voting, so we don't cause Twitter
> fatigue before we're promoting the final agenda and key themes

> - Level the playing field for speakers from startups, new community
> members, etc. who may not have an established network in the
> community for voting

While all the above intentions are noble and I agree with them, I do
believe that the alternate approach which I'm humbly proposing has all
those merits as well.

And I do understand that this also means additional work on the voting
toolchain. So even in case there is agreement on the approach, I am not
suggesting for it to be implemented for Barcelona. Instead, my
suggestion would be for the voting process to be unchanged for the
upcoming Summit in spite of its imperfections, and for the new approach
to be considered for Boston.

[Claire]
> We initially started the voting process for good reasons and we do
> think there's value, but we're reaching a point where the costs are
> starting to outweigh the benefits. We'd like to get your input before
> we open the call for speakers in early June for the Barcelona
> Summit.

May I offer a couple of additional thoughts on why I believe killing
voting altogether is bad:

- Firstly, we've seen some track chair lobbying in the run-up to the
Austin cycle already, and in our track tried to nip it in the bud, but I
think this would be far worse if track chairs were the only people
influencing the talk selection. They already have the final say, but
it's a different quality if they have the *only* say.
- Secondly and more importantly, I do think of OpenStack as a community,
and I much appreciate the fact that the Foundation has chosen a model
involving broad individual membership. We as a community ought to have a
stake in the Summit selection process; rather than weakening or
eliminating that stake, strengthening it would be prudent.

What do others think?

Cheers,
Florian


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/community/attachments/20160517/6647a56d/attachment.pgp>