Open Stack

Hi Lana!

On Thu, May 19, 2016 at 3:23 AM, Lana Brindley
<openstack at lanabrindley.com> wrote:
> I'm actually really starting to like Florian's proposed method, with one exception: I don't like the idea of limiting it to talk submitters. The reason I say this is that, before I was a PTL, I submitted talks to every summit. That stopped when I became a PTL because, quite simply, I spend most of my time running sessions in the Design Summit, only popping over to the main conference for the talks I absolutely don't want to miss. I think by limiting the voting to only people submitting talks, you will miss the voice of people who deliberately *don't* submit a talk, because they're massively invested in other aspects of the conference, especially those on the more technical side of the house.
>
> Perhaps, instead of limiting it to talk submitters, maybe make it available to a different subset: people who have attended previously, maybe?

Hmmm. Well I'm afraid limiting talk votes to talk submitters is
exactly what makes the proposed approach meaningful. :)

If you have a minute, please consider reviewing Prof. Merrifield's
remarks in the video when Brady asks his question starting with "call
me a cynic", about https://youtu.be/7c0CoXFApnM?t=6m25s — this is
exactly the part that makes this system self-policing, and it goes out
the window if your own proposal isn't at stake.

Side note, if your assessment badly disagrees with what everyone else
has been thinking about a proposal, then this is not necessarily
because you're naughty and you want to game the system — you may just
be a shoddy reviewer who went over their reviewed proposals in a rush
whereas everyone else gave them more time. That, too, is something
that the system *should* penalize, because it ensures the quality of
the review process.

There is one other criticism to this, which is the opposite: what if
I'm being *extremely* diligent and I detect an issue that no-one else
detects? This is addressed here: https://youtu.be/bplncn4xC74?t=1m48s
(tl;dw: have public, anonymized free-form comments available to all
reviewers).

At any rate though, I can't think of a way to do this that does *not*
make the group of reviewers identical with the group of submitters.
And quite frankly, I quite like it as it is, considering the fact that
the proposed system forces everyone not only to think "how would I
rank this", but also "how would *others* rank this", which is exactly
what you want for the benefit of the much greater group of conference
attendees (as opposed to speakers).

What are your thoughts on that?

Cheers,
Florian