Re: [PHP-DEV] Capturing reasons for votes for historical sake?

This is only part of a thread. view whole thread
  109093
March 17, 2020 03:01 mike@newclarity.net (Mike Schinkel)
> On Mar 16, 2020, at 9:20 PM, Dan Ackroyd <Danack@basereality.com> wrote: > > On Mon, 16 Mar 2020 at 18:29, Mike Schinkel <mike@newclarity.net> wrote: >> >> Would it be possible to add a feature when voting were people either need to type in >> a one to two sentence reason why they voted "no" on a proposal > > Pretty strong no for me.
Thank you for your reply. Let me start with the end of your reply, what you state was your biggest issue:
> And the big issue, where possibly I disagree with your reasons for wanting this: > >> and their reasons are really important when it comes >> to future consideration of the same issue > > This kind of makes an assumption that all RFCs should have some path > to being passed. > > For some RFCs, there just isn't a good path forward for them.
Actually no. You are assuming wrong. If we had history we could better see the RFCs that had valid reasons for not passing and that should not be revisited vs. the ones that didn't pass for some technicality and didn't pass because the timing was not right that should be revisited. If we knew WHY RFCs were turned down then the next time someone brings the issue up again — which we they will — we could say "Here, read this. Chances are this idea will never pass for the reasons explained, so don't waste all of our bandwidth (unless you can counter all those objections, which you probably cannot.)" So actually part of what I am asking for is a way to put the bad ones down, and for good.
> btw I think there's a bigger problem on some RFCs being accepted, > particularly where people who aren't core maintainers are voting on > things that really need an informed understanding of internals.
I can certainly understand that perspective. But what's the proposal to improve that situation? Actually, I think you are under-appreciating what I proposed. Consider if instead of a "Yes/No" prompting people to vote, there was a list of reasons to vote "no" from those who already voted no. Isn't it likely that people who got to review that list would be more likely to vote "no" in the case of something that should not actually be accepted? (To be fair, if we did that we'd probably need a list of reason why people votes "yes" too so that voters could see why others voted yes and no.) (BTW, I do think having a good history of why things failed might keep bad ones from being approved in the future "accidentally.")
>> However in PHP we have no way of knowing why people voted against a proposal > > I think this type of analysis belongs off-list. Which is why: > https://github.com/Danack/RfcCodex/blob/master/rfc_codex.md
That's nice, but requires superhuman efforts on one individual's part, is contained on the GitHub account controlled by one individual, and is inherently slanted by one individual's bias, even if only unconscious bias. So it is really not a solution for the PHP community in general, notwithstanding that it is a nice resource that you have provided of your own industriousness.
> * Explaining exactly why you're voting no can be hard as there can be > multiple overlapping reasons for voting no.
1. Nothing I proposed would stop someone from listing multiple reasons. Free-form text would allow someone to write a dissertation, if they desire. 2. If someone votes "no" but cannot put it into words, it is really fair that are driving the destiny of PHP used by ~5 million programmers? 3. There is nothing that In I am proposing that would stop you from just typing in "I really don't like this" and leave it at that.
> * It sets up arguments about what is and isn't a valid reason for voting no.
Not sure how this is relevant. Once it is voted down it doesn't matter if people felt the reasons were valid or not, it would still be voted down. BTW, I am not proposing that the reasons be tied to individuals. Those would be separated from the vote and displayed anonymously. So who specifically would they argue with? Unless of course you are earnestly concerned about this syndrome: https://imgur.com/gallery/91sn32Q
> * It drives people who might want to vote no away from the project, if > they have to take the extra time to justify their position.
That is why I proposed that they could select from the list of all other things people have provided. If someone does not want to type a reason then just pick from one of the reasons other people already submitted.
> * Shifts the part of the burden for coming up with a plan for how to > get the RFC implemented onto people who are not in favour of the RFC.
Currently it takes herculean effort to get almost anything approved, but it takes effectively zero effort to stifle the hard work someone invests in trying to improve PHP. Is it really just that all their work can be nullified by a simple thumbs down like an emperor deciding the death of a gladiator? (sorry, couldn't resist using that analogy. ;-) BTW, I am thinking of the outrageous amount of work Paul M Jones is putting into Server-Side Request and Response Objects (v2) and fear for him that all his effort will be for naught, and he won't even have a concise list of reasons why it was voted down. The best he will be able to do is infer from the comments in thousands of the messages why people voted down. But he still won't know. Frankly it also feels to me that people seeing that will be a lot more likely to be be driven away from the project than the fact that people who have already gotten the right to vote on the project actually have to write a sentence or two to go along with their votes. I know that if people vote down his work after all his effort I will feel less like contributing for fear of the same outcome. But being told that I am expected to help future people by recording my reason to vote? Hell, I'd feel like I was being granted an honor and I would want to rise to the occassion. It is almost like you are arguing the with great power comes no responsibility (sorry, couldn't resist using allusion, either. ;-)
> * The phrase "this is a terrible idea and I don't have enough crayons > to explain why to you" would be more likely to be used, and wouldn't > add much to the discussion.
In which case we would be no worse off than we are now. However, if even only 1/3 the people who voted actually did provide an intelligible reason then we'd be much farther along to know how and why we should reconsider something in the future, OR NOT. And this might stop a *lot* of wasted effort. Otherwise we continue to be "Those who are ignorant of the past are doomed to repeat it." -Mike P.S. Some modifications to the proposal might include: 1. As Jakob Givoni proposed: a list of predetermined reasons even the first voter course pick from a list , OR they would write their own if they want. 2. Create a two (2) phase process. Phase 1: Solicit people enter a short concise reason to vote "no" or "yes", or to select to join one of the existing reasons. This is basically everyones way to lobby for yes or no. Phase 2: Do the actually vote where people can select a reason from the list to go with their vote, or enter a new one.
  109153
March 19, 2020 20:46 rowan.collins@gmail.com (Rowan Tommins)
Hi Mike,

On 17/03/2020 03:01, Mike Schinkel wrote:
> Currently it takes herculean effort to get almost anything approved, but it takes effectively zero effort to stifle the hard work someone invests in trying to improve PHP. Is it really just that all their work can be nullified by a simple thumbs down like an emperor deciding the death of a gladiator? (sorry, couldn't resist using that analogy. ;-) > > BTW, I am thinking of the outrageous amount of work Paul M Jones is putting into Server-Side Request and Response Objects (v2) and fear for him that all his effort will be for naught, and he won't even have a concise list of reasons why it was voted down. The best he will be able to do is infer from the comments in thousands of the messages why people voted down. But he still won't know.
I wanted to pull this point out from further up the thread, because I think it's a very real concern, but I think it's about something more fundamental than how people vote. Large RFCs, where there's significant implementation work, are essentially software projects. If we ran them that way, we'd all collaborate on some sequence of steps (or several agile iterations, or whatever), such as: * Identifying the problem or aim * Defining the scope * Identifying risks * Agreeing an approach * Initial implementation or prototype * Discovering problems based on the initial work * Testing * Final sign-off for release (this is what RFC votes should be) The problem is that the RFC process only really covers a small fraction of that process, and mostly the later parts. Most of the effort, and crucially most of the decisions, are left to whoever is drafting the RFC. So we end up with a process that feels more like a sales pitch for a shrink-wrapped product: * The RFC author identifies a problem, defines the scope, tries to identify risks, designs a solution, maybe builds an initial implementation or prototype, and starts refining and testing it * This is pitched to the community * There is some negotiation, usually over details rather than substantial rewrites, and often involving the author defending the decisions they've already made * The community decides whether to "buy" the proposal (this is what RFC votes often turn into) Some RFCs are rejected because the community doesn't actually agree on the definition of the problem or scope; capturing that in a reason for voting No might help someone later, but it's already far too late to save the effort of the author. Other RFCs are rejected because the community agrees on the problem, but not the details of the solution; writing that down encourages someone to try again, but repeatedly trying variations until one passes might not be the most efficient process. Just to be clear, I am not blaming authors for bringing ready-made pitches like this - it's what the current process tells them to do. Nor do I have a brilliant proposal that would change it overnight. But I do think we need to spend more time collaborating on ideas, and less on saying "yes" or "no" to each other's ready-made solutions. Regards, -- Rowan Tommins (né Collins) [IMSoP]
  109155
March 19, 2020 21:25 mike@newclarity.net (Mike Schinkel)
> On Mar 19, 2020, at 4:46 PM, Rowan Tommins collins@gmail.com> wrote: > > Hi Mike, > > On 17/03/2020 03:01, Mike Schinkel wrote: >> Currently it takes herculean effort to get almost anything approved, but it takes effectively zero effort to stifle the hard work someone invests in trying to improve PHP. Is it really just that all their work can be nullified by a simple thumbs down like an emperor deciding the death of a gladiator? (sorry, couldn't resist using that analogy. ;-) >> >> BTW, I am thinking of the outrageous amount of work Paul M Jones is putting into Server-Side Request and Response Objects (v2) and fear for him that all his effort will be for naught, and he won't even have a concise list of reasons why it was voted down. The best he will be able to do is infer from the comments in thousands of the messages why people voted down. But he still won't know. > > > I wanted to pull this point out from further up the thread, because I think it's a very real concern, but I think it's about something more fundamental than how people vote. > > Large RFCs, where there's significant implementation work, are essentially software projects. If we ran them that way, we'd all collaborate on some sequence of steps (or several agile iterations, or whatever), such as: > > * Identifying the problem or aim > * Defining the scope > * Identifying risks > * Agreeing an approach > * Initial implementation or prototype > * Discovering problems based on the initial work > * Testing > * Final sign-off for release (this is what RFC votes should be) > > > The problem is that the RFC process only really covers a small fraction of that process, and mostly the later parts. Most of the effort, and crucially most of the decisions, are left to whoever is drafting the RFC. So we end up with a process that feels more like a sales pitch for a shrink-wrapped product: > > * The RFC author identifies a problem, defines the scope, tries to identify risks, designs a solution, maybe builds an initial implementation or prototype, and starts refining and testing it > * This is pitched to the community > * There is some negotiation, usually over details rather than substantial rewrites, and often involving the author defending the decisions they've already made > * The community decides whether to "buy" the proposal (this is what RFC votes often turn into) > > Some RFCs are rejected because the community doesn't actually agree on the definition of the problem or scope; capturing that in a reason for voting No might help someone later, but it's already far too late to save the effort of the author. Other RFCs are rejected because the community agrees on the problem, but not the details of the solution; writing that down encourages someone to try again, but repeatedly trying variations until one passes might not be the most efficient process. > > > Just to be clear, I am not blaming authors for bringing ready-made pitches like this - it's what the current process tells them to do. Nor do I have a brilliant proposal that would change it overnight. But I do think we need to spend more time collaborating on ideas, and less on saying "yes" or "no" to each other's ready-made solutions.
I completely agree that the current process is sub-optimal. It is basically like throwing a gladiator into the lion's den to see if they can survive. If the person writing the RFC is good, they might get an upvote. If they have no experience with writing RFCs and/or can't implement the C code, they are most likely doomed to failure. And PHP collectively is worse off because of this. There is rarely if ever a collaborative process, except for what may happen ad-hoc between people off-list. It would be great if instead of what we have know we could as a community discuss and determine a list of the top five (5) things we collectively want for PHP and put that out as a roadmap for PHP. Maybe even plan a series of remote "conferences" over Zoom to collaborate on such a top five (5) list. Ironically I just today read about Docker publishing their roadmap on Github: - Docker Roadmap: https://github.com/docker/roadmap/projects/1 From there we could solicit a working group for each of the five (5) areas to each go off to a GitHub repo to collaborate and come back with a proposal for implementing in PHP, with the proposal to include a plan for all aspects of the what's needed, not just the idea. From there, we rinse and repeat. Others: Does this sound viable? -Mike P.S. However, even though I completely agree that the current process is suboptimal, that does not mean asking for reasons for voting for posterity should be tabled as the two concerns are orthogonal and some feedback would be more helpful to those looking to understand past votes than no feedback at all. More importantly, adding reasons for voting would only require an update implemented an then an vote to use it, but fixing the process is an open-ended concern that would require herded lots of cats and then to impose a much bigger change, so I wouldn't want perfect to be the enemy of the good here.
  109157
March 19, 2020 22:42 ajf@ajf.me (Andrea Faulds)
Hi everyone,

I want to break off this thing from the “Capturing reasons for votes for 
historical sake?” thread.

Background and problem
======================

Rowan Tommins wrote:
> Hi Mike, > > On 17/03/2020 03:01, Mike Schinkel wrote: >> Currently it takes herculean effort to get almost anything approved, >> but it takes effectively zero effort to stifle the hard work someone >> invests in trying to improve PHP. Is it really just that all their >> work can be nullified by a simple thumbs down like an emperor deciding >> the death of a gladiator? (sorry, couldn't resist using that analogy. ;-) >> >> BTW, I am thinking of the outrageous amount of work Paul M Jones is >> putting into Server-Side Request and Response Objects (v2) and fear >> for him that all his effort will be for naught, and he won't even have >> a concise list of reasons why it was voted down. The best he will be >> able to do is infer from the comments in thousands of the messages why >> people voted down. But he still won't know. > > > I wanted to pull this point out from further up the thread, because I > think it's a very real concern, but I think it's about something more > fundamental than how people vote. > > Large RFCs, where there's significant implementation work, are > essentially software projects. If we ran them that way, we'd all > collaborate on some sequence of steps (or several agile iterations, or > whatever), such as: > > * Identifying the problem or aim > * Defining the scope > * Identifying risks > * Agreeing an approach > * Initial implementation or prototype > * Discovering problems based on the initial work > * Testing > * Final sign-off for release (this is what RFC votes should be) > > > The problem is that the RFC process only really covers a small fraction > of that process, and mostly the later parts. Most of the effort, and > crucially most of the decisions, are left to whoever is drafting the > RFC. So we end up with a process that feels more like a sales pitch for > a shrink-wrapped product: > > * The RFC author identifies a problem, defines the scope, tries to > identify risks, designs a solution, maybe builds an initial > implementation or prototype, and starts refining and testing it > * This is pitched to the community > * There is some negotiation, usually over details rather than > substantial rewrites, and often involving the author defending the > decisions they've already made > * The community decides whether to "buy" the proposal (this is what RFC > votes often turn into) > > Some RFCs are rejected because the community doesn't actually agree on > the definition of the problem or scope; capturing that in a reason for > voting No might help someone later, but it's already far too late to > save the effort of the author. Other RFCs are rejected because the > community agrees on the problem, but not the details of the solution; > writing that down encourages someone to try again, but repeatedly trying > variations until one passes might not be the most efficient process. > > > Just to be clear, I am not blaming authors for bringing ready-made > pitches like this - it's what the current process tells them to do. Nor > do I have a brilliant proposal that would change it overnight. But I do > think we need to spend more time collaborating on ideas, and less on > saying "yes" or "no" to each other's ready-made solutions. > > > Regards, >
I agree with those quoted on this, it matches my own experience from the… I want to say around two dozen?… RFCs I have created or contributed to. For most ideas, they get one real chance at life — someone writes a draft RFC, attempts implementation, puts it out to discussion, perhaps makes minor changes after feedback, and then puts it to one big vote. If the vote passes, the feature is finalised then and there. If the vote fails, it is dead. Of course, it does not have to be this way in every case, but it usually is. There are several angles you could take to criticise this process. Here are just two on my mind: * Ideas that are popular enough to get voted through are not necessarily fully refined. The typical life of an RFC — from draft to final merged patch — might be as little as a few weeks. That is not a long time, and it is probably not long enough to fully evaluate how well the idea has been executed. The first significant code using the feature is probably going to be written long after it's too late to make significant changes. * Ideas that do not get voted through can die an untimely death. Perhaps with further refineements, they would have been acccepted — sometimes there is agreement with an idea in principle, but not in execution. Possible solution ================= I have been thinking about these problems for a while now, and wondering if there is a better way. I am aware that some projects, e.g. the Chromium web browser and the Rust programming language, land things on trunk before they are fully “stable” and keep them gated behind a flag until they are done. I must admit I am not that familiar with it, but it seems like it could be the basis of a better approach. Essentially, what if we don't have one vote on an RFC, but two? The first vote would be the conclusion of a similar RFC process to now, but instead of voting on merging the feature into the next release, it would be on merging it to an official development/unstable branch of PHP (it could just be master but with some sort of flag). Then, over the next few months, interested parties could play around with this bleeding-edge version of PHP, and the feature could be gradually improved. At some point, a second vote can then be held on whether we think the feature in its current state is now polished enough to be finalised (frozen in place with a promise of compatibility) and released in a regular PHP version. The hope would be that this longer period of refinement could: * Give better engagement with the community — they could actually have not only the opportunity (publicly available binaries?), but time to try out and write perhaps real code with proposed features before they are finalised * Reduce rough edges on features that currently successfully pass as RFCs • Give features that are in principle popular but fail in the RFC process due to implementation detail disagreements a second chance — maybe people would be more willing to vote for things if they know it is not going to be released in its current form, and with more experimentation there is a chance of finding an acceptable final version This is just a rough idea though. And sorry for the long email! How does the list feel about this idea? :) Regards, Andrea