An incubation phase for RFCs?

This is only part of a thread. view whole thread
  109157
March 19, 2020 22:42 ajf@ajf.me (Andrea Faulds)
Hi everyone,

I want to break off this thing from the “Capturing reasons for votes for 
historical sake?” thread.

Background and problem
======================

Rowan Tommins wrote:
> Hi Mike, > > On 17/03/2020 03:01, Mike Schinkel wrote: >> Currently it takes herculean effort to get almost anything approved, >> but it takes effectively zero effort to stifle the hard work someone >> invests in trying to improve PHP. Is it really just that all their >> work can be nullified by a simple thumbs down like an emperor deciding >> the death of a gladiator? (sorry, couldn't resist using that analogy. ;-) >> >> BTW, I am thinking of the outrageous amount of work Paul M Jones is >> putting into Server-Side Request and Response Objects (v2) and fear >> for him that all his effort will be for naught, and he won't even have >> a concise list of reasons why it was voted down. The best he will be >> able to do is infer from the comments in thousands of the messages why >> people voted down. But he still won't know. > > > I wanted to pull this point out from further up the thread, because I > think it's a very real concern, but I think it's about something more > fundamental than how people vote. > > Large RFCs, where there's significant implementation work, are > essentially software projects. If we ran them that way, we'd all > collaborate on some sequence of steps (or several agile iterations, or > whatever), such as: > > * Identifying the problem or aim > * Defining the scope > * Identifying risks > * Agreeing an approach > * Initial implementation or prototype > * Discovering problems based on the initial work > * Testing > * Final sign-off for release (this is what RFC votes should be) > > > The problem is that the RFC process only really covers a small fraction > of that process, and mostly the later parts. Most of the effort, and > crucially most of the decisions, are left to whoever is drafting the > RFC. So we end up with a process that feels more like a sales pitch for > a shrink-wrapped product: > > * The RFC author identifies a problem, defines the scope, tries to > identify risks, designs a solution, maybe builds an initial > implementation or prototype, and starts refining and testing it > * This is pitched to the community > * There is some negotiation, usually over details rather than > substantial rewrites, and often involving the author defending the > decisions they've already made > * The community decides whether to "buy" the proposal (this is what RFC > votes often turn into) > > Some RFCs are rejected because the community doesn't actually agree on > the definition of the problem or scope; capturing that in a reason for > voting No might help someone later, but it's already far too late to > save the effort of the author. Other RFCs are rejected because the > community agrees on the problem, but not the details of the solution; > writing that down encourages someone to try again, but repeatedly trying > variations until one passes might not be the most efficient process. > > > Just to be clear, I am not blaming authors for bringing ready-made > pitches like this - it's what the current process tells them to do. Nor > do I have a brilliant proposal that would change it overnight. But I do > think we need to spend more time collaborating on ideas, and less on > saying "yes" or "no" to each other's ready-made solutions. > > > Regards, >
I agree with those quoted on this, it matches my own experience from the… I want to say around two dozen?… RFCs I have created or contributed to. For most ideas, they get one real chance at life — someone writes a draft RFC, attempts implementation, puts it out to discussion, perhaps makes minor changes after feedback, and then puts it to one big vote. If the vote passes, the feature is finalised then and there. If the vote fails, it is dead. Of course, it does not have to be this way in every case, but it usually is. There are several angles you could take to criticise this process. Here are just two on my mind: * Ideas that are popular enough to get voted through are not necessarily fully refined. The typical life of an RFC — from draft to final merged patch — might be as little as a few weeks. That is not a long time, and it is probably not long enough to fully evaluate how well the idea has been executed. The first significant code using the feature is probably going to be written long after it's too late to make significant changes. * Ideas that do not get voted through can die an untimely death. Perhaps with further refineements, they would have been acccepted — sometimes there is agreement with an idea in principle, but not in execution. Possible solution ================= I have been thinking about these problems for a while now, and wondering if there is a better way. I am aware that some projects, e.g. the Chromium web browser and the Rust programming language, land things on trunk before they are fully “stable” and keep them gated behind a flag until they are done. I must admit I am not that familiar with it, but it seems like it could be the basis of a better approach. Essentially, what if we don't have one vote on an RFC, but two? The first vote would be the conclusion of a similar RFC process to now, but instead of voting on merging the feature into the next release, it would be on merging it to an official development/unstable branch of PHP (it could just be master but with some sort of flag). Then, over the next few months, interested parties could play around with this bleeding-edge version of PHP, and the feature could be gradually improved. At some point, a second vote can then be held on whether we think the feature in its current state is now polished enough to be finalised (frozen in place with a promise of compatibility) and released in a regular PHP version. The hope would be that this longer period of refinement could: * Give better engagement with the community — they could actually have not only the opportunity (publicly available binaries?), but time to try out and write perhaps real code with proposed features before they are finalised * Reduce rough edges on features that currently successfully pass as RFCs • Give features that are in principle popular but fail in the RFC process due to implementation detail disagreements a second chance — maybe people would be more willing to vote for things if they know it is not going to be released in its current form, and with more experimentation there is a chance of finding an acceptable final version This is just a rough idea though. And sorry for the long email! How does the list feel about this idea? :) Regards, Andrea