Improve visibility of RFC negative feedback

  106377
August 5, 2019 13:33 Danack@basereality.com (Dan Ackroyd)
So, recently there was some discussion about RFCs that have passed
despite there being some strong objections to them.

I think there is a fundamental problem that could be addressed here,
in that the arguments 'for' an RFC have much higher visibility than
the arguments 'against' an RFC. The RFC page, which presents the
arguments for an RFC, is a link that is emailed round and can be
linked to from other sites. The arguments against an RFC are contained
only within an internals email thread. Not only is it much more
difficult for people to discover these, as there are part of an email
thread they might not be written as concisely and as clearly as they
could be.

That sounds like an uneven balance of power which tips the balance
towards RFCs passing. I think we could balance this by doing something
similar to the following:

# Proposal - improve visibility of negative feedback

When someone creates an RFC, near the top of that page they should
create a link to a separate page that will contain negative feedback.
People other that the RFC author are free to put whatever negative
feedback think is appropriate on that 'negative feedback' page.

In the (hopefully) rare cases where the people providing negative
feedback can't agree on how that page should be formatted, they may
create a new negative feedback page and also put a link to it on the
actual RFC page, next to the other 'negative feedback' link.

To indicate agreement with any negative feedback, people are free to
put their own name (not other peoples names) as a signature after the
relevant 'negative feedback' link.

Changing the process to add more visibility to negative feedback with
something like the above wouldn't add much burden to the RFC process,
and could improve the outcomes in some cases.


# Proposal - set a standard text to used when announcing voting

When the vote is opened, and the announcement is made we should have
some standard text to be used to remind people to read the dissenting
feedback. i.e. something like:

 "Voters are reminded to please read all feedback given. If some
feedback hasn't been addressed by the RFC, and has to be added by
someone other than the RFC author, voters should read that dissenting
feedback, and weigh that in whether to vote in favour of the RFC or
not."


## Reasons for not putting the negative feedback on the same page as the RFC

Putting the negative feedback on the same page as the RFC would have
problems with multiple people editing one document and possibly not
agreeing on the formatting.

It could also have problems in making the RFC harder to read, if for
example someone decides to write 20,000 words on why an RFC is bad,
that would make it more difficult to read the RFC.

## Reasons for signing negative feedback

If people similar to the people who were kicked off the internals list
for being too negative want to write some negative feedback, they're
free to do that, and then other people free to not put much value on
that.

If people who have more weight in the community are giving feedback,
and they all object strongly enough to sign their name, then it
behooves everyone to read that feedback.

As a side benefit, I think this would also help the 'discussion only
starting when the voting is opened' problem, as it would allow people
to indicate how they are going to vote.

## Why not just enforce the current RFC rule.

So we apparently already have a rule that says:

https://wiki.php.net/rfc/howto

> Listen to the feedback, and try to answer/resolve all questions. Update > your RFC to document all the issues and discussions. Cover both the > positive and negative arguments. Put the RFC URL into all your replies.
I think one of the reasons no-one is doing that is that it's just too much work. It's also just not possible to summarise some of the arguments in a neutral way, and instead if the RFC author attempts to summarise the negative arguments, it would provoke heated arguments. Changing the responsibility of documenting the negative feedback from the RFC author to people saying that negative feedback, would mean: * it's more likely to be done. * it doesn't add to the work needed to be done by an RFC author. * it avoids heated debate about whether some feedback is relevant or not. In addition, this proposal make it clear and straight-forward for people who wish to leave negative feedback what to do if the RFC author forgets to create the page for negative feedback (create it themselves), rather than trying to force the RFC author to do more work. Any thoughts before I spend the time to write this as an RFC? cheers Dan Ack
  106378
August 5, 2019 18:40 zeev@php.net (Zeev Suraski)
> > > On Mon, Aug 5, 2019 at 4:34 PM Dan Ackroyd <Danack@basereality.com> wrote:
> So, recently there was some discussion about RFCs that have passed > despite there being some strong objections to them. > > I think there is a fundamental problem that could be addressed here, > in that the arguments 'for' an RFC have much higher visibility than > the arguments 'against' an RFC. The RFC page, which presents the > arguments for an RFC, is a link that is emailed round and can be > linked to from other sites. The arguments against an RFC are contained > only within an internals email thread. Not only is it much more > difficult for people to discover these, as there are part of an email > thread they might not be written as concisely and as clearly as they > could be. >
Dan - thanks a lot for your work on this. I think you articulated the problem very well and came up with good solutions.
> # Proposal - improve visibility of negative feedback > > When someone creates an RFC, near the top of that page they should > create a link to a separate page that will contain negative feedback. > People other that the RFC author are free to put whatever negative > feedback think is appropriate on that 'negative feedback' page. >
Do we need it for every RFC? Many (if not most) RFCs don't have inherent 'opposers', so it might be a bit too much to always require the existence of a negative feedback page. Some may go as much as saying it encourages negative feedback... That said - it's a huge improvement over the current situation. Perhaps instead we can say that we allow the creation of such a negative feedback page - whether by the author (preemptively / later on) or by anyone else. That means that if the need arises - there'll be a standard, guaranteed way of doing it. Another thing to think about is where in the RFC this should be positioned. In a lengthy RFC, people may notice it if it's somewhere in the middle. This in itself may become a point of contention. I think that if there's a negative feedback page - there should be a pointer at the RFC header, that will in turn point to a section in the RFC - which will include negative feedback link(s) and signatories. The section itself should probably be at a standard place as well, probably at the end of the RFC above the voting options. Changing the process to add more visibility to negative feedback with
> something like the above wouldn't add much burden to the RFC process,
> and could improve the outcomes in some cases. >
Agreed.
> # Proposal - set a standard text to used when announcing voting > > When the vote is opened, and the announcement is made we should have > some standard text to be used to remind people to read the dissenting > feedback. i.e. something like: > > "Voters are reminded to please read all feedback given. If some > feedback hasn't been addressed by the RFC, and has to be added by > someone other than the RFC author, voters should read that dissenting > feedback, and weigh that in whether to vote in favour of the RFC or > not." >
If we do that - I would consider adding the negative feedback URL to the message. People will quickly learn to ignore any boilerplate... ## Reasons for not putting the negative feedback on the same page as the RFC
> > Putting the negative feedback on the same page as the RFC would have > problems with multiple people editing one document and possibly not > agreeing on the formatting. > > It could also have problems in making the RFC harder to read, if for > example someone decides to write 20,000 words on why an RFC is bad, > that would make it more difficult to read the RFC. >
I agree. I think it should be allowed to include the feedback within the RFC text itself if at least one of the RFC authors explicitly allows it, but not otherwise. It probably makes sense to allow said author with withdraw the agreement, and move to the link approach. It's not quite the same but https://wiki.php.net/rfc/php6 included two opposing views, and was still managed reasonably. This requires good faith and agreement from both sides.
> ## Reasons for signing negative feedback > > If people similar to the people who were kicked off the internals list > for being too negative want to write some negative feedback, they're > free to do that, and then other people free to not put much value on > that. > > If people who have more weight in the community are giving feedback, > and they all object strongly enough to sign their name, then it > behooves everyone to read that feedback. > > As a side benefit, I think this would also help the 'discussion only > starting when the voting is opened' problem, as it would allow people > to indicate how they are going to vote. >
In general, I think the negative feedback should be predominantly a summary of points raised and discussed on internals@. The discussion should continue to happen on internals - with the negative feedback page (if one is needed) being the summary of the points brought up on the list. That's in the same way that positive feedback, ideas for improvement or constructive criticism aren't conveyed by edits to the RFC, but first on internals - and (as needed) are later merged into the RFC by the author (or, from time to time, the author would let others improve it directly instead). If that happens - the folks behind the feedback will be known anyway. That said - I think it's fine to require a signature to make the information easier to access.
> ## Why not just enforce the current RFC rule. > > So we apparently already have a rule that says: > > https://wiki.php.net/rfc/howto > > > Listen to the feedback, and try to answer/resolve all questions. Update > > your RFC to document all the issues and discussions. Cover both the > > positive and negative arguments. Put the RFC URL into all your replies. > > I think one of the reasons no-one is doing that is that it's just too much > work
I think the fact that few people like arguing against their own ideas probably also plays a role. Combined - it's - as you say - a lot of work, but also a lot of not-so-pleasant work, which means it does not get done.
> > It's also just not possible to summarise some of the arguments in a
> neutral way, and instead if the RFC author attempts to summarise the > negative arguments, it would provoke heated arguments. > > Changing the responsibility of documenting the negative feedback from > the RFC author to people saying that negative feedback, would mean: > > * it's more likely to be done. > * it doesn't add to the work needed to be done by an RFC author. > * it avoids heated debate about whether some feedback is relevant or not. > > In addition, this proposal make it clear and straight-forward for > people who wish to leave negative feedback what to do if the RFC > author forgets to create the page for negative feedback (create it > themselves), rather than trying to force the RFC author to do more > work. >
Agreed.
> Any thoughts before I spend the time to write this as an RFC? >
I think one part which may be missing here (emphasis on 'may') is encouraging the RFC author to address the negative feedback - both on the list, and later on in the RFC itself. I.e. - what are their thoughts on it, why they still think it's a good idea, or how they will address the flaws. I don't think we can require it, but I think we should encourage it. Thanks again for your work on this! Zeev
  106386
August 6, 2019 12:33 rowan.collins@gmail.com (Rowan Collins)
On 5 August 2019 14:33:53 BST, Dan Ackroyd <Danack@basereality.com> wrote:
>When someone creates an RFC, near the top of that page they should >create a link to a separate page that will contain negative feedback. >People other that the RFC author are free to put whatever negative >feedback think is appropriate on that 'negative feedback' page.
Hi Dan, I think this is a really sensible idea. The key difference between an RFC and a discussion thread is that it presents a summary or synthesis, which can be much more easily digested than a full discussion. It is also, crucially, editable, so can be reworded or corrected to clarify points; in an email thread, a reader often has to read the first attempt at conveying something, then follow a series of errata down the sub-thread. As Zeev mentioned, it might be enough to have a standard format for this, rather than always requiring it. I also think the term "negative feedback" might be a bit ... negative. Elsewhere in your message, you used "dissenting", which I think captures the essence better. The difference from the main page is not inherently about positive vs negative, but about allowing different voices. Regards, -- Rowan Collins [IMSoP]
  106387
August 6, 2019 12:53 kjarli@gmail.com (Lynn)
On Tue, Aug 6, 2019 at 2:33 PM Rowan Collins collins@gmail.com>
wrote:

> The key difference between an RFC and a discussion thread is that it > presents a summary or synthesis, which can be much more easily digested > than a full discussion. It is also, crucially, editable, so can be reworded > or corrected to clarify points; in an email thread, a reader often has to > read the first attempt at conveying something, then follow a series of > errata down the sub-thread. > > As Zeev mentioned, it might be enough to have a standard format for this, > rather than always requiring it. > > Hi,
Taking the current RFC ( https://wiki.php.net/rfc/counterargument/deprecate_php_short_tags) as example, how do we as reader differentiate between people's counter arguments? When I read those points, I feel like this is something agreed upon by the group as a whole, rather than a person and I know that not everyone might these points as (valid) counter arguments or have different opinions about each. My proposal is to add a name to either a section or argument itself, or perhaps each person could create a page with their counter arguments, meaning the current page becomes an index. This makes it very clear to see who provides which arguments. Regards, Lynn van der Berg
  106388
August 6, 2019 13:19 rowan.collins@gmail.com (Rowan Collins)
On Tue, 6 Aug 2019 at 13:54, Lynn <kjarli@gmail.com> wrote:

> Taking the current RFC ( > https://wiki.php.net/rfc/counterargument/deprecate_php_short_tags) as > example, how do we as reader differentiate between people's counter > arguments? When I read those points, I feel like this is something agreed > upon by the group as a whole, rather than a person and I know that not > everyone might these points as (valid) counter arguments or have different > opinions about each. > > My proposal is to add a name to either a section or argument itself, or > perhaps each person could create a page with their counter arguments, > meaning the current page becomes an index. This makes it very clear to see > who provides which arguments. >
Firstly, I would somewhat question why you need to know who holds an opinion. RFCs, and any dissenting opinions, are not manifestoes in elections, they are information presented so that you can form your own opinion. They should not be read as representative of "the group as a whole", but nor should the author be particularly important in most cases. That said, the current RFC template has an "author" field in the header, and Dan already proposed a convention of contributors "signing" dissenting opinions they agree with. The example you link to says "Author: Zeev Suraski", so I'm not sure what change you're proposing. Regards, -- Rowan Collins [IMSoP]
  106389
August 6, 2019 13:46 kjarli@gmail.com (Lynn)
On Tue, Aug 6, 2019 at 3:19 PM Rowan Collins collins@gmail.com>
wrote:

> > Firstly, I would somewhat question why you need to know who holds an > opinion. RFCs, and any dissenting opinions, are not manifestoes in > elections, they are information presented so that you can form your own > opinion. They should not be read as representative of "the group as a > whole", but nor should the author be particularly important in most cases. > > That said, the current RFC template has an "author" field in the header, > and Dan already proposed a convention of contributors "signing" dissenting > opinions they agree with. The example you link to says "Author: Zeev > Suraski", so I'm not sure what change you're proposing. >
The current setup allows for a single author to write down counter arguments. As the counter arguments seem to primarily be opinionated, I'm interested to see who's opinion it is, as two people can have different opinions on the same subject. If person 1 writes down "option A is bad because of X", person 2 wants to write down that option A is also bad, but not for the reason mentioned by person 2, and person 3 wants thinks the arguments mentioned are actually pros and not cons, I don't see how that is possible right now. That being said, I feel like this should be more of a personal summary per person so everyone can look back what the opinions were and why someone voted yes or no. The mailing list is rather chaotic, even when using an interface such as externals.io, and it's hard to get a summary of opinions, and some people might not write down their opinions in the mailing list and will simply vote. The counter argument page feels like a good idea to see why a certain RFC should not be accepted, but why not extend it to a who votes on what and why page? This would be very useful information for people who are not active on the mailing list or externals.io and leeitheraves information behind of why something wasn't accepted at the time. At times when an RFC is accepted or rejected, a lot of people wonder why this happened. "Why isn't this awesome RFC accepted?", "Why is the RFC accepted while it brings more problems than it fixes?", "Why was a partial solution accepted?". There's valuable information in the mailing list, yet some conversations go off-topic fast or get spammed with mails that should probably dealt with in direct conversations between some people, and this makes it hard to follow.
> so I'm not sure what change you're proposing.
I suppose a page per person allowed to vote where they can summarize their opinion on an RFC, whether it is in favor of, or against. The current setup for counter arguments serves as a nice starting point, as we can read back why people were against an RFC. This means instead of a direct link to a single counter argument page with a single author, a link could be provided to a page where multiple authors post their summary. This could of course also be a page linking to other dedicated pages with one author per page. Regards, Lynn van der Berg
  106390
August 6, 2019 16:01 rowan.collins@gmail.com (Rowan Collins)
On Tue, 6 Aug 2019 at 14:46, Lynn <kjarli@gmail.com> wrote:

> The current setup allows for a single author to write down counter > arguments. As the counter arguments seem to primarily be opinionated, I'm > interested to see who's opinion it is, as two people can have different > opinions on the same subject. If person 1 writes down "option A is bad > because of X", person 2 wants to write down that option A is also bad, but > not for the reason mentioned by person 2, and person 3 wants thinks the > arguments mentioned are actually pros and not cons, I don't see how that is > possible right now. >
Dan's proposal covered that:
> In the (hopefully) rare cases where the people providing negative > feedback can't agree on how that page should be formatted, they may > create a new negative feedback page and also put a link to it on the > actual RFC page, next to the other 'negative feedback' link.
> That being said, I feel like this should be more of a personal summary per > person so everyone can look back what the opinions were and why someone > voted yes or no. >
That sounds like something rather different from what Dan was proposing, and something that's been discussed before: require voters to give reasons for their votes. I won't go into the pros and cons of that right now, but will highlight why it's fundamentally different: A "negative feedback" / "counterargument" / "dissenting opinion" page is by design a *summary*, designed to *reduce* the amount of reading required to understand the additional viewpoint; a page for every user on the list would not achieve that design goal. This also comes back to the distinction between consensus and majority. If no two participants in a discussion can agree on even a summary of what the issues are, then we have a far bigger problem than how many wiki pages to create. Most voting reasons would amount to "I agree with point 4, but disagree with points 3 and 8", rather than needing to restate the whole case. Regards, -- Rowan Collins [IMSoP]
  106399
August 6, 2019 21:54 neclimdul@gmail.com (James Gilliland)
On Mon, Aug 5, 2019 at 8:34 AM Dan Ackroyd <Danack@basereality.com> wrote:

> > # Proposal - improve visibility of negative feedback > > When someone creates an RFC, near the top of that page they should > create a link to a separate page that will contain negative feedback. > People other that the RFC author are free to put whatever negative > feedback think is appropriate on that 'negative feedback' page. >
A kind of in the weeds question but would this require karma or just wiki registration? Current discussions are pretty open on the list but karma would lock down this part of the process a lot for better worse.
  106500
August 9, 2019 16:49 Danack@basereality.com (Dan Ackroyd)
While I think there still might be a seed of a good idea here, I am
not going to pursue it at this time. I believe that it could be
misused to distract from productive conversations.

Before it can be discussed further,  the PHP project needs to have a
Code of Conduct to prevent people who are in sum being detrimental to
the project from being too distracting.

cheers
Dan
Ack

On Tue, 6 Aug 2019 at 22:54, James Gilliland <neclimdul@gmail.com> wrote:
> > > > On Mon, Aug 5, 2019 at 8:34 AM Dan Ackroyd <Danack@basereality.com> wrote: >> >> >> # Proposal - improve visibility of negative feedback >> >> When someone creates an RFC, near the top of that page they should >> create a link to a separate page that will contain negative feedback. >> People other that the RFC author are free to put whatever negative >> feedback think is appropriate on that 'negative feedback' page. > > > A kind of in the weeds question but would this require karma or just wiki registration? Current discussions are pretty open on the list but karma would lock down this part of the process a lot for better worse.