[RFC] Pipe Operator, take 2

  114770
June 7, 2021 19:00 larry@garfieldtech.com ("Larry Garfield")
Hi folks. Me again.

A year ago, I posted an RFC for a pipe operator, |>, aka function concatenation.  At the time, the main thrust of the feedback was "cool, like, but we need partial function application first so that the syntax for callables isn't so crappy."

The PFA RFC is winding down now and is looking quite good, so it's time to revisit pipes.

https://wiki.php.net/rfc/pipe-operator-v2

Nothing radical has changed in the proposal since last year.  I have updated it against the latest master.  I also updated the RFC to use more examples that assume PFA, as the result is legit much nicer.  i also tested it locally with a combined partials-and-pipes branch to make sure they play nicely together, and they do.  (Yay!)  Assuming PFA passes I will include those tests in the pipes branch before this one goes to a vote.

-- 
  Larry Garfield
  larry@garfieldtech.com
  114771
June 7, 2021 19:35 bjorn.x.larsson@telia.com (=?UTF-8?Q?Bj=c3=b6rn_Larsson?=)
Den 2021-06-07 kl. 21:00, skrev Larry Garfield:
> Hi folks. Me again. > > A year ago, I posted an RFC for a pipe operator, |>, aka function concatenation. At the time, the main thrust of the feedback was "cool, like, but we need partial function application first so that the syntax for callables isn't so crappy." > > The PFA RFC is winding down now and is looking quite good, so it's time to revisit pipes. > > https://wiki.php.net/rfc/pipe-operator-v2 > > Nothing radical has changed in the proposal since last year. I have updated it against the latest master. I also updated the RFC to use more examples that assume PFA, as the result is legit much nicer. i also tested it locally with a combined partials-and-pipes branch to make sure they play nicely together, and they do. (Yay!) Assuming PFA passes I will include those tests in the pipes branch before this one goes to a vote. >
Thanks! I have been waiting for this :) r//Björn Larsson
  114773
June 7, 2021 21:00 gen.work@gmail.com (Eugene Leonovich)
On Mon, Jun 7, 2021 at 9:03 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Hi folks. Me again. > > A year ago, I posted an RFC for a pipe operator, |>, aka function > concatenation. At the time, the main thrust of the feedback was "cool, > like, but we need partial function application first so that the syntax for > callables isn't so crappy." > > The PFA RFC is winding down now and is looking quite good, so it's time to > revisit pipes. > > https://wiki.php.net/rfc/pipe-operator-v2 > > Nothing radical has changed in the proposal since last year. I have > updated it against the latest master. I also updated the RFC to use more > examples that assume PFA, as the result is legit much nicer. i also tested > it locally with a combined partials-and-pipes branch to make sure they play > nicely together, and they do. (Yay!) Assuming PFA passes I will include > those tests in the pipes branch before this one goes to a vote. > > FTR, there are several typos in the "Hello World" examples (*strto*t*upper,
htmlent*i*ties*). Also, these examples will not work as written because explode() expects two arguments and will fail if you pass only one: https://3v4l.org/tLO0s. I wonder what the correct version of the pipe example (the one that uses strings as callbacks) would look like, given that you have to pass two arguments for explode()? -- Thank you and best regards, Eugene Leonovich
  114775
June 7, 2021 22:09 larry@garfieldtech.com ("Larry Garfield")
On Mon, Jun 7, 2021, at 4:00 PM, Eugene Leonovich wrote:
> On Mon, Jun 7, 2021 at 9:03 PM Larry Garfield <larry@garfieldtech.com> > wrote: > > > Hi folks. Me again. > > > > A year ago, I posted an RFC for a pipe operator, |>, aka function > > concatenation. At the time, the main thrust of the feedback was "cool, > > like, but we need partial function application first so that the syntax for > > callables isn't so crappy." > > > > The PFA RFC is winding down now and is looking quite good, so it's time to > > revisit pipes. > > > > https://wiki.php.net/rfc/pipe-operator-v2 > > > > Nothing radical has changed in the proposal since last year. I have > > updated it against the latest master. I also updated the RFC to use more > > examples that assume PFA, as the result is legit much nicer. i also tested > > it locally with a combined partials-and-pipes branch to make sure they play > > nicely together, and they do. (Yay!) Assuming PFA passes I will include > > those tests in the pipes branch before this one goes to a vote. > > > > > FTR, there are several typos in the "Hello World" examples (*strto*t*upper, > htmlent*i*ties*). Also, these examples will not work as written because > explode() expects two arguments and will fail if you pass only one: > https://3v4l.org/tLO0s. I wonder what the correct version of the pipe > example (the one that uses strings as callbacks) would look like, given > that you have to pass two arguments for explode()?
Hm. You're right. It used to, but it's been a very long time since explode() allowed an empty split, apparently. I updated the example to use str_split, which is what I'd intended to do in this case. Thanks. If you wanted to explode with a separator in a non-PFA pipes world, you'd need to wrap it in an arrow function. (Hence why a PFA-world pipe is all around better.) --Larry Garfield
  114780
June 8, 2021 10:41 guilliam.xavier@gmail.com (Guilliam Xavier)
Hi,

On Tue, Jun 8, 2021 at 12:09 AM Larry Garfield <larry@garfieldtech.com>
wrote:

> On Mon, Jun 7, 2021, at 4:00 PM, Eugene Leonovich wrote: > > On Mon, Jun 7, 2021 at 9:03 PM Larry Garfield <larry@garfieldtech.com> > > wrote: > > > > > https://wiki.php.net/rfc/pipe-operator-v2 > > > > > FTR, there are several typos in the "Hello World" examples > (*strto*t*upper, > > htmlent*i*ties*). Also, these examples will not work as written because > > explode() expects two arguments and will fail if you pass only one: > > https://3v4l.org/tLO0s. > > Hm. You're right. It used to, but it's been a very long time since > explode() allowed an empty split, apparently. I updated the example to use > str_split, which is what I'd intended to do in this case. Thanks. >
Are you thinking to implode()? Anyway, you forgot to update one `explode(?)` to `str_split(?)`, and also, the first `fn($v) => 'strtoupper'` should be just `'strtoupper'`. About Haskell, rather than (or in addition to) the function composition [not "concatenation"] (.), I would mention the reverse application operator (&): https://hackage.haskell.org/package/base-4.15.0.0/docs/Data-Function.html#v:-38- One thing I note is that even with PFA, some examples still need an arrow function, e.g. the PSR-7 one: ``` ServerRequest::fromGlobals() |> authenticate(?) |> $router->resolveAction(?) |> fn($request) => $request->getAttribute('action')($request) /* ... */; ``` while in Hack you would write it as: ``` ServerRequest::fromGlobals() |> authenticate($$) |> $router->resolveAction($$) |> $$->getAttribute('action')($$) /* ... */; ``` Also, quoting from https://wiki.php.net/rfc/first_class_callable_syntax#partial_function_application : """ Both approaches to the pipe operator have their advantages. The $$ based variant allows using more than plain function calls in each pipeline step (e.g. you could have $$->getName() as a step, something not possible with PFA), and is also trivially free. A PFA-based optimization would entail significant overhead relative to simple function calls, unless special optimization for the pipe operator usage is introduced (which may not be possible, depending on precise semantics). """ Could you (or Nikita) expand a bit on this (esp. the advantages of the PFA approach / disadvantages of Hack's approach)? Regards, -- Guilliam Xavier
  114785
June 8, 2021 14:09 larry@garfieldtech.com ("Larry Garfield")
On Tue, Jun 8, 2021, at 5:41 AM, Guilliam Xavier wrote:

> > Hm. You're right. It used to, but it's been a very long time since > > explode() allowed an empty split, apparently. I updated the example to use > > str_split, which is what I'd intended to do in this case. Thanks. > > > > Are you thinking to implode()? Anyway, you forgot to update one > `explode(?)` to `str_split(?)`, and also, the first `fn($v) => > 'strtoupper'` should be just `'strtoupper'`.
I deliberately made that example extra verbose to show how ugly it can get, but I can shorten it.
> About Haskell, rather than (or in addition to) the function composition > [not "concatenation"] (.), I would mention the reverse application operator > (&): > https://hackage.haskell.org/package/base-4.15.0.0/docs/Data-Function.html#v:-38- > > One thing I note is that even with PFA, some examples still need an arrow > function, e.g. the PSR-7 one: > > ``` > ServerRequest::fromGlobals() > |> authenticate(?) > |> $router->resolveAction(?) > |> fn($request) => $request->getAttribute('action')($request) > /* ... */; > ``` > > while in Hack you would write it as: > > ``` > ServerRequest::fromGlobals() > |> authenticate($$) > |> $router->resolveAction($$) > |> $$->getAttribute('action')($$) > /* ... */; > ``` > > Also, quoting from > https://wiki.php.net/rfc/first_class_callable_syntax#partial_function_application > : > > """ > Both approaches to the pipe operator have their advantages. The $$ based > variant allows using more than plain function calls in each pipeline step > (e.g. you could have $$->getName() as a step, something not possible with > PFA), and is also trivially free. A PFA-based optimization would entail > significant overhead relative to simple function calls, unless special > optimization for the pipe operator usage is introduced (which may not be > possible, depending on precise semantics). > """ > > Could you (or Nikita) expand a bit on this (esp. the advantages of the PFA > approach / disadvantages of Hack's approach)? > > Regards,
It's true PFA doesn't cover every possible RHS of pipes. In practice, I think using the piped value as an object on which to invoke a method is the only major gap. Normally in functional code you would use a lens in that case, which (if I am understanding those correctly; that's roughly at the edge of my functional understanding) is essentially a function call that wraps accessing a property or calling a method so that it feels more functional, and thus pipes cleanly. However, piping with callables has a number of advantages. 1) The implementation is vastly simpler. It's simple enough that even I can manage it, whereas Hack-style would be more considerably implementation work. 2) I would argue it's more flexible. Once you start thinking of callables/functions in a first class way, producing functions on the fly that do what you want becomes natural, and fits better with a pipe-to-callable model. For instance, the comprehension-esque example (which I suspect will be one of the most common use cases of pipes) is far cleaner with a callable, as it can obviate any question about parameter order. Another example I threw together last night is this proof of concept last night, which works when pipes, enums, and partials are combined. I don't think Hack-style would be capable of this, at least not as elegantly. https://gist.github.com/Crell/e484bb27372e7bc93516331a15069f97 (That's essentially a "naked either monad".) 3) I disagree that the overhead of arbitrary callables is "significant." It's there, but at that point you're talking about optimizing function call counts, mostly on partials; unless you're using pipes for absolutely everything, go remove an SQL query or two and you'll get a bigger performance boost. 4) Far more languages have callable pipes. Hack is, as far as I am aware, entirely alone in having pipes be combined with a custom expression syntax rather than just using functions/callables. That isn't conclusive proof of anything, but it's certainly suggestive. I'm going to be moving forward with this approach one way or another (if for point 1 if nothing else). I do believe it is the more flexible, more robust approach, and fits with the general strategy I recommend of small, targeted changes that combine with other small, targeted changes to offer more functionality than either of them alone. That's exactly what we're doing here. --Larry Garfield
  114787
June 8, 2021 14:53 guilliam.xavier@gmail.com (Guilliam Xavier)
On Tue, Jun 8, 2021 at 4:09 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> On Tue, Jun 8, 2021, at 5:41 AM, Guilliam Xavier wrote: > > > you forgot to update one > > `explode(?)` to `str_split(?)`, and also, the first `fn($v) => > > 'strtoupper'` should be just `'strtoupper'`. > > I deliberately made that example extra verbose to show how ugly it can > get, but I can shorten it. >
Extra verbose would have been `fn($v) => strtoupper($v)`, there was obviously a typo (already correct in the second equivalent code fragment). Anyway, I see you fixed it, and also updated the Haskell section :thumbsup:
> > > Also, quoting from > > > https://wiki.php.net/rfc/first_class_callable_syntax#partial_function_application > > : > > > > """ > > Both approaches to the pipe operator have their advantages. The $$ based > > variant allows using more than plain function calls in each pipeline step > > (e.g. you could have $$->getName() as a step, something not possible with > > PFA), and is also trivially free. A PFA-based optimization would entail > > significant overhead relative to simple function calls, unless special > > optimization for the pipe operator usage is introduced (which may not be > > possible, depending on precise semantics). > > """ > > > > Could you (or Nikita) expand a bit on this (esp. the advantages of the > PFA > > approach / disadvantages of Hack's approach)? > > It's true PFA doesn't cover every possible RHS of pipes. In practice, I > think using the piped value as an object on which to invoke a method is the > only major gap. Normally in functional code you would use a lens in that > case, which (if I am understanding those correctly; that's roughly at the > edge of my functional understanding) is essentially a function call that > wraps accessing a property or calling a method so that it feels more > functional, and thus pipes cleanly. > > However, piping with callables has a number of advantages. > > 1) The implementation is vastly simpler. It's simple enough that even I > can manage it, whereas Hack-style would be more considerably implementation > work. > > 2) I would argue it's more flexible. Once you start thinking of > callables/functions in a first class way, producing functions on the fly > that do what you want becomes natural, and fits better with a > pipe-to-callable model. For instance, the comprehension-esque example > (which I suspect will be one of the most common use cases of pipes) is far > cleaner with a callable, as it can obviate any question about parameter > order. > > Another example I threw together last night is this proof of concept last > night, which works when pipes, enums, and partials are combined. I don't > think Hack-style would be capable of this, at least not as elegantly. > > https://gist.github.com/Crell/e484bb27372e7bc93516331a15069f97 > > (That's essentially a "naked either monad".) > > 3) I disagree that the overhead of arbitrary callables is "significant." > It's there, but at that point you're talking about optimizing function call > counts, mostly on partials; unless you're using pipes for absolutely > everything, go remove an SQL query or two and you'll get a bigger > performance boost. > > 4) Far more languages have callable pipes. Hack is, as far as I am aware, > entirely alone in having pipes be combined with a custom expression syntax > rather than just using functions/callables. That isn't conclusive proof of > anything, but it's certainly suggestive. > > I'm going to be moving forward with this approach one way or another (if > for point 1 if nothing else). I do believe it is the more flexible, more > robust approach, and fits with the general strategy I recommend of small, > targeted changes that combine with other small, targeted changes to offer > more functionality than either of them alone. That's exactly what we're > doing here. >
All good points IMHO. Thanks! -- Guilliam Xavier
  114776
June 8, 2021 01:09 mike@newclarity.net (Mike Schinkel)
> On Jun 7, 2021, at 3:00 PM, Larry Garfield <larry@garfieldtech.com> wrote: > > Hi folks. Me again. > > A year ago, I posted an RFC for a pipe operator, |>, aka function concatenation. At the time, the main thrust of the feedback was "cool, like, but we need partial function application first so that the syntax for callables isn't so crappy." > > The PFA RFC is winding down now and is looking quite good, so it's time to revisit pipes. > > https://wiki.php.net/rfc/pipe-operator-v2 > > Nothing radical has changed in the proposal since last year. I have updated it against the latest master. I also updated the RFC to use more examples that assume PFA, as the result is legit much nicer. i also tested it locally with a combined partials-and-pipes branch to make sure they play nicely together, and they do. (Yay!) Assuming PFA passes I will include those tests in the pipes branch before this one goes to a vote.
In general, much nicer with PFA than before. A few questions though, although #1 is probably best answered by Derick Rethans: 1. Will PHP consider a piped sequence a single expression? More specifically, will XDEBUG consider it a single expression, or could XDEBUG be able to set a breakpoint at each pipe assuming they are on separate lines? I ask because if pipes were an indivisible expression from the standpoint of XDEBUG and breakpoints I would not want to see them included in PHP. Alternately if included in PHP I would avoid using them like the plague just like I avoid using fluent interfaces because of their inability to be breakpointed at each step in XDEBUG. 2. Besides `throw` will there be a way to short-circuit evaluation mid-way through the pipes without having to write all of the callables to respect said short-circuiting and just return? For example, could returning an instance of an object that implements a `ShortCircuitPipeInterface` allow the pipe to short-circuit, or some other way to short-circuit and then determine after the pipe if it was short-circuited? -Mike
  114777
June 8, 2021 01:39 larry@garfieldtech.com ("Larry Garfield")
On Mon, Jun 7, 2021, at 8:09 PM, Mike Schinkel wrote:
> > > On Jun 7, 2021, at 3:00 PM, Larry Garfield <larry@garfieldtech.com> wrote: > > > > Hi folks. Me again. > > > > A year ago, I posted an RFC for a pipe operator, |>, aka function concatenation. At the time, the main thrust of the feedback was "cool, like, but we need partial function application first so that the syntax for callables isn't so crappy." > > > > The PFA RFC is winding down now and is looking quite good, so it's time to revisit pipes. > > > > https://wiki.php.net/rfc/pipe-operator-v2 > > > > Nothing radical has changed in the proposal since last year. I have updated it against the latest master. I also updated the RFC to use more examples that assume PFA, as the result is legit much nicer. i also tested it locally with a combined partials-and-pipes branch to make sure they play nicely together, and they do. (Yay!) Assuming PFA passes I will include those tests in the pipes branch before this one goes to a vote. > > In general, much nicer with PFA than before. > > A few questions though, although #1 is probably best answered by Derick Rethans: > > 1. Will PHP consider a piped sequence a single expression? More > specifically, will XDEBUG consider it a single expression, or could > XDEBUG be able to set a breakpoint at each pipe assuming they are on > separate lines? > > I ask because if pipes were an indivisible expression from the > standpoint of XDEBUG and breakpoints I would not want to see them > included in PHP. Alternately if included in PHP I would avoid using > them like the plague just like I avoid using fluent interfaces because > of their inability to be breakpointed at each step in XDEBUG.
$foo |> bar(?) |> baz(?); will produce the same AST as baz(bar($foo)); So whatever Xdebug can do with that AST, it should still be able to do.
> 2. Besides `throw` will there be a way to short-circuit evaluation > mid-way through the pipes without having to write all of the callables > to respect said short-circuiting and just return? > > For example, could returning an instance of an object that implements a > `ShortCircuitPipeInterface` allow the pipe to short-circuit, or some > other way to short-circuit and then determine after the pipe if it was > short-circuited?
What you're describing is effectively monads. (There's that scary word again.) That is not covered here, although I mention it in the future scope section. There's a couple of ways it could be implemented, although you can already do so today in user-space with a method instead of a dedicated operator. The caveat is monads essentially have to be either untyped or use generics, so for now we're stuck with untyped monads. Alternatively, you can wrap each callable in another function that wraps the short-circuiting behavior around it. That's essentially "railroad oriented programming", a term coined by Scott Wlaschin: https://fsharpforfunandprofit.com/rop/ In other words, "I want that, but it comes later, via something more robust." --Larry Garfield
  114778
June 8, 2021 02:44 mike@newclarity.net (Mike Schinkel)
> On Jun 7, 2021, at 9:39 PM, Larry Garfield <larry@garfieldtech.com> wrote: > > On Mon, Jun 7, 2021, at 8:09 PM, Mike Schinkel wrote: >> >>> On Jun 7, 2021, at 3:00 PM, Larry Garfield <larry@garfieldtech.com> wrote: >>> >>> Hi folks. Me again. >>> >>> A year ago, I posted an RFC for a pipe operator, |>, aka function concatenation. At the time, the main thrust of the feedback was "cool, like, but we need partial function application first so that the syntax for callables isn't so crappy." >>> >>> The PFA RFC is winding down now and is looking quite good, so it's time to revisit pipes. >>> >>> https://wiki.php.net/rfc/pipe-operator-v2 >>> >>> Nothing radical has changed in the proposal since last year. I have updated it against the latest master. I also updated the RFC to use more examples that assume PFA, as the result is legit much nicer. i also tested it locally with a combined partials-and-pipes branch to make sure they play nicely together, and they do. (Yay!) Assuming PFA passes I will include those tests in the pipes branch before this one goes to a vote. >> >> In general, much nicer with PFA than before. >> >> A few questions though, although #1 is probably best answered by Derick Rethans: >> >> 1. Will PHP consider a piped sequence a single expression? More >> specifically, will XDEBUG consider it a single expression, or could >> XDEBUG be able to set a breakpoint at each pipe assuming they are on >> separate lines? >> >> I ask because if pipes were an indivisible expression from the >> standpoint of XDEBUG and breakpoints I would not want to see them >> included in PHP. Alternately if included in PHP I would avoid using >> them like the plague just like I avoid using fluent interfaces because >> of their inability to be breakpointed at each step in XDEBUG. > > $foo |> bar(?) |> baz(?); > > will produce the same AST as > > baz(bar($foo)); > > So whatever Xdebug can do with that AST, it should still be able to do.
That is unfortunate. Few people write code or would write code like bazoom(baz(bar(foo($x)))) because the trailing parenthesis make it far from ergonomic. What you are proposing therefore is an ergonomic way to convert a program into a single large expression (when taken to the limit.) Given that your pipe RFC makes practical a style of coding that was previously not practical it begs the question of addressing intermediate steps in the AST to support breakpoints, which also might require XDEBUG to introspect the values on the calling stack. Unfortunately I do not know enough about how the internal representations work to argue that this be considered a must; hopefully Derick Rethans will comment on this. -Mike