RFC Draft: Comprehensions

  104637
March 10, 2019 21:44 larry@garfieldtech.com ("Larry Garfield")
Hello, peoples.  I know it's been discussed once or twice before on the list, many years ago, but not recently.  I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP:

https://wiki.php.net/rfc/comprehensions

Sara Golemon has written a preliminary patch that is partially complete (see the RFC link for details, it's impressively simple), but unfortunately doesn't have the bandwidth to complete it at this time.  I am therefore looking for collaborators with more knowledge of internals than I (which is almost everyone) to help finish it up.

The syntax proposed is also subject to revision if a terser but still lexer-friendly alternative can be proposed.

At the moment I'm not calling this Proposed yet, as I don't feel comfortable doing so until someone else is on board to finish coding it.  That said, if someone wants to weigh in on the concept for now (hopefully supportively) that's also fine.

Anyone excited enough to help finish the job?

(This is my first RFC collaboration, so if you're going to smack me for goofing something please be gentle about it.)

-- 
  Larry Garfield
  larry@garfieldtech.com
  104638
March 10, 2019 22:25 weirdan@gmail.com (Bruce Weirdan)
> array_map and array_filter combined
This example has array_map and array_filter in wrong order (duplicated once or twice below as well). The RFC proposes to allow multiple `for`s in comprehensions, and really could benefit from an example of such usage.
  104640
March 10, 2019 22:41 netmo.php@gmail.com (Wes)
Hi!

proposed syntax:

> $gen = [for $list as $x if $x % 2 yield $x*2];
> current php + short closures:
$gen = () => foreach($list as $x) if($x % 2) yield $x * 2; 1- parentheses could be optional around control structures, so `if(true){}` could be simply `if true{}` 2- "for" could accept the syntax of "foreach" 3- we need short closures
  104641
March 10, 2019 23:16 bobwei9@hotmail.com (Bob Weinand)
> Am 10.03.2019 um 22:44 schrieb Larry Garfield <larry@garfieldtech.com>: > > Hello, peoples. I know it's been discussed once or twice before on the list, many years ago, but not recently. I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP: > > https://wiki.php.net/rfc/comprehensions > > Sara Golemon has written a preliminary patch that is partially complete (see the RFC link for details, it's impressively simple), but unfortunately doesn't have the bandwidth to complete it at this time. I am therefore looking for collaborators with more knowledge of internals than I (which is almost everyone) to help finish it up. > > The syntax proposed is also subject to revision if a terser but still lexer-friendly alternative can be proposed. > > At the moment I'm not calling this Proposed yet, as I don't feel comfortable doing so until someone else is on board to finish coding it. That said, if someone wants to weigh in on the concept for now (hopefully supportively) that's also fine. > > Anyone excited enough to help finish the job? > > (This is my first RFC collaboration, so if you're going to smack me for goofing something please be gentle about it.) > > -- > Larry Garfield > larry@garfieldtech.com
Hey Larry, First, I like it. (the idea/intention behind of providing a compact way for list comprehensions) Though in general, I think this would be better off implemented with a more flexible short closure; it feels like a special case of these. I.e. that the comprehension syntax can be simply desugared to an immediately invoked short closure. At that point also, it can be a compiler only patch, if we have short closures with the appropriate binding behavior. I guess it should be pretty easy to build that on top of my existing short closures patch. Regarding the syntax, I'd not conflate for with foreach, the distinction is built into PHP and as such should remain. We also might have trivial argument-less short closures written as "{ stmt }" simply, having the comprehension written as $gen = {foreach($list as $x) if($x % 2) yield $x * 2;}(); (note the trailing "()" function call - if we desire to, we might make the semicolon at the end of that single-statement closure optional.) Bob
  104644
March 11, 2019 04:29 larry@garfieldtech.com ("Larry Garfield")
On Sun, Mar 10, 2019, at 7:16 PM, Bob Weinand wrote:
> > Am 10.03.2019 um 22:44 schrieb Larry Garfield <larry@garfieldtech.com>: > > > > Hello, peoples. I know it's been discussed once or twice before on the list, many years ago, but not recently. I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP: > > > > https://wiki.php.net/rfc/comprehensions > > > > Sara Golemon has written a preliminary patch that is partially complete (see the RFC link for details, it's impressively simple), but unfortunately doesn't have the bandwidth to complete it at this time. I am therefore looking for collaborators with more knowledge of internals than I (which is almost everyone) to help finish it up. > > > > The syntax proposed is also subject to revision if a terser but still lexer-friendly alternative can be proposed. > > > > At the moment I'm not calling this Proposed yet, as I don't feel comfortable doing so until someone else is on board to finish coding it. That said, if someone wants to weigh in on the concept for now (hopefully supportively) that's also fine. > > > > Anyone excited enough to help finish the job? > > > > (This is my first RFC collaboration, so if you're going to smack me for goofing something please be gentle about it.) > > > > -- > > Larry Garfield > > larry@garfieldtech.com > > Hey Larry, > First, I like it. (the idea/intention behind of providing a compact way > for list comprehensions) > > Though in general, I think this would be better off implemented with a > more flexible short closure; it feels like a special case of these. > I.e. that the comprehension syntax can be simply desugared to an > immediately invoked short closure. At that point also, it can be a > compiler only patch, if we have short closures with the appropriate > binding behavior. > I guess it should be pretty easy to build that on top of my existing > short closures patch. > > Regarding the syntax, I'd not conflate for with foreach, the > distinction is built into PHP and as such should remain. > We also might have trivial argument-less short closures written as "{ > stmt }" simply, having the comprehension written as $gen = > {foreach($list as $x) if($x % 2) yield $x * 2;}(); (note the trailing > "()" function call - if we desire to, we might make the semicolon at > the end of that single-statement closure optional.) > > Bob
Replying to a couple of people at once here: 1) I am 100% in favor of adopting short closures, the less syntax ceremony in it the better. Bob, if there's a logistical/non-code way I can help make that happen let me know, because we wants it, precious. 2) That said, I'm not sure it would fully obviate the need for a generator-specific syntax. Even if the examples from Bob and Wes are shorter than what I had envisioned for a short-closure version (nice!), they're still notably longer than a purpose-built generator version. Too, and I acknowledge this is a highly subjective statement, they have a lot of syntax salad. ";)();" at the end is a superb thing to mistype more often than you get it right. If there were some way to reduce that then I might be convinced but I'd have to see it first. (And that means we need short closures still, which we don't have yet.) 3) I'd still be in favor of adding a type check to comprehensions, too; I don't know how or if that would be possible if they were "just" a fancy use of short-closures. 4) Regarding the specific syntax proposed, it's honestly more verbose than I'd like as is. Sara and I tried putting the expression first, as in Python, but that made the lexer very unhappy since "[$foo ... ]" is already legal syntax for an array of one iterable. The yield keyword is there to help the parser (and reader) tell where the expression begins since, being after the conditional, it needs some kind of separator. If anything, I'd be interested in considering entirely different syntaxes borrowed from other languages (see the linked Wikipedia page) that would involve even fewer keywords, not using longer keywords. That's why, for instance, it uses for instead of foreach. 5) Regarding Javascript comprehensions, well drat. I didn't realize it had been withdrawn. That's disappointing. I'll adjust the RFC at some point soonish (it's after midnight here right now) to not mention them. 6) array_map/array_filter order reversed: Bruce, I think you're right. It took a while for me to grok that it is. Which IMO is even more reason we want some more comprehensible and compact syntax for that use case, as it's surprisingly common. :-) 7) Bruce, there is an example of multiple foreach()s. See the "Iterate a 2D array" section in the long code block. 8) To George: yes, since comprehensions as envisioned would work on any iterable, and generators are iterables, and comprehensions are generators, passing the generator produced by one comprehension to another comprehension is 100% legit. Strings are not currently iterable, though, so they wouldn't work. Making strings iterable is an entirely separate discussion so let's not go there, but were that to ever happen then they would "just work" in comprehensions as well. 9) To Alex: Because the syntax feels more akin to Generators at the user-level, that's how I described them. You're writing code that yields items one by one. That said, if under the hood it fits the code better to have it nominally return an \Iterator rather than \Generator I don't much care either way. That's an implementation detail that I leave to whoever I manage to bribe to finish the code. :-) --Larry Garfield
  104642
March 10, 2019 23:47 george.banyard@gmail.com ("G. P. B.")
On Sun, 10 Mar 2019 at 22:45, Larry Garfield <larry@garfieldtech.com> wrote:

> Hello, peoples. I know it's been discussed once or twice before on the > list, many years ago, but not recently. I therefore feel OK putting forth > the following draft proposal for Comprehensions, aka "compact generators", > in PHP: > > https://wiki.php.net/rfc/comprehensions > > Sara Golemon has written a preliminary patch that is partially complete > (see the RFC link for details, it's impressively simple), but unfortunately > doesn't have the bandwidth to complete it at this time. I am therefore > looking for collaborators with more knowledge of internals than I (which is > almost everyone) to help finish it up. > > The syntax proposed is also subject to revision if a terser but still > lexer-friendly alternative can be proposed. > > At the moment I'm not calling this Proposed yet, as I don't feel > comfortable doing so until someone else is on board to finish coding it. > That said, if someone wants to weigh in on the concept for now (hopefully > supportively) that's also fine. > > Anyone excited enough to help finish the job? > > (This is my first RFC collaboration, so if you're going to smack me for > goofing something please be gentle about it.) > > -- > Larry Garfield > larry@garfieldtech.com > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php >
Hello Larry, I do like the proposal of adding comprehensions to PHP, and I do agree with you that short closures do not replace a short closure synstax (as even with them it is still not ideal). However I have a couple of complaints, JS comprehensions have not become a standard and have been removed from my understanding [1][2]. Secondly I don't really understand the argument for using `for` instead of ` foreach` Sure it saves 4 chars but the PHP version compared to the Python uses `yield ` which already lengthen it, I do understand why it is there because it is a generator. But Python using `for` makes more sense, imho, because its syntax is `expression for item in iterator (if condition)`. Now maybe what I am going to say is total BS because I don't know *how* the engine works, but wouldn't also use foreach make it easier for the lexer as it already know the `as` keyword? Maybe if we want to mimic the Python syntax why not use a syntax like: `(yield expression) foreach iterator as item (if condition)` I also suppose that comprehension chaining is possible because it returns a generator. Small nitpick but shouldn't the `current` method link to the Generator->current instead of the function `current` which only works with arrays? Also would it be possible to use a string as an iterable expression? Because currently it's not possible to foreach a string (but that's maybe more the scope of another RFC). Best regards George P. Banyard [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Array_comprehensions [2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Generator_comprehensions
  104643
March 11, 2019 04:07 drealecs@gmail.com (=?UTF-8?Q?Alexandru_P=C4=83tr=C4=83nescu?=)
Hi Larry,

I'm still digesting the syntax and people have already said about it the
most important things.

I mostly wanted to mention that I think comprehensions should return
\Iterator instead of \Generator. There is nothing that stopped us from
having lazy iterators, even before generators were available.
IMO, all points in *Why generators?* section are valid with \Iterator and
you also noted that send() method is kind of useless.

Just to make it clear, I agree that behind the scene it can use the same
logic of a \Generator but I think userland type should be \Iterator.

For example, if we look at this two important functions widely used for
processing lazy/infinite iterators, the return type is \Iterator
function map(callable $function, iterable $iterable): \Iterator
https://github.com/nikic/iter/blob/master/src/iter.php#L79
function filter(callable $predicate, iterable $iterable): \Iterator
https://github.com/nikic/iter/blob/master/src/iter.php#L203

Regards,
Alex

On Sun, Mar 10, 2019 at 11:45 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Hello, peoples. I know it's been discussed once or twice before on the > list, many years ago, but not recently. I therefore feel OK putting forth > the following draft proposal for Comprehensions, aka "compact generators", > in PHP: > > https://wiki.php.net/rfc/comprehensions > > Sara Golemon has written a preliminary patch that is partially complete > (see the RFC link for details, it's impressively simple), but unfortunately > doesn't have the bandwidth to complete it at this time. I am therefore > looking for collaborators with more knowledge of internals than I (which is > almost everyone) to help finish it up. > > The syntax proposed is also subject to revision if a terser but still > lexer-friendly alternative can be proposed. > > At the moment I'm not calling this Proposed yet, as I don't feel > comfortable doing so until someone else is on board to finish coding it. > That said, if someone wants to weigh in on the concept for now (hopefully > supportively) that's also fine. > > Anyone excited enough to help finish the job? > > (This is my first RFC collaboration, so if you're going to smack me for > goofing something please be gentle about it.) > > -- > Larry Garfield > larry@garfieldtech.com > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
  104650
March 11, 2019 12:16 nikita.ppv@gmail.com (Nikita Popov)
On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Hello, peoples. I know it's been discussed once or twice before on the > list, many years ago, but not recently. I therefore feel OK putting forth > the following draft proposal for Comprehensions, aka "compact generators", > in PHP: > > https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions > <https://wiki.php.net/rfc/comprehensions> > > Sara Golemon has written a preliminary patch that is partially complete > (see the RFC link for details, it's impressively simple), but unfortunately > doesn't have the bandwidth to complete it at this time. I am therefore > looking for collaborators with more knowledge of internals than I (which is > almost everyone) to help finish it up. > > The syntax proposed is also subject to revision if a terser but still > lexer-friendly alternative can be proposed. > > At the moment I'm not calling this Proposed yet, as I don't feel > comfortable doing so until someone else is on board to finish coding it. > That said, if someone wants to weigh in on the concept for now (hopefully > supportively) that's also fine. > > Anyone excited enough to help finish the job? > > (This is my first RFC collaboration, so if you're going to smack me for > goofing something please be gentle about it.) >
I've brought up a similar proposal a few years ago, probably around the time generators were first introduced: https://externals.io/message/61021 I think my main point of feedback would be to stick closer to existing PHP syntax, even if it costs us some brevity. I would prefer $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2]; over $gen = [for $list as $x if $x % 2 yield $x * 2]; The latter is nice in languages that generally use "for" for this purpose and generally allow omitting parentheses, but I don't think it's good to introduce this kind of syntax inconsistency in one place. Similarly, I found the ability to omit the "yield" expression somewhat confusing. Especially once you get to nesting $gen = [for $a as $b for $b as $c]; the behavior becomes non-obvious. Also wondering how this part interacts with keys, do you decide whether or not to yield the keys based on whether they are part of the for expression? $gen = [for $list as $v]; // drops keys $gen = [for $list as $k => $v]; // keeps keys Finally, Python makes a distinction between list comprehensions using [] and generator expressions using (). This proposal effectively corresponds to generator expressions, but uses the [] syntax. I'm wondering if that will cause confusion. Regards, Nikita
  104663
March 12, 2019 00:07 larry@garfieldtech.com ("Larry Garfield")
On Mon, Mar 11, 2019, at 8:16 AM, Nikita Popov wrote:
> On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield <larry@garfieldtech.com> > wrote: > > > Hello, peoples. I know it's been discussed once or twice before on the > > list, many years ago, but not recently. I therefore feel OK putting forth > > the following draft proposal for Comprehensions, aka "compact generators", > > in PHP: > > > > https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions > > <https://wiki.php.net/rfc/comprehensions> > > > > Sara Golemon has written a preliminary patch that is partially complete > > (see the RFC link for details, it's impressively simple), but unfortunately > > doesn't have the bandwidth to complete it at this time. I am therefore > > looking for collaborators with more knowledge of internals than I (which is > > almost everyone) to help finish it up. > > > > The syntax proposed is also subject to revision if a terser but still > > lexer-friendly alternative can be proposed. > > > > At the moment I'm not calling this Proposed yet, as I don't feel > > comfortable doing so until someone else is on board to finish coding it. > > That said, if someone wants to weigh in on the concept for now (hopefully > > supportively) that's also fine. > > > > Anyone excited enough to help finish the job? > > > > (This is my first RFC collaboration, so if you're going to smack me for > > goofing something please be gentle about it.) > > > > I've brought up a similar proposal a few years ago, probably around the > time generators were first introduced: https://externals.io/message/61021
Ah, I knew I'd seen it on the list once before. Thanks.
> I think my main point of feedback would be to stick closer to existing PHP > syntax, even if it costs us some brevity. I would prefer > > $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2]; > > over > > $gen = [for $list as $x if $x % 2 yield $x * 2]; > > The latter is nice in languages that generally use "for" for this purpose > and generally allow omitting parentheses, but I don't think it's good to > introduce this kind of syntax inconsistency in one place. > > Similarly, I found the ability to omit the "yield" expression somewhat > confusing. Especially once you get to nesting > > $gen = [for $a as $b for $b as $c];
In that case you'd get a list of $c returned (implicit yield $c), although I can see where that's non-obvious. OTOH, requiring the yield statement every time in the majority case seems like excessive verbosity and ceremony.
> the behavior becomes non-obvious. Also wondering how this part interacts > with keys, do you decide whether or not to yield the keys based on whether > they are part of the for expression? > > $gen = [for $list as $v]; // drops keys > $gen = [for $list as $k => $v]; // keeps keys
Yes, that's the intent. No yield means implicit "key and value"; if key is omitted then it's implicitly just 0-based numbers.
> Finally, Python makes a distinction between list comprehensions using [] > and generator expressions using (). This proposal effectively corresponds > to generator expressions, but uses the [] syntax. I'm wondering if that > will cause confusion. > > Regards, > Nikita
I don't think so. For one, although we started from Python syntax the result is decidedly not Python, so I wouldn't expect people to assume the same symbols mean the same things. For another, Python uses [] for a list-producing comprehension, and {} for a dictionary-producting comprehension. Whether those return generators or not is a version question, not a syntax question. Since PHP doesn't differentiate between lists and dictionaries to begin with there's no need for different syntax. Our "dictionaries" already use []. --Larry Garfield
  104677
March 12, 2019 15:24 bjorn.x.larsson@telia.com (=?UTF-8?Q?Bj=c3=b6rn_Larsson?=)
Den 2019-03-11 kl. 13:16, skrev Nikita Popov:

> On Sun, Mar 10, 2019 at 10:45 PM Larry Garfield <larry@garfieldtech.com> > wrote: > >> Hello, peoples. I know it's been discussed once or twice before on the >> list, many years ago, but not recently. I therefore feel OK putting forth >> the following draft proposal for Comprehensions, aka "compact generators", >> in PHP: >> >> https://wiki.php.net/rfc/cohttps://externals.io/message/61021mprehensions >> <https://wiki.php.net/rfc/comprehensions> >> >> Sara Golemon has written a preliminary patch that is partially complete >> (see the RFC link for details, it's impressively simple), but unfortunately >> doesn't have the bandwidth to complete it at this time. I am therefore >> looking for collaborators with more knowledge of internals than I (which is >> almost everyone) to help finish it up. >> >> The syntax proposed is also subject to revision if a terser but still >> lexer-friendly alternative can be proposed. >> >> At the moment I'm not calling this Proposed yet, as I don't feel >> comfortable doing so until someone else is on board to finish coding it. >> That said, if someone wants to weigh in on the concept for now (hopefully >> supportively) that's also fine. >> >> Anyone excited enough to help finish the job? >> >> (This is my first RFC collaboration, so if you're going to smack me for >> goofing something please be gentle about it.) >> > I've brought up a similar proposal a few years ago, probably around the > time generators were first introduced: https://externals.io/message/61021 > > I think my main point of feedback would be to stick closer to existing PHP > syntax, even if it costs us some brevity. I would prefer > > $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2]; > > over > > $gen = [for $list as $x if $x % 2 yield $x * 2]; > > The latter is nice in languages that generally use "for" for this purpose > and generally allow omitting parentheses, but I don't think it's good to > introduce this kind of syntax inconsistency in one place. > > Similarly, I found the ability to omit the "yield" expression somewhat > confusing. Especially once you get to nesting > > $gen = [for $a as $b for $b as $c]; > > the behavior becomes non-obvious. Also wondering how this part interacts > with keys, do you decide whether or not to yield the keys based on whether > they are part of the for expression? > > $gen = [for $list as $v]; // drops keys > $gen = [for $list as $k => $v]; // keeps keys > > Finally, Python makes a distinction between list comprehensions using [] > and generator expressions using (). This proposal effectively corresponds > to generator expressions, but uses the [] syntax. I'm wondering if that > will cause confusion. > > Regards, > Nikita
As a userland developer I found the example parsing large log files from the proposal a few years back a good use case for this feature. Besides that, I recall that for arrow functions a few different syntax proposals were on the table. Wonder which one of those plays best with this proposal? Maybe that should be taken into consideration when looking at this RFC. r//Björn L
  104698
March 13, 2019 20:42 smalyshev@gmail.com (Stanislav Malyshev)
Hi!

> I think my main point of feedback would be to stick closer to existing PHP > syntax, even if it costs us some brevity. I would prefer > > $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2]; > > over > > $gen = [for $list as $x if $x % 2 yield $x * 2];
Agree here. There's a principle of least surprise - "for" in PHP looks the certain way, and sequence iteration looks certain way, and it's better to work within those paradigms than import different way of doing the same from Python. Especially given that for/foreach look pretty close, and remembering in which context "for" is ok for sequences and in which not is going to be very annoying. It's much easier to remember "for is always for(x;y;z), foreach is always foreach(x as y=>z)". Also agree about parentheses - reading a complex expression without separators would be annoying. I know python loves to do this but even in an IDE with highlighter it's not always easy to quickly figure out which part belongs where. Delimiting it would make it easier IMHO, and also allows again to carry over the intuition of how foreach() and if() work from already known constructs.
> Finally, Python makes a distinction between list comprehensions using [] > and generator expressions using (). This proposal effectively corresponds > to generator expressions, but uses the [] syntax. I'm wondering if that > will cause confusion.
Do we need this distinction for anything useful? -- Stas Malyshev smalyshev@gmail.com
  104778
March 17, 2019 19:46 ajf@ajf.me (Andrea Faulds)
Hi,

Stanislav Malyshev wrote:
> >> Finally, Python makes a distinction between list comprehensions using [] >> and generator expressions using (). This proposal effectively corresponds >> to generator expressions, but uses the [] syntax. I'm wondering if that >> will cause confusion. > > Do we need this distinction for anything useful? >
In Python, the difference is that []-syntax gives you a list (pre-comupted), whereas without the [] you get a generator (generate-on-demand). This distinction is important because the generator might be iterating over something non-repeatable (e.g. another generator), or have some destructive or mutating effect. You also might not want to take the performance penalty of computing it every time you iterate over it. Why not apply the same approach to PHP? There is iterator_to_array() to convert a generator to an array, so we may not need both syntaxes. However, I think using [] for something that is *not an array* is counter-intuitive. Thanks, Andrea
  104664
March 12, 2019 08:54 kalle@php.net (Kalle Sommer Nielsen)
Hi

Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield <larry@garfieldtech.com>:
> > Hello, peoples. I know it's been discussed once or twice before on the list, many years ago, but not recently. I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP: > > https://wiki.php.net/rfc/comprehensions
While I think believe the RFC has great intentions, I am not a fan by any means to introduce convoluted syntaxes like this into PHP, I think it is rather hard to read vs the gain of saving a few key strokes. I think that both the shorthand closures and this makes PHP code potentially even harder to read and I would prefer we did not add such essential sugar where you need to understand the semantics very well to understand what a piece of foreign code may do, in what order (like the no yield example given). One more thing that kinda "annoys" me / what I base my above statement on is; when reading some of the example is that the code examples given, as in the code inside a Comprehension does not look very PHP-ish besides the $-sigil and I do not like the idea of having "two" syntaxes for PHP where one is only available in a very narrow-, specific context and I feel it would be prone to potential errors. Another thing that has been mentioned by multiple people now is the for-vs-foreach; If it is a foreach, which you mention in the RFC then it should be so, not for. A little background on where I am coming from here; I think it is great that we look at other languages and technologies when designing PHP and adding more features the language, however while I do believe it is an important point to keep cross language consistency, I do think that consistency with PHP itself is more important, which I believe too, is the reason for the expression for using foreach in this context comes from (not only from my PoV). -- regards, Kalle Sommer Nielsen kalle@php.net
  104674
March 12, 2019 13:49 chasepeeler@gmail.com (Chase Peeler)
On Tue, Mar 12, 2019 at 4:55 AM Kalle Sommer Nielsen <kalle@php.net> wrote:

> Hi > > Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield < > larry@garfieldtech.com>: > > > > Hello, peoples. I know it's been discussed once or twice before on the > list, many years ago, but not recently. I therefore feel OK putting forth > the following draft proposal for Comprehensions, aka "compact generators", > in PHP: > > > > https://wiki.php.net/rfc/comprehensions > > While I think believe the RFC has great intentions, I am not a fan by > any means to introduce convoluted syntaxes like this into PHP, I think > it is rather hard to read vs the gain of saving a few key strokes. I > think that both the shorthand closures and this makes PHP code > potentially even harder to read and I would prefer we did not add such > essential sugar where you need to understand the semantics very well > to understand what a piece of foreign code may do, in what order (like > the no yield example given). > > One more thing that kinda "annoys" me / what I base my above statement > on is; when reading some of the example is that the code examples > given, as in the code inside a Comprehension does not look very > PHP-ish besides the $-sigil and I do not like the idea of having "two" > syntaxes for PHP where one is only available in a very narrow-, > specific context and I feel it would be prone to potential errors. > > Everything looks weird and "non-phpish" when it's new. OO constructs weren't PHP-ish at first, because PHP didn't originally support OO. Imagine
"foreach" didn't exist in PHP and we were still using while(list($key,$value) = each($arr)) syntax. I agree that some of the examples are a bit hard to read and understand - but that's only because we haven't been exposed to them. This isn't meant as an endorsement of the proposal. I just don't think the fact that it adds something different than what we are used to is a good justification for not supporting it. As a non-voting userland developer, I'm personally ambivalent. They seem cool, and I'm sure I'll use them, but it won't be a big deal if they aren't added.
> Another thing that has been mentioned by multiple people now is the > for-vs-foreach; If it is a foreach, which you mention in the RFC then > it should be so, not for. A little background on where I am coming > from here; I think it is great that we look at other languages and > technologies when designing PHP and adding more features the language, > however while I do believe it is an important point to keep cross > language consistency, I do think that consistency with PHP itself is > more important, which I believe too, is the reason for the expression > for using foreach in this context comes from (not only from my PoV). > > I was originally going to say I agreed with this position, but, after thinking about it, I've changed my mind. A "for" loop is just a method of
specifying the keys that you are iterating over. Since you aren't specifying the keys inside a comprehension, the traditional "for" loop doesn't really exist. I'd suggest supporting both keywords, though.
> -- > regards, > > Kalle Sommer Nielsen > kalle@php.net > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > > -- -- Chase
chasepeeler@gmail.com
  104675
March 12, 2019 14:12 kalle@php.net (Kalle Sommer Nielsen)
Hi

Den tir. 12. mar. 2019 kl. 15.49 skrev Chase Peeler <chasepeeler@gmail.com>:
> Everything looks weird and "non-phpish" when it's new. OO constructs weren't PHP-ish at first, because PHP didn't originally support OO. Imagine "foreach" didn't exist in PHP and we were still using while(list($key,$value) = each($arr)) syntax. > > I agree that some of the examples are a bit hard to read and understand - but that's only because we haven't been exposed to them.
What I mean here is that the lack of things like the parentheses makes the code generally harder to read, imagine this for large comprehensions as userland would certainly do things like that and you end up with monstrosities of code that is less verbose than the counter examples given in the RFC. I mean sure I could just rewrite a comprehension with a lot of whitespace, but it kinda seem to defeat the purpose of short circuited generators and I might as well just write it the more verbose way. I like the idea, just not the syntax because it has the potential (along with short closures) to be annoying to read in the long run. -- regards, Kalle Sommer Nielsen kalle@php.net
  104681
March 12, 2019 16:36 chasepeeler@gmail.com (Chase Peeler)
On Tue, Mar 12, 2019 at 10:12 AM Kalle Sommer Nielsen <kalle@php.net> wrote:

> Hi > > Den tir. 12. mar. 2019 kl. 15.49 skrev Chase Peeler <chasepeeler@gmail.com > >: > > Everything looks weird and "non-phpish" when it's new. OO constructs > weren't PHP-ish at first, because PHP didn't originally support OO. Imagine > "foreach" didn't exist in PHP and we were still using > while(list($key,$value) = each($arr)) syntax. > > > > I agree that some of the examples are a bit hard to read and understand > - but that's only because we haven't been exposed to them. > > What I mean here is that the lack of things like the parentheses makes > the code generally harder to read, imagine this for large > comprehensions as userland would certainly do things like that and you > end up with monstrosities of code that is less verbose than the > counter examples given in the RFC. I mean sure I could just rewrite a > comprehension with a lot of whitespace, but it kinda seem to defeat > the purpose of short circuited generators and I might as well just > write it the more verbose way. > > I've never liked "developers might use it wrong" as reason to not implement something, especially if hard to read/complex code is the worst
impact to improper use.
> I like the idea, just not the syntax because it has the potential > (along with short closures) to be annoying to read in the long run. >
I understand that you are talking about the syntax (which others have said is still up for discussion) and not the feature itself. I still wanted to make the comment above more a general response to that line of thinking.
> > -- > regards, > > Kalle Sommer Nielsen > kalle@php.net > --
-- Chase chasepeeler@gmail.com
  104702
March 13, 2019 21:26 kalle@php.net (Kalle Sommer Nielsen)
Hi

Den tir. 12. mar. 2019 kl. 18.36 skrev Chase Peeler <chasepeeler@gmail.com>:
> I've never liked "developers might use it wrong" as reason to not implement something, especially if hard to read/complex code is the worst impact to improper use. Me neither to be fair, but I think it is a fair consideration to have
when designing syntax. I agree with Stas reply in this thread regarding this.
> I understand that you are talking about the syntax (which others have said is still up for discussion) and not the feature itself. I still wanted to make the comment above more a general response to that line of thinking. I like the feature as it is, and if the syntax is right for it then I
would most likely vote for it if it comes to that state. -- regards, Kalle Sommer Nielsen kalle@php.net
  104678
March 12, 2019 15:27 php@hristov.com (Andrey Hristov)
Hi,
the real fun starts when people start to put list comprehension in a 
list comprehension in a list comprehension. And the result is one-liners 
that are pretty dense and write once never read.

The problem during software development are not the key strokes, it's 
the time which later has to be invested to understand the code that lack 
the additional keystrokes. Of course one can learn to understand dense 
stuff. One can also learn to read German words with a length of 20 and 
up. It ain't that hard and needs practice. Is it worth when non-compound 
words also work (hint: German words can be written alternatively with 
hypens and CamelCaps, which really improves the readability).

Best,
Andrey


On 12.03.19 г. 15:49 ч., Chase Peeler wrote:
> On Tue, Mar 12, 2019 at 4:55 AM Kalle Sommer Nielsen <kalle@php.net> wrote: > >> Hi >> >> Den søn. 10. mar. 2019 kl. 23.45 skrev Larry Garfield < >> larry@garfieldtech.com>: >>> >>> Hello, peoples. I know it's been discussed once or twice before on the >> list, many years ago, but not recently. I therefore feel OK putting forth >> the following draft proposal for Comprehensions, aka "compact generators", >> in PHP: >>> >>> https://wiki.php.net/rfc/comprehensions >> >> While I think believe the RFC has great intentions, I am not a fan by >> any means to introduce convoluted syntaxes like this into PHP, I think >> it is rather hard to read vs the gain of saving a few key strokes. I >> think that both the shorthand closures and this makes PHP code >> potentially even harder to read and I would prefer we did not add such >> essential sugar where you need to understand the semantics very well >> to understand what a piece of foreign code may do, in what order (like >> the no yield example given). >> >> One more thing that kinda "annoys" me / what I base my above statement >> on is; when reading some of the example is that the code examples >> given, as in the code inside a Comprehension does not look very >> PHP-ish besides the $-sigil and I do not like the idea of having "two" >> syntaxes for PHP where one is only available in a very narrow-, >> specific context and I feel it would be prone to potential errors. >> >> Everything looks weird and "non-phpish" when it's new. OO constructs > weren't PHP-ish at first, because PHP didn't originally support OO. Imagine > "foreach" didn't exist in PHP and we were still using > while(list($key,$value) = each($arr)) syntax. > > I agree that some of the examples are a bit hard to read and understand - > but that's only because we haven't been exposed to them. > > This isn't meant as an endorsement of the proposal. I just don't think the > fact that it adds something different than what we are used to is a good > justification for not supporting it. As a non-voting userland developer, > I'm personally ambivalent. They seem cool, and I'm sure I'll use them, but > it won't be a big deal if they aren't added. > > >> Another thing that has been mentioned by multiple people now is the >> for-vs-foreach; If it is a foreach, which you mention in the RFC then >> it should be so, not for. A little background on where I am coming >> from here; I think it is great that we look at other languages and >> technologies when designing PHP and adding more features the language, >> however while I do believe it is an important point to keep cross >> language consistency, I do think that consistency with PHP itself is >> more important, which I believe too, is the reason for the expression >> for using foreach in this context comes from (not only from my PoV). >> >> I was originally going to say I agreed with this position, but, after > thinking about it, I've changed my mind. A "for" loop is just a method of > specifying the keys that you are iterating over. Since you aren't > specifying the keys inside a comprehension, the traditional "for" loop > doesn't really exist. I'd suggest supporting both keywords, though. > > >> -- >> regards, >> >> Kalle Sommer Nielsen >> kalle@php.net >> >> -- >> PHP Internals - PHP Runtime Development Mailing List >> To unsubscribe, visit: http://www.php.net/unsub.php >> >> -- > -- Chase > chasepeeler@gmail.com >
  104701
March 13, 2019 21:10 d.takken@xs4all.nl (Dik Takken)
On 11-03-19 05:29, Larry Garfield wrote:
> Sara and I tried putting the expression first, as in Python, but that made the lexer very unhappy
There is another possible reason to put the expression last: It preserves the ordering of the intended foreach loop. What I mean to say is this: One can change a loop into a comprehension by simply removing the newlines, braces, etc. Other than that, the various elements stay in the same place. For me, this makes it easier to 'see' the intended loop in the comprehension. I have always found the Python syntax to be confusing in this regard. On 11-03-19 13:16, Nikita Popov wrote:
> I've brought up a similar proposal a few years ago, probably around the > time generators were first introduced: https://externals.io/message/61021
May I please point at one particularly nice detail in Nikita's proposal: A dual syntax for both array comprehensions and generator expressions: $arr = [foreach ($users as $user) yield $user->firstName] $gen = (foreach ($users as $user) yield $user->firstName) I would love to see a dual syntax included in the current proposal as well. I think it makes sense considering the fact that many functions in PHP only accept arrays, not generators. So in practice, I expect that using comprehensions as proposed in the new RFC will also require doing a lot of iterator_to_array(). A dual comprehension syntax could fix that. One little remark about the array comprehension syntax: Using a keyword other than "yield" would make the difference between the two more obvious. Perhaps the array comprehension could use "return": $arr = [foreach ($users as $user) return $user->firstName] $gen = (foreach ($users as $user) yield $user->firstName)
> I think my main point of feedback would be to stick closer to existing PHP > syntax, even if it costs us some brevity. I would prefer > > $gen = [foreach ($list as $x) if ($x % 2) yield $x * 2]; > > over > > $gen = [for $list as $x if $x % 2 yield $x * 2]; > > The latter is nice in languages that generally use "for" for this purpose > and generally allow omitting parentheses, but I don't think it's good to > introduce this kind of syntax inconsistency in one place.
Keeping the parenthesis would seem to be more consistent indeed. Omitting the curly brackets of a "foreach" loop is valid syntax in current PHP while omitting the parenthesis from the "foreach" and "if" is not. However, the RFC also points out that the syntax should be limited compared to a "foreach" loop, which makes sense. This makes me want to favor the syntax as proposed, without the parenthesis. The "full syntax" with parenthesis kind of suggests that comprehension syntax is no different than a regular foreach loop, supporting the existing if / else syntax, etc, but on a single line. That level of similarity might be confusing. For example, if I can write: if ($x % 2) yield $x * 2; and if ($x % 2) yield $x * 2; else yield $x; then why is it that I can write: [foreach ($list as $x) if ($x % 2) yield $x * 2] but not: [foreach ($list as $x) if ($x % 2) yield $x * 2; else yield $x] Bottom line: If comprehensions aim to provide an alternative syntax to foreach loops, with different rules, why try to make them look exactly like foreach loops? That could be confusing. Also note that Python list comprehensions basically show the same "inconsistency" as the proposed PHP comprehensions: In PHP, parenthesis are required after "foreach" or "if". The RFC proposes to omit them in comprehensions. In Python, colons are required after "for" and "if". Python comprehensions omit these. Not that "Python has it too" is a valid argument for anything at all though... I do agree that using "foreach" in stead of "for" may be a better choice. The proposed comprehensions are compressed foreach loops. In my mind, I want to think "foreach" when I see a comprehension, not "for". On 12-03-19 09:54, Kalle Sommer Nielsen wrote:
> One more thing that kinda "annoys" me / what I base my above statement > on is; when reading some of the example is that the code examples > given, as in the code inside a Comprehension does not look very > PHP-ish besides the $-sigil and I do not like the idea of having "two" > syntaxes for PHP where one is only available in a very narrow-, > specific context and I feel it would be prone to potential errors.
I fully understand this feeling. It's new to PHP and looks quite weird at first. Many of us know comprehensions from the Python world. Maybe the association with Python is so strong that the very concept of comprehensions feels Pythonic and not something that fits into PHP. I was skeptical about comprehensions myself when I started coding in Python. However, after getting used to them, I must admit that I find comprehensions easier to read than traditional loops. When they are short, that is. :) Looking at a comprehension, I can still 'see' the intended loop. Only quicker, because there is less code to scan. I would really like to encourage trying to step over that initial gosh-that's-weird feeling and consider what it could offer for PHP. On 12-03-19 16:42, Levi Morrison wrote:
> Look at this example from the RFC: > > $result = (function() use ($table) { > foreach ($table as $num => $row) { > if ($num % 2 == 0) { > foreach ($row as $col => $val) { > if ($col >= 3) { > yield $num => $val; > } > } > } > } > })(); > > We could use this syntax today: > > $result = (function () { > foreach ($table as $num => $row) > if ($num % 2 == 0) > foreach ($row as $col => $val) > if ($col >= 3) > yield $num => $val; > })(); > > Compare that to this code using comprehensions, using the same whitespace style: > > $result = (for $table as $num => $row > if $num %2 ==0 > for $row as $col => $value > if $col >= 3 > yield $num => $val > ); > > In my opinion there are no meaningful differences here, and nothing > prevents you from using the all-in-one-line style if you care to.
For bigger loop constructs, the advantage of comprehensions is indeed less obvious. In Python code, I find myself using comprehensions to replace mostly simple loops. That is where they really shine I think. Omitting curly brackets and using the all-in-one-line style is indeed possible, which closes the gap with closures even further. However, in practice the brackets are typically enforced by code style checkers, for good reason. In comprehensions, they can be safely omitted.
> Oof, that sucks and I didn't even have to bind any variables. But if > we had shorter closures (this is JavaScript style) then it's fine > again: > > $result = array_map(($num) => intval($num, 16), $hex_numbers); > > My point is, closures are the major culprit. We really need shorter closures.
That is one great example of how short closures might do the job. However, this assumes that what you need as output is an array. As proposed in the RFC, comprehensions would be generators. In the case where the above example would need to create a generator in stead of an array, a comprehension would be preferable over a closure. I think both short closures and comprehensions have great potential, I don't think we should pick one or the other. On 13-03-19 21:42, Stanislav Malyshev wrote:
> Also agree about parentheses - reading a complex expression without > separators would be annoying. I know python loves to do this but even in > an IDE with highlighter it's not always easy to quickly figure out which > part belongs where. Delimiting it would make it easier IMHO, and also > allows again to carry over the intuition of how foreach() and if() work > from already known constructs.
Only, the intuition of how "foreach" and "if" works does not fully apply due to the limited syntax of comprehensions. For example, you can't pair the "if" with an "else" in the current proposal. Maybe that could be supported as well, but that is not my point. If comprehensions start looking like familiar foreach loops but do not work the same way, would that not be confusing? Kind regards, Dik Takken
  104703
March 13, 2019 22:30 rowan.collins@gmail.com (Rowan Collins)
On 13/03/2019 21:10, Dik Takken wrote:
> So in practice, I expect that > using comprehensions as proposed in the new RFC will also require doing > a lot of iterator_to_array(). A dual comprehension syntax could fix that.
At risk of complicating things further, might the solution to that be to have a shorter syntax for iterator_to_array in general? It's a shame array-casts are defined for arbitrary objects, else we could have (array)$iterator - and therefore (array)[foreach ($users as $user) yield $user->firstName] Regards, -- Rowan Collins [IMSoP]
  104706
March 14, 2019 03:21 larry@garfieldtech.com ("Larry Garfield")
On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote:
> On 13/03/2019 21:10, Dik Takken wrote: > > So in practice, I expect that > > using comprehensions as proposed in the new RFC will also require doing > > a lot of iterator_to_array(). A dual comprehension syntax could fix that. > > > At risk of complicating things further, might the solution to that be to > have a shorter syntax for iterator_to_array in general? > > It's a shame array-casts are defined for arbitrary objects, else we > could have (array)$iterator - and therefore (array)[foreach ($users as > $user) yield $user->firstName]
I am again going to reply to a bunch of people at once here... If I can summarize the responses so far, they seem to fall into one of two categories: 1) Love the idea, but wouldn't short-closures be close enough? 2) Love the idea, but hate the particular syntax proposed. On the plus side, it seems almost everyone is on board in concept, so yay. That of course just leaves the syntax bikeshedding, which is always the fun part. As an aside, someone up-thread said that comprehensions were "an easier way to write foreach loops", which is only true by accident. Comprehensions are more formally a way of defining one set in relation to another set. That is, they are a declarative relationship between one set and another. While in PHP that ends up effectively being a short-hand for foreach loops, that's more an accidental implementation detail. The syntax used by many other languages to achieve the same thing doesn't look at all like loop syntax. To the question of having both a generator and array version, I would have to say no. As noted in the RFC, most cases where you'd want to use a comprehension are not places where you'd be feeding the result into an array function. On the off chance that you are converting the iterable into an array is trivial enough that supporting, documenting, and learning two slightly different syntaxes seems a net negative. To Rowan's point, I would be fully in favor of an easier syntax alternative to iterator_to_array(). I think that's rather similar (although not identical) to the "run out an iterator" add-on mentioned in the RFC. I would support that, but I think it's a bit orthogonal and should not be a blocker for short-closures or for comprehensions. As for the specific syntax, I see a couple of options. 1) Assuming that short-lambdas get adopted and they can transparently support generators, the following syntax becomes automatically possible: $gen = (fn() => foreach($arr as $k => $v) if ($k % 2) yield $v;)(); While that does work, there's an awful lot of symbol salad there: (fn() => and ;)(); are both just gross and hard to type. I would consider that not a full solution for comprehensions because of how clumsy it is. 2) We could include an even-shorter-lambda syntax, potentially, or perhaps a short-lambda-based comprehension syntax. For example (and this may not be parser friendly but it's just to demonstrate the idea): $gen = fn{ foreach($arr as $k => $v) if ($k % 2) yield $v }; That would be a short-hand for a short-closure that has no parameters, and we could detect the yield and self-execute. The language inside the function body would still be a bit verbose, but it would technically be any legal single statement, which would offer some potentially interesting (scary?) options. I would consider this an acceptable solution for comprehensions-ish in PHP. 3) The specific syntax proposed in the RFC is Python-inspired and PHP-ified, but there's no reason we need to stick to that. There are a myriad of other syntaxes for comprehensions in other languages that we could steal if they fit better, some of which wouldn't at all resemble foreach loops and thus avoid the for/foreach confusion. Wikipedia of course has a large index of them we can mine: https://en.wikipedia.org/wiki/Comparison_of_programming_languages_(list_comprehension) It appears that the most common syntax involves [] of some variety, which pose parsing problems for PHP, but a few other options jump out at me as possible syntaxes to pilfer: C# has this SQL-esque syntax (which may involve too many additional language keywords): var ns = from x in Enumerable.Range(0,100) where x*x > 3 select x*2; Elixr, Erlang, and Haskell use the <- symbol, which... I don't think we use anywhere else currently? In Elixir: for x <- 0..100, x * x > 3, do: x * 2 Java 8, Ruby, Rust, and Swift are very very similar, and use a fluent syntax. The Rust example: (0..100).filter(|x| x * x > 3).map(|x| 2 * x).collect(); While that could not be taken as-is, of course, it does propose an interesting alternative approach, if we limit comprehensions to Traversable objects rather than any iterable (that is, exclude arrays): $t->filter(fn($v) => expression)->filter(fn($k, $v) => expression)->map(fn($v) => expression); Which would, in turn, each produce a generator that reduces the set or finally yields. I am not sure I fully like this one, to be honest, as the multiple inline short closures make it rather verbose and harder to follow with the proposed short-closure syntax (and it would involve more function calls internally), but it's an option. (collect() in these languages seems like it's the equivalent of iterator_to_array(); maybe that's another alternative there as well?) Nemerle, which I've never heard of before, has this: $[x*2 | x in [0 .. 100], x*x > 3] Which, while $ is obviously already used, does suggest using one of the other not-yet-used sigils that Nikita identified, which would let us reorder the parameters to put the expression first if we wanted. For example: ^[$x *2 | $k => $v in $arr if $k %2] In general, I see two alternatives: 1) Pass short closures and then include a special case of that special case that effectively gives us comprehensions over foreach, if, and yield, but with fewer seemingly-stray characters. 2) Steal a completely different syntax from some other language that is still terse but less confusing. The main alternatives to "square brackets around a for loop" syntax seem to be: A) Chained filter() and map() methods B) SQL-like keywords C) Use <- somehow. D) Use a different starting character before the [] so that the parser knows some new funky order of stuff is coming. I am open to both options, of course contingent on someone willing and able to code it. --Larry Garfield
  104709
March 14, 2019 07:16 michal.brzuchalski@gmail.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
czw., 14 mar 2019, 04:22 użytkownik Larry Garfield <larry@garfieldtech..com>
napisał:

> On Wed, Mar 13, 2019, at 6:30 PM, Rowan Collins wrote: > > On 13/03/2019 21:10, Dik Takken wrote: > > > So in practice, I expect that > > > using comprehensions as proposed in the new RFC will also require doing > > > a lot of iterator_to_array(). A dual comprehension syntax could fix > that. > > > > > > At risk of complicating things further, might the solution to that be to > > have a shorter syntax for iterator_to_array in general? > > > > It's a shame array-casts are defined for arbitrary objects, else we > > could have (array)$iterator - and therefore (array)[foreach ($users as > > $user) yield $user->firstName] > > I am again going to reply to a bunch of people at once here... > > > If I can summarize the responses so far, they seem to fall into one of two > categories: > > 1) Love the idea, but wouldn't short-closures be close enough? > > 2) Love the idea, but hate the particular syntax proposed. > > On the plus side, it seems almost everyone is on board in concept, so > yay. That of course just leaves the syntax bikeshedding, which is always > the fun part. > > As an aside, someone up-thread said that comprehensions were "an easier > way to write foreach loops", which is only true by accident. > Comprehensions are more formally a way of defining one set in relation to > another set. That is, they are a declarative relationship between one set > and another. While in PHP that ends up effectively being a short-hand for > foreach loops, that's more an accidental implementation detail. The syntax > used by many other languages to achieve the same thing doesn't look at all > like loop syntax. > > To the question of having both a generator and array version, I would have > to say no. As noted in the RFC, most cases where you'd want to use a > comprehension are not places where you'd be feeding the result into an > array function. On the off chance that you are converting the iterable > into an array is trivial enough that supporting, documenting, and learning > two slightly different syntaxes seems a net negative. > > To Rowan's point, I would be fully in favor of an easier syntax > alternative to iterator_to_array(). I think that's rather similar > (although not identical) to the "run out an iterator" add-on mentioned in > the RFC. I would support that, but I think it's a bit orthogonal and > should not be a blocker for short-closures or for comprehensions. > > As for the specific syntax, I see a couple of options. > > 1) Assuming that short-lambdas get adopted and they can transparently > support generators, the following syntax becomes automatically possible: > > $gen = (fn() => foreach($arr as $k => $v) if ($k % 2) yield $v;)(); > > While that does work, there's an awful lot of symbol salad there: (fn() => > and ;)(); are both just gross and hard to type. I would consider that not > a full solution for comprehensions because of how clumsy it is. >
Have you thought about adopting a keyword which fill look similar as short closures but by default yield instead of return. I'm thinking of something like that which looks close to short functions but shorter: $arr = [ 'apple' => ['a' => 1], 'orange' => ['a' => 2], ] $comprehension = from($arr as $k => ['a' => $a]) => if ($k === 'orange' || $a === 1) $a; $comprehension = from($arr as $k => ['a' => $a]) => { if ($k === 'orange') yield $a; if ($a === 1) yield $a; }; The 'from' keyword indicates that it is going to yield from what put in the parenthesis. Probably it won't be ambiguous when nesting comprehension. The example includes also a list array destruct on purpose to show all what foreach can do.
> 2) We could include an even-shorter-lambda syntax, potentially, or perhaps > a short-lambda-based comprehension syntax. For example (and this may not > be parser friendly but it's just to demonstrate the idea): > > $gen = fn{ foreach($arr as $k => $v) if ($k % 2) yield $v }; > > That would be a short-hand for a short-closure that has no parameters, and > we could detect the yield and self-execute. The language inside the > function body would still be a bit verbose, but it would technically be any > legal single statement, which would offer some potentially interesting > (scary?) options. I would consider this an acceptable solution for > comprehensions-ish in PHP. > > 3) The specific syntax proposed in the RFC is Python-inspired and > PHP-ified, but there's no reason we need to stick to that. There are a > myriad of other syntaxes for comprehensions in other languages that we > could steal if they fit better, some of which wouldn't at all resemble > foreach loops and thus avoid the for/foreach confusion. > > Wikipedia of course has a large index of them we can mine: > > > https://en.wikipedia.org/wiki/Comparison_of_programming_languages_(list_comprehension) > > It appears that the most common syntax involves [] of some variety, which > pose parsing problems for PHP, but a few other options jump out at me as > possible syntaxes to pilfer: > > C# has this SQL-esque syntax (which may involve too many additional > language keywords): > > var ns = from x in Enumerable.Range(0,100) > where x*x > 3 > select x*2; > > Elixr, Erlang, and Haskell use the <- symbol, which... I don't think we > use anywhere else currently? In Elixir: > > for x <- 0..100, x * x > 3, do: x * 2 > > Java 8, Ruby, Rust, and Swift are very very similar, and use a fluent > syntax. The Rust example: > > (0..100).filter(|x| x * x > 3).map(|x| 2 * x).collect(); > > While that could not be taken as-is, of course, it does propose an > interesting alternative approach, if we limit comprehensions to Traversable > objects rather than any iterable (that is, exclude arrays): > > $t->filter(fn($v) => expression)->filter(fn($k, $v) => > expression)->map(fn($v) => expression); > > Which would, in turn, each produce a generator that reduces the set or > finally yields. I am not sure I fully like this one, to be honest, as the > multiple inline short closures make it rather verbose and harder to follow > with the proposed short-closure syntax (and it would involve more function > calls internally), but it's an option. (collect() in these languages seems > like it's the equivalent of iterator_to_array(); maybe that's another > alternative there as well?) > > Nemerle, which I've never heard of before, has this: > > $[x*2 | x in [0 .. 100], x*x > 3] > > Which, while $ is obviously already used, does suggest using one of the > other not-yet-used sigils that Nikita identified, which would let us > reorder the parameters to put the expression first if we wanted. For > example: > > ^[$x *2 | $k => $v in $arr if $k %2] > > > In general, I see two alternatives: > > 1) Pass short closures and then include a special case of that special > case that effectively gives us comprehensions over foreach, if, and yield, > but with fewer seemingly-stray characters. > > 2) Steal a completely different syntax from some other language that is > still terse but less confusing. The main alternatives to "square brackets > around a for loop" syntax seem to be: > > A) Chained filter() and map() methods > B) SQL-like keywords > C) Use <- somehow. > D) Use a different starting character before the [] so that the parser > knows some new funky order of stuff is coming. > > I am open to both options, of course contingent on someone willing and > able to code it. > > --Larry Garfield > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php
Great work and I like the idea in general. BR, -- Michał Brzuchalski
  104727
March 14, 2019 21:34 d.takken@xs4all.nl (Dik Takken)
On 14-03-19 04:21, Larry Garfield wrote:
> To the question of having both a generator and array version, I would have to say no. As noted in the RFC, most cases where you'd want to use a comprehension are not places where you'd be feeding the result into an array function. On the off chance that you are converting the iterable into an array is trivial enough that supporting, documenting, and learning two slightly different syntaxes seems a net negative.
I fully agree that comprehensions can replace the usual array_*() functions in many cases. However, when you just happen to use a library that has an array oriented API, then you still need to pass it arrays. It would be disappointing when comprehensions turn out to be tedious to use in these situations, because they are no good fit. A dual generator / array syntax will make comprehensions fit in anywhere. Then there are the cases where you want to access the output of the comprehension multiple times. This may be expensive when a comprehension is a generator. So I still think that using a comprehension to create an array isn't an off chance but a common use case. Regards, Dik Takken
  104728
March 14, 2019 21:48 d.takken@xs4all.nl (Dik Takken)
On 13-03-19 23:30, Rowan Collins wrote:
> At risk of complicating things further, might the solution to that be to > have a shorter syntax for iterator_to_array in general? > > It's a shame array-casts are defined for arbitrary objects, else we > could have (array)$iterator - and therefore (array)[foreach ($users as > $user) yield $user->firstName]
A shorter alternative for iterator_to_array() could work as well I think. Casting would have been stellar, but not feasible. However, could using the array() built-in work? Something like: array([foreach ($users as $user) yield $user->firstName]) Just thinking out loud here... Regards, Dik Takken
  104676
March 12, 2019 15:13 pollita@php.net (Sara Golemon)
On Sun, Mar 10, 2019 at 4:45 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Sara Golemon has written a preliminary patch that is partially complete > (see the RFC link for details, it's impressively simple), but unfortunately > doesn't have the bandwidth to complete it at this time. I am therefore > looking for collaborators with more knowledge of internals than I (which is > almost everyone) to help finish it up. > > Just want to clarify; The proof-of-concept I made for Larry is *just* a proof of concept to illustrate one potential syntax. The actual
implementation is a dumpster fire that was hastily assembled between conference sessions. Please ignore the mechanics of how it was put together and focus on the intended syntax and resulting behavior. -Sara
  104679
March 12, 2019 15:42 levim@php.net (Levi Morrison)
On Sun, Mar 10, 2019 at 3:45 PM Larry Garfield <larry@garfieldtech.com> wrote:
> > Hello, peoples. I know it's been discussed once or twice before on the list, many years ago, but not recently. I therefore feel OK putting forth the following draft proposal for Comprehensions, aka "compact generators", in PHP: > > https://wiki.php.net/rfc/comprehensions > > Sara Golemon has written a preliminary patch that is partially complete (see the RFC link for details, it's impressively simple), but unfortunately doesn't have the bandwidth to complete it at this time. I am therefore looking for collaborators with more knowledge of internals than I (which is almost everyone) to help finish it up. > > The syntax proposed is also subject to revision if a terser but still lexer-friendly alternative can be proposed. > > At the moment I'm not calling this Proposed yet, as I don't feel comfortable doing so until someone else is on board to finish coding it. That said, if someone wants to weigh in on the concept for now (hopefully supportively) that's also fine. > > Anyone excited enough to help finish the job? > > (This is my first RFC collaboration, so if you're going to smack me for goofing something please be gentle about it.) > > -- > Larry Garfield > larry@garfieldtech.com > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php
I want to like the idea of list comprehensions. However, after thinking about it for quite a bit over the past year as well as since discussion started, the issue is really that closures are terrible in PHP. Look at this example from the RFC: $result = (function() use ($table) { foreach ($table as $num => $row) { if ($num % 2 == 0) { foreach ($row as $col => $val) { if ($col >= 3) { yield $num => $val; } } } } })(); We could use this syntax today: $result = (function () { foreach ($table as $num => $row) if ($num % 2 == 0) foreach ($row as $col => $val) if ($col >= 3) yield $num => $val; })(); Compare that to this code using comprehensions, using the same whitespace style: $result = (for $table as $num => $row if $num %2 ==0 for $row as $col => $value if $col >= 3 yield $num => $val ); In my opinion there are no meaningful differences here, and nothing prevents you from using the all-in-one-line style if you care to. Let's look at a shorter piece, and let's also assume that the following code is eagerly evaluated and stored into an array (which is not what the RFC is proposing, but it helps for this example): $result = [ for ($hex_numbers as $num) yield intval($num, 16); ]; With closures as they are today: $result = array_map(function($num) { return intval($num, 16); }, $hex_numbers); Oof, that sucks and I didn't even have to bind any variables. But if we had shorter closures (this is JavaScript style) then it's fine again: $result = array_map(($num) => intval($num, 16), $hex_numbers); My point is, closures are the major culprit. We really need shorter closures. Again, I want to like comprehensions. I think we should revisit list comprehensions after we've had shorter closures for a little while (hopefully we can get something passed for 7.4 or 8.0).