Proposal for a RFC

  105589
May 4, 2019 14:58 stevenwadejr@gmail.com (Steven Wade)
Hi Internals team!

I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. 

The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays.

I would personally love this feature and those I've run it by were also excited by the idea. So I'm soliciting feedback in hopes that things go well and I can officially write the RFC. As for implementation, Sara Golemon is awesome and while chatting a few months back, knocked out a proof-of-concept implementation <https://github.com/sgolemon/php-src/tree/experimental.toarray>.  There's still work to be done if this proposal gets to the RFC phase, but again, just gauging interest here.

I appreciate any feedback you all can provide.

Thanks,

- Steven Wade
  105590
May 4, 2019 15:59 pmjones88@gmail.com (Paul Jones)
Hi Steven,

> The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays.
I'd like to see this as well. -- Paul M. Jones pmjones@pmjones.io http://paul-m-jones.com Modernizing Legacy Applications in PHP https://leanpub.com/mlaphp Solving the N+1 Problem in PHP https://leanpub.com/sn1php
  105591
May 4, 2019 16:08 kalle@php.net (Kalle Sommer Nielsen)
Hi

Den lør. 4. maj 2019 kl. 17.58 skrev Steven Wade <stevenwadejr@gmail.com>:
> > Hi Internals team! > > I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. > > The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays.
While this sounds great and all, I do wonder about the implications it may have on the already existing behavior of type casting an object into an array which returns current public/protected/private property values. While this functionality is not an ideal or by design, it is fairly heavily (ab-)used, meaning that it is a potential BC break without the ability to an alternative. See this example: https://3v4l.org/ONMoi -- regards, Kalle Sommer Nielsen kalle@php.net
  105592
May 4, 2019 16:10 pmjones88@gmail.com (Paul Jones)
> On May 4, 2019, at 11:08, Kalle Sommer Nielsen <kalle@php.net> wrote: > > Hi > > Den lør. 4. maj 2019 kl. 17.58 skrev Steven Wade <stevenwadejr@gmail.com>: >> >> Hi Internals team! >> >> I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. >> >> The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays. > > While this sounds great and all, I do wonder about the implications it > may have on the already existing behavior of type casting an object > into an array which returns current public/protected/private property > values.
I find it easy to imagine that objects not implementing __toArray() would continue to exhibit that behavior. -- Paul M. Jones pmjones@pmjones.io http://paul-m-jones.com Modernizing Legacy Applications in PHP https://leanpub.com/mlaphp Solving the N+1 Problem in PHP https://leanpub.com/sn1php
  105642
May 8, 2019 14:13 stevenwadejr@gmail.com (Steven Wade)
Hi Kalle,

> On May 4, 2019, at 12:08 PM, Kalle Sommer Nielsen <kalle@php.net> wrote: > > Hi > > Den lør. 4. maj 2019 kl. 17.58 skrev Steven Wade <stevenwadejr@gmail.com>: >> >> Hi Internals team! >> >> I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. >> >> The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays. > > While this sounds great and all, I do wonder about the implications it > may have on the already existing behavior of type casting an object > into an array which returns current public/protected/private property > values. While this functionality is not an ideal or by design, it is > fairly heavily (ab-)used, meaning that it is a potential BC break > without the ability to an alternative. > > See this example: https://3v4l.org/ONMoi <https://3v4l.org/ONMoi>
That's a great example, thanks for that. I can understand how casting now can be as you say "abused". With my proposal, the default behavior wouldn't change unless someone implemented the __toArray() method on their class, which would simply give them greater control over how that class is represented. In regards to the output of the example you linked, the same is possible today with reflection: https://3v4l.org/Ab51l <https://3v4l.org/Ab51l> If libraries are using casting to gain insight into objects, it seems that they could be updated to use reflection without losing any functionality. -- Regards, Steven Wade stevenwadejr@gmail.com
  105594
May 4, 2019 16:36 ocramius@gmail.com (Marco Pivetta)
Hi Steven,

As it currently stands, the array cast is the only operation capable of
exposing object state without triggering any kind of access guards: it is
very much required for anything that works with reflection and typed
properties, and possibly the only operation in PHP that operates on state
without some contraption intercepting its execution.

It is also necessary to distinguish dynamic properties from declared object
state.

For comparison, all of the following have side-effects due to the
complexity of the language:

 * isset()
 * unset()
 * property read
 * property write
 * ReflectionProperty#getValue()
 * ReflectionProperty#setValue()

Overall, this is problematic, and introduces more magic that I'd gladly
avoid, in an endpoint used to work around all the engine magic (the only
stable one so far).

From my end, this sounds like a bad idea, because it removes one of the
very very few referentially transparent guarantees (if not the only one,
when dealing with objects) from the language.

Greets,

Marco

On Sat, 4 May 2019, 16:58 Steven Wade, <stevenwadejr@gmail.com> wrote:

> Hi Internals team! > > I have an idea for a feature that I'd love to see in the language one day > and wanted to run the idea by you all. > > The idea is to add a new magic method "__toArray()" that would allow a > developer to specifiy how a class is cast to an array. The idea is the same > mentality of __toString(), but, for arrays. > > I would personally love this feature and those I've run it by were also > excited by the idea. So I'm soliciting feedback in hopes that things go > well and I can officially write the RFC. As for implementation, Sara > Golemon is awesome and while chatting a few months back, knocked out a > proof-of-concept implementation < > https://github.com/sgolemon/php-src/tree/experimental.toarray>. There's > still work to be done if this proposal gets to the RFC phase, but again, > just gauging interest here. > > I appreciate any feedback you all can provide. > > Thanks, > > - Steven Wade
  105597
May 5, 2019 15:07 nicolas.grekas@gmail.com (Nicolas Grekas)
Le sam. 4 mai 2019 à 18:37, Marco Pivetta <ocramius@gmail.com> a écrit :

> Hi Steven, > > As it currently stands, the array cast is the only operation capable of > exposing object state without triggering any kind of access guards: it is > very much required for anything that works with reflection and typed > properties, and possibly the only operation in PHP that operates on state > without some contraption intercepting its execution. > > It is also necessary to distinguish dynamic properties from declared object > state. > > For comparison, all of the following have side-effects due to the > complexity of the language: > > * isset() > * unset() > * property read > * property write > * ReflectionProperty#getValue() > * ReflectionProperty#setValue() > > Overall, this is problematic, and introduces more magic that I'd gladly > avoid, in an endpoint used to work around all the engine magic (the only > stable one so far). > > From my end, this sounds like a bad idea, because it removes one of the > very very few referentially transparent guarantees (if not the only one, > when dealing with objects) from the language. > > Greets, > > Marco >
I want to weight in with what Marco expressed. I have the very same concerns and they are major ones for many use cases. Mine is VarDumper. Please don't do this the way it is described. Nicolas
  105614
May 7, 2019 12:19 stevenwadejr@gmail.com (Steven Wade)
> I want to weight in with what Marco expressed. I have the very same concerns and they are major ones for many use cases. Mine is VarDumper. > > Please don't do this the way it is described.
Is there a way that you'd suggest? First of all, TIL some cool things about "(array) $foo" and how VarDumper uses that, so thank you! Secondly, I believe the casting syntax is the best user experience and would be a good addition to the language, so is there any other way at all to maintain something like VarDumper's functionality without using the casting syntax? I'd like to find a way to not distrupt too much of what's out there, while also adding value to the end user. -- Steven Wade stevenwadejr@gmail.com
  105615
May 7, 2019 12:22 nikita.ppv@gmail.com (Nikita Popov)
On Tue, May 7, 2019 at 2:20 PM Steven Wade <stevenwadejr@gmail.com> wrote:

> > I want to weight in with what Marco expressed. I have the very same > concerns and they are major ones for many use cases. Mine is VarDumper. > > > > Please don't do this the way it is described. > > > Is there a way that you'd suggest? First of all, TIL some cool things > about "(array) $foo" and how VarDumper uses that, so thank you! Secondly, I > believe the casting syntax is the best user experience and would be a good > addition to the language, so is there any other way at all to maintain > something like VarDumper's functionality without using the casting syntax? > I'd like to find a way to not distrupt too much of what's out there, while > also adding value to the end user. >
We can add a separate function to provide this functionality. We should do that anyway because it's both clearer and because (array) already requires some special handling for ArrayObject that could be avoided. Nikita On Tue, May 7, 2019 at 2:20 PM Steven Wade <stevenwadejr@gmail.com> wrote:
> > I want to weight in with what Marco expressed. I have the very same > concerns and they are major ones for many use cases. Mine is VarDumper. > > > > Please don't do this the way it is described. > > > Is there a way that you'd suggest? First of all, TIL some cool things > about "(array) $foo" and how VarDumper uses that, so thank you! Secondly, I > believe the casting syntax is the best user experience and would be a good > addition to the language, so is there any other way at all to maintain > something like VarDumper's functionality without using the casting syntax? > I'd like to find a way to not distrupt too much of what's out there, while > also adding value to the end user. > > -- > Steven Wade > stevenwadejr@gmail.com > > > > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
  105617
May 7, 2019 12:28 stevenwadejr@gmail.com (Steven Wade)
> We can add a separate function to provide this functionality. We should do that anyway because it's both clearer and because (array) already requires some special handling for ArrayObject that could be avoided.
Are you talking about adding a separate function to replace my proposed __toArray() and casting functionality? Or did you mean adding a separate function to add functionality and insight into objects that packages like VarDumper have been using casting to access? -- Steven Wade stevenwadejr@gmail.com
  105618
May 7, 2019 12:31 php-lists@koalephant.com (Stephen Reay)
> On 7 May 2019, at 19:22, Nikita Popov ppv@gmail.com> wrote: > > On Tue, May 7, 2019 at 2:20 PM Steven Wade <stevenwadejr@gmail.com> wrote: > >>> I want to weight in with what Marco expressed. I have the very same >> concerns and they are major ones for many use cases. Mine is VarDumper. >>> >>> Please don't do this the way it is described. >> >> >> Is there a way that you'd suggest? First of all, TIL some cool things >> about "(array) $foo" and how VarDumper uses that, so thank you! Secondly, I >> believe the casting syntax is the best user experience and would be a good >> addition to the language, so is there any other way at all to maintain >> something like VarDumper's functionality without using the casting syntax? >> I'd like to find a way to not distrupt too much of what's out there, while >> also adding value to the end user. >> > > We can add a separate function to provide this functionality. We should do > that anyway because it's both clearer and because (array) already requires > some special handling for ArrayObject that could be avoided. > > Nikita > > On Tue, May 7, 2019 at 2:20 PM Steven Wade <stevenwadejr@gmail.com> wrote: > >>> I want to weight in with what Marco expressed. I have the very same >> concerns and they are major ones for many use cases. Mine is VarDumper. >>> >>> Please don't do this the way it is described. >> >> >> Is there a way that you'd suggest? First of all, TIL some cool things >> about "(array) $foo" and how VarDumper uses that, so thank you! Secondly, I >> believe the casting syntax is the best user experience and would be a good >> addition to the language, so is there any other way at all to maintain >> something like VarDumper's functionality without using the casting syntax? >> I'd like to find a way to not distrupt too much of what's out there, while >> also adding value to the end user. >> >> -- >> Steven Wade >> stevenwadejr@gmail.com >> >> >> >> >> -- >> PHP Internals - PHP Runtime Development Mailing List >> To unsubscribe, visit: http://www.php.net/unsub.php >> >>
Maybe I’m missing the point (I’ve never used it) of VarDumper, but isn’t this type of thing exactly why the `__debugInfo` magic method exists?
  105619
May 7, 2019 12:47 stevenwadejr@gmail.com (Steven Wade)
> Maybe I’m missing the point (I’ve never used it) of VarDumper, but isn’t this type of thing exactly why the `__debugInfo` magic method exists?
I can't speak for the exact reason a library like VarDumper is using casting versus __debugInfo, but in trying to find a substitute this morning, I played with the method you mentioned and found that it can be overwritten by classes and be commanded to not return anything (similar to my __toArray proposal), whereas current functionality prevents overriding casting to an array so casting will always give you insight but users can overwrite __debugInfo and cause unintended effects. I assume that's the reason for not using __debugInfo. -- Steven Wade stevenwadejr@gmail.com
  105620
May 7, 2019 12:52 php-lists@koalephant.com (Stephen Reay)
> On 7 May 2019, at 19:47, Steven Wade <stevenwadejr@gmail.com> wrote: > >> Maybe I’m missing the point (I’ve never used it) of VarDumper, but isn’t this type of thing exactly why the `__debugInfo` magic method exists? > > > I can't speak for the exact reason a library like VarDumper is using casting versus __debugInfo, but in trying to find a substitute this morning, I played with the method you mentioned and found that it can be overwritten by classes and be commanded to not return anything (similar to my __toArray proposal), whereas current functionality prevents overriding casting to an array so casting will always give you insight but users can overwrite __debugInfo and cause unintended effects. I assume that's the reason for not using __debugInfo. > > -- > Steven Wade > stevenwadejr@gmail.com > > >
Right, I understand that it *can* be overridden, but surely if someone’s doing that, its because they want more useful information provided in, e.g. var_dump.
  105621
May 7, 2019 12:57 stevenwadejr@gmail.com (Steven Wade)
> On May 7, 2019, at 8:52 AM, Stephen Reay <php-lists@koalephant.com> wrote: > > >> On 7 May 2019, at 19:47, Steven Wade <stevenwadejr@gmail.com> wrote: >> >>> Maybe I’m missing the point (I’ve never used it) of VarDumper, but isn’t this type of thing exactly why the `__debugInfo` magic method exists? >> >> >> I can't speak for the exact reason a library like VarDumper is using casting versus __debugInfo, but in trying to find a substitute this morning, I played with the method you mentioned and found that it can be overwritten by classes and be commanded to not return anything (similar to my __toArray proposal), whereas current functionality prevents overriding casting to an array so casting will always give you insight but users can overwrite __debugInfo and cause unintended effects. I assume that's the reason for not using __debugInfo. >> >> -- >> Steven Wade >> stevenwadejr@gmail.com >> >> >> > > Right, I understand that it *can* be overridden, but surely if someone’s doing that, its because they want more useful information provided in, e.g. var_dump.
I think that's a good point. If we follow that mentality, then the proposed __toArray and it's subsequent ability to affect array casting isn't a worry too much as the proposal is to only call __toArray if it exists and an object is cast to an array, so if a user manually implements __toArray, then we could assume "they want more useful information provided in, e.g. var_dump." -- Steven Wade stevenwadejr@gmail.com
  106783
August 29, 2019 03:09 kontakt@beberlei.de (Benjamin Eberlei)
On Sun, May 5, 2019 at 5:08 PM Nicolas Grekas grekas@gmail.com>
wrote:

> Le sam. 4 mai 2019 à 18:37, Marco Pivetta <ocramius@gmail.com> a écrit : > > > Hi Steven, > > > > As it currently stands, the array cast is the only operation capable of > > exposing object state without triggering any kind of access guards: it is > > very much required for anything that works with reflection and typed > > properties, and possibly the only operation in PHP that operates on state > > without some contraption intercepting its execution. > > > > It is also necessary to distinguish dynamic properties from declared > object > > state. > > > > For comparison, all of the following have side-effects due to the > > complexity of the language: > > > > * isset() > > * unset() > > * property read > > * property write > > * ReflectionProperty#getValue() > > * ReflectionProperty#setValue() > > > > Overall, this is problematic, and introduces more magic that I'd gladly > > avoid, in an endpoint used to work around all the engine magic (the only > > stable one so far). > > > > From my end, this sounds like a bad idea, because it removes one of the > > very very few referentially transparent guarantees (if not the only one, > > when dealing with objects) from the language. > > > > Greets, > > > > Marco > > > > I want to weight in with what Marco expressed. I have the very same > concerns and they are major ones for many use cases. Mine is VarDumper. > > Please don't do this the way it is described. >
Because Steve asked for wiki access to work on an RFC for his proposal, I went back and re-read and just wanted to come in his support as I think it would be a great addition to the existing __magic functionality, I want to add a few more arguments why __toArray() should be added regardless of the objections from Marco & Nicolas: 1. PHP already has __debugInfo() which affects the behavior of var_dump() allowing users to control what gets exposed to it in a debugging context. This is new magic method would affect Symfony's VarDumper+ VarCloner in a similar consistent way that gives class owners back control of what gets exposed. One primary benefit of OOP is encapsulation and guarding against access of internal state from the outside, so it is questionable anyways why an idiomatic and simple to use syntax like (array) $object would expose internal state and a developer would have no means of preventing that. 2. Adding __toArray would not constitute a backwards compatibility break, because all existing code does not implement __toArray and would keep the same exact behavior. 3. You can't argue against the concept of magic methods overall by citing your own use case that is objectively an extremely magic use of the existing behavior of (array) $object itself. I can see a very good case to change (array) $object to return only public properties as part of the general push to cleanup inconsistent behavior in PHP 8. As Nikita mentioned, it might make sense to add a function that is specifically designed for your both use-cases instead and that would honestly make this part of the language cleaner in the end and we can get rid of this * protected and nullbyte private variable returning behavior once and for all.. 4. ReflectionProperty::getValue() does not trigger a guard clause if you call setAccessible(true) first, so I don't think the objection that this is the only way to access private data is true here. Correct me if I forgot an edge case that prevents this. 5. Arguments of API design that toSomethingString() is somehow "better" than using __toString() and therefore the addition of __toArray() is bad pit one paradigm (explicit OOP vs magic methods) and should be avoided in an RFC discussion. We are also not discussing to remove functions in PHP because we now have classes and methods and this falls into the same category. Its a subject design decision and shouldn't influence the expansion of an existing feature/paradigm that one doesn't use themselves.
> Nicolas >
  106786
August 29, 2019 05:44 ocramius@gmail.com (Marco Pivetta)
Hey Benjamin,

On Thu, Aug 29, 2019, 05:09 Benjamin Eberlei <kontakt@beberlei.de> wrote:

> > > On Sun, May 5, 2019 at 5:08 PM Nicolas Grekas grekas@gmail.com> > wrote: > >> Le sam. 4 mai 2019 à 18:37, Marco Pivetta <ocramius@gmail.com> a écrit : >> >> > Hi Steven, >> > >> > As it currently stands, the array cast is the only operation capable of >> > exposing object state without triggering any kind of access guards: it >> is >> > very much required for anything that works with reflection and typed >> > properties, and possibly the only operation in PHP that operates on >> state >> > without some contraption intercepting its execution. >> > >> > It is also necessary to distinguish dynamic properties from declared >> object >> > state. >> > >> > For comparison, all of the following have side-effects due to the >> > complexity of the language: >> > >> > * isset() >> > * unset() >> > * property read >> > * property write >> > * ReflectionProperty#getValue() >> > * ReflectionProperty#setValue() >> > >> > Overall, this is problematic, and introduces more magic that I'd gladly >> > avoid, in an endpoint used to work around all the engine magic (the only >> > stable one so far). >> > >> > From my end, this sounds like a bad idea, because it removes one of the >> > very very few referentially transparent guarantees (if not the only one, >> > when dealing with objects) from the language. >> > >> > Greets, >> > >> > Marco >> > >> >> I want to weight in with what Marco expressed. I have the very same >> concerns and they are major ones for many use cases. Mine is VarDumper. >> >> Please don't do this the way it is described. >> > > Because Steve asked for wiki access to work on an RFC for his proposal, I > went back and re-read and just wanted to come in his support as I think it > would be a great addition to the existing __magic functionality, > > I want to add a few more arguments why __toArray() should be added > regardless of the objections from Marco & Nicolas: > > 1. PHP already has __debugInfo() which affects the behavior of var_dump() > allowing users to control what gets exposed to it in a debugging context. > This is new magic method would affect Symfony's VarDumper+ VarCloner in a > similar consistent way that gives class owners back control of what gets > exposed. One primary benefit of OOP is encapsulation and guarding against > access of internal state from the outside, so it is questionable anyways > why an idiomatic and simple to use syntax like (array) $object would expose > internal state and a developer would have no means of preventing that. >
__debugInfo is already arguably added complexity and debugging time for those looking at a dump result and not understanding it. From my PoV, I'd love to also see __debugInfo gone, since it only ever caused me to lose hair.
> 2. Adding __toArray would not constitute a backwards compatibility break, > because all existing code does not implement __toArray and would keep the > same exact behavior. >
It would break any code relying on current `(array)` semantics against the general `object` type. Can certainly propose it for 8.x, and then we go on a hunt for `(array)` casts, and disallowing them explicitly (cs, static analysis).
> 3. You can't argue against the concept of magic methods overall by citing > your own use case that is objectively an extremely magic use of the > existing behavior of (array) $object itself. I can see a very good case to > change (array) $object to return only public properties as part of the > general push to cleanup inconsistent behavior in PHP 8. As Nikita > mentioned, it might make sense to add a function that is specifically > designed for your both use-cases instead and that would honestly make this > part of the language cleaner in the end and we can get rid of this * > protected and nullbyte private variable returning behavior once and for all. >
It's fine to break this for 8.x, but I think 7.4 already has an added API for this? Can't remember the name, but something about unmangled vars.
> 4. ReflectionProperty::getValue() does not trigger a guard clause if you > call setAccessible(true) first, so I don't think the objection that this is > the only way to access private data is true here. Correct me if I forgot an > edge case that prevents this. >
Guards are triggered for unset properties, even with reflection.
> 5. Arguments of API design that toSomethingString() is somehow "better" > than using __toString() and therefore the addition of __toArray() is bad > pit one paradigm (explicit OOP vs magic methods) and should be avoided in > an RFC discussion. We are also not discussing to remove functions in PHP > because we now have classes and methods and this falls into the same > category. Its a subject design decision and shouldn't influence the > expansion of an existing feature/paradigm that one doesn't use themselves.. >
I'd say that the paradigm is at the core of the discussion: magic calls are some of the worst traps in the language, leading to monstrous amounts of complexity due to them always being active (rather than being explicitly declared via interface).
  105595
May 4, 2019 18:12 markyr@gmail.com (Mark Randall)
On 04/05/2019 15:58, Steven Wade wrote:
> The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays.
I see possibilities for it, and on a slightly wider approach, I do think it would be nice if PHP could eventually have full operator overloading. It goes against my better judgement for enforcing return types, but maybe a generic __cast function taking a single argument of the desired type might be an option. The engine could enforce the return type being the same as the argument type, and the main IDE tools in use support metadata patterns to perform static analysis. Question is, would it be either a supports/cast function pair, an exception thrower, or something else... class X { /* ... */ public function __cast($type) { if ($type === 'array') { return (array)$this; } if ($type === MyObject::class) { return new MyObject($this->x, $this->y); } throw new InvalidCastError($type); } public function __castable(): array { return [ 'array', MyObject::class ]; } } Eh, my suggestion is probably a terrible ideal. -- Mark Randall
  105596
May 5, 2019 14:59 ben@benramsey.com (Ben Ramsey)
> On May 4, 2019, at 09:58, Steven Wade <stevenwadejr@gmail.com> wrote: > > Hi Internals team! > > I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. > > The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays. > > I would personally love this feature and those I've run it by were also excited by the idea. So I'm soliciting feedback in hopes that things go well and I can officially write the RFC. As for implementation, Sara Golemon is awesome and while chatting a few months back, knocked out a proof-of-concept implementation <https://github.com/sgolemon/php-src/tree/experimental.toarray>. There's still work to be done if this proposal gets to the RFC phase, but again, just gauging interest here. > > I appreciate any feedback you all can provide. > > Thanks, > > - Steven Wade
Using existing language functionality, would it satisfy the similar needs if you implement IteratorAggregate and return an ArrayIterator from the getIterator() method? Obviously, the (array) cast wouldn’t use this, but you could do $object->getIterator()->getArrayCopy(). I’m not against the proposal. I’m mainly asking if there are other ways in the language today to accomplish similar things. I would prefer to see interfaces used over more magic methods (something like ArraySerializable). -Ben
  105598
May 6, 2019 14:37 stevenwadejr@gmail.com (Steven Wade)
> On May 5, 2019, at 10:59 AM, Ben Ramsey <ben@benramsey.com> wrote: > > >> On May 4, 2019, at 09:58, Steven Wade <stevenwadejr@gmail.com> wrote: >> >> Hi Internals team! >> >> I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. >> >> The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays. >> >> I would personally love this feature and those I've run it by were also excited by the idea. So I'm soliciting feedback in hopes that things go well and I can officially write the RFC. As for implementation, Sara Golemon is awesome and while chatting a few months back, knocked out a proof-of-concept implementation <https://github.com/sgolemon/php-src/tree/experimental.toarray>. There's still work to be done if this proposal gets to the RFC phase, but again, just gauging interest here. >> >> I appreciate any feedback you all can provide. >> >> Thanks, >> >> - Steven Wade > > Using existing language functionality, would it satisfy the similar needs if you implement IteratorAggregate and return an ArrayIterator from the getIterator() method? Obviously, the (array) cast wouldn’t use this, but you could do $object->getIterator()->getArrayCopy(). > > I’m not against the proposal. I’m mainly asking if there are other ways in the language today to accomplish similar things. I would prefer to see interfaces used over more magic methods (something like ArraySerializable).
PHP already has the magic built in. The proposal is really just a way to control the magic. We already have "(string) $foo" and "__toString()", so the idea of implementing a "__toArray()" method is bringing a feature more inline with what's already there. I think it reads easier and cleaner "(array) $foo", and a simpler as a developer to implement than a mix of chained methods and interfaces. That being said, adding an interface like `ArraySerializable` might be nice too. I'm seeing technical arguments against adding magic casts, but (just spit-balling here), what if a class implemented a new `ArraySerializable` interace with whatever method it demands, and if that is attempted to be cast to an array either manually by calling "(array) $foo" or PHP attempting to, then the array serialize method is called. ¯\_(ツ)_/¯
  105599
May 6, 2019 14:45 ben@benramsey.com (Ben Ramsey)
> On May 6, 2019, at 09:37, Steven Wade <stevenwadejr@gmail.com> wrote: > > That being said, adding an interface like `ArraySerializable` might be nice too. I'm seeing technical arguments against adding magic casts, but (just spit-balling here), what if a class implemented a new `ArraySerializable` interace with whatever method it demands, and if that is attempted to be cast to an array either manually by calling "(array) $foo" or PHP attempting to, then the array serialize method is called. ¯\_(ツ)_/¯
That’s how I would prefer to see it work. BTW, I’m not suggesting `ArraySerializable` as the name. I used that to make a connection to `JsonSerializable`, which I think did a good job of using an interface for this kind of thing (even though it’s not quite the same because there’s no cast for a JSON type). In some of my open source projects, I’ve had to add `__toString()` to the interfaces to ensure that implementers implement it. It would be nice for my interfaces to instead extend an internal interface like `Stringable`. -Ben
  105616
May 7, 2019 12:25 stevenwadejr@gmail.com (Steven Wade)
> I’m not against the proposal. I’m mainly asking if there are other ways in the language today to accomplish similar things. I would prefer to see interfaces used over more magic methods (something like ArraySerializable).
A Twitter user pointed out that 7.4 is adding two new magic methods <https://wiki.php.net/rfc/custom_object_serialization> - __serialize() and __unserialize(). So adding more magic methods to PHP isn't unprecedented. On the same note, casting is already magical, so controling the implementation via a magic method is more in line with current functionality. -- Steven Wade stevenwadejr@gmail.com
  105622
May 7, 2019 12:58 ocramius@gmail.com (Marco Pivetta)
Hey Steven,


On Tue, 7 May 2019, 14:25 Steven Wade, <stevenwadejr@gmail.com> wrote:

> > I’m not against the proposal. I’m mainly asking if there are other ways > in the language today to accomplish similar things. I would prefer to see > interfaces used over more magic methods (something like ArraySerializable). > > A Twitter user pointed out that 7.4 is adding two new magic methods < > https://wiki.php.net/rfc/custom_object_serialization> - __serialize() and > __unserialize(). So adding more magic methods to PHP isn't unprecedented. > > On the same note, casting is already magical, so controling the > implementation via a magic method is more in line with current > functionality. >
With your current proposal so far, I'd simply have to throw an exception: `if (method_exists($object, '__toArray')) { throw UnsupportedObject::from($object); }` The alternative is to expand the reflection API with an array cast operation in there, and then migrate existing userland usages to that to retain BC. In addition to that, as someone that used to write a lot of `__toString()` (and now doesn't do that anymore), a clear API and interfaced (non-magic) `toArray()` is more powerful and useful. Greets, Marco
  105623
May 7, 2019 13:08 stevenwadejr@gmail.com (Steven Wade)
Hi Marco,

> The alternative is to expand the reflection API with an array cast operation in there, and then migrate existing userland usages to that to retain BC.
That's an interesting proposal - adding new reflection abilities. Hm..
> In addition to that, as someone that used to write a lot of `__toString()` (and now doesn't do that anymore), a clear API and interfaced (non-magic) `toArray()` is more powerful and useful.
I see you and I hear you. After 12 years, still like the magic of PHP. I like that casting is quick, simple, and is less verbose when writing/reading. But I'm not opposed to further discussing possible solutions. If this gets to the RFC stage, I'd prefer it be the best option for the language and most likely to pass. -- Steven Wade stevenwadejr@gmail.com
  105686
May 13, 2019 13:46 stevenwadejr@gmail.com (Steven Wade)
> On May 4, 2019, at 10:58 AM, Steven Wade <stevenwadejr@gmail.com> wrote: > > Hi Internals team! > > I have an idea for a feature that I'd love to see in the language one day and wanted to run the idea by you all. > > The idea is to add a new magic method "__toArray()" that would allow a developer to specifiy how a class is cast to an array. The idea is the same mentality of __toString(), but, for arrays. > > I would personally love this feature and those I've run it by were also excited by the idea. So I'm soliciting feedback in hopes that things go well and I can officially write the RFC. As for implementation, Sara Golemon is awesome and while chatting a few months back, knocked out a proof-of-concept implementation <https://github.com/sgolemon/php-src/tree/experimental.toarray>. There's still work to be done if this proposal gets to the RFC phase, but again, just gauging interest here. > > I appreciate any feedback you all can provide. > > Thanks, > > - Steven Wade
Hi all, I wanted to re-ping the list to see if there is any more feedback on this proposal? Any technical concerns or true BC changes? This feature wouldn't be as exciting as the others in 7.4, but I think it'd be a nice little helper, and the community feedback I've received from developers has been positive, so I'd like to keep the conversation going. -- Steven Wade stevenwadejr@gmail.com
  105688
May 13, 2019 14:17 rowan.collins@gmail.com (Rowan Collins)
On Mon, 13 May 2019 at 14:46, Steven Wade <stevenwadejr@gmail.com> wrote:

> > Hi all, I wanted to re-ping the list to see if there is any more feedback > on this proposal? Any technical concerns or true BC changes? >
I'm personally unconvinced of the value of this, and would probably propose it was blocked by coding standards in my team if it was added, because its meaning is so ambiguous. I actually see quite a lot of classes with normal methods called things like "toArray", and my comment is always "to *what* array?" Most objects do not have a single "natural"/"canonical" array representation, and such a transform is usually actually used as part of some particular helper or code pattern - e.g. an intermediate form for serializing to XML/JSON, or a compatibility-wrapper for legacy code. There's nearly always a better name for the method that properly indicates its intent. As a thought experiment, imagine a similar method which allowed you to overload (object)$foo. Although (array)$foo tells you slightly more than that, I'm not convinced it tells you enough that you're not just hiding meaning behind cute syntax. JsonSerializable actually suffers from similar problems, and is IMO useful only because it's automatically recursive. I presume the proposed mechanism would not be, i.e. return [$foo] would not be interpreted as return [(array)$foo]. Regards, -- Rowan Collins [IMSoP]
  105692
May 13, 2019 18:55 stevenwadejr@gmail.com (Steven Wade)
> I'm personally unconvinced of the value of this, and would probably propose > it was blocked by coding standards in my team if it was added, because its > meaning is so ambiguous.
That's perfectly reasonable. Do you also block use of casting to a string with (string) $foo as well? I ask because this proposal is simply on par with the idea behind string casting for objects. I don't expect that everyone would use custom array casting via __toArray(), but for those that would like to have that control and ease of use, it'd be valuable for them.
> I actually see quite a lot of classes with normal methods called things > like "toArray", and my comment is always "to *what* array?" Most objects do > not have a single "natural"/"canonical" array representation
I think the same could be said about "__toString()". But with that, some classes can be boiled down to a single representation, such as the Ramsey\Uuid <https://github.com/ramsey/uuid/> package. The same is with arrays. You can have a single entity such as person to where its array representation can be first name, last name, age, race, gender, email, etc..., or you can have a collection of items, to where in that representation as an array, you have control over what information is returned and what isn't.
> As a thought experiment, imagine a similar method which allowed you to > overload (object)$foo. Although (array)$foo tells you slightly more than > that, I'm not convinced it tells you enough that you're not just hiding > meaning behind cute syntax.
I'm confused by example, as there's no real need to overload casting to an object as a class is already an object. Whereas, a class is not already an array. It's not about "cute syntax", it's honestly about providing a simple clutter free helper for developers to take control over how their classes are transformed to array representations.
> JsonSerializable actually suffers from similar problems, and is IMO useful > only because it's automatically recursive. I presume the proposed mechanism > would not be, i.e. return [$foo] would not be interpreted as return > [(array)$foo].
You bring up a good point. Could you for a moment pretend like you're behind this proposal and expand upon this question? If PHP were to have a __toArray() method, would you see it as being recursive? In your opinion, how should/would it react? I know not everyone will want or use this feature, but for those who would, it'd be a great addition IMO. So my goal of this initial thread was to see, "what if we had this feature, what would we want it to do?", and go from there. -- Steven Wade stevenwadejr@gmail.com
  105694
May 14, 2019 10:11 rowan.collins@gmail.com (Rowan Collins)
On Mon, 13 May 2019 at 19:55, Steven Wade <stevenwadejr@gmail.com> wrote:

> I'm personally unconvinced of the value of this, and would probably propose > it was blocked by coding standards in my team if it was added, because its > meaning is so ambiguous. > > > That's perfectly reasonable. Do you also block use of casting to a string > with (string) $foo as well? I ask because this proposal is simply on par > with the idea behind string casting for objects. >
I have seen valid uses of __toString(), but I would certainly approach it cautiously. For a complex object, it's not at all obvious if (string)$foo will give you a debug representation, a JSON serialisation, an HTML rendering, etc.
> Most objects do not have a single "natural"/"canonical" array > representation > > > I think the same could be said about "__toString()". But with that, some > classes can be boiled down to a single representation, such as the > Ramsey\Uuid <https://github.com/ramsey/uuid/> package. >
Indeed it could. I think the difference is that a "one-dimensional" object, like a UUID, probably does lend itself to a single canonical string representation. You wouldn't expect it to return XML, or JSON, or any other string format, so (string)$uuid is fairly unambiguous.
> The same is with arrays. You can have a single entity such as person to > where its array representation can be first name, last name, age, race, > gender, email, etc..., >
This is exactly the kind of place I would *not* want a simple toArray() function. Should (array)$person (or $person->toArray()) return ['firstName'=>'Rowan', 'lastName'=>'Collins'], or ['name' => 'Rowan Collins'], or ['name' => ['Rowan', 'Collins']]? What date format should 'dateOfBirth' be formatted to? If 'address' is an object, should that be converted to an object as well, and into what format? The answers to these questions are going to be different in different contexts, and it doesn't make sense for the Person class to determine the "one true array representation" - the only canonical representation is the object itself.
> or you can have a collection of items, to where in that representation as an array, you have control over what information is returned and what
isn't. This is a more reasonable case; given that objects can't completely mimic arrays, I can see value in a custom List class implementing an array cast as a quick "back door" for using existing array functionality.
> As a thought experiment, imagine a similar method which allowed you to > overload (object)$foo. Although (array)$foo tells you slightly more than > that, I'm not convinced it tells you enough that you're not just hiding > meaning behind cute syntax. > > > I'm confused by example, as there's no real need to overload casting to an > object as a class is already an object. Whereas, a class is not already an > array. >
Sure, it's extra vague because "return $this" would be a valid response, but imagine there was something other than objects - structs, or custom resources, or whatever - and there was special syntax to say "give me an object based on this thing". The immediate question would surely be "what object? what are you using it for?" I feel the same way about "give me an array based on this object" - it tells me very little about what you actually want, and why.
> It's not about "cute syntax", it's honestly about providing a simple > clutter free helper for developers to take control over how their classes > are transformed to array representations. >
If it's not recursive, it's just syntactic sugar - which can be fine, if it serves a common use case, but it adds an extra "trick" that readers need to know about. It doesn't let you do anything you can't already - (array)$foo would just be a funny way of spelling $foo->__toArray()
> JsonSerializable actually suffers from similar problems, and is IMO useful > only because it's automatically recursive. I presume the proposed mechanism > would not be, i.e. return [$foo] would not be interpreted as return > [(array)$foo]. > > > You bring up a good point. Could you for a moment pretend like you're > behind this proposal and expand upon this question? If PHP were to have a > __toArray() method, would you see it as being recursive? In your opinion, > how should/would it react? >
A recursive method would certainly have more value, because it actually does something more than translate one syntax to another. On the other hand, the use case that comes to mind is serialization, and we already have more specific methods and systems for that. I guess that's what it comes down to, what *specific* use cases would this feature be intended to help with? Is there some code of your own that inspired you to propose it, or something you've seen publically that would benefit from it? Regards, -- Rowan Collins [IMSoP]
  105906
June 13, 2019 14:23 stevenwadejr@gmail.com (Steven Wade)
Apologies for the super late response:

> A recursive method would certainly have more value, because it actually > does something more than translate one syntax to another. On the other > hand, the use case that comes to mind is serialization, and we already have > more specific methods and systems for that.
How could this new magic method be recursive? If it only works if you manually declare __toArray() in your class, wouldn't you then as the user be in charge of casting anything manually in your method implementation?
> > I guess that's what it comes down to, what *specific* use cases would this > feature be intended to help with? Is there some code of your own that > inspired you to propose it, or something you've seen publically that would > benefit from it?
Originally, it was inspired by seeing Laravel's use of Arrayable as an interface and if something implements that, calling that class' `toArray()` method, and wishing that was built in so that frameworks didn't re-invent the wheel every time. As far as in my code, collections being cast as an array easily would be nice. Models with relationships, being able to implement that cast and control how your model and its children are (or aren't) represented. That's useful for returning an array in a controller for an API, or for simply adding context to a log message. IMO, the point is, it's another tool in the developers arsenal that they can use when they see fit. Not everyone will use it and not everyone will see the benefit of it, and that's ok, but for those that would and could, __toArray() is for them (and me). -- Steven Wade stevenwadejr@gmail.com
  105909
June 13, 2019 14:35 ocramius@gmail.com (Marco Pivetta)
On Thu, Jun 13, 2019 at 4:23 PM Steven Wade <stevenwadejr@gmail.com> wrote:

> > I guess that's what it comes down to, what *specific* use cases would > this > > feature be intended to help with? Is there some code of your own that > > inspired you to propose it, or something you've seen publically that > would > > benefit from it? > > Originally, it was inspired by seeing Laravel's use of Arrayable as an > interface and if something implements that, calling that class' `toArray()` > method, and wishing that was built in so that frameworks didn't re-invent > the wheel every time. >
Interestingly, my work day today is spent mostly removing this kind of behavior from a codebase riddled by it, replacing it with explicit conversions where needed. Marco Pivetta http://twitter.com/Ocramius http://ocramius.github.com/
  105690
May 13, 2019 17:51 ocramius@gmail.com (Marco Pivetta)
Hi Steven,

On Mon, 13 May 2019, 15:46 Steven Wade, <stevenwadejr@gmail.com> wrote:

> > On May 4, 2019, at 10:58 AM, Steven Wade <stevenwadejr@gmail.com> wrote: > > > > Hi Internals team! > > > > I have an idea for a feature that I'd love to see in the language one > day and wanted to run the idea by you all. > > > > The idea is to add a new magic method "__toArray()" that would allow a > developer to specifiy how a class is cast to an array. The idea is the same > mentality of __toString(), but, for arrays. > > > > I would personally love this feature and those I've run it by were also > excited by the idea. So I'm soliciting feedback in hopes that things go > well and I can officially write the RFC. As for implementation, Sara > Golemon is awesome and while chatting a few months back, knocked out a > proof-of-concept implementation < > https://github.com/sgolemon/php-src/tree/experimental.toarray>. There's > still work to be done if this proposal gets to the RFC phase, but again, > just gauging interest here. > > > > I appreciate any feedback you all can provide. > > > > Thanks, > > > > - Steven Wade > > Hi all, I wanted to re-ping the list to see if there is any more feedback > on this proposal? Any technical concerns or true BC changes? > > This feature wouldn't be as exciting as the others in 7.4, but I think > it'd be a nice little helper, and the community feedback I've received from > developers has been positive, so I'd like to keep the conversation going. >
I don't think any of the discussion points mentioned above is resolved: a heavy BC break on the `(array)` cast instead of introducing a clear `SomeArrayableInterfaceType#toArray()` (to be explicitly called) is a no-go from my end, as no clear value is added besides more language complexity, and more mixed cast is encouraged. Greets, Marco
>
  105691
May 13, 2019 18:43 stevenwadejr@gmail.com (Steven Wade)
> I don't think any of the discussion points mentioned above is resolved
I believe that most of the discussion points mentioned have been addressed and resolved. Regarding the concern that an array cast is the only operation capable of exposing object state, the same functionality can be achieved with reflection - as demonstrated here: https://3v4l.org/Dh3PO <https://3v4l.org/Dh3PO>.
> a heavy BC break on the `(array)` cast instead of introducing a clear `SomeArrayableInterfaceType#toArray()` (to be explicitly called) is a no-go from my end
As far as backwards compatibility breaks, I don't believe that's an issue here. My proposal and subsequent conversations state that __toArray() is only called if it is explicitly declared and implemented by a developer, otherwise casting an class to an array behaves exactly as it does today. The PHP documentation on magic methods <https://www.php.net/manual/en/language.oop5.magic.php> even specifically mentions: "PHP reserves all function names starting with __ as magical" and cautions "it is recommended that you do not use function names with __ in PHP unless you want some documented magic functionality." So the language itself reserves the double underscore methods for itself and future versions.
> as no clear value is added besides more language complexity, and more mixed cast is encouraged.
I don't think supporting another magic method and giving developers control over how an object they create is handled when cast to an array adds complexity. It's on par with the already existing __toString() method. Many libraries use this already to determine how their objects are handled as strings, and to make it easy to convert between types without needing to explicitly implement interfaces: such as Ramsey\Uuid <https://github.com/ramsey/uuid>, and League\Uri <https://github.com/thephpleague/uri-components>. The proposal to add a __toArray() isn't meant to add language complexity, it's meant to add an optional helper for developers to simply converting their objects to arrays, and doing so with magic methods isn't without precedence and is more in line with PHP. -- Steven Wade stevenwadejr@gmail.com
  105693
May 13, 2019 18:55 ocramius@gmail.com (Marco Pivetta)
On Mon, 13 May 2019, 20:43 Steven Wade, <stevenwadejr@gmail.com> wrote:

> I don't think any of the discussion points mentioned above is resolved > > > I believe that most of the discussion points mentioned have been addressed > and resolved. Regarding the concern that an array cast is the only > operation capable of exposing object state, the same functionality can be > achieved with reflection - as demonstrated here: https://3v4l.org/Dh3PO. >
That example misses the fact that `ReflectionProperty#getValue()` triggers property access guards, which is an extremely important detail for unset and typed properties. See https://3v4l.org/BtDs5 for an example of why `(array)` is vital in reflection/serialisation layers/libraries.
> a heavy BC break on the `(array)` cast instead of introducing a clear > `SomeArrayableInterfaceType#toArray()` (to be explicitly called) is a no-go > from my end > > > As far as backwards compatibility breaks, I don't believe that's an issue > here. My proposal and subsequent conversations state that __toArray() is > only called if it is explicitly declared and implemented by a developer, > otherwise casting an class to an array behaves exactly as it does today. > The PHP documentation on magic methods > <https://www.php.net/manual/en/language.oop5.magic.php> even specifically > mentions: "PHP reserves all function names starting with __ as magical" and > cautions "it is recommended that you do not use function names with __ in > PHP unless you want some documented magic functionality." So the language > itself reserves the double underscore methods for itself and future > versions. >
See example above on the BC break: existing libraries would need to be adapted to throw eagerly (and reject interactions with) objects implementing `__toArray`.
> as no clear value is added besides more language complexity, and more > mixed cast is encouraged. > > > I don't think supporting another magic method and giving developers > control over how an object they create is handled when cast to an array > adds complexity. It's on par with the already existing __toString() method. > Many libraries use this already to determine how their objects are handled > as strings, and to make it easy to convert between types without needing to > explicitly implement interfaces: such as Ramsey\Uuid > <https://github.com/ramsey/uuid>, and League\Uri > <https://github.com/thephpleague/uri-components>. >
The added complexity comes from consumers: a `(string) $something` cast is problematic if `$something` is of doubtful type, while `$something->toString()` already restricts the possible types of that `$something`. The same accidental complexity comes with unsafe `(array)` casts.
> The proposal to add a __toArray() isn't meant to add language complexity, > it's meant to add an optional helper for developers to simply converting > their objects to arrays, and doing so with magic methods isn't without > precedence and is more in line with PHP. >
Helpers need to be helpful: this ain't, as explained above, since a clear type declaration is a more useful and introspectible way of handling type conversions, especially with the already excessively complex and bloated design of objects in the PHP language. Greets, Marco