Make namespace part of runtime

  115885
August 27, 2021 19:47 olleharstedt@gmail.com (=?UTF-8?Q?Olle_H=C3=A4rstedt?=)
Hi,

As a followup for my previous post, I think this could be a good place
to start to enable attribute writers to create encapsulation mechanics
for composition, like namespace visibility, "internal" access or
friend classes.

There's an outline to an idea by Nikic here:
https://github.com/php/php-src/pull/947#issuecomment-224934615

> In principle this should be relatively simple: Introduce a struct _zend_namespace { zend_string *name, uint32_t start; uint32_t end; } structure (where start/end are opline offsets), and store an array of these in the op array. Usually this array will have only a single element (or be empty). Generating and storing this data should be relatively simple. The most difficult part is probably adding support for this structure in the opcache optimizer (particularly the block pass), but this can later be done by someone else, who is familiar with the component.
The missing part is exposing it to attribute authors using a magic constant like __NAMESPACE_RUNTIME__, reflection, or something else I didn't think of. :) Eventually, idiomatic attributes could be made part of the core language as new keywords, for performance reasons (instead of #[Internal] attribute, add an "internal" keyword, and so on). Good idea or bad? Olle
  115887
August 27, 2021 21:07 rowan.collins@gmail.com (Rowan Tommins)
On 27/08/2021 20:47, Olle Härstedt wrote:
> As a followup for my previous post, I think this could be a good place > to start to enable attribute writers to create encapsulation mechanics > for composition, like namespace visibility, "internal" access or > friend classes.
In my experience PHP projects tend to use namespace hierarchies which are deep rather than broad, and I'm not sure how easily those hierarchies could be re-purposed as scopes. Clicking through the Symfony source at random for an example, I found "namespace Symfony\Component\Messenger\Transport\Serialization\Normalizer", which contains only one class. A trivial implementation of namespace visibility which just matched the exact namespace string would be completely useless for that class (it would be the same as "private"). A slightly less trivial implementationmight allow access from "child namespaces", but that wouldn't help in this case. However, it might be really useful to restrict usage of that class, or some of its members, to classes in the "Symfony\Component\Messenger" namespace; or maybe to those in the "Symfony\Component\Messenger\Transport\Serialization" namespace. I can see two ways of enabling that: - Allowing visibility modifiers to specify exactly what level of prefix they refer to. This is apparently how Scala approaches things, allowing you to write the equivalent of "private[Symfony\Component\Messenger] $foo; protected[Symfony\Component\Messenger\Transport\Serialization] $bar;" - Introducing a separate concept of "package", either orthogonal to namespaces or overlapping with them, e.g. "package Symfony\Component\Messenger; namespace Transport\Serialization\Normalizer;" would still give a class name beginning "Symfony\Component\Messenger\Transport\Serialization\Normalizer", but would define "internal" to mean accessible within the "Symfony\Component\Messenger" package. I'm personally quite keen on the idea of packages, because they're how a lot of PHP code is developed in practice - as modular libraries linked by Composer - and I think we could optimise the language for that use (e.g. applying more cross-file optimisations within a fully preloaded package). Regards, -- Rowan Tommins [IMSoP]
  115888
August 28, 2021 00:02 mike@newclarity.net (Mike Schinkel)
> On Aug 27, 2021, at 5:07 PM, Rowan Tommins collins@gmail.com> wrote: > > On 27/08/2021 20:47, Olle Härstedt wrote: >> As a followup for my previous post, I think this could be a good place >> to start to enable attribute writers to create encapsulation mechanics >> for composition, like namespace visibility, "internal" access or >> friend classes. > > > In my experience PHP projects tend to use namespace hierarchies which are deep rather than broad, and I'm not sure how easily those hierarchies could be re-purposed as scopes. > > Clicking through the Symfony source at random for an example, I found "namespace Symfony\Component\Messenger\Transport\Serialization\Normalizer", which contains only one class.
I have noticed the same, including in some of my earlier PHP code soon after namespaces became available. I often wonder if those implement deep namespaces did so because "it seemed like a good idea at the time?" I have since worked hard to minimize depth of any PHP namespace hierarchy I have worked with, and as a happy coincidence it reduces the complexity of the code in the projects that use those libraries. #fwiw
> A trivial implementation of namespace visibility which just matched the exact namespace string would be completely useless for that class (it would be the same as "private"). A slightly less trivial implementationmight allow access from "child namespaces", but that wouldn't help in this case. > > However, it might be really useful to restrict usage of that class, or some of its members, to classes in the "Symfony\Component\Messenger" namespace; or maybe to those in the "Symfony\Component\Messenger\Transport\Serialization" namespace. > > I can see two ways of enabling that: > > - Allowing visibility modifiers to specify exactly what level of prefix they refer to. This is apparently how Scala approaches things, allowing you to write the equivalent of "private[Symfony\Component\Messenger] $foo; protected[Symfony\Component\Messenger\Transport\Serialization] $bar;"
+1
> - Introducing a separate concept of "package", either orthogonal to namespaces or overlapping with them, e.g. "package Symfony\Component\Messenger; namespace Transport\Serialization\Normalizer;" would still give a class name beginning "Symfony\Component\Messenger\Transport\Serialization\Normalizer", but would define "internal" to mean accessible within the "Symfony\Component\Messenger" package.
+100
> 'm personally quite keen on the idea of packages, because they're how a lot of PHP code is developed in practice - as modular libraries linked by Composer - and I think we could optimise the language for that use (e.g. applying more cross-file optimisations within a fully preloaded package).
Yes, agree. -Mike
  115892
August 28, 2021 21:12 olleharstedt@gmail.com (=?UTF-8?Q?Olle_H=C3=A4rstedt?=)
2021-08-27 23:07 GMT+02:00, Rowan Tommins collins@gmail.com>:
> On 27/08/2021 20:47, Olle Härstedt wrote: >> As a followup for my previous post, I think this could be a good place >> to start to enable attribute writers to create encapsulation mechanics >> for composition, like namespace visibility, "internal" access or >> friend classes. > > > In my experience PHP projects tend to use namespace hierarchies which > are deep rather than broad, and I'm not sure how easily those > hierarchies could be re-purposed as scopes. > > Clicking through the Symfony source at random for an example, I found > "namespace > Symfony\Component\Messenger\Transport\Serialization\Normalizer", which > contains only one class. > > A trivial implementation of namespace visibility which just matched the > exact namespace string would be completely useless for that class (it > would be the same as "private"). A slightly less trivial > implementationmight allow access from "child namespaces", but that > wouldn't help in this case. > > However, it might be really useful to restrict usage of that class, or > some of its members, to classes in the "Symfony\Component\Messenger" > namespace; or maybe to those in the > "Symfony\Component\Messenger\Transport\Serialization" namespace. > > I can see two ways of enabling that: > > - Allowing visibility modifiers to specify exactly what level of prefix > they refer to. This is apparently how Scala approaches things, allowing > you to write the equivalent of "private[Symfony\Component\Messenger] > $foo; protected[Symfony\Component\Messenger\Transport\Serialization] $bar;"
Yes, I assume this is why Psalm introduced a new annotation @psalm-internal , which does take such a parameter. A PHP attribute can do this, _if_ it has access to runtime namespace.
> - Introducing a separate concept of "package", either orthogonal to > namespaces or overlapping with them, e.g. "package > Symfony\Component\Messenger; namespace > Transport\Serialization\Normalizer;" would still give a class name > beginning > "Symfony\Component\Messenger\Transport\Serialization\Normalizer", but > would define "internal" to mean accessible within the > "Symfony\Component\Messenger" package.
What's the estimated effort for adding package support to PHP? Did anyone outline a possible implementation? Olle
  115893
August 29, 2021 03:12 mike@newclarity.net (Mike Schinkel)
> On Aug 28, 2021, at 5:12 PM, Olle Härstedt <olleharstedt@gmail.com> wrote: > > 2021-08-27 23:07 GMT+02:00, Rowan Tommins collins@gmail.com <mailto:rowan.collins@gmail.com>>: >> On 27/08/2021 20:47, Olle Härstedt wrote: >>> As a followup for my previous post, I think this could be a good place >>> to start to enable attribute writers to create encapsulation mechanics >>> for composition, like namespace visibility, "internal" access or >>> friend classes. >> >> >> In my experience PHP projects tend to use namespace hierarchies which >> are deep rather than broad, and I'm not sure how easily those >> hierarchies could be re-purposed as scopes. >> >> Clicking through the Symfony source at random for an example, I found >> "namespace >> Symfony\Component\Messenger\Transport\Serialization\Normalizer", which >> contains only one class. >> >> A trivial implementation of namespace visibility which just matched the >> exact namespace string would be completely useless for that class (it >> would be the same as "private"). A slightly less trivial >> implementationmight allow access from "child namespaces", but that >> wouldn't help in this case. >> >> However, it might be really useful to restrict usage of that class, or >> some of its members, to classes in the "Symfony\Component\Messenger" >> namespace; or maybe to those in the >> "Symfony\Component\Messenger\Transport\Serialization" namespace. >> >> I can see two ways of enabling that: >> >> - Allowing visibility modifiers to specify exactly what level of prefix >> they refer to. This is apparently how Scala approaches things, allowing >> you to write the equivalent of "private[Symfony\Component\Messenger] >> $foo; protected[Symfony\Component\Messenger\Transport\Serialization] $bar;" > > Yes, I assume this is why Psalm introduced a new annotation > @psalm-internal , which does take such a parameter. A PHP > attribute can do this, _if_ it has access to runtime namespace. > >> - Introducing a separate concept of "package", either orthogonal to >> namespaces or overlapping with them, e.g. "package >> Symfony\Component\Messenger; namespace >> Transport\Serialization\Normalizer;" would still give a class name >> beginning >> "Symfony\Component\Messenger\Transport\Serialization\Normalizer", but >> would define "internal" to mean accessible within the >> "Symfony\Component\Messenger" package. > > What's the estimated effort for adding package support to PHP? Did > anyone outline a possible implementation?
In case you have not seen these, which discuss possibilities but at a higher level than actually how to implement: https://github.com/nikic/php-rfcs/blob/language-evolution/rfcs/0000-language-evolution.md <https://github.com/nikic/php-rfcs/blob/language-evolution/rfcs/0000-language-evolution.md> https://phpinternals.news/45 <https://phpinternals.news/45> -Mike
  115894
August 29, 2021 08:33 olleharstedt@gmail.com (=?UTF-8?Q?Olle_H=C3=A4rstedt?=)
2021-08-29 5:12 GMT+02:00, Mike Schinkel <mike@newclarity.net>:
> > >> On Aug 28, 2021, at 5:12 PM, Olle Härstedt <olleharstedt@gmail.com> >> wrote: >> >> 2021-08-27 23:07 GMT+02:00, Rowan Tommins collins@gmail.com >> <mailto:rowan.collins@gmail.com>>: >>> On 27/08/2021 20:47, Olle Härstedt wrote: >>>> As a followup for my previous post, I think this could be a good place >>>> to start to enable attribute writers to create encapsulation mechanics >>>> for composition, like namespace visibility, "internal" access or >>>> friend classes. >>> >>> >>> In my experience PHP projects tend to use namespace hierarchies which >>> are deep rather than broad, and I'm not sure how easily those >>> hierarchies could be re-purposed as scopes. >>> >>> Clicking through the Symfony source at random for an example, I found >>> "namespace >>> Symfony\Component\Messenger\Transport\Serialization\Normalizer", which >>> contains only one class. >>> >>> A trivial implementation of namespace visibility which just matched the >>> exact namespace string would be completely useless for that class (it >>> would be the same as "private"). A slightly less trivial >>> implementationmight allow access from "child namespaces", but that >>> wouldn't help in this case. >>> >>> However, it might be really useful to restrict usage of that class, or >>> some of its members, to classes in the "Symfony\Component\Messenger" >>> namespace; or maybe to those in the >>> "Symfony\Component\Messenger\Transport\Serialization" namespace. >>> >>> I can see two ways of enabling that: >>> >>> - Allowing visibility modifiers to specify exactly what level of prefix >>> they refer to. This is apparently how Scala approaches things, allowing >>> you to write the equivalent of "private[Symfony\Component\Messenger] >>> $foo; protected[Symfony\Component\Messenger\Transport\Serialization] >>> $bar;" >> >> Yes, I assume this is why Psalm introduced a new annotation >> @psalm-internal , which does take such a parameter. A PHP >> attribute can do this, _if_ it has access to runtime namespace. >> >>> - Introducing a separate concept of "package", either orthogonal to >>> namespaces or overlapping with them, e.g. "package >>> Symfony\Component\Messenger; namespace >>> Transport\Serialization\Normalizer;" would still give a class name >>> beginning >>> "Symfony\Component\Messenger\Transport\Serialization\Normalizer", but >>> would define "internal" to mean accessible within the >>> "Symfony\Component\Messenger" package. >> >> What's the estimated effort for adding package support to PHP? Did >> anyone outline a possible implementation? > > In case you have not seen these, which discuss possibilities but at a higher > level than actually how to implement: > > https://github.com/nikic/php-rfcs/blob/language-evolution/rfcs/0000-language-evolution.md > <https://github.com/nikic/php-rfcs/blob/language-evolution/rfcs/0000-language-evolution.md> > https://phpinternals.news/45 <https://phpinternals.news/45> > > -Mike
Oh cool, thanks! So there's more support in internals for doing package-visibility instead of namespace-visibility in the end? Or maybe the topic is not so hot? Olle