Re: [PHP-DEV] [RFC] Namespace-scoped declares, again

This is only part of a thread. view whole thread
  106578
August 14, 2019 06:45 michal@brzuchalski.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
Hi Rowan,

wt., 13 sie 2019 o 22:26 Rowan Collins collins@gmail.com> napisał(a):

> On 13/08/2019 18:45, Mark Randall wrote: > > I thought about this as my first consideration, however it effectively > > requires that the PHP code within the package class is fully parsed > > and executed in order for it to retrieve the data. > > > > Consider pre-loading where the code is compiled, but not run, it would > > not be possible to use namespace level defines to add compile-level > > optimizations or checks (at least without some AST hackery). > > > Ah, that makes sense. Does that necessarily mean we need a dummy class, > though? The autoloading logic in the engine knows that it called the > autoload callback expecting a package definition, so can count as > success that the package is now defined. > > In other words, the file the autoloader included would look like this > (again, sticking to the notion that "package" is separate from > "namespace"): > > > # > # File: /lib/company/project1.php > # > > > packagedef company/project1 { > strict_types=1; > strict_operators=1; > upgrade_errors_to_exceptions=E_ALL; > } >
Was thinking a while about proper syntax and would like to mention what I said before. It doesn't have to be a new syntax if we agree to put package definiction in sort of separate configuration file. In the end all what we need is package name and a bunch of declare directives. Given that it could be easily done using separate conf file like: # package.ini [package] name = MyVendor\MyLibrary strict_types = 1 encoding = UTF-8 # package.json { "name": "MyVendor\\MyLibrary", "declare": { "strict_types": 1, "encoding": "UTF-8" } } # package.toml [[package]] name = "MyVendor\\MyLibrary" strict_types = 1 encoding = "UTF-8" # package.yaml name: MyVendor\MyLibrary strict_types: 1 encoding: UTF-8 That way it would prevent to mix normal PHP code with package definition, cause every possible PHP syntax we would agree, opens the door to add something more to that file, for eg.: # package.php MyLibrary:FooBar\Baz; Then it would be possible to extract it and load package definition before class autoload would attempt to load source file and meet: https://wiki.php.net/rfc/namespace_scoped_declares#motivation_and_vision [2] https://www.php.net/manual/en/function.register-tick-function.php -- regards / pozdrawiam, -- Michał Brzuchalski about.me/brzuchal brzuchalski.com
  106582
August 14, 2019 09:01 rowan.collins@gmail.com (Rowan Collins)
On 14/08/2019 07:45, Michał Brzuchalski wrote:
> It doesn't have to be a new syntax if we agree to put package definiction > in sort of separate configuration file.
With the exception of INI files, all your examples are new syntax, as far as the internal mechanisms in PHP are concerned. They have the advantage of not inventing our own, but the disadvantage of needing our parser to handle all the ways people might write them (particularly true for YAML).
> That way it would prevent to mix normal PHP code with package definition, > cause every possible PHP syntax > we would agree, opens the door to add something more to that file, for eg.: > > # package.php > declare package MyVendor\MyLibrary { > strict_types = 1, > encoding = "UTF-8" > } > function foo() {}
I don't see this as a problem. Right now, PHP doesn't care how many files you put your code in. As far as I know, you could concatenate the entirety of Laravel into one PHP file, and applications would not be able to tell the difference. Similarly, you could put the whole thing in a database and use eval() to execute it without touching any files. In the same way, we can have a package definition not as a configuration file, but as a piece of PHP code that needs to have been executed. In practice, people wouldn't write what you have there for the same reason they don't write this: # foo.php 1,        'strict_operators' => 1     ]; } That way, we can probably reuse the whole parser infrastructure, and it looks entirely natural to a PHP programmer. [1]: https://blog.jetbrains.com/phpstorm/2019/02/new-phpstorm-meta-php-features/ -- Rowan Collins [IMSoP]
  106586
August 14, 2019 09:27 michal@brzuchalski.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
śr., 14 sie 2019 o 11:01 Rowan Collins collins@gmail.com> napisał(a):

> I don't see this as a problem. Right now, PHP doesn't care how many > files you put your code in. As far as I know, you could concatenate the > entirety of Laravel into one PHP file, and applications would not be > able to tell the difference. Similarly, you could put the whole thing in > a database and use eval() to execute it without touching any files. > > That is true but not if there are any declare statements. If so it could
IMO work only like that: > I think what attracts me to this idea is precisely that it doesn't > require much extra machinery. We could even use the trick that PHPStorm > uses for metadata stubs [1], and make the package definition look like > valid executable PHP, but never actually execute it: > > package Foo { > const declare = [ > 'strict_types' => 1, > 'strict_operators' => 1 > ]; > } >
How could that know when need look for package definition? Example: # src/Foo.php
  106587
August 14, 2019 09:49 rowan.collins@gmail.com (Rowan Collins)
On 14/08/2019 10:27, Michał Brzuchalski wrote:
> But that's conflicting with you above idea for package definition like > that: > package MyVendorMyLibrary { > declare(strict_types = 1); > }
That's not what I'm suggesting; perhaps my example could have been clearer. I envisage two new keywords: - To put some code in a package, you would write "package Foo;" at the top of a file, or "package Foo { }" around a block, in the same way you can do now for namespaces and declare() statements. - To define a package before it is used, you would write "packagedef Foo { }" around some block of pseudo-PHP that defined the options for that package. That might involve using declare() directly, or might just be an array that defines the options.
> You suggest that it would trigger autoload to load "MyVendor\MyPackage" > but current autoload machinery is able to load only classes, > not even functions or consts! cause it gets the only class name now. > It would need to be changed anyway.
As I said in another reply, this is only the same change that needed to be made to support "trait" alongside "class" and "interface", or would be needed to support "enum" or "struct". The userland part of the autoloader already doesn't know which of those it's looking for, so the only constraint is that the names can't collide, so you couldn't name a package the same thing as a class. There are other problems with autoloading functions and constants, but the relevant point is that you can define function foo() {} and class foo {} in the same namespace; as long as you couldn't also have a separate package foo{}, it could share the same autoloader as classes, interfaces, and traits. Regards, -- Rowan Collins [IMSoP]
  106589
August 14, 2019 10:07 michal@brzuchalski.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
śr., 14 sie 2019 o 11:49 Rowan Collins collins@gmail.com> napisał(a):

> > You suggest that it would trigger autoload to load "MyVendor\MyPackage" > > but current autoload machinery is able to load only classes, > > not even functions or consts! cause it gets the only class name now. > > It would need to be changed anyway. > > > As I said in another reply, this is only the same change that needed to > be made to support "trait" alongside "class" and "interface", or would > be needed to support "enum" or "struct". The userland part of the > autoloader already doesn't know which of those it's looking for, so the > only constraint is that the names can't collide, so you couldn't name a > package the same thing as a class. > > Exactly so how would it know from string name either it should load class
from src/Foo.php or src/__nsmeta.php if there is no information? -- regards / pozdrawiam, -- Michał Brzuchalski about.me/brzuchal brzuchalski.com
  106591
August 14, 2019 10:11 rowan.collins@gmail.com (Rowan Collins)
On 14/08/2019 11:07, Michał Brzuchalski wrote:
> Exactly so how would it know from string name either it should load class > from src/Foo.php or src/__nsmeta.php if there is no information?
It wouldn't. It would include src/Foo.php, and that would have the definition of something with the name "Foo" - either a class, an interface, a trait, or a package. If it wasn't what the engine was expecting, it would be an error, just as it is right now if you write "implements ClassName", or "new TraitName();" Regards, -- Rowan Collins [IMSoP]
  106592
August 14, 2019 10:17 michal@brzuchalski.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
śr., 14 sie 2019 o 12:11 Rowan Collins collins@gmail.com> napisał(a):

> On 14/08/2019 11:07, Michał Brzuchalski wrote: > > Exactly so how would it know from string name either it should load class > > from src/Foo.php or src/__nsmeta.php if there is no information? > > > It wouldn't. It would include src/Foo.php, and that would have the > definition of something with the name "Foo" - either a class, an > interface, a trait, or a package. If it wasn't what the engine was > expecting, it would be an error, just as it is right now if you write > "implements ClassName", or "new TraitName();" >
Following that would introduce unneeded additional directory hierarchy level in a usual library which uses PSR-4 which is the most used one, right? /composer.json /src/Foo.php /src/Foo/ <- all package classes should go here? -- regards / pozdrawiam, -- Michał Brzuchalski about.me/brzuchal brzuchalski.com
  106594
August 14, 2019 10:26 rowan.collins@gmail.com (Rowan Collins)
On 14/08/2019 11:17, Michał Brzuchalski wrote:
> Following that would introduce unneeded additional directory hierarchy > level in a usual library > which uses PSR-4 which is the most used one, right? > > /composer.json > /src/Foo.php > /src/Foo/ <- all package classes should go here?
That would be one place to put it, yes. But it would be entirely up to how people wanted to define their autoloader, that's the beauty of it. There's no reason something couldn't generate an autoloader that essentially said this: function autoload($name) {    if ( $name == self::PACKAGE_NAME ) {       require self::SRC_DIR . '/__packagedef.php';    } elseif ( str_begins_with($name, self::BASE_NAMESPACE) ) {       require self::SRC_DIR . str_replace('\\', '/', $name) . '.php';    } } Please don't pick holes in that implementation; my point is, if this was how packages were implemented, people would decide how they wanted to use it, and PSR-4 would probably be superseded by something which accounted for packages existing. Regards, -- Rowan Collins [IMSoP]
  106595
August 14, 2019 10:26 michal@brzuchalski.com (=?UTF-8?Q?Micha=C5=82_Brzuchalski?=)
śr., 14 sie 2019 o 12:17 Michał Brzuchalski <michal@brzuchalski.com>
napisał(a):

> > > śr., 14 sie 2019 o 12:11 Rowan Collins collins@gmail.com> > napisał(a): > >> On 14/08/2019 11:07, Michał Brzuchalski wrote: >> > Exactly so how would it know from string name either it should load >> class >> > from src/Foo.php or src/__nsmeta.php if there is no information? >> >> >> It wouldn't. It would include src/Foo.php, and that would have the >> definition of something with the name "Foo" - either a class, an >> interface, a trait, or a package. If it wasn't what the engine was >> expecting, it would be an error, just as it is right now if you write >> "implements ClassName", or "new TraitName();" >> > > Following that would introduce unneeded additional directory hierarchy > level in a usual library > which uses PSR-4 which is the most used one, right? > > /composer.json > /src/Foo.php > /src/Foo/ <- all package classes should go here? >
Going even further by the example of Doctrine libraries: # doctrine/collections - Composer Package [1] uses PSR-4 autoload with "Doctrine\\Common\\Collections\\" prefix to adopt package concept it would have to change the prefix to "Doctrine\\Common\\" to find Collections.php and all the rest of source code in Collections directory # doctrine/common - Composer Package [2] uses "Doctrine\\Common\\" prefix should change into "Doctrine\\" to be able to load Common.php and the rest of source code in Common directory # ocramius/package-versions - Composer Package [3] uses "PackageVersions\\" prefix and then what? [1] https://github.com/doctrine/collections/blob/master/composer.json [2] https://github.com/doctrine/common/blob/master/composer.json [3] https://github.com/Ocramius/PackageVersions/blob/master/composer.json -- regards / pozdrawiam, -- Michał Brzuchalski about.me/brzuchal brzuchalski.com
  106861
September 4, 2019 22:16 andreas@dqxtech.net (Andreas Hennings)
I would like to make a wild proposal.

## Proposal

Similar to spl_autoload_register(), we would provide a function that
allows to register a callback to provide the declares / edition /
compatibility information for a given file, before this file is
loaded.

To address performance concerns:
- The result of the callback is cached per file, and per "application
cache key". (*)
- The callback has the option to additionally provide compatibility
information for an entire directory, or a path pattern.

Callback signature:
- The primary parameter is the absolute path of the file, after
symlinks are resolved.
- Additional parameters could provide the version of the path that was
passed to include / require.
- An additional parameter could provide the class name, if we are
currently doing autoload lookup (ouch, this could be complicated).
- An additional parameter could contain the namespace. This would
require that the parsing already begins, before the compatibility
information is available.
- An additional parameter could contain class names and function names
declared in the file. This would imply that the file is already parsed
before the compatibility information is available.

(*) Caching mechanism:
- Different PHP "applications" can have different cache keys, so that
they do not poison each other. This could be provided in the initial
function call along with the callback.
- If no cache key is provided, the starting script path is used (e.g.
/path/to/index.php)
- There needs to be a function to clear this cache for the current application.
- The opcache could contain different versions of a file for different
compatibility levels.

Combination with other proposals:
There could still be functions to set compatibility information for an
entire directory or namespace preemptively.
There could also be ways to disable this special file-by-file callback
for specific directories or namespaces.


## Motivation

We saw proposals where the "edition" or "declares" would be on file
level, on namespace level, on directory level, or on package level.
The basic idea (mostly) was that the author provides compatibility
information which is then picked up by PHP.
The author of a package should be responsible for this, and it should
be protected from side effects from other packages.
Any file or package should have one definitive compatibility info,
which should not be changed from outside.

The proposal turns this on its head.
It is now the responsibility of the package manager (e.g. Composer) or
the framework to organize different compatibility levels in the
application.

Once some conventions have been established by the community (e.g.
Composer), the IDE can extract information from e.g. composer.json to
apply the appropriate checks on each file.

Benefits:
- Fine-grained control where needed, e.g. override the directory-level
or namespace-level compatibility information for specific files.
- Keep boilerplate out of the file header. No need to update all files
when a new compatibility type / edition is released.
- No verbose and noisy git commits in the package history.
- Option for shared compatibility information across "packages", e.g.
for the entire application.
- No need for an artificial "package" concept on language level. Let
community tools (e.g. Composer) define what a package is.
- Possibility to test-run an application or specific files with
different compatibility level.
- Possibility to control compatibility level based on dynamic file
lists, which can be updated based on error statistics, or which can be
updated file by file to slowly make an existing codebase compliant
with a new version.
- The average developer only needs to learn the API of the framework
or package manager, and does not need to deal with this on language
level.

Flaws:
- No explicit compatibility information when looking at a specific file.


## Assumptions

It is a assumed that "compatibility level" does NOT change the meaning of code.
When running a file with a different compatibility level than it was
designed for, we may get errors and warnings, but we do not want to
get silent changes in behavior.


Now I want to see this idea being barbecued..

On Wed, 14 Aug 2019 at 12:27, Michał Brzuchalski <michal@brzuchalski.com> wrote:
> > śr., 14 sie 2019 o 12:17 Michał Brzuchalski <michal@brzuchalski..com> > napisał(a): > > > > > > > śr., 14 sie 2019 o 12:11 Rowan Collins collins@gmail.com> > > napisał(a): > > > >> On 14/08/2019 11:07, Michał Brzuchalski wrote: > >> > Exactly so how would it know from string name either it should load > >> class > >> > from src/Foo.php or src/__nsmeta.php if there is no information? > >> > >> > >> It wouldn't. It would include src/Foo.php, and that would have the > >> definition of something with the name "Foo" - either a class, an > >> interface, a trait, or a package. If it wasn't what the engine was > >> expecting, it would be an error, just as it is right now if you write > >> "implements ClassName", or "new TraitName();" > >> > > > > Following that would introduce unneeded additional directory hierarchy > > level in a usual library > > which uses PSR-4 which is the most used one, right? > > > > /composer.json > > /src/Foo.php > > /src/Foo/ <- all package classes should go here? > > > > Going even further by the example of Doctrine libraries: > # doctrine/collections - Composer Package [1] > uses PSR-4 autoload with "Doctrine\\Common\\Collections\\" prefix > to adopt package concept it would have to change the prefix to > "Doctrine\\Common\\" > to find Collections.php and all the rest of source code in Collections > directory > > # doctrine/common - Composer Package [2] > uses "Doctrine\\Common\\" prefix should change into "Doctrine\\" to be able > to load > Common.php and the rest of source code in Common directory > > # ocramius/package-versions - Composer Package [3] > uses "PackageVersions\\" prefix and then what? > > [1] https://github.com/doctrine/collections/blob/master/composer.json > [2] https://github.com/doctrine/common/blob/master/composer.json > [3] https://github.com/Ocramius/PackageVersions/blob/master/composer.json > -- > regards / pozdrawiam, > -- > Michał Brzuchalski > about.me/brzuchal > brzuchalski.com