High performance function autoloading

  105757
May 20, 2019 17:17 theodorejb@outlook.com (Theodore Brown)
Every time function autoloading been brought up in the past, there
have been concerns about performance issues when calling a global
function from inside a namespace. E.g. calling `strlen` in a loop
would become far slower if every call has to trigger the autoloader.

But isn't it extremely rare to override global functions in a
namespace? And even if they are overridden, how common is it to call
an overridden function in the same namespace that exists in a
different file?

Would it be possible to handle this rare case with a per-file option
like `declare(root_fallback=0);`?

Consider the following code examples:

# example_1.php
```php
namespace Foo;

echo strlen('test');
// PHP would first check if Foo\strlen exists. If not, it would check
// whether \strlen exists. Since it does, it would call the built-in
// function as expected without triggering the autoloader.
```

# example_2.php
```php
namespace Foo;

function strlen(string $str) {...}

echo strlen('test');
// PHP would first check if Foo\strlen exists. Since it is defined in
// the same file, it would be called without falling back to the
// global function or triggering the autoloader.
```

# example_3.php
```php
namespace Foo;

use function Bar\strlen;

echo strlen('test');
// PHP would first check if Bar\strlen exists. If not, it would
// trigger the autoloader rather than falling back to the global
// function. The same thing would occur for any qualified function
// call. E.g. without "use function" the following could be written:
echo \Bar\strlen('test');
```

# example_4.php
```php
namespace Foo;

echo my_function('test');
// PHP would first check if Foo\my_function exists. If not, it would
// check whether \my_function exists. If not, the autoloader would be
// triggered.
```

# example_5.php
```php
declare(root_fallback=0);

namespace Foo;

echo strlen('test');
// PHP would first check if Foo\strlen exists. If not, it would
// trigger the autoloader rather than falling back to the global
// function, since root fallback is disabled in the file.
```

What are your thoughts about this approach?

Personally I would rarely have to disable root fallback, since I like
to group functions in a single file per namespace. But it seems like
this would be a good way to enable function autoloading that works
with namespaces spread across multiple files and doesn't slow down
performance.
  105758
May 20, 2019 17:56 ocramius@gmail.com (Marco Pivetta)
Hey Theodore,

On Mon, May 20, 2019 at 7:17 PM Theodore Brown <theodorejb@outlook.com>
wrote:

> Every time function autoloading been brought up in the past, there > have been concerns about performance issues when calling a global > function from inside a namespace. E.g. calling `strlen` in a loop > would become far slower if every call has to trigger the autoloader. > > But isn't it extremely rare to override global functions in a > namespace? And even if they are overridden, how common is it to call > an overridden function in the same namespace that exists in a > different file? > > Would it be possible to handle this rare case with a per-file option > like `declare(root_fallback=0);`? > > Consider the following code examples: > > # example_1.php > ```php > namespace Foo; > > echo strlen('test'); > // PHP would first check if Foo\strlen exists. If not, it would check > // whether \strlen exists. Since it does, it would call the built-in > // function as expected without triggering the autoloader. > ``` > > # example_2.php > ```php > namespace Foo; > > function strlen(string $str) {...} > > echo strlen('test'); > // PHP would first check if Foo\strlen exists. Since it is defined in > // the same file, it would be called without falling back to the > // global function or triggering the autoloader. > ``` > > # example_3.php > ```php > namespace Foo; > > use function Bar\strlen; > > echo strlen('test'); > // PHP would first check if Bar\strlen exists. If not, it would > // trigger the autoloader rather than falling back to the global > // function. The same thing would occur for any qualified function > // call. E.g. without "use function" the following could be written: > echo \Bar\strlen('test'); > ``` > > # example_4.php > ```php > namespace Foo; > > echo my_function('test'); > // PHP would first check if Foo\my_function exists. If not, it would > // check whether \my_function exists. If not, the autoloader would be > // triggered. > ``` > > # example_5.php > ```php > declare(root_fallback=0); > > namespace Foo; > > echo strlen('test'); > // PHP would first check if Foo\strlen exists. If not, it would > // trigger the autoloader rather than falling back to the global > // function, since root fallback is disabled in the file. > ``` > > What are your thoughts about this approach? > > Personally I would rarely have to disable root fallback, since I like > to group functions in a single file per namespace. But it seems like > this would be a good way to enable function autoloading that works > with namespaces spread across multiple files and doesn't slow down > performance. >
I do like the approach, as it makes the (better) fallback-less function usage opt-in, which is currently only fixed by coding-standards tooling. To be clear, would this lead to a "Fatal error: Uncaught Error: Call to undefined function" in case of autoloader failure? Greets, Marco Pivetta http://twitter.com/Ocramius http://ocramius.github.com/
  105764
May 20, 2019 19:11 theodorejb@outlook.com (Theodore Brown)
On Mon, May 20, 2019 at 12:56 PM Marco Pivetta <ocramius@gmail.com> wrote:

> I do like the approach, as it makes the (better) fallback-less function > usage opt-in, which is currently only fixed by coding-standards tooling.
> To be clear, would this lead to a "Fatal error: Uncaught Error: Call > to undefined function" in case of autoloader failure?
Yes, that would be my expectation. Though I didn't mention it in my first email, the same approach should work for autoloading constants as well as functions. If others are in favor, I'd be happy to collaborate on writing an RFC. I'd need someone more experienced with PHP's internals to help with the implementation, though. Regards, Theodore
  105759
May 20, 2019 18:00 gadelat@gmail.com (Gabriel O)
On 20 May 2019 7:17:58 PM Theodore Brown <theodorejb@outlook.com> wrote:

> Every time function autoloading been brought up in the past, there > have been concerns about performance issues when calling a global > function from inside a namespace. E.g. calling `strlen` in a loop > would become far slower if every call has to trigger the autoloader.
This trick for perf improvement is overblown. It's misconception that it does provide speed advantage for most functions and reasons behind it. It does so only for those implemented as opcodes. People started to abuse it by importing ALL functions. Such overzealous approach completely prevents useful things like function mocking.
  105760
May 20, 2019 18:42 theodorejb@outlook.com (Theodore Brown)
On Mon, May 20, 2019 at 1:00 PM Gabriel O <gadelat@gmail.com> wrote:

> > Every time function autoloading been brought up in the past, there > > have been concerns about performance issues when calling a global > > function from inside a namespace. E.g. calling `strlen` in a loop > > would become far slower if every call has to trigger the autoloader. > > This trick for perf improvement is overblown. It's misconception that it > does provide speed advantage for most functions and reasons behind it. It > does so only for those implemented as opcodes. People started to abuse it > by importing ALL functions. Such overzealous approach completely prevents > useful things like function mocking.
Can you clarify? I'm not sure what performance improvement trick you are referring to, or what you mean by "importing all functions". Are you saying it's a misconception that triggering an autoloader on unqualified function calls in a namespace will slow down performance? As I understand it, this is the roadblock for function autoloading. Best regards, Theodore
  105765
May 20, 2019 19:36 gadelat@gmail.com (Gabriel O)
On 20. May 2019, at 20:42, Theodore Brown <theodorejb@outlook.com> wrote:
> > Can you clarify? I'm not sure what performance improvement trick you > are referring to, or what you mean by "importing all functions". Are > you saying it's a misconception that triggering an autoloader on > unqualified function calls in a namespace will slow down performance? > As I understand it, this is the roadblock for function autoloading. > > Best regards, > Theodore
Yes, this is misconception. Namespace resolutions are cached. First fixer tool bringing support for this was correct: PHP-CS-Fixer auto imports only optimized functions for this very same reason.
> On 20. May 2019, at 20:45, Marco Pivetta <ocramius@gmail.com> wrote: > > Disabling function mocking is good ©️ > > It was a terrible practice in first place, and it is usually done for > impure functions that should be wrapped in integration-tested adapters.
…which dramatically affects design of code. Nobody builds injectable replacements for these calls. As always, you are speaking from POV of library maintainer. As a user of your library, you don’t have perfect design. Nobody has. As a user of the library, I want to have ability to replace every aspect of it without copying it, while preserving simplicity - this is impossible to do, because these goals are in opposition. As a contributor of the library, I have found it impossible to write tests for some parts of very same library, because its code style guide dictates importing all the functions which makes it impossible to mock function call. And replacing them with injectable adapters was deemed overkill (and is subject of BC concerns). Anyways, it doesn’t matter. Disabling function fallback for all functions has very little benefit like I mentioned.
  105766
May 20, 2019 20:30 theodorejb@outlook.com (Theodore Brown)
On Mon, May 20, 2019 at 2:36 PM Gabriel O <gadelat@gmail.com> wrote:

> > Can you clarify? I'm not sure what performance improvement trick you > > are referring to, or what you mean by "importing all functions". Are > > you saying it's a misconception that triggering an autoloader on > > unqualified function calls in a namespace will slow down performance? > > As I understand it, this is the roadblock for function autoloading. > > Yes, this is misconception. Namespace resolutions are cached. First > fixer tool bringing support for this was correct: PHP-CS-Fixer auto > imports only optimized functions for this very same reason.
I'm confused, since this doesn't match what I've read elsewhere. For example, Nikita wrote the following last time function autoloading was discussed: [1]
> Calling the autoloader if a global function with the name exists will > totally kill performance. This means that every call to strpos() or > any of the other functions in the PHP standard library will have to > go through the autoloader first, unless people use fully qualified > names (which, currently, they don't). This is completely out of the > question.
In the same thread, it was suggested that a cache could be used to avoid repeatedly triggering the autoloader, but concerns were also pointed out with this: [2]
> > Of course calling e.g. strpos() should not trigger the auto-loader > > repeatedly - can we cache the information that the auto-loader was > > attempted once during the current script execution? so that e.g. > > only the first call to strpos() triggers the auto-loader? > > I think triggering it even once for every internal function in the > code may be too much. > > > I suppose it would still happen once for every namespace from which > > strpos() gets called, so maybe this optimization doesn't help much. > > Exactly. Also, caching stuff assumes static environment. What if it > changes, e.g. autoloader gets different configuration?
This is why I suggested the alternative of by default falling back to the global namespace for unqualified functions before triggering the autoloader, and allowing this fallback to be disabled for the rare case where functions in a namespace are spread across files and need to override a global function. I don't think this proposal changes anything related to function mocking. As you pointed out, libraries can already prevent mocking by importing or qualifying global function calls. Sincerely, Theodore [1]: https://externals.io/message/94895#94897 [2]: https://externals.io/message/94895#94956
  105769
May 21, 2019 08:16 gadelat@gmail.com (Gabriel O)
> On 20. May 2019, at 22:30, Theodore Brown <theodorejb@outlook.com> wrote: > > On Mon, May 20, 2019 at 2:36 PM Gabriel O <gadelat@gmail.com> wrote: >> >> Yes, this is misconception. Namespace resolutions are cached. First >> fixer tool bringing support for this was correct: PHP-CS-Fixer auto >> imports only optimized functions for this very same reason. > > I'm confused, since this doesn't match what I've read elsewhere. For > example, Nikita wrote the following last time function autoloading was > discussed: [1] > >> Calling the autoloader if a global function with the name exists will >> totally kill performance. This means that every call to strpos() or >> any of the other functions in the PHP standard library will have to >> go through the autoloader first, unless people use fully qualified >> names (which, currently, they don't). This is completely out of the >> question.
Check https://www.reddit.com/r/PHP/comments/b107c5/optimizing_your_php_app_speed/eim2lti <https://www.reddit.com/r/PHP/comments/b107c5/optimizing_your_php_app_speed/eim2lti> which is also comment from nikic. I guess there is some confusion core maintainers should clear up. Anyway, my point was if it improves perf of only some functions, this config should be named differently and not have impact on non-optimized ones.
  105761
May 20, 2019 18:45 ocramius@gmail.com (Marco Pivetta)
On Mon, 20 May 2019, 20:01 Gabriel O, <gadelat@gmail.com> wrote:

> > On 20 May 2019 7:17:58 PM Theodore Brown <theodorejb@outlook.com> wrote: > > > Every time function autoloading been brought up in the past, there > > have been concerns about performance issues when calling a global > > function from inside a namespace. E.g. calling `strlen` in a loop > > would become far slower if every call has to trigger the autoloader. > > This trick for perf improvement is overblown. It's misconception that it > does provide speed advantage for most functions and reasons behind it. It > does so only for those implemented as opcodes. People started to abuse it > by importing ALL functions. Such overzealous approach completely prevents > useful things like function mocking. >
Disabling function mocking is good ©️ It was a terrible practice in first place, and it is usually done for impure functions that should be wrapped in integration-tested adapters.
  105762
May 20, 2019 19:01 mo.mu.wss@gmail.com ("M. W. Moe")
Hello,

"Disabling function mocking is good", in my life I read many idioties; this
one gets the "Palme d'Or";
This majesty; crowned Emperor.

On Mon, May 20, 2019 at 11:45 AM Marco Pivetta <ocramius@gmail.com> wrote:

> On Mon, 20 May 2019, 20:01 Gabriel O, <gadelat@gmail.com> wrote: > > > > > On 20 May 2019 7:17:58 PM Theodore Brown <theodorejb@outlook.com> wrote: > > > > > Every time function autoloading been brought up in the past, there > > > have been concerns about performance issues when calling a global > > > function from inside a namespace. E.g. calling `strlen` in a loop > > > would become far slower if every call has to trigger the autoloader. > > > > This trick for perf improvement is overblown. It's misconception that it > > does provide speed advantage for most functions and reasons behind it. It > > does so only for those implemented as opcodes. People started to abuse it > > by importing ALL functions. Such overzealous approach completely prevents > > useful things like function mocking. > > > > Disabling function mocking is good ©️ > > It was a terrible practice in first place, and it is usually done for > impure functions that should be wrapped in integration-tested adapters. >
  105767
May 20, 2019 20:54 Danack@basereality.com (Dan Ackroyd)
On Mon, 20 May 2019 at 20:01, M. W. Moe wss@gmail.com> wrote:
> > *abusive stuff that shouldn't be on this list*
Please don't troll. You might be aiming for being amusing, but you're just being disruptive. cheers Dan Ack
  105763
May 20, 2019 19:06 sebastian@php.net (Sebastian Bergmann)
On 5/20/19 8:45 PM, Marco Pivetta wrote:
> Disabling function mocking is good ©️ > > It was a terrible practice in first place, and it is usually done for > impure functions that should be wrapped in integration-tested adapters.
Hear, hear :)