Re: [PHP-DEV] [RFC] token_get_all() TOKEN_AS_OBJECT mode

This is only part of a thread. view whole thread
February 14, 2020 14:41 (Nikita Popov)
On Fri, Feb 14, 2020 at 2:44 PM Marco Pivetta <> wrote:

> On Fri, Feb 14, 2020 at 2:38 PM Sara Golemon <> wrote: > >> Thanks for picking it up, and I agree with your response to Larry. As >> nice >> as it would be to lazy iterate, the scanner is just in NO shape to >> tolerate >> reentering userspace and potentially reinvoking the scanner before the >> first run through is done. >> > > If this is the current state, maybe it would suffice to declare the return > type as `iterable`, and return a strict (fully populated) structure in a > first implementation, later to be changed to an iterator, if applicable? > > Marco Pivetta >
The fact that it returns an array is an important part of the contract. If an iterator variant is added in the future, it should be added as a separate method. I don't want to be writing $tokens = PhpToken::getAll($code); if (!is_array($tokens)) { $tokens = iterator_to_array($tokens); } to convert this to the right type. And I also don't want my usage to be implicitly "upgraded" to an iterator in the future: The iterator will always be less efficient, and I don't want to be forced to use it if I need an array anyway. Nikita