Replaced the bundled libgd with upstream (aka. system) libgd

  101801
February 7, 2018 16:31 cmbecker69@gmx.de ("Christoph M. Becker")
Hi everybody!

During the discussion of PR 3080[1] the idea to replace our bundled
libgd with an (unmodified) upstream libgd[2] has come up again.  It
seems this issue deserves its own discussion, and probably an RFC.

Our bundled libgd (particularly as found in master) is mostly compatible
with upstream libgd 2.2.5 – actually, the API is compatible, but there
are several (mostly minor) implementation differences.  For instance,
the $threshold parameter of imagecropauto($im, IMG_CROP_THRESHOLD) is
interpreted differently[3], the result of imagettfbbox() is slightly
different[4] and imagetruecolortopalette() uses a completely different
algorithm if built with libimagequant (which is not available for the
bundled libgd)[5].  Another important difference is that our bundled
libgd uses ZendMM, but upstream libgd does not[6].

For most Linux environments PHP is built with an upstream (system)
libgd; on Windows usually the bundled libgd is used.  Users targeting
both platforms likely already have conditional code (using `GD_BUNDLED`)
to cater to relevant implementation differences.  So, assuming we would
not set `GD_BUNDLED` anymore after bundling upstream libgd, most users
would not notice a difference.  However, the PHP manual explicitly
recommends to use the bundled libgd[7], so users heeding this advice
would face several minor BC breaks.

The main benefit of bundling upstream libgd, though, would be to save
time by avoiding the 2-way synchronizations (which often occur on a per
commit basis, and may even sometimes be overlooked).

Any thoughts are welcome!

[1] <https://github.com/php/php-src/pull/3080>
[2] <http://libgd.github.io/>
[3] <https://bugs.php.net/73291>
[4] <https://github.com/php/php-src/pull/1845>
[5]
<http://libgd.github.io/manuals/2.2.5/files/gd-h.html#gdPaletteQuantizationMethod>
[6] <https://github.com/libgd/libgd/issues/335>
[7] <http://www.php.net/manual/en/image.installation.php>

-- 
Christoph M. Becker
  101806
February 7, 2018 20:04 smalyshev@gmail.com (Stanislav Malyshev)
Hi!

> During the discussion of PR 3080[1] the idea to replace our bundled > libgd with an (unmodified) upstream libgd[2] has come up again. It > seems this issue deserves its own discussion, and probably an RFC.
I think this is a good idea. In general, as open source ecosystem matures (and more and more people move to use established OSS vendors or non-OSS vendors well-integrated into OSS system, like macOS/homebrew setups) most people move to use distributions which supply wide array of libraries usually covering ones that we used to bundle, and often doing better job in keeping with updates and security fixes than we can. So in general I think phasing out bundling, unless absolutely necessary, is the idea whose time has come.
> bundled libgd)[5]. Another important difference is that our bundled > libgd uses ZendMM, but upstream libgd does not[6].
This one we need to find a solution for. GD is often exposed to the unfiltered user input, has a potential to consume large amounts of memory and not having ZendMM memory limits in place can be a serious issue.
> For most Linux environments PHP is built with an upstream (system) > libgd; on Windows usually the bundled libgd is used. Users targeting
Windows is another concern - are there viable solutions for non-bundled GD for Windows that we can recommend to the users? If not, that means we still have to keep and maintain bundled GD, and if so, there's no point to spend any time on un-bundling before we find solution to this. -- Stas Malyshev smalyshev@gmail.com
  101807
February 7, 2018 22:50 cmbecker69@gmx.de ("Christoph M. Becker")
On 07.02.2018 at 21:04, Stanislav Malyshev wrote:

>> bundled libgd)[5]. Another important difference is that our bundled >> libgd uses ZendMM, but upstream libgd does not[6]. > > This one we need to find a solution for. GD is often exposed to the > unfiltered user input, has a potential to consume large amounts of > memory and not having ZendMM memory limits in place can be a serious issue.
I fully agree. Presumably few users are aware of this, and even if they were, it's not easy to cater to this generally. Until upstream libgd adds an API to attach custom memory allocators, we might have to patch the bundled copy (at least this would be a single, well located modification, instead of the current mess).
>> For most Linux environments PHP is built with an upstream (system) >> libgd; on Windows usually the bundled libgd is used. Users targeting > > Windows is another concern - are there viable solutions for non-bundled > GD for Windows that we can recommend to the users? If not, that means we > still have to keep and maintain bundled GD, and if so, there's no point > to spend any time on un-bundling before we find solution to this.
I don't see a real issue here. All other external PHP dependencies are provided by windows.php.net (amongst others, all GD dependencies, such as libjpeg), so a libgd.dll could also be provided, especially since upstream already provides a native Windows build "toolchain" which already relies on the dependencies provided by windows.php.net (see <https://github.com/libgd/libgd/tree/gd-2.2.5/windows>). -- Christoph M. Becker
  101844
February 12, 2018 17:57 ab@php.net (Anatol Belski)
Hi,

> -----Original Message----- > From: Christoph M. Becker [mailto:cmbecker69@gmx.de] > Sent: Wednesday, February 7, 2018 11:51 PM > To: Stanislav Malyshev <smalyshev@gmail.com>; PHP Internals > <internals@lists.php.net> > Subject: Re: [PHP-DEV] Replaced the bundled libgd with upstream (aka. > system)libgd > > On 07.02.2018 at 21:04, Stanislav Malyshev wrote: > > >> bundled libgd)[5]. Another important difference is that our bundled > >> libgd uses ZendMM, but upstream libgd does not[6]. > > > > This one we need to find a solution for. GD is often exposed to the > > unfiltered user input, has a potential to consume large amounts of > > memory and not having ZendMM memory limits in place can be a serious > issue. > > I fully agree. Presumably few users are aware of this, and even if they were, it's > not easy to cater to this generally. Until upstream libgd adds an API to attach > custom memory allocators, we might have to patch the bundled copy (at least > this would be a single, well located modification, instead of the current mess). > There's gdIOCtx in libgd, which allows usage of the PHP streams. Until similar is available for the memory management, we probably stick with a bundled lib in one or another form. Especially on 64-bit.
> >> For most Linux environments PHP is built with an upstream (system) > >> libgd; on Windows usually the bundled libgd is used. Users targeting > > > > Windows is another concern - are there viable solutions for > > non-bundled GD for Windows that we can recommend to the users? If not, > > that means we still have to keep and maintain bundled GD, and if so, > > there's no point to spend any time on un-bundling before we find solution to > this. > > I don't see a real issue here. All other external PHP dependencies are provided > by windows.php.net (amongst others, all GD dependencies, such as libjpeg), so a > libgd.dll could also be provided, especially since upstream already provides a > native Windows build "toolchain" which already relies on the dependencies > provided by windows.php.net (see <https://github.com/libgd/libgd/tree/gd- > 2.2.5/windows>). > Certainly an external libgd is doable. Once the PHP compatibility is there, libgd could be added to the dependency lists. Same scenario as it was done for libzip in 7.2.
From the current POV, it seems that bundling of the patched upstream version, that supports ZMM, would be the optimal solution for both maintenance and security. The unbundling topic can be revived, once libgd provides custom allocator functionality. Regards Anatol