HTTP/1.1 by default in PHP 8.0

  110237
May 21, 2020 21:53 rowan.collins@gmail.com (Rowan Tommins)
Hi all,

A few years ago, I posted a message suggesting that PHP improve support 
for HTTP/1.1 in its stream wrapper functions: 
https://externals.io/message/96192

A quick summary of the current situation:

* HTTP/1.1 was officially standardised in January 1997, and most web 
browsers had already implemented it by then
* PHP has a very simple HTTP client implementation, used by the "http:" 
and "https:" stream wrappers, and also by extensions which make HTTP 
requests, such as ext/soap
* The client implementation defaults to advertising HTTP/1.0 requests, 
unless over-ridden by a stream context option
* Since a lot of servers only actually talk HTTP/1.1, the client mostly 
acts as an HTTP/1.1 client even when advertising HTTP/1.0


In my previous message, I identified four requirements in HTTP/1.1 but 
not HTTP/1.0 that are relevant to a client:

a) Send a "Host" header with every request. (RFC 7230 Section 5.4)
b) Support persistent connections, or send "Connection: Close" with each 
request. (RFC 7230 Section 6.1)
c) Ignore 1xx status lines (notably, "100 Continue") "even if the client 
does not expect one" (RFC 7231 Section 6.2)
d) Support "chunked" transfer encoding (RFC 7230 Section 4.1)


The PHP client now supports all four regardless of protocol version 
configured, i.e. it always sends "Host:" and "Connection: Close" 
headers; and always handles "100 Continue" and "Transfer-Encoding: 
Chunked" if returned by the server.

I would like to propose that the client advertises HTTP/1.1 in its 
requests by default in PHP 8.0.  Users can opt out of this behaviour in 
a fully backwards- and forwards-compatible way if necessary using a 
stream context option, e.g.: 
https://gist.github.com/IMSoP/a685fed6589435530102d57138755511


What are people's opinions? Does this need an RFC, or should I just 
submit a PR if nobody objects?


Regards,

-- 
Rowan Tommins (né Collins)
[IMSoP]
  110238
May 21, 2020 22:06 larry@garfieldtech.com ("Larry Garfield")
On Thu, May 21, 2020, at 4:53 PM, Rowan Tommins wrote:
> Hi all, > > A few years ago, I posted a message suggesting that PHP improve support > for HTTP/1.1 in its stream wrapper functions: > https://externals.io/message/96192 > > A quick summary of the current situation: > > * HTTP/1.1 was officially standardised in January 1997, and most web > browsers had already implemented it by then > * PHP has a very simple HTTP client implementation, used by the "http:" > and "https:" stream wrappers, and also by extensions which make HTTP > requests, such as ext/soap > * The client implementation defaults to advertising HTTP/1.0 requests, > unless over-ridden by a stream context option > * Since a lot of servers only actually talk HTTP/1.1, the client mostly > acts as an HTTP/1.1 client even when advertising HTTP/1.0 > > > In my previous message, I identified four requirements in HTTP/1.1 but > not HTTP/1.0 that are relevant to a client: > > a) Send a "Host" header with every request. (RFC 7230 Section 5.4) > b) Support persistent connections, or send "Connection: Close" with each > request. (RFC 7230 Section 6.1) > c) Ignore 1xx status lines (notably, "100 Continue") "even if the client > does not expect one" (RFC 7231 Section 6.2) > d) Support "chunked" transfer encoding (RFC 7230 Section 4.1) > > > The PHP client now supports all four regardless of protocol version > configured, i.e. it always sends "Host:" and "Connection: Close" > headers; and always handles "100 Continue" and "Transfer-Encoding: > Chunked" if returned by the server. > > I would like to propose that the client advertises HTTP/1.1 in its > requests by default in PHP 8.0.  Users can opt out of this behaviour in > a fully backwards- and forwards-compatible way if necessary using a > stream context option, e.g.: > https://gist.github.com/IMSoP/a685fed6589435530102d57138755511 > > > What are people's opinions? Does this need an RFC, or should I just > submit a PR if nobody objects? > > > Regards, > > -- > Rowan Tommins (né Collins) > [IMSoP]
This makes complete sense to me. Whether it needs a formal RFC or not is above my pay grade, but I expect it would be just a formality if so. +1 --Larry Garfield
  110240
May 22, 2020 09:55 lear@lear.ch (Eliot Lear)
[2nd try, this time with proper etiquette]

Hi Rowan and Larry and others,

On 21.05.20 23:53, Rowan Tommins wrote:
> > I would like to propose that the client advertises HTTP/1.1 in its > requests by default in PHP 8.0.  Users can opt out of this behaviour > in a fully backwards- and forwards-compatible way if necessary using a > stream context option, e.g.: > https://gist.github.com/IMSoP/a685fed6589435530102d57138755511 > > > What are people's opinions? Does this need an RFC, or should I just > submit a PR if nobody objects? >
To me the only question is whether you want HTTP1.1 or 2 to be the default.  version 2 is a bit more complex, to be sure, but quite powerful in avoiding blocking. Eliot
  110241
May 22, 2020 10:36 maxsem.wiki@gmail.com (Max Semenik)
On Fri, May 22, 2020 at 12:54 AM Rowan Tommins collins@gmail.com>
wrote:

> * PHP has a very simple HTTP client implementation, used by the "http:" > and "https:" stream wrappers, and also by extensions which make HTTP > requests, such as ext/soap
Sorry, I'm an internals noob, but why does PHP implement this at all, instead of using an external library like cURL which is already fairly prominent in our ecosystem? -- Best regards, Max Semenik
  110242
May 22, 2020 10:57 nikita.ppv@gmail.com (Nikita Popov)
On Thu, May 21, 2020 at 11:54 PM Rowan Tommins collins@gmail.com>
wrote:

> Hi all, > > A few years ago, I posted a message suggesting that PHP improve support > for HTTP/1.1 in its stream wrapper functions: > https://externals.io/message/96192 > > A quick summary of the current situation: > > * HTTP/1.1 was officially standardised in January 1997, and most web > browsers had already implemented it by then > * PHP has a very simple HTTP client implementation, used by the "http:" > and "https:" stream wrappers, and also by extensions which make HTTP > requests, such as ext/soap > * The client implementation defaults to advertising HTTP/1.0 requests, > unless over-ridden by a stream context option > * Since a lot of servers only actually talk HTTP/1.1, the client mostly > acts as an HTTP/1.1 client even when advertising HTTP/1.0 > > > In my previous message, I identified four requirements in HTTP/1.1 but > not HTTP/1.0 that are relevant to a client: > > a) Send a "Host" header with every request. (RFC 7230 Section 5.4) > b) Support persistent connections, or send "Connection: Close" with each > request. (RFC 7230 Section 6.1) > c) Ignore 1xx status lines (notably, "100 Continue") "even if the client > does not expect one" (RFC 7231 Section 6.2) > d) Support "chunked" transfer encoding (RFC 7230 Section 4.1) > > > The PHP client now supports all four regardless of protocol version > configured, i.e. it always sends "Host:" and "Connection: Close" > headers; and always handles "100 Continue" and "Transfer-Encoding: > Chunked" if returned by the server. > > I would like to propose that the client advertises HTTP/1.1 in its > requests by default in PHP 8.0. Users can opt out of this behaviour in > a fully backwards- and forwards-compatible way if necessary using a > stream context option, e.g.: > https://gist.github.com/IMSoP/a685fed6589435530102d57138755511 > > > What are people's opinions? Does this need an RFC, or should I just > submit a PR if nobody objects? >
Sounds good to me. Assuming there are no objections, feel free to just send a PR. @Eliot: Unfortunately we don't implement HTTP 2 in the http:// stream wrapper, so HTTP 1.1 is the best we can do there right now. We only provide HTTP 2 support through the curl extension. Implementing HTTP 2 support would certainly be a possibility, but I don't think it's particularly easy to do so without pulling in a dependency like nghttp2. @Max: I'm only guessing here, because I'm not familiar with the historical context, but I imagine part of the motivation is to support HTTP requests in a minimal build of PHP, which does not have any dependencies (that we do not bundle ourselves). We did actually provide an implementation of the http:// stream wrapper via curl for a long time, but dropped it at some point, because there were many subtle behavior differences to our native implementation. Regards, Nikita
  110266
May 23, 2020 07:08 davey@php.net (Davey Shafik)
dopOn Fri, May 22, 2020 at 3:58 AM Nikita Popov ppv@gmail.com>
wrote:

> On Thu, May 21, 2020 at 11:54 PM Rowan Tommins collins@gmail.com> > wrote: > > > Hi all, > > > > A few years ago, I posted a message suggesting that PHP improve support > > for HTTP/1.1 in its stream wrapper functions: > > https://externals.io/message/96192 > > > > A quick summary of the current situation: > > > > * HTTP/1.1 was officially standardised in January 1997, and most web > > browsers had already implemented it by then > > * PHP has a very simple HTTP client implementation, used by the "http:" > > and "https:" stream wrappers, and also by extensions which make HTTP > > requests, such as ext/soap > > * The client implementation defaults to advertising HTTP/1.0 requests, > > unless over-ridden by a stream context option > > * Since a lot of servers only actually talk HTTP/1.1, the client mostly > > acts as an HTTP/1.1 client even when advertising HTTP/1.0 > > > > > > In my previous message, I identified four requirements in HTTP/1.1 but > > not HTTP/1.0 that are relevant to a client: > > > > a) Send a "Host" header with every request. (RFC 7230 Section 5.4) > > b) Support persistent connections, or send "Connection: Close" with each > > request. (RFC 7230 Section 6.1) > > c) Ignore 1xx status lines (notably, "100 Continue") "even if the client > > does not expect one" (RFC 7231 Section 6.2) > > d) Support "chunked" transfer encoding (RFC 7230 Section 4.1) > > > > > > The PHP client now supports all four regardless of protocol version > > configured, i.e. it always sends "Host:" and "Connection: Close" > > headers; and always handles "100 Continue" and "Transfer-Encoding: > > Chunked" if returned by the server. > > > > I would like to propose that the client advertises HTTP/1.1 in its > > requests by default in PHP 8.0. Users can opt out of this behaviour in > > a fully backwards- and forwards-compatible way if necessary using a > > stream context option, e.g.: > > https://gist.github.com/IMSoP/a685fed6589435530102d57138755511 > > > > > > What are people's opinions? Does this need an RFC, or should I just > > submit a PR if nobody objects? > > > > Sounds good to me. Assuming there are no objections, feel free to just send > a PR. > > @Eliot: Unfortunately we don't implement HTTP 2 in the http:// stream > wrapper, so HTTP 1.1 is the best we can do there right now. We only provide > HTTP 2 support through the curl extension. Implementing HTTP 2 support > would certainly be a possibility, but I don't think it's particularly easy > to do so without pulling in a dependency like nghttp2. > > @Max: I'm only guessing here, because I'm not familiar with the historical > context, but I imagine part of the motivation is to support HTTP requests > in a minimal build of PHP, which does not have any dependencies (that we do > not bundle ourselves). > > We did actually provide an implementation of the http:// stream wrapper > via > curl for a long time, but dropped it at some point, because there were many > subtle behavior differences to our native implementation. > > Regards, > Nikita >
This is ridiculously timely as I've been spending my evening working on HTTP/2 stuff in PHP. This might be a good time to reopen discussion of adding support for HTTP/2 in PHP with the inclusion of libnghttp2. I posted a long time ago about adding support for HTTP/2 to the CLI server and the http stream using libnghttp2 [1]. I think PHP 8.0 would be a perfect opportunity to revisit this given that HTTP/2 has now reached ~97% adoption[2] and HTTP/3 is on the horizon. I believe that HTTP/2 has the potential to dramatically change how we serve content on the web, and PHP should jump on the bandwagon. - Davey [1] https://externals.io/message/89932 [2] https://caniuse.com/#search=http%2F2
  110269
May 23, 2020 19:42 ben@benramsey.com (Ben Ramsey)
> On May 23, 2020, at 02:08, Davey Shafik <davey@php.net> wrote: > > dopOn Fri, May 22, 2020 at 3:58 AM Nikita Popov ppv@gmail..com> > wrote: > >>> On Thu, May 21, 2020 at 11:54 PM Rowan Tommins collins@gmail.com> >>> wrote: >>> >>> Hi all, >>> >>> A few years ago, I posted a message suggesting that PHP improve support >>> for HTTP/1.1 in its stream wrapper functions: >>> https://externals.io/message/96192 >>> >>> A quick summary of the current situation: >>> >>> * HTTP/1.1 was officially standardised in January 1997, and most web >>> browsers had already implemented it by then >>> * PHP has a very simple HTTP client implementation, used by the "http:" >>> and "https:" stream wrappers, and also by extensions which make HTTP >>> requests, such as ext/soap >>> * The client implementation defaults to advertising HTTP/1.0 requests, >>> unless over-ridden by a stream context option >>> * Since a lot of servers only actually talk HTTP/1.1, the client mostly >>> acts as an HTTP/1.1 client even when advertising HTTP/1.0 >>> >>> >>> In my previous message, I identified four requirements in HTTP/1.1 but >>> not HTTP/1.0 that are relevant to a client: >>> >>> a) Send a "Host" header with every request. (RFC 7230 Section 5.4) >>> b) Support persistent connections, or send "Connection: Close" with each >>> request. (RFC 7230 Section 6.1) >>> c) Ignore 1xx status lines (notably, "100 Continue") "even if the client >>> does not expect one" (RFC 7231 Section 6.2) >>> d) Support "chunked" transfer encoding (RFC 7230 Section 4.1) >>> >>> >>> The PHP client now supports all four regardless of protocol version >>> configured, i.e. it always sends "Host:" and "Connection: Close" >>> headers; and always handles "100 Continue" and "Transfer-Encoding: >>> Chunked" if returned by the server. >>> >>> I would like to propose that the client advertises HTTP/1.1 in its >>> requests by default in PHP 8.0. Users can opt out of this behaviour in >>> a fully backwards- and forwards-compatible way if necessary using a >>> stream context option, e.g.: >>> https://gist.github.com/IMSoP/a685fed6589435530102d57138755511 >>> >>> >>> What are people's opinions? Does this need an RFC, or should I just >>> submit a PR if nobody objects? >>> >> >> Sounds good to me. Assuming there are no objections, feel free to just send >> a PR. >> >> @Eliot: Unfortunately we don't implement HTTP 2 in the http:// stream >> wrapper, so HTTP 1.1 is the best we can do there right now. We only provide >> HTTP 2 support through the curl extension. Implementing HTTP 2 support >> would certainly be a possibility, but I don't think it's particularly easy >> to do so without pulling in a dependency like nghttp2. >> >> @Max: I'm only guessing here, because I'm not familiar with the historical >> context, but I imagine part of the motivation is to support HTTP requests >> in a minimal build of PHP, which does not have any dependencies (that we do >> not bundle ourselves). >> >> We did actually provide an implementation of the http:// stream wrapper >> via >> curl for a long time, but dropped it at some point, because there were many >> subtle behavior differences to our native implementation. >> >> Regards, >> Nikita >> > > This is ridiculously timely as I've been spending my evening working on > HTTP/2 stuff in PHP. > > This might be a good time to reopen discussion of adding support for HTTP/2 > in PHP with the inclusion of libnghttp2. I posted a long time ago about > adding support for HTTP/2 to the CLI server and the http stream using > libnghttp2 [1]. > > I think PHP 8.0 would be a perfect opportunity to revisit this given that > HTTP/2 has now reached ~97% adoption[2] and HTTP/3 is on the horizon. > > I believe that HTTP/2 has the potential to dramatically change how we serve > content on the web, and PHP should jump on the bandwagon. > > - Davey > > [1] https://externals.io/message/89932 > [2] https://caniuse.com/#search=http%2F2
Agree 100% with Davey on this! -Ben
  110270
May 24, 2020 15:05 rowan.collins@gmail.com (Rowan Tommins)
On 23/05/2020 08:08, Davey Shafik wrote:
> This is ridiculously timely as I've been spending my evening working > on HTTP/2 stuff in PHP. > [...] > I believe that HTTP/2 has the potential to dramatically change how we > serve content on the web, and PHP should jump on the bandwagon.
Hi Davey, I'm glad someone's working on this, because PHP should definitely be trying to keep up with such things. My only hesitation is that other than as a proof-of-concept, just allowing 'protocol_version'=>2 as a stream option isn't going to bring much advantage, because it doesn't give access to any of the interesting new features - it's still a single, one-way, synchronous request. Given it's always been a web programming language, PHP has surprisingly poor native support for HTTP. On the server side, we have the SAPI layer, which delegates most of the protocol to external applications in generally CGI-type ways. On the client side, we have the very minimal stream wrapper (which doesn't even support pipelining with Connection:Keep-Alive), or some painfully low-level bindings around libcurl. To really make use of HTTP/2, I would imagine both would need quite significant re-thinks: On the server side, can the SAPI and FastCGI definitions be extended, or do they need to be replaced, as Python defined ASGI [1] to replace WSGI? On the client side, should there be a better built-in HTTP client, whether based on libcurl or lower-level pieces like nghttp; or do we keep it low-level and expose nghttp2 itself in some way? [1] https://asgi.readthedocs.io/en/latest/introduction.html Regards, -- Rowan Tommins (né Collins) [IMSoP]