8243246: HTTP Client sometimes gets java.io.IOException -> Invalid chunk header byte 32

[Daniel Fuchs]

Hi,

Please find below a fix for:

8243246: HTTP Client sometimes gets java.io.IOException -> Invalid
          chunk header byte 32
https://bugs.openjdk.java.net/browse/JDK-8243246

webrev:
http://cr.openjdk.java.net/~dfuchs/webrev_8243246/webrev.00/

Some websites add extraneous white space after the chunk size
in a chunked response. The HttpClient HTTP/1.1 chunk body parser
was expecting CRLF to immediately follow the chunk size and
complains about the extraneous space (ASCII code 32).

Though whitespace after chunk size don't seem to be allowed
by the RFC [1] the chunk size could be followed by a
chunk-ext semi-colon separated list of attributes.

After looking at the legacy HTTP ChunkedInputStream implementation
I decided to handle extensions in the same way:
1. Introduce a maximum value for the number of characters
    that can be found in chunk-size + chunk-ext
2. Simply discard anything between the last character in
    chunk-size and the following CR.

The test contains Lorem-Ipsum-ized versions of the responses
that were sent by the problematic websites listed in the
JBS issue (that is - I kept the structure of the response
and just replaced any content by lorem ipsum data).
I also added an additional response with fake extensions
at the end.

Note: this is HTTP/1.1 only since HTTP/2 doesn't have chunked
responses (DATA frames serve the same purpose).

best regards,

-- daniel


[1] https://tools.ietf.org/html/rfc7230#section-4.1