Code review request 6858865: Fix for 6728376 causes regression if the size of "data" is 0 and malloc returns Null for 0-length

[Xueming Shen]

On 11/19/2010 01:31, Alan Bateman wrote:
> Xueming Shen wrote:
>> Alan,
>>
>> It might not be a real "regression" if only consider the supported 
>> platforms
>> (and yes, the malloc manpageI can found does clearly indicate NULL is 
>> for error).
>> However I prefer to add some checks to make sure it behaves the same
>> (compared to before the #6728376 change went it), even on the "weird OS"
>> that Mario has. Anyway, a 0-length really malloc should not trigger a 
>> OOME.
>>
>> http://cr.openjdk.java.net/~sherman/6858865/webrev
>>
>> The webrev for #6728376 is at
>> http://cr.openjdk.java.net/~sherman/6728376/webrev
>>
>> Thanks,
>> -Sherman
>>
>>
> I think this one has come up before [1].  Looking at it now, I wonder 
> if it would be simpe for inflate to just return 0 if the input buffer 
> or the max number of uncompressed bytes is 0. That is, just don't 
> attempt the mallocs for these cases.
>
> -Alan
>
>
>
> [1] 
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-July/002028.html

We can probably do that for Inflater.c (and probably better do that at 
java level before
we even "come down" here), but thing gets a little complicated for 
Deflater. One of the
paths of the deflateBytes is to deal with parameter(s) change, so if 
malloc does not
fail for 0-length request (true on all supported platforms), the 
deflateBytes goes down
to zlib's deflateParams() and if there is nothing left to flush, "this" 
invocation can successfully
re-set the param(s) and return 0. The reason why I decided to go with 
the "safe and simple"
solution last night, the proposed webrev at least should behave the 
exactly the same as it
was before.

-Sherman