[PATCH] RFC: XML encoder can cause a StackOverflowError (PR295)

[Andrew Haley]

Omair Majid wrote:

> IcedTea6 and apparently OpenJDK6, IcedTea, OpenJDK and the Xalan-J2 all
> have a bug in their xml encoder, which can result in a
> StackOverflowError [1].
> 
> The test case which tries to encode character 4096 to 321 easily
> reproduces the issue.  The issue appears to be this: When finding the
> encoding for (char) 4096, an EncodingImpl object is created which
> explicitly manages characters 4096 to 4223. So far so good. But when the
> transformer tries to find the encoding for (char) 4095, a new
> EncodingImpl delegate object is created which manages the values 4095 to
> 4222. Effectively, this object only manages the value 4095 (since the
> parent already manages 4096 to 4222). To find the value for 4094, a new
> delegate is created. Do this a few more times, and you have one stack
> frame for each value. Trying to encode characters from 0xffff to 0x0000
> will result in thousands of stack frames.
> 
> The proposed patch makes sure that each delegate manages 128 values
> which dont overlap with any other delegate. The patch brings down the
> maximum number of delegates in the chain to (0xffff + 1)/RANGE = 512
> (also the max number of stack frames that can be used by EndcodingImpl).

Thanks.  This is OK for IcedTea.

Please create an upstream bug for this, attaching the test case and your
patch.

Andrew.