[PATCH] RFC: XML encoder can cause a StackOverflowError (PR295)

[Omair Majid]

Hi,

IcedTea6 and apparently OpenJDK6, IcedTea, OpenJDK and the Xalan-J2 all 
have a bug in their xml encoder, which can result in a 
StackOverflowError [1].

The test case which tries to encode character 4096 to 321 easily 
reproduces the issue.  The issue appears to be this: When finding the 
encoding for (char) 4096, an EncodingImpl object is created which 
explicitly manages characters 4096 to 4223. So far so good. But when the 
transformer tries to find the encoding for (char) 4095, a new 
EncodingImpl delegate object is created which manages the values 4095 to 
4222. Effectively, this object only manages the value 4095 (since the 
parent already manages 4096 to 4222). To find the value for 4094, a new 
delegate is created. Do this a few more times, and you have one stack 
frame for each value. Trying to encode characters from 0xffff to 0x0000 
will result in thousands of stack frames.

The proposed patch makes sure that each delegate manages 128 values
which dont overlap with any other delegate. The patch brings down the
maximum number of delegates in the chain to (0xffff + 1)/RANGE = 512
(also the max number of stack frames that can be used by EndcodingImpl).

Cheers,
Omair

[1] http://icedtea.classpath.org/bugzilla/show_bug.cgi?id=295
-------------- next part --------------
A non-text attachment was scrubbed...
Name: encodinginfo.patch
Type: text/x-patch
Size: 681 bytes
Desc: not available
Url : http://mail.openjdk.java.net/pipermail/distro-pkg-dev/attachments/20090225/c6c76111/encodinginfo.patch