Depth and String handling in ObjectInputFilter

[Remi Forax]

Hi Robert,
first this is the wrong mailing list,
this mailing list is about how to implement the JDK, not how it works,
usually stackoverflow is better for the kind of questions you are asking.

----- Mail original -----
> De: "Robert Olofsson" <robert.olofsson at digpro.se>
> À: "core-libs-dev" <core-libs-dev at openjdk.java.net>
> Envoyé: Vendredi 17 Janvier 2020 16:26:25
> Objet: Depth and String handling in ObjectInputFilter

> Hi!
> 
> I hope this is the correct mailing list, if not please tell me.
> 
> I was looking into using a ObjectInputFilter and try to improve the
> security in some old code that uses serialization. After reading and
> testing a bit I think the filtering works a bit strange and that it
> does not provide enough information.
> 
> I have added a small test program below that I think illustrates the
> problem. Running this on java 11 or java 13 gives the same result. The
> program writes a small data structure to a an ObjectOutputStream and
> then reads it in again, just to see what we get from the
> ObjectInputFilter.
> 
> Output of example program:
> roberto at roberto-d4:~/src/java$ ~/pkg/java/jdk-13.0.1+9/bin/java
> SerializationTest
> Depth: 1, class: class Payload, array: -1
> Depth: 2, class: class java.util.ArrayList, array: -1
> Depth: 2, class: class [Ljava.lang.Object;, array: 2
> Depth: 3, class: class X, array: -1
> Depth: 3, class: null, array: -1
> 
> Problems:
> 1) It is not possible to filter long strings. So even when using
>   a filter a long string will lead to OutOfMemoryError. The
>   javadoc for ObjectInputFilter clearly states that this is how
>   it is supposed to work. My thoughts are that I ought to be
>   able to reject too long strings, at least TC_LONGSTRING that
>   have an 8 byte length if I understand how ObjectInputStream
>   works internally.

If you want to avoid long strings, the best is to detect big array of char[],
so using a combination  of arrayLength() + serialClass() should give yo what you want.
A more brutal solution is to use streamBytes() and reject any big values, independent of the serialClass().

> 
> 2) I do not understand how the depth is supposed to work in the
>   example program. I get depth 2 for both the ArrayList and its
>   contained Object[]. My expectations is that I ought to get the
>   array at depth 3. Is depth 2 how it is supposed to work?

It's the depth of the serialized graph, not the depth of the object in memory.
ArrayList (like any lists) replace itself by an array of objects during the serialization,
that why the depth is 2 for ArrayList and LObject.

> 
> Do you have any clarification how should I think regarding the
> ObjectInputFilter?
> 
> Thanks!
> /robo

regards,
Rémi