RFR: 8278087: Deserialization filter and filter factory property error reporting under specified

[Roger Riggs]

On Mon, 6 Dec 2021 04:40:01 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> The effects of invalid values of `jdk.serialFilter` and `jdk.serialFilterFactory` properties are 
>> incompletely specified. The behavior for invalid values of the properties is different and
>> use an unconventional exception type, `ExceptionInInitializerError` and leave the `OIF.Config` class
>> uninitialized. 
>> 
>> The exceptions in the `ObjectInputFilter.Config` class initialization caused by invalid values of the two properties, 
>> either by system properties supplied on the command line or security properties are logged.
>> The `Config` class marks either or both the filter and filter factory values as unusable
>> and remembers the exception message.
>> 
>> Subsequent calls to the methods that get or set the filter or filter factory or create 
>> an `ObjectInputStream` throw `java.lang.IllegalStateException` with the remembered exception message.
>> Constructing an `ObjectInputStream` calls both `Config.getSerialFilter` and `Config.getSerialFilterFactory`.
>> The nature of the invalid property is reported as an `IllegalStateException` on first use.
>> 
>> This PR supercedes https://github.com/openjdk/jdk/pull/6508 Document that setting an invalid property jdk.serialFilter disables deserialization
>
> src/java.base/share/classes/java/io/ObjectInputFilter.java line 751:
> 
>> 749:                 if (serialFilter != null) {
>> 750:                     throw new IllegalStateException("Serial filter can only be set once");
>> 751:                 } else if (invalidFilterMessage != null) {
> 
> The `invalidFilterMessage` is a `static` `final` `String` which gets initialized during the static initialization of the class and as such doesn't get changed after that. Do you think this lock on `serialFilterLock` is necessary for this check or it can be moved outside this `synchronized` block?

Checking `serialFilter` for non-null required the synchronization.
The check of `invalidFilterMessage` can be pulled out of the synchronization since in expected usage the method is only called once so the performance isn't significant.

-------------

PR: https://git.openjdk.java.net/jdk/pull/6645