February 2016 - core-libs-dev

RE: DeserializationPermission Proposal
by Peter Firmstone 11 Feb '16

11 Feb '16

The use case here is: Do we trust the code to deserialize unverified data from an untrusted source? For example, an attacker might look for the following: 1. Deserialization into privileged context 2. Deserialization with a class that catches and ignores IOException. 3. A hash collission that will cause equals to be invoked on a proxy and invocation handler. History has shown us that no permission was required to perform these actions. In this case, I'm not concerned who or which object might obtain a reference to some object that's serializable, because anyone can do so using a crafted stream reference. An attacker can obtain a reference to any object in the stream using a stream reference. Objects that shouldn't escape a permission check in a constructor or being returned from some method call containing a permission check, shouldn't be serializable. What I'm concerned about is, do we trust all classes belonging to this object's heirarchy to read unverified data from the stream and verify it before creating an object? Ideally the stream should be deserialized into a context that has no permission, but currently we have no way of restricting the attack surface to include only those classes necessary to perform some function. We could use the traditional stack context of the calling thread, as you suggest. It is true, an object that contains a field that didn't have deserialization permission, would itself become unserializable. Regards, Peter. > On 02/08/2016 10:19 PM, Peter Firmstone wrote: > >/ Why not, just prior to instantiating an object just prior to deserializing, add each class' ProtectionDomain in the objects hierarchy to an AccessControlContext and pass this to the SecurityManager's two argument checkPermission call? > />/ > />/ This permission could never be granted to a principal, it is only ever a code trust concern. This would allow an administrator to minimise the attack surface of Serializable classes. > / > I think rather than using the ProtectionDomain of the objects in the > serialization stream, would it not make more sense to capture and use > (exclusively) the access control context of the entity which is > constructing the stream? The reason I say this is that it's very > possible for a less-privileged object to have references to > more-privileged objects and vice-versa; also, in some cases you're > predicating the success of deserialization upon the order in which > objects are deserialized. > > For example, if I have a four-object graph of A, B, C, and D, in a > diamond like this: > > A->B > A->C > B->D > C->D > > If B's class does not have privileges to construct D, deserialization > will fail, even if C does. On the other hand, if B has permission to > construct D, but C doesn't, C can escape its restriction by relying on B > to have already deserialized the object. > > But by using one permission set - the captured context of the creator of > the stream, or perhaps the captured context of the "root" readObject() > call, you cannot change the authorization outcome of the deserialization > just by fiddling with the bits. A graph in this case is either valid or > not. > > -- > - DML

2 2

Compact Strings and APIs for fast decoding of string data
by Chris Vest 11 Feb '16

11 Feb '16

Hi, Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we’d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I’ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. From an API standpoint, CharsetDecoder is close to home, but is not quite there. It’s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I’m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. The CharBuffers are allocated with a length up front, which is nice, but I can’t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. StringDecoder decoder = Charset.forName(“latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don’t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. What do you guys think? Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ <http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/> It doesn’t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It’d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. Cheers, Chris

5 9

RFR 9: 8142539 : Incorrect definition of ZoneOffset.MIN and other javadoc cleanup
by Roger Riggs 10 Feb '16

10 Feb '16

Please review a trio of javadoc fixes for java.time. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-typos-8142539/ Issues: 8142539: Incorrect definition of ZoneOffset.MIN 8140211: Example in the Documentation is wrong for java.time.ZonedDateTime.minusHours 8139529: java.time.temporal.ChronoUnit.FOREVER typo Thanks, Roger

3 2

OpenJDK bug question (JDK-6519463)
by David Hoffer 10 Feb '16

10 Feb '16

We have a production environment where we are getting errors similar to what was reported in JDK-6519463. I would like to test our code with the test file (buggedfile.bin) used in this bug report but the file is posted on an ftp server that is not publicly available. Can this file be attached to the bug report or just email me a copy? Thanks, -Dave

2 5

New Collection framework?
by Thomas Kamps 10 Feb '16

10 Feb '16

Hi everyone, Did you ever think about a new Collection framework for Java? My main complains about the java.util Collections are: * No co/contra variance for assigning. Which forbids assigments like List<Object> foo = new List<Number>(); Currently it can be bypassed when using wildcards, but then you cannot add items to foo. Even more problematic are interfaces with some kind of List<List<...>> as parameter * The main Collection interface allowes too much. If you get a Collection and you should not modify it, your programm will either behave stragly with a hard to detect bug, or crash with some kind of UnsupportedOperationException, when you do it. I have my problems with UnsupprtedOperationException, because when i get a Collection, i have to read the documentation (if existing) to know what i am allowed to do, or run unexpectedly into an exception. Would it be great to hava an "readonly" interface, which simply does not have and modifing methods? The other possible reason could be the Valhalla project, which tries to introduce value types, to enable Collections with primitive type parameters. There is (or was?) a discussion how to handle the remove methods on List: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2015-December/… There is a suggestion for partial methods to solve the problem, but i think partial methods makes the language even more complicated to compile and more difficult to understand. All theese problems could be resolved by just letting the java.util Collection Framework as it is. And make a new one. I have created a proposal for it, which can be easily (modulo class names) used together with the java.util Collections. This is achieved by having wrapper Collections and converter in both directions. https://github.com/Blahord/BetterCollections Note: This is the first release with the target to have a clean and small as possible API. Since it makes heavily use of wildcards, it is some kind of bureaucratic. This could be reduced, by * Adding a shortcur for wildcards like "?Type" or "? extends Type" and "!Type" for "? super Type" or * Allowing co and contra variance for type parameters.

4 5

Re: RFR JDK-8141491: Unaligned memory access in Bits.c
by Coleen Phillimore 10 Feb '16

10 Feb '16

Sending to core-libs mailing list. On 11/25/15 2:19 PM, Alexander Smundak wrote: > Please take a look at > http://cr.openjdk.java.net/~asmundak/8141491/jdk/webrev.00 > that fixes the problem. > > It utilizes the ability of some (GCC and Clang) to declare data > alignment explicitly. > I have verified it works on x86_64 Linux by running > jdk/test/java/nio/Buffer/Basic.java test > > I need a sponsor. > > Sasha

8 26

RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory
by Aleksey Shipilev 10 Feb '16

10 Feb '16

Hi, I would like to make StringConcatFactory more coherent with existing LambdaMetafactory: https://bugs.openjdk.java.net/browse/JDK-8149459 http://cr.openjdk.java.net/~shade/8149459/webrev.01/ Testing: jdk/test/java/String/concat jtreg, JPRT Cheers, -Aleksey

5 6

RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause
by Ramanand Patil 10 Feb '16

10 Feb '16

Hi all Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8135259 Bug Description: Attempts to resolve a host unknown to the DNS server cause an UnknownHostException stating "unknown error" instead of "Name or service not known". Webrev: http://cr.openjdk.java.net/~rpatil/8135259/webrev.00/ Fix: Using a function call directly "gai_strerror(gai_error)" instead of using the function pointer which was declared but not initialized. Apart from this I have removed few other unused variables and function pointers. Brief description of how this bug was introduced is added to the bug comment. Regards, Ramanand.

2 2

RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk repository
by Volker Simonis 10 Feb '16

10 Feb '16

Hi, can I please have a review for this boring change which simply changes all the SAP copyright messages from "SAP AG" to "SAP SE". The reason for this change is that SAP changed from a German stock corporation (AG) to a European one (SE). http://cr.openjdk.java.net/~simonis/webrevs/2016/8149529 https://bugs.openjdk.java.net/browse/JDK-8149529 Thank you and best regards, Volker

3 3

RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance()
by Tagir F. Valeev 10 Feb '16

10 Feb '16

Please review and sponsor this fix: https://bugs.openjdk.java.net/browse/JDK-8148838 http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ When buffer traversal is already started, and split is requested, then the existing buffer should be carefully transferred to the prefix part. The only problem I see here is the testing time. Due to permuteFunctions() call it increases significantly what I added the flatMap() test. If this becomes an issue, I can write new simple test which tests flatMap() only (without permutations with other intermediate operations). With best regards, Tagir Valeev.

3 10