Towards a JSON API for the JDK

forax at univ-mlv.fr forax at univ-mlv.fr
Fri May 16 12:26:54 UTC 2025 

> From: "Ethan McCue" <ethan at mccue.dev>
> To: "Remi Forax" <forax at univ-mlv.fr>
> Cc: "Paul Sandoz" <paul.sandoz at oracle.com>, "core-libs-dev"
> <core-libs-dev at openjdk.org>
> Sent: Friday, May 16, 2025 1:44:52 AM
> Subject: Re: Towards a JSON API for the JDK

> I present for your consideration the library I made when spiraling about this
> problem space a few years ago

> [ https://github.com/bowbahdoe/json | https://github.com/bowbahdoe/json ]

> [
> https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html
> |
> https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html
> ]

> Notably missing during the design process here were patterns, hence the
> JsonDecoder design. I haven't been able to evaluate how patterns affect that on
> account of them not being out.

> I will more thoroughly peruse the draft of java.util.json at a later date, but
> my initial observations/comments:

> * I am not sure having JsonValue be distinct from Json has value.
> * toUntyped feels a little strange to me - the only type information presumably
> lost is the sealed-ness of the hierarchy. The interplay between that and
> toNumber is also a little unnerving.
> * One notion that I found helpful was that a class could be "json encodable,"
> meaning there is a method to call to obtain a canonical json representation.

> record Person(String name) implements JsonEncodable {
> @Override
> public Json toJson() {
> return Json.objectBuilder()
> .put("namen", name)
> .build();
> }
> }

> Which helper methods like Json#of(List<? extends JsonEncodable>) could make use
> of. Json itself ( JsonValue in your prototype) could then have a vacuous
> implementation.

> * Terminology wise - I went with reading/writing for the actual
> parsing/generation of json and encoding/decoding for the mapping of those
> representations to/from specific classes. The merits are not top of mind, just
> noting the difference. read/write vs parse/toString+toDisplayString
> * One thing I did was make the helper methods in Json null tolerant and the ones
> in the specific subtypes like JsonString not. This was because from what I saw
> of usages of javax.json/jakarta.json that nullability was a footgun and
> correcting for it required changes to code structure (breaking up builder
> chains with if (x != null) checks)
> * The functionality you want from JsonNumber could be achieved by making it just
> extend Number ( [
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java
> |
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java
> ] ) instead of a bespoke toNumber . You need the extra methods to go to big
> decimal and co, but it's just an extension to the behavior of Number at that
> point.
> * JsonObject and JsonArray could implement Map<String, Json> and List<Json>
> respectively. This lowers the need for toUntyped() - since presumably one of
> the use cases for that is turning the json tree into something that more
> generic map/list traversal code can handle. It also complicates any lazy
> loading somewhat.
> * Assuming patterns can be placed on interfaces, you might want to consider
> something similar to JsonDecoder , but with a pattern instead of a method that
> throws an exception.

> // Where here fromJson would box up the logic for testing and extracting from
> each element in the array.
> List<Person> people = array(json, Person::fromJson);

> * I don't think there is sufficient cause for anything to be non-sealed at this
> point.
> * JsonBoolean and JsonNull do not have reasonable alternative implementations -
> as far as I can imagine, maybe i'm wrong - so maybe those can just be final
> classes?
> * If you seal up the whole hierarchy then its pretty trivial to make it
> serializable ( [
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java
> |
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java
> ] )

Hello, 
I think your library is more high-level than java.util.json. 
Moreover, Paul proposes a design closer to DOP (Data Oriented Programming) than OOP. 

JsonNumber should not extend Number, JsonObject and JsonArray should not implements Map and List, the idea is to use delegation instead. 
(BTW extending Number will become an issue in the future if the VM tries to flatten filed typed with a value capable abstract class like Number) 

JSON is fundamentally untyped and recursive, but also describe data structures, for me, the proposed API exposes that structure (the subtypes of JsonValue) and make it easy to move from a JSON text to Java objects or vice-versa. 
The java.util.json API is very simple, 
- Json.fromUntyped() see a Java object to a structured JSON value (a JsonValue), 
- Json.toUntyped() unwrap a structured JSON value to extract the underlying Java object. 
- Json.parse() from String to a structured JSON value 
- Json.toDisplayString() (i would prefer "prettyString") goes from a structured JSON value to a pretty string. 

Having a JsonEncoder and JsonDecoder, JsonDecodable, etc is more high level than what is proposed here, so you can write objects like JsonEncoder/Decoder on top of it. 
Your example of JsonEncodable can be re-written to use the java.util.json API like this 

record Person(String name) implements JsonEncodable { 
@Override 
public JsonValue toJson() { 
return Json.fromUntyped(Map.of("name", name)); 
} 
} 

Compared to your code, java.util.json uses immutable data structures instead of a mutable builder (and yes, you lose the order, until SequenceMap.of() exists). 

regards, 
Rémi 

> On Thu, May 15, 2025 at 11:29 PM Remi Forax < [ mailto:forax at univ-mlv.fr |
> forax at univ-mlv.fr ] > wrote:

>> Hi Paul,
>> yes, not having a simple JSON API in Java is an issue for beginners.

>> It's not clear to me why JsonArray (for example) has to be an interface instead
>> of a record ?

>> I understand why Json.parse() only works on String and char[] but the API make
>> it too easy to have many performance issues.
>> I think you need versions using a Reader and a Path.
>> Bonus point, if there is a method walk() that also returns a JsonValue but the
>> List/Map inside JsonArray/JsonObject are populated lazily.

>> Minor point: Json.toDisplayString() should takes a second parameters indicating
>> the number of spaces used for the indentation (like JSON.stringify in JS).

>> regards,
>> Rémi

>> ----- Original Message -----
>>> From: "Paul Sandoz" < [ mailto:paul.sandoz at oracle.com | paul.sandoz at oracle.com ]
>> > >
>>> To: "core-libs-dev" < [ mailto:core-libs-dev at openjdk.org |
>> > core-libs-dev at openjdk.org ] >
>> > Sent: Thursday, May 15, 2025 10:30:42 PM
>> > Subject: Towards a JSON API for the JDK

>> > Hi,

>> > We would like to share with you our thoughts and plans towards a JSON API for
>> > the JDK.
>> > Please see the document below.

>> > -

>> > We have had the pleasure of using a clone of this API in some experiments we are
>> > conducting with
>> > ONNX and code reflection [1]. Using the API we were able to quickly write code
>> > to ingest and convert
>> > a JSON document representing ONNX operation schema into instances of records
>> > modeling the schema
>> > (see here [2]).

>> > The overall out-of-box experience with such a minimal "batteries included” API
>> > has so far been positive.

>> > Thanks,
>> > Paul.

>>> [1] [ https://openjdk.org/projects/babylon/ |
>> > https://openjdk.org/projects/babylon/ ]
>> > [2]
>>> [
>>> https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87
>>> |
>>> https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87
>> > ]

>> > # Towards a JSON API for the JDK

>> > One of the most common requests for the JDK is an API for parsing and generating
>> > JSON. While JSON originated as a text-based serialization format for JSON
>> > objects ("JSON" stands for "JavaScript Object Notation"), because of its simple
>> > and flexible syntax, it eventually found use outside the JavaScript ecosystem as
>> > a general data interchange format, such as framework configuration files and web
>> > service requests/response formats.

>> > While the JDK cannot, and should not, provide libraries for every conceivable
>> > file format or protocol, the JDK philosophy is one of "batteries included",
>> > which is to say we should be able to write basic programs that use common
>> > protocols such as HTTP, without having to appeal to third party libraries.
>> > The Java ecosystem already has plenty of JSON libraries, so inclusion in
>> > the JDK is largely meant to be a convenience, rather than needing to be the "one
>> > true" JSON library to meet the needs of all users. Users with specific needs
>> > are always free to select one of the existing third-party libraries.

>> > ## Goals and requirements

>> > Our primary goal is that the library be simple to use for parsing, traversing,
>> > and generating conformant JSON documents. Advanced features, such as data
>> > binding or path-based traversal should be possible to implement as layered
>> > features, but for simplicity are not included in the core API. We adopt a goal
>> > that the performance should be "good enough", but where performance
>> > considerations conflict with simplicity and usability, we will choose in favor
>> > of the latter.

>> > ## API design approach

>>> The description of JSON at `https::// [ http://json.org/ | json.org ] `
>> > describes a JSON document using
>> > the familiar "railroad diagram":
>>> ![image]( [ https://www.json.org/img/value.png |
>> > https://www.json.org/img/value.png ] )

>> > This diagram describes an algebraic data type (a sum of products), which we
>> > model directly with a set of Java interfaces:

>> > ```
>> > interface JsonValue { }
>> > interface JsonArray extends JsonValue { List<JsonValue> values(); }
>> > interface JsonObject extends JsonValue { Map<String, JsonValue> members(); }
>> > interface JsonNumber extends JsonValue { Number toNumber(); }
>> > interface JsonString extends JsonValue { String value(); }
>> > interface JsonBoolean extends JsonValue { boolean value(); }
>> > interface JsonNull extends JsonValue { }
>> > ```

>> > These interfaces have (hidden) companion implementation classes that admit
>> > greater flexibility of implementation than modeling them directly with records
>> > would permit.
>> > Further, these interfaces are unsealed. We compromise on the sealed sum of
>> > products to enable
>> > alternative implementations, for example to support alternative formats that
>> > encode the same information in a JSON document but in a more efficient form than
>> > text.

>> > The API has static methods for parsing strings into a `JsonValue`, conversion to
>> > and from purely untyped representations (lists and maps), and factory methods
>> > for building JSON documents. We apply composition consistently, e.g, a
>> > JsonString has a string, a JsonObject has a map of string to JsonValue, as
>> > opposed to extension for structural JSON values.

>> > It turns out that this simple API is almost all we need for traversal. It gives
>> > us an immutable representation of a document, and we can use pattern matching to
>> > answer the myriad questions that will come up (Does this object have key X? Does
>> > it map to a number? Is that number representable as an integer?) when going
>> > from an untyped format like JSON to a more strongly typed domain model.
>> > Given a simple document like:

>> > ```
>> > {
>> > "name": "John”,
>> > "age": 30
>> > }
>> > ```

>> > we can parse and traverse the document as follows:

>> > ```
>> > JsonValue doc = Json.parse(inputString);
>> > if (doc instanceof JsonObject o
>> > && o.members().get("name") instanceof JsonString s
>> > && s.value() instanceof String name
>> > && o.members().get("age") instanceof JsonNumber n
>> > && n.toNumber() instanceof Long l && l instanceof int age) {
>> > // use "name" and "age"
>> > }
>> > ```

>> > Later, when the language acquires the ability to expose deconstruction patterns
>> > for arbitrary interfaces (similar to today's record patterns, see
>>> [
>>> https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns
>>> |
>>> https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns
>> > ] ),
>> > this will be simplifiable to:

>> > ```
>> > JsonValue doc = Json.parse(inputString);
>> > if (doc instanceof JsonObject(var members)
>> > && members.get("name") instanceof JsonString(String name)
>> > && members.get("age") instanceof JsonNumber(int age)) {
>> > // use "name" and "age"
>> > }
>> > ```

>> > So, overtime, as more pattern matching features are introduced we anticipate
>> > improved use of the API. This is a primary reason why the API is so minimal.
>> > Convenience methods we add today, such as a method that accesses a JSON
>> > object component as say a JSON string or throws an exception, will become
>> > redundant in the future.

>> > ## JSON numbers

>> > The specification of JSON number makes no explicit distinction between integral
>> > and decimal numbers, nor specifies limits on the size of those numbers.
>> > This is a common source of interoperability issues when consuming JSON
>> > documents. Generally users cannot always but often do assume JSON numbers are
>> > parsable, without loss of precision, to IEEE double-precision floating point
>> > numbers or 32-bit signed integers.

>> > In this respect the API provides three means to operate on the JSON number,
>> > giving the user full control:

>> > 1. Underlying string representation can be obtained, if preserving syntactic
>> > details such as leading or trailing zeros is important.
>> > 2. The string representation can be parsed to an instance of `BigDecimal`, using
>> > `toBigDecimal` if preserving decimal numbers is important.
>> > 3. The string representation can be parsed into an instance of `Long`, `Double`,
>> > `BigInteger`, or `BigDecimal`, using `toNumber`. The result of this method
>> > depends on how the representation can be parsed, possibly losing precision,
>> > choosing a suitably convenient numeric type that can then be pattern
>> > matched on.

>> > Primitive pattern matching will help as will further pattern matching features
>> > enabling the user to partially match.

>> > ## Prototype implementation

>> > The prototype implementation is currently located into the JDK sandbox
>> > repository
>> > under the `json` branch, see
>> > here
>>> [
>>> https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json
>>> |
>>> https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json
>> > ]
>> > The prototype API javadoc generated from the repository is also available at
>>> [
>>> https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html
>>> |
>>> https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html
>> > ]

>> > ### Testing and conformance

>> > The prototype implementation passes all conformance test cases but two,
>> > available
>>> on [ https://github.com/nst/JSONTestSuite | https://github.com/nst/JSONTestSuite
>> > ] . The two exceptions are the ones which
>> > the
>> > prototype specifically prohibits, i.e, duplicated names in JSON objects
>>> ( [ https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35 |
>> > https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35 ] ).

>> > ### Performance

>> > Our main focus so far has been on the API design and a functional
>> > implementation.
>> > Hence, there has been less focus on performance even though we know there are a
>> > number of performance enhancements we can make eventually.
>> > We are reasonably happy with the current performance. The
>> > implementation performs well when compared to other JSON implementations
>> > parsing from string instances and traversing documents.

>> > An example of where we may choose simplicity and usability over performance
>> > is the rejection of JSON documents containing objects that in turn contain
>> > members
>> > with duplicate names. That may increase the cost of parsing, but simplifies the
>> > user
>> > experience for the majority of cases since if we reasonably assume JsonObjects
>> > are
>> > map-like, what should the user do with such members, pick one the last one?
>> > merge
>> > the values? or reject?

>> > ## A JSON JEP?

>> > We plan to draft JEP when we are ready. Attentive readers will observe that
>> > a JEP already exists, JEP 198: Light-Weight JSON API
>> > ( [ https://openjdk.org/jeps/198 | https://openjdk.org/jeps/198 ] ). We will
>> > either update this JEP, or withdraw it and draft a new one.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20250516/e08f6de7/attachment-0001.htm>

More information about the core-libs-dev mailing list