Towards a JSON API for the JDK
Ethan McCue
ethan at mccue.dev
Sat May 17 05:58:37 UTC 2025
I fail to see a fundamental difference in level (high vs. low) - we should
consider extensibility, ergonomics, and how people are likely to
use the API.
"It gets in the way of laziness" is a valid reason to have JsonObject not
be a Map<String, Json> (though I still want to interrogate it), "the idea
is to use delegation" and a desire for "DOP over OOP" is not.
This is part of why fromUntyped/toUntyped is uncomfortable - it's choosing
one particular encoding of JSON and making it central to the API. Is that
really the most important encoding?
And I get a lot of mileage out of Json extending JsonEncodable - both in
terms of ergonomics for constructing Json structures (being able to turn a
List<Person> into Json using the same overload as you would a List<Json>,
etc.) as well as an extension point for interacting with other json
libraries (easy to teach jackson and co. to treat something implementing an
interface differently).
On Fri, May 16, 2025 at 2:26 PM <forax at univ-mlv.fr> wrote:
>
>
> ------------------------------
>
> *From: *"Ethan McCue" <ethan at mccue.dev>
> *To: *"Remi Forax" <forax at univ-mlv.fr>
> *Cc: *"Paul Sandoz" <paul.sandoz at oracle.com>, "core-libs-dev" <
> core-libs-dev at openjdk.org>
> *Sent: *Friday, May 16, 2025 1:44:52 AM
> *Subject: *Re: Towards a JSON API for the JDK
>
> I present for your consideration the library I made when spiraling about
> this problem space a few years ago
>
> https://github.com/bowbahdoe/json
>
>
> https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html
>
> Notably missing during the design process here were patterns, hence the
> JsonDecoder design. I haven't been able to evaluate how patterns affect
> that on account of them not being out.
>
> I will more thoroughly peruse the draft of java.util.json at a later date,
> but my initial observations/comments:
>
> * I am not sure having JsonValue be distinct from Json has value.
> * toUntyped feels a little strange to me - the only type information
> presumably lost is the sealed-ness of the hierarchy. The interplay between
> that and toNumber is also a little unnerving.
> * One notion that I found helpful was that a class could be "json
> encodable," meaning there is a method to call to obtain a canonical json
> representation.
>
> record Person(String name) implements JsonEncodable {
> @Override
> public Json toJson() {
> return Json.objectBuilder()
> .put("namen", name)
> .build();
> }
> }
>
> Which helper methods like Json#of(List<? extends JsonEncodable>) could
> make use of. Json itself (JsonValue in your prototype) could then have a
> vacuous implementation.
>
> * Terminology wise - I went with reading/writing for the actual
> parsing/generation of json and encoding/decoding for the mapping of those
> representations to/from specific classes. The merits are not top of mind,
> just noting the difference. read/write vs parse/toString+toDisplayString
> * One thing I did was make the helper methods in Json null tolerant and
> the ones in the specific subtypes like JsonString not. This was because
> from what I saw of usages of javax.json/jakarta.json that nullability was a
> footgun and correcting for it required changes to code structure (breaking
> up builder chains with if (x != null) checks)
> * The functionality you want from JsonNumber could be achieved by making
> it just extend Number (
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java)
> instead of a bespoke toNumber. You need the extra methods to go to big
> decimal and co, but it's just an extension to the behavior of Number at
> that point.
> * JsonObject and JsonArray could implement Map<String, Json> and
> List<Json> respectively. This lowers the need for toUntyped() - since
> presumably one of the use cases for that is turning the json tree into
> something that more generic map/list traversal code can handle. It also
> complicates any lazy loading somewhat.
> * Assuming patterns can be placed on interfaces, you might want to
> consider something similar to JsonDecoder, but with a pattern instead of
> a method that throws an exception.
>
> // Where here fromJson would box up the logic for testing and extracting
> from each element in the array.
> List<Person> people = array(json, Person::fromJson);
>
> * I don't think there is sufficient cause for anything to be non-sealed at
> this point.
> * JsonBoolean and JsonNull do not have reasonable alternative
> implementations - as far as I can imagine, maybe i'm wrong - so maybe those
> can just be final classes?
> * If you seal up the whole hierarchy then its pretty trivial to make it
> serializable (
> https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java
> )
>
>
> Hello,
> I think your library is more high-level than java.util.json.
> Moreover, Paul proposes a design closer to DOP (Data Oriented Programming)
> than OOP.
>
> JsonNumber should not extend Number, JsonObject and JsonArray should not
> implements Map and List, the idea is to use delegation instead.
> (BTW extending Number will become an issue in the future if the VM tries
> to flatten filed typed with a value capable abstract class like Number)
>
> JSON is fundamentally untyped and recursive, but also describe data
> structures, for me, the proposed API exposes that structure (the subtypes
> of JsonValue) and make it easy to move from a JSON text to Java objects or
> vice-versa.
> The java.util.json API is very simple,
> - Json.fromUntyped() see a Java object to a structured JSON value (a
> JsonValue),
> - Json.toUntyped() unwrap a structured JSON value to extract the
> underlying Java object.
> - Json.parse() from String to a structured JSON value
> - Json.toDisplayString() (i would prefer "prettyString") goes from a
> structured JSON value to a pretty string.
>
> Having a JsonEncoder and JsonDecoder, JsonDecodable, etc is more high
> level than what is proposed here, so you can write objects like
> JsonEncoder/Decoder on top of it.
> Your example of JsonEncodable can be re-written to use the java.util.json
> API like this
>
> record Person(String name) implements JsonEncodable {
> @Override
> public JsonValue toJson() {
> return Json.fromUntyped(Map.of("name", name));
> }
> }
>
> Compared to your code, java.util.json uses immutable data structures
> instead of a mutable builder (and yes, you lose the order, until
> SequenceMap.of() exists).
>
> regards,
> Rémi
>
>
>
>
> On Thu, May 15, 2025 at 11:29 PM Remi Forax <forax at univ-mlv.fr> wrote:
>
>> Hi Paul,
>> yes, not having a simple JSON API in Java is an issue for beginners.
>>
>> It's not clear to me why JsonArray (for example) has to be an interface
>> instead of a record ?
>>
>> I understand why Json.parse() only works on String and char[] but the API
>> make it too easy to have many performance issues.
>> I think you need versions using a Reader and a Path.
>> Bonus point, if there is a method walk() that also returns a JsonValue
>> but the List/Map inside JsonArray/JsonObject are populated lazily.
>>
>> Minor point: Json.toDisplayString() should takes a second parameters
>> indicating the number of spaces used for the indentation (like
>> JSON.stringify in JS).
>>
>> regards,
>> Rémi
>>
>> ----- Original Message -----
>> > From: "Paul Sandoz" <paul.sandoz at oracle.com>
>> > To: "core-libs-dev" <core-libs-dev at openjdk.org>
>> > Sent: Thursday, May 15, 2025 10:30:42 PM
>> > Subject: Towards a JSON API for the JDK
>>
>> > Hi,
>> >
>> > We would like to share with you our thoughts and plans towards a JSON
>> API for
>> > the JDK.
>> > Please see the document below.
>> >
>> > -
>> >
>> > We have had the pleasure of using a clone of this API in some
>> experiments we are
>> > conducting with
>> > ONNX and code reflection [1]. Using the API we were able to quickly
>> write code
>> > to ingest and convert
>> > a JSON document representing ONNX operation schema into instances of
>> records
>> > modeling the schema
>> > (see here [2]).
>> >
>> > The overall out-of-box experience with such a minimal "batteries
>> included” API
>> > has so far been positive.
>> >
>> > Thanks,
>> > Paul.
>> >
>> > [1] https://openjdk.org/projects/babylon/
>> > [2]
>> >
>> https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87
>> >
>> > # Towards a JSON API for the JDK
>> >
>> > One of the most common requests for the JDK is an API for parsing and
>> generating
>> > JSON. While JSON originated as a text-based serialization format for
>> JSON
>> > objects ("JSON" stands for "JavaScript Object Notation"), because of
>> its simple
>> > and flexible syntax, it eventually found use outside the JavaScript
>> ecosystem as
>> > a general data interchange format, such as framework configuration
>> files and web
>> > service requests/response formats.
>> >
>> > While the JDK cannot, and should not, provide libraries for every
>> conceivable
>> > file format or protocol, the JDK philosophy is one of "batteries
>> included",
>> > which is to say we should be able to write basic programs that use
>> common
>> > protocols such as HTTP, without having to appeal to third party
>> libraries.
>> > The Java ecosystem already has plenty of JSON libraries, so inclusion in
>> > the JDK is largely meant to be a convenience, rather than needing to be
>> the "one
>> > true" JSON library to meet the needs of all users. Users with specific
>> needs
>> > are always free to select one of the existing third-party libraries.
>> >
>> > ## Goals and requirements
>> >
>> > Our primary goal is that the library be simple to use for parsing,
>> traversing,
>> > and generating conformant JSON documents. Advanced features, such as
>> data
>> > binding or path-based traversal should be possible to implement as
>> layered
>> > features, but for simplicity are not included in the core API. We adopt
>> a goal
>> > that the performance should be "good enough", but where performance
>> > considerations conflict with simplicity and usability, we will choose
>> in favor
>> > of the latter.
>> >
>> > ## API design approach
>> >
>> > The description of JSON at `https:://json.org` describes a JSON
>> document using
>> > the familiar "railroad diagram":
>> > 
>> >
>> > This diagram describes an algebraic data type (a sum of products),
>> which we
>> > model directly with a set of Java interfaces:
>> >
>> > ```
>> > interface JsonValue { }
>> > interface JsonArray extends JsonValue { List<JsonValue> values(); }
>> > interface JsonObject extends JsonValue { Map<String, JsonValue>
>> members(); }
>> > interface JsonNumber extends JsonValue { Number toNumber(); }
>> > interface JsonString extends JsonValue { String value(); }
>> > interface JsonBoolean extends JsonValue { boolean value(); }
>> > interface JsonNull extends JsonValue { }
>> > ```
>> >
>> > These interfaces have (hidden) companion implementation classes that
>> admit
>> > greater flexibility of implementation than modeling them directly with
>> records
>> > would permit.
>> > Further, these interfaces are unsealed. We compromise on the sealed sum
>> of
>> > products to enable
>> > alternative implementations, for example to support alternative formats
>> that
>> > encode the same information in a JSON document but in a more efficient
>> form than
>> > text.
>> >
>> > The API has static methods for parsing strings into a `JsonValue`,
>> conversion to
>> > and from purely untyped representations (lists and maps), and factory
>> methods
>> > for building JSON documents. We apply composition consistently, e.g, a
>> > JsonString has a string, a JsonObject has a map of string to JsonValue,
>> as
>> > opposed to extension for structural JSON values.
>> >
>> > It turns out that this simple API is almost all we need for traversal.
>> It gives
>> > us an immutable representation of a document, and we can use pattern
>> matching to
>> > answer the myriad questions that will come up (Does this object have
>> key X? Does
>> > it map to a number? Is that number representable as an integer?) when
>> going
>> > from an untyped format like JSON to a more strongly typed domain model.
>> > Given a simple document like:
>> >
>> > ```
>> > {
>> > "name": "John”,
>> > "age": 30
>> > }
>> > ```
>> >
>> > we can parse and traverse the document as follows:
>> >
>> > ```
>> > JsonValue doc = Json.parse(inputString);
>> > if (doc instanceof JsonObject o
>> > && o.members().get("name") instanceof JsonString s
>> > && s.value() instanceof String name
>> > && o.members().get("age") instanceof JsonNumber n
>> > && n.toNumber() instanceof Long l && l instanceof int age) {
>> > // use "name" and "age"
>> > }
>> > ```
>> >
>> > Later, when the language acquires the ability to expose deconstruction
>> patterns
>> > for arbitrary interfaces (similar to today's record patterns, see
>> >
>> https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns
>> ),
>> > this will be simplifiable to:
>> >
>> > ```
>> > JsonValue doc = Json.parse(inputString);
>> > if (doc instanceof JsonObject(var members)
>> > && members.get("name") instanceof JsonString(String name)
>> > && members.get("age") instanceof JsonNumber(int age)) {
>> > // use "name" and "age"
>> > }
>> > ```
>> >
>> > So, overtime, as more pattern matching features are introduced we
>> anticipate
>> > improved use of the API. This is a primary reason why the API is so
>> minimal.
>> > Convenience methods we add today, such as a method that accesses a JSON
>> > object component as say a JSON string or throws an exception, will
>> become
>> > redundant in the future.
>> >
>> > ## JSON numbers
>> >
>> > The specification of JSON number makes no explicit distinction between
>> integral
>> > and decimal numbers, nor specifies limits on the size of those numbers.
>> > This is a common source of interoperability issues when consuming JSON
>> > documents. Generally users cannot always but often do assume JSON
>> numbers are
>> > parsable, without loss of precision, to IEEE double-precision floating
>> point
>> > numbers or 32-bit signed integers.
>> >
>> > In this respect the API provides three means to operate on the JSON
>> number,
>> > giving the user full control:
>> >
>> > 1. Underlying string representation can be obtained, if preserving
>> syntactic
>> > details such as leading or trailing zeros is important.
>> > 2. The string representation can be parsed to an instance of
>> `BigDecimal`, using
>> > `toBigDecimal` if preserving decimal numbers is important.
>> > 3. The string representation can be parsed into an instance of `Long`,
>> `Double`,
>> > `BigInteger`, or `BigDecimal`, using `toNumber`. The result of this
>> method
>> > depends on how the representation can be parsed, possibly losing
>> precision,
>> > choosing a suitably convenient numeric type that can then be pattern
>> > matched on.
>> >
>> > Primitive pattern matching will help as will further pattern matching
>> features
>> > enabling the user to partially match.
>> >
>> > ## Prototype implementation
>> >
>> > The prototype implementation is currently located into the JDK sandbox
>> > repository
>> > under the `json` branch, see
>> > here
>> >
>> https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json
>> > The prototype API javadoc generated from the repository is also
>> available at
>> >
>> https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html
>> >
>> > ### Testing and conformance
>> >
>> > The prototype implementation passes all conformance test cases but two,
>> > available
>> > on https://github.com/nst/JSONTestSuite. The two exceptions are the
>> ones which
>> > the
>> > prototype specifically prohibits, i.e, duplicated names in JSON objects
>> > (https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35
>> ).
>> >
>> > ### Performance
>> >
>> > Our main focus so far has been on the API design and a functional
>> > implementation.
>> > Hence, there has been less focus on performance even though we know
>> there are a
>> > number of performance enhancements we can make eventually.
>> > We are reasonably happy with the current performance. The
>> > implementation performs well when compared to other JSON implementations
>> > parsing from string instances and traversing documents.
>> >
>> > An example of where we may choose simplicity and usability over
>> performance
>> > is the rejection of JSON documents containing objects that in turn
>> contain
>> > members
>> > with duplicate names. That may increase the cost of parsing, but
>> simplifies the
>> > user
>> > experience for the majority of cases since if we reasonably assume
>> JsonObjects
>> > are
>> > map-like, what should the user do with such members, pick one the last
>> one?
>> > merge
>> > the values? or reject?
>> >
>> > ## A JSON JEP?
>> >
>> > We plan to draft JEP when we are ready. Attentive readers will observe
>> that
>> > a JEP already exists, JEP 198: Light-Weight JSON API
>> > (https://openjdk.org/jeps/198). We will
>> > either update this JEP, or withdraw it and draft a new one.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/core-libs-dev/attachments/20250517/c66591c3/attachment-0001.htm>
More information about the core-libs-dev
mailing list