<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<p>Ahh, as expected some great thoughts have gone into this. :-) <br>
</p>
<p>Allowing the ignore of comments when parsing is not "inventing a
new format", IMO. More so because the RFC allows it (section 9)
and because it would be strictly opt-in. So the parser can still
be said to be strictly RFC-8259 while optionally supporting such
feature. I'm speculating the authors of the RFC has thought of it
as a form a pre-processing when they used the word "extension" and
that is indeed a good way to think about it.<br>
</p>
<p>In any case my goal was purely comments in the JSON, not about
allowing other forms of leniency, like trailing commas, flexible
quotes, etc. I feel those are in another category and not
something that should (ever) be supported. I can see why someone
might ask for that if comments where allowed to be ignored by the
lib. The argument about slippery slope can be made. I just fear
that newbies will try to build their own pre-processor, using
regexp or whatever .. and get it wrong.<br>
</p>
<p>I agree with you that JSON5 sneaks in too many odd things. <br>
</p>
<p>Keep up the good work.<br>
</p>
<p>/Lars<br>
</p>
<br>
<div class="moz-cite-prefix">On 16/05/2025 15.17, Brian Goetz wrote:<br>
</div>
<blockquote type="cite"
cite="mid:a5db35c2-800d-4b4a-99f4-91a47008aaac@oracle.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
At first, we were hopeful that we could jump right to JSON5, which
appears at first glance to be a strictly lexical, more permissive
grammar for JSON (supporting comments, trailing commas, more
flexible quoting, white space, etc.) If that were actually true,
this would have been a slam dunk, since all these lexical niceties
don't have an impact on the parsed results. And JSON5 has been
gaining some traction, so it probably could have been a
justifiable move to jump right to that. <br>
<br>
But then we discovered that JSON5 also sneaks in some semantics,
by also supporting the exotic numeric values (NaN, infinities,
signed zero), which now has consequences for "what is a number",
the numeric representation, the API for unpacking numeric values,
etc. (Having multiple parsers is one thing; having multiple
parsers that produce different semantics is entirely another.)
And inventing a new "JSON5 but not quite" subset would be doing no
one any favors. <br>
<br>
Jsonc seems to be entirely a MS-ecosystem thing; it does not have
broad enough traction to be the "one grammar" we accept. So pure
JSON, as specification-challenged as it is, is the logical, though
sad, conclusion (for now.)<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 5/16/2025 9:02 AM, Lars
Bruun-Hansen wrote:<br>
</div>
<blockquote type="cite"
cite="mid:b0e7ff8e-f6dc-48b1-ad0f-5de52d8191b3@apache.org">
<p><br>
Great work.</p>
<p><br>
</p>
<p>I feel the elephant in the room needs to addressed: JSON
comments. I haven't tested the proposed lib but I cannot see
it mentioned so I'm assuming that comments are not supported.<br>
</p>
<p>For better or worse, the use of jsonc (JSON with comments) is
everywhere in some ecosystems. Unsurprisingly this happens
often when JSON is used as a config file format. Looking at
you, Microsoft.</p>
<p>It would be nice if the JDK's build-in JSON parser at least
recognized this.</p>
<p>I'm well aware that comments are frowned upon in JSON and not
part of neither the spec at <a
class="moz-txt-link-abbreviated" href="http://www.json.org"
moz-do-not-send="true">www.json.org</a> nor the RFC-8259.</p>
<p>Yet, I advocate the JDK JSON library should optionally allow
comments to be ignored when PARSING. This should be an opt-in
feature that would technically treat comments as whitespace
during the parsing process.<br>
</p>
<p>This would also be in line with what many other parsers do.
For example, Jackson has "ALLOW_COMMENTS" feature [1]. Also,
by comparison, the build-in parser in the .NET world, known as
System.Text.Json, also supports this [2]. <br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>The "discoverer" of JSON, Douglas Crowford, had this to say
[3] on the topic:</p>
<p><br>
</p>
<p>[QUOTE]<br>
</p>
<p>I removed comments from <abbr
title="JavaScript Object Notation">JSON</abbr> because I saw
people were using them to hold parsing directives, a practice
which would have destroyed interoperability. I know that the
lack of comments makes some people sad, but it shouldn't. </p>
<p>Suppose you are using <abbr
title="JavaScript Object Notation">JSON</abbr> to keep
configuration files, which you would like to annotate. Go
ahead and insert all the comments you like. Then pipe it
through JSMin before handing it to your <abbr
title="JavaScript Object Notation">JSON</abbr> parser.</p>
<p> [/QUOTE]</p>
<p><br>
</p>
<p>By not having the ability to ignore comments when parsing we
would effectively force users to use another parser first or a
minifier. I doubt beginners would appreciate that. <br>
</p>
<p><br>
</p>
<p>BTW: The test suite already has tests for comments.</p>
<p><br>
</p>
<p>/Lars<br>
</p>
<p><br>
</p>
<p>[1]: <a class="moz-txt-link-freetext"
href="https://www.javadoc.io/static/com.fasterxml.jackson.core/jackson-core/2.19.0/com/fasterxml/jackson/core/JsonParser.Feature.html#ALLOW_COMMENTS"
moz-do-not-send="true">https://www.javadoc.io/static/com.fasterxml.jackson.core/jackson-core/2.19.0/com/fasterxml/jackson/core/JsonParser.Feature.html#ALLOW_COMMENTS</a></p>
<p>[2]: <a class="moz-txt-link-freetext"
href="https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonreaderoptions?view=net-9.0#properties"
moz-do-not-send="true">https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonreaderoptions?view=net-9.0#properties</a></p>
<p>[3]: <a class="moz-txt-link-freetext"
href="https://plus.google.com/118095276221607585885/posts/RK8qyGVaGSr"
moz-do-not-send="true">https://plus.google.com/118095276221607585885/posts/RK8qyGVaGSr</a></p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 16/05/2025 01.44, Ethan McCue
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CA+NR86gt3GNjDPS2m-2wha5AwgumMvBRxKw+bwgTCNQ1xCiDfg@mail.gmail.com">
<div dir="ltr">I present for your consideration the library I
made when spiraling about this problem space a few years ago<br>
<br>
<a href="https://github.com/bowbahdoe/json" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/bowbahdoe/json</a><br>
<br>
<a
href="https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html</a><br>
<br>
Notably missing during the design process here were
patterns, hence the JsonDecoder design. I haven't been able
to evaluate how patterns affect that on account of them not
being out.<br>
<br>
I will more thoroughly peruse the draft of java.util.json at
a later date, but my initial observations/comments:<br>
<br>
* I am not sure having JsonValue be distinct from Json has
value.<br>
* toUntyped feels a little strange to me - the only type
information presumably lost is the sealed-ness of the
hierarchy. The interplay between that and toNumber is also a
little unnerving.<br>
* One notion that I found helpful was that a class could be
"json encodable," meaning there is a method to call to
obtain a canonical json representation.<br>
<br>
<font face="monospace">record Person(String name) implements
JsonEncodable {</font><br>
<font face="monospace"> </font><a class="gmail_plusreply"
id="m_-2004856809941181964plusReplyChip-0"
style="font-family:monospace" moz-do-not-send="true">@Override<br>
</a><font face="monospace"> public Json toJson() {</font><br>
<font face="monospace"> return Json.objectBuilder()</font><br>
<font face="monospace"> .put("namen", name)</font><br>
<font face="monospace"> .build();</font><br>
<font face="monospace"> }</font><br>
<font face="monospace">}</font><br>
<br>
<font face="arial, sans-serif">Which helper methods like </font><font
face="monospace">Json#of(List<? extends
JsonEncodable>)</font><font face="arial, sans-serif"> could
make use of. </font><font face="monospace">Json</font><font
face="arial, sans-serif"> itself (</font><font
face="monospace">JsonValue</font><font
face="arial, sans-serif"> in your prototype) could then
have a vacuous implementation.<br>
<br>
* Terminology wise - I went with reading/writing for the
actual parsing/generation of json and encoding/decoding
for the mapping of those representations to/from specific
classes. The merits are not top of mind, just noting the
difference. read/write vs parse/toString+toDisplayString<br>
* One thing I did was make the helper methods in </font><font
face="monospace">Json</font><font face="arial, sans-serif"> null
tolerant and the ones in the specific subtypes like </font><font
face="monospace">JsonString</font><font
face="arial, sans-serif"> not. This was because from what
I saw of usages of javax.json/jakarta.json that
nullability was a footgun and correcting for it required
changes to code structure (breaking up builder chains with
</font><font face="monospace">if (x != null)</font><font
face="arial, sans-serif"> checks)<br>
</font>* The functionality you want from <font
face="monospace">JsonNumber </font><font
face="arial, sans-serif">could be achieved by making it
just extend </font><font face="monospace">Number (</font><a
href="https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java</a>)
instead of a bespoke <font face="monospace">toNumber</font><font
face="arial, sans-serif">. You need the extra methods to
go to big decimal and co, but it's just an extension to
the behavior of Number at that point.<br>
</font>* JsonObject and JsonArray could implement
Map<String, Json> and List<Json> respectively.
This lowers the need for toUntyped() - since presumably one
of the use cases for that is turning the json tree into
something that more generic map/list traversal code can
handle. It also complicates any lazy loading somewhat.<br>
* Assuming patterns can be placed on interfaces, you might
want to consider something similar to <font face="monospace">JsonDecoder</font><font
face="arial, sans-serif">, but with a pattern instead of a
method that throws an exception.<br>
</font><br>
<span style="font-family:monospace">// Where here fromJson
would box up the logic for testing and extracting from
each element in the array.</span><br>
<font face="monospace">List<Person> people =
array(json, Person::fromJson);<br>
</font><br>
* I don't think there is sufficient cause for anything to be
non-sealed at this point.<br>
* JsonBoolean and JsonNull do not have reasonable
alternative implementations - as far as I can imagine, maybe
i'm wrong - so maybe those can just be final classes?<br>
* If you seal up the whole hierarchy then its pretty trivial
to make it serializable (<a
href="https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java"
moz-do-not-send="true" class="moz-txt-link-freetext">https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java</a>)<br>
<br>
<br>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, May 15, 2025 at
11:29 PM Remi Forax <<a href="mailto:forax@univ-mlv.fr"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">forax@univ-mlv.fr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi
Paul,<br>
yes, not having a simple JSON API in Java is an issue for
beginners.<br>
<br>
It's not clear to me why JsonArray (for example) has to be
an interface instead of a record ?<br>
<br>
I understand why Json.parse() only works on String and
char[] but the API make it too easy to have many
performance issues.<br>
I think you need versions using a Reader and a Path.<br>
Bonus point, if there is a method walk() that also returns
a JsonValue but the List/Map inside JsonArray/JsonObject
are populated lazily.<br>
<br>
Minor point: Json.toDisplayString() should takes a second
parameters indicating the number of spaces used for the
indentation (like JSON.stringify in JS).<br>
<br>
regards,<br>
Rémi<br>
<br>
----- Original Message -----<br>
> From: "Paul Sandoz" <<a
href="mailto:paul.sandoz@oracle.com" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">paul.sandoz@oracle.com</a>><br>
> To: "core-libs-dev" <<a
href="mailto:core-libs-dev@openjdk.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">core-libs-dev@openjdk.org</a>><br>
> Sent: Thursday, May 15, 2025 10:30:42 PM<br>
> Subject: Towards a JSON API for the JDK<br>
<br>
> Hi,<br>
> <br>
> We would like to share with you our thoughts and
plans towards a JSON API for<br>
> the JDK.<br>
> Please see the document below.<br>
> <br>
> -<br>
> <br>
> We have had the pleasure of using a clone of this API
in some experiments we are<br>
> conducting with<br>
> ONNX and code reflection [1]. Using the API we were
able to quickly write code<br>
> to ingest and convert<br>
> a JSON document representing ONNX operation schema
into instances of records<br>
> modeling the schema<br>
> (see here [2]).<br>
> <br>
> The overall out-of-box experience with such a minimal
"batteries included” API<br>
> has so far been positive.<br>
> <br>
> Thanks,<br>
> Paul.<br>
> <br>
> [1] <a href="https://openjdk.org/projects/babylon/"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://openjdk.org/projects/babylon/</a><br>
> [2]<br>
> <a
href="https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87</a><br>
> <br>
> # Towards a JSON API for the JDK<br>
> <br>
> One of the most common requests for the JDK is an API
for parsing and generating<br>
> JSON. While JSON originated as a text-based
serialization format for JSON<br>
> objects ("JSON" stands for "JavaScript Object
Notation"), because of its simple<br>
> and flexible syntax, it eventually found use outside
the JavaScript ecosystem as<br>
> a general data interchange format, such as framework
configuration files and web<br>
> service requests/response formats.<br>
> <br>
> While the JDK cannot, and should not, provide
libraries for every conceivable<br>
> file format or protocol, the JDK philosophy is one of
"batteries included",<br>
> which is to say we should be able to write basic
programs that use common<br>
> protocols such as HTTP, without having to appeal to
third party libraries.<br>
> The Java ecosystem already has plenty of JSON
libraries, so inclusion in<br>
> the JDK is largely meant to be a convenience, rather
than needing to be the "one<br>
> true" JSON library to meet the needs of all users.
Users with specific needs<br>
> are always free to select one of the existing
third-party libraries.<br>
> <br>
> ## Goals and requirements<br>
> <br>
> Our primary goal is that the library be simple to use
for parsing, traversing,<br>
> and generating conformant JSON documents. Advanced
features, such as data<br>
> binding or path-based traversal should be possible to
implement as layered<br>
> features, but for simplicity are not included in the
core API. We adopt a goal<br>
> that the performance should be "good enough", but
where performance<br>
> considerations conflict with simplicity and
usability, we will choose in favor<br>
> of the latter.<br>
> <br>
> ## API design approach<br>
> <br>
> The description of JSON at `https:://<a
href="http://json.org" rel="noreferrer" target="_blank"
moz-do-not-send="true">json.org</a>` describes a JSON
document using<br>
> the familiar "railroad diagram":<br>
> ![image](<a href="https://www.json.org/img/value.png"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://www.json.org/img/value.png</a>)<br>
> <br>
> This diagram describes an algebraic data type (a sum
of products), which we<br>
> model directly with a set of Java interfaces:<br>
> <br>
> ```<br>
> interface JsonValue { }<br>
> interface JsonArray extends JsonValue {
List<JsonValue> values(); }<br>
> interface JsonObject extends JsonValue {
Map<String, JsonValue> members(); }<br>
> interface JsonNumber extends JsonValue { Number
toNumber(); }<br>
> interface JsonString extends JsonValue { String
value(); }<br>
> interface JsonBoolean extends JsonValue { boolean
value(); }<br>
> interface JsonNull extends JsonValue { }<br>
> ```<br>
> <br>
> These interfaces have (hidden) companion
implementation classes that admit<br>
> greater flexibility of implementation than modeling
them directly with records<br>
> would permit.<br>
> Further, these interfaces are unsealed. We compromise
on the sealed sum of<br>
> products to enable<br>
> alternative implementations, for example to support
alternative formats that<br>
> encode the same information in a JSON document but in
a more efficient form than<br>
> text.<br>
> <br>
> The API has static methods for parsing strings into a
`JsonValue`, conversion to<br>
> and from purely untyped representations (lists and
maps), and factory methods<br>
> for building JSON documents. We apply composition
consistently, e.g, a<br>
> JsonString has a string, a JsonObject has a map of
string to JsonValue, as<br>
> opposed to extension for structural JSON values.<br>
> <br>
> It turns out that this simple API is almost all we
need for traversal. It gives<br>
> us an immutable representation of a document, and we
can use pattern matching to<br>
> answer the myriad questions that will come up (Does
this object have key X? Does<br>
> it map to a number? Is that number representable as
an integer?) when going<br>
> from an untyped format like JSON to a more strongly
typed domain model.<br>
> Given a simple document like:<br>
> <br>
> ```<br>
> {<br>
> "name": "John”,<br>
> "age": 30<br>
> }<br>
> ```<br>
> <br>
> we can parse and traverse the document as follows:<br>
> <br>
> ```<br>
> JsonValue doc = Json.parse(inputString);<br>
> if (doc instanceof JsonObject o<br>
> && o.members().get("name") instanceof
JsonString s<br>
> && s.value() instanceof String name<br>
> && o.members().get("age") instanceof
JsonNumber n<br>
> && n.toNumber() instanceof Long l
&& l instanceof int age) {<br>
> // use "name" and "age"<br>
> }<br>
> ```<br>
> <br>
> Later, when the language acquires the ability to
expose deconstruction patterns<br>
> for arbitrary interfaces (similar to today's record
patterns, see<br>
> <a
href="https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns</a>),<br>
> this will be simplifiable to:<br>
> <br>
> ```<br>
> JsonValue doc = Json.parse(inputString);<br>
> if (doc instanceof JsonObject(var members)<br>
> && members.get("name") instanceof
JsonString(String name)<br>
> && members.get("age") instanceof
JsonNumber(int age)) {<br>
> // use "name" and "age"<br>
> }<br>
> ```<br>
> <br>
> So, overtime, as more pattern matching features are
introduced we anticipate<br>
> improved use of the API. This is a primary reason why
the API is so minimal.<br>
> Convenience methods we add today, such as a method
that accesses a JSON<br>
> object component as say a JSON string or throws an
exception, will become<br>
> redundant in the future.<br>
> <br>
> ## JSON numbers<br>
> <br>
> The specification of JSON number makes no explicit
distinction between integral<br>
> and decimal numbers, nor specifies limits on the size
of those numbers.<br>
> This is a common source of interoperability issues
when consuming JSON<br>
> documents. Generally users cannot always but often do
assume JSON numbers are<br>
> parsable, without loss of precision, to IEEE
double-precision floating point<br>
> numbers or 32-bit signed integers.<br>
> <br>
> In this respect the API provides three means to
operate on the JSON number,<br>
> giving the user full control:<br>
> <br>
> 1. Underlying string representation can be obtained,
if preserving syntactic<br>
> details such as leading or trailing zeros is
important.<br>
> 2. The string representation can be parsed to an
instance of `BigDecimal`, using<br>
> `toBigDecimal` if preserving decimal numbers is
important.<br>
> 3. The string representation can be parsed into an
instance of `Long`, `Double`,<br>
> `BigInteger`, or `BigDecimal`, using `toNumber`.
The result of this method<br>
> depends on how the representation can be parsed,
possibly losing precision,<br>
> choosing a suitably convenient numeric type that
can then be pattern<br>
> matched on.<br>
> <br>
> Primitive pattern matching will help as will further
pattern matching features<br>
> enabling the user to partially match.<br>
> <br>
> ## Prototype implementation<br>
> <br>
> The prototype implementation is currently located
into the JDK sandbox<br>
> repository<br>
> under the `json` branch, see<br>
> here<br>
> <a
href="https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json</a><br>
> The prototype API javadoc generated from the
repository is also available at<br>
> <a
href="https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html</a><br>
> <br>
> ### Testing and conformance<br>
> <br>
> The prototype implementation passes all conformance
test cases but two,<br>
> available<br>
> on <a href="https://github.com/nst/JSONTestSuite"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://github.com/nst/JSONTestSuite</a>.
The two exceptions are the ones which<br>
> the<br>
> prototype specifically prohibits, i.e, duplicated
names in JSON objects<br>
> (<a
href="https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35</a>).<br>
> <br>
> ### Performance<br>
> <br>
> Our main focus so far has been on the API design and
a functional<br>
> implementation.<br>
> Hence, there has been less focus on performance even
though we know there are a<br>
> number of performance enhancements we can make
eventually.<br>
> We are reasonably happy with the current performance.
The<br>
> implementation performs well when compared to other
JSON implementations<br>
> parsing from string instances and traversing
documents.<br>
> <br>
> An example of where we may choose simplicity and
usability over performance<br>
> is the rejection of JSON documents containing objects
that in turn contain<br>
> members<br>
> with duplicate names. That may increase the cost of
parsing, but simplifies the<br>
> user<br>
> experience for the majority of cases since if we
reasonably assume JsonObjects<br>
> are<br>
> map-like, what should the user do with such members,
pick one the last one?<br>
> merge<br>
> the values? or reject?<br>
> <br>
> ## A JSON JEP?<br>
> <br>
> We plan to draft JEP when we are ready. Attentive
readers will observe that<br>
> a JEP already exists, JEP 198: Light-Weight JSON API<br>
> (<a href="https://openjdk.org/jeps/198"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://openjdk.org/jeps/198</a>).
We will<br>
> either update this JEP, or withdraw it and draft a
new one.<br>
</blockquote>
</div>
</blockquote>
</blockquote>
<br>
</blockquote>
</body>
</html>