Example of a Flow-based JDBC API

[Douglas Surber]

Answers in line.

Douglas

> On Oct 25, 2017, at 2:07 AM, Viktor Klang <v at lightbend.com> wrote:
> 
> Hi Douglas,
> 
> thanks for your patience, the past 2 weeks have been crazy.
> 
> On Mon, Oct 16, 2017 at 6:56 PM, Douglas Surber <douglas.surber at oracle.com <mailto:douglas.surber at oracle.com>> wrote:
> The target app is a large scale app running many instances in parallel against a large database system, probably geographically distributed. Maximum throughput with low and stable latency.
> 
> The goals for the implementation are
> 
>  - vendor neutral
>  - avoid features that reduce scalability
>  - every feature must balance user benefit vs performance cost
>  - every feature must balance user benefit vs code to implement
> 
> The goals for the user API are
> 
>  - asynchronous. No user thread blocks
> 
> This is fairly simple with current JDBC though, just put all blocking ops in another threadpool.
> I'd recommend that the goal be set higher.

It is out of scope for a Java standard to specify details of how that standard is implemented. The EG hopes and the user community would certainly expect any implementation of the spec to be as deeply asynchronous as possible. Oracle has developed an internal prototype for the Oracle Database  that does not block internally.
>  
>  - plays well with Java SE
> 
> What does this mean in practice (i.e. how can this be evaluated?).

 - Use what Java SE provides
 - Provide what Java SE uses.

>  
>  - minimal. Every feature must be important to the target app. 80/20 rule
> What is "the target app" in this?

See the first line of previous reply above.
>  
>  - consistent. No special cases
> 
> So all operations on primitives require boxing?

In the current version. The only places auto-boxing appears are Result gets and Parameter sets. Hotspot can do a lot to optimize these. And at least from an implementers POV this decision reduces the amount of code in these areas by a factor of seven. Code that doesn’t exist doesn’t have bugs.
>  
>  - transparent. Principle of least surprise
> 
> Principle of least surprise is good.
>  
>  - ready by Java 10/19.x
> 
> That's a great goal. How many implementations of "drivers" are required to make this pass the bar?

No implementations are required, only the API being included in Java 19.x.
>  
>  - independent of java.sql and javax.sql
> 
> Completely independent?

Yes, completely independent. I have removed the few dependencies that were present in the sandbox version.
>  
> 
> We haven’t discussed TCK or testing.
> 
> Then I'd suggest doing that early—a specification tends to evolve alongside its canonical interpretation (TCK).

Agreed.

>  
> 
> We made the following design choices. They aren’t requirements. But having made those choices consistency requires that we be rigorous in abiding by them.
> 
>  - Rigorous use of types
>  - Builder pattern
>  - Fluent API
>  - Immutable after initialization
>  - One way to do something is enough
>  - Avoid SQL processing
>  - Avoid callback hell
> 
> Sounds like a great set of design choices!
>  
> 
> As we started work we looked at existing async database access APIs both in Java and other languages. We looked at reactive streams but did not follow that path because there was no support in Java SE. We looked at java.util.stream but that blocks and cannot/will not be made non-blocking. The Core Library Team encouraged us to build around CompletableFuture. CompletableFuture seems too low level so this API provides a higher level abstraction, well defined execution of Operations, to support the most common use cases while still making use of CompletableFuture. (As noted elsewhere replacing CompletableFuture with CompletionStage would be reasonable.)
> 
> I think there needs to be some separation between what is intended to be used by "application developers" and what is for "library developers" vs what is for "implementors". Needs and capabilities are different amongst the groups.

The goal of JDBC was to expose the underlying database. Unfortunately, JDBC adds a bunch of abstraction on top of the database that greatly complicates any implementation. These complications make drivers, bigger, slower, and less reliable. With ADBA we have tried to eliminate everything that is not strictly necessary to expose the underlying database with a fully async API. Application developers and library developers are of equal importance but with the caveat that the spec not require a “smart” implementation. A thin implementation will be faster and more transparent than an implementation that requires a neural network to implement.

In all honestly, I would say that the needs of the implementer are given priority. If the resulting API does not provide a feature that app developers need that is an opportunity for library developers. If the API does not expose a database capability that the target app needs, that is a limitation in the API. A small, transparent API makes the library developers’ job easier. Fast implementations, which are enabled by a small, transparent API, make everyone happier.

[Note: the above strongly suggests that POJO support is superfluous and should be removed.]

>  
> 
> Douglas
> 
> 
> > On Oct 14, 2017, at 6:07 AM, Viktor Klang <v at lightbend.com <mailto:v at lightbend.com>> wrote:
> >
> > What would help me, is to learn what the requirements for success is for the following aspects:
> >
> > * Implementor SPI—ergonomics, performance, scalability
> > * User API—ergonomics, performance, scalability
> > * Verification of implementations—TCK for implementations
> > * Verification of usage—how to write tests for JDBC 2
> >
> 
> 
> 
> 
> -- 
> Cheers,
> √
> 
> ——————
> Viktor Klang
> Deputy CTO
> Lightbend, Inc.
> 
>