Example of a Flow-based JDBC API

[Viktor Klang]

Hi Douglas,

thanks for your patience, the past 2 weeks have been crazy.

On Mon, Oct 16, 2017 at 6:56 PM, Douglas Surber <douglas.surber at oracle.com>
wrote:

> The target app is a large scale app running many instances in parallel
> against a large database system, probably geographically distributed.
> Maximum throughput with low and stable latency.
>
> The goals for the implementation are
>
>  - vendor neutral
>  - avoid features that reduce scalability
>  - every feature must balance user benefit vs performance cost
>  - every feature must balance user benefit vs code to implement
>
> The goals for the user API are
>
>  - asynchronous. No user thread blocks
>

This is fairly simple with current JDBC though, just put all blocking ops
in another threadpool.
I'd recommend that the goal be set higher.


>  - plays well with Java SE
>

What does this mean in practice (i.e. how can this be evaluated?).


>  - minimal. Every feature must be important to the target app. 80/20 rule
>
What is "the target app" in this?


>  - consistent. No special cases
>

So all operations on primitives require boxing?


>  - transparent. Principle of least surprise
>

Principle of least surprise is good.


>  - ready by Java 10/19.x
>

That's a great goal. How many implementations of "drivers" are required to
make this pass the bar?


>  - independent of java.sql and javax.sql
>

Completely independent?


>
> We haven’t discussed TCK or testing.
>

Then I'd suggest doing that early—a specification tends to evolve alongside
its canonical interpretation (TCK).


>
> We made the following design choices. They aren’t requirements. But having
> made those choices consistency requires that we be rigorous in abiding by
> them.
>
>  - Rigorous use of types
>  - Builder pattern
>  - Fluent API
>  - Immutable after initialization
>  - One way to do something is enough
>  - Avoid SQL processing
>  - Avoid callback hell
>

Sounds like a great set of design choices!


>
> As we started work we looked at existing async database access APIs both
> in Java and other languages. We looked at reactive streams but did not
> follow that path because there was no support in Java SE. We looked at
> java.util.stream but that blocks and cannot/will not be made non-blocking.
> The Core Library Team encouraged us to build around CompletableFuture.
> CompletableFuture seems too low level so this API provides a higher level
> abstraction, well defined execution of Operations, to support the most
> common use cases while still making use of CompletableFuture. (As noted
> elsewhere replacing CompletableFuture with CompletionStage would be
> reasonable.)
>

I think there needs to be some separation between what is intended to be
used by "application developers" and what is for "library developers" vs
what is for "implementors". Needs and capabilities are different amongst
the groups.


>
> Douglas
>
>
> > On Oct 14, 2017, at 6:07 AM, Viktor Klang <v at lightbend.com> wrote:
> >
> > What would help me, is to learn what the requirements for success is for
> the following aspects:
> >
> > * Implementor SPI—ergonomics, performance, scalability
> > * User API—ergonomics, performance, scalability
> > * Verification of implementations—TCK for implementations
> > * Verification of usage—how to write tests for JDBC 2
> >
>
>


-- 
Cheers,
√

——————
*Viktor Klang*
Deputy CTO
Lightbend, Inc.