SPI vs Third party dependency

[Viktor Klang]

Douglas,

I'm looking forward to having a look at the next upload.

Have a great weekend!

On Wed, Oct 25, 2017 at 4:24 PM, Douglas Surber <douglas.surber at oracle.com>
wrote:

> In other words, backpressure is an integral part of this model in order to
> allow the queues which mediate between threads to be bounded.
>
> — https://github.com/reactive-streams/reactive-streams-jvm/
> blob/v1.0.1/README.md
>
> We have tried to use reactive streams (Flow) where appropriate. With the
> inspiration of the discussion in this list I think I have developed Flow
> based mechanisms for Operation creation and row processing. Those will be
> included in the next upload to the sandbox. The one other place where back
> pressure is required is interacting with LOBs. The current API is more
> primitive than Flow. I think it would be straightforward to implement a
> Flow API on top of the one proposed.
>
> And this is my approach toward a reactive stream API. Back pressure is
> important in a few places in the API. In those places the API must support
> back pressure either through Flow or through some other mechanism. If
> someone wants a fully Flow based API the hooks are in place to support one.
> If one does not require a fully Flow based API then there is no extra
> weight in the spec (and the implementations) trying to cram something that
> does not require it into a Flow shaped box.
>
> If you want a 100% Flow based database access library, by all means write
> one. If there is something about the spec that prevents you from doing so,
> please let us know. Ideally include a concrete suggestion. We will do our
> best to accommodate you.
>
> Douglas
>
>
> On Oct 25, 2017, at 2:39 AM, Viktor Klang <v at lightbend.com> wrote:
>
> Hi Douglas!
>
> Answers inline.
>
> On Tue, Oct 17, 2017 at 8:29 PM, Douglas Surber <douglas.surber at oracle.com
> > wrote:
>
>> It is not the lack of implementations of Flow.X that is the problem. It
>> is the lack of a support library that manipulates streams.
>>
>
> Fortunately, in this case, there's nothing which prevents the JDK to
> incorporate more support for that in the existing j.u.stream package.
>
>
>>
>> If you look at the example code in the sketch David Karnok referenced in
>> the Example of a Flow-based JDBC API thread you will see that the code
>> makes extensive use of a reactive stream library. Writing the same example
>> using the same sketch API but without a reactive stream library is
>> impractical. There is (currently) no reactive stream library in Java SE so
>> it would be impractical to use the sketch API in Java SE. Doing so would
>> require some non standard reactive stream library. This fails to meet one
>> of the goals of this project which is integration with Java SE, not
>> integration with Java SE plus some random other library.
>>
>
> While I sympathize with the argument, the point of Flow/RS is
> *interoperability*. There's already ongoing work to add Flow ot other parts
> of Java, and given that the more things support it, the network effect is
> exponential.
>
> I assume that the goal of this effort is not to replace Hibernate—i.e. do
> I need to be able to map objects to the database, do lazy-loading etc[A]?
>
> To me, there are a couple of orthogonal requirements:
>
> 1. Create an SPI which makes it possible to implement RDBMs access in a
> standardized way which does not necessitate blocking threads—because that
> severely impacts scalability.
>     * This is where current JDBC has a ton of issues.
>     * This is where Flow/RS/CompletionStage can really help since it
> provides a standardized way to integrate with the desired properties.
>
> 2. Create an RDBMS API for library developers to build higher-level
> constructs/integrations which does not block any user threads—because that
> severely impacts scalability, and introduces the risk of
> deadlocks/starvation.
>     * This where Flow/RS/CompletionStage can be used to integrate with the
> underlying standard SPI—adding any transformation/integration without
> breaking the underlying properties or compromising the ability to not block
> user threads in the above layer.
>
> 3. Create an application developer SQL API to interact with an RDBMS in a
> scalable way—allowing for many concurrent users/requests.
>      * This is where JPA/Hibernate/every ORM ever built lives. Replacing
> this is going to be a huge effort.
>
>
> IMO: If 1 and 2 is achieved then support in 3 from current solutions there
> is just a matter of time[B].
>
>
>
> A: Disclaimer: I might have misunderstood the following, but: The purpose
> of JDBC is to provide an integration layer—not to provide the best possible
> application developer API for dealing with RDBMS:es—I hope not given that
> it has practically not evolved since it was created—and practically no
> developer deals with raw JDBC.
>
> B: I could be wrong, but I'm willing to make a bet on it.
>
>
>>
>> Douglas
>>
>> > On Oct 17, 2017, at 12:29 AM, Jens Schauder <jschauder at pivotal.io>
>> wrote:
>> >
>> > Sorry if I'm a little dense but there is something I don't really
>> > understand.
>> >
>> > The existing JDBC, as well as the upcoming new version, is what I
>> consider
>> > an SPI: a set of interfaces plus rules how they should behave, which
>> won't
>> > get implemented in the JDK itself. Instead, other parties (in this case
>> > probably database vendors) will provide implementations.
>> >
>> > In my eyes, the Flow API as well is such an SPI. It is intended to get
>> > implemented by third parties.
>> >
>> > Yet you don't want to depend on it although it seems to fit the
>> > requirements quite well, especially "Maximum throughput with low and
>> stable
>> > latency"
>> >
>> > So my question is: what is the fundamental difference between the Flow
>> API
>> > and the new async JDBC that makes it ok for one to depend on third-party
>> > implementations and the other not?
>> >
>> > Also, if this difference is relevant beyond the scope of JDBC the
>> question
>> > arises how/if the Flow API is supposed to be used at all in the JDK?
>> > Although that question might be better suited for a different mailing
>> list?
>> >
>> > Just to make sure: I'm not trying to change anybody's mind. I'm just
>> trying
>> > to understand the underlying principles for those decisions.
>> >
>> > Regards,
>> > Jens Schauder
>>
>>
>
>
> --
> Cheers,
> √
>
> ——————
> *Viktor Klang*
> Deputy CTO
> Lightbend, Inc.
>
>
>


-- 
Cheers,
√

——————
*Viktor Klang*
Deputy CTO
Lightbend, Inc.