<div dir="ltr"><div dir="ltr">Thanks for following up David, </div><div dir="ltr"><br></div><div dir="ltr">You just made my day. Really. I have finally something concrete in front of me that I can address. </div><div dir="ltr">In interest of keeping this discussion in focus, I will bypass the rest and address a couple of key points from your response:</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><br></div><div dir="ltr">> Having a special Selector type for SSL channels is weird, period. It's ugly, it smells. It doesn't solve the problem in a clean way.</div><div dir="ltr"><br></div></span><div dir="ltr">Cut & pasting from a previous message of mine in this same thread, here is the way a java.nio.channels loop works as of today (no SSL):</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><pre style="white-space:pre-wrap;box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:13.6px;margin-bottom:16px;margin-top:0px;background-color:rgb(246,248,250);border-radius:3px;line-height:1.45;overflow:auto;padding:16px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:13.6px;background:transparent;border-radius:3px;margin:0px;padding:0px;border:0px;word-break:normal;display:inline;line-height:inherit;overflow:visible">Selector selector = Selector.open();
SocketChannel socketChannel = SocketChannel.open();
socketChannel.register(selector, SelectionKey.OP_READ);
while(true) {
selector.select();
for (SocketChannel sc : selector.selectedKeys() {
sc.read(...)
}
}</code></pre></div></span><div dir="ltr">and here it is how it works with my implementation (which I am considering offering it to JDK):</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><pre style="white-space:pre-wrap;box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:13.6px;margin-bottom:16px;margin-top:0px;background-color:rgb(246,248,250);border-radius:3px;line-height:1.45;overflow:auto;padding:16px;color:rgb(36,41,46)"><code style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:13.6px;background:transparent;border-radius:3px;margin:0px;padding:0px;border:0px;word-break:normal;display:inline;line-height:inherit;overflow:visible">Selector selector = SSLSelector.open(); // notice the SSL
SocketChannel socketChannel = SSLSocketChannel.open(); // notice the SSL
socketChannel.register(selector, SelectionKey.OP_READ);
while(true) {
selector.select();
for (SocketChannel sc : selector.selectedKeys() {
sc.read(...)
}
}</code></pre></div></span><div dir="ltr">Adding the following block to show the same (and some) if it ever made it to JDK -- is quite evident, but adding it here to avoid any possible misunderstanding:</div><div dir="ltr"><pre style="white-space:pre-wrap;box-sizing:border-box;margin-bottom:16px;margin-top:0px;background-color:rgb(246,248,250);border-radius:3px;line-height:1.45;overflow:auto;padding:16px"><font color="#24292e" face="SFMono-Regular, Consolas, Liberation Mono, Menlo, Courier, monospace"><span style="font-size:13.6px">Selector selector = Selector.open(); // notice no SSL, but internally the SelectorProvider pics the SSLSelector
SocketChannel socketChannel1 = SocketChannel.open(); // notice Plain
SocketChannel socketChannel2 = SSLSocketChannel.open(); // notice SSL
socketChannel1.register(selector, SelectionKey.OP_READ); // notice mixture of channels
socketChannel2.register(selector, SelectionKey.OP_WRITE); // notice mixture of channels
while(true) {
selector.select();
for (SocketChannel sc : selector.selectedKeys() {
if (sc.isReadable()) sc.read(...)
if (sc.isWritable()) sc.write(...)
}
}</span></font></pre></div><div dir="ltr">So:</div><div dir="ltr"><br></div><div dir="ltr">1. There is no special Selector (that the user sees), just the good old Selector.</div><div dir="ltr">2. The user creates SSLSocketChannels (and possibly SocketChannels, SSLServerSocketChannels, ServerSocketChannels) and registers them altogether with the Selector.</div><div dir="ltr"><br></div><div dir="ltr">What is ugly and smelly here? How much cleaner can we bridge today's world and tomorrow's? At the end of the day, a developer wants to use the same model, to achieve performant SSL communication the same way it used to with non-SSL communication. Just like in the blocking world, an InputStream can be obtained from a Socket or an SSLSocket, then it can be read upon in indistinguishable manner.</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><br></div><div dir="ltr">> If it ever comes to the JDK though, it would have to come in the form of a JDK-level event loop abstraction, with callback-driven channels that extend NIO. I don't see that happening unless it would be part of a long term plan to deprecate NIO: the JDK folks hate needless duplication. This seems unlikely. But hey, maybe it's time for that idea to get its day.</div><div dir="ltr"><br></div></span><div>In light of the previous point, I couldn't disagree more. As it is painfully clear, the proposed approach stays within the confines of nio, and no duplication is needed (I also hate duplication).</div><div><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div>I am commenting on the rest, but I believe this is nearly not as important as the points above. Feel free to discard:</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><br></div><div dir="ltr">> This, for the record, has nothing to do with anything. Some things designed long ago are good; some are not good. Age doesn't make ideas go rotten. </div></span><div dir="ltr">Agreed. My answer has to be taken in the context of "... opted to not spend the money/time on creating an SSLSelector knowing people could program it on top"</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><br></div><div dir="ltr">> The correct abstraction you're looking for is not a Selector, it's an "event loop". </div><div dir="ltr"><br></div></span><div dir="ltr">The "event loop" or whatever pattern that can be chosen today will be possible tomorrow with the proposed API. XNIO should be able to ingest this right in, same way (I assume) it does with Selector/SocketChannel, and without need to use SSLEngine to encrypt/decrypt SSL.</div><span class="gmail-im" style="color:rgb(80,0,80)"><div dir="ltr"><br></div><div dir="ltr">> Selector is just too low-level to abstract the concept neatly (see: countless discussions about the difficulty of implementing the lowest-level event loop in every network daemon software written in the past 30 years).</div><div dir="ltr"><br></div></span><div dir="ltr">Not to say I don't follow discussions, but why should I when I can develop something myself, see it work perfectly, test it to work with hundreds of thousands of SSL connections on a single host, handling 23GBPS of SSL traffic on a 25GBPS network (75% usage over 48 CPUs)?</div><div dir="ltr"><br></div><div>Thanks again,</div><div>--Andi</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Mar 8, 2019 at 6:21 PM David Lloyd <<a href="mailto:david.lloyd@redhat.com">david.lloyd@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, Mar 8, 2019 at 4:17 PM Andi Mullaraj <<a href="mailto:andimullaraj@gmail.com" target="_blank">andimullaraj@gmail.com</a>> wrote:<br>
><br>
> Inline my comments but putting this upfront so that it doesn't go unnoticed for the occasional reader:<br>
><br>
> I was directed to this forum for discussing the merits and technicals details of a possible SSLSelector/SSLSocketChannels. The level of engagement seems to be very minimal though (I appreciate the effort of the two members pitching their feedback). The best I have heard was "[..] make SSL with Selectors easier, because right now it is very painful. But this approach is unlikely to succeed, at least in the JDK.", but without saying why.<br>
<br>
The answer is that selectors are for implementing event loops. SSL is<br>
a thing that applies to sockets. Combining them doesn't make sense<br>
from any rational engineering perspective. Having a special Selector<br>
type for SSL channels is weird, period. It's ugly, it smells. It<br>
doesn't solve the problem in a clean way. It's certainly not the<br>
*best* solution. And what other things will we then need special<br>
Selector types for? If the answer is "nothing", then why do we need a<br>
special Selector subclass for SSL to begin with, instead of just using<br>
Selector?<br>
<br>
> Every Java developer I pitch this idea to, says the same thing "Are you sure that SSLSelector/SSLSocketChannels are not in JDK?"<br>
<br>
All this says is that the Java developers you have talked to have (a)<br>
never implemented non-blocking SSL (as evidenced by the fact that they<br>
do not have any familiarity with the APIs) and (b) therefore do not<br>
understand the complexities therein, or to put it more bluntly, they<br>
have no idea what they're talking about. I mean saying that Joe<br>
Developer thinks some complex thing they've never looked at or<br>
bothered to understand ought to be simple isn't really a great<br>
argument for a lot of otherwise very busy people to drop everything to<br>
review something that they know, based on their own expertise, isn't<br>
really a workable solution to the problem.<br>
<br>
> > but they chose to separate the SSL from the nio such that it was more re-usable and opted to not spend the money/time on creating an SSLSelector knowing people could program it on top. (ie. the history).<br>
> That decision must have been made 15 years ago.<br>
<br>
This, for the record, has nothing to do with anything. Some things<br>
designed long ago are good; some are not good. Age doesn't make ideas<br>
go rotten. These decisions were made for good reasons; the result may<br>
not be perfect but it's not significantly worse now than in the 1.4<br>
days. The OS-level APIs that underpin Selector have not changed that<br>
much either.<br>
<br>
> > Most stuff should be done outside the jdk anyways. ie. look at logback and how good it is and how the jdk logging which tried to copy log4j(and failed) kinda sucks.<br>
> The reason I tackled this problem 2 years ago is that I could find no third-party implementation of SSLSocketChannels with Selectors. I checked how other applications dealt with that and felt the pain they must have felt to get huge dependencies in, just to get a functionality as primordial as this.<br>
<br>
The correct abstraction you're looking for is not a Selector, it's an<br>
"event loop". It is possible to have an API based on Channels that is<br>
ready-callback driven instead of Selector-driven, and such an API can<br>
support SSL cleanly. This is the basis of my XNIO project. It is<br>
only slightly higher level than Selector; the Channels are otherwise<br>
largely similar.<br>
<br>
Selector is just too low-level to abstract the concept neatly (see:<br>
countless discussions about the difficulty of implementing the<br>
lowest-level event loop in every network daemon software written in<br>
the past 30 years).<br>
<br>
> This must be related to the pain threshold. I believe (and David also mentioned) that working with SSLEngine is very painful. I mean, in order to achieve a robust application with no edge cases especially when security comes into play. This is not about making it perfect, it is about offering something decent that, IMO, should have been there in first place.<br>
<br>
What you're looking for is a better non-blocking approach to SSL, and<br>
I too would love to see it. If it ever comes to the JDK though, it<br>
would have to come in the form of a JDK-level event loop abstraction,<br>
with callback-driven channels that extend NIO. I don't see that<br>
happening unless it would be part of a long term plan to deprecate<br>
NIO: the JDK folks hate needless duplication. This seems unlikely.<br>
But hey, maybe it's time for that idea to get its day.<br>
<br>
-- <br>
- DML<br>
</blockquote></div>