[sctp-dev] Basic Questions

Mon Jun 22 05:51:42 PDT 2009

more comments inline. You provided quite a lot of data so please forgive 
me if I missed something...

David Ryan wrote:
> Hi Chris,
> 
> thanks for the quick response.  Comments inline too..
> 
> On Fri, Jun 19, 2009 at 1:31 AM, Christopher Hegarty - Sun Microsystems 
> Ireland <Christopher.Hegarty at sun.com 
> <mailto:Christopher.Hegarty at sun.com>> wrote:
> 
>     Hi David,
> 
>     comments inline...
> 
> 
>     David Ryan wrote:
> 
>         Hi,
> 
>         I have recently downloaded the Client/Server example from Chris
>         Hegarty's Blog.  I've been playing around with it and getting a
>         feel for SCTP.  My environment is Java 7 b59 running on Open
>         Solaris 2009.06  (in VMware Fusion on OSX).
> 
>     Great that you are playing around with this. I really appreciate the
>     feedback/questions. Interesting environment, good to see that you
>     are on  09.06
> 
> 
> For any interested on lookers, I found it very easy to setup following 
> this guide.
> 
> http://wikis.sun.com/display/OpenSolarisInfo/How+to+Run+OpenSolaris+in+VMware+Fusion+for+Mac
> 
> I used the package manager to install eclipse.  Actually the only change 
> from the guide above was to increase the memory from 780mb to 1.5Gb.  
> Eclipse and/or java uses a lot of memory.
> Of course I also installed Java7 which was easy too.
>  
> 
> 
> 
>         I changed the example code to send different sized packets and
>         found that the maximum size of data returned by receive was
>         16,000 bytes.  Is this the exact same size of the underlying
>         network packet?  I'm assuming it is and that the packet size is
>         large because it didn't actually go over a network.
> 
>     I assume you are sending messages larger that 16,000 bytes but the
>     receive is return an incomplete message, i.e. isComplete returns
>     false, right?
> 
> 
> Yes, I was sending packets larger than 16,000 bytes.  isComplete was 
> always returning true on the receiver; more on this below.
>  
> 
> 
>     The packet size does indeed look very large for a single PDU. I
>     would suspect that several PDU's are available in the input buffer
>     and being combined before returning to the user. Can you see the MTU
>     of interface that you are sending/receiving on, 'ifconfig -a'?
> 
> 
> I think you're right about this.  The MTU size was 1500 bytes.  I ended 
> up making a few other changes and ran my test again and found that I was 
> receiving upto 100,000 bytes in each message.  I must have been lucky 
> sending at the edge of a timer which caused the packets to split.  I 
> found that the underlying send/receive buffers were around 100,000 bytes 
> and received a message to large exception when trying to send larger 
> packets.  ie
Yes, it looks like your messages are larger than the underlying buffer 
sizes. You can of course configure the sizes using SO_RCVBUF and SO_SNDBUF.

For sending messages larger than the buffer size you can use 
SCTP_EXPLICIT_COMPLETE. It is implementation specific whether or not 
this option is supported, but I can tell you that it is not supported on 
Solaris.

> java.net.SocketException: Message too long
>     at sun.nio.ch.SctpChannelImpl.send0(Native Method)
>     at 
> sun.nio.ch.SctpChannelImpl.sendFromNativeBuffer(SctpChannelImpl.java:1005)
>     at sun.nio.ch.SctpChannelImpl.send(SctpChannelImpl.java:973)
>     at sun.nio.ch.SctpChannelImpl.send(SctpChannelImpl.java:948)
>     at DaytimeClient$DataSender.run(DaytimeClient.java:146)
>     at java.lang.Thread.run(Thread.java:717)
> 
>     The API supports SCTP_DISABLE_FRAGMENTS socket option to disable the
>     implementation from fragmenting messages. On your platform I suspect
>     that this option is true by default. Can you check the value of
>     this? Maybe you could set it to false to determine the exact PMTU,
>     if it is not actually 16,000 which is may be.
> 
> 
> I ran a few tests to retrieve all the socket options available.  On the 
> client I got the following:
> 
> disableFragments = Option not supported by protocol
> explicit complete = Option not supported by protocol
> fragment interleave = Option not supported by protocol
> init max streams = 
> com.sun.nio.sctp.SctpStandardSocketOption$InitMaxStreams at 7 
> [maxInStreams:32maxOutStreams:32]
> no delay = false
> primary address = Option not supported by protocol
> linger = -1
> rcv buffer = 102265
Sorry, I forgot SCTP_DISABLE_FRAGMENTS was not supported on Solaris. 
However, messages are fragmented by default. I added this option to 
ensure that fragmentation was possible on platforms where it was not the 
default. The main motivation behind this was to support TLS/SSL over 
SCTP, see http://tools.ietf.org/html/rfc3436.

Some of the more advanced socket options are not supported on all 
platforms. The reason is that the underlying platform does not implement 
the option yet. I included them in the Java specification because I 
considered them important. They are specified to be optional in 
SctpStandardSocketOption.

> I thought it was odd that some of those options weren't supported.  I 
> tried the same thing on the server.  I got different exceptions/results 
> from doing them on the SctpServerChannel and SctpChannel.
> 
> On the SctpServerChannel the following were returned:
> 
> disableFragments = 'SCTP_DISABLE_FRAGMENTS' not supported
> explicit complete = 'SCTP_EXPLICIT_COMPLETE' not supported
> fragment interleave = 'SCTP_FRAGMENT_INTERLEAVE' not supported
> init max streams = 
> com.sun.nio.sctp.SctpStandardSocketOption$InitMaxStreams at 7 
> [maxInStreams:32maxOutStreams:32]
> no delay = 'SCTP_NODELAY' not supported
> primary address = 'SCTP_PRIMARY_ADDR' not supported
> linger = 'SO_LINGER' not supported
> rcv buffer = 'SO_RCVBUF' not supported
> snd buffer = 'SO_SNDBUF' not supported
> 
> On the SctpChannel (returned from ssc.accept() ) the following were 
> returned:
> 
> disableFragments = Option not supported by protocol
> explicit complete = Option not supported by protocol
> fragment interleave = Option not supported by protocol
> init max streams = 
> com.sun.nio.sctp.SctpStandardSocketOption$InitMaxStreams at 7 
> [maxInStreams:32maxOutStreams:32]
> no delay = false
> primary address = Option not supported by protocol
> linger = -1
> rcv buffer = 102400
> snd buffer = 102400
> 
> One thing to note is that the exception returned from the 
> SctpServerChannel getOption was different from the SctpServerChannel 
> getOption call.  It would probably be nicer if these were the same.
Ah yes, this is a little funny. The reason is that SctpServerChannel 
does not support SCTP_DISABLE_FRAGMENT, you can see this from its 
supported options in the class description. While SctpChannel does 
"support" SCTP_DISABLE_FRAGMENT (one of the supported options listed in 
the class description), so it will accept this option, but because the 
option itself if optional, i.e. platform dependant, it may or may not be 
set. I may need to revisit this.

> I tried playing around with the various options available from 
> MessageInfo and found that setting complete to true or false on the 
> sender didn't change anything; on the receiver the setting always 
> returned true.
The completeness of a message is only meaningful when 
SCTP_EXPLICIT_COMPLETE is set. Unfortunately, Solaris doe not yet 
support this. Otherwise the message is always considered complete, i.e. 
single send.

>  I also found the the TTL on the receiver was always 
> zero, although I'm not sure if this is expected.
The TTL is only meaningful for messages being sent.

> Another thing I noticed was that InvalidStreamException is an undeclared 
> exception.  It would be nice if this extended from IOException and could 
> be caught in the declared catch block.
InvalidStreamException is a RuntimeException (more specifically an 
IllegalArgumentException) and as such is not required to be caught. It 
is listed in the list of exceptions thrown by send, but not declared in 
the method description so you are not required to provide a catch block. 
Does this make sense or answer you question?

> 
>         Is it possible to discover the PMTU size from the API so that a
>         protocol can be optimized for SCTP?
> 
>     It is not currently possible to retrieve the PMTU through the API,
>     but it should possible to implement an implementation specific
>     socket option to support this, and I could add it to the API if
>     necessary.
> 
>     Are you sure you want to do this? The reason I ask is that the PMTU
>     may be different for the different routes on an association with
>     multiple interfaces. Typical applications will not directly control
>     which destination address to use when sending, so trying to size
>     your messages to an specific size based on the PMTU may not be the
>     best approach. Or do you have a different scenario?
> 
> 
> At the moment I'm not exactly sure if I need the PMTU or not.  This is 
> my first experience with SCTP so am attempting to work out how best to 
> build application protocols to use it.  At the most basic level I'm 
> wondering how to construct application level framing packets.  Do I rely 
> on SCTP to frame the packets or do I need to include start of packet 
> headers and such for framing?
I would personally leave it to SCTP to provide the message framing. I 
don't see any issues with allowing SCTP do the framing unless you 
require very large messages that the platform does not support.

> I figure there's probably two options.  If the PMTU is available and 
> guarantees that each message is received completely then I can split 
> large messages into packets and know that any headers will be at the 
> start of the message buffer.  This can be easily split off before 
> passing data to applications, etc.  If the PMTU is not available then I 
> can't rely on the message buffer being aligned and will need to use 
> standard TCP style protocol methods of assuming that data is an 
> unbounded stream.  The third option is that SCTP handles message 
> boundaries even for large packets and any size packet up to the send 
> buffer size will be gauranteed to arrive at the destination with the 
> message boundary intact.  In this case I still need headers to confirm 
> the start of packets, but have a slightly easier time finding the end of 
> the packet (ie won't need to use too many offsets for start of following 
> packets into buffers, etc).  If you or anyone on the list has any 
> experience with this I'd appreciate some input.
Sorry, I have no experience with this, but as I said I would let SCTP do 
the message framing.

Please let me know your comments. Have I answers some of your questions 
or just confused things more? ;-)

-Chris.

> Personally, I like the idea of having the PMTU and exposing the 
> underlying IP nature of protocols to higher level applications 
> protocols.  I see this as one of the many nice features in SCTP.
> 
> Eventually the plan is to build a couple of layers above SCTP.  The 
> first layer being a kind of SSL (based on web of trust concepts) and 
> second being a session layer for managing the streams.  Hopefully I'll 
> end up with a higher level API which can assign bi-directional 
> streams/packets to higher level applications.
>  
> Thanks again for your help,
> David.
> 
> 
> 
>         Also, great to see SCTP on Java.  Disapointing its not on all
>         platforms yet, but I'm sure that will be fixed in good time.
> 
>     Yes, hopefully over time other platforms/ports can be supported.
> 
>     -Chris.
> 
> 
>         Regards,
>         David.
> 
> 