[mosh-devel] mosh and SCTP.

[Hari Balakrishnan]

On Apr 11, 2012, at 7:47 AM, Mark Blackman wrote:

> 
> On 11 Apr 2012, at 12:23, Hari Balakrishnan wrote:
> 
>> Hi Mark,
>> 
>> Good question.  Here are some relevant differences:
>> 
>> 1) As far as I know, SCTP doesn't support the ability to resume after suspension (and changing IP), particularly if data was outstanding over the connection when the suspension occurred.
> 
> Well, there's SCTP multihoming, but that probably requires knowing all the IPs in advance.

In addition, the issue is, suppose we get disconnected and there's a bunch of outstanding data waiting to be delivered from the SCTP on the server (the machine we're logged into).  If the disconnection lasts more than some time, T, chances are the SCTP server will terminate.  We don't want that for Mosh.  

>> 2) SCTP delivers data in order and does not have the ability to skip data to allow the receiver to reach the current state promptly.  This capability is important for interactivity.
> 
> The wikipedia entry indicates "However, message ordering is optional in SCTP; a receiving application may choose to process messages in the order they are received instead of the order they were sent." and " As in UDP, in SCTP a sender sends a message in one operation, and that exact message is passed to the receiving application process in one operation." Perhaps there's a wrinkle I'm missing though.

The ordering issue I'm referring to is the ability to "pull back" data from the transport once the app puts it there.  Suppose we have data corresponding to 7 screen refreshes the server has put into the transport layer and the client, after going through some very flaky connectivity, got disconnected.  We don't want all those refreshes to show up when connectivity resumes; we want to jump to the most current state.  

>> 3) SCTP, like TCP is bulk-transfer-optimized.  So if a buffer were to fill up for whatever reason from the session, it would reduce interactivity.  Mosh's State Synchronization Protocol (SSP) has a different end-to-end flow control method than SCTP (or TCP), which optimizes for interactivity (rather than for bulk throughput). 
> 
> Ok, it's not clear to me what the flow-control method is in SCTP, so I can't really comment on that. :)

Please see Section 2.4 of http://mosh.mit.edu/mosh-paper-draft.pdf (& if you have time, the rest of section 2 leading up to it; you can skip the crypto subsection because it isn't relevant to this issue).

>> 4) I'm not sure how widely deployed SCTP is, across different operating systems.
> 
> Wikipedia suggests an option for most of the common ones, but I'm sure they're all a bit
> on the immature side.
> 
>> 
>> Please note that Mosh doesn't run atop UDP; there's a state synchronization protocol that sits in between.  That protocol uses the principles of "application-level framing", which is a design philosophy that says that one can get better behavior and performance by designing a protocol in concert with the application.  In our case, SSP uses out-of-order delivery (i.e., skipping data when appropriate), roaming across IPs, and a different flow control strategy, to better support an interactive terminal.
> 
> Thanks, I did appreciate that SSP was the key layer in this discussion. I suppose I was thinking that SSP might
> be implemented in terms of SCTP instead of UDP. UDP is, of course, a very mature protocol, but I got the impression
> the SSP layer could offload more work into the SCTP layer than it could to the UDP transport. Ultimately, it may
> be just that SCTP would just represent an alternate UDP transport with no extra benefits and the disadvantage of
> only partial or incomplete support across OSes.

I think that's a fair assessment.  We feel that the trickier issues relating to synchronizing terminal state across suspend/resume stages and providing good interactive response are not provided by any generic transport layer today.

Hari

> 
> Cheers,
> Mark