[mosh-devel] Mosh and multi-path

[john hood]

Hi Matthieu,

I'm following up to our discussion on Github about your multipath Mosh
branch at
<https://github.com/boutier/mosh/commit/5e4c6df1aacbf4c6a8785146c8540cc92347d32f#commitcomment-12313177>.

I finally did review your code with an eye towards compatibility with
existing Mosh versions, and it looks like good quality code.  It is
incompatible with existing versions of mosh, so I do have a few questions:

* If I'm understanding correctly, the two major changes you make to the
network message format are 1) inserting a flow ID into the message
sequence number, and 2) adding a 16-bit field for a message type to the
encrypted message string, immediately after the two timestamps. That
second change allows you to add two new message types-- a probe message,
and a message to report server addresses to the client.  Have I got that
more or less right?

* Does the flow ID need to be in the sequence number?  Is there any
reason it couldn't be in a separate protobuf field?  Along with the
compatibility issues, this is a significant change to Mosh's
cryptography.  I'm no cryptographic expert, but reducing the namespace
for sequence IDs and adding maybe-correlatable info to the sequence ID
worries me.

* Does your message type need to be in the encrypted message string, but
outside the protobuf message?  In an ideal world, mosh would have used
another level of protobuf serialization for the timestamps, and adding
the message type wouldn't cause a compatibility problem.  But mosh is in
the same less-than-ideal world as the rest of us.

* Your code makes the server responsible for discovering and reporting
its listening addresses to the client.  Can you explain why you chose
this approach?

'm not sure this is the right approach, because the server may not
correctly know its public addresses in an IPv4 world-- it may have NAT,
or load balancers, or a proxy in front of it.  My thinking is that the
*client* should be responsible for discovering server addresses-- it
should discover a list of possible server addresses at session start.
Perhaps it should also update that list when it roams, because the
client may roam from a public IPv4 network with public DNS records for
the server to a private network with different routing and private DNS
records with different addresses for the same server.  If the server is
behind NAT or an L3 proxy, then its local addresses might not be correct
for either of these cases.

* Do we need to have a message type to describe the probe messages?
Could we instead say that an empty TransportInstruction protobuf string
represents a probe message?

All these questions I'm asking are attempts to discover whether we need
to explicitly describe these various pieces of information in
incompatible ways.  I think we might be able to make some of it
implicit, and move some of it to session setup, or to optional protobuf
fields that would offer better compatibility.

I've recently done some work on the mosh script that I think might be
relevant to multipath session setup, see
<https://github.com/keithw/mosh/pull/653>.  This collects server
addresses in a way that might be useful for a multipath mosh
implementation.  I think clients may still need to do name lookups later
to really fully handle roaming, though.  (This isn't specific to your
multipath work, though I think the need might be stronger there.)

Also have a look at my thoughts on implementing a multiplexing moshd
server:
<https://github.com/keithw/mosh/issues/295#issuecomment-109861097>  This
is not the same problem, but I think a similar solution for discovering
unnamed flows might be appropriate here.

Also, do we need the full 5-tuple (ports, address, IPv4/6) to name
flows?  Can we safely use a 3-tuple of (client-port, client-address,
IPv4/6) instead?

regards,

  --jh