[mosh-devel] multipath mosh

[Matthieu Boutier]

Hi all,

I have written a multipath version of mosh.  Essentially, it builds all possible paths between local and remote addresses, probe them, and selects the best (RTT-based metric).  This is especially usefull when the machines have multiple addresses.  Some examples of multi-addresses are:

- combination with wifi, ethernet, 3/4G (all users)
- more generally multihomed networks (cf. IETF homenet working group, for all users)
- dual-stack IP (v4 / v6, all users)
- switching to link-local addresses (only for bad administrators, like me)

The code is available at :

    git clone https://github.com/boutier/mosh

The code works on Linux and MacOS, but has *not been tested on FreeBSD*.  For strange reasons, it seems the control message argument of "sendmsg" has the same macro constants on Linux and MacOS.

The current branch has been rebased for easier review.  I have just keep some "fix" that I deem interesting, as the MacOS kernel panic's feature:

    (http://www.pps.univ-paris-diderot.fr/~boutier/mac-crash.html)


Some details on different branches:

The main branch is "master" (== "uni-socket-rebased").  In this branch, I have one socket per stack (v4/v6), and I use special options of "sendmsg" and "recvmsg" to respectively set and get the local address (i.e. the src and the dst).

The branch "uni-socket" is exactly the same, but has not been rebased : probably very difficult to review.

The branch "multi-socket" uses one socket per flow(*): each socket is bound to a local address of the host, and should be used for only one given destination.  I didn't use the "connect" system call, but I ought to have.  Anyway, even if this branch works, it should be rewritten.  Note this branch is NOT COMPATIBLE with the uni-socket's one.

(*) a flow is a pair (src addr, dst addr) -- with related data (cf. network.h).


Security details (uni-socket branch):

In mosh, the nonce has two usage: cryptographic one, and sequence number.  As the sequence number depends on the flow, it can anymore be global in multipath-mosh.  One elegant solution ought to separate the nonce from the sequence number... Currently, I cut the nonce in two parts (three):

  - direction (already present)
  - flow id
  - seqno

Flows ID are only set by the client, and strictly increasing.  When the client obtains flow ID limit number (currently 0xFFFF -- if memory serves), then it crashes.  The server gets the flow ID from the client message (and so never remove flows).  The seqno is incremented as before (one per flow).


Protocol details:

My current network protocol is:

64 bits: nonce
  1 bit: direction
  15 bits: flow id (or sock_id, depending of the implementation)
  48 bits: seqno

16 bits: flags
  0x1: probe flag /* check RTT */
  0x2: address flag /* ask the server for its other addresses */


Logger:

Currently, the program write two log files:

    /tmp/mosh_client.log
    /tmp/mosh_server.log


Any comments are welcome, enjoy !

Matthieu