[mosh-devel] X forwarding support via devious means

[Nathaniel Smith]

Hi all,

Just saw mosh and read the paper. Excellent work!

Obviously it would be fabulous if X forwarding could work with mosh,
but just as obviously, tunneling the X protocol is going to be
horrible (or at least, mosh can't do any better than ssh does). It
seems like you'd need some way to turn X remote display into a
state-synchronization problem.

A few years ago I wrote (and then, uh, abandoned) a remote display
program for X, called "xpra". The maintained fork is here:
http://xpra.org, and if you want a technical summary, the best place
to start is probably here:
  http://xpra.org/trac/browser/trunk/src/README.xpra
Traditional X is horribly latency sensitive; xpra fixes that by using,
a, well, state-synchronization protocol instead. That way we're
basically round-trip free, and also get to throw away window updates
that get immediately overwritten, all the positions your mouse pointer
passed through on the way between A and B, etc. It does have a
problem, though -- as designed, it relies on the underlying network
transport to manage buffers properly. It never buffers messages
itself; to minimize latency, it generates them on-demand whenever its
socket becomes writeable. While tunneling over ssh over TCP. Isn't
that charmingly naive? (It does seem to have grown some sort of
by-hand round-trip-estimator buffer control since I last looked.) And
of course if you get disconnected you have to reconnect by hand, and
all that.

So... I'm too busy to actually follow up on this myself. But I wanted
to point out that there are some potential Valuable Synergies(tm)
here, in hopes it might spark something.

Cheers,
-- Nathaniel

P.S.: There aren't more docs on the protocol, but here's some links to
the key bits of code. It's pretty straightforward. I'm linking to my
obsolete, mostly-abandoned version because I am confused by the worker
threads and x264 encoding and whatnot that seem to have grown up in my
absence.
Server-side protocol:
  compressing sends:
    https://code.google.com/p/partiwm/source/browse/xpra/server.py#150
  the list of incoming message types:
    https://code.google.com/p/partiwm/source/browse/xpra/server.py#651
Client-side protocol:
  compressing sends:
    https://code.google.com/p/partiwm/source/browse/xpra/client.py#24
  the list of incoming message types:
    https://code.google.com/p/partiwm/source/browse/xpra/client.py#397