[mosh-devel] X forwarding support via devious means

[Nathaniel Smith]

On Fri, Apr 13, 2012 at 8:16 PM, Keith Winstein <keithw at mit.edu> wrote:
> Hi Nathaniel,
>
> Thanks for your kind words and very interesting pointer!
>
> My concern here is that I'm far from an X protocol expert, and
> successfully importing xpra (with a dependence on Xvfb) and then
> polishing and maintaining it in mosh looks like a major project. It
> sounds like you may have other maintainers who have continued to work
> on xpra? If you or some of them want to take a stab at this (and
> maintaining it!) and want to make contact to work together, we'd be
> interested to talk. But without that brain trust, I think our
> temptation will be to do the simplest thing possible, because if we
> try to MITM an X connection, suddenly we have to have considerable
> expertise to maintain and debug that.

To be clear: I have no special knowledge of the X protocol either;
xpra doesn't actually MITM an X connection directly. It lets clients
connect to Xvfb, and then it connects to Xvfb as another X client
(using gtk+ and xlib), and basically has a clever scheme for taking
screenshots of each window and sending those over the wire. There's
some nasty stuff in xpra's innards involving ICCCM compatibility and X
keycodes and stuff, but that's all abstracted away from the protocol
layer, which just synchronizes bitmaps and simple "button pressed",
"window closed", kinds of events. That said, maintaining a window
manager has its own challenges :-)

Is there a library interface to the SUPDUP stuff, or any mechanism for
two SUPDUP clients to multiplex over a single connection? (I suppose
you could just have one mosh connection command that spawns two
servers with two UDP ports. Hmm.)

I'll forward this to the current maintainers and see if anyone bites.

-- Nathaniel

> Thanks again for sending, and best regards,
> Keith
>
> On Thu, Apr 12, 2012 at 4:07 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> Hi all,
>>
>> Just saw mosh and read the paper. Excellent work!
>>
>> Obviously it would be fabulous if X forwarding could work with mosh,
>> but just as obviously, tunneling the X protocol is going to be
>> horrible (or at least, mosh can't do any better than ssh does). It
>> seems like you'd need some way to turn X remote display into a
>> state-synchronization problem.
>>
>> A few years ago I wrote (and then, uh, abandoned) a remote display
>> program for X, called "xpra". The maintained fork is here:
>> http://xpra.org, and if you want a technical summary, the best place
>> to start is probably here:
>>  http://xpra.org/trac/browser/trunk/src/README.xpra
>> Traditional X is horribly latency sensitive; xpra fixes that by using,
>> a, well, state-synchronization protocol instead. That way we're
>> basically round-trip free, and also get to throw away window updates
>> that get immediately overwritten, all the positions your mouse pointer
>> passed through on the way between A and B, etc. It does have a
>> problem, though -- as designed, it relies on the underlying network
>> transport to manage buffers properly. It never buffers messages
>> itself; to minimize latency, it generates them on-demand whenever its
>> socket becomes writeable. While tunneling over ssh over TCP. Isn't
>> that charmingly naive? (It does seem to have grown some sort of
>> by-hand round-trip-estimator buffer control since I last looked.) And
>> of course if you get disconnected you have to reconnect by hand, and
>> all that.
>>
>> So... I'm too busy to actually follow up on this myself. But I wanted
>> to point out that there are some potential Valuable Synergies(tm)
>> here, in hopes it might spark something.
>>
>> Cheers,
>> -- Nathaniel
>>
>> P.S.: There aren't more docs on the protocol, but here's some links to
>> the key bits of code. It's pretty straightforward. I'm linking to my
>> obsolete, mostly-abandoned version because I am confused by the worker
>> threads and x264 encoding and whatnot that seem to have grown up in my
>> absence.
>> Server-side protocol:
>>  compressing sends:
>>    https://code.google.com/p/partiwm/source/browse/xpra/server.py#150
>>  the list of incoming message types:
>>    https://code.google.com/p/partiwm/source/browse/xpra/server.py#651
>> Client-side protocol:
>>  compressing sends:
>>    https://code.google.com/p/partiwm/source/browse/xpra/client.py#24
>>  the list of incoming message types:
>>    https://code.google.com/p/partiwm/source/browse/xpra/client.py#397
>> _______________________________________________
>> mosh-devel mailing list
>> mosh-devel at mit.edu
>> http://mailman.mit.edu/mailman/listinfo/mosh-devel