[mosh-devel] Mosh on Kuberbnetes

[Thomas Buckley-Houston]

Thanks Jim,

That really clears things up for me.

So but you're saying that Keith's idea of having the outgoing
datagram's source IP remapped is not possible?

I totally agree about the traditional approach of the "bastion host",
it's basically like having SSH act as the "load balancer" - as I'll
have some shell script open up a connection to a random server on the
bastion's CLI. The only problem there is that it doesn't take
advantage of traditional cluster architecture so well. For instance
I'll need to come up with a custom method of keeping a live list of
the available internal IP addresses to which I can forward the
session. Any idea how many connections an average 512MB VM could
handle like this? I suspect so many that you'd only ever need one VM?

Tom

On 13 July 2018 at 12:54, Jim Cheetham <jim.cheetham at otago.ac.nz> wrote:
> Excerpts from Thomas Buckley-Houston's message of July 13, 2018 3:58 pm:
>> So
>> because UDP is a connectionless protocol, every datagram has to
>> contain the source IP in order for the receiver to send responses?
>
> Every packet does have the source IP address in it, but not because it's a UDP packet, it's because that's what all *IP* packets have. UDP is a protocol built on top of IP, as is TCP - we're used to the "full name" of TCP being TCP/IP, but it's unusual to say "UDP/IP" :-)
>
>> It's not enough for either end to assume an IP based on the *initial*
>> datagram's IP?
>
> Because UDP is connectionless, there's no concept of an initial packet in the first place; all packets that arrive are completely self-contained entities with no relation to any other packets that might arrive before or after. It's a bit like HTTP in that respect.
>
> The *application* that's using UDP may have a concept of a long-term session that involved multiple packets, but UDP itself doesn't. Again looking at HTTP, that's like using cookies to describe a session.
>
>> So based on that assumption, the best way for a
>> datagram to get its IP:port is to query the OS. Which in our scenario
>> is a privately networked container and so will be unreachable from the
>> outside.
>
> You have a middle layer load balancer, which will be keeping track of the state of sessions going through; this is easy for TCP because the session identifiers are basically in every packet. However, in UDP they are not present at all - you have to be aware of the actual application using UDP; you have to be explicitly aware of mosh itself in order to be able to load-balance/proxy it correctly.
>
> A simple assumption for most protocols that want to have cheap lightweight sessions is to assume that the source IP address stays the same throughout. This happens to have been true for most of the network's history, but has never been guaranteed.
>
> In the case of mosh, this is explicitly not true; the protocol *expects* that the source IP address will change unannounced throughout the application's session.
>
> It is this issue that you need to be looking in to. You need a proxy/LB layer that looks at a UDP packet coming through, recognises that it is using SSP (State Synchronization Protocol), and keeps track of which backend system the packet should be forwarded to.
>
> Unfortunately, this will be very difficult; datagrams are encrypted, and your proxy will not be aware of the key in use. The only way I can see to make this happen will be to alter the mosh server to inform the proxy directly, and that's a lot of engineering that will introduce a number of security tradeoffs that might not be worthwhile.
>
> Overall, your better approach is to have a traditional "bastion host" in the middle, use mosh to connect to that, and from there make inbound ssh connections to your service hosts (because you can get away with ssh on an internal "reliable" network, and because you're using TCP for this, you can go through a load balancer if required).
>
> -jim
>
> --
> Jim Cheetham, Information Security, University of Otago, Dunedin, N.Z.
> ✉ jim.cheetham at otago.ac.nz    ☏ +64 3 470 4670    ☏ m +64 21 279 4670
> ⚷ OpenPGP: B50F BE3B D49B 3A8A 9CC3 8966 9374 82CD C982 0605