[mosh-devel] Mosh connections didn't come back after ca. 18000 sec over 2x NAT

[Axel Beckert]

Hi Quentin,

On Sun, Dec 30, 2012 at 11:22:42AM -0500, Quentin Smith wrote:
> On Sun, 30 Dec 2012, Axel Beckert wrote:
> > Any idea what could have cause such a bad lockup in a mosh connection?
> > IIRC as of now, mosh does DNS lookups only once at start, so it
> > couldn't be a cached bad DNS reply or such.
> 
> Just to check the obvious first - to your knowledge, the server
> resolved to the same IP address before and after you restarted the
> AP? (That is, the server didn't appear to move for any reason?)

That would have meant a compromise of DNS servers in at least two
domains. :-)

Nevertheless, I checked all for machines where this happened and for
two of them I know the IP addresses by mind and they didn't change.
And the other three look at least familiar.

> What version of mosh are you using? Mosh 1.2.3 adds a new behavior
> where it will try opening a new connection on a new port if it
> hasn't heard from the server in a while (I think 10 seconds?). This
> is to work around some braindead NAT devices that have behavior
> similar to what you're describing.

It was always 1.2.3 (from the offical Debian Wheezy package) on the
client side and 2x 1.2.3 (same package) on the server side and 3x
1.2.2 from Debian Backports on the server side.

> > I still kept one of the non-responding mosh sessions open (now at
> > 21521 seconds), so I can possibly debug that session.
>
> If you capture a tcpdump on both the client and server for ~30
> seconds or so, that should show conclusively if your network is
> eating the packets, or the client or server is confused.

Looks like the latter. I captured this on the client side:

# tcpdump -i wlan0 host 78.46.73.201
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 65535 bytes
18:08:38.183852 IP xenlink.noone.org.60001 > c-crosser.local.55013: UDP, length 116
18:08:38.295591 IP c-crosser.local.55013 > xenlink.noone.org.60001: UDP, length 71
18:08:39.143872 IP xenlink.noone.org.60001 > c-crosser.local.55013: UDP, length 133
18:08:39.253217 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 82
18:08:40.064163 IP xenlink.noone.org.60001 > c-crosser.local.55013: UDP, length 132
18:08:40.172792 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 78
18:08:40.865203 IP xenlink.noone.org.60001 > c-crosser.local.55013: UDP, length 133
18:08:40.973333 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 77
18:08:41.785033 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 124
18:08:41.894189 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 69
18:08:41.963862 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 114
18:08:42.074045 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 81
18:08:42.725320 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 121
18:08:42.837176 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 79
18:08:44.046276 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 124
18:08:44.162170 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 81
18:08:44.665467 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 114
18:08:44.774038 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 83
18:08:45.983868 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 126
18:08:46.094821 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 81
18:08:46.602694 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 127
18:08:46.716342 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 75
18:08:47.544721 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 130
18:08:47.651795 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 76
18:08:49.003503 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 120
18:08:49.110928 IP c-crosser.local.53665 > xenlink.noone.org.60001: UDP, length 78
18:08:49.746999 IP xenlink.noone.org.60001 > c-crosser.local.53665: UDP, length 129
18:08:49.855992 IP c-crosser.local.45626 > xenlink.noone.org.60001: UDP, length 69

Nevertheless, the top line of that mosh session (and I only have one
to that host) still says "mosh: Last reply 26184 seconds ago. [To
quit: Ctrl-^ .]"

		Kind regards, Axel
-- 
/~\  Plain Text Ribbon Campaign                   | Axel Beckert
\ /  Say No to HTML in E-Mail and News            | abe at deuxchevaux.org  (Mail)
 X   See http://www.asciiribbon.org/              | abe at noone.org (Mail+Jabber)
/ \  I love long mails: http://email.is-not-s.ms/ | http://noone.org/abe/ (Web)