[mosh-devel] Mosh connections didn't come back after ca. 18000 sec over 2x NAT

[Keith Winstein]

Something is awry here but I am a little confused, especially about why 
packets are being sent so often. I wonder if somehow two sessions got 
slotted in to the same spot, somehow, on the intermediate NATted link. But 
really I don't quite understand what is going on.

Is the display (visible on the client) updating the screen often or at 
all? Do you know what is running on the server (is it continually changing 
the screen state)?

We may need to figure out a way for you to break in to the server with gdb 
and turn on verbose debugging. Is the server running the wheezy 1.2.3 
package on x86, amd64, or another architecture?

-Keith

On Sun, 30 Dec 2012, Axel Beckert wrote:

> Hi Keith,
>
> On Sun, Dec 30, 2012 at 12:31:46PM -0500, Keith Winstein wrote:
>> Thanks for the detailed report. "Last reply" means that the _server_
>> is not getting (or at least not acknowledging) packets from the
>> _client_. (If the client were not getting packets at all, it would
>> say "Last contact.")
>
> I see.
>
>> So the client-side tcpdump is somewhat as expected. Are you able to
>> send a similar tcpdump from the server side? I hope that might help
>> resolve the mystery.
>
> Yeah, I can do that via SSH. :-)
>
> # tcpdump -i eth0 host 212.23.103.125 and not port 22 and not host 78.46.73.207 and not host 178.63.92.236
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> 19:20:26.118258 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 114
> 19:20:26.648257 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 67
> 19:20:27.117409 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 120
> 19:20:28.120633 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 114
> 19:20:28.383553 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 79
> 19:20:28.923537 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 73
> 19:20:29.119180 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 124
> 19:20:30.117066 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 140
> 19:20:30.162439 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 77
> 19:20:31.121057 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 134
> 19:20:32.118384 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 133
> 19:20:32.165360 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 73
> 19:20:33.120720 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 123
> 19:20:33.745353 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 82
> 19:20:34.118490 IP xenlink.noone.org.60001 > 212.23.103.125.52834: UDP, length 122
> 19:20:34.165574 IP 212.23.103.125.52834 > xenlink.noone.org.60001: UDP, length 68
> 19:20:34.866453 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 71
> 19:20:34.963754 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 76
> 19:20:35.118092 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 126
> 19:20:36.121004 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 116
> 19:20:36.423953 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 68
> 19:20:37.117481 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 129
> 19:20:37.783474 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 75
> 19:20:38.120108 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 123
> 19:20:38.241532 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 69
> 19:20:39.119904 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 126
> 19:20:39.284710 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 72
> 19:20:39.882758 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 81
> 19:20:40.118030 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 126
> 19:20:41.119489 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 121
> 19:20:41.343944 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 78
> 19:20:41.905053 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 81
> 19:20:42.117682 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 125
> 19:20:43.121191 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 124
> 19:20:43.342843 IP 212.23.103.125.48736 > xenlink.noone.org.60001: UDP, length 74
> 19:20:44.121087 IP xenlink.noone.org.60001 > 212.23.103.125.48736: UDP, length 118
> 19:20:44.903687 IP 212.23.103.125.58270 > xenlink.noone.org.60001: UDP, length 71
> 19:20:45.120348 IP xenlink.noone.org.60001 > 212.23.103.125.58270: UDP, length 127
> 19:20:45.765318 IP 212.23.103.125.58270 > xenlink.noone.org.60001: UDP, length 74
> 19:20:46.122775 IP xenlink.noone.org.60001 > 212.23.103.125.58270: UDP, length 132
> 19:20:46.708392 IP 212.23.103.125.58270 > xenlink.noone.org.60001: UDP, length 76
> 19:20:46.710505 IP 212.23.103.125.58270 > xenlink.noone.org.60001: UDP, length 70
> 19:20:47.119126 IP xenlink.noone.org.60001 > 212.23.103.125.58270: UDP, length 122
>
> (I had to exclude some other hosts which are reachable through that
> host and to which I have (more recent) mosh connections, too.)
>
>> I do see from this tcpdump that the mosh-client is doing its
>> port-hopping and the proximal (client-side) NAT correctly responded
>> to at least one port hop (from 55013 to 53665). I'm curious what it
>> looks like from the other end of the "distal" NAT.
>
> I waited a little bit longer and think we see 2x port hopping in that
> dump. Hope it helps to solve the mystery. If not, feel free to ask for
> some more sniffing.
>
> 		Kind regards, Axel
> -- 
> /~\  Plain Text Ribbon Campaign                   | Axel Beckert
> \ /  Say No to HTML in E-Mail and News            | abe at deuxchevaux.org  (Mail)
> X   See http://www.asciiribbon.org/              | abe at noone.org (Mail+Jabber)
> / \  I love long mails: http://email.is-not-s.ms/ | http://noone.org/abe/ (Web)
> _______________________________________________
> mosh-devel mailing list
> mosh-devel at mit.edu
> http://mailman.mit.edu/mailman/listinfo/mosh-devel
>