bugGNU GRUB - Bugs: bug #56391, Easily-triggered TCP bug(s)

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #56391: Easily-triggered TCP bug(s)

Submitter:  Faidon Liambotis <paravoid>
Submitted:  Sun 26 May 2019 02:59:11 AM UTC
Votes: 1
 
Category:  Network Severity:  Major
Priority:  5 - Normal Item Group:  Software Error
Status:  None Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Open Release:  Git master
Release:  Reproducibility:  Every Time
Planned Release:  None

Sun 26 May 2019 02:59:11 AM UTC, original submission:  

GRUB (git master @ 53e70d30cf0d18e6c28bab0ab8d223a90d3e1b46), PXE booted, with this configuration:
  set prefix=(http)/grub
  menuentry 'memtest86+' --id memtest {
    linux16 /boot/memtest86+.bin
  }

Boot the host (say, a QEMU). Reboot and try it again, relatively quickly.
Boot gets stuck at "Welcome to GRUB" and cannot fetch files for a while. This is the simplified version -- I started debugging this when in a complicated menu I was getting a "connection timed out" every now and then when selecting an entry like memtest.

The HTTP server is a Linux 4.9, with nginx, in the same LAN (no firewalls or anything like that). Nothing out of the ordinary there.

It appears that GRUB has a fixed local port that it starts with (21550); the (src IP, src port, dst IP, dst port) tuple remains the same in the two fetches across reboots, so the socket has remained in TIME_WAIT on the server. Thus, the server attempts to reuse the same ack nr, per RFC. All hell breaks loose after that. Multiple bugs I think -- GRUB gets something unexpected but doesn't notice, doesn't RST, and keeps sending the same ACKs over and over. Plus, Wireshark doesn't seem to be happy with the FIN/ACK of the first connection either.

(I took a look at the source, which seems ridden with FIXMEs. Have you thought about using a third-party TCP/IP stack like lwIP or PicoTCP? No idea how easy it would be to integrate this, apologies if this is a stupid idea :)

Packet capture from tshark (tshark -n -o tcp.relative_sequence_numbers:FALSE -r grub.pcap 'tcp.port == 21550'):

<first boot>
    1   0.000000 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [SYN] Seq=96 Win=8192 Len=0
    2   0.000290 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [SYN, ACK] Seq=150886417 Ack=97 Win=29200 Len=0 MSS=1460
    3   0.000689 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=97 Ack=150886418 Win=8192 Len=0
    4   0.001023 192.168.24.104 → 192.168.24.1 HTTP 162 GET /grub/i386-pc/command.lst HTTP/1.1
    5   0.001175 192.168.24.104 → 192.168.24.1 TCP 162 [TCP Retransmission] 21550 → 80 [PSH, ACK] Seq=97 Ack=150886418 Win=8192 Len=108
    6   0.001208 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [ACK] Seq=150886418 Ack=205 Win=29200 Len=0
    7   0.001335 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 6#1] 80 → 21550 [ACK] Seq=150886418 Ack=205 Win=29200 Len=0
    8   0.001576 192.168.24.1 → 192.168.24.104 TCP 590 HTTP/1.1 200 OK  [TCP segment of a reassembled PDU]
    9   0.001581 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150886954 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   10   0.001583 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150887490 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   11   0.001585 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150888026 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   12   0.001587 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150888562 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   13   0.001646 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150889098 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   14   0.001651 192.168.24.1 → 192.168.24.104 TCP 590 80 → 21550 [ACK] Seq=150889634 Ack=205 Win=29200 Len=536 [TCP segment of a reassembled PDU]
   15   0.001653 192.168.24.1 → 192.168.24.104 HTTP 393 HTTP/1.1 200 OK
   16   0.001894 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150886954 Win=8192 Len=0
   17   0.002365 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150887490 Win=8192 Len=0
   18   0.002554 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150888026 Win=8192 Len=0
   19   0.002721 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150888562 Win=8192 Len=0
   20   0.002885 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150889098 Win=8192 Len=0
   21   0.003047 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150889634 Win=8192 Len=0
   22   0.003213 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150890170 Win=8192 Len=0
   23   0.003393 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [ACK] Seq=205 Ack=150890510 Win=8192 Len=0
   24   0.003566 192.168.24.104 → 192.168.24.1 TCP 54 21550 → 80 [FIN, ACK] Seq=205 Ack=150890510 Win=0 Len=0
   25   0.003770 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Out-Of-Order] 21550 → 80 [FIN, ACK] Seq=205 Ack=150890510 Win=0 Len=0
   26   0.003798 192.168.24.1 → 192.168.24.104 TCP 60 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   27   0.003945 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 26#1] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
<reboot here>
   71  36.235831 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Port numbers reused] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   72  36.236082 192.168.24.1 → 192.168.24.104 TCP 60 [TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   73  36.236502 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#1] 21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   74  36.635996 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   75  37.035883 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   76  37.036117 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#1] [TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   77  37.036428 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#2] 21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   78  37.435783 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   79  37.835738 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   80  37.835972 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#2] [TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   81  37.836155 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#3] 21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   82  38.235704 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   83  38.635605 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   84  38.635810 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#3] [TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
   85  38.635962 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Dup ACK 0#4] 21550 → 80 [ACK] Seq=94 Ack=0 Win=8192 Len=0
   86  39.035594 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   87  39.435549 192.168.24.104 → 192.168.24.1 TCP 54 [TCP Retransmission] 21550 → 80 [SYN] Seq=93 Win=8192 Len=0
   88  39.435754 192.168.24.1 → 192.168.24.104 TCP 60 [TCP Dup ACK 72#4] [TCP ACKed unseen segment] 80 → 21550 [ACK] Seq=150890510 Ack=206 Win=29200 Len=0
<etc. etc. etc., keeps repeating for a while>

 Faidon Liambotis <paravoid>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jjw
  • -email is unavailable- added by paravoid (Submitted the item)

    •  

    There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-06-18 jjw Carbon-Copy- Added jjw

    Back to the top

    Copyright © 2024 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.13-1b59.
    Corresponding source code