bugGNU GRUB - Bugs: bug #63245, PXE boot with GRUB 2.06 sometimes...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #63245: PXE boot with GRUB 2.06 sometimes fail on generation 11 Intel NUCs

Submitter:  Matttias Andersson <jerrymattias>
Submitted:  Thu 20 Oct 2022 06:30:08 AM UTC
   
 
Category:  None Severity:  Major
Priority:  5 - Normal Item Group:  Hardware-specific
Status:  None Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Open Release:  other
Release:  Reproducibility:  Intermittent
Planned Release:  None

Thu 27 Oct 2022 09:02:50 AM UTC, comment #2: 

I came up with a pretty efficeient method to reproduce this issue.

I edit the grub.cfg so it only contatins this lines:

set timeout="0"

menuentry "Reboot" {
              reboot
}

This will of course just reboot as soon as the grub.cfg is successfully fetched but as soon as grub fail to fetch it will stop at the grub prompt, hopefully making it possible for someone that knows the internals of GRUB to shine some light on what is preventing GRUB to make the fetch request for the grub.cfg.

As staded before, I can reproduce the issue on all Intel NUC generation 11 models, I have not tried NUC 12. It cannot be reproduced on any device pre gen 11.

Matttias Andersson <jerrymattias>
Thu 20 Oct 2022 06:40:50 AM UTC, comment #1: 

Link to video showing behaviour after failure:
https://1drv.ms/v/s!AggiUAzQLDWUooY9XvGgiR0vsa8n-w?e=AT7SNt

One idea Intel had was to try to PXE boot with a USB dongle, I bought one that was able to PXE boot on DELL but did not show up as an boot option on these devices. Intel was reluctant to point me in the right direction to a device that actually is supported by the UEFI firmware in NUCs.

Matttias Andersson <jerrymattias>
Thu 20 Oct 2022 06:30:08 AM UTC, original submission:  

Short background:
We sell a commercial server system that is made to handle large Audi/Video installations handling both device control and media management and interaction. The media players (Intel NUC in general) runs Porteus Kiosk https://porteus-kiosk.org/. That system uses grub to PXE boot from our server. It has always worked very well in the past. The OS runs as read only on ramdisk and the boot files are kept on our server.

Problem:
With the 11th generation Intel NUC, all flavours i3 to i7 we started to get reports from installations where they saw the kiosks sometimes does not finish the boot process but instead ends up showing the grub prompt. Since installations may be with 100 or more computers all running without a human user that can reboot, it is a problem.

Way to reproduce:
Since our system (pixilab.se/blocks) also can easily be programmed doing different tasks I set up a task that as soon as the kiosk browser connects to our web-server we send a power down, wait for a bit then send a WakeOnLan to cold boot again. With this loop it is pretty easy to reproduce the issue and there is a counter and a log setup that keep track on how many successful loops we made and how long any boot process took.

 
Error frequency: Around 1 in 1000 attempts. It may happen at the third attempt; it may run up so 700 attempts then fail. It seems to be something that can happen at every boot but the probability is pretty low.

What I found:
I have made wireshark recordings from successful and unsuccessful attempts. The fetch and execution of the grup  packageThe only thing found there was that when it fails, there is never any request to fetch the grub.cfg file from server.

If I try to fetch the grub.cfg from manually from grub command line the screen goes black for a second or two then the grub prompt returns again. ( I have this on video but it seem to be problem to submit with that attached)

It seem grub is crippled in the way it cannot sent anything out on the network at all. Not tftp and not http. 

From what I can see Grub seem to have a network device that is up with a valid IP.

Who is affected:
Probably anyone running PXE boot with GRUB 2.06 or 2.16 but mainly userless applications such as IoT and Signage and cluster applications running on Intel NUC gen 11 hardware. (Perhaps any Intel device gen 11, NUC is what I have here)  

I am more than happy to provide any data that I can,but will need help from someone to provide some syntax that may help to shine a light in what goes wrong.

I have raised this concern with Intel Custome Service case No 05585598. I have had some rather good response, but since they do not have the facilities to create the loop they gave up after just 20 attempts. They even provided some test firmware to see if it helped. But I figure, perhaps someone that is fluent in GRUB 2 may be of assistance to try to figure out why and what happens.


I believe this problem may affect any future product from Intel unless fixed by Intel or Grub.
I am happy to assist since I can reproduce without too much effort.



 

Matttias Andersson <jerrymattias>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #53885:  grubnetx64.efi.signed added by jerrymattias (1MiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jerrymattias (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-20 jerrymattias Attached File- Added grubnetx64.efi.signed, #53885

    Back to the top

    Powered by Savane 3.13-54b4.
    Corresponding source code