I came up with a pretty efficeient method to reproduce this issue.

I edit the grub.cfg so it only contatins this lines:

set timeout="0"

menuentry "Reboot" {
              reboot
}

This will of course just reboot as soon as the grub.cfg is successfully fetched but as soon as grub fail to fetch it will stop at the grub prompt, hopefully making it possible for someone that knows the internals of GRUB to shine some light on what is preventing GRUB to make the fetch request for the grub.cfg.

As staded before, I can reproduce the issue on all Intel NUC generation 11 models, I have not tried NUC 12. It cannot be reproduced on any device pre gen 11.

Short background:
We sell a commercial server system that is made to handle large Audi/Video installations handling both device control and media management and interaction. The media players (Intel NUC in general) runs Porteus Kiosk https://porteus-kiosk.org/. That system uses grub to PXE boot from our server. It has always worked very well in the past. The OS runs as read only on ramdisk and the boot files are kept on our server.

Problem:
With the 11th generation Intel NUC, all flavours i3 to i7 we started to get reports from installations where they saw the kiosks sometimes does not finish the boot process but instead ends up showing the grub prompt. Since installations may be with 100 or more computers all running without a human user that can reboot, it is a problem.

Way to reproduce:
Since our system (pixilab.se/blocks) also can easily be programmed doing different tasks I set up a task that as soon as the kiosk browser connects to our web-server we send a power down, wait for a bit then send a WakeOnLan to cold boot again. With this loop it is pretty easy to reproduce the issue and there is a counter and a log setup that keep track on how many successful loops we made and how long any boot process took.

 
Error frequency: Around 1 in 1000 attempts. It may happen at the third attempt; it may run up so 700 attempts then fail. It seems to be something that can happen at every boot but the probability is pretty low.

What I found:
I have made wireshark recordings from successful and unsuccessful attempts. The fetch and execution of the grup  packageThe only thing found there was that when it fails, there is never any request to fetch the grub.cfg file from server.

If I try to fetch the grub.cfg from manually from grub command line the screen goes black for a second or two then the grub prompt returns again. ( I have this on video but it seem to be problem to submit with that attached)

It seem grub is crippled in the way it cannot sent anything out on the network at all. Not tftp and not http. 

From what I can see Grub seem to have a network device that is up with a valid IP.

Who is affected:
Probably anyone running PXE boot with GRUB 2.06 or 2.16 but mainly userless applications such as IoT and Signage and cluster applications running on Intel NUC gen 11 hardware. (Perhaps any Intel device gen 11, NUC is what I have here)  

I am more than happy to provide any data that I can,but will need help from someone to provide some syntax that may help to shine a light in what goes wrong.

I have raised this concern with Intel Custome Service case No 05585598. I have had some rather good response, but since they do not have the facilities to create the loop they gave up after just 20 attempts. They even provided some test firmware to see if it helped. But I figure, perhaps someone that is fluent in GRUB 2 may be of assistance to try to figure out why and what happens.


I believe this problem may affect any future product from Intel unless fixed by Intel or Grub.
I am happy to assist since I can reproduce without too much effort.