When I was doing a recursive retrieval with the -E option, it was downloading the robots.txt, saving it as robots.txt.html, and then trying to read robots.txt (unsuccessfully), and retrying to fetch robots.txt. I didn't keep the details when this happened, so some of this may be off. I'll have to spend some time reproducing this.

My guess is that the robots.txt file may have been served as text/html, in which case it's debatable what wget's reaction should be.