This bug also applies to other anchors (\<,\>,\b, etc.)
echo A llittle problem | grep -o "\<l"
responds with two matches instead of one.
Also,
echo A llittle problem | grep --color "\<l"
colors both l's in llittle, when it should only color the first one.

I suspect, though I have not looked at the code, that since grep was originally designed to show the entire line of input if any part of the line matches the expression, it originally stopped searching for a pattern after the first match. When the -o (and later the related --color) option was added, to be able to show all the matches on a line, the regular expression search was restarted on the line at the point of the last match as though it was the beginning of the line. Hence,
echo This is a  liittle problem | grep -E --color "(\<l)|(^i)"
colors the lii in liitle.