bugGNU GRUB - Bugs: bug #43653, grub-probe fails to find ZFS VDEVs...


You are not allowed to post comments on this tracker with your current authentication level.

bug #43653: grub-probe fails to find ZFS VDEVs in /dev/mapper

Submitted by:  Bart Kus <bkus>
Submitted on:  Fri 21 Nov 2014 01:26:54 AM UTC  
Category:  Disk & Partition Severity:  Major
Priority:  6 Item Group:  Software Error
Status:  Need Info Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Open Release: 
Release:  2.02~beta1 Reproducibility:  Every Time
Planned Release:  2.02

( Jump to the original submission)

Wed 03 May 2017 03:55:26 PM UTC, comment #16:

~ # zfs list
rpool 163G 760G 19K none
rpool/ROOT 2.55G 760G 19K none
rpool/ROOT/linux 2.55G 760G 1.61G /target
rpool/ROOT/linux/home 19K 760G 19K legacy
rpool/ROOT/linux/var 963M 760G 963M legacy
rpool/VARSHARE 19K 760G 19K /target/var/share
rpool/dump 64.0G 824G 8K -
rpool/swap 96.0G 856G 8

~ # zpool status
pool: rpool
state: ONLINE
scan: none requested

rpool ONLINE 0 0 0
nvme0n1p2 ONLINE 0 0 0

~ # zpool status -P
pool: rpool
state: ONLINE
scan: none requested

rpool ONLINE 0 0 0
/dev/nvme0n1p2 ONLINE 0 0 0

May 3 15:37:27 in-target: grub-common is already the newest version (2.02~beta2-36ubuntu3.9).
May 3 15:37:28 main-menu[422]: (process:4764): grub-probe: error: failed to get canonical path of `rpool/ROOT/linux'.

Jens Elkner <jelmd>
Fri 04 Mar 2016 12:06:33 AM UTC, comment #15:

Andrei, here it is, working.

$ sudo grub-probe /

$ sudo strace -v -f grub-probe / 2>&1 |grep /zpool
[pid 11679] execve("/sbin/zpool", ["zpool", "status", "gelb"], ["LC_PAPER=en_AU.UTF-8", "LC_ADDRESS=en_AU.UTF-8", "LC_MONETARY=en_AU.UTF-8", "TERM=xterm-256color", "LC_NUMERIC=en_AU.UTF-8", "LC_TELEPHONE=en_AU.UTF-8", "LC_IDENTIFICATION=en_AU.UTF-8", "LANG=en_US.UTF-8", "LC_MEASUREMENT=en_AU.UTF-8", "LANGUAGE=en", "LC_TIME=en_AU.UTF-8", "LC_NAME=en_AU.UTF-8", "SHELL=/bin/bash", "MAIL=/var/mail/root", "PATH=/usr/local/sbin:/usr/local/"..., "LOGNAME=root", "USER=root", "USERNAME=root", "HOME=/root", "SUDO_COMMAND=/usr/bin/strace -v "..., "SUDO_USER=cmiller", "SUDO_UID=1000", "SUDO_GID=1000", "http_proxy="..., "ZPOOL_VDEV_NAME_PATH=YES", "LC_ALL=C"]) = 0

# zpool status gelb

gelb ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-ST31000333AS_9TE1QFL7-part1 ONLINE 0 0 0

# ZPOOL_VDEV_NAME_PATH=yes zpool status gelb

gelb ONLINE 0 0 0
raidz1 ONLINE 0 0 0
/dev/disk/by-id/ata-ST31000333AS_9TE1QFL7-part1 ONLINE 0 0 0


Chad Miller <chadmiller>
Fri 26 Feb 2016 07:32:30 PM UTC, comment #14:

Did you actually test it with updated ZFS?

Andrei Borzenkov <arvidjaar>
Project Member
Fri 26 Feb 2016 04:19:45 PM UTC, comment #13:

zfs-linux now supports environment variable to signal full path output of zpool.


Now I think a change to grub is justified and useful.

(file #36473)

Chad Miller <chadmiller>
Tue 16 Feb 2016 06:12:07 PM UTC, comment #12:

arvidjaar, it's not used yet. I proposed a change in a bug report earlier today.


The beauty of an environment variable is that it adds optional functionality and doesn't make Grub depend on a particular version of ZFS utils. Grub doesn't have to know ZFS accepts a parameter or even this variable name. At worst, nothing changes.

Chad Miller <chadmiller>
Tue 16 Feb 2016 05:22:38 PM UTC, comment #11:

I do not see this variable being used in current ZFSoL GIT. Could you provide link to discussion? Thank you.

Andrei Borzenkov <arvidjaar>
Project Member
Tue 16 Feb 2016 04:48:02 PM UTC, comment #10:

This patch is a suggestion to zfs zpool to emit full device names. If the system zpool command supports it, it gives us the data we need for all users. If it doesn't, older users still continue to work as they did before. A new parameter would break older installs, and complicate the code a great deal in detecting parameter errors and retrying without parameter.

Chad Miller <chadmiller>
Fri 12 Feb 2016 08:54:51 AM UTC, comment #9:

I may be mistaken, but my last information is that zdb is for internal use only, its output is not stable, is not guaranteed to remain compatible.

I'm fine to use whatever zfs developers say has long term stability.

Andrei Borzenkov <arvidjaar>
Project Member
Fri 12 Feb 2016 08:12:01 AM UTC, comment #8:

Another example (to show what a raidz looks like, mirror is probably similar, my previous example is a single drive pool)

This is not best example since this one is using /dev/sdX for it's members, so would probably work as-is, but if it was using /dev/disk/by-id it wouldn't, so just use your imagination. :)

Sadly, my C-fu is too weak to attempt writing any patches myself at the moment. I haven't done any "real" C programming in probably a decade. I glanced at the code and the output parsing would need to be rewritten since the output is in a different format, but if that's done it should be fine.

Jonathan Vaughn <biosehnsucht>
Fri 12 Feb 2016 08:01:57 AM UTC, comment #7:

Instead of 'zpool status POOL', more useful information could be gleaned from 'zdb -C POOL'. Among other things, zdb provides the full device path for each member of the pool.
For comparison, using zpool:

And using zdb:

Jonathan Vaughn <biosehnsucht>
Sat 09 Jan 2016 01:28:12 PM UTC, comment #6:

You want us to be holier than zfs? Why you do not ask ZFSoL folks to provide official interface to list full vdev names?

Anyway, I will agree to a patch that is using ZPOOL_IMPORT_PATH (and probably default zpool import path) to resolve short names. The obvious issue with default import path is that we have no idea what your particular ZFSoL installtion is using.

I have no way to test it, so someone will need to provide patch.

Andrei Borzenkov <arvidjaar>
Project Member
Fri 08 Jan 2016 03:54:27 PM UTC, comment #5:

grub and libzfs have incompatible licenses, so they will never be linked in any place that cares about licenses. No distro will do that.

SO, fallback code has to be good enough.

Chad Miller <chadmiller>
Sat 26 Dec 2015 07:29:12 AM UTC, comment #4:

As you noticed this bug is not marked as resolved.

What do you mean "third party"? If your system has zpool your system has libzfs. If your system has libzfs why you cannot link grub with it?

The proper fix should go into ZFS. Either zpool should provide option to return full pathname(s) or zpool should resolve its output to primary names. Otherwise we will always have some directory that is not covered (/dev/mapper/, /dev/md/, /dev/cciss/ to name few).

Andrei Borzenkov <arvidjaar>
Project Member
Sat 26 Dec 2015 04:15:58 AM UTC, comment #3:

Oh, a third party library could fix it, if only it were linked in. Great.

Prepending /dev/ is still wrong. And that's in grub. Erase it or guard it with configure-time conditions or something, but don't mark this as resolved until the bad code is gone.

There's a fix attached to another bug.


Chad Miller <chadmiller>
Fri 21 Nov 2014 08:29:46 AM UTC, comment #2:

ii libzfs2 0.6.3-1~wheezy amd64 Native ZFS filesystem library for Linux
ii libnvpair1 0.6.3-1~wheezy amd64 Solaris name-value library for Linux

Debian is reporting those libs as installed. However, I see they're coming from zfsonlinux.org and not from the Debian repo, whereas grub is coming from the Debian repo, so it's not linked against these libs:

root@debian:~# ldd /usr/sbin/grub-probe
linux-vdso.so.1 (0x00007fffb53a1000)
libdevmapper.so.1.02.1 => /lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 (0x00007fe77abc4000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe77a81b000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fe77a5f5000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fe77a3e6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe77a1c9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe77ae14000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe779f5a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe779d56000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe779b4e000)

Glad to hear the support exists and will come into distros with time. However, I still think it'd be a good idea for grub-probe to be more thorough about finding devices in /dev sub-dirs when it falls back to the zpool method.

Bart Kus <bkus>
Fri 21 Nov 2014 06:17:36 AM UTC, comment #1:

grub will fall back to zpool only if libzfs/libnvpair are not found; otherwise it fetches vdev path information directly. You may need to install development packages for those libraries and rebuild grub. Please test using libzfs - does it work?

Andrei Borzenkov <arvidjaar>
Project Member
Fri 21 Nov 2014 01:26:54 AM UTC, original submission:

When the root filesystem is a ZFS composed of VDEVs from /dev/mapper (eg: /dev/mapper/cryptroot), grub-probe fails to find the underlying device(s). This is because it calls out to zpool to find the VDEVs of rpool, but they're listed as non-absolute names in zpool's output, and grub-probe then tries to find the full paths in /dev, but not any sub-dir of /dev, like /dev/mapper.

A workaround is to create symlinks in /dev using the exact same device names as listed in zpool's output. grub-probe then succeeds and things like grub-mkconfig start working.

Please enhance grub-probe's support for ZFS by searching deeper in the /dev tree, or find an alternate more precise way of obtaining the VDEV information from ZFS.

Bart Kus <bkus>


Attached Files
file #36473:  zpool_full_device_name.patch added by chadmiller (808B - text/x-patch - Signal to zpool to emit full vdev path names. Replaces earlier suggestion file #36357)
file #36357:  zpool-device-full-paths-envvar.patch added by chadmiller (303B - text/x-patch - Ask ZFS pool command for device names, hopefully with full paths. Use environment variable for maximum compatibility.)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by delx
  • -email is unavailable- added by jelmd (Posted a comment)
  • -email is unavailable- added by seschwar
  • -email is unavailable- added by phcoder (Updated the item)
  • -email is unavailable- added by biosehnsucht (Posted a comment)
  • -email is unavailable- added by chadmiller (Posted a comment)
  • -email is unavailable- added by arvidjaar (Posted a comment)
  • -email is unavailable- added by bkus (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-09-08 delx Carbon-Copy- => Added delx
    2016-10-11 phcoder Priority5 - Normal => 6
    2016-09-18 seschwar Carbon-Copy- => Added seschwar
    2016-03-02 phcoder Planned Release2.03+ => 2.02
    2016-03-02 phcoder Planned ReleaseNone => 2.03+
    2016-02-26 chadmiller Attached File- => Added zpool_full_device_name.patch, #36473
    2016-02-16 chadmiller Attached File- => Added zpool-device-full-paths-envvar.patch, #36357
    2014-11-21 arvidjaar StatusNone => Need Info

    Back to the top

    Powered by Savane 3.3