bugGNU GRUB - Bugs: bug #43653, grub-probe fails to find ZFS VDEVs...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #43653: grub-probe fails to find ZFS VDEVs in /dev/mapper

Submitter:  Bart Kus <bkus>
Submitted:  Fri 21 Nov 2014 01:26:54 AM UTC
   
 
Category:  Disk & Partition Severity:  Major
Priority:  6 Item Group:  Software Error
Status:  Need Info Privacy:  Public
Assigned to:  None Originator Name: 
Open/Closed:  Open Release:  2.02~beta1
Release:  Reproducibility:  Every Time
Planned Release:  2.02

Jump to the original submission

Wed 03 May 2017 03:55:26 PM UTC, comment #16: 

~ # zfs list
NAME                    USED  AVAIL  REFER  MOUNTPOINT
rpool                   163G   760G    19K  none
rpool/ROOT             2.55G   760G    19K  none
rpool/ROOT/linux       2.55G   760G  1.61G  /target
rpool/ROOT/linux/home    19K   760G    19K  legacy
rpool/ROOT/linux/var    963M   760G   963M  legacy
rpool/VARSHARE           19K   760G    19K  /target/var/share
rpool/dump             64.0G   824G     8K  -
rpool/swap             96.0G   856G     8

~ # zpool status
  pool: rpool
 state: ONLINE
  scan: none requested
config:

NAME         STATE     READ WRITE CKSUM
rpool        ONLINE       0     0     0
  nvme0n1p2  ONLINE       0     0     0

~ # zpool status -P
  pool: rpool
 state: ONLINE
  scan: none requested
config:

NAME              STATE     READ WRITE CKSUM
rpool             ONLINE       0     0     0
  /dev/nvme0n1p2  ONLINE       0     0     0

May  3 15:37:27 in-target: grub-common is already the newest version (2.02~beta2-36ubuntu3.9).
...
May  3 15:37:28 main-menu[422]: (process:4764): grub-probe: error: failed to get canonical path of `rpool/ROOT/linux'.


Jens Elkner <jelmd>
Fri 04 Mar 2016 12:06:33 AM UTC, comment #15: 

Andrei, here it is, working.

$ sudo grub-probe  /
zfs


$ sudo strace -v -f grub-probe / 2>&1  |grep /zpool
[pid 11679] execve("/sbin/zpool", ["zpool", "status", "gelb"], ["LC_PAPER=en_AU.UTF-8", "LC_ADDRESS=en_AU.UTF-8", "LC_MONETARY=en_AU.UTF-8", "TERM=xterm-256color", "LC_NUMERIC=en_AU.UTF-8", "LC_TELEPHONE=en_AU.UTF-8", "LC_IDENTIFICATION=en_AU.UTF-8", "LANG=en_US.UTF-8", "LC_MEASUREMENT=en_AU.UTF-8", "LANGUAGE=en", "LC_TIME=en_AU.UTF-8", "LC_NAME=en_AU.UTF-8", "SHELL=/bin/bash", "MAIL=/var/mail/root", "PATH=/usr/local/sbin:/usr/local/"..., "LOGNAME=root", "USER=root", "USERNAME=root", "HOME=/root", "SUDO_COMMAND=/usr/bin/strace -v "..., "SUDO_USER=cmiller", "SUDO_UID=1000", "SUDO_GID=1000", "http_proxy=http://172.30.100.1:8"..., "ZPOOL_VDEV_NAME_PATH=YES", "LC_ALL=C"]) = 0


# zpool status gelb
...
config:

NAME                                                   STATE     READ WRITE CKSUM
gelb                                                   ONLINE       0     0     0
  raidz1-0                                             ONLINE       0     0     0
    ata-ST31000333AS_9TE1QFL7-part1                    ONLINE       0     0     0
...


# ZPOOL_VDEV_NAME_PATH=yes zpool status gelb
...
config:

NAME                                                                   STATE     READ WRITE CKSUM
gelb                                                                   ONLINE       0     0     0
  raidz1                                                               ONLINE       0     0     0
    /dev/disk/by-id/ata-ST31000333AS_9TE1QFL7-part1                    ONLINE       0     0     0

...


Chad Miller <chadmiller>
Fri 26 Feb 2016 07:32:30 PM UTC, comment #14: 

Did you actually test it with updated ZFS?

Andrei Borzenkov <arvidjaar>
Group Member
Fri 26 Feb 2016 04:19:45 PM UTC, comment #13: 

zfs-linux now supports environment variable to signal full path output of zpool.

https://github.com/zfsonlinux/zfs/commit/d2f3e292dccab23e47ade3c67677a10f353b9e85

Now I think a change to grub is justified and useful.

(file #36473)

Chad Miller <chadmiller>
Tue 16 Feb 2016 06:12:07 PM UTC, comment #12: 

arvidjaar, it's not used yet. I proposed a change in a bug report earlier today.

https://github.com/zfsonlinux/zfs/issues/4341

The beauty of an environment variable is that it adds optional functionality and doesn't make Grub depend on a particular version of ZFS utils. Grub doesn't have to know ZFS accepts a parameter or even this variable name. At worst, nothing changes.

Chad Miller <chadmiller>
Tue 16 Feb 2016 05:22:38 PM UTC, comment #11: 

I do not see this variable being used in current ZFSoL GIT. Could you provide link to discussion? Thank you.

Andrei Borzenkov <arvidjaar>
Group Member
Tue 16 Feb 2016 04:48:02 PM UTC, comment #10: 

This patch is a suggestion to zfs zpool to emit full device names. If the system zpool command supports it, it gives us the data we need for all users. If it doesn't, older users still continue to work as they did before. A new parameter would break older installs, and complicate the code a great deal in detecting parameter errors and retrying without parameter.

Chad Miller <chadmiller>
Fri 12 Feb 2016 08:54:51 AM UTC, comment #9: 

I may be mistaken, but my last information is that zdb is for internal use only, its output is not stable, is not guaranteed to remain compatible.

I'm fine to use whatever zfs developers say has long term stability.

Andrei Borzenkov <arvidjaar>
Group Member
Fri 12 Feb 2016 08:12:01 AM UTC, comment #8: 

Another example (to show what a raidz looks like, mirror is probably similar, my previous example is a single drive pool)


[root@localhost ~]# zdb -C storage

MOS Configuration:
        version: 5000
        name: 'storage'
        state: 0
        txg: 140131
        pool_guid: 17167584455341853717
        errata: 0
        hostname: 'localhost'
        vdev_children: 2
        vdev_tree:
            type: 'root'
            id: 0
            guid: 17167584455341853717
            create_txg: 4
            children[0]:
                type: 'raidz'
                id: 0
                guid: 16890424298360256069
                nparity: 2
                metaslab_array: 34
                metaslab_shift: 37
                ashift: 12
                asize: 20003866804224
                is_log: 0
                create_txg: 4
                children[0]:
                    type: 'disk'
                    id: 0
                    guid: 17777217965918563459
                    path: '/dev/sdb1'
                    whole_disk: 1
                    create_txg: 4
                children[1]:
                    type: 'disk'
                    id: 1
                    guid: 7290186694997986110
                    path: '/dev/sdg1'
                    whole_disk: 1
                    create_txg: 4
                children[2]:
                    type: 'disk'
                    id: 2
                    guid: 5903874119169633124
                    path: '/dev/sdc1'
                    whole_disk: 1
                    create_txg: 4
                children[3]:
                    type: 'disk'
                    id: 3
                    guid: 9042132936038593189
                    path: '/dev/sdf1'
                    whole_disk: 1
                    create_txg: 4
            children[1]:
                type: 'disk'
                id: 1
                guid: 10552128460515355461
                path: '/dev/disk/by-id/ata-SanDisk_SDSSDA120G_999999999999-part1'
                whole_disk: 0
                metaslab_array: 107
                metaslab_shift: 24
                ashift: 9
                asize: 1018691584
                is_log: 1
                create_txg: 60548
        features_for_read:
            com.delphix:hole_birth
            com.delphix:embedded_data


This is not best example since this one is using /dev/sdX for it's members, so would probably work as-is, but if it was using /dev/disk/by-id it wouldn't, so just use your imagination. :)

Sadly, my C-fu is too weak to attempt writing any patches myself at the moment. I haven't done any "real" C programming in probably a decade. I glanced at the code and the output parsing would need to be rewritten since the output is in a different format, but if that's done it should be fine.

Jonathan Vaughn <biosehnsucht>
Fri 12 Feb 2016 08:01:57 AM UTC, comment #7: 

Instead of 'zpool status POOL', more useful information could be gleaned from 'zdb -C POOL'. Among other things, zdb provides the full device path for each member of the pool.
For comparison, using zpool:


[root@localhost /]# zpool status rpool
  pool: rpool
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(5) for details.
  scan: none requested
config:

        NAME                                      STATE     READ WRITE CKSUM
        rpool                              ONLINE       0     0     0
          ata-TOSHIBA_DT01ACA050_999999999-part1  ONLINE       0     0     0

errors: No known data errors


And using zdb:


[root@localhost /]# zdb -C rpool

MOS Configuration:
        version: 5000
        name: 'rpool'
        state: 0
        txg: 4
        pool_guid: 11603639135903192856
        errata: 0
        hostname: 'localhost'
        vdev_children: 1
        vdev_tree:
            type: 'root'
            id: 0
            guid: 11603639135903192856
            create_txg: 4
            children[0]:
                type: 'disk'
                id: 0
                guid: 13527569036254253061
                path: '/dev/disk/by-id/ata-TOSHIBA_DT01ACA050_999999999-part1'
                whole_disk: 0
                metaslab_array: 34
                metaslab_shift: 32
                ashift: 12
                asize: 499565199360
                is_log: 0
                create_txg: 4
        features_for_read:


Jonathan Vaughn <biosehnsucht>
Sat 09 Jan 2016 01:28:12 PM UTC, comment #6: 

You want us to be holier than zfs? Why you do not ask ZFSoL folks to provide official interface to list full vdev names?

Anyway, I will agree to a patch that is using ZPOOL_IMPORT_PATH (and probably default zpool import path) to resolve short names. The obvious issue with default import path is that we have no idea what your particular ZFSoL installtion is using.

I have no way to test it, so someone will need to provide patch.

Andrei Borzenkov <arvidjaar>
Group Member
Fri 08 Jan 2016 03:54:27 PM UTC, comment #5: 

grub and libzfs have incompatible licenses, so they will never be linked in any place that cares about licenses. No distro will do that.

SO, fallback code has to be good enough.

Chad Miller <chadmiller>
Sat 26 Dec 2015 07:29:12 AM UTC, comment #4: 

As you noticed this bug is not marked as resolved.

What do you mean "third party"? If your system has zpool your system has libzfs. If your system has libzfs why you cannot link grub with it?

The proper fix should go into ZFS. Either zpool should provide option to return full pathname(s) or zpool should resolve its output to primary names. Otherwise we will always have some directory that is not covered (/dev/mapper/, /dev/md/, /dev/cciss/ to name few).

Andrei Borzenkov <arvidjaar>
Group Member
Sat 26 Dec 2015 04:15:58 AM UTC, comment #3: 

Oh, a third party library could fix it, if only it were linked in. Great.

Prepending /dev/ is still wrong. And that's in grub. Erase it or guard it with configure-time conditions or something, but don't mark this as resolved until the bad code is gone.

There's a fix attached to another bug.

https://savannah.gnu.org/bugs/download.php?file_id=35855


Chad Miller <chadmiller>
Fri 21 Nov 2014 08:29:46 AM UTC, comment #2: 

ii  libzfs2                        0.6.3-1~wheezy                amd64        Native ZFS filesystem library for Linux
ii  libnvpair1                     0.6.3-1~wheezy                amd64        Solaris name-value library for Linux

Debian is reporting those libs as installed.  However, I see they're coming from zfsonlinux.org and not from the Debian repo, whereas grub is coming from the Debian repo, so it's not linked against these libs:

root@debian:~# ldd /usr/sbin/grub-probe
        linux-vdso.so.1 (0x00007fffb53a1000)
        libdevmapper.so.1.02.1 => /lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 (0x00007fe77abc4000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe77a81b000)
        libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007fe77a5f5000)
        libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fe77a3e6000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe77a1c9000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe77ae14000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe779f5a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe779d56000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe779b4e000)

Glad to hear the support exists and will come into distros with time.  However, I still think it'd be a good idea for grub-probe to be more thorough about finding devices in /dev sub-dirs when it falls back to the zpool method.

Bart Kus <bkus>
Fri 21 Nov 2014 06:17:36 AM UTC, comment #1: 

grub will fall back to zpool only if libzfs/libnvpair are not found; otherwise it fetches vdev path information directly. You may need to install development packages for those libraries and rebuild grub. Please test using libzfs - does it work?

Andrei Borzenkov <arvidjaar>
Group Member
Fri 21 Nov 2014 01:26:54 AM UTC, original submission:  

When the root filesystem is a ZFS composed of VDEVs from /dev/mapper (eg: /dev/mapper/cryptroot), grub-probe fails to find the underlying device(s).  This is because it calls out to zpool to find the VDEVs of rpool, but they're listed as non-absolute names in zpool's output, and grub-probe then tries to find the full paths in /dev, but not any sub-dir of /dev, like /dev/mapper.

A workaround is to create symlinks in /dev using the exact same device names as listed in zpool's output.  grub-probe then succeeds and things like grub-mkconfig start working.

Please enhance grub-probe's support for ZFS by searching deeper in the /dev tree, or find an alternate more precise way of obtaining the VDEV information from ZFS.

Bart Kus <bkus>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #36473:  zpool_full_device_name.patch added by chadmiller (808B - text/x-patch - Signal to zpool to emit full vdev path names. Replaces earlier suggestion file #36357)
file #36357:  zpool-device-full-paths-envvar.patch added by chadmiller (303B - text/x-patch - Ask ZFS pool command for device names, hopefully with full paths. Use environment variable for maximum compatibility.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by delx
  • -email is unavailable- added by jelmd (Posted a comment)
  • -email is unavailable- added by seschwar
  • -email is unavailable- added by phcoder (Updated the item)
  • -email is unavailable- added by biosehnsucht (Posted a comment)
  • -email is unavailable- added by chadmiller (Posted a comment)
  • -email is unavailable- added by arvidjaar (Posted a comment)
  • -email is unavailable- added by bkus (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-09-08 delx Carbon-Copy- Added delx
    2016-10-11 phcoder Priority5 - Normal 6
    2016-09-18 seschwar Carbon-Copy- Added seschwar
    2016-03-02 phcoder Planned Release2.03+ 2.02
    2016-03-02 phcoder Planned ReleaseNone 2.03+
    2016-02-26 chadmiller Attached File- Added zpool_full_device_name.patch, #36473
    2016-02-16 chadmiller Attached File- Added zpool-device-full-paths-envvar.patch, #36357
    2014-11-21 arvidjaar StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code