mainThe GNU Bourne-Again SHell - Support: sr #110964, BASH 5.1.16(1)-release on WSL2...

 
 

sr #110964: BASH 5.1.16(1)-release on WSL2 weird behavior in string splitting

Submitter:  None
Submitted:  Thu 30 Nov 2023 11:42:43 AM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  Microsoft Windows
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 07 Dec 2023 10:13:50 AM UTC, comment #2: 

Okay my bad wasn't expecting this kind of behavior, I guess I could turn off pathname generation temporarily with set -o noglob.. Thanks for the answer!

Anonymous
Mon 04 Dec 2023 04:52:52 PM UTC, comment #1: 

Do you happen to have a file or directory named `backend-components' in the directory where you're running this script?

The list of words following `in' in the for command is expanded, and those expansions include pathname generation, which happens after word splitting.

So the word splitting produces a set of individual patterns, each with the leading and trailing `*', and each undergoes pathname expansion. If you have a file or directory named `backend-components', `*backend-components*' will match it and be replaced with `backend-components'. A non-matching pattern is left unchanged.

Chet Ramey <chet>
Group administrator
Thu 30 Nov 2023 11:42:43 AM UTC, original submission:  

I came across something which I can't explain:

Consider this bash script:

set -x
RA_DO_NOT_PACKAGE_FILEGLOBS="*ci-components*|*external-files*|*backend-components*|*another-component*|*cli-component*|*random-word*"
defIFS=$IFS
IFS='|'
for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
do
exclude_string+=(" --exclude ${pattern}")
echo Excluding "${pattern}"
done
IFS=$defIFS
echo Excluding in total: "${exclude_string[@]}"

Running this in this kind of environment ( output from bashbug ):
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wall
uname output: Linux kalampakas 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.1
Patch Level: 16
Release Status: release

produces the following:
Syncing components to minio
+ RA_DO_NOT_PACKAGE_FILEGLOBS='*ci-components*|*external-files*|*dickend-components*|*backend-components*|*cli-component*|*random-word*'
+ IFS='|'
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding '*ci-components*'
Excluding ci-components
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding '*external-files*'
Excluding external-files
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding '*dickend-components*'
Excluding dickend-components
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding backend-components
Excluding backend-components
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding '*cli-component*'
Excluding cli-component
+ for pattern in $RA_DO_NOT_PACKAGE_FILEGLOBS
+ exclude_string+=("${pattern}")
+ echo Excluding '*random-word*'
Excluding random-word
+ unset IFS
+ echo Excluding in total: '*ci-components*' '*external-files*' '*dickend-components*' backend-components '*cli-component*' '*random-word*'
Excluding in total: ci-components *external-files* dickend-components backend-components cli-component *random-word*
+ mc mirror --insecure --overwrite --retry --remove --dry-run '*ci-components*' '*external-files*' '*dickend-components*' backend-components '*cli-component*' '*random-word*' temp 'minio/export-debug/CI-package/New Demo'

If I replace the `*backend-components*` string with something with the same number of characters it works as expected ( i.e. backend-components is quoted correctly and the wildcard character preserved. Replace all the non-wildcard characters with the character 'x', change the size/word to `*backend-component*` works as well. It is like the string `*backend-components*` is a special case somehow. Adding another string like this has the same result.

But when run on a Ubuntu VM with bash 5.0.17(1)-release (x86_64-pc-linux-gnu) it works normally ( i.e. the `*backend-components*` is quoted and the wildcards preserved ). From bashbug for the VM working example:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -fdebug-prefix-map=/build/bash-Smvct5/bash-5.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wno-parentheses -Wno-format-security
uname output: Linux ubuntuvm 5.4.0-165-generic #182-Ubuntu SMP Mon Oct 2 19:43:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.0
Patch Level: 17
Release Status: release

Same for the debian bookworm installation :
bash 5.2.15(1)-release (x86_64-pc-linux-gnu) debian bookworm works
linux debian 6.1.0-13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64 GNU/Linux

I guess this must be some WSL2 quirk but I can't for the life me find anything related in WSL2 Github.

Shopt options are identical in the working/non-working examples.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-12-07 chet StatusNeed Info Done
    2023-12-04 chet StatusNone Need Info

    Back to the top

    Powered by Savane 3.14-f13d.
    Corresponding source code