mainThe GNU Bourne-Again SHell - Support: sr #110924, bash 5.2: no way to test if...

 
 

sr #110924: bash 5.2: no way to test if associative array is set

Submitter:  None
Submitted:  Fri 18 Aug 2023 04:00:51 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Works For Me
Privacy:  Public Assigned to:  None
Originator Email:  -email is unavailable- Open/Closed:  Open
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 26 Aug 2023 03:33:52 PM UTC, comment #6: 


> An indexed array can have set elements other than 0.


Ohhh.  I've never used that feature.  When tempted, one should just switch to associative arrays.  (Hmmm .... lua's unification of indexed and associative arrays now seems like a good design.)

It is very surprising that [[ -v varname ]] is equivalent to [[ -v varname[0] ]] even for associative arrays, where "0" is not an obviously special key.  Understandable given the history, where associative arrays were added much later than indexed arrays.

(Thanks for bash.  We keep using it at industrial scale (with set -eu -o pipefail), despite its many warts.)

Anonymous
Sat 26 Aug 2023 02:52:44 PM UTC, comment #5: 

Here is a slightly more elegant way to test if an associative array is set, with more awareness of the definition of "is set":


(
  is-associative-array-set() {
    local -nr ref=$1
    shopt -oq nounset; local -ir u=$?
    (( u )) || set +u
    local -ir n="${#ref[@]}"
    (( u )) || set -u
    (( n > 0 ))
  }

  test-is-associative-array-set() {
    declare -A aa=([k]=v)
    set +eu
    if is-associative-array-set aa; then echo set 1; fi
    set -eu
    if is-associative-array-set aa; then echo set 2; fi

    unset aa
    set +eu
    if is-associative-array-set aa; then echo set 3; fi
    set -eu
    if is-associative-array-set aa; then echo set 4; fi
  }
  test-is-associative-array-set
)


Anonymous
Fri 25 Aug 2023 04:06:04 PM UTC, comment #4: 


>> If you use test -v with an array name supplied without a subscript, you'll
> test element 0
>
> Ohh!  That was non-obvious!  I now see that the docs state:
>
>
> Referencing an array variable without a subscript is equivalent to referencing
> the array with a subscript of 0
>
>
> but there remains ambiguity about when an array as a whole or its "0" element
> is under consideration.  The docs for [[ -v varname ]] should make this
> clearer.

It's the normal case. The documentation should call out exceptions to the
without-subscript rule, and this isn't one of them.

(If you can find exceptions that should be called out and are not, please
let me know.)


> Perhaps also document the special treatment of varname[@]:
>
>
> When arrayname is the name of an indexed array, arrayname[@] is treated the
> same as arrayname[0], i.e. true if arrayname has any elements.

That is not true. An indexed array can have set elements other than 0. It's
reasonable to call out the exception and the difference in behavior between
indexed and associative arrays, though.


> I see in the docs:
>
>
> An array variable is considered set if a subscript has been assigned a value.
>
>
> but I don't think that is quite true.  Here's a counterexample that shows that
> an array with no subscripts is considered set (at least treated differently
> from an array variable not declared at all):
>
>
> bash-5.3$ ( set -eu; declare -A aa=(); echo ${#aa[@]}; )
> 0
> bash-5.3$ ( set -eu; declare -A aa=(); echo ${#xx[@]}; )
> bash: xx: unbound variable

That's an exception that parallels the treatment of $@ and $* when there
are no positional parameters and -u is set:

"When the shell tries to expand an unset parameter other than the '@' and '*' special parameters, it shall write..."

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_25

It means that $@ and $* are always `set' for the purposes of `set -u'.

So we support that by saying that an array variable that exists but is
unset (like a variable declared with `export xx' exists but is unset) is
subject to the same rules as $* and $@ when `set -u' is enabled. This
came in in bash-4.4 after user requests.

Chet Ramey <chet>
Group administrator
Thu 24 Aug 2023 11:09:56 PM UTC, comment #3: 

Still on the hunt for implementing is-associative-array-variable-name without creating a subshell or doing an O(N) copy.  We can locally disable set -u thus:


(
  is-associative-array-variable-name() {
    local -n ref=$1
    if [[ "$-" =~ u ]]; then local -ir u=1; else local -ir u=0; fi
    (( u == 0 )) || set +u
    local -ri n="${#ref[@]}"
    (( u == 0 )) || set -u
    (( n > 0 ))
  }
  test-is-associative-array-variable-name() {
    unset aa
    if is-associative-array-variable-name aa; then echo set 1; fi
    set -eu
    if is-associative-array-variable-name aa; then echo set 2; fi

    set +eu
    declare -A aa=([k]=v)
    if is-associative-array-variable-name aa; then echo set 3; fi
    set -eu
    if is-associative-array-variable-name aa; then echo set 4; fi
  }
  test-is-associative-array-variable-name
)


Anonymous
Thu 24 Aug 2023 10:22:36 PM UTC, comment #2: 


> If you use test -v with an array name supplied without a subscript, you'll test element 0


Ohh!  That was non-obvious!  I now see that the docs state:


Referencing an array variable without a subscript is equivalent to referencing the array with a subscript of 0


but there remains ambiguity about when an array as a whole or its "0" element is under consideration.  The docs for [[ -v varname ]] should make this clearer.


-v varname
  True if the shell variable varname is set (has been  assigned  a value).


Perhaps simply add to the doc above:


When varname is an array name supplied without a subscript, true if varname[0] is set.


Perhaps also document the special treatment of varname[@]:


When arrayname is the name of an indexed array, arrayname[@] is treated the same as arrayname[0], i.e. true if arrayname has any elements.


I see in the docs:


An array variable is considered set if a subscript has been assigned a value.


but I don't think that is quite true.  Here's a counterexample that shows that an array with no subscripts is considered set (at least treated differently from an array variable not declared at all):


bash-5.3$ ( set -eu; declare -A aa=(); echo ${#aa[@]}; )
0
bash-5.3$ ( set -eu; declare -A aa=(); echo ${#xx[@]}; )
bash: xx: unbound variable



Anonymous
Thu 24 Aug 2023 03:29:31 PM UTC, comment #1: 

First, it's clearly possible to test whether or not an array has any set elements in bash-5.2+. The difficulty depends, as always, on how much trouble you want to make for yourself with set -eu.

If you insist on using set -e, you have to wrap the test in a command for which set -e is not active. If you have to use set -u, you want a subshell whose exit status you can test.

If you don't want to use set -eu, it's as simple as

(( ${#var[@]} > 0 ))

If you want to use set -eu, it gets a little more complicated, but still not difficult:

varset=0
( (( ${#var[@]} > 0 )) ) && varset=1

The reason for the change in bash-5.2 is that there was no way to test whether an an associative array has a key named `@' in previous versions, while there are other ways to test whether it has any set elements. Incompatible change or no, it's something that should be possible.

The old behavior is available if you set the compatibility mode to 51.

There are some problems with your examples. If you use test -v with an array name supplied without a subscript, you'll test element 0 (or "0"). That's why the first and third examples differ. I didn't change the behavior of indexed arrays with `@' as a key because that's not a valid arithmetic expression anyway, and there's no reason to use it for any other purpose.

Chet Ramey <chet>
Group administrator
Fri 18 Aug 2023 04:00:51 PM UTC, original submission:  

The construct [[ -v a ]] should work whether a is an associative array or an indexed array.  Currently it only works for indexed arrays.

In bash 5.2+, there is no obvious way to test if an associative array is set (that works in the presence of set -eu)

bash git devel branch built from source:


./bash --norc
bash-5.3$ ( declare -A aa=([k]=v); if [[ -v aa ]]; then echo set; fi; )
bash-5.3$ ( declare -A aa=([k]=v); if [[ -v aa[@] ]]; then echo set; fi; )
bash-5.3$ ( declare -a ia=(e); if [[ -v ia ]]; then echo set; fi; )
set
bash-5.3$ ( declare -a ia=(e); if [[ -v ia[@] ]]; then echo set; fi; )
set


bash 5.1 allows the traditional idiom [[ -v aa[@] ]] to work:


/bin/bash --norc
bash-5.1$ ( declare -A aa=([k]=v); if [[ -v aa ]]; then echo set; fi; )
bash-5.1$ ( declare -A aa=([k]=v); if [[ -v aa[@] ]]; then echo set; fi; )
set
bash-5.1$ ( declare -a ia=(e); if [[ -v ia ]]; then echo set; fi; )
set
bash-5.1$ ( declare -a ia=(e); if [[ -v ia[@] ]]; then echo set; fi; )
set


I couldn't find any good portable way to detect whether an associative array is set.
The best I could do is the ugly and slow:


# bash 5.2 broke the construct [[ -v aa[@] ]]
# - [compat51](
#   https://www.gnu.org/software/bash/manual/bash.html#Shell-Compatibility-Mode)
# - https://unix.stackexchange.com/a/742451
is-associative-array-variable-name() {
  local -n ref=$1
  local -ar keys=( "${!ref[@]}" )
  [[ -v keys[@] ]]
}


(I don't think changing the meaning of aa[@] in bash 5.2 was a good idea.  bash is too old for such incompatible changes)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by chet (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-08-24 chet StatusNone Works For Me

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code