bugPSPP - Bugs: bug #62267, Debian 1.5.4-2: Regression fails...

 
 

bug #62267: Debian 1.5.4-2: Regression fails for test 521, 522, 525, 528 on mips64el, mipsel, ia64

Submitter:  Friedrich Beckmann <beckmanf>
Submitted:  Fri 08 Apr 2022 09:49:19 AM UTC
   
 
Category:  Compilation/Portability Severity:  5 - Average
Status:  Fixed Assigned to:  None
Open/Closed:  Closed Release:  None
Effort:  0.00
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 11 Apr 2022 04:41:00 AM UTC, comment #4: 

I made a 1.5.4-3 release where I included your patch

https://git.savannah.gnu.org/cgit/pspp.git/commit/?id=435b06ea1e6cefd2d540552e161a84a9b2cbef42

and that made the regression work.

Friedrich Beckmann <beckmanf>
Group Member
Sun 10 Apr 2022 11:25:59 PM UTC, comment #3: 

Based on https://buildd.debian.org/status/package.php?p=pspp, I think that my bug fix worked, so I'm closing this as fixed.

Ben Pfaff <blp>
Group administrator
Sun 10 Apr 2022 03:52:00 PM UTC, comment #2: 

Good catch! Thanks, I pushed a fix. I'd appreciate it if you could verify the fix.

> I guess code like this: ... from optimize.inc will not work either, because the get_number_arg is cast to int and then the int result is checked with "is_valid" where the parameter is double again.


I think that this code will work OK because DATE.DMY is not marked "absorb_miss" and expr_optimize() will catch it here:


  if (n_sysmis && (op->flags & OPF_ABSORB_MISS) == 0)
    {
      /* Most operations produce SYSMIS given any SYSMIS
         argument. */


I guess that the checking in optimize.inc is redundant there.  Maybe that's an oversight, but it seems harmless.

Ben Pfaff <blp>
Group administrator
Sun 10 Apr 2022 10:16:22 AM UTC, comment #1: 

I think he reason for regression failure for some architectures is a bug in the handling of system-missing values in the expression processing.

The expression handling code is generated via generate.py.

The result for the "replace" function is


case OP_REPLACE_sssn:
  {
    struct substring arg_haystack = get_string_arg (node, 0);
    struct substring arg_needle = get_string_arg (node, 1);
    struct substring arg_replacement = get_string_arg (node, 2);
    int arg_n = get_number_arg (node, 3);
    return expr_allocate_string (e, eval_OP_REPLACE_sssn (arg_haystack, arg_needle, arg_replacement, arg_n, e));
  }


The result type of "get_number_arg" is double. The result of "get_number_arg" is cast to the integer variable "arg_n". However if the argument of get_number_arg is for example "1.5" as in the test szenario, then this is converted to system-missing here in the generated "evaluate.h" file:


static inline double
eval_OP_NUM_TO_INTEGER (double x, struct expression *e, const struct expr_node *n)
{
  if (x == floor (x) && x > INT_MIN && x <= INT_MAX)
    return x;

  msg_at (SE, expr_location (e, n),
          _("Treating unexpected non-integer value %g as missing."), x);
  return SYSMIS;
}


which results in the system-missing value which is defined as -DBL_MAX, something -10**308. A cast of a double type to integer that cannot be represented is undefined. So here this happens.

This handling of integer is done systematiclly. I guess code like this:


case OP_DATE_DMY_nnn:
  {
    int arg_d = get_number_arg (node, 0);
    int arg_m = get_number_arg (node, 1);
    int arg_y = get_number_arg (node, 2);
    bool force_sysmis = !is_valid (arg_d) || !is_valid (arg_m) || !is_valid (arg_y);
    double result = force_sysmis ? SYSMIS : eval_OP_DATE_DMY_nnn (arg_d, arg_m, arg_y, e, node);
    return expr_allocate_number (e, result);
  }


from optimize.inc will not work either, because the get_number_arg is cast to int and then the int result is checked with "is_valid" where the parameter is double again.


Friedrich Beckmann <beckmanf>
Group Member
Fri 08 Apr 2022 09:49:19 AM UTC, original submission:  

The regression fails for some architectures for the 1.5.4-2 debian release. The failed tests are:

mips64el: 521, 522, 525 and 528
mipsel: 521, 522, 525 and 528
ia64: 528
 
Here is the log for tests 521, 522, 525 and 528 for mips64el:

## ------------- ##
## Test results. ##
## ------------- ##

ERROR: 1510 tests were run,
4 failed unexpectedly.
2 tests were skipped.

## ------------------------ ##
## Summary of the failures. ##
## ------------------------ ##
Failed tests:
GNU PSPP 1.5.4 test suite test groups:

 NUM: FILE-NAME:LINE     TEST-GROUP-NAME
      KEYWORDS

 521: evaluate.at:3255   expressions - REPLACE
      expression expressions evaluate
 522: evaluate.at:3313   expressions - LPAD
      expression expressions evaluate
 525: evaluate.at:3645   expressions - RPAD
      expression expressions evaluate
 528: evaluate.at:3979   expressions - STRUNC
      expression expressions evaluate

## ---------------------- ##
## Detailed failed tests. ##
## ---------------------- ##

#                             -*- compilation -*-
521. evaluate.at:3255: testing expressions - REPLACE ...
## --- ##
## OPT ##
## --- ##
./evaluate.at:3278: pspp --testing-mode evaluate.sps
--- -        2022-04-08 07:41:12.783375731 +0000
+++ /<<PKGBUILDDIR>>/tests/testsuite.dir/at-groups/521/stdout        2022-04-08 07:41:12.776029600 +0000
@@ -17,7 +17,7 @@
    10 | DEBUG EVALUATE /replace('banana', 'an', 'xyzzy', 1.5).
       |                                                  ^~~

-replace('banana', 'an', 'xyzzy', 1.5) => "banana"
+replace('banana', 'an', 'xyzzy', 1.5) => "bxyzzyxyzzya"

 replace('banana', 'bananana', 'xyzzy') => "banana"

@@ -27,5 +27,5 @@

 replace('banana', 'ba', '', -1) => "banana"

-replace('banana', 'ba', '', $sysmis) => "banana"
+replace('banana', 'ba', '', $sysmis) => "nana"

521. evaluate.at:3255: 521. expressions - REPLACE (evaluate.at:3255): FAILED (evaluate.at:3278)

#                             -*- compilation -*-
522. evaluate.at:3313: testing expressions - LPAD ...
## --- ##
## OPT ##
## --- ##
./evaluate.at:3355: pspp --testing-mode evaluate.sps
--- -        2022-04-08 07:41:12.792446599 +0000
+++ /<<PKGBUILDDIR>>/tests/testsuite.dir/at-groups/522/stdout        2022-04-08 07:41:12.780029623 +0000
@@ -28,6 +28,16 @@

 lpad('abc', 32768) => "abc"

+evaluate.sps:11.17-11.36: error: DEBUG EVALUATE: The length argument to LPAD
+must be between 0 and 32767.
+   11 | DEBUG EVALUATE /lpad('abc', $sysmis).
+      |                 ^~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:11.29-11.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   11 | DEBUG EVALUATE /lpad('abc', $sysmis).
+      |                             ^~~~~~~
+
 lpad('abc', $sysmis) => "abc"

 evaluate.sps:13.17-13.36: error: DEBUG EVALUATE: The length argument to LPAD
@@ -70,10 +80,40 @@

 lpad('abc', 32768, '*') => "abc"

+evaluate.sps:26.17-26.41: error: DEBUG EVALUATE: The length argument to LPAD
+must be between 0 and 32767.
+   26 | DEBUG EVALUATE /lpad('abc', $sysmis, '*').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:26.29-26.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   26 | DEBUG EVALUATE /lpad('abc', $sysmis, '*').
+      |                             ^~~~~~~
+
 lpad('abc', $sysmis, '*') => "abc"

+evaluate.sps:27.17-27.40: error: DEBUG EVALUATE: The length argument to LPAD
+must be between 0 and 32767.
+   27 | DEBUG EVALUATE /lpad('abc', $sysmis, '').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:27.29-27.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   27 | DEBUG EVALUATE /lpad('abc', $sysmis, '').
+      |                             ^~~~~~~
+
 lpad('abc', $sysmis, '') => "abc"

+evaluate.sps:28.17-28.42: error: DEBUG EVALUATE: The length argument to LPAD
+must be between 0 and 32767.
+   28 | DEBUG EVALUATE /lpad('abc', $sysmis, 'xy').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:28.29-28.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   28 | DEBUG EVALUATE /lpad('abc', $sysmis, 'xy').
+      |                             ^~~~~~~
+
 lpad('abc', $sysmis, 'xy') => "abc"

 evaluate.sps:29.17-29.27: error: DEBUG EVALUATE: Function invocation
522. evaluate.at:3313: 522. expressions - LPAD (evaluate.at:3313): FAILED (evaluate.at:3355)

#                             -*- compilation -*-
525. evaluate.at:3645: testing expressions - RPAD ...
## --- ##
## OPT ##
## --- ##
./evaluate.at:3687: pspp --testing-mode evaluate.sps
--- -        2022-04-08 07:41:13.009342185 +0000
+++ /<<PKGBUILDDIR>>/tests/testsuite.dir/at-groups/525/stdout        2022-04-08 07:41:13.000030876 +0000
@@ -28,6 +28,16 @@

 rpad('abc', 32768) => "abc"

+evaluate.sps:11.17-11.36: error: DEBUG EVALUATE: The length argument to RPAD
+must be between 0 and 32767.
+   11 | DEBUG EVALUATE /rpad('abc', $sysmis).
+      |                 ^~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:11.29-11.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   11 | DEBUG EVALUATE /rpad('abc', $sysmis).
+      |                             ^~~~~~~
+
 rpad('abc', $sysmis) => "abc"

 evaluate.sps:13.17-13.36: error: DEBUG EVALUATE: The length argument to RPAD
@@ -70,10 +80,40 @@

 rpad('abc', 32768, '*') => "abc"

+evaluate.sps:26.17-26.41: error: DEBUG EVALUATE: The length argument to RPAD
+must be between 0 and 32767.
+   26 | DEBUG EVALUATE /rpad('abc', $sysmis, '*').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:26.29-26.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   26 | DEBUG EVALUATE /rpad('abc', $sysmis, '*').
+      |                             ^~~~~~~
+
 rpad('abc', $sysmis, '*') => "abc"

+evaluate.sps:27.17-27.40: error: DEBUG EVALUATE: The length argument to RPAD
+must be between 0 and 32767.
+   27 | DEBUG EVALUATE /rpad('abc', $sysmis, '').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:27.29-27.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   27 | DEBUG EVALUATE /rpad('abc', $sysmis, '').
+      |                             ^~~~~~~
+
 rpad('abc', $sysmis, '') => "abc"

+evaluate.sps:28.17-28.42: error: DEBUG EVALUATE: The length argument to RPAD
+must be between 0 and 32767.
+   28 | DEBUG EVALUATE /rpad('abc', $sysmis, 'xy').
+      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~
+
+evaluate.sps:28.29-28.35: note: DEBUG EVALUATE: The length argument is
+2147483647.
+   28 | DEBUG EVALUATE /rpad('abc', $sysmis, 'xy').
+      |                             ^~~~~~~
+
 rpad('abc', $sysmis, 'xy') => "abc"

 evaluate.sps:29.17-29.27: error: DEBUG EVALUATE: Function invocation
525. evaluate.at:3645: 525. expressions - RPAD (evaluate.at:3645): FAILED (evaluate.at:3687)

#                             -*- compilation -*-
528. evaluate.at:3979: testing expressions - STRUNC ...
## --- ##
## OPT ##
## --- ##
./evaluate.at:4016: pspp --testing-mode evaluate.sps
--- -        2022-04-08 07:41:13.385656684 +0000
+++ /<<PKGBUILDDIR>>/tests/testsuite.dir/at-groups/528/stdout        2022-04-08 07:41:13.364032950 +0000
@@ -21,11 +21,11 @@
    13 | DEBUG EVALUATE /strunc('a c   ', 0.75).
       |                                  ^~~~

-strunc('a c   ', 0.75) => "a c   "
+strunc('a c   ', 0.75) => "a c"

 strunc('a c   ', -1) => ""

-strunc('a c   ', $sysmis) => "a c   "
+strunc('a c   ', $sysmis) => "a c"

 strunc('  abc  ', 9) => "  abc"

@@ -50,9 +50,9 @@
    27 | DEBUG EVALUATE /strunc('  abc  ', 1.5).
       |                                   ^~~

-strunc('  abc  ', 1.5) => "  abc  "
+strunc('  abc  ', 1.5) => "  abc"

 strunc('  abc  ', -1) => ""

-strunc('  abc  ', $sysmis) => "  abc  "
+strunc('  abc  ', $sysmis) => "  abc"

528. evaluate.at:3979: 528. expressions - STRUNC (evaluate.at:3979): FAILED (evaluate.at:4016)


The ia64 log:


## ---------------------- ##
## Detailed failed tests. ##
## ---------------------- ##

#                             -*- compilation -*-
528. evaluate.at:3979: testing expressions - STRUNC ...
## --- ##
## OPT ##
## --- ##
./evaluate.at:4016: pspp --testing-mode evaluate.sps
--- -        2022-04-08 07:48:00.871055259 +0000
+++ /<<PKGBUILDDIR>>/tests/testsuite.dir/at-groups/528/stdout        2022-04-08 07:48:00.868143852 +0000
@@ -21,11 +21,11 @@
    13 | DEBUG EVALUATE /strunc('a c   ', 0.75).
       |                                  ^~~~

-strunc('a c   ', 0.75) => "a c   "
+strunc('a c   ', 0.75) => ""

 strunc('a c   ', -1) => ""

-strunc('a c   ', $sysmis) => "a c   "
+strunc('a c   ', $sysmis) => ""

 strunc('  abc  ', 9) => "  abc"

@@ -50,9 +50,9 @@
    27 | DEBUG EVALUATE /strunc('  abc  ', 1.5).
       |                                   ^~~

-strunc('  abc  ', 1.5) => "  abc  "
+strunc('  abc  ', 1.5) => ""

 strunc('  abc  ', -1) => ""

-strunc('  abc  ', $sysmis) => "  abc  "
+strunc('  abc  ', $sysmis) => ""

528. evaluate.at:3979: 528. expressions - STRUNC (evaluate.at:3979): FAILED (evaluate.at:4016)





Friedrich Beckmann <beckmanf>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by blp (Posted a comment)
  • -email is unavailable- added by beckmanf (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-04-10 blp StatusConfirmed Fixed
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-b921.
    Corresponding source code