bugGNU a2ps - Bugs: bug #66853, a2ps SIGSEGVs on recent Solaris...

 
 

bug #66853: a2ps SIGSEGVs on recent Solaris 11.4

Submitter:  Martin Rehak <mrehak>
Submitted:  Wed 26 Feb 2025 03:34:44 PM UTC
   
 
Category:  None Severity:  3 - Normal
Item Group:  None Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 26 Feb 2025 03:34:44 PM UTC, original submission:  

Hello,

a2ps-4.15.6 generate SIGSEGVs on Oracle Solaris.

I have found that configure script decides the Solaris malloc() is not POSIX compliant after running a strange looking test. Because of that a2ps wraps some part of memory allocator by some macros which all makes some memory blocks to be allocated using system malloc()/calloc(), but passed to GC_realloc(). That generates SIGSEGV.

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#define ptrdiff_t long
#ifndef PTRDIFF_MAX
# define PTRDIFF_MAX ((ptrdiff_t) ((1UL << (8 * sizeof (ptrdiff_t) - 1)) - 1))
#endif

int main ()
{
  void *p;

  fprintf (stderr, "PTRDIFF_MAX = %lu\n", (unsigned long) PTRDIFF_MAX);

  errno = 0;
  p = malloc ((unsigned long) PTRDIFF_MAX + 1);
  fprintf (stderr, "p=%p errno=%d\n", p, errno);

  errno = 0;
  p = calloc (PTRDIFF_MAX / 2 + 1, 2);
  fprintf (stderr, "p=%p errno=%d\n", p, errno);

  errno = 0;
  p = realloc (NULL, (unsigned long) PTRDIFF_MAX + 1);
  fprintf (stderr, "p=%p errno=%d\n", p, errno);

  return 0;
}
$ gcc a.c
a.c: In function ‘main’:
a.c:17:11: warning: argument 1 value ‘9223372036854775808’ exceeds maximum object size 9223372036854775807 [
-Walloc-size-larger-than=]
   17 |       p = malloc ((unsigned long) PTRDIFF_MAX + 1);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/stdlib.h:12,
                 from a.c:4:
/usr/include/iso/stdlib_iso.h:179:14: note: in a call to allocation function ‘malloc’ declared here
  179 | extern void *malloc(size_t);
      |              ^~~~~~
a.c:21:11: warning: product ‘4611686018427387904 * 2’ of arguments 1 and 2 exceeds maximum object size 92233
72036854775807 [-Walloc-size-larger-than=]
   21 |       p = calloc (PTRDIFF_MAX / 2 + 1, 2);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/iso/stdlib_iso.h:172:14: note: in a call to allocation function ‘calloc’ declared here
  172 | extern void *calloc(size_t, size_t);
      |              ^~~~~~
a.c:25:11: warning: argument 1 value ‘9223372036854775808’ exceeds maximum object size 9223372036854775807 [
-Walloc-size-larger-than=]
   25 |       p = realloc (NULL, (unsigned long) PTRDIFF_MAX + 1);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.c:25:11: note: in a call to built-in allocation function ‘__builtin_malloc’
$ ./a.out
PTRDIFF_MAX = 9223372036854775807
p=0 errno=12
p=0 errno=12
p=0 errno=12
mrehak@virtuals:~$ uname -a
SunOS virtuals 5.11 11.4.76.182.1 sun4v sparc sun4v non-virtualized

Why the test for calloc limit is passing pointer difference limit as an argument? The limit is defined as SIZE_MAX in C99 standard. What the test is actually testing, please?

I even doubt that the tests proved something in the past as Solaris was ever strictly POSIX compliant against POSIX.1-1988.

Also the comment says

On Solaris 11.3, the three functions return NULL with errno set to EAGAIN, not ENOMEM, when the argument is larger than PTRDIFF_MAX.

But this is the output from the test on Solaris 11.3:

# ./a.out
PTRDIFF_MAX = 2147483647
p=211d0 errno=0
p=0 errno=12
p=0 errno=12

Following patch stops running the test and let assume the allocator is POSIX compliant. Since then a2ps works like a charm.

--- a/m4/malloc.m4
+++ b/m4/malloc.m4
@@ -130,12 +130,9 @@
       case "$host_os" in
         mingw*)
           gl_cv_func_malloc_posix=no ;;
-        irix* | solaris*)
+        irix*)
           dnl On IRIX 6.5, the three functions return NULL with errno unset
           dnl when the argument is larger than PTRDIFF_MAX.
-          dnl On Solaris 11.3, the three functions return NULL with errno set
-          dnl to EAGAIN, not ENOMEM, when the argument is larger than
-          dnl PTRDIFF_MAX.
           dnl Here is a test program:
 m4_divert_push([KILL])
 #include <errno.h>

Please consider updating the test. Let me know if you need to test something on Solaris.

Thank you,
m.

Martin Rehak <mrehak>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mrehak (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.14-962f.
    Corresponding source code