mainAutoconf - Support: sr #111238, downstream package build failure...

 
 

sr #111238: downstream package build failure with autoreconf+gettext-0.24.1

Submitter:  Funda Wang <fundawang>
Submitted:  Thu 08 May 2025 05:01:51 AM UTC
   
 
Priority:  * 5 - Unprioritized Severity:  3 - Normal
Status:  Confirmed Privacy:  Public
Assigned to:  None Open/Closed:  Open
Operating System:  None
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Jump to the original submission

Wed 11 Jun 2025 07:44:10 PM UTC, comment #8: 


comment #7:

> I apologize for saying "the distros". I meant to state what I observed several important distros are doing. If you are working on a distro that generally respects upstream tarballs, the "tension" that I was describing does not exist between package maintainers and your distro.
>>
> > > 2) ... They run autoreconf anyway.
> > We don't care if the "correct" command is autoreconf or autogen.sh
>
> Glad to hear that your distro is so pragmatic. I was relating my experience with at least one major distro. Sorry for the over-generalization.



No problem. :)


> > I suppose that Zack has the option of pushing back against false information...
>
> I still claim that this tension exists, between specific distros and upstream.
>
> Specifically Debian: https://wiki.debi ... an.org/Autoreconf



That page is a bit confusing as it gives multiple justifications.

In Gentoo's experience, updating config.guess and config.sub tends to be reliable for the vast majority of cases -- we handle this separately from anything else. We also have an "eltpatch" program that applies backported patches to ltmain.sh for notable issues, so that we can continue using the existing build system while getting critical fixes from later versions.

They also note the diff patches issue which... Okay? For packages where one is applying patches to the configure.ac it's only reasonable to also re-run autoreconf.

And then they note the ideological issue of "ensure that it's possible to modify the actual source". Debian has a lot of opinions here. It fits quite well with the opinion of some that if a project has included vendored dependencies along with a configure --without-bundled-foobar, it's a Debian social contract violation if one doesn't delete the bundled code copy before creating a Debian source package. i.e. it's not acceptable to use the upstream release tarball even though the project is very respectful and supports both approaches, and it isn't even non-free software. They refuse to ship a .prog.tar.xz that contains the bundles code copies.

Anyway, this policy predates Jia Tan, and Debian doesn't seem to require that you clean out the existing sources. That's how the malicious m4 file with a later serial number survived, after all. Probably it is because they can't enforce this, you don't know which .m4 files were automatically copied in by aclocal --install and which were handwritten... Totally impractical to enforce this.

(Arch Linux is gunning for a very different policy, to add git as a build dependency for all the world's software and do a git clone from the canonical upstream repository instead of using release tarballs. That means you can't cache them offline and you can't build software if the upstream website disappears.)


> Specifically openSUSE: https://lists.ope ... FINYCNRNPI3BRD6G/



This says exactly the opposite. Someone thought the policy was to run autoreconf, didn't like it, and asked if they were really forced to do so. And got told "no, in fact please do not, we would prefer the upstream generated configure script if it is in good working order".

In fact, insisting to run autoreconf even prevented backporting the package to an older SLES, which had an older autotools that was buggy and failed to build, whereas the upstream generated configure script was generated on a new system that had those fixes.

...

The recent Patch release that I persuaded Andreas et al. to put together, was because over the years we collected so many patches to Patch that we had to rerun autoreconf to apply changes to configure.ac and Makefile.am; this is a big problem for us because it broke our ability to bootstrap Gentoo Prefix, where we don't yet have the Autotools and aren't cross compiling from a CBUILD that does. And we need Patch in order to build them.

Even if it wasn't a matter of badly motivated FOSS politics, for purely practical reasons any distro that cares about bootstrapping simply cannot put up with nonsense like "we don't trust the upstream maintainer generated configure scripts". It's a pointless frippery that sets back progress.

If one is concerned about the next Jia Tan, auditing source tarballs by verifying that they can be reproducibly recreated (documenting the autoconf/automake/libtool/gettext etc versions used) is the sensible and practical solution to this.

If one is concerned about fixing bugs via newer autoconf/automake/libtool/gettext, it is simple to apply a two line update (in Gentoo this is "inherit the `autotools` build style") per package known to have issues and test that it works before releasing.

Eli Schwartz <eschwartz>
Wed 11 Jun 2025 06:00:40 PM UTC, comment #7: 

Re comment #6:

> > 1) ... But then the distros say "we don't trust upstream tarballs any more, because of the xz backdoor drama", and regenerate everything with their own versions of autoconf, automake, ...
>
> Please do not tar and feather us all with the same brush! This is unreasonable and insulting.


I apologize for saying "the distros". I meant to state what I observed several important distros are doing. If you are working on a distro that generally respects upstream tarballs, the "tension" that I was describing does not exist between package maintainers and your distro.

> > 2) ... They run autoreconf anyway.
> We don't care if the "correct" command is autoreconf or autogen.sh


Glad to hear that your distro is so pragmatic. I was relating my experience with at least one major distro. Sorry for the over-generalization.

> I've submitted patches to autoreconf to solve the case where commonly used tools such as the GLib ecosystem ones aren't invoked by autoreconf, and an autogen.sh was the only thing that worked. Now autoreconf works too!


Your patches regarding gtkdocize and intltoolize surely made the job of distributors easier.

> I suppose that Zack has the option of pushing back against false information...


I still claim that this tension exists, between specific distros and upstream.

Specifically Debian: https://wiki.debi ... an.org/Autoreconf
Specifically openSUSE: https://lists.ope ... FINYCNRNPI3BRD6G/

Bruno Haible <haible>
Wed 11 Jun 2025 04:32:20 PM UTC, comment #6: 

comment #5:

> This is a symptom of the tension between package maintainers and distros:
>
> 1) When package maintainers do a release, they test a specific tarball, with generated files from a specific autoconf version, a specific automake version, a specific GNU bison version, a specific GNU gperf version, a specific makeinfo version, a specific groff version, and so on. But then the distros say "we don't trust upstream tarballs any more, because of the xz backdoor drama", and regenerate everything with their own versions of autoconf, automake, bison, gperf, makeinfo, groff, etc.



Please do not tar and feather us all with the same brush! This is unreasonable and insulting.


> 2) Distros typically ignore it when the upstream maintainer says "my package does not support autoreconf". They run autoreconf anyway.



This one I don't specially recall seeing at all. Distros aren't in the business of intentionally failing to build a package. We don't care if the "correct" command is autoreconf or autogen.sh, although the latter has a nasty habit of hardcoding specific micro releases of the various tools -- versions we may or may not package, and would rather backport an upstream patch to support newer versions rather than add a deprecated package installing autoconf-2.58 or whatever. So, like everyone else, we appreciate when autoreconf works seamlessly.

I've submitted patches to autoreconf to solve the case where commonly used tools such as the GLib ecosystem ones aren't invoked by autoreconf, and an autogen.sh was the only thing that worked. Now autoreconf works too! But I don't call that "ignoring what the upstream maintainer says".


> You (or we, in the GNU Build System) cannot solve this tension between package maintainers and distros.



I suppose that Zack has the option of pushing back against false information...

Eli Schwartz <eschwartz>
Tue 03 Jun 2025 04:46:43 PM UTC, comment #5: 

Zack:

> the intended purpose of autoreconf is to be a complete substitute for project-specific autogen.sh scripts.


The documentation https://www.gnu.o ... f-Invocation.html disagrees: It implies that autoreconf is a complete substitute for a project-specific autogen.sh only if that autogen.sh script makes use only of autoconf, autoheader, aclocal, automake, libtoolize, intltoolize, gtkdocize, autopoint and nothing else. Nothing else: No gnulib-tool, no libtool.m4 patches, no 'git clone' of other repositories, no 'git submodule init'. (I'm not arguing for 'autoreconf' to do 'git clone' or 'git submodule init': It has become clear that fetching sources through the network and generating files are two tasks that are best kept separate; this is where the distinction between 'autopull.sh' and 'autogen.sh' comes from.)

> If I understand the discussion at https://savannah. ... u.org/bugs/?67090 and https://savannah. ... s/index.php?66968 correctly, a change in gettext broke building certain older packages from source.


To keep a long story short, I'll focus on two points:

1) In the beginning (before 2002), there was only one tool for adding gettext's build infrastructure to a package: gettextize. Then it became clear that the manual invocation (wizard-like tool) and the automated invocation (part of what autogen.sh does) are two different situations, and 'autopoint' was created for the latter. One of the main differences is that 'gettextize' installs the infrastructure for the current gettext version, whereas 'autopoint' installs the infrastructure for the gettext version that has been agreed upon among the package developer team.

Both 'gettextize' and 'autopoint' make sure that what they install is consistent, that is, that po.m4 and po/Makefile.in.in come from matching versions. This constraint is important, because without it, po.m4 could not evolve and po/Makefile.in.in could not evolve either.

2) The gettext .m4 files were installed into $prefix/share/aclocal/ without the intention to make them available for 'aclocal' or 'autoreconf'. It was simply because other packages install their .m4 files there as well.

When a user has two different ways to do the same thing, it is tedious to keep the two ways in sync, that is, do the equivalent things. And this is what happened: For some packages, 'autoreconf --install' invoked just aclocal, not autopoint. For many years, this behavioural difference 'autoreconf --install' vs. 'autopoint' hardly mattered, because po.m4 hardly changed. But with gettext 0.24, there were significant changes in po.m4 and po/Makefile.in.in, and while 'autopoint' handled the situation well, 'autoreconf --install' did not.

Therefore the fix that I did in 0.24.1 was to make it impossible for 'autoreconf --install' to produce the broken situations that were reported (in https://savannah. ... u.org/bugs/?66968 , https://savannah. ... u.org/bugs/?67090 , https://lists.gnu ... -03/msg00040.html ). This change should have been done at the moment 'autopoint' was invented, but I did not realize it at that time (in 2002).

> I'd appreciate it if you could suggest any approach to threading a path through all these semi-contradictory requirements.


Here are some thoughts about it.

1) The GNU Build System should be more consequent in offering to the user one and only one way to do a specific thing. I discussed 'autopoint' above.

The same problem exists for 'libtoolize': It has its .m4 macros stored in $prefix/share/aclocal/ as well (see https://packages. ... /libtool/filelist ). If someone runs 'autoreconf --install' and autoreconf guesses wrong about the need to invoke libtoolize, the user will get a libtool.m4 file that does not match the version of their ltmain.sh.

Another example (unfortunately) is that the gettext manual documents the AM_ICONV macro. This should be in Gnulib only, but some old packages use 'autopoint' to get AM_ICONV.

2) autoreconf should have a more solid recognition which tools it should invoke.

There is the case of packages with AM_GNU_GETTEXT invocations without AM_GNU_GETTEXT_VERSION.

There is also the case when no .m4 files are present in the package, and the package invokes some PKG_CHECK_MODULES macro. I've seen autoreconf fail in such a situation, IIRC, because autoreconf runs autom4te and autom4te fails.

3) autopoint should handle all cases. Currently in the case of packages with AM_GNU_GETTEXT invocations without AM_GNU_GETTEXT_VERSION, it bails out.

4) autoreconf should invoke autopoint in such cases as well.

> perhaps it is not doing it correctly.  What it currently does is grep configure.ac for occurrences of AM_GNU_GETTEXT_(REQUIRE_)?VERSION.


There is room for improvement, indeed: https://savannah. ... u.org/bugs/?52424

> there exist projects that "intentionally don't call AM_GNU_GETTEXT_(REQUIRE_)VERSION because they have all of the gettext infrastructure checked into version control and they want [autoreconf] to not run autopoint"; any change will need to not break such projects.


This is a symptom of the tension between package maintainers and distros:

1) When package maintainers do a release, they test a specific tarball, with generated files from a specific autoconf version, a specific automake version, a specific GNU bison version, a specific GNU gperf version, a specific makeinfo version, a specific groff version, and so on. But then the distros say "we don't trust upstream tarballs any more, because of the xz backdoor drama", and regenerate everything with their own versions of autoconf, automake, bison, gperf, makeinfo, groff, etc.

2) Distros typically ignore it when the upstream maintainer says "my package does not support autoreconf". They run autoreconf anyway.

You (or we, in the GNU Build System) cannot solve this tension between package maintainers and distros.

Bruno Haible <haible>
Tue 03 Jun 2025 01:58:24 PM UTC, comment #4: 

Bruno: No, I'm not going to close this as invalid.  The specific remedy requested may be incorrect, but:

1) If I understand the discussion at https://savannah. ... u.org/bugs/?67090 and https://savannah. ... s/index.php?66968 correctly, a change in gettext broke building certain older packages from source.  On its face, that is a legitimate bug.  A request for restored backward compatibility should always be taken seriously, even if it appears that part of the problem is user error.  Of course it may not be possible to accomplish restored backward compatibility without causing other problems, but that's no reason to be dismissive.

2) From autoconf's perspective, `autoreconf -i [-f]` is supposed to be sufficient to prepare one to run ./configure, except for very unusual cases, such as trying to build Autoconf itself from a VCS checkout without any pre-installed copy of Autoconf available.  To put it another way, the intended purpose of autoreconf is to be a complete substitute for project-specific autogen.sh scripts.

Thus, to the extent "don't use autoreconf" is correct advice for the problem Funda Wang was experiencing, that is a bug in autoreconf, and I see no reason not to treat this as a report of that bug.

autoreconf does know how to run autopoint, but perhaps it is not doing it correctly.  What it currently does is grep configure.ac for occurrences of AM_GNU_GETTEXT_(REQUIRE_)?VERSION.  If any are found, it runs autopoint, as its very first operation (in particular, before aclocal).

In https://savannah. ... s/index.php?66968 the recommended sequence seems instead to be to run aclocal with --install first, then run autopoint, then run aclocal again without --install.  The problem with that is, per the commentary in bin/autoconf.in, if you run aclocal on a source tree with no pre-existing po.m4, without having run autopoint first, aclocal will fail because it can't find the gettext macros.

Complicating matters still further, there exist projects that "intentionally don't call AM_GNU_GETTEXT_(REQUIRE_)VERSION because they have all of the gettext infrastructure checked into version control and they want [autoreconf] to not run autopoint"; any change will need to not break such projects.

I'd appreciate it if you could suggest any approach to threading a path through all these semi-contradictory requirements.

Zack Weinberg <zackw>
Group administrator
Sun 18 May 2025 12:50:23 PM UTC, comment #3: 

Funda Wang,

I gave an answer to your question in https://savannah. ... u.org/bugs/?67090 on May 7, 2025. The next day, you come here to the Autoconf tracker and suggest to the Autoconf people to make a change that makes my bug fix in GNU gettext ineffective. And you do so without CCing me. (There is a CC field at the bottome of every savannah ticket.)

This behaviour is anti-social. NEVER DO THIS AGAIN!!!

Bruno Haible <haible>
Sun 18 May 2025 12:45:16 PM UTC, comment #2: 

This issue has been discussed in https://savannah. ... u.org/bugs/?67090 and closed. The resolution was:

The problem is the use of autoreconf.

> May autoreconf script be tweaked, so that "-I/usr/share/gettext/m4/" be used when involking autoconf?


This is the worst possible suggestion. The gettext .m4 files were moved to a different directory, so that autoreconf would not mistakenly install them. To install the gettext .m4 files, 'autopoint' needs to be used. If necessary, a line with AM_GNU_GETTEXT_VERSION needs to be added to configure.ac first.

Please close this ticket as INVALID.

Bruno Haible <haible>
Thu 08 May 2025 05:22:09 AM UTC, comment #1: 
Funda Wang <fundawang>
Thu 08 May 2025 05:01:51 AM UTC, original submission:  

Since gettext 0.24.1, *.m4 was installed into /usr/share/gettext/m4/, instead of /usr/share/aclocal. Such a change lead to build failure of some old downstream packages, such as libbonobo-2.0, GConf-2.0, when using autoreconf to regenerate configure script and Makfiles.

See: https://savannah. ... u.org/bugs/?67090

May autoreconf script be tweaked, so that "-I/usr/share/gettext/m4/" be used when involking autoconf?

Funda Wang <fundawang>

 

Attached Files

This item currently has no attached files.

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by eschwartz (Posted a comment)
  • -email is unavailable- added by zackw (Posted a comment)
  • -email is unavailable- added by haible (Posted a comment)
  • -email is unavailable- added by fundawang (Submitted the item)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    History

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2025-06-03 zackw StatusNone Confirmed

    Back to the top

    Powered by Savane 3.16-ed84.
    Corresponding source code