GNU gettext - Bugs: bug #59658, xgettext and msginit output isn't...
You are not allowed to post comments on this tracker with your current authentication level.
bug #59658: xgettext and msginit output isn't reproducible
Submitter: | Miguel Ángel Arruga Vivas <m1gu3l> | ||
Submitted: | Fri 11 Dec 2020 01:19:33 PM UTC | ||
Votes: | 1 | ||
Category: | Programmer tools | Severity: | 3 - Normal |
Item Group: | None | Status: | In Progress |
Privacy: | Public | Assigned to: | m1gu3l |
Open/Closed: | Open |
( Jump to the original submission )
Sat 07 Aug 2021 06:22:59 PM UTC, comment #10: |
Eric Gallager <egallager> |
Fri 18 Dec 2020 05:55:10 PM UTC, comment #9:
That link (at least on my computer) contains on the second paragraph of the header "timestamps are best avoided":
But you, and I agree on that, insist on the usefulness of the timestamp generated by xgettext---which was the main issue at hand on both bug reports as the msginit header year wasn't something to look at before rolling the hill up---, therefore their preferred way isn't omitting the timestamp but providing a meaningful value based on the source date and not the build date.
Dangerous? Human response to danger is fear, which isn't a reason but a feeling, therefore I'm trying to answer from that point of view, because those feelings may be there and they should be acknowledged and treated as such. I try to understand them along the responsibility of maintaining a project so useful and long lived, and as user I have to thank you for all your dedication and contributions towards making software accessible to everybody regardless their language. For that reason, I'd like to move the terms of the discourse to advantages/benefits/disadvantages/risks related to them, and analyze the problem with that in mind.
This is obviously a risk, but that would be reported as a bug by any translator to the creator of the POT, as it's their responsibility to provide a useful date---xgettext can only provide a mechanism for that. Paraphrasing from the other thread, if any project maintainer wants to enforce the usage of the system clock, SOURCE_DATE_EPOCH can be unset right before the desired invocation, therefore this risk has a low final impact from my point of view.
A system clock without battery nor ntp produces a similar output. I guess you're here confusing the responsibilities of the one who executes the code from the one who creates it: the first one has the responsibility of providing the inputs, the second of processing the inputs and generating the output, again for the usage of the former. To call "bogus" the desired output based on the input provided by the user may be a bit loaded. The environment variables, as any command line parameter, are completely controlled by the caller: only a person who doesn't know how the execution process works can be fooled that way, which again is out of the scope of gettext.
That isn't true on any interactive session on any current system, and I'd say that you don't have anything to worry in that sense in the near future. Be sure that I'd raise as many bug reports as needed to anybody who has the nefarious idea of defining that environment variable by default on any kind of session used for actual user interaction.
There is a big issue with that macro: the changelog date of the spec isn't a meaningful one because it isn't related to the source date and it's allowed to have a common spec for several versions of the same software. The variable must be set to the maximum of either the source date or the spec changelog latest date, not only the latter. Clearly any distro using that flag doesn't comply with the spirit of SOURCE_DATE_EPOCH spec.
It's a shame that this was directly closed as NOTABUG even when it clearly is. The reaction wasn't very polite neither. Sad. :(
Sure, because they use the same rpm flag: https://en.opensuse.org/openSUSE:Reproducible_Builds
That isn't the case in any of these projects. They'd produce POT files with the latest .spec changelog entry date, which also is quite bad but nowhere close to your example.
Distributions don't generate POT files for projects except for their software, so even in that case they will be well aware of the issue as they probably want to have their software translated, and they probably don't want to receive continuous complaints from their translators.
|
Miguel Ángel Arruga Vivas <m1gu3l>![]() ![]() |
Tue 15 Dec 2020 05:59:16 PM UTC, comment #8: 3) What the "reproducible builds" initiative is about: See https://reproducible-builds.org/docs/definition/ .
|
Bruno Haible <haible>![]() |
Tue 15 Dec 2020 02:36:48 AM UTC, comment #7: As a general comment: v3 patch only define a macro and include the header into xgettext.c and msginit.c, no other modification is needed and all the handling has been moved to gnulib (patch included too for the sake of completeness.) The tests remain mostly the same, with the removal of the sleep at xgettext-17, and the fixed timezone..
That definition of artifacts mentions distribution packages explicitly, nowhere "installed". Even rpm and deb can create source packages too, but nowadays GNU tarballs are artifacts of that kind by themselves, as they are generated from their corresponding VCS used for the actual development by make dist.
That may be a common case, but at least most of my gettext compilations start from a git checkout, even when the one I usually use is the one compiled by the distro or from a release when it isn't possible. Isn't that your case too?
I'm not aware of any current project that generates and modify manually POT files, nor it would be the scope of xgettext in that case. From the same paragraph of the GPL:
It can easily be interpreted that POT files emitted by xgettext, exactly the same as -email is unavailable- by msginit, and unlike po files from translators, are "object code" for the GPL because they are not used for modifications of the work itself but subproducts of its generation, just like the .o files.
As I said, they are symptoms, not the issue. The impossibility of controlling one of the implicit inputs (the time) is the issue here.
Sorry, they are identical or they are not: the return codes of diff are different for exactly that reason. The issue 1 fixed means that they are identical given the same inputs: sources, build environment (including SOURCE_DATE_EPOCH and TZ) and build instructions.
As I said, I'm well aware of this, but nobody defines SOURCE_DATE_EPOCH on their environment without a concrete objective, even less by default.
Of course it would impact the workflow negatively, but that date can be modified anyway with sed -e 's,(POT-Creation-Date:)[^\\]*,\\1 1970-01-01 00:00+0000'. I cannot understand the reason behind your concern, as their suggested mechanism for git projects is:
This is already encoded in the project version, something of the output that already can be controlled unlike POT-Creation-Date. The proposed mechanism with git gets the best POT-Creation-Date you can expect based on the current format---julian/gregorian based dates are "good enough" too, but that's another social+cultural+political issue and not the one at hand.
These patches allow exactly this without modifying the current build scripts for any project. Please try this with the patches applied, either v2 or v3+gnulib:
|
Miguel Ángel Arruga Vivas <m1gu3l>![]() ![]() |
Sun 13 Dec 2020 12:41:22 PM UTC, comment #6: We need to separate the issues.
|
Bruno Haible <haible>![]() |
Sun 13 Dec 2020 01:41:19 AM UTC, comment #5: From comment #2:
IMHO this doesn't address the main issue: the output of xgettext and msginit cannot be easily generated again after one minute (xgettext) or after one year (msginit) in the scenario you suggest---msginit also replaces the YEAR template from the source header. These tags are the symptom, not the actual issue.
I'm glad you ask. I though about this while working on it, as any time/localtime call introduce that kind of reproducibility problems. Reading gnulib's manual I see that at least one project has to use it. GCC has some code for it, which I'm using to discover missing points from my implementation: errno, long vs long long...
That's the main issue with wall clock tests, they need an annoying amount of time. It would be worse for the msginit test, as a call to 'sleep 1y' would have been indeed even less acceptable. :-)
|
Miguel Ángel Arruga Vivas <m1gu3l>![]() ![]() |
Sat 12 Dec 2020 02:11:23 PM UTC, comment #4:
A time requirement of 1 or 2 seconds is acceptable. A time requirement of 61 seconds is not; this is excessive.
|
Bruno Haible <haible>![]() |
Sat 12 Dec 2020 02:05:53 PM UTC, comment #3: Independently of GNU gettext: The SOURCE_DATE_EPOCH handling that you implemented (especially w.r.t. the time zone) seems to be non-trivial. Since https://reproducible-builds.org/specs/source-date-epoch/ applies to many programs used in a build process, a Gnulib module with this functionality would be useful.
|
Bruno Haible <haible>![]() |
Sat 12 Dec 2020 02:00:55 PM UTC, comment #2:
3. -email is unavailable- is created through
|
Bruno Haible <haible>![]() |
Sat 12 Dec 2020 12:18:24 AM UTC, comment #1: The v2 adds changes a bit the interface of po_strftime in order to provide the UTC field, fixes the missing call to time/localtime on msginit and adds two new tests to the test suite: msginit-5 and xgettext-17.
|
Miguel Ángel Arruga Vivas <m1gu3l>![]() ![]() |
Fri 11 Dec 2020 01:19:33 PM UTC, original submission:
This is an extension of the problem reported and solved by https://savannah.gnu.org/bugs/?49654
|
Miguel Ángel Arruga Vivas <m1gu3l>![]() ![]() |
Depends on the following items: None found
Items that depend on this one: None found
There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.
Follow 12 latest changes.
Date | Changed by | Updated Field | Previous Value | => | Replaced by |
---|---|---|---|---|---|
2021-08-07 | egallager | Carbon-Copy | - | ![]() |
Added egallager |
2020-12-22 | m1gu3l | Summary | xgettext and msginit don't honor SOURCE_DATE_EPOCH | ![]() |
xgettext and msginit output isn't reproducible |
2020-12-18 | m1gu3l | Attached File | - | ![]() |
Added 0001-msginit-Do-not-use-POT-Creation-Date.patch, #50513 |
2020-12-15 | m1gu3l | Attached File | - | ![]() |
Added 0001-source-date-epoch-New-module.patch, #50466 |
2020-12-15 | m1gu3l | Attached File | #50465 | ![]() |
Removed |
2020-12-15 | m1gu3l | Attached File | - | ![]() |
Added v3-0001-xgettext-msginit-Honor-SOURCE_DATE_EPOCH-for-time.patch, #50464 |
Attached File | - | ![]() |
Added 0001-source-date-epoch-New-module.patch, #50465 | ||
2020-12-12 | haible | Status | Ready For Test | ![]() |
In Progress |
2020-12-12 | m1gu3l | Status | In Progress | ![]() |
Ready For Test |
2020-12-12 | m1gu3l | Attached File | - | ![]() |
Added 0001-xgettext-msginit-Honor-SOURCE_DATE_EPOCH-for-timesta.patch, #50444 |
2020-12-11 | haible | Summary | xgettext and msgint don't honor SOURCE_DATE_EPOCH | ![]() |
xgettext and msginit don't honor SOURCE_DATE_EPOCH |
2020-12-11 | m1gu3l | Attached File | - | ![]() |
Added 0001-xgettext-msginit-Honor-SOURCE_DATE_EPOCH-for-timesta.patch, #50438 |
I wonder if this is related to the following Battle for Wesnoth bug? https://github.com/wesnoth/wesnoth/issues/5989
A commenter seems to think some of the lack of reproducibility there is due to msgmerge...