bugGNU Wget - Bugs: bug #62110, HSTS broken on 32 bit big endian...

 
 

bug #62110: HSTS broken on 32 bit big endian devices

Submitter:  martian <martian>
Submitted:  Thu 24 Feb 2022 10:50:28 AM UTC
   
 
Category:  Feature Request Severity:  3 - Normal
Priority:  5 - Normal Status:  Fixed
Privacy:  Public Assigned to:  rockdaboot
Originator Name:  Open/Closed:  Closed
Release:  trunk Operating System:  GNU/Linux
Reproducibility:  Every Time Fixed Release:  None
Planned Release:  None Regression:  None
Work Required:  None Patch Included:  Yes
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 21 Mar 2022 07:34:25 AM UTC, comment #16: 

Hi Tim,

just confirming that the fix works, thanks for the quick solution.

Regards,
Michal

Michal Ruprich <formaiko>
Sun 20 Mar 2022 08:02:05 PM UTC, comment #15: 

Thanks for testing <3

The Gitlab PR #31 has been merged manually into master.

Tim Ruehsen <rockdaboot>
Group administrator
Sun 20 Mar 2022 07:55:37 PM UTC, comment #14: 

I compile PR#31 on 32bit mips big endian with musl 1.1 and 1.2, both work as expected.

martian <martian>
Sun 20 Mar 2022 11:25:06 AM UTC, comment #13: 

Created https://gitlab.com/gnuwget/wget/-/merge_requests/31 to address this issue.

Can someone please provide feedback for 32 bit little and big endian ?

Tim Ruehsen <rockdaboot>
Group administrator
Sun 20 Mar 2022 10:53:52 AM UTC, comment #12: 

IMO, the problem is that the code uses a time_t variable and uses sscanf SCNd64. That means that time_t must be a 64bit integer (signed or unsigned doesn't matter), which is of course not always true.

C99/C11 does not define a format specifier for time_t.

There are several ways to address this, but using SCNd64 is not among them.

In Wget2 we use uint64_t instead of time_t after getting complaints from user previously. We failed to backport this properly to Wget 1.x, sorry for that. We should fix these issues once and forever (there might be more code that needs to be amended beyond hsts.c).

Tim Ruehsen <rockdaboot>
Group administrator
Fri 18 Mar 2022 06:43:02 AM UTC, comment #11: 

Indeed. A 32-bit big-endian MIPS:

$ file staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl/lib/gcc/mips-openwrt-linux-musl/8.4.0/crtbegin.o
staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl/lib/gcc/mips-openwrt-linux-musl/8.4.0/crtbegin.o: ELF 32-bit MSB relocatable, MIPS, MIPS32 rel2 version 1 (SYSV), not stripped

I compiled with it a simple code to check the sizes: time_t is 4 bytes, int is 4 bytes, int64_t is supported and has 8 bytes.

I tried adding glibc's -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64, but that did not change anything. musl C library does not support it. musl supports 64-bit time_t since version 1.2.0 <https://musl.libc.org/time64.html>. And of course on needs "recent" Linux. I guess this is not the case of OpenWRT.

Petr Pisar <petrp>
Fri 18 Mar 2022 12:40:41 AM UTC, comment #10: 

Yeah, I compile wget 1.21.3 with OpenWRT sdk from downloaded https://downloads.openwrt.org/releases/21.02.2/targets/ath79/generic/openwrt-sdk-21.02.2-ath79-generic_gcc-8.4.0_musl.Linux-x86_64.tar.xz

It's big endian and has 32bit time_t provided by musl 1.1.

The hsts file will be like after downloading a file.

# HSTS 1.0 Known Hosts database for GNU Wget.
# Edit at your own risk.
# <hostname> <port> <incl. subdomains> <created> <max-age>
api.github.com 0 1 0 0
codeload.github.com 0 0 0 0
raw.githubusercontent.com 0 0 1638583977 31536000
github.com 0 1 0 0

martian <martian>
Thu 17 Mar 2022 05:16:40 PM UTC, comment #9: 

Yes, i686 is 32-bit little-endian and GCC supports int64_t there. However, the original reporter wrote "broken on 32 bit big endian devices".

Petr Pisar <petrp>
Thu 17 Mar 2022 07:59:56 AM UTC, comment #8: 

Hi Pert,

thanks for the input. The failing hsts test was on i686 in Fedora and I am no 100% sure but that should be little-endian right?

Michal Ruprich <formaiko>
Thu 17 Mar 2022 07:31:32 AM UTC, comment #7: 

That gnulib "int64" code does not work on all platforms. It only works on platforms where a 64-bit integer type is supported by the compiler (int64_t in stdint.h). For the sake of completeness, it is supported e.g. by GCC on 32-bit little-endian x86. I have no idea about the platform of the original reporter.

Petr Pisar <petrp>
Thu 17 Mar 2022 02:54:13 AM UTC, comment #6: 

IMO, the following code should always get the correct result on any platform. If not, that should be a bug of gnulib.

int64 x;
sscanf(buf, "%" SCNd64, &x);

martian <martian>
Wed 16 Mar 2022 09:10:39 AM UTC, comment #5: 

The problem in the test is that the sscanf in hsts_read_database tries to read values 1647421226 and 123456 into variables 'created' and 'max_age; and the results are numbers 530239482494976 and 6224032784512362224. In Fedora, I changed the sscanf to read SCNd32 on i686 only but I was just wondering whether there is a more general solution to this(I did not figure any yet).

Michal Ruprich <formaiko>
Tue 15 Mar 2022 08:02:13 AM UTC, comment #4: 

Reversing commit e1bacd2fa5f026f9493bd74e9179b859fa9ce8f1 - src/hsts.c (hsts_read_database): Read time_t values as long long helps.

Michal Ruprich <formaiko>
Tue 15 Mar 2022 07:58:27 AM UTC, comment #3: 

Hi,

when building new wget for i686, Test-hsts.py is the only test that fails. Seems to me that there is something wrong with the server in this test:

Reading HSTS entries from /root/rpmbuild/BUILD/wget-1.21.3/testenv/.wget-hsts-testenv
URI encoding = 'ANSI_X3.4-1968'
converted 'http://localhost:46041/hw' (ANSI_X3.4-1968) -> 'http://localhost:46041/hw' (UTF-8)
Converted file name 'hw' (UTF-8) -> 'hw' (ANSI_X3.4-1968)
--2022-03-15 03:48:20--  http://localhost:46041/hw
Resolving localhost (localhost)... ::1, 127.0.0.1
Caching localhost => ::1 127.0.0.1
Connecting to localhost (localhost)|::1|:46041... Closed fd 3
failed: Connection refused.
Connecting to localhost (localhost)|127.0.0.1|:46041... connected.
Created socket 3.
Releasing 0x572d4340 (new refcount 1).

Then there is a bunch of failed connects:

---request begin---
GET /hw HTTP/1.1^M
Host: localhost:46041^M
User-Agent: Wget/1.21.3^M
Accept: /^M
Accept-Encoding: identity^M
Connection: Keep-Alive^M
^M
---request end---
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Closed fd 3
Retrying.

This goes on for a while until it ends up with the actual test:

---request end---
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Closed fd 3
Giving up.

Running Test Test-hsts.py
/root/rpmbuild/BUILD/wget-1.21.3/src/wget --debug --no-config --hsts-file=/root/rpmbuild/BUILD/wget-1.21.3/testenv/.wget-hsts-testenv --ca-certificate=/root/rpmbuild/BUILD/wget-1.21.3/testenv/certs/ca-cert.pem http://localhost:46041/hw
['/root/rpmbuild/BUILD/wget-1.21.3/src/wget', '--debug', '--no-config', '--hsts-file=/root/rpmbuild/BUILD/wget-1.21.3/testenv/.wget-hsts-testenv', '--ca-certificate=/root/rpmbuild/BUILD/wget-1.21.3/testenv/certs/ca-cert.pem', 'http://localhost:46041/hw']
{'HOME': '/root/rpmbuild/BUILD/wget-1.21.3/testenv/Test-hsts.py-test'}
Error: Expected file hw not found..
Traceback (most recent call last):
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/./Test-hsts.py", line 79, in <module>
    err = test.begin()
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/test/http_test.py", line 41, in begin
    self.do_test()
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/test/base_test.py", line 198, in do_test
    self.post_hook_call()
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/test/base_test.py", line 217, in post_hook_call
    self.hook_call(self.post_configs, 'Post Test Function')
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/test/base_test.py", line 207, in hook_call
    conf.find_conf(conf_name)(conf_arg)(self)
  File "/root/rpmbuild/BUILD/wget-1.21.3/testenv/conf/expected_files.py", line 55, in _call_
    raise TestFailed('Expected file %s not found.' % file.name)
exc.test_failed.TestFailed: Expected file hw not found.
FAIL Test-hsts.py (exit status: 1)

Michal Ruprich <formaiko>
Fri 11 Mar 2022 02:41:22 PM UTC, comment #2: 

Excuse me, what failure did you get? I'm not getting any problems.

martian <martian>
Fri 11 Mar 2022 10:53:48 AM UTC, comment #1: 

Just adding a bit of info. When --disable-year2038 is used, hsts fails with this patch but metalink works. Without the option, all metalink tests fail but hsts runs ok.

Michal Ruprich <formaiko>
Thu 24 Feb 2022 10:50:28 AM UTC, original submission:  

e1bacd2fa5f026f9493bd74e9179b859fa9ce8f1 fix the 64 bit time_t, but break the behavior on 32 bit big endian devices

martian <martian>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #52918:  fix-hsts-time-on-mips-musl1.1.patch added by martian (787B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rockdaboot (Posted a comment)
  • -email is unavailable- added by petrp (Posted a comment)
  • -email is unavailable- added by formaiko (Posted a comment)
  • -email is unavailable- added by martian
  • -email is unavailable- added by martian (Submitted the item)
  •  

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-03-20 rockdaboot StatusNone Fixed
        Assigned toNone rockdaboot
        Open/ClosedOpen Closed
    2022-02-24 martian Carbon-Copy- Added -email is unavailable-
    2022-02-24 martian Attached File- Added fix-hsts-time-on-mips-musl1.1.patch, #52918

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code