bugGNU Octave - Bugs: bug #31079, DATESTR function buggy before year...

 
 

bug #31079: DATESTR function buggy before year 1970

Submitter:  J-D Bonjour <jdb>
Submitted:  Sun 19 Sep 2010 10:27:29 PM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 3.2.4 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 09 Dec 2010 11:20:50 PM UTC, comment #12: 

OK, I assume they failed before?  I'm closing this report.  If there are any further problems, reopen it, or open a new bug report.

John W. Eaton <jwe>
Group administrator
Thu 09 Dec 2010 02:45:29 PM UTC, comment #11: 

Running on MacOS X, the datestr tests now all pass for me!


test datestr
PASSES 35 out of 35 tests


Ben Abbott <bpabbott>
Group Member
Thu 09 Dec 2010 07:44:40 AM UTC, comment #10: 

The datestr function from Octave 3.0.3 had other problems that prompted me to change to using strftime.  See the following thread:

https://www-old.cae.wisc.edu/pipermail/bug-octave/2008-September/006730.html

I checked in the following changes:

http://hg.savannah.gnu.org/hgweb/octave/rev/f286a874617c
http://hg.savannah.gnu.org/hgweb/octave/rev/6b073da9d7fc

These changes do not handle DST.  It would be nice if they did, but I don't know how to easily do that without localtime and mktime.  But I agree that this is probably less of a problem than not handling dates prior to 1970.

Please let me know whether this fixes the problem for you.

John W. Eaton <jwe>
Group administrator
Fri 24 Sep 2010 08:30:47 PM UTC, comment #9: 

FYI, here's what I get on that freshly built octave-3.3.52:

[philip@deskprn octave-3.3.52]$ ./run-octave
GNU Octave, version 3.3.52+
<snip>

octave:1> tm = localtime (time)
tm =
{
  usec =  795444
  sec =  25
  min =  35
  hour =  21
  mday =  24
  mon =  8
  year =  110
  wday =  5
  yday =  266
  isdst =  1
  zone = CEST
}

octave:2> tm.year = -23
tm =
{
  usec =  795444
  sec =  25
  min =  35                                                                                             
  hour =  21                                                                                            
  mday =  24
  mon =  8
  year = -23
  wday =  5
  yday =  266
  isdst =  1
  zone = CEST
}

octave:3> mktime (tm)
ans = -0.20456
octave:4> localtime (mktime (tm))
ans =
{
  usec = -204555
  sec = 0
  min = 0
  hour =  1
  mday =  1
  mon = 0
  year =  70
  wday =  4
  yday = 0
  isdst = 0
  zone = CET
}

A while earlier (see the 2nd thread I mentioned some posts ago) you've asked that same question: what are we to do?
I wouldn't know. Relying on system functions doesn't seem to work.
There is much SW around that has similar limitations: some have epoch around 1-1-1970, others around 1-1-1900. Few SW allows earlier dates.

Perhaps first of all we might need to set some goals.
ML compatibility?
As to datenum <-> datestr, TMW says "The year 0000 is merely a reference point and is not intended to be interpreted as a real year in time." I suppose anything from before Pope Gregorius' intervention in October 1582 had better be treated similarly.
So I'd opt for something that yields proper date/time (i.e. valid for countries that followed Gregorius immediately) at least starting from October 15, 1582, 00:00:00.

Still the fact that datestr.m from Octave-3.0.3 and earlier worked OK (or should I say:satisfactorily for me) intrigues me. While this is an Octave-only solution, we do have it already. I have no idea about its bugs or issues, but perhaps these bugs + issues/limitations could be fixed? Or was it just too hopeless?

Philip Nienhuis <philipnienhuis>
Group Member
Fri 24 Sep 2010 05:04:56 PM UTC, comment #8: 

I think the gnulib mktime tests don't check whether dates before the epoch can be handled.

Since you have a glibc-based system, I'm not sure why you see results that are different from what I see.  On my system, I get the following results:

octave3.2:2> tm = localtime (time)
tm =
{
  usec =  135222
  sec =  41
  min =  57
  hour =  12
  mday =  24
  mon =  8
  year =  110
  wday =  5
  yday =  266
  isdst =  1
  zone = EDT
}

octave3.2:3> tm.year = -23
tm =
{
  usec =  135222
  sec =  41
  min =  57
  hour =  12
  mday =  24
  mon =  8
  year = -23
  wday =  5
  yday =  266
  isdst =  1
  zone = EDT
}

octave3.2:4> mktime (tm)
ans = -2.9117e+09
octave3.2:5> localtime (mktime (tm))
ans =
{
  usec = -864778
  sec =  42
  min =  57
  hour =  12
  mday =  24
  mon =  8
  year = -23
  wday =  1
  yday =  266
  isdst = 0
  zone = LMT
}

So mktime just generates negative values for dates before the epoch and localtime converts those back to a time structure.  But there are apparently some strange things happening because we don't exactly recover the original time structure.

If the C library time functions are worthless, then what are we supposed to do?  I don't think we should be inventing a date/time library just for Octave.  If we are going to write something like that, I think it should be a gnulib module that could be used by more than just Octave.

Or, is there something that already exists to do the job?  Surely Octave is not the only program that needs functionality like this.

John W. Eaton <jwe>
Group administrator
Fri 24 Sep 2010 04:41:16 PM UTC, comment #7: 

Oh I forgot to mention my system, sorry:
Mandriva Linux 2010.1 (Spring) with all updates.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 24 Sep 2010 04:39:39 PM UTC, comment #6: 

I "cloned" a fresh copy of the 3.3.52 sources, did ./autogen.sh, ./configure, make, and then ./run-octave:

octave:1> datestr (datenum (1969, 1, 1), 0)
ans = 01-Jan-1969 00:00:00
octave:2> datestr (datenum (1901, 1, 1), 0)
ans = 01-Jan-1970 00:59:59
octave:3> datestr (datenum (1795, 1, 1), 0)
ans = 01-Jan-1970 00:59:59

So no joy.

I'm wondering a bit what to expect from mktime().
Is the goal really to be able to use datestr() for any date after AD and -say- some centuries in the future? (as Matlab does?)

I perused through ./gnulib/lib/mktime.c and I'm not very confident as to whether it will help solve the datestr.m "bug", judging from this stanza around line 115:
---<snip>
#define EPOCH_YEAR 1970
#define TM_YEAR_BASE 1900
---</snip>

For the record, in config.log & config.status:

philip@deskprn octave-3.3.52]$ grep -i mktime < config.log
configure:19699: checking for working mktime
| #define GNULIB_TEST_MKTIME 1
<lines 5...72 = same message as above>
| #define GNULIB_TEST_MKTIME 1
configure:50547: defining UGLY_DEFS to be -DPACKAGE_NAME=\\\\\"GNU\ Octave  <long snip, line contains "-DGNULIB_TEST_MKTIME=1">
ac_cv_func_working_mktime=yes
GNULIB_MKTIME='1'
REPLACE_MKTIME='0'
UGLY_DEFS='-DPACKAGE  <long snip, line contains "-DGNULIB_TEST_MKTIME=1">
#define GNULIB_TEST_MKTIME 1
[philip@deskprn octave-3.3.52]$ grep -i mktime < config.status
"CPY=1 -DGNULIB_TEST_MEMPCPY=1 -DHAVE_MEMRCHR=1 -DGNULIB_TEST_MEMRCHR=1 -DGNULIB_TEST_MKFIFO=1 -DGNULIB_TEST_MKTIME=1 -DGNULIB_TEST_NANOSLEEP=1 -DGNU"\
S["REPLACE_MKTIME"]="0"
S["GNULIB_MKTIME"]="1"
D["GNULIB_TEST_MKTIME"]=" 1"
[philip@deskprn octave-3.3.52]$

...so perhaps the gnulib version hasn't been included?

Philip Nienhuis <philipnienhuis>
Group Member
Wed 22 Sep 2010 11:48:32 AM UTC, comment #5: 

I'd assumed that gnulib:mktime would now be used. I think it is, but I'm not familiar enough with the configure flags to be certain.

In my config.log, for MacOS X, I see mktime mentioned in 4 places.

55764 ac_cv_func_wmempcpy=no
55765 ac_cv_func_working_mktime=yes
55766 ac_cv_func_x_utime=no

56688 GNULIB_MKSTEMPS='0'
56689 GNULIB_MKTIME='1'
56690 GNULIB_NANOSLEEP='1'

57195 REPLACE_MKSTEMP='0'
57196 REPLACE_MKTIME='0'
57197 REPLACE_NAN='0'

12150 configure:19540: result: no
12151 configure:19583: checking for working mktime
12152 configure:19775: gcc-4.2 -o conftest -I/sw/include -m32 -O2 -g -I/sw/lib/flex/include -I/sw/include -m32 -I/sw/include/freetype2 -I/sw/include      /qhull -I/usr/include -L/sw/lib/flex/lib -L/sw/lib -L/usr/lib -m32 conftest.c  >&5
12153 configure:19775: $? = 0

In config.status ...

bens-macbook:local_clone bpabbott$ fgrep -in mktime config.status
986:"R=1 -DMKFIFO_TRAILING_SLASH_BUG=1 -DGNULIB_TEST_MKFIFO=1 -DGNULIB_TEST_MKTIME=1 -DGNULIB_TEST_NANOSLEEP=1 -DOPEN_TRAILING_SLASH_BUG=1 -DGNULIB_TEST_"\
1347:S["REPLACE_MKTIME"]="0"
1357:S["GNULIB_MKTIME"]="1"
2297:D["GNULIB_TEST_MKTIME"]=" 1"

Ben Abbott <bpabbott>
Group Member
Wed 22 Sep 2010 04:54:02 AM UTC, comment #4: 

Are you sure you are using the gnulib mktime?  What did configure determine about whether you have a working mktime?

John W. Eaton <jwe>
Group administrator
Tue 21 Sep 2010 05:55:14 PM UTC, comment #3: 

This problem existed for me on MacOS 10.6.4 before your change. With Octave built using the gnulib mktime, I see the same problem.

octave:1> test time.cc
  *** assert (datestr (datenum (1901, 1, 1), 0), "01-Jan-1901 00:00:00")
!!!!! test failed
assert (datestr (datenum (1901, 1, 1), 0),"01-Jan-1901 00:00:00") expected
01-Jan-1901 00:00:00
but got
31-Dec-1969 18:59:59

I've also installed a copy of Ubuntu 10.04 using VirtualBox. I get the same (incorrect) result there.

Ben Abbott <bpabbott>
Group Member
Tue 21 Sep 2010 01:52:29 AM UTC, comment #2: 

Here is what I see with the current development sources of Octave:

octave:1> datestr (datenum (1969, 1, 1), 0)
ans = 01-Jan-1969 00:00:00
octave:2> datestr (datenum (1901, 1, 1), 0)
ans = 01-Jan-1901 00:00:00
octave:3> datestr (datenum (1795, 1, 1), 0)
ans = 01-Jan-1795 00:00:00

I also see the same results with Octave 3.2.4 installed from the Debian package.

But my system has glibc.

Maybe using the mktime module from gnulib would solve the problem, as that is based on code from glibc?  I checked in the following change:

  http://hg.savannah.gnu.org/hgweb/octave/rev/0de4eff677d6

This change includes tests based on the examples in this bug report.  If you have additional tests that would help, please submit a changeset.

It would be helpful if someone could test my change on systems where the failure was previously happening and report back.

John W. Eaton <jwe>
Group administrator
Mon 20 Sep 2010 11:41:51 AM UTC, comment #1: 

A nabble search shows up that this has been discussed several times before, and not only for MingW. The most recent thread on bug-octave is here

https://www-old.cae.wisc.edu/pipermail/bug-octave/2009-June/008887.html

and M. Goffioul supplied an answer (for MingW) here:

https://www-old.cae.wisc.edu/pipermail/bug-octave/2009-June/008905.html

IMO the underlying issue is that Octave uses system calls (in MingW: mktime() ) that have limitations.

Personally I use a datestr.m from octave 3.0.3 for my octave-3.2.4 installation that doesn't suffer from this issue (don't get fooled by octave's version):

octave-3.2.4.exe:2> datestr (datenum (1969, 1, 1), 0)
ans = 01-Jan-1969 00:00:00
octave-3.2.4.exe:3> datestr (datenum (1901, 1, 1), 0)
ans = 01-Jan-1901 00:00:00
octave-3.2.4.exe:4> datestr (datenum (1795, 1, 1), 0)
ans = 01-Jan-1795 00:00:00
octave-3.2.4.exe:5>

I need this as I often work with time series starting in the late 19th century.
A problem with this old datestr.m seems to be (please correct me if I'm wrong) that it doesn't accept multidimensional input (it doesn't work on arrays). However as regards date range limitations it is more robust than later versions.

I think the test section of datestr.m should be augmented with earlier dates than referring to the year 2005. This bug (or limitation) would have shown up much earlier if such tests would have been in place.
I'm not proficient enough in octave script for fixing datestr.m, but I can at least send patches for the test section (later this week).

Philip Nienhuis <philipnienhuis>
Group Member
Sun 19 Sep 2010 10:27:29 PM UTC, original submission:  

On Octave 3.2.4 Windows MinGW (from 3.2.0 ?), when the argument corresponds to a date before 1st January 1970 at 1:00, result returned by DATESTR is wrong. Example :

>> datestr(now)

ans = 17-Sep-2010 16:12:57
% it's OK

>> datestr(datenum(1970,1,1,1,0,01))

ans = 01-Jan-1970 01:00:01
% still OK

>> datestr(datenum(1970,1,1,0,59,30))

ans = 00-Jan-1900 00:00:00
% bad, shoud be 01-Jan-1970 00:59:30

>> datestr(1,'dd-mmm-yyyy HH:MM:SS')

ans = 00-Jan-1900 00:00:00
% bad, shoud be 01-Jan-0000 00:00:00

>> datestr(1.0001,'dd-mmm-yyyy HH:MM:SS')

ans = 01-Jan-1970 01:00:00
% bad, shoud be 01-Jan-0000 00:00:08
% and why suddenly back in year 1970 and not 1900 like previous command ???


On Linux Ubuntu 10.04, Octave 3.2.3 is buggy when the argument is earlier than 13 December 1901 at 21:45:52, string allways returned is :  01-Jan-1970 00:59:59

J-D Bonjour <jdb>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bpabbott (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by philipnienhuis (Posted a comment)
  • -email is unavailable- added by jdb (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-12-09 jwe StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2010-09-21 jwe StatusNone Ready For Test

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code