bugGNU Octave - Bugs: bug #58457, [octave forge] (io)...

 
 

bug #58457: [octave forge] (io) dbfread/dbfwrite failure on big-endian systems

Submitter:  Rafael Laboissière <rlaboiss>
Submitted:  Wed 27 May 2020 09:26:32 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 29 May 2020 01:41:20 PM UTC, comment #9: 

Thanks for the commit and for the acknowledgement.

Thanks also for the heads up regarding the tempfile warning.  This is problem in dh-octave, which has just been fixed.

Rafael Laboissière <rlaboiss>
Fri 29 May 2020 07:22:14 AM UTC, comment #8: 

<push credentials> I don't know either. If you want those you might ask Olaf Till.

After some searching in my archives and on the web I found that info on endianness of numeric data in dbf files is inconclusive at best. So let's hope that you and I are doing the right thing :-) yet I do have some faith in it.

Anyway after some editing I pushed your fix, incl. a NEWS entry, under your name, here:
http://hg.code.sf.net/p/octave/io/rev/80ae14a84ec3

Thanks very much.

A little OT:
In the s390x log I saw references to deprecated tempfile - you may want to look at that.

Closing report.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 29 May 2020 06:25:04 AM UTC, comment #7: 

I do not think I have write access to the repository, or do I?

Rafael Laboissière <rlaboiss>
Thu 28 May 2020 10:46:03 PM UTC, comment #6: 

In that format spec (which I recognize, got it on my HD somewhere) only endianness of numeric data in the file header is mentioned. I have several more file specs, I'll check there as well later on, but I'm sure you're right.

Can you commit your cset to the repo? if not I'll do it tomorrow.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 28 May 2020 07:51:11 PM UTC, comment #5: 

To sum up: dbfwrite does the right thing, by opening the .dbf file for writing with the "ieee-le" option.  On the other hand, the current version of dbfread does the right thing only on little-endian systems.  It fails on big-endian systems, unless the option "ieee-le" is used.

Rafael Laboissière <rlaboiss>
Thu 28 May 2020 07:47:12 PM UTC, comment #4: 

The dbase file (.dbf) format specification indicates that 16- and 32-bit numbers should be written with least significant byte first.

So, my patch is indeed simplistic, but it does fix the problem on s390x, powerpc and ppc64.  The bug was revealed by a unit test in the mapping package, which was exercised at build time of the octave-mapping Debian packages (s390x, powerpc, and ppc64, respectively). The build is now working for the three architectures (s390x, powerpc, and ppc64, respectively).

As regards the need for a similar fix in line 267 of dbfread.m, I really do not know. It seems that only 8-bit chars are read from .dbt files, so I would say that there is no need for a fix here.

Rafael Laboissière <rlaboiss>
Thu 28 May 2020 05:45:59 PM UTC, comment #3: 

Thanks Rafael.
Looks like a suspiciously simple patch. Do the tests pas and don't we need a similar fix for L.267 ?

I'd have to check the dbf/xbf docs whether a dbf file really needs to be written with little-endian data (could be, I suppose I didn't write it like that for nothing, but it's a looong time ago :-) )

Philip Nienhuis <philipnienhuis>
Group Member
Thu 28 May 2020 07:31:27 AM UTC, comment #2: 

The patch attached to this comment fixes the problem for me on a s390x machine.

(file #49182)

Rafael Laboissière <rlaboiss>
Thu 28 May 2020 07:19:26 AM UTC, comment #1: 

Thanks Rafael, I'll look into it, early June I'll have more time for this.
But I don't have a big-endian system at my disposal. Hopefully you can help with testing.


Philip Nienhuis <philipnienhuis>
Group Member
Wed 27 May 2020 09:26:32 PM UTC, original submission:  

Functions dbfread/dbfwrite of the io package fail on big-endian system (in Debian: s390x, powerpc, ppc64). The following code works fine on little-endian systems:


f = tempname ();
c = cell (2,1);
c {1, 1} = "a";
c {2, 1} = 1;
dbfwrite (f, c);
x = dbfread (f);


but fails on a 390x system, with the following error message:


error: fdesc(11): out of bound 0
error: called from
    dbfread at line 212 column 25


The bytes in the file are not written/read in the right order on big-endian machines.

Rafael Laboissière <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49182:  dbfread-big-endian.patch added by rlaboiss (589B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-05-29 philipnienhuis StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
    2020-05-28 philipnienhuis StatusNone Patch Submitted
        Carbon-Copy- Added philipnienhuis
    2020-05-28 rlaboiss Attached File- Added dbfread-big-endian.patch, #49182
    2020-05-28 mtmiller Summary[octave forge] (io) dfbread/dbfwrite failure on big-endian systems [octave forge] (io) dbfread/dbfwrite failure on big-endian systems

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code