bugGNU Octave - Bugs: bug #45687, fread does not allow different...

 
 

bug #45687: fread does not allow different ARCH when file fopen'd with ARCH

Submitted by:  Emmanuel Goua de Baix <egouadeb>
Submitted on:  Tue 04 Aug 2015 08:44:34 AM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: FixedAssigned to: None
Originator Name: Open/Closed: Closed
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 11 Aug 2015 05:16:46 PM UTC, comment #9:

I added further clarification to the documentation for fread() that the ARCH specified there overrides the ARCH specified with fopen() (http://hg.savannah.gnu.org/hgweb/octave/rev/97edbce9b975). I think this has been resolved so I'm marking as fixed and closing the report.

Rik <rik5>
Project Administrator
Tue 11 Aug 2015 08:24:09 AM UTC, comment #8:

Rik, your results with the corrected version seem OK for me.
Here are the results when displaying the values (same as yours except that obviously I still get the error for b/ieee-le since I made no change to my Octave):

Now that I think of it, the results b/nothing are unexpected: it behaves like b/ieee-be instead of b/native.
The documentation of Octave about fread() is silent about the default value but the documentation of ML states that 'native' is the default ARCH. Still in fread()'s doc, I think there is no reference about the ARCH used by fopen(). No reference about ARCH used by fread() neither in fopen().

So I feel like the implementation of fread() uses the ARCH of fopen() by default, not necessarly 'native'. The documentation of ML is wrong on this point.

I discovered this issue with a binary file written by a C program where int are big-endian and float are little-endian.
One can also imagine a file where some floats are LE and some others are BE (headaches granted !).
I think that the feature to let the programmer force the ARCH in fread() independantly to the one used by fopen() is good and shall remain IMHO.

Emmanuel Goua de Baix <egouadeb>
Mon 10 Aug 2015 03:20:43 AM UTC, comment #7:

I changed the documentation on the stable branch to explain the one-letter shortcut codes (http://hg.savannah.gnu.org/hgweb/octave/rev/f02c22f71cdb).

There was a problem in the C++ code that called a non-existent function name which happened to be an enum. It was sort of miraculous that the compiler didn't flag it as an error. I fixed that on the default branch here (http://hg.savannah.gnu.org/hgweb/octave/rev/a76e20f2f156). Octave now runs through to completion for each of the different combinations. Still, it would be useful to see what the actual value is that should be returned. Could the original reporter run the newly attached rpt_arch_combinations.m and report the results. Running it in Octave I get the following.

(file #34619)

Rik <rik5>
Project Administrator
Fri 07 Aug 2015 11:01:11 PM UTC, comment #6:

Ok, I'm able to reproduce on Linux. But I have no idea if this is a valid situation, or whether we just want to have a better error handler for this condition. When the file is opened with ARCH = "native", you can choose the same or different ARCH when doing fread. But if fopen is called with ARCH != "native", then a different ARCH becomes an error in fread.

The ARCH values "b", "l", and "n" are undocumented synonyms for "ieee-be", "ieee-le", and "native", respectively.

Mike Miller <mtmiller>
Project Administrator
Thu 06 Aug 2015 11:12:29 AM UTC, comment #5:

ARRGG ! B***** H*** ! I found my mistake! Sorry, the bug is more subtle than I first thought.

Some explanations: Since several people reported that my example worked on their machine, I tried it again on mine ... and it worked also!
In fact, the example I gave in the original post is a very simplified version of a Matlab script that I am trying to port for Ocatve.
In this original Matlab script (not written by me), the file is opened with: fopen(filename, 'r', 'b') ... not 'rb' !!
Stupidly, I didn't pay attention to this, thinking that it was equivalent!
My first tests to write the example was done with "'r','b'". Then for an unknown reason I written "'rb'" (maybe I was lazy and wanted to save 3 chars ...) in order to test on the 32bits PC. And I didn't try again with "'rb'" on my 64bits PC. I posted the example with this option, thinking it was a particular bug on 64bits machine!
Now, after carefully reading the doc of fopen on both Matlab and Octave, I understand that the third parameter of fopen is an ARCH indicator. 'b' seems to be a short for 'ieee-be' (undocumented on Ocatve).

So to make it short, my issue seems to be that I tell fopen I work in ieee-be then I try to read in ieee-le?

I wrote the attached script to test several combinations of ARCH for fopen and fread.
It prints “OK” when fread succeeds in reading the double in the binary file attached in original post (it does not check the correctness of the value).
It prints “ERROR !“ if an error of fread is caught (assuming it is the same error as reported in original post)

Here are the results:

Thus, it seems that the anomaly appears when the file is opened in ieee-be and then the value is read in ieee-le .

On PC, no ARCH for fopen should be equivalent to ieee-le which doesn’t result in error. Sorry for the mistake in the original post and thank you all for your reactivity.

NOTE: On both my 64 and 32 bits PCs :

(file #34579)

Emmanuel Goua de Baix <egouadeb>
Wed 05 Aug 2015 05:25:38 PM UTC, comment #4:

This works for me as well on GNU/Linux 64-bit and Windows XP 32-bit. Someone else with access to a Windows 64-bit installation will have to try and verify this.

The code in question is in liboctave/util/data-conv.cc and is shown below.

The only way it can get to the default branch of the case statement is if the conversion format is not ieee-le or ieee-be. The fact that it gets there suggests that something is wrong with the initial determination of the machine's floating point format. That determination occurs in liboctave/system/mach-info.cc.

Could the original reporter try the following at an Octave prompt?

On my Linux Intel machine this returns "ieee-le" as it should. I suspect that this will return "unknown" for you.

Rik <rik5>
Project Administrator
Wed 05 Aug 2015 12:52:31 AM UTC, comment #3:

Hi, thanks for your bug report. Someone will have to try to reproduce this on Windows 64-bit. It works correctly on a GNU/Linux 64-bit system.

Mike Miller <mtmiller>
Project Administrator
Tue 04 Aug 2015 10:09:03 AM UTC, comment #2:

After some few additional tests, I found that the issue is the same with float (single precision)

I specified my workaround solution ( compatible 32/64 bits Matlab/Octave but only on PC (little endian))

Emmanuel Goua de Baix <egouadeb>
Tue 04 Aug 2015 08:55:03 AM UTC, comment #1:

Sorry for the poor text formatting, it is may first post on this site.
I don't know how to edit the orginal post.
The file content given in PS, is not shown correctlly due to tags, the value starts with a zero byte :
00 48 AB 19 8A 98 0A 41

Emmanuel Goua de Baix <egouadeb>
Tue 04 Aug 2015 08:44:34 AM UTC, original submission:

I found an issue to read a double ieee-le in a binary file :
returns this error message :

Additonal information :
Error observed on a Windows7 64bits PC with Octave 4.0.0 (installed from octave-4.0.0_0-installer.exe).
On a Windows XP 32bits PC, BOTH Matlab AND Octave 4.0.0 give the expected result :
Moreover, on the 64bits PC, the fread command without ARCH option does return a result but not the expected one since bytes need to be swapped (validated by swapping manually the bytes):
Using the option 'native' results in the same error message than 'ieee-le' (expected since PC is little-endian).
Using the option 'ieee-be' provides the same unwanted value than without option - and no error message !
Reading to type 'uint32' or 'uint64' does not trigger an error for any ARCH.
Not tested with other types (except uint above).
Not tested on Linux nor Mac.
Not tested with other bytes values.
Not tested on other Octave release.

Due to the 64bits dependancy of this issue, this bug might be related to #41672 : fread for "float" not matlab compatible

Workaround :
I will add a swap command in my parsing script and use fread without ARCH ...

PS:
'test_octave_double_01.hex' simply contains the following bytes (can be created/edited by HexEdit) :

    1. 48 AB 19 8A 98 0A 41

PPS about Item group :
I don't know if it is a Regression, and I don't think it is an Incorrect/Inaccurate Result, and it is more a Failure than a Crash (error message) => Other

Emmanuel Goua de Baix <egouadeb>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #34619:  rpt_arch_combinations.m added by rik5 (2KiB - text/x-objcsrc)
file #34579:  test_arch_combinations.m added by egouadeb (2KiB - application/octet-stream)
file #34573:  test_octave_double_01.hex added by egouadeb (8B - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by egouadeb (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 11 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 11 Aug 2015 05:16:46 PM UTCrik5StatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Mon 10 Aug 2015 03:20:43 AM UTCrik5Attached File-=>Added rpt_arch_combinations.m, #34619
    Fri 07 Aug 2015 11:01:11 PM UTCmtmillerStatusNeed Info=>Confirmed
      Release4.0.0=>dev
      Operating SystemMicrosoft Windows=>Any
      Summaryfread does not support ARCH \'ieee-le\' for double on 64bits PC=>fread does not allow different ARCH when file fopen'd with ARCH
    Thu 06 Aug 2015 11:12:29 AM UTCegouadebAttached File-=>Added test_arch_combinations.m, #34579
    Wed 05 Aug 2015 12:52:31 AM UTCmtmillerItem GroupOther=>Incorrect Result
      StatusNone=>Need Info
    Tue 04 Aug 2015 08:44:34 AM UTCegouadebAttached File-=>Added test_octave_double_01.hex, #34573

    Back to the top


    Powered by Savane 3.1-cleanup1