bugGNU Octave - Bugs: bug #42353, ncread reading data incorrectly

 
 

bug #42353: ncread reading data incorrectly

Submitter:  Muhali <muhali>
Submitted:  Wed 14 May 2014 12:21:30 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Wont Fix Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 10 Nov 2015 02:50:59 PM UTC, comment #6: 

I propose to close the bug report. Missing value are not replaced by NaNs (as asked by the submitter) for compatibility with matlab.

Alexander Barth <abarth>
Thu 11 Jun 2015 10:05:26 AM UTC, comment #5: 

Ok, thanks. Values equal to the attribute missing_value are not automatically replaced by NaN (only _FillValue). This is compatible with Matlab (from the doc http://nl.mathworks.com/help/matlab/ref/ncread.html).

I prefer to keep compatibility with Matlab even it is true that in most cases you want to replace the missing value by NaN.

For NetCDF files that I create myself, I always use now _FillValue instead of missing_value.

I propose to close this bug report.

Alexander Barth <abarth>
Thu 11 Jun 2015 06:41:24 AM UTC, comment #4: 

I meant the missing_value Attribute.

Muhali <muhali>
Wed 10 Jun 2015 07:04:07 PM UTC, comment #3: 

Just a clarification, when you say "with an int16 for undefined", do you mean that the missing_value attribute of the variable was an int16 ? Or do you mean that the _FillValue attribute was int16 ?


Alexander Barth <abarth>
Wed 10 Jun 2015 04:27:07 PM UTC, comment #2: 

As the original file was changed it does no longer show the int16 problem. Unfortunately, I don't have a static nc file with an int16 for undefined.

Muhali <muhali>
Wed 10 Jun 2015 01:40:23 PM UTC, comment #1: 

In octave 4.0.0 and netcdf 1.0.6, I get this:


>> url = "http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/udel.airt.precip/precip.mon.total.v301.nc"
url = http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/udel.airt.precip/precip.mon.total.v301.nc
>> x = ncread(url, "precip",[1 1 1], [720 360 1], [1 1 1]);
>> xm = max(x(:))
xm =  99.530
>> max(x(x~=xm))
ans =  93
>> xmin = min(x(:))
xm =   -9.9692e+36


I get the same values for xm, max(x(x~=xm)) and xmin than in matlab R2013a.

I think that the file has changed since you reported this bug.

Can you reproduce the issue with a static NetCDF file (not to big to be attached to this report for instance)?

As far as I am aware, Matlab converts only converts _FillValues to NaN and not missing_value. Thus the minimum is equal to the missing_value in octave and Matlab.

In your example the missing_value attribute has the type float now (from ncdump -h <url>):


...
        float precip(time, lat, lon) ;
                precip:missing_value = -9.96921e+36f ;
...



Alexander Barth <abarth>
Wed 14 May 2014 12:21:30 PM UTC, original submission:  

The following  example illustrates that ncread reads data incorrect:


pkg load netcdf
url = "http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/udel.airt.precip/precip.mon.total.v301.nc"
x = ncread(url, "precip",[1 1 1], [720 360 1], [1 1 1]);
xm = max(x(:))
max(x(x~=xm))


which gives xm = 3276.7 and max(x(x~=xm)) = 99.5. The latter value is the correct maximum, as one can test with other netcdf packages (e.g. cdo).

I suspect that the error occurs because the missing value


undef = ncinfo(file).Variables(4).Attributes(3)


is not recognized as such; it is an int16 variable with value 32767, which after scaling results in the maximum value of 3276.7.

Muhali <muhali>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Updated the item)
  • -email is unavailable- added by abarth (Posted a comment)
  • -email is unavailable- added by muhali (added netcdf maintainer)
  • -email is unavailable- added by muhali (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-11-10 mtmiller StatusNone Wont Fix
        Open/ClosedOpen Closed
    2014-05-14 muhali Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code