taskGNU Astronomy Utilities - Tasks: task #14299, Not writing NaN in ASCII (plain or...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #14299: Not writing NaN in ASCII (plain or FITS) tables

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Wed 04 Jan 2017 12:01:24 PM UTC
   
 
Should Start On:  Tue 03 Jan 2017 03:00:00 PM UTC Should be Finished on:  Tue 03 Jan 2017 03:00:00 PM UTC
Category:  Libraries Priority:  5 - Normal
Item Group:  Enhancement Status:  Postponed
Privacy:  Public Assigned to:  None
Percent Complete:  0% Open/Closed:  Open
Effort:  0.00

Wed 04 Jan 2017 12:01:24 PM UTC, original submission:  

While I was working on reading and writing of FITS ASCII tables, I got in touch with William Pence (author of CFITSIO) for some issues I had and fortunately with his guidance they were all fixed.

One issue that he raised was that of printing NaN values in an ASCII table and that some compilers cannot correctly read ASCII printd `nan' (or `NaN') as a floating point number (IEEE 754's NaN). So he suggested using some extreme floating point number instead and also noted that CFITSIO comes with `FLOATNULLVALUE' and `DOUBLENULLVALUE' for this purpose.

So I am opening this task as a discussion (and hopefully later, implementation) to account for this issue. One way that comes to my mind now is that we can add an argument/flag to the table producing library, that when the table is in text format (FITS ASCII or plain text), it doesn't print NaN values, but uses extreme numbers. Since those numbers will be included in the column's meta-data (both in FITS ASCII and also plain text, see below), the readers won't have any problem.

I had a look at the portability issues with strtod in Gnulib which confirms William Pence's point and the many issues different systems/compilers have with reading floating points from printed characters. So in order to avoid all these problems in Gnuastro (when compiled on those systems), Gnulib's `strtod' module is now also imported into Gnuastro during bootstrapping.

About FITS ASCII tables, I don't think there will be a problem, because according to the FITS standrard, the `TNULLn' keywords for FITS ASCII are strings, not numbers (and all the columns have a fixed width), so the column string can be compared with the string given to the `TNULLn' keyword before it is read as a number, so the string can be anything (including `nan').

So the problem is mainly in plain text tables, and thanks to Gnulib, Gnuastro won't have many of the problems other programs might have. But still, people generate plain text for portability, not to use with one particular program (like Gnuastro), so we need to give them the ability to account for this deficiancy in other software.

Until we find a solution, people who can't read `nan' values can do use tools like SED to chance all occurances of `nan' in the plain text tables to some out-of-range number they like.

Just as a side note, Gnuastro's plain text table format is now fully described in the Gnuastro text table format of the book. Just note that since this version of the book is not yet released, this link points to a line number in the source, which might change with updates to the book. So search for the section name if it has changed.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code