taskGNU Astronomy Utilities - Tasks: task #14340, Defining arbitrary variable and...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #14340: Defining arbitrary variable and its value in FITS headers

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Tue 31 Jan 2017 12:09:49 AM UTC
   
 
Should Start On:  Mon 30 Jan 2017 03:00:00 PM UTC Should be Finished on:  Mon 30 Jan 2017 03:00:00 PM UTC
Category:  All Gnuastro Priority:  5 - Normal
Item Group:  Enhancement Status:  Postponed
Privacy:  Public Assigned to:  None
Percent Complete:  0% Open/Closed:  Open
Effort:  0.00

Tue 31 Jan 2017 05:17:38 PM UTC, comment #5: 

Sorry, I meant the `gal_data_string_to_number' function of lib/data.c. This function will determine the type of a number written as a string and store it automatically. So we won't need to explicitly keep the type of the variable in the header keyword. If the returned pointer is NULL, we know the value was a string and we can store it as such.

`gal_data_string_to_type' (that I mentioned in the previous comment), takes a given type and reads the string in that, so it isn't useful for this job.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Tue 31 Jan 2017 05:10:20 PM UTC, comment #4: 

Thank you very much Boud and Lee for the great comments, I personally subscribed to the fitsbits mailing list and will post issues I have with the standard there. Until now, I have tried really hard to do everything with the existing conventions and make as little change as possible. Fortunately Lee's suggestion was the solution :-).

I had forgot about the HIERARCH convention! Thanks Lee for bringing it up. Infact after thinking a little more, I noticed that as part of task #14245 (Root data structure), the new `gal_data_string_to_type' (currently last function in lib/data.c) function has been defined to read a string number into the proper data-type. So we can use it after calling CFITSIO to read the value into the proper type. The name and units of the variable can also be added to the `gal_data_t' representing the variable.

So if there is no better suggestion, we will use the HIERARCH convention. I currently don't have the time to implement it, so I will keep this task as open. In the meantime, if you have the chance to implement it, post a comment here and I would be happy to help navigating the new `gal_data_t' and all its great new features. Also, if possibly better solutions come to you, please don't hesitate to share them here.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Tue 31 Jan 2017 04:58:09 PM UTC, comment #3: 

Boud posted the following message to this task but directory to the gnuastro-devel mailing list. So I am also sending it here for the record.

It can happen that we mistkanely reply to the mail instead of following the link and posting the reply on Savannah. The categorization offered by Savannah makes navigating the discussions here much more easier than a mailing list. So in order to not miss any replys/comments in the future, I have updated the settings of the gnuastro-devel mailing list. From now on, all direct mails to that list will be rejected and an informative reply will be sent back describing the recommended way to post a reply/comment. Thank you very much Boud for bringing up this issue ;-).



Boud's message
--------------

In case gnuastro people don't know it: there's a fitsbits@ discussion
list where it might be worth coordinating some of these sorts of
questions:

https://listmgr.nrao.edu/mailman/listinfo/fitsbits

One of the recurring themes on that list is how to continue updating
FITS standards to follow or precede what really gets used in
astronomical FITS files, and to encourage people to implement the
standards. If gnuastro becomes widely used, then it may lead to
new de facto standards - so some coordination now at an early stage
might be useful.

Cheers
Boud

Mohammad Akhlaghi <makhlaghi>
Group administrator
Tue 31 Jan 2017 11:49:24 AM UTC, comment #2: 

Apologies, the link in my previous post seems to be broken. Here it is again, for reference: https://heasarc.gsfc.nasa.gov/docs/software/fitsio/c/f_user/node28.html

Lee Kelvin <leeskelvin>
Tue 31 Jan 2017 11:47:55 AM UTC, comment #1: 

Hi Mohammad,

Another option I've come across before to allow for long FITS keyword names is use of the HIERARCH convention:

https://heasarc.gsfc.nasa.gov/docs/software/fitsio/c/f_user/node28.html

This is a standard developed by ESO and fully supported by CFITSIO. Many FITS viewer applications also support it.

To quote the link above: "Basically, this convention uses the FITS keyword 'HIERARCH' to indicate that this convention is being used, then the actual keyword name begins in column 10 and can contain any printable ASCII text characters, including spaces. The equals sign marks the end of the keyword name and is followed by the usual value and comment fields just as in standard FITS keywords."

This could be another potential option to allow for long keyword names.

Best,
Lee Kelvin

Lee Kelvin <leeskelvin>
Tue 31 Jan 2017 12:09:49 AM UTC, original submission:  

It is very important for reproducibility that the programs which produce FITS images also add all their input option values into the produced extension. One of the reasons this is currently not done was that the option names can be arbitrarily long and have many types. While the FITS keywords have a strict limit on their length and there is nothing about specifying a type. So there was no robust way to keep this information.

After reading the Draft FITS 4.0 standard, where it defines `ZNAMEi' and `ZVALi' in section 10.1.2, page 44, I thought of extending that concept to solve this problem. My proposed style is like the following, for variable `i':


VNi     = 'type variable_name' / [unit] Variable i type and name.
VVi     =      3.141592653589  / [unit] Variable i value.


`VN' stands for "Variable Name", and `VV' for "Variable Value". The only requirement is that `VNi' is placed before/above `VVi'.

For the `type' we can have any of the standard C types and `variable_name' can be any name (of any length). The type of the value from `VNi' can then be used to read `VVi' into the proper data type. In Section 4.3.2 (Units in comment fields) of the same FITS 4.0 standard, a method is defined to store the units of the variable, which can be stored in one or both of the two comment sections.

A parser can be written to easily parse these and use them in programs, so for example you can say that you want to use these values/variables instead of configuration files and so you can process another input dataset with exactly the same paramters you used on another image.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by leeskelvin (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code