taskGNU Astronomy Utilities - Tasks: task #15912, VOTable input-output

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #15912: VOTable input-output

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Sun 07 Mar 2021 02:39:52 AM UTC
   
 
Should Start On:  Sun 07 Mar 2021 12:00:00 AM UTC Should be Finished on:  Sun 07 Mar 2021 12:00:00 AM UTC
Category:  Libraries Priority:  5 - Normal
Item Group:  Enhancement Status:  None
Privacy:  Public Assigned to:  None
Percent Complete:  0% Open/Closed:  Open
Effort:  0.00

Tue 09 Mar 2021 07:09:21 PM UTC, comment #3: 

I'm not sure to understand what is a "MACRO" in the first item about semantics in comment #2 ?

Yes a PARAM is like a FITS keyword — the problems with FITS keywords (other than the standard ones described in the FITS documents) is that they lack a lot of details (units, explanations, possibility of arrays, precision, etc.).

About parsing a VOTable in C, a colleague wrote a set of functions based on libxml2 which seems to work well − I should check whether it could be reused; it is rather small (1000 lines) but it only  deals with TABLEDATA (pure xml) serialization. On the other hand, the XML structure is simple enough to envisage a dedicated code for VOTable without the complex machinery of libxml2...

There is also the code in the voclient by Mike Fitzpatrick: https://github.com/mjfitzpatrick/voclient/blob/master/libvotable/votParseP.h

Cheers, François










Francois Ochsenbein <fochsenbein>
Sun 07 Mar 2021 09:00:02 PM UTC, comment #2: 

Thanks a lot for the great feedback François. Indeed, I really look forward to expanding the elements within 'gal_data_t' to preserve the rich metadata of VOTable. Hopefully as we start implementing VOTable, we'll see what is necessary at each level and implement it.

Here are some brainstorms on each point that just occurred to me and I am adding here only for the record (may be useful later!). We should discuss them each in separate tasks during the implementation phase of VOTable (tasks that spin-off from this one).

  • A semantic element like IVOA's Unified Content Descriptors (UCD) is indeed a great point! We can define headers to associate MACROs for each of them. And even define it as a two-element integer array ('int semantics[2]'): the first element will have the macro of the standard (for example UCD or UCD1+ in IVOA, but also allow adding other semantic standards in the future), and the second can be the macro for the the semantic type of that particular dataset.


  • About the grouping of columns, it is very good to think about a good way to implement it. I also loved this feature when I read the VOTable standard and already envision so many usage examples of it (in the 'asttable' or 'astmkcatalog' programs). Regarding its implementation, we may be able to define higher-level structures and not necessarily need to touch the low-level 'gal_data_t'). But we'll have to go into practice and see it it evolves ;-).


  • Linking columns (to extract the data from a URI when they are actually necessary) is also a very good feature to implement! We can use all the good new functions that you are currently adding to the library for that, thanks a lot for them ;-).


  • Adding parameters for columns is also interesting to consider. Currently for FITS files (where "keywords" play the role of VOTable's "parameters"), we only read the necessary keywords as input and each program writes its own new keywords in the output. I like how CFITSIO deals with keywords: you can read/write/delete/update them independent of the data in a FITS file. So on first thought, probably we don't need high-level parameters in the low-level 'gal_data_t': we can add functions to read/write/delete/update certain parameters from the VOTable based on the program that wants to use the input data, and write its own output data. But again, this is just my first thoughts now, let's get our hands dirty and see how it evolves in practice ;-).


Adding VOTable features in Gnuastro will be a very productive step forward, it is an honor to do it with your help and guidance as the author of the standard :-). Thank you François.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 07 Mar 2021 08:00:00 PM UTC, comment #1: 

The Generic data container will likely have to evolve to ensure that metadata are not lost between reading a VOTable and re-writing it ; for instance the UCD would have to be added, but also the groups of columns, the references linking columns & parameters ; I did not yet have a deep look at the section 10.3.6 which details this, but I will !

Francois Ochsenbein <fochsenbein>
Sun 07 Mar 2021 02:39:52 AM UTC, original submission:  

The Virtual Observatory's VOTable format is an XML-based format for storing and sharing table data. It is one of the core formats used in the Virtual Observatory and thus heavily used in astronomy (for example see task #15911).

We should add features to read each necessary column of a VOTable into Gnuastro's Generic data container, very similar to what we do with plain-text tables or FITS tables.

There are several XML parser libraries that we can use, like libxml2 or expat.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by fochsenbein (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  • -email is unavailable- added by makhlaghi
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-03-07 makhlaghi Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code