taskGNU Astronomy Utilities - Tasks: task #14300, Option management using new root...


You are not allowed to post comments on this tracker with your current authentication level.

task #14300: Option management using new root data structure.

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Wed 04 Jan 2017 03:19:03 PM UTC
Should Start On:  Tue 03 Jan 2017 03:00:00 PM UTC Should be Finished on:  Tue 03 Jan 2017 03:00:00 PM UTC
Category:  All Gnuastro Priority:  5 - Normal
Item Group:  Enhancement Status:  Done
Privacy:  Public Percent Complete:  100%
Assigned to:  makhlaghi Open/Closed:  Closed
Effort:  0.00

Mon 16 Jan 2017 03:22:25 AM UTC, comment #4: 

This task has been completed and is now pushed onto my development fork. This was a major update and makes managing of the options and configuration files much more easier than before.

There are no longer any more macros, and there is only one instance of every option's short or long identifier string. New options can now be added very easily and more generally, it is much more easier to define new programs.

If you want to work on new programs/options, please pull this branch and base your work on this branch until it is merger with master, do not work on the master branch. Things are so much more clear and managable now :-)...

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 08 Jan 2017 02:05:27 AM UTC, comment #3: 

This was a very intriguing thought, and since I am going through the programs for using `gal_data_t', I thought it would be very useful if this system is adopted also, so the two jobs can be done in one round of learning/remembering how the programs work!

After thinking about it, I came up with a better solution that I have just pushed to my development branch. I think I came up with a better solution that what was suggested in previous comments on this task. They needed to define a new array in parallel to `argp_option' and its management could its self cause bugs/management-nightmares.

So now, we edit the definition of `argp_option' from `argp.h' during bootstrapping. An AWK script was included in `bootstrap.conf' to add the new `void *value', and `void *type' elements to this structure in `argp.h' after all the source files are imported from Gnulib. As a test, I implemented it in the Table program, by setting values for these elements in `args.h' and it is compiling with no errors.

So the new data structure only came in handy by giving us a standard way to work with types/void-*, but nevertheless was still instrumental in the inception of this approach. 

Also, defining a flag for every option as suggested bellow would make the flags too much and hard to manage. The library will just read the values, store them in the appropriate type, and check if all values are given, Then the program's `ui.c' will go over them and put them into the respective element of the main program's data structure. During this phase, it can also check the validity of their values (for example if the number must be positive, or less than 1 and so on).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Wed 04 Jan 2017 11:58:22 PM UTC, comment #2: 

I just noticed that since we are using GNU C Library's Argp for parsing of arguments, we can use the each programs' argp_option array (and the `gal_commonargs_options' in `lib/commonargs.h' for the common options) to look up all the long option names automatically. 

Since the `argp_option' array is statically defined, we can easily get the number of elements in it with ((sizeof options)/(sizeof options[0])) and parse over them to pull out the option names that were not set on the command-line and put them in a list to pass onto the library function as suggested below.

This will make each program's `args.h' the only place where the long option names are mentioned, greatly simplifying the process of reading in options. With the programs essentially shrinking to library calls, this will make the creation of new programs or management of existing ones an order of magnitude more easier than before and most importantly the code will be much more elegant.

I don't have the time to do this now, so if you are up for the challenge, you are most welcome to start working on it, I would be very happy to help ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Wed 04 Jan 2017 03:38:59 PM UTC, comment #1: 

Building further on the suggested process, we can tell the library function how to interpret the values (for example keep the string value, or convert it to integer, or float) to the options with the `type' element of `gal_data_t' and define bit flags in its `status' element for further checks like a requirement that the value be larger than zero, or should be less than 1.

Another bit-flag can be added to let the configuration file parser know that the option shouldn't be popped from the input linked list. This can be used for options that can accept multiple occurances when multiple occurances, like the as in column indexs/labels in the Table program.

The final value to the option can also be stored in the `array' element of `gal_data_t', and when multiple occurances of an option are acceptable, we can increase the array size and put more elements into it as they are read.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Wed 04 Jan 2017 03:19:03 PM UTC, original submission:  

With the expansion of Gnuastro and its libraries (which will centralize many operations), the number of common options will increase. So it would be much more easier for the gnuastro users to set the values of common options only once for all of Gnuastro's programs (when they want to).

One example maybe that you want your work to be reproducible, and so all the programs must run with the same version fo Gnuastro, and you want them to only use the current directory configuration file. To do this, you have to manually add the two `onlydirconf' and `onlyversion' to the configuration files of all the Gnuastro programs that you use.

This can be buggy and cause errors, for example, during your research (before you publish/release the reproduction pipeline), a new version of Gnuastro comes and you want to use it instead, this step of checking/changing all those configuration files can be hard and a cause of avoiding this update.

There are many other examples like log files, or (later) printing/file formats for programs that output tables.

So here, I am suggesting to add a `gnuastro.conf' configuration file which will only be used in reading common options to all the programs. Its precedence would be lower than the configuration file for that utility in the same directory.

It would also be great if as part of this transition we could also re-write the way each program's configuration files are read, and values are checked. Currently each program does this in its own `ui.c' file and uses macros a lot. There might be better/cleaner ways to do all this in the library so the programs just have to call the appropriate function.

This might be possible with the new root data structure (task #14245), in a similar way to how we read table columns (see `gal_table_read' in the new lib/table.c). In short, each program makes a linked list of `gal_data_t' with the `name' field set to the long option name (short options are only for the command-line). This linked list is passed to the library function along with the program's name (to determine the name of the configuration file).

The library function will then start looking for the directory, user-wide and system-wide configuration files and fill in the option values as they are read from the lines. Every option that is read will be popped from the input linked list (and added to the output linked list) so any future occurance of that option (in lower precedence configuration file) is never checked (since it is no longer in the list that is searched).

This will also remove the need for the `*set' variables in the `uiparams' structure of each program's `main.h' file. Because the options that have been set on the command-line will not be in the linked list that is passed onto the library function for reading configuration files. This can be a good way forward to greatly simplifying the process of option management in Gnuastro's programs and simplifying the life of developers/users :-).

Mohammad Akhlaghi <makhlaghi>
Group administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-01-16 makhlaghi StatusPostponed Done
        Percent Complete10% 100%
        Assigned toNone makhlaghi
        Open/ClosedOpen Closed
    2017-01-08 makhlaghi Percent Complete0% 10%
    2017-01-04 makhlaghi Summarygnuastro.conf for common option, better .conf file/option management Option management using new root data structure.

    Back to the top

    Powered by Savane 3.12