taskGNU Astronomy Utilities - Tasks: task #15799, Add the autocomplete feature to...


You are not allowed to post comments on this tracker with your current authentication level.

task #15799: Add the autocomplete feature to gnuastro

Submitter:  Pedram Ashofteh-Ardakani <pedram>
Submitted:  Sun 25 Oct 2020 04:00:53 PM UTC
Should Start On:  Sun 25 Oct 2020 12:00:00 AM UTC Should be Finished on:  Sun 25 Oct 2020 12:00:00 AM UTC
Category:  Development Priority:  5 - Normal
Item Group:  New feature Status:  None
Privacy:  Public Assigned to:  pedram
Percent Complete:  90% Open/Closed:  Open
Effort:  0.00

Jump to the original submission

Sun 30 May 2021 04:34:45 PM UTC, comment #33: 

Due to bug #60618, in Commit 4c01082f8, I added notices above the relevant parts of the documentation. In the notices, we highlight that TAB completion is still under development and not yet complete. I also cite this task for interested people to see the progress on this.

Just for the record, as mentioned in the commit message, TAB completion is currently only available for these programs:

  • Arithmetic
  • BuildProgram
  • ConvertType
  • Convolve
  • CosmicCalculator
  • Crop
  • Fits
  • Table

We'll continue with the rest once we find a solution to bug #60618.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Fri 14 May 2021 11:51:46 AM UTC, comment #32: 

As a small progress report, I am happy to say that the TAB completion has matured nicely and has already been fully implemented in 8 programs so far (12 more programs and scripts to go)!

The process of adding it to every program takes about an hour or two for me (which is hard to find, given my busy schedule!), so to avoid delays in testing, I have merged this branch into the 'master' branch (the latest program was Fits in Commit b947f79228).

Within the manual, a new "Shell TAB completion" title (Section 4.1.3) has been added to Chapter 4 on "Common program behavior". Also, the tutorial and general "for developers" description of the process has been improved for the current implementation in Section 12.8.

I am now using it in my daily usage of Gnuastro (for the programs that have it), and will try to find bugs and fix them. But it would be great if you (whom ever is reading this message!) can also update your installed Gnuastro and use it (I'm sure you'll love it!) and report any bugs or inconvenience you may find!

The programs that have TAB completion are the ones with an 'astPROGNAME-complete.bash' in their source directory.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 03 May 2021 12:14:04 PM UTC, comment #31: 

Indeed, modularity is the key to doing any type of project (if you later want to allow the potential for growing and scaling up).

As one further step in modularity, I have now broken the file into two: bin/gnuastro-complete.bash and bin/table/asttable-complete.bash. The first one only has generic shell functions to be used by all programs, and the second one is only for Table. In the comments of the generic file I have explained how you can easily debug in this scenario ;-).

By the way, in the previous comment I forgot to mention why I defined a new branch: because Gnuastro had grew a lot in the last two months, I just "rebased" the old branch over the most recent 'master' branch and continued developing there. Hopefully we'll merge this in shortly and no further re-basing will be necessary.

Another minor note I forgot to mention is that four new options were added to the Fits program to greatly simplify and speedup the completion (in in Commit ffc760820041). With these, instead of having to parse plain-text outputs, 'astfits' will directly give us the answers we need :-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 03 May 2021 06:03:18 AM UTC, comment #30: 

Great! You made my day Mohammad :)

I had a glimpse of the new commit and it looks easier on the eye! The formatting is pretty neat. Defining helper functions is a smart move. I am learning a lot through this process :)

Thank you for the opportunity and the encouragements.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 02 May 2021 11:41:06 PM UTC, comment #29: 

I am happy to say that I was finally able to open up some time this weekend and do a complete review of your great work Pedram!

After studying it in detail, in order to make it readable and easily scalable to all programs, I done a full re-write in the 'auto-complete' branch of CodeBerg (see the new bin/table/completion.bash).

Fortunately the rest of the job (to install it and add completions for the other programs) is very easy and since it involves Automake, I guess its easier if I do it myself (but if you are interested and have time this week, let me know by tomorrow so I don't do it).

For everyone else reading this, I should say that this is a SUPER COOL and CONVENIENT new feature! It automatically suggests FITS files, tables, HDUs, column names based on the options!

For example if you are using 'asttable' and press [TAB][TAB], it only  suggests files that have a table! Same for HDUs: if you are in 'asttable' and press [TAB][TAB] on '--hdu', you only get a list of HDUs that have a table!

Gone are the days that you had to run 'astfits' or 'asttable' to remember the example HDU or column name/number!


Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 28 Mar 2021 07:33:20 PM UTC, comment #28: 

Thanks for the encouragement Mohammad :)

Yes it looks like everything works just fine. Please let me know if you have any questions or concerns.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sat 27 Mar 2021 05:21:02 PM UTC, comment #27: 

Great! Its a really good point to discuss formattting issues independently of the actual job! Thanks for defining task #15924 for this.

About my review, I was waiting for your green light (if everything is complete from your side). From your last message, it appeared that you are still working on the task and its not ready for a review yet. If everything is ready on your side, I'll try to look into as soon as possible ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 27 Mar 2021 05:08:05 PM UTC, comment #26: 

Hi again :)

I have submitted the new task #15924 under the name "Auto-completion: Output Formatting". Now, we can focus on core of the 'auto-completion feature' here, and work on formatting in its own dedicated task :)

Mohammad, what do you think about the current status of the auto- completion feature? Should we start implementing it in other sub-programs too?

Pedram Ashofteh-Ardakani <pedram>
Group Member
Thu 04 Mar 2021 09:57:49 PM UTC, comment #25: 

Hi Mohammad!

I have pushed three new commits on the dev-auto-complete branch. Also, after lots of investigation to no avail, I started asking if it is possible to control and manipulate the bash autocomplete output formatting.

I have opened a new task to add a unit test script that checks on our completion script.

Please let me know if things are looking good :)


P.S. Commit Messages:

Auto-completion: Refactor ..._is_table function

With this commit, I replaced 'return 0' (or 1) with 'break' as it used to
be. Because we are not using the function as a boolean query. However, we
can se if there is a valid FITS or plaintext file withing the command
line by checking if the $last_table is populated: [ x$last_table != x ]

Also, I replaced an indented 'if' with 'elif' for better code readablity.

 Auto-completion: Call ..._last_table where needed

Until now, the '..._last_table' function ran everytime. This function is
responsible for populating the $last_table variable. Which means there
has been a valid FITS or plaintext table file called in the commandline.
Since we do not need to check for this everytime the script runs, I have
moved it to more specific places where we need to check the $last_table
variable. This might help reduce overhead.

Also, I noticed and removed the extra space between the command prompt
and the user commands.

 Auto-completion: Debug -i, improve performance

Until now, autocompletion checked if there is a valid FITS file or a
plaintext table available while also the '-i' or '--information' option
is called. In this case, it would print a warning message that it is safe
to press ENTER since all other options will be disabled by the 'table'
program. However, this behavior broke in previous commits. This commit
fixes the problem.

Also, previously the 'awk' command in the '..._last_table' function would
print all the words in the command line -- named as 'token'. This
includes the short and long options. The problem is that bash is later
instructed to check if each 'token' is a valid file -- [ -f $token]. As
Mohammad pointed out before, this is a computationaly exhaustive process.
So, we would only need to check tokens that we already know are not the
program options! These tokens would start with a dash '-'. Hence, I have
added a regex condition check in the 'awk' program to take care of that.
Now, it will only print tokens NOT starting with a '-' sign.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Tue 02 Mar 2021 09:08:12 PM UTC, comment #24: 

Hello again Mohammad :) I have pushed 6 new atomic commits, so you can cherry pick if need be. I will try to work on your previous suggestions soon :)


P.S.Commit Message

Auto-completion: Refactor FITS related functions

Until now, each function that had something to do with fits file had a
different approach of making sure the input file is actually a fits file.
However, Mohammad wrote a special function that checks if an input file
is an actual existing FITS file, regardless of its extension. With this
commit, I moved those functions below the '..._is_fits' function.

From now on, we should be careful to feed only valid FITS file to fits
accepting functions, or there will be confusing errors.

Also, I revised some comments on different functions and replaced some
variables with their human-readable substitutes.

Auto-completion: return value from last table

Until now, the _gnuastro_autocomplete_last_table function would not
return anything. It would just set a global variable. However, it might
be a better idea to use return values instead of breaking the loops.
Because later we can use the return value as a signal if need be.

Auto-completion: Rename boolean query functions

Until now, functions that validated if a file is FITS or a plaintext
table were named like: _gnuastro_autocomplete_x_is_extension
However, it might be a better naming style as if we were asking the
functions whether the input file is the extension we want or not as:
_gnuastro_autocomplete_is_x_extension (?) and we'd expect a 0 or 1 as
return. With this commit, I have renamed two functions accordingly.

Auto-completion: Human readable arguments

Until now, some functions just used the raw '$1', '$2', etc. to refer to
given arguments. With this commit, some of these arguments are given more
specific names to improve and maintain code readability.

Also, I moved some comment blocks out of functions to respect the coding
standards as Mohammad mentioned previously.

Auto-completion: Suggest plaintext tables too

Until now, autocomplete only suggested FITS files as input. With this
commit, plaintext files will be shown too.

Also, I have added the _gnuastro_autocomplete_list_all_valid_files
function to prevent redundant iteration over a list of files to check
for all acceptable file types.

Auto-completion: Call astfits/asttable consistently

With this commit, the auto-completion script calls for astfits and
asttable only using their stored name in the script beginning. They are
'$_gnuastro_astfits' and '$_gnuastro_asttable'.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 01 Mar 2021 09:55:12 PM UTC, comment #23: 

My pleasure Mohammad! I sure learned some useful tricks coding with you :) I pushed two commits, latest commit b001e7c272.

I smartened up the way functions look inside directories for valid FITS files. Of course, I used 'astfits' since it does its job pretty well! -- as mentioned in 'Master Foo and the shell tools' ;)

> For FITS files, you can use `astfits FILE.FITS -q` and look into the third column [...]

I guess there is no need for that. I just use the 'astfits' return value as a reference. Is this OK?

> [...] when listing options, files and directories, we separate them [...]

Nice idea! I will see what I can do ASAP :) It is fun to play around with formatting options that make the outputs easier on the eye.

> [...] It is becoming EXTREMELY USEFUL and I am really excited to be so close to implementing this in all the programs! [...]

Same here! Thank you for the fast response and helpful tweaks!

P.S. Commit Message:

Auto-completion: Faster fits parse, use 'local'

With this commit, I added the '-q' option to 'astfits' program so it will
print less information, hence, it looks might be a bit faster.

Since we can use the 'local' declaration, I removed the 'unset' at the
end of functions and declared their variables as 'local' instead.

Also, there was a misleading typo in comments about return value of the
_gnuastro_autocomplete_plaintext_is_table function. It said the function
would return '1' if the given file is a table, on the contrary, it
returns '0', as it should by the conventions.

Auto-completion: Smart list fits files, fix var

Until now, the function responsible for listing the fits files only
checked if the file extension ended with 'fits'. This is not a rebust
test, because other fits files -- that might have different extensions
or no extensions at all -- would be left out.

With this commit, I used 'astfits' program return value to check their
validity regardless of their name or extension. However, this has a bit
of overhead and slows response. We can ignore it if it is not that
significant, and will not ruin user experience.

Also, I replaced a hard-coded value with intended $inputfile variable.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 01 Mar 2021 01:18:46 AM UTC, comment #22: 

Thanks a lot Pedram, I made some modifications in Commit 2f43d29151f (message in P.S., this was a good idea!)

In the case that '--information' is called before a file is given, we do indeed want to list all possible tables. But don't forget that tables can also be plain-text files (not just FITS). So we need a new function (maybe called '_gnuastro_autocomplete_file_is_table'). Can you implement it? All its components are already there ;-).

In short, we want to get a listing of all the files in this directory and pass them through this function so we only suggest files that are tables. The '_gnuastro_autocomplete_plaintext_is_table' function will already do the check for non-FITS files. For FITS files, you can use `astfits FILE.FITS -q` and look into the third column, if it has a 'table' in any of the rows/HDUs, you can show that file, otherwise, don't bother showing it!

This will be SO USEFUL! When we have a directory with image and table files, when we call 'asttable' and press TAB, we only see the tables :-D!

Another suggestion: after playing a little with it, I feel that mixing options and file names is very annoying! Now that we can print our own terminal prompt, can you implement a feature like this: when listing options, files and directories, we separate them like this:

astfits <TAB><TAB>

a/    b/    c/

a.fits  b.fits c.fits

--column --information --quiet ...

We can use the 'column' command to sort the outputs into columns internally. For the titles, we can also give different shell colors and make them bold. This will allow us to remove the empty lines. Ofcourse, when no directories, files or options are present, we can remove the title also.

It is becoming EXTREMELY USEFUL and I am really excited to be so close to implementing this in all the programs!

P.S.Commit message:

Auto-completion: Minor polishing of the warning message

Until now, the warning message for the '--information' option was a single
string that was repeated in two places (adding a check for it in general
was a good idea Pedram!). However, based on previous experience, this can
cause bugs if we later edit one of them and forget to the edit the other!

With this commit a new 'infowarning' variable is defined at the top of the
function and message is put there. That variable can now be used anywhere
in the code, and all the time it will have the same value.

Other changes with this commit:

 - When the 'last_table' variable is not empty, the existance of its file
   has already been checked. So we should just check if its an empty string
   or not. Don't forget that checking the existance of a file is
   computationally expensive (requires Bash to leave its internal
   evironment and go to the operating system!).

 - In my case the previous '$' wasn't enough to print the prompt. The new
   line simply started with a '$' sign, not my standard prompt. So I added
   '${PS1@P}'. As mentioned above it, there is a portability issue here
   with older Bash systems. We should find a more portable solution.

 - I removed the extra '-----' lines: they are distracting and waste a lot
   of valuable terminal lines ;-)!

[To help in future reference to commits related to auto-completion, let's
start the commit title with 'Auto-completion'.]

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 28 Feb 2021 09:38:11 PM UTC, comment #21: 

Hi Mohammad! OK, I pushed the commit 968bb7c07b and changed the syntax a bit. Now it should be more clear :)

> [...] the user's prompt is no longer printed [...]

It should have printed the prompt in a new line :/ I used the command below to print the prompt:

printf "%s" "$COMP_LINE"

A sample output:

[pedram@ardakani table]$ asttable gaia.fits --information
The '--information' (or '-i') will disable all other options. You can safely press ENTER now.
$ asttable gaia.fits --information

Do you want me to print the part before '$' too? It is stored in '$PS1', I can look it up and see if we can prepend it to the $COMP_LINE.

Would you mind just checking it again and letting me know if it works?


=== Commit Message: ===

/bin/table/completion.bash: Refactor code

Until now, I used a case inside an 'if' statement. The syntax looked
pretty confusing and redundant. In this commit, the completion script
will scan the entire prompt for '-i' or '--information' option. If it
finds any instances, it will check if there is a valid fits file
specified. If both cases are TRUE, the completion will print a message
and let the user know that s/he can press ENTER safely.

Also, I declared a new _gnuastro_autocomplete_print_message function.
This function will take only one argument, and pretty prints it. Then,
it will reprint the user prompt with a '$' at line beginning for a
natural and familiar experience.

Last but not least, I used 'grep' to scan the prompt. This is POSIX
compatible and easier to implement when there is more than one pattern
to check.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 28 Feb 2021 07:34:23 PM UTC, comment #20: 

Thanks Pedram, I just pulled the changes and tried it. The problem is that in this mode, the user's prompt is no longer printed and the command stars at the beginning of the line! This is not what people expect.

Can you look into this and find a way to print the warning and then print the full prompt with the existing command?

Another point: Using the 'if' statement before the 'case' statement, only to check for something that is also checked later, makes the code very hard to read and un-elegant generally ;-).

We already have a case statement for '-i' or '--information'. So we should keep all checks related to this option under that case check (the same way it originally was).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 28 Feb 2021 05:39:49 AM UTC, comment #19: 

Oh I got caught up in replying comment #17 and forgot about comment #16. Sure! I will do it in a separate task soon :-)

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 28 Feb 2021 05:35:59 AM UTC, comment #18: 

Hi Mohammad, thank you for the swift response and code check!

I like the way you used the 'astfits' program instead of 'reinventing the wheel' ;-) Great!

There you go, problem solved in Commit 32084abdd7 ;-)

Just a tip, printing messages using 'printf' instead of 'COMPREPLY' might be a better approach. It can prevent confusions in the future. It might be better to distinguish communicating with the user (using 'echo', 'printf', 'read', etc.) and providing autocomplete suggestions (using 'COMPREPLY').

More explanation in the commit message. Please let me know if you have any questions :-)

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 28 Feb 2021 03:29:40 AM UTC, comment #17: 

Hi again Pedram, I made some good progress with the script, but I a problem that didn't let me progress fast and I didn't have time to figure out its solution :-(!

The problem (and all the changes I made) is described in Commit e18d46e476.

When you get a chance, can you look into it? I think you'll be able to find the solution to it pretty fast! This issue (of printing a warning when there is an inconsistency) would be a very good feature to have.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 27 Feb 2021 04:08:59 PM UTC, comment #16: 

I am happy you enjoyed the comments, a good work will have good feedback ;-).

I like your idea on adding some of these points within the "Developing" chapter. Since you have read it, can you find a good place to put them? I am too busy to focus on this right now :-(.

Since these tips are general, can you do this over a new branch on the most recent commit of the 'master' branch?

Since the GSoC application period is approaching, these tips can be very useful for new people who want to apply this year (and many people will be grateful to you ;-)).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 27 Feb 2021 06:59:45 AM UTC, comment #15: 

You made my day Mohammad! Thank you for the wise and informative feedback :) Mentioning the references points enthusiastic people - your truly ;-) - to the right direction. I am having a good time reading them. Thank you so much.

In fact, the Commit 69208a5aeb9ca64a taught me a good deal about both GNU conventions and Emacs shortcuts for better TexInfo content management. Maybe they can help other developers too! How about adding them to the GNU manual? Since we should respect the conventions and make sure every piece of code follows them anyway.

OK, I will continue the work on https://codeberg.org then :) Please let me know if I could help.

Cheers ^_^

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sat 27 Feb 2021 03:13:25 AM UTC, comment #14: 

I just went through the newly added section in the book and REALLY ENJOYED IT! Wonderful work Pedram. I really recommend the tutorial that Pedram has written to anyone using the shell and would like to customize their shell completions (even outside of Gnuastro!).

I just made Commit 69208a5aeb9ca64a over your branch in Codeberg with some minor edits in the book (see the commit message for full explanation on every change). In fact, I also added you as a member of this Gnuastro development fork on Codeberg, so its best that you commit to this repository/branch directly yourself from now on too ;-).

About this branch, you are the manager of this branch on Codeberg from now on. So feel free to commit to this branch on Codeberg, I'll send merge requests for you if parallel work comes up.

I will try to go over the actual implementation of the autocomplete features in the next few days and will post the results here.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sat 20 Feb 2021 08:25:31 PM UTC, comment #13: 

Hi Mohammad!

Thanks for letting me know :) I got this far in the autocompletion code and the documentation.

I am looking forward to hearing your thoughts on this. Can't wait to have the autocompletion feature implemented in the entire Gnuastro!

It would be great if you could let me know the autocompletion is working as expected in a Mac device too. Because I am a bit skeptical about the 'compopt' command. I will just use another approach if 'compopt' does not work in Mac.

Please let me know if you have any questions or concerns.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 08 Feb 2021 12:41:50 AM UTC, comment #12: 

Great work Pedram.

> It looks like arrays are not supported in POSIX.

No problem, the completion we are designing is only for Bash anyway. Things like Dash don't have completion (as far as I know), and really have no idea how other shells (like zsh) treat completion. So you just have to assume that the feature exists in a certain version of Bash.

Its fine to assume Bash 3.2! We can just add some checks at configure time and if certain features aren't available in the user's bash (or the user's shell isn't Bash at all, like on Debian systems that use Dash: a minimal shell), we won't activate autocomplete, and just print a warning. Don't worry about the test in './configure', I'll add it ;-)..

Of course, the same applies to 'compgen'. It is available in Bash, and that is the only important thing.

So you don't have to worry about POSIX here ;-).

The links you send were very good! I am sure I'm going to come back to this task a lot for those links ;-).

I'm a little busy in the next day or two, but I'll try my best to test the completed work! Very good job! In the meantime, you can define a new section in the Developing chapter of the book to explain the basic concepts. I'll start reading from that (as a new developer), then I'll look into the code and commits ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 07 Feb 2021 07:26:35 PM UTC, comment #11: 

Great news! check commit f8425b57..75b02a5d. I can not access gitlab right now, so please help yourself with the last commit on 'dev-auto-complete'. Try it out and let me know Mohammad :) Just download the 'completion.sh' file, and call 'source completion.sh'. Let me know if you have any tweaks in mind :)

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 07 Feb 2021 09:09:06 AM UTC, comment #10: 

I read a few articles about portability to Mac and previous versions. Keeping them here might be helpful.

This can be really confusing ... Should we keep things compatible with bash-3.2 [https://tiswww.case.edu/php/chet/bash/COMPAT (which is
still the `standard' version for Mac OS X)] or even older ones?

Pedram Ashofteh-Ardakani <pedram>
Group Member
Sun 07 Feb 2021 08:32:28 AM UTC, comment #9: 

Hi Mohammad. Speaking of portability, I stumbled upon the program: shellcheck. This program shows warnings in case of any error or POSIX incompatibility. It looks like arrays are not supported in POSIX. Is it OK if I use them in autocompletion or should I find a replacement? E.g. using awk, known shell workarounds, etc.

Also, there is a warning on compgen, a bash built-in function for autocompletion. I guess I can replace this too. I am just wondering how much should I worry about POSIX compatibility and  portability.

Your help always keeps me motivated and on track. Thanks in advance.

=== Helpful links ===

  • You can install flycheck to embed shellcheck in emacs from the MELPA repository. A simple tutorial here.

M-x package-install RET flycheck

Pedram Ashofteh-Ardakani <pedram>
Group Member
Fri 05 Feb 2021 02:39:12 PM UTC, comment #8: 

As Mohammad pointed out, it is better to submit a new task for using the autocompletion in emacs. Now things are simpler, and we can focus on tasks one at a time.

I agree with you on portability Mohammad (comment #6)!

Thanks for keeping things simple, yet effective ^_^

I will keep you posted. Cheers.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Thu 04 Feb 2021 03:39:26 PM UTC, comment #7: 

As Samaeh pointed out, some Gnuastro users write their scripts in emacs. So it would be nice if the completions could work there too. Fortunately, that is possible already: https://github.com/szermatt/emacs-bash-completion

Pedram Ashofteh-Ardakani <pedram>
Group Member
Thu 04 Feb 2021 12:56:40 PM UTC, comment #6: 

This is very good progress, thanks Pedram ;-)!

Also, the link was interesting, thanks! I'd love to read it in detail but I am too busy these days :-(! I trust that you have read them and will follow the best practices ;-). But IT IS GREAT to have the links here for future reference, so please continue sharing relevant best practices here.

Another issue just occurred to me: the version of the host's Bash. For example I know some recent macOS systems that come with the Bash of 2007!!!! So to help in portability of the solution, please also check to see from when Bash completion was available. And within the functions/scripts, we should also avoid using too many recent shell constructs to help in the portability of this work.

Let me know when the full option set of the Table program is complete and I'll try it out and hopefully expand it to the other programs ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 04 Feb 2021 12:45:54 PM UTC, comment #5: 

I had some progress with the completion script in this commit. As we are thinking of more complicated output and helpers, this article might be useful: https://brbsix.github.io/2015/11/29/accessing-tab-completion-programmatically-in-bash/

Pedram Ashofteh-Ardakani <pedram>
Group Member
Wed 03 Feb 2021 02:06:59 AM UTC, comment #4: 

I just came across this project: https://github.com/scop/bash-completion

It has autocomplete features for many common command-line programs. For example here is the Bash completion they have designed for Make: https://github.com/scop/bash-completion/blob/master/completions/make

Of course, they clearly aren't fans of writing comments (which is bad!). But there must be some good lessons to be learnt from their various solutions, or good conventions they may have adopted that may be useful in our scenario also.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 28 Jan 2021 03:55:00 PM UTC, comment #3: 

Hi Mohammad, Thanks for the commit c826434952!

I have made some progress in Commit 2b52b3d3 setting up a bash completion script. For the time being, this works specifically for the CosmicCalculator program.

We can build up on these atomic commits step by step with the aim of creating an `auto-generated` bash completion script for the entire Gnuastro!

As you pointed out, `we can extract the list of HDUs or column names to show to the user` for the next step.

Pedram Ashofteh-Ardakani <pedram>
Group Member
Wed 27 Jan 2021 10:09:46 PM UTC, comment #2: 

As one step towards this, in Commit c826434952 the options that accept FITS (or FITS or TXT) are now marked as 'FITS' or 'FITS/TXT'.

This can help the auto-complete script to see what types of files it should show when a user presses TAB.

For the other string arguments (like HDU names in FITS files or column names in tables), the auto-complete script can call 'astfits' or 'asttable' and extract the list of HDUs or column names to show to the user.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 25 Oct 2020 09:39:05 PM UTC, comment #1: 

This is indeed a very useful feature! Thanks a lot Pedram for suggesting it.

If you can get familiar with the completion concept and commands, you can write a "completion script" for one of the Gnuastro programs. The process for the other programs will be very similar.

We can then easily write a script to be included in Gnuastro's building steps (in particular in 'doc/Makefile.am') to automatically generate this script for all the programs when the developer is building Gnuastro. We can use the output of the '--help' option to do this, or at a lower-level, go and parse all the 'args.h' files of the programs.

In fact, 'doc/Makefile.am' already uses the output of '--help' to automatically construct the Man pages. Like the Man pages, the Bash completion scripts can also be installed in 'doc/Makefile.am'.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 25 Oct 2020 04:00:53 PM UTC, original submission:  

Using autocomplete (using the <TAB> key) not only helps write commands faster, but also reduces the chance of making typing mistakes, and helps users explore all the options available.

I looked up the way one can achieve this feat, and found links and tutorials similar to Bash Programmable Completion Tutorial.

It might be a good idea to visit the GNU Programmable Completion Builtins too.

Pedram Ashofteh-Ardakani <pedram>
Group Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by makhlaghi (Posted a comment)
  • -email is unavailable- added by pedram (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.


    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-05-02 makhlaghi Percent Complete30% 90%
    2021-02-28 makhlaghi Assigned toNone pedram
    2021-02-27 makhlaghi Percent Complete0% 30%
    2020-11-27 pedram Carbon-CopyRemoved makhlaghi -
    2020-10-25 pedram Carbon-Copy- Added makhlaghi

    Back to the top

    Powered by Savane 3.13-1277.
    Corresponding source code