GNU Astronomy Utilities - News
Gnuastro 0.3 released
Item posted by Mohammad Akhlaghi <makhlaghi> on Thu 01 Jun 2017 10:41:51 PM UTC.
The third (version 0.3) release of GNU Astronomy Utilities (Gnuastro) is now available.
Gnuastro is a collection of programs and libraries for the manipulation and analysis of astronomical data. All the programs share the same basic command-line user interface for the comfort of both the users and developers.
This release is a full re-write of Gnuastro. Most importantly,
Gnuastro now has a new generic data container (`gal_data_t'). This new container can now deal natively with all standard numeric data types, work in RAM or HDD/SSD, keep data in any dimensions and has enabled many other very useful features in the programs and libraries. Some of the most prominent of the new features are discussed in the NEWS file below [*]. It is strongly recommended to review the respective section of the Gnuastro manual/book for a better feeling of all the new features.
As discussed below, some program names have changed, if you have a previous version of Gnuastro installed from source, it is recommended to uninstall it first (with `make uninstall' using the corresponding tarball), then install this new version. Building Gnuastro 0.3 can be slower than previous versions, so please build in parallel with Make's `-j8' option (to build on 8 threads for example).
Here are the compressed sources:
- http://ftp.gnu.org/gnu/gnuastro/gnuastro-0.3.tar.gz (4.4MB)
- http://ftp.gnu.org/gnu/gnuastro/gnuastro-0.3.tar.lz (2.9MB)
Here are the GPG detached signatures[**]:
- http://ftp.gnu.org/gnu/gnuastro/gnuastro-0.3.tar.gz.sig
- http://ftp.gnu.org/gnu/gnuastro/gnuastro-0.3.tar.lz.sig
Use a mirror for higher download bandwidth:
Here are the MD5 and SHA1 checksums:
04f2cb5eae45725aeeae575080c835a0 gnuastro-0.3.tar.gz
aa787aaf80383519322727ff5ee02258 gnuastro-0.3.tar.lz
7b1e0f1cbfc81fd042eccbe617b23c78e9feaa6d gnuastro-0.3.tar.gz
50730e84d3c43fadccbd079e3159fd7e42b9ee3d gnuastro-0.3.tar.lz
Gnuastro 0.3 tarball was bootstrapped with the following tools:
- Texinfo 6.3
- Autoconf 2.69
- Automake 1.15
- Libtool 2.4.6
- Help2man 1.47.4
- Gnulib v0.1-1405-g6ec17baf9
- GNU Autoconf Archives v2017.03.21-99-gcfa14e9
(These are not installation dependencies, for those, please see
https://www.gnu.org/software/gnuastro/manual/html_node/Dependencies.html )
We are most greatful to Boud Roukema and Vladimir Markelov for the contributions to the code and to Lee Kelvin, Ole Streicher, Alan Lefor, Mosè Giordano, Guillaume Mahler, Rosa Calvi, Nicolas Bouché, Francesco Montanari, William Pence, David Valls-Gabaud and Christopher Willmer for the great comments, suggestions, bug reports and helps in testing.
NEWS file
New programs or library features
- Library functions that deal with datasets now use this generic data container for inputs and outputs, significantly simplifying their API. Nearly all library functions have been re-written with much more clear names, argument lists and individual purpose. Some example library functions are shown below, also see the Library demos section of the book for some complete working example:
** `gal_table_read' and `gal_table_write' will read and write data to plain text, FITS ASCII and FITS Binary formats.
** `gal_fits_img_read' and `gal_fits_img_write' can read a FITS image to memory or write a FITS image from memory.
- Gnuastro now defines a simple comment line format to keep basic information in a plain text table, see the Gnuastro text table format section of the book. In short for every column, a comment line like below can be used to give a name, units, comments, or a type to a column. This allows a FITS binary table for example to be written to plain text and converted back to binary without loosing any information (except for very small floating point errors if not enough decimals are printed).
# Column N: NAME [UNIT, TYPE, BLANK] COMMENT
- The new Fits program replaces the old Header program. But besides reading/checking FITS header keywords, it can now also work on FITS extensions/HDUs. For example with no options, it will list all the HDUs in a FITS file along with basic information. It can copy a whole HDU to another file, or delete a HDU from a FITS file. To get the previous behavior of listing all the keywords in a FITS HDU, you can run it with the `-p' option.
- All programs now write data into the second HDU of a FITS file to allow a clean first HDU. Note that following CFITSIO, HDU counting still starts from zero, so FITS images and tables written by Gnuastro in a new file can always be accessed with the `--hdu=1' option (which is now the default).
- If any program is run within a Git version controlled directory, a `COMMIT' header keyword will be added to the created FITS files, see the Output headers section of the book for a discussion on the usefulness of this new feature.
- BuildProgram: a new program to easily compile, link and run a C program you have written with Gnuastro's libraries without having to worry about which libraries (Gnuastro dependencies) your program needs. Debugging (`-g'), optimizations (`-O'), warnings (`-W'), include search path (`-I'), link search path (`-L'), and linked libraries (`-l') compiler options are also supported. BuildProgram will greatly facilitate the easy usage of Gnuastro's libraries.
Removed programs or library features
- The following program names have been renamed:
** ImageCrop --> Crop
** ImageWarp --> Warp
** ImageStatistics --> Statistics
** Header --> Fits
- SubtractSky has been removed. The Statistics program now has tools to estimate the Sky value and it can be subtracted with Arithmetic.
New features
- All Gnuastro programs that read and write tables can now do so in plain text table format or in FITS ASCII or FITS Binary tables. Depending on the filename or with the new `--tableformat' common option to all programs.
- The option management system in all Gnuastro programs has been completely re-written with many new features, some of the most important ones are listed below. For developers, you will notice that there is no more usage of macros and adding new options has become much more easier.
** All programs will now also look for a `gnuastro.conf' configuration file to keep common options for all programs in every directory.
** The `--lastconfig' opion can be used on the command-line or in any configuration file to stop parsing any further configuration files.
** The `--config' option can now be used to identify any arbitrary file to be parsed as a configuration file. Any file that is given to this option is parsed immediately.
** The `--printparams' option now also prints the short documentation of each option (same description in `--help') after its value.
- It is now possible to choose columns in tables based on column names as well as column numbers. It is also possible to search for columns based on searching in their units or comments. When using column numbers, counting now starts from 1 (one), not 0 (as before). See the new Selecting table columns section for more on these new features.
- Where relevant, all programs now accept a `--type' option that you can use to specify the numerical datatype of the output.
- With the new common option `--minmapsize', you can specify a minimum size of an array (in bytes) to store data in SSD/HDD and not in RAM. This can be instrumental when you are dealing with large datasets, or even smaller ones, but when your RAM is getting full.
- Making a log file is now optional and users have to explicitly ask for it with the `--log' option.
- Slower building of Gnuastro: Binary operators (e.g., plus or multiply) are now done in the native type of the input dataset. Doing so for all the different combinations of types, greatly slows down the initial compilation of Gnuastro (after running `make'). So for every type there is now a `--enable-bin-op-*' configure time option. When the dataset's type isn't compiled (only for the binary operators), it will be converted to a compiled type and then converted back in the end.
- Arithmetic: all operations are done in the native data type of the dataset. Until now, it would convert the data internally to double precision floating point, do the requested operation and write the data back in the proper type. So this new implementation is much more efficient.
- Arithmetic: two new classes of operators: type conversion operators to all standard types and integer-only operators (all C bit-wise operators and the modulo operator).
- ConvertType: can also print the input dataset to the command-line (`stdout'). To use this feature set the output filename to `stdout'.
- Convolve: has the `--minsharpspec' option to specify the minimum spectrum value to use in deconvolution (matching PSFs).
- Crop: when in WCS mode it can still only work on aligned images. However, very small floating point errors in writing the WCS (for example 10e-13 degrees) are now acceptable. Until now these would cause Crop to complain and abort.
- Crop: the name of the crop can be pre-determined based on the values in a given in a table column. This allows your object's IDs to be directly used as the crop's file name for example.
- MakeCatalog: also reports the surface brightness in mag/arcsec^2. Until now, MakeCatalog would only print the dataset's surface brightness in units of magnitudes/pixel. But that is not nicely comparable to other datasets. Hence, using a simple calculation (from the pixel projected size, fully derived in the book), it now also reports the surface brightness in magnitudes/arcsec^2 also.
- MakeProfiles: Profile codes now start from `1' (until now they started from `0').
- MakeProfiles: now accepts the radial function of profiles as human-readable strings instead of a code for each profile (which was very cryptic, although codes are also still acceptable). For example in the profile column you can now write `sersic' instead of the code `1'.
- NoiseChisel: the new `--cleandilated' option will remove dilated objects that have a low S/N (it is mainly useful on very clean or mock images). For non-clean noise, it will result in a decrease of completeness. With this new option, NoiseChisel will also print detection S/N values when run with the `--checkdetsn' option.
- Statistics: now reads table columns as well as images and does basic operations on them. It can also only work on a certain range of the data (instead of the whole set). Alternatively, you can define the range on another reference column, but use values of the main column.
- Statistics: all its single-valued measurements can now be done on a tessellation (tile grid) over the input dataset.
- Statistics: can now estimate the Sky value on the input dataset using the mode's quantile similar to what NoiseChisel does to find its initial threshold.
- Statistics: has several new single valued calculations: `--quantile', `--quantfunc' (quantile function), `--mode', `--modequant', `--modesym', and `--modesymvalue'.
- Warp: align the image with the celestial coordiates using the `--align' option.
- Warp: standard modular warpings can now be requested without an input matrix, using the following options: `--shear', `--flip', `--project', `--rotate', `--scale', `--translate'. Any number of these transformations (along with the `--align' option) can be called on the command-line and they will be applied in the same order to create one warping matrix. By default the WCS will also be corrected.
Changes in behavior
- Mask image options have been removed from all programs. Instead, all programs can work directly on data with blank values. So when some pixels must be masked, the Arithmetic program's `where' operator can be used to select special pixels and set them to blank. In particular bit-wise operations are now available in Arithmetic to use bit-mask images. Managing all these different choices in every program would only confuse the user (with too many options).
- Arithmetic: the `x' letter is now used to represent the multiplication operator. Previously it was `*' which needed quotation and was thus very inconvenient.
- Convolve: the old `--frequency' and `--spatial' options have been removed and are replaced by `--domain' which accepts values of `frequency' and `spatial'.
- Convolve: the old `--viewfreqsteps' was changed to `--checkfreqsteps' to fit with the general style of such check images in all Gnuastro's programs.
- Crop: `--section' syntax is now inclusive in both bounds.
- Crop: only checks if the center of a crop is filled when the crop was defined by its center (for example with `--ra' and `--dec'). The verbose outputs of Crop are also not cryptic 0s or 1s. The are human readable text.
- Crop: doesn't have separate `--imgmode' and `--wcsmode' options any more. There is now a single `--mode' option which accepts values of `img' or `wcs'.
- MakeProfiles: the old `--inputascanvas' is now called `--clearcanvas'.
- MakeProfiles: until now, it would abort with an error when the input columns had blank values. But for masking, it might happen that you set a blank magnitude. So this check has now been removed when reading the magnitude column.
- NoiseChisel: default value of the `--minskyfrac' option (new name for the old `--minbfrac') is now 0.7 as opposed to 0.5. This will allow much better estimation of noise properties (by default). It may be slightly too high for a crowded field, but the users can change it on the command-line (or in a configuration file) for such datasets.
- NoiseChisel: when it is run with any of the `--check' options, it will abort after all the check images have been created. This is very useful for checking your parameters until each step and not be distracted (or have to wait) for later steps to finish.
- Statistics: will not make a histogram and cumulative frequency files, or calculate sigma-clipped results by default (with no options). It will just print some basic information.
- Table: Previously, if a column was requested, the `-i' option would be ignored. But it often happens that the users forget a column name after already typing several of their desired columns. So the opposite behavior is preferred. Because when more than a couple of columns are needed, you will probably forget the column identifiers of the last few and having to retype everything is very frustrating. Something like how `--help' takes precedence over all other options.
- Table: to select column(s) by regular expression searching, the name now has to be put in `/ /' (similar to AWK). If a value isn't in `/ /', the programs will only select a column with the exact match.
- Warp: when a 2 by 2 matrix is given, the FITS pixel positions (which define the center of a pixel as an integer) are automatically implemented internally, see "Invoking Warp" in the manual for more.
- Warp: the old `--nofitscorrect' option has been changed to `--centeroncorner' to be more clear. The new option is now more general than before and also works on warping with a matrix, not just on modular warpings.
- Warp: the old `--nowcscorrection' option has been given a more clear name of `--keepwcs'. With this option, Warp will not apply the warp the input's WCS structure.
- Warp: the old `--maxblankfrac' option has been changed to `--coveredfrac'. Until now, Warp would only look for the fraction of input blank/NaN pixel area over the output pixel. But this would be useless on the edges of the image. So the new `--coveredfrac' option takes the acceptable fraction of output pixel area that must be covered by input pixels in order to give that output pixel a value. You can use this to set edge pixels that are not fully covered in the new grid to blank and have a flat warped image.
Bug fixes
- Using `%zu' to print `size_t' variables for clean build on 32-bit systems.
- Crash in Table for some operating systems due to memory is now fixed (bug #49347).
- Table's man-page is now created and installed (bug #49418).
- Check for malloc returning valid pointer (bug #49459).
- Segfault in mesh interpolation corrected (bug #49588).
- Corrected bad status usage in calls to wcsp2s and wcss2p (bug #49752).
- Strickter checking in Crop's polygon point list (bug #48978).
- Correction in alignment and getting pixel scale (bug #50072).
- Decomposing PCi_j and CDELTi matices in output WCS (bug #50073).
- Using image naxes[n] when checkcenter is larger (bug #50099).
- Memory leak in MakeCatalog corrected (bug #51118).
- Fix copy-paste error in MakeCatalog flag arrays (bug #51130).
Checking with signature
Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this:
gpg --verify gnuastro-0.3.tar.gz.sig
If that command fails because you don't have the required public key, then run this command to import it:
gpg --keyserver keys.gnupg.net --recv-keys 16A8A4B2AEC42AFF
and rerun the 'gpg --verify' command.
Powered by Savane 3.14-8aba.
Corresponding source code