bugGNU Astronomy Utilities - Bugs: bug #52295, Cosmology library integrals crash...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #52295: Cosmology library integrals crash for high z

Submitter:  Mohammad Akhlaghi <makhlaghi>
Submitted:  Fri 27 Oct 2017 12:18:21 PM UTC
   
 
Category:  Libraries Severity:  3 - Normal
Item Group:  Crash Status:  Fixed
Privacy:  Public Assigned to:  alteholz
Open/Closed:  Closed

Jump to the original submission

Fri 02 Feb 2024 12:42:56 AM UTC, comment #7: 

Thanks a lot Thorsten for the nice fix; and thank you very much Boud for confirming the fix :-).

The fix has been merged into the 'master' branch of Gnuastro as Commit d6447d2b26 (I rebased it over the lastest 'master' branch and done some stylistic fixes to make it fit into Gnuastro's coding conventions). Some tips in this regard:

  • Keep lines below 75 characters.
  • We do not use any capital letters in variable names.
  • Start the comments with capital letters (as in how you write a normal English sentence: a comment is basic English, not code).


Another important tip: do not put two independent bug fixes or tasks to be merged in the same branch (this commit was placed on top of the commit for bug #46225). I had to separate your commit from that of bug #46225, since I don't have time to check both at this moment ;-).

Thanks again for the nice fix Thorsten.

Thanks also Boud for the comments regarding speed. I agree that things can be greatly improved, as you mentioned, the problem is primarily the time needed to develop this; especially that the speed of this calculation has never been our bottleneck so far; so there has always been higher priority bugs/tasks.

Ultimately, as mentioned in the main Gnuastro tutorial, if this is indeed the bottleneck for some project, they can fix it with custom program (which is easy through Gnuastro's library) ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Wed 24 Jan 2024 09:10:43 PM UTC, comment #6: 

Commit d74dbcbe on branch bug52295 looks fine to me to handle the higher redshifts:

 for z in 0.01 0.1 1 10 20 100 200 1000 1200 2000; do ./bin/cosmiccal/astcosmiccal --quiet --properdistance --oradiation=0.0 --omatter=0.3 --olambda=0.7 --H0=70 --redshift=$z; done |& grep "^[0-9]"
42.730926
418.454488
3303.828806
9440.249626
10742.296524
12598.733839
13051.764583
13660.529297
13703.558079
13805.213886


As a side comment, changing from qng to qag doesn't hurt, but there are more significant issues for calculational speed. For the extreme slowdown due to shell overhead, see task 14691: https://savannah.gnu.org/task/?14691 ; cosmdist is 50000 or so times faster, which will be non-negligible for LSST/Rubin photo-z conversions.

Boud Roukema <boud>
Group Member
Wed 24 Jan 2024 06:05:34 AM UTC, comment #5: 
Thorsten Alteholz <alteholz>
Group Member
Tue 23 Jan 2024 06:35:22 PM UTC, comment #4: 

comment #3:

> > gives me nothing. Is the site down?
>
> with this command you are querying only the A-record of that host. As this is an IPv6-only host, you need to use "-t AAAA" for the IPv6 address. As IPv6 does not seem to be widely spread here, I am moving the repository to codeberg ...


The dig -t AAAA option works fine for me. I should probably wait until your Codeberg mirror is ready, but from the command line I had a failure too (which is why I checked with dig):


Connecting to git.alteholz.dev (git.alteholz.dev)|2a01:4f9:3b:41e7::20|:443... failed: Network is unreachable.


It's certainly possible that I don't have IPv6 set up properly for wget or curl - I've assumed up to now that Debian would do this out-of-the-box when it's needed ...

Boud Roukema <boud>
Group Member
Tue 23 Jan 2024 06:11:51 PM UTC, comment #3: 

comment #2:

> On three different machines, as of around Tue Jan 23 13:40 UTC 2024 or so
>
> dig git.alteholz.dev
>
> gives me nothing. Is the site down?


with this command you are querying only the A-record of that host. As this is an IPv6-only host, you need to use "-t AAAA" for the IPv6 address. As IPv6 does not seem to be widely spread here, I am moving the repository to codeberg ...


Thorsten Alteholz <alteholz>
Group Member
Tue 23 Jan 2024 04:04:16 PM UTC, comment #2: 

On three different machines, as of around Tue Jan 23 13:40 UTC 2024 or so

dig git.alteholz.dev

gives me nothing. Is the site down?

Redshifts up to about z=1100 are in principle useful for anything to the last scattering surface of the CMB; z=200 is higher than the epoch of reionisation. Stellar mass primordial black hole mergers from prior to the population III epoch would be rather speculative. Whether we'll detect any individual objects about z=20 in the coming decades is highly speculative. But up to z=1100 is certainly within our observational sphere, it's useful to observational cosmic topology, and excluding the possibility of astronomical objects existing because "we know that they shouldn't" is unwise, given the history of astronomy. :) Bottom line: up to z=1100 is physically meaningful, yes. Beyond that the Universe is electromagnetically opaque, but it couldn't hurt to go slightly further (depending on the Omega_{*0} parameters).

Version 0.21.75-4aca of gnuastro/CosmicCalculator matches Version 0.3.12 of cosmdist for the radial proper distance. https://codeberg.org/boud/cosmdist is well-tested code that started nearly 20 years ago (2004-11-13 https://codeberg.org/boud/cosmdist/src/branch/main/ChangeLog). In principle I accepted to integrate it into gnuastro, but it rather seems complementary and I don't have the time to do that. Two particular missing features:

  • astcosmiccal doesn't accept FLRW radial comoving distance or FLRW cosmological time as an input parameter, while cosmdist does;
  • astcosmiccal does not accept input from stdin, while cosmdist does.


Going from a model distance or age to a redshift requires the inverse functions. Accepting values from stdin makes cosmdist like a typical command line tool for putting in pipes and doing rapid conversions. Anyway, that's just an aside to the bug listed here.


$ ./bin/cosmiccal/astcosmiccal --version
CosmicCalculator (GNU Astronomy Utilities) 0.21.75-4aca
Copyright (C) 2015-2024 Free Software Foundation, Inc.
License GPLv3+: GNU General public license version 3 or later.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written/developed by Mohammad Akhlaghi

$ for z in 0.01 0.1 1 10 20 100; do ./bin/cosmiccal/astcosmiccal --quiet --properdistance --oradiation=0.0 --omatter=0.3 --olambda=0.7 --H0=70 --redshift=$z; done |& grep "^[0-9]"
42.730926
418.454488
3303.828806
9440.249626
10742.296524
12598.733839


z=200 gives

gsl: qng.c:189: ERROR: failed to reach tolerance with highest-order rule
Default GSL error handler invoked.
Aborted

so the bug is reproducible.


$ ./cosmdist --version
cosmdist 0.3.12
Written by Boud Roukema.

Copyright (C) 2004-2021 Boud Roukema
This program is free software; you may redistribute it under the terms of
the GNU General Public License.  This program has absolutely no warranty.

$ for z in 0.01 0.1 1 10 20 100 200 1000 1200 2000; do echo $z | ./cosmdist -r 0 ; done
42.73
418.45
3303.83
9440.25
10742.30
12598.73
13051.76
13660.53
13703.56
13805.21


Boud Roukema <boud>
Group Member
Tue 16 Jan 2024 06:14:22 PM UTC, comment #1: 

I assume gsl_integration_qng() was choosen for the reason that it is fast. So I propose to check whether it could successful integrate and if it fails just choose a slower function. Using gsl_integration_qag() did work in all cases when gsl_integration_qng() fails, so I propose the following commit:

https://git.alteholz.dev/alteholz/gnuastro/-/commit/d74dbcbeca0f7893918189dc80dd22c1665cb973

The variable "error" needs to be renamed in order to use the error() function.

I have no idea whether the numbers for higher values of z are calculated correctly and have any physical meaning. At least up to z=2000 there is no longer a core dump ...

Thorsten Alteholz <alteholz>
Group Member
Fri 27 Oct 2017 12:18:21 PM UTC, original submission:  

The cosmology library functions use GSL's qng integration function for their integrations. However, when you run them with a high redshift (for example 200, as shown in the example below with the CosmicCalculator program which uses those library functions), they crash:


$ astcosmiccal -z200
gsl: qng.c:189: ERROR: failed to reach tolerance with highest-order rule
Default GSL error handler invoked.
Aborted (core dumped)


As the error suggests, this may be fixed by decreasing the tollerance level, but the question is this: how much can the tolerance level be decreased to give a reasonable result at all redshifts? Or maybe, should we set a varying tolerance level based on redshift?

As an observer, I hardly ever need any calculation above redshift of ~10, but I guess simulations do need it. So another question is this: should we set a limit on the highest redshift (maybe until recombination at z=1100)?

Please share your thoughts on the best way forward.

Mohammad Akhlaghi <makhlaghi>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by boud (Posted a comment)
  • -email is unavailable- added by alteholz (Posted a comment)
  • -email is unavailable- added by makhlaghi (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-02-02 makhlaghi StatusConfirmed Fixed
        Assigned toNone alteholz
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code