bugGNU Astronomy Utilities - Bugs: bug #63171, Strange patterns on warped images

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #63171: Strange patterns on warped images

Submitter:  Raul Infante-Sainz <infantesainz>
Submitted:  Thu 06 Oct 2022 03:59:57 PM UTC
   
 
Category:  Warp Severity:  3 - Normal
Item Group:  Output not reasonable Status:  Fixed
Privacy:  Public Assigned to:  pedram
Open/Closed:  Closed

Jump to the original submission

Mon 24 Oct 2022 08:00:41 AM UTC, comment #19: 

Hi all 👋

We've made the new commit [1] which discusses where these Moire patterns come from, and some ways to mitigate/remove them in the 'Moire pattern and its correction' section under 'Warp'.

[1] https://git.savannah.gnu.org/cgit/gnuastro.git/commit/?id=b58b49872b

Thanks to Mohammad's nice example demonstration in the Book [2] and the new '--checkmaxfrac' option in Warp, we have a way to visually inspect the effect - see (file jplus-warp-reg.jpg) and (file jplus-warp-reg-maxfrac.jpg), and quantify it by interpreting the 'Statistics' of the 'MAX-FRAC' HDU which is added to the Warped image if '--checkmaxfrac' is given, see [2].

[2] https://www.gnu.org/software/gnuastro/manual/html_node/Moire-pattern-and-its-correction.html

Also, we can see that stacking the 'dithered' images helps resolve this issue in the most precise way, potentially even improving the spatial resolution of the image. It is worthy of note that stacking 'dithered' images can create deeper images - see (file jplus-stack.jpg).

Please note that scales and limits are not exactly the same in the images attached here, I tried to make them look similar. But you can make all of them yourself following the instructions in [2].

Hopefully now things are making more sense. Please feel free to let us know and continue the conversation if anything comes up.

Best,
Pedram


Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 17 Oct 2022 04:25:47 PM UTC, comment #18: 

While I was talking about this feature (Moiré pattern) with David Muniesa (at CEFCA), he mentioned that the SWarp manual also talks about it in Figure 7 on page 20 (of the latest version from December 17, 2010). The proposed solution there is also oversampling!

Pedram, do you have any thoughts on how to reduce/improve it without requiring an oversampling (which is very expensive in CPU and RAM usage)?

Mohammad Akhlaghi <makhlaghi>
Group administrator
Sun 16 Oct 2022 04:29:20 PM UTC, comment #17: 

Hi there everyone, I've added the '--pixareaonwcs' option as Mohammad suggested. Here is the result for the two images you shared here, the JPLUS 'm51.fits' and Raul's 'img.fits'.



You can clearly see the gradients are because of the different area each pixel takes in the image (despite of the unique pixel scale given).

Pedram Ashofteh-Ardakani <pedram>
Group Member
Wed 12 Oct 2022 09:18:19 AM UTC, comment #16: 

Nice catch Raul! I forgot about '--envseed' 🤦

Here's a cleaner version of the previous code. Now you can easily read the image properties from environment and use them to Warp or SWarp the image. Of course this is just for quick analysis, and you should be careful only to run these when environment variables are set. If not, they will produce errors (before doing anything).


# Set the random seed for arith noise, don't forget to use '--envseed'.
export GSL_RNG_TYPE=ranlxs1
export GSL_RNG_SEED=1665844464

# Set pixel scale, image size, and center values for Warp and SWarp.
CDELT=0.555594  # in arcsec
WIDTH='700,700' # in pixels
CENTER='109.5309438,40.17599612'

RUN_SWARP_ON_TO ()
{
  swarp "$1" -IMAGEOUT_NAME "$2" \
        -SUBTRACT_BACK N -IMAGE_SIZE $WIDTH \
        -CENTER_TYPE MANUAL -CENTER $CENTER \
        -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE $CDELT
}

RUN_ASTWARP_ON_TO ()
{
  astwarp "$1" --width $WIDTH --widthinpix \
          --center $CENTER --cdelt ${CDELT}/3600 -o "$2"
}

REMOVE_PV_FROM_TO ()
{
  cp "$1" "$2"

  # Remove all PV keywords from file
  PKEYS=$(astfits "$1" -h1 --printkeynames | grep --color=never ^PV)
  for pv in $PKEYS; do
    astfits "$2" -h1 --delete $pv
  done

  # Update the projection from TPV to TAN
  astfits "$2" -h1 --update CTYPE1,RA---TAN --update CTYPE2,DEC--TAN
}


I also found out using crop will produce very tiny dot patterns on the image. The difference is very small and negligible. But it is there. They look like Fourier transform side-effects, see file 'ones-pv-warp.jpg'.

To check this, I used the original 'ones.fits' image with a 700 by 700 crop of image center 'crop/ones.fits', and a crop of the original image size (to check if indeed astcrop is making that pattern or if it is a side-effect of working on a smaller crop of the image). You can find the region file attached as 'ones-pv.reg'.

I chose 700x700 since it could hold 9 squares of the pattern seen in m51-noise-pv.fits .

So I guess you can ignore this pattern if you see it in the cropped images.


mkdir crop
astcrop m51.fits -h1 --center 109.5309438,40.17599612 \
        --width 700,700 --widthinpix --mode wcs -o crop/m51.fits
cd crop
astarithmetic m51.fits -h1 set-i i i / -o ones.fits
RUN_SWARP_ON_TO ones.fits ones-swarp.fits
RUN_ASTWARP_ON_TO ones.fits ones-warp.fits


## The large circle

This circle is completely tied with the PV coefficients. I tried altering several coefficients, and it completely changed shape. So in essence, Warp is using WCSLIB to accurately calculate the distortion effects; which create these patterns.

file #53842)

Pedram Ashofteh-Ardakani <pedram>
Group Member
Wed 12 Oct 2022 07:07:19 AM UTC, comment #15: 

I solved my previous doubts. Pedram, just by using your code the noise image is different each time. You need also to use the option '--envseed' and specify the type of the random seed generator. So, something like:


export GSL_RNG_TYPE=ranlxs1
export GSL_RNG_SEED=1665844464
astarithmetic m51.fits -h1 set-i i i / 10 mknoise-sigma -o noise.fits --envseed


would generate always the same noised image. I will keep looking at this. Thanks!


Raul Infante-Sainz <infantesainz>
Group Member
Wed 12 Oct 2022 06:59:02 AM UTC, comment #14: 

Thanks a lot Pedram and Mohammad.

With the code you posted it is much easy to play with. I am still looking at some points that come to my mind. Probably they have no sense but I found something that is disturbing me.

If you remember, in my first post of this thread I converted the original image to 'uint8'. Why? just because I needed to upload the image and I wanted to reduce the size. Then I realized some of the things we are investigating.

Now, I did something similar. I consider the input image as 'float64' instead of the original 'float32' data type. So, basically I added to your code 'float64' in the lines where the ones and noise images are created:


astarithmetic m51.fits -h1 float64 set-i i i / -o ones.fits

astarithmetic m51.fits -h1 float64 set-i i i / 10 mknoise-sigma -o noise.fits


Before running it, my guess was that nothing should change significantly. I am right? The result is that they are actually different. Of course, the final results of swarping/resampling also differ.

Raul Infante-Sainz <infantesainz>
Group Member
Tue 11 Oct 2022 08:16:02 PM UTC, comment #13: 

;; This buffer is for text that is not saved, and for Lisp evaluation.
;; To create a file, visit it with C-x C-f and enter text in its buffer.

Thank you both Mohammad and Raul for your replies!

Mohammad, I kept working on the big images, but I'll do it with the smaller
ones as you suggested as well! Thanks for the tip :-)

Raul you've done a great job with this investigation :-)

I've got some INTERESTING results :-) But I don't know how to put the
images under each section (comment #7), so please refer to them and scroll
back up while reading this.

## Considerations

First, while making the noise, I thought maybe using the same random seed
could be beneficial, so I exported the GSL_RNG_SEED for reproducible
results.

Second, I found out my 'grep' alias which has the '--color=always' option,
is not letting 'astfits' actually '--delete' the PV coefficients! That's
why I added the '--color=never' option.

Third, just removing the PV coefficients weren't enough! Please don't
forget to update the CTYPE1 and CTYPE2 values to a TAN projection as well!

Fourth, when investigating the images with ds9 regions, I chose the 'box'
shape, and set the size to 100x100 and angle to '0' manually. That is
because using 'circles' on 'square' pixels can mess up the statistics
easily!

Now the code I used after Raul's additions (essentially they're the same,
just with taking the notes above into account):

## Initialization


# DRY! To keep things tidy and error-free
RUN_SWARP_ON_TO ()
{
  swarp "$1" -SUBTRACT_BACK N \
             -IMAGEOUT_NAME "$2" \
             -CENTER_TYPE MANUAL \
             -PIXELSCALE_TYPE MANUAL \
             -IMAGE_SIZE 9245,9245   \
             -PIXEL_SCALE 0.555594   \
             -CENTER 109.5309438,40.17599612
}
RUN_ASTWARP_ON_TO ()
{
  astwarp "$1" --cdelt 0.555594/3600 -o "$2"  \
               --width 9245,9245 --widthinpix \
               --center 109.5309438,40.17599612
}

REMOVE_PV_FROM_TO ()
{
  cp "$1" "$2"

  # Remove all PV keywords from file
  PKEYS=$(astfits "$1" -h1 --printkeynames | grep --color=never ^PV)
  for pv in $PKEYS; do
    astfits "$2" -h1 --delete $pv
  done

  # Update the projection from TPV to TAN
  astfits "$2" -h1 --update CTYPE1,RA---TAN --update CTYPE2,DEC--TAN
}


## Ones with PV


# To make the noise.fits file reproducible
export GSL_RNG_SEED=1665844464

# Create an image with ones
astarithmetic m51.fits -h1 set-i i i / -o ones.fits

# SWarp and Warp _with_ PV
RUN_SWARP_ON_TO   ones.fits ones-swarp.fits
RUN_ASTWARP_ON_TO ones.fits ones-warp.fits


Look at m51-ones-pv.jpg file #53834 (hopefully I've got the number correct!)

Indeed, as Raul pointed out we can see a circle in the middle. I've pointed
out the scale values. Just using 'Statistics' you can see the properties:


aststatistics ones-warp.fits -h1

Statistics
-------
Input: ones-warp.fits (hdu: 1)
-------
  Number of elements:                      84820444
  Minimum:                                 0.994662
  Maximum:                                 1.02064
  Mode:                                    0.9948816299
  Mode quantile:                           0.01426795189
  Median:                                  1.00243
  Mean:                                    1.002846947
  Standard deviation:                      0.005231166437



## Ones without the PV


# Remove the PV parameters
REMOVE_PV_FROM_TO ones.fits ones-no-pv.fits

# SWarp and Warp _without_ PV
RUN_ASTWARP_ON_TO ones-no-pv.fits ones-no-pv-warp.fits
RUN_SWARP_ON_TO   ones-no-pv.fits ones-no-pv-swarp.fits


Look at m51-ones-no-pv.jpg file #53835

Warp has this strange pixel distribution. But the difference is of order
10e-6. I guess it is because of the edges of the image (I will try it out
again as Mohammad suggested, now with the functions, it is very easy, we
only have to reset the center and size for RUN_SWARP_ON_TO and RUN_ASTWARP_ON_TO
functions).


## Noise with PV


# Create an image with noise
astarithmetic m51.fits -h1 set-i i i / 10 mknoise-sigma -o noise.fits

# SWarp and Warp _with_ PV
RUN_ASTWARP_ON_TO noise.fits noise-warp.fits
RUN_SWARP_ON_TO   noise.fits noise-swarp.fits


Look at m51-noise-pv.jpg file #53836

Indeed, again BOTH SWarp and Warp have this pattern on them. I think this
is not a 'dot' pattern. This looks like the sky grid to me! You'll get what
I mean after seeing the next output (thanks Raul for creating these
wonderful examples).

## Noise without PV


# Remove the PV parameters
REMOVE_PV_FROM_TO noise.fits noise-no-pv.fits

# SWarp and Warp _without_ PV
RUN_ASTWARP_ON_TO noise-no-pv.fits noise-no-pv-warp.fits
RUN_SWARP_ON_TO   noise-no-pv.fits noise-no-pv-swarp.fits


Look at m51-noise-no-pv.jpg file #53837

The pattern is here AGAIN! But this time, it is NOT distorted! It looks
like this is a low-level WCSLIB sky-grid calculation kind of effect.

## Conclusion

This pattern does NOT emerge from the distortions or Warp. It is
lower-level!

## Question

Why is Warp creating that spot in 'ones with PV' section? The difference
from the edges to the center is about 0.03 (~3%), where the edges have the
larger values.


Pedram Ashofteh-Ardakani <pedram>
Group Member
Tue 11 Oct 2022 03:26:33 PM UTC, comment #12: 

Thanks for the nice tests Pedram and Raul.

To help speed up the tests, Pedram, can you see if this also happens on a small '--width' also? If so, you can set the '--center' and '--width' to a small size around a small region like the one I showed in my original post, select one pixel in the high and another in the low resolution regions, and see what is happening!?!

The fact that setting '--cdelt' to a smaller value reduces the problem hints that it has to do with sampling.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 10 Oct 2022 10:16:35 PM UTC, comment #11: 

Thanks a lot for investigating this Pedram! You are right, I have been playing a bit with the images and I can clearly see the same pattern on SWarped images. But the effect is less strong so we didn't notice that until now.

I am pasting here a few command that I have been playing with. Basically I tried to check what is the difference between having and not having the distortion coefficients in the header. I also tried to simplify the problem by constructing images with only ones as pixel values, and random noise. That would simplify the problem of having astronomical objects that make difficult the analysis of the data.


# Create an image with ones
astarithmetic m51.fits -h1 set-i i i / -o ones.fits


# SWarp and Warp _with_ PV
swarp ones.fits -SUBTRACT_BACK N         -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE 0.555594         -IMAGEOUT_NAME ones-swarp.fits -IMAGE_SIZE 9245,9245         -CENTER_TYPE MANUAL -CENTER 109.5309438,40.17599612

 astwarp ones.fits --cdelt 0.555594/3600           -o ones-warp.fits --width 9245,9245           --widthinpix --center 109.5309438,40.17599612


# Remove the PV parameters
cp ones.fits ones-no-pv.fits

for pv in $(astfits ones.fits -h1 --printkeynames | grep ^PV); do astfits ones-no-pv.fits -h1 --delete $pv; done


# SWarp and Warp _without_ PV
swarp ones-no-pv.fits -SUBTRACT_BACK N         -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE 0.555594         -IMAGEOUT_NAME ones-no-pv-swarp.fits -IMAGE_SIZE 9245,9245         -CENTER_TYPE MANUAL -CENTER 109.5309438,40.17599612

astwarp ones-no-pv.fits --cdelt 0.555594/3600           -o ones-no-pv-warp.fits --width 9245,9245           --widthinpix --center 109.5309438,40.17599612


# Create an image with noise
astarithmetic m51.fits -h1 set-i i i / 10 mknoise-sigma -o noise.fits


# SWarp and Warp _with_ PV
swarp noise.fits -SUBTRACT_BACK N         -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE 0.555594         -IMAGEOUT_NAME noise-swarp.fits -IMAGE_SIZE 9245,9245         -CENTER_TYPE MANUAL -CENTER 109.5309438,40.17599612

astwarp noise.fits --cdelt 0.555594/3600           -o noise-warp.fits --width 9245,9245           --widthinpix --center 109.5309438,40.17599612


# Remove the PV parameters
cp noise.fits noise-no-pv.fits

for pv in $(astfits noise.fits -h1 --printkeynames | grep ^PV); do astfits noise-no-pv.fits -h1 --delete $pv; done


# SWarp and Warp _with_ PV
swarp noise-no-pv.fits -SUBTRACT_BACK N         -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE 0.555594         -IMAGEOUT_NAME noise-no-pv-swarp.fits -IMAGE_SIZE 9245,9245         -CENTER_TYPE MANUAL -CENTER 109.5309438,40.17599612

astwarp noise-no-pv.fits --cdelt 0.555594/3600           -o noise-no-pv-warp.fits --width 9245,9245           --widthinpix --center 109.5309438,40.17599612


An interesting thing:

In the warped images of ones with PV you can clearly see a circular shape. While in the SWarped image it is flat.


astscript-fits-view ones-warp.fits


On the SWarped images, with two different regions of the same size on different parts of the image, I can see that the properties (statistics) are different.

Sorry for the long lines and etc. I will try to polish that and re-post when I have some time, but I wanted to let you know what I have done quickly so you can have some clues to keep investigating.


Raul Infante-Sainz <infantesainz>
Group Member
Mon 10 Oct 2022 05:22:49 PM UTC, comment #10: 

SWarp is showing the same pattern in the JPLUS M51 image as well (comment #5)! If you can't spot it, download the image and look more closer (it is visible, but Warp has more pronounced intensity). Here are the steps to make these images so you'd be able to play with it yourselves 🤩


$ wget http://archive.cefca.es/catalogues/vo/siap/jplus-dr2/reduced/get_fits?id=771824 -Om51-single-exposure.fits.fz
$ astarithmetic m51-single-exposure.fits.fz -o m51.fits
$ astwarp m51.fits --cdelt 0.555594/3600 \
          -o m51-warp.fits --width 9245,9245 \
          --widthinpix --center 109.5309438,40.17599612
$ swarp m51.fits -SUBTRACT_BACK N \
        -PIXELSCALE_TYPE MANUAL -PIXEL_SCALE 0.555594 \
        -IMAGEOUT_NAME m51-swarp.fits -IMAGE_SIZE 9245,9245 \
        -CENTER_TYPE MANUAL -CENTER 109.5309438,40.17599612



Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 10 Oct 2022 04:27:37 PM UTC, comment #9: 

I thought maybe we can spot some difference between Warp and SWarp. So I made these two images:

+verbose+
$ astwarp img.fits --width 601,601 --widthinpix  \
          --center 20.53208198,0.9266444499     \
          --cdelt 0.390897/3600 -o img_warp.fits
$ swarp img.fits -SUBTRACT_BACK N -PIXELSCALE_TYPE MANUAL   \
        -PIXEL_SCALE 0.390897 -IMAGEOUT_NAME img_swarp.fits \
        -IMAGE_SIZE 601,601 -CENTER_TYPE MANUAL -CENTER 20.53208198,0.9266444499
-verbose-

Then took the difference:

+verbose+
$ astarithmetic img_warp.fits -h1 img_swarp.fits -h0 - \
                -o warp-vs-swarp.fits
-verbose-

I tried different scales (and limits) to spot the patterns, but it wasn't visible ... Does it mean that both Warp and SWarp actually ARE producing the SAME pattern but we can see it better in Warp?


Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 10 Oct 2022 04:06:31 PM UTC, comment #8: 

I've compared the Original image with SWarp and Warp in the 'bug_63171.jpg' file. It looks like this pattern emerges around '100' value. So the scale is set from 50 to 150 where this is best visible.

Any tips on analyzing the SWarp image? I tried checking the regions with 'histogram' but I can't interpret anything useful. I can only see that in these fields the histogram of 50 to 150 is pretty symmetric (when not on an object).


Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 10 Oct 2022 02:50:45 PM UTC, comment #7: 

Nice catch! First I tried --edgesampling but then I thought of smaller CDELT as well.

I just updated task #16272 and gave the developing branch [1]. But the output looks just like what Mohammad shows in comment #3! See files 'img.jpg' and 'img_pixelareaonwcs.jpg'.

[1] https://codeberg.org/pedramardakani/gnuastro/src/branch/pixelonarea

This debugging image is added as the last HDU if you give the '--pixelareaonwcs' option to Warp. It is implemented in Library level.

(By the way, how do you guys know what is the image id so you can show them in your posts? 🤔)


Pedram Ashofteh-Ardakani <pedram>
Group Member
Mon 10 Oct 2022 02:33:47 PM UTC, comment #6: 

However, the work-around suggested below (to use a smaller pixel scale) will be slow; so its not an optimal solution; just a work-around for now.

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 10 Oct 2022 02:16:10 PM UTC, comment #5: 

I just noticed that this problem also occurs on J-PLUS images. For example, with the command below, you can download a single-exposure J-PLUS image and see the effect after running Warp:


$ wget http://archive.cefca.es/catalogues/vo/siap/jplus-dr2/reduced/get_fits?id=771824 -Om51-single-exposure.fits.fz

$ astwarp m51-single-exposure.fits.fz


So it seems to be more comman than we thought! Therefore more urgent to find/fix the cause.

However, I did find a solution: to resample each exposure such that the pixel scale is half (we have 2x2 times more pixels!). This feature becomes much weaker as you use a higher sampling. In the case of J-PLUS, I resampled the three exposures to double the sampling, and after stacking, I resampled it back to the original resolution and these features were gone.

If sampling is the only problem, we may be able to fix it by having a finer sampled grid. But I am still puzzled by the varying smoothing level!

Mohammad Akhlaghi <makhlaghi>
Group administrator
Mon 10 Oct 2022 07:16:09 AM UTC, comment #4: 

Thanks for pointing this out Raul! And Mohammad, thank you for investigating in detail and defining task #16272 🙌

As you know I was busy with the announcement [1] recordings [2] and didn't get the chance to follow tasks/bugs. But now, I'll try to keep up with you 😃

This looks very interesting 👍

[1] https://lists.gnu.org/archive/html/info-gnuastro/2022-10/msg00000.html
[2] https://peertube.stream/w/uq7SBDYZS1HRtJwCkbcDsz

Pedram Ashofteh-Ardakani <pedram>
Group Member
Thu 06 Oct 2022 05:58:27 PM UTC, comment #3: 

This bug inspired me to define task #16272. It will be very useful in understanding the distortion of a camera, and visualizing things like this.

While writing that bug, the following test occurred to me (making an image with all input pixels equal to one and then applying the warp. This should show how many input pixels have been used in each output pixel


$ astarithmetic img.fits set-i i i - 1 + -o one.fits

$ astwarp one.fits


Here is the output: as we see, we don't see any such pattern in the number of pixels that are used in the output pixel grid.



So this is indeed, very confusing for me! We don't see this effect in the noise of other images with distortion that we have tested!

The strong change in smoothing level that we see in the second image of Comment #1 hints that a significantly different number of input pixels are being used, but the test above doesn't show that! Either the test is wrong, or there is another cause for this strongly different smoothing!)

We have to investigate what is happening here!

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 06 Oct 2022 05:25:01 PM UTC, comment #2: 

I just noticed small typo in the second example command demonstrating overflow, in the last comment line, I should have written "# Because 518 = 256*2 + 6".

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 06 Oct 2022 05:21:28 PM UTC, comment #1: 

What an interesting scenario! There are two different factors at play here which can be summarized in this image:



1) ["Fringes"] When you simply convert a value to an unsigned integer, if it is larger than 256 (=2^8), it overflows and its remainder (modulo) from a multiple of 256 is kept. To see what I mean try the following commands. As you see, how much ever you go up (as expected from the overflow logic!) the value will be between 0 255

$ astarithmetic 257 uint8 -q
1                             # Because 257 = 256*1 + 1

$ astarithmetic 518 uint8 -q
6                             # Because 257 = 256*2 + 6


In the case of your image, we see that it has the following minimum and maximum values:

$ aststatistics img.fits --minimum --maximum
-207.176 8272.66


So the "fringing" or contour-like pattern you see in 'uint8.fits' over the galaxy is because of overflow, and has nothing to do with Warp. To do this properly, you should first scale your image to a range of 0 to 254 (because 255 is considered as BLANK in the unint8 type!), then call the 'uint8' operator. Like the Arithmetic command below: I am first subtracting the image from its minimum value (so afterwards, its minimum value is 0), then I am dividing it by the maximum value (so the maximum is 1). Now that I know all pixels are between 0 and 1, I multiply it by 254 to change the range from 0 to 254:

astarithmetic img.fits set-i i i minvalue - i maxvalue / 254 x uint8 \
              -ouint8-correct.fits


2) [Noise changes] These also happen in the pure noise warping that you created. These are caused by the distortion: input pixels have strongly different sizes on the sky!

Have a look at the image below which is a zoom-in of one of the purple regions above. You clearly see the different resolutions/smoothings. In effect the pixel size is hugely different on such small scales!!! Since Warp uses a "pixel mixing" based resampling, in effect, it acts like a convolution, allowing you to see such things :-).



This is a VERY BAD distortion pattern! I really wonder how SWARP does the job! Pedram: here is a new section for your paper ;-). We should really go into the detail of these extreme cases, because it is very important/interesting ;-).

Mohammad Akhlaghi <makhlaghi>
Group administrator
Thu 06 Oct 2022 03:59:57 PM UTC, original submission:  

In Gnuastro 0.18.65-7933d, the output of warped images have strange patterns. I found this issue when testing the new features on some images. Let me reproduce and describe the problem step by step.

I attach an image to star with: 'img.fits'. It is a cropped region nearby to the galaxy NGC 493. Image of 'float32' data type. Nothing special.

I convert this image to another one of type 'uint8'.


astarithmetic img.fits uint8 -o uint8.fits


Now just warp this image and have a look.


astwarp uint8.fits -o uint8-warp.fits

astscript-fits-view uint8.fits uint8-warp.fits


There is a clear pattern across the image, like an interference pattern.

If you are wondering why I am doing such a strange things, it is just because I was warping bigger images and I found the same pattern (on normal 'float32' images). Attached a screenshot (warped-float32.png). Then, when I tried to upload the image to Savannah I had to decrease the size (by converting it to integers). On this converted images the problem is even more clear.

To confirm that it is not a problem that comes from the original image I did the following test: create an image with pure noise, warp and check it.


astarithmetic img.fits set-i i i - 3 mknoise-sigma -o noise.fits

astwarp noise.fits -o noise-warp.fits

astscript-fits-view noise-warp.fits



Raul Infante-Sainz <infantesainz>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #53899:  jplus-warp-oversample.jpg added by pedram (575KiB - image/jpeg)
file #53900:  jplus-warp-reg-maxfrac.jpg added by pedram (287KiB - image/jpeg)
file #53901:  jplus-warp-reg.jpg added by pedram (448KiB - image/jpeg)
file #53902:  jplus-stack.jpg added by pedram (440KiB - image/jpeg)
file #53865:  m51_and_img_pixarea.jpg added by pedram (70KiB - image/jpeg)
file #53841:  ones-pv-warp.jpg added by pedram (82KiB - image/jpeg)
file #53842:  ones-pv.reg added by pedram (1KiB - text/x-ms-regedit)
file #53837:  m51-ones-pv.jpg added by pedram (73KiB - image/jpeg)
file #53838:  m51-noise-pv.jpg added by pedram (494KiB - image/jpeg)
file #53839:  m51-ones-no-pv.jpg added by pedram (78KiB - image/jpeg)
file #53840:  m51-noise-no-pv.jpg added by pedram (499KiB - image/jpeg)
file #53833:  m51-jplus.jpg added by pedram (369KiB - image/jpeg)
file #53831:  warp-minus-swarp.jpg added by pedram (194KiB - image/jpeg)
file #53830:  bug_63171.jpg added by pedram (236KiB - image/jpeg)
file #53827:  img.jpg added by pedram (86KiB - image/jpeg)
file #53828:  img_pixelareaonwcs.jpg added by pedram (28KiB - image/jpeg)
file #53808:  area-after-warp.jpg added by makhlaghi (25KiB - image/jpeg)
file #53807:  strong-distortion.png added by makhlaghi (104KiB - image/png)
file #53806:  two-features.png added by makhlaghi (822KiB - image/png)
file #53803:  warped-float32.png added by infantesainz (678KiB - image/png)
file #53804:  img.fits added by infantesainz (1MiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pedram (Posted a comment)
  • -email is unavailable- added by makhlaghi (Updated the item)
  • -email is unavailable- added by infantesainz (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-24 pedram StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2022-10-24 pedram Attached File- Added jplus-warp-oversample.jpg, #53899
        Attached File- Added jplus-warp-reg-maxfrac.jpg, #53900
        Attached File- Added jplus-warp-reg.jpg, #53901
        Attached File- Added jplus-stack.jpg, #53902
    2022-10-16 pedram Attached File- Added m51_and_img_pixarea.jpg, #53865
    2022-10-12 pedram Attached File- Added ones-pv-warp.jpg, #53841
        Attached File- Added ones-pv.reg, #53842
    2022-10-11 pedram Attached File- Added m51-ones-pv.jpg, #53837
        Attached File- Added m51-noise-pv.jpg, #53838
        Attached File- Added m51-ones-no-pv.jpg, #53839
        Attached File- Added m51-noise-no-pv.jpg, #53840
    2022-10-10 pedram Attached File- Added m51-jplus.jpg, #53833
    2022-10-10 pedram Attached File- Added warp-minus-swarp.jpg, #53831
    2022-10-10 pedram Attached File- Added bug_63171.jpg, #53830
    2022-10-10 pedram Attached File- Added img.jpg, #53827
        Attached File- Added img_pixelareaonwcs.jpg, #53828
    2022-10-10 makhlaghi StatusNone In Progress
        Assigned toNone pedram
    2022-10-06 makhlaghi Attached File- Added area-after-warp.jpg, #53808
    2022-10-06 makhlaghi Attached File- Added strong-distortion.png, #53807
    2022-10-06 makhlaghi Attached File- Added two-features.png, #53806
    2022-10-06 infantesainz Attached File- Added warped-float32.png, #53803
        Attached File- Added img.fits, #53804

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code