taskGNU Astronomy Utilities - Tasks: task #14363, Option for scaling to nearest full...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #14363: Option for scaling to nearest full pixel fraction

Submitted by:  Lee Kelvin <leeskelvin>
Submitted on:  Wed 08 Feb 2017 08:16:03 PM UTC  
 
Should Start On: Wed 08 Feb 2017 12:00:00 AM UTCShould be Finished on: Wed 08 Feb 2017 12:00:00 AM UTC
Category: WarpPriority: 5 - Normal
Item Group: EnhancementStatus: None
Privacy: PublicPercent Complete: 0%
Assigned to: NoneOpen/Closed: Open
Effort: 0.00

(Jump to the original submission Jump to the original submission)

Mon 13 Feb 2017 11:31:25 PM UTC, comment #8:

Warp (the new name for ImageWarp) now comes with the new option `--coveredfrac' (replacing the old `--maxblankfrac' function, see P.S.), which can address the problem you mentioned about a flat image.

What this option does is this: you give it a fraction (between 0 and 1). If the output pixel doesn't cover the input pixels by a fraction larger than the number you gave, the output pixel will be blank.

So, to ensure flat-ness in your example, you can run Warp with `--coveredfrac=1'. In this way, if the output pixel isn't fully covered by the input image pixels, it will have no value, so all your output pixels will also be flat. This comes at the expense of missing the pixels immediately touching the edge, so the total sum of the pixel values will be less than the input image. But you do have access to a continuous warping space.

The great thing with this feature is that it doesn't just apply to `--scale', it applies to any kind of warping.

This isn't a solution to this task, but I am just mentioning it here as one alternative that can be used until this task is implemented.

P.S. the old `--maxblankfrac' just counted the fraction of blank pixels over the output image, so it was useless for edge effects. The new `--coveredfrac' now accounts for both blank pixels and edges of the image.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Fri 10 Feb 2017 03:37:19 PM UTC, comment #7:

Sorry, just one note, if you do get the chance to update the book, please use this doc/gnuastro.texi version (on my development branch as part of task #14245). As I am making changes/additions, I am also updating the book when necessary in every commit, the link above will always be up to date with my work, making my job to merge your changes easier later).

You can just replace this `gnuastro.texi' with the one in the tarball that you have, and then run `make pdf' (in the top source directory or the `doc/' directory), in both cases, the PDF output will be created in `doc/gnuastro.pdf'.

You can find the top of the ImageWarp section by searching for `@node ImageWarp'.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Fri 10 Feb 2017 03:24:07 PM UTC, comment #6:

Thanks, I am really happy that we were able to reach a common understanding. I have modified the title of this task to "Option for scaling to nearest full pixel fraction", if you can think of a better title, please feel free to update it (or if you can't, comment here so I do it).

Since you have read the book and you brought up this problem with ImageWarp not operating as you expected it, it would be great if you could propose some corrections so this experience doesn't repeat for other people who read the book later. The book source is in `doc/gnuastro.texi', it is plain text and written in GNU Texinfo, as a first step, you can see it as a TeX source just replacing `\' with `@'. To see your corrections in compiled format, you can just run `make pdf' and see the updated results in an updated `gnuastro.pdf' output.

It would be great if you could bootstrap and fork Gnuastro, but if you don't have the time, you can just send me the updated `gnuastro.texi' and I will commit the changes with your name, so there is no problem. It would be great if the text could avoid such un-expected behavior for the future users.

Gnuastro has been built and grown starting from the low-level (most general) solution, and we have slowly built our way up to the high-level (more specific, but limited) solutions. As we grow and with great suggestions like yours, more high-level features will certainly be added, thank you so much ...

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Fri 10 Feb 2017 12:28:22 PM UTC, comment #5:

Yes, I see your point, and I accept that scaling in this manner has its uses. As mentioned earlier, I find your recommendation to add a --discretescale or --autoscale to be a good compromise.

The current mode of operation for scaling is not without its dangers. If I have a perfect flat field image whereby every pixel has exactly the same value and I scale it using --scale, I will end up introducing an irreversible discrepancy into the image at the top and right edges. My flat field image would no longer be flat.

Until such time as the new options above come into existence, I am happy to be more careful in choosing my scaling ratios! The addition of Task 14365 would greatly help in this regard.

Lee Kelvin <leeskelvin>
Thu 09 Feb 2017 06:29:36 PM UTC, comment #4:

Sorry, this is correction to the previous sentence to make it more clear ("sizes" should have been "numbers":

Would you do science (for example measure the colors) with two images that have a different resolutions? (even though the pixel NUMBERS are the same)

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Thu 09 Feb 2017 06:24:33 PM UTC, comment #3:

Thanks for the explanation. I am happy that you agree to having a secondary option for he higher-level suggestion you made.

Generally, the main thing you are not considering is that unlike GIMP, in astronomy the WCS information (resolution) is the defining factor of a dataset, not the number of pixels. Let me give you one example:

In your own example, suppose your input 8x1 image is from telescope+camera A. Also, suppose you also have another image from telescope+camera B which has 1/3 the resolution of the telescope+camera A. And you want to find the color of your target. If `--scale' behaved as you suggested, it would be impossible to correctly scale the two to one grid. The best you could do is scale A by 1/2.6666, not 1/3.

Would you do science (for example measure the colors) with two images that have a different resolutions? (even though the pixel sizes are the same)

The pixel numbers are irrelevant for science, what defines our data (connects it to the outside world) is the WCS (which is continues). In any deep survey (where multiple images are warped then added with each other), the edges become very inaccurate, both because of such processing issues and also because their depth (exposure time) will not be as much as the inner regions. So there is no harm in loosing accuracy on the pixel immediately touching the edge! The edges are NOT going to be used for science any way. The important thing is that the main body of the image (inner to the immediate edge) are accurate (and can be warped to ANY grid).

Just on the side, when I say we work on a continues grid, it doesn't mean that for example we fit the pixel values to some continues mathematical function, do the warping and then convert them back into a discrete pixel as you described! I mean that the output pixel area is mapped onto the input image (using the inverse of the input warping matrix) and the fractional area/value of the input pixels that overlap with it is used to calculate the output pixel value. So the output pixels are assumed to be discrete from the very beginning. What is continues is the transformation matrix space. You are suggesting to have a discrete transformation space, where some transformations would be impossible.

Every step in the source code of ImageWarp (and all Gnuastro generally, is heavily commented), you don't have to know C or even read the code to see what is going on. You can understand each step of any program/library exactly by following the functions and reading the comments. I have described the main source files of each program in the Mandatory source code files section of the manual.

The manual is only the tip of the ice-burg! There is much more explanation on the details of each operation in the comments of the code ;-)! This is the foundation principle of Gnuastro.

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Thu 09 Feb 2017 04:49:40 PM UTC, comment #2:

Thank you for your detailed reply. As we have discussed, I do not consider this issue to be a misunderstanding, but rather something more fundamental in the way image manipulation with respect to scaling should operate. However, I find your suggestions to add new arguments which do behave in such a manner to be good compromise if I'm the only one who sees an issue with the current operation.

With regards continuous space vs discrete pixel space, I am more than happy for ImageWarp to internally treat data as laying on a continuous axis. This is as it should be for the purposes of maintaining accuracy during potentially multiple image transformations. However, what I think you have missed in my original post is that when it comes to the point at which an output file must be saved, ImageWarp will always have to make a conversion back into discretised pixel space. In that sense, yes, I am thinking in terms of discrete pixel space because this is fundamentally how our data is observed at the telescope, saved, and output. When a photon hits a CCD pixel, we have no further information on whereabouts across the area of this pixel the photon was absorbed. Therefore, our input data is discretised by design from its inception. Again, to re-iterate what I wrote above, it then makes sense to treat an input image as continuous for the purposes of image manipulation, pixel mixing, resampling, etc, but ultimately the final output must be re-discretised by the software.

However, I think the discussion of continuous vs discrete space, whilst interesting, does not fully address the issue I originally raised here. I am also well aware that the bottom left corner of the first pixel in a FITS image has coordinates [0.5,0.5] as defined by the FITS standard, and I also do not think that this addresses the issue either.

Lets work in continuous space. My x-axis begins at 0.5 and ends at 8.5, as per the FITS standard definition. Using the real numbers along this axis we now have a continuous space within which to operate of length 8.5 - 0.5 = 8.0. I now want to scale my continuous spatial axis by 1/3. Performing the calculation, my continuous spatial axis is now only of length 2.666. All my image manipulation is now finished. I now want to save this file back out to a FITS image, however, the length of the axis is not an integer, which makes it impossible to write an image - what do we do? The current mode of operation just begins chopping up the image pixel-by-pixel, starting in the bottom-left corner, and when it gets to the last one it says "tough luck, you don't have a full pixel area, so we're just going to have to upgrade you from having an area of 0.666 pixels to 1 pixel." Why should the final pixel alone be singled out and penalised for an image transformation which affected the entire image? This imposes all of the image manipulation sizing discrepancy onto the final pixel.

Why do I believe this to be counter-intuitive? Consider the example FITS file I proposed in my original post: 8 pixels in a row with each pixel having the value 1. The input image is flat and homogeneous. After scaling using the current set-up of ImageWarp, the homogeneity of the image has been lost. Instead, the final pixel is now different from the remaining pixels, imposing a modification of a previously flat and homogeneous dataset.

My suggestion is that it is fairer to penalise all pixels by some small amount by stretching the continuous spatial axis from 2.666 back up to 3, such that it maps perfectly onto a discretised pixel space. Or, in effect, you modify your initial scaling value such that only one transformation need take place.

In summary, I do not believe that a modification of ImageWarp in such a manner as proposed would cripple ImageWarp for science use. Indeed, I find the current operation to be unexpected and violating the principle of least surprise, making science application potentially hazardous. However, this is merely my opinion as an end user. I realise (from our correspondence) that this opinion may not be universal(!), however, I wanted to leave it here as a task for completeness sake.

Lee Kelvin <leeskelvin>
Thu 09 Feb 2017 02:11:53 AM UTC, comment #1:

This is an interesting suggestion, thank you. It would be great if you could mention what part of the ImageWarp documentation created this impression?

The solution to this mis-understanding is to think in terms of a continues grid, not in terms of a discrete pixel space. You are thinking in terms of a continues space when dealing with rotation, translation, or projection for example, but for `scale' you want it to be on a discrete space.

If this feature is implemented for `--scale', the real range of `--scale's available to the user will not be continues any more, it will be discrete! This can have many real-life implications: as you know, astronomical datasets have a very wide range of resolutions that don't easily divide into each other, if `--scale' is modified as you described, it will be impossible to warp such images into the same WCS grid (if their pixel sizes are identical).

To make things more clear, I should also explain that when you use the `--scale' option (or any of the modular transformation options, not the `--matrix' option). The center of the coordinate space is set at the bottom left of the first pixel. So it can exactly cover the pixel. If you directly use the `--matrix' option (`--matrix=0.3333,0,0,0.33333') for raw transformation, the center of coordinates will be what is defined by the FITS standard (half a pixel before the first pixel, since the center of the first pixel is defined to have a value of `1.0' in the FITS standard). Which will even be more confusing when you are thinking in terms of a discrete pixel space!

In ImageWarp, you can also shift the center of coordinates before the scaling using the `--translate' option (on a continues grid). So, you see, even the center isn't fixed, you are free to define ANY kind of transformation! ImageWarp is defined to be very low level to allow any kind of generic tranformation for ANY kind of data and science.

I hope the explanations above have clarified the incorrect points below:

  • I must have explained badly in our private communication, I am sorry for that ;-). ImageWarp DOES NOT "modif[y] the input image such that it is extended by one extra blank pixel.". ImageWarp works based on the continues space of COORDINATES, not discrete pixels! So in your example, the last output pixel (defined by a SCALED COORDINATE GRID), only covers the last two input pixels. ImageWarp doesn't add a new pixel, there is just nothing for it to add! It is very important to think in terms of coordinates (a continues grid) rather than pixels (that are discrete).
  • I hope this also explains how this interpretation is wrong: "That is, no pixel mixing has taken place. Instead, the pixels have been grouped seemingly irregularly into bins of 3, 3 and 2 pixels, respectively.". Pixel mixing has indeed taken place: the first output pixel covers 3 input pixels, so their values have been "mixed" (or added) to generate 3. The same for the last pixel: as described above, the COORDINATES have scaled, so the last output pixel covers only two of the input pixels.

What you are suggesting is indeed a desired feature, I don't disagree with that. So, I suggest that we use another option for that (maybe something like `--discrete-scale', or `--automatic-scale'), so the user knows that what they get is NOT necessarily what they asked for. This fact (that the output is NOT necessarily what the user expected, when looking at the WCS, not pixel numbers) is very important.

In the case of applications like GIMP this is the default because GIMP only cares about pixels. In other words, it doesn't have something WCS, to attribute the pixels to some other reference! For a GIMP user, the mere fact that two images having the same size is enough for any further processing.

However, in our science images, depending on the telescope pixel scale (in units of arcseconds/pixel for example), two images of exactly the same number of pixels in width and hight can cover vastly different regions of the sky.

As a summary, making this the default behavior would cripple ImageWarp and make it non-usable in science applications. Adding another option would be a good solution, so when WCS information doesn't matter and the user only cares about the number of pixels, they can use that option.

I would be very happy to help anyone interested implement this new option ;-)...

Mohammad Akhlaghi <makhlaghi>
Project Administrator
Wed 08 Feb 2017 08:16:03 PM UTC, original submission:

The current method by which image scaling takes place in ImageWarp may be confusing to many users. The value taken from --scale should be made flexible in order to perform scalings which are non-trivial.

By way of example, consider an image of dimension 8x1 pixels, i.e., 8 pixels in a row. For the sake of explanation, imagine that each pixel has a value of 1. I now perform the command:

As is obvious, scaling 8 pixels by 1/3 on an axis is non-trivial, as 8 does not divide into 3 perfectly. In this situation, ImageWarp must make a decision on how to handle this request.

Having read through the relevant sections of the ImageWarp manual, I would have expected ImageWarp to employ its pixel mixing routines to split those 8 pixels into 3 regions of equal area, and generate three output pixels. In the case whereby our 8 hypothetical input pixels each have a value of 1, I would therefore have expected the three output pixels to have a value of 8/3 each (~2.666).

However, if you run the command above on the hypothetical 8x1 image, the output image produced by ImageWarp produces pixels with values of [3,3,2]. That is, no pixel mixing has taken place. Instead, the pixels have been grouped seemingly irregularly into bins of 3, 3 and 2 pixels, respectively.

In order to further explain the differences between expected and actual outputs, the attached image shows this in graphical form.

I found this behaviour to be most confusing. On discussing this matter with Mohammad he informs me that in order to perform the command above ImageWarp modifies the input image such that it is extended by one extra blank pixel. That is, effectively the input image dimension is internally assumed to have dimensions of 9x1 prior to performing any image scaling.

I do not believe that the current behaviour ImageWarp exhibits in this situation is either desirable nor obvious. It seems to me that the current mode of operation effectively performs a redefinition of the input image, changing the dimensions of the image to suit the scaling. I believe that any redefinition of input imaging is not what the user would expect, and may even be dangerous to the integrity of a dataset.

In order to see how other programs deal with such a situation, I turned to the image manipulation software GIMP. When asking GIMP to scale an 8x1 pixel image by a factor of 0.333, the scale factor is automatically changed to 0.375 (3/8). That is, the scale value is automatically rounded to the nearest sensible value. In essence, this is what I believe ImageWarp should do also.

TL;DR: In summary, when presented with a non-trivial image scaling request, ImageWarp has two options: 1) to modify the input image such that it better suits the requested scaling value, or 2) to modify the requested scaling value such that it maps directly onto an output pixel map. The current mode of operation is 1), which I believe to be unexpected and potentially dangerous. I advocate a change to option 2), in line with many other contemporary image manipulation software.

Best,
Lee Kelvin

Lee Kelvin <leeskelvin>

 

Attached Files
file #39705:  pixmix.png added by leeskelvin (16KiB - image/png)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by makhlaghi (Posted a comment)
  • -unavailable- added by leeskelvin (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 10 Feb 2017 03:24:07 PM UTCmakhlaghiSummaryChange the default behaviour of --scale in ImageWarp=>Option for scaling to nearest full pixel fraction
    Wed 08 Feb 2017 08:16:03 PM UTCleeskelvinAttached File-=>Added pixmix.png, #39705

    Back to the top


    Powered by Savane 3.1-cleanup1