When the Warp program does WCS operations (which is the default). It consumes a very large amount of RAM! Such that in large pipelines, the warping step becomes the main bottleneck (in RAM).

Here is an example (the second Warp command)

$ echo "1 101 101 1 20 0.5 30 0.7 15 5" \
       | astmkprof --mergedsize=201,201 --oversample=1 -oraw.fits \
                   --pc=-0.939692,-0.3420201,-0.3420201,0.9396926

$ /usr/bin/time --format=%M astwarp raw.fits
23680

$ ls -lh raw.fits
-rw-r--r-- 1 mohammad mohammad 169K Jun 19 22:16 raw.fits

We see that while the file is only 169K, at its maximum, Warp is consuming 23680K of memory!!!! In a normal reduction pipeline these days images are about 300-400 Megaybtes, so this excessive usage of RAM by Warp easily fills up any RAM and creates a strong bottleneck for parallel operations!

This excessive usage occurs because of the two phase operation of Warp in WCS mode: first it initializes the output image's pixel grid, and then it will warp the input pixels into it. But since the operation is non-linear, we need the coordinates of all four corners of every pixel.

My first guess is that if we can re-design the program to get the output pixel grid with minimal points (for example only warp the outer corners of the the pixels on the border of the image), and then do the warp of each input pixel independently in its own thread, this excessive RAM usage will be removed. But we should look into it closer!

I should greatly thank Pedram for his wonderful work on completing the WCS-based feature of Warp! We knew about this problem at that time, but in the spirit of progress, needed to finish the work and couldn't invest more in the RAM optimization at that time.