bugGNU Octave - Bugs: bug #41674, [octave forge] (image) temporary...

 
 

bug #41674: [octave forge] (image) temporary arrays in imresize are doubles

Submitter:  Ian Journeaux <ijourneaux>
Submitted:  Sat 22 Feb 2014 11:30:23 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  Fixed Assigned to:  None
Originator Name:  Ian Journeaux Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 12 Sep 2021 04:08:33 PM UTC, comment #21: 

I also do not see any feasable way to fully avoid the intermediate conversion to double in the imresize function. We do need interpolation in imresize.m, and this interpolation only works easily with doubles.

There was mentioning, of maybe only do the intermediate conversion to the floating point type "single" instead of "double" to save memory. But when I understood coorectly this is kind of solved by not converting single input images at all at the moment. That's probably a good way to deal with this.

To summarize: I do not have any objections to CLOSE this bug report now.

Hartmut <hardy>
Sat 28 Aug 2021 01:28:20 PM UTC, comment #20: 

If there is no objection, I will close this bug report as fixed.

Avinoam Kalma <avinoam>
Group Member
Fri 27 Aug 2021 07:51:20 PM UTC, comment #19: 

stepping through imresize to see if the issue has been resolved, i'm not certain it has. currently, it pushes most of the calculation over to imremap. 

just testing a simple 2x linear scaling on the first 7x10 'test' image with a uint8 input, line 89 in impremap (on default branch) forces im to be recast as a double before running the interp, so the output is double before being recast in the last step back to uint8. 

So this appears to be better than before, where meshgrid first made two full n x m arrays before doing the same process described above.  So, this reduces the 'extra' memory overhead by a factor of 3. 

Assuming there's no other way to get around the temporary output being a double before being recast to match the input class, this can probably be called complete and this old bug report closed.

Nicholas Jankowski <nrjank>
Group Member
Fri 27 Aug 2021 07:29:51 PM UTC, comment #18: 

fyi regarding the comment #17 mention of meshgrid in imresize - an older patch #8369 addressed reducing memory size related to meshgrid usage in imresize.  it appears, however that that patch was superseded by more recent code changes and the current default branch version of imresize no longer calls meshgrid at all.

Nicholas Jankowski <nrjank>
Group Member
Sat 29 Sep 2018 04:37:52 PM UTC, comment #17: 

The original bug is about memory usage on imresize. Half through the discussion (comment #10) there's mentioned that imremap also uses meshgrid. For that case, I pushed 736f6c0cd527 which completely removes the use of meshgrid (it wasn't really necessary).

I didn't look into imresize yet.

Carnë Draug <carandraug>
Group Member
Sat 29 Sep 2018 04:10:49 PM UTC, comment #16: 

PLease ignore my last comment (comment #15). I posted on the wrong bug report.

Carnë Draug <carandraug>
Group Member
Sat 29 Sep 2018 03:55:45 PM UTC, comment #15: 

I have pushed c353b598d7bd which checks whether input is class single instead of always converting to double for all real input.

This is a lot of code duplication and macros. This should probably be cleaned up with templates, but oh well.

Carnë Draug <carandraug>
Group Member
Sat 04 Mar 2017 02:56:09 AM UTC, comment #14: 

Here is my result after trying some codes. See comment #5.


XD = linspace(1, 5, 10)';
all(abs(XD - single(XD)) == 0)

ans =

 1



XD = linspace(1, 5, 10)';
eps(single(XD))

ans =

  1.1921e-007
  1.1921e-007
  1.1921e-007
  2.3842e-007
  2.3842e-007
  2.3842e-007
  2.3842e-007
  4.7684e-007
  4.7684e-007
  4.7684e-007


Xie Rui <xierui>
Thu 15 Dec 2016 12:08:58 PM UTC, comment #13: 

This issue seems to be still present in Octave 4.2.0 with image 2.6.1.

(I could find none of the discussed improvements in the repos.)

Hartmut <hardy>
Thu 27 Feb 2014 11:43:02 PM UTC, comment #12: 

for me:

>> class(linspace(single(1), 2))

ans = single

although

>> class(linspace(1, 2, single(10)))

ans = double

both as expected, I'd say.


XD and YD may contain fractional values, so we need to check if single precision is sufficient to store these values.  their maximum value is not expected to be anywhere near the maximum integer used for indexing.


Felipe G. Nievinski <fgnievinski>
Thu 27 Feb 2014 10:49:58 PM UTC, comment #11: 

Another thought...

If you pass single arguments to a function like meshgrid, you get single outputs.  But there appears to be no way to tell linspace to generate single values instead of double.  Maybe we should consider an extension to Matlab so that if the final argument of linspace may be a character string, either "single" or "double" to specify the type of result that should be produced.

Would that help here?

I'm not sure why you need to use the isequal test.  Are you trying to check to see whether single precision is sufficient for the given operations, or whether double is needed because of the range of values?

John W. Eaton <jwe>
Group administrator
Thu 27 Feb 2014 10:38:52 PM UTC, comment #10: 

Thanks John/Filipe  I am not sure of the process to submit a patch but will look into. I also use imremap which also uses meshgrid. A similar fix would help there as well.

Ian Journeaux <ijourneaux>
Thu 27 Feb 2014 09:40:54 PM UTC, comment #9: 

I think the all() is unnecessary, as in Octave that is implicitly applied to the operands of the && operator.

I'm not a big fan of using clear inside functions as that operates on the symbol table.  Simply assigning [] to those variables should release the memory, which I assume is what you want to do.

John W. Eaton <jwe>
Group administrator
Thu 27 Feb 2014 09:12:38 PM UTC, comment #8: 

Thank you for taking the time to report and test it.  If you feel like, please consider submitting a patch, based on your experience.  Maybe a reusable separate function would help.  Thanks.

Felipe G. Nievinski <fgnievinski>
Thu 27 Feb 2014 08:29:41 PM UTC, comment #7: 

Atleast in my test cases, I did not have to relax the tolerance so

if(all(isequalAbs(XD, XDs, eps(XDs))) ...
&& all(isequalAbs(YD, YDs, eps(YDs))))

worked. Thanks for the help. There are several other places where meshgrid is used in the image package. This could make it more useful in a memory constrained situation.

Ian Journeaux <ijourneaux>
Thu 27 Feb 2014 08:18:02 PM UTC, comment #6: 

Using eps is a great way to accomplish this. IS there a way to set the k value without having to modify the function?

Ian Journeaux <ijourneaux>
Thu 27 Feb 2014 07:52:06 PM UTC, comment #5: 

you're right, I was wrong.

although, hard coded tolerance (1e-6) is to be avoided, not least because Octave might be compiled in 64 bits.

I'd go for absolute comparison and take tol = eps(XDs):

if(all(isequalAbs(XD, XDs, k*eps(XDs))) ...
&& all(isequalAbs(YD, YDs, k*eps(YDs))))

if you need to relax the tolerance, then set k = 10.


here's a quick test showing it's viable:


XD = linspace(1, 5, 10)';
[XD, abs(XD-single(XD)), eps(single(XD)), abs(XD-single(XD)) < eps(single(XD))]

ans =

               1               0    1.192093e-07               1
        1.444444    2.649095e-08    1.192093e-07               1
        1.888889    5.298191e-08    1.192093e-07               1
        2.333333    7.947286e-08    2.384186e-07               1
        2.777778    1.059638e-07    2.384186e-07               1
        3.222222    1.059638e-07    2.384186e-07               1
        3.666667    7.947286e-08    2.384186e-07               1
        4.111111    5.298191e-08    4.768372e-07               1
        4.555555    2.119276e-07    4.768372e-07               1
               5               0    4.768372e-07               1


Felipe G. Nievinski <fgnievinski>
Thu 27 Feb 2014 05:07:06 PM UTC, comment #4: 

I am not sure your proposal works since it is virtually guaranteed that

all(XD == double(single(XD)))

will not be true due to differences in precision.

I believe you have to factor in what would be an acceptable tolerance for the difference. You can do that with a relative tolerance or an absolute tolerance. My preference is an absolute tolerance but I have the definition for the relative tolerance comparison in case it is useful.

Also my choice of 1e-6 as a tolerance might not be appropriate.

Since you are creating grid coordinates for an interpolation, how many significant digits are required for an accurate interpolation?

If valid, it might be applicable to other places where meshgrid is used as a method of reducing the memory footprint.

I have tested this and it seems to work correctly and is more robust than my original proposal.

XD = linspace (1, inCols, outCols);
YD = linspace (1, inRows, outRows);
XDs = single(XD);
YDs = single(YD);
%# absolute tolerance equality
isequalAbs = @(x,y,tol) (abs(x-y) <= tol );
%# relative tolerance equality
isequalRel = @(x,y,tol) ( abs(x-y) <= ( tol*max(abs(x),abs(y)) + eps) );

if(all(isequalAbs(XD, XDs, 1e-6)) && all(isequalAbs(YD, YDs, 1e-6)))
XD = XDs;
YD = YDs;
endif
clear XDs YDs
[XI, YI] = meshgrid (XD, YD);
        im = imremap (im, XI, YI, method);

Ian Journeaux <ijourneaux>
Thu 27 Feb 2014 03:48:36 PM UTC, comment #3: 

the tolerance would have to be different for each value.

so yes, you could check for:

all( abs(XD - double(single(XD))) < double(eps(single(XD))) )

but I believe that's equivalent to:

all(XD == double(single(XD)))

Felipe G. Nievinski <fgnievinski>
Thu 27 Feb 2014 02:45:12 PM UTC, comment #2: 

For the comparisons to be valid, would you need to specify a tolerance when doing the comparison?

Perhaps something like this?

XD = linspace (1, 10, 12);
YD = linspace (1, 20, 15);
XDs = single(XD);
YDs = single(YD);
%# absolute tolerance equality
isequalAbs = @(x,y,tol) (abs(x-y) <= tol );

if(all(isequalAbs(XD, XDs, 1e-6)) && all(isequalAbs(YD, YDs, 1e-6)))
%if all(XD == XDs) && all(YD == YDs)
   XD = XDs;
   YD = YDs;
endif
clear XDs YDs

Ian Journeaux <ijourneaux>
Thu 27 Feb 2014 02:58:20 AM UTC, comment #1: 

this seems safe to do:


XD = linspace (1, inCols, outCols);
YD = linspace (1, inRows, outRows);
XDs = single(XD);
YDs = single(YD);
if all(XD == XDs) && all(YD == XDs)
   XD = XDs;
   YD = YDs;
endif
clear XDs YDs

[XI, YI] = meshgrid (XD, YD);

im = imremap (im, XI, YI, method);


can you test if it works, please.


Felipe G. Nievinski <fgnievinski>
Sat 22 Feb 2014 11:30:23 PM UTC, original submission:  

When using imresize on larger images, I end up running out of memory. In the process of troubleshooting this, I noticed that in
imresize there is a call to meshgrid
 
[XI, YI] = meshgrid (linspace (1, inCols, outCols), linspace (1, inRows, outRows));

im = imremap (im, XI, YI, method);

Since the default cast for XI and YI is double, the memory footprint is very large for large images.

If the arrays XI and YI are cast as single the out of memory condition is avoided.

This is probably not the ideal solution (or even a good solution) but it dramatically reduces the memory footprint of imresize.

Ian

Ian Journeaux <ijourneaux>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by avinoam
  • -email is unavailable- added by xierui (Posted a comment)
  • -email is unavailable- added by hardy (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by fgnievinski (Posted a comment)
  • -email is unavailable- added by ijourneaux (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 14 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-09-12 nrjank StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2021-08-29 avinoam StatusNone In Progress
    2018-09-29 carandraug StatusFixed None
        Open/ClosedClosed Open
    2018-09-29 carandraug StatusNone Fixed
        Open/ClosedOpen Closed
    2018-03-06 avinoam Carbon-Copy- Added avinoam
    2018-03-06 avinoam Carbon-CopyRemoved -email is unavailable- -
    2018-03-06 avinoam Carbon-Copy- Added -email is unavailable-
    2017-08-13 jwe Summaryimage package: temporary arrays in imresize are doubles [octave forge] (image) temporary arrays in imresize are doubles
    2014-02-23 carandraug Release3.8.0 other
        Operating SystemMicrosoft Windows Any
        SummaryTemporary arrays in imresize are doubles image package: temporary arrays in imresize are doubles

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code