bugGNU Octave - Bugs: bug #31512, Problem with code on Linux octave...

 
 

bug #31512: Problem with code on Linux octave 3.2.3 and 3.2.4/// OK on mac w/ 3.2.3

Submitter:  None
Submitted:  Mon 01 Nov 2010 07:38:52 PM UTC
   
 
Category:  None Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Works For Me Assigned to:  None
Originator Name:  kody Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 3.2.3
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 19 Nov 2010 10:47:52 PM UTC, comment #17: 

I was going to start a new thread, but apparently I am no longer authorized to submit to this group .... if anyone knows how to be readmitted, I would appreciate it.

So, the new peculiarity is attached.  Any suggestions about how to speed it up would be greatly appreciated.  Thanks a lot for all your help!

A takes about 2.5 seconds on matlab, while B takes 2 seconds.  On octave the difference is more than an order of magnitude!

%%%% when A is uncommented and B is commented,
%%%% the routine takes 40 secs!

%%%% when B is uncommented and A is commented,
%%%% the routine takes 2 secs!

%%%% each time, the program sorts a permutation into
%%%% cycles, going through each element.  'lens' lists
%%%% the number of elements in permutations of each length

%%%% longer permutations =
%%%% fewer entries into while loop =
%%%% faster -- too much faster!

Thanks again, best


(file #22063)

kody law <kodo>
Fri 19 Nov 2010 10:34:09 PM UTC, comment #16: 

Thank you for the thoughtful and informative comments, Rik.

I will try installing another Linux distribution, although being a non-expert, I may not be successful!

About the radnomness, it actually does not play a role and the term with the randomness can be set to 0 (line 110).  Sorry, the code is a little cluttered and there is a lot of stuff in there which is not used.  For instance, in the evolution, there is a random term pre-multiplied by the variable "sto" which is set to 0.

Incidentally, I have a routine in another program which runs in 2 seconds in matlab, but takes 40 seconds in octave! - even on the same system!  Will post another thread.

Thanks again, cheers

kody law <kodo>
Fri 19 Nov 2010 03:41:37 AM UTC, comment #15: 

Opensuse desktop should have been vanilla enough.  It is perhaps only coincidental, but the two failing systems are Opensuse (desktop and enterprise) whereas working systems include your Mac OS and my Kubuntu 9.04.  My next suggestion, therefore, is to try something besides Opensuse.  Ubuntu is pretty easy to download as a liveCD and install on a USB memory stick.  Or if you have a free partition it will be simpler to just do a hard drive install.

Some background on Octave which is germane to your problem.  Octave is an interpreter.  It parses your commands, but hands off large fractions of the actual computation to other libraries, such as the FFTW library for FFTs or BLAS for matrix multiplications.  Given that the code runs correctly on different versions of Octave (3.2.3, 3.2.4, and development version), I don't think there is an issue with the parser.  This leaves the next most obvious candidate as the libraries that Octave is using for backend calculation.  That is why I'm suggesting trying another Linux distribution, where the libraries could have been compiled very differently.

A final thought, your code does initialize the random number generators to get reproducible results.  Perhaps this initialization is not working as expected.  Maybe try simplifying the initalization to just:

rand('twister',1)
randn('state',2)


Rik <rik5>
Group administrator
Sun 14 Nov 2010 12:46:45 AM UTC, comment #14: 

Many thanks, John, and Rik.  I do understand you are volunteers and learning, my apologies for referring to you as octave support. 

I am not sure exactly what vanilla system means, but there were 2 systems on which it didn't work.  The cluster, from which the system specifications came AND

Suse v11, installed as is, default from the internet.

Nonetheless, I am still very hopeful that you can understand this, if you are interested and I will forward your suggestions about the cluster install to support here.

Thanks,
Kody.

kody law <kodo>
Tue 09 Nov 2010 06:20:14 AM UTC, comment #13: 

I think the following changeset improves Matlab compatibiliy of mod and rem:

  http://hg.savannah.gnu.org/hgweb/octave/rev/94d9d412a2a0

See also bug #31579

John W. Eaton <jwe>
Group administrator
Sun 07 Nov 2010 10:31:11 PM UTC, comment #12: 

I didn't realize that you were running your own custom-compiled Octave with custom-built libraries.  I think the first thing to do is try your code on a vanilla system (i386, vendor-supplied libraries and Octave binaries).  You can use any of the distros which support LiveCD such as Debian, Ubuntu, RedHat, etc.

The compilation flags you list ("-O3 -fpmath=sse") are rather aggressive.  In particular, "-O3" is known to occasionally break pieces of code.  I also quote from the gcc documentation regarding "-fpmath=sse"

-- Start of Quote --
Use scalar floating point instructions present in the SSE instruction set. This instruction set is supported by Pentium3 and newer chips, in the AMD line by Athlon-4, Athlon-xp and Athlon-mp chips. The earlier version of SSE instruction set supports only single precision arithmetics, thus the double and extended precision arithmetics is still done using 387. Later version, present only in Pentium4 and the future AMD x86-64 chips supports double precision arithmetics too.

 For the i386 compiler, you need to use -march=cpu-type, -msse or -msse2 switches to enable SSE extensions and make this option effective. For the x86-64 compiler, these extensions are enabled by default.

 The resulting code should be considerably faster in the majority of cases and avoid the numerical instability problems of 387 code, but may break some existing code that expects temporaries to be 80bit. [Emphasis mine]

 This is the default choice for the x86-64 compiler.

-- End of Quote --

This might be part of the problem as well.  Finally, I note my gcc is more recent, 4.4.1, than yours.  But really, do start with a vanilla system and see if that works.

Rik <rik5>
Group administrator
Sun 07 Nov 2010 04:23:37 PM UTC, comment #11: 

Please stop referring to us as "Octave support".  There is no such thing.  We are all individual volunteers here, just like you.  I agree that the mod/rem function could perform better, but I would probably not use it to compute remainders for floating point numbers since that is likely to give unpredictable results.  So for me, the problem is relatively low priority.  I have no idea about the other problem you are seeing, but since it does work on some systems and not others, that again points to problems with a system library, not Octave itself.  On the system where there are problems, have you tried building Octave or the supporting libraries with different optimization options to see if that might be the source of the problem?  No, I do not have access to any other systems to use for building or testing Octave.

John W. Eaton <jwe>
Group administrator
Sun 07 Nov 2010 02:03:14 PM UTC, comment #10: 

Many thanks for your attention, John.

I should clarify that there are TWO issues here, and we are focusing on the trivial (perhaps also easier) of the two.  The remainder problem always exists, across all systems.  This is intrinsic to octave, and a difference between matlab which I agree would be a good thing to resolve, but I was mentioning as a side-comment, and I am not too concerned about. 

Nonetheless, again, a completely separate issue is central to this string and that is the blow-up on 2 distinct systems of the code I attached below.  And, again, this code runs properly on matlab and on octave on my mac, as well as Kubuntu 9.10, with a 64-bit kernel. 

It does not work on

SuSe v.11

Francesca is running 64-bit SLES10 SP3 with kernel 2.6.16.60-0.69.1-smp. GCC
version is 4.1.2 and Octave was built with compiler options "-O3 -mfpmath=sse"
and linked against libbgsl and fftw3.

in the sense described below, that the variable en blows up.



AGAIN, forwarded priority sub-string of this submission:

Sat 06 Nov 2010 01:00:39 PM GMT, comment #6:

Further to the first comment of Rik. We have failure on 2 systems... is it possible for you to run on another system?

REPLY of support here:

Well, we also know the code does not work on ITS Linux desktops (SuSe v.11). So it's
not exactly an issue affecting a single system as suggested by Octave support.
Also, they only tested on one configuration and we have failures on two
independent systems and across two Octave versions, so perhaps they are drawing
conclusions prematurely.

For Octave support:

Francesca is running 64-bit SLES10 SP3 with kernel 2.6.16.60-0.69.1-smp. GCC
version is 4.1.2 and Octave was built with compiler options "-O3 -mfpmath=sse"
and linked against libbgsl and fftw3.







Re: comment #8 of John, I get the same results indeed.  Again, I do not find the problem with rem/mod to be system specific.




Many thanks again to both for your attention and help!  I really hope we are able to sort out the cause of the blow-up (and not ` bad idea to sort out the rem/mod business too).


Kody.




kody law <kodo>
Sat 06 Nov 2010 03:01:42 PM UTC, comment #9: 

Rik wrote: "If Matlab is playing tricks with the divisor it would be very interesting to know that."

The current Matlab documentation for rem and mod says that it uses something like

  x - y * floor (x / y)  (for rem, replace floor with fix)

and that

"If Y is not an integer and the quotient X./Y is within roundoff error of an integer, then n is that integer."

So yes, I guess they are doing some tricks.  I'm not sure that is really the best thing to do.  I mean, maybe the correct result is actually not an integer.  So why perform the rounding ?  How can you know when that is appropriate?result an integer.  But maybe for compatibility, we will have to do the same.  Otherwise, I'm sure that we will just see complaints about the difference.

John W. Eaton <jwe>
Group administrator
Sat 06 Nov 2010 02:54:59 PM UTC, comment #8: 

That you get "correct" results with Octave on one system and not another would seem to indicate a problem with the system math library, not Octave, but I would still like to understand what is happening.  I can see some issues with the way ranges are implemented in the current sources (but not 3.2.x, I think).  I will start a discussion on the Octave maintainers mailing list about that.

I also see what I think is unusual behavior of fmod on my system.  What does the following program print on your system where you have the problem with rem/mod?  Here, I see


x = 0.5
y = 0.1
floor (x / y) = 5
x - y * floor (x / y) = 0
fmod (x, y) = 0.1


which is definitely a surprising result.  Since fmod is the function used by Octave to compute the remainder, this is the same result that Octave will compute on my system.  If this is a bug, then I think the right place to fix it is in the system math library, or perhaps in gnulib.


#include <iostream>
#include <cmath>

int
main (void)
{
  double x = 0.5;
  double y = 0.1;
  std::cout << "x = " << x << std::endl;
  std::cout << "y = " << y << std::endl;
  std::cout << "floor (x / y) = " << floor (x / y) << std::endl;
  std::cout << "x - y * floor (x / y) = " << x - y * floor (x / y) << std::endl;
  std::cout << "fmod (x, y) = " << fmod (x, y) << std::endl;

  return 0;
}


John W. Eaton <jwe>
Group administrator
Sat 06 Nov 2010 01:11:02 PM UTC, comment #7: 



As for the new comment of Rik, I get correct remainders in octave (3.2.3 on mac) AND matlab (7.9.0.529 (R2009b) on mac)....

octave:4> format long
octave:5> rem([2;2.1],.1)
ans =

   0
   0


>> format long
>> rem([2;2.1],.1)


ans =

     0
     0

kody law <kodo>
Sat 06 Nov 2010 01:00:39 PM UTC, comment #6: 

Further to the first comment of Rik.  We have failure on 2 systems... is it possible for you to run on another system?

REPLY of support here:

Well, we also know the code does not work on ITS Linux desktops (SuSe v.11). So it's
not exactly an issue affecting a single system as suggested by Octave support.
Also, they only tested on one configuration and we have failures on two
independent systems and across two Octave versions, so perhaps they are drawing
conclusions prematurely.

For Octave support:

Francesca is running 64-bit SLES10 SP3 with kernel 2.6.16.60-0.69.1-smp. GCC
version is 4.1.2 and Octave was built with compiler options "-O3 -mfpmath=sse"
and linked against libbgsl and fftw3.

kody law <kodo>
Fri 05 Nov 2010 11:49:43 PM UTC, comment #5: 

I think JWE is right.  You might want to try the following code in Octave and Matlab.


format long
rem([2; 2.1], 0.1)
ans =

   0.0999999999999999
   0.1000000000000000


As you can see, 2.0/0.1 leaves a remainder when calculated with binary numbers which would not be the case for base10 numbers.

If Matlab is playing tricks with the divisor it would be very interesting to know that.

Rik <rik5>
Group administrator
Fri 05 Nov 2010 08:51:25 PM UTC, comment #4: 

"I don't think this is a bug. Neither .025 or .1 are exactly representable in binary, so you are not computing what you think you are."


Yet, the exact same sequence of commands leads to the correct answer with matlab ....

perhaps matlab recognizes non-integers in the command and multiplies everything by 10? ....

also, should .25 not be representable either, then?

kody law <kodo>
Fri 05 Nov 2010 08:36:09 PM UTC, comment #3: 

"FYI - another thing I noticed consistently with octave is that it can compute the correct mod operation only with integers. For example, try

for pp=1:1e4
if mod(pp*.025,.1)==0"

I don't think this is a bug.  Neither .025 or .1 are exactly representable in binary, so you are not computing what you think you are.

John W. Eaton <jwe>
Group administrator
Fri 05 Nov 2010 08:12:37 PM UTC, comment #2: 

Thanks a lot for your help!

This is a predicament.  I know that one machine is running a new(er) version of SuSe, with (I think) a 32-bit kernel.  I will find out for sure.


FYI - another thing I noticed consistently with octave is that it can compute the correct mod operation only with integers. For example, try

for pp=1:1e4
if mod(pp*.025,.1)==0
q=q+1;
end
end

pp/q


Now, see

for pp=1:1e4
if mod(pp*.25,1)==0
q=q+1;
end
end

pp/q


Thanks again,
Kody


kody law <kodo>
Tue 02 Nov 2010 03:46:20 PM UTC, comment #1: 

Good news/bad news.  Good news is that your code seems sound.  I ran it on octave versions 3.2.3, 3.2.4, and a development version.  In none of them does the variable en blow up.  For all of them I get:

max(en)
ans =  9.5609


The bad news is that your issue is not reproducible, which means it is likely a problem with the specific libraries, compilation options for octave, etc. on your Linux machine.  For reference, I tested your code on a Linux machine running Kubuntu 9.10, with a 64-bit kernel.

Unless you can more specifically isolate the problem to Octave, I don't believe this is an Octave issue and this bug report will eventually be closed.

Rik <rik5>
Group administrator
Mon 01 Nov 2010 07:38:52 PM UTC, original submission:  

Hello,

I have a code which runs forward solves of Navier Stokes with a 4th order etd method.  It works fine on matlab and on octave 3.2.3 on my mac, but when i submit to queue or even run it on a linux workstation running 3.2.4, it consistently blows up!  (as though something were wrong with the code - such a situation would ordinarily be remedied by).  Other than standard operations, it runs a number of FFTs.

Further to this, I have now run the exact same code on another Linux machine (SuSe v.11) with octave 3.2.3 and it also blows up!

The code (attached) is run by the command EVG and then the energy should remain roughly constant and order 1.  On the first machine (3.2.4) it blows up at 482

octave:14> en(482)
ans =  2.1751e+05

The second (3.2.3) it blows up later

octave:17> en(533)
ans =  9.6420e+05

Many thanks in advance. 

Best regards,
Kody

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #22063:  per.tar added by kodo (103KiB - application/x-tar)
file #21893:  test.tar added by None (100KiB - application/x-tar)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by kodo (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-09-04 rik5 Open/ClosedOpen Closed
    2010-11-19 kodo Attached File- Added per.tar, #22063
    2010-11-02 rik5 StatusNone Works For Me
    2010-11-01 None Attached File- Added test.tar, #21893

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code