bugGNU Octave - Bugs: bug #38616, memory leak in regexprep

 
 

bug #38616: memory leak in regexprep

Submitted by:  None
Submitted on:  Thu 28 Mar 2013 08:17:36 AM UTC  
 
Category: InterpreterSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Crash
Status: Need InfoAssigned to: None
Originator Name: ceOriginator Email: -unavailable-
Open/Closed: ClosedRelease: dev
Operating System: Any

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 13 Oct 2013 04:38:19 AM UTC, comment #10:

We are getting ready to do the 3.8 release. Can you verify whether this bug is still a problem for you with the development code from Mercurial? My guess is that this is no longer a problem.

Rik <rik5>
Project Administrator
Thu 11 Apr 2013 09:15:00 PM UTC, comment #9:

Correction: results from last comment was a build of the latest stable -> 3.6.4 .
Building latest tip now.

Christoph Ellenberger <ellchr00>
Thu 11 Apr 2013 08:54:29 PM UTC, comment #8:

Ok I managed to build the latest tip on my box (Gentoo Core Duo E6750) at home.

I found that the following worked as well without a glitch:

but with your second suggestion I get the out of memory (As I don't have much experience with gdb...did I do it right?):

free shows the following before I start:

Additionally I tested the inital problem. This gives no Error here. This seems to be strictly Windows related and wil need the help of someone able to build under Windows.

Christoph Ellenberger <ellchr00>
Thu 11 Apr 2013 03:01:13 PM UTC, comment #7:

At least on Linux you can use the 'free' command at the command line to see how much memory you have. This will give you both actual memory and swap memory. If 2^23 is bigger than actual + swap then that would be the obvious problem, independent of Octave.

Rik <rik5>
Project Administrator
Thu 11 Apr 2013 01:00:25 PM UTC, comment #6:

Actually the 3.7.2+ I am running on a i5 750 (64 bit) [windows xp] but it is compiled for i686-pc-mingw32 from Michael Goffioul (VS2010) and here unfortunately I do not have a build environment. But at home under Linux I can do the tests (given some time).

And already 2^22 seems to be too big, as
s = repmat ('2', 1, 2^22);
s = regexprep (s, '2', '3');
gives me the "out of memory..." error.

There might be probably 2 things, as the indexing problem in my oppinion wouldn't explain for the initial error, that after the clear all, the second read-in with fscanf would fail.

Christoph Ellenberger <ellchr00>
Wed 10 Apr 2013 08:12:08 PM UTC, comment #5:

I started by attempting to reproduce the segfault you describe with this code.

Unfortunately this works fine for me with the system I have available for testing (3.6.4 on x86_64 Linux).

Are you running on a 32-bit or 64-bit architecture? This would affect the size of pointers which could be a problem.

Second, the final segfault is replacing 3/10 of the entries in s with a new value. 3/10 of the size of s is 6291456 or approximately 2^22.6. So in order to untangle things, does the following standalone code also cause a segfault?

Finally, would it be possible for you to compile with debugging and run under gdb to produce a backtrace?

Something like the following should do it assuming you are using a Bourne sh.

Rik <rik5>
Project Administrator
Thu 28 Mar 2013 04:53:02 PM UTC, comment #4:

I also attach TestData3.dat after:

fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
fclose(fid);
s = regexprep(s,'0','2');
save "TestData3.dat" s

(file #27703)

Christoph Ellenberger <ellchr00>
Thu 28 Mar 2013 04:37:59 PM UTC, comment #3:

Sure. As all files are not really small (20 MB) I zipped them. Surprisingly for me is that the binary version of a string is not really binary... All are the same also size(s) is always giving the same answer.

(file #27702)

Christoph Ellenberger <ellchr00>
Thu 28 Mar 2013 02:53:25 PM UTC, comment #2:

Could you upload some of the intermediate files to see if there is a problem with them?

I would like to see TestData.dat, TestData1.dat, and TestData2.dat after executing the following:

Rik <rik5>
Project Administrator
Thu 28 Mar 2013 09:22:08 AM UTC, comment #1:

As a reference, I show the original discussion on octave-help list:

http://octave.1599824.n4.nabble.com/memory-leak-in-regexprep-windows-3-7-2-VS-td4651218.html

Tatsuro MATSUOKA <tmacchant>
Thu 28 Mar 2013 08:17:36 AM UTC, original submission:

The following code produces an out of memory / memory exhausted error on octave 3.7.2+(VS), 3.6.2(VS, MinGW), 3.4.3 (MinGW) and 3.2.4 (MinGW) as well as on octave-3.6.4 under Linux:

#**************************************
s="0123456789";
for i=1:21, s=cstrcat(s,s);,endfor

s = regexprep(s,'0','2');
s = regexprep(s,'1','2');
s = regexprep(s,'2','3');
#**************************************

Also the following (it seems to be something persistent):
**********
s="0123456789";
for i=1:21, s=cstrcat(s,s);,endfor
save "TestData.dat" s

*****************************
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
warning: range error for conversion to character value
fclose(fid);
s = regexprep(s,'0','2');
clear all
fid = fopen("TestData.dat","r");
s = fscanf(fid,'%c');
******************************

results in (octave-3.6.2 VS and 3.6.2 MinGW [win 7 64 bit Home premium, 4GB memory])
error: memory exhausted or requested size too large for range of Octave's index type -- trying to return to prompt
and (octave 3.7.2+ VS) [win 7 64 bit Home premium, 4GB memory] and [Win XP, 4GB]
error: out of memory or dimension too large for Octave's
index type
but succeds in Octave-3.6.2 Cygwin, -3.2.4 MinGW and -3.4.3 MinGW

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #27703:  TestData3.zip added by ellchr00 (40kB - application/x-zip-compressed)
file #27702:  TestData.zip added by ellchr00 (120kB - application/x-zip-compressed)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by ellchr00 (Updated the item)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by tmacchant (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 17 Oct 2013 04:56:12 PM UTCrik5Open/ClosedOpen=>Closed
    Thu 28 Mar 2013 04:53:02 PM UTCellchr00Attached File-=>Added TestData3.zip, #27703
    Thu 28 Mar 2013 04:37:59 PM UTCellchr00Attached File-=>Added TestData.zip, #27702
    Thu 28 Mar 2013 02:55:56 PM UTCrik5StatusNone=>Need Info
    Thu 28 Mar 2013 09:23:03 AM UTCtmacchantCarbon-CopyRemoved -unavailable-=>-
    Thu 28 Mar 2013 09:22:08 AM UTCtmacchantCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup