bugGNU Octave - Bugs: bug #52355, load 4 times slower than dlmread

 
 

bug #52355: load 4 times slower than dlmread

Submitted by:  A.R. Burgers <arb>
Submitted on:  Mon 06 Nov 2017 07:07:24 PM UTC  
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  None Assigned to:  None
Originator Name:  Open/Closed:  Open
Release:  dev Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 07 Nov 2017 11:05:32 AM UTC, comment #3: 

check on EOL_tst.txt line endings:

file EOL_tst.txt
EOL_tst.txt: ASCII text, with CRLF, CR, LF line terminators

matlab output:

8.6.0.267246 (R2015b)

x =
     1     2     3
     4     5     6
     7     8     9
     1     2     3
     4     5     6
     7     8     9

A.R. Burgers <arb>
Mon 06 Nov 2017 11:16:05 PM UTC, comment #2: 

See also bug #51871.  One of the issues there is that the EOL delimiter could theoretically change on a per-line basis between UNIX, Mac, and Windows line endings.  If Octave only needed to support a single EOL per file, that would help a lot.  That my be an issue in this case as well.  If someone with access to Matlab could try loading the demonstration file EOL_tst.txt and report back what it prints that would be useful.

For Octave, results are

 x = load ('EOL_tst.txt')
x =

   1   2   3
   4   5   6
   7   8   9
   1   2   3
   4   5   6
   7   8   9

Rik <rik5>
Project Administrator
Mon 06 Nov 2017 09:21:28 PM UTC, comment #1: 

That's because load can handle a range of field separators such as " ", "\t", "," (and perhaps more), and they can even be mixed wildly in your input file, which still will be correctly read. Probably there is no valid use-case for this feature (btw, what does matlab do for mixed field delimiters?), but that flexibility will cost you.

Anonymous
Mon 06 Nov 2017 07:07:24 PM UTC, original submission:  

load is about 4 times slower on a csv type file than dlmread.
I did expect the more specialized dlmread to be faster, but didn't expect a factor 4.

ver = 4.3.0+
hg_id = 2ac103974d00+
time for load 2.60886
time for dlmwrite 0.757713
ans = 0

A.R. Burgers <arb>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #42336:  load_slow.m added by arb (273B - application/octet-stream)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by arb (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-11-06 rik5 Dependencies- => Depends on bugs #51871
    2017-11-06 arb Attached File- => Added load_slow.m, #42336

    Back to the top


    Powered by Savane 3.5