bugGNU Octave - Bugs: bug #52355, load 4 times slower than dlmread

 
 

bug #52355: load 4 times slower than dlmread

Submitter:  A.R. Burgers <arb>
Submitted:  Mon 06 Nov 2017 07:07:24 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  None Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 07 Nov 2017 11:05:32 AM UTC, comment #3: 

check on EOL_tst.txt line endings:


file EOL_tst.txt
EOL_tst.txt: ASCII text, with CRLF, CR, LF line terminators


matlab output:


8.6.0.267246 (R2015b)

x =
     1     2     3
     4     5     6
     7     8     9
     1     2     3
     4     5     6
     7     8     9


A.R. Burgers <arb>
Mon 06 Nov 2017 11:16:05 PM UTC, comment #2: 

See also bug #51871.  One of the issues there is that the EOL delimiter could theoretically change on a per-line basis between UNIX, Mac, and Windows line endings.  If Octave only needed to support a single EOL per file, that would help a lot.  That my be an issue in this case as well.  If someone with access to Matlab could try loading the demonstration file EOL_tst.txt and report back what it prints that would be useful.

For Octave, results are


 x = load ('EOL_tst.txt')
x =

   1   2   3
   4   5   6
   7   8   9
   1   2   3
   4   5   6
   7   8   9



Rik <rik5>
Group administrator
Mon 06 Nov 2017 09:21:28 PM UTC, comment #1: 

That's because load can handle a range of field separators such as " ", "\t", "," (and perhaps more), and they can even be mixed wildly in your input file, which still will be correctly read. Probably there is no valid use-case for this feature (btw, what does matlab do for mixed field delimiters?), but that flexibility will cost you.

Anonymous
Mon 06 Nov 2017 07:07:24 PM UTC, original submission:  

load is about 4 times slower on a csv type file than dlmread.
I did expect the more specialized dlmread to be faster, but didn't expect a factor 4.


ver = 4.3.0+
hg_id = 2ac103974d00+
time for load 2.60886
time for dlmwrite 0.757713
ans = 0


A.R. Burgers <arb>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #42336:  load_slow.m added by arb (273B - application/octet-stream)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by arb (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-11-06 rik5 Dependencies- Depends on bugs #51871
    2017-11-06 arb Attached File- Added load_slow.m, #42336

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code