bugGNU Octave - Bugs: bug #39505, dlmwrite with coffset and roffset...

 
 

bug #39505: dlmwrite with coffset and roffset >0, inserts extra characters

Submitted by:  imartinez <imartinez>
Submitted on:  Wed 17 Jul 2013 06:22:35 AM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: InvalidAssigned to: None
Originator Name: imartinezOpen/Closed: Closed
Release: 3.6.3Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Fri 19 Jul 2013 12:34:32 PM UTC, comment #17:

Like Octave, Matlab does write the tabs on file, but you (and wordpad, and your microwave S2P program) don't see them because they're "whitespace" (if you don't know what that is look it up on google).

Friendly advice: please dump wordpad asap - use a real editor.
Hint: I find notepad++ to be very handy for editing even very large data files (and I run it in Linux under wine).

You did hit single vs. double quote issues in Octave, sorry for that; but that has been fixed now.
Double vs. single quotes is a bit of an Octave idiosyncrasy (or I'd rather say: a Matlab deficiency), you're not the first to stumble over it.

Anyway the second I press the "submit" button will be the last one I'll waste on this.

Philip Nienhuis <philipnienhuis>
Project Member
Wed 17 Jul 2013 10:08:46 PM UTC, comment #16:

Calm down, Philip Nienhuis.... it needs several cases to check
what is going on.

in matlab with tabs first and then with commas...

dlmwrite('matrix_test.txt',a,'-append','delimiter', '\t','roffset',0,'coffset',0);

>> dlmwrite('matrix_test.txt',b,'-append','delimiter', '\t','roffset',0,'coffset',0);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', '\t','roffset',4,'coffset',0);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', '\t','roffset',4,'coffset',4);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', ',','roffset',0,'coffset',0);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', ',','roffset',1,'coffset',0);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', ',','roffset',2,'coffset',0);
>> dlmwrite('matrix_test.txt',b,'-append','delimiter', ',','roffset',3,'coffset',0);
>>


Tabs are ok in matlab, empty lines are generate when you open the file in wordpad. This is what a microwave S2P file needs

I do recognize that with commas, it is inserting commas on the blank lines.

So, I think it still holds water at about 50%

1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
6 5 4 3 2
6 5 4 3 2
6 5 4 3 2

6 5 4 3 2
6 5 4 3 2
6 5 4 3 2

6 5 4 3 2
6 5 4 3 2
6 5 4 3 2
6,5,4,3,2
6,5,4,3,2
6,5,4,3,2
,,,,
6,5,4,3,2
6,5,4,3,2
6,5,4,3,2
,,,,
,,,,
6,5,4,3,2
6,5,4,3,2
6,5,4,3,2
,,,,
,,,,
,,,,
6,5,4,3,2
6,5,4,3,2
6,5,4,3,2

imartinez <imartinez>
Wed 17 Jul 2013 10:08:43 PM UTC, comment #15:

I think this bug report is done. Matlab and Octave both print out lines full of delimiters if the roffset option is used. Along the way we at least found and fixed the '\t' interpretation issue.

Rik <rik5>
Project Administrator
Wed 17 Jul 2013 09:57:57 PM UTC, comment #14:

@comment #12 (ML r2013b prerel.):

Philip Nienhuis <philipnienhuis>
Project Member
Wed 17 Jul 2013 09:48:49 PM UTC, comment #13:

Mike, I fixed the interpretation of '\t' for dlmwrite (http://hg.savannah.gnu.org/hgweb/octave/rev/1a1e831fe6b4). It was already being handled correctly for dlmread.

Rik <rik5>
Project Administrator
Wed 17 Jul 2013 09:30:25 PM UTC, comment #12:

I don't think there is any issue with coffset, although it is worth checking.

What file does the following code sample produce?

We need to check csvwrite because Octave uses dlmwrite internally for csvwrite. I don't know whether it will also produce blank lines or not.

Rik <rik5>
Project Administrator
Wed 17 Jul 2013 09:25:37 PM UTC, comment #11:

(Trying again, my comment was cut off half-way....)

No I didn't read back in Octave, I just tried your original example in Octave with double quotes around \t. That gave a beautiful text file with -seemingly- empty lines - but read on.

Trying you original example in Matlab r2013b pre-release gives:

so your example doesn't work as-is in Matlab; 'append' is not a valid user-configurable option in ML.

About your comment #4, did you literally try it in Matlab?
If I try (in ML r2013b pre-release):

I get a text file looking like

so your claim that ML writes empty lines corresponding to empty data rows in case of printable characters doesn't hold water. Not even for non-printable delimiters like tabs, my example in comment #5 gives the exact same results in ML (but with single quotes around \t in the dlmwrite argument list).

As to your comment #7, I don't follow. Matlab says:

All in all I think you'd better re-read the docs.
I tend to close this bug with "invalid" but I'll await Jordi's and Rik's opinions.

The "\t" versus '\t' thing is a different issue. Is this documented? If not it may be worth another bug report.

Philip Nienhuis <philipnienhuis>
Project Member
Wed 17 Jul 2013 09:16:13 PM UTC, comment #10:

No I didn't read back in Octave, I just tried your original example in Octave with double quotes around \t. That gave a beautiful text file with -seemingly- empty lines - but read on.

Trying you original example in Matlab r2013b pre-release gives:

so your example doesn't work as-is in Matlab; 'append' is not a valid user-configurable option in ML.

About your comment #4, did you literally try it in Matlab?
If I try (in ML r2013b pre-release):

Philip Nienhuis <philipnienhuis>
Project Member
Wed 17 Jul 2013 09:01:32 PM UTC, comment #9:

I now see in the MATLAB description that dlmread and dlmwrite both interpret the string '\t' as a tab character. I think we may simply need to handle this for compatibility the same way we're doing it in importdata.

http://hg.savannah.gnu.org/hgweb/octave/file/78f57b14535c/scripts/io/importdata.m#l303

Mike Miller <mtmiller>
Project Administrator
Wed 17 Jul 2013 08:44:13 PM UTC, comment #8:

Also jumping in...

Reading through the comments and running the provided snippets, there are two potential problems here, and I'm also unclear which one the reporter is claiming as the bug.

1. OP's comment 4 seems to indicate that a positive "roffset" inserts blank lines in MATLAB rather than lines with back-to-back delimiters. I cannot verify this at the moment, and it's unclear to me from the API documentation whether the offset lines should be blank or filled with delimiters. It's also unclear whether the OP meant that the line appeared blank in a text editor but was actually filled with tab characters.

2. There is definitely some inconsistency in dlmwrite when a single-quoted string is used with what would be an escape sequence if it were double-quoted, see below.

I'm using cat -T so the actual tab characters are shown as ^I, while "\t" is two characters.

Mike Miller <mtmiller>
Project Administrator
Wed 17 Jul 2013 08:43:37 PM UTC, comment #7:

Those zeros are not supposed to be there.....
They are supposed to be
,,,,,,,,,

imartinez <imartinez>
Wed 17 Jul 2013 08:34:26 PM UTC, comment #6:

Are you reading the file back on octave??

I am writing on octave and opening the file in a text editor to check it.

if you read back on octave, it looks ok

x = dlmread ('matrix_text.txt', ',')
x =

2 3 4 5 6
2 3 4 5 6
2 3 4 5 6
0 0 0 0 0
0 0 0 0 0
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5

octave does not have problems because the extra characters are read are valid empty lines.

READ THE NEXT STATEMENT CAREFULLY

if you open the file in an EXTERNAL TEXT EDITOR (like KWRITE)

2,3,4,5,6
2,3,4,5,6
2,3,4,5,6
,,,,
,,,,
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5
,,,,
,,,,
,,,,
,,,,
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5

If the file goes to an external application, then the little ,,,,,
are a problem.

it is HIS code

imartinez <imartinez>
Wed 17 Jul 2013 07:17:36 PM UTC, comment #5:

PMFJI

Do tabs need to be entered between double quotes rather than single quotes to write tab characters? On my WinXP box with octave-3.7.5 that is indeed the case.
Maybe that is the OP's problem.

If I run his example in his (her?) original post with "\t" rather than '\t', I get (hopefully savannah leaves tab formatting intact):

Philip Nienhuis <philipnienhuis>
Project Member
Wed 17 Jul 2013 06:54:00 PM UTC, comment #4:

>>a=[1 2 3 4 5;1 2 3 4 5;1 2 3 4 5];


>> b=[2 3 4 5 6;2 3 4 5 6;2 3 4 5 6]

b =

2 3 4 5 6
2 3 4 5 6
2 3 4 5 6

>>dlmwrite('matrix_text.txt',b,'-append','on','delimiter', ',','roffset',0,'coffset',0);


%leave or try to leave a blank line

>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', ',','roffset',2,'coffset',0);


2,3,4,5,6
2,3,4,5,6
2,3,4,5,6
,,,, (this is supposed to be blank)
,,,, (this is also supposed to be blank)
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5

MATLAB:

2,3,4,5,6
2,3,4,5,6
2,3,4,5,6

(BLANK as expected)
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5

imartinez <imartinez>
Wed 17 Jul 2013 06:41:09 PM UTC, comment #3:

Bug was re-opened. What is the actual problem? dlmwrite seems to be respecting the documentation regarding inserting delimiters if roffset or coffset is used. The Mathworks documentation is here (http://www.mathworks.com/help/matlab/ref/dlmwrite.html).

I changed the sample code to use a comma as the delimiter so that I can see what is going on.

If I then read the file back in with dlmread all of the test matrices are in their proper positions.

"z

I'm attaching the actual file generated as matrix_text.octave.txt. If you have access to Matlab you could run the code in this comment and compare the generated file to matrix_text.octave.txt. If there is a difference then we can look into it, but Octave appears to be following the documentation correctly.

(file #28606)

Rik <rik5>
Project Administrator
Wed 17 Jul 2013 05:23:24 PM UTC, comment #2:

This bug report was followed in bug #39511. I have requested the reporter to continue here instead.

Jordi GutiƩrrez Hermoso <jordigh>
Project Administrator
Wed 17 Jul 2013 03:34:35 PM UTC, comment #1:

Which part is a bug? roffset and coffset are used to position the matrix being written in the destination file. For example, roffset=1 means one entire line of delimiters will be written. A coffset=2 means 2 delimiters will be prepended to every row written out.

Example

Notice how the second matrix portion has been offset from the first as specified by roffset and coffset.

Rik <rik5>
Project Administrator
Wed 17 Jul 2013 06:22:35 AM UTC, original submission:

I think this is a bug... I ran a few things on the command line

dlmwrite with roffset and coffset =0 seems ok

with roffset > 0

it is inserting tabs characters..

code

>> a=[1 2 3 4 5;1 2 3 4 5;1 2 3 4 5];


>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', '\t','roffset',0,'coffset',0);


>> b=[2 3 4 5 6;2 3 4 5 6;2 3 4 5 6];


>>dlmwrite('matrix_text.txt',b,'-append','on','delimiter', '\t','roffset',0,'coffset',0);


>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', '\t','roffset',1,'coffset',0);


>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', '\t','roffset',2,'coffset',0);


>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', '\t','newline','unix','roffset',3,'coffset',0);


>>dlmwrite('matrix_text.txt',a,'-append','on','delimiter', '\t','newline','unix','roffset',3,'coffset',3);


file looks like:

1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
2 3 4 5 6
2 3 4 5 6
2 3 4 5 6
\t\t\t\t
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
\t\t\t\t
\t\t\t\t
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
\t\t\t\t
\t\t\t\t
\t\t\t\t
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
\t\t\t\t\t\t\t
\t\t\t\t\t\t\t
\t\t\t\t\t\t\t
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5

imartinez <imartinez>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #28606:  matrix_text.octave.txt added by rik5 (243B - text/plain)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by jordigh (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by imartinez (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 17 Jul 2013 10:08:43 PM UTCrik5StatusConfirmed=>Invalid
      Open/ClosedOpen=>Closed
    Wed 17 Jul 2013 06:41:09 PM UTCrik5Attached File-=>Added matrix_text.octave.txt, #28606
    Wed 17 Jul 2013 05:24:15 PM UTCjordighOpen/ClosedClosed=>Open
    Wed 17 Jul 2013 05:23:24 PM UTCjordighStatusInvalid=>Confirmed
      Dependencies-=>Depends on bugs #39511
    Wed 17 Jul 2013 03:34:35 PM UTCrik5StatusNone=>Invalid
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1