bugGNU Octave - Bugs: bug #63930, fprintf problem

 
 

bug #63930: fprintf problem

Submitter:  Charles Praplan <charprap>
Submitted:  Wed 15 Mar 2023 05:09:22 PM UTC
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Need Info Assigned to:  None
Originator Name:  Charles Praplan Open/Closed:  * Open
Release:  * 8.1.0 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

( Jump to the original submission )

Mon 20 Mar 2023 10:20:41 PM UTC, comment #15: 

@Charles, I see that you linked bug #63931. There's also a related bug #63415. Most of the discussion is happening there if you're interested in tracking this problem.

A good summary of what went wrong is here:
https://savannah.gnu.org/bugs/?63415#comment33


Arun Giridhar <arungiridhar>
Project Member
Sun 19 Mar 2023 05:49:53 PM UTC, comment #14: 

Yes, this is a serious bug and switching to utf-8 is a temp workaround.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 19 Mar 2023 05:44:27 PM UTC, comment #13: 

Thank you for your answer,

I agree that changing the encoding seems to solve the problem even for binary files.

But I assume that your proposal is only a workaround and that the problem will be corrected soon. Right?

What's the point of having the best computation software, if the results cannot be output reliably?

And regarding binary files, from my point of view, write operations should not be dependent on the encoding. The following simple code should always output the same file, but the result depends on the encoding.

file = 'file_251.bin';
fid = fopen(file, 'w');
fwrite(fid, 251);
f_error = fclose(fid);



Charles Praplan <charprap>
Sun 19 Mar 2023 02:26:30 PM UTC, comment #12: 

Change your editor encodings to "UTF-8"

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 19 Mar 2023 02:19:50 PM UTC, comment #11: 

It has everything to do with encoding. If I change my encoding
(in editor setting) to iso-8859-1 the problem appears again:



>> t1
val = 193
cw = 1
cr1 = 1
x = 193
y = 63
------
val = 194
cw = 1
cr1 = 0
x = 194
error: b(1): out of bound 0 (dimensions are 0x0)
error: called from
    t1 at line 15 column 5
>> t2
val = 127
cw = 1
cr1 = 1
x = 127
y = 127
------
val = 128
cw = 1
cr1 = 1
x = 128
y = 63
------
val = 129
cw = 1
cr1 = 1
x = 129
y = 63
------


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 19 Mar 2023 02:08:08 PM UTC, comment #10: 

Bugs https://savannah.gnu.org/bugs/?63930 and https://savannah.gnu.org/bugs/?63931 are probably related.

When writing bytes with large value to a file (either with fprintf for text files or with fwrite for binary files) the desired value is sometimes replaced by 0x3F.

Thus, this has probably nothing to do with code pages and encoding...

Charles Praplan <charprap>
Sat 18 Mar 2023 10:06:26 AM UTC, comment #9: 

I guess I didn't mention my settings in the octave editor:

Text encoding ... : SYSTEM (CP1252)

Charles Praplan <charprap>
Sat 18 Mar 2023 09:59:23 AM UTC, comment #8: 

I found some systematical behaviour of the problem. This is described below:

Test cases and comparisons are appended (fprintf_bug.tgz). I also join pictures of the
comparisons because I'm not sure that you see the same thing than me.

1)
If the m-file only writes special characters to the file, the proble does not appear.

Test case : fprintf_bug4.m => 04.txt

2)
Once a standard character is outputted (incl \n), each 16th character is wrong (replaced by char 3F).

Test case : fprintf_bug5.m   =>  05.txt


3)
If a second standard character is outputted, the problem disappear for the rest of the file.

Test case : fprintf_bug6.m   =>  06.txt

4)
if a third standard character is outputted, the problem appears again for the rest of the file.

Test case : fprintf_bug7.m   =>  07.txt


(file #54508)

Charles Praplan <charprap>
Thu 16 Mar 2023 03:56:23 PM UTC, comment #7: 

The 2 text files are now appended.
Thank you.

(file #54499, file #54500)

Charles Praplan <charprap>
Thu 16 Mar 2023 01:33:03 PM UTC, comment #6: 

Thanks.

The encoding of the strings is actually converted twice.
1. When parsing the .m file, the entire file is converted from codepage 1252 to UTF-8. (That is done so that Octave can potentially feed those strings to other libraries, like for regex matching or printing text in figures.)
2. When writing to the file, each character array is converted from UTF-8 to the target encoding. That is either the one specified when opening the file (with `fopen`), or, if none is specified, the encoding that is used for parsing .m files (the encoding returned by `__mfile_encoding__`). That was done to be compatible with Matlab.

I'm guessing that something might be going wrong in the second conversion if the output is buffered, and the buffer is flushed when the end of it happens to be "between" bytes of a multi-byte UTF-8 character...

Markus Mützel <mmuetzel>
Project Member
Thu 16 Mar 2023 12:08:45 PM UTC, comment #5: 

No, the 2 text files are created on the same PC.
But I tried another PC to be more convinced that the problem is not in my PC.
I will attach the generate files later.

Running the script creates 2 files which should have only one difference.
But I get one second difference.

Effectively the problem occurs with
&#251; | \373 | =FB | u ^ | (û) | Small u, circumflex accent

Running fprintf_bug1.m, octave outputs this character most of the time correctly but sometimes not: char 0x3F is sometimes outputted instead of 0xFB.

I do not want to use UTF-8 encoding. I want to generate pure text files.

Charles Praplan <charprap>
Thu 16 Mar 2023 07:27:22 AM UTC, comment #4: 

Do I understand correctly that the two text files were created on two different PCs with different locale?

Instead of images of text, could you please attach the actual generated text files?

Markus Mützel <mmuetzel>
Project Member
Wed 15 Mar 2023 11:30:01 PM UTC, comment #3: 

Yes the scripts apparently are using "latin1" encoding


&#233; | \351 | =E9 | e ' | (é) | Small e, acute accent

and

&#251; | \373 | =FB | u ^ | (û) | Small u, circumflex accent


I think octave expects UTF-8 encoding, so it gets confused.
If I change those to UTF-8 characters, the output looks OK.
Converted .m files are attached.

Dmitri.
--





(file #54495, file #54496)

Dmitri A. Sergatskov <dasergatskov>
Wed 15 Mar 2023 09:25:41 PM UTC, comment #2: 

It seems that there might be an issue with the encoding of the text file being written by the fprintf function in Octave 8.1.0. The character on line 29 that is being printed incorrectly might be due to a mismatch in the encoding being used by the function and the encoding of the file.

One possible solution to this issue is to explicitly specify the encoding being used by the fprintf function. For example, if the text file is in UTF-8 encoding, the fprintf function can be called with the '-encoding' option set to 'UTF-8'

PRIYANSHU SHUKLA <priyanshu>
Wed 15 Mar 2023 06:55:00 PM UTC, comment #1: 

I cannot reproduce this (or I misunderstood the bug report) on
Win11 octave gui codepage 437


GNU Octave Version: 8.1.0 (hg id: 75b1efe727d4)
GNU Octave License: GNU General Public License
Operating System: MINGW32_NT-6.2 Windows 6.2  x86_64



$ diff -b 01_D.txt 02_D.txt
4c4
< A           no 1 : aaaaa
---
> A           no 1 : aaaaab


Files are attached.

Dmitri.
--


(file #54491, file #54492)

Dmitri A. Sergatskov <dasergatskov>
Wed 15 Mar 2023 05:09:22 PM UTC, original submission:  

fprintf_bug1.m illustrates a strange behavior of the fprintf function which appeared with the new version of Octave (8.1.0).

The script writes 2 text files with the same content, except on line 5, but the difference is small.
One of the two files is written correctly, while the other one contains an error on another line (29) containing a special character.

The comparison between the 2 text file is plotted in the appended file File_comparison_1.png.

The character which is printed where a ? in the comparison is 0x3F instead of 0xFB.

Any small change in the m-file can modify the result. Therefore additional test files with the corresponding comparison.

I tried this on 2 different PCs with different codepages:
PS C:\> chcp
Active code page: 850

PS C:\> chcp
Active code page: 437

Charles Praplan <charprap>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54508:  fprintf_bug.tgz added by charprap (147KiB - application/x-compressed)
file #54499:  01_D.txt added by charprap (907B - text/plain)
file #54500:  02_D.txt added by charprap (908B - text/plain)
file #54495:  fprintf_bug1c.m added by dasergatskov (2KiB - text/x-objcsrc)
file #54496:  fprintf_bug2c.m added by dasergatskov (2KiB - text/x-objcsrc)
file #54491:  01_D.txt.gz added by dasergatskov (177B - application/x-gzip)
file #54492:  02_D.txt.gz added by dasergatskov (178B - application/x-gzip)
file #54487:  fprintf_bug1.m added by charprap (2KiB - text/plain)
file #54488:  fprintf_bug2.m added by charprap (2KiB - text/plain)
file #54489:  File_comparison_1.png added by charprap (27KiB - image/png)
file #54490:  File_comparison_2.png added by charprap (26KiB - image/png)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by arungiridhar (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by priyanshu (Posted a comment)
  • -email is unavailable- added by dasergatskov (Updated the item)
  • -email is unavailable- added by charprap (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.

     

     

     

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-03-18 charprap Attached File- Added fprintf_bug.tgz, #54508
    2023-03-16 charprap Attached File- Added 01_D.txt, #54499
        Attached File- Added 02_D.txt, #54500
    2023-03-16 mmuetzel StatusNone Need Info
    2023-03-15 dasergatskov Attached File- Added fprintf_bug1c.m, #54495
        Attached File- Added fprintf_bug2c.m, #54496
    2023-03-15 dasergatskov Attached File- Added 01_D.txt.gz, #54491
        Attached File- Added 02_D.txt.gz, #54492
    2023-03-15 charprap Attached File- Added fprintf_bug1.m, #54487
        Attached File- Added fprintf_bug2.m, #54488
        Attached File- Added File_comparison_1.png, #54489
        Attached File- Added File_comparison_2.png, #54490

    Back to the top

    Powered by Savane 3.11