bugGNU Octave - Bugs: bug #55826, Use system encoding for file...

 
 

bug #55826: Use system encoding for file streams (fopen, ...)

Submitter:  Charles Praplan <charprap>
Submitted:  Mon 04 Mar 2019 05:39:42 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Other
Status:  Fixed Assigned to:  None
Originator Name:  Charles Praplan Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 09 May 2020 07:21:31 PM UTC, comment #6: 

Pushed three months ago and no negative reports yet. So I assume this is working as expected.

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Sun 09 Feb 2020 02:24:15 PM UTC, comment #5: 

I finally pushed a slightly modified version of the patch here (using the mfile encoding instead of the locale's encoding):
http://hg.savannah.gnu.org/hgweb/octave/rev/118606de9359

Marking as Ready For Test.

Markus Mützel <mmuetzel>
Group administrator
Fri 25 Oct 2019 05:15:51 PM UTC, comment #4: 

The attached patch changes fopen to use the locale's encoding by default.

On my Ubuntu test box, this is still UTF-8 like before. I haven't tried on Windows yet.

(file #47740)

Markus Mützel <mmuetzel>
Group administrator
Tue 05 Mar 2019 09:01:10 AM UTC, comment #3: 

Thank you for your help!

I could manage writing my output files correctly with latest Octave version.

Yes, the same behavior as Matlab, when not specifying an encoding, would be great.

Charles Praplan <charprap>
Tue 05 Mar 2019 08:02:29 AM UTC, comment #2: 

Matlab seems to write files with the system encoding when not specifying an encoding with fopen.

Re-titling accordingly and targeting dev.

This depends on bug #55452.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 Mar 2019 06:58:50 PM UTC, comment #1: 

Octave is in the process of better supporting international scripts in all parts of its code. As part of that process, the internal representation of char arrays has changed to be consistently in UTF-8.
So even if you are using .m files that are encoded in Windows-1252, the resulting strings are internally UTF-8.

If you want to write in a specific encoding, you would have to convert the strings to that encoding before writing to the file:

fid = fopen('test.txt', 'w');
fprintf(fid, "%s\n", unicode2native('Décompte', 'Windows-1252'));
f_error = fclose(fid);


In a future version of Octave, you'll be able to specify an encoding when opening a file:

fid = fopen('test.txt', 'w', 'n', 'Windows-1252');


That might also default to the system encoding on Windows (or to UTF-8).

Markus Mützel <mmuetzel>
Group administrator
Mon 04 Mar 2019 05:39:42 PM UTC, original submission:  

With Octave 5, the files generated by code (fopen fprints fclose) have their encoding which depends on the editor settings:
    Menu: Edit -> Preferences -> Editor -> File Handling -> Text encoding...

This is reasonable.
   
On my system, I have selected SYSTEM for this encoding.

My system settings are as follows
   (obtained with following powershell command: [System.Text.Encoding]::Default)

  IsSingleByte      : True
  BodyName          : iso-8859-1
  EncodingName      : Western European (Windows)
  HeaderName        : Windows-1252
  WebName           : Windows-1252
  WindowsCodePage   : 1252
  IsBrowserDisplay  : True
  IsBrowserSave     : True
  IsMailNewsDisplay : True
  IsMailNewsSave    : True
  EncoderFallback   : System.Text.InternalEncoderBestFitFallback
  DecoderFallback   : System.Text.InternalDecoderBestFitFallback
  IsReadOnly        : True
  CodePage          : 1252

However, when I execute the appended script, I get an UTF-8 file (10 bytes) instead of an Windows-1252 file (8 bytes).

The situation is the same if I choose Windows-1252 in the settings.

This was different in Octave 4.4.1.

Charles Praplan <charprap>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #47740:  bug55862_fopen_locale_encoding.patch added by mmuetzel (2KiB - application/octet-stream)
file #46413:  test.m added by charprap (83B - application/octet-stream)

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by charprap (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-05-09 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2020-02-09 mmuetzel StatusPatch Submitted Ready For Test
    2019-10-25 mmuetzel Attached File- Added bug55862_fopen_locale_encoding.patch, #47740
        StatusConfirmed Patch Submitted
    2019-10-23 mmuetzel Dependencies- bugs #57107 is dependent
    2019-03-05 mmuetzel Dependencies- Depends on bugs #55452
    2019-03-05 mmuetzel CategoryNone Interpreter
        StatusNone Confirmed
        Release5.1.0 dev
        Operating SystemMicrosoft Windows Any
        SummaryWrong encoding of files generated by code (fopen fprints fclose) Use system encoding for file streams (fopen, ...)
    2019-03-04 charprap Attached File- Added test.m, #46413

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code