bugGNU Octave - Bugs: bug #52905, Can't manage paths that contain...

 
 

bug #52905: Can't manage paths that contain extended unicode characters

Submitter:  None
Submitted:  Tue 16 Jan 2018 12:34:48 AM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Duplicate Assigned to:  None
Originator Name:  Hale Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * dev
Operating System:  * Microsoft Windows Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 17 Jan 2018 10:18:26 AM UTC, comment #4: 

At the moment Octave is transitioning from Qt4 to Qt5 for its GUI. Both handle non-ASCII character just fine "behind the scenes".

Like you've already written, what the interpreter does is a separate thing.
I wouldn't say that Octave "misuses WinAPI". I've nowhere seen that Windows' ANSI character API (the one that uses "char" and has "A" at the end of most API function calls) is obsolete. In fact it would probably break a lot of programs that are out there if it would be deprecated. And I believe that this won't happen in quite some time from now.

The patch in bug #49118 changes the file IO of Octave's core to use Windows' wide character API (the one that uses "wchar_t" and has "W" at the end of most API function calls) in many places. If you have suggestions on how this should be done, you are welcome to comment at that bug report.

Markus Mützel <mmuetzel>
Group administrator
Wed 17 Jan 2018 05:19:37 AM UTC, comment #3: 

Well, it does not look like an API misuse in the GUI binary.
Windows GUI shell opens "unicoded" filenames without questions.

It seems to me, the problem is in interpreter. It looks like standard path/name handling in such functions as cd(), or readcvs/dlm do not  handle, or do not allow handling strings (probably corrupting them) as Unicode when passing to Windows system.

You see, "TESTテスト①.csv" looks like exact 8-bit conversion of "TESTテスト①.csv", so there is no data corruption, only data-type is undetermined in Ocave, since there is no UTF-char type.
However, (AFAIK) WinAPI functions expect the data in specific format for further handling. And I foresee three possibilities:
1)Octave cripples the string with precaution procedures when passing it to WinAPI. I.e. like truncating or extending the string basing on difference in byte/character count. Or replacing "unknown" bytes with "?"...
2)Octave misuses WinAPI, calling obsolete functions(like winapi OpenFile, fopen instead of _wfopen)

Anonymous
Tue 16 Jan 2018 08:59:40 PM UTC, comment #2: 

This will be fixed once the patch from bug #49118 (or a similar one) should be applied.

Closing as duplicate.

Markus Mützel <mmuetzel>
Group administrator
Tue 16 Jan 2018 05:51:13 PM UTC, comment #1: 

This might be a duplicate of bug #49118.
I haven't tried yet whether the patch there fixes all your failing cases. As soon as I have tested, I will report back.

Markus Mützel <mmuetzel>
Group administrator
Tue 16 Jan 2018 12:34:48 AM UTC, original submission:  

When getting the file name with
[F_name, F_path, fltidx] = uigetfile(...)

neither F_name nor F_path strings can be used further for traversing folders and accessing the file data.

Example:
Try creating the file like "TESTテスト①.csv"

execute:
[Fin_name, Fin_path, fltidx] = uigetfile  ({'*.csv', 'CSV data (*.csv)';},  'Pick files', "MultiSelect", "on");

Check variables:

>> Fin_name

Fin_name = TESTテスト①.csv

>> Fin_path

Fin_path = C:\Users\user\Documents\Octave\Test\TEST玉②\

Therefore cd(Fin_path) fails with
cd(Fin_path)
error: C:\Users\user\Documents\Octave\Test\TEST玉②\: No such file or directory

The same is for CSVREAD
csvread([Fin_path,Fin_name])
error: dlmread: unable to open file 'C:\Users\user\Documents\Octave\Test\TEST玉②\TESTテスト①.csv'
error: called from
    csvread at line 39 column 5

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-01-16 mmuetzel StatusConfirmed Duplicate
        Open/ClosedOpen Closed
        Release4.2.1 dev
    2018-01-16 mmuetzel StatusNone Confirmed
        Dependencies- Depends on bugs #49118

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code