bugGNU Octave - Bugs: bug #49842, Missing functions native2unicode...

 
 

bug #49842: Missing functions native2unicode and unicode2native

Submitter:  Markus Mützel <mmuetzel>
Submitted:  Wed 14 Dec 2016 05:47:16 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 19 Jun 2017 02:13:06 PM UTC, comment #10: 

Closing as fixed as of comment #5 in bug #50774.

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Jun 2017 07:04:01 PM UTC, comment #9: 

I went ahead and pushed the patch. Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Thu 23 Mar 2017 05:32:59 PM UTC, comment #8: 

I re-worked the patch and included a link to libiconv (which was always part of MXE Octave) instead of to the Microsoft page.

The tests for these two files are still failing for me on Windows. Could you please consider reviewing the attached patch?


(file #40095)

Markus Mützel <mmuetzel>
Group administrator
Sun 05 Feb 2017 03:08:51 PM UTC, comment #7: 

Thank you, for pushing and reworking the patch.

I tried on Windows and it looks like it is stricter when it comes to names of the code pages. I thought I tried on Windows before and the tests passed. Maybe that was because I had to include libunistring in MXE than and it is not needed any longer with your change.
The attached patch fixes the tests on Windows for me.

Should we additionally include libiconv to have a consistent set of available code pages on all platforms? [1]

[1]: https://www.gnu.org/software/libiconv/

(file #39672)

Markus Mützel <mmuetzel>
Group administrator
Tue 31 Jan 2017 07:30:35 PM UTC, comment #6: 

I pushed your changeset.

I also made some more changes.  Now we just ask gnulib to provide the functions we use, use wrappers for them.

For future patches, please take a look at what I did.

Thanks.

John W. Eaton <jwe>
Group administrator
Sun 22 Jan 2017 01:01:00 PM UTC, comment #5: 

Forgot to free the memory after use. Attached is the updated patch.

(file #39528)

Markus Mützel <mmuetzel>
Group administrator
Sat 21 Jan 2017 08:14:41 PM UTC, comment #4: 

The attached patch uses libunistring from libgnu instead of ICU for the conversion. That makes it actually easier since we no longer need to convert to UTF-32 intermittently, and these functions natively work on strings containing 0.

To compile successfully with this patch, you need to run the full bootstrap-configure-make cycle.

With libunistring added, I am going to rework the patch in bug #49118, too. The conversion between UTF-8 <-> UTF-16 needed there can easily done with libunistring, too, without the need for a newer compiler.

(file #39525)

Markus Mützel <mmuetzel>
Group administrator
Wed 18 Jan 2017 11:32:14 PM UTC, comment #3: 

I was not aware of libunistring. Thank you for the hint.

I briefly browsed the docs at [1]. u8_conv_from_encoding and u8_conv_to_encoding should perfectly fit what is needed.
I will have a deeper look at it when I find the time.

[1] https://www.gnu.org/software/libunistring/manual/libunistring.html#uniconv_002eh

Markus Mützel <mmuetzel>
Group administrator
Wed 18 Jan 2017 10:27:38 PM UTC, comment #2: 

What about using the unicode string functions that are part of gnulib?  I think they may also be installed on some systems as libunistring.  If we used those functions instead, we wouldn't need to add a new dependency.

John W. Eaton <jwe>
Group administrator
Wed 14 Dec 2016 05:54:29 PM UTC, comment #1: 

Attached is the patch to add native2unicode and unicode2native.
The Wiki mentions ICU as a open-source library for the conversion of strings between different codepages. [1]
So I hope it is OK that I used that.

The necessary files were already installed in my Ubuntu (icu-devtools). It might be necessary to apt-get it manually on other systems.

[1] http://wiki.octave.org/International_Characters_Support#Development

(file #39224)

Markus Mützel <mmuetzel>
Group administrator
Wed 14 Dec 2016 05:47:16 PM UTC, original submission:  

The functions native2unicode and unicode2native are currently missing in Octave.
I will submit a patch that adds both as soon as I have a bug number.

Markus Mützel <mmuetzel>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #39224:  bug49842_native2unicode_unicode2native.patch added by mmuetzel (13KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mmuetzel (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-06-19 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2017-06-08 mmuetzel StatusPatch Submitted Ready For Test
    2017-03-23 mmuetzel Attached File- Added bug49842_native2unicode_unicode2native_tests_v2.patch, #40095
    2017-02-05 mmuetzel Attached File- Added bug49842_native2unicode_unicode2native_tests.patch, #39672
        StatusFixed Patch Submitted
        Open/ClosedClosed Open
    2017-01-31 jwe StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
    2017-01-22 mmuetzel Attached File- Added bug49842_native2unicode_unicode2native_v3.patch, #39528
    2017-01-21 mmuetzel Attached File- Added bug49842_native2unicode_unicode2native_v2.patch, #39525
    2016-12-14 mmuetzel Attached File- Added bug49842_native2unicode_unicode2native.patch, #39224
        StatusNone Patch Submitted

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code