bugGNU Octave - Bugs: bug #49842, Missing functions native2unicode...

 
 

bug #49842: Missing functions native2unicode and unicode2native

Submitted by:  Markus Mützel <mmuetzel>
Submitted on:  Wed 14 Dec 2016 05:47:16 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Matlab Compatibility
Status: FixedAssigned to: None
Originator Name: Open/Closed: Closed
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 19 Jun 2017 02:13:06 PM UTC, comment #10:

Closing as fixed as of comment #5 in bug #50774.

Markus Mützel <mmuetzel>
Project Member
Thu 08 Jun 2017 07:04:01 PM UTC, comment #9:

I went ahead and pushed the patch. Marking as ready for test.

Markus Mützel <mmuetzel>
Project Member
Thu 23 Mar 2017 05:32:59 PM UTC, comment #8:

I re-worked the patch and included a link to libiconv (which was always part of MXE Octave) instead of to the Microsoft page.

The tests for these two files are still failing for me on Windows. Could you please consider reviewing the attached patch?

(file #40095)

Markus Mützel <mmuetzel>
Project Member
Sun 05 Feb 2017 03:08:51 PM UTC, comment #7:

Thank you, for pushing and reworking the patch.

I tried on Windows and it looks like it is stricter when it comes to names of the code pages. I thought I tried on Windows before and the tests passed. Maybe that was because I had to include libunistring in MXE than and it is not needed any longer with your change.
The attached patch fixes the tests on Windows for me.

Should we additionally include libiconv to have a consistent set of available code pages on all platforms? [1]

[1]: https://www.gnu.org/software/libiconv/

(file #39672)

Markus Mützel <mmuetzel>
Project Member
Tue 31 Jan 2017 07:30:35 PM UTC, comment #6:

I pushed your changeset.

I also made some more changes. Now we just ask gnulib to provide the functions we use, use wrappers for them.

For future patches, please take a look at what I did.

Thanks.

John W. Eaton <jwe>
Project Administrator
Sun 22 Jan 2017 01:01:00 PM UTC, comment #5:

Forgot to free the memory after use. Attached is the updated patch.

(file #39528)

Markus Mützel <mmuetzel>
Project Member
Sat 21 Jan 2017 08:14:41 PM UTC, comment #4:

The attached patch uses libunistring from libgnu instead of ICU for the conversion. That makes it actually easier since we no longer need to convert to UTF-32 intermittently, and these functions natively work on strings containing 0.

To compile successfully with this patch, you need to run the full bootstrap-configure-make cycle.

With libunistring added, I am going to rework the patch in bug #49118, too. The conversion between UTF-8 <-> UTF-16 needed there can easily done with libunistring, too, without the need for a newer compiler.

(file #39525)

Markus Mützel <mmuetzel>
Project Member
Wed 18 Jan 2017 11:32:14 PM UTC, comment #3:

I was not aware of libunistring. Thank you for the hint.

I briefly browsed the docs at [1]. u8_conv_from_encoding and u8_conv_to_encoding should perfectly fit what is needed.
I will have a deeper look at it when I find the time.

[1] https://www.gnu.org/software/libunistring/manual/libunistring.html#uniconv_002eh

Markus Mützel <mmuetzel>
Project Member
Wed 18 Jan 2017 10:27:38 PM UTC, comment #2:

What about using the unicode string functions that are part of gnulib? I think they may also be installed on some systems as libunistring. If we used those functions instead, we wouldn't need to add a new dependency.

John W. Eaton <jwe>
Project Administrator
Wed 14 Dec 2016 05:54:29 PM UTC, comment #1:

Attached is the patch to add native2unicode and unicode2native.
The Wiki mentions ICU as a open-source library for the conversion of strings between different codepages. [1]
So I hope it is OK that I used that.

The necessary files were already installed in my Ubuntu (icu-devtools). It might be necessary to apt-get it manually on other systems.

[1] http://wiki.octave.org/International_Characters_Support#Development

(file #39224)

Markus Mützel <mmuetzel>
Project Member
Wed 14 Dec 2016 05:47:16 PM UTC, original submission:

The functions native2unicode and unicode2native are currently missing in Octave.
I will submit a patch that adds both as soon as I have a bug number.

Markus Mützel <mmuetzel>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #39224:  bug49842_native2unicode_unicode2native.patch added by mmuetzel (13KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by mmuetzel (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 13 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 19 Jun 2017 02:13:06 PM UTCmmuetzelStatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
    Thu 08 Jun 2017 07:04:01 PM UTCmmuetzelStatusPatch Submitted=>Ready For Test
    Thu 23 Mar 2017 05:32:59 PM UTCmmuetzelAttached File-=>Added bug49842_native2unicode_unicode2native_tests_v2.patch, #40095
    Sun 05 Feb 2017 03:08:51 PM UTCmmuetzelAttached File-=>Added bug49842_native2unicode_unicode2native_tests.patch, #39672
      StatusFixed=>Patch Submitted
      Open/ClosedClosed=>Open
    Tue 31 Jan 2017 07:30:35 PM UTCjweStatusPatch Submitted=>Fixed
      Open/ClosedOpen=>Closed
    Sun 22 Jan 2017 01:01:00 PM UTCmmuetzelAttached File-=>Added bug49842_native2unicode_unicode2native_v3.patch, #39528
    Sat 21 Jan 2017 08:14:41 PM UTCmmuetzelAttached File-=>Added bug49842_native2unicode_unicode2native_v2.patch, #39525
    Wed 14 Dec 2016 05:54:29 PM UTCmmuetzelAttached File-=>Added bug49842_native2unicode_unicode2native.patch, #39224
      StatusNone=>Patch Submitted

    Back to the top


    Powered by Savane 3.1-cleanup1