bugGNU Octave - Bugs: bug #41461, cast from numeric to char...

 
 

bug #41461: cast from numeric to char inconsistent and incompatible

Submitter:  Mike Sander <msander>
Submitted:  Wed 05 Feb 2014 02:44:26 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Confirmed Assigned to:  None
Originator Name:  mike sander Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 11 Aug 2023 08:57:32 PM UTC, comment #10: 

comment #0 behavior is unchanged in Octave 9.0.0:

- char(x) results in an error for any int types x<0 and x>255
- if x is double or single, the above range returns char(0)
- for any number type, char(x) for x>127 and x<256 produce non-ascii characters of the correct value.


>> x = 100; z = char(x), single(z)
z = d
ans = 100

>> x = int8(100); z = char(x), single(z)
z = d
ans = 100

>> x = -128; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0

>> x = int8(-128); z = char(x), single(z)
warning: range error for conversation to character value
error: value on right hand side of assignment is undefined

>> x = 256; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0

>> x = int16(256); z = char(x), single(z)
warning: range error for conversation to character value
error: value on right hand side of assignment is undefined

>> x = 1000; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0



Matlab 2023 public facing help on char currently states:

Numeric array
char converts numbers into characters. Valid numeric values range from 0 to 65535 and correspond to Unicode® code units. Values from 0 to 127 also correspond to 7-bit ASCII characters. The char function:
1 Rounds nonintegers toward zero.
2 Treats values less than 0 as 0.
3 Treats values greater than 65535 as 65535.

https://www.mathworks.com/help/matlab/ref/char.html

Matlab 2023a output for commands above:


>> x = 100; z = char(x), single(z)
z =
    'd'
ans =
  single
   100
>> x = int8(100); z = char(x), single(z)
z =
    'd'
ans =
  single
   100
>> x = -128; z = char(x), single(z)
z =
    ' '
ans =
  single
     0
>> x = int8(-128); z = char(x), single(z)
z =
    ' '
ans =
  single
     0
>> x = 256; z = char(x), single(z)
z =
    'Ā'
ans =
  single
   256
>> x = int16(256); z = char(x), single(z)
z =
    'Ā'
ans =
  single
   256
>> x = 1000; z = char(x), single(z)
z =
    'Ϩ'
ans =
  single
        1000


regarding rounding, it seems that we also aren't compatible there:

Octave 9.0.0:

>> char(120)
ans = x
>> char(121)
ans = y
>> char(120.1)
ans = x
>> char(120.9)
ans = y
>> int8(120.9)
ans = 121


Matlab 2023a:

>> char(120)
ans =
    'x'
>> char(121)
ans =
    'y'
>> char(120.1)
ans =
    'x'
>> char(120.9)
ans =
    'x'
>> int8(120.9)
ans =
  int8
   121


I didn't find another bug report on the rounding but would think something like this had to have been reported before.

Michael's attached patch still applies cleanly to default and passes a make check. Here are the above commands after the patch:


>> x = 100; z = char(x), single(z)
z = d
ans = 100

>> x = int8(100); z = char(x), single(z)
z = d
ans = 100

>> x = -128; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0

>> x = int8(-128); z = char(x), single(z)
z =
ans = 0

>> x = 256; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0

>> x = int16(256); z = char(x), single(z)
z = ÿ
ans = 255

>> x = 1000; z = char(x), single(z)
warning: range error for conversation to character value
z =
ans = 0

>> x = int16(1000); z = char(x), single(z)
z = ÿ
ans = 255


So it fixes the int-specific error, and negative values map to 0.   it only changes the behavior of ints > 255 and maps them to 255. And it removes the range warning on ints that is still there for floats.  this leaves a significant difference in behavior between ints and floats, where the latter are still mapped to zero.

I agree with his initial approach that we should try to follow matlab's lead here. negatives of any type should map to 0. anything greater than maxval of the system's char type should map to the max value not 0 (which jwe says may be signed or unsigned int8 depending on system.  and if we ever handle 16bit chars the rules won't change). I think an out of range warning should still be issued when either case occurs. And ideally, we'd fix the rounding behavior to match matlab as well.

last: the warning message:
warning: range error for conversation to character value

seems ill-phrased. it's either a warning or an error. if it's just going to be a warning, the message shouldn't use the word error.

Nicholas Jankowski <nrjank>
Group Member
Fri 15 Jan 2016 11:51:17 PM UTC, comment #9: 

Mike has raised the valid points that  char(int8(-128))  is invalid, while  char(-128) == char(0).

I agree that giving an error for char(x) with x>255 is better than giving a value different from Matlab's, but Octave currently gives char(256) == char(0), which is no better than Mike's suggestion.

I propose the following behaviour:
1. char (x) = char(0)  for x<0 for all (non-complex) numeric types.  Based on Michael's experiment, I assume this is what Matlab does.
2. char (x) undefined for x > 255 for all numeric types.

Mike, would you be interested in modifying your patch to implement this, of others approve, or have you moved on?

Lachlan Andrew <lachlan>
Fri 07 Feb 2014 09:27:06 PM UTC, comment #8: 

Could someone from the octave community please clarify what are the requirement for cast of integer values to char?

my goal is to finish this patch, get it right and have it accepted/applied.   presently, i am unclear what the requirements are.


Mike Sander <msander>
Fri 07 Feb 2014 09:16:32 PM UTC, comment #7: 

response to comment #6.

i do follow your point "Before making this another permanent Matlab-incompatibility,...".    As far as i can determine octaves current treatment of casting integer type to char is NOT compatible with matlab.  I believe my patch improves [or at least my goal is to improve] compatability constrained by the fact that octave char is 8 bits [signedness system dependent], matlab appears to treat char as 16 bit [unsigned].

i assume changing octaves char type would be a major project.  I would not propose such a change without support of the wider octave community.

Mike Sander <msander>
Wed 05 Feb 2014 08:52:07 PM UTC, comment #6: 

Before making this another permanent Matlab-incompatibility, please consider, that that char is probably intended to stand for character, i.e. hold the numerical value for a character. In the long run, I would hope that Octave would move to Unicode encoding, and what value range would you give char then? I suggest using the same range as Matlab does.

Anonymous
Wed 05 Feb 2014 06:32:56 PM UTC, comment #5: 

comment 4:  good point regarding system dependence of char.  with regards to the patch, should the tests be written as
std::numeric_limits<char>::min ()  ?

this way limits fall into the ranges [-128,127] or [0,255] as determined by signedness of char on that system.   Is there a better approach?


In regards to matlab compatibility brought up in earlier comments, here is the matlab documentation for char():

Description

S = char(X) converts array X of nonnegative integer codes into a character array. Valid codes range from 0 to 65535, where codes 0 through 127 correspond to 7-bit ASCII characters. The characters that MATLAB® can process (other than 7-bit ASCII characters) depend upon your current locale setting. To convert characters into a numeric array, use the double function.


So, octave has a char type which is sytem dependent, and 8 bits.  matlab appears to treat char as signed & 16 bit.  Richard Crozier (in linked discussion) confirmed negative values truncate to 0. he tested +1000 & matlab returned 1000 which is consistent with documentation.  

In the current implementation of octave cast to char of negative is undefined.   Cast to char of value > 127 results in 0.  neither result is consistent with matlab.

Ultimately it comes down to requirements.  If the requirement is identical behavior as matlab, then this implies reworking the internal char representation of octave.  Seems unlikely to me.

If the requirement is to return a value limited to the range of char on the system in question, then I think the patch i provided is close to that.   If the latter is the requirement, could i get some concrete feedback regarding the patch so I can make it right?




Mike Sander <msander>
Wed 05 Feb 2014 04:10:54 PM UTC, comment #4: 

Internally, Octave uses the C type "char" to represent characters.  Whether this is signed or unsigned is system dependent.  On my system (GNU/Linux with GCC) it is signed.  That it may sometimes appear to properly handle character codes greater than 127 is purely by chance.

John W. Eaton <jwe>
Group administrator
Wed 05 Feb 2014 01:03:39 PM UTC, comment #3: 

Part of what you say seems correct, but what I meant to
say is that checking with what Matlab does is important.
That is why I tried your examples in Matlab.



Michael Godfrey <godfrey>
Group Member
Wed 05 Feb 2014 11:51:07 AM UTC, comment #2: 

Michael, I did check.  See http://octave.1599824.n4.nabble.com/cast-to-char-td4661457.html.

From matlab (online) documentation it treats char as unsigned 16 bit value.  Octave (from what i can determine) treats char as unsigned 8 bit.

this patch removes the undefined result for negative inputs.   It also returns 255 for values > 255, rather than 0.   I believe these changes improve compatability, within the octave 8 bit char type.

if my arguments above are valid, are there any technical issues with the checks i have implemented?

Mike Sander <msander>
Wed 05 Feb 2014 03:27:15 AM UTC, comment #1: 

In cases like this it is appropriate
to check for matlab behavior before making
proposals no matter how "reasonable" they may
seem.

Matlab (R2009B) says:

>> x=int8(-128)


x =

 -128

>> z=char(x)

Warning: Out of range or non-integer values truncated during conversion to character.

z =

 

>> x=256   


x =

   256

>> z=char(x)


z =



>> single(z)


ans =

   256

>>

====================
So, your suggestions will introduce further
Matlab incompatibility.  It is not clear why
Matlab seems to think that 256 is within char range.

Michael Godfrey <godfrey>
Group Member
Wed 05 Feb 2014 02:44:26 AM UTC, original submission:  


octave:1> x=int8(-128)
x = -128
octave:2> z=char(x)
warning: range error for conversion to character value
error: value on right hand side of assignment is undefined


I believe this should return 0.   This assumes char maps to unsigned 8 bit type.



octave:2> x=256
x =  256
octave:3> z=char(x)
warning: range error for conversion to character value
z =
octave:4> single(z)  # to show value
ans = 0


this should return 255.

The attached patch (cast_integer01.diff) has a proposed fix for integer types.  I am open to comments/suggestions.   Specifically is use of std::numeric_limits appropriate in this context?

NB. bug #38956 is related, but only considers +ve portion, not the undefined value.

Mike Sander <msander>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #30460:  cast_integer01.diff added by msander (1KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by lachlan (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by godfrey (Posted a comment)
  • -email is unavailable- added by msander (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-08-11 nrjank StatusNeed Info Confirmed
    2023-08-11 nrjank StatusNone Need Info
        Summarycast to integer type incorrect cast from numeric to char inconsistent and incompatible
    2014-02-05 msander Attached File- Added cast_integer01.diff, #30460

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code