buglibiconv - Bugs: bug #24216, UTF7 invariant violation

 
 

bug #24216: UTF7 invariant violation

Submitted by:  None
Submitted on:  Sat 06 Sep 2008 08:38:04 PM UTC  
 
Category: ConvertersSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Bruno Haible <haible>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 07 Sep 2008 11:31:43 PM UTC, comment #2:

I've improved the handling of shift sequence before invalid input
now, in the libiconv CVS. The change was major, a simple change
to utf7.h did not do it.

2008-09-07 Bruno Haible <bruno@clisp.org>

Make behaviour when encountering invalid input after a shift sequence
more consistent.

  • lib/converters.h (RET_SHIFT_ILSEQ): New macro.

(RET_ILSEQ): Define in terms of RET_SHIFT_ILSEQ.
(RET_TOOFEW): Change to avoid collisions with RET_SHIFT_ILSEQ.
(DECODE_SHIFT_ILSEQ, DECODE_TOOFEW): New macros.

  • lib/loop_unicode.h (unicode_loop_convert): Take into account the

shift count contained in the negative return values.

  • lib/hz.h (hz_mbtowc): When encountering invalid input, store the

modified state and return RET_SHIFT_ILSEQ.

  • lib/iso2022_cn.h (iso2022_cn_mbtowc): Likewise.
  • lib/iso2022_cnext.h (iso2022_cn_ext_mbtowc): Likewise.
  • lib/iso2022_jp.h (iso2022_jp_mbtowc): Likewise.
  • lib/iso2022_jp1.h (iso2022_jp1_mbtowc): Likewise.
  • lib/iso2022_jp2.h (iso2022_jp2_mbtowc): Likewise.
  • lib/iso2022_jp3.h (iso2022_jp3_mbtowc): Likewise.
  • lib/iso2022_kr.h (iso2022_kr_mbtowc): Likewise.
  • lib/ucs2.h (ucs2_mbtowc): Likewise.
  • lib/ucs4.h (ucs4_mbtowc): Likewise.
  • lib/utf16.h (utf16_mbtowc): Likewise.
  • lib/utf32.h (utf32_mbtowc): Likewise.
  • lib/utf7.h (utf7_mbtowc): Likewise.
  • lib/utf16be.h (utf16be_mbtowcutf16be_mbtowc): When encountering

invalid input, return RET_SHIFT_ILSEQ.

  • lib/utf16le.h (utf16le_mbtowc): Likewise.
  • tests/test-shiftseq.c: New file.
  • tests/Makefile.in (check): Run test-shiftseq.

(test-shiftseq, test-shiftseq.@OBJEXT@): New rules.
(clean): Remove test-shiftseq executable.
Reported by Roman Rybalko <roman_rybalko@users.sourceforge.net>
at <http://savannah.gnu.org/bugs/?24216>.

  • man/iconv.3: Clarify the processing of shift-sequences.

Thank you very much for the report. It was a dark corner of
the libiconv implementation.

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Sun 07 Sep 2008 10:00:45 PM UTC, comment #1:

I'm attaching a test case that is based on your input, involving
only the public API of iconv(). (The xxx_mbtowc functions are not
public API, it makes no sense to discuss their behaviour.)

What I see is that indeed the libiconv behaviour is "strange",
but I would not qualify it as a bug. Up to today, the iconv.3
manual page did not even mention how shift sequences are converted.

(file #16446)

Bruno Haible <haible>
Project AdministratorIn charge of this item.
Sat 06 Sep 2008 08:38:04 PM UTC, original submission:

1st case:
utf7_mbtowc(conv, &wc, "+2D/YQNhB2ELYQ9hE2EXYRthH2EjYSdhK2EvYTN", 20) = -1

2nd case:
utf7_mbtowc(conv, &wc, "+2D/YQNhB2ELYQ9hE2EXYRthH2EjYSdhK2EvYTN", 5) = -3,
utf7_mbtowc(conv, &wc, "2D/YQNhB2ELYQ9hE2EXYRthH2EjYSdhK2EvYTN", 10) = -1

Both cases should produce the same results. But in the second case we are on different position in being converted stream.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #16446:  foo.c added by haible (2KiB - text/x-c)
file #16438:  utf7-invariant-fix.patch added by None (611B - text/x-diff - Suggested fix)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by haible (Updated the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 7 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 24 Nov 2016 12:00:55 AM UTChaibleCategoryNone=>Converters
    Sun 07 Sep 2008 11:31:43 PM UTChaibleStatusIn Progress=>Fixed
      Open/ClosedOpen=>Closed
    Sun 07 Sep 2008 10:00:45 PM UTChaibleStatusNone=>In Progress
      Assigned toNone=>haible
    Sun 07 Sep 2008 10:00:44 PM UTChaibleAttached File-=>Added foo.c, #16446
    Sat 06 Sep 2008 08:38:04 PM UTCNoneAttached File-=>Added utf7-invariant-fix.patch, #16438

    Back to the top


    Powered by Savane 3.1-cleanup1