bugPSPP - Bugs: bug #35688, encoding-guesser could detect bad...


bug #35688: encoding-guesser could detect bad fallback encoding more generally

Submitted by:  Ben Pfaff <blp>
Submitted on:  Fri 02 Mar 2012 05:28:58 AM UTC  
Category:  Internationalization/Localization Severity:  5 - Average
Status:  None Assigned to:  None
Open/Closed:  Open Release:  None
Effort:  0.00

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


Fri 02 Mar 2012 05:28:58 AM UTC, original submission:

See http://lists.gnu.org/archive/html/pspp-dev/2012-03/msg00008.html for full context.

John Darrington <john@darrington.wattle.id.au> writes:

> Would it be possible to generalize it? That is to say, could we make
> sure, that when the fallback encoding is X, where X is multi-byte encoding,
> but we know that the input is not X, that it also falls back to windows-1252?
> On Wed, Feb 29, 2012 at 10:44:30PM -0800, Ben Pfaff wrote:
> Until now the encoding-guesser code has used UTF-8 as a fallback in
> situations where we can tell that the file is not valid UTF-8. In
> this kind of situation having a single-byte character set as a
> fallback makes more sense. This commit hard-codes windows-1252 as
> that fallback, since it is a widely encountered encoding (and
> compatible with ISO-8859-1 as well).

Ben Pfaff <blp>
Project Administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by blp (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    No changes have been made to this item

    Back to the top

    Powered by Savane 3.3