bugGNU Octave - Bugs: bug #46855, [OF] io package OCT interface:...

 
 

bug #46855: [OF] io package OCT interface: XML/Unicode issues

Submitter:  Philip Nienhuis <philipnienhuis>
Submitted:  Fri 08 Jan 2016 10:42:48 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  philipnienhuis
Originator Name:  Philip Nienhuis Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 15 Jan 2016 09:50:12 PM UTC, comment #1: 

I've pushed the following cs:
http://hg.code.sf.net/p/octave/io/rev/6d8a647d1b8c

....merely stripping out all characters < 32 or > 126.

It is made so that only text strings from or to the Octave workspace are treated.  When writing to existing files, the lower level routines leave strings in worksheets (incl. any double-byte strings) intact as much as possible (i.e., when not overwritten by new data) and simply rewrite them back to file. Only if they are conveyed to the user (workspace) they are stripped from double-byte characters.

Closing as fixed.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 08 Jan 2016 10:42:48 PM UTC, original submission:  

Writing text strings containing non-alphanumeric characters to xlsx/ods files using OCT interface seems to go fine but Excel nor LibreOffice can read the produced files w/o errors.
LibreOffice flat-out refuses to read the file; Excel deletes the entire string container (xl/sharedStrings.xml) from the zipped xlsx file so no strings appear in the opened file.

I've just applied a first fix, pertaining to an (in hindsight obvious) oversight when developing the XML string I/O code:

> < & ' "

(which are illegal characters in XML) are now translated back and forth into XML escape sequences (fix to be in future io-2.4.1+).
But most other non-alphanumeric characters still give problems. When non-alphanumeric chars are input directly in Excel or LibreOffice themselves and saved to file from those programs, reading the files with the OCT interface returns double-byte sequences.
A workaround could be filtering out any non-alphanumeric characters before writing and after reading respectively, but that'll require costly regexprep calls :-(  and when writing it may screw up files produced by other programs.

Does Octave have any binary function, or is there an easy and fast procedure for filtering out alphanumeric characters?

Philip Nienhuis <philipnienhuis>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-01-15 philipnienhuis StatusNone Fixed
        Assigned toNone philipnienhuis
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code