Sat 11 Aug 2012 10:33:50 AM UTC, comment #7:
After some toying around with xml_html_parser I found it too complicated:
- Where should utf8->cork conversion happen? Inside the xml_html_parser?
- When is strict xml parsing mode activated and should I use this to escape the entities?
- A few changes were needed to expand_entity() and co. whose side effects I'm not sure of. Regression tests should be made, etc.
So, I just modified a bit my previous patches:
- I renamed the functions with "cdata" in their name to "chardata".
- I added more functions for conversion to and from CDATA, now escaping ">" in these sections.
Your example still works and hopefully so should any containing the characters "]]>" inside any CDATA section.
(file #26348)
|
Sat 11 Aug 2012 08:52:59 AM UTC, comment #6:
A bit more on this "preferred way":
- I think the parsing of CDATA and CharData should happen inside xml_html_parser (in parsexml.cpp). Specifically:
- xml_html_parser::parse_until() should not always try to expand all entities, since in a CDATA section only ">" may be escaped.
- xml_cdata_to_tm() and friends from analyze.cpp should be part of the xml_html_parser if I understand its role correctly: no extra conversions on the data in the tree (as opposed to its structure, which needs further formatting via xmlin) should be necessary after a call to (parse-xml).
- It seems a bit asymmetrical that xml->tm is done using xml_html_parser but tm->xml is done in tmtmml.scm. Is this ok?
|
Sat 11 Aug 2012 08:29:07 AM UTC, comment #5:
Hi. Sorry for the incomplete patch. The new one I'm attaching has another function like the one proposed below, this one working and added to the glue, so importing back should work.
The problem is that although everything seems fine (except for the escaping of ">" in CDATA), there is probably some preferred way of dealing with these conversions that I'm not aware of (using all those "converter" dictionaries and classes for instance).
(file #26347)
|
Sat 11 Aug 2012 07:50:08 AM UTC, comment #4:
Hi Miguel,
Brief testing shows tmml export/import basically works with your patch (apart from re-import of htlm entities, of course). Good job Miguel!
If I want to test your proposed function where should I call it from?
As for adding to the glue, it's not imposible. I proposed a string conversion function recently and it went through.
|
Tue 31 Jul 2012 08:47:18 PM UTC, comment #3:
I meant we must escape any occurences of ">" inside CDATA, not &. It really sucks comments cannot be edited.
|
Tue 31 Jul 2012 08:45:01 PM UTC, comment #2:
Hi Philippe,
thanks for the, as usual, very detailed report. Thanks to it I've been to reproduce and locate the problems quite quickly. I have a "solution" for the unescaped <, >, &, so documents won't break so easily. The assumption was made in the code that utf8 is valid as CharData in XML, which as far as I understand is not (W3C).
I've also tried a bit with images, treating raw-data tags especially. Exporting and importing back work, but there are a couple of things missing:
- The CDATA section corresponding to an image's raw data must have any &'s escaped which occur in ]]>
- Conversion back from tmml of strings uses utf8->cork which obviously fails for &, < and > I'd propose a new function
which should go into the glue. But every time I did such thing it was reverted or ignored based on the arguments that there was some plan to do it better or that my idea was just bad. Because both are good arguments, I'd rather wait for some comments before I commit anything. In the meantime you can try the attached patch.
(file #26296)
|
Tue 22 May 2012 01:27:29 PM UTC, comment #1:
As a follow-up on the initial report I have found other cases when the exported tmml file is is not a valid xml structure.
See attached tm file and resulting tmml export.
As I say, the xml is invalid and most programs just refuse to process it. Even texmacs is not able to reload it (blank document).
I bumped into that bug while creating svg images of texmacs content ("copy-to-image" feature), and I was hoping to embed tmml into the svg to allow for re-edition of the initial content.
(file #25903, file #25904)
|
Mon 07 May 2012 08:54:13 AM UTC, original submission:
in xml the character < is strictly reserved for tags. However, in texmacs if a chunk of text contains < , it is passed untounched in the tmml export instead of being converted to <. This completely breaks xml for any use. Texmacs can't even re-open that file.
Note that part of the mechanism for converting < to < is already implemented since the character & is correctly converted to & when converting to tmml (but not correctly re-imported -- see bug #30801)
|