bugLiberty Eiffel - Bugs: bug #41339, XML_TREE is slow

 
 

bug #41339: XML_TREE is slow

Submitter:  Paolo Redaelli <Tybor>
Submitted:  Fri 24 Jan 2014 07:03:06 PM UTC
   
 
Category:  Kernel libraries Severity:  3 - Normal
Priority:  * 3 - Low Status:  Need Info
Assigned to:  None Open/Closed:  Closed
Release:  None Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 24 Jan 2016 08:36:23 PM UTC, comment #7: 

Closing, due to lack of measurability. - Of course having a big XML file as DOM completly in RAM is not the fastest thing ever. so I think we can close this until it turns out to be a real problem in some application.

Feel free to reopen in case you see any benefit from keeping it open...

Raphael Mack <ramack>
Group administrator
Tue 28 Jan 2014 02:35:12 PM UTC, comment #6: 

You can use bdw in ace files, just use

collect("bdw")

:-)

Cyril Adrian <cadrian>
Group administrator
Tue 28 Jan 2014 10:42:23 AM UTC, comment #5: 

Cyril you are definitively right, that's a GC-related issue.
The wrapper generator (naively?) uses DOM, since the XML nodes often refeers to elements later in the file.

I recompiled wrappers-generator without GC in boost mode and run:

   wrappers-generator --local gobject.gcc-xml /usr/include/glib-2.0/gobject/*

in less than a second. Then

   wrappers-generator --local qt4.gccxml /usr/include/qt4/QtGui/*

crashed after 5 seconds but it had already loaded the whole XML file and were outputting the wrappers.
Then I recompiled wrappers-generator with require assertions, but it doesn't tell me anything interesting.
So it's definitively a GC related issue.
I'm now running the same examples with all assertions. When interrupted after a minute or two they just crash and that's pretty puzzling.
Using -bdw_gc runs fine... is BWD-GC usable in ACE files?
Btw, it's not listed in the command line help. I'll solve it

Paolo Redaelli <Tybor>
Group administrator
Tue 28 Jan 2014 07:08:59 AM UTC, comment #4: 

tutorial/xml/sax/example.e takes two seconds to parse qt4 on my machine (provided I send the output to /dev/null) :-)

tutorial/xml/dom/example1.e takes a bit less than 2 minutes to do the same (with GC), and two SECONDS with -no_gc...

Does wrappers-generator use SAX or DOM? The latter is bound to take more time since it has to build the whole (huge) tree.

Anyway we are far from the two hours. Maybe some problem in the WG data structures? Could you try deactivating the GC?

Cyril Adrian <cadrian>
Group administrator
Mon 27 Jan 2014 11:40:47 AM UTC, comment #3: 

I answer myself. No it's not worth the effort, at least right now.
It seems that the parser is not linear in size, since it parses the attacked file in less than a second. It's the gccxml output of Gobject library.

(file #30384)

Anonymous
Mon 27 Jan 2014 11:16:29 AM UTC, comment #2: 

Here's the beast.
Do you think is it worth the time to think about wrapping libxml?

(file #30382)

Anonymous
Mon 27 Jan 2014 08:29:34 AM UTC, comment #1: 

I don't think it is the entities, but just the mere size of the beast :-)

Could you attach the (gzipped) xml file for me to look at?

Thanks

Cyril Adrian <cadrian>
Group administrator
Fri 24 Jan 2014 07:03:06 PM UTC, original submission:  

I've started the work needed to add C++ support to wrappers-generator, discovering that XML_TREE is very slow when parsing the output of gccxml when it actually contains the description of C++ source code.

I tried to parse Qt4 sources creating a gccxml file with

gccxml -I /usr/include/qt4/ /usr/include/qt4/QtGui/QtGui  -o qt4.gccxml

I do acknoledge that it's not the easiest C++ library to start with, expecially when recounting that the XML description of QtGui is 12 Mb...
Parsing the 1,3Mb description of Gobject is almost istantaneuous on a "recent" machine, while XMLTREE is parsing that qt4.gccxml after two full hours of runtime.

You may see it in action in tag

https://github.com/tybor/Liberty/tree/xml_tree_is_slow

I suspect this low performance is caused by XML entities ("&lt;"), since C++ templates like QList<QAbstractButton> is translated into XML like this:

<Class id="_1048" name="Qlist&lt;QAbstractButton&gt;" ...

In fact there are more than 9 thousands "&lt;" in that qt4.gccxml while only one in gobject.gccxml (as produced by the makefile in wrappers).
I'm sorry I haven't been able to trace it, I'll do ASAP.

Paolo Redaelli <Tybor>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #30384:  gobject.gcc-xml.bz2 added by None (112KiB - application/x-bzip)
file #30382:  qt4.gccxml.bz2 added by None (928KiB - application/x-bzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ramack (Updated the item)
  • -email is unavailable- added by Tybor (Submitted the item)
  • -email is unavailable- added by Tybor
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-01-24 ramack Open/ClosedOpen Closed
    2015-12-26 ramack Priority5 - Normal 3 - Low
    2014-01-27 None Attached File- Added gobject.gcc-xml.bz2, #30384
    2014-01-27 None Attached File- Added qt4.gccxml.bz2, #30382
    2014-01-24 Tybor Carbon-Copy- Added cadrian

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code