bugLiberty Eiffel - Bugs: bug #41339, XML_TREE is slow

 
 

bug #41339: XML_TREE is slow

Submitted by:  Paolo Redaelli <Tybor>
Submitted on:  Fri 24 Jan 2014 07:03:06 PM UTC  
 
Category: Kernel librariesSeverity: 3 - Normal
Priority: 5 - NormalStatus: Need Info
Assigned to: NoneOpen/Closed: Open
Release: NoneFixed Release: None
Planned Release: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 28 Jan 2014 02:35:12 PM UTC, comment #6:

You can use bdw in ace files, just use

collect("bdw")

:-)

Cyril Adrian <cadrian>
Project Administrator
Tue 28 Jan 2014 10:42:23 AM UTC, comment #5:

Cyril you are definitively right, that's a GC-related issue.
The wrapper generator (naively?) uses DOM, since the XML nodes often refeers to elements later in the file.

I recompiled wrappers-generator without GC in boost mode and run:

wrappers-generator --local gobject.gcc-xml /usr/include/glib-2.0/gobject/*

in less than a second. Then

wrappers-generator --local qt4.gccxml /usr/include/qt4/QtGui/*

crashed after 5 seconds but it had already loaded the whole XML file and were outputting the wrappers.
Then I recompiled wrappers-generator with require assertions, but it doesn't tell me anything interesting.
So it's definitively a GC related issue.
I'm now running the same examples with all assertions. When interrupted after a minute or two they just crash and that's pretty puzzling.
Using -bdw_gc runs fine... is BWD-GC usable in ACE files?
Btw, it's not listed in the command line help. I'll solve it

Paolo Redaelli <Tybor>
Project Administrator
Tue 28 Jan 2014 07:08:59 AM UTC, comment #4:

tutorial/xml/sax/example.e takes two seconds to parse qt4 on my machine (provided I send the output to /dev/null) :-)

tutorial/xml/dom/example1.e takes a bit less than 2 minutes to do the same (with GC), and two SECONDS with -no_gc...

Does wrappers-generator use SAX or DOM? The latter is bound to take more time since it has to build the whole (huge) tree.

Anyway we are far from the two hours. Maybe some problem in the WG data structures? Could you try deactivating the GC?

Cyril Adrian <cadrian>
Project Administrator
Mon 27 Jan 2014 11:40:47 AM UTC, comment #3:

I answer myself. No it's not worth the effort, at least right now.
It seems that the parser is not linear in size, since it parses the attacked file in less than a second. It's the gccxml output of Gobject library.

(file #30384)

Anonymous
Mon 27 Jan 2014 11:16:29 AM UTC, comment #2:

Here's the beast.
Do you think is it worth the time to think about wrapping libxml?

(file #30382)

Anonymous
Mon 27 Jan 2014 08:29:34 AM UTC, comment #1:

I don't think it is the entities, but just the mere size of the beast :-)

Could you attach the (gzipped) xml file for me to look at?

Thanks

Cyril Adrian <cadrian>
Project Administrator
Fri 24 Jan 2014 07:03:06 PM UTC, original submission:

I've started the work needed to add C++ support to wrappers-generator, discovering that XML_TREE is very slow when parsing the output of gccxml when it actually contains the description of C++ source code.

I tried to parse Qt4 sources creating a gccxml file with

gccxml -I /usr/include/qt4/ /usr/include/qt4/QtGui/QtGui -o qt4.gccxml

I do acknoledge that it's not the easiest C++ library to start with, expecially when recounting that the XML description of QtGui is 12 Mb...
Parsing the 1,3Mb description of Gobject is almost istantaneuous on a "recent" machine, while XMLTREE is parsing that qt4.gccxml after two full hours of runtime.

You may see it in action in tag

https://github.com/tybor/Liberty/tree/xml_tree_is_slow

I suspect this low performance is caused by XML entities ("&lt;"), since C++ templates like QList<QAbstractButton> is translated into XML like this:

<Class id="_1048" name="Qlist&lt;QAbstractButton&gt;" ...

In fact there are more than 9 thousands "&lt;" in that qt4.gccxml while only one in gobject.gccxml (as produced by the makefile in wrappers).
I'm sorry I haven't been able to trace it, I'll do ASAP.

Paolo Redaelli <Tybor>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #30384:  gobject.gcc-xml.bz2 added by None (112kB - application/x-bzip)
file #30382:  qt4.gccxml.bz2 added by None (928kB - application/x-bzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by Tybor (Submitted the item)
  • -unavailable- added by Tybor
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 27 Jan 2014 11:40:47 AM UTCNoneAttached File-=>Added gobject.gcc-xml.bz2, #30384
    Mon 27 Jan 2014 11:16:29 AM UTCNoneAttached File-=>Added qt4.gccxml.bz2, #30382
    Fri 24 Jan 2014 07:03:06 PM UTCTyborCarbon-Copy-=>Added cadrian

    Back to the top


    Powered by Savane 3.1-cleanup