bugPSPP - Bugs: bug #33260, terrible performance when data...

 
 

bug #33260: terrible performance when data paged to disk

Submitted by:  Ben Pfaff <blp>
Submitted on:  Sat 07 May 2011 06:10:19 PM UTC  
 
Category: Graphical User InterfaceSeverity: 5 - Average
Status: FixedAssigned to: None
Open/Closed: ClosedRelease: Before 0.8.0
Effort: 0.00

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 25 Jun 2013 03:46:11 PM UTC, comment #7:

It certainly is MUCH faster now. Thanks.

Closing this item.

John Darrington <jmd>
Project Administrator
Tue 25 Jun 2013 06:08:25 AM UTC, comment #6:

I pushed a commit that made your test case much faster. I believe that it reduces the time to read a row from O(N**2) in the number of columns to O(N), in the common case.

John, can you try it out too?

Ben Pfaff <blp>
Project Administrator
Tue 28 May 2013 03:57:37 PM UTC, comment #5:

> and produced the following syntax:


Thanks, that helps.

Ben Pfaff <blp>
Project Administrator
Tue 28 May 2013 12:50:42 PM UTC, comment #4:

Your test is not representative of typical GUI usage.

For the following experiment I used the dimensions reported by this user:
http://lists.gnu.org/archive/html/pspp-users/2013-02/msg00005.html

and produced the following syntax:

input program.
vector vec(1174).
loop #c = 1 to 27500.
loop #v = 1 to 1174.
compute vec(#v) = #v.
end loop.
end case.
end loop.
end file.
end input program.

frequencies /variables vec1.

The results:

1. Running the entire syntax from the command line:

real 0m21.848s
user 0m18.893s
sys 0m2.000s

Let us say 20 seconds. Acceptable.

2. Load the syntax into the gui, thus:
psppire file.sps
Then click Run|All :
I timed that at 31 seconds. I think this is also acceptable.

3. a) Load the syntax into the gui, thus:
psppire file.sps
b) Then select lines 1 to 10. (from INPUT PROGRAM thru END INPUT PROGRAM).
c ) Click Run|Selection - this loads the data into the sheet.
This also takes 31 seconds. Acceptable.
d) Now select the line "frequencies /variables vec1." and
Click Run|Selection
I waited more than 7 minutes for this to complete (then I got fed up with waiting). This is NOT acceptable.

These results would agree with what was reported in the bug report above.

So it would seem that there is a very poor performance reading data from the gui's datasheet.

John Darrington <jmd>
Project Administrator
Tue 28 May 2013 05:03:30 AM UTC, comment #3:

I'm not sure that this is really a problem anymore. My test case below run in .283 seconds in the text UI. It looks like it takes about the same amount of time in the GUI.

John, any opinion?

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 08:11:35 PM UTC, comment #2:

I pushed some commits that reduce runtime from 2.5 seconds to 1.5 seconds in the terminal UI for this FREQUENCIES test case (with the default workspace size).

The GUI performance is still worse than the terminal UI performance. I'll look into it.

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 07:08:38 PM UTC, comment #1:

The problem seems to be that performance for data paged to disk is terrible. If I run "set workspace=100000000." before loading data, then I get much better performance, only about a second or so from either UI for FREQUENCIES runs. It's still perceptibly faster from the terminal UI, but not bad from the GUI.

I'm updating the summary and category accordingly.

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 06:10:19 PM UTC, original submission:

Running FREQUENCIES on a data set with about 300,000 cases from the text UI, it takes a few seconds. This is slow (under a second would be what I'd consider "normal"). But running the same FREQUENCIES from the GUI takes over a minute! This needs to get tracked down--it shouldn't be any slower than the terminal UI.

Ben Pfaff <blp>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jmd (Original reporter)
  • -unavailable- added by jmd (Posted a comment)
  • -unavailable- added by blp (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 25 Jun 2013 03:46:11 PM UTCjmdStatusReady for Test/Review=>Fixed
      Open/ClosedOpen=>Closed
      Carbon-Copy-=>Added -unavailable-
    Tue 25 Jun 2013 06:08:25 AM UTCblpStatusNone=>Ready for Test/Review
    Tue 28 May 2013 12:50:42 PM UTCjmdCategoryOther=>Graphical User Interface
    Sat 14 May 2011 04:40:00 AM UTCblpReleaseNone=>Before 0.8.0
    Sat 07 May 2011 07:08:38 PM UTCblpCategoryGraphical User Interface=>Other
      Summaryterrible performance in GUI for large data sets=>terrible performance when data paged to disk

    Back to the top


    Powered by Savane 3.1-cleanup