bugPSPP - Bugs: bug #33260, terrible performance when data...


bug #33260: terrible performance when data paged to disk

Submitted by:  Ben Pfaff <blp>
Submitted on:  Sat 07 May 2011 06:10:19 PM UTC  
Category:  Graphical User Interface Severity:  5 - Average
Status:  Fixed Assigned to:  None
Open/Closed:  Closed Release:  Before 0.8.0
Effort:  0.00

Add a New Comment(Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


( Jump to the original submission)

Tue 25 Jun 2013 03:46:11 PM UTC, comment #7:

It certainly is MUCH faster now. Thanks.

Closing this item.

John Darrington <jmd>
Project Administrator
Tue 25 Jun 2013 06:08:25 AM UTC, comment #6:

I pushed a commit that made your test case much faster. I believe that it reduces the time to read a row from O(N**2) in the number of columns to O(N), in the common case.

John, can you try it out too?

Ben Pfaff <blp>
Project Administrator
Tue 28 May 2013 03:57:37 PM UTC, comment #5:

> and produced the following syntax:

Thanks, that helps.

Ben Pfaff <blp>
Project Administrator
Tue 28 May 2013 12:50:42 PM UTC, comment #4:

Your test is not representative of typical GUI usage.

For the following experiment I used the dimensions reported by this user:

and produced the following syntax:

input program.
vector vec(1174).
loop #c = 1 to 27500.
loop #v = 1 to 1174.
compute vec(#v) = #v.
end loop.
end case.
end loop.
end file.
end input program.

frequencies /variables vec1.

The results:

1. Running the entire syntax from the command line:

real 0m21.848s
user 0m18.893s
sys 0m2.000s

Let us say 20 seconds. Acceptable.

2. Load the syntax into the gui, thus:
psppire file.sps
Then click Run|All :
I timed that at 31 seconds. I think this is also acceptable.

3. a) Load the syntax into the gui, thus:
psppire file.sps
b) Then select lines 1 to 10. (from INPUT PROGRAM thru END INPUT PROGRAM).
c ) Click Run|Selection - this loads the data into the sheet.
This also takes 31 seconds. Acceptable.
d) Now select the line "frequencies /variables vec1." and
Click Run|Selection
I waited more than 7 minutes for this to complete (then I got fed up with waiting). This is NOT acceptable.

These results would agree with what was reported in the bug report above.

So it would seem that there is a very poor performance reading data from the gui's datasheet.

John Darrington <jmd>
Project Administrator
Tue 28 May 2013 05:03:30 AM UTC, comment #3:

I'm not sure that this is really a problem anymore. My test case below run in .283 seconds in the text UI. It looks like it takes about the same amount of time in the GUI.

John, any opinion?

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 08:11:35 PM UTC, comment #2:

I pushed some commits that reduce runtime from 2.5 seconds to 1.5 seconds in the terminal UI for this FREQUENCIES test case (with the default workspace size).

The GUI performance is still worse than the terminal UI performance. I'll look into it.

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 07:08:38 PM UTC, comment #1:

The problem seems to be that performance for data paged to disk is terrible. If I run "set workspace=100000000." before loading data, then I get much better performance, only about a second or so from either UI for FREQUENCIES runs. It's still perceptibly faster from the terminal UI, but not bad from the GUI.

I'm updating the summary and category accordingly.

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 06:10:19 PM UTC, original submission:

Running FREQUENCIES on a data set with about 300,000 cases from the text UI, it takes a few seconds. This is slow (under a second would be what I'd consider "normal"). But running the same FREQUENCIES from the GUI takes over a minute! This needs to get tracked down--it shouldn't be any slower than the terminal UI.

Ben Pfaff <blp>
Project Administrator


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by jmd (Original reporter)
  • -email is unavailable- added by jmd (Posted a comment)
  • -email is unavailable- added by blp (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-06-25 jmd StatusReady for Test/Review => Fixed
        Open/ClosedOpen => Closed
        Carbon-Copy- => Added -email is unavailable-
    2013-06-25 blp StatusNone => Ready for Test/Review
    2013-05-28 jmd CategoryOther => Graphical User Interface
    2011-05-14 blp ReleaseNone => Before 0.8.0
    2011-05-07 blp CategoryGraphical User Interface => Other
        Summaryterrible performance in GUI for large data sets => terrible performance when data paged to disk

    Back to the top

    Powered by Savane 3.3