bugGNU Octave - Bugs: bug #58564, classdef instance field access...

 
 

bug #58564: classdef instance field access painfully slow vs. struct field access

Submitted by:  Neil Konzen <waterdog1>
Submitted on:  Sun 14 Jun 2020 11:05:18 AM UTC  
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  Confirmed Assigned to:  None
Originator Name:  waterdog1 Open/Closed:  Open
Release:  dev Operating System:  Any

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Tue 16 Jun 2020 04:04:29 AM UTC, comment #7: 

> I saw your stack overflow Q&A: nice!


Thanks! Nice tool you got here yourself.

> That's a pretty old thread, it's probably worth the trouble to start a new question and answer for people who are searching for perf info as I was.


No way, man: I've got 223 upvotes there, and that question is the top Google result for "matlab oop slow". This is an institution of the internet now. :)

> The results you posted here are Octave results, right? It says Matlab in the benchmark output; that's worth fixing.


Oops! Yep, those are Octave results. Fixed. Thanks for catching that.

Andrew Janke <apjanke>
Tue 16 Jun 2020 03:00:50 AM UTC, comment #6: 

In Matlab, class .() access is quite a bit slower than struct .() access, but thanks to the JIT class direct field access is as fast as for structs.

For reference, here are the Matlab timings on the same machine as the Octave results I posted earlier.  The direct field access times are essentially the same, within the variance of the measurements.

.() access is

struct: obj.field access time = 0.025 usec
struct: obj.('field') access time = 0.643 usec
struct: subsref(obj,struct) access time = 5.836 usec

value_class: obj.field access time = 0.031 usec
value_class: obj.('field') access time = 1.972 usec
value_class: subsref(obj,struct) access time = 7.588 usec

handle_class: obj.field access time = 0.036 usec
handle_class: obj.('field') access time = 1.969 usec
handle_class: subsref(obj,struct) access time = 7.499 usec

You can see that Matlab's JIT is really pretty awesome.  Class .() is quite a bit slower than struct .(), roughly by the same factor of 3 that is seen in the Octave results.  I'll bet there are more dictionaries to search to handle subclassing.

This lousy performance for direct field access, plus the fact that we still can't set breakpoints in classdef code makes Octave a pretty impractical  platform for OO class development.

@apjanke I saw your stack overflow Q&A: nice!  That's a pretty old thread, it's probably worth the trouble to start a new question and answer for people who are searching for perf info as I was.  I almost overlooked your answer updates when I saw it the first time.

The results you posted here are Octave results, right? It says Matlab in the benchmark output; that's worth fixing.

Neil Konzen <waterdog1>
Tue 16 Jun 2020 01:50:02 AM UTC, comment #5: 

I added Octave compatibility to my matlab-bench suite: https://github.com/apjanke/matlab-bench

Here's what I get:

octave:4> bench_matlab_nops

Matlab R on x86_64-apple-darwin18.7.0
Matlab 5.2.0 / Java 13.0.2 on x86_64-apple-darwin18.7.0 Mac OS X 10.14.6 (angharad)
Machine: ???, ??? GB RAM
nIters = 100000

Java tests errored: [java] java.lang.IllegalAccessException: class org.octave.ClassHelper cannot access a member of class net.apjanke.matlab_bench.bench_nops.DummyJavaClass with modifiers "public". Skipping.
MEX tests errored: 'mexnop' undefined near line 367 column 9. Skipping.
Operation                        Time (�sec)
nop() function:                         4.86
nop() subfunction:                      3.70
@()[] anonymous function:               4.87
nop(obj) method:                        7.68
nop() private fcn on @class:            4.05
classdef nop(obj):                     11.25
classdef obj.nop():                    10.19
classdef pivate_nop(obj):              12.02
classdef class.static_nop():           10.11
classdef constant:                      6.83
classdef property:                      7.68
classdef property with getter:         22.44
+pkg.nop() function:                   11.21
+pkg.nop() from inside +pkg:           10.25
feval('nop'):                           8.83
feval(@nop):                            8.20
eval('nop'):                           16.42
Java tests errored: [java] java.lang.IllegalAccessException: class org.octave.ClassHelper cannot access a member of class net.apjanke.matlab_bench.bench_nops.DummyJavaClass with modifiers "public". Skipping.
MEX tests errored: 'mexnop' undefined near line 367 column 9. Skipping.
builtin j():                            3.92
struct s.foo field access:              2.79
isempty(persistent):                    4.88

Andrew Janke <apjanke>
Tue 16 Jun 2020 12:27:31 AM UTC, comment #4: 

I did a Whole Thing on this Matlab OOP performance stuff a while back. Check it out: https://stackoverflow.com/a/1745686/105904.

Short answer: In Matlab, classdef property access used to be way slower than struct field access, but in recent versions, it's just as fast (as long as you don't define a getter method).

Andrew Janke <apjanke>
Mon 15 Jun 2020 08:26:23 PM UTC, comment #3: 

Do you have access to Matlab?  Is there a comparable slowdown there?

Rik <rik5>
Project Administrator
Mon 15 Jun 2020 08:26:00 PM UTC, comment #2: 

I ran another configuration where I converted the class object to a struct first, and then use direct field access.

Code is

%% Convert to struct and then use direct . access
warning ("off", "Octave:classdef-to-struct");
sum = struct (obj).foo;
tic;
for i=1:iters
  sum = sum + struct (obj).foo;
end
dt = toc*1e6/iters - loop_overhead;
fprintf('%s: struct(obj).field access time = %.3f usec\n', class(obj), dt);

Results are

octave:7> field_access_timing
struct: obj.field access time = 1.183 usec
struct: struct(obj).field access time = 5.350 usec
struct: obj.('field') access time = 1.675 usec
struct: subsref(obj,struct) access time = 5.937 usec

value_class: obj.field access time = 5.471 usec
value_class: struct(obj).field access time = 14.032 usec
value_class: obj.('field') access time = 5.375 usec
value_class: subsref(obj,struct) access time = 10.259 usec

handle_class: obj.field access time = 6.119 usec
handle_class: struct(obj).field access time = 15.243 usec
handle_class: obj.('field') access time = 7.334 usec
handle_class: subsref(obj,struct) access time = 13.133 usec

Clearly it is no help, but it is suggestive that running

struct (value_class).field

has nearly the same run time as calling

struct (struct).field

It may be that internally the classdef object is being converted to a struct before accessing the field members.  That's just a wild guess, have to check the code in libinterp/octave-value.

Rik <rik5>
Project Administrator
Mon 15 Jun 2020 08:16:56 PM UTC, comment #1: 

Confirmed.  I change the Operating System to Any because I was able to reproduce this on Linux as well.

Here are the results from the stable branch that is about to become Octave version 6.1.

octave:1> field_access_timing
struct: obj.field access time = 1.181 usec
struct: obj.('field') access time = 1.869 usec
struct: subsref(obj,struct) access time = 6.431 usec

value_class: obj.field access time = 5.897 usec
value_class: obj.('field') access time = 5.543 usec
value_class: subsref(obj,struct) access time = 11.797 usec

handle_class: obj.field access time = 7.304 usec
handle_class: obj.('field') access time = 6.718 usec
handle_class: subsref(obj,struct) access time = 14.590 usec

And here are the results from the development branch that will eventually become Octave 7.1.

octave:1> field_access_timing
struct: obj.field access time = 1.238 usec
struct: obj.('field') access time = 1.669 usec
struct: subsref(obj,struct) access time = 5.740 usec

value_class: obj.field access time = 5.025 usec
value_class: obj.('field') access time = 5.353 usec
value_class: subsref(obj,struct) access time = 10.455 usec

handle_class: obj.field access time = 6.176 usec
handle_class: obj.('field') access time = 7.538 usec
handle_class: subsref(obj,struct) access time = 14.024 usec

Results are all generally reflective of the same behavior from Octave 5.2 which is that struct access is quick, while class access is slow.

Rik <rik5>
Project Administrator
Sun 14 Jun 2020 11:05:18 AM UTC, original submission:  

Direct access to a classdef instance field is 3x slower than accessing a struct field.  The attached 3 .m files demonstrate the timing; below are the timings I get with Octave 5.2 on my AMD 3900x machine with 32Gb memory.

struct: obj.field access time = 1.484 usec
struct: obj.('field') access time = 1.733 usec
struct: subsref(obj,struct) access time = 5.875 usec

value_class: obj.field access time = 4.483 usec
value_class: obj.('field') access time = 4.707 usec
value_class: subsref(obj,struct) access time = 10.059 usec

handle_class: obj.field access time = 4.620 usec
handle_class: obj.('field') access time = 4.797 usec
handle_class: subsref(obj,struct) access time = 9.911 usec

(I'm not concerned too much about subsref() timing; it's shown for reference only)

Neil Konzen <waterdog1>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49258:  field_access_timing.m added by waterdog1 (1KiB - text/plain - all three files in current directory, run field_access_timing.m)
file #49259:  value_class.m added by waterdog1 (74B - text/plain - all three files in current directory, run field_access_timing.m)
file #49260:  handle_class.m added by waterdog1 (85B - text/plain - all three files in current directory, run field_access_timing.m)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by waterdog1 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-06-15 rik5 StatusNone => Confirmed
        Release5.2.0 => dev
        Operating SystemMicrosoft Windows => Any
    2020-06-14 waterdog1 Attached File- => Added field_access_timing.m, #49258
        Attached File- => Added value_class.m, #49259
        Attached File- => Added handle_class.m, #49260

    Back to the top


    Powered by Savane 3.5