ML Benchmarks

Benchmarks

They are ran with Protractor, a tool to perform e2e tests. They measure performances of ng2 code vs the equivalent in plain Javascript.
Each benchmark is in fact a simple HTML page with a button for each scenario. Each button is clicked a given number of times. It is possible to force a GC between each run.
After each iteration (i.e. a button click), the data presented in the report are computed based on the performance logs retrieved from the browser through the webdrivers.
These data are:

• script: total duration of the JavaScript execution
• render: total duration of he rendering preocesses (RecalculateStyles, Layout, UpdateLayerTree, Paint, Rasterize, CompositeLayers)
• gcAmount: heap amount recovered in GC
• gcAmountInScript: heap amount recovered in GC from JavaScript execution
• gcTime: total time spent in GC

The benchmarks code is located im modules/benchmarks (ng2) and modules/benchmarks_external (ng1).
Benchpress is the tool which runs and measures a single benchmark. It is available in tools/benchpress.

Change detection

• baseline.changeDetection: creates a chain of 50k objects. Then it iterates over them with a while loop.
• ng2.changeDetection: creates a ChangeDetector from a RecordRange with 50k child RecordRange. Then it triggers a change detection on it.

Compiler

Bindings: templates for the benchmarks which are concatanated 150 times in the test setup.

• ng2.compile.withBindings: compiles the template with binding
• ng2.compile.noBindings: compiles the template without bindings
• ng1.compile.withBindings: same in Angular 1.3
• ng1.compile.noBindings: same in Angular 1.3

Selector

• ng2.selector.parse: parses 10000 random selectors (CssSelector.parse)
• ng2.selector.match: performs 10000 selector matches(SelectorMatcher.match)
• ng2.selector.addSelectable: adds 10000 selectables to a matcher (SelectorMatcher.add)

DI

An Injector is created with some children. Then the objects are retrieved 20000 times in various way.

• ng2.di.getByToken: injector.get(D)
• ng2.di.getByKey: injector.get(D_KEY)
• ng2.di.getChild: childInjector.get(D)
• ng2.di.instantiate: instantiates a child injector and get an object child.get(E)

Element Injector

• ng2.elementInjector.instantiate: instantiates an Element Injector, and then instantiates the directives 20000 times
• ng2.elementInjector.instantiateDirectives: same, but clear directives between each instantiation

Tree

The tests creates a tree structure in data model, then it generates and destroy a DOM tree in each iteration.

• baseline.tree: DOM created/destroyed with vanilla JavaScript.
• ng2.tree: DOM created/destroyed with directives and change detection.
• ng1.tree: DOM created/destroyed with directives and $apply.

Large table

Angular 1.3 only for now.

Questions

• [benchmarks.es6] Is there documentation about the record object retrieved from webdriver performance logs (type, data.usedHeapSizeDelta) ?

Results

Coming from a run a a local machine, format: script | render | gcAmount | gcAmountInScript

• ng2.changeDetection: 47.51±26% | 1.25±20% | 257359.60±436% | 0.00±NaN%
• baseline.changeDetection: 6.48±33% | 1.31±28% | 163408.60±436% | 0.00±NaN%
• ng2.compile.withBindings: 134.24±10% | 5.60±131% | 13072560.60±21% | 10968023.40±34%
• ng2.compile.noBindings: 43.64±23% | 1.58±22% | 2459267.40±139% | 1648670.0±175%
• ng1.compile.withBindings: 128.21±13% | 2.01±51% | 10944117.00±35% | 7191367.80±29%
• ng1.compile.noBindings: 77.15±15% | 1.63±31% | 7950137.80±17% | 6923360.40±35%
• ng2.di.getByToken: 30.26±24% | 1.97±23% | 1655556.40±202% | 1543716.00±200%
• ng2.di.getByKey: 4.81±36% | 1.58±30% | 1737852.60±179% | 1548087.20±200%
• ng2.di.getChild: 80.79±9% | 1.61±22% | 8826529.00±26% | 8052127.40±0%
• ng2.di.instantiate: 86.25±10% | 1.62±21% | 12194497.40±34% | 11703840.00±34%
• ng2.elementInjector.instantiate: 23.09±34% | 1.54±29% | 2921303.60±139% | 2747992.80±136%
• ng2.elementInjector.instantiateDirectives: 18.94±32% | 1.73±29% | 908441.40±250% | 367938.80±436%
• baseline.tree: 49.35±14% | 22.64±14% | 397173.20±301% | 0.00±NaN%
• ng2.tree: 118.20±14% | 33.18±25% | 4068837.40±77% | 2170002.20±128%
• ng1.tree: 183.83±5% | 46.51±51% | 9135352.00±54% | 1900083.40±150%
• ng2.selector.parse: 45.48±28% | 1.42±28% | 11726877.60±108% | 7593364.40±37%
• ng2.selector.match: 100.60±11% | 1.22±21% | 541742.00±310% | 0.00±NaN%
• ng2.selector.addSelectable: N/A