criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.091594041034686e-5 8.092091053864229e-5 8.092706505040016e-5
Standard deviation 1.467933940674516e-8 1.8471114932548828e-8 2.455026343862431e-8

Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.882807141640935e-4 5.883726264555551e-4 5.885267249498629e-4
Standard deviation 2.453528204719336e-7 3.928724502321958e-7 6.153549337114866e-7

Outlying measurements have slight (1.2193263222069806e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.754451874247458e-6 5.7548241459919545e-6 5.756129979781408e-6
Standard deviation 5.381498517651273e-10 2.0995217550281856e-9 4.260172994766873e-9

Outlying measurements have no (5.681632653061087e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.7616626711131035e-6 5.761889990742393e-6 5.762145529329323e-6
Standard deviation 6.970184495831387e-10 8.253880035320092e-10 1.0106292367378116e-9

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.762589956659088e-6 5.76285575636608e-6 5.76319281443714e-6
Standard deviation 7.593783798715292e-10 9.75816667531193e-10 1.3335662403744115e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.620177199081375e-6 8.620472275756653e-6 8.620917254257966e-6
Standard deviation 8.906471854011784e-10 1.1589824354371925e-9 1.6934808173822908e-9

Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.021030264002661e-5 9.021490790034398e-5 9.022112509582912e-5
Standard deviation 1.4394629454281839e-8 1.770527916486982e-8 2.2388961084140825e-8

Outlying measurements have no (8.332744862650761e-3%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1476337083925503e-5 1.1476634584963182e-5 1.1477233094081153e-5
Standard deviation 9.956645044557004e-10 1.349320593873456e-9 2.1120185823227134e-9

Outlying measurements have no (6.172601365688052e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.460116122695241e-5 8.465304689114422e-5 8.474560968110133e-5
Standard deviation 1.6277341065556115e-7 2.2660871991410745e-7 3.43875603427452e-7

Outlying measurements have no (8.26388888888889e-3%) effect on estimated standard deviation.

monte carlo/conduit, stream fusion

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.253131673242632e-3 8.258963177408768e-3 8.271730043859865e-3
Standard deviation 1.5186713255778434e-5 2.4870772148564208e-5 4.353505877505575e-5

Outlying measurements have slight (2.9384756657483774e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.4326715574835697e-3 3.4360382644052613e-3 3.4401614024322812e-3
Standard deviation 9.770973151965788e-6 1.189664304664663e-5 1.4718211246790648e-5

Outlying measurements have slight (2.0823902218198277e-2%) effect on estimated standard deviation.

monte carlo/conduit, stream

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.773133562804749e-3 4.776566171183987e-3 4.781786256970655e-3
Standard deviation 9.659095669042616e-6 1.251577503451541e-5 1.6588050242117556e-5

Outlying measurements have slight (2.3242630385487528e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.271996282088649e-3 5.276192006542395e-3 5.280354508561703e-3
Standard deviation 1.0246037551717183e-5 1.2504696240291832e-5 1.5017443369794541e-5

Outlying measurements have slight (2.4375e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives, Codensity

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.514092514398126e-3 6.518285240206057e-3 6.523643717865826e-3
Standard deviation 1.092032674096062e-5 1.3770368097696992e-5 1.711226624009795e-5

Outlying measurements have slight (2.629656683710726e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives, explicit binding order

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.270688682633458e-3 5.275350945494108e-3 5.281676481234446e-3
Standard deviation 1.1905966164587227e-5 1.6275432998620384e-5 2.397554880146709e-5

Outlying measurements have slight (2.4375000000000004e-2%) effect on estimated standard deviation.

monte carlo/conduit, Pipe primitives

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6172263308478655e-2 1.618851681197337e-2 1.622009326794064e-2
Standard deviation 3.122623255850061e-5 5.4330843341624857e-5 1.0006949620985029e-4

Outlying measurements have slight (3.993055555555545e-2%) effect on estimated standard deviation.

monte carlo/conduit, Pipe constructos

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.876059306318267e-3 4.881512085857501e-3 4.888425190470118e-3
Standard deviation 1.5619571586911622e-5 1.9092012359922357e-5 2.43813547469023e-5

Outlying measurements have slight (2.3795359904818368e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6472466155062454e-3 1.6488329634360597e-3 1.651442325180042e-3
Standard deviation 4.1392184366538905e-6 6.925345097909881e-6 1.0854306634670237e-5

Outlying measurements have slight (1.6124697661918756e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7238692659812308e-3 1.72496672961846e-3 1.7268567392209769e-3
Standard deviation 3.598844949525555e-6 4.680746553573781e-6 6.5856427457470115e-6

Outlying measurements have slight (1.638888888888889e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.134348099438947e-3 2.1358658523623945e-3 2.138424418764505e-3
Standard deviation 4.008225873152173e-6 6.607722930046053e-6 1.0798875298825376e-5

Outlying measurements have slight (1.7538265306122448e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.013972934849674e-3 2.0158282355402894e-3 2.018834914203063e-3
Standard deviation 4.781289874382635e-6 7.9132819557204e-6 1.1960434120144488e-5

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.173020453811222e-3 4.1760583176076085e-3 4.180595428157465e-3
Standard deviation 8.587043523405868e-6 1.1130551302547828e-5 1.4588360661710835e-5

Outlying measurements have slight (2.2210743801652756e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.256983024257013e-3 4.2605806506288675e-3 4.266666916835026e-3
Standard deviation 9.46820068374062e-6 1.4357434797362394e-5 2.2855067640173208e-5

Outlying measurements have slight (2.221074380165263e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.5137412948345317e-3 3.5157234276002314e-3 3.519069714181917e-3
Standard deviation 5.567827254140382e-6 8.205647116724825e-6 1.1332444517297673e-5

Outlying measurements have slight (2.0823902218198277e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.022581938940886e-3 2.0242389679873845e-3 2.0264197189779993e-3
Standard deviation 4.751064059886647e-6 6.343857323377534e-6 9.303477207141359e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.0585655457193465e-3 2.0600730466896087e-3 2.0626747607942167e-3
Standard deviation 4.9353685118565595e-6 6.697327254039141e-6 9.765613152073928e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.038248649331312e-3 4.0408734303492575e-3 4.0444781158894385e-3
Standard deviation 6.8824313745636505e-6 9.702839416241223e-6 1.2312820932448278e-5

Outlying measurements have slight (2.1728395061728308e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.425089055953215e-3 4.428718966604835e-3 4.435648508397139e-3
Standard deviation 9.275900646519568e-6 1.575208119097481e-5 2.5799454591808568e-5

Outlying measurements have slight (2.271498107084876e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.185057921864447e-3 4.1875439782907056e-3 4.1920821005788965e-3
Standard deviation 6.7895805377417215e-6 1.01565203463177e-5 1.7134990362178224e-5

Outlying measurements have slight (2.2210743801652895e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.666569026488468e-3 9.67186678133863e-3 9.678945188845987e-3
Standard deviation 1.4179855573171124e-5 1.746119421534582e-5 2.195613840001095e-5

Outlying measurements have slight (3.1217481789801948e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1014579748012056e-3 3.1034435570137416e-3 3.107857267922585e-3
Standard deviation 4.804819242657887e-6 9.153556967839823e-6 1.6710713707851066e-5

Outlying measurements have slight (1.999167013744273e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.0304989432279553e-3 2.0322560420480327e-3 2.0345497323940856e-3
Standard deviation 4.773047824183959e-6 6.276983697458527e-6 8.360152180537352e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.027123674601107e-2 1.0280761436659503e-2 1.0294514100885901e-2
Standard deviation 1.9821299233230366e-5 3.2297514802670876e-5 4.71850138714446e-5

Outlying measurements have slight (3.2222222222222215e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.4573894162667745e-3 5.460872564143798e-3 5.4666960094290305e-3
Standard deviation 8.698494533026172e-6 1.336161422312811e-5 2.0078336476992952e-5

Outlying measurements have slight (2.4375e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.1354323306087605e-3 4.138474197330554e-3 4.143908282995059e-3
Standard deviation 7.873948312858421e-6 1.2501873003743917e-5 2.0565676397992192e-5

Outlying measurements have slight (2.2210743801652895e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.549517627459404e-2 7.553873940648279e-2 7.5575813760112e-2
Standard deviation 5.0658841147182914e-5 6.822366967348134e-5 8.482328442133266e-5

Outlying measurements have slight (8.264462809917356e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5818468973285444e-2 1.583254513077027e-2 1.5863700975511683e-2
Standard deviation 2.382920719541112e-5 5.2831917706781453e-5 9.156277684755489e-5

Outlying measurements have slight (3.99305555555555e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.834554917745861e-3 1.8741992999316674e-3 2.029652802126416e-3
Standard deviation 5.818656394509239e-6 2.52274220573295e-4 5.362656527467201e-4

Outlying measurements have severe (0.8005810571968061%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.620182935490194e-2 7.627375197048512e-2 7.635256317248823e-2
Standard deviation 9.534141245131711e-5 1.2052232108558411e-4 1.495646031843688e-4

Outlying measurements have slight (9.000000000000001e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.813423821009181e-2 1.8152172845586262e-2 1.817317383146529e-2
Standard deviation 3.767581318787379e-5 4.579297736529254e-5 5.6355819079756716e-5

Outlying measurements have slight (4.338842975206612e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.034788300917793e-3 4.045083064460538e-3 4.057004414324823e-3
Standard deviation 2.841073486330649e-5 3.588873357342879e-5 4.545595149447844e-5

Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.