criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.063063029420278e-5 8.065897550341402e-5 8.071415096222987e-5
Standard deviation 7.663069068862863e-8 1.327323891267452e-7 2.3083552702591478e-7

Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.189942927317045e-4 7.193530769653197e-4 7.199154696906915e-4
Standard deviation 1.045862614139102e-6 1.4230835295881444e-6 2.002348102539738e-6

Outlying measurements have slight (1.2818350480688143e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.757741569810475e-6 5.7598079308809575e-6 5.763531002540638e-6
Standard deviation 6.990243647923174e-9 9.280499647332705e-9 1.3593084365625443e-8

Outlying measurements have no (5.681632653061159e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.765150155104442e-6 5.767255169375647e-6 5.770620972714444e-6
Standard deviation 6.380614426067382e-9 9.38937902612314e-9 1.5944622158737647e-8

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.767311862827392e-6 5.77040100096304e-6 5.773740689528214e-6
Standard deviation 8.290610325459581e-9 1.0204066211777313e-8 1.2928606953595316e-8

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8600268439299784e-4 1.8617203089119788e-4 1.865557815934372e-4
Standard deviation 3.6720167441228147e-7 7.774383572330349e-7 1.4890518903865035e-6

Outlying measurements have no (9.522928994082696e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.431128636293632e-4 6.435972211719634e-4 6.4437580186269e-4
Standard deviation 1.4826210434768177e-6 2.1245531661440643e-6 3.1547409665286144e-6

Outlying measurements have slight (1.2497997115846798e-2%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8199038512444043e-4 1.8211225545118777e-4 1.8231370696794792e-4
Standard deviation 3.966750998647503e-7 5.299316148465181e-7 7.056862301349035e-7

Outlying measurements have no (9.433106575963721e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0897770695941402e-3 1.090563077556839e-3 1.092182006313877e-3
Standard deviation 2.3767714475987275e-6 3.5318279815887916e-6 5.68732526846758e-6

Outlying measurements have slight (1.4282713715605966e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.436368788019751e-3 3.4411476811440824e-3 3.4535248780676363e-3
Standard deviation 9.35068834394308e-6 2.4285764988793634e-5 4.859492714612691e-5

Outlying measurements have slight (2.0823902218198225e-2%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3758543549873749e-2 1.3771832930007118e-2 1.3803698327291884e-2
Standard deviation 2.4138006747208774e-5 4.9503134496067194e-5 8.838647783109975e-5

Outlying measurements have slight (3.6982248520710026e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.42781826654655e-3 1.428730628848882e-3 1.430184320117427e-3
Standard deviation 2.767720280962022e-6 3.9513225409450175e-6 5.835316954420705e-6

Outlying measurements have slight (1.538085937499993e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.929878600155207e-3 1.9310010587875873e-3 1.9326936770412801e-3
Standard deviation 3.287071496457978e-6 4.490044765967474e-6 6.271518280913007e-6

Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2628214323297995e-3 2.2644459335772497e-3 2.2679841473479422e-3
Standard deviation 3.5903050712835083e-6 8.082993788214103e-6 1.3603951748724889e-5

Outlying measurements have slight (1.7851239669421485e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.93727374687814e-3 1.9376048189204138e-3 1.9380752516770425e-3
Standard deviation 9.243368667978584e-7 1.2715207891688293e-6 2.0708913423181766e-6

Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.6468888046920475e-3 7.648111763838822e-3 7.649661508562464e-3
Standard deviation 2.715584561590118e-6 3.85980990384062e-6 5.520246684113479e-6

Outlying measurements have slight (2.8546712802768087e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.889739212307733e-3 8.898657234297091e-3 8.914334993402318e-3
Standard deviation 2.166109620324039e-5 3.2129676400528114e-5 5.137098553331747e-5

Outlying measurements have slight (3.0273437499999997e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.33612249926989e-3 3.338345351742686e-3 3.342114700605778e-3
Standard deviation 6.8585641670523305e-6 9.531883160356786e-6 1.5053871483557248e-5

Outlying measurements have slight (2.0399305555555445e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.253045985113646e-3 2.254665518083557e-3 2.2565438546336804e-3
Standard deviation 5.019554127712189e-6 5.9442814947032415e-6 7.1398013915182515e-6

Outlying measurements have slight (1.785123966942149e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1773500561592034e-3 2.179186840933559e-3 2.1827929379180724e-3
Standard deviation 5.038267945024815e-6 8.228130476236825e-6 1.4189661353765168e-5

Outlying measurements have slight (1.7538265306122448e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.9080638265094e-3 3.911895366985719e-3 3.916049489872195e-3
Standard deviation 1.1166939490785406e-5 1.3103111896983346e-5 1.5724786510927567e-5

Outlying measurements have slight (2.1728395061728398e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.877141526969857e-3 7.879886206085344e-3 7.884330391143779e-3
Standard deviation 6.684619487053192e-6 9.44898255244797e-6 1.2492258844205642e-5

Outlying measurements have slight (2.8546712802768163e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.570179825563668e-3 8.571253400481373e-3 8.57322097023658e-3
Standard deviation 2.5256086321448867e-6 3.954593537757722e-6 6.409232222255917e-6

Outlying measurements have slight (3.02734375e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.278988110832042e-3 9.279824145191757e-3 9.280935348223721e-3
Standard deviation 2.1153865237437864e-6 2.5833978775357055e-6 3.301377491377686e-6

Outlying measurements have slight (3.1217481789802257e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.3353702880973093e-3 3.336789240086499e-3 3.3392387371834866e-3
Standard deviation 4.421748749246466e-6 6.1608492149706705e-6 8.477918039471495e-6

Outlying measurements have slight (2.0399305555555417e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1415998779392787e-3 2.1429420098334306e-3 2.1446815352777386e-3
Standard deviation 3.5407420115837796e-6 4.832921007246594e-6 7.28163909258246e-6

Outlying measurements have slight (1.7538265306122448e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0039100395739489e-2 1.0043807334417645e-2 1.0049016688482546e-2
Standard deviation 1.0203841849338177e-5 1.3604603673583366e-5 1.8896738449877486e-5

Outlying measurements have slight (3.222222222222222e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.951308823727012e-3 8.95518935359405e-3 8.960625047601556e-3
Standard deviation 1.0188165025557004e-5 1.3038533768811844e-5 1.7894073228999347e-5

Outlying measurements have slight (3.02734375e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.520297125418751e-3 8.524712419695978e-3 8.53485059828594e-3
Standard deviation 1.0287528711214386e-5 1.7168950818292636e-5 2.969145929394518e-5

Outlying measurements have slight (2.9384756657483836e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.243571900686445e-2 7.249615777911027e-2 7.257204407514717e-2
Standard deviation 7.942839983065064e-5 1.1517486685289366e-4 1.6232674238888774e-4

Outlying measurements have slight (8.264462809917344e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.607676502578943e-2 1.6092113933720543e-2 1.611037848487328e-2
Standard deviation 3.074400943663239e-5 4.299297577378736e-5 5.947111462400733e-5

Outlying measurements have slight (3.9930555555555275e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9731589412736567e-3 2.014259199211689e-3 2.258877992510061e-3
Standard deviation 4.438444300604463e-6 2.6337434960623917e-4 6.433587428086761e-4

Outlying measurements have severe (0.7965752747427772%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.337837596934052e-2 7.345607489210423e-2 7.35413418554026e-2
Standard deviation 1.0115651062866028e-4 1.4247675921479143e-4 1.9755613167214938e-4

Outlying measurements have slight (8.264462809917347e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1539218112913525e-2 2.1558591350141338e-2 2.1589507363043037e-2
Standard deviation 3.483109135469768e-5 5.548371239516179e-5 9.707757808583772e-5

Outlying measurements have slight (4.5351473922902494e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.92948856231594e-3 7.935192174253832e-3 7.94577162161664e-3
Standard deviation 1.4163187222666306e-5 2.1773510566004588e-5 3.425729870985561e-5

Outlying measurements have slight (2.8546712802768166e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.