criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.125316176138798e-5 8.129950172508953e-5 8.137666413766647e-5
Standard deviation 1.4950914624388357e-7 1.983194924969662e-7 2.7163417347049075e-7

Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.970346379198125e-4 5.977796388793879e-4 5.990180558626078e-4
Standard deviation 2.180148198638494e-6 3.235689596549572e-6 5.170808668108526e-6

Outlying measurements have slight (1.2193263222069489e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.74917050167312e-6 5.751115626359779e-6 5.756154127680543e-6
Standard deviation 3.1104170628972967e-9 9.48729140960894e-9 1.9843895484271095e-8

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/boxed vectors, I/O

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.626847257683185e-6 8.632236009522514e-6 8.640011559949345e-6
Standard deviation 1.560335427712077e-8 2.0842443902262948e-8 2.8879826064439297e-8

Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.760031243369299e-6 5.7632922171468895e-6 5.768525214922163e-6
Standard deviation 9.31140679725454e-9 1.337981803828594e-8 1.9026709301940808e-8

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.760911904295693e-6 5.764134129512608e-6 5.769178710191435e-6
Standard deviation 9.569107560952662e-9 1.3327712203772554e-8 1.8456233828659987e-8

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.658250544958975e-6 8.670044563538928e-6 8.684790709797013e-6
Standard deviation 3.26033243764104e-8 4.550681658801341e-8 7.303554058221922e-8

Outlying measurements have no (5.952167521245102e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.821563358180641e-5 8.830089802709136e-5 8.839810116904793e-5
Standard deviation 2.6817164633854646e-7 3.163036119646593e-7 3.7799353045138296e-7

Outlying measurements have no (8.26388888888888e-3%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.150584978950254e-5 1.151665380363164e-5 1.1531257197248227e-5
Standard deviation 3.4034732044646935e-8 4.116353829802586e-8 5.4301470649938765e-8

Outlying measurements have no (6.172601365688123e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.541814080744619e-5 8.548909113057639e-5 8.560752438366873e-5
Standard deviation 2.160914920762318e-7 2.923407083554638e-7 4.946731517124888e-7

Outlying measurements have no (8.263888888888817e-3%) effect on estimated standard deviation.

map + sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.9439598862756175e-6 5.947833169398621e-6 5.95274655729247e-6
Standard deviation 1.0592913544256484e-8 1.5119260988432043e-8 1.9004523054116664e-8

Outlying measurements have no (5.681632653061116e-3%) effect on estimated standard deviation.

map + sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.9441870491081675e-6 5.947608576622326e-6 5.953498082636309e-6
Standard deviation 1.0807515959704501e-8 1.4522819553884204e-8 1.9837412869724338e-8

Outlying measurements have no (5.681632653061038e-3%) effect on estimated standard deviation.

map + sum/conduit, connect1

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.346018117738933e-4 6.351524807383135e-4 6.358271052969135e-4
Standard deviation 1.6100504599029886e-6 2.1071766357934644e-6 3.164524625804439e-6

Outlying measurements have slight (1.234375e-2%) effect on estimated standard deviation.

map + sum/conduit, connect2

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.835227223314004e-4 5.841739742789186e-4 5.850700709913633e-4
Standard deviation 2.0143033599048285e-6 2.491538084187158e-6 3.414297092489558e-6

Outlying measurements have slight (1.219326322206954e-2%) effect on estimated standard deviation.

map + sum/conduit, connect3

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.212660927278225e-4 5.216787671221704e-4 5.220611986999907e-4
Standard deviation 1.1635518484684425e-6 1.359931877074997e-6 1.5851546981021736e-6

Outlying measurements have slight (1.1763038548752517e-2%) effect on estimated standard deviation.

map + sum/conduit, inner fuse

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.159679296444914e-4 5.162312145165072e-4 5.165685971503418e-4
Standard deviation 8.629287734623248e-7 1.0312744661618293e-6 1.2363714638740263e-6

Outlying measurements have slight (1.1763038548752833e-2%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.337752357755636e-3 5.341387399622476e-3 5.345641809962247e-3
Standard deviation 9.163657804482997e-6 1.1983688747878939e-5 1.7016394863520117e-5

Outlying measurements have slight (2.4375e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.469690096068386e-3 3.472505362842744e-3 3.475169388674715e-3
Standard deviation 7.393091054977912e-6 8.81977029467318e-6 1.1130287799556675e-5

Outlying measurements have slight (2.0823902218198246e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.529804435415924e-3 1.5306554769249166e-3 1.5318890438274257e-3
Standard deviation 2.3133951605296183e-6 3.4138693812621976e-6 4.68910264557361e-6

Outlying measurements have slight (1.586888657648283e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.895266964848574e-3 1.8964374719388835e-3 1.8981077149174527e-3
Standard deviation 3.6687718870054974e-6 4.955572410650941e-6 6.951032819667142e-6

Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7406561519314513e-3 1.741939632318957e-3 1.7436531568878564e-3
Standard deviation 3.7630369847545036e-6 5.018410470291441e-6 7.100229305760256e-6

Outlying measurements have slight (1.6388888888888765e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.3738622252696138e-3 3.3772485115569255e-3 3.380390894162564e-3
Standard deviation 9.140074293865328e-6 1.1063406082921327e-5 1.453425448512702e-5

Outlying measurements have slight (2.0399305555555407e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2536634444670026e-3 2.256027479874095e-3 2.2592819377073377e-3
Standard deviation 6.901651230070295e-6 9.471391923659867e-6 1.3489158752961605e-5

Outlying measurements have slight (1.7851239669421485e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.050057168988636e-3 4.053880984223189e-3 4.058201754660315e-3
Standard deviation 1.1508142758778834e-5 1.3414450150568548e-5 1.6807254330294353e-5

Outlying measurements have slight (2.1728395061728332e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.488088791891946e-3 3.4912062388519982e-3 3.4955318766590983e-3
Standard deviation 9.75745212306961e-6 1.2161178334645656e-5 1.839821031998955e-5

Outlying measurements have slight (2.0823902218198086e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.8650690706926692e-3 3.869798311763203e-3 3.877608202355911e-3
Standard deviation 1.292593158609748e-5 1.8341622777346755e-5 3.207563776555454e-5

Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.051900639890731e-3 2.054345243358931e-3 2.05787126196837e-3
Standard deviation 8.049945736456613e-6 9.912556814281058e-6 1.3357516982010825e-5

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.627659341255608e-3 3.630339017537377e-3 3.6336355764432214e-3
Standard deviation 7.85272573375388e-6 9.436273945621291e-6 1.1644265646039392e-5

Outlying measurements have slight (2.1266540642722116e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1608228986743517e-3 2.162461153573839e-3 2.1647970162875737e-3
Standard deviation 5.17800693066317e-6 6.656050077728642e-6 8.713346527463114e-6

Outlying measurements have slight (1.753826530612239e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.0094770227215165e-3 4.012490221826493e-3 4.015862310766827e-3
Standard deviation 9.50160469723356e-6 1.0582747216048797e-5 1.2344263211285676e-5

Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.344503452173528e-3 9.353007797115101e-3 9.363459198792765e-3
Standard deviation 2.1279327809914752e-5 2.5585780586932992e-5 3.153678238688347e-5

Outlying measurements have slight (3.1217481789802288e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.824938809433224e-3 9.836556157354745e-3 9.850644915101724e-3
Standard deviation 2.779953075755451e-5 3.341004018329876e-5 4.244120695122805e-5

Outlying measurements have slight (3.222222222222212e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.146702076942491e-3 3.149892052009288e-3 3.154463973700717e-3
Standard deviation 1.034315997146129e-5 1.2636528799015154e-5 1.598267882766762e-5

Outlying measurements have slight (1.9991670137442685e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.738346828025309e-3 4.743465813639795e-3 4.751041220133328e-3
Standard deviation 1.3266788626527203e-5 1.7828675788135046e-5 2.7296740862656733e-5

Outlying measurements have slight (2.3242630385487528e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1255024943930513e-3 2.12770583295492e-3 2.129817302294408e-3
Standard deviation 5.795958353823756e-6 6.913033971040142e-6 8.717893646257134e-6

Outlying measurements have slight (1.753826530612224e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.8082676123562423e-3 3.8117676248770184e-3 3.819182834511613e-3
Standard deviation 8.808481281834631e-6 1.5216637048430866e-5 2.6463279608461232e-5

Outlying measurements have slight (2.126654064272189e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.244176556205949e-2 7.249819651903675e-2 7.253895809743349e-2
Standard deviation 4.931905984781488e-5 7.949844836545032e-5 1.1541054270055978e-4

Outlying measurements have slight (8.264462809917356e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.308817519517671e-2 7.312123432277165e-2 7.315699327812208e-2
Standard deviation 4.076294829648093e-5 5.7990582204541764e-5 9.109413807601336e-5

Outlying measurements have slight (8.264462809917353e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5869719157128963e-2 1.5883937261823185e-2 1.5919733583385665e-2
Standard deviation 2.1915650169220274e-5 5.342285348505935e-5 1.0717886418015289e-4

Outlying measurements have slight (3.9930555555555386e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7573684913960205e-2 1.759278527365787e-2 1.7613066047824796e-2
Standard deviation 3.6780585940390876e-5 4.520613930989e-5 6.063317070350517e-5

Outlying measurements have slight (4.158790170132321e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9110541278629114e-3 1.9511527546967613e-3 2.149945330213388e-3
Standard deviation 4.5823728945283805e-6 2.600572915024288e-4 5.975796726098554e-4

Outlying measurements have severe (0.800386733302099%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.5540277941793442e-3 3.562887659406095e-3 3.5715242014596484e-3
Standard deviation 2.318402848927872e-5 2.7655880612574962e-5 3.475554340462335e-5

Outlying measurements have slight (2.082390221819827e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.