criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.033215144745678e-5 8.034831556854684e-5 8.042360293997327e-5
Standard deviation 2.1489161408174037e-8 9.274221760105146e-8 2.054350807787325e-7

Outlying measurements have no (8.196161464380783e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.893094470139216e-4 5.893446356729368e-4 5.893754358000811e-4
Standard deviation 9.264251947650342e-8 1.1301214782298991e-7 1.3834803574739785e-7

Outlying measurements have slight (1.2193263222069792e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.75494638297788e-6 5.755071343980865e-6 5.755257389199044e-6
Standard deviation 3.8674883335235203e-10 5.273355820450178e-10 7.494206786073123e-10

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.763064279419362e-6 5.763528265144594e-6 5.764447684665212e-6
Standard deviation 1.0255648544522794e-9 2.041725058859452e-9 4.014662386186746e-9

Outlying measurements have no (5.68163265306107e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.763014577011684e-6 5.763280678098752e-6 5.763567271439581e-6
Standard deviation 7.92330947441478e-10 9.496775994187801e-10 1.1942499826072816e-9

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8527384426530305e-4 1.8528562883799775e-4 1.852990039060086e-4
Standard deviation 3.552313730721171e-8 4.3716257725035075e-8 5.4794658608699845e-8

Outlying measurements have no (9.522928994082842e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.519044416990382e-4 6.519587755424521e-4 6.52023846948967e-4
Standard deviation 1.780542747784484e-7 2.0865198471971387e-7 2.6103930933105816e-7

Outlying measurements have slight (1.2497997115846685e-2%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8160556750120154e-4 1.8161841898503824e-4 1.816390147490908e-4
Standard deviation 4.0363888362977336e-8 5.5699862212050825e-8 8.569937310360774e-8

Outlying measurements have no (9.433106575963617e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0414163579356862e-3 1.0415719830232397e-3 1.0417403314087678e-3
Standard deviation 4.699858152942583e-7 5.627672536384494e-7 6.799521598362818e-7

Outlying measurements have slight (1.4081632653061097e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.431719735607036e-3 3.4330595895722483e-3 3.4343207703042143e-3
Standard deviation 3.2538746618349146e-6 3.951941254809009e-6 5.015220484912488e-6

Outlying measurements have slight (2.082390221819791e-2%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3342102033429564e-2 1.3345094344236209e-2 1.3348103017699425e-2
Standard deviation 5.9987052702949865e-6 7.885306057091114e-6 1.1098175126678947e-5

Outlying measurements have slight (3.698224852070998e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5121463990432327e-3 1.5124610641006704e-3 1.5127383247081961e-3
Standard deviation 8.293010402216655e-7 1.0171638161084653e-6 1.4014222820957787e-6

Outlying measurements have slight (1.5621063240110861e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.664269094192341e-3 1.6644938623150208e-3 1.6647332772209151e-3
Standard deviation 6.448873822578524e-7 8.044664284956318e-7 9.988888896329453e-7

Outlying measurements have slight (1.6124697661918687e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.30110197398555e-3 2.3016037241758444e-3 2.302504020535686e-3
Standard deviation 1.3802311659640296e-6 2.2589086343959683e-6 3.736603773703366e-6

Outlying measurements have slight (1.7851239669421298e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.3081524492889e-3 2.3084829304719683e-3 2.3088741228707026e-3
Standard deviation 9.907349367854003e-7 1.224293389727956e-6 1.612018000108829e-6

Outlying measurements have slight (1.785123966942134e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.920406317601597e-3 8.923635929750448e-3 8.931110143842859e-3
Standard deviation 6.940930897089675e-6 1.3226464464423624e-5 2.4144553404562662e-5

Outlying measurements have slight (3.0273437499999948e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.62189931495284e-3 9.624173179040381e-3 9.627410837876303e-3
Standard deviation 4.9086661296791175e-6 7.482098489619955e-6 1.0612225646254854e-5

Outlying measurements have slight (3.121748178980213e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.410905895908723e-3 3.4120902830370144e-3 3.413930393504338e-3
Standard deviation 3.3696685596587884e-6 4.5934133395930765e-6 7.249340637863483e-6

Outlying measurements have slight (2.039930555555541e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9778998200768016e-3 1.979106359854091e-3 1.9823000597584788e-3
Standard deviation 2.9178522675574305e-6 6.242964702195969e-6 1.179638083825667e-5

Outlying measurements have slight (1.6944114149821585e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2348945282162553e-3 2.2356248901398e-3 2.236665512095786e-3
Standard deviation 2.216450632998817e-6 2.9575828766017627e-6 4.332828966295815e-6

Outlying measurements have slight (1.785123966942149e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.3197094537074255e-3 4.326742351604152e-3 4.340300153702711e-3
Standard deviation 1.2703045078104195e-5 2.952731885415408e-5 4.7431595128878856e-5

Outlying measurements have slight (2.271498107084897e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.005120778213919e-3 9.016016444117973e-3 9.052080672038847e-3
Standard deviation 1.0114750400221084e-5 5.1288065647645144e-5 1.0480072096212416e-4

Outlying measurements have slight (3.02734375e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.156705331664747e-3 9.161889902371882e-3 9.178407171242131e-3
Standard deviation 8.996614056003973e-6 2.3436167445973076e-5 4.524110307646063e-5

Outlying measurements have slight (3.1217481789802035e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.58894987486149e-3 9.60188056114661e-3 9.65019223138805e-3
Standard deviation 5.322718813972647e-6 6.587233703316086e-5 1.360222127418246e-4

Outlying measurements have slight (3.121748178980223e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.068415838213535e-3 3.0718637004761283e-3 3.078299872354803e-3
Standard deviation 1.0457825956475865e-5 1.4220792106493137e-5 2.201785995492172e-5

Outlying measurements have slight (1.9991670137442505e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1247138982424256e-3 2.1265712833072616e-3 2.1321980619154316e-3
Standard deviation 4.568357107618319e-6 9.609144097562137e-6 2.118487701368696e-5

Outlying measurements have slight (1.7538265306122347e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0591848600051624e-2 1.0598037849423834e-2 1.0604878473370358e-2
Standard deviation 1.3272513969825497e-5 1.7192781503532566e-5 2.34960352671549e-5

Outlying measurements have slight (3.3293697978596756e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.941056463180237e-3 9.951146940081446e-3 9.975642177625671e-3
Standard deviation 1.826633073396144e-5 4.079095311454e-5 8.009561942036741e-5

Outlying measurements have slight (3.222222222222222e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.902904071285762e-3 8.910083643299656e-3 8.916947784902342e-3
Standard deviation 1.6365889731320598e-5 2.005669081499954e-5 2.452411465464508e-5

Outlying measurements have slight (3.02734375e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.560325440858234e-2 7.565166190421009e-2 7.569043554164255e-2
Standard deviation 4.524297756868892e-5 7.212727185778656e-5 1.1390209690250106e-4

Outlying measurements have slight (8.264462809917357e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5837691442957436e-2 1.5857220380524326e-2 1.5876813159322378e-2
Standard deviation 4.00235343327669e-5 4.9329588104041284e-5 6.060645782116215e-5

Outlying measurements have slight (3.993055555555522e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8902010564375157e-3 1.929666723096848e-3 2.0827646790144917e-3
Standard deviation 9.01911685821813e-6 2.5089858563317156e-4 5.340067622438734e-4

Outlying measurements have severe (0.7998807741188889%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.675560602349576e-2 7.689161844875034e-2 7.697809272700434e-2
Standard deviation 9.962167516221392e-5 1.7143225079380256e-4 2.51493720574029e-4

Outlying measurements have slight (9.0e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1787315776038874e-2 2.1818103048044123e-2 2.184140613003472e-2
Standard deviation 4.1205098967301716e-5 5.9750319224589364e-5 8.751135254976553e-5

Outlying measurements have slight (4.75e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.945312644122484e-3 7.959994978019648e-3 7.976564376473996e-3
Standard deviation 3.3326724912734826e-5 4.2462388987713765e-5 5.684597145209705e-5

Outlying measurements have slight (2.8546712802768166e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.