criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.200978996488641e-5 8.202553370945025e-5 8.203848190398795e-5
Standard deviation 4.031395721879059e-8 4.95265222174238e-8 6.620617444442145e-8

Outlying measurements have no (8.19616146438085e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.935622845741719e-4 5.936611177339033e-4 5.93774483868275e-4
Standard deviation 2.818885603614142e-7 3.5220578115183574e-7 4.7362994662733584e-7

Outlying measurements have slight (1.2193263222069806e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.776022499397066e-6 5.77695282342469e-6 5.777980197756415e-6
Standard deviation 2.531601333668295e-9 3.2014742056370086e-9 4.4383334357611646e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/boxed vectors, I/O

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.65422130262644e-6 8.655524523032863e-6 8.657106453454752e-6
Standard deviation 3.4507511769374674e-9 4.628221065322133e-9 5.827296172744394e-9

Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.779622274905699e-6 5.7804819553811155e-6 5.781556557827429e-6
Standard deviation 2.4525770665652345e-9 3.1966636180466597e-9 4.773569472910662e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.7805925299422755e-6 5.781805431648853e-6 5.783403978260931e-6
Standard deviation 3.015054690198222e-9 4.55805957774908e-9 6.79402372252481e-9

Outlying measurements have no (5.681632653060961e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.787553733523127e-6 5.788793552161907e-6 5.791350035790199e-6
Standard deviation 3.4079704158918935e-9 5.657074380384473e-9 1.0150155387158191e-8

Outlying measurements have no (5.68163265306106e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.786337928956522e-6 5.78739384030936e-6 5.788948083492559e-6
Standard deviation 3.0257002067603437e-9 4.299111162963674e-9 7.01209108283508e-9

Outlying measurements have no (5.68163265306122e-3%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.652192346721054e-6 8.653941275512476e-6 8.656189834686456e-6
Standard deviation 5.073848744417303e-9 6.432133999305006e-9 8.209418397789077e-9

Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.651663970932305e-6 8.65343925706902e-6 8.655519364346464e-6
Standard deviation 4.7438272558891766e-9 6.571393373474417e-9 9.113543464177411e-9

Outlying measurements have no (5.9521675212449365e-3%) effect on estimated standard deviation.

map + sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.791481995439159e-6 5.792491465239114e-6 5.793630034280189e-6
Standard deviation 3.108281610282875e-9 3.784244104367871e-9 4.701067417226551e-9

Outlying measurements have no (5.681632653061161e-3%) effect on estimated standard deviation.

map + sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.7904958952209705e-6 5.791776469169844e-6 5.793595761123648e-6
Standard deviation 3.454390736204477e-9 5.057724554392284e-9 8.215463943673575e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

map + sum/conduit, connect1

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.956316659219372e-6 5.957102726333318e-6 5.9580909247098275e-6
Standard deviation 2.304907335194339e-9 3.005040044090488e-9 4.15469232156602e-9

Outlying measurements have no (5.681632653061143e-3%) effect on estimated standard deviation.

map + sum/conduit, connect2

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.784478302566626e-6 5.785723004540315e-6 5.7871305909879625e-6
Standard deviation 3.348375229285051e-9 4.583656701200334e-9 7.25732882829443e-9

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

map + sum/conduit, connect3

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.9573341489626185e-6 5.958464858360363e-6 5.9598780885018285e-6
Standard deviation 3.3552160369997932e-9 4.288740273134074e-9 5.7674223573126565e-9

Outlying measurements have no (5.68163265306101e-3%) effect on estimated standard deviation.

map + sum/conduit, inner fuse

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.783047228632048e-6 5.7841056571364484e-6 5.785185731927168e-6
Standard deviation 2.563820051252382e-9 3.2134290845889695e-9 4.05636406588982e-9

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.7053907369126843e-3 3.706791647763219e-3 3.7083291036388117e-3
Standard deviation 3.873955250479329e-6 4.738114709807256e-6 6.184151639671708e-6

Outlying measurements have slight (2.126654064272212e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.4556362344830673e-3 3.45719678271476e-3 3.4586424551346804e-3
Standard deviation 4.115104586763387e-6 4.989897759786735e-6 6.2188050926133415e-6

Outlying measurements have slight (2.082390221819816e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.580983237443337e-3 1.5817556944039248e-3 1.5825095535383874e-3
Standard deviation 2.1011268336207535e-6 2.5405713508821805e-6 3.0465361760675156e-6

Outlying measurements have slight (1.586888657648283e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5306013820385453e-3 1.531006550700252e-3 1.531469064993544e-3
Standard deviation 1.2589758013340123e-6 1.5111118357784166e-6 1.8319069810285986e-6

Outlying measurements have slight (1.586888657648283e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.761552783694778e-3 1.7618001679377204e-3 1.7621373543549707e-3
Standard deviation 7.895751185161201e-7 9.488290340483354e-7 1.2018635370196671e-6

Outlying measurements have slight (1.638888888888889e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8380398363491701e-3 1.8383996129084697e-3 1.8388186263489257e-3
Standard deviation 1.026599512531777e-6 1.2372681724326358e-6 1.5420322364096063e-6

Outlying measurements have slight (1.6661878770468257e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2164263451369863e-3 2.2169722180510575e-3 2.2177700890978636e-3
Standard deviation 1.4773313678212622e-6 2.1254164967558233e-6 3.5186250932827607e-6

Outlying measurements have slight (1.785123966942149e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.331117755479155e-3 2.3317175309939928e-3 2.332778507190131e-3
Standard deviation 1.6730599294731362e-6 2.6063198597114557e-6 4.028197890848815e-6

Outlying measurements have slight (1.8175582990397777e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.3162132958799617e-3 3.3201823659654425e-3 3.324114552457549e-3
Standard deviation 9.544392677924176e-6 1.2809813188976118e-5 1.8326779393974202e-5

Outlying measurements have slight (2.039930555555533e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.4226103417070337e-3 3.423505873366238e-3 3.424597715333968e-3
Standard deviation 2.5146207916647364e-6 3.15449099599542e-6 4.195879802717394e-6

Outlying measurements have slight (2.0399305555555462e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.0548297406293935e-3 2.0550546547607076e-3 2.0553298161353007e-3
Standard deviation 6.492175118002178e-7 8.56253665074788e-7 1.2116110383843353e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.134072015589363e-3 2.1341749895299514e-3 2.1342701123879346e-3
Standard deviation 2.61000767359339e-7 3.2309176571161156e-7 4.183874784194477e-7

Outlying measurements have slight (1.753826530612226e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1337883552397393e-3 2.1339794668293966e-3 2.1341509462245237e-3
Standard deviation 4.719156811558417e-7 6.128193893333282e-7 8.328766286836367e-7

Outlying measurements have slight (1.753826530612237e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2357208387347237e-3 2.2359215486883388e-3 2.2361640030694224e-3
Standard deviation 5.896233702178153e-7 7.291679546990665e-7 9.127037582900374e-7

Outlying measurements have slight (1.785123966942149e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.19490862454675e-3 9.196054414158827e-3 9.197451882398391e-3
Standard deviation 2.480610663044064e-6 3.290493541858537e-6 4.306214856852169e-6

Outlying measurements have slight (3.121748178980213e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.35301081590698e-3 9.35387965827983e-3 9.354725613766258e-3
Standard deviation 1.9743618270583887e-6 2.3319546711869324e-6 2.8392557691486518e-6

Outlying measurements have slight (3.1217481789802253e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1347893780895976e-3 3.1351144190362515e-3 3.135660320866398e-3
Standard deviation 6.814769531715e-7 1.3815081634592919e-6 2.1362567032016577e-6

Outlying measurements have slight (1.999167013744264e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.207831623189105e-3 3.2085186493118305e-3 3.209054140358104e-3
Standard deviation 1.895205514587815e-6 2.0401049220293815e-6 2.212136250068326e-6

Outlying measurements have slight (1.9991670137442685e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1205763036524577e-3 2.121414811143434e-3 2.122317418448557e-3
Standard deviation 2.3914934035522996e-6 3.044181744121338e-6 3.79109269791041e-6

Outlying measurements have slight (1.7538265306122385e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1894265673342373e-3 2.189653230345583e-3 2.1899114232467654e-3
Standard deviation 6.714921554210775e-7 8.285676189668749e-7 1.081865862632918e-6

Outlying measurements have slight (1.7538265306122437e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.21898080841322e-2 7.21920416539635e-2 7.219432761003247e-2
Standard deviation 2.752290061931607e-6 4.000395634915852e-6 5.733581981899156e-6

Outlying measurements have slight (8.264462809917354e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.239093402776332e-2 7.23943163354075e-2 7.239724526915335e-2
Standard deviation 3.871524933174877e-6 5.5112808897136165e-6 7.2663725350667915e-6

Outlying measurements have slight (8.264462809917347e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5847870102747935e-2 1.5852788617769294e-2 1.585519915317417e-2
Standard deviation 4.3604649795804345e-6 8.957696878483026e-6 1.6970976363305467e-5

Outlying measurements have slight (3.9930555555555414e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.609981136245623e-2 1.6120786265971555e-2 1.6142991722375426e-2
Standard deviation 4.246516012670192e-5 5.349157363533747e-5 6.88401629187692e-5

Outlying measurements have slight (3.993055555555555e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9829243132246962e-3 2.0244187600326766e-3 2.1872941802311926e-3
Standard deviation 9.644658355205605e-6 2.6270104866730923e-4 5.578997258806828e-4

Outlying measurements have severe (0.7964155337038413%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.054666560306108e-3 2.0561044109321954e-3 2.0581373109528865e-3
Standard deviation 4.078843124762703e-6 5.626079814636857e-6 8.682358621964176e-6

Outlying measurements have slight (1.7236072637734613e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.