criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.065956996647648e-5 8.066851301303836e-5 8.068619896731302e-5
Standard deviation 2.5407974813797872e-8 3.992244904950421e-8 7.016597168503847e-8

Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.842372592887739e-4 5.842749061738654e-4 5.843173848219795e-4
Standard deviation 1.2085413784013693e-7 1.4094098389233178e-7 1.7222275017001099e-7

Outlying measurements have slight (1.2193263222069765e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.754412621539532e-6 5.754632705563855e-6 5.755010938189453e-6
Standard deviation 5.47747471539394e-10 1.0040736429956428e-9 1.7581982860606895e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.764025791163836e-6 5.764466679942933e-6 5.765059895718508e-6
Standard deviation 1.2308615719524405e-9 1.5949801852768564e-9 2.0803121436023114e-9

Outlying measurements have no (5.6816326530611215e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.764494243171081e-6 5.764974949877927e-6 5.765459679948825e-6
Standard deviation 1.2882712650142253e-9 1.547526192651043e-9 1.9141609548929535e-9

Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.614887296052375e-6 8.615085207403243e-6 8.615323179221518e-6
Standard deviation 6.085672730335036e-10 7.532895394042403e-10 9.894627759196977e-10

Outlying measurements have no (5.9521675212449165e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.355516630429168e-5 8.356186224010829e-5 8.356848006844966e-5
Standard deviation 1.8455259235061703e-8 2.2316897523092854e-8 2.8057865665749975e-8

Outlying measurements have no (8.196161464380788e-3%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1480558754269431e-5 1.1480941658506174e-5 1.1481437410450051e-5
Standard deviation 1.0353700679403412e-9 1.4325234731143843e-9 2.113261641926567e-9

Outlying measurements have no (6.1726013656880525e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.930164619411127e-5 7.931796017897935e-5 7.933540854746231e-5
Standard deviation 4.588026657164087e-8 5.608861481065683e-8 6.807596589133285e-8

Outlying measurements have no (8.129535071217353e-3%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.468621603600873e-3 3.4702818571291758e-3 3.471678181398824e-3
Standard deviation 4.128110783919427e-6 5.122190451742985e-6 7.24818757703174e-6

Outlying measurements have slight (2.0823902218198184e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5761479391294346e-2 1.5784378490364525e-2 1.5802088419419755e-2
Standard deviation 3.476611060925457e-5 4.911114248078068e-5 6.997118398669895e-5

Outlying measurements have slight (3.993055555555551e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives, Codensity

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.201116136559248e-2 2.2025541652676645e-2 2.2051873339856847e-2
Standard deviation 2.7627936315397216e-5 4.246870603651231e-5 6.671778234140492e-5

Outlying measurements have slight (4.750000000000001e-2%) effect on estimated standard deviation.

monte carlo/conduit, ConduitM primitives, explicit binding order

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.1898400560597075e-3 5.1913310144913235e-3 5.193175718369111e-3
Standard deviation 4.163248472560582e-6 5.2708228667269145e-6 6.730119285616033e-6

Outlying measurements have slight (2.4375e-2%) effect on estimated standard deviation.

monte carlo/conduit, Pipe primitives

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5471177838481424e-2 1.5476630052212212e-2 1.5483700611579635e-2
Standard deviation 1.1127094419313163e-5 1.6238946583610072e-5 2.1673924738439524e-5

Outlying measurements have slight (3.9930555555555414e-2%) effect on estimated standard deviation.

monte carlo/conduit, Pipe constructos

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.843396575441765e-3 4.84489279452913e-3 4.846964816251458e-3
Standard deviation 3.9763328214235104e-6 5.475680511907445e-6 8.140812372889953e-6

Outlying measurements have slight (2.324263038548744e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3526646782027522e-3 1.3527744204972923e-3 1.3530110911294906e-3
Standard deviation 2.955116223106822e-7 4.923769164000065e-7 9.156764929057416e-7

Outlying measurements have slight (1.514792899408277e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7391117923924691e-3 1.739234968656599e-3 1.739370792696255e-3
Standard deviation 3.724059867868702e-7 4.4971597482458963e-7 5.582546330272438e-7

Outlying measurements have slight (1.6388888888888887e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.172913667485704e-3 2.1731267104408755e-3 2.1733628466452564e-3
Standard deviation 6.19946584788987e-7 7.435238469265922e-7 9.59480865088757e-7

Outlying measurements have slight (1.75382653061223e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7915872086630353e-3 1.7919233339981039e-3 1.7923226085130443e-3
Standard deviation 1.0090555216299087e-6 1.3017997467562972e-6 1.679277054803635e-6

Outlying measurements have slight (1.666187877046811e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.45759740319892e-3 7.4598164581618e-3 7.462183437193626e-3
Standard deviation 5.5962068439259386e-6 6.883815071047344e-6 8.597655809479361e-6

Outlying measurements have slight (2.7755102040816156e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.81414950730993e-3 7.814932461556926e-3 7.815830839631022e-3
Standard deviation 1.7551130329018508e-6 2.3649953159083152e-6 3.1524156060397802e-6

Outlying measurements have slight (2.8546712802768166e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.4405274229914753e-3 3.441171289421724e-3 3.4419414614259257e-3
Standard deviation 1.953306393646736e-6 2.3408970131815534e-6 2.8610807748247496e-6

Outlying measurements have slight (2.082390221819824e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.046244686873941e-3 2.046380204237562e-3 2.0465732948174134e-3
Standard deviation 4.0082898659279606e-7 5.388829801286559e-7 8.501956395930682e-7

Outlying measurements have slight (1.723607263773469e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.096093102977742e-3 2.096323997175508e-3 2.096703507874978e-3
Standard deviation 6.920533289166901e-7 1.0065051061323785e-6 1.4638456080296715e-6

Outlying measurements have slight (1.7538265306122448e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.913969908763736e-3 3.91508245156891e-3 3.916534769363819e-3
Standard deviation 2.8729017346099914e-6 4.121290939337299e-6 6.624060782700059e-6

Outlying measurements have slight (2.1728395061728398e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.489493127027179e-3 7.490832107944578e-3 7.4926498271106045e-3
Standard deviation 3.190156869053635e-6 4.293593710447965e-6 6.203008045855209e-6

Outlying measurements have slight (2.775510204081632e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.481192546048042e-3 7.482533644135223e-3 7.4843317077191345e-3
Standard deviation 3.3479624234482957e-6 4.5539397162950845e-6 6.4418015245584885e-6

Outlying measurements have slight (2.7755102040816326e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0478306520018505e-2 1.0483906307161684e-2 1.0503725060803581e-2
Standard deviation 4.813960757882597e-6 2.622430034130168e-5 5.352473368020901e-5

Outlying measurements have slight (3.2222222222222124e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1369570330653253e-3 3.138080515392989e-3 3.1396499610659157e-3
Standard deviation 3.166483720689194e-6 4.25627352565169e-6 6.111857096747686e-6

Outlying measurements have slight (1.9991670137442612e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.067595994587502e-3 2.069617322345205e-3 2.0720676423318647e-3
Standard deviation 6.502625748664143e-6 7.171425585100934e-6 7.862403558722112e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1065321176221086e-2 1.106848685807341e-2 1.1074477423462759e-2
Standard deviation 6.9610497217988226e-6 1.076503532349315e-5 1.5911454262575558e-5

Outlying measurements have slight (3.329369797859683e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.558120379218341e-3 8.561677027165104e-3 8.566627975010328e-3
Standard deviation 8.299667112775489e-6 1.1430816741259301e-5 1.7500341899429326e-5

Outlying measurements have slight (3.0273437499999997e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.252947219999909e-3 7.2710024774443665e-3 7.342785992038761e-3
Standard deviation 5.593476853765493e-6 9.895345340489151e-5 2.0616188665884558e-4

Outlying measurements have slight (2.7755102040816312e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.963374602889505e-2 8.96450087307657e-2 8.966334081171347e-2
Standard deviation 5.649205118194016e-6 2.3740108640893234e-5 2.971847964501684e-5

Outlying measurements have slight (9.0e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5871278593983726e-2 1.5875131819062714e-2 1.587987282643851e-2
Standard deviation 7.701224599638815e-6 1.081501717441649e-5 1.483094304996885e-5

Outlying measurements have slight (3.993055555555543e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.869313377274502e-3 1.9092890056287243e-3 2.0639528971656876e-3
Standard deviation 5.871802030097757e-6 2.5250205015426295e-4 5.362143068626006e-4

Outlying measurements have severe (0.8002303073563125%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.034155810663917e-2 9.035802410967525e-2 9.037295629949921e-2
Standard deviation 1.7977765625972805e-5 2.4718588080055677e-5 3.432225493671321e-5

Outlying measurements have slight (9.000000000000001e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.0996476066078275e-2 2.1007513503568144e-2 2.1020887649093954e-2
Standard deviation 2.0692045915082105e-5 2.7772798747798234e-5 3.584090473511911e-5

Outlying measurements have slight (4.5351473922902494e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.727542535482776e-3 6.733433662674848e-3 6.739396751063569e-3
Standard deviation 1.2337249169071239e-5 1.6293028607252863e-5 2.1809990325873184e-5

Outlying measurements have slight (2.7006172839506046e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.