criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.076625071921971e-5 8.07735276067103e-5 8.078190712255119e-5
Standard deviation 2.2319285480238645e-8 2.6598291055114367e-8 3.5057953315985684e-8

Outlying measurements have no (8.196161464380642e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.96307493222459e-4 5.963515205726343e-4 5.964017958334409e-4
Standard deviation 1.2955126986251228e-7 1.6283731166833463e-7 2.4753998739041407e-7

Outlying measurements have slight (1.2193263222069805e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.7543410315100955e-6 5.754453620738119e-6 5.754572129041224e-6
Standard deviation 3.030039514718732e-10 3.6794184636499837e-10 4.494446348773569e-10

Outlying measurements have no (5.681632653060895e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.762209002143261e-6 5.762455871605136e-6 5.762746398693253e-6
Standard deviation 7.072097454620155e-10 8.617953865852868e-10 1.0829667761090853e-9

Outlying measurements have no (5.6816326530612065e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.761926293457681e-6 5.762200981699823e-6 5.762549214643014e-6
Standard deviation 7.411016130622529e-10 1.0212932391870627e-9 1.488771995327922e-9

Outlying measurements have no (5.681632653061203e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8543616796795728e-4 1.8545189747245103e-4 1.8546706711665547e-4
Standard deviation 4.500373013929295e-8 5.3052376310271026e-8 6.306961355727904e-8

Outlying measurements have no (9.522928994082842e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.56245178285084e-4 6.56297140820494e-4 6.563445564418866e-4
Standard deviation 1.434853327981052e-7 1.7625135321985597e-7 2.247625650036078e-7

Outlying measurements have slight (1.2497997115846819e-2%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8112654483517154e-4 1.811377614755824e-4 1.8115071627119932e-4
Standard deviation 3.2569087723018645e-8 4.049611338535633e-8 5.6269029365643614e-8

Outlying measurements have no (9.433106575963702e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.180219297888376e-3 1.1811758201898004e-3 1.182023229809276e-3
Standard deviation 2.399053657311654e-6 2.961573335971435e-6 3.515611916946113e-6

Outlying measurements have slight (1.4702606371129428e-2%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.4274717669539683e-3 3.4292498974775156e-3 3.431689746083928e-3
Standard deviation 4.908269002337966e-6 6.743784210920239e-6 9.525074927614487e-6

Outlying measurements have slight (2.0399305555555546e-2%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.345992398479883e-2 1.3463418316743732e-2 1.346650472446629e-2
Standard deviation 6.145845876924239e-6 8.136416223368085e-6 1.1526928278230706e-5

Outlying measurements have slight (3.698224852070986e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6263447235447326e-3 1.626645041708454e-3 1.6273441556666735e-3
Standard deviation 6.767390956517569e-7 1.4998791881261344e-6 2.7799252208028605e-6

Outlying measurements have slight (1.6124697661918656e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6738785786272728e-3 1.6741446828618313e-3 1.6744343997189914e-3
Standard deviation 6.337661574807985e-7 9.57914753696589e-7 1.4375572011064374e-6

Outlying measurements have slight (1.612469766191867e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.183041137200511e-3 2.1835075996438627e-3 2.184178942810925e-3
Standard deviation 1.3494292053777067e-6 1.905171973854273e-6 2.882305422603209e-6

Outlying measurements have slight (1.7538265306122295e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.4072982480577557e-3 2.407711408953955e-3 2.4082319520334597e-3
Standard deviation 1.1299652528381399e-6 1.4737237472919676e-6 2.2714616975018044e-6

Outlying measurements have slight (1.8175582990397753e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.833145856415537e-3 7.834836410644948e-3 7.83753764368557e-3
Standard deviation 4.162845406784194e-6 6.475020235199359e-6 1.0369345058512056e-5

Outlying measurements have slight (2.8546712802768055e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.715663619936172e-3 8.716943263996729e-3 8.718462160342048e-3
Standard deviation 3.1965389831137047e-6 4.053034127698587e-6 5.454948622038106e-6

Outlying measurements have slight (3.02734375e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.5727840205980266e-3 3.57370170938355e-3 3.574703198598677e-3
Standard deviation 2.323114539904125e-6 2.9855264839360846e-6 4.257701224321091e-6

Outlying measurements have slight (2.082390221819813e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.9735229063932267e-3 1.974398965218041e-3 1.9769623658940053e-3
Standard deviation 1.8241465060781432e-6 4.682797093650898e-6 9.06122397969522e-6

Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1072799330538027e-3 2.107512060330707e-3 2.107740273174009e-3
Standard deviation 6.013831493455535e-7 7.602571850678668e-7 9.71447868433487e-7

Outlying measurements have slight (1.7538265306122448e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 4.4703347673155916e-3 4.471458589679281e-3 4.472581657016265e-3
Standard deviation 2.8548287234460216e-6 3.5584389960381064e-6 4.7520948024812996e-6

Outlying measurements have slight (2.271498107084911e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.07972241890516e-3 8.087312801160344e-3 8.097501139332677e-3
Standard deviation 1.8852823360199703e-5 2.5163927507080477e-5 3.3426165795874874e-5

Outlying measurements have slight (2.938475665748393e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.407689643070726e-3 8.40868808615223e-3 8.40980789284093e-3
Standard deviation 2.4968105333668423e-6 3.0207778755796944e-6 4.158030032443996e-6

Outlying measurements have slight (2.938475665748387e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.684477831990638e-3 9.68565790406188e-3 9.687801553428224e-3
Standard deviation 2.686690833137668e-6 3.979919172945676e-6 6.267001602502878e-6

Outlying measurements have slight (3.1217481789802066e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.053628608192978e-3 3.0537886384262654e-3 3.0539868124331133e-3
Standard deviation 4.639818330885839e-7 6.12186567860757e-7 8.032629504280269e-7

Outlying measurements have slight (1.9599999999999885e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.077074270049753e-3 2.07780896517155e-3 2.0783592312242096e-3
Standard deviation 1.7492015268540709e-6 2.243942336778958e-6 3.0735059816964964e-6

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0662185623535204e-2 1.0663944410453534e-2 1.0665662121510945e-2
Standard deviation 3.618261629000133e-6 4.5440534637129575e-6 6.1590133555100105e-6

Outlying measurements have slight (3.329369797859691e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.167151109252389e-3 9.172107203258126e-3 9.178715075191957e-3
Standard deviation 1.1965084013062493e-5 1.5385634120618966e-5 2.172482561829554e-5

Outlying measurements have slight (3.1217481789802017e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.30892777945231e-3 8.31012496276381e-3 8.311076511085433e-3
Standard deviation 2.348718591453453e-6 3.0971270751338295e-6 4.157974660609815e-6

Outlying measurements have slight (2.938475665748393e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.553959456596546e-2 7.554151402465234e-2 7.554255016656282e-2
Standard deviation 1.1487686995627327e-6 2.430921004592891e-6 3.824837860889446e-6

Outlying measurements have slight (8.264462809917356e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5768389859687072e-2 1.5773725895057912e-2 1.5777333314063616e-2
Standard deviation 6.768839439854426e-6 1.0941519994623677e-5 1.8720031587695113e-5

Outlying measurements have slight (3.993055555555529e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8927674276428236e-3 1.930939937881697e-3 2.1199686149731565e-3
Standard deviation 2.1366589360845364e-6 2.4846110441386124e-4 5.713494681362112e-4

Outlying measurements have severe (0.7828688119209188%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.664374988128649e-2 7.667583474340897e-2 7.672582492570584e-2
Standard deviation 4.641719302352641e-5 6.373653591338158e-5 8.198058133691391e-5

Outlying measurements have slight (9.000000000000001e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1661521343311254e-2 2.168172667486782e-2 2.1702026094503175e-2
Standard deviation 3.873307201380898e-5 4.664165415880615e-5 5.599863179705253e-5

Outlying measurements have slight (4.75e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.778304639894896e-3 7.788923688839331e-3 7.8062040252015975e-3
Standard deviation 2.4884879990756348e-5 3.703941534927384e-5 6.343481053787911e-5

Outlying measurements have slight (2.8546712802768066e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.