criterion performance measurements

overview

want to understand this report?

sum/Data.List.foldl'

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.067303208425206e-5 8.07198463185624e-5 8.08050211294153e-5
Standard deviation 1.3741523335500288e-7 1.993119473968741e-7 2.8839446754337276e-7

Outlying measurements have no (8.19616146438081e-3%) effect on estimated standard deviation.

sum/Control.Monad.foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.011624034055862e-4 6.031708996852516e-4 6.049039686555583e-4
Standard deviation 6.252635968113406e-6 6.586160131626698e-6 7.105309310976541e-6

Outlying measurements have slight (1.2193263222069874e-2%) effect on estimated standard deviation.

sum/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.757158740711335e-6 5.758173538378857e-6 5.759522634893695e-6
Standard deviation 3.0238294718937004e-9 3.929781740737584e-9 4.72111496397018e-9

Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.

sum/boxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.771454572836373e-6 5.775174134617033e-6 5.781837127867366e-6
Standard deviation 7.753079241314347e-9 1.6378391127046583e-8 2.6329095276813736e-8

Outlying measurements have no (5.681632653061076e-3%) effect on estimated standard deviation.

sum/unboxed vectors

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.766356230513231e-6 5.766778256344031e-6 5.76734072722361e-6
Standard deviation 1.2997998422312048e-9 1.6143520094956084e-9 2.2619525392647665e-9

Outlying measurements have no (5.681632653061177e-3%) effect on estimated standard deviation.

sum/conduit, pure, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.62459851076306e-6 8.625796494570802e-6 8.627537790435071e-6
Standard deviation 3.564970051601737e-9 4.9405916952920725e-9 6.230131748053822e-9

Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.

sum/conduit, pure, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.821371144386792e-5 8.822673235381544e-5 8.824615506803203e-5
Standard deviation 3.8570299084823315e-8 5.1818351959837755e-8 7.415505566028389e-8

Outlying measurements have no (8.26388888888865e-3%) effect on estimated standard deviation.

sum/conduit, IO, fold

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.1477972328638122e-5 1.1478934435329431e-5 1.1481367623826323e-5
Standard deviation 1.753773708530283e-9 4.798642971445351e-9 8.43961750438064e-9

Outlying measurements have no (6.172601365687805e-3%) effect on estimated standard deviation.

sum/conduit, IO, foldM

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.111347742752568e-5 8.113185444448489e-5 8.11538149541583e-5
Standard deviation 6.082237398947374e-8 6.974800400973953e-8 7.905170082731158e-8

Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.

monte carlo/low level

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.423556355966472e-3 3.42474788588011e-3 3.4260789705983013e-3
Standard deviation 3.3855170696501694e-6 4.278017349977473e-6 6.098110547192784e-6

Outlying measurements have slight (2.0399305555555254e-2%) effect on estimated standard deviation.

monte carlo/conduit

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0633766583864291e-2 1.0636014074026142e-2 1.0638635787666472e-2
Standard deviation 5.295440342475829e-6 6.740976289896081e-6 8.337337210665644e-6

Outlying measurements have slight (3.329369797859686e-2%) effect on estimated standard deviation.

sliding window/10/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.433755546658281e-3 1.4339300840904015e-3 1.4342158138119234e-3
Standard deviation 5.404100456766932e-7 7.470791046397037e-7 1.2639584188300813e-6

Outlying measurements have slight (1.5380859374999998e-2%) effect on estimated standard deviation.

sliding window/10/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7647708959538603e-3 1.7649582202758436e-3 1.7652004910999463e-3
Standard deviation 5.307439889567579e-7 7.264711147800921e-7 1.0122487370753399e-6

Outlying measurements have slight (1.638888888888889e-2%) effect on estimated standard deviation.

sliding window/10/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.307697060735648e-3 2.308006579834558e-3 2.3083765254495233e-3
Standard deviation 9.130993974824338e-7 1.1013327346373966e-6 1.3573300895270382e-6

Outlying measurements have slight (1.7851239669421485e-2%) effect on estimated standard deviation.

sliding window/10/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.7909291128899653e-3 1.791186780587607e-3 1.7915446670484303e-3
Standard deviation 8.052321296656053e-7 1.0199337342563257e-6 1.3248693509748036e-6

Outlying measurements have slight (1.666187877046796e-2%) effect on estimated standard deviation.

sliding window/10/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.492638633722915e-3 7.497309170257165e-3 7.5018286917592596e-3
Standard deviation 8.958303247166203e-6 1.2978583831175979e-5 1.9269404459665193e-5

Outlying measurements have slight (2.8546712802768163e-2%) effect on estimated standard deviation.

sliding window/10/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.725029114238423e-3 7.725981873724272e-3 7.727161865733911e-3
Standard deviation 2.328863803851482e-6 2.9470742696865243e-6 3.843473786994938e-6

Outlying measurements have slight (2.8546712802768163e-2%) effect on estimated standard deviation.

sliding window/30/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.357581256788434e-3 3.3582667053036848e-3 3.358927718743588e-3
Standard deviation 1.892226914464939e-6 2.2842022373262424e-6 2.721080456445116e-6

Outlying measurements have slight (2.039930555555551e-2%) effect on estimated standard deviation.

sliding window/30/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.0643995919089196e-3 2.064562874339967e-3 2.064741202968844e-3
Standard deviation 4.3560881646484476e-7 5.547882633920739e-7 7.177021379008462e-7

Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.

sliding window/30/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.229676347674519e-3 2.229800648755914e-3 2.229923388731439e-3
Standard deviation 3.3282526063210793e-7 4.0800343337655994e-7 5.046786180861538e-7

Outlying measurements have slight (1.7851239669421485e-2%) effect on estimated standard deviation.

sliding window/30/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.957798654771626e-3 3.958506472645054e-3 3.959462085920789e-3
Standard deviation 2.1117310309652683e-6 2.6409404694030396e-6 3.3904349321111623e-6

Outlying measurements have slight (2.1728395061728398e-2%) effect on estimated standard deviation.

sliding window/30/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.5065420331595235e-3 7.528469387942153e-3 7.566212917795683e-3
Standard deviation 5.159849193849123e-5 8.00279617108593e-5 1.0804689725973929e-4

Outlying measurements have slight (2.8546712802768163e-2%) effect on estimated standard deviation.

sliding window/30/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.4405893728543655e-3 7.441502143418648e-3 7.442474401818854e-3
Standard deviation 2.1144367911220585e-6 2.770153660832705e-6 3.696140396227849e-6

Outlying measurements have slight (2.775510204081624e-2%) effect on estimated standard deviation.

sliding window/100/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 9.593669889036474e-3 9.594700869461029e-3 9.595763102057226e-3
Standard deviation 2.4618938219140506e-6 2.9134866260795166e-6 3.425138397203845e-6

Outlying measurements have slight (3.121748178980215e-2%) effect on estimated standard deviation.

sliding window/100/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.141508681383835e-3 3.1416328320063103e-3 3.141793088232728e-3
Standard deviation 3.947490112035477e-7 4.7484400957515567e-7 5.99146987060793e-7

Outlying measurements have slight (1.999167013744262e-2%) effect on estimated standard deviation.

sliding window/100/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.17952784524866e-3 2.1800626044716116e-3 2.180606331858677e-3
Standard deviation 1.4633739568878131e-6 1.8421181070354314e-6 2.4506437190570987e-6

Outlying measurements have slight (1.753826530612243e-2%) effect on estimated standard deviation.

sliding window/100/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.0187372886290342e-2 1.0190313993952033e-2 1.019334422283646e-2
Standard deviation 6.827903764299878e-6 7.872081427667579e-6 9.109704225905723e-6

Outlying measurements have slight (3.222222222222223e-2%) effect on estimated standard deviation.

sliding window/100/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 8.51763968662028e-3 8.524843434786959e-3 8.531044038348415e-3
Standard deviation 1.5288887251111184e-5 1.9548750025840968e-5 2.6329269504838102e-5

Outlying measurements have slight (2.9384756657483767e-2%) effect on estimated standard deviation.

sliding window/100/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.350768714445918e-3 7.352400156351931e-3 7.353998280948293e-3
Standard deviation 3.866234922000348e-6 4.795074542533123e-6 6.439970525842065e-6

Outlying measurements have slight (2.7755102040816212e-2%) effect on estimated standard deviation.

sliding window/1000/low level, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.552938935706177e-2 7.553355166585181e-2 7.554137867065626e-2
Standard deviation 3.9721385783036414e-6 9.390559833373168e-6 1.5396647604988064e-5

Outlying measurements have slight (8.264462809917332e-2%) effect on estimated standard deviation.

sliding window/1000/low level, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.5866343710220197e-2 1.5872063299647e-2 1.5874626687573914e-2
Standard deviation 4.714874326436047e-6 8.768389303695783e-6 1.658745027992085e-5

Outlying measurements have slight (3.9930555555555476e-2%) effect on estimated standard deviation.

sliding window/1000/low level, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.8934266901516161e-3 1.934962532754441e-3 2.0997918294427074e-3
Standard deviation 3.364332245320802e-6 2.7001672239327837e-4 5.742529961738408e-4

Outlying measurements have severe (0.8181111392845363%) effect on estimated standard deviation.

sliding window/1000/conduit, Seq

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 7.627237162916116e-2 7.627623603909162e-2 7.627925627369173e-2
Standard deviation 4.182556273872257e-6 5.751377002891961e-6 7.256836411057105e-6

Outlying measurements have slight (8.999999999999994e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, boxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.1005194166719608e-2 2.1028253514759104e-2 2.1050742105979972e-2
Standard deviation 4.4705933127223344e-5 5.5149306958155336e-5 7.005159116694605e-5

Outlying measurements have slight (4.5351473922902376e-2%) effect on estimated standard deviation.

sliding window/1000/conduit, unboxed Vector

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 6.806760532508497e-3 6.810780296934259e-3 6.815663726705223e-3
Standard deviation 9.153676997484933e-6 1.2779837626246097e-5 1.7865725200316664e-5

Outlying measurements have slight (2.7006172839506133e-2%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.