criterion performance measurements
overview
want to understand this report?
sum/Data.List.foldl'
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.125316176138798e-5 | 8.129950172508953e-5 | 8.137666413766647e-5 |
| Standard deviation | 1.4950914624388357e-7 | 1.983194924969662e-7 | 2.7163417347049075e-7 |
Outlying measurements have no (8.196161464380848e-3%) effect on estimated standard deviation.
sum/Control.Monad.foldM
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.970346379198125e-4 | 5.977796388793879e-4 | 5.990180558626078e-4 |
| Standard deviation | 2.180148198638494e-6 | 3.235689596549572e-6 | 5.170808668108526e-6 |
Outlying measurements have slight (1.2193263222069489e-2%) effect on estimated standard deviation.
sum/low level
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.74917050167312e-6 | 5.751115626359779e-6 | 5.756154127680543e-6 |
| Standard deviation | 3.1104170628972967e-9 | 9.48729140960894e-9 | 1.9843895484271095e-8 |
Outlying measurements have no (5.681632653061224e-3%) effect on estimated standard deviation.
sum/boxed vectors, I/O
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.626847257683185e-6 | 8.632236009522514e-6 | 8.640011559949345e-6 |
| Standard deviation | 1.560335427712077e-8 | 2.0842443902262948e-8 | 2.8879826064439297e-8 |
Outlying measurements have no (5.952167521244936e-3%) effect on estimated standard deviation.
sum/boxed vectors
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.760031243369299e-6 | 5.7632922171468895e-6 | 5.768525214922163e-6 |
| Standard deviation | 9.31140679725454e-9 | 1.337981803828594e-8 | 1.9026709301940808e-8 |
Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.
sum/unboxed vectors
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.760911904295693e-6 | 5.764134129512608e-6 | 5.769178710191435e-6 |
| Standard deviation | 9.569107560952662e-9 | 1.3327712203772554e-8 | 1.8456233828659987e-8 |
Outlying measurements have no (5.681632653061225e-3%) effect on estimated standard deviation.
sum/conduit, pure, fold
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.658250544958975e-6 | 8.670044563538928e-6 | 8.684790709797013e-6 |
| Standard deviation | 3.26033243764104e-8 | 4.550681658801341e-8 | 7.303554058221922e-8 |
Outlying measurements have no (5.952167521245102e-3%) effect on estimated standard deviation.
sum/conduit, pure, foldM
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.821563358180641e-5 | 8.830089802709136e-5 | 8.839810116904793e-5 |
| Standard deviation | 2.6817164633854646e-7 | 3.163036119646593e-7 | 3.7799353045138296e-7 |
Outlying measurements have no (8.26388888888888e-3%) effect on estimated standard deviation.
sum/conduit, IO, fold
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.150584978950254e-5 | 1.151665380363164e-5 | 1.1531257197248227e-5 |
| Standard deviation | 3.4034732044646935e-8 | 4.116353829802586e-8 | 5.4301470649938765e-8 |
Outlying measurements have no (6.172601365688123e-3%) effect on estimated standard deviation.
sum/conduit, IO, foldM
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.541814080744619e-5 | 8.548909113057639e-5 | 8.560752438366873e-5 |
| Standard deviation | 2.160914920762318e-7 | 2.923407083554638e-7 | 4.946731517124888e-7 |
Outlying measurements have no (8.263888888888817e-3%) effect on estimated standard deviation.
map + sum/boxed vectors
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.9439598862756175e-6 | 5.947833169398621e-6 | 5.95274655729247e-6 |
| Standard deviation | 1.0592913544256484e-8 | 1.5119260988432043e-8 | 1.9004523054116664e-8 |
Outlying measurements have no (5.681632653061116e-3%) effect on estimated standard deviation.
map + sum/unboxed vectors
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.9441870491081675e-6 | 5.947608576622326e-6 | 5.953498082636309e-6 |
| Standard deviation | 1.0807515959704501e-8 | 1.4522819553884204e-8 | 1.9837412869724338e-8 |
Outlying measurements have no (5.681632653061038e-3%) effect on estimated standard deviation.
map + sum/conduit, connect1
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.346018117738933e-4 | 6.351524807383135e-4 | 6.358271052969135e-4 |
| Standard deviation | 1.6100504599029886e-6 | 2.1071766357934644e-6 | 3.164524625804439e-6 |
Outlying measurements have slight (1.234375e-2%) effect on estimated standard deviation.
map + sum/conduit, connect2
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.835227223314004e-4 | 5.841739742789186e-4 | 5.850700709913633e-4 |
| Standard deviation | 2.0143033599048285e-6 | 2.491538084187158e-6 | 3.414297092489558e-6 |
Outlying measurements have slight (1.219326322206954e-2%) effect on estimated standard deviation.
map + sum/conduit, connect3
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.212660927278225e-4 | 5.216787671221704e-4 | 5.220611986999907e-4 |
| Standard deviation | 1.1635518484684425e-6 | 1.359931877074997e-6 | 1.5851546981021736e-6 |
Outlying measurements have slight (1.1763038548752517e-2%) effect on estimated standard deviation.
map + sum/conduit, inner fuse
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.159679296444914e-4 | 5.162312145165072e-4 | 5.165685971503418e-4 |
| Standard deviation | 8.629287734623248e-7 | 1.0312744661618293e-6 | 1.2363714638740263e-6 |
Outlying measurements have slight (1.1763038548752833e-2%) effect on estimated standard deviation.
monte carlo/conduit
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.337752357755636e-3 | 5.341387399622476e-3 | 5.345641809962247e-3 |
| Standard deviation | 9.163657804482997e-6 | 1.1983688747878939e-5 | 1.7016394863520117e-5 |
Outlying measurements have slight (2.4375e-2%) effect on estimated standard deviation.
monte carlo/low level
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.469690096068386e-3 | 3.472505362842744e-3 | 3.475169388674715e-3 |
| Standard deviation | 7.393091054977912e-6 | 8.81977029467318e-6 | 1.1130287799556675e-5 |
Outlying measurements have slight (2.0823902218198246e-2%) effect on estimated standard deviation.
sliding window/10/low level, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.529804435415924e-3 | 1.5306554769249166e-3 | 1.5318890438274257e-3 |
| Standard deviation | 2.3133951605296183e-6 | 3.4138693812621976e-6 | 4.68910264557361e-6 |
Outlying measurements have slight (1.586888657648283e-2%) effect on estimated standard deviation.
sliding window/10/conduit, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.895266964848574e-3 | 1.8964374719388835e-3 | 1.8981077149174527e-3 |
| Standard deviation | 3.6687718870054974e-6 | 4.955572410650941e-6 | 6.951032819667142e-6 |
Outlying measurements have slight (1.694411414982164e-2%) effect on estimated standard deviation.
sliding window/10/low level, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.7406561519314513e-3 | 1.741939632318957e-3 | 1.7436531568878564e-3 |
| Standard deviation | 3.7630369847545036e-6 | 5.018410470291441e-6 | 7.100229305760256e-6 |
Outlying measurements have slight (1.6388888888888765e-2%) effect on estimated standard deviation.
sliding window/10/conduit, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.3738622252696138e-3 | 3.3772485115569255e-3 | 3.380390894162564e-3 |
| Standard deviation | 9.140074293865328e-6 | 1.1063406082921327e-5 | 1.453425448512702e-5 |
Outlying measurements have slight (2.0399305555555407e-2%) effect on estimated standard deviation.
sliding window/10/low level, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.2536634444670026e-3 | 2.256027479874095e-3 | 2.2592819377073377e-3 |
| Standard deviation | 6.901651230070295e-6 | 9.471391923659867e-6 | 1.3489158752961605e-5 |
Outlying measurements have slight (1.7851239669421485e-2%) effect on estimated standard deviation.
sliding window/10/conduit, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.050057168988636e-3 | 4.053880984223189e-3 | 4.058201754660315e-3 |
| Standard deviation | 1.1508142758778834e-5 | 1.3414450150568548e-5 | 1.6807254330294353e-5 |
Outlying measurements have slight (2.1728395061728332e-2%) effect on estimated standard deviation.
sliding window/30/low level, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.488088791891946e-3 | 3.4912062388519982e-3 | 3.4955318766590983e-3 |
| Standard deviation | 9.75745212306961e-6 | 1.2161178334645656e-5 | 1.839821031998955e-5 |
Outlying measurements have slight (2.0823902218198086e-2%) effect on estimated standard deviation.
sliding window/30/conduit, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.8650690706926692e-3 | 3.869798311763203e-3 | 3.877608202355911e-3 |
| Standard deviation | 1.292593158609748e-5 | 1.8341622777346755e-5 | 3.207563776555454e-5 |
Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.
sliding window/30/low level, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.051900639890731e-3 | 2.054345243358931e-3 | 2.05787126196837e-3 |
| Standard deviation | 8.049945736456613e-6 | 9.912556814281058e-6 | 1.3357516982010825e-5 |
Outlying measurements have slight (1.7236072637734686e-2%) effect on estimated standard deviation.
sliding window/30/conduit, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.627659341255608e-3 | 3.630339017537377e-3 | 3.6336355764432214e-3 |
| Standard deviation | 7.85272573375388e-6 | 9.436273945621291e-6 | 1.1644265646039392e-5 |
Outlying measurements have slight (2.1266540642722116e-2%) effect on estimated standard deviation.
sliding window/30/low level, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1608228986743517e-3 | 2.162461153573839e-3 | 2.1647970162875737e-3 |
| Standard deviation | 5.17800693066317e-6 | 6.656050077728642e-6 | 8.713346527463114e-6 |
Outlying measurements have slight (1.753826530612239e-2%) effect on estimated standard deviation.
sliding window/30/conduit, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.0094770227215165e-3 | 4.012490221826493e-3 | 4.015862310766827e-3 |
| Standard deviation | 9.50160469723356e-6 | 1.0582747216048797e-5 | 1.2344263211285676e-5 |
Outlying measurements have slight (2.1728395061728394e-2%) effect on estimated standard deviation.
sliding window/100/low level, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.344503452173528e-3 | 9.353007797115101e-3 | 9.363459198792765e-3 |
| Standard deviation | 2.1279327809914752e-5 | 2.5585780586932992e-5 | 3.153678238688347e-5 |
Outlying measurements have slight (3.1217481789802288e-2%) effect on estimated standard deviation.
sliding window/100/conduit, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.824938809433224e-3 | 9.836556157354745e-3 | 9.850644915101724e-3 |
| Standard deviation | 2.779953075755451e-5 | 3.341004018329876e-5 | 4.244120695122805e-5 |
Outlying measurements have slight (3.222222222222212e-2%) effect on estimated standard deviation.
sliding window/100/low level, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.146702076942491e-3 | 3.149892052009288e-3 | 3.154463973700717e-3 |
| Standard deviation | 1.034315997146129e-5 | 1.2636528799015154e-5 | 1.598267882766762e-5 |
Outlying measurements have slight (1.9991670137442685e-2%) effect on estimated standard deviation.
sliding window/100/conduit, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.738346828025309e-3 | 4.743465813639795e-3 | 4.751041220133328e-3 |
| Standard deviation | 1.3266788626527203e-5 | 1.7828675788135046e-5 | 2.7296740862656733e-5 |
Outlying measurements have slight (2.3242630385487528e-2%) effect on estimated standard deviation.
sliding window/100/low level, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.1255024943930513e-3 | 2.12770583295492e-3 | 2.129817302294408e-3 |
| Standard deviation | 5.795958353823756e-6 | 6.913033971040142e-6 | 8.717893646257134e-6 |
Outlying measurements have slight (1.753826530612224e-2%) effect on estimated standard deviation.
sliding window/100/conduit, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.8082676123562423e-3 | 3.8117676248770184e-3 | 3.819182834511613e-3 |
| Standard deviation | 8.808481281834631e-6 | 1.5216637048430866e-5 | 2.6463279608461232e-5 |
Outlying measurements have slight (2.126654064272189e-2%) effect on estimated standard deviation.
sliding window/1000/low level, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.244176556205949e-2 | 7.249819651903675e-2 | 7.253895809743349e-2 |
| Standard deviation | 4.931905984781488e-5 | 7.949844836545032e-5 | 1.1541054270055978e-4 |
Outlying measurements have slight (8.264462809917356e-2%) effect on estimated standard deviation.
sliding window/1000/conduit, Seq
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.308817519517671e-2 | 7.312123432277165e-2 | 7.315699327812208e-2 |
| Standard deviation | 4.076294829648093e-5 | 5.7990582204541764e-5 | 9.109413807601336e-5 |
Outlying measurements have slight (8.264462809917353e-2%) effect on estimated standard deviation.
sliding window/1000/low level, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.5869719157128963e-2 | 1.5883937261823185e-2 | 1.5919733583385665e-2 |
| Standard deviation | 2.1915650169220274e-5 | 5.342285348505935e-5 | 1.0717886418015289e-4 |
Outlying measurements have slight (3.9930555555555386e-2%) effect on estimated standard deviation.
sliding window/1000/conduit, boxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.7573684913960205e-2 | 1.759278527365787e-2 | 1.7613066047824796e-2 |
| Standard deviation | 3.6780585940390876e-5 | 4.520613930989e-5 | 6.063317070350517e-5 |
Outlying measurements have slight (4.158790170132321e-2%) effect on estimated standard deviation.
sliding window/1000/low level, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.9110541278629114e-3 | 1.9511527546967613e-3 | 2.149945330213388e-3 |
| Standard deviation | 4.5823728945283805e-6 | 2.600572915024288e-4 | 5.975796726098554e-4 |
Outlying measurements have severe (0.800386733302099%) effect on estimated standard deviation.
sliding window/1000/conduit, unboxed Vector
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.5540277941793442e-3 | 3.562887659406095e-3 | 3.5715242014596484e-3 |
| Standard deviation | 2.318402848927872e-5 | 2.7655880612574962e-5 | 3.475554340462335e-5 |
Outlying measurements have slight (2.082390221819827e-2%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.