criterion performance measurements
overview
want to understand this report?
mapM_ yield, pure sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.400875059074314e-5 | 4.401410435388406e-5 | 4.402237434838216e-5 |
| Standard deviation | 1.5565632782045e-8 | 2.1594584496708315e-8 | 3.128247738377716e-8 |
Outlying measurements have no (7.40699487636442e-3%) effect on estimated standard deviation.
mapM_ yield, pure sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3688123368138356e-4 | 1.3695519972886357e-4 | 1.3707087822500502e-4 |
| Standard deviation | 2.1941598877806215e-7 | 3.0465888585565676e-7 | 4.6529089684500936e-7 |
Outlying measurements have no (8.927846765684526e-3%) effect on estimated standard deviation.
mapM_ yield, pure sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.092458973315044e-5 | 2.092691568219637e-5 | 2.0929892755835145e-5 |
| Standard deviation | 6.817732185490189e-9 | 8.688493690577864e-9 | 1.2809544104504889e-8 |
Outlying measurements have no (6.66636637989237e-3%) effect on estimated standard deviation.
mapM_ yield, pure sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.881498646271264e-5 | 7.882794449351718e-5 | 7.884334548037175e-5 |
| Standard deviation | 3.678263981921684e-8 | 4.7143593635012985e-8 | 6.066604053576739e-8 |
Outlying measurements have no (8.129535071217416e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.998675700543882e-5 | 3.999228032443045e-5 | 3.999748875976593e-5 |
| Standard deviation | 1.596694742236384e-8 | 1.8914265131887194e-8 | 2.3823460203790895e-8 |
Outlying measurements have no (7.2988754325258726e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.6180679491582603e-4 | 3.6191687556420853e-4 | 3.6209157044558263e-4 |
| Standard deviation | 3.243331982305874e-7 | 4.5228131150133843e-7 | 6.673143738675141e-7 |
Outlying measurements have slight (1.0868252626494386e-2%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.726010749331589e-5 | 1.7262085414719766e-5 | 1.7264068449812644e-5 |
| Standard deviation | 5.6934243140183304e-9 | 6.977369803913803e-9 | 8.462561492470274e-9 |
Outlying measurements have no (6.493229099918812e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3030240522125897e-4 | 1.3033339687656904e-4 | 1.3041507588872783e-4 |
| Standard deviation | 7.212844885831337e-8 | 1.52637872404749e-7 | 2.778569486038193e-7 |
Outlying measurements have no (8.848852040816327e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.49227225944977e-5 | 8.493398703163033e-5 | 8.495190878022553e-5 |
| Standard deviation | 3.632807806574497e-8 | 4.8010358430589225e-8 | 6.756997726390734e-8 |
Outlying measurements have no (8.263888888888652e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.3000990547139126e-4 | 3.302183062931343e-4 | 3.3051292472710265e-4 |
| Standard deviation | 7.248346044469621e-7 | 8.748896448095989e-7 | 1.1731843167243998e-6 |
Outlying measurements have slight (1.0637067869117818e-2%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.221474678743142e-5 | 4.222176421802219e-5 | 4.222986269841479e-5 |
| Standard deviation | 2.330580404748021e-8 | 2.5897673791358662e-8 | 2.9423379430761153e-8 |
Outlying measurements have no (7.352537722907762e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.4492211332952026e-4 | 2.451002700249174e-4 | 2.4539036158384626e-4 |
| Standard deviation | 4.857722975869838e-7 | 7.135541031227803e-7 | 1.0994901037164895e-6 |
Outlying measurements have no (9.998979695949392e-3%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.227641000433038e-5 | 9.228689016359127e-5 | 9.229655492174828e-5 |
| Standard deviation | 2.8067847506427372e-8 | 3.415138473996696e-8 | 4.406972707172734e-8 |
Outlying measurements have no (8.332744862650654e-3%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.993721804089985e-4 | 6.995655606694611e-4 | 6.999573061108173e-4 |
| Standard deviation | 4.760360381592106e-7 | 8.830034757063757e-7 | 1.6501474894266544e-6 |
Outlying measurements have slight (1.2656147271531887e-2%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.655920771924677e-5 | 3.656955226976974e-5 | 3.6582941795292086e-5 |
| Standard deviation | 3.189442450497552e-8 | 3.8544154083797506e-8 | 5.364674027744572e-8 |
Outlying measurements have no (7.193866834698374e-3%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.0336873273968954e-4 | 4.03560880524151e-4 | 4.0400263641901876e-4 |
| Standard deviation | 5.239194375720462e-7 | 9.799916137069279e-7 | 1.7566318847342245e-6 |
Outlying measurements have slight (1.1109708370155102e-2%) effect on estimated standard deviation.
enumFromTo, pure sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.006123657103667e-5 | 5.006677368982412e-5 | 5.007214114461833e-5 |
| Standard deviation | 1.5776459585283523e-8 | 1.8201498062765646e-8 | 2.1555602433051127e-8 |
Outlying measurements have no (7.575316123769012e-3%) effect on estimated standard deviation.
enumFromTo, pure sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4307452160745065e-4 | 1.4314698661917026e-4 | 1.4326579859545572e-4 |
| Standard deviation | 2.2216832846374912e-7 | 3.349160030875882e-7 | 5.140177197597963e-7 |
Outlying measurements have no (9.008264462809916e-3%) effect on estimated standard deviation.
enumFromTo, pure sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.855503159626786e-5 | 3.855666328884397e-5 | 3.8558969133814634e-5 |
| Standard deviation | 5.3010585306135726e-9 | 7.045711541250786e-9 | 1.051911137967118e-8 |
Outlying measurements have no (7.298875432525902e-3%) effect on estimated standard deviation.
enumFromTo, pure sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.542259731044736e-5 | 9.544725156480883e-5 | 9.54772049636822e-5 |
| Standard deviation | 6.911741196209278e-8 | 9.659194008392487e-8 | 1.4852051791545683e-7 |
Outlying measurements have no (8.402757828209994e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.9716916254500424e-5 | 3.972203939897696e-5 | 3.972819677175226e-5 |
| Standard deviation | 1.724483757921393e-8 | 1.9609833442346977e-8 | 2.2625264762258724e-8 |
Outlying measurements have no (7.298875432525951e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.4656685338275707e-4 | 3.466616136616291e-4 | 3.468608616853056e-4 |
| Standard deviation | 2.0832738163396648e-7 | 4.396041444234597e-7 | 8.598099707126696e-7 |
Outlying measurements have slight (1.0751417769376001e-2%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.7555952589790056e-5 | 1.7558183245506937e-5 | 1.7560334636870458e-5 |
| Standard deviation | 5.895384964657462e-9 | 7.2524944387988535e-9 | 9.025783385303583e-9 |
Outlying measurements have no (6.493229099918745e-3%) effect on estimated standard deviation.
mapM_ yield, monadic sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.4360820439733392e-4 | 1.436459226444844e-4 | 1.4369527684784012e-4 |
| Standard deviation | 1.1315980293747149e-7 | 1.520639788031505e-7 | 2.2942782107360634e-7 |
Outlying measurements have no (9.008264462809916e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.072539949853035e-5 | 9.073621760958772e-5 | 9.075311954919738e-5 |
| Standard deviation | 3.223583485959508e-8 | 4.4626115377389626e-8 | 6.988713838704283e-8 |
Outlying measurements have no (8.33274486265082e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.545824223872964e-4 | 3.549254359499204e-4 | 3.551910779343896e-4 |
| Standard deviation | 7.623521794886255e-7 | 1.0256699215383634e-6 | 1.4550719982527183e-6 |
Outlying measurements have slight (1.0868252626494372e-2%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.888766418679812e-5 | 5.889562313725415e-5 | 5.891298906743173e-5 |
| Standard deviation | 2.2502435135856112e-8 | 3.7344404253040925e-8 | 6.40059172026597e-8 |
Outlying measurements have no (7.751464843749999e-3%) effect on estimated standard deviation.
mapM_ yield, map, pure sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.750123602534095e-4 | 2.751759701681939e-4 | 2.754767094525218e-4 |
| Standard deviation | 4.5051302747379587e-7 | 7.260032829746587e-7 | 1.318438419510707e-6 |
Outlying measurements have slight (1.0308159722222222e-2%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.919409930876294e-5 | 8.921943061849822e-5 | 8.92627280496477e-5 |
| Standard deviation | 6.764873112532912e-8 | 1.0957614704766407e-7 | 1.6932079899564003e-7 |
Outlying measurements have no (8.332744862650782e-3%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.97932416302825e-4 | 6.980944235989052e-4 | 6.98747132259926e-4 |
| Standard deviation | 2.3376177336674218e-7 | 8.413127782492702e-7 | 1.6654031938782148e-6 |
Outlying measurements have slight (1.2656147271531887e-2%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.7140379906103314e-5 | 3.7144933619757854e-5 | 3.714967594858966e-5 |
| Standard deviation | 1.3424533907644068e-8 | 1.5755595846174405e-8 | 1.874318391039073e-8 |
Outlying measurements have no (7.245990729394213e-3%) effect on estimated standard deviation.
mapM_ yield, map, monadic sum/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.438410551231393e-4 | 4.4405822279434807e-4 | 4.445225111783592e-4 |
| Standard deviation | 6.452185943957408e-7 | 1.0207803230472612e-6 | 1.7122920385559202e-6 |
Outlying measurements have slight (1.1362135024441767e-2%) effect on estimated standard deviation.
mix compositions/Standard
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.611337493157821e-4 | 4.614465872345989e-4 | 4.6176181502847525e-4 |
| Standard deviation | 8.140557553937851e-7 | 1.0026880524215584e-6 | 1.275427077752118e-6 |
Outlying measurements have slight (1.149269875608437e-2%) effect on estimated standard deviation.
mix compositions/Free
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.3737230574684986e-3 | 1.374973328249584e-3 | 1.3764900056773277e-3 |
| Standard deviation | 3.8834733032127455e-6 | 4.568689318016509e-6 | 5.598136107215079e-6 |
Outlying measurements have slight (1.5380859374999993e-2%) effect on estimated standard deviation.
mix compositions/Codensity
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.39121033471213e-4 | 2.3929936924400796e-4 | 2.3952034272272844e-4 |
| Standard deviation | 5.490744067700067e-7 | 6.840893965872789e-7 | 9.737188314120105e-7 |
Outlying measurements have no (9.998979695949253e-3%) effect on estimated standard deviation.
mix compositions/Church
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 1.0264099777096442e-3 | 1.0279757188137993e-3 | 1.0296397280403873e-3 |
| Standard deviation | 4.283186126496901e-6 | 5.257085012670669e-6 | 6.837331683820626e-6 |
Outlying measurements have slight (1.4081632653061223e-2%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.