Bootstrap Thresholds


1. Basics

The bootstrap provides a powerful, general procedure for estimating the accuracy of a statistical estimate derived from a set of experimental data. The illustration below shows how bootstrap standard deviations are calculated from a sample stimulus-response curve or psychometric function.

bootstrap.gif (34495 bytes)

Panel (a) shows a model psychometric function (continuous curve) used to generate a sample set of data (open symbols) each representing the percentage of correct responses in 10 trials at each x level. These data are intended to be typical of those obtained in practice; the model function thus represents the unknown, 'true' psychometric function. The broken line shows the best (maximum likelihood) estimate of the true function derived from the sample set, along with the resulting estimate of the midpoint m and slope g of the function.

Panels (b) and (c) show individual bootstrap replicates generated from the estimated psychometric function fitted to the sample set in (a). Notice the different values of midpoint and slope in the two replicates.

Panels (d) and (e) show histograms of the values of midpoint and slope obtained from 100 bootstrap replicates like those in (b) and (c). For comparison, the smooth curves are normal distributions with the same means and standard deviations as the bootstrap histograms. Their goodness of fit is shown by the chi-squared values.

For further details see Foster, D. H. & Bischof, W. F. (1991) Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators, Psychological Bulletin, 109, 152-159.


2. Portable software for threshold estimates

David Foster and Walter Bischof have developed a portable version of this bootstrap threshold program called bootprob. This program fits a Gaussian cumulative distribution function to a set of binomial data and saves the fitted curve in a separate file called fitted_curve.txt.  It gives the threshold (for a given criterion level of performance), the gradient (slope), and 1/gradient (spread) for the fitted curve.  It then estimates the standard deviations (SDs) and confidence intervals (CIs) of the threshold, slope, and spread by a bootstrap procedure (the 'parametric' bootstrap).

The bootstrap procedure is similar to that described in Foster and Bischof (1991), but a more robust procedure has been adopted here in that SDs are computed from centiles, assuming an approximately normal distribution. When that assumption is violated, CIs should be used anyway.  For an introduction to the software, see Foster, D.H. & Bischof, W.F. (1997) Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions. Spatial Vision, 11, 135-139.

The program bootprob is available in several versions:

  1. boot.txt, a plain text file (with source code, readme, notice, and test files)
  2. bootpc.zip, a PC version in zip format (containing a Windows executable program, and source code, readme, notice, and test files)
  3. bootunix.tar, a Unix version (in tar format, containing source code, readme, notice, and test files)

If you use these routines, would you please cite this source publication: Foster, D.H. & Bischof, W.F. (1997) Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions. Spatial Vision, 11, 135-139.


3. Portable software for threshold difference estimates

Another portable bootstrap program is available called bootprobdiff for comparing thresholds and other properties of two psychometric functions.

This program fits Gaussian cumulative distribution functions to two sets of binomial data and saves the fitted curves in separate files. It gives the threshold (for a given criterion level of performance), the slope, and spread for each of the fitted curves, and the differences in threshold, slope, and spread. It then estimates the standard deviations (SDs) and confidence intervals (CIs) of these differences by the bootstrap procedure. The confidence intervals can be used to decide whether the two thresholds (or two slopes or two spreads) are significantly different from each other, as explained in the readme file.

The program bootprobdiff is available in several versions:

  1. bootdiff.txt, a plain text file (with source code, readme, notice, and test files)
  2. bootdiffpc.zip, a PC version in zip format (containing a Windows executable program, and source code, readme, notice, and test files)
  3. bootdiffunix.tar, a Unix version (in tar format, containing source code, readme, notice, and test files)

If you use these routines, would you please cite this source publication: Foster, D.H. & Bischof, W.F. (1997) Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions. Spatial Vision, 11, 135-139.


Mirrored at https://webdocs.cs.ualberta.ca/~wfb/bootstrap.html


4. Model-free psychometric functions

Software for fitting a stimulus-response curve and estimating a threshold and standard deviation without assuming a Gaussian cumulative distribution function, is available here. One package is for MATLAB (The Mathworks, Inc., Natick, MA, http://www.mathworks.co.uk/index.html) and the other is for R (http://www.r-project.org/),


(c) D. H. Foster and W. F. Bischof, 2016


d.h.foster@manchester.ac.uk | +44 (0)161 306 3888 | www.eee.manchester.ac.uk/d.h.foster 
School of Electrical and Electronic Engineering, University of Manchester, Manchester M13 9PL, UK