Bootstrap thresholds

The bootstrap provides a powerful, general procedure for estimating the accuracy of a statistical estimate derived from a set of experimental data. The illustration belows shows how bootstrap standard deviations are calculated from a sample psychometric function.

(a) The continuous curve is a model psychometric function from which a sample set of data (open symbols) has been generated based on 10 trials at each x level. These data are intended to be typical of those obtained in practice; the model function thus represents the unknown, 'true' psychometric function. The broken line shows the best (maximum likelihood) estimate of the true function derived from the sample set, along with the resulting estimate of the midpoint m and slope g of the function. (b) and (c) Individual bootstrap replicates generated from the estimated psychometric function fitted to the sample set in (a). Notice the different values of midpoint and slope in the two replicates. (d) and (e) Histograms of the values of midpoint and slope obtained from 100 bootstrap replicates like those in (b) and (c). For comparison, the smooth curves are normal distributions with the same means and standard deviations as the bootstrap histograms. Their goodness of fit is shown by the chi-squared values. For further details see Foster, D. H. & Bischof, W. F. (1991) Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators, Psychological Bulletin, 109, 152-159.

David Foster and Walter Bischof have developed a portable version of this bootstrap program called bootprob. This program fits a cumulative Gaussian psychometric function to a set of binomial data and saves the fitted curve in a separate file called fitted_curve.txt.  It gives the threshold (for a given criterion level of performance), the gradient (slope), and 1/gradient (spread) for the fitted curve.  It then estimates the standard deviations (SDs) and confidence intervals (CIs) of the threshold, slope, and spread by a bootstrap procedure (the 'parametric' bootstrap).

The bootstrap procedure is similar to that described in Foster and Bischof (1991), but a more robust procedure has been adopted here in that SDs are computed from centiles, assuming an approximately normal distribution. When that assumption is violated, CIs should be used anyway.  For an introduction to the software, see Foster, D.H. & Bischof, W.F. (1997) Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions. Spatial Vision, 11, 135-139.

For more background to the bootstrap procedure, including comparison with classical asymptotic estimates, see bootunix.tar, >Foster, D. H. & Bischof, W. F. (1991) Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators, Psychological Bulletin, 109, 152-159.

The program bootprob is available in several versions:

• boot.txt, a plain text file (with source code, readme, notice, and test files)
  
• bootpc.zip, a PC version in zip format (containing a Windows executable program, and source code, readme, notice, and test files)
• bootunix.tar, a Unix version (in tar format, containing source code, readme, notice, and test files)

Another portable bootstrap program is available called bootprobdiff for comparing thresholds and other properties of two psychometric functions.

This program fits cumulative Gaussian psychometric functions to two sets of binomial data and saves the fitted curves in separate files. It gives the threshold (for a given criterion level of performance), the slope, and spread for each of the fitted curves, and the differences in threshold, slope, and spread. It then estimates the standard deviations (SDs) and confidence intervals (CIs) of these differences by the bootstrap procedure. The confidence intervals can be used to decide whether the two thresholds (or two slopes or two spreads) are significantly different from each other, as explained in the readme file. 

The program bootprobdiff is available in several versions:          

• bootdiff.txt, a plain text file (with source code, readme, notice, and test files)
• bootdiffpc.zip, a PC version in zip format (containing a Windows executable program, and source code, readme, notice, and test files)
• bootdiffunix.tar, a Unix version (in tar format, containing source code, readme, notice, and test files)