QUEST allows rapid estimation of psychophysical thresholds and has become a cornerstone tool in modern behavioral testing (Watson AB, Pelli DG, 1983). However, there is widespread variability in how QUEST is implemented. Consequently, choosing from the vast number of possible strategies and combinations of adjustable parameters can be daunting when implementing QUEST. To address this, we created a generalized framework for optimizing and informing QUEST implementations. Central to our framework is a system for generating automated responses. In this system, a specifiable psychometric function controls simulated response accuracy across a range of psychophysical parameter values during implementations of QUEST sessions on users’ actual visual stimuli. Because the psychometric function is known, thresholds and functions estimated via QUEST during these simulations can then be compared against it as a veridical “ground truth.” This enables quantitative comparison of the accuracy and reliability of QUEST implementations across a range of parameter values of interest including QUEST starting parameters, target thresholds, trials per session, and number of interleaved QUESTs. The psychometric function is also manipulable, allowing one to test across a range of plausible response curves. We recently used this system to inform the number of trials used for threshold estimation on a two-alternative forced-choice experiment. In these simulations, we ran 3 interleaved QUESTs (65, 70, and 85% accuracy threshold targets) with either 20, 30, 40, or 50 trials/QUEST. We iterated each configuration 20x and repeated with 3 different simulated Naka Rushton sigmoidal psychometric curves (Naka KI, Rushton WA, 1966). Each time, data from the interleaved QUESTs were pooled, binned, and a separate Naka Rushton function was estimated to fit the computed accuracy values. The simulated data showed up to a 51% decrease in variability with increasing trials. However, it also suggested this benefit wains considerably as the psychometric function’s slope gets shallower.
Acknowledgements: This work was supported by a grant from the National Science Foundation (SBE 2122866 to A.G.)