ABX Testing Without Fooling Yourself
Published: August 20, 2026 · Read Time: 5 min read · Category: DSP & Audio
Author: Quilio Research (Signal Processing & Computing Group)
A listening test is only useful when its procedure controls expectation, loudness, order and the limits of what the result can support.
The Question Comes First
ABX is not a ritual for proving that one algorithm is better. It answers a narrower question: given two hidden alternatives and a reference, can a listener identify which alternative they heard above chance?
That distinction determines the material, the instructions and the analysis. "Can listeners tell?" is different from "Which do they prefer?" and different again from "Which is more accurate?"
Control the Easy Clues
Match loudness before comparing quality. Randomise trial order. Keep the source, duration and playback path fixed. Give the listener a way to switch quickly and repeat a section without revealing labels.
The test should be comfortable enough that fatigue is not the main variable. If the difference is only audible at one transient, make that transient easy to revisit rather than asking the listener to remember it for a minute.
Count Decisions, Not Impressions
Record each answer and report the number of correct identifications, the number of trials and the uncertainty around the result. A listener getting 7 of 10 correct is evidence of a different strength than getting 70 of 100, even though the proportions match.
A negative result is usefulFailure to distinguish two versions does not prove they are identical. It says that this test, with these listeners and this material, did not reveal a reliable difference.
Pair Listening with Measurement
Measurements can show where to listen; listening can show which measurements matter. For pitch processing, inspect transient timing and formant movement, then test whether those changes are audible. For a reverb, compare decay structure and early reflections, then ask whether the perceptual distinction survives level matching.
The strongest claims are modest, reproducible and clear about what the test did not establish.