Testing Code That Processes Signals: What a Unit Test Can and Cannot Prove

A unit test for a string-formatting function or a validation rule has a clean shape: given this input, expect exactly this output. Signal-processing code — filters, Fourier transforms, resampling, feature extraction — breaks that shape in a few specific ways, and the breaks are worth naming before writing the tests, not after debugging why a passing suite still shipped something wrong.

Exact equality is the wrong assertion

assertEqual(result, expected) works when the output is deterministic down to the bit — an integer count, a boolean flag, a string. A band-pass filter’s output is a sequence of floating-point numbers produced by repeated multiplication and summation, and two mathematically equivalent implementations (different FFT library, different platform, even a different compiler flag) can legitimately disagree in the fifteenth decimal place. Testing signal-processing code means asserting closeness, not equality — assertAlmostEqual, numpy.allclose, or an explicit tolerance — and picking that tolerance is itself a decision that needs a reason, not a default copied from another test file.

# Wrong: will flake on floating-point noise that means nothing
assert filtered_signal[100] == 0.42315

# Right: states the tolerance the test actually cares about
assert abs(filtered_signal[100] - 0.42315) < 1e-6

A test can prove consistency, not correctness

A test that feeds a known sine wave into a band-pass filter and checks the output amplitude dropped outside the pass-band proves the filter behaves like a filter — it doesn’t prove 1-40 Hz was the right pass-band for the task. That’s a domain decision, made by reading about what frequency range the signal of interest lives in, not something a test suite can verify from first principles. The useful tests here check that the mechanism does what it claims (a notch filter at 60 Hz actually suppresses 60 Hz, a resample from 1000 Hz to 250 Hz actually produces a quarter as many samples) — not that the mechanism was pointed at the right target.

Property-based tests catch what example-based tests miss

A handful of hand-picked example inputs tests the cases you thought of. Signal-processing bugs tend to live in the cases you didn’t: a zero-length input, a signal shorter than the filter’s window, all-NaN data from a sensor dropout, a sampling rate that doesn’t evenly divide the window length. A property-based test — generate many random (and edge-case) inputs, assert an invariant that should hold for all of them — covers more of that space than examples alone:

@given(arrays(dtype=np.float64, shape=st.integers(0, 1000)))
def test_filter_never_changes_signal_length(signal):
    result = bandpass_filter(signal, low=1, high=40, fs=160)
    assert len(result) == len(signal)

This doesn’t replace example-based tests — a test asserting “this known input filters to this known output” is still the clearest regression check when the implementation changes — but it catches the shape-and-edge-case bugs that fixed examples structurally can’t.

Golden-file tests, used deliberately

For something complex enough that hand-computing the expected output isn’t practical — a full preprocessing pipeline, a multi-stage feature extractor — a golden-file test (save a known-good output once, compare future runs against it) is often the only practical option. The risk is using it as a substitute for understanding: a golden file that was generated by the buggy version of the code will happily certify the bug as correct forever after. A golden file earns its place only when the output it was generated from was independently checked at least once — by a domain expert, a published reference value, or a simpler method computing the same thing a different way.

The honest limit

None of this closes the real gap between “the code does what I wrote” and “the code does what the signal actually needs.” That gap is exactly why, in the EEG research thread on this blog, a passing pipeline and a sensible-looking result still got checked against a null model, a sample-size caveat, and an honest “this might not generalize” — a test suite is necessary for catching mechanical bugs, and it was never going to be sufficient for catching a bad scientific assumption.