Experiment building - non-software-specific notes, and timing

Notes related to setting up behavioural experiments and such. Experiment design Hardware and timing Experiment building - on timing · on online experiments · on counterbalancing E-Prime notes · PsychoPy notes · Experiment builder notes · Gorilla notes · PsychToolbox notes · OpenSesame notes · DMDX notes

Experiment builders are one possible term for 'software that lets you fairly easily create and run a well-controlled behavioural experiment, to research the validity of a theory or belief'.

This may sound no harder than making a nice powerpoint, but since various experiments care about reaction speed (e.g. as a measure of confusion), you care about precise stimuli timing, and precise response timing.

The lower you want to push this, the more specialized both the software and other practical details become.

On online experiments

Online experiments, as convenient as they are, means there are many things you can no longer control for - display time, hardware response time, browser details, whether it is a computer or a phone (I have a years-old phone and I wouldn't trust its timing),

Browsers

Assume that browsers tend to merge movement into 60Hz intervals - or whatever monitor-limited speed it's drawing at - so negating the effects of a 1000Hz keyboard / mouse / button device.

Also, multitasking in browsers varies more between browsers. Maybe there's a video stream in another tab making things... more varied.

Since it's not something you can control, at all, it's not a good environment for precision timing.

Easy for questionnaire style stuff, though.

Hardware and timing

Buttons and timing

Electric output from a switch that just closed and is still bouncing (most of the mess in 2ms)

The hardware buttons bounce. This is literal: metal contacts bounce physically, which is reflected in the electric signal. (and yes, there are e.g. optical and magnetic switches that do not bounce in this way, though they are not free of other issues)

You can assume this bouncing lasts at least a few milliseconds, up to a dozen or two.

Debouncing refers to some way of ensuring that this mass of flip-flopping is registered as a single press.

A lot of buttons react dozens of milliseconds, later and you never noticed, so in general this is a non-issue, but in reaction time experiments this is pretty central.

Consider that if your hardware assumes a switch never bounces longer than 10ms, then if it actually is still bouncing after that, you will register multiple presses. Rather fewer than if you didn't debounce, but still multiple.

There is often a tradeoff where the more sure you want to be that it never triggers weirdly, the longer the delay must be.

In the "can we do better" camp, absolutely, but only if you are willing to build in some specific assumptions that better be true for your hardware setup.

If you know specific hardware (ideally down to capacitance of the wires that connect it and the size of the pullups/pulldowns), you can safely have a shorter debounce time.

But also, consider that in experiments, it is enough to afterwards have a record of down to the millisecond, without the need to have something react to down to the millisecond.

you can record both when activity started and whether it was later verified to be a real press.

As long as you timestamp it, this is much more accurate than "I am now reporting a debounced press that probably started idunno milliseconds ago"

you could do this in an external device, if you synchronize clocks to do that timestamping

if you want to avoid that complexity, you you could choose to do this in post-processing

This isn't perfect (consider e.g. some EMI noise right before a real press).

...but is typically good for reporting response to with ~1ms accuracy later.

Keyboards and timing

tl;dr:

• Assume a keyboard may have 8-25ms of latency, probably on the 20ms end.

It might well be lower, but you should never assume it.

• sometimes more importantly, there is variation in the response time, which will be reflected in a standard deviation in your data even if triggered by atomic-clock precision

• Avoid wireless. In particular bluetooth can adds a handful of milliseconds.

• So if we care about 1ms accuracy, do not use keyboards.

The OS or hardware does not care about precisely the human acted. It's not that it couldn't, it's that it doesn't.

And when you step back, there is a very pragmatic reason for this imprecision: most keyboardists don't care.

You probably never noticed that your keystrokes arrived two dozen milliseconds later, and no one presses the same key more than 50 times per second, or types at 3000 characters per minute it would take for it to probably start losing presses(verify).

There are also low level reasons. One is debounce, as just mentioned. A relatively safe 10 to 20ms of debounce time is perfectly fine for all typing needs - same numbers as just mentioned.

Another is an OS / USB / driver level choice: with most USB keyboards, the PC will only check whether the keyboard had anything to say at most 125 times per second - again, because that's faster than most people type.

Variation

When you do care about timing at the millisecond scale, you have two issues. One, you know it arrives late. You do not know how much.

Two, you do not know this "fetch from USB" schedule. While it's probably strictly regular, it's on its own timer, at a different rate from your screen updates.

So relative to screen updates (specifically the first in a stimulus, which determines timing start), the keystroke arrives in an also-regular but unrelated schedule. Which, to frame-based timing, looks like it is variable.

Gamer keyboards may both have a faster readout (e.g. 1000Hz=1ms interval instead of 8ms), and may choose a shorter debounce time.

Note that "1000Hz keyboard" means it reduces the average latency as well as reducing the variation, but is unrelated to the delay from debounce -- which was larger to start with. Assume even fancy stuff never gets under a handful of milliseconds - and measure to actually know.

Displays and timing

✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

Sound and timing

✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

Semi-sorted

D Bridges et al. (2020) "The timing mega-study: comparing a range of experiment generators, both lab-based and online"