Experiment building - on counterbalancing: Difference between revisions
Jump to navigation
Jump to search
mNo edit summary |
mNo edit summary |
||
| Line 2: | Line 2: | ||
===Thinking about counterbalancing=== | ===Thinking about counterbalancing=== | ||
<!-- | |||
You often want to arrange experimental conditions | |||
in a way that minimizes the influence of extraneous factors. | |||
Say, order effects are common when you present a bunch of stimuli to a person, | |||
but you may ''want' to ask them a bunch, because you specifically do within-subject study (as much as between-subject, or more so), | |||
and/or want to use your subjects for more than one question, because they're not easy to come by. | |||
So instead you would like to reduce said order effects. (or whatever other effects are in your way) | |||
The best way to reduce such effects depends on the kind of experiment design, | |||
both in the sense of e.g. is it within-subject or between-subject or such, | |||
and also just because of the actual task. | |||
Say, if you want to reduce priming, or at least ''detect'' that it happened, | |||
maybe you show different people the same things in different orders. | |||
Maybe 'the same' means 'all' and that is almost ''certain'' to prime them, | |||
and you want to spread your cases. | |||
Maybe they get better at the task because it's the only thing they're doing. | |||
Maybe a bunch of those cases seem related, meaning they are biased by what they've already seen a few minutes ago. | |||
Maybe that makes the react slower as they get confused about that, maybe they react faster by leading you to an answer. | |||
Maybe they react ''differently'' because they got primed. | |||
Maybe your neutralizing context is itself potentially related to the actual test words. | |||
Maybe the order did that. | |||
Maybe that's less of an issue because the task is so complex it taxes short term memory | |||
but it's still an issue when two similar cases are in close proximity. | |||
Maybe get get slower over time, as they get tired. | |||
Maybe they get faster purely because they are effectively practicing your task. | |||
You ''have'' to think of all the possible ways someone could do that and do it, | |||
because if you don't, a reviewer will, and they will usually have a really good point. | |||
So think like a reviewer, trying to shoot holes in this research. Ask a friend. | |||
If it's impossible for you and some people near you to think of ways it might be messy, | |||
that's generally better science. | |||
Counterbalancing sometimes refers more specifically to a sort of A-B testing: | |||
do something two different ways, and split it | |||
...or more ways and more groups. | |||
* Do it purely randomly | |||
: the fewer participants, the more | |||
* Do it entirely manually | |||
: this feels wrong, | |||
: and you can absolutely do this without enough shuffling | |||
: but it does mean you have complete overview of the ordering. | |||
* Make a set of rules to adhere to | |||
: I've seen it too often | |||
: that is ''far'' from random, but may still be what you want | |||
* "Make a list, and make an entirely-reversed" | |||
In software, a setting called 'counterbalance' | |||
might mean as little as "randomness based on participant number" | |||
or as much as "please adhere to all of these rules" | |||
https://dictionary.apa.org/within-subjects-design | |||
--> | |||
Revision as of 00:26, 24 April 2024
Notes related to setting up behavioural experiments and such.
|