to uncover causal relationships, cease making an attempt to invent a time machine and run an experiment as a substitute! Understanding causal relationships provides the information wanted to supply desired outcomes by motion. On this article, I’m going as an example the facility of experimental design by utilizing a time-machine-based conceptual train. My purpose is to persuade you that extra will be realized about causality by experimentation than utilizing a time machine.
Why are time machines helpful in a causal thought experiment?
Utilizing a time machine for a thought experiment feels ridiculous, and in some ways it’s. Nevertheless it additionally has a attribute that makes it precious for exploring hypothetical outcomes. Time machines might give us one thing that we, in our time-bound state, can’t see – counterfactuals. Because the identify implies, a counterfactual is one thing that didn’t occur. They aren’t observable by definition as a result of they by no means occurred. Counterfactuals what would’ve occurred underneath completely different circumstances. They offer solutions to questions like – “Would I’ve gotten sick if I didn’t eat that gasoline station sushi?” If we had a time machine nonetheless, we might reverse the clock, do one thing completely different and see what occurs. Within the case of the sushi, I might restart the day, not eat the sushi and see if I nonetheless get sick. In different phrases, we might observe the in any other case unobservable counterfactuals.
The counterfactuals realized by the point machine might then be in comparison with what really occurred (we might name it a ‘factual’ I suppose…) to know the impression of an intervention. For our unlucky sushi instance, me getting sick is the ‘factual’ – it really occurred. If I had a time machine, I might rewind time, not eat the sushi and observe what would’ve occurred, that is the counterfactual. I might then examine the factual with the counter factual to ascertain causality. Let’s say that I went again in time, stored every part in my day the identical besides consuming the sushi. If I nonetheless received sick (factual = counterfactual), I do know that the sushi didn’t trigger the sickness as a result of I’d’ve been sick both means. If I didn’t get sick nonetheless (factual ≠ counterfactual), then I can conclude that the sushi brought about my sickness. With a time machine, establishing causality for particular person occasions could be that simple!
At first look, it looks as if our time machine can be an superior causality deducing machine! Having the ability to observe counterfactuals could be very highly effective, however we are able to really make extra helpful causal deductions utilizing well-designed experiments. Which is nice as a result of, time machines don’t exist, however well-designed experiments do! Let’s get into how designed experiments will be higher than utilizing a time machine.
The causality of particular person occasions just isn’t generalizable
Whereas a time machine would reply quite a lot of curiosity-driven ‘what if’ causal questions, the learnings we’d acquire from observing counterfactuals wouldn’t be generalizable to different, related (however not the identical) conditions. In my sushi instance, I’d fulfill my curiosity by understanding if the sushi made me sick – however the information I gained wouldn’t serve any pragmatic function for future choices. All I do know is that on that particular day, at that particular gasoline station, at that particular time, that particular serving of sushi made me sick. I don’t know what would occur if I modified any of the bolded circumstances.
We will acquire generalizable information, which we wouldn’t get from the time machine, by designing an experiment. Generalizable information may be very helpful as a result of it could actually assist us make good choices sooner or later!
Think about that I ran an experiment that randomly assigned a number of courageous souls to eat gasoline station sushi or restaurant sushi. This experiment would inform me if on common, gasoline station sushi makes folks sicker than restaurant sushi. That is already an enchancment from the ‘time machine’ method as a result of the outcomes apply to the inhabitants of people who I sampled as a substitute making use of to me solely.
However, I could possibly be smarter in regards to the design of the experiment to get much more information! As an alternative of merely assigning folks to gasoline station or restuarant sushi, I might assign folks specifc gasoline stations at particular occasions or the restuarant at particular occasions. By including these two new variables (time and gasoline station location) I cannot solely be taught if gasoline station sushi makes folks sick extra typically, I may be taught if there are variations between the three gasoline stations that serve sushi in my city and if time of day additionally has an impression.
On this experiment, I don’t straight observe counterfactuals, however the randomized project helps confounders common out so I can estimate the typical therapy impact (ATE) virtually as if I might observe counterfactuals.
How do the experiment learnings differ from my time machine learnings? The experiment is (1) utilizing a number of folks, (2) a number of sushi servings, (3) a number of gasoline stations and (4) a number of occasions of day. Because of this, I can take away quite a lot of causal insights that I and different folks can use. For instance, I’d perceive if usually, gasoline station sushi makes folks sicker than restaurant sushi in my city. I’d additionally be taught if some gasoline stations make folks extra sick than others and if shopping for sushi at some occasions is worse than others. This information can assist me, and different folks make future choices. It’s far more helpful than understanding that the sushi from one gasoline station and one time made me sick!
Along with the entire variables that we are able to management, we are able to embody covariates in our evaluation. Covariates are elements that we can’t management however are essential. On this instance, covariates could possibly be issues like earlier medical situations or age. By together with covariates within the evaluation, we are able to additionally be taught if there are any interplay results between the covariates and the remedies.
Under is a abstract that compares what we might be taught with a time machine to what we are able to be taught with experiments.
Now that we perceive the wealthy depth of causal relationships that we are able to perceive utilizing experimentation, let’s transition to discussing how the number of outcomes underneath an experiment is extra highly effective than a single final result (the one counterfactual) that we’d observe with a time-machine run.
Designed experiments quantify the causal relationships; single counterfactuals don’t
Direct remark of a single counterfactual doesn’t give any thought of the energy of the final causal relationship. If I’m going again in time after I received sick as soon as to check if the sushi made me sick, I’d be taught that it did, or it didn’t trigger my sickness. I nonetheless wouldn’t have any thought of the chance that I’ll get sick if I fulfill my sushi craving at a gasoline station once more sooner or later! Is it deterministic, i.e., will get sick each single time I eat gasoline station sushi? Is it probabilistic, will I get sick fifty % of the time? I simply don’t have sufficient data to know.
The experiment we designed within the earlier part wouldn’t solely assist us perceive if gasoline station sushi makes folks sick, it could additionally assist quantify the connection. For instance, the experiment would possibly discover that on common, consuming gasoline station sushi makes you 5 occasions extra more likely to get sick than restaurant sushi.
Experimental design generalizes higher, and it additionally quantifies the causal relationship higher! If we return in time and take a look at one counterfactual, we are able to’t know the chance of observing the identical outcome underneath related situations, with experimentation we are able to!
Wrapping it up
My purpose in writing this text was to debate why I’d nonetheless use experimental design to study causal relationships even when I had a time machine that allowed me to watch counterfactuals.
The principle causes experimental design is best is as a result of:
- It generates generalizable causal learnings (versus one particular case)
- It gives the energy of relationships to tell future choices
I hope this thought experiment deepened your understanding of the strengths of experimental design!
