Types of Variables
Formulating a Scientific Study, Part 1
Looking back at my previous articles, I’ve realized I’ve neglected an important part of science research - how to set up a scientific study, and what kinds there are. Thus, the beginning of a new series. For now, we’ll talk about the different variables that go into formulating a study.
Independent variables are aspects of the study that do not depend on anything else. Obvious examples of this are things like time or temperature, but studies can also be designed to attempt to make the variable of interest independent, such as “amount of hot chocolate you’ve ordered at Starbucks”.
Dependent variables are exactly what they sound like - aspects of the study that depend on the independent variables. You could create a study that attempts to determine the effect of the amount of hot chocolate you’ve ordered at Starbucks on the amount of times you have to shovel your driveway. The hot chocolate would still be the independent variable, and the shoveling would be the dependent variable. This may sound absurd, but it is not uncommon for researchers to be given large datasets with lots of variables to do analysis on and see what relationships pop up.
Confounding variables are variables in a study that affect both independent and dependent variables. This can make it look like the independent variable caused the effect on the dependent variable, but in reality it did not. In our hot chocolate/driveway shoveling example, the obvious confounding variable is the weather. These can cause “spurious correlations”, where it appears that things are directly correlated when in fact they are not. These correlations are very common in modern life, and there are various websites where you can see different examples. Not fully understanding your confounding variables can lead to invalid scientific conclusions, as well as omitted variable bias, in which you have attempted to define a mathematical model for the behavior you are seeing, but you don’t have all the correct variable defined, so it’s not a good representation of reality.
The null hypothesis is the concept that the effect you are studying does not exist, and it is the framing for many scientific studies. In our hot chocolate/shoveling example, the null hypothesis would be that the amount of hot chocolate you bought at Starbucks does not have an effect on the amount of time you spent shoveling your driveway. The goal of your experiment would be to perform some sort of statistical analysis on your data to find out if that is a true assumption.
P-value is a metric that is commonly used to determine if a statistic is relevant. Assuming that our null hypothesis is true, it determines the probability that the data we are seeing came from that null hypothesis. If that probability is small (typically less than 5%), we can say that our null hypothesis is incorrect. So, if we collected all our data and got a p-value of 1%, we can say that our null hypothesis is incorrect, and the amount of hot chocolate you buy does affect the amount of time you spent shoveling. This is a metric that is really easy to be confused by, and even easier to manipulate with p-value hacking, which I will go into more detail on in another article.
Control groups are groups in comparative studies that do not receive whatever is being tested- the people who aren’t buying any hot chocolate.
Benchmarks set standards or measurements to compare new methods, machines, or applications against. They are found in most areas of scientific research, with common examples being the Pascal Visual Object Classification leaderboard in machine learning, or the Dhrystone metric for computer processor performance. This doesn’t fit cleanly into our hot chocolate example study, but if we didn’t have a control group in our experiment, we could potentially look at very large amounts of data on the amount of time people spent shoveling their driveway, and compare it to people we’ve surveyed.