• Determine the question of interest
• Identify the response to be measured or observed
• Identify the factor that you think may be causing the difference, the treatment factor
• Define the treatment group and the control group
• Compare the groups to see if your hypothesis is consistent with the results
• Unacceptable: ``He said, she said (hearsay)'', over-reporting of rare events, non-falsifiable hypotheses, etc.

Experimental design is fundamental

Conduct a study to determine whether a relationship exists between variables.

• Compare the treatment and control groups to see if an association exists
• If the treatment has no effect the two groups should have the same response
• The two groups should be similar in all respects except for the treatment, so that differences are due to the treatment

Treatment vs. control

We want to make sure that treatment and control groups are as similar as possible.

• Other factors could be contributing to the difference in responses of the groups
• Placebos: ``fake'' treatment to make it impossible for subjects to know they are controls
• Double-blind studies: designed so that neither the subjects nor the experimentors who interact with them know which are the controls
• Confounding variables: difference in response is due to some factor other than treatment; this other factor is also responsible for the fact that subject is in the treatment group.

Controlled experiment

In a controlled experiment, the experimenter chooses who will receive treatment and who will not.

• For best results, choose people eligible for either, and then randomly assign to treatment or control: randomized controlled
• Chances are the two groups will be similar in all respects except for the treatment
• Avoid bias of the experimenters in assigning groups and self-selection of experimental subjects.

What if we cannot choose whether someone will be treated or not?

Observational study

In an observational study, the experimenter cannot choose who will receive treatment and who will not.

• Possible that the ``treatment'' variable to be studied is a disease or a behavior that cannot be changed or forced
• More difficult to establish similar treatment and control groups
• Can only establish association, but not causation

Historical controls

It's sometimes necessary to compare a treatment group in the present with a control group from the past

• Conditions in the past are often not the same as those in the present (e.g. improved medical procedures, changes in environment and social structure)
• Unable to observe those patients from the past to re-evaluate
• If possible, it's best to compare contemporaneous groups

More about confounding

Since the risk of confounding variables is such a problem with observational studies, how can we deal with this issue?

• First try to understand how controls were selected
• Try to control for confounding factors by making comparisons of smaller groups of like subjects
• Examples include subsetting by gender, age, etc.
• Using randomized, controlled experiments (when possible) is the most effective way to limit confounding