12. Experimental Designs

The  design of an experiment and its statistical analysis are intimately connected. These pages describe various designs with numerical examples of their statistical analysis. There may be  some overlap with the pages on Statistical Analysis. Sample size  is discussed  separately.The designs, in most cases with a  numerical example, are considered under the following three headings:

Note that this page has three sub-pages. To access them click on “More details”. To return to this page click on “12. Experimental designs” in the navigation bar.

1. Between-subject,  single factor & completely randomised designs

These are the most common designs. They involve a single factor, usually called the "treatment". This can have any number of levels which can  be quantitative (e.g. doses of a test compound) or qualitative (e.g. diets A, B, C). Unequal numbers in each group  present no problems.

Experimental subjects are  assigned to treatments strictly at random without regard to any characteristics, so these are sometimes called "completely randomised between-subjects designs". However, surveys such as comparisons of  different strains involve the same statistical methods, so are also discussed here.

These designs are only suitable for  experiments where the experimental material is homogeneous. If the  experimental units are highly variable (e.g. in body weight, age, genotype  etc.) or if they are so large that they become unwieldy, the use of a blocked design should be considered otherwise the experiment will need to be large or it will lack power.

These designs are usually  analysed by a one-way analysis of variance (possibly following a scale  transformation) if the outcome is quantitative or a chi-squared analysis if the outcome is counts/proportions. Non-parametric tests such as the Mann-Whitney or Kruskal Wallace test may be necessary when the assumptions  underlying parametric methods are not met (see 11. Statistical analysis). More details

2. Blocked or  stratified designs

The designs include randomised block, crossover (within-subject) and Latin square designs as well as some others not discussed here. They are very useful  designs, and should be used more widely in biomedical research.Blocking breaks the experiment up into smaller "mini-experiments":

    For convenience. It is often  easier to do an experiment which has been split up into smaller bits.  Blocking ensures that this is done correctly so as to avoid bias and improve  power.

    To increase statistical power.  It can remove the effects of any observable heterogeneity of the experimental material (e.g. it may be difficult to get a group of animals of sufficiently uniform age or body weight or it might be impossible to do all  the measurements at the same time of day).

    To take  account of any natural structure of the experimental material, such as experiments involving neonatal animals which come in litters.

    To remove  time effects such as those due to diurnal or seasonal rhythms or just  random fluctuations in the environment.

    To ensure repeatability. If an experiment is done in small bits over a  period of days, weeks or months significant differences among treatments will only be detected if each mini-experiment gives consistent results,  subject to sampling error. These designs are common in in-vitro studies  where the investigator sometimes uses words like "The experiment was repeated three times". This implies that they did a randomised block experiment with three blocks, with the blocking factor being time.

These designs are usually  analysed by a two-way analysis of variance without interaction (see numerical examples). The statistical analysis is much  easier if there are equal numbers in each treatment group within a block. More details

3. Factorial designs

These designs look at the effect of two or more "factors" simultaneously. One factor is usually the  "treatment", others could be sex, genotype, age, previous treatment, diet etc. There can be any number of factors and any number of levels of each  factor, but designs with more than four factors are rare in biomedical research, though common in industry.

These designs are extremely  useful and some statisticians suggest that virtually all experiments should have this type of design because they provide extra information at little or no extra cost.

They are used to:

    Study any interactions among factors. It is often important to know what factors influence the  outcome of an experiment.
    Increase the amount of information
    from an experiment without increasing the numbers of animals. They are almost like doing two or more experiments simultaneously with the same animals. Thus they are a useful "Reduction" strategy
    Optimise subsequent  experiments. They can be used to find the combination of factors which  produces most sensitivity in subsequent similar experiments.

    Blocking and factorial designs are not mutually exclusive. Both can be used in a single experiment. More details