Multiple hypothesis testing
Time and place
Hanken School of Economics, 23 to 27 May 2026
Basic information
Languages of learning: English
Grading scale: General scale, 0-5
Course type: Special course unit (guest lecturer)
Course level: Graduate and Doctoral studies
Organiser: The course is organized by the Finnish Statistical Society and the Finnish Society of Biostatistics in collaboration with the Hanken School of Economics.
Higher education institution: Hanken School of Economics
Content and Goals
Learning outcomes:
After completing the course, the student has a good overview of the most important
multiple testing criteria. The student realizes that even if the most standard multiple
testing procedure (Bonferroni) provides no result, useful insights can be gained from the
data. The student can choose a suitable method for their data analysis problem, and is
aware of software implementations for the methods.
Content:
This course introduces doctoral students and other interested researchers to the field
of multiple hypothesis testing. In economics and many other fields of science, it is
common to test hypotheses or compute confidence intervals. When we test multiple
hypotheses, there is a risk of a large number of false discoveries. Multiple testing
methods allow us to have control over such false discoveries. This course covers all the
basics that you need in order to validly test multiple hypotheses in your research. The
course shows how to maximize statistical power, and how to draw additional
conclusions on sets of (post hoc) selected hypotheses.
This course starts with theory on testing a single hypothesis, using simple classical
tests, permutation tests, bootstrap tests or e-values. Subsequently, di erent multiple
testing criteria will be covered, including: global testing, familywise error rate control
(e.g. Bonferroni, and more powerful variants based on e.g. closed testing), false
discovery rate control, false discovery exceedance control and simultaneous false
discovery proportion control. For each of these criteria, multiple methods exist, and the
most important ones will be covered – including the methods popular in economics
(e.g. the Romano & Wolf procedure) and biostatistics. For several of these methods,
software implementations (mostly in R) will be explained.
Target audience
The course is aimed at doctoral students in statistics, economics or other quantitative
field where hypotheses are tested. Students should have knowledge of probability
theory and statistics at the university level.
Teaching
The course consists of 20 hours of instruction and pen-and-paper exercise sessions.
Examination
The final grade is determined by a written, 2-hour take-home exam, with mostly open
questions. The questions test if the student can apply and understand the methods
from the course. Limited self-made notes can be brought to the exam.
Exam dates: 2 dates (to be decided).
Credits
5 ECTS
Prerequisites
Description of prerequisites:
No knowledge of multiple testing is assumed. Basic knowledge of probability theory and
statistics is assumed. Some abstract mathematical thinking will be required (e.g., an
event corresponds to a subset of the sample space, the sample space may be multi-
dimensional, and a hypothesis is a set of distributions). Almost no knowledge of linear
algebra is assumed.
Course materials
Digital lecture notes will be provided at the start of the course. The course is not based
on any book in particular, but here is a list of some relevant sources.
- Clarke, D., Romano, J. P., & Wolf, M. (2020). The Romano-Wolf Multiple Hypothesis Correction in Stata. The Stata Journal, 20(4), 812-843.
- Goeman, J. J., & Solari, A. (2011). Multiple Testing for Exploratory Research. Statistical Science, 26(4), 584-597.
- Goeman, J. J., & Solari, A. (2014). Multiple Hypothesis Testing in Genomics. Statistics in Medicine, 33(11), 1946-1978.
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing, Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300.
- Westfall, P. H., & Young, S. S. (1993). Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. John Wiley & Sons.
- Cui, X., Dickhaus, T., Ding, Y., & Hsu, J. C. (Eds.). (2021). Handbook of Multiple Comparisons. CRC Press.
Registration
The course is fully funded by Hanken School of Economics and there is no course fee.
Students will have to fund their travel and accommodation costs.
Send the completed application form to professor Niklas Ahlgren (niklas.ahlgren@hanken.fi).
Application deadline: 27 April 2026.
Responsible persons and contact information
Lecturer: Assistant Professor Jesse Hemerik (hemerik@ese.eur.nl), Econometric Institute, Erasmus University
Rotterdam, PO Box 1738, 3000DR Rotterdam, The Netherlands.
Course coordinator: Niklas Ahlgren (niklas.ahlgren@hanken.fi)