Multiple hypothesis testing

Time and place

Time: 23 to 27 May 2026

Place: Hanken School of Economics, Arkadiankatu 22, 00101 Helsinki, Finland.
 

Schedule

Course_schedule_310326.pdf

Basic information

Languages of learning: English
Grading scale: General scale, 0-5
Course type: Special course unit (guest lecturer)
Course level: Graduate and Doctoral studies
Organiser: The course is organized by the Finnish Statistical Society and the Finnish Society of Biostatistics in collaboration with the Hanken School of Economics.
Higher education institution: Hanken School of Economics

Content and Goals

Learning outcomes:

After completing the course, the student has a good overview of the most important multiple testing criteria. The student realizes that even if the most standard multiple testing procedure (Bonferroni) provides no result, useful insights can be gained from the data. The student can choose a suitable method for their data analysis problem, and is aware of software implementations for the methods.

Content:

This course introduces doctoral students and other interested researchers to the field of multiple hypothesis testing. In economics and many other fields of science, it is common to test hypotheses or compute confidence intervals. When we test multiple
hypotheses, there is a risk of a large number of false discoveries. Multiple testing methods allow us to have control over such false discoveries. This course covers all the basics that you need in order to validly test multiple hypotheses in your research. The
course shows how to maximize statistical power, and how to draw additional conclusions on sets of (post hoc) selected hypotheses.

This course starts with theory on testing a single hypothesis, using simple classical tests, permutation tests, bootstrap tests or e-values. Subsequently, different multiple testing criteria will be covered, including: global testing, familywise error rate control (e.g. Bonferroni, and more powerful variants based on e.g. closed testing), false discovery rate control, false discovery exceedance control and simultaneous false discovery proportion control. For each of these criteria, multiple methods exist, and the most important ones will be covered – including the methods popular in economics (e.g. the Romano & Wolf procedure) and biostatistics. For several of these methods, software implementations (mostly in R) will be explained.
 

Target audience

The course is aimed at doctoral students in statistics, economics or other quantitative field where hypotheses are tested. Students should have knowledge of probability theory and statistics at the university level.

Teaching

The course consists of 20 hours of lectures and exercise sessions and concludes with an exam at Hanken. The lecturer will provide some additional practice exercises which students in the course can do in the weeks before the exam.

Examination

Exam dates: 18 June 2026, 9 am - 11 am (two hours) and 17 August 2026,  9 am - 11 am. The second exam is a retake. Students from outside Helsinki and abroad can ask for a special arrangement so that they take the exam at  their home university. In this case it is the student’s responsibility to arrange a faculty member to monitor  the exam.

Credits

5 ECTS

Prerequisites

Description of prerequisites:

No knowledge of multiple testing is assumed. Basic knowledge of probability theory and statistics is assumed. Some abstract mathematical thinking will be required (e.g., an event corresponds to a subset of the sample space, the sample space may be multidimensional, and a hypothesis is a set of distributions). Almost no knowledge of linear algebra is assumed.
 

Course materials

Digital lecture notes will be provided at the start of the course. The course is not based on any book in particular, but here is a list of some relevant sources.

  • Clarke, D., Romano, J. P., & Wolf, M. (2020). The Romano-Wolf Multiple Hypothesis Correction in Stata. The Stata Journal, 20(4), 812-843.
  • Goeman, J. J., & Solari, A. (2011).  Multiple Testing for Exploratory Research. Statistical Science, 26(4), 584-597.
  • Goeman, J. J., & Solari, A. (2014). Multiple Hypothesis Testing in Genomics. Statistics in Medicine, 33(11), 1946-1978.
  • Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing, Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300.
  • Westfall, P. H., & Young, S. S. (1993). Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. John Wiley & Sons.
  • Cui, X., Dickhaus, T., Ding, Y., & Hsu, J. C. (Eds.). (2021). Handbook of Multiple Comparisons. CRC Press.

Registration

The course is fully funded by Hanken School of Economics and there is no course fee. Students will have to fund their travel and accommodation costs.

Send the completed application form to professor Niklas Ahlgren (niklas.ahlgren@hanken.fi).

Application deadline: 27 April 2026.

Responsible persons and contact information

Lecturer: Assistant Professor Jesse Hemerik (hemerik@ese.eur.nl), Econometric Institute, Erasmus University Rotterdam, PO Box 1738, 3000 DR Rotterdam, The Netherlands. 

Course coordinator: Niklas Ahlgren  (niklas.ahlgren@hanken.fi)