skip to content

Department of Psychology

Postgraduate Research Methods Class

Prof Mark Haggard

Thursdays 4-5pm from 20 January to 24 February 2022. 

Mode of delivery: pre-recorded Zoom recording, to be viewed in advance before timed discussion, (on Zoom or Teams platform, to be notified), on Thursday afternoons at 4pm on 20 Jan 2022 and the subsequent 5 weeks (27 Jan, 3 Feb, 10 Feb, 17 Feb, 24 Feb).  For each Discussion, class member are encouraged also to raise apparently important methods issues they have seen in reading or issues they have recently faced themselves. If there is sufficient demand, a further specialised class of a workshop nature on Structural Equation Modelling, also open to non-class-members, will be scheduled.

Suitable for: all postgraduates. Primarily aimed at research students who have not taken it before (ie including those beyond 1st year PhD registrants who think they may benefit). In past years some who attended in their 1st year have re-attended for particular topics.

The 5 educational aims of the course are:

  1. To give postgraduates confidence to acquire and use statistical techniques for diverse types of problem, beyond the undergraduate minimum of by conveying an intuitive understanding of principles concerning what is legitimate and why
  2. To convey a profound understanding of the design trade-offs, including real-world compromises, that govern planning and execution of competent research, and to help them avoid acquiring oddly structured data without prior clear idea of appropriate analysis
  3. To help avoid mistakes that would rightly lead an examiner or reviewer to reject (eg undeclared and unadjusted multiple testing)
  4. To oppose the trivialisation of research seen in recent decades, via advocated principles for planning and reporting that increase generalisability and replicability; to
  5. To inform students on how to specify a design/analysis issue clearly in correct ‘data-language’, and so facilitate searches or discussions to solve an analysis problem (preparation for efficient and productive consultations with experts, including eg the class organiser)
Class 1: 20 January 2022 at 4pm online

Strategy, principles, power and errors; putting the Replication Crisis behind us

Recommended Reading:

Brunton, & Beyeler, M. (2019). Data-driven models in human neuroscience and neuroengineering. Current Opinion in Neurobiology, 58, 21–29.

Davis, Redshaw, J., Suddendorf, T., Nielsen, M., Kennedy-Costantini, S., Oostenbroek, J., & Slaughter, V. (2021). Does Neonatal Imitation Exist? Insights From a Meta-Analysis of 336 Effect Sizes. Perspectives on Psychological Science, 16(6), 1373–1397.

Guest, & Martin, A. . (2021). How computational modeling can force theory building in psychological science. Perspectives on Psychological Science, 16(4), 789–802.

Peterson, Bourgin, D. D., Agrawal, M., Reichman, D., & Griffiths, T. L. (2021). Using large-scale experiments and machine learning to discover theories of human decision-making. Science (American Association for the Advancement of Science), 372(6547), 1209–1214.

Press, Yon, D., & Heyes, C. (2022). Building better theories. Current Biology, 32(1), R13–R17.



Watch preview lecture - Week 1 (Raven only)

Watch a recording of the lecture - Week 1 (20 January 2022) - (Raven only)

Slides in PDF - Week 1 (Raven only)

Class 2: 27 January 2022 at 4pm in Psychology Classroom, Downing Site

Level of measurement and the General Linear Model underlying ANOVA, regression etc

Recommended reading:


Watch a recording of the lecture - Week 2 (27 January 2022) - (Raven only)

Slides in PDF - Week 2 (Raven only)

Class 3: 3 February 2022 at 4pm in Psychology Classroom, Downing Site

Interactions in ANOVA: planning for interpretation, adjustment for multiple testing

Recommended reading:

Ioannou, Green, P., Fan, V. S., Dominitz, J. A., O’Hare, A. M., Backus, L. I., Locke, E., Eastment, M. C., Osborne, T. F., Ioannou, N. G., & Berry, K. (2021). Development of COVIDVax Model to Estimate the Risk of SARS-CoV-2-Related Death among 7.6 Million US Veterans for Use in Vaccination Prioritization. JAMA Network Open, 4(4), e214347–e214347.

Carrie E. Smith, & Robert Cribbie. (2014). Factorial Anova with Unbalanced Data: A Fresh Look at the Types of Sums of Squares. Journal of Data Science, 12(3), 385–404.

Tanguma, & Speed, F. M. (2000). Interpreting the Four Types of Sums of Squares in SPSS. Null.

Suzuki, & Yamamoto, Y. (2021). Characterizing the Influence of Confirmation Bias on Web Search Behavior. Frontiers in Psychology, 12, 771948–771948.

Krueger, & Heck, P. R. (2017). The heuristic value of p in inductive statistical inference. Frontiers in Psychology, 8, 908–908.

Antônio Alves Tôrres Fernandes, Dalson Britto Figueiredo Filho, Enivaldo Carvalho da Rocha, & Willber da Silva Nascimento. (2021). Read this paper if you want to learn logistic regression. Revista de Sociologia e Política, 28(74).

Dmitrienko, & D’Agostino, R. B. (2018). Multiplicity Considerations in Clinical Trials. The New England Journal of Medicine, 378(22), 2115–2122.

Félix, & Menezes, A. F. B. (2018). Comparisons of ten corrections methods for t-test in multiple comparisons via Monte Carlo study (Vol. 11, Issue 1, pp. 74–91) [Data set].

Huang. (2022). Alternatives to logistic regression models in experimental studies. The Journal of Experimental Education, 90(1), 213–228. 


Watch a recording of the lecture - Week 3 (3 February  2022) - (Raven only)

Slides in PDF - Week 3 (Raven only)

Class 4: 10 February  2022 at 4pm in Psychology Classroom, Downing Site

Generalisability and parsimony: the df-ratio and multi-versing

Recommended Reading 

Yarkoni. (2020). The generalizability crisis. The Behavioral and Brain Sciences, 1–37.

Daniel Lakens. (2021). There is no generalizability crisis. Pre-Print.

Grahek, Schaller, M., & Tackett, J. L. (2021). Anatomy of a Psychological Theory: Integrating Construct-Validation and Computational-Modeling Methods to Advance Theorizing. Perspectives on Psychological Science, 16(4), 803–815.

Brittany I Davidson. (2021). Measurement practices exacerbate the generalizability crisis: Novel digital measures can help. Pre-Print.

Carpenter, & Bithell, J. (2000). Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19(9), 1141–1164.<1141::AID-SIM479>3.0.CO;2-F 


Watch a recording of the lecture - Week 4 (10 February  2022) - (Raven only)

Slides in PDF - Week 4 (Raven only)

Class 5: 17 February 2022 at 4pm in Psychology Classroom, Downing Site

Data-reduction: principal components, factor analysis and introduction to SEM#

Recommended Reading 

Thalheimer, & Cook, S. (2009). How to calculate effect sizes from published research: A simplified methodology. N/A. 

Patel, Burford, B., & Ioannidis, J. P. . (2015). Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. Journal of Clinical Epidemiology, 68(9), 1046–1058.

Lewis. (n.d.). An Introduction to Classification and Regression Tree (CART) Analysis. N/A. 



Watch a recording of the lecture - Week 5 (17 February 2022) - (Raven only)

Slides in PDF - Week 5 (Raven only)

Class 6: 24 February 2022 at 4pm in Psychology Classroom, Downing Site

Structural equation modelling and confirmatory factor analysis


Recommended Reading

Wang, & Rhemtulla, M. (2021). Power Analysis for Parameter Estimation in Structural Equation Modeling: A Discussion and Tutorial. Advances in Methods and Practices in Psychological Science, 4(1), 251524592091825–.

Deng, Yang, M., & Marcoulides, K. M. (2018). Structural equation modeling with many variables: A systematic review of issues and developments. Frontiers in Psychology, 9, 580–580.

Ringle, Sarstedt, M., Mitchell, R., & Gudergan, S. P. (2020). Partial least squares structural equation modeling in HRM research. International Journal of Human Resource Management, 31(12), 1617–1643.


Joel E. Collier. (2020). Applied structural equation modeling using AMOS : basic to advanced techniques / Joel E. Collie. Routledge. 


Watch a recording of the lecture - Week 6 (24 February 2022) - (Raven only)

Slides in PDF - Week 6 (Raven only)

#It is hoped to be able hold again with sufficient physical isolation, an SEM workshop in the department -- later in the Lent term or in the Easter term.