skip to content

Department of Psychology

Postgraduate Research Methods Class

Prof Mark Haggard

Thursdays 4-5pm from 20 January to 24 February 2022. 

Mode of delivery: pre-recorded Zoom recording, to be viewed in advance before timed discussion, (on Zoom or Teams platform, to be notified), on Thursday afternoons at 4pm on 20 Jan 2022 and the subsequent 5 weeks (27 Jan, 3 Feb, 10 Feb, 17 Feb, 24 Feb).  For each Discussion, class member are encouraged also to raise apparently important methods issues they have seen in reading or issues they have recently faced themselves. If there is sufficient demand, a further specialised class of a workshop nature on Structural Equation Modelling, also open to non-class-members, will be scheduled.

Suitable for: all postgraduates. Primarily aimed at research students who have not taken it before (ie including those beyond 1st year PhD registrants who think they may benefit). In past years some who attended in their 1st year have re-attended for particular topics.

The 5 educational aims of the course are:

  1. To give postgraduates confidence to acquire and use statistical techniques for diverse types of problem, beyond the undergraduate minimum of by conveying an intuitive understanding of principles concerning what is legitimate and why
  2. To convey a profound understanding of the design trade-offs, including real-world compromises, that govern planning and execution of competent research, and to help them avoid acquiring oddly structured data without prior clear idea of appropriate analysis
  3. To help avoid mistakes that would rightly lead an examiner or reviewer to reject (eg undeclared and unadjusted multiple testing)
  4. To oppose the trivialisation of research seen in recent decades, via advocated principles for planning and reporting that increase generalisability and replicability; to
  5. To inform students on how to specify a design/analysis issue clearly in correct ‘data-language’, and so facilitate searches or discussions to solve an analysis problem (preparation for efficient and productive consultations with experts, including eg the class organiser)
Class 1: 20 January 2022 at 4pm online

Strategy, principles, power and errors; putting the Replication Crisis behind us

Recommended Reading:

Brunton, & Beyeler, M. (2019). Data-driven models in human neuroscience and neuroengineering. Current Opinion in Neurobiology, 58, 21–29. https://doi.org/10.1016/j.conb.2019.06.008
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19103041730003606?auth=SAML

Davis, Redshaw, J., Suddendorf, T., Nielsen, M., Kennedy-Costantini, S., Oostenbroek, J., & Slaughter, V. (2021). Does Neonatal Imitation Exist? Insights From a Meta-Analysis of 336 Effect Sizes. Perspectives on Psychological Science, 16(6), 1373–1397. https://doi.org/10.1177/1745691620959834
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19103045410003606?auth=SAML

Guest, & Martin, A. . (2021). How computational modeling can force theory building in psychological science. Perspectives on Psychological Science, 16(4), 789–802. https://doi.org/10.1177/1745691620970585
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19103050890003606?auth=SAML

Peterson, Bourgin, D. D., Agrawal, M., Reichman, D., & Griffiths, T. L. (2021). Using large-scale experiments and machine learning to discover theories of human decision-making. Science (American Association for the Advancement of Science), 372(6547), 1209–1214. https://doi.org/10.1126/science.abe2629
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19103056070003606?auth=SAML

Press, Yon, D., & Heyes, C. (2022). Building better theories. Current Biology, 32(1), R13–R17. https://doi.org/10.1016/j.cub.2021.11.027
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19103080290003606?auth=SAML

 

null

Watch preview lecture - Week 1 (Raven only)

Watch a recording of the lecture - Week 1 (20 January 2022) - (Raven only)

Slides in PDF - Week 1 (Raven only)

Class 2: 27 January 2022 at 4pm in Psychology Classroom, Downing Site

Level of measurement and the General Linear Model underlying ANOVA, regression etc

Recommended reading:

null

Watch a recording of the lecture - Week 2 (27 January 2022) - (Raven only)

Slides in PDF - Week 2 (Raven only)

Class 3: 3 February 2022 at 4pm in Psychology Classroom, Downing Site

Interactions in ANOVA: planning for interpretation, adjustment for multiple testing

Recommended reading:

Ioannou, Green, P., Fan, V. S., Dominitz, J. A., O’Hare, A. M., Backus, L. I., Locke, E., Eastment, M. C., Osborne, T. F., Ioannou, N. G., & Berry, K. (2021). Development of COVIDVax Model to Estimate the Risk of SARS-CoV-2-Related Death among 7.6 Million US Veterans for Use in Vaccination Prioritization. JAMA Network Open, 4(4), e214347–e214347. https://doi.org/10.1001/jamanetworkopen.2021.4347
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19204777450003606?auth=SAML

Carrie E. Smith, & Robert Cribbie. (2014). Factorial Anova with Unbalanced Data: A Fresh Look at the Types of Sums of Squares. Journal of Data Science, 12(3), 385–404.
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19204826090003606?auth=SAML

Tanguma, & Speed, F. M. (2000). Interpreting the Four Types of Sums of Squares in SPSS. Null. https://files.eric.ed.gov/fulltext/ED448170.pdf
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19205033890003606?auth=SAML

Suzuki, & Yamamoto, Y. (2021). Characterizing the Influence of Confirmation Bias on Web Search Behavior. Frontiers in Psychology, 12, 771948–771948. https://doi.org/10.3389/fpsyg.2021.771948
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19205051480003606?auth=SAML

Krueger, & Heck, P. R. (2017). The heuristic value of p in inductive statistical inference. Frontiers in Psychology, 8, 908–908. https://doi.org/10.3389/fpsyg.2017.00908
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19205056940003606?auth=SAML

Antônio Alves Tôrres Fernandes, Dalson Britto Figueiredo Filho, Enivaldo Carvalho da Rocha, & Willber da Silva Nascimento. (2021). Read this paper if you want to learn logistic regression. Revista de Sociologia e Política, 28(74). https://doi.org/10.1590/1678-987320287406en https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19242328770003606?auth=SAML

Dmitrienko, & D’Agostino, R. B. (2018). Multiplicity Considerations in Clinical Trials. The New England Journal of Medicine, 378(22), 2115–2122. https://doi.org/10.1056/NEJMra1709701
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19242340540003606?auth=SAML

Félix, & Menezes, A. F. B. (2018). Comparisons of ten corrections methods for t-test in multiple comparisons via Monte Carlo study (Vol. 11, Issue 1, pp. 74–91) [Data set]. https://doi.org/10.1285/I20705948V11N1P74
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19242387030003606?auth=SAML

Huang. (2022). Alternatives to logistic regression models in experimental studies. The Journal of Experimental Education, 90(1), 213–228.https://doi.org/10.1080/00220973.2019.1699769  https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19247141110003606?auth=SAML 

null

Watch a recording of the lecture - Week 3 (3 February  2022) - (Raven only)

Slides in PDF - Week 3 (Raven only)

Class 4: 10 February  2022 at 4pm in Psychology Classroom, Downing Site

Generalisability and parsimony: the df-ratio and multi-versing

Recommended Reading 

Yarkoni. (2020). The generalizability crisis. The Behavioral and Brain Sciences, 1–37. https://doi.org/10.1017/S0140525X20001685
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19255990690003606?auth=SAML

Daniel Lakens. (2021). There is no generalizability crisis. Pre-Print. https://doi.org/10.31234/osf.io/tm8jy
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19256025160003606?auth=SAML

Grahek, Schaller, M., & Tackett, J. L. (2021). Anatomy of a Psychological Theory: Integrating Construct-Validation and Computational-Modeling Methods to Advance Theorizing. Perspectives on Psychological Science, 16(4), 803–815. https://doi.org/10.1177/1745691620966794
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19256034940003606?auth=SAML

Brittany I Davidson. (2021). Measurement practices exacerbate the generalizability crisis: Novel digital measures can help. Pre-Print. https://doi.org/10.31234/osf.io/8abzy
https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19256053790003606?auth=SAML

Carpenter, & Bithell, J. (2000). Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19(9), 1141–1164. https://doi.org/10.1002/(SICI)1097-0258(20000515)19:9<1141::AID-SIM479>3.0.CO;2-F  https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19263494130003606?auth=SAML 

null

Watch a recording of the lecture - Week 4 (10 February  2022) - (Raven only)

Slides in PDF - Week 4 (Raven only)

Class 5: 17 February 2022 at 4pm in Psychology Classroom, Downing Site

Data-reduction: principal components, factor analysis and introduction to SEM#

Recommended Reading 

Thalheimer, & Cook, S. (2009). How to calculate effect sizes from published research: A simplified methodology. N/A. https://www.researchgate.net/publication/253642160_How_to_calculate_effe... 

Patel, Burford, B., & Ioannidis, J. P. . (2015). Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. Journal of Clinical Epidemiology, 68(9), 1046–1058. https://doi.org/10.1016/j.jclinepi.2015.05.029 https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/19378387530003606?auth=SAML

Lewis. (n.d.). An Introduction to Classification and Regression Tree (CART) Analysis. N/A. https://www.researchgate.net/publication/240719582_An_Introduction_to_Cl... 

 

null

Watch a recording of the lecture - Week 5 (17 February 2022) - (Raven only)

Slides in PDF - Week 5 (Raven only)

Class 6: 24 February 2022 at 4pm in Psychology Classroom, Downing Site

Structural equation modelling and confirmatory factor analysis

 

Recommended Reading

Wang, & Rhemtulla, M. (2021). Power Analysis for Parameter Estimation in Structural Equation Modeling: A Discussion and Tutorial. Advances in Methods and Practices in Psychological Science, 4(1), 251524592091825–. https://doi.org/10.1177/2515245920918253 https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/20160280050003606?auth=SAML

Deng, Yang, M., & Marcoulides, K. M. (2018). Structural equation modeling with many variables: A systematic review of issues and developments. Frontiers in Psychology, 9, 580–580. https://doi.org/10.3389/fpsyg.2018.00580 https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/20160302040003606?auth=SAML

Ringle, Sarstedt, M., Mitchell, R., & Gudergan, S. P. (2020). Partial least squares structural equation modeling in HRM research. International Journal of Human Resource Management, 31(12), 1617–1643. https://doi.org/10.1080/09585192.2017.1416655 https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/20160312510003606?auth=SAML

 

Joel E. Collier. (2020). Applied structural equation modeling using AMOS : basic to advanced techniques / Joel E. Collie. Routledge. https://cam.alma.exlibrisgroup.com/leganto/public/44CAM_INST/citation/20279102310003606?auth=SAML 

null

Watch a recording of the lecture - Week 6 (24 February 2022) - (Raven only)

Slides in PDF - Week 6 (Raven only)

#It is hoped to be able hold again with sufficient physical isolation, an SEM workshop in the department -- later in the Lent term or in the Easter term.