Inspiration

At its core, psychometrics offers a nuanced approach to measuring (and thereby understanding) mental capacities and processes. Psychometric methods allow for accurate assessment of individual differences in cognitive skills, along with tools for theorizing about and testing psychological constructs. Why do cognitive scientists not use psychometric models? We suggest that the underutilization of modern psychometric methods in cognitive science practice today is largely a result of unfamiliarity with psychometrics and how to apply it in a given domain. This workshop will help bridge this gap, explaining how psychometric methods can enhance cognitive science research, and showcasing example applications of these methods in a variety of domains. By bringing together psychometric experts and practitioners, this workshop will serve as a catalyst for cognitive scientists who seek to produce highly reliable and valid results, and who are also interested in creating testable theoretical frameworks (Smaldino, 2020; Guest & Martin, 2021).

In particular, we seek to explore what psychometric methods can bring to cognitive science. There are at least three key fronts which can be advanced with psychometrics: (1) understanding individual differences, (2) understanding variability across items, and (3) constructing formal models. The first front is clearly an important goal in cognitive science, because it allows for exploration of the mechanisms and development of various cognitive functions and abilities. For example, capturing individual-level variability across a range of cognitive abilities allows us to study how these abilities change over time, and whether they may lie along developmental cascades (Oakes & Rakison, 2019), whereby the development of one ability causes further downstream changes in another. Reciprocal interactions during the development of these abilities reflects the theory of mutualism (van der Maas et al., 2006), which provides a model for why many cognitive abilities are positively correlated. Understanding individual differences allows for the more careful examination of such dynamically interactive development, since the particular temporal characteristics are likely to vary substantially among individuals.

Psychometric methods also allow for the study of variability across items. Unlike classical test theory, which assumes that all items contribute equally to the measurement of any particular latent factor, more sophisticated psychometric models such as item response-theoretic (IRT) models incorporate the observation that items themselves may differ. These approaches have enabled innovations in measurement tools—for example, careful, parametrized selection of items using adaptive testing allows for an accurate measurement of an individual’s ability while using markedly shorter assessments (e.g., Kachergis, Marchman, Dale, et al., 2022). Furthermore, psychometric approaches can shed light on whether particular items exhibit measurement equivalence across different subgroups (e.g., across cultures or genders), and can also elucidate the relationship between different items (e.g., using factor analysis).

Finally, psychometric models can serve as the basis for defining formal, extensible theories – which psychology largely lacks (Muthukrishna & Henrich, 2019). Formalizing theories in a model requires making assumptions explicit, which clarifies thinking, allows the theory to generate testable predictions, and enables direct comparisons to other theories (Smaldino, 2020; Guest & Martin, 2021). Psychometric models can formally unite studies of different aspects of a phenomenon, for example connecting per-child language input measures to per-child and per-word uptake (Kachergis, Marchman, & Frank, 2022).

References