Title: A Review of Generalizability and Transportability
Author(s) and Year: Irina Degtiar and Sherri Rose, 2023.
Journal: Annual Review of Statistics and Its Application (Open Access)
In an increasingly data-driven world, the ability to draw accurate conclusions from research and apply them to a broader context is essential. Enter generalizability and transportability, two critical concepts researchers consider when assessing the applicability of their findings to different populations and settings. In their article “A Review of Generalizability and Transportability,” published in the Annual Review of Statistics and Its Application in 2023, Irina Degtiar and Sherri Rose delve into these critical concepts, providing valuable insights into their nuances and potential applications.
Generalizability: One Size Fits All?
Generalizability refers to the extent to which research findings can be applied to populations beyond the study sample. In other words, can the results of a study be generalized to different groups of people or contexts? In their paper, Degtiar and Rose discuss the importance of generalizability in research, highlighting its role in ensuring that results apply to a broader population.
According to Degtiar and Rose, “When assessing causal effects, determining the target population to which the results are intended to generalize is a critical decision.” They go on to explain that both randomized and observational studies have strengths and limitations when it comes to estimating causal effects in a target population. While estimates from randomized data may have internal validity, they are often not representative of the target population. On the other hand, observational data may better reflect the target population and be more likely to have external validity but are subject to potential bias due to unmeasured confounding.
Figure 1. The authors illustrate the differences between biases that affect the accuracy of research findings within the study population (internal validity) and biases that affect the applicability of research findings to the broader target population (external validity) and how these biases relate to the populations being studied and analyzed. Image reproduced without modification from Figure 1 of Degtiar and Rose, 2023.
Transportability: Taking the Findings on a Trip
Transportability refers to the ability to apply research findings from one context to another. This concept is essential when considering the applicability of results across different settings or populations. In their work, Degtiar and Rose explore the idea of transportability in depth, discussing its importance in ensuring that research findings can be applied in different contexts.
According to Degtiar and Rose, “observational data may better reflect the target population, and hence be more likely to have external validity.” They explain that while much of the causal inference literature has focused on addressing internal validity bias (i.e., bias due to unmeasured confounding), both internal and external validity are necessary for unbiased estimates in a target population. To address external validity bias, they present a synthesis of approaches for generalizability and transportability as well as tests for treatment effect heterogeneity and differences between study and target populations.
Figure 2. The authors outline a structured approach for evaluating and addressing any biases that may affect the applicability of research findings to populations beyond the study sample (external validity) after the data has been collected. Image reproduced without modification from Figure 2 of Degtiar and Rose, 2023.
By considering these factors, researchers can improve their ability to transport their results from one context to another. This is particularly important when considering the applicability of research findings across different settings or populations.
Future Directions: The Road Ahead
In their review, Degtiar and Rose suggest that future research should focus on synthesizing approaches for generalizability and transportability. Developing new methods for assessing these concepts can improve researchers’ ability to apply their results in different contexts. Additionally, further exploration into treatment effect heterogeneity could help researchers better understand how their conclusions may vary across populations.
Degtiar and Rose’s article provides a valuable foundation for future research into generalizability and transportability. Building on their work can improve our understanding of these concepts and their importance in ensuring research findings are applicable in various contexts. By bridging the gap between research and its applicability to different populations, researchers can make a significant impact by ensuring their results are relevant to a broad audience.
Edited by Alyssa Columbus
Cover image credit: Marvin Kuhn on Unsplash
Figures used under CC BY 4.0