Monte Carlo Simulation-Based Recommendations for Reducing the Risk of Bias in Multilevel Models Introduced by Mis-Measuring the Neighborhood

Claudia L. Nau, Johns Hopkins Bloomberg School of Public Health

This paper develops recommendations on how to minimize the risk and size of bias in multilevel models if the scale of the neighborhood effect is unknown or data is not available to model the neighborhood-scale properly. These recommendations are derived from a comprehensive set of “hybrid” simulation experiments that combine a Monte Carlo simulation approach with census information at the block-, block-group-, and tract-level to generate plausible data scenarios. Results suggest that, all else equal, the larger scale is a safer choice. Furthermore, results caution against the use and interpretation of level- two variances and intra-class-correlation coefficients. Three-level models should be estimated if possible, and modelfit statistics should be used to assess the scale, or, combination of scales, at which neighborhood effects operate. Results also suggest that the scale needs to be grossly mis-measured in order to cause significant bias.