Importance: Better patient management can reduce emergency department (ED) use. Performance measures should reward plans for reducing utilization by predictably high-use patients, rather than rewarding plans that shun them. Objective: The objective of this study was to develop a quality measure for ED use for people diagnosed with serious mental illness or substance use disorder, accounting for both medical and social determinants of health (SDH) risks. Design: Regression modeling to predict ED use rates using diagnosis-based and SDH-augmented models, to compare accuracy overall and for vulnerable populations. Setting: MassHealth, Massachusetts’ Medicaid and Children’s Health Insurance Program. Participants: MassHealth members ages 18–64, continuously enrolled for the calendar year 2016, with a diagnosis of serious mental illness or substance use disorder. Exposures: Diagnosis-based model predictors are diagnoses from medical encounters, age, and sex. Additional SDH predictors describe housing problems, behavioral health issues, disability, and neighborhood-level stress. Main Outcome and Measures: We predicted ED use rates: (1) using age/sex and distinguishing between single or dual diagnoses; (2) adding summarized medical risk (DxCG); and (3) further adding social risk (SDH). Results: Among 144,981 study subjects, 57% were women, 25% dually diagnosed, 67% White/non-Hispanic, 18% unstably housed, and 37% disabled. Utilization was higher by 77% for those dually diagnosed, 50% for members with housing problems, and 18% for members living in the highest-stress neighborhoods. SDH modeling predicted best for these high-use populations and was most accurate for plans with complex patients. Conclusion: To set appropriate benchmarks for comparing health plans, quality measures for ED visits should be adjusted for both medical and social risks.