An Integrated Framework of Global Sensitivity Analysis and Calibration for Spatially Explicit ABMs

In the past we have written about the challenges of validation and to some extent the calibration of agent-based models but never really went into much detail about the calibration process. To this end, Jeon-Young Kang, Alexander Michels, Jared Aldstadt, Shaowen Wang and myself recently had a paper published in Transactions in GIS entitled “An Integrated Framework of Global Sensitivity Analysis and Calibration for Spatially Explicit Agent-Based Models.” In the paper we have present an integrated framework for global sensitivity analysis and calibration (GSA-CAL), and then apply the framework to a spatially explicit agent-based model of influenza transmission as a case study in the city of Miami, FL. If this sounds of interest, below you can read the abstract to the paper, see some of the figures from the paper including the general workflow and some of the results. At the bottom of the post you can find the full citation and a link to the paper.

Calibration of agent-based models (ABMs) is a major challenge due to the complex nature of the systems being modeled, the heterogeneous nature of geographical regions, the varying effects of model inputs on the outputs, and computational intensity. Nevertheless, ABMs need to be carefully tuned to achieve the desirable goal of simulating spatiotemporal phenomena of interest, and a well-calibrated model is expected to achieve an improved understanding of the phenomena. To address some of the above challenges, this article proposes an integrated framework of global sensitivity analysis (GSA) and calibration, called GSA-CAL. Specifically, variance-based GSA is applied to identify input parameters with less influence on differences between simulated outputs and observations. By dropping these less influential input parameters in the calibration process, this research reduces the computational intensity of calibration. Since GSA requires many simulation runs, due to ABMs’ stochasticity, we leverage the high-performance computing power provided by the advanced cyberinfrastructure. A spatially explicit ABM of influenza transmission is used as the case study to demonstrate the utility of the framework. Leveraging GSA, we were able to exclude less influential parameters in the model calibration process and demonstrate the importance of revising local settings for an epidemic pattern in an outbreak.

Workflow of global sensitivity analysis and calibration

Study area

Daily activity-based contact network construction

Results from calibration distance-based mobility (DBM): (a) simulated results from an initial model; (b) simulated results from the calibrated model; and (c) sum of RMSE

Full Reference:

Kang, J-Y., Michels, A., Crooks, A.T., Aldstradt, J. and Wang, S. (2021), An Integrated Framework of Global Sensitivity Analysis and Calibration for Spatially Explicit Agent-Based Models, Transactions in GIS. https://doi.org/10.1111/tgis.12837 (pdf)