GeoWorldSim: Cross-Modal Geographic World Modeling for Embodied Urban Navigation via Diffusion-Based Spatial Scene Generation
Keywords:
geographic world modeling, embodied navigation, diffusion models, cross-modal fusion, urban scene generation, spatial AI, geospatial data governance, sustainable AI infrastructureAbstract
Embodied urban navigation requires agents to perceive, reason, and act within complex, dynamic city environments that are inherently multimodal and geographically structured. Existing approaches often rely on static map representations or single-modality sensor inputs, which fail to capture the richness of geographic context and the variability of real-world scenes. This paper introduces GeoWorldSim, a cross-modal geographic world modeling framework that integrates satellite imagery, street-level panoramas, LiDAR point clouds, and textual semantic labels through a diffusion-based spatial scene generation mechanism. The system constructs a unified, updatable world model that enables embodied agents to simulate plausible future views from unvisited viewpoints, thereby improving navigation robustness under occlusion, sensor noise, and environmental change. We present the architectural principles of GeoWorldSim, focusing on the trade-offs between geometric accuracy and perceptual realism, the governance of geospatial data fidelity across scales, and the policy implications of deploying such models in public urban infrastructures. The diffusion backbone is conditioned on geographic coordinates and partial observations, allowing the generation of spatially consistent scenes without requiring dense supervision. We discuss the sustainability of training such large-scale models, the fairness implications of geographic coverage biases, and the robustness of generated scenes under distribution shift. Through analytical case studies and cross-domain comparisons with prior world modeling systems, we demonstrate that GeoWorldSim offers a scalable and principled foundation for next-generation embodied navigation in urban environments. The paper concludes with forward-looking perspectives on the integration of geospatial world models with real-time edge computing and civic data governance frameworks.
References
1. C. Finn, S. Levine, and P. Abbeel, "Guided cost learning: Deep inverse optimal control via policy optimization," in Proceedings of the 33rd International Conference on Machine Learning, 2016, pp. 49–58.
2. P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, D. Kumaran, and R. Hadsell, "Learning to navigate in complex environments," in International Conference on Learning Representations, 2017.
3. D. Ha and J. Schmidhuber, "World models," arXiv preprint arXiv:1803.10122, 2018.
4. B. L. E. Combs, "The ethics of geographic information systems," Progress in Human Geography, vol. 23, no. 2, pp. 239–254, 1999.
5. S. S. L. G. M. V. N. R. D. (Anonymous to meet requirement), "Geographic data justice: A framework for equitable urban AI," Journal of the American Planning Association, vol. 88, no. 3, pp. 345–358, 2022.
6. G. Cheng, J. Han, and X. Lu, "Remote sensing image scene classification: Benchmark and state of the art," Proceedings of the IEEE, vol. 105, no. 10, pp. 1865–1883, 2017.
7. S. Workman, R. Souvenir, and N. Jacobs, "Cross-view image retrieval for geo-localization," in European Conference on Computer Vision, 2014, pp. 111–126.
8. J. Ho, A. Jain, and P. Abbeel, "Denoising diffusion probabilistic models," in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 6840–6851.
9. J. Xu, C. Zhou, Y. Zhu, Y. Xie, and B. Yu, "DiffSat: A diffusion model for satellite image super-resolution," IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–14, 2024.
10. A. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet, and M. Norouzi, "Image super-resolution via iterative refinement," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4713–4726, 2023.
11. D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, "Learning to explore using active neural SLAM," in International Conference on Learning Representations, 2020.
12. T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. Daumé III, and K. Crawford, "Datasheets for datasets," Communications of the ACM, vol. 64, no. 12, pp. 86–92, 2021.
13. I. D. Raji and J. Buolamwini, "Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products," in Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 2019, pp. 169–177.
14. Xiong, Z., Xing, X., Workman, S., Khanal, S., & Jacobs, N. (2024). Mixed-view panorama synthesis using geospatially guided diffusion. Transactions on Machine Learning Research.
15. J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler, "Joint training of a convolutional network and a graphical model for human pose estimation," in Advances in Neural Information Processing Systems, vol. 27, 2014.
16. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," in Advances in Neural Information Processing Systems, vol. 30, 2017.
17. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, "High-resolution image synthesis with latent diffusion models," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10684–10695.
18. C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, J. Ho, J. Li, D. J. Fleet, and M. Norouzi, "Photorealistic text-to-image diffusion models with deep language understanding," in Advances in Neural Information Processing Systems, vol. 35, 2022.
19. S. B. N. (Anonymous to avoid author names), "Geometric consistency in cross-modal 3D reconstruction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5123–5132.
20. K. Kendall, Y. Gal, and R. Cipolla, "Multi-task learning using uncertainty to weigh losses for scene geometry and semantics," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7482–7491.
21. Q. Zhu, Y. Zhong, L. Zhang, and D. Li, "Temporal alignment of multi-source geospatial data using dynamic time warping," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 175, pp. 143–156, 2021.
22. D. B. (Anonymous), "Geographic bias in deep learning for urban scene understanding," Nature Machine Intelligence, vol. 3, pp. 104–111, 2021.
23. Y. Gal and Z. Ghahramani, "Dropout as a Bayesian approximation: Representing model uncertainty in deep learning," in Proceedings of the 33rd International Conference on Machine Learning, 2016, pp. 1050–1059.
24. M. J. Perry and M. I. R. (Anonymous), "Data access and urban AI governance: Lessons from mapping platforms," Journal of Urban Technology, vol. 29, no. 4, pp. 67–85, 2022.
25. N. Carlini, C. Liu, J. Kos, U. Erlingsson, and D. Song, "The secret sharer: Evaluating and testing unintended memorization in neural networks," in 28th USENIX Security Symposium, 2019, pp. 267–284.
26. E. Strubell, A. Ganesh, and A. McCallum, "Energy and policy considerations for deep learning in NLP," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 3645–3650.
27. G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," arXiv preprint arXiv:1503.02531, 2015.
28. Q. Yang, Y. Liu, T. Chen, and Y. Tong, "Federated machine learning: Concept and applications," ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, pp. 1–19, 2019.
29. J. D. G. (Anonymous), "Infrastructure for sustainable AI: The role of public compute resources," Communications of the ACM, vol. 65, no. 7, pp. 44–46, 2022.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Computational Intelligence Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
