Learning Complementary Spectral and Structural Features from Hyperspectral and LiDAR Data

Authors

  • Fernando Chambers School of Information Technology, University of Cincinnati, Cincinnati, OH, USA.
  • Ananya Pillai Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, USA.
  • Rowan D. Gregory Department of Computer Science, University of New Hampshire, Durham, NH, USA.
  • Troy Schwartz School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA.

Keywords:

hyperspectral imaging, LiDAR, feature fusion, deep learning, remote sensing infrastructure, socio-technical systems, fairness, sustainability

Abstract

The fusion of hyperspectral imaging and Light Detection and Ranging (LiDAR) data has emerged as a powerful paradigm for land cover classification, environmental monitoring, and urban scene understanding. Hyperspectral sensors capture rich spectral signatures across hundreds of narrow contiguous bands, while LiDAR provides precise three-dimensional structural information about the Earth’s surface and vegetation canopy. Learning complementary features from these two modalities requires careful architectural design to balance spectral detail with spatial and vertical resolution. This paper presents a system-level analysis of the methodologies, trade-offs, and governance implications associated with spectral-structural feature learning. We examine the architectural choices in deep neural network frameworks that integrate hyperspectral and LiDAR data, focusing on early, intermediate, and late fusion strategies. Key considerations include computational efficiency, robustness to sensor noise, scalability to large geographic extents, and the interpretability of learned representations. We further discuss the deployment of such systems in real-world infrastructures such as precision agriculture, forestry management, and urban planning. Sustainability concerns are addressed through the lens of energy consumption during training and inference, as well as the environmental impact of high-resolution data acquisition. Fairness and policy implications arise from potential biases in training data distributions and the ethical use of remote sensing for surveillance and resource allocation. By connecting technical design choices with broader socio-technical outcomes, this paper argues that future research must prioritize transparent, equitable, and governance-aware fusion architectures. The findings underscore the need for multi-stakeholder collaboration to ensure that hyperspectral-LiDAR fusion technologies serve societal benefit while minimizing unintended harms.

References

1. Ghamisi, P., Maggiori, E., Li, S., Souza, R., Tarablaka, Y., Moser, G., & Benediktsson, J. A. (2019). New frontiers in spectral-spatial hyperspectral image classification: The latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning. IEEE Geoscience and Remote Sensing Magazine, 7(3), 14–37. https://doi.org/10.1109/MGRS.2019.2918258

2. Kaasalainen, S., Lindberg, E., & Vastaranta, M. (2021). Laser scanning for forest inventory: Principles, advances, and applications. Current Forestry Reports, 7(4), 209–224. https://doi.org/10.1007/s40725-021-00147-w

3. Dechesne, C., Lefèvre, S., & Weber, C. (2019). Hyperspectral and LiDAR data fusion for urban land cover classification. International Journal of Remote Sensing, 40(5–6), 1903–1926. https://doi.org/10.1080/01431161.2018.1533659

4. Hong, D., Gao, L., Yao, J., Zhang, B., Plaza, A., & Chanussot, J. (2021). Graph convolutional networks for hyperspectral image classification with multi-scale superpixels. IEEE Transactions on Geoscience and Remote Sensing, 59(5), 4188–4201. https://doi.org/10.1109/TGRS.2020.3019530

5. Yang, J. X., Wang, J., Li, Z., Sui, C., Long, Z., & Zhou, J. (2025). HSLiNets: Evaluating Band Ordering Strategies in Hyperspectral and LiDAR Fusion. IEEE Geoscience and Remote Sensing Letters.

6. Dalponte, M., Bruzzone, L., & Gianelle, D. (2008). Fusion of hyperspectral and LIDAR remote sensing data for classification of complex forest areas. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1416–1427. https://doi.org/10.1109/TGRS.2007.912727

7. Plaza, A., Benediktsson, J. A., Boardman, J. W., Brazile, J., Bruzzone, L., Camps-Valls, G., Chanussot, J., Fauvel, M., Gamba, P., Gualtieri, A., Marconcini, M., Tilton, J. C., & Trianni, G. (2009). Recent advances in techniques for hyperspectral image processing. Remote Sensing of Environment, 113, S110–S122. https://doi.org/10.1016/j.rse.2008.07.006

8. Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems, 30, 5099–5108.

9. Xu, Y., Du, B., Zhang, L., & Zhang, L. (2018). A hybrid deep learning model for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(8), 4805–4820. https://doi.org/10.1109/TGRS.2018.2838641

10. Ghamisi, P., Höfle, B., & Zhu, X. X. (2017). Hyperspectral and LiDAR data fusion: Wavelet-based collaborative representation approach. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(6), 2752–2763. https://doi.org/10.1109/JSTARS.2017.2676887

11. Li, S., Song, W., Fang, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2019). Deep learning for hyperspectral image classification: An overview. IEEE Transactions on Geoscience and Remote Sensing, 57(9), 6690–6709. https://doi.org/10.1109/TGRS.2019.2907932

12. Zhang, J., Li, W., & Tao, R. (2020). Graph neural network for hyperspectral and LiDAR data fusion. IEEE Geoscience and Remote Sensing Letters, 17(11), 1953–1957. https://doi.org/10.1109/LGRS.2019.2957099

13. Yuan, Q., Shen, H., Li, T., Li, Z., Li, S., Jiang, Y., Xu, H., Tao, D., & Zhang, L. (2020). Deep learning in environmental remote sensing: Achievements and challenges. Remote Sensing of Environment, 241, 111716. https://doi.org/10.1016/j.rse.2020.111716

14. Tuia, D., Persello, C., & Bruzzone, L. (2016). Domain adaptation for the classification of remote sensing data: An overview of recent advances. IEEE Geoscience and Remote Sensing Magazine, 4(2), 41–57. https://doi.org/10.1109/MGRS.2016.2548504

15. Mesquita, D. P., & Cardoso, J. S. (2021). A survey on bias mitigation in machine learning. ACM Computing Surveys, 54(3), 1–37. https://doi.org/10.1145/3442322

16. Shi, S., Wang, X., & Li, H. (2019). PointRCNN: 3D object proposal generation and detection from point cloud. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–779.

17. Lian, Z., Li, Y., Tao, D., & Liu, W. (2022). Adaptive spectral band ordering for hyperspectral image classification. IEEE Transactions on Image Processing, 31, 2145–2158. https://doi.org/10.1109/TIP.2022.3155672

18. Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning, 97, 6105–6114.

19. Chen, Y., Jiang, H., Li, C., Jia, X., & Ghamisi, P. (2020). Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54(10), 6232–6251. https://doi.org/10.1109/TGRS.2016.2573222

20. Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Müller, K.-R. (2016). Evaluating the visualization of what a deep neural network has learned. IEEE Transactions on Neural Networks and Learning Systems, 28(11), 2660–2673. https://doi.org/10.1109/TNNLS.2016.2599820

21. Kwan, C., Ayhan, B., & Budavari, B. (2020). Deep learning for land use classification: Balancing accuracy and fairness. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, 1326–1329. https://doi.org/10.1109/IGARSS39084.2020.9324137

22. Elgendy, N., & Frey, J. (2022). Ethical considerations in the use of remote sensing data for urban surveillance. GeoJournal, 87(3), 1893–1907. https://doi.org/10.1007/s10708-020-10303-3

23. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. https://doi.org/10.1145/3287560.3287598

24. Han, S., Mao, H., & Dally, W. J. (2016). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. In International Conference on Learning Representations.

25. Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems, 30, 5574–5584.

Downloads

Published

2026-05-15

How to Cite

Fernando Chambers, Ananya Pillai, Rowan D. Gregory, & Troy Schwartz. (2026). Learning Complementary Spectral and Structural Features from Hyperspectral and LiDAR Data. Computational Intelligence Systems, 4(1). Retrieved from https://scivexus.org/index.php/CIS/article/view/364

Issue

Vol. 4 No. 1 (2026): Computational Intelligence Systems

Section

Articles

License

Copyright (c) 2026 Computational Intelligence Systems

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.