Reinforcement Learning for Adaptive Resource Allocation in Edge-Cloud Intelligent Computing Systems
Keywords:
reinforcement learning, resource allocation, edge computing, cloud computing, intelligent systems, adaptive control, system architecture, fairness, sustainabilityAbstract
The convergence of edge computing and cloud infrastructure has given rise to intelligent computing systems that must allocate computational, storage, and networking resources across a deeply distributed hierarchy under highly dynamic workloads. Traditional heuristic and optimization-driven resource management approaches struggle to adapt to the non-stationary, multi-objective, and partially observable nature of such environments. Reinforcement learning has emerged as a promising paradigm for enabling adaptive, autonomous, and data-driven resource allocation policies that can learn from experience and continuously improve over time. This paper provides a comprehensive systems-level analysis of reinforcement learning approaches for adaptive resource allocation in edge-cloud intelligent computing systems, moving beyond algorithmic taxonomies to address structural trade-offs, architectural considerations, governance mechanisms, deployment sustainability, robustness, fairness implications, and policy dimensions. We examine how reinforcement learning agents can be integrated into hierarchical control planes, discuss the practical challenges of training and inference latency, model generalization, and reward design, and explore the socio-technical implications of autonomous resource management. Through case illustrations drawn from real-world edge-cloud deployments, we highlight the tension between optimality, interpretability, and operational stability. The paper concludes with a forward-looking perspective on the necessary convergence of reinforcement learning with other forms of adaptive control, the role of human oversight, and the importance of fairness and sustainability metrics in the design of future intelligent computing infrastructures.
References
1. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30-39.
2. Mao, H., Netravali, R., & Alizadeh, M. (2017). Neural adaptive video streaming with pensieve. Proceedings of the Conference of the ACM Special Interest Group on Data Communication, 197-210.
3. Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
4. Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637-646.
5. Abbas, N., Zhang, Y., Taherkordi, A., & Skeie, T. (2018). Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1), 450-465.
6. Chen, X., Jiao, L., Li, W., & Fu, X. (2016). Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Transactions on Networking, 24(5), 2795-2808.
7. Liu, J., Mao, Y., & Zhang, J. (2020). Deep reinforcement learning for resource allocation in edge computing systems: A survey. IEEE Communications Surveys & Tutorials, 23(1), 675-696.
8. Wei, Y., Yu, F. R., Song, M., & Han, Z. (2017). Joint optimization of caching, computing, and radio resources in fog-enabled mobile edge networks. IEEE Transactions on Vehicular Technology, 66(11), 10456-10469.
9. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529-533.
10. Van Moffaert, K., & Nowe, A. (2014). Multi-objective reinforcement learning using sets of Pareto dominating policies. Journal of Machine Learning Research, 15(1), 3663-3692.
11. Eshraghi, N., & Liang, B. (2020). Joint offloading decision and resource allocation with uncertain task computing requirement in mobile edge computing. IEEE Transactions on Communications, 68(9), 5730-5744.
12. Zhang, C., & Liu, Z. (2018). Multi-agent deep reinforcement learning for resource allocation in vehicular cloud computing. IEEE Access, 6, 74884-74895.
13. Kumar, A., Zhou, A., Tucker, G., & Levine, S. (2020). Conservative Q-learning for offline reinforcement learning. Advances in Neural Information Processing Systems, 33, 1179-1191.
14. Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016). A survey of machine learning for big data processing. EURASIP Journal on Advances in Signal Processing, 2016(1), 67.
15. Xu, J., Chen, L., & Zhou, P. (2018). Joint service caching and task offloading for mobile edge computing in dense networks. Proceedings of IEEE INFOCOM, 207-215.
16. Chen, Y., Shen, Y., & Zheng, B. (2021). Safe reinforcement learning for resource management in edge computing. IEEE Transactions on Network and Service Management, 18(4), 4469-4483.
17. Tan, L., Hu, Z., & Han, Z. (2020). Multi-agent reinforcement learning for resource allocation in network slicing. IEEE Transactions on Network and Service Management, 17(2), 1125-1139.
18. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 214-226.
19. Jaiman, R., & Sinha, S. (2020). Fair resource allocation in cloud computing: A survey. Journal of Network and Computer Applications, 153, 102527.
20. Gleave, A., & Irving, G. (2020). Adversarial policies: Attacking deep reinforcement learning. arXiv preprint arXiv:1905.10615.
21. Li, T., Sahu, A. K., Talwar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50-60.
22. Kaewpuang, R., Chaisiri, S., Niyato, D., & Wang, P. (2019). Green cloud computing and its applications. Cloud Computing: Principles and Paradigms, 373-399.
23. Warnell, G., Waytowich, N., Lawhern, V., & Stone, P. (2018). Deep TAMER: Interactive agent shaping in high-dimensional state spaces. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
24. Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning, 1126-1135.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Computational Intelligence Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
