Lightweight Federated Multi-Agent Learning for Behind-the-Meter Resource Coordination: A Survey
Keywords:
Edge intelligence, federated reinforcement learning, energy management, multi-agent reinforcement learning, privacy controlAbstract
Small devices that do not reveal private data are required to coordinate behind-the-meter solar systems, batteries, electric cars, and flexible loads; yet, these systems might be advantageous to distribution grids and homeowners alike. This work is an investigation of lightweight federated multi-agent learning for this purpose. From recent literature survey, and categorizes the field based on learning topology, model-efficiency mechanism, coordinated resource, and evaluation realism. Thanks to the evidence, it is clear that personalized and sparse federated reinforcement learning can be deployed more easily than the approach of full-model synchronous training, and that the majority of the multi-agent literature continues to focus on simulations. Safe coordination among non-independent and identically distributed (non-IID) households under intermittent communication, with limited memory and latency budgets, is the central open problem. Thus, the deployment-oriented benchmark and research agenda are suggested.
References
[1] J. R. Vázquez-Canteli and Z. Nagy, “Reinforcement learning for demand response: A review of algorithms and modeling techniques,” Applied Energy, vol. 235, pp. 1072-1089, 2019, doi: 10.1016/j.apenergy.2018.11.002.
[2] A. T. D. Perera and P. Kamalaruban, “Applications of reinforcement learning in energy systems,” Renewable and Sustainable Energy Reviews, vol. 137, Art. no. 110618, 2021, doi: 10.1016/j.rser.2020.110618.
[3] H. B. Gooi, T. Wang, and Y. Tang, “Edge intelligence for smart grid: A survey on application potentials,” CSEE Journal of Power and Energy Systems, vol. 9, no. 5, pp. 1623-1640, 2023, doi: 10.17775/CSEEJPES.2022.02210.
[4] A. Grataloup, S. Jonas, and A. Meyer, “A review of federated learning in renewable energy applications: Potential, challenges, and future directions,” Energy and AI, vol. 17, Art. no. 100375, 2024, doi: 10.1016/j.egyai.2024.100375.
[5] S. Lee and D.-H. Choi, “Federated reinforcement learning for energy management of multiple smart homes with distributed energy resources,” IEEE Transactions on Industrial Informatics, vol. 18, no. 1, pp. 488-497, 2022, doi: 10.1109/TII.2020.3035451.
[6] S. Lee, L. Xie, and D.-H. Choi, “Privacy-preserving energy management of a shared energy storage system for smart buildings: A federated deep reinforcement learning approach,” Sensors, vol. 21, no. 14, Art. no. 4898, 2021, doi: 10.3390/s21144898.
[7] J.-H. Lee, J.-Y. Park, H.-S. Sim, and H.-S. Lee, “Multi-residential energy scheduling under time-of-use and demand charge tariffs with federated reinforcement learning,” IEEE Transactions on Smart Grid, vol. 14, no. 6, pp. 4360-4372, 2023, doi: 10.1109/TSG.2023.3251956.
[8] Y. Chu, Z. Wei, X. Fang, S. Chen, and Y. Zhou, “A multiagent federated reinforcement learning approach for plug-in electric vehicle fleet charging coordination in a residential community,” IEEE Access, vol. 10, pp. 98535-98548, 2022, doi: 10.1109/ACCESS.2022.3206020.
[9] F. Rezazadeh and N. Bartzoudis, “A federated DRL approach for smart micro-grid energy control with distributed energy resources,” Proc. IEEE 27th Int. Workshop Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), pp. 108-114, 2022, doi: 10.1109/CAMAD55695.2022.9966919.
[10] J. Gao, W. Wang, F. Nikseresht, V. G. Rajan, and B. Campbell, “PFDRL: Personalized federated deep reinforcement learning for residential energy management,” Proc. 52nd Int. Conf. Parallel Processing (ICPP), pp. 402-411, 2023, doi: 10.1145/3605573.3605641.
[11] D. Qiu, J. Xue, T. Zhang, J. Wang, and M. Sun, “Federated reinforcement learning for smart building joint peer-to-peer energy and carbon allowance trading,” Applied Energy, vol. 333, Art. no. 120526, 2023, doi: 10.1016/j.apenergy.2022.120526.
[12] T. Wang and Z. Y. Dong, “Adaptive personalized federated reinforcement learning for multiple-ESS optimal market dispatch strategy with electric vehicles and photovoltaic power generations,” Applied Energy, vol. 365, Art. no. 123107, 2024, doi: 10.1016/j.apenergy.2024.123107.
[13] M. Tan et al., “Federated reinforcement learning for smart and privacy-preserving energy management of residential microgrids clusters,” Engineering Applications of Artificial Intelligence, vol. 139, Art. no. 109579, 2025, doi: 10.1016/j.engappai.2024.109579.
[14] J. Sievers et al., “Federated reinforcement learning for sustainable and cost-efficient energy management,” Energy and AI, vol. 21, Art. no. 100521, 2025, doi: 10.1016/j.egyai.2025.100521.
[15] Y. Zhang et al., “Federated deep reinforcement learning for varying-scale multi-energy microgrids energy management considering comprehensive security,” Applied Energy, vol. 380, Art. no. 125072, 2025, doi: 10.1016/j.apenergy.2024.125072.
[16] Z. Chen, J. He, L. Yu, and Q. Zhao, “Personalized federated deep reinforcement learning for smart home energy management,” IFAC-PapersOnLine, vol. 59, no. 9, pp. 193-198, 2025, doi: 10.1016/j.ifacol.2025.08.135.
[17] Y. Li, X. Chen, and Y. Wang, “EdgeHEM: Sparse federated reinforcement learning for home energy management at the edge,” IEEE Transactions on Smart Grid, vol. 16, no. 6, pp. 5602-5614, 2025, doi: 10.1109/TSG.2025.3598438.
[18] Z. Ye, D. Qiu, S. Li, Z. Fan, and G. Strbac, “Federated reinforcement learning for decentralized peer-to-peer energy trading,” Energy and AI, vol. 20, Art. no. 100500, 2025, doi: 10.1016/j.egyai.2025.100500.
[19] W. Pinthurat, T. Surinkaew, and B. Hredzak, “An overview of reinforcement learning-based approaches for smart home energy management systems with energy storages,” Renewable and Sustainable Energy Reviews, vol. 202, Art. no. 114648, 2024, doi: 10.1016/j.rser.2024.114648.
[20] L. Yu et al., “Deep reinforcement learning for smart home energy management,” IEEE Internet of Things Journal, vol. 7, no. 4, pp. 2751-2762, 2020, doi: 10.1109/JIOT.2019.2957289.
[21] C. Huang, H. Zhang, L. Wang, X. Luo, and Y. Song, “Mixed deep reinforcement learning considering discrete-continuous hybrid action space for smart home energy management,” Journal of Modern Power Systems and Clean Energy, vol. 10, no. 3, pp. 743-754, 2022, doi: 10.35833/MPCE.2021.000394.
[22] A. A. Amer, K. Shaban, and A. M. Massoud, “DRL-HEMS: Deep reinforcement learning agent for demand response in home energy management systems considering customers and operators perspectives,” IEEE Transactions on Smart Grid, vol. 14, no. 1, pp. 239-250, 2023, doi: 10.1109/TSG.2022.3198401.
[23] M. Dorokhova, Y. Martinson, C. Ballif, and N. Wyrsch, “Deep reinforcement learning control of electric vehicle charging in the presence of photovoltaic generation,” Applied Energy, vol. 301, Art. no. 117504, 2021, doi: 10.1016/j.apenergy.2021.117504.
[24] Y. Yang, J. Hao, Y. Zheng, and C. Yu, “Large-scale home energy management using entropy-based collective multiagent deep reinforcement learning framework,” Proc. 28th Int. Joint Conf. Artificial Intelligence (IJCAI), pp. 630-636, 2019, doi: 10.24963/ijcai.2019/89.
[25] D. Qiu, Y. Ye, D. Papadaskalopoulos, and G. Strbac, “Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach,” Applied Energy, vol. 292, Art. no. 116940, 2021, doi: 10.1016/j.apenergy.2021.116940.
[26] D. Qiu, J. Wang, Z. Dong, Y. Wang, and G. Strbac, “Mean-field multi-agent reinforcement learning for peer-to-peer multi-energy trading,” IEEE Transactions on Power Systems, vol. 38, no. 5, pp. 4853-4866, 2023, doi: 10.1109/TPWRS.2022.3217922.
[27] T. Chen, S. Bu, X. Liu, J. Kang, F. R. Yu, and Z. Han, “Peer-to-peer energy trading and energy conversion in interconnected multi-energy microgrids using multi-agent deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 13, no. 1, pp. 715-727, 2022, doi: 10.1109/TSG.2021.3124465.
[28] Y. Ye, D. Papadaskalopoulos, Q. Yuan, Y. Tang, and G. Strbac, “Multi-agent deep reinforcement learning for coordinated energy trading and flexibility services provision in local electricity markets,” IEEE Transactions on Smart Grid, vol. 14, no. 2, pp. 1541-1554, 2023, doi: 10.1109/TSG.2022.3149266.
[29] J. Xie, A. Ajagekar, and F. You, “Multi-agent attention-based deep reinforcement learning for demand response in grid-responsive buildings,” Applied Energy, vol. 342, Art. no. 121162, 2023, doi: 10.1016/j.apenergy.2023.121162.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Intelligent Engineering and Informatics
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
