BRFedTD: A Novel Framework for Federated Reinforcement Learning with Byzantine-Resilient Policy Evaluation
作者:
时间:2026-11-20
阅读量:59次
  • 演讲人: 毛晓军(上海交通大学,教授)
  • 时间:2026年12月4日14:00
  • 地点:浙江大学紫金港校区行政楼1312会议室
  • 主办单位:浙江大学数据科学研究中心

简介:  Federated reinforcement learning (FRL) enables distributed agents to collaboratively evaluate policies without sharing raw data, making it a promising approach for privacy-preserving and scalable decision-making. However, the presence of Byzantine agents, which may behave arbitrarily or maliciously, poses significant challenges to the robustness and reliability of FRL. To address this issue, in this paper, we propose a novel trimmed mean-based robust federated policy evaluation framework called Byzantine-resilient federated temporal difference learning (BRFedTD), and establish a finite-time convergence theory of BRFedTD. This framework effectively addresses the combined challenges of linear function approximation, heterogeneous Markov decision processes (MDPs), multiple local updates, and robust aggregation. To support more accurate confidence interval estimation and policy uncertainty analysis, we further derive the asymptotic distribution of the estimation error, showing that BRFedTD achieves asymptotic normality and efficiency in the case of identical MDPs without Byzantine attacks. This represents, to the best of our knowledge, the first asymptotic normality result established in FRL. Extensive numerical experiments demonstrate the robustness and effectiveness of the proposed algorithm, and corroborate that it generalizes to deep reinforcement learning and performs well on complex control tasks.


个人简介:毛晓军,上海交通大学教授。他的研究领域包括分布式统计推断和高维数据分析。主要研究成果发表于AOS、JASA、JMLR、IEEE (TPAMI, TIT, TSP, TON, TIFS)、ICML、NeurIPS, 《管理世界》等期刊及会议上。先后主持国家自然科学基金优秀青年基金、面上项目等。