（疫情原因，取消）Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference
时间：2022-03-10
阅读量：355次
- 演讲人： 刘卫东（上海交通大学 教授）
- 时间：2022年03月17日 周四下午3:00
- 地点：腾讯会议415-192-228
- 主办单位：数据科学研究中心
摘要：This paper develops an efficient distributed inference algorithm, which is robust against a moderate fraction of Byzantine nodes, namely arbitrary and possibly adversarial machines in a distributed learning system. In robust statistics, the median-of-means (MOM) has been a popular approach to hedge against Byzantine failures due to its ease of implementation and computational efficiency. However, the MOM estimator has the shortcoming in terms of statistical efficiency. The first main contribution of the paper is to propose a variance reduced median-of-means (VRMOM) estimator, which improves the statistical efficiency over the vanilla MOM estimator and is computationally as efficient as the MOM. Based on the proposed VRMOM estimator, we develop a general distributed inference algorithm that is robust against Byzantine failures. Theoretically, our distributed algorithm achieves a fast convergence rate with only a constant number of rounds of communications. We also provide the asymptotic normality result for the purpose of statistical inference. To the best of our knowledge, this is the first normality result in the setting of Byzantine-robust distributed learning. The simulation results are also presented to illustrate the effectiveness of our method.
报告人简介：刘卫东，上海交通大学教授，博士生导师，“国家杰出青年科学基金”获得者。2003年本科毕业于浙江大学数学系，2008年获得浙江大学博士学位，2008-2011年在香港科技大学和美国宾夕法尼亚大学沃顿商学院担任博士后研究员。主要研究方向为统计学理论和机器学习等，目前已在Annals of Statistics、Annals of Probability、Bernoulli、 Journal of Machine Learning Research 等国际学术期刊上发表论文数篇。
欢迎参加！