Abstract: Although distributional reinforcement learning (DRL) has been widely examined in the past few years, very few studies investigate the validity of the obtained Q-function estimator in the distributional setting. We discuss some of our works in ensuring the monotonicity of the obtained quantile estimates and the theoretical necessity. Moreover, we undertake a comprehensive analysis of how the approximation errors within the Q-function impact the overall training process in DRL. We both theoretically analyze and empirically demonstrate techniques to reduce both bias and variance in these error terms, ultimately resulting in improved performance in practical applications.

Bio: 周帆，上海财经大学统计与管理学院副教授，博士毕业于美国北卡罗莱纳大学教堂山分校。主要研究方向包括强化学习，深度学习，因果推断。多项研究成果发表于NeurIPS, ICML, KDD, IJCAI等国际人工智能会议和Journal of American Statistical Association，Biometrics, Nature Genetics等统计学期刊，曾获泛华统计协会新研究者奖，北卡教堂山分校Barry H. Margolin Award，入选上海市扬帆计划，晨光计划。