哈尔滨工业大学（深圳）学术讲座

演讲人Speaker：夏俐 教授

题目Title: Risk-Sensitive Markov Decision Processes with Long-Run CVaR Criterion

时间Date：2024年 3月 18日       Time：上午 10：00 ~ 11：00

地点Venue： T2 栋 T2410  室  

内容摘要Abstract：

CVaR(Conditional Value at Risk) is an important risk measure in finance engineering. Traditional studies on the optimization of CVaR metrics are usually for single-stage problem. When extended to multi-stage scenarios, the CVaR risk function is not additive per stage, which does not fit the standard MDP(Markov decision process) model and the principle of dynamic programming fails. In this talk, we study the MDP optimization problem for long-run CVaR criterion using a new tool called the sensitivity-based optimization. By introducing a pseudo CVaR metric, we convert the original problem as a bilevel MDP problem: the inner is a standard MDP optimizing the pseudo CVaR, the outer is an optimization problem for a single auxiliary variable. We derive a CVaR difference formula which quantifies the difference of long-run CVaR values under any two randomized policies. With this difference formula, we prove the optimality of deterministic policies. We also obtain a so-called Bellman local optimality equation for CVaR, which is a necessary and sufficient condition for local optimal policies and only necessary for global optimal policies. We further develop a policy iteration type algorithm to efficiently optimize CVaR. We prove that the iterative algorithm can converge to local optima in the mixed policy space. Finally, we conduct a numerical experiment about portfolio management to demonstrate the main results. Our work may shed light on dynamically optimizing CVaR from a sensitivity viewpoint.

个人简介(About the speaker):

夏俐，中山大学管理学院教授。分别于2002年和2007年在清华大学自动化系获得学士和博士学位，博士生期间在香港科技大学联合培养，博士毕业后分别在IBM中国研究院、沙特国王科技大学从事科研工作，2011年至2019年在清华大学自动化系任教，历任讲师、副教授（博士生导师），2019年调入中山大学。主要研究方向为马氏决策过程、强化学习、排队论、博弈论等理论研究，以及在能源、金融等领域的应用研究。发表论文100余篇，获得中国专利10余项、美国专利3项，主持5项国家自然科学基金项目、3项国家重点研发计划子课题、多项华为腾讯等公司的合作研发项目。担任IEEE Transactions on Automation Science and Engineering、Discrete Event Dynamic Systems等国际权威SCI期刊的副主编（AE），曾两次获教育部高等学校自然科学二等奖等学术奖励。