Posts by Collection

Model-based Reinforcement Learning (RL) integrates learning and planning and has received increasing attention in recent years. However, learning the model can incur a significant cost (in terms of sample complexity), due to the need to obtain a sufficient number of samples for each state-action pair. In this paper, we investigate the benefits of leveraging structural information about the system in terms of reducing sample complexity. Specifically, we consider the setting where the transition probability matrix is a known function of a number of structural parameters, whose values are initially unknown. We then consider the problem of estimating those parameters based on the interactions with the environment. We characterize the difference between the Q estimates and the optimal Q value as a function of the number of samples. Our analysis shows that there can be a significant saving in sample complexity by leveraging structural information about the model. We illustrate the findings by considering how to control a queuing system with heterogeneous servers.

Synergy Unleashed: The Convergence of Control, Game Theory, and Reinforcement Learning in Cyber-Physical Systems

Published: November 05, 2023

Nowadays, with the integration of information technology into plant automation (traditional control systems), cyber-physical systems (CPSs) technology has been widely applied to various infrastructures. The pursuit of resilient and robust controllers withstanding adversarial disturbances and interventions has stood as a formidable challenge for years.

teaching

Introduction to the Security of Industrial Cyber-physical Systems

Graduate level course, Zhejiang University, Department of Control Science and Engineering, 2016

This course provides an introduction to the fundamentals of cyber-physical systems, emphasizing mathematical formulation with a focus on control theory and game theory principles.

Jiajun Shen (沈佳骏)

Posts by Collection

portfolio

publications

talks

teaching