Reinforcement Learning

Mitigating Metastable Failures in Distributed Systems with Offline Reinforcement Learning
This paper introduces a load-shedding mechanism that mitigates metastable failures through offline reinforcement learning (RL). Previous studies have heavily focused on heuristics that are reactive and limited in generalization, while online RL algorithms face challenges in accurately simulating system dynamics and acquiring data with sufficient coverage. In contrast, our algorithm leverages offline RL to learn directly from existing log data. Through extensive empirical experiments, we demonstrate that our algorithm outperforms rule-based methods and supervised learning algorithms in a proactive, adaptive, generalizable, and safe manner. Deployed in a Java compute service with diverse execution times and configurations, our algorithm exhibits faster reaction time and attains the Pareto frontier between throughput and tail latency.
Yueying Li et al
PDF Cite
Autocat, Reinforcement learning for automated exploration of cache-timing attacks
The aggressive performance optimizations in modern microprocessors can result in security vulnerabilities. For example, timing-based …
Mulong Luo, Wenjie Xiong, Geunbae Lee, Yueying Li, Xiaomeng Yang, Amy Zhang, Yuandong Tian, Hsien-Hsin S Lee, G Edward Suh
PDF Cite
Efficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces. However, scaling such methods to high-dimensional action spaces remains challenging. We propose Trajectory Autoencoding Planner (TAP), which learns a compact discrete latent action space from offline data for efficient planning, enabling continuous control in high-dimensional control with a learned model.
Zhengyao Jiang, Tianjun Zhang, Michael Janner, Yueying Li, Tim Rocktäschel, Edward Grefenstette, Yuandong Tian
PDF Cite Code Project
AutoCAT: Reinforcement Learning for Automated Exploration of Cache-Timing Attacks
Security can be seen as a competitive game between an attacker and a defender RL can be used to automatically explore attacks (and defenses) on a black-box system
Oct 28, 2022 New Hampshire
Mulong Luo, Wenjie Xiong, Geunbae Lee, Yueying Li, Xiaomeng Yang, Amy Zhang, Yuandong Tian, Hsien-Hsin S. Lee, and Edward Suh
Slides
AutoCAT: Reinforcement Learning for Automated Exploration of Cache-Timing Attacks
Efficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces. However, scaling such methods to high-dimensional action spaces remains challenging. We propose Trajectory Autoencoding Planner (TAP), which learns a compact discrete latent action space from offline data for efficient planning, enabling continuous control in high-dimensional control with a learned model.
Follow
Efficient Planning in a Compact Latent Action Space