Academic
Academic
Home
Projects
Talks
Publications
Contact
Light
Dark
Automatic
1
Mitigating Metastable Failures in Distributed Systems with Offline Reinforcement Learning
This paper introduces a load-shedding mechanism that mitigates metastable failures through offline reinforcement learning (RL). Previous studies have heavily focused on heuristics that are reactive and limited in generalization, while online RL algorithms face challenges in accurately simulating system dynamics and acquiring data with sufficient coverage. In contrast, our algorithm leverages offline RL to learn directly from existing log data. Through extensive empirical experiments, we demonstrate that our algorithm outperforms rule-based methods and supervised learning algorithms in a proactive, adaptive, generalizable, and safe manner. Deployed in a Java compute service with diverse execution times and configurations, our algorithm exhibits faster reaction time and attains the Pareto frontier between throughput and tail latency.
Yueying Li et al
PDF
Cite
Autocat, Reinforcement learning for automated exploration of cache-timing attacks
The aggressive performance optimizations in modern microprocessors can result in security vulnerabilities. For example, timing-based …
Mulong Luo
,
Wenjie Xiong
,
Geunbae Lee
,
Yueying Li
,
Xiaomeng Yang
,
Amy Zhang
,
Yuandong Tian
,
Hsien-Hsin S Lee
,
G Edward Suh
PDF
Cite
Efficient Planning in a Compact Latent Action Space
Planning-based reinforcement learning has shown strong performance in tasks in discrete and low-dimensional continuous action spaces. However, scaling such methods to high-dimensional action spaces remains challenging. We propose Trajectory Autoencoding Planner (TAP), which learns a compact discrete latent action space from offline data for efficient planning, enabling continuous control in high-dimensional control with a learned model.
Zhengyao Jiang
,
Tianjun Zhang
,
Michael Janner
,
Yueying Li
,
Tim Rocktäschel
,
Edward Grefenstette
,
Yuandong Tian
PDF
Cite
Code
Project
Cite
×