Anurag Ajay

I am a Ph.D. student in EECS at MIT CSAIL advised by Professor Pulkit Agrawal. I am grateful for support from MIT presidential fellowship.

In Summer 2020, I did a research internship with Ofir Nachum and Sergey Levine at Google Brain, working on unsupervised skill discovery for improving offline deep reinforcement learning.

In May 2019, I received my S.M. in EECS from MIT advised by Professor Leslie Kaelbling and Professor Josh Tenenbaum. Before MIT, I completed my bachelor's degree in EECS from UC Berkeley where I worked with Professor Pieter Abbeel and Professor Sergey Levine.

Email  /  Resume  /  Google Scholar  /  LinkedIn

profile photo

My research goal is to design algorithms that can enable agents to continuously interact, learn, and perform complex tasks in their environments. To this end, I am interested in learning Visuomotor Priors for Behavior Transfer: leveraging past interactions to learn both visual and control priors that allows agent to quickly adapt to new environments.

In the past, I have worked on using deep learning for state estimation, building model-based deep reinforcement learning algorithms for robotics and learning dynamics model, with the help of physics engines, in the robotics domain.

PontTuset OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
Anurag Ajay, Aviral Kumar, Pulkit Agrawal, Sergey Levine, Ofir Nachum
ICLR, 2021
arXiv / bibtex / project page

When presented with offline data composed of a variety of behaviors, an effective way to leverage this data is to extract a continuous space of recurring and temporally extended primitive behaviors before using these primitives for downstream task learning. Primitives extracted in this way serve two purposes: they delineate the behaviors that are supported by the data from those that are not, making them useful for avoiding distributional shift in offline RL; and they provide a degree of temporal abstraction, which reduces the effective horizon yielding better learning in theory, and improved offline RL in practice.

PontTuset Learning Action Priors for Visuomotor transfer
Anurag Ajay, Pulkit Agrawal,
ICML Inductive Biases, Invariances and Generalization in RL (BIG) Workshop, 2020

We learn a latent space for past useful action trajectories using an autoencoding model and explore in the learned space which improves performance in the context of transfer learning across tasks and in multi-task reinforcement learning.

PontTuset Long-Horizon Prediction and Uncertainty Propagation with Residual Point Contact Learners
Nima Fazeli, Anurag Ajay, Alberto Rodriguez
ICRA, 2020
arXiv / bibtex

We propose a self-supervised approach to learning residual models for rigid-body simulators that exploits corrections of contact models to refine predictive performance and propagate uncertainty.

PontTuset Learning skill hierarchies from predicate descriptions and self-supervision
Tom Silver*, Rohan Chitnis*, Anurag Ajay, Leslie Kaelbling, Josh Tenenbaum
AAAI GenPlan Workshop, 2020

We learn lifted, goal-conditioned policies and use STRIPS planning with learned operator descriptions to solve a large suite of unseen test tasks.

PontTuset Learning to Navigate Endoscopic Capsule Robots
Mehmet Turan, Yasin Almalioglu, Hunter Gilbert, Faisal Mahmood, Nicholas Durr, Helder Araujo, Alp Eren Sarı, Anurag Ajay, Metin Sitti
IEEE RAL, 2019
paper / bibtex

We use deep reinforcement learning algorithms for controlling endoscopic capsule robots

PontTuset Combining Physical Simulators and Object-Based Networks for Control
Anurag Ajay, Maria Bauza, Jiajun Wu, Nima Fazeli, Josh Tenenbaum, Alberto Rodriguez, Leslie Kaelbling
ICRA, 2019
arXiv / bibtex / project page

We propose a hybrid dynamics model, simulator-augmented interaction networks (SAIN), combining a physics engine with an object-based neural network for dynamics modeling. Compared with existing models that are purely analytical or purely data-driven, our hybrid model captures the dynamics of interacting objects in a more accurate and data-efficient manner.

PontTuset Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing
Anurag Ajay, Jiajun Wu, Nima Fazeli, Maria Bauza, Leslie Kaelbling, Josh Tenenbaum, Alberto Rodriguez
IROS, 2018   (Best Paper on Cognitive Robotics)
arXiv / bibtex / project page

Combining symbolic, deterministic simulators with learnable, stochastic neural nets provides us with expressiveness, efficiency, and generalizability simultaneously. Our model outperforms both purely analytical and purely learned simulators consistently on real, standard benchmarks.

PontTuset Reset-Free Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States
William Montgomery*, Anurag Ajay*, Chelsea Finn, Pieter Abbeel, Sergey Levine
ICRA, 2017
arXiv / bibtex / project page

We present a new guided policy search algorithm that allows the method to be used in domains where the initial conditions are stochastic, which makes the method more applicable to general reinforcement learning problems and improves generalization performance in our robotic manipulation experiments.

PontTuset Backprop KF: Learning Discriminative Deterministic State Estimators
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel
NIPS, 2016
arXiv / bibtex

We represent kalman filter as a computational graph and use it in place of recurrent neural networks for data-efficient end to end visual state estimation.

Reviewer for IROS, IEEE-RAL, ICRA, ICML, NeurIPS
uGSI for CS 189 (Introduction to Machine Learning) Fall 2016, Spring 2017