Reinforcement Learning

←All Research

The most opaque and intriguing system

Reinforcement learning is a framework open to such a deep level of investigation. There are theoretical proofs showing convergence, new algorithms, relations to biology, and my personal favorite: applications. RL is becoming feasible to use in real-world systems and this has potentially huge implications (see this write-up by a colleague) because it’s interactions and problem definition are not well-posed. Legislating this so safety and usefulness are preserved is an active area of my work.

I spend most of my time thinking about the variant of model-based reinforcement learning, which involves very similar optimizations, but has a structured and modular learning setup. Learning a dynamics model lends itself to interpretability and generalization (see model-learning).

Model-based reinforcement learning.

Open areas of study:

  • interpretable RL algorithms: what can we learn about how an agent comes to a decision?
  • non sample-based optimization with a learned model.
  • multi-agent learning (100s or 1000s of agents).
We explored how MBRL can learn multi-step, nonlinear controllers!
Full Page
Trying to reframe the MBRL framework with long-term predictions instead of one-step predictions!
Full Page
Studying the numerical effects of a dual-optimization problem in model-based reinforcement learning -- control and dynamics. When optimizing model accuracy, there is no guarantee on improving task performance!
Full Page
Learning how to walk with a real-world hexapod using a hierarchy of model-free RL for basic motion primitives with model-based RL for higher level planning.
Full Page
We used deep model-based reinforcement learning to have a quadrotor learn to hover from less than 5 minutes of all experimental training data.
Full Page