Model Learning

←All Research

Structured tools for predicting into the future

Modeling learning is the process of taking logged data and utilizing it to create a tool for predicting into the future. My work here started in the area of model-based reinforcement learning, but it is broad enough now that it warrants its own category. Model learning from batch data is also of great interest. If we can learn a useful model, we can leverage all the data we have logged to its fullest extent.

Predicting with a model into the future!

Open areas of study:

  • Models for long-term predictions,
  • Changing model training to prioritize task performance over accuracy,
  • Model predictions with multi-modal data,
  • Applications of low-data model learning.
Where is model-based RL heading 4 years after the seminal paper of my Ph.D.
My thesis on model-based RL. Let's make models work with tasks!
We flip the script on Offline RL research and ask the question of "what is the best dataset to collect?" rather than "what is the best algorithm?"
In this paper we set out to understand the causes of compounding prediction errors in one-step learned models. With this, we hope a next generation of models can be used to improve model-based reinforcement learning.
An open-source PyTorch repository designed from the bottom up for model-based reinforcement learning research.
We showed that advancements in AutoML when paired with common deep RL tasks, MBRL algorithms perform so well they break the simulator.
We explored how MBRL can learn multi-step, nonlinear controllers!
Trying to reframe the MBRL framework with long-term predictions instead of one-step predictions!
Studying the numerical effects of a dual-optimization problem in model-based reinforcement learning -- control and dynamics. When optimizing model accuracy, there is no guarantee on improving task performance!
We used deep model-based reinforcement learning to have a quadrotor learn to hover from less than 5 minutes of all experimental training data.