Here are the challenges I have faced in graduate school.
(Last updated Jan. 2021)
Stalled, not failed, projects
List of projects that stalled that I remember (reverse chronological order).
- Undergrad project on scaling multi-agent communications into multi-agent RL frameworks (think merging BICNET with QMIX), (Summer 2020 — Fall 2020)
- Controlled Flight of the Ionocraft, (Spring 2018 — Fall 2020).
- Constrained optimization for control in MBRL, (Tried a couple times — Summer 2019).
- Audio transmission with electrohydrodynamic thrusters, (Summer 2018 — Spring 2019).
- Multiple projects on controlled coordination and learning trajectories with a drone and MBRL, building on this, (Fall 2018 — Spring 2019).
- A project that was almost identical to the idea that turned into the MBPO algorithm (pseudo-scoop), (Fall 2018 — Summer 2019).
- Using 0th order optimizers to generate control parameters from learned one-step dynamics models, (Fall 2018 — Summer 2019, during internship :/).
- Imitating human drivers interactions with pedestrians, Berkeley Deep Drive, (Winter 2017).
Paper submissions also historically are a total mess. Making this list I think I have actually been somewhat fortunate in the amount of rejections.
List of papers that were not accepted the first submission:
- On the Importance of Hyperparameter Optimization forModel-based Reinforcement Learning, page soon, AISTATS 2021 (I got added to this late, but it was rejected at least once from ICML).
- Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning, TBD 2021 (CoRL 2020 rejection).
- Objective Mismatch in Model-based Reinforcement Learning, L4DC 2019 (CoRl 2019 weird rejection, ICRL 2020 rejection).
- Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learning, IROS 2018 (ICRA 2018 rejection).
Wild Project Journeys
Some projects are not defined by resubmissions. Here are couple projects that I did that are not the imagined path for research.
- Nonholonomic Yaw Control of an Underactuated Flying Robot with Model-based Reinforcement Learning: This paper started in my second semester of graduate school, had a two year gap in the middle, and then was published in the start of my fourth year after my Qualifying Exam Chair seemed to like how it sounded.
- A project on simulating swarm-v-swarm combat in my lab has been started and restarted by various groups of undergrads. Now on the third wave. It doesn’t count as stalled, but it is special.
I have submitted numerous grants and fellowships that I have not won or heard anything back from. At this point figuring out all of them is too much of a hassle. Needless to say it includes multiple years of Facebook, Google, Microsoft fellowships and Google, Amazon research awards. Many others have come and past too.