Reinforcement Learning for Low Earth Orbit Momentum Stabilization of CubeSats
Contributors
Anirudh Aatresh, Joseph Breeden, Kevin Tan, Liliang Wang.
Summary
Space is commonly assumed to be a pure vacuum, but in reality, it is not, especially for satellites in low earth orbit (LEO). Real satellites encounter disturbances including upper atmospheric drag, solar radiation pressure, gravity gradient, magnetic field interactions, etc. These disturbances tend to make a spacecraft's orbit decay and disorient it. Often, onboard reaction wheels are used to counter these torques, but if the torque is non-zero-mean (as drag usually is), these wheels will spin faster and faster until it is necessary to unload the wheels' angular momentum. A cost-effective way to unload this momentum is to use magnetic torquer bars (MTBs) that use the Earth's magnetic field to induce torques.
Existing control algorithms are ineffective due to the irregular nature of the Earth's magnetic field. In this project, we attempted to use the success of reinforcement learning (RL) algorithms to learn an effective policy on an offline dataset for a CubeSat. We compared the developed policies' performance in three simulated disturbance environments. We use NASA's open-source general-purpose spacecraft trajectory simulator, 42, in our experiments. We concluded that RL could perform better than a classical control law, but questions remain regarding the stability of the RL approach.