Optimization of a Learned Dynamic Model for Inverted Pendulum

Approach 

To solve this problem in the context of optimization, I decided to pick a simple Feed Forward network architecture and have the following design variables:

• Number of hidden layers

• Number of hidden nodes (constant across layers)

• Batch Size

• Activation function (ReLu, Sigmoid, Tanh)

• Loss Function (MSE, MAE)

• Optimizer (Adam, SGD, LBFGS)

I used PyTorch Lightning to perform the actual model training. Training data was generated from a random state, input pair and the state at the next time step was found using the analytical model of the inverted pendulum. The final format of the training data is:

network input: [theta, theta_dot, torque] 

network output:   [theta, theta_dot] at the next time step

To handle the optimization, I used the Ray Tune optimization package, with the Optuna Tree-Structured Parzen Estimator (TPE) Algorithm. This algorithm is a Bayesian method, which means that it constructs a model that can estimate the performance of a set of hyper parameters. Each time a new model is trained, the TPE model gets better, resulting in a better learned model of the dynamics.

The optimization objective is to maximize theta accuracy.  A prediction is considered accurate if it is within 0.01 radians (0.57 degrees) of the ground truth. For this optimization, I did not consider theta_dot because the final position of the pendulum was more important. 

For the control of the inverted pendulum I used NEMPC, a non-linear MPC developed at BYU. Essentially NEMPC forecasts a certain number of trajectories using evolutionary methods and then selects the best trajectory. For more information, see the linked paper.