Posts for: #optimal-control

There's been a lot of noise about large language models (LLMs) and them possibly replacing programmers in the near future -- a perspective shared by many technology enthusiasts but met with a tad more skepticism from researchers and software engineers. OpenAI's release of ChatGPT has generated lots of discussion on the subject. Although complete automation might be far off, some folks have already found some use for Copilot, automating things like rote boilerplate code. However, that does not necessarily translate to correct, let alone efficient, code especially in complex applications and specific domains. An obvious caveat: different language models lead to different results, and the randomness in the inference process can create a lot of variability -- some friends have told me OpenAI playground does generate more reliable code.

Consider once again an optimal control problem (OCP) in continuous time: $$ \begin{equation} \begin{alignedat}{2} &\min_{x, u}{}&& \int_0^T \ell(x(t), u(t)) dt + h(x(T)) \\ &\suchthat&& \dot{x} = f(x, u), \quad t \in [0, T) \end{alignedat} \end{equation} $$

The Differential Dynamic Programming (DDP) is a nowadays quite popular optimal control algorithm, which was introduced by Jacobson all the way back in the 1970s. Recently, it has made a resurgence following work by Tassa and Todorov.

Most control loops out there used in real-world systems are simple feedback loops proportional to the error, its derivative or integral (this is called PID control). However, this kind of control can exhibit undesirable behavior such as oscillations or failing to converge to a given setpoint quickly if at all. Some more complex systems such robots, satellites or cars can come with precise performance requirements, and more carefully constructed control actions need to be supplied with guarantees about their optimality.