Skip to content
Digital Control

LQ optimal control and model predictive control

The linear quadratic regulator with the Riccati equation proved by dynamic programming, a closed-form scalar example, LQG, predictive control with receding horizon and constraints.

4 min read 0 comments

Pole placement leaves a question open: where to put the poles? Faster means larger commands, slower means errors that last longer. Optimal control answers by writing the trade-off as a cost to be minimised, and predictive control extends the idea to the constraints every real plant has: saturating actuators, temperatures and pressures not to be exceeded. In this lesson we look at the discrete linear quadratic (LQ) regulator with the derivation of the Riccati equation, a scalar example solved in closed form, and predictive control with the receding horizon, constraints and its applications.

The linear quadratic regulator

For the system x(k+1) = \Phi x(k) + \Gamma u(k) we look for the command sequence that minimises

    \[ J = \sum_{k=0}^{\infty}\left(x(k)^TQx(k) + u(k)^TRu(k)\right) \]

with Q positive semidefinite (how much the error weighs) and R positive definite (how much the command weighs). Increasing R relative to Q asks for a more “frugal” controller.

Theorem. The optimal solution is a state feedback u = -Kx with

    \[ K = (R + \Gamma^TP\Gamma)^{-1}\Gamma^TP\Phi \]

where P is the symmetric positive definite solution of the algebraic Riccati equation

    \[ P = Q + \Phi^TP\Phi - \Phi^TP\Gamma(R + \Gamma^TP\Gamma)^{-1}\Gamma^TP\Phi \]

and the minimum cost starting from x is x^TPx.

Proof (dynamic programming). Let V(x) be the minimum cost starting from x. By Bellman’s principle of optimality, the optimal first step minimises the cost of the step plus the minimum cost from the next state:

    \[ V(x) = \min_u\left[x^TQx + u^TRu + V(\Phi x + \Gamma u)\right] \]

Try V(x) = x^TPx. The quantity in brackets is quadratic in u; its gradient with respect to u is 2Ru + 2\Gamma^TP(\Phi x + \Gamma u), and setting it to zero:

    \[ u = -(R + \Gamma^TP\Gamma)^{-1}\Gamma^TP\Phi x = -Kx \]

It is a minimum because the matrix R + \Gamma^TP\Gamma is positive definite. Substituting u, all the terms are of the form x^T(\cdot)x, and the equality V(x) = x^TPx holds for every x if and only if P satisfies the Riccati equation. ∎

If (\Phi, \Gamma) is reachable and the state is “seen” by the cost ((\Phi, Q^{1/2}) observable), the solution exists, is unique and the closed loop is asymptotically stable. In practice the equation is solved numerically, with a single instruction in computing packages.

Example: an integrator

Take x(k+1) = x(k) + u(k), with Q = 1 and R = r. With \Phi = \Gamma = 1 the Riccati equation becomes

    \[ P = 1 + P - \frac{P^2}{r + P} \quad\Rightarrow\quad P^2 - P - r = 0 \quad\Rightarrow\quad P = \frac{1 + \sqrt{1 + 4r}}{2} \]

and K = P/(r + P), with closed-loop pole 1 - K = r/(r + P).

rPKpole 1 - K
0110 (deadbeat)
11.6180.6180.382
103.700.2700.730
10010.50.0950.905

With a free command (r = 0) the optimum is deadbeat (lesson on deadbeat control); with an expensive command the controller becomes slower and slower. For r = 1 the golden ratio appears, P = (1 + \sqrt{5})/2. The designer no longer chooses the poles, but the weights, which have a direct physical meaning. Continuous LQ control also has guaranteed margins (infinite gain margin, at least 60° of phase); in discrete time the margins are somewhat smaller but remain good.

Together with the Kalman filter (lesson on observers) the LQ regulator forms LQG control, optimal when disturbances and noise are Gaussian, and thanks to the separation principle the two are designed independently.

Predictive control

The LQ regulator knows nothing about limits: if the optimal command exceeds the actuator’s saturation, it is simply clipped, and optimality and sometimes stability are lost. Predictive control (MPC, model predictive control) puts the constraints inside the problem.

Receding horizon of predictive control
To the left of instant k the measured samples of the output y; to the right the dashed prediction of the output, reaching the reference r, obtained with the planned command sequence u over the horizon of N steps. Only the first command, in red, is applied; at the next instant everything is recomputed.

At every sample the controller:

  1. measures (or estimates with an observer) the current state;
  2. using the model, predicts the evolution of the output over the next N steps as a function of the future command sequence u(k), \dots, u(k+N-1);
  3. chooses the sequence that minimises a quadratic cost (error from the reference plus command effort) while respecting the constraints: limits on commands, on their rate of change, on outputs and on states;
  4. applies only the first command, discards the others, and at the next sample repeats everything with the new measurement.

Step 3 is a quadratic programming problem: quadratic cost, linear constraints, a unique solution that efficient algorithms find in microseconds or milliseconds. Step 4, the receding horizon, makes the scheme a true feedback: model errors and disturbances are corrected at every cycle.

Some properties:

  • with no active constraints, predictive control is a fixed linear feedback; with an infinite horizon it coincides with the LQ regulator;
  • stability is guaranteed by adding a terminal cost equal to the LQ cost x^TPx of the final state, and possibly a terminal constraint;
  • it knows reference changes and measurable disturbances in advance, and can start reacting before they occur;
  • it naturally handles systems with many coupled inputs and outputs (Process Control lesson on multivariable systems).

Predictive control was born in the late 1970s in refineries, where processes are slow and there is time to compute, and today it is the standard for advanced process control. With modern processors it has reached fast systems: engine management and autonomous driving in cars, drones, power converters and traction electronics, where the “finite control set” version directly chooses the state of the switches (Power Electronics lessons).

Related articles

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x