Skip to content
Math Analysis 2

Constrained extrema: the method of Lagrange multipliers

The geometric idea of tangency, proof of the gradient condition, the Lagrangian, the box of largest volume, the meaning of the multiplier and several constraints.

4 min read 0 comments

Often you are not looking for the maximum of a function everywhere, but only among points that satisfy a constraint: the box of largest volume from a given amount of cardboard, the largest power with a limited current, the shortest distance from a curve. These are constrained extremum problems, and the method of Lagrange multipliers solves them with an elegant geometric idea.

The problem

We look for maxima and minima of f(x, y) among the points that satisfy the constraint g(x, y) = 0. If one variable can be solved for from the constraint (for example y = 1 - x), substitute it and get a one-variable problem. But often this is impossible or awkward: a constraint like x^2 + y^2 = 1 gives two branches, and in three variables things get worse.

The geometric idea

Imagine walking along the constraint curve g = 0 while watching the level curves of f. As long as our curve crosses the level curves, we can move towards higher or lower values of f. At a constrained maximum or minimum that is no longer possible: there the constraint curve is tangent to a level curve of f.

Level curves of f tangent to the constraint at the maximum, with parallel gradients
Maximising f = x + y on the circle g = x^2 + y^2 - 1 = 0. The dashed lines are the level curves of f; the red one is the last that still touches the circle, and it is tangent to it. At that point \nabla f and \nabla g are parallel.

Two tangent curves share the same normal line. The gradients are perpendicular to their level curves, so at the point of tangency \nabla f and \nabla g are parallel:

    \[ \nabla f(\mathbf x_0) = \lambda\,\nabla g(\mathbf x_0) \]

for some number \lambda, the Lagrange multiplier.

Proof

Assume \nabla g(\mathbf x_0) \ne \mathbf 0. Then, by the implicit function theorem, near \mathbf x_0 the constraint is a smooth curve that can be parametrised by \mathbf r(t), \mathbf r(0) = \mathbf x_0. The function f(\mathbf r(t)) has an extremum at t = 0, so by the chain rule \nabla f(\mathbf x_0)\cdot\mathbf r'(0) = 0: the gradient of f is perpendicular to the tangent of the constraint. So is \nabla g(\mathbf x_0), because g(\mathbf r(t)) = 0 for every t and therefore \nabla g\cdot\mathbf r'(0) = 0. In the plane, two vectors perpendicular to the same direction are parallel.

The method

Define the Lagrangian

    \[ \mathcal L(x, y, \lambda) = f(x, y) - \lambda\,g(x, y) \]

and look for its critical points, that is solve the system

    \[ f_x = \lambda g_x,\qquad f_y = \lambda g_y,\qquad g(x, y) = 0 \]

Three equations in three unknowns. The solutions include every constrained extremum (along with any points where \nabla g = \mathbf 0, to be checked separately). If the constraint is a compact set, like a circle, the maximum and minimum exist by Weierstrass: just compare the values of f at the candidates.

The example in the figure. f = x + y, g = x^2 + y^2 - 1: 1 = 2\lambda x, 1 = 2\lambda y, so x = y, and from the constraint x = y = \pm\frac{1}{\sqrt2}. Maximum \sqrt2 at \left(\frac1{\sqrt2}, \frac1{\sqrt2}\right), minimum -\sqrt2 at the opposite point.

Example: the box of largest volume

A closed box shaped like a rectangular block must have surface area S = 2(xy + yz + zx) = 6 m². What is the largest volume V = xyz? With the Lagrangian in three variables:

    \[ yz = \lambda\cdot 2(y + z),\qquad xz = \lambda\cdot 2(x + z),\qquad xy = \lambda\cdot 2(x + y) \]

Multiplying the first by x and the second by y: xyz = 2\lambda(xy + xz) = 2\lambda(xy + yz), so xz = yz and x = y. In the same way y = z. The best box is a cube: 6x^2 = 6, side 1 m, volume 1 m³.

The meaning of the multiplier

\lambda is not just an auxiliary unknown. If the constraint is g(x, y) = c, the optimal value f^*(c) changes with c, and

    \[ \frac{df^*}{dc} = \lambda \]

The multiplier says how much the optimum improves if the constraint is relaxed by one unit. In economics it is called the “shadow price” of the resource; in physics, in problems with mechanical constraints, the multipliers are the constraint forces.

Several constraints and inequalities

With several constraints g_1 = 0, \dots, g_k = 0 there is one multiplier for each: \nabla f = \lambda_1\nabla g_1 + \dots + \lambda_k\nabla g_k. For example, the extrema of a function on a curve in space, the intersection of two surfaces.

When the constraint is an inequality, g \le 0, as in the search for absolute extrema on a disc, interior points (zero gradient) and the boundary g = 0 (Lagrange) are studied separately. The systematic generalisation is the Karush-Kuhn-Tucker conditions, the basis of the numerical optimisation used in circuit design and machine learning.

An electronics example: maximum power transfer

A voltage source V with internal resistance R_s drives a load R_L. The power in the load is P = \frac{V^2 R_L}{(R_s + R_L)^2}. Here the problem is unconstrained in one variable: dP/dR_L = 0 gives R_L = R_s, with maximum power V^2/(4R_s). If instead the current may not exceed a maximum value, the constraint becomes active and the optimum moves to the edge of the allowed region: exactly the situation that calls for the methods of this lesson.

Related articles

Subscribe
Notify of
guest
0 Comments
Oldest
Newest Most Voted
0
Would love your thoughts, please comment.x
()
x