## Lagrangians – derivation under constraints

In this post I will describe a formal technique which can help in finding derivatives of functions under certain constraints. In order to see what I mean, take the following two examples:

**1. **If you differentiate the function with respect to , you get zero, obviously.

**2. **If you differentiate the function under the constraint that (supposing that ) then it is not hard to see that and the derivative will be .

This shows that adding certain constraints will change the derivative and when dealing with more sophisticated constraints, like PDE or differential equations, things get less clear.

The question of the *existence* of derivatives with respect to some variables, when dealing with constraints, is usually dealt with by using the implicit function theorem: this basically says that if some variables are linked by some smooth equations and that you can locally invert the dependence, then you can infer differentiability.

The method presented below skips this step and goes directly for the computation of the derivative. This is a common used technique in control theory or shape optimization, where computing derivatives is essential in numerical simulations. I will come back to derivatives linked to shape optimization in some further posts. For now, let’s see how one can use the Lagrangian method for computing derivatives in some simple cases.

**Example 1.** Let’s compute the derivative of under the constraint . In order to do this we introduce a Lagrangian, which has the following form

The penalization of the constraints is done using *Lagrange multipliers*. In our case we can define the Lagrangian in the following way:

where takes the place of and is the Lagrange multiplier. When introducing the Lagrangian it is important to make all variables **independent** so that when dealing with derivatives, we won’t have to worry about derivatives of composite functions. Moreover, note that if we take then

so we recover the functional we wish to differentiate. Now imagine we differentiate directly w.r.t :

The fact that we have the derivatives of and which appear makes the formulas more complicated. In order to eliminate such terms we may search for the so-called *saddle-points* w.r.t. variables and . Let’s investigate where these partial derivatives vanish:

- (a) The derivative w.r.t. :
which gives . This is the

*state equation*, and obtaining it is natural when differentiating w.r.t. Lagrange multipliers. - (b) The derivative w.r.t. :
giving . This is the

*adjoint equation*, giving the Lagrange multiplier once the state is known.

Now let’s suppose that and verifies the state equation . Then

Differntiating with respect to we get

and this holds for all . Note that the second term vanishes, as is a solution of the state equation. We are free to choose a convenient value for and the natural choice is to cancel the third term. For this we choose . Now it remains to compute the derivative of w.r.t. and this gives

In conclusion we have

where we used the fact that and therefore .

**Example 2.** Let’s find the derivative of under the constraint . This is equivalent to finding the derivative of (for , of course).

Define the Lagrangian

As before, we look for points where derivatives of w.r.t. and vanish:

- (a) The derivative w.r.t. :
which gives .

- (b) The derivative w.r.t. :
giving .

Now we recall that if then

for all . When differentiating, if we replace using the relation given by the adjoint state then we’ll cancel the partial derivative w.r.t. . Therefore we are left with

as expected.

**Example 3.** (do it yourself) Use the Lagrangian formulation to recover the computation of the derivative of the inverse function.