Mathematics · live now

Optimisation Foundations

Extend calculus to functions of several variables and use it to optimise them. Learn the multivariable chain rule, the gradient, directional derivatives and tangent planes. Then use the Hessian and convexity to find and classify critical points.

See pricing Preview a lesson

No account needed to read a lesson · 30-day money-back guarantee

See the full learning path

lessons

112

practice questions

111

prerequisite lessons

See it taught

2 of the 10 lessons here are free to preview - the whole explanation and worked example, with no account and no email.

Free preview

Steepest ascent and descent

Learn how the rate of change depends on the angle to the gradient, find the directions in which a function rises and falls fastest and how fast, and read the direction of the gradient off a contour plot.

D_{\vec{u}}f(a,b) = \|\nabla f(a,b)\| \cos\theta

Free preview

Tangent planes and linear approximation

Learn how to write the equation of the tangent plane to a surface at a point from the height and the two partial derivatives there, and how to use it to estimate values of a function of two variables near that point.

z = 3 + 2(x-1) + 4(y-1)

Live, not a screenshot

Lesson 06 · Steepest ascent and descent

0,0

(a, b)

(a, b)

\nabla f (a, b)

\vec{u}

θ

Move the slider to turn the unit direction $\vec{u}$ .

θ = 0^{\circ}

θ = 15^{\circ}

θ = 30^{\circ}

θ = 45^{\circ}

θ = 60^{\circ}

θ = 75^{\circ}

θ = 90^{\circ}

θ = 105^{\circ}

θ = 120^{\circ}

θ = 135^{\circ}

θ = 150^{\circ}

θ = 165^{\circ}

θ = 180^{\circ}

θ = 165^{\circ}

θ = 150^{\circ}

θ = 135^{\circ}

θ = 120^{\circ}

θ = 105^{\circ}

θ = 90^{\circ}

θ = 75^{\circ}

θ = 60^{\circ}

θ = 45^{\circ}

θ = 30^{\circ}

θ = 15^{\circ}

θ = 0^{\circ}

\cos θ = 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.24

\cos θ = 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.16

\cos θ = 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.94

\cos θ = 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.58

\cos θ = 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.12

\cos θ = 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.58

\cos θ = 0.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.00

\cos θ = - 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 0.58

\cos θ = - 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.12

\cos θ = - 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.58

\cos θ = - 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.94

\cos θ = - 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.16

\cos θ = - 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.24

\cos θ = - 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 2.16

\cos θ = - 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.94

\cos θ = - 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.58

\cos θ = - 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 1.12

\cos θ = - 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = - 0.58

\cos θ = 0.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.00

\cos θ = 0.26

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 0.58

\cos θ = 0.50

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.12

\cos θ = 0.71

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.58

\cos θ = 0.87

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 1.94

\cos θ = 0.97

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.16

\cos θ = 1.00

D_{\vec{u}} f (a, b) = ‖ \nabla f (a, b) ‖ \cos θ = 2.24

Here $\nabla f (a, b) = [\begin{matrix} 2 \\ 1 \end{matrix}]$ , so $‖ \nabla f (a, b) ‖ = \sqrt{5} \approx 2.24$ . The angle $θ$ is always the smaller angle between $\vec{u}$ and $\nabla f (a, b)$ , so it rises to $180^{\circ}$ and then falls back to $0^{\circ}$ .

The diagram from the lesson itself, running live.

10 of 15 lessons

10 lessons you can take today, and 5 more being written. Expand any one to see what it covers. Each unlocks when its own prerequisites are passed.

The multivariable chain rule

12 questions

Learn how to draw a dependency diagram for a function whose inputs each depend on one further variable, read the multivariable chain rule from it, and write the chain rule equation for functions given by formulas.

Covers

Dependency diagrams

The multivariable chain rule

Writing the chain rule for given functions

Computing with the multivariable chain rule

12 questions

Learn how to compute with the multivariable chain rule when the functions are given as formulas, first with one independent variable and then with two, where every derivative is partial.

Covers

One independent variable

Two independent variables

Computing both partial derivatives

The gradient vector

6 questions

Learn how to find the gradient of a function of two or more variables by collecting its first partial derivatives into a single column vector, and how to evaluate it at a point.

Covers

Computing the gradient

Rates of change along a line

14 questions

Learn how to find the point reached by stepping along a line in a unit direction, and how fast a function of two variables changes as you move along that line.

Covers

The point reached after a step

How direction and position affect the rate

Computing the rate at a point

Directional derivatives

13 questions

Learn how to find the rate of change of a function in any direction as the dot product of its gradient with a unit vector, normalise a direction that is not a unit vector, and interpret what the value tells you about the function.

Covers

The directional derivative

Computing a directional derivative

Interpreting a directional derivative

Steepest ascent and descent

Preview

13 questions

Covers

The rate as $\|\nabla f\|\cos\theta$

Steepest ascent is along the gradient

The gradient on a contour plot

Preview this lesson - no account needed

Tangent planes and linear approximation

Preview

8 questions

Covers

The tangent plane to a surface

Linear approximation in two variables

Preview this lesson - no account needed

Critical points of a function of two variables

14 questions

Learn why every local maximum or minimum of a function of two variables occurs at a critical point, and how to find the critical points by solving for where the gradient is zero, including systems that split into cases.

Covers

The gradient at a local extremum

Finding critical points

Critical points when an equation factors

Convex functions

9 questions

Learn what it means for a function of one or two variables to be convex, by comparing its graph with its secant lines.

Covers

What convex means

Convexity in two variables

Minimising convex functions

11 questions

Learn why a local minimum of a convex function is its global minimum, and how to find the global minimum of a convex function of one or two variables by solving for where its derivative or gradient is zero.

Covers

A local minimum of a convex function is global

Finding the global minimum at a critical point

—

The Hessian matrix

In drafting

—

Second directional derivatives

In drafting

—

Classifying critical points

In drafting

—

Convexity and the Hessian

In drafting

—

Gradient descent

In drafting

Where this unit sits in the curriculum

This unit is part of our Essential Calculus for ML learning path, which contains 192 lessons. Every one of them is drawn below.

The diagnostic test lets you skip any lesson you already know, including the ones in this unit.

Hover any dot to name its lesson.

10 lessons in this unit

111 prerequisite lessons across the unit, counting every step back to the start

71 other lessons in the learning path - after this unit, or alongside it

Every dot is a lesson, every line a prerequisite. You can only start a lesson once you have mastered all of its prerequisites, so you are always building on solid foundations.

Builds on: Derivatives, Limits, Matrices and linear systems, Partial derivatives, Vectors in Euclidean space

Start Optimisation Foundations

One subscription covers every learning path, and you can test out of anything you already know.

See pricing Preview a lesson

One subscription covers every learning path · 30-day money-back guarantee

More in Calculus: Limits, Derivatives, Integrals, Partial derivatives, Double integrals