vector calculus_ understanding the gradient _ betterexplained

1/2/2015 VectorCalculus:UnderstandingtheGradient|BetterExplained

http://betterexplained.com/articles/vectorcalculusunderstandingthegradient/ 1/30

The gradient is a fancy word for derivative, or the rate of change of a function. Its a vector (a direction to

move) that

Points in the direction of greatest increase of a function (intuition on why)

Is zero at a local maximum or local minimum (because there is no single direction of increase)

The term gradient (grad) typically refers to the derivative of vector functions, or functions of more than

one variable. Yes, you can say a line has a gradient (its slope), but using the term gradient for single-

variable functions is unnecessarily confusing. Keep it simple.

Gradient can refer to gradual changes of color, but well stick to the math definition if thats ok with you.

Youll see the meanings are related.

Properties of the GradientNow that we know the gradient is the derivative of a multi-variable function, lets derive some properties.

The regular, plain-old derivative gives us the rate of change of a single variable, usually x. For example,

dF/dx tells us how much the function F changes for a change in x. But if a function takes multiple variables,

such as x and y, it will have multiple derivatives: the value of the function will change when we wiggle x

(dF/dx) and when we wiggle y (dF/dy).

We can represent these multiple rates of change in a vector, with one component for each derivative. Thus,

a function that takes 3 variables will have a gradient with 3 components:

F(x) has one variable and a single derivative: dF/dx

F(x,y,z) has three variables and three derivatives: (dF/dx, dF/dy, dF/dz)

The gradient of a multi-variable function has a component for each direction.

And just like the regular derivative, the gradient points in the direction of greatest increase. However, now

that we have multiple directions to consider (x, y and z), the direction of greatest increase is no longer

simply forward or backward along the x-axis, like it is with functions of a single variable.

Vector Calculus: Understanding the Gradient

Search...
http://betterexplained.com/articles/understanding-pythagorean-distance-and-the-gradient/http://betterexplained.com/



If we have two variables, then our 2-component gradient can specify any direction on a plane. Likewise,

with 3 variables, the gradient can specify and direction in 3D space to move to increase our function.

A Twisted ExampleIm a big fan of examples to help solidify an explanation. Suppose we have a magical oven, with coordinates

written on it and a special display screen:

We can type any 3 coordinates (like 3,5,2) and the display shows us the gradient of the temperature atthat point.

The microwave also comes with a convenient clock. Unfortunately, the clock comes at a price the

temperature inside the microwave varies drastically from location to location. But this was well worth it: we

really wanted that clock.

With me so far? We type in any coordinate, and the microwave spits out the gradient at that location.

Be careful not to confuse the coordinates and the gradient. The coordinates are the current location,

measured on the x-y-z axis. The gradient is a direction to move from our current location, such as move

up, down, left or right.

Now suppose we are in need of psychiatric help and put the Pillsbury Dough Boy inside the oven because we

think he would taste good. Hes made of cookie dough, right? We place him in a random location inside the

oven, and our goal is to cook him as fast as possible. The gradient can help!

The gradient at any location points in the direction of greatest increase of a function. In this case, our



function measures temperature. So, the gradient tells us which direction to move the doughboy to get him

to a location with a higher temperature, to cook him even faster. Remember that the gradient does not give

us the coordinates of where to go; it gives us the direction to move to increase our temperature.

Thus, we would start at a random point like (3,5,2) and check the gradient. In this case, the gradient there is

(3,4,5). Now, we wouldnt actually move an entire 3 units to the right, 4 units back, and 5 units up. The

gradient is just a direction, so wed follow this trajectory for a tiny bit, and then check the gradient again.

We get to a new point, pretty close to our original, which has its own gradient. This new gradient is the new

best direction to follow. Wed keep repeating this process: move a bit in the gradient direction, check the

gradient, and move a bit in the new gradient direction. Every time we nudged along and follow the gradient,

wed get to a warmer and warmer location.

Eventually, wed get to the hottest part of the oven and thats where wed stay, about to enjoy our fresh

cookies.

Dont eat that cookie!But before you eat those cookies, lets make some observations about the gradient. Thats more fun, right?

First, when we reach the hottest point in the oven, what is the gradient there?

Zero. Nada. Zilch. Why? Well, once you are at the maximum location, there is no direction of greatest

increase. Any direction you follow will lead to a decrease in temperature. Its like being at the top of a

mountain: any direction you move is downhill. A zero gradient tells you to stay put you are at the max of

the function, and cant do better.

But what if there are two nearby maximums, like two mountains next to each other? You could be at the top

of one mountain, but have a bigger peak next to you. In order to get to the highest point, you have to go

downhill first.

Ah, now we are venturing into the not-so-pretty underbelly of the gradient. Finding the maximum in

regular (single variable) functions means we find all the places where the derivative is zero: there is no

direction of greatest increase. If you recall, the regular derivative will point to local minimums and

maximums, and the absolute max/min must be tested from these candidate locations.

The same principle applies to the gradient, a generalization of the derivative. You must find multiple

locations where the gradient is zero youll have to test these points to see which one is the global

maximum. Again, the top of each hill has a zero gradient you need to compare the height at each to see

which one is higher. Now that we have cleared that up, go enjoy your cookie.

Mathematics



We know the definition of the gradient: a derivative for each variable of a function. The gradient symbol is

usually an upside-down delta, and called del (this makes a bit of sense delta indicates change in one

variable, and the gradient is the change in for all variables). Taking our group of 3 derivatives above

Notice how the x-component of the gradient is the partial derivative with respect to x (similar for y and z).

For a one variable function, there is no y-component at all, so the gradient reduces to the derivative.

Also, notice how the gradient can itself be a function!

If we want to find the direction to move to increase our function the fastest, we plug in our current

coordinates (such as 3,4,5) into the equation and get:

So, this new vector (1, 8, 75) would be the direction wed move in to increase the value of our function. In

this case, our x-component doesnt add much to the value of the function: the partial derivative is always 1.

Obvious applications of the gradient are finding the max/min of multivariable functions. Another less

obvious but related application is finding the maximum of a constrained function: a function whose x and y

values have to lie in a certain domain, i.e. find the maximum of all points constrained to lie along a circle.

Solving this calls for my boy Lagrange, but all in due time, all in due time: enjoy the gradient for now.

The key insight is to recognize the gradient as the generalization of the derivative. The gradient points to

the maximum of the function; follow the gradient, and you will reach the local maximum.

Questions

Why is the gradient perpendicular to lines of equal potential?

Lines of equal potential (equipotential) are the points with the same energy (or value for F(x,y,z)). In the

simplest case, a circle represents all items the same distance from the center.

The gradient represents the direction of greatest change. If it had any component along the line of

equipotential, then that energy would be wasted (as its moving closer to a point at the same energy). When

the gradient is perpendicular to the equipotential points, it is moving as far from them as possible (this
http://betterexplained.com/articles/understanding-pythagorean-distance-and-the-gradient/



article explains why the gradient is the direction of greatest increase its the direction that maximizes

the varying tradeoffs inside a circle).

Other Posts In This Series

1. Vector Calculus: Understanding Flux

2. Vector Calculus: Understanding Divergence

3. Vector Calculus: Understanding Circulation and Curl

4. Vector Calculus: Understanding the Gradient

5. Understanding Pythagorean Distance and the Gradient

6. Vector Calculus: Understanding the Dot Product

Vector Calculus Printable version

153 Comments

1.

2.

3.

i like it well explained.

me

Super!!!

Jane

You are the man! Nice work!

Chris

Kalid

Better Explained is dedicated to helping you grasp concepts, and serves

over 250k readers each month.

Enjoy this article? Try the site guide or join the newsletter:

[email protected] Get Free Lessons + Bonus PDF
http://betterexplained.com/articles/understanding-pythagorean-distance-and-the-gradient/http://betterexplained.com/abouthttp://betterexplained.com/articles/flux/http://betterexplained.com/cheatsheethttp://betterexplained.com/articles/vector-calculus-understanding-the-gradient/print/http://betterexplained.com/articles/vector-calculus-understanding-the-dot-product/http://betterexplained.com/articles/vector-calculus-understanding-the-gradient/http://betterexplained.com/articles/understanding-pythagorean-distance-and-the-gradient/http://betterexplained.com/articles/category/math/vector-calculus/http://betterexplained.com/articles/vector-calculus-understanding-circulation-and-curl/http://betterexplained.com/articles/divergence/



4.

5.

6.

7.

8.

9.

Thanks, glad it was helpful for you.

Kalid

i was always looking for conceptual and practical examples and yes i finally got.

gaurav

Awsome!

Harry

well you made a good explanation, that even a not-so-smart guy gets it, but i think you missed the

obvious -> WHY does gradient show the direction of the greatest increase.

I think that the principle of the gradient is quite easy, but understanding why does it work the way it

does is a bit tricky and you should have focued on it more.

It would be interesting if you would somehow add it to this good article. Inspiration

http://mathforum.org/library/drmath/view/68326.html

good luck !

Palo

Hi Palo, thats a great point! Ive been feeling a bit guilty, if you can imagine it, because Ive lacked that

explanation

Im probably going to do a separate article on the reason *why* the gradient points in the direction of

greatest increase I have another explanation that it works well with. Thanks for the link and feedback!

Kalid

Your introduction is not quite correct:

You claim: Points in the direction of greatest increase of a function.

John Gabriel
http://instacalc.com/http://mathforum.org/library/drmath/view/68326.html



10.

11.

12.

Why? It can also point in the direction of greatest decrease of a function.

A gradient is one or more directional derivatives. These derivatives are considered in a particular

direction. In the case of single variable calculus, we generally talk about a directional derivative when

we consider multiples of the x unit vector, i.e. k*(1,0). To consider the y unit vector, we deal with the

partial derivatives with respect to y in a given direction. In three dimensions, the 3 partial derivatives

form what we now call a gradient.

So in fact it is incorrect to call this a slope or anything else except to say that it describes the partial

derivatives of a point in the direction of a given vector in space.

Does this make sense? Please visit my blog for some more interesting reading.

http://mathphile.blogspot.com/

Hi John, thanks for writing. Youre right, the formal definition of a gradient is a set of directional

derivatives.

But when thinking about the intuitive meaning, I think its ok to consider the gradient as a vector that

points in the direction of greatest increase (i.e. if you follow that direction your function will tend

towards a local maximum).

Unless Im mistaken, the gradient vector always points in the direction of greatest increase (greatest

decrease would be in the opposite direction).

Kalid

What I was saying is that it points either one way or the other, it is not restricted to the direction of

greatest increase. As a simple example, consider what happens when you differentiate a parabola: You

set the derivative equal to 0 and then you determine that it has either a maximum or a minimum at its

turning point. It is not always a maximum just as it is not always a minimum. Think I have explained

this correctly now.

John Gabriel

good john you have done a great job.

sqib
http://betterexplained.com/http://mathphile.blogspot.com/



13.

14.

15.

16.

17.

18.

Hi John, thanks for the clarification. Id still politely disagree and say that in general, the gradient points

in the direction of greatest increase :).

In the case of 2 dimensions, the gradient/slope only gives a forward or backward direction. A positive

slope means travel forward and a negative slope means travel backwards.

Consider f(x) = x^2, a regular parobola. The gradient is zero at the minimum (x=0), and there is no

*single* direction to go. At x = -1, the slope is negative, which means travel backwards (to x = -2) to

increase your value. Similarly, at x = 1, you travel forward (to x = 2) to increase your value.

But, as you mention, strange things can happen when the derivative = 0. It can mean you are at a local

maximum (no way to improve), or at a local minimum (no single direction to improve your position

forward or back will help). I consider the corner case of zero an exception to the general rule / intuition

that the gradient is the direction to follow if you want to improve your function.

Kalid

Wonderful explanation!

Vidhya

Thanks Vidhya, glad you liked it.

Kalid

hi john keep it up you done a great job

bihazo

Thanks a bunch! I didnt think it could be this simple to find the maximum increase at a point, so I

thought Id look it up. Thanks to your great explaination, it turn out it was as easy as it seemed it should

be. Great job! Thanks!

Travis

Travis

Kalid
http://www.betterexplained.com/http://instacalc.com/



19.

20.

21.

22.

23.

Awesome, glad it worked for you

thanks!!!!

caitlyn

Hi Caitlyn, youre welcome.

Kalid

Thanks! The sadistic microwave example helped a lot.

Derek

Awesome, glad it was useful :).

Kalid

Hello Kalid,

Did not read your reply for some

time. Am sorry you do not agree.

Let me give you an example:

Suppose we are dealing with pressure

and height in a certain cubic

area. Suppose that the middle of the

cube height is 0 meters. Also suppose

that we have a whirlpool generated in the

cube such that the pressure rate increases

as we go below the middle of the cube.

Anything below is negative height and anything above

is positive height. Now, as one rises

higher in the cube, the pressure decreases.

If we find the gradient, then according to

your definition (and many others), then

the gradient vector for the rate of greatest

increase will point below the middle of the

John Gabriel
http://flickr.com/photos/markfive



24.

25.

26.

27.

increase will point below the middle of the

cube, not above. But above the middle we

find the greatest decrease in rate of pressure.

In this example, greatest increase points

downwards and greatest decrease upwards.

It would probably be better to define

gradient as a vector that points in a

direction of greatest increase or decrease.

Its additive inverse will point in the

diretion of greatest decrease or increase

respectively. For most physical phenomena,

your definition would generally be true.

But what happens when you have an anomaly?

Make sense?

I do not believe I have the best answer to this question but like yourself, I am a believer in trying to find

the best possible explanation. Once again, I like your website. Keep up the good work Kalid!

John Gabriel

Okay, I think I have the best answer. If f is a real-valued function, then del(f) or gradient of f points to

the greatest increase, whereas -del(f) points t0 the greatest decrease.

For once planet math has some decent information on this since I last checked:

http://planetmath.org/encyclopedia/Gradient.html

I do not endorse everything Planet Math publishes but this particular information appears to be correct.

In any event, it clears up the previous confusion I think.

John Gabriel

Hi John, thanks for the comment! Yes, thats an important distinction to make: the positive gradient is

the greatest increase, and the negative gradient is the greatest decrease. Thanks for helping clarify :).

Kalid

Jared
http://www.betterexplained.com/http://planetmath.org/encyclopedia/Gradient.html



28.

29.

30.

31.

32.

33.

Thank you!

This actually makes sense to me. Thanks!

Bigmouth

@Jared, Bigmouth: Cool, glad it was helpful!

Kalid

did not grasp the idea

Anonymous

Be more specific. The gradient is the direction to move that gives you the biggest increase.

Kalid

It helps me a lot. But I have some doubt still now.Is it the same concept for gradient of each vertex in a

triangle mesh?

Thanks so much.

Shaheen

Kalid

Thanks for the great explanations! I thought I was math-retarded for some time; however your writings

actually make sense to me!

Take care!

Johnny T

JohnnyT



34.

35.

36.

37.

38.

39.

40.

@Shaheen: Thanks, glad you enjoyed it. Im not sure I understand the question: in a triangle mesh, you

could measure the gradient at each vertex to find the best direction to move. Again, not sure if this is

your question.

@Johnny T: Thank you for the comment! Yes, when a subject seems difficult (as vector calculus was for

me) sometimes its just because the explanation wasnt clicking properly. Thanks for dropping by.

Kalid

well done,excellent explaination with solid examples

wali khan

Thanks Wali, glad you enjoyed it.

Kalid

thanks

but i have some doubts.how the differentaion gives the maximum space rate of change. as per my

understandings differentiation only is difference between two point in the region say p1 and p2.can u

clarify

j.sathish kumar

Thanks a lot for explaining the concept.

leon

i was having so much trouble understanding this and now its all clear thank you so much!

sophie

@lon, sophie: Thanks, glad you enjoyed it!

Kalid

Ryan Johnson
http://betterexplained.com/



41.

42.

43.

44.

45.

46.

47.

48.

Jesus. This was a lot better explained than in my text book and by my professor. I thought we were using

the gradient as the normal vector but I really doubted that it could be that.

Ryan Johnson

@Ryan: Thanks! I struggled with this concept for a while also.

Kalid

thanks ! this explanation made me clear how to find the direction of smallest change.It is just the 90

degree rotation of gradiant(the direction of largest change).

Ranjeet Kumar

Thanks very much for your effort

Shakeel Ahmed

Um in your microwave example, arent you pushing the doughboy out the back of the microwave?

(Just wanted to understand the concept). I love these essays, btw, keep them coming!

Bill

I loved the microwave analogy.also thanks for clarifying the upsidedown delta now everything makes

more sense

Hehehe

stil im confused between scalar field and vector field.

RAHUL

how can such a mathematical expression denote the max change? pls i didnt understand the relation of

this with mathematics. pls reply sir.

aradhita chattopadhyay



49.

50.

51.

52.

53.

54.

thank you soo much!!

its a big help for our project

Can we have your number?hehe

nat2_bam2

@Rahul: A scalar field returns a single value (x), but a vector field returns multiple values (x,y,z).

Usually the multiple values (x,y,z) are taken as a direction to follow.

@aradhita: Hi, thats a question I need to get into in a later post.

@nat2_bam2: Thanks!

Kalid

Hi kalid! i read your explanation. oh this is very helpful! by the way can you give an example on how to

apply this on a situation of the classic mountain and mountain climber problem? hope you will reply.

thanks again your explanations were clear

Migs

@Migs: Great question. The classic mountain climber problem is when the vector field gives the

height of the mountain (z) at a certain position (x,y), so z = f(x,y).

The gradient at any position x,y will give you the direction of the _greatest increase_ in z. That is, the

gradient will point in the most uphill. Following the gradient will give you the shortest path the the

top of the mountain (technically, the top of the nearest local maximum). How this helps!

Kalid

beautifulwell said

vignesh

thanks a lot for the wonderful explanation!!!

akansha



55.

56.

57.

58.

59.

60.

61.

thanks a lot for the wonderful explanation!!!

@akansha: Youre welcome!

Kalid

Very nice! Keep up. Thanks a lot

anonymous

Very nice article!!

Hope to see how to find the maximum of a constrained function soon!!

Thanks a lot!!

Florencia

@Florencia: Glad you liked it! Thanks for the suggestion.

Kalid

Very good explanation by the way. So if you are on a landscape given by z=cosy-cosx and u want to get

from (0,0,0) to (4pi,0,0) by moving in the direction of the gradient in the positive x-direction how would

u explain that? What would that path look like?

ab

Thanks for the great explanation. Another topic that would be very interesting for you to cover is the

Jacobian, which causes pain for many, many students (including myself).

P-F

@P-F: Thanks for the note I think the Jacobian, and linear algebra in general, would be great to cover.

Ive forgotten a lot of it and am looking to relearning :).

Kalid

Mark Soric
http://www.islamalways.com/



62.

63.

64.

65.

66.

67.

68.

Just wondering something. In that case of f(x,y) = X^2 + y^2, a paraboloid how can the gradient by

perpendicular to the tangent plane at all point and only have components in x and y

gradF(X,Y) = 2x + 2y

How can it point in any other direction other than parallel to the xy plane?

Im lost here.

Mark Soric

thank you kalil. wonderful explanation.

prabu

@prabu: Glad it helped!

Kalid

It was a great explanation! But I have a specific problem with gradients. Is there any functions that cant

be expressed as gradient of any parameter? What could be the properties of that function?

Ashraful

May I could be more specific about my previous problem. If a function is constant in all direction, is it

possible to express the function as gradient?

Ashraful

Im not sure if I understand the question the gradient of a constant function would be a 0 vector

[perhaps technically (0,0)], that is, there is no direction of greatest increase. If it helps, think of the

gradient in terms of a derivative (the derivative of a constant function is 0).

Kalid

Math professional!

Kinar



69.

70.

71.

72.

73.

74.

75.

76.

Thank you for getting to the heart of why del is required and how to intuitively understand it. Its the

first time I understand it so well despite reading so much about it before!

Anonymous

damn! i got it now

Anonymous

math is so beautiful

Anonymous

WOW! great explanation. thanks dude.:D

bob clear

@bob: Thanks!

Kalid

@Anonymous: Agreed :).

Kalid

Great explanation helped me explain my brother! Nice job! Gonna bookmark it for further needs I might

have with it.

Jose

great explanation and example

js

Kalid



77.

78.

79.

80.

81.

82.

83.

@js: Thanks!

Kalid

hey, explained really well. But still you didnt provide any sign of why the gradient would always point in

the direction of maximum increase

shreedhar

I dont usually comment on blogs, but this is a great explanation. Way better than my text book.

A+++++++++

Nick Pellatz

@shreedhar: Thanks Id like to cover that in a follow-up article. I need to get a nice, intuitive

explanation for it first ;).

@Nick: Thanks, glad it helped.

Kalid

Man! I just love this kind of explanation. Its so clear and concise, and it shows me that the author really

understands the concept himself.

All mathematics should be taught this way. Go from the specific to the general (abstract). Not the other

way around, which is the path usually followed by the type who wants to show off his prowess with math

symbols and equations.

Al Paquette

nice explanation

Anonymous

dont eat that cookie!

Anonymous



84.

85.

86.

87.

88.

89.

90.

91.

@Al: Thank you! I think one of the big problems in math teaching (especially) is just trying to get things

explained without the professors prowess getting in the way, as you say.

@Anonymous: How could you eat cookies when theres gradients to be studied?

Kalid

Nice work!! Thanks man:)

Pandia

@Pandia: No prob!

Kalid

Thanks!

Gaton

Thanks alot,I loved your way explaining this, very helpful indeed.

Keep it up.

Marwa

@Marwa: Thanks, glad it helped!

Kalid

Great !! Congrats

Vitor P.F.

Awesome! I was cracking my head trying to figure out HW, only to realize how basic it was after reading

through ur page. Thanks!

Alex



92.

93.

94.

95.

96.

97.

goodluck with my exam tom. ^^

dianne

It wasnt a bad explanation but I wish you had explained why the gradient is the perpendicular vector

of the function its derivatives were derived from. This still bothers me a little.

Also, if we have a function with three variables, shouldnt the independent variable be considered? By

considered I mean, if I have a function F(x, y z), then I am saying that w = F(x, y, z), and this function

can not be graphed since it has 4 dimentions. A normal F(x, y) can be graphed since you considered the

Z, X, and Y of the graph.

From the book I read, I interpreted that the original function has a constant value for w, hence

producing a graph with a new function F2(x, y). However I still didnt see the math that proves that the

gradient of the function F(x, y, z) is actually the vector that is perpendicular to the surface of the graph

from which its derivatives were derived from. If you could prove this, it would be really helpful.

Anonymous

thx man very much

I understood it totally from u

my regards

mohamed

Thank you very much! This made perfect sense and it really helped me out.

Burton

@Burton: Thanks!

Kalid

Cool! Thanks

panchito
http://reallyniceexplanation/



98.

99.

100.

101.

102.

103.

104.

105.

good explain, it solved my problem

Donglin

kalid , are u professor

Anonymous

such wonderful explanation..wow

shrikant

@shrikant: Thanks!

Kalid

so very easy method

jayakumar.g

I love you!

Mrigeh

@jayakumar.g: Glad it helped

@Mrigeh:

Kalid

i want to ask, once knowing the maximum rate of change of temperature in yours microwave example,

how we can attain that particular place without moving our coordinates positions as mentioned by

microwave for example when we choose coordinates (3,5,2) we obtain gradient as (3,4,5). now from

where we get the information that which coordinates should be selected next time that gives us

maximum gradient? should we choose (3,4,5) coordinates?

hyaa
http://hadagali/



106.

107.

108.

109.

110.

111.

112.

maximum gradient? should we choose (3,4,5) coordinates?

@hyaa: Im sorry, I dont think I understand the question. The gradient gives you the direction (not

coordinates) of the greatest increase in your current value. You have to follow the gradient for a bit, get

to a new point, get the gradient there, follow it for a bit and so on to maximize your value.

Think of the gradient as a compass which points towards your greatest increase. A compass doesnt give

you the coordinates of North, but tells you how to get there from your current position. Hope that helps.

Kalid

Hi, I still have a question. If there is a function h(x,y)to denote the height of a mountain at position(x,y).

Can I use the knowledge of gradient to locate the top of the mountain and how?

Zita

@Zita: Yep you start at any point, and keep following the gradient of h to find the top.

kalid

great, really good thank you,it would be comprehensive if you explain that why the gradient is the

perpendicular vector of the function its derivatives were derived from

max

@max: Great question. Going to add it as a Q & A at the end of the article.

kalid

Excellent explanation, I think if you provide your ebook for free of cost it would really be helpful for the

poorer students to strengthen thier grass-roots.

GOOD JOB, KEEP IT UP.

Patel Ankit

Consider the directional derivative, f_u.

Chico



113.

114.

115.

116.

117.

Consider the directional derivative, f_u.

f_u = f_x u_1 + f_y u_2 (it takes some effort to see this definition of f_u)

=grad(f) dot u (u is a unit vector)

=|grad(f)| cos@ (@ is the angle between grad(f) and u)

Thus, it is clear that the directional derivative, f_u, is maxed when cos@=1.

It follows that @=0 and the directional derivative, f_u, is attained when u is in the direction of the

gradient. Therefore, the gradient does indeed give the direction of greatest increase.

Note that f_u is minimized when cos@=-1. Thus, @=pi, and u is in the opposite direction of the

gradient. QED

ps

I am a nerdy math professor who likes demonstrating mathematical prowess. Thanks for the microwave

intuition builder. My students are going to like that.

I already knew this but you gave me a better intuition of it and I like your style of writing! Thank you!

Deniz

@Chico: Awesome, thanks for sharing! I like that a lot lining up with the gradient (out of all possible

directional derivatives) will give you the best return (cosine = 1). That clicks for me.

Glad you enjoyed the microwave intuition, I love searching for little analogies.

kalid

@Deniz: Thanks! And youre welcome :).

kalid

ah highly informative and excellently defined

beant singh

@beant: Great, glad it helped!

kalid

madhu



118.

119.

120.

121.

122.

123.

that cookie tasted awsome

madhu

could u explain why does gradient is zero for local minimum?

madhu

Madhu: Those cookies are delicious :). The gradient behaves like the first derivative (rate of change). In

regular calculus, d/dx = 0 means your function is not changing [therefore you are at a max or min].

Similarly, when the gradient is zero, it means your function is not changing when you move.

kalid

wow gr8

Anonymous

Hey, could you explain WHY gradient points in the direction of maximum increase?

I mean given that this article is mindblowing but I still cant get Why maximum increase..

Also, could you explain a vector field? In case of scalar field, what I imagine is as follows: Consider the

3D space, and for each point my scalar function returns a value.. And with a intensity proportional to

that value, a black point appears(Greyscale).. I cant picture anything about a vector field though.. Please

consider..

And again, thanks for all the Vector calculus explanations..

Yogesh

Hi Yogesh, thanks for the note.

This article explains why the gradients points in the direction of greatest increase:

http://betterexplained.com/articles/vector-calculus-understanding-the-gradient

The essence is you have a circle of possible directions, the individual derivatives (df/dx and df/dy) give

you the tradeoff as you change directions, so find the direction that makes the best use of that tradeoff.

A vector field is tricky. Imagine your same 3d space, but instead of a point (a single value) imagine that

there is wind blowing through it. Each position in your space could feel a different push (strength and

kalid
http://betterexplained.com/articles/vector-calculus-understanding-the-gradient



124.

125.

there is wind blowing through it. Each position in your space could feel a different push (strength and

angle) from the wind. For example, in a cyclone, the push might be in a circle (each individual point is

pushed in a different instantaneous direction), with a dead spot in the middle. With a steady wind, every

point might feel the same push.

You are a gift to mankind! Well to me you are.

someone

People confused by gradient pointing towards maximally increasing direction, or perpendicularity of

gradient to equipotential.

When you have a function, you know what I mean by function, something that looks like a hill, whether

its a 2d hill you drew on the paper or a 3d hill you made out of clay on your table, stick your finger at a

single point on the hill. Now ask what is the change height of the hill as I move across the hill a little

bit in a flat direction? In the 2d case, you go left or right, the only flat direction. You trace the line

one inch to the right (right = positive), following it, and you find the line goes up (up = positive) one

inch, so you say the slope is one inch (up) per inch (to the right) near that point. The per means divide.

Positive / positive = positive. Going back to the left (negative) you say you went down (negative) one

inch. Neg/Neg = Positive. The gradient for this 2D example, since its defined a vector pointing along x,

can only point along the x axis, but which way? Left or right? The gradient also has a magnitude, which

can be very positive, a little positive, a little negative, or very negative. Its magnitude is whatever you

got for the slope. When the slope was positive, the gradient will point along x, for sure, but along the

positive direction. When the slope was negative, the gradient still points along x, but towards the

negative direction (left). So the gradient points along x towards the direction of increasing height,

whichever way it is. If the increasing height was to the left of your finger, you would find the slope was

negative there after defining left to be negative and up to be positive, and the gradient would point

negative along x, saying look at me, Im pointing the direction you travel along x to see the hill rise.

In the 3d case, the clay on your table, you stick your finger and soon realize you get different slopes

depending on if you move your finger forwards, backwards, left, right, or diagonal. Lets say you move

your finger to the left and it goes up. Then the portion of the gradient pointing to the left is the slope

your finger measured, since your finger went up moving that direction a component of the gradient

points somewhat in that direction. Then you go back to where you started from. You go forward, and

find your finger went up even more than it did going left. Lets say your finger went up two inches going

an inch forward, and it went up one inch going an inch to the left. We know a portion of the gradient

points forwards (cause your finger went up, not down), and about half that portion (you went up an inch,

not two inches like going forwards) points to the left. So the gradient points very forward, and a little

left (2 inch up per inch forward, pointing forward, 1 inch up per inch left, pointing left). You will notice

this is obviously uphill. It turns out it is EXACTLY uphill. If you go 2 forward for every 1 left you move

near that point, you will gain the most height possible. When you gain the most height possible, you are

moving exactly perpendicular to the direction you would not gain height if you were moving, aka the

Jose



126.

127.

128.

129.

130.

131.

132.

moving exactly perpendicular to the direction you would not gain height if you were moving, aka the

equipotential path, or path of same height.

its awsomeit clearly explains the actual physical significance of a gradient..thank you..:)

Baljeet Kaur

Hi Baljeet, glad it helped!

kalid

BRILLIANT EXPLANATION! Thanks a lot

jayakrishnan.k.j

thks for complete explanation

manju

I stumbled across your site looking for one specific aspect of gradients and ended up reading the whole

post. You did a great job of distilling these concepts Kalid.

Kevin @ http://kldavenport.com

Thanks Kevin, I appreciate it!

Kalid

I went over to Wikipedia and read an article on a similar topic. It was so much more difficult to

understand, and Wikipedia is easier than most math texts. It makes me so angry that most math books

seem to go out of their way to make mathematics unnecessarily difficult. Maybe, with more people

beginning to write internet articles like this, the math obfuscators wont be able to get away with it

much longer. I look forward to the day when students realize that math can be the easiest class in

school.

Rick
http://google/



133.

134.

135.

136.

137.

138.

139.

Thanks and keep up the good work Kalid!

Thanks Rick! I can relate to the feeling of frustration, its what drove me to write (after having an aha

momne, it quickly turned to: Why couldnt they explain it like that in the first place?).

I do think math has the potential to become the easiest subject. Its objectivity, which could be seen as

offputting, is a great indicator of when something has truly clicked. As a result, we can quickly

determine whether an analogy is helping solve the problem before us.

Appreciate the support :).

Kalid

Lovely!

Marco P

Jazak Allah khair

Khasud

An easy and interesting approach. Hats off!

Qammar Abbas

awesme dude!!gud work!!!well explainedws really helpful!

ATHUL P ANAND

Thank u so much,your explanations really helped- must confess am not all that good in mathematics I

could really use some more help.but thank you

lashiwe

Kalid
http://betterexplained.com/articles/vector-calculus-understanding-the-gradient/



140.

Another explanation that I posted on reddit:

I imagine a vector field like a grid. Off in the distance is a billboard showing the amount of money well

get for being at a certain position on the grid (value of the function).

From where were standing, we can take a step in any direction. The work is the same (i.e., I moved 1

unit) but the payoff (net amount of cash we gain) can differ drastically depending on if we go North,

West, South, NorthEast, etc.

The gradient is the payoff in the sense it points to the direction of greatest reward (if its zero, it

means you are already at the max reward, i.e. any step you take will diminish your earnings).

A surface like z = 3 is basically saying show me all the positions in this grid where the reward is $3.On the grid a path is drawn, highlighting all the positions of this equal payoff. If you started following

this path, your payoff would never change.

But what would the gradient be? The gradient is pointing in the direction of greatest increase, so

should have nothing in common with the path of 0 increase. In other words, the projection of the

gradient onto the surface should be 0, i.e. its normal. The gradient has zero inclination for you to go

anywhere near the path of zero gains.

Ok, fine, thats what the gradient should do. How do we show it actually maximizes the payoff?

Heres how I figure it: on a circle (showing all possible paths), we can basically make any tradeoff of x

and y that we want. At 45-degrees we can trade them 1-for-1, at higher values we can get 2 units x-

distance for 1 unit of y distance, or 10 to 1, or a million to 1 (at angles close to 0 or 90).

The direction of the gradient is calculated to maximize the tradeoff based on dz/dx and dz/dy, i.e. it

figures out how much reward we get for moving in each direction and allocates effort appropriately.

If we get $20 for moving the x direction and $10 for moving in the y direction, then our direction should

favor x, but only at a 1:2 tradeoff. I.e., if we can trade 1 x for 3 ys then we should keep trading (adjusting

the angle) until we get 1x for 2ys.

In other words, you can prove that the gradient direction is the direction which maximizes z assuming

you are moving 1 infinitesimal unit and are getting rewarded by dz/dx and dz/dy. And by definition, this

profit-maximizing direction would not waste any energy along the profit-maintaining path that must

have both dz/dx and dz/dy of 0 (the equal-valued path must not change the amount of z).

Really great explanation! The only thing is that in your definition of the del the partial derivatives

should have a lower-case delta ( instead of d

Rob
http://n/a



141.

142.

143.

144.

145.

146.

147.

should have a lower-case delta ( instead of d

Really helpful

I would be really glad if you could tell me the derivation for the formula for gradient of a scalar in terms

of the nabla/del operator

Or if you could tell a link where I can find it

Ajinkya

Hi kalid, it is a great explanation at lest for people like me.

I think one should get this overview before getting into the actual concept.

thanks a lot!

pari

Thanks a bunch !

Naveen

very good article

Rajmohan

MashaAllah you posted this in 2007and to this date people are getting benefit from it. Loved it. Thank

you

Aymun

what is greens theorem? can anyone explain.

Anonymous

Before reading this, I wasted 2 days for this gradient!!

but the world of mathematics is very very very interesting when people like you teaching us.

thanks a lot.

sanu



148.

149.

150.

151.

152.

153.

Awesome work But if you can help me with this problem,? My professor assigned me physical

significance of gradient that would be my first assignment confused what to submitt? ??

(application/physical significance)

Osama

Good explanation

Amir sultan

Thank you so much

Taps patel

Thanks a lot. this really helped me understand it better!

J Scrib

A gradient is Vector differentiation operator applied on a scalar function. Not strictly the derivative of

vector functions as you said in opening paragraph, both are different. However, gradient can be treated

as a derivative of a special vector for which all the vector components have same function, whose

magnitude is the scalar function and directed in the direction of f_x=f_y=f_z from notation f(vector) =

(f_x,f_y,f_z).

Manikanta

Iam adding this Just to clarify the things,

1. Derivative of a vector function is called divergence.

reference:http://ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter09/section01.html,

2. Gradient of a vector function is called Jacobian.

reference:http://en.wikipedia.org/wiki/Gradient#Gradient_of_a_vector

Manikanta
http://en.wikipedia.org/wiki/Gradient#Gradient_of_a_vectorhttp://ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter09/section01.html

vector calculus_ understanding the gradient _ betterexplained

Documents

component gradient

dfdzthe gradient

gradient points

single variable

single derivative

multivariable function

singlevariable functions

derivative of vector