derivative of 2-differentiable function w.r.t. unit vectors

cantor.set · New Member Offline Joined: 07 Oct 2009 Posts: 9

Suppose $g(x)$ is a twice differentiable function in the open neighborhood of the origin, and let $\mathbf{x}$ and $\mathbf{y}$ be two unit vectors in $\mathbb{R}^n$ , not necessarily orthogonal to each other. Show that in an open neighborhood of the origin, $\partial_{\mathbf{x}} \partial_{\mathbf{y}} g(\mathbf{0}) = \partial_{\mathbf{y}} \partial_{\mathbf{x}} g(\mathbf{0})$ . Why is it to be expected that if $\mathbf{0}$ is a local maximum of the function $g$ , then usually we will have $\partial_{\mathbf{x}} \partial_{\mathbf{x}} g(\mathbf{0}) < 0$ ? Demonstrate an instance where this is not so.

**Posted:** Thu Oct 29, 2009 12:55 pm

cantor.set · New Member Offline Joined: 07 Oct 2009 Posts: 9

uhhhh sry maybe this should be moved to the unsolved probs section.

sry.

**Posted:** Thu Oct 29, 2009 1:05 pm

Kent Merryfield

For a sufficiently smooth $f,$ we have

$f(a+\mathbf{u})=f(a)+Df\cdot\mathbf{u}+\frac12\mathbf{u}^TH\mathbf{u}+o(\|\mathbf{u}\|^2)$

where $H$ is the Hessian matrix consisting of all of the second partial derivatives of $f.$ Note that $Df$ and $H$ are evaluated at $a$ in the statement above. Note also that $H$ is a symmetric matrix.

One directional derivative works out to

$\partial_{\mathbf{x}}f(a+\mathbf{u})=Df\cdot\mathbf{x}+\frac12(\mathbf{x}^TH\mathbf{u}+\mathbf{u}^TH\mathbf{x})+o(\|\mathbf{u...$

$=Df\cdot\mathbf{x}+\mathbf{x}^TH\mathbf{u}+o(\|\mathbf{u}\|)$

where we just used that $H$ is a symmetric matrix.

A second directional derivative gives

$\partial_{\mathbf{y}}\partial_{\mathbf{x}}f=\mathbf{x}^TH\mathbf{y}$

Since $\mathbf{x}^TH\mathbf{y}=\mathbf{y}^TH\mathbf{x},$ it's not going to matter which order we take this in. That answers your first question.

As for the second question: there is no good reason at all to suppose that we "usually" have $\partial_{\mathbf{y}}\partial_{\mathbf{x}}f(0)<0$ at a maximum. In fact, $\mathbf{x}^TH\mathbf{y}$ is an inner product of $\mathbf{x}$ and $\mathbf{y}.$ It's bilinear. Merely taking $-1$ times one of those vectors will reverse the sign of that inner product. Generically, if we choose $\mathbf{x}$ and $\mathbf{y}$ at random, the probability that $\partial_{\mathbf{y}}\partial_{\mathbf{x}}f(0)<0$ should be $\frac12,$ and that's the same whether the point is a maximum, a minimum, a saddle point, or not even a critical point at all.

**Posted:** Fri Oct 30, 2009 8:42 pm