Searching in Continuous Space

Louis Oliphant

oliphant@cs.wisc.edu

cs540 section 2

Environment Types

Up until now we have been working in discrete environments

given a state, there is a clear, finite set of neighbors

Now we will look at continuous environments

No clear, finite set of neighbors

Still trying to maximize an objective function f(x)

Different Approaches

Discritize the state space

Not much to say about this

Use the derivative of f(x)

solve for f '(x)=0

gradient descent

Newton's method

Discritizing State Space

Binning (one method of)

For each continuous variable in state space divide its domain into a fixed number of equally spaced bins

Use standard discrete search methods

Using the Derivative

Remember when f '(x)=0 then f(x) is:

at a maximum

at a minimum

at a point of inflection

Solve f '(x)=0

So Really we need to find the roots of f '(x)

Check each x in f(x) for a maximum

or check f ''(x) for each x

f ''(x)==0 then point of inflection

f ''(x)==positive then minimum

f ''(x)==negative then maximum

Using the Derivative

What if you can't solve f '(x)=0 but you can still figure out the derivative?

Use Gradient Descent

Use Newton's Method

Gradient Descent (Ascent)

pick an initial value for x

pick a step size

if f '(x) is:

positive – move right one step size

negative – move left one step size

Repeat until sign of f '(x) changes or f '(x)=0.

if f '(x)=0 then return x else return (xi+xi-1)/2.

try it with x=2, step size=0.25

Picking a Step Size

Step size to small

take forever to reach maximum

Step size to large

“step over” the maximum

Picking a Step Size

Use a varying step size

Start by using a large step-size and find the range where a maximum exists

repeat using a smaller step-size inside that range

continue this until you have enough precision

Alternate method

Start by using a small step-size

As long as the sign of f '(x) doesn't change double the step size

When sign changes drop back to initial small step size and continue search

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

xo=2

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

xo=2

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

xo=2

Assuming g(x) is a line

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

xo=2

x1=0.2

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

x1=0.2

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

x1=0.2

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

x1=0.2

Assuming g(x) is a line

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

x1=0.2

x3=1.1

Newton's Method of finding Roots

We are trying to find the roots of some function g(x).

Remember this is the derivative of the Real function that we want to maximize

Assume g(x) is linear and that you know the value of g(x) and g'(x) for your initial guess, x0

x3=1.1

Newton's Method of finding Roots

g(x) and g'(x) are given

xo=initial guess of root

repeat

until little change made in x

Multi-dimensional Environments

f(x) but now x is a vector
x={x1,x2,...,xn}

Now use the Gradient of f(x) written as f

f is just a vector of the partial derivatives of f

gradient always points to higher ground

Multi-dimensional Environments

Multi-dimensional Environments

Multi-dimensional Environments

Follow the direction of the gradient

Take small step sizes until the gradient is zero

You can also vary the step size just like before

Conclusions

Binning

Solve for f '(x)=0

Gradient descent

Newton's method

Understand what the Gradient of
a multi-dimensional function is:

always points to higher ground

Project Ideas – Ant Colony Optimization

Project Ideas – Root Finding Methods

2005-09-21T14:40:26