Planning (Chapter 11.1 - 11.4)

The Problem

• Find a sequence of actions that achieves a given goal when performed starting in a given state
• In other words, given a set of operator descriptions (defining the possible primitive actions by the agent), an initial state description, and a goal state description or predicate, compute a plan, which is a sequence of operator instances, such that "executing" them in the initial state will change the world to a state satisfying the goal-state description.
• Goals are usually specified as a conjunction of goals to be achieved
• Assumptions
  • Atomic time: Each action is indivisible
  • No concurrent actions allowed
  • Deterministic actions: Result of each action is completely determined by the definition of the action, and there is no uncertainty in performing it in the world
  • Agent is the sole cause of change in the world
  • Agent is omniscient -- has complete knowledge of the state of the world
  • Closed World Assumption -- everything known to be true in the world is included in a state description. Anything not listed is false.

Some Possible Approaches

• Situation Calculus
  • Augment FOL so that it can reason about actions in time
  • Add situation variables to specify time. A situation is a snapshot of the world at an interval of time when nothing changes
  • Every true or false statement is made with respect to a particular situation. For example, instead of predicate P(x1, ..., xn) use P(x1, ..., xn, s) to mean that P is true in situation (i.e., state) s. Alternatively, add a special predicate holds(f,s) that means "f is true in situation s."
  • Add a new special function called result(a,s) that maps current situation s into a new situation as a result of performing action a. For example, result(grasp, s) is a function that returns the successor state (situation) to s
  • Example
    The action agent-walks-to-location-y could be represented by
    (Ax)(Ay)(As)(at(Agent,x,s) ^ ~onbox(s)) -> at(Agent,y,result(walk(y),s))
  • The Frame Problem
    Actions in Situation Calculus describe what they change, not what they don't change. So, how do we know what's still true in the new situation?
    • To fix this problem add a set of frame axioms that explicitly state what doesn't change.
    • Example
      When the agent walks to a location, the locations of most other objects in the world do not change. So, for each such object, e.g., a bunch of bananas, add an axiom like:
      (Ax)(Ay)(As)at(Bananas,x,s) -> at(Bananas,x,result(walk(y),s))
    • Find a solution plan by posing a query to be solved by the theorem prover (e.g., using the resolution algorithm). For example,
      (Es)on(A,B,s) ^ on(B,C,s) is a query that, after resolution and answer extraction results in an answer of the form:
      result(stack(A,B), result(stack(B,C), result(unstack(A,C), S0)))
      meaning that starting in the initial situation S0 and performing the plan steps unstack(A,C), stack(B,C), stack(A,B) will result in a situation that satisfies the query.
    • The problem with this approach is there must be lots of frame axioms, and whenever a new predicate is added, every operator must be updated to add appropriate frame axioms referring to the predicate. So, it's not a modular solution.
  • Situation calculus is representation strong, but reasoning weak because of the poor efficiency of resolution and other reasoners that use lots of very fine-grained units of information (the clauses).
• State-Space Search
  • Let initial state = initial situation, goal-test predicate = goal state description, and successor function computed from the set of operators. Once goal is found, solution plan is the sequence of operators in the path from the start node to the goal node.
  • In search, operators are used simply to generate successor states and we could not look "inside" an operator to see how it's defined. Consequently, heuristic functions could only be applied to states in order to rank states according to some estimate of their distance to the goal.
  • Goal-test predicate also is used as a "black box" to test if a state is a goal or not. Cannot use properties of how a goal is defined in order to reason about finding path to that goal.
  • Hence, this approach is all algorithm and representation weak.

Planning Emphasizes What's in Operator and Goal Representations

• "Open up" representation of state, goals and operators so that reasoner can more intelligently select actions when they are needed
• State/Situation Representation
  A conjunction of "facts," (i.e., ground literals that do not contain variable symbols). For example, {pyramid(C), on(A,B), on(B,Table), clear(A), handempty} where the commas implicitly represent conjunction.
• Goal Description
  A conjunction of positive literals. Any variables are assumed to be existentially quantified. For example, {on(A,B), on(B,C)} where the comma represents conjunction.
• Operators
  Operators usually given as "STRIPS-type" if-then rules:
  1. Operator/Action Name.
  2. Preconditions: conjunction of positive literals. Represents what must be true (immediately) before the action is done. Hence defines when the action is legal/applicable to be performed.
  3. Effects: conjunction of positive (called the add list) and negative (called the delete list) literals.
• Example
  

    Name:  pickup(x)
    Preconditions:  ontable(x), clear(x), handempty
    Add list:  holding(x)
    Delete list:  ontable(x), clear(x), handempty
  

  Note: Instead of the two Add and Delete lists, we could have specified a single "Effects list:"

    Effects list: holding(x), ~ontable(x), ~clear(x), ~handempty
  

• The frame problem is handled with these operators by assuming that everything that is true in the state where an action is performed will still be true after that action unless it is on the Delete list. And all things that were not true in the state where the action is performed but are true after the action, are included in the Add list.

Planning as Search

Two main approaches to solving planning problems, depending on the kind of search space that is explored:

• Situation-Space Search
  The search space is the space of all possible states or situations of the world. Initial state defines one node. A Goal node is a state where all goals in the conjunctive goal list are satisfied. A solution plan is the sequence of actions (i.e., operator instances) in the path from the start node to a goal node.
• Plan-Space Search
  The search space is the space of all possible (partial) plans. A node corresponds to a partial plan. Initially, we will specify an "initial plan," which is one node in this space. A goal node is a node containing a plan which is complete, satisfying all of the goals in the conjunctive goal list. The node itself contains all of the information for determining a solution plan (i.e., sequence of actions). Arcs connecting nodes in this space corresponds to "plan transformations" that add/delete/modify one plan to produce a new plan.

Situation-Space Planners

Two approaches to situation-space planning:

• Progression
  
  • Forward chaining from initial state to goal state
  • Start from the initial state, find all operators whose preconditions are true in the initial state, "compute effects" to generate successor states, and repeat. Stop when a new state satisfies goal conditions. Use any search method you like, e.g., BFS, DFS, A*.
  • Simplest application of the STRIPS representation
  • Looks just like state-space search except STRIPS operators specified instead of a set of next-move functions
  • Problem with this approach: Huge search space to explore, so usually very inefficient
  • Algorithm for generating successor state T from state S
    1. Select operator O such that all Preconditions(O) unify with literals in state S using most general unifier (mgu) u.
    2. If no such operator, then fail.
    3. T = S ; copy S to T
    4. T = T - (Di)u for all Di in Delete-list(O), and Du means to make substitutions u into literal D ; remove Delete-list literals from T
    5. T = T + (Ai)u for all Ai in Add-list(O) ; add Add-list literals to T
  • Example
    S = {clear(A), clear(B), clear(C), ontable(A), ontable(B), handempty}

    O = pickup(x) with u = {x/A} means

    T = {clear(B), clear(C), ontable(B), ontable(C), holding(A)}

    and the operator instance associated with the action from S to T is pickup(A).

  • Generating successor state is slightly more complicated if a variable occurs in the Add-list and/or Delete-list that does not occur in the Preconditions list. In this case there may be multiple possible ways to unify with these variables in the Add-list and Delete-list. Represents another choice point for the search algorithm, since we have to consider all possible consistent substitutions v for these variables in order to create successor states that are both consistent and contain only ground literals.
• Regression
  
  • Backward chaining from goal state to initial state.
  • Start with goal node corresponding to the conjunction of goals to be achieved, choose an operator that will add one of the goals, then replace that goal with the operator's preconditions, and repeat.
  • Usually more efficient than progression because many operators are applicable at each state, yet only a small number of operators are applicable for achieving a given goal. Hence, more goal-directed than progression.
  • Start from goal state defined by the conjunctive goal list
  • Generating a predecessor state by replacing one goal by an operator's preconditions is not correct because that operator may make other changes (defined by its Add and Delete lists) that cause other goals to also be solved (good luck) or to be negated (bad luck because we create a contradiction of the form P ^ ~P in the new state). So, we need to verify how all goals are affected by the selected operator, as indicated by the following algorithm.
  • Algorithm for generating predecessor state S from state T
    1. Select operator O such that unify(Add-list(O), Gi) = u where Gi is the ith literal in state T
    2. If no such operator, then fail
    3. S = (Pi)u + (Gj')u for all i,j where Pi is the ith precondition of O, Gj is the jth literal in T, and Gj' is the regression of Gj through O as defined by:

       Gj' = true, if (Gj)u in (Add-list(O))u
       Gj' = false, if (Gj)u in (Delete-list(O))u
       Gj' = (Gj)u, otherwise

STRIPS Algorithm -- A Goal-Stack Based Regression Planner

• Instead of using the regression algorithm for generating a predecessor state and backward-chaining from goal to initial state, uses a goal stack to maintain the set of conjunctive goals yet to be achieved
• Maintains a description of the current state, which is initially the given initial state
• Approach: Pick an order for achieving each of the goals, then when each goal is popped from the stack, if it is not already true in the current state, push onto the stack an operator that adds that goal. Then, push onto the stack each of the preconditions of the operator in some order, and solve each of these sub-goals in the same fashion. When all of the preconditions are solved, and hence popped from the stack, re-verify that all of the preconditions are still true in the current state, and, if so, then apply the operator to create a new successor state description.
• Algorithm

  procedure strips(I, G1 ^ ... ^ Gn, OPS)
    ;; I = initial-state
    ;; G1 ^ ... ^ Gn = conjunctive goal list
    ;; OPS = list of operators
    push(achieve(G1, ..., Gn), Goal-Stack)
    loop:
      if empty(Goal-Stack) then success
      N = pop(Goal-Stack)
      if N of the form "achieve(g1, ..., gm)" then
         begin
  	 if N true in state I then goto loop
  	 if all possible orders of gi already attempted, then fail
  	 push(N, Goal-Stack)
  	 Choose a new order for the gi
  	 push(achieve(gi), Goal-Stack) in selected order
         end
      else if N of the form "achieve(g)" then
         begin
  	 if N true in state I then goto loop
  	 Choose an operator O from OPS that possibly adds g
  	 If none, fail
  	 Choose a set of variable bindings that makes O add g
  	 push(apply(O), Goal-Stack)
  	 push(achieve(preconditions(O)), Goal-Stack)
         end
      else if N of the form "apply(O)" then
         I = apply(O,I)  ;  apply operator O in state I to
                         ;  produce successor state
  

• Notes
  • If the order of solving a set of goals (either the original goals or a set of sub-goals which are the preconditions of an operator) fails because solving a later goal undoes an earlier goal, then this version of the STRIPS algorithm fails.
  • While backward-chaining is performed by STRIPS in terms of the generation of goals, sub-goals, sub-sub-goals, etc, operators are used in the forward direction to generate successor states, starting from the initial state, until a goal state is found.
  • If goal G1 is above goal G2 on the goal stack, then if a solution is found, the plan will contain all of the steps associated with solving G1 before all of the steps associated with solving G2. Hence, STRIPS does not allow for interleaving of steps in any solution that it finds.
  • Re-verifying a conjunctive goal list is STRIPS way of checking for goal interactions whereby solving a later goal undoes the solution to an earlier goal.
• Example
  • Operators
    1. Pickup(x)

        P (Preconditions): ontable(x), clear(x), handempty
        E (Effects): holding(x), ~ontable(x), ~clear(x),
       	      ~handempty
       

    2. Putdown(x)

        P: holding(x)
        E: ontable(x), clear(x), handempty, ~holding(x)
       

    3. Stack(x,y)

        P: holding(x), clear(y)
        E: on(x,y), clear(x), handempty, ~holding(x), ~clear(y)
       

    4. Unstack(x,y)

        P: clear(x), on(x,y), handempty
        E: holding(x), clear(y), ~clear(x), ~on(x,y),
           ~handempty
       

  • Initial State

    clear(B)      -------
    clear(C)      |     |
    on(C,A)       |  C  |
    ontable(A)    |     |
    ontable(B)    -------     -------
    handempty     |     |     |     |
                  |  A  |     |  B  |
                  |     |     |     |
               -------------------------
    

  • Goal

    on(A,C)       -------
    on(C,B)       |     |
    	      |  A  |
    	      |     |
    	      -------
    	      |     |
    	      |  C  |
    	      |     |
    	      -------
    	      |     |
    	      |  B  |
    	      |     |
               -------------
    

  • Steps of Algorithm
    1. Goal Stack = [achieve(on(C,B), on(A,C))]
       State = {clear(B), clear(C), on(C,A), ontable(A), ontable(B), handempty}
       Note: Stack contains original conjunctive goal to be achieved, and State contains initial state description.
    2. Stack = [achieve(on(C,B)), achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Ordering of two goals chosen arbitrarily.
    3. Stack = [achieve(clear(B), holding(C)), apply(Stack(C,B)), achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Top goal was popped and the Stack operator was selected as one that could solve that goal (because it's Add-list contains a literal that unified with the goal on(C,B). The preconditions of this goal were then pushed on the stack.
    4. Stack = [achieve(holding(C)), achieve(clear(B)), achieve(clear(B), holding(C)), apply(Stack(C,B)), achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Sub-goals ordered arbitrarily.
    5. Stack = [achieve(handempty, clear(C), on(C,y)), apply(Unstack(C,y)), achieve(clear(B)), achieve(clear(B), holding(C)), apply(Stack(C,B)), achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Choose operator Unstack.
    6. Stack = [achieve(clear(B)), achieve(clear(B), holding(C)), apply(Stack(C,B)), achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State = {clear(B), ontable(A), ontable(B), holding(C), clear(A)}
       Note: Top goal achieve(handempty, clear(C), on(C,y)) true in current state with {y/A}, so pop and apply step Unstack(C,A) to create new successor state of initial state. Plan so far = [Unstack(C,A)].
    7. Stack = [achieve(on(A,C)), achieve(on(C,B), on(A,C))]
       State = {ontable(A), ontable(B), clear(A), on(C,B), clear(C), handempty}
       Note: Popped top goal because it is true in current state. Then verified that conjunctive goals defining the preconditions of the operator Stack(C,B) are all still true in current state, so popped that and then applied the operator to generate a new successor state. Plan so far = [Unstack(C,A), Stack(C,B)].
    8. Stack = [achieve(clear(C), holding(A)), apply(Stack(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Choose operator Stack to solve goal popped from top of goal stack.
    9. Stack = [achieve(holding(A)), achieve(clear(C)), achieve(clear(C), holding(A)), apply(Stack(A,C)), achieve(on(C,B), on(A,C))]
       State same as previous step.
       Note: Order sub-goals arbitrarily.
    10. Stack = [achieve(ontable(A), clear(A), handempty), apply(Pickup(A)), achieve(clear(C)), achieve(clear(C), holding(A)), apply(Stack(A,C)), achieve(on(C,B), on(A,C))]
        State same as previous step.
        Note: Choose operator Pickup to solve goal popped from top of goal stack.
    11. Stack = [achieve(clear(C)), achieve(clear(C), holding(A)), apply(Stack(A,C)), achieve(on(C,B), on(A,C))]
        State = {ontable(B), on(C,B), clear(C), holding(A)}
        Note: Top goal true in current state, so pop it and apply operator Pickup(A). Plan so far = [Unstack(C,A), Stack(C,B), Pickup(A)].
    12. Stack = [achieve(on(C,B), on(A,C))]
        State = {ontable(B), on(C,B), on(A,C), clear(A), handempty}
        Note: Top goal achieve(clear(C)) true, so pop it. Re-verify that conjunctive goals that are the preconditions of the Stack(A,C) operator are still true, then pop that and the operator is then applied. Plan so far = [Unstack(C,A), Stack(C,B), Pickup(A), Stack(A,C)].
    13. Stack = []
        State same as previous step.
        Note: Re-verify that both original goals are true in current state, then pop and halt with empty goal stack and state description satisfying original goal.

Goal Interaction

• Most planning algorithms assume that the goals to be achieved are independent or nearly independent in the sense that each can be solved separately and then the solutions concatenated together (assuming independence) or else there is a small amount of interaction between sub-goals that can be detected and fixed locally in the plan.
• Sussman Anomaly
  The following planning problem, called the "Sussman Anomaly," is the classic example of the goal interaction problem:

  				-------
  				|     |
  				|  A  |
  				|     |
    -------                       -------
    |     |                       |     |
    |  C  |                       |  B  |
    |     |                       |     |
    -------    -------            -------
    |     |    |     |            |     |
    |  A  |    |  B  |            |  C  |
    |     |    |     |            |     |
  ----------------------        -----------
  
      Initial State              Goal State
  

  Solving on(A,B) first (by doing unstack(C,A), stack(A,B) will be undone when solving the second goal on(B,C) (by doing unstack(A,B), stack(B,C)). But, similarly, solving on(B,C) first will be undone when solving on(A,B). Hence, either order of solving the two goals means the plan that solves the second goal inadvertently undoes the solution to the first as a side-effect.

• The STRIPS algorithm given above cannot solve the Sussman Anomaly because it assumes no goal interaction (though it does test for goal interaction when it does the "verification" step for a conjunctive goal)
• STRIPS could be made to solve the Sussman Anomaly if it tried to resolve goals that were undone after the verification check fails.

Copyright © 1996-2003 by Charles R. Dyer. All rights reserved.