Week 12: Informed and Adversarial Search

Best-first search uses a given heuristic to prioritize which node to visit next. You can imagine that it's similar to breadth-first search, only instead of expanding its search frontier in increasing layer order, it goes in increasing order of heuristic value (lower is better).

def best_first_search_iterative(start:SearchState) -> SearchState:
    pqueue = [start]
    while pqueue:
        # Removes last element and returns it
        first = stack.pop()
        if first.is_goal():
            return first
        for next_s in first.next_states():
            pqueue.append(next_s)
        # Put cheapest states earliest
        pqueue.sort(key=lambda s: s.heuristic(), reversed=True)
    return None

You can see for yourself that this follows the iterative search template! The only difference is that it does some sorting of the queue. We can avoid re-sorting the entire queue if we use a specialized structure called a PriorityQueue (then we just use pop() and append() and the code is exactly like depth-first or breadth-first search), but we'll leave that for a future CS course.

Best-first search has the flaw that it's easily misled by a heuristic that's too optimistic. Challenge problem: Thinking of finding a path through a maze, can you come up with a case where "prefer the spot closest to the goal" will make best-first search perform poorly? Can you outline a tweak to best-first search that will help it perform better in cases like that?

Assignment 10 used a simpler mechanism for guiding search with a heuristic, taking advantage of the structure of Sudoku problems: instead of considering every empty square, just pick a square with the fewest possible candidate values. This is a special case of the more general "most constrained variable" heuristic for search and constraint satisfaction problems. The following code works about the same way, except it doesn't refuse to consider other spots; in problems like Sudoku, it doesn't really matter what order squares are filled in, but in some other problems the order might be important.

def guided_dfs(state):
   if is_goal_state(state):
       return True
   nexts = successor_states(node):
   nexts.sort(key=lambda s: s.heuristic())
   for next_state in nexts:
      if guided_dfs(next_state):
         return True
   return False

Challenge problem: Come up with a search problem where guided_dfs and best_first_search will traverse nodes in a different order.

A* is a popular search algorithm which combines the best aspects of best-first-search (taking promising routes earlier) and Dijkstra's algorithm (which is like our best-first outline above, but sorts states based on how "far away" they are from the starting point). It is especially well-suited to path planning but works well for any search problem where actions have costs and the problem is to find a lowest-cost plan.

The key idea of A* is that instead of sorting by the heuristic (as in best first) or by the distance from the start (as in Dijkstra's), we should sort states by the sum of the heuristic and the distance so far. This strikes a balance between preferring paths we know are short and preferring paths which seem likely to reach the goal soon.

We've read about several heuristics so far—straight-line distance to the goal in a maze problem, using the number of valid values for a variable as the heuristic—but we don't have much practice coming up with heuristics.

One useful way to come up with heuristics is to imagine a simpler version of the problem and solve that. For example, in the eight-puzzle example we looked at a simpler problem where tiles could just be swapped arbitrarily, and in path planning we can imagine we can walk through walls. It's ideal if the heuristic we come up with under-estimates the true cost, but not by too much—we want it to be optimistic, but not overly so (and we consider heuristics that get closer to the true cost to be "better"). We considered earlier what can happen if the heuristic is too greedy, but it's also a problem if the heuristic is too pessimistic.

Challenge problem: Imagine a search problem (for example, maze navigation) where the heuristic is too pessimistic, for example that it never gets smaller than "3 steps from the goal" no matter how close the tile is to the goal (so for spaces three or more away from the goal it's perfectly correct, but for spaces that are closer it's an overestimate). Can you design a maze where following this wrong heuristic with best first search will lead to a worse path than using depth first search?

The "most constrained variable" heuristic is especially interesting for constraint satisfaction problems (where we're trying to assign values to variables, e.g. columns to queens) since it seems to be pretty good regardless of the specific problem. If we can represent a search problem as one of giving values to variables, we can unlock a variety of relatively general-purpose heuristics like this. Can you think of any others?