Overview

We prove a real algorithm---insertion sort---correct. This chapter is a taste of what Coq is actually used for: reasoning carefully about critical algorithms.

The Insertion-Sort Program

Our work this week will be proving a sort correct. We'll write a function sort that takes a list of natural numbers and rearranges it to be in ascending order. The algorithm we'll implement is called insertion sort, which works in the following way:

• to insert an element i into an already sorted list l, simply walk down the list until we find an item greater than or equal to i---then put i into the list; and
• given an unsorted list, insert each element into the recursive result of sorting the list.

Does our sorting function work? We can try a few examples:

The tail of this list, 12::14::18::nil, is not disturbed or rebuilt by the insert algorithm. The nodes 1::3::4::7::_ are new, constructed by insert. If you're having trouble following exactly how the algorithm works, try working out how the following evalutes on the board:
    sort [3;2;1].

Simply believing that this algorithm works isn't enough. Let's prove it correct!

Permutations

A permutation of a list is a rearrangement of its elements. We can define permutations using the following inductive proposition. We'll worry about whether or not this is exactly the right proposition later on.

Exercise: 1 star, standard (Permutation_refl)

Exercise: 1 star, standard (Permutation_length)

How might Permutation_length look as in informal proof?

• Theorem: if l₁ is a permutation of l₂, then l₁ and l₂ have the same length.

Proof: Let lists l₁ and l₂ be given such that l₁ is a permutation of l₂. We go by induction on the derivation of Permutation l₁ l₂. There are four cases.

• (perm_nil) We have l₁ = [] and l₂ = [], both of which have length 0.
• (perm_skip) We have l₁ = x::l₁' and l₂ = x::l₂' and Permutation l₁' l₂'; our IH shows that length l₁' = length l₂'. We have length (x::l₁') = length (x::l₂') immediately by the IH.
• (perm_swap) We have l₁ = y::x::l and l₂ = x::y::l. We have length (y::x::l) = length (x::y::l) immediately.
• (perm_trans) We have l₁ = l and l₂ = l'' and a list l' such that Permutation l l' and Permutation l' l''; our IHs show that length l = length l' and length l' = length l''. We can find length l₁ = l₂ by transitivity of equality and our IHs. Qed.

Notice that we get IHs in the perm_skip and perm_trans cases but not in perm_nil and perm_swap. Why is that? Induction hypothesis come from "smaller" things---for nats, we get an IH for the n = S n' case because there's a smaller nat involved---n'. For inductive propositions, we'll get an IH when there's a recursive reference to the proposition we're defining. Both perm_skip and perm_trans refer back to the Permutation definition as a premise (and so we get an IH) but perm_nil and perm_swap don't (and so we don't get an IH). More generally, induction on propositions works as follows. We have the following rules:

	(perm_nil)
Permutation [] []

Permutation l l'	(perm_skip)
Permutation (x::l) (x::l')

	(perm_swap)
Permutation (y::x::l) (x::y::l)

Permutation l l     Permutation l' l''	(perm_trans)
Permutation l l''

We want to prove that for all l₁ and l₂, if Permutation l₁ l₂ and then length l₁ = length l₂. Having let l₁ and l₂ be given such that Permutation l₁ l₂, we go by induction on the derivation to show that length l₁ = length l₂. What cases do we have? When do we have an IH and where does it come from? We will have one case for each rule. If the rule is of the form Permutation a b, then we will have to show our goal for a and b. Here, that means showing that length a = length b. So in the perm_nil case, we have to prove that length [] = length [], since perm_nil only applies when both l₁ = [] and l₂ = []. In perm_trans, we'll get to keep our hypotheses (that Permutation l l' and Permutation l' l'') and we'll have to prove that length l = length l'' (since the a and b in the goal are l and l'', respectively). We get an IH for each premise that is a recursive reference. That is, perm_nil and perm_swap are axioms, so there's no IH. But perm_skip and perm_trans both have premises that involve recursive references to the inductive relation we're looking at, so we get IHs. In a case where we have a recursive reference of the form Permutation a b, we'll have a corresponding IH based on our goal, using a and b. That is, for this proof, we get an IH saying that length a = length b. Concretely, the perm_skip rule will give us length l = length l'; the perm_trans rule will give us two IHs: one saying that length l = length l' and one saying that length l' = length l''. One final remark: we've called something an axiom when it has no premises and a rule otherwise. But not every rule gets an IH! We could have given the following rule instead of perm_nil:

l = []   l' = []	(perm_nil')
Permutation l l'

In this case perm_nil' isn't strictly speaking an axiom, but we still wouldn't get an IH. You only get an IH for each recursive use of the inductive predicate.

Exercise: 1 star, standard (Permutation_sym)

Exercise: 2 stars, standard (Forall_perm)

To close, a useful utility lemma. Prove this by induction; but is it induction on al, or on bl, or on Permutation al bl, or on Forall f al? Some choices are much easier than others! If you're stuck, try a different one.

In order to compute the permutations of a list, we define a function everywhere that takes an element and puts it in every possible position.

Suppose we have a list h::t. Given an element h and a list permutatons ls of the tail t, how might we collect all possible insertions of h? The function flat_map from Poly will serve:

What does our function permutations have to do with the inductive proposition Permutation? We can prove that our function permutations is in some sense complete: everything it produces is a permutation of the given list.

Exercise: 2 stars, standard (everywhere_perm)

Exercise: 2 stars, standard (permutations_complete)

Exercise: 5 stars, advanced, optional (permutations_correct)

It's possible to prove the converse, that Permutation l l' implies In l' (permutations l). Feel free to give it a go, but it's very challenging! ☐

Sortedness

A sorting algorithm must rearrange the elements into a list that is totally ordered. What does it mean for a list to be sorted? We can define an inductive proposition that seems to do the trick:

Is this really the right definition of what it means for a list to be sorted? One might have thought that it should refer to list indices, i.e., for valid indices i,j into the list, the ith item is less than or equal to the jth item:

Note: the notation i < j < length al really means i < j ∧ j < length al:

This is a reasonable definition too. It should be equivalent. Later on, we'll prove that the two definitions really are equivalent. For now, let's use the first one to define what it means to be a correct sorting algorthm.

That is: the result (f al) should not only be a sorted sequence, but it should be some rearrangement (Permutation) of the input sequence.

Proofs with Sortedness

Before proving that we can write a sorting algorithm, let's get some intution for how the sorted predicate behaves. In particular, what operations preserve sortedness? Which functions will return a sorted list when given a sorted list? We can formalize the notion of preservation as follows:

What operations on lists do we have? Let's start with simple ones, like mapping (map). If we map the successor function S over a sorted list, the list should still be sorted.

We can do more than add one... we can add two!

There are lots of functions that preserve sortedness. So many, in fact, that we can characterize a set of them: the monotonic functions are those that preserve ordering, i.e., if x ≤ y then f x ≤ f y. Both S and (plus 2) are monotonic.

Every monotonic function preserves sortedness.

Exercise: 3 stars, standard (addn_preserves_sortedness)

Now prove that adding any number preserves sortedness.

Other operations preserve sortedness, too. For example, we can remove any elements we want from a sorted list and the list will still be sorted. We can formalize this idea using the filter function---for any predicate p, filter p preserves sortedness.

Exercise: 4 stars, standard, optional (sorted_filter_cons)

Our proof goes in two stages. First, we show that if something is good at the front of a sorted list, it's also good at the front of a filtered sorted list. Think carefully about what you do induction on. You may need a helper lemma in a particularly tricky case!

Exercise: 3 stars, standard (filter_preserves_sortedness)

Using the (optional) lemma above, we can prove the more general property: (filter p) preserves sortedness for all predicates p. There are two ways to prove this function, and one is much easier than the other... but both require sorted_filter_cons.

Exercise: 3 stars, standard (bad_function_breaks_sortedness)

Not every function preserves sortedness. Define a non-recursive function that doesn't preserve sortedness. You may use any other function we've defined so far. (You can even write a recursive function, if you want, but it's possible to solve this exercise without it.

Now prove that your function doesn't preserve sortedness.

Proof of Correctness

Exercise: 3 stars, standard (insert_perm)

Prove the following auxiliary lemma, insert_perm, which will be useful for proving sort_perm below. Your proof will be by induction, but you'll need some of the permutation facts from the above.

Exercise: 3 stars, standard (sort_perm)

Now prove that sort is a permutation.

Exercise: 4 stars, standard (insert_sorted)

This one is a bit tricky. However, there is just a single induction right at the beginning, and you do not need to use insert_perm or sort_perm. The leb_spec lemma from IndProp.v may come in handy.

Exercise: 2 stars, standard (sort_sorted)

This one is shorter.

Now we wrap it all up.

Exercise: 2 stars, standard, optional (sort_stable)

It's a nice exercise to prove the idempotence and stability lemmas from the informal work in Coq. Two terms of art are used here: idempotent and stable. A function f is idempotent when f (f x) = f x. A sort is stable when it doesn't change the original orderings of two elements. To generally show stability, we'd have to prove that for any two elements x and y in a list l, if x ≤ y and x comes before y in l, then x comes before y in sort l. We'll prove something slightly weaker here, abusing terminology as an excuse to introduce a cool concept.

Making Sure the Specification is Right

It's really important to get the specification right. You can prove that your program satisfies its specification (and Coq will check that proof for you), but you can't prove that you have the right specification. Therefore, we take the trouble to write two different specifications of sortedness (sorted and sorted'), and prove that they mean the same thing. This increases our confidence that we have the right specification, though of course it doesn't prove that we do.

Exercise: 4 stars, standard, optional (sorted_sorted')

Hint: Instead of doing induction on the list al, do induction on the sortedness of al. This proof is a bit tricky, so you may have to think about how to approach it, and try out one or two different ideas.

Exercise: 3 stars, standard, optional (sorted'_sorted)

Here, you can't do induction on the sorted'-ness of the list, because sorted' is not an inductive predicate.

Proving Correctness from the Alternate Spec

Depending on how you write the specification of a program, it can be much harder or easier to prove correctness. We saw that the predicates sorted and sorted' are equivalent; but it is really difficult to prove correctness of insertion sort directly from sorted'. Try it yourself, if you dare! I managed it, but my proof is quite long and complicated. I found that I needed all these facts:

• insert_perm, sort_perm
• Forall_perm, Permutation_length
• Permutation_sym, Permutation_trans
• a new lemma Forall_nth, stated below.

Maybe you will find a better way that's not so complicated. DO NOT USE sorted_sorted', sorted'_sorted, insert_sorted, or sort_sorted in these proofs!

Exercise: 3 stars, standard, optional (Forall_nth)

Exercise: 4 stars, standard, optional (insert_sorted')

Exercise: 4 stars, standard, optional (insert_sorted')

The Moral of This Story

The proofs of insert_sorted and sort_sorted were easy; the proofs of insert_sorted' and sort_sorted' were difficult; and yet sorted al ↔ sorted' al. Different formulations of the functional specification can lead to great differences in the difficulty of the correctness proofs. Suppose someone required you to prove sort_sorted', and never mentioned the sorted predicate to you. Instead of proving sort_sorted' directly, it would be much easier to design a new predicate (sorted), and then prove sort_sorted and sorted_sorted'.