SortInsertion Sort

Require Export IndProp.

Permutations

A permutation of a list is a rearrangement of its elements.
We can define permutations using the following inductive proposition. We'll worry about whether or not this is exactly the right proposition later on.
Inductive Permutation {A : Type} : list A → list A → Prop :=
| perm_nil: Permutation [] []
| perm_skip : ∀ x l l', Permutation l l' → Permutation (x::l) (x::l')
| perm_swap : ∀ x y l, Permutation (y::x::l) (x::y::l)
| perm_trans : ∀ l l' l'',
    Permutation l l' → Permutation l' l'' → Permutation l l''.
Example permutation_eg :
  Permutation [true;true;false] [false;true;true].
Proof.
  apply perm_trans with [true;false;true].
  { apply perm_skip.
    apply perm_swap. }
  { apply perm_swap. }
Qed.

Exercise: 2 stars (Permutation_properties)

Think of some properties of the Permutation relation and write them down informally in English, or a mix of Coq and English. Here are four to get you started:
  • 1. If Permutation al bl, then length al = length bl.
  • 2. If Permutation al bl, then Permutation bl al.
  • 3. [1;1] is NOT a permutation of [1;2].
  • 4. [1;2;3;4] IS a permutation of [3;4;2;1].
YOUR ASSIGNMENT: Add three more properties that hold.
Write them down here, but no need to prove them: ☐

Exercise: 1 star (Permutation_refl)

Theorem Permutation_refl : ∀ A (l : list A),
    Permutation l l.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 1 star (Permutation_length)

Theorem Permutation_length : ∀ A (l1 l2 : list A),
  Permutation l1 l2 → length l1 = length l2.
Proof.
  (* FILL IN HERE *) Admitted.
☐
How might Permutation_length look as in informal proof?
  • Theorem: if l1 is a permutation of l2, then l1 and l2 have the same length.
Proof: Let lists l1 and l2 be given such that l1 is a permutation of l2. We go by induction on the derivation of Permutation l1 l2. There are four cases.
  • (perm_nil) We have l1 = [] and l2 = [], both of which have length 0.
  • (perm_skip) We have l1 = x::l1' and l2 = x::l2' and Permutation l1' l2'; our IH shows that length l1' = length l2'. We have length (x::l1') = length (x::l2') immediately by the IH.
  • (perm_swap) We have l1 = y::x::l and l2 = x::y::l. We have length (y::x::l) = length (x::y::l) immediately.
  • (perm_trans) We have l1 = l and l2 = l'' and a list l' such that Permutation l l' and Permutation l' l''; our IHs show that length l = length l' and length l' = length l''. We can find length l1 = l2 by transitivity of equality and our IHs. Qed.
Notice that we get IHs in the perm_skip and perm_trans cases but not in perm_nil and perm_swap. Why is that?
Induction hypothesis come from "smaller" things—-for nats, we get an IH for the n = S n' case because there's a smaller nat involved—-n'. For inductive propositions, we'll get an IH when there's a recursive reference to the proposition we're defining. Both perm_skip and perm_trans refer back to the Permutation definition as a premise (and so we get an IH) but perm_nil and perm_swap don't (and so we don't get an IH).
More generally, induction on propositions works as follows. We have the following rules:
   (perm_nil)  
Permutation [] []
Permutation l l' (perm_skip)  
Permutation (x::l) (x::l')
   (perm_swap)  
Permutation (y::x::l) (x::y::l)
Permutation l l'    Permutation l' l'' (perm_trans)  
Permutation l l''
We want to prove that for all l1 and l2, if Permutation l1 l2 and then length l1 = length l2.
Having let l1 and l2 be given such that Permutation l1 l2, we go by induction on the derivation to show that length l1 = length l2. What cases do we have? When do we have an IH and where does it come from?
We will have one case for each rule. If the rule is of the form Permutation a b, then we will have to show our goal for a and b. Here, that means showing that length a = length b. So in the perm_nil case, we have to prove that length [] = length [], since perm_nil only applies when both l1 = [] and l2 = []. In perm_trans, we'll get to keep our hypotheses (that Permutation l l' and Permutation l' l'') and we'll have to prove that length l = length l'' (since the a and b in the goal are l and l'', respectively).
We get an IH for each premise that is a recursive reference. That is, perm_nil and perm_swap are axioms, so there's no IH. But perm_skip and perm_trans both have premises that involve recursive references to the inductive relation we're looking at, so we get IHs.
In a case where we have a recursive reference of the form Permutation a b, we'll have a corresponding IH based on our goal, using a and b. That is, for this proof, we get an IH saying that length a = length b. Concretely, the perm_skip rule will give us length l = length l'; the perm_trans rule will give us two IHs: one saying that length l = length l' and one saying that length l' = length l''. One final remark: we've called something an axiom when it has no premises and a rule otherwise. But not every rule gets an IH!
We could have given the following rule instead of perm_nil:
l = []   l' = [] (perm_nil')  
Permutation l l'
In this case perm_nil' isn't strictly speaking an axiom, but we still wouldn't get an IH. You only get an IH for each recursive use of the inductive predicate.

Exercise: 1 star (Permutation_sym)

Lemma Permutation_sym : ∀ A (l1 l2 : list A),
  Permutation l1 l2 → Permutation l2 l1.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 2 stars (Permutation_sym_informal)

Write an informal proof of Permutation_sym.
  • Theorem: the permutation relation is symmetric, i.e., if l1
is a permutation of l2, then l2 is a permutation of l1.
Proof: (* FILL IN HERE *)
☐

Exercise: 2 stars (Forall_perm)

To close, a useful utility lemma. Prove this by induction; but is it induction on al, or on bl, or on Permutation al bl, or on Forall f al? Some choices are much easier than others! If you're stuck, try a different one.
Inductive Forall {A : Type} (P : A → Prop) : list A → Prop :=
    Forall_nil : Forall P []
  | Forall_cons : ∀ (x : A) (l : list A), P x → Forall P l → Forall P (x :: l).
Theorem Forall_perm: ∀ {A} (f: A → Prop) al bl,
  Permutation al bl →
  Forall f al → Forall f bl.
Proof.
  (* FILL IN HERE *) Admitted.
☐

The Insertion-Sort Program

Our work this week will be proving a sort correct. We'll write a function sort that takes a list of natural numbers and rearranges it to be in ascending order.
The algorithm we'll implement is called insertion sort, which works in the following way:
  • to insert an element i into an already sorted list l, simply walk down the list until we find an item greater than or equal to i—-then put i into the list; and
  • given an unsorted list, insert each element into the recursive result of sorting the list.
Fixpoint insert (i:nat) (l: list nat) :=
  match l with
  | nil ⇒ i::nil
  | h::t ⇒ if leb i h then i::h::t else h :: insert i t
 end.
Fixpoint sort (l: list nat) : list nat :=
  match l with
  | nil ⇒ nil
  | h::t ⇒ insert h (sort t)
end.
Does our sorting function work? We can try a few examples:
Compute (sort [10;9;8;7;6;5;4;3;2;1]).
Example sort_pi: sort [3;1;4;1;5;9;2;6;5;3;5]
                    = [1;1;2;3;3;4;5;5;5;6;9].
Proof. simpl. reflexivity. Qed.

Compute (insert 7 [1; 3; 4; 8; 12; 14; 18]).
(* = 1; 3; 4; 7; 8; 12; 14; 18 *)
The tail of this list, 12::14::18::nil, is not disturbed or rebuilt by the insert algorithm. The nodes 1::3::4::7::_ are new, constructed by insert.
If you're having trouble following exactly how the algorithm works, try working out how the following evalutes on the board:
    sort [3;2;1].
Simply believing that this algorithm works isn't enough. Let's prove it correct!

Specification of Correctness

A sorting algorithm must rearrange the elements into a list that is totally ordered.
What does it mean for a list to be sorted? We can define an inductive proposition that seems to do the trick:
Inductive sorted: list nat → Prop :=
  | sorted_nil : sorted []
  | sorted_1 : ∀ x, sorted [x]
  | sorted_cons : ∀ x y l, x ≤ y → sorted (y::l) → sorted (x::y::l).
Example sorted_one_through_four :
  sorted [1;2;3;4].
Proof.
  (* WORKED IN CLASS *)
  apply sorted_cons. { apply le_S. apply le_n. }
  apply sorted_cons. { apply le_S. apply le_n. }
  apply sorted_cons. { apply le_S. apply le_n. }
  apply sorted_1.
Qed.
Is this really the right definition of what it means for a list to be sorted? One might have thought that it should refer to list indices, i.e., for valid indices i,j into the list, the ith item is less than or equal to the jth item:
Fixpoint nth {X:Type} (n:nat) (l:list X) (default:X) : X :=
  match n,l with
  | _,[] ⇒ default
  | 0,h::_ ⇒ h
  | (S n'),_::t ⇒ nth n' t default
  end.
Example nth_in_list : nth 3 [1;2;3;4;5] 0 = 4.
Proof. reflexivity. Qed.
Example nth_default : nth 7 [1;2;3;4;5] 0 = 0.
Proof. reflexivity. Qed.
Definition sorted' (al: list nat) :=
 ∀ i j, i < j < length al → nth i al 0 ≤ nth j al 0.
Note: the notation i < j < length al really means i < j ∧ j < length al:
Compute (0 < 1 < 2).
This is a reasonable definition too. It should be equivalent. Later on, we'll prove that the two definitions really are equivalent. For now, let's use the first one to define what it means to be a correct sorting algorthm.
Definition is_a_sorting_algorithm (f: list nat → list nat) :=
  ∀ al, Permutation al (f al) ∧ sorted (f al).
That is: the result (f al) should not only be a sorted sequence, but it should be some rearrangement (Permutation) of the input sequence.

Proof of Correctness

Exercise: 3 stars (insert_perm)

Prove the following auxiliary lemma, insert_perm, which will be useful for proving sort_perm below. Your proof will be by induction, but you'll need some of the permutation facts from the above.
Lemma insert_perm: ∀ x l, Permutation (x::l) (insert x l).
Proof.
(* FILL IN HERE *) Admitted.
☐

Exercise: 3 stars (sort_perm)

Now prove that sort is a permutation.
Theorem sort_perm: ∀ l, Permutation l (sort l).
Proof.
(* FILL IN HERE *) Admitted.
☐

Exercise: 4 stars (insert_sorted)

This one is a bit tricky. However, there is just a single induction right at the beginning, and you do not need to use insert_perm or sort_perm. The leb_spec lemma from IndProp.v may come in handy.
Lemma insert_sorted:
  ∀ a l, sorted l → sorted (insert a l).
Proof.
(* FILL IN HERE *) Admitted.
☐

Exercise: 2 stars (sort_sorted)

This one is shorter.
Theorem sort_sorted: ∀ l, sorted (sort l).
Proof.
(* FILL IN HERE *) Admitted.
☐
Now we wrap it all up.
Theorem insertion_sort_correct:
    is_a_sorting_algorithm sort.
Proof.
  split. apply sort_perm. apply sort_sorted.
Qed.

Exercise: 3 stars (sort_idempotent_informal)

Prove that sort (sort l) = sort l. To do so, you'll want to prove the following lemma:
  • Lemma: If l is sorted, then sort l = l.
    Proof: (* FILL IN HERE *)
Okay: now prove the theorem!
  • Theorem: sort (sort l) = sort l.
    Proof: (* FILL IN HERE *)
☐

Exercise: 2 stars, optional (sort_stable)

It's a nice exercise to prove the above lemmas in Coq.
Two terms of art are used here: idempotent and stable. A function f is idempotent when f (f x) = f x.
A sort is stable when it doesn't change the original orderings of two elements. To generally show stability, we'd have to prove that for any two elements x and y in a list l, if x ≤ y and x comes before y in l, then x comes before y in sort l. We'll prove something slightly weaker here, abusing terminology as an excuse to introduce a cool concept.
Lemma sort_stable : ∀ l,
    sorted l → sort l = l.
Proof.
  (* FILL IN HERE *) Admitted.
Corollary sort_idempotent : ∀ l,
    sort (sort l) = sort l.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Making Sure the Specification is Right

It's really important to get the specification right. You can prove that your program satisfies its specification (and Coq will check that proof for you), but you can't prove that you have the right specification. Therefore, we take the trouble to write two different specifications of sortedness (sorted and sorted'), and prove that they mean the same thing. This increases our confidence that we have the right specification, though of course it doesn't prove that we do.

Exercise: 4 stars (sorted_sorted')

Lemma sorted_sorted': ∀ al, sorted al → sorted' al.
Hint: Instead of doing induction on the list al, do induction on the sortedness of al. This proof is a bit tricky, so you may have to think about how to approach it, and try out one or two different ideas.
(* FILL IN HERE *) Admitted.
☐

Exercise: 3 stars (sorted'_sorted)

Lemma sorted'_sorted: ∀ al, sorted' al → sorted al.
Here, you can't do induction on the sorted'-ness of the list, because sorted' is not an inductive predicate.
Proof.
(* FILL IN HERE *) Admitted.
☐

Proving Correctness from the Alternate Spec

Depending on how you write the specification of a program, it can be much harder or easier to prove correctness. We saw that the predicates sorted and sorted' are equivalent; but it is really difficult to prove correctness of insertion sort directly from sorted'.
Try it yourself, if you dare! I managed it, but my proof is quite long and complicated. I found that I needed all these facts:
  • insert_perm, sort_perm
  • Forall_perm, Permutation_length
  • Permutation_sym, Permutation_trans
  • a new lemma Forall_nth, stated below.
Maybe you will find a better way that's not so complicated.
DO NOT USE sorted_sorted', sorted'_sorted, insert_sorted, or sort_sorted in these proofs!

Exercise: 3 stars (Forall_nth)

Lemma Forall_nth:
  ∀ {A: Type} (P: A → Prop) d (al: list A),
     Forall P al ↔ (∀ i, i < length al → P (nth i al d)).
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 4 stars, optional (insert_sorted')

(* Prove that inserting into a sorted list yields a sorted list, for
   the index-based notion of sorted.
   
   You'll find leb_spec handy. If you find that your context gets
   cluttered, you can run clear H to get rid of the hypothesis H;
   you can run clear - H to get rid of everything _but_ H. Be
   careful---you can't undo these tactics! *)
Lemma insert_sorted':
  ∀ a l, sorted' l → sorted' (insert a l).
(* FILL IN HERE *) Admitted.
☐

Exercise: 4 stars, optional (insert_sorted')

Theorem sort_sorted': ∀ l, sorted' (sort l).
(* FILL IN HERE *) Admitted.
☐

The Moral of This Story

The proofs of insert_sorted and sort_sorted were easy; the proofs of insert_sorted' and sort_sorted' were difficult; and yet sorted al ↔ sorted' al. Different formulations of the functional specification can lead to great differences in the difficulty of the correctness proofs.
Suppose someone required you to prove sort_sorted', and never mentioned the sorted predicate to you. Instead of proving sort_sorted' directly, it would be much easier to design a new predicate (sorted), and then prove sort_sorted and sorted_sorted'.