ComboCombinatorics


From DMFP Require Export Sort.

Overview

Combinatorics is the study of counting. Counting ways to perform a task, or ways to arrange a set of objects, or ways to build something. Such modes of counting find general application in computer science to analyze algorithms and data structures.
We'll look at a variety of methods for counting, starting from the simplest rules---e.g., how many three-letter acronyms are there in English---to more complex theorems---e.g., how does the expression (x + y)^n expand into a polynomial?

Sums and products: simple counting

Suppose you're trying to plan your route into LA for an evening of contemporary dance at REDCAT.
You could drive, taking the 210/605/10, just taking the 10, or taking the 10 and then switching to the 60. That is, there are three ways to go.
But you could also take a train! You could take the CalTrain to Union Station, or you could bike to Azusa and take the Gold Line. That is, there are two ways to go via train.
In total, there are five different ways to get to REDCAT.
The principle we just applied is called the sum rule: 
    Suppose there [n] ways to do something, and there are another [m]
    ways to do it differently, then there are [n + m] ways to do it.
If we model our options as lists, we might proceed as follows. For driving, write down the dominant highway.

Definition driving : list nat := [210;10;60].
For the train, we'll just use arbitrary other numbers to represent the options.
Definition train : list nat := [1 (* CalTrain *); 2 (* Gold Line *)].
Altogether, we can model the choice of driving or taking the train using list append, ++.
Example route_options : length (driving ++ train) = 5.
Proof. reflexivity. Qed.
In fact, youv'e already proved a general theorem validating the sum rule. It's called app_length, and you proved it way back in Poly.v!

Check app_length.
  • Question: suppose there are three different kinds of tacos (vegetarian, carnitas, al pastor) and two different kinds of enchilada (cheese, chicken). How many different dishes are there?
    Answer: Five. There are three different taco dishes and two difference enchilada dishes.
Let's say you're opening an Italian restaurant, "Oh, the Pastabilities!", focused on pasta dishes: diners choose a shape, a topping, and a sauce.
You offer five shapes of pasta. (But check out check out https://en.wikipedia.org/wiki/List_of_pasta, there'ss a whole world of shapes with delightful names!

Inductive pasta :=
| spaghetti
| tagliatelle
| fettuccine
| fusilli
| penne.
Here's the listing on your menu:
Definition pastas : list pasta := [spaghetti; tagliatelle; fettuccine; fusilli; penne].
Inductive topping :=
| none
| vegetables
| meatballs
| seafood.
Definition toppings : list topping := [none; vegetables; meatballs; seafood].
Inductive sauce :=
| aglio_e_olio (* olive oil and garlic---simple but delicious! *)
| marinara (* tomato sauce with herbs, garlic, and onions *)
| alfredo. (* traditionally just butter and parmagianno reggiano,
                          but often with a hint of cream, too *)
Definition sauces : list sauce := [aglio_e_olio; marinara; alfredo].
Given our menu of pastas, toppings, and sauces, we might want to know how many different dishes you can make.
To start, how many different topping/sauce combinations are there?
To enumerate them manually, it helps to be methodical: for each possible topping, consider the sauces it could go with:
  • none with aglio_e_olio
  • none with marinara
  • none with alfredo
Okay, that's three options. Next we consider the vegetables:
  • vegetables with aglio_e_olio
  • vegetables with marinara
  • vegetables with alfredo
Similarly, we can come up with lists for meatballs and seafood:
  • meatballs with aglio_e_olio
  • meatballs with marinara
  • meatballs with alfredo
  • seafood with aglio_e_olio
  • seafood with marinara
  • seafood with alfredo
Totting it all up, there are twelve options. You might have noticed that there are 4 toppings and 3 sauces, and 12 = 4 × 3. It's no coincidence! We can apply what is called the product rule: 
    Suppose you are performing two-part task. If there are n ways to
    do the first part and m ways to do the second part, there n*m ways
    to do the whole task.

Exercise: 2 stars, standard (product_times)

Formalize the product rule in Coq.
Write a function that takes two lists---these will represent our options. Return a list that represents every possible combination of the two options.
That is, write a function that performs the enumeration procedure described above. There's more than one way to write this function, and the precise order of elements that comes out may vary.
Fixpoint product {X Y:Type} (l1 : list X) (l2 : list Y) : list (X × Y)
  (* REPLACE THIS LINE WITH ":= _your_definition_ ." *). Admitted.
Example product_example1 :
  length (product toppings sauces) = 12.
Proof. (* FILL IN HERE *) Admitted.
Example product_example2 :
  length (product [1;2;3] [true;false]) = 6.
Proof. (* FILL IN HERE *) Admitted.
Lemma product_times : ∀ X Y (l1 : list X) (l2 : list Y),
  length (product l1 l2) = length l1 × length l2.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Order in the court: factorial and permutations

We've already studied permutations in Sort.v. Let's count them!
Suppose we have a list of three elements, like sauces = [aglio_e_olio; marinara; alfredo]. How many different ways could we order the elements of the list sauces?
  • aglio_e_olio, marinara, alfredo
  • aglio_e_olio, alfredo, marinara
  • alfredo, aglio_e_olio, marinara
  • alfredo, marinara, aglio_e_olio
  • marinara, alfredo, aglio_e_olio
  • marinara, aglio_e_olio, alfredo
You can verify for yourself that we've listed all six of them. But why is the answer six?
Here's an explanation using the product rule:
  • There are three ways to choose the first sauce---it could be any of them.
  • Once we've chosen the first sauce, there's only two possible chocies for the next sauce.
  • Finally, after we've picked the first two sauces, there's only one choice for the third sauce.
That is, there are 3 × 2 × 1 = 6 possible choices.
We can generalize that argument: in a list of n things, there are n × (n-1) × ... × 2 × 1 possible choices. There's a name for this operation---factorial, which we defined way back in Basics.v!

Fixpoint factorial (n:nat) : nat :=
  match n with
    | 0 ⇒ 1
    | S n' ⇒ n × factorial n'
  end.
Example fact_0__1 : factorial 0 = 1.
Proof. reflexivity. Qed.
Example fact_5__120 : factorial 5 = 120.
Proof. reflexivity. Qed.
One might ask why factorial 0 = 1. To phrase it another way: how many different orderings are there of the empty list? Just one!
We can in fact able to say more than that: for all n, factorial n is nonzero.

Exercise: 1 star, standard (factorial_nz)

Lemma factorial_nz : ∀ n : nat,
    factorial n ≠ 0.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Relating factorial to permutations

We can also connect permutations with factorial: if we have a list l of length n, then permutations l should have length factorial n.

Exercise: 2 stars, standard, optional (permutation_length_aux)

Lemma In_everywhere_length :
  ∀ A (a:A) (l perm:list A),
    In perm (everywhere a l) →
    length perm = S (length l).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma length_flat_map :
  ∀ A B (f:A → list B) (l:list A) n,
  (∀ y, In y l → length (f y) = n) →
  length (flat_map f l) = n × length l.
Proof.
  (* FILL IN HERE *) Admitted.
Lemma length_permutation :
  ∀ A n (l:list A),
    length l = n →
    ∀ y, In y (permutations l) → length y = n.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 3 stars, standard, optional (permutation_length)

You may need to define and prove an auxiliary lemma in order to see how everywhere and length interact.

Lemma permutation_length :
  ∀ A (l l':list A) (a:A),
  In l' (permutations l) → length (everywhere a l') = S (length l).
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 2 stars, standard (permutations_length)

We can finally prove the desired result: there are factorial n permutations of a list of length n.
Lemma permutations_length :
  ∀ A (l:list A) n,
    length l = n →
    length (permutations l) = factorial n.
Proof.
  (* FILL IN HERE *) Admitted.
☐

The division rule

The multiplication rule has a corresponding division rule. If there are n ways to do something and for every way of doing it there are m equivalent ways, then there are n/m ways to do it.
One of the standard uses of the division rule is to ignore order. Suppose we want to pick two numbers from the list [1;2;3;4] . There are twelve possibilities:
  • [1;2]
  • [1;3]
  • [1;4]
  • [2;1]
  • [2;3]
  • [2;4]
  • [3;1]
  • [3;2]
  • [3;4]
  • [4;1]
  • [4;2]
  • [4;3]
If we don't care about order, however, then there's really fewer: what's the difference between [1;2] and [2;1] ? For each "way" of choosing two numbers n and m, there are two equivalent ways: [n;m] and [m;n] . So if we were to choose two such numbers not caring about order, then there would be 12/2 = 6 possible orderings:
  • [1;2]
  • [1;3]
  • [1;4]
  • [2;3]
  • [2;4]
  • [3;4]
How can we tell that we haven't inappropriately removed an option or accidentally double-counted? We can use the order of our enumerations to make sure we cover all of our bases---if every set of options is sorted, we can be certain that only the right things show up. That is, we're using sorted lists as a canonical form.

Disregarding order: choose


Fixpoint choose (n m : nat) : nat :=
  match n, m with
  | _, O ⇒ 1
  | O, S m' ⇒ 0
  | S n', S m' ⇒ choose n' (S m') + choose n' m'
  end.
Example choose_two_of_six : choose 6 2 = 15.
Proof. reflexivity. Qed.
We can characterize choose in terms of a few useful equations.

Lemma choose_n_0 : ∀ n : nat, choose n 0 = 1.
Proof.
  (* FILL IN HERE *) Admitted.
Lemma choose_n_lt_m : ∀ n m : nat,
    n < m → choose n m = 0.
Proof.
  (* FILL IN HERE *) Admitted.
Lemma choose_n_n : ∀ n : nat, choose n n = 1.
Proof.
  (* FILL IN HERE *) Admitted.
We built the following into our definition, but in fact it's Pascal's Identity, named after Blaise Pascal.
Lemma choose_pascal : ∀ n k : nat,
    choose (S n) (S k) = choose n (S k) + choose n k.
Proof.
  simpl. reflexivity.
Qed.

Defining choose in terms of factorial

There's another way to think of choose: we can define it in terms of factorial itself.
To choose n things from a list of m things, there are m*(m-1)*...*(m-n) ways to order the n things. But if we don't care about order, there are n! ways to order the things we've selected.
First, note that m*(m-1)*...*(m-n) = m!/(m-n)!. So we'd expect to have: choose m n = (factorial m)/(factorial (m - n) × factorial n).
We haven't defined division, though. And the subtraction is inconvenient---what if n > m? Instead we'll phrase our theorem as follows, where n + m is the number of things we're choose n things from:

    (choose (n+m) n) × (factorial n × factorial m) = factorial (n + m)
Lemma choose_fact : ∀ m n : nat,
 choose (n + m) n × (factorial n × factorial m) = factorial (n + m).
Proof.
  intros m. induction m as [|m' IHm']; intros n; simpl.
  - rewrite <- plus_n_O. rewrite choose_n_n.
    simpl. rewrite <- plus_n_O. rewrite mult_comm. apply mult_1_l.
  - induction n as [|n' IHn'].
    + simpl. rewrite <- plus_n_O. rewrite <- plus_n_O. reflexivity.
    + assert (S n' + S m' = S (S n' + m')) as HeqSS.
      { simpl. rewrite <- plus_n_Sm. reflexivity. }
      rewrite HeqSS. clear HeqSS.
      rewrite choose_pascal. rewrite mult_plus_distr_r.
            apply eq_trans with (y := factorial (S n' + m') × S m' + factorial (n' + S m') × S n').
      apply f_equal2.
      { rewrite <- IHm'.
        rewrite <- mult_assoc.
        rewrite <- mult_assoc.
        rewrite (mult_comm (factorial m')).
        reflexivity. }
      { rewrite <- IHn'.
        rewrite <- plus_n_Sm.
        rewrite <- mult_assoc.
        rewrite <- mult_assoc.
        rewrite <- (mult_comm (S n')).
        rewrite (mult_assoc (factorial n')).
        rewrite <- (mult_comm (S n')).
        reflexivity. }
      apply eq_trans with (y := (S m' + S n') × factorial (S n' + m')).
      rewrite mult_plus_distr_r. apply f_equal2.
      { apply mult_comm. }
      { rewrite mult_comm.
        rewrite <- plus_n_Sm.
        reflexivity. }
      unfold factorial. fold factorial.
      rewrite plus_n_Sm. rewrite plus_comm.
      reflexivity.
Qed.

Other identities for choice

There are many other relevant identities for choice. We'll prove just one here:

    choose n m = choose n (n - m)
Since we want to avoid subtraction, we'll prove something slightly different:

    choose (n+m) m = choose (n+m) n
You'll need this lemma about arithmetic, which may in turn need other lemmas we haven't yet proved.

Exercise: 3 stars, standard, optional (mult_nz)

Lemma mult_nz : ∀ m n nz : nat,
    m × nz = n × nz →
    nz ≠ 0 →
    m = n.
Proof.
  (* FILL IN HERE *) Admitted.
☐

Exercise: 2 stars, standard (choose_swap)

Lemma choose_swap : ∀ m n : nat,
    choose (n + m) m = choose (n + m) n.
Proof.
  (* FILL IN HERE *) Admitted.
☐

The binomial theorem

Finally, there is a profound result relating choice to binomials, which are polynomials over two variables.
In particular, we will prove that:

    exp (x + y) n =
    sum_nm n 0 (fun k : nat ⇒ choose n k × (exp x (n - k) × exp y k)).
Where sum_nm is a function we'll use for summing, defined below.
The binomial theorem has plenty of interesting history, going back about 2600 years to early mathematics happening around the Mediterranean (Euclid, for n=2) and in India (Aryabhatiya for n=3; Pingala). Others involved include Omar Khayyam and Chu-Shih-Chieh---binomials were truly of a very global interest! The credit for the modern statement goes to Blaise Pascal. (For more information on this, check out "The Story of the Binomial Theorem" by J. L. Coolidge, appearing in The American Mathematical Monthly, Vol. 56, No. 3 (Mar., 1949), pp. 147-157.
You can understand the following sum function as:

    sum_nm n m (fun i ⇒ e) == sum_i=m^{n+m} e
Note carefully that the upper bound of the sum is n+m, not n.
Fixpoint sum_nm (n m : nat) (f : nat → nat) : nat :=
  match n with
  | O ⇒ f m
  | S n' ⇒ f m + sum_nm n' (S m) f
  end.
Lemma sum_nm_i : ∀ (m n : nat) (f : nat → nat),
   sum_nm (S n) m f = f m + sum_nm n (S m) f.
Proof.
  intros.
  simpl.
  reflexivity.
Qed.

Exercise: 2 stars, standard, optional (sum_identities)

The following make good practice and will familiarize you with sum_nm. That said, we won't use sum_nm again in the course.
Lemma sum_nm_f : ∀ (m n : nat) (f : nat → nat),
   sum_nm (S n) m f = sum_nm n m f + f (m + S n).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma sum_nm_ext : ∀ (m n : nat) (f g : nat → nat),
 (∀ x : nat, x ≤ n → f (m + x) = g (m + x)) →
 sum_nm n m f = sum_nm n m g.
Proof.
  (* FILL IN HERE *) Admitted.
Lemma sum_nm_add : ∀ (m n : nat) (f g : nat → nat),
 sum_nm n m f + sum_nm n m g = sum_nm n m (fun i : nat ⇒ f i + g i).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma sum_nm_times : ∀ (m n x : nat) (f : nat → nat),
 x × sum_nm n m f = sum_nm n m (fun i : nat ⇒ x × f i).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma t_sum_Svars : ∀ (m n : nat) (f : nat → nat),
 sum_nm n m f = sum_nm n (S m) (fun i : nat ⇒ f (pred i)).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma sum_zeroton : ∀ n : nat,
    2 × sum_nm n 0 (fun i : nat ⇒ i) = n × (n + 1).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma sum_odds : ∀ n : nat,
    sum_nm n 0 (fun i : nat ⇒ S (2×i)) = (S n) × (S n).
Proof.
  (* FILL IN HERE *) Admitted.
☐

Lemma minus_Sn_m: ∀ n m : nat,
    m ≤ n → S (n - m) = S n - m.
Proof.
  (* FILL IN HERE *) Admitted.
Theorem binomial: ∀ x y n : nat,
 exp (x + y) n =
 sum_nm n 0 (fun k : nat ⇒ choose n k × (exp x (n - k) × exp y k)).
Proof.
  intros x y n. induction n as [|n' IHn'].
  - simpl. reflexivity.
  - destruct n' as [|n'].
    + simpl.
      repeat rewrite (mult_comm _ 1).
      repeat rewrite mult_1_l.
      repeat rewrite <- plus_n_O. reflexivity.
    + replace (exp (x + y) (S (S n'))) with ((x + y) × (exp (x + y) (S n'))) by reflexivity.
      rewrite IHn'. rewrite mult_plus_distr_r.
      rewrite sum_nm_times. rewrite sum_nm_times.
      rewrite sum_nm_i. rewrite choose_n_0.
      replace (exp y 0) with 1 by reflexivity.
      rewrite mult_1_l. rewrite (mult_comm _ 1). rewrite mult_1_l. rewrite <- minus_n_O.
      rewrite sum_nm_f. rewrite choose_n_n.
      rewrite plus_O_n. rewrite minus_diag. replace (exp x 0) with 1 by reflexivity.
      repeat rewrite mult_1_l.
      rewrite (t_sum_Svars 0 n').
      replace
        (x × exp x (S n') +
         sum_nm n' 1
                (fun i : nat ⇒
                   x × (choose (S n') i × (exp x (S n' - i) × exp y i))) +
         (sum_nm n' 1
                 (fun i : nat ⇒
                    y ×
                    (choose (S n') (pred i) ×
                     (exp x (S n' - pred i) × exp y (pred i)))) +
          y × exp y (S n'))) with
          (exp x (S (S n')) +
           (sum_nm n' 1
                   (fun i : nat ⇒
                      choose (S (S n')) i × (exp x (S (S n') - i) × exp y i)) +
            exp y (S (S n')))).
      rewrite (sum_nm_i 0). rewrite (sum_nm_f 1).
      rewrite choose_n_0. rewrite choose_n_n.
      rewrite <- minus_n_O.
      rewrite minus_diag.
      replace (exp x 0) with 1 by reflexivity.
      replace (exp y 0) with 1 by reflexivity.
      rewrite (mult_comm _ 1). repeat rewrite mult_1_l.
      replace (1 + S n') with (S (S n')) by reflexivity.
      reflexivity. (* !!! *)
      (* old proof obligation from replace *)
      { replace (x × exp x (S n')) with (exp x (S (S n'))) by reflexivity.
        replace (y × exp y (S n')) with (exp y (S (S n'))) by reflexivity.
        repeat rewrite <- plus_assoc.
        apply f_equal2. reflexivity.
        rewrite plus_assoc.
        apply f_equal2.
        - rewrite sum_nm_add.
          apply sum_nm_ext.
          intros i Hi.
          replace (pred (1 + i)) with i by reflexivity.
          replace (1 + i) with (S i) by reflexivity.
          replace (S (S n') - S i) with (S n' - i) by reflexivity.
          replace (S n' - S i) with (n' - i) by reflexivity.
          rewrite (choose_pascal (S n')).
          rewrite mult_plus_distr_r.
          apply f_equal2.
          × rewrite (mult_comm x). rewrite <- mult_assoc. apply f_equal2.
            { reflexivity. }
            { rewrite <- mult_assoc. rewrite (mult_comm _ x).
              rewrite mult_assoc. apply f_equal2.
              rewrite <- minus_Sn_m. rewrite mult_comm. reflexivity. apply Hi.
              reflexivity. }
          × rewrite (mult_comm y). rewrite <- mult_assoc. apply f_equal2.
            { reflexivity. }
            { rewrite <- mult_assoc. apply f_equal2. reflexivity.
              rewrite mult_comm. reflexivity. }
        - reflexivity.
      }
Qed.
We can instantiate the binomial theorem with particular numbers for x, y, and n---producing some remarkable identities! For example, we can see that exp 2 n is equal to the sum of choose n k for k ranging from 0 to n.

Lemma exp_one : ∀ n,
    exp 1 n = 1.
Proof.
  induction n as [|n IHn'].
  - reflexivity.
  - simpl. rewrite IHn'. reflexivity.
Qed.
Lemma binomial_two_to_the_n : ∀ n,
    exp 2 n =
    sum_nm n 0 (fun k : nat ⇒ choose n k).
Proof.
  intros n.
  rewrite → (sum_nm_ext _ _ _ (fun k ⇒ choose n k × (exp 1 (n - k) × exp 1 k))).
  apply (binomial 1 1).
  intros x Hlt.
  simpl. rewrite exp_one. rewrite exp_one. rewrite mult_comm.
  rewrite mult_1_l. reflexivity.
Qed.

Exercise: 2 stars, standard, optional (binomial_consequences)

Prove these other consequences of the binomial theorem.

Lemma binomial_x_plus_1 : ∀ x n,
    exp (x + 1) n =
    sum_nm n 0 (fun k : nat ⇒ choose n k × exp x (n - k)).
Proof.
  (* FILL IN HERE *) Admitted.
Lemma binomial_three_to_the_n : ∀ n,
    exp 3 n =
    sum_nm n 0 (fun k : nat ⇒ choose n k × exp 2 (n - k)).
Proof.
  (* FILL IN HERE *) Admitted.
☐

(* Mon Apr 6 09:16:56 PDT 2020 *)