1. Recursion & Induction

Recursion & Induction

& Proofs of Correctness

Many algorithms can be designed with recursive algorithms. Once you are used to them, they can be easier to understand (& prove correct) than iterative algorithms:

protected void recSelSort(int lastIndex, 
                                        Comparable[] elts)
{
    if (lastIndex > 0)   // more than 1 element to sort
    {
        int extreme = 0;    // index of element w/ largest value 
                    
        // Find "extreme", index of elts w/ largest value. 
        for (int searchIndex = 1; searchIndex <= lastIndex; searchIndex++) 
        {
            if (elts[extreme].lessThan(elts[searchIndex])) 
                extreme = searchIndex;
        }
       // elt at "extreme" <= elts[index..lastIndex] 

        // swap largest elt (at extreme) w/ one at lastIndex
        Comparable tempElt = elts[extreme]; 
        elts[extreme] = elts[lastIndex];
        elts[lastIndex] = tempElt;
        // elts[lastIndex] now largest in   
        //  elts[0..lastIndex]
            
        recSelSort(lastIndex-1,elts);
       // elts[0..lastIndex] are sorted.
    } 

Mathematical Induction:

A. Prove base case(s). (Usually this is trivial)

B. Show that if algorithm works correctly for all simpler input, then will work for current input.

Reason by induction on size of array (i.e. on lastIndex)

Base: If lastIndex <= 0 then at most one element in elts, and hence sorted - correct.

Induction: Suppose works if lastIndex < n. show it works if last = n (> 0)

(I.e. believe recursive call works, try to show for entire algorithm given call works!)

Loop finds largest element and then swaps with elts[lastIndex].

Thus elts[lastIndex] holds largest elt of list, while others held in elts[0..lastIndex-1]

Since lastIndex- 1 < lastIndex, know (by induction hypothesis) that
recSelSort(lastIndex-1,elts) sorts elts[0..lastIndex-1].

Because elts[0..lastIndex-1] in order and elts[lastIndex] is >= all of them, elts[0..lastIndex] is sorted.

Claim: recSelSort(n-1,elts) (i.e, on n elements) involves n*(n-1)/2 comparisons of elements of array.

Base: n = 0 or 1, 0 comparisons and n*(n-1)/2 = 0.

Induction hypothesis:
Suppose recSelSort(k-1,elts) takes k*(k-1)/2 comparisons for all k < n. Show true for n as well!

Look at algorithm: Run algorithm on recSelSort(n-1,elts). Therefore, last = n-1.

Go through for loop "last" = n-1 times, w/ one comparison each time through.

Thus n-1 comparisons. Only other comparisons are in recursive call:
recSelSort(last-1,elts) where last = n-1.

But by induction (since last < n), this takes last*(last-1)/2 = (n-1)*(n-2)/2 comparisons.

Therefore altogether have (n-1) + (n-1)*(n-2)/2 = (n-1)*2/2 + (n-1)*(n-2)/2
= (n-1)*(2 + n-2)/2 = (n-1)*n/2 = n(n-1)/2 comparisons.

Finished proof.

Therefore RecSelSort takes O(n²) comparisons.

Note: Iterative version of SelectionSort is similar, but needs an extra for loop. See on-line code in Sort.

Look at another example:

/**
    pre:  exponent >= 0
    post - Returns baseexponent
**/ 
public int fastPower(int base, int exponent) {
    if (exponent == 0) 
        return 1
    else if (exponent%2 == 1)       // exponent is odd
        return base * fastPower(base, exponent-1)
    else                            // exponent is even 
        return fastPower(base * base, exponent / 2)
}

Show FastPower(base,exp) = base^exp.

By induction on size of exp:

Base case: exp = 0, FastPower(base,0) = 1 = base⁰ -- correct!

Suppose FastPower(base,exp) = base^exp for all exp < n.

Show FastPower(base,n) = baseⁿ

Two cases:

    n is odd:  FastPower(base,n)    = base * FastPower(base, n-1)

                                    = base * basen-1   (by induction)

                                    = basen

    n is even: FastPower(base,n)    = FastPower(base*base,n / 2)

                                    = (base2)(n / 2)  (by induction)

                                    = basen  (since n is even)

Note can also show complexity of algorithms by induction:

PRO & CON of Recursion

CON:

• Any recursive program requires additional storage on (implicit) run-time stack.

• Sometimes (as in maze program) can replace user-defined stack with recursion.

• If not stack then will get extra space usage. Overhead.

• Also slight run-time overhead in procedure call since must push activation records on stack.

• Often slightly faster and less space if can do iteratively.

• On other hand often easier to construct recursive solution.

• Resulting code may be significantly clearer.

• Smart compilers (particularly for functional languages) will remove "tail" recursion.

protected void recInsSort(int last, Comparable[] elts)
    {
        if (last > 0)
        {
            recInsSort(last-1, elts); // Sort elts[0..last-1]
            
            int posn= last-1;   // index where last shd be inserted

            // Search for first elt (from rear) <=  elts[last]
            while (posn >= 0 && elts[last].lessThan(elts[posn]))
                posn--;

            posn++; // insert elts[index] at posn

            // move elts[posn .. last-1] to put in elts[last] 
            Comparable tempElt = elts[last];
            for (int moveIndex = last-1; moveIndex >= posn; moveIndex--)
                elts[moveIndex+1] = elts[moveIndex];

        // insert element into proper posn
            elts[posn] = tempElt;
        // now elts[0..last] are in order
        }
    }

Correctness straightforward: By induction on size of last parameter:

Clearly works for recInsSort(0, elts) -- array of size 1.

Suppose recInsSort(k, elts) correctly sorts elts[0..k] for k < n.

Show for recInsSort(n, elts).

Algorithm first calls recInsSort(n-1, elts).
By induction correctly sorts elts[0..n-1].

Next find first posn (from rear) where elts[posn].lessThan(elts[n]).

Update posn = posn +1, so elts[posn] >= elts[n]

Moves each of elts[posn..last-1] one step to right, and inserts value of elts[last] in elts[posn].

Now elts[posn+1..last] >= elts[posn] > elts[posn-1] and elts[0..posn-1] are sorted.

Therefore elts[0..last] is sorted.

Complexity is O(n²) again because can show recInsSort(n-1,elts) takes <= n*(n-1)/2 comparisons by induction. (I.e., show for n = 1. Then assume induction hypothesis for k < n, and show for n.)

Because while loop may quit early, insertion sort only uses half as many comparisons (on average) than selection sort. Thus usually twice as fast (but same "O" complexity).