10 Lecture 5 and 6 – Proof by Induction

In Lecture 4 – Type Systems we learend about type systems. The point of a type system is to make predictions. But we don’t just want to make predictions about a single term, like z. We want to make predictions about whole classes of terms, like all terms of type Nat, or any well-typed term.

A prediction is a theorem, so we need to write a proof to be sure our prediction is true. So far, we’ve seen one proof technique: build a derivation. This essentially corresponds proof by Implication. Judgments express implication: if A then B. To show B, I build a derivation B, by build a derivation of A, and appling some rule.

Unfortunately, we can only build derivations when we have concrete, particular terms. Rules refer to particular terms, like z, or true, or (s e). How do we prove a theorem about an arbitrary term e? There is no known structure to e, so we can’t immediately build a derivation.

For this, we need Induction, our second axiom:

This is an axiom, not a provable fact. This is an assertion by mathematicians that if we do this, then we use this proof technique, then we can believe the proof. We can’t prove this axiom is reasonable; we trust that it is reasonable. It’s counter-intuitive, since it allows us to assume the very thing we’re trying to prove (albeit, assume it in a "smaller" way). But so far, no proofs by induction have been at the root of an proof of an untrue theorem, so it seems okay to use.

A good rule of thumb is that any time we have a theorem with some meta-varaible with no additional structure, like e, we need induction. This isn’t always true. We may be able to appeal to previously proved lemmas that will let us build generic derivations. Sometimes, our judgments will happen to let us build derivations even when we have no structure. But it’s a good rule of thumb if you’re not sure where to start.

10.1 Some Facts About Natural Numbers

Let’s warm up to induction by proving some things about natural numbers. Natural numbers are an inductively defined judgment, given below (as a syntax):

Or below, as a judgment:

We also define a simple reduction judgment and evaluation judgment:

I leave the conversion judgment undefined, but it’s the obvious reflexive, transitive, compatible closure of the reduction judgment.

We’ll start simple:

Theorem: (For all m,) eval(0 + m) = m

This theorem says that evaluating 0 plus any m equals m. Often, we would leave out of the explicit quantification over all possible m in the theorem statement. I’ve written it explicitly in parentheses to suggest this quantification is "obvious" (to a working PL theorist) from context.

The theorem is probably intuitively true to anyone reading it. And of course, it is true. But true is not the same as being proven, so let’s see if we can prove it.

Theorem: (For all m,) eval(0 + m) = m

Huh. That proof worked without any kind of induction even those it had a for all quantification.

But this was an accident of our reduction judgment. It just so happens that the [Step-Plus-Z] rule works for any m, as long as the left hand side is 0. If we try to do the proof that eval(m + 0) = m, it would be so easy:

Theorem: (For all m,) eval(m + 0) = m

In general, when stuck in a proof, we have two options: change our theorem, or change our model. We could try to separate that n + (s 0) ->* s n, and that might work. It certainly ought to be true. But actually, we can more easily complete the proof by changing our reduction rule to something equivalent, but simpler to prove things about.

This is a pretty common trick in programming languages work. If you see a judgment that seems correct, but different than the obvious thing, you should ask yourself if this judgment is easier to use in proofs.

We change the reduction judgment to the following:

Note that now addition reduces to an addition where both sub-expressions are inductively smaller. This is a good sign that it will work better in inductive proofs.

Theorem: (For all m,) eval(m + 0) = m

10.2 An inductive proof is a recursive function over derivations

It’s sometimes mysterious to see an on-paper proof. What tells us that the series of words I just wrote constitutes a proof? What is a proof? How do I understand a proof that assumes the very thing it is proving? What is an induction hypothesis?

These are all questions I’ve had too. I didn’t really understand proofs until I understood that proofs are programs. A theorem is a function: is has inputs (its premises) and produces outputs (its conclusion). An inductive proof is a recursive function: it pattern matches on its input, deconstructing a derivation, builds a new derivation, and occassionally recurs on a sub-derivation.

Understanding that proofs are programs is useful sometimes. It’s actually useful, sometimes, to structure a proof as loop with an accumulator. I’ve done this in one paper. It’s hard to think about that until you understand proofs as programs.

We don’t need any fancy proof assistant to see this. We can write recursive functions in any language. I’ll choose Racket since Redex provides derivations that Racket understands.

Below, I give a model of the natural numbers in Redex. I use the judgment style for defining them. I redefine n and m to be aliases for any, to make the judgments more readable.

The proof transforms one derivation, that z is a natural number, into another, that eval (z + z) = z.

In the second case of this proof, we know that the list of premises contains one sub-derivation, which we name so we can use it later. We use the "unquote" pattern, ,n, to bind whatever expression the s is applied to. This is part of Racket’s pattern language; we know there is some term there, but we don’t know what. Since it’s a quasiquoted list, we need to use comma to tell Racket to treat that as a pattern variable instead of a symbol.

Then we proceed to build a derivation that eval ((s n) + z) = (s n). Most of this is standard, building derivations by hand, appealing to rules, and building derivations for premises.

But, eventually we get to a point where we can’t apply any rule directly. We need to prove that (n + z) ->* n, and there is no rule we can apply, since we don’t know anything about n. Thankfully, we can gleam this fact from induction! We recursively call the function on the sub-derivation that n is a natural number. We get back a derivation about eval, so we destruct it. We could have written the code more cleanly by using a helper function, i.e., a lemma.

Unfortunately, when we try to finish the proof case for n + m, we discover that our theorem isn’t general enough. I leave it as an exercise to the reader to discover the fix. (Hint: evaluation and equality are different, but similar, judgments.)

Formally, a proof can’t just be any function, though. It must be a complete, bug-free function. It must always return something of the right type, handle all cases of any data it consumes, and always terminate. It’s hard to trust a proof written in Racket since Racket can’t check any of these things, but it’s easier to get started writing such a proof. This is why many proof assistants aren’t Turing-complete; it’s very useful to avoid the halting problem when one wants to check that a function terminates by making it impossible to write an infinite loop.

But, thinking of proofs as functions can be helpful.