Lecture 1 : Speedup theorem

Outline

Webpage of the course

We’ll see complexity classes, such as:

  • SPACE classes
  • Polynomial time hierarchy

  • Probabilistic classes

    • TM that are probabilistic instead of being non-deterministic. Acceptance is made with some probability.

    • Non-deterministic: you want one accepting run VS probabilistic one with a certain probability ⟶ you either do many runs to have the right answer, OR run on many machines

Reminder: Turing Machines ( TM )

Useful for low-level complexity: match the idea of a theoretical computers.

A TM $M$ is given by:

  • Finitely many states: $Q$, among which:

    • a starting state $q_0$
    • final states $F ⊆ Q$
    • rejecting states $R⊆ Q$
  • a finite alphabet: denoted by $Σ, Γ, \ldots$
    • the symbols $$$ (resp. $B$ (blank), or $\square$) is used to express the fact that we’re at the beginning (resp. at the end) of the written tape

  • a reading/writing head

    • based on the current state and what the head sees, the TM can change the head’s direction and the state

  • rules of behavior \(Δ : Q × Σ ∪ \lbrace \$, B \rbrace ⟶ Q × Σ × \lbrace ←, → \rbrace\)

  • A configuration of $M$ is some $C ∈ Q × (Σ^\ast ∪ \lbrace B \rbrace) × ℕ$

    • for instance: $C ≝ (q, ab01, 2)$ means that we’re on $b$, in the state $q$

    • OR: we could write $C = ab, q, 01$ alternatively

from $Δ$ we get $C ⟶ C’$ (for instance)

More general: Non-deterministic TM

Instead of having a function $Δ : Q × Σ ∪ \lbrace $, B \rbrace ⟶ Q × Σ × \lbrace ←, → \rbrace$, one have a relation \(Δ ⊆ \Big(Q × Σ ∪ \lbrace \$, B \rbrace \Big) × \Big(Q × Σ × \lbrace ←, → \rbrace \Big)\)

Going from $C$ to $C’$ is a step.

A run of TM:

is a sequence $C_0 ⟶ ⋯ ⟶ C_n ⟶ ⋯$ (it can be infinite) where $C_0 ≝ (q_0, x, 0)$ ($x$ is the input of the TM)

The run $C_0 ≝ (q_0, x, 0) ⟶ ⋯ ⟶ C_n ≝ (q, x’, m)$ accepts/recognizes $x$:

if $q ∈ F$ ($x$ is then said to be accepted/recognized).

For a non-deterministic TM, $x$ is accepted iff there exists a run that accepts $x$.

On can also have several tapes: as it happens

  • there are one input tape, $k$ working tapes, and one output tape

    • the length of $x$ doesn’t count as used space, it’s supposed given

  • one never writes on the input tape

  • one never reads on the output tape, one only writes (the head can only go to the right, on the printer)

Not moving the head

  • for one tape, it doesn’t change anything

  • for several tapes, it’s not trivially the case: if you just want stay put on the first tape only and you have two tapes, you can’t for parity reasons.

  • but all these models are “more or less equivalent”, that is, when it comes to the asymptotic complexity, it doesn’t change anything.

Space complexity

Let $f: ℕ ⟶ ℕ$.

A Language $L$ is in $SPACE(f(n))$:

if there exists a TM $M$ s.t.

  • $M$ recognizes $L$

    • so $L$ is on the fixed alphabet of $M$
  • $M$ is deterministic
  • for every word $x ∈ Σ^\ast$, $M$ halts using at most $f(\vert x \vert)$ space (that is, we add the space used on both tapes)
A Language $L$ is in $NSPACE(f(n))$:

if there exists a TM $M$ s.t.

  • $M$ recognizes $L$

    • so $L$ is on the fixed alphabet of $M$
  • $M$ is NON-deterministic
  • for every word $x ∈ Σ^\ast$, $M$ halts using at most $f(\vert x \vert)$ space (that is, we add the space used on both tapes)

NB: an finite automaton can be seen as a TM using NO working space whatsoever (since everything is stored in the sates).

So regular languages are in $SPACE(0)$

And any TM using no working space is a 2-way automaton (it reads back on the input tape), and it is as expressive as a finite automaton.

So

\[REG = SPACE(0)\]

And even

\[REG = SPACE(k)\]

NB: pebble automata: more expressive than finite automata.

Speedup Theorem

Theorem: \(SPACE(O(f(n))) ≝ \bigcup\limits_{k ∈ℕ} SPACE(k f(n)) ⊆ SPACE(f(n))\)

Proof:

Let $L$ be a language in $SPACE(k f(n))$. It can recognized by a new TM whose alphabet is comprised of $k$-tuples of letters.

Likewise:

Theorem: \(TIME(O(f(n))) ⊆ TIME(f(n) + n + 2)\)

Both of these theorems are true for non-deterministic TM as well (put a ‘N’ before each class).

\[NL ≝ NSPACE(\log(n))\]

In logarithmic space, one can

  • count the length of the input

  • recognize $L ≝ \lbrace a^n b^n \mid n ∈ ℕ \rbrace$, for instance, in $2 \log (\vert x\vert)$ (but with the speedup theorem, there’s no problem)

  • \(REACH ∈ NL\) (a.k.a. $GAP$ (Graph Accessibility Problem)): is there a path from one node to another in a directed graph (DAG)

\[REACH ≝ \lbrace (G, s, t) \mid s,t∈ node(G) \text{ and } s ⟶ ⋯ ⟶ t\rbrace\]

where $s$ (source) and $t$ (target) and the triples are written in a specified form.

But: we have to

  • use pointers to refer to nodes
  • use a decrementing counter of the number of edges to avoid cycles
  • use non-determinism to guess a path

We can even do that in $SPACE(\log^2(n))$

# is there a path from s to t of length ≤ 2^p
reach(s, t, p) = | yes  if s=t or s→t
                 | no   if p=0
                 | OR_{for each node v} (reach(s, v, p-1) ∧ reach(v, t, p-1))

call reach(s, t, log |number of edges|)

There are a logarithmic number of recursive calls, each of which takes logarithmic space.

Leave a comment