MODULE 4:   Limits of Computation
Recitation 4
1
Countability Cheat Sheet

There are 2 equivalent (and useful) definitions for “countable set”:

  1. Set AA is countable if AN|A| \leq |\mathbb N| (there is an injection from AA to N\mathbb N).

  2. Set AA is countable if AΣ|A| \leq |\Sigma^*| for some finite alphabet Σ\Sigma (AA is encodable).

You are given a set AA. Is it countable or uncountable?

Options for showing AA is countable:

  • Show AA is “listable”: The elements of AA can be listed so that every element in AA eventually appears in the list. This is equivalent to showing there is a surjection ff from N\mathbb N to AA: we define f(n)f(n) as the nn’th element in the list.

  • Show AB|A|\leq |B|, where BB is a known countable set like Z\mathbb Z, Z×Z\mathbb Z\times\mathbb Z, Q\mathbb Q, Q[x]\mathbb Q[x], etc.

  • Show that AA is encodable, i.e. the elements of AA have finite-length string representations/encodings.

Options for showing AA is uncountable:

  • Assume for the sake of contradiction that AA is countable, then diagonalize against AA to reach a contradiction.

  • Show BA|B| \leq |A|, where BB is a known uncountable set like {0,1}\{0, 1\}^\infty (the set of all infinite-length binary strings). Note that {0,1}(N)\{0, 1\}^\infty \leftrightarrow \wp(\mathbb N).

2
These Decidable Definitions Have Undecidable Ends
  • A decider is a TM that halts on all inputs.

  • A language LL is undecidable if there is no decider TM such that M(x)M(x) accepts if and only if xLx \in L.

  • A language AA reduces to BB (i.e. solving AA reduces to solving BB) if it is possible to decide AA using an algorithm that decides BB as a subroutine. Denote this as ABA \leq B (read: BB can be used to solve AA so AA is at most as hard as BB.)

3
Counting to Infinity

Is the set N\mathbb N^* countable? Does it make a difference if we replace N\mathbb N with another countable set?

Solution

First, let’s make sure we have the correct understanding of what N\mathbb N^* is: N={(x1,x2,,xk):kN,  i  xiN}.\mathbb N^* = \{(x_1,x_2,\ldots,x_k) : k \in \mathbb N, \; \forall i \; x_i \in \mathbb N\}. So N\mathbb N^* denotes the set of all finite-length tuples where each coordinate is an element of N\mathbb N.

We’ll show N\mathbb N^* is encodable/countable. Consider the alphabet Σ={0,1,2,3,4,5,6,7,8,9,$}\Sigma = \{{\texttt{0}},{\texttt{1}},{\texttt{2}},{\texttt{3}},{\texttt{4}},{\texttt{5}},{\texttt{6}},{\texttt{7}},{\texttt{8}},{\texttt{9}},{\texttt{\$}}\}. Any element of N\mathbb N^* can be uniquely represented with a finite-length string where commas are replaced with ${\texttt{\$}}. (Note that there is really no need to put the opening and closing parentheses for the tuple.) To make the encoding definition a bit more explicit, if Enc:NΣ9\text{Enc} : \mathbb{N} \to \Sigma_9^* is the usual encoding of the naturals with digits (where Σ9={0,1,,9}\Sigma_9 = \{{\texttt{0}},{\texttt{1}},\ldots,{\texttt{9}}\}), then Enc:NΣ\text{Enc}':\mathbb N^* \to \Sigma^* is defined as follows: for (x1,x2,,xk)N(x_1,x_2,\ldots,x_k) \in \mathbb N^*, Enc((x1,x2,,xk))=Enc(x1)$Enc(x2)$$Enc(xk).\text{Enc}'((x_1,x_2,\ldots,x_k)) = \text{Enc}(x_1){\texttt{\$}}\text{Enc}(x_2){\texttt{\$}} \ldots {\texttt{\$}}\text{Enc}(x_k). In this case, it is rather obvious that Enc\text{Enc}' is an injective function (i.e. that every element of N\mathbb N^* maps to a unique finite-length string.) Or in other words, if two elements of N\mathbb N^* map to the same string, then it is clear that they represent the same element of N\mathbb N^*. (Explicitly writing a proof of this would be too pedantic in our opinion.) This shows N\mathbb N^* is encodable, i.e. countable.

Similarly, we can conclude that AA^* is countable whenever AA is a countable set.

4
I’m Undecided About These

Prove that the following languages are undecidable (below, MM, M1M_1, M2M_2 refer to TMs).

  1. REGULAR={M:L(M) is regular}\text{REGULAR} = \{\langle M \rangle : L(M) \text{ is regular}\}.

  2. DOLORES={M1,M2:wΣ such that both M1(w) and M2(w) accept}\text{DOLORES} = \{\langle M_1, M_2 \rangle : \exists w \in \Sigma^* \text{ such that both $M_1(w)$ and $M_2(w)$ accept}\}.

Solution

Part 1: We show that REGULAR\text{REGULAR} is undecidable via a reduction from HALTS\text{HALTS} (i.e. we show that HALTS\text{HALTS} reduces to REGULAR\text{REGULAR}).

Suppose MREGM_\text{REG} decides REGULAR\text{REGULAR}. We define a decider for HALTS\text{HALTS} as follows

def M_HALTS(<TM M, string x>):
    <HELP> = 
    """def HELP(w):
        if w is of the form 0^n1^n for some natural number n:
            ACCEPT
        else:
            run M(x)
            ACCEPT"""
    return M_REG(<HELP>)

We now have to prove that MHALTSM_\text{HALTS} is a correct decider for HALTS\text{HALTS}. In order to do this, we consider all possible inputs to MHALTSM_\text{HALTS} and argue that it gives the correct output.

First, consider the case when the input is M,x\langle M, x \rangle such that M(x)M(x) halts. Then notice that the helper TM we define accepts all strings, i.e. L(HELP)=ΣL(HELP) = \Sigma^*. Therefore MREG(HELP)M_\text{REG}(\langle HELP \rangle) accepts (because Σ\Sigma^* is a regular language), which means MHALTSM_\text{HALTS} accepts, as desired.

Next, consider the case when the input is M,x\langle M, x \rangle such that M(x)M(x) loops (in the context of TMs, the word “loops” is synonymous with “does not halt”). Then notice that L(HELP)={0n1nnN}L(HELP) = \{0^n 1^n | n \in \mathbb{N} \} is irregular. Therefore MREG(HELP)M_\text{REG}(\langle HELP \rangle) rejects, which means MHALTSM_\text{HALTS} rejects, as desired.

Thus, we’ve shown that HALTSREGULAR\text{HALTS} \leq \text{REGULAR}, so REGULAR\text{REGULAR} is undecidable.

(Note that there is also the case when the input string to MHALTSM_\text{HALTS} does not correspond to a valid encoding of a TM MM together with a string xx. We would like to remind you once again that even though this is not explicitly written, we implicitly assume that the first thing our machine MHALTSM_\text{HALTS} does is check whether the input is a valid encoding of a TM MM together with a string xx. If it is not, the machine rejects. If it is, then it will carry on with the specified instructions. Recall that this step of checking whether the input string has the correct form is never explicitly written. And in the proof of correctness, we do not have to explicitly argue what happens if the input string does not have the expected format since we can assume this step is handled properly.)

Part 2: We show that DOLORES\text{DOLORES} is undecidable via a reduction from HALTS\text{HALTS} (i.e. we show that HALTS\text{HALTS} reduces to DOLORES\text{DOLORES}).

Suppose MDOLORESM_\text{DOLORES} decides DOLORES\text{DOLORES}. We define a decider for HALTS\text{HALTS} as follows

def M_HALTS(<TM M, string x>):
    <HELP> = 
    """def HELP(w):
        run M(x)
        ACCEPT"""
    return M_DOLORES(<HELP, HELP>)

We now argue we have a correct decider for HALTS\text{HALTS} as in the previous part.

Suppose the input is M,x\langle M, x \rangle such that M(x)M(x) halts. Then HELPHELP accepts all inputs. Therefore MDOLORES(HELP,HELP)M_\text{DOLORES}(\langle HELP, HELP \rangle) accepts, which means MHALTSM_\text{HALTS} accepts, as desired.

Suppose the input is M,x\langle M, x \rangle such that M(x)M(x) loops. Then HELPHELP does not accept any inputs. Therefore MDOLORES(HELP,HELP)M_\text{DOLORES}(\langle HELP, HELP \rangle) rejects, which means MHALTSM_\text{HALTS} rejects, as desired.

Thus, we’ve shown that HALTSDOLORES\text{HALTS} \leq \text{DOLORES}, so DOLORES\text{DOLORES} is undecidable.

5
(Extra) Lose All Scripted Responses. Improvisation Only

Let FINITE={M:M is a TM and L(M) is finite}\text{FINITE} = \{\langle M \rangle : \text{$M$ is a TM and $L(M)$ is finite}\}.
Let TOTAL={M:M is a TM which halts on all inputs}\text{TOTAL} = \{\langle M \rangle : \text{$M$ is a TM which halts on all inputs}\}.

Show that TOTALFINITE\text{TOTAL} \leq \text{FINITE}.

Solution

Suppose that MFINITEM_\text{FINITE} decides FINITE\text{FINITE}.

We define a decider for TOTAL\text{TOTAL}. In the description below, we use the symbol <=\texttt{<=} (less than or equal to) to compare two strings. If string xx is less than or equal to string yy, it means xx is lexicographically less than or equal to yy (this is a total order on the set of all finite-length strings).

def M_TOTAL(<TM M>):
    <HELP> = 
    """def HELP(w):
        for y <= w:
            run M(y)
        ACCEPT"""
    return not M_FINITE(<HELP>)

We now have to prove that MTOTALM_\text{TOTAL} is a correct decider for TOTAL\text{TOTAL}. In order to do this, we consider all possible inputs to MTOTALM_\text{TOTAL} and argue that it gives the correct output.

Suppose the input string is M\langle M \rangle such that MM halts on all inputs. Then notice that L(HELP)=ΣL(HELP) = \Sigma^*. Therefore MFINITE(HELP)M_\text{FINITE}(\langle HELP \rangle) rejects, which means MTOTALM_\text{TOTAL} accepts, as desired.

Suppose the input string is M\langle M \rangle such that MM does not halt on all inputs. Let xx be the lexicographically smallest string that MM does not halt on. Then HELPHELP will not accept any string lexicographically greater (or equal to) than xx, as M(x)M(x) will not halt. Since there are only finitely many strings lexicographically smaller than xx, L(HELP)L(HELP) is finite. Thus, MFINITE(HELP)M_\text{FINITE}(\langle HELP \rangle) accepts, which means MTOTALM_\text{TOTAL} rejects, as desired.

6
(Bonus) Spicy Reals

Let SS be a subset of the positive reals with the property that for every real number xx, the set {yS:y>x}\{y \in S : y > x\} has a minimum element. For example N+\mathbb N^+ is such a set, but Q+\mathbb Q^+ is not. Determine whether SS is necessarily countable.

Solution

The set is countable. We’ll use the fact that rationals are dense in the reals. We sketch the main idea without giving a full proof: Inject SS to Q\mathbb Q by mapping each element ss of SS to some rational between ss and the next largest element of SS.

Note: If you try to inject to N\mathbb N by mapping the smallest element of SS to 00, the second smallest to 11, etc. it won’t work. If S={1,1.5,1.55,1.555,,3,}S=\{1, 1.5, 1.55, 1.555, \ldots , 3, \ldots\}, the 33 will not be assigned to anything.