1.1-1
Give a real-world example that requires sorting or a real-world example that requires computing a convex hull.
- Sorting: browse the price of the restaurants with ascending prices on NTU street.
- Convex hull: computing the diameter of set of points.
1.1-2
Other than speed, what other measures of efficiency might one use in a real-world setting?
Memory efficiency and coding efficiency.
1.1-3
Select a data structure that you have seen previously, and discuss its strengths and limitations.
Linked-list:
- Strengths: insertion and deletion.
- Limitations: random access.
1.1-4
How are the shortest-path and traveling-salesman problems given above similar? How are they different?
- Similar: finding path with shortest distance.
- Different: traveling-salesman has more constrains.
1.1-5
Come up with a real-world problem in which only the best solution will do. Then come up with one in which a solution that is ‘‘approximately’’ the best is good enough.
- Best: find the GCD of two positive integer numbers.
- Approximately: find the solution of differential equations.
1.2-1
Give an example of an application that requires algorithmic content at the application level, and discuss the function of the algorithms involved.
Drive navigation.
1.2-2
Suppose we are comparing implementations of insertion sort and merge sort on the same machine. For inputs of size , insertion sort runs in steps, while merge sort runs in steps. For which values of does insertion sort beat merge sort?
\begin{align}
8n^2 & < 64n\lg n \\
2^n & < n^8 \\
n & \le 43.
\end{align}
1.2-3
What is the smallest value of such that an algorithm whose running time is runs faster than an algorithm whose running time is on the same machine?
\begin{align}
100n^2 & < 2^n \\
n & \ge 15.
\end{align}
For each function and time in the following table, determine the largest size of a problem that can be solved in time , assuming that the algorithm to solve the problem takes microseconds.
For each function and time in the following table, determine the largest size of a problem that can be solved in time , assuming that the algorithm to solve the problem takes microseconds.
2-1
Although merge sort runs in worst-case time and insertion sort runs in worst-case time, the constant factors in insertion sort can make it faster in practice for small problem sizes on many machines. Thus, it makes sense to coarsen the leaves of the recursion by using insertion sort within merge sort when subproblems become sufficiently small. Consider a modification to merge sort in which sublists of length are sorted using insertion sort and then merged using the standard merging mechanism, where is a value to be determined.
a. Show that insertion sort can sort the sublists, each of length , in worst-case time.
b. Show how to merge the sublists in worst-case time.
c. Given that the modified algorithm runs in worst-case time, what is the largest value of as a function of for which the modified algorithm has the same running time as standard merge sort, in terms of -notation?
d. How should we chosse in practice?
a. Insertion sort takes time per -element list in the worst case. Therefore, sorting lists of elements each takes worst-case time.
b. Just extending the -list merge to merge all the lists at once would take time ( from copying each element once into the result list, from examining lists at each step to select next item for result list).
To achieve -time merging, we merge the lists pairwise, then merge the resulting lists pairwise, and so on, until there’s just one list. The pairwise merging requires work at each level, since we are still working on elements, even if they are partitioned among sublists. The number of levels, starting with lists (with elements each) and finishing with 1 list (with elements), is . Therefore, the total running time for the merging is .
c. The modified algorithm has the same asymptotic running time as standard merge sort when . The largest asymptotic value of as a function of that satisfies this condition is .
To see why, first observe that cannot be more than (i.e., it can’t have a higher-order term than ), for otherwise the left-hand expression wouldn’t be (because it would have a higher-order term than ). So all we need to do is verify that works, which we can do by plugging into
to get
which by taking just the high-order term and ignorin the constant coefficient, equals .
d. In practice, should be the largest list length on which insertion sort is faster than merge sort.
2-2
Bubblesort is a popular, but inefficient, sorting algorithm. It works by repeatedly swapping adjacent elements that are out of order.
BUBBLESORT(A) for i = 1 to A.length - 1 for j = A.length downto i + 1 if A[j] < A[j - 1] exchange A[j] with A[j - 1]
a. Let denote the output of To prove that is correct, we need to prove that it terminates and that
where . In order to show that BUBBLESORT actually sorts, what else do we need to prove?
The next two parts will prove inequality .
b. State precisely a loop invariant for the for loop in lines 2–4, and prove that this loop invariant holds. Your proof should use the structure of the loop invariant proof presented in this chapter.
c. Using the termination condition of the loop invariant proved in part (b), state a loop invariant for the for loop in lines 1–4 that will allow you to prove inequality . Your proof should use the structure of the loop invariant proof presented in this chapter.
d. What is the worst-case running time of bubblesort? How does it compare to the running time of insertion sort?
a. We need to show that the elements of form a permutation of the elements of .
b. Loop invariant: At the start of each iteration of the for loop of lines 2–4, and the subarray is a permutation of the values that were in at the time that the loop started.
Initialization: Initially, , and the subarray consists of single element . The loop invariant trivially holds.
Maintenance: Consider an iteration for a given value of . By the loop invariant, is the smallest value in . Lines 3–4 exchange and if is less than , and so will be the smallest value in afterward. Since the only change to the subarray is this possible exchange, and the subarray is a permutation of the values that were in at the time that the loop started, we see that is a permutation of the values that were in at the time that the loop started. Decrementing for the next iteration maintains the invariant.
Termination: The loop terminates when reaches . By the statement of the loop invariant, and is a permutation of the values that were in at the time that the loop started.
c. Loop invariant: At the start of each iteration of the for loop of lines 1–4, the subarray consists of the smallest values originally in , in sorted order, and consists of the remaining values originally in .
Initialization: Before the first iteration of the loop, . The subarray is empty, and so the loop invariant vacuously holds.
Maintenance: Consider an iteration for a given value of . By the loop invariant, consists of the smallest values in , in sorted order. Part (b) showed that after executing the for loop of lines 2–4, is the smallest value in , and so is now the smallest values originally in , in sorted order. Moreover, since the for loop of lines 2–4 permutes , the subarray consists of the remaining values originally in .
Termination: The for loop of lines 1–4 terminates when , so that . By the statement of the loop invariant, is the subarray , and it consists of the smallest values originally in , in sorted order. The remaining element must be the largest value in , and it is in . Therefore, the entire array is sorted.
Note: Tn the second edition, the for loop of lines 1–4 had an upper bound of . The last iteration of the outer for loop would then result in no iterations of the inner for loop of lines 1–4, but the termination argument would simplify: would be the entire array , which, by the loop invariant, is sorted.
d. The running time depends on the number of iterations of the for loop of lines 2–4. For a given value of , this loop makes iterations, and takes on the values . The total number of iterations, therefore, is
\begin{align}
\sum_{i = 1}^{n - 1} (n - i)
& = \sum_{i = 1}^{n - 1} n - \sum_{i = 1}^{n - 1} i \\
& = n(n - 1) - \frac{n(n - 1)}{2} \\
& = \frac{n(n - 1)}{2} \\
& = \frac{n^2}{2} - \frac{n}{2}.
\end{align}
Thus, the running time of bubblesort is in all cases. The worst-case running time is the same as that of insertion sort.
2-3
The following code fragment implements Horner’s rule for evaluating a polynomial
\begin{align}
P(x) & = \sum_{k = 0}^n a_k x^k \\
& = a_0 + x(a_1 + x (a_2 + \cdots + x(a_{n - 1} + x a_n) \cdots)),
\end{align}given the coefficients and a value of :
y = 0 for i = n downto 0 y = a[i] + x * y
a. In terms of -notation, what is the running time of this code fragment for Horner’s rule?
b. Write pseudocode to implement the naive polynomial-evaluation algorithm that computes each term of the polynomial from scratch. What is the running time of this algorithm? How does it compare to Horner’s rule
c. Consider the following loop invariant: At the start of each iteration of the for loop of lines 2-3,
Interpret a summation with no terms as equaling . Following the structure of the loop invariant proof presented in this chapter, use this loop invariant to show that, at termination, .
d. Conclude by arguing that the given code fragment correctly evaluates a polynomial characterized by the coefficients .
a. .
b.
NAIVE-HORNER()
y = 0
for k = 0 to n
temp = 1
for i = 1 to k
temp = temp * x
y = y + a[i] * m
The running time is , because of the nested loop. It is obviously slower.
c. Initialization: It is pretty trivial, since the summation has no terms which implies .
Maintenance: By using the loop invariant, in the end of the -the iteration, we have
\begin{align}
y & = a_i + x \sum_{k = 0}^{n - (i + 1)} a_{k + i + 1} x^k \\
& = a_i x^0 + \sum_{k = 0}^{n - i - 1} a_{k + i + 1} x^{k + 1} \\
& = a_i x^0 \sum_{k = 1}^{n - i} a_{k + i} x^k \\
& = \sum_{k = 0}^{n - i} a_{k + i} x^k.
\end{align}
Termination: The loop terminates at . If we substitute,
d. The invariant of the loop is a sum that equals a polynomial with the given coefficients.
2-4
Let be an array of distinct numbers. If and , then the pair is called an inversion of .
a. List the five inversions in the array .
b. What array with elements from the set has the most inversions? How many does it have?
c. What is the relationship between the running time of insertion sort and the number of inversions in the input array? Justify your answer.
d. Give an algorithm that determines the number of inversions in any permutation of elements in worst-case time. ( Modify merge sort).
a. The inversions are , , , , . (Remember that inversions are specified by indices rather than by the values in the array.)
b. The array with elements from with the most inversions is . For all , there is an inversion . The number of such inversions is .
c. Suppose that the array starts out with an inversion . Then and . At the time that the outer for loop of lines 1–8 sets , the value that started in is still somewhere to the left of . That is, it’s in , where , and so the inversion has become . Some iteration of the while loop of lines 5–7 moves one position to the right. Line 8 will eventually drop to the left of this element, thus eliminating the inversion. Because line 5 moves only elements that are greater than , it moves only elements that correspond to inversions. In other words, each iteration of the while loop of lines 5–7 corresponds to the elimination of one inversion.
d. We follow the hint and modify merge sort to count the number of inversions in time.
To start, let us define a merge-inversion as a situation within the execution of merge sort in which the procedure, after copying to and to , has values in and in such that . Consider an inversion , and let and , so that and . We claim that if we were to run merge sort, there would be exactly one mergeinversion involving and . To see why, observe that the only way in which array elements change their positions is within the procedure. Moreover, since keeps elements within in the same relative order to each other, and correspondingly for , the only way in which two elements can change their ordering relative to each other is for the greater one to appear in and the lesser one to appear in . Thus, there is at least one merge-inversion involving and . To see that there is exactly one such merge-inversion, observe that after any call of that involves both and , they are in the same sorted subarray and will therefore both appear in or both appear in in any given call thereafter. Thus, we have proven the claim.
We have shown that every inversion implies one merge-inversion. In fact, the correspondence between inversions and merge-inversions is one-to-one. Suppose we have a merge-inversion involving values and , where originally was and was originally . Since we have a merge-inversion, . And since is in and is in , must be within a subarray preceding the subarray containing . Therefore started out in a position preceding 's original position , and so is an inversion.
Having shown a one-to-one correspondence between inversions and mergeinversions, it suffices for us to count merge-inversions.
Consider a merge-inversion involving in . Let be the smallest value in that is greater than . At some point during the merging process, and will be the “exposed” values in and , i.e., we will have and in line 13 of . At that time, there will be merge-inversions involving and , and these merge-inversions will be the only ones involving . Therefore, we need to detect the first time that and become exposed during the procedure and add the value of at that time to our total count of merge-inversions.
The following pseudocode, modeled on merge sort, works as we have just described. It also sorts the array .
COUNT-INVERSIONS(A, p, r)
inversions = 0
if p < r
q = floor((p + r) / 2)
inversions = inversions + COUNT-INVERSIONS(A, p, q)
inversions = inversions + COUNT-INVERSIONS(A, q + 1, r)
inversions = inversions + MERGE-INVERSIONS(A, p, q, r)
return inversions
MERGE-INVERSIONS(A, p, q, r)
n[1] = q - p + 1
n[2] = r - q
let L[1..n[1] + 1] and R[1..n[2] + 1] be new arrays
for i = 1 to n[1]
L[i] = A[p + i - 1]
for j = 1 to n[2]
R[j] = A[q + j]
L[n[1] + 1] = ∞
L[n[2] + 1] = ∞
i = 1
j = 1
inversions = 0
for k = p to r
if R[j] < L[i]
inversions = inversions + n[1] - i + 1
A[k] = R[j]
j = j + 1
else A[k] = L[i]
i = i + 1
return inversions
The initial call is .
In , whenever is exposed and a value greater than becomes exposed in the array, we increase inersions by the number of remaining elements in . Then because becomes exposed, can never be exposed again. We don’t have to worry about merge-inversions involving the sentinel in , since no value in will be greater than .
Since we have added only a constant amount of additional work to each procedure call and to each iteration of the last for loop of the merging procedure, the total running time of the above pseudocode is the same as for merge sort: .
2.1-1
Using Figure 2.2 as a model, illustrate the operation of on the array .
\begin{align}
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\
A & = \langle 31, 41, 59, 26, 41, 58 \rangle \\
A & = \langle 26, 31, 41, 59, 41, 58 \rangle \\
A & = \langle 26, 31, 41, 41, 59, 58 \rangle \\
A & = \langle 26, 31, 41, 41, 58, 59 \rangle
\end{align}
2.1-2
Rewrite the procedure to sort into nonincreasing instead of nondecreasing order.
INSERTION-SORT(A)
for j = 2 to A.length
key = A[j]
i = j - 1
while i > 0 and A[i] < key
A[i + 1] = A[i]
i = i - 1
A[i + 1] = key
2.1-3
Consider the searching problem:
Input: A sequence of numbers and a value .
Output: An index such that or the special value if does not appear in .
Write pseudocode for linear search, which scans through the sequence, looking for . Using a loop invariant, prove that your algorithm is correct. Make sure that your loop invariant fulfills the three necessary properties.
LINEAR-SEARCH(A, v)
for i = 1 to A.length
if A[i] == v
return i
return NIL
Loop invariant: At the start of each iteration of the for loop, the subarray consists of elements that are different than .
Initialization: Initially the subarray is the empty array, so the prove is trivial.
Maintenance: On each step, we know that does not contain . We compare it with . If they are the same, we return , which is a correct result. Otherwise, we continue to the next step. We have already insured that does not contain and that is different from , so this step preserves the invariant.
Termination: The loop terminates when . Since increases by and , we know that all the elements in have been checked and it has been found that is not among them. Thus, we return .
2.1-4
Consider the problem of adding two -bit binary integers, stored in two -element arrays and . The sum of the two integers should be stored in binary form in an -element array . State the problem formally and write pseudocode for adding the two integers.
Input: An array of booleans and an array of booleans , each representing an integer stored in binary format (each digit is a number, either or , least-significant digit first) and each of length .
Output: An array such that where , and are the integers, represented by , and .
ADD-BINARY(A, B)
C = new integer[A.length + 1]
carry = 0
for i = 1 to A.length
C[i] = (A[i] + B[i] + carry) % 2 // remainder
carry = (A[i] + B[i] + carry) / 2 // quotient
C[i] = carry
return C
2.2-1
Express the function in terms of -notation.
.
2.2-2
Consider sorting numbers stored in array by first finding the smallest element of and exchanging it with the element in . Then find the second smallest element of , and exchange it with . Continue in this manner for the first elements of . Write pseudocode for this algorithm, which is known as selection sort. What loop invariant does this algorithm maintain? Why does it need to run for only the first elements, rather than for all elements? Give the best-case and worst-case running times of selection sort in -notation.
SELECTION-SORT(A)
n = A.length
for j = 1 to n - 1
smallest = j
for i = j + 1 to n
if A[i] < A[smallest]
smallest = i
exchange A[j] with A[smallest]
The algorithm maintains the loop invariant that at the start of each iteration of the outer for loop, the subarray consists of the smallest elements in the array , and this subarray is in sorted order. After the first elements, the subarray contains the smallest elements, sorted, and therefore element must be the largest element.
The running time of the algorithm is for all cases.
2.2-3
Consider linear search again (see Exercise 2.1-3). How many elements of the in- put sequence need to be checked on the average, assuming that the element being searched for is equally likely to be any element in the array? How about in the worst case? What are the average-case and worst-case running times of linear search in -notation? Justify your answers.
If the element is present in the sequence, half of the elements are likely to be checked before it is found in the average case. In the worst case, all of them will be checked. That is, checks for the average case and for the worst case. Both of them are .
2.2-4
How can we modify almost any algorithm to have a good best-case running time?
Modify the algorithm so it tests whether the input satisfies some special-case condition and, if it does, output a pre-computed answer. The best-case running time is generally not a good measure of an algorithm.
2.3-1
Using Figure 2.4 as a model, illustrate the operation of merge sort on the array .
2.3-2
Rewrite the procedure so that it does not use sentinels, instead stopping once either array or has had all its elements copied back to and then copying the remainder of the other array back into .
MERGE(A, p, q, r)
n[1] = q - p + 1
n[2] = r - q
let L[1..n[1]] and R[1..n[2]] be new arrays
for i = 1 to n[1]
L[i] = A[p + i - 1]
for j = 1 to n[2]
R[j] = A[q + j]
i = 1
j = 1
for k = p to r
if i > n[1]
A[k] = R[j]
j = j + 1
else if j > n[2]
A[k] = L[i]
i = i + 1
else if L[i] ≤ R[j]
A[k] = L[i]
i = i + 1
else
A[k] = R[j]
j = j + 1
2.3-3
Use mathematical induction to show that when is an exact power of , the solution of the recurrence
is .
The base case is when , and we have .
For the inductive step, our inductive hypothesis is that . Then
\begin{align}
T(n) & = 2T(n / 2) + n \\
& = 2(n / 2) \lg(n / 2) + n \\
& = n(\lg n - 1) + n \\
& = n\lg n - n + n \\
& = n\lg n,
\end{align}
which completes the inductive proof for exact powers of .
2.3-4
We can express insertion sort as a recursive procedure as follows. In order to sort , we recursively sort and then insert into the sorted array . Write a recurrence for the running time of this recursive version of insertion sort.
Since it takes time in the worst case to insert into the sorted array , we get the recurrence
Although the exercise does not ask you to solve this recurrence, its solution is .
2.3-5
Referring back to the searching problem (see Exercise 2.1-3), observe that if the sequence is sorted, we can check the midpoint of the sequence against and eliminate half of the sequence from further consideration. The binary search algorithm repeats this procedure, halving the size of the remaining portion of the sequence each time. Write pseudocode, either iterative or recursive, for binary search. Argue that the worst-case running time of binary search is .
Procedure takes a sorted array , a value , and a range of the array, in which we search for the value . The procedure compares to the array entry at the midpoint of the range and decides to eliminate half the range from further consideration. We give both iterative and recursive versions, each of which returns either an index such that , or if no entry of contains the value . The initial call to either version should have the parameters , , , .
ITERATIVE-BINARY-SEARCH(A, v, low, high)
while low ≤ high
mid = floor((low + high) / 2)
if v == A[mid]
return mid
else if v > A[mid]
low = mid + 1
else high = mid - 1
return NIL
RECURSIVE-BINARY-SEARCH(A, v, low, high)
if low > high
return NIL
mid = floor((low + high) / 2)
if v == A[mid]
return mid
else if v > A[mid]
return RECURSIVE-BINARY-SEARCH(A, v, mid + 1, high)
else return RECURSIVE-BINARY-SEARCH(A, v, low, mid - 1)
Both procedures terminate the search unsuccessfully when the range is empty (i.e., ) and terminate it successfully if the value has been found. Based on the comparison of to the middle element in the searched range, the search continues with the range halved. The recurrence for these procedures is therefore , whose solution is .
2.3-6
Observe that the while loop of lines 5–7 of the procedure in Section 2.1 uses a linear search to scan (backward) through the sorted subarray . Can we use a binary search (see Exercise 2.3-5) instead to improve the overall worst-case running time of insertion sort to ?
The while loop of lines 5–7 of procedure scans backward through the sorted array to find the appropriate place for . The hitch is that the loop not only searches for the proper place for , but that it also moves each of the array elements that are bigger than one position to the right (line 6). These movements can take as much as time, which occurs when all the elements preceding are larger than . We can use binary search to improve the running time of the search to , but binary search will have no effect on the running time of moving the elements. Therefore, binary search alone cannot improve the worst-case running time of to .
2.3-7
Describe a -time algorithm that, given a set of integers and another integer , determines whether or not there exist two elements in whose sum is exactly .
The following algorithm solves the problem:
- Sort the elements in .
- Form the set KaTeX parse error: Expected '}', got 'EOF' at end of input: …\\{z: z = x - y for some KaTeX parse error: Expected 'EOF', got '}' at position 10: y \in S\\}̲.
- Sort the elements in .
- Merge the two sorted sets and .
- There exist two elements in whose sum is exactly if and only if the same value appears in consecutive positions in the merged output.
To justify the claim in step 4, first observe that if any value appears twice in the merged output, it must appear in consecutive positions. Thus, we can restate the condition in step 5 as there exist two elements in whose sum is exactly if and only if the same value appears twice in the merged output.
Suppose that some value appears twice. Then appeared once in and once in . Because appeared in , there exists some such that , or . Since , the elements and are in and sum to .
Conversely, suppose that there are values such that . Then, since , the value appears in . Thus, is in both and , and so it will appear twice in the merged output.
Steps 1 and 3 require steps. Steps 2, 4, 5, and 6 require steps. Thus the overall running time is .
A reader submitted a simpler solution that also runs in time. First, sort the elements in , taking time. Then, for each element in , perform a binary search in for . Each binary search takes time, and there are are most of them, and so the time for all the binary searches is . The overall running time is .
Another reader pointed out that since is a set, if the value appears in , it appears in just once, and so cannot be a solution.
3.1-1
Let be asymptotically nonnegative functions. Using the basic definition of -notation, prove that .
First, let’s clarify what the function is. Let’s define the function . Then
Since and are asymptotically nonnegative, there exists such that and for all . Thus for , and . Since for any particular , is either or , we have , which shows that
for all (with in the definition of ).
Similarly, since for any particular , is the larger of and , we have for all , and . Adding these two inequalities yields , or equivalently , which shows that
for all (with in the definition of ).
3.1-2
Show that for any real constants and , where ,
To show that , we want to find constants such that for all .
Note that
\begin{align}
n + a & \le n + |a| & \\
& \le 2n & \text{ when } |a| \le n,
\end{align}
and
\begin{align}
n + a & \ge n - |a| & \\
& \ge \frac{1}{2}n & \text{ when } |a| \le \frac{1}{2}n.
\end{align}
Thus, when ,
Since , the inequality still holds when all parts are raised to the power :
\begin{align}
0 \le \Big(\frac{1}{2}n\Big)^b & \le (n + a)^b \le (2n)^b, \\
0 \le \Big(\frac{1}{2}\Big)^b n^b & \le (n + a)^b \le 2^b n^b.
\end{align}
Thus, , , and satisfy the definition.
3.1-3
Explain why the statement, ‘‘The running time of algorithm is at least ,’’ is meaningless.
Let the running time be . means that for some function in the set . This statement holds for any running time , since the function for all is in , and running times are always nonnegative. Thus, the statement tells us nothing about the running time.
3.1-4
Is ? Is ?
, but .
-
To show that , we must find constants ; such that
Since for all , we can satisfy the definition with and .
-
To show that , assume there exist constants such that
Then . But no constant is greater than all , and so the assumption leads to a contradiction.
3.1-5
Prove Theorem 3.1.
The theorem states:
For any two functions and , we have if and only if and .
From , we have that
We can pick the constants from here and use them in the definitions of and to show that both hold.
From and , we have that
\begin{align}
& 0 \le c_3g(n) \le f(n) & \text{ for all } n \ge n_1 \\
\text{and } & 0 \le f(n) \le c_4g(n) & \text{ for all } n \ge n_2.
\end{align}
If we let and merge the inequalities, we get
Which is the definition of .
3.1-6
Prove that the running time of an algorithm is if and only if its worst-case running time is and its best-case running time is .
If is the worst-case running time and is the best-case running time, we know that
\begin{align}
& 0 \le c_1g(n) \le T_b(n) & \text{ for } n > n_b \\
\text{and } & 0 \le T_w(n) \le c_2g(n) & \text{ for } n > n_w.
\end{align}
Combining them we get
Since the running time is bound between and and the above is the definition of the -notation, proved.
3.1-7
Prove is the empty set.
We know that for any ,
\begin{align}
& \exists n_1 > 0: 0 \le f(n) < cg(n) \\
\text{and } & \exists n_2 > 0: 0 \le cg(n) < f(n).
\end{align}
If we pick , from the problem definition we get
There is no solutions, which means that the intersection is the empty set.
3.1-8
We can extend our notation to the case of two parameters and that can go to infinity independently at different rates. For a given function we denote the set of functions:
\begin{align}
O(g(n, m)) = \{f(n, m):
& \text{ there exist positive constants } c, n_0, \text{ and } m_0 \\
& \text{ such that } 0 \le f(n, m) \le cg(n, m) \\
& \text{ for all } n \ge n_0 \text{ or } m \ge m_0.\}
\end{align}Give corresponding definitions for and .
\begin{align}
\Omega(g(n, m)) = \{f(n, m):
& \text{ there exist positive constants } c, n_0, \text{ and } m_0 \\
& \text{ such that } 0 \le cg(n, m) \le f(n, m) \\
& \text{ for all } n \ge n_0 \text{ or } m \ge m_0.\}
\end{align}
\begin{align}
\Theta(g(n, m)) = \{f(n, m):
& \text{ there exist positive constants } c_1, c_2, n_0, \text{ and } m_0 \\
& \text{ such that } 0 \le c_1 g(n, m) \le f(n, m) \le c_2 g(n, m) \\
& \text{ for all } n \ge n_0 \text{ or } m \ge m_0.\}
\end{align}
3.2-1
Show that if and are monotonically increasing functions, then so are the functions and , and if and are in addition nonnegative, then is monotonically increasing.
\begin{align}
f(m) \le f(n) \quad \text{ for } m \le n \\
g(m) \le g(n) \quad \text{ for } m \le n, \\
\to f(m) + g(m) \le f(n) + g(n),
\end{align}
which proves the first function.
Then
This is true, since and is monotonically increasing.
If both functions are nonnegative, then we can multiply the two equalities and we get
3.2-2
Prove equation .
\begin{align}
a^{\log_b c} = a^\frac{\log_a c}{\log_a b} = (a^{\log_a c})^{\frac{1}{\log_a b}} = c^{\log_b a}
\end{align}
3.2-3
Prove equation . Also prove that and .
We use Stirling’s approximation:
\begin{align}
\lg(n!)
& = \lg\Bigg(\sqrt{2\pi n}\Big(\frac{n}{e}\Big)^n\Big(1 + \Theta(\frac{1}{n})\Big)\Bigg) \\
& = \lg\sqrt{2\pi n } + \lg\Big(\frac{n}{e}\Big)^n + \lg\Big(1+\Theta(\frac{1}{n})\Big) \\
& = \Theta(\sqrt n) + n\lg{\frac{n}{e}} + \lg\Big(\Theta(1) + \Theta(\frac{1}{n})\Big) \\
& = \Theta(\sqrt n) + \Theta(n\lg n) + \Theta(\frac{1}{n}) \\
& = \Theta(n\lg n).
\end{align}
The other two are
and
3.2-4
Is the function polynomially bounded? Is the function polynomially bounded?
is not polynomially bounded, but is.
Proving that a function is polynomially bounded is equivalent to proving that for the following reasons.
- If is polynomially bounded, then there exist constants , , such that for all , . Hence, , which, since and are constants, means that .
- Similarly, if , then is polynomially bounded.
In the following proofs, we will make use of the following two facts:
- (by equation ).
-
, because
\begin{align}
\lg(\lceil \lg n \rceil!) & = \Theta(\lceil \lg n \rceil \lg \lceil \lg n \rceil) \\
& = \Theta(\lg n\lg\lg n) \\
& = \omega(\lg n).
\end{align}
Therefore, , and so is not polynomially bounded.
\begin{align}
\lg(\lceil \lg\lg n \rceil!) & = \Theta(\lceil \lg\lg n \rceil \lg \lceil \lg\lg n \rceil) \\
& = \Theta(\lg\lg n\lg\lg\lg n) \\
& = o((\lg\lg n)^2) \\
& = o(\lg^2(\lg n)) \\
& = o(\lg n).
\end{align}
3.2-5
Which is asymptotically larger: KaTeX parse error: Expected group after '^' at position 8: \lg(\lg^̲\*n) or KaTeX parse error: Expected group after '^' at position 4: \lg^̲\*(\lg n)?
KaTeX parse error: Expected group after '^' at position 4: \lg^̲\*(\lg n) is asymptotically larger because KaTeX parse error: Expected group after '^' at position 4: \lg^̲\*(\lg n) = \lg….
3.2-6
Show that the golden ratio and its conjugate both satisfy the equation .
\begin{align}
\phi^2 - \phi - 1
& = \big(\frac{1 + \sqrt 5}{2}\big)^2 - \frac{1 + \sqrt 5}{2} - 1 \\
& = \frac{1 + 2\sqrt 5 + 5 - 2 - 2\sqrt 5 - 4}{4} \\
& = 0.
\end{align}
\begin{align}
\hat\phi^2 - \hat\phi - 1
& = \big(\frac{1 - \sqrt 5}{2}\big)^2 - \frac{1 - \sqrt 5}{2} - 1 \\
& = \frac{1 - 2\sqrt 5 + 5 - 2 + 2\sqrt 5 - 4}{4} \\
& = 0.
\end{align}
3.2-7
Prove by induction that the th Fibonacci number satisfies the equality
where is the golden ratio and is its conjugate.
We have two base cases: and . For , we have
\frac{\phi^0 - \hat\phi^0}{\sqrt 5}
& = \frac{1 - 1}{\sqrt 5} \\
& = 0 \\
& = F_0,
and for , we have
\begin{align}
\frac{\phi^1 - \hat\phi^1}{\sqrt 5}
& = \frac{(1 + \sqrt 5) - (1 - \sqrt 5)}{2\sqrt 5} \\
& = \frac{2\sqrt 5}{2\sqrt 5} \\
& = 1 \\
& = F_1.
\end{align}
For the inductive case, the inductive hypothesis is that and . We have
F_i & = F_{i - 1} + F_{i - 2} & \text{(equation (3.22)} \\
& = \frac{\phi^{i - 1} - \hat\phi^{i - 1}}{\sqrt 5} + \frac{\phi^{i - 2} - \hat\phi^{i - 2}}{\sqrt 5} & \text{(inductive hypothesis)} \\
& = \frac{\phi^{i - 2}(\phi + 1) - \hat\phi^{i - 2}(\hat\phi + 1)}{\sqrt 5} \\
& = \frac{\phi^{i - 2}\phi^2 - \hat\phi^{i - 2}\hat\phi^2}{\sqrt 5} & \text{(Exercise 3.2-6)} \\
& = \frac{\phi^i - \hat\phi^i}{\sqrt 5}.
3.2-8
Show that implies .
From the symmetry of ,
Let’s find ,
Let’s divide the two,
The last step above follows from the property that any polylogarithmic function grows more slowly than any positive polynomial function, i.e., that for constants , we have . Substitute for , for , and for , giving .
Therefore, , and so is polynomially bounded.
3-1
Let
where , be a degree- polynomial in , and let be a constant. Use the definitions of the asymptotic notations to prove the following properties.
a. If , then .
b. If , then .
c. If , then .
d. If , then .
e. If , then .
Let’s see that . We need do pick , such that
When we divide by , we get
and
If we choose , then we can choose ,
Now we have and , such that
which is the definition of .
By chosing we can prove the inequality and thus the inequality.
It is very similar to prove the other inequalities.
3-2
Indicate for each pair of expressions in the table below, whether is , , , , or of . Assume that , , and are constants. Your answer should be in the form of the table with ‘‘yes’’ or ‘‘no’’ written in each box.
\begin{array}{ccccccc}
A & B & O & o & \Omega & \omega & \Theta \\
\hline
\lg^k n & n^\epsilon & yes & yes & no & no & no \\
n^k & c^n & yes & yes & no & no & no \\
\sqrt n & n^{\sin n} & no & no & no & no & no \\
2^n & 2^{n / 2} & no & no & yes & yes & no \\
n^{\lg c} & c^{\lg n} & yes & no & yes & no & yes \\
\lg(n!) & \lg(n^n) & yes & no & yes & no & yes
\end{array}