Starry Code

Detecting a Loop in Linked Lists and Finding a Start of the Loop

2021-07-12T00:00:00-05:00

A Linked List is a linear data structure where, unlike the structure of an array, the elements’ data part and address part are stored separately. Each element inside a linked list is linked using pointers and addresses. Each element is called a node. Due to the structure of the linked list, an element cannot be accessed directly. However, the insertions and deletions of the linked list are far less complicated compared to those of the array. Therefore, preferences are given to using linked lists over arrays.

Contents

1Linked List

1. Linked List¶

A linked list would normally have a start and an end. The first node pointing to the second node, the second node pointing to the third node, etc. The last element in the linked list points to null, but nothing points to the first element in the linked list.

Figure 1: Typical linked list

1.1. Detecting a Loop¶

How about when there’s a loop inside a list? In this post, detecting a loop inside a list, and detecting the start of the loop will be discussed. Consider the following linked list in Figure (2).

Figure 2: Linked list with a loop

Detecting a loop inside a linked list is the first step. There are two pointers: a slower pointer ‘S’ and a faster pointer ‘F’ will be used to detect a loop.

The slower pointer will be moving one node at a time whereas the faster pointer will be moving two nodes at a time. A linked list without a loop would never have two pointers pointing to the same node since the faster pointer is always ahead of slower pointer and the distance between the two pointers will always be increasing every time the pointers move. This is Floyd’s Cycle-finding algorithm, and it is also called the “tortoise and the hare algorithm”. (The faster pointer is a hare, and the slower pointer is a tortoise.) Therefore, if the two pointers wind up pointing to the same node, it implies that there is a loop inside a list.

This is the strategy to detect a loop inside a list. The following is the illustration of the two pointers moving and winding up pointing to the same node.

Detecting a Loop Illustration

Figure 3: Loop detection 1

Figure 4: Loop detection 2

Figure 5: Loop detection 3

Figure 6: Loop detection 4

Figure 7: Loop detection 5

Figure 8: Loop detection 6

Figure 9: Loop detection 7

Figure (10) is the gif represention of the Floyd's Cycle-finding algorithm described above. Note that the faster pointer moves two nodes at a time whereas the slower pointer moves one node at a time. You can see that both pointers pointed to the node 3, which implies that this linked list has a loop inside itself.

Figure 10: Loop detection gif

1.2. Finding the Start of the Loop¶

The next step is to figure out the start of the loop. For this step, the faster pointer stays pointing at the same node continuing from the previous step, and the slower pointer points to the starting node of the list. Then both pointers move one node at a time until they meet. Once the pointers meet or point to the same node, then that particular node is the start of the loop.

Finding the start of the Loop Illustration

Figure 11: Finding start 1

Figure 12: Finding start 2

Figure 13: Finding start 3

Figure 14: Finding start 4

Figure 15: Finding start 5

Figure 16: Finding start 6

Figure (17) is the gif represention of finding the start of the loop as described above. Note that the faster pointer stays at the same node from Figure (9) and the slower pointer starts from the very first node of the list. Both of the pointers move one node at a time and they meet at the start of the loop.

Figure 17: Finding start gif

1.3. Algorithm Explanation¶

How does this algorithm work? Let l be the length of the loop, and m be the distance between the start of the loop and the very first node of the list.

l and m Illustration

In the following diagram, l is 5 since 5 nodes are forming a loop and m is 4 since the distance between node 6 and node 7 is 4.

Figure 18: Algorithm explanation 1

k Illustration

Let k be the distance between the start of the loop and the node that was pointed by both pointers while detecting the loop. Recall that in the first step (Floyd’s cycle-finding algorithm), both pointers met at the node 3. Therefore, in the following diagram, k is 1 since the distance between the node 7 and the node 3 is 1.

Figure 19: Algorithm explanation 2

Using these three variables, the distance traveled by the slower pointer is illustrated as the following:

$$ Distance\_S = m + p * l + k \tag{1}$$

Since the pointer traveled from the start of the list, entered the loop and looped p times, and k amount since that is the end of the travel.

Similarly, the distance traveled by the faster pointer is illustrated as the following:

$$ Distance\_F = m + q * l + k \tag{2}$$

Note that p is less than q since the speed of the faster pointer is faster than that of the slower pointer and therefore the faster pointer travels more than the slower pointer.

Since the faster pointer traveled twice as fast as the slower pointer, the distance traveled by the faster pointer is as twice as the distance traveled by the slower pointer as well.

$$ Distance\_F = Distance\_S * 2 \tag{3}$$

$$ m + q*l + k = 2*(m + p*l + k) \tag{4}$$

By simplifying the above equation, it is equivalent to the following:

$$ m + k = (q – 2p)*l \tag{5}$$

Which implies that m + k is a multiple of l. Note that (q - 2p) will be replaced with x.

$$ m + k = x * l \tag{6}$$

$$ m = x * l – k \tag{7}$$

Hence, by the time the slower pointer entered the loop and traveled the distance of m, the faster pointer had also traveled the distance of m, or x * l – k since both pointers are moving at the same pace. Since m + k is a multiple of l and the faster pointer started from k, both pointers meet at the start of the loop.

1.4. Code Implementation¶

Source Code in Python

            
               # Author: Violet Oh
                Class ListNode:
                    def __init__(self, x):
                        self.val = x
                        self.next = None
                Class LinkedList:
                    def hasCycle(self, head:ListNode) -> bool:
                        slow = fast = head
                        while fast and fast.next:
                            slow = slow.next
                            fast = fast.next.next
                            if fast == slow:               
                                return True
                        return False

                # Create a linked list
                linked_list = LinkedList()
                linked_list.push(4)
                linked_list.push(9)
                linked_list.push(8)
                linked_list.push(3)
                linked_list.push(7)
                linked_list.push(5)
                linked_list.push(2)
                linked_list.push(1)
                linked_list.push(6)
                answer = linked_list.hasCycle(linked_list.head)
                print(answer)

                # Make a loop inside the list
                linked_list.head.next.next.next.next.next.next.next.next.next = linked_list.head.next.next.next.next
                answer = linked_list.hasCycle(linked_list.head)
                print(answer)

Out[24]:

False
True

Source Code in Java

            
               @author: Violet Oh 
                class ListNode {
                      int val;
                      ListNode next;
                      ListNode(int x) {
                          val = x;
                          next = null;
                      }
                  }

                public class LinkedList {
                    public boolean hasCycle(ListNode head) {
                        if(head==null) return false;
                        ListNode walker = head;
                        ListNode runner = head;
                        while(runner.next!=null && runner.next.next!=null) {
                            walker = walker.next;
                            runner = runner.next.next;
                            if(walker==runner) 
                                return true;
                        }
                        return false;
                    }
                }

Visual Understanding of Insertion Sort Algorithm

2020-09-01T00:00:00-05:00

In computer science, a sorting algorithm is an algorithm that puts elements of a list in a certain order. Efficient sorting is important for optimizing the efficiency of other algorithms (such as search and merge algorithms) that require input data to be in sorted lists. Sorting is also often useful for canonicalizing data and for producing human-readable output. - From Wikipedia

There are many sorting algorithms. One of the most important factor when choosing which sorting algorithm to use is its algorithm complexity, represented in Big-O notations. Figure (1) summarizes the the algorithm complexity of various sorting methods. Note that $N$ is is an array size, $O$ is the worst-case scenario, $\Omega$ is the best-case scenario, and $\theta$ is the average-case scenario. This post focuses on Insertion Sort.

Figure 1: Algorithm complexities and their efficiencies

Contents

1Insertion Sort

1. Insertion Sort¶

Insertion sort is an in-place comparison-based algorithm. In the best case scenario, in which the array is nearly-sorted, it has $O(N)$ time complexity. In the worst case scenario, in which the array is reserve-sorted, it has $O(N^2)$ time complexity, according to figure (1). It has the following pros and cons:

Pros

When array size is small, insertion sort is faster than divide-and-conquer algorithms (quicksort, mergesort, heapsort), because they have extra overhead from the recursive function calls.
If data is almost sorted, it can be very fast, approaching $O(N)$ time complexity
In-place; i.e., only requires a constant amount of $O(1)$ of additional memory space
More efficient in practice than most other simple quadratic $O(N^2)$ algorithms such as selection or bubble sort.
Simple and easy to implement

Cons

Bad for large data sets due to its quadratic nature $O(N^2)$.
Writes more than selection sort $O(1)$, whereas insertion is $O(N^2)$.

1.1. Understanding Insertion Sort¶

Insertion sort works in a similar way that you sort playing cards in your hands. The algorithm searchs an array sequentially, and elements in the unsorted sub-list is inserted into the sorted sub-list. Similar to selection sort, the algorithm divides the list into two parts:

The sub-list which is already sorted.
Remaining sub-list which is unsorted.

An element which is to be 'inserted' in this sorted sub-list, has to find its appropriate position to be inserted, by comparing itself to its predecessor. Hence the name, insertion sort. Note that insertion sort is very similar to selection sort.

Let's dive into the illustrations for better understanding of insertion sort. Imagine that you want to sort the books shown in figure (2) in ascending order of sizes.

Figure 2: Books to sort

1st Iteration

Insertion sort divides a list into sorted & unsorted sublists. The grey divider represents the split between sorted (left) vs unsorted (right) array. At the beginning, nothing is sorted. We move the divider one index to the right.

Figure 3: Insertion sort 1

2nd Iteration

We compare book 2 (left) and book 6 (right) adjacent to the grey divider. If left < right, which means that the books are already sorted, move the grey divider one index to the right. Anything on the left side of the divider is sorted. Anything on the right side of the divider is unsorted, and will be "inserted" into the appropriate index of the sorted sub-list.

Figure 4: Insertion sort 2

3rd Iteration

We compare book 6 (left) and book 1 (right) adjacent to the grey divider. It turned out that the left item is greater than the right item. We take out the current unsorted item (right, book 1) and briefly keep it in the temporary storage (the hand icon), until it finds the appropriate index to be inserted.

Figure 5: Insertion sort 3

The books on the left side of the divider shifts to the right, until the current block (book 1) in the temporary storage finds its right position.

Figure 6: Insertion sort 4

Once the current block's (book 1's) appropriate index is found, the block is inserted into that index. Once the left side is sorted again, move the grey divider one index to the right.

Figure 7: Insertion sort 5

4th Iteration

We repeat the same steps from above.

Figure 8: Insertion sort 6

Figure 9: Insertion sort 7

Figure 10: Insertion sort 8

5th Iteration

Figure 11: Insertion sort 9

Figure 12: Insertion sort 10

Figure 13: Insertion sort 11

6th Iteration

Figure 14: Insertion sort 12

Figure 15: Insertion sort 13

The grey divider reached the end and the books are now sorted.

Figure 16: Insertion sort 14

Figure (17) is the gif represention of the insertion sort algorithm described above. Notice that the left side of the grey divider is a sorted sub-list, and right side of the divider is an unsorted sub-list. At the last iteration, the grey wall is at the end of the books, indicating the fact that all books are now sorted in an ascending order of sizes.

Figure 17: Insertion sort gif

1.2. Code Implementation¶

    
import java.util.Arrays;
//@author: Violet Oh

public class Insertion_Sort {
    public static void main(String[] args) {
        int[] array = {2, 6, 1, 5, 3, 4}; // The array we discussed.
        System.out.println(Arrays.toString(array));
        InsertionSort(array);
        System.out.println(Arrays.toString(array));
    }
    public static void InsertionSort(int[] array) {
        for(int i = 1; i < array.length; ++i)   // Starts from the second element.
        {
            int temp = array[i];    // Save the second element to another variable, "temp".

            int j;
            for(j = i; j > 0; --j)  // Shifts the elements until they are in order.
            {
                if(array[j-1] > temp)   // Checks if the first element is NOT SMALLER than the second element.
                {
                    array[j] = array[j-1];      // If so, the first element gets shifted to right (index + 1).
                }
                else
                {
                    break;  // If the first element is SMALLER than the second element already, nothing happens.
                }
            }
            array[j] = temp;    // The proper position for the element that was being compared with the element right before this.
            System.out.println(Arrays.toString(array)); // Print it out to see the changes.
        }
    }
}

Out[24]:

[2, 6, 1, 5, 3, 4]
[2, 6, 1, 5, 3, 4]
[1, 2, 6, 5, 3, 4]
[1, 2, 5, 6, 3, 4]
[1, 2, 3, 5, 6, 4]
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]

Visual Understanding of Selection Sort Algorithm

2020-08-19T00:00:00-05:00

In computer science, a sorting algorithm is an algorithm that puts elements of a list in a certain order. Efficient sorting is important for optimizing the efficiency of other algorithms (such as search and merge algorithms) that require input data to be in sorted lists. Sorting is also often useful for canonicalizing data and for producing human-readable output. - From Wikipedia

Figure 1: Algorithm complexities and their efficiencies

Contents

1Selection Sort

1. Selection Sort¶

Selection sort is an in-place comparison sorting algorithm. It has an $O(N^2)$ time complexity at all times, according to figure (1). Effectively, the only reason that schools still teach selection sort is because it's an easy-to-understand, teachable technique, on the path to more complex and powerful algorithms. It has the follwoing Pros and Cons:

Pros

It is a straightforward and teachable technique.
It does no more than $N$ swaps, and thus is useful where swapping is expensive. However, this is rarely an important design factor, because there are better algorihtms for it.
It is an in-place algorithm. No additional temporary storage is required beyond what is needed to hold the original list (space complexity = $O(1)$)

Cons

Bad for large lists due to $O(N^2)$ time complexity
Other sorting algorithms are better.

1.1. Understanding Selection Sort¶

The selection sort algorithm works by repeatedly finding the smallest element from unsorted part and putting it at the beginning of the sorted part. Effectively, the algorithm divides the list into two parts:

The sublist which is already sorted.
Remaining sublist which is unsorted.

For each iteration, the smallest element from the unsorted sublist is picked and moved to the end of the sorted sublist. Let's dive into the illustrations for better understanding of selection sort. Imagine that you want to sort the books shown in figure (2) in ascending order of sizes.

Figure 2: Books to sort

1st Iteration

Selection sort divides a list into sorted & unsorted sublists. The grey divider represents the split between sorted (left) vs unsorted (right) array. At the beginning, nothing is sorted.

Figure 3: Selection sort 1

The algorithm finds the smallest value in the unsorted sublist, and then put it in the last index of the sorted sublist. Since this is the first iteration, the smallest value in the unsorted sublist becomes the first element in the sorted sublist.

Figure 4: Selection sort 2

2nd Iteration

After the 1st iteration, the grey divider (that splits sorted vs unsorted) moves 1 index to the right. The smallest value in the new unsorted sublist is identified, and then moved to the last index of the sorted sublist. From now on, the same procedures are repeated until sorting is completed.

Figure 5: Selection sort 3

3rd Iteration

Figure 6: Selection sort 4

4th Iteration

Figure 7: Selection sort 5

5th Iteration

Figure 8: Selection sort 6

6th Iteration

Sorting is now over. Notice that the grey divider moved N=6 times total, as mentioned in the 2nd Pros of selection sort above.

Figure 9: Selection sort 7

Figure (10) is the gif represention of the selection sort algorithm described above. Notice that the left of grey divider is sorted sublist, and right is the unsorted sublist. At the last iteration, the grey wall is at the end of the books, representing the fact that all books are now sorted in ascending order. Also notice that the grey divider moved only 6 times, which supports the 2nd pros of selection sort mentioned above.

Figure 10: Selection sort gif

1.2. Code Implementation¶

    
import java.util.Arrays;
//@author: Violet Oh

public class Selection_Sort {
    public static void main(String[] args) {
        int[] array = {2, 6, 1, 5, 3, 4}; // The array we discussed.
        System.out.println(Arrays.toString(array));
        SelectionSort(array);
        System.out.println(Arrays.toString(array));
    }
    public static void SelectionSort(int[] array) {
        for (int i = 0; i < array.length - 1; ++i)
        {
            int idxMin = i;
            for (int j = i + 1; j < array.length; ++j)  // Using nested loop.
            {
                if (array[j] < array[idxMin])   // If value at the index of j is smaller than that of i.
                {
                    idxMin = j;
                }
            }
        int temp = array[i];    // The value at the index of i gets stored in an instance variable.
        array[i] = array[idxMin]; // Put the smaller value to the index of i.
        array[idxMin] = temp;   // Replace the swapped value with the smaller value.

        System.out.println(Arrays.toString(array)); // Print it out to see the changes.
        }
    }
}

Out[24]:

[2, 6, 1, 5, 3, 4]
[1, 6, 2, 5, 3, 4]
[1, 2, 6, 5, 3, 4]
[1, 2, 3, 5, 6, 4]
[1, 2, 3, 4, 6, 5]
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]