Mark1626 Home Page

Deep Dive: On Parallel graph algorithms

2022-05-27T00:00:00+00:00

Traditional graph traversal algorithms DFS and BFS are recursive. There are iterative versions as well but our next question would be on how to scale these to multiple CPU cores, or across muliple GPU cores. This deep dive is to explore parallel graph algorithms and on what type of hardware they would shine

Parallel BFS

In a parallel BFS we create a list of all the nodes in the layer, then we distribute this to threads for processing, a barrier is needed after between layers in the graph to ensure consistency.

A small psuedo code of this would be as follows. This wouldn’t be the best way of doing this in parallel, the example is for representation purposes

void bfs_parallel(source s) {
  Curr_Nodes = {}
  Next_Nodes = {}

  push(s, Curr_Nodes)

  #pragma omp parallel shared(Curr_Nodes, Next_Nodes)
  {

    while (Curr_Nodes !empty) {

      #pragma omp for
      for (node : Curr_Nodes) {

        // Add neighbours for next level
        for (v : u.neighbour()) {
          #pragma omp critical
          {
            push(v, Next_Nodes)
          }
        }

        process(node)
      }

      #pragma omp barrier

      #pragma omp master
      {
        Current_Nodes = Next_Nodes;
        Next_Nodes = {};
        level += 1;
      }

      #pragma omp barrier
    }
  }
}

This idea runs into some problems

This cannot be used for algorithms where the order is important like range based algorithms.
The amount of workload received by the threads will change in different layers of the graph.
Synchronization between layers would mean that we need to have a decent number of nodes in a layer to actually benefit from parallelism
Data locality is a challenge
- If we were in a distributed system it would mean that the data has to be transferred to the remote system adding communication challenges
- If we were using a multicore / threading data being in the same cache line would be a challenge

Note: On a distributed system an alternative way is to partition the graph across nodes

Graph algorithms in the GPU

A recursive BFS would perform very badly on the GPU. If we were to use an iterative approach we would run into the problem of thread divergence. GPUs have SIMT architecture, where we would get max performance if all the threads in a thread warp execute the same instruction. A relatively simple GPU kernel without a lot of conditions would have the best performance.

There is a case study from Nvidia’s Technical Blog Tree Traversal on GPU on graph traversal in which they assign one thread per leaf node in a BVH and threads will process objects that are nearby in 3D space.

References

Wikipedia. Parallel breadth-first search. URL https://en.wikipedia.org/wiki/Parallel_breadth-first_search
Tero Karras. Thinking Parallel, Part 2: Tree Traversal on GPU. Nvidia Technical Blog . URL https://developer.nvidia.com/blog/thinking-parallel-part-ii-tree-traversal-gpu/
Tero Karras. Thinking Parallel, Part 3: Tree Construction on GPU. Nvidia Technical Blog . URL https://developer.nvidia.com/blog/thinking-parallel-part-iii-tree-construction-gpu/

Deep Dive - The Cost of Integer Division

2022-05-20T00:00:00+00:00

Integer division and modulo has always been expensive in the hardware. Modern CPUs convert ASM into uops internally which can then even be processed out of order, but even so division takes a lot of clock cycles to complete and stalls the pipeline. In this case study we will be compare different approaches to unsigned integer division.

Approaches

x86 unsigned integer division

When we do not know the divisor before hand the compiler uses the idiv instruction. The idiv instruction calculates both the divisor and reminder.

For the following code GCC will generate an idiv instruction for both the function

uint32_t mod(uint32_t n, uint32_t d) {
    return n % d;
}

uint32_t quo(uint32_t n, uint32_t d) {
    return n / d;
}

Between the assembly generated for division and reminder the only difference is the movl %edx, %eax done to extract the reminder from the edx register.

mod(unsigned int, unsigned int):
        movl    %edi, %eax
        xorl    %edx, %edx
        divl    %esi
        movl    %edx, %eax
        ret
quo(unsigned int, unsigned int):
        movl    %edi, %eax
        xorl    %edx, %edx
        divl    %esi
        ret

This instruction is very expensive, if we view the CPU execution timeline in llvm-mca the div instruction takes a very long to finish execution and also stalls the pipeline.

// The timeline is truncated to fit, it takes is close to 80 cycles

Timeline view:
          0         1         2  ...  7 
                    0123456789   ...  0123456789
Index     0123456789          01 ...            

[0,0]     DeER .    .    .    .  ...  .    .   .   movl	%edi, %eax
[0,1]     D--R .    .    .    .  ...  .    .   .   xorl	%edx, %edx
[0,2]     .Deeeeeeeeeeeeeeeeeee  ... eeeeeeeeeER   divl	%esi

Granlund-Montgomery algorithm

The compiler uses Granlund-Montgomery’s division algorithm for division when the divisor is known.

/* Compiler optimizes to multiplication and shift instructions*/
uint32_t mod(uint32_t n, uint32_t d) {
    return n % 23;
}

Although this has a lot more instructions than the earlier version with idiv, the division is calculated in a single multiplication

mod(unsigned int, unsigned int):
        movl    %edi, %eax
        movl    $2987803337, %edx
        imulq   %rdx, %rax
        shrq    $36, %rax
        imull   $23, %eax, %edx
        movl    %edi, %eax
        subl    %edx, %eax
        ret

The timeline view for a AMD Zen3 architecture CPU shows the modulo being run for 3 iterations. The amount of cycles needed is much less than using idiv

Timeline view:
                    0123456
Index     0123456789       

[0,0]     DeER .    .    ..   movl	%edi, %eax
[0,1]     DeER .    .    ..   movl	$2987803337, %edx
[0,2]     D=eeeER   .    ..   imulq	%rdx, %rax
[0,3]     D====eER  .    ..   shrq	$36, %rax
[0,4]     D=====eeeER    ..   imull	$23, %eax, %edx
[0,5]     DeE-------R    ..   movl	%edi, %eax
[0,6]     .D=======eER   ..   subl	%edx, %eax
[0,7]     .DeeeeeeeE-R   ..   retq
[1,0]     .DeE-------R   ..   movl	%edi, %eax
[1,1]     .D=eE------R   ..   movl	$2987803337, %edx
[1,2]     . D=eeeE---R   ..   imulq	%rdx, %rax
[1,3]     . D====eE--R   ..   shrq	$36, %rax
[1,4]     . D=====eeeER  ..   imull	$23, %eax, %edx
[1,5]     . DeE-------R  ..   movl	%edi, %eax
[1,6]     . D========eER ..   subl	%edx, %eax
[1,7]     .  DeeeeeeeE-R ..   retq
[2,0]     .  DeE-------R ..   movl	%edi, %eax
[2,1]     .  D=eE------R ..   movl	$2987803337, %edx
[2,2]     .  D===eeeE--R ..   imulq	%rdx, %rax
[2,3]     .   D=====eE-R ..   shrq	$36, %rax
[2,4]     .   D======eeeER.   imull	$23, %eax, %edx
[2,5]     .   DeE--------R.   movl	%edi, %eax
[2,6]     .   D=========eER   subl	%edx, %eax
[2,7]     .    DeeeeeeeE--R   retq

libdivide

libdivide replace the expensive integer divides with multiplication and bitshifts similar to the GM algorithm done by the compiler. But unlike the compiler libdivide can be used to optimize runtime constants. libdivide also allows division for SIMD vectors.

LKK algorithm

Lemire Kaser Kurz(LKK) is another algorithm for unsigned integer division. It has similar performance to Granlund-Montgomery and is better for some divisors. This algorithm works best when we precompute the inverse of the divisor beforehand, and the divisor is to an extent a runtime constant.

A full implementation of LKK algorithm can be found in fastmod

const uint32_t d = 9;
const uint64_t c = UINT64_C(0xFFFFFFFFFFFFFFFF) / d + 1;

uint32_t fastmod(uint32_t n) {
  uint64_t lowbits = c * n;
  return ((__uint128_t)lowbits * d) >> 64;
}

fastmod(unsigned int):
        movl    %edi, %eax
        movabsq $2049638230412172402, %rdx
        imulq   %rdx, %rax
        movl    $9, %edx
        mulq    %rdx
        movq    %rdx, %rax
        ret

Timeline view is similar to GM algorithm in term of the number of cycles and instruction level parallelism used.

Timeline view:
                    0123456
Index     0123456789       

[0,0]     DeER .    .    ..   movl	%edi, %eax
[0,1]     DeER .    .    ..   movabsq	$2049638230412172402, %rdx
[0,2]     D=eeeER   .    ..   imulq	%rdx, %rax
[0,3]     DeE---R   .    ..   movl	$9, %edx
[0,4]     D====eeeeER    ..   mulq	%rdx
[0,5]     .D=======eER   ..   movq	%rdx, %rax
[0,6]     .DeeeeeeeE-R   ..   retq
[1,0]     .DeE-------R   ..   movl	%edi, %eax
[1,1]     .D=eE------R   ..   movabsq	$2049638230412172402, %rdx
[1,2]     . D=eeeE---R   ..   imulq	%rdx, %rax
[1,3]     . DeE------R   ..   movl	$9, %edx
[1,4]     . D====eeeeER  ..   mulq	%rdx
[1,5]     . D========eER ..   movq	%rdx, %rax
[1,6]     .  DeeeeeeeE-R ..   retq
[2,0]     .  DeE-------R ..   movl	%edi, %eax
[2,1]     .  D=eE------R ..   movabsq	$2049638230412172402, %rdx
[2,2]     .  D==eeeE---R ..   imulq	%rdx, %rax
[2,3]     .   DeE------R ..   movl	$9, %edx
[2,4]     .   D=====eeeeER.   mulq	%rdx
[2,5]     .   D=========eER   movq	%rdx, %rax
[2,6]     .    DeeeeeeeE--R   retq

Performance

Performance for 100 iterations

Method	Instructions	Total Cycles	Total uOps	uOps Per Cycle	IPC
idiv	500	1462	3800	2.60	0.34
compiler	800	234	1000	4.27	3.42
fastmod	700	235	1000	4.26	2.98

References

Torbjörn Granlund and Peter L. Montgomery. Division by invariant integers using multiplication. SIGPLAN Not., 29(6):61-72, jun 1994. ISSN 0362-1340. doi: 10.1145/773473.178249. URL https://doi.org/10.1145/773473.178249.
Daniel Lemire, Owen Kaser, and Nathan Kurz. Faster remainder by direct computation: Applications to compilers and software libraries. CoRR, abs/1902.01961, 2019. URL http://arxiv.org/abs/1902.01961.
Daniel Lemire @lemire. fastmod. URL https://github.com/lemire/fastmod.
Peter Ammon @ridiculousfish. libdivide. URL https://github.com/ridiculousfish/libdivide.

Auto Vectorization Case Study 2

2021-10-09T00:00:00+00:00

For this case study let’s try to auto-vectorize an implementation of Abelian sandpile model

void stabilize() {
  while(1) {
    int spills = 0;

    for (size_t y = 1; y <= pixel; ++y) {
      for (size_t x = 1; x <= pixel; ++x) {
          const int pos = y * points + x;
          char currSand = buffer[pos];
          char newSand = currSand >= 4 ? currSand - 4 : currSand;
          spills += currSand >= 4;

          newSand = newSand + buffer[pos - points] >= 4;
          newSand = newSand + buffer[pos + points] >= 4;
          newSand = newSand + buffer[pos-1] >= 4;
          newSand = newSand + buffer[pos+1] >= 4;

          state[pos] = newSand;
      }
    }

    std::swap(buffer, state);
    if (!spills) {
      return;
    }
  }
}

Explore this example in [Godbolt]https://godbolt.org/#z:OYLghAFBqd5TKALEBjA9gEwKYFFMCWALugE4A0BIEAZgQDbYB2AhgLbYgDkAjF%2BTXRMiAZVQtGIHgCYBQogFUAztgAKAD24AGfgCsp5eiyahUAUmkAhC5fIrGqIgSHVmmAMLp6AVzZMDbgAyBEzYAHK%2BAEbYpFIA7OQADuhKxM5Mnj5%2BBsmpTkLBoRFs0bE8CfbYjukiRCykRJm%2B/jx22A75TLX1RIXhUTHxdnUNTdmtSiO9If0lg%2BUAlHbo3qSonFwWAMwhqD44ANRmW%2B6TpCHAx7hmWgCCGEyTByFEB4kE6u1HWwAiBzzHdyAgAcx2sdweTxeb3QLyU3z%2BAJOILBN1uIXoMwOqQAXtgAPqvUjYJReABu2AAkph1BBcQTXgBPcjYgh4wkHdQLI5xSwHYlEVZMA6Mg4AKhhcKOVk5YJ5PzRaLJsMwBxoTAgqCQ9QlkW8NBoMRZWp12LqRGw3LMvLRB2ewmx73o9Hhxz%2BWlRd1tarIBzpbIZIoR/zlosBbreH3acpsNkZVptdztdu2iVILGAbBYB3QbESrLYmG9dsEpD99I56mDSL5VfDv0jn3oMasNi5PPBt2TyZNZdQq1IImMqojeoNMTMAFZLMTSfQKdTaczOQspwqtp3u3bewdQgB3IdMEcN/ukQfD77XBsAFm%2BADEDqfz0eDgBaA63kCPgeHosb4vJkoToutK1gnj%2BF5XBG16el2W4APTwQcIjATmFJljQpC5ru2AEMASCRCspBKABdr7r%2BoGjvqhqkFOM4kuSVI0lAorvjwSwrmul7QbBpFmiwFp0bOjGLhAy5clxEbkcOsHdtaCpJvKipxD8XBLPQ3CTvw/hcDo5DoNwQKtjKpKrOs0pbHw5BENoalLAA1iAWxbAAdAAbFoWzXtePBbHEbmTpOWhubIGlcNe/BsCAPkuQAnNIcRxDwWjAtezlxFsk7kDpekGVw/BKCAWjWbZSxwLAKAYDg%2BDEGQlDUHQjCsBw3BWYIwhiBInA8NecjCMoaiaLpeggKFximLGVhtB06SuEeYwtFs5BBDMxSlFIxW5GkQgLVIS1bZ0fRrfMxWVNUQjdKMXjNHt01VJ0l3TEUAxlKdUy7b5ww9EdL0bUsplrBs2y7Ps2DfKcRDnCYUEQkIUIOu8TbVvW7igv%2BsOPK80LJFKEZIkCJxo52GJYhWRIMfOTG0mTIosjT7bWnyApCkGEo48IroyuoMYqYqdzKgQqrqpq2qkLq1FGo%2BosSpMAmWh23rQkBDAgRGHro7c3qluWAYcmGDY1kG9Z/Ij0YbpRVjxgriklr6/rsq8dYG3KTsEybUbNubk2WAziZwd2O5PhRVHjrR07CZToniauk7rpuW47tJL4RkHkFbFefy3scD6py%2B76ft%2BZ6/rJW7K86nNgX8ufHhnH4l92iHIah6DoWqWFsDheEEURJE28mScjq2DZjjRQkUwuzFiW%2B/wcRJsfcTevF93asuCeH49U2JLJz38UnYAeMka3JvOKfJymqepmnabZ%2BmGd72JEeZ2zSPwNnDQsSxINgLA4LEECX%2BFSKIBJzAhcloOI0hpDAlijwScbkQpuWvLFNy2Ub55QKkVEq78HKjWkC5IK0CeAhWBAFOIPlrxZTClsa%2Bw1b75SwToMqiByrIDQLmJ0RoqCanYQwQYwAeAyAEAwC0xFqCRBvpEEI9RGStX4BgNgHBhAAHkmD0BkbQnAWYTCSA0QQYk1QKSFVodgT4/YLSyMoMIdoN9MSRHTKQRkngcA30hgQKKvA1ICCMMAJQAA1Ag%2B8lGJGYBY9qohxCSB6n1RQKgNA330K0IwJg0De0MAQSIhVIBLHQIkToRjXxKK2G%2BdU6BXyQ2wNgV8FJHBkADMUnJzA8wFXaPdWaEA3AfWWkeH6cwyhJBSNtDI11xj9LyOkHp60JgtPOl0d6wyWh3RmY9CZ8wvpXSyAs2WDQVllH%2Bo/bq1liQbD4AArSqDaF5XUCQ18iCDjAFQKgf4PAXLSD9EZawU0Dg1RIGWbYrQDieDzLw350hLILFfqVHBSCXIQNioQ2KcC4jAgSok7gEVyBRSCuc3K3AMHFTfow8gLCICVR4YwCgXD5EcNiPwwRjURGFQgOI2hkjWAOIsfIxRRAVFqJvpo8aOi9KEH0U4QxN8TFVG8OYjxliLRhT0rY%2BxjisAbD0q49xJyvEZj8QEvcQSQkyrCZ1SJvUwkDTibQ/QY1knmGMrYWxmT/76VyekfJhTmkzRcO0%2Ba8yAjdNWr9VoB10idODQUANvSDBnQenMjZUbpkxu%2BhGyZazGi%2BomFMHZUg9lmU4LIcpxzPFhTOTlfglzrm3PuY8gRLy3mpK%2BYQH5FlZAArJTEZt4KGF2XIF/H%2BgwnVhXRZi4qpa6F4q7R/cgjlpCxXcvFNyMhYoZUSiQyh3BqHYrLbiidRLmEoEIAaeqtB5DGu6qa%2BQ5qhoKqQIVRJN6D00CIIyYJmDSA3tGuQN9SgH1PpfVoU5NCcVcB%2BAQA0Bx/H73bVctyNzbxVqebWsk8JoOwbuQ8hDL8J1QriDCuF144jgK2Mg/D0hYGGDRYBrd9DCr4sheRrgmHR3oKw%2BQdCqQXDXiAA%3D%3D%3D) or here

Restructuring for auto vectorization

const int pixel = 1<<8;
const int points = 1<<8;

void stabilize(char* buffer, char* state) {
  int spills = 0;

    for (size_t y = 1; y <= pixel; ++y) {
      for (size_t x = 1; x <= pixel; ++x) {
          const int pos = y * points + x;
          char currSand = buffer[pos];
          char newSand = currSand >= 4 ? currSand - 4 : currSand;
          spills += currSand >= 4;

          newSand = newSand + buffer[pos - points] >= 4;
          newSand = newSand + buffer[pos + points] >= 4;
          newSand = newSand + buffer[pos-1] >= 4;
          newSand = newSand + buffer[pos+1] >= 4;

          state[pos] = newSand;
      }
    }

    std::swap(buffer, state);
    if (!spills) {
      return;
    }
}

To make this reable I’m going to be extracting the y*points + x as an inline function

const int pixel = 1<<8;
const int points = 1<<8;
inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }

void stabilize(char* buffer, char* state) {
  int spills = 0;

  for (size_t y = 1; y <= pixel; ++y) {
    for (size_t x = 1; x <= pixel; ++x) {
      char currSand = buffer[resolveIdx(y, x)];
      char newSand = currSand >= 4 ? currSand - 4 : currSand;
      spills += currSand >= 4;
      // Spill over from neighbours
      newSand += buffer[resolveIdx((y - 1), x)] >= 4;
      newSand += buffer[resolveIdx((y + 1), x)] >= 4;
      newSand += buffer[resolveIdx(y, (x - 1))] >= 4;
      newSand += buffer[resolveIdx(y, (x + 1))] >= 4;

      state[resolveIdx(y, x)] = newSand;
    }
  }
}

OpenMP

const int pixel = 1<<8;
const int points = 1<<8;
inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }

void stabilize(char* buffer, char* state) {
  int spills = 0;

  for (size_t y = 1; y <= pixel; ++y) {
    #pragma omp simd
    for (size_t x = 1; x <= pixel; ++x) {
      char currSand = buffer[resolveIdx(y, x)];
      char newSand = currSand >= 4 ? currSand - 4 : currSand;
      spills += currSand >= 4;
      // Spill over from neighbours
      newSand += buffer[resolveIdx((y - 1), x)] >= 4;
      newSand += buffer[resolveIdx((y + 1), x)] >= 4;
      newSand += buffer[resolveIdx(y, (x - 1))] >= 4;
      newSand += buffer[resolveIdx(y, (x + 1))] >= 4;

      state[resolveIdx(y, x)] = newSand;
    }
  }
}

Encapsulating the function as a class

const size_t pixel = 1 << 8;
const size_t points = pixel;
const size_t size = points * points;

class Sandpile {
  void stabilize(char *state, char *buffer);
  inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }
};

void Sandpile::stabilize(char *state, char *buffer) {
    size_t spills = 0;
    for (size_t y = 1; y <= pixel; ++y) {
      #pragma omp simd
      for (size_t x = 1; x <= pixel; ++x) {
        const int pos = y * points + x;
        char currSand = buffer[pos];
        char newSand = currSand >= 4 ? currSand - 4 : currSand;
        spills += currSand >= 4;
        // Spill over from neighbours
        newSand = newSand + buffer[pos - points] >= 4;
        newSand = newSand + buffer[pos + points] >= 4;
        newSand = newSand + buffer[pos-1] >= 4;
        newSand = newSand + buffer[pos+1] >= 4;

        state[pos] = newSand;
      }
    }

    // print();
    std::swap(buffer, state);
    if (!spills) {
      return;
    }
}

For some reason with the signature I wasn’t able to auto vectorize it

// Work
class Sandpile {
  void stabilize(char *state, char *buffer); 
  inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }
};

// Does not allow for vectorization
class Sandpile {
  char *state;
  char *buffer;
  void stabilize();
  inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }
};

Final auto vectorized verison.

const size_t pixel = 1 << 8;
const size_t points = pixel;
const size_t size = points * points;

class Sandpile {
  vector<char> _state;
  vector<char> _buffer;
private:
  inline size_t resolveIdx(size_t y, size_t x) { return y * points + x; }
  // __restrict__ as we know there will be no overlap
  void stabilize(char *__restrict__ state, char *__restrict__ buffer) {
    size_t spills = 0;
    for (size_t y = 1; y <= pixel; ++y) {
      #pragma omp simd
      for (size_t x = 1; x <= pixel; ++x) {
        const int pos = y * points + x;
        char currSand = buffer[pos];
        char newSand = currSand >= 4 ? currSand - 4 : currSand;
        spills += currSand >= 4;
        // Spill over from neighbours
        newSand = newSand + buffer[pos - points] >= 4;
        newSand = newSand + buffer[pos + points] >= 4;
        newSand = newSand + buffer[pos-1] >= 4;
        newSand = newSand + buffer[pos+1] >= 4;

        state[pos] = newSand;
      }
    }

    // print();
    std::swap(buffer, state);
    if (!spills) {
      return;
    }
  }

  void computeIdentity() {
    // f(ones(n)*6 - f(ones(n)*6)
    char* buffer = _buffer.data();
    char* state = _state.data();

    for (size_t y = 1; y <= pixel; ++y) {
      for (size_t x = 1; x <= pixel; ++x) {
          const int pos = y * points + x;
        buffer[resolveIdx(y, x)] = 6;
      }
    }

    stabilize(buffer, state);

    for (size_t y = 1; y <= pixel; ++y) {
      for (size_t x = 1; x <= pixel; ++x) {
        buffer[resolveIdx(y, x)] = 6 - state[resolveIdx(y, x)];
      }
    }

    stabilize(buffer, state);
  }

};

Perf Scripts

2021-08-17T00:00:00+00:00

perf is a Performance analysis tools for Linux. perf can be used to provide useful statistics about your application with perf stat <app> or sampled and analysed with perf record <app>. When recording with perf we are left with a binary file perf.data which contains information of the all sampled events.

Events from perf.data can be extracted and scripted on with perf script. There is very limited documentation and examples of perf script, so I’m going to be walking through the exploration I did with perf script. For this I’ll be using a perf.data generated from an execution of an albeian sandpile model program.

Dump events from perf.data

First let’s understand what data can be extracted from perf.data. Running perf script will output events from perf.data

perf record
# Walk through perf file and output contents of each record
perf script > out.perf

        sandpile  1751 13289.335292:   10101010 cpu-clock:uhH:      559a3f21248b Fractal::Sandpile::stabilize+0x24b (/home/bench/sandpile)

What each field means

comm - sandpile
tid - 1751
event - cpu-clock:uhpppH
ip - 559a3f21248b
sym - Fractal::Sandpile::stabilize
symoff - +0x24b
time - 13289.335292
dso - (/home/bench/sandpile)

If perf was recorded with -g, the output contains trace like this

perf record -g
perf script > out.perf

sandpile   131 11005.728397:   10101010 cpu-clock:uhH: 
	    560cdca6739a Fractal::Sandpile::stabilize+0x15a (/home/bench/sandpile)
	    560cdca67984 Fractal::Sandpile::computeIdentity+0x84 (/home/bench/sandpile)
	    560cdca670ef main+0x1f (/home/bench/sandpile)
	    7f36527fd09b __libc_start_main+0xeb (/lib/x86_64-linux-gnu/libc-2.28.so)
	41fd89415541f689 [unknown] ([unknown])

If we want to extracting a custom set of fields from perf events

perf script -F  comm,tid,event,ip,sym,srcline,time > out.perf

        sandpile  1751 13289.335292: cpu-clock:uhH:      559a3f21248b Fractal::Sandpile::stabilize
  sandpile.cc:72
        sandpile  1751 13289.345516: cpu-clock:uhH:      559a3f212494 Fractal::Sandpile::stabilize
  sandpile.cc:72

# Add extra field
perf script -F+srcline > out.perf

        sandpile  1751 13289.335292:   10101010 cpu-clock:uhH:      559a3f21248b Fractal::Sandpile::stabilize+0x24b (/home/bench/sandpile)
  sandpile.cc:72
        sandpile  1751 13289.345516:   10101010 cpu-clock:uhH:      559a3f212494 Fractal::Sandpile::stabilize+0x254 (/home/bench/sandpile)

# Remove field
perf script -F+srcline -F-period > out.perf

        sandpile  1751 13289.335292: cpu-clock:uhH:      559a3f21248b Fractal::Sandpile::stabilize+0x24b (/home/bench/sandpile)
  sandpile.cc:72
        sandpile  1751 13289.345516: cpu-clock:uhH:      559a3f212494 Fractal::Sandpile::stabilize+0x254 (/home/bench/sandpile)
  sandpile.cc:72

List of all events in perf we can extract(this was taken from the man page)

Valid types: hw,sw,trace,raw,synth. Fields: comm,tid,pid,time,cpu,event,trace,ip,sym,dso,addr,symoff,srcline,period,iregs,uregs,brstack,brstacksym,flags,bpf-output,brstackinsn,brstackoff,callindent,insn,insnlen,synth,phys_addr,metric,misc

Scripting

There are two ways of scripting the data. You can generate a script from the default template with perf script -g perl or perf script -g python

Or you can create a custom script, for this example we’ll generate a custom script, I’ll cover using the generated script in a separate blog

perf script -F+srcline -F-period -F-time -F-dso -F+sym -F-symoff > out.perf

        sandpile  1751/1751  cpu-clock:uhH:      559a3f21248b Fractal::Sandpile::stabilize
  sandpile.cc:72

The structure of the file is constant so scripting this is similar to scripting any file. The following is a script to extract the occurrence of a particular source line number in the samples, giving us an idea on which symbol and line took time

Case - 1: Identifying a hotspot in the code with source line number and symbol

#!/usr/bin/env perl
# srcline-occurance.pl <perf output>
# Usage perf script -F+srcline -F-period -F-time -F-dso -F+sym -F-symoff | srcline-occurance.pl
use strict;

my %sym_occurance = ();

while (<>) {
  chomp;
  next if $_ =~ /^#/;

  if (/^\s*(\S.+?)\s+(\d+)\/*(\d+)*\s+/) {
    my ($comm, $pid, $tid) = ($1, $2, $3);
    if (not $tid) {
			$tid = $pid;
			$pid = "?";
		}

    if (/(\S+):\s*(\S+)\s+(\S+)/) {
      my ($event, $ip, $sym) = ($1, $2, $3);

      $_ = <>;
      chomp;
      my $srcline;
      if (/\s+(\S+:\d+)/) {
        $srcline = $1;
      } else {
        $srcline = 'Unknown';
        $sym = '?';
        next;
      }

      if (exists $sym_occurance{$srcline}) {
        $sym_occurance{$srcline}{occ}++;
      } else {
        $sym_occurance{$srcline}{symbol} = $sym;
        $sym_occurance{$srcline}{occ} = 1;
      }
    }
  } else {
    print STDERR "Unknown line";
  }
}

foreach my $k (sort { $a cmp $b } keys %sym_occurance) {
	print "$k $sym_occurance{$k}{symbol} $sym_occurance{$k}{occ}\n";
}

sandpile.cc:62 Fractal::Sandpile::stabilize 1
sandpile.cc:63 Fractal::Sandpile::stabilize 1
sandpile.cc:64 Fractal::Sandpile::stabilize 5
sandpile.cc:67 Fractal::Sandpile::stabilize 35
sandpile.cc:70 Fractal::Sandpile::stabilize 3
sandpile.cc:71 Fractal::Sandpile::stabilize 2
sandpile.cc:72 Fractal::Sandpile::stabilize 4
sandpile.cc:73 Fractal::Sandpile::stabilize 15
sandpile.cc:75 Fractal::Sandpile::stabilize 3
sandpile.cc:85 Fractal::Sandpile::stabilize 1

Case - 2: Spitting perf.data of a MPI run into separate files for individual processes

#!/usr/bin/env perl
# split-process.pl <perf output>
# Usage perf script -F+pid | split-process.pl
use strict;

my %files = {};
my $pid = '';
my $comm = '';
my $tid = '';

while (<>) {
  chomp;
  next if $_ =~ /^#/;
  if (/^\s*(\S.+?)\s+(\d+)\/*(\d+)*\s+/) {
    ($pid, $tid) = ($2, $3);
    if (not $tid) {
			$tid = $pid;
			$pid = "?";
		}
  }
  open my $fh, '>>', "out-$pid.perf";
  print $fh $_ . "\n";
}

will generate out-<pid1>.perf, out-<pid2>.perf, out-<pid3>.perf

Auto Vectorization Case Study 1

2021-08-12T00:00:00+00:00

A small case study of auto vectorization.

The Challenge

One challenge in auto vectorization this is that the compiler has no way to know that the pointers a, b and c don’t overlap; meaning the function can be called as case_study_1(x, y, z) or case_study_1(x, x+8, x+16), the latter if vectorized leads to undefined behavior

void case_study_1(int *a, int *b, int *c) {
  for (int i = 0; i < 1<<20; i++) {
    c[i] = a[i] + b[i];
  }
}

Example at Godbolt

Runtime Check of Pointers

Both clang and GCC use runtime checks of pointers to vectorize the loops as there is no guarantee that the pointers don’t overlap - Clang Runtime Check of Pointers

void case_study_1(int *a, int *b, int *c) {
  for (int i = 0; i < 1<<20; i++) {
    c[i] = a[i] + b[i];
  }
}

They generate two sets of ASM one vectorized and another normal and call the appropriate one

case_study_1(int*, int*, int*):
        lea     rcx, [rdi+4]
        mov     rax, rdx
        sub     rax, rcx              # Basic address arithmetic to look for overlap
        cmp     rax, 8                # Check for overlap
        jbe     .L5                   # Fallback to non vectorized .L5 if overlap is detected
        lea     rcx, [rsi+4]
        mov     rax, rdx
        sub     rax, rcx
        cmp     rax, 8                # Check for overlap
        jbe     .L5                   # Fallback to non vectorized .L5 if overlap is detected
        xor     eax, eax
.L3:
        movdqu  xmm0, XMMWORD PTR [rdi+rax]
        movdqu  xmm1, XMMWORD PTR [rsi+rax]
        paddd   xmm0, xmm1
        movups  XMMWORD PTR [rdx+rax], xmm0
        add     rax, 16
        cmp     rax, 4194304
        jne     .L3
        ret
.L5:
        xor     eax, eax
.L2:
        mov     ecx, DWORD PTR [rsi+rax]
        add     ecx, DWORD PTR [rdi+rax]
        mov     DWORD PTR [rdx+rax], ecx
        add     rax, 4
        cmp     rax, 4194304
        jne     .L2
        ret

restrict

This can be simplified by telling the compiler that these pointers will not overlap with the __restrict__, this will result in the compiler generating only vector instructions

void case_study_1(int *__restrict__ a, int *__restrict__ b, int *__restrict__ c) {
  for (int i = 0; i < 1<<20; i++) {
    c[i] = a[i] + b[i];
  }
}

case_study_1(int*, int*, int*):
        xor     eax, eax
.L2:                                            # Vectorized loop
        movdqu  xmm0, XMMWORD PTR [rsi+rax]
        movdqu  xmm1, XMMWORD PTR [rdi+rax]
        paddd   xmm0, xmm1
        movups  XMMWORD PTR [rdx+rax], xmm0
        add     rax, 16
        cmp     rax, 4194304
        jne     .L2
        ret

OpenMP

Another way to achieve vectorization here is with the OpenMP directive #pragma omp simd. By defining the pragma we force the compiler to emit vector instructions for the loop

void case_study_1(int *a, int *b, int *c) {
  // Note: Compiler assumes there is no overlap so this has to used when you are sure there will not be a overlap
  #pragma omp simd
  for (int i = 0; i < 1<<20; i++) {
    c[i] = a[i] + b[i];
  }
}

case_study_1(int*, int*, int*):
        xor     eax, eax
.L2:                                          # Vectorized loop
        movdqu  xmm0, XMMWORD PTR [rsi+rax]
        movdqu  xmm1, XMMWORD PTR [rdi+rax]
        paddd   xmm0, xmm1
        movups  XMMWORD PTR [rdx+rax], xmm0
        add     rax, 16
        cmp     rax, 4194304
        jne     .L2
        ret

Exploring Obsured Code: Part -1

2021-06-04T00:00:00+00:00

This is an snippet I saved back in college. I’m not sure who the original author is for this. Let’s try to understand how this works

// A decimal to binary convertor
// All Credits to the original author
(_$=($,_=[]+[])=>$?_$($>>+!![],($&+!![])+_):_)(255);

Round 1

Let’s format this

(
  _$=($,_=[]+[]) => {
    return $?_$($>>+!![],($&+!![])+_):_
  }
)(255);

Round 2

Rename to some readable variables

// number is the number we are trying to convert to binary
// binary is the binary representation constructed so far
(
  fn = (number, binary=[]+[]) => {
    return number ? fn(number>>+!![],(number&+!![])+binary) : binary
  }
)(255);

Round 3

Simplyfing some expressions

// []+[] === ''
// !![] === true
// +true === 1
// (number&1) + '' === '0' | '1'
(
  fn = (number, binary = "") => {
    return number ? fn(number >> 1, (number & 1) + binary) : binary;
  }
)(255);

Round 4

Simplyfing this down even further. Now this is a standard recursive decimal to binary convertor

// From this (_$=($,_=[]+[])=>$?_$($>>+!![],($&+!![])+_):_)(255); to 
const fn = (number, binary = "") => {
  if (number) {
    binary = (number & 1) + binary;
    number >>= 1;
    return fn(number, binary);
  }
  return binary;
};
fn(255);

Node Module System

2021-05-09T00:00:00+00:00

Similar to the article I wrote about Lua module loaders, the same can be done in nodejs. We can override the require to be able to load custom extensions

Wait a minute is there any practical use case other than a syntax hack?

Well then how about we build a loader so nodejs can read yaml

require("./loader")
const config = require("config.yaml")
const port = config.port || 3000;
const http = require('http');

const requestListener = function (req, res) {
  res.writeHead(200);
  res.end('Hello, World!');
}

const server = http.createServer(requestListener);
server.listen(config.port);

This looks way more practical so let’s begin.

TLDR; Leaving a link to the source code I made for this example

Understanding the Module system

Before we get into this, have you seen stack traces similar to these before? In this case I tried to run node on a non existent file

❯ node test
internal/modules/cjs/loader.js:883
  throw err;
  ^

Error: Cannot find module 'path/to/file'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

What I’m trying to highlight here is the internal/modules/cjs/loader.js, internal/modules/run_main.js, MODULE_NOT_FOUND

Module System

Source@nodejs

Every module in nodejs is converted into a function wrapper of the following structure

(function(exports, require, module, __filename, __dirname) {
// Module code actually lives in here
});

The module object is where we export our code in module.exports
Next this conversion is done by a module loader

Main Module

When a file is run with node, it is set as the main module require.main, this is what we saw in the first call internal/modules/run_main.js earlier in the stack trace

The main module is run inside the VM

Module._load

Source@nodejs Source@nodejs

First it checks if the module is present in cache
Create a new module and save it to cache
Call module.load
- Call Module._extensions[extension](this, filename);

Module._extensions

_extensions contains loaders for all the different file extensions in nodejs. Loaders finally populate the module.exports

In the stacktrace earlier the file is loaded with the cjs loader hence the internal/modules/cjs/loader.js in the trace

In the JSON loader it’s just a simple module.exports = JSON.parse(content)

Native module loaders in NodeJS

Module._compile

Source@nodejs Create Require Source

This does the actual work. It create a new require instance

Run the file contents in the correct scope. Expose the correct helper variables (require, module, exports) to the file.
Returns exception, if any.

Unfortunately I can’t get into the depths of this in this article, so let’s save this for later

Creating our own module extensions

Note: There will always be a better way to do this than to override require, do not use this in production without knowing what you are doing. As per the Nodejs docs this is considered deprecated as it could mess with performance.

Despite the warning this is how things work under the hood, let’s continue for our understanding

Case 1: Module loader for YAML

// Load the yaml as JSON into the variable config
const config = require("./config.yaml")

Since this is a POC I’m going to be using the npm package yaml for the parsing

const fs = require("fs");
const yaml = require("yaml");

/*
 Loader for yaml, based on the json loader in nodejs
 https://github.com/nodejs/node/blob/master/lib/internal/modules/cjs/loader.js#L1143-L1157
*/
function yamlLoader(mod, filename) {
  const content = fs.readFileSync(filename, "utf8");

  try {
    mod.exports = yaml.parse(content)
  } catch (err) {
    err.message = filename + ": " + err.message;
    throw err;
  }
}

Module._extensions[".yaml"] = yamlLoader;
Module._extensions[".yml"] = yamlLoader;

That’s pretty much it!!!!

How does this work?

Module._extensions contains the loaders for each extension
We simply need to add a function to handle .yaml and .yml files
The loader is a function which takes module and filename as arguments
- filename is self explanatory, the name of the file
- The module refers to the module which I described in the previous section. By setting the module.exports we define how a file has to be loaded

Case 2: Extending JS file syntax

// cube.js
class Cube {
  constructor(side) {
    $.side = side
  }

  area() {
    return $.side * $.side
  }
}
// fn.js
const greet = fn (nm) {
  console.log('Hello ' +  nm)
}

greet('mark')

For this I’m going ahead with the same loader I used in my Lua module loader article

const transform = (code) => {
  const patterns = [
    { patt: /\$/g, repl: "this" },
    { patt: /fn \(/g, repl: "function (" },
  ];

  patterns.forEach((pattern) => {
    code = code.replace(pattern.patt, pattern.repl);
  });
  return code;
};

Now we additionally also have to extend the existing cjs loader, this is not quite straight forward as the yaml loader we made

const oldLoader = Module._extensions[".js"];
/**
 * Simplified version of the code from pirates
 * MIT License
 * Copyright (c) 2016-2018 Ari Porad
 * https://github.com/ariporad/pirates/blob/master/LICENSE
 */
Module._extensions[".js"] = function customLoader(mod, filename) {
  let compile = mod._compile;
  mod._compile = function _compile(code) {
    // reset the compile immediately as otherwise we end up having the
    // compile function being changed even though this loader might be reverted
    // Not reverting it here leads to long useless compile chains when doing
    // addHook -> revert -> addHook -> revert -> ...
    // The compile function is also anyway created new when the loader is called a second time.
    mod._compile = compile;
    const newCode = transform(code);

    return mod._compile(newCode, filename);
  };

  // Run the original loader
  // https://github.com/nodejs/node/blob/master/lib/internal/modules/cjs/loader.js#L1118-L1139
  oldLoader(mod, filename);
};

How it works

JS files will have to be compiled for them to work, since a new compile function is too complex, we will have to rely on the old one

Store the original CJS loader
Creating a new mod._compile function
Store the original _compile function
Run transform over the original code
Run mod._compile the original compile function over the new code
Run the original CJS loader, this will now compile the module

Reference

Lua Module Loader

2021-04-21T00:00:00+00:00

What if I told you there is a way to extend the default syntax of Lua? In the below example I use @ as an alias for self, and fn as an alias of function. This and may more can be achived with the module loader in Lua

-- myfile.lua
local greet = fn(name)
  print('Hello ' .. name)
end

greet('mark1626')

---
-- cube.lua
Cube = {a = 1}

function Cube:new(o)
  o = o or {}
  setmetatable(o, @)
  @.__index = @
  return o
end

function Cube:area()
  return @.a * @.a
end

return Cube

Module Loader

The package loaders is a table used by require to load modules. This can be overridden with a simple table.insert to add our custom logic

In the below I implement a custom loader that runs a function transform over the contents of the file

local custom_loader = function(modulename)
  local modulepath = string.gsub(modulename, "%.", "/")
  for path in string.gmatch(package.path, "([^;]+)") do
    local filename = string.gsub(path, "%?", modulepath)
    local file = io.open(filename, "rb")
    if file then
      local content = assert(file:read("*a"))
      local transformed_file = transform(content)
      return assert(loadstring(transformed_file, modulename))
    end
  end
  return "Unable to load file " .. modulename
end

-- Override the default loader
table.insert(package.loaders, 2, custom_loader)

Note: This will work on all the future require after importing this module Note: In Lua 5.3 this is called package searchers

Transformation Use Case

Now let’s have a look at the first example I posted

local greet = fn(name)
  print('Hello ' .. name)
end

This is the transform function I use to achieve this uses a simple string replace. fn( is converted into function(

-- This will transform some patterns in our file
local function transform(s)
  local str = s
  local int = "([%d]+)"
  
  local patterns = {
    { patt = "@", repl = "self" },
    { patt = "&&", repl = " and " },
    { patt = "||", repl = " or " },
    { patt = "fn%(", repl = "function(" },
  }

  for _, v in ipairs(patterns) do str = str:gsub(v.patt, v.repl) end
  return str
end

References

The Drunken Bishop

2021-03-16T00:00:00+00:00

Have you ever seen something similar to this??

+----[RSA 2048]---+
|        . o.+o  .|
|     . + *  +o...|
|      + * .. ... |
|       o + .   . |
|        S o   .  |
|         o     . |
|          .     o|
|               .o|
|               Eo|
+------[MD5]------+

Usually when we create a new SSH key there is an image printed at the last step and we mostly ignore it, this time I wanted to understand what this image meant.

The image is a visualization of the fingerprint created with the Drunken Bishop Algorithm

What is this image?

The Drunken Bishop Algorithm is an algorithm used by OpenSSH for visualizing the fingerprints of SSH keys.

Why do we need a Fingerprint Visualization Algorithm?

❯ ssh [email protected]
Do you want to connect to authentic host with fingeprint fc94b0c1e5b0987c5533997697ee9fb7
+-----------------+
|        . o.+o  .|
|     . + *  +o...|
|      + * .. ... |
|       o + .   . |
|        S o   .  |
|         o     . |
|          .     o|
|               .o|
|               Eo|
+-----------------+

❯ ssh user@my.__host
Do you want to connect to malicious host with fingeprint fc94b0c1e5b0987c5843997697ee9fb7
+-----------------+
|       .=o.  .   |
|     . *+*. o    |
|      =.*..o     |
|       o + ..    |
|        S o.     |
|         o  .    |
|          .  . . |
|              o .|
|               E.|
+-----------------+

In the above example it would be nearly impossible to find out that the host is different just by looking at the hashes fc94b0c1e5b0987c5533997697ee9fb7 and fc94b0c1e5b0987c5843997697ee9fb7. The visual aid can make a big difference in this scenario

How does it work?

Consider a bishop in the center of a 17x9 board

0   1   2   3   4   5   6 ... 14  15  16
17  18  19  20  21 .......    31  32  33
...
...               76
...
...
136 137 ....                  150 151 152

Each bit of the fingerprint represents the path traversed by the bishop.

Fingerprint        fc         :     94      : ... : b7
Bits        11   11   11   00 : 10 01 01 00 : ... : 10  11  01  11
            |    |    |    |    |  |  |  |          |   |   |   |
Step        4    3    2    1    8  7  6  5    ...   64  63  62  61

For each bit we move in the following direction

- Upper left
- Upper right
- Lower right
- Lower left

When the bishop tries to move outside the grid it stays in the same square it started from

When the bishop visits a grid we increment a count on how many times it has visited that grid

To visualize the grid we use the following characters. S and E are special characters we use to mark the start and end of the bishop’s journey

Value	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
Character		.	o	+	=	*	B	O	X	@	%	&	#	/	^	S	E

Is is this good to rely on visualization?

The next question we may have is that, can there be two hashes with similar visuals. The answer is yes, but finding one would be similar to finding a hash collision

These visualization are surprisingly resistant to collision

How to Enable this in SSH Config

You can make it be printed when doing SSH related operations by setting

// ~/.ssh/config
VisualHostKey true

References and Further links

Retro video games used similar approach of relying on visuals for checkpoints. An example of how this was used in Castlevania 3

Castlevania 3 Password Algorithm

Reverse Engineering Code Art - Part 6 - Noisy Donuts

2021-03-09T00:00:00+00:00

Based on this dwitter by Pascal

c.width=w=70;for(i=2800;i--;)x.fillRect(X=i%w,Y=(i-X)/w,0<T(T(S(X/7+5*C(t))+S(Y/7-5*t))),1)

Format & Analysis

The code is simple, there is a lot of math to explain

const u = (t) => {
  c.width = w = 70;
  for (i = 2800; i--; ) {
    const X = i % w;
    const Y = (i - X) / w;
    const angle = S(X / 7 + 5 * t) + S(Y / 7 - 5 * t) 
    const val = T(T(angle));
    x.fillRect(X, Y, 0 < val, 1);
  }
};

I’m making a small change to make things easier to understand, isolating the moving effect to one direction by changing S(X / 7 + 5 * C(t)) to S(X / 7 + 5 * t)

Our main focus is around two functions

sin(x) + sin(x)

This is rather a simple function

We can visualizing it by

x.fillRect(X, Y, S(X / 7 + 5 * t) + S(Y / 7 - 5 * t), 1)

It’s seeing the 2D graph in a 1D pixel space

Another example dwitter, but with sin(x) + cos(x)

tan(tan(x))

The function y = tan(tan(x)) has multiple bands.

In each band these are infinitely transitions from Inf to -Inf, this causes the noise effect that we see

The below is the function from x -1.6 to -1.5

By adding the factor of time the noise circles behave chaotically

Mark1626 Home Page

Deep Dive: On Parallel graph algorithms

Parallel BFS

Graph algorithms in the GPU

References

Deep Dive - The Cost of Integer Division

Approaches

x86 unsigned integer division

Granlund-Montgomery algorithm

libdivide

LKK algorithm

Performance

References

Auto Vectorization Case Study 2

Restructuring for auto vectorization

OpenMP

Encapsulating the function as a class

Perf Scripts

Dump events from perf.data

Scripting

Case - 1: Identifying a hotspot in the code with source line number and symbol

Case - 2: Spitting perf.data of a MPI run into separate files for individual processes

Auto Vectorization Case Study 1

The Challenge

Runtime Check of Pointers

__restrict__

OpenMP

Exploring Obsured Code: Part -1

Round 1

Round 2

Round 3

Round 4

Node Module System

Understanding the Module system

Module System

Main Module

Module._load

Module._extensions

Native module loaders in NodeJS

Module._compile

Creating our own module extensions

Case 1: Module loader for YAML

How does this work?

Case 2: Extending JS file syntax

How it works

Reference

See Also

Lua Module Loader

Module Loader

Transformation Use Case

References

See Also

The Drunken Bishop

What is this image?

Why do we need a Fingerprint Visualization Algorithm?

How does it work?

Is is this good to rely on visualization?

How to Enable this in SSH Config

References and Further links

Reverse Engineering Code Art - Part 6 - Noisy Donuts

Format & Analysis

sin(x) + sin(x)

tan(tan(x))

restrict