Micro-benchmarks for individual operations in a WaveNet neural network inference engine, comparing hand-optimized inline implementations against Eigen linear algebra expressions.

The primary motivation is optimizing WaveNet inference for real-time audio on ARM Cortex-M7 microcontrollers (specifically the Electrosmith Daisy platform with STM32H750), where Eigen's GEMM path triggers malloc in hot loops and expression template overhead is measurable for tiny matrices. The desktop benchmarks reveal which optimizations are universally beneficial vs. architecture-specific.

• C++17 compiler (GCC 7+, Clang 5+, or Apple Clang)
• Eigen (header-only, downloaded automatically or manually)

• DaisyToolchain (provides arm-none-eabi-gcc + newlib)
  • macOS: install to /Library/DaisyToolchain/0.2.0/ (default) or set DAISY_TOOLCHAIN
  • Linux: install and add to PATH, or set DAISY_TOOLCHAIN
• libDaisy built from source
• Eigen headers (same as desktop)
• Electrosmith Daisy Seed hardware + USB cable

git clone https://github.com/jfsantos/nam-inference-benchmarks.git
cd nam-inference-benchmarks

# Download Eigen (header-only, ~5 MB)
mkdir -p third_party
git clone --depth 1 --branch 3.4.0 https://gitlab.com/libeigen/eigen.git third_party/eigen

# Build all benchmarks
make

# Run all benchmarks sequentially
make run

# Or run individual benchmarks
./bench_gemm
./bench_hardswish
./bench_lut_activation

# Build libDaisy first (if not already done)
cd /path/to/libDaisy && make
cd -

# Cross-compile, passing the path to libDaisy
make -f Makefile.daisy LIBDAISY_DIR=/path/to/libDaisy

# Flash via DFU (hold BOOT button, press RESET, then release BOOT)
make -f Makefile.daisy LIBDAISY_DIR=/path/to/libDaisy program-dfu

# Connect to USB serial to see results
screen /dev/ttyACM0 115200        # Linux
screen /dev/tty.usbmodem* 115200   # macOS

The Daisy firmware runs all benchmarks sequentially on boot and outputs results over USB serial. The LED blinks when complete.

.
├── common.h              # Desktop benchmark utilities (chrono timing)
├── common_daisy.h        # Daisy benchmark utilities (DWT cycle counter)
├── bench_gemm.cpp        # Desktop: GEMM benchmarks
├── bench_restrict.cpp    # Desktop: __restrict__ effect
├── bench_memcpy_vs_eigen.cpp
├── bench_elementwise.cpp
├── bench_bias_broadcast.cpp
├── bench_hardswish.cpp
├── bench_activation_unroll.cpp
├── bench_lut_activation.cpp
├── bench_strided_copy.cpp
├── bench_depthwise.cpp
├── bench_film.cpp
├── bench_ringbuffer.cpp
├── bench_daisy.cpp       # Daisy: all benchmarks in one firmware binary
├── Makefile              # Desktop build
├── Makefile.daisy        # ARM cross-compile build

Desktop benchmarks are separate executables (one per category) for flexibility. The Daisy benchmark is a single firmware binary that runs all tests sequentially, since flashing is slow and USB serial output is the only interface.

• Desktop: std::chrono::high_resolution_clock, 1000--10000 iterations after 100--200 warmup. Reports mean, stddev, min, max in nanoseconds.
• Daisy: DWT CYCCNT register (cycle-accurate, zero overhead), 1000 iterations after 100 warmup. Reports cycles and microseconds at the configured CPU frequency. FTZ (Flush-to-Zero) and DN (Default-NaN) are enabled to avoid subnormal float slowdowns.

These benchmarks were developed as part of optimizing NeuralAmpModelerCore for real-time guitar amp simulation on the Electrosmith Daisy platform. The WaveNet architecture uses small matrix multiplications (4x8, 8x8) thousands of times per audio buffer, making per-operation overhead critical.

MIT