Skip to content

hayabusa-cloud/iobuf

Repository files navigation

iobuf

Go Reference Go Report Card codecov License: MIT

Lock-free, memory-friendly bounded buffer pools for Go optimized for low-latency systems.

English | 简体中文 | Español | 日本語 | Français

Three-Tier Progress Model

iobuf utilizes the Spin and Adapt layers of our performance ecosystem:

  1. Strike: System call → Direct kernel hit.
  2. Spin: Hardware yield (spin) → Local atomic synchronization.
  3. Adapt: Software backoff (iox.Backoff) → External I/O readiness.

Features

  • Bounded lock-free buffer pools for low-latency systems.
  • Page-aligned memory allocation for DMA and io_uring compatibility.
  • Zero-copy IoVec generation for vectored I/O syscalls.
  • Cooperative back-off: Uses iox.Backoff to handle resource exhaustion gracefully.

Requirements

  • Go 1.26+
  • 64-bit CPU (amd64, arm64, riscv64, loong64, ppc64, s390x, mips64, etc.)

Note: 32-bit architectures are not supported due to 64-bit atomic operations in the lock-free pool implementation.

Installation

go get code.hybscloud.com/iobuf

Quick Start

Buffer Pools

// Create a pool of 1024 small buffers (2 KiB each)
pool := iobuf.NewSmallBufferPool(1024)
pool.Fill(iobuf.NewSmallBuffer)

// Acquire a buffer index
idx, err := pool.Get()
if err != nil {
    panic(err)
}

// Access the buffer directly (zero-copy)
buf := pool.Value(idx)
...

// Return to pool
pool.Put(idx)

Page-Aligned Memory

// Single page-aligned block (default page size)
block := iobuf.AlignedMemBlock()

// Custom size with explicit alignment
mem := iobuf.AlignedMem(65536, iobuf.PageSize)

// Multiple aligned blocks
blocks := iobuf.AlignedMemBlocks(16, iobuf.PageSize)

IoVec for Vectored I/O

// Convert tiered buffers to iovec for readv/writev
buffers := make([]iobuf.SmallBuffer, 8)
iovecs := iobuf.IoVecFrom(buffers)

// Get raw pointer and count for syscalls
addr, n := iobuf.IoVecAddrLen(iovecs)

Buffer Tiers

Power-of-4 progression starting at 32 bytes (12 tiers, 32 B to 128 MiB):

Tier Size Use Case
Pico 32 B UUIDs, flags, tiny control messages
Nano 128 B HTTP headers, JSON tokens, small RPC payloads
Micro 512 B DNS packets, MQTT messages, protocol frames
Small 2 KiB WebSocket frames, small HTTP responses
Medium 8 KiB TCP segments, gRPC messages, page I/O
Big 32 KiB TLS records (16 KiB max), stream chunks
Large 128 KiB io_uring buffer rings, bulk network transfers
Great 512 KiB Database pages, large API responses
Huge 2 MiB Huge page aligned, memory-mapped files
Vast 8 MiB Image processing, compressed archives
Giant 32 MiB Video frames, ML model weights
Titan 128 MiB Large datasets, maximum stack-safe buffer

API Overview

Pool Interfaces

// Generic pool interface
type Pool[T any] interface {
    Put(item T) error
    Get() (item T, err error)
}

// Index-based pool for zero-copy buffer access
type IndirectPool[T BufferType] interface {
    Pool[int]
    Value(indirect int) T
    SetValue(indirect int, item T)
}

Pool Constructors

func NewPicoBufferPool(capacity int) *PicoBufferBoundedPool
func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool

Memory Allocation

// Page-aligned memory
func AlignedMem(size int, pageSize uintptr) []byte
func AlignedMemBlocks(n int, pageSize uintptr) [][]byte
func AlignedMemBlock() []byte

// Cache-line-aligned memory (prevents false sharing)
func CacheLineAlignedMem(size int) []byte
func CacheLineAlignedMemBlocks(n int, blockSize int) [][]byte
const CacheLineSize  // 64 or 128 depending on architecture

IoVec Generation

func IoVecFrom[T BufferType](buffers []T) []IoVec
func IoVecFromBytesSlice(iov [][]byte) (addr uintptr, n int)
func IoVecFromRegisteredBuffers(buffers []RegisterBuffer) []IoVec
func IoVecAddrLen(vec []IoVec) (addr uintptr, n int)

Design

The bounded pool implementation is based on lock-free queue algorithms:

  • Memory-efficient: O(n) space for n-capacity pool
  • Lock-free progress: Guaranteed global progress bounds
  • Cache-friendly: Minimizes false sharing and cache-line bouncing

References

License

MIT License - see LICENSE for details.

© 2025 Hayabusa Cloud Co., Ltd.

About

Lock-free bounded buffer pools for I/O in Go

Topics

Resources

License

Stars

Watchers

Forks

Sponsor this project

 

Packages

 
 
 

Contributors