Skip to content

Smaller MAX_PACKET_SIZE and better batch cache locality#816

Merged
zakird merged 1 commit intozmap:mainfrom
droe:droe/batch-perf
Mar 10, 2024
Merged

Smaller MAX_PACKET_SIZE and better batch cache locality#816
zakird merged 1 commit intozmap:mainfrom
droe:droe/batch-perf

Conversation

@droe
Copy link
Contributor

@droe droe commented Mar 9, 2024

This is a very arguable change, feel free to reject. I'm seeing a consistent ~ 0.7 % send rate improvement with lower-end hardware at 10 GbE close to but not quite at max send rate.

  • Change how batch is laid out in memory: Since all access is one packet at a time and never all IPs or all lengths, lay out all fields pertaining to the same packet adjacent in memory, for sequential access (single array of packets with metadata instead of separate arrays for packet data, lengths and ips)
  • Reduce MAX_PACKET_SIZE from 4k to 2k minus metadata, which is still way above the standard MTU of 1500 that we probably don't want to exceed in order to avoid IP fragmentation

Extra benefits:

  • Arguably cleaner code for accessing the packets in batches (no manual offset calculations)
  • Single definition of MAX_PACKET_SIZE instead of the previous two definitions of MAX_PACKET_SIZE

Tested on the usual macOS Sonoma, FreeBSD 14 and Ubuntu 23.10 with and without netmap.

@zakird
Copy link
Member

zakird commented Mar 10, 2024

Thanks. Performance increase is slim, but it's a cleaner design for sure, so happy to merge.

@zakird zakird merged commit 7dce05c into zmap:main Mar 10, 2024
@droe droe deleted the droe/batch-perf branch March 11, 2024 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants