You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An Arcaflow plugin that wraps the
rusty-comms IPC
benchmark suite. It executes the ipc-benchmark binary, parses the
structured JSON output, and returns strongly-typed latency and
throughput results to the Arcaflow engine.
Supported IPC Mechanisms
Mechanism
CLI Name
Description
Unix Domain Socket
uds
Local inter-process socket
TCP Socket
tcp
TCP loopback socket
Shared Memory
shm
POSIX shared memory (with optional direct mode)
POSIX Message Queue
pmq
Kernel-managed message queue
Use all to benchmark every mechanism in a single run.
Building the Container Image
The plugin ships as a multi-stage container image that compiles
ipc-benchmark from source (Rust) and bundles it with the Python
plugin code.
Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below
Run IPC Benchmark (run-benchmark)
Executes one or more rusty-comms IPC benchmark test runs and returns structured latency and throughput results.
Input
Type:
scope
Root object:
InputParams
Properties
tests (list[reference[TestRunConfig]])
Name:
Test Runs
Description:
List of benchmark test configurations. Each entry specifies mechanisms and parameters for one run.
Required:
Yes
Type:
list[reference[TestRunConfig]]
Minimum items:
1
List items
Type:
reference[TestRunConfig]
Referenced object:
TestRunConfig
ObjectsInputParams (object)
Type:
object
Properties
tests (list[reference[TestRunConfig]])
Name:
Test Runs
Description:
List of benchmark test configurations. Each entry specifies mechanisms and parameters for one run.
Required:
Yes
Type:
list[reference[TestRunConfig]]
Minimum items:
1
List items
Type:
reference[TestRunConfig]
Referenced object:
TestRunConfig
TestRunConfig (object)
Type:
object
Properties
blocking (bool)
Name:
Blocking Mode
Description:
Use blocking I/O. When not set, the binary's own default applies. Set to true for blocking mode or false for async Tokio mode.
Required:
No
Type:
bool
buffer_size (int)
Name:
Buffer Size
Description:
Internal buffer size in bytes for message queues and shared memory.
Required:
No
Type:
int
Minimum:
1
client_affinity (int)
Name:
Client CPU Affinity
Description:
Pin the client (sender) process to this CPU core.
Required:
No
Type:
int
Minimum:
0
concurrency (int)
Name:
Concurrency
Description:
Number of concurrent workers/threads.
Required:
No
Type:
int
Minimum:
1
continue_on_error (bool)
Name:
Continue on Error
Description:
Continue running remaining mechanisms if one fails.
Required:
No
Type:
bool
duration (string)
Name:
Duration
Description:
Fixed test duration, e.g. '30s', '5m', '1h'. Takes precedence over message count.
Required:
No
Type:
string
extra_args (list[string])
Name:
Extra Arguments
Description:
Additional CLI arguments passed directly to the ipc-benchmark binary for advanced usage.
Required:
No
Type:
list[string]
List items
Type:
string
host (string)
Name:
TCP Host
Description:
Host address for TCP socket tests. Default: 127.0.0.1.
Required:
No
Type:
string
include_first_message (bool)
Name:
Include First Message
Description:
Include the first (canary) message in results instead of discarding it.
Required:
No
Type:
bool
iterations (int)
Name:
Iterations
Description:
Number of times to repeat this test configuration. Results from all iterations are collected and statistical aggregates (mean, stddev, min, max) are computed per mechanism. Set to a higher value (e.g. 5) for statistically significant results.
Required:
No
Default (JSON encoded):
1
Type:
int
Minimum:
1
mechanisms (list[enum[string]])
Name:
IPC Mechanisms
Description:
IPC mechanisms to benchmark. Use 'all' to test every available mechanism. Valid values: uds, shm, tcp, pmq, all.
Required:
Yes
Type:
list[enum[string]]
Minimum items:
1
List items
Type:
enum[string]
Values
all: all
pmq: pmq
shm: shm
tcp: tcp
uds: uds
message_size (int)
Name:
Message Size
Description:
Message payload size in bytes.
Required:
No
Type:
int
Minimum:
1
msg_count (int)
Name:
Message Count
Description:
Number of messages to send. Ignored when duration is set.
Required:
No
Type:
int
Minimum:
1
one_way (bool)
Name:
One-Way Test
Description:
Run one-way latency tests. If neither one_way nor round_trip is set, both run by default.
Required:
No
Type:
bool
percentiles (list[float])
Name:
Percentiles
Description:
Percentile values to compute, e.g. [50.0, 95.0, 99.0, 99.9].
Required:
No
Type:
list[float]
List items
Type:
float
pmq_priority (int)
Name:
PMQ Priority
Description:
Message priority for POSIX Message Queues.
Required:
No
Type:
int
Minimum:
0
port (int)
Name:
TCP Port
Description:
Port for TCP socket tests. Default: 8080.
Required:
No
Type:
int
Minimum:
1
quiet (bool)
Name:
Quiet Mode
Description:
Silence informational console output.
Required:
No
Type:
bool
round_trip (bool)
Name:
Round-Trip Test
Description:
Run round-trip latency tests. If neither one_way nor round_trip is set, both run by default.
Required:
No
Type:
bool
send_delay (string)
Name:
Send Delay
Description:
Delay between sends, e.g. '10ms', '50us'. Useful for latency-focused testing under controlled load.
Required:
No
Type:
string
server_affinity (int)
Name:
Server CPU Affinity
Description:
Pin the server (receiver) process to this CPU core.
Required:
No
Type:
int
Minimum:
0
shm_direct (bool)
Name:
SHM Direct Memory
Description:
Use high-performance direct memory shared memory. Auto-enables blocking mode. Unix only, 8KB max payload.
Required:
No
Type:
bool
timeout (int)
Name:
Timeout
Description:
Maximum seconds to wait for the benchmark to complete before killing the process. Increase for large benchmark matrices or long-duration tests.
Required:
No
Default (JSON encoded):
3600
Type:
int
Minimum:
1
warmup_iterations (int)
Name:
Warmup Iterations
Description:
Number of warmup messages before measurement starts.
Per-mechanism statistical aggregates computed across iterations. Populated when tests are run with iterations > 0. Contains mean, stddev, min, and max for throughput and latency metrics.
Required:
No
Type:
reference[IterationAggregates]
Referenced object:
IterationAggregates
metadata (reference[BenchmarkMetadata])
Name:
Metadata
Description:
Run metadata and system information.
Required:
Yes
Type:
reference[BenchmarkMetadata]
Referenced object:
BenchmarkMetadata
results (list[reference[BenchmarkResult]])
Name:
Results
Description:
Per-mechanism benchmark results.
Required:
Yes
Type:
list[reference[BenchmarkResult]]
List items
Type:
reference[BenchmarkResult]
Referenced object:
BenchmarkResult
summary (reference[OverallSummary])
Name:
Overall Summary
Description:
Aggregated summary across mechanisms.
Required:
Yes
Type:
reference[OverallSummary]
Referenced object:
OverallSummary
ObjectsBenchmarkMetadata (object)
Type:
object
Properties
system_info (reference[SystemInfo])
Name:
System Info
Description:
System information.
Required:
Yes
Type:
reference[SystemInfo]
Referenced object:
SystemInfo
timestamp (string)
Name:
Timestamp
Description:
ISO-8601 timestamp when run started.
Required:
Yes
Type:
string
total_tests (int)
Name:
Total Tests
Description:
Number of mechanism tests executed.
Required:
Yes
Type:
int
version (string)
Name:
Version
Description:
rusty-comms version.
Required:
Yes
Type:
string
BenchmarkResult (object)
Type:
object
Properties
failure_reason (string)
Name:
Failure Reason
Description:
Error description when status is 'Failure'. None when the test succeeded.
Required:
No
Type:
string
input_blocking (bool)
Name:
Input: Blocking
Description:
Original blocking flag from the test input. True = blocking, False = async, None = default.
Required:
No
Type:
bool
input_concurrency (int)
Name:
Input: Concurrency
Description:
Original concurrency value from the test input.
Required:
No
Type:
int
input_one_way (bool)
Name:
Input: One-Way
Description:
Original one_way flag from the test input.
Required:
No
Type:
bool
input_round_trip (bool)
Name:
Input: Round-Trip
Description:
Original round_trip flag from the test input.
Required:
No
Type:
bool
input_send_delay (string)
Name:
Input: Send Delay
Description:
Original send_delay value from the test input.
Required:
No
Type:
string
input_shm_direct (bool)
Name:
Input: SHM Direct
Description:
Original shm_direct flag from the test input.
Required:
No
Type:
bool
mechanism (string)
Name:
Mechanism
Description:
IPC mechanism name, e.g. UnixDomainSocket, SharedMemory, TcpSocket, PosixMessageQueue.
Per-test-configuration aggregate statistics. Each entry represents a unique combination of mechanism, message size, and direction.
Required:
Yes
Type:
list[reference[TestIterationAggregate]]
List items
Type:
reference[TestIterationAggregate]
Referenced object:
TestIterationAggregate
LatencyMetrics (object)
Type:
object
Properties
histogram_data (list[int])
Name:
Histogram Data
Description:
Raw HDR histogram quantile samples for advanced analysis and visualization. Each entry is a latency value in nanoseconds from the quantile iteration of the underlying HdrHistogram.
Required:
Yes
Type:
list[int]
List items
Type:
int
latency_type (string)
Name:
Latency Type
Description:
'OneWay' or 'RoundTrip'.
Required:
Yes
Type:
string
max_ns (int)
Name:
Max (ns)
Description:
Maximum observed latency in nanoseconds.
Required:
Yes
Type:
int
mean_ns (float)
Name:
Mean (ns)
Description:
Mean latency in nanoseconds.
Required:
Yes
Type:
float
median_ns (float)
Name:
Median (ns)
Description:
Median (P50) latency in nanoseconds.
Required:
Yes
Type:
float
min_ns (int)
Name:
Min (ns)
Description:
Minimum observed latency in nanoseconds.
Required:
Yes
Type:
int
percentiles (list[reference[PercentileValue]])
Name:
Percentiles
Description:
Requested percentile values.
Required:
Yes
Type:
list[reference[PercentileValue]]
List items
Type:
reference[PercentileValue]
Referenced object:
PercentileValue
std_dev_ns (float)
Name:
Std Dev (ns)
Description:
Standard deviation of latency in nanoseconds.
Required:
Yes
Type:
float
total_samples (int)
Name:
Total Samples
Description:
Number of latency samples collected.
Required:
Yes
Type:
int
MechanismSummary (object)
Type:
object
Properties
average_throughput_mbps (float)
Name:
Average Throughput (MB/s)
Description:
Average throughput for this mechanism.
Required:
Yes
Type:
float
mechanism (string)
Name:
Mechanism
Description:
IPC mechanism enum variant name.
Required:
Yes
Type:
string
p95_latency_ns (int)
Name:
P95 Latency (ns)
Description:
P95 latency in nanoseconds.
Required:
No
Type:
int
p99_latency_ns (int)
Name:
P99 Latency (ns)
Description:
P99 latency in nanoseconds.
Required:
No
Type:
int
total_messages (int)
Name:
Total Messages
Description:
Total messages for this mechanism.
Required:
Yes
Type:
int
MetricStatistics (object)
Type:
object
Properties
max_value (float)
Name:
Maximum
Description:
Maximum observed value.
Required:
Yes
Type:
float
mean (float)
Name:
Mean
Description:
Arithmetic mean across iterations.
Required:
Yes
Type:
float
min_value (float)
Name:
Minimum
Description:
Minimum observed value.
Required:
Yes
Type:
float
sample_count (int)
Name:
Sample Count
Description:
Number of iterations contributing to this statistic.
Required:
Yes
Type:
int
Minimum:
1
stddev (float)
Name:
Standard Deviation
Description:
Sample standard deviation across iterations. Zero when only one iteration was run.
Per-mechanism statistical aggregates computed across iterations. Populated when tests are run with iterations > 0. Contains mean, stddev, min, and max for throughput and latency metrics.
Required:
No
Type:
reference[IterationAggregates]
Referenced object:
IterationAggregates
metadata (reference[BenchmarkMetadata])
Name:
Metadata
Description:
Run metadata and system information.
Required:
Yes
Type:
reference[BenchmarkMetadata]
Referenced object:
BenchmarkMetadata
results (list[reference[BenchmarkResult]])
Name:
Results
Description:
Per-mechanism benchmark results.
Required:
Yes
Type:
list[reference[BenchmarkResult]]
List items
Type:
reference[BenchmarkResult]
Referenced object:
BenchmarkResult
summary (reference[OverallSummary])
Name:
Overall Summary
Description:
Aggregated summary across mechanisms.
Required:
Yes
Type:
reference[OverallSummary]
Referenced object:
OverallSummary
SystemInfo (object)
Type:
object
Properties
architecture (string)
Name:
Architecture
Description:
CPU architecture, e.g. x86_64.
Required:
Yes
Type:
string
benchmark_version (string)
Name:
Benchmark Version
Description:
rusty-comms version string.
Required:
Yes
Type:
string
cpu_cores (int)
Name:
CPU Cores
Description:
Number of CPU cores.
Required:
Yes
Type:
int
memory_gb (float)
Name:
Memory (GB)
Description:
System memory in gigabytes.
Required:
Yes
Type:
float
os (string)
Name:
OS
Description:
Operating system name.
Required:
Yes
Type:
string
rust_version (string)
Name:
Rust Version
Description:
Rust compiler version used to build.
Required:
Yes
Type:
string
TestConfiguration (object)
Type:
object
Properties
buffer_size (int)
Name:
Buffer Size
Description:
Internal buffer size in bytes.
Required:
Yes
Type:
int
concurrency (int)
Name:
Concurrency
Description:
Number of concurrent workers.
Required:
Yes
Type:
int
duration (map[string,int])
Name:
Duration
Description:
Configured duration as {secs, nanos} (None if count).
Required:
No
Type:
map[string,int]
Key type
Type:
string
Value type
Type:
int
message_size (int)
Name:
Message Size
Description:
Payload size in bytes.
Required:
Yes
Type:
int
msg_count (int)
Name:
Message Count
Description:
Configured message count (None if duration).
Required:
No
Type:
int
one_way_enabled (bool)
Name:
One-Way Enabled
Description:
Whether one-way tests were enabled.
Required:
Yes
Type:
bool
percentiles (list[float])
Name:
Percentiles
Description:
Percentile levels computed.
Required:
Yes
Type:
list[float]
List items
Type:
float
round_trip_enabled (bool)
Name:
Round-Trip Enabled
Description:
Whether round-trip tests were enabled.
Required:
Yes
Type:
bool
warmup_iterations (int)
Name:
Warmup Iterations
Description:
Warmup messages before measurement.
Required:
Yes
Type:
int
TestIterationAggregate (object)
Type:
object
Properties
direction (string)
Name:
Direction
Description:
Test direction: 'one_way' or 'round_trip'.
Required:
Yes
Type:
string
iterations_completed (int)
Name:
Iterations Completed
Description:
Number of iterations that produced results for this test configuration.
Required:
Yes
Type:
int
Minimum:
1
mean_latency_ns (reference[MetricStatistics])
Name:
Mean Latency (ns)
Description:
Statistical summary of mean latency across iterations. None when no latency data was collected.
Required:
No
Type:
reference[MetricStatistics]
Referenced object:
MetricStatistics
mechanism (string)
Name:
Mechanism
Description:
IPC mechanism enum variant name.
Required:
Yes
Type:
string
message_size (int)
Name:
Message Size
Description:
Payload size in bytes for this test.
Required:
Yes
Type:
int
p95_latency_ns (reference[MetricStatistics])
Name:
P95 Latency (ns)
Description:
Statistical summary of P95 latency across iterations.
Required:
No
Type:
reference[MetricStatistics]
Referenced object:
MetricStatistics
p99_latency_ns (reference[MetricStatistics])
Name:
P99 Latency (ns)
Description:
Statistical summary of P99 latency across iterations.
Required:
No
Type:
reference[MetricStatistics]
Referenced object:
MetricStatistics
throughput_mbps (reference[MetricStatistics])
Name:
Throughput (MB/s)
Description:
Statistical summary of average throughput across iterations.
Required:
Yes
Type:
reference[MetricStatistics]
Referenced object:
MetricStatistics
ThroughputMetrics (object)
Type:
object
Properties
bytes_per_second (float)
Name:
Bytes/sec
Description:
Data transmission rate in bytes per second.
Required:
Yes
Type:
float
duration_ns (int)
Name:
Duration (ns)
Description:
Measurement duration in nanoseconds.
Required:
Yes
Type:
int
messages_per_second (float)
Name:
Messages/sec
Description:
Message transmission rate.
Required:
Yes
Type:
float
total_bytes (int)
Name:
Total Bytes
Description:
Total bytes transmitted.
Required:
Yes
Type:
int
total_messages (int)
Name:
Total Messages
Description:
Total messages transmitted.
Required:
Yes
Type:
int
Utilities
The utils/ directory contains helper scripts for multi-run
benchmarking and result analysis:
utils/run_benchmarks.sh — Builds the container and runs the
comprehensive suite N times (default 5), producing an averaged CSV.
utils/python/run_comprehensive.py — Python runner that
executes multiple Arcaflow iterations and averages results.
utils/python/parse_arcaflow_output.py — Parses Arcaflow engine
output YAML into a CSV summary for analysis.
Running the Full Benchmark Suite
# Run 5 iterations and produce averaged CSV
./utils/run_benchmarks.sh
# Run 3 iterations
./utils/run_benchmarks.sh 3
# Parse existing outputs only (skip running)
./utils/run_benchmarks.sh --skip
Results are written to utils/out/comprehensive_averaged.csv.