Post-Quantum Cryptography Benchmark Suite
Multithreaded benchmarking harness for post-quantum signature schemes with configurable concurrency, message sizes, and cache-eviction support to simulate more realistic signing and verification workloads.
Problem
Evaluating real-world performance of post-quantum signature schemes requires controlled benchmarking that accounts for threading behavior, message size variation, and cache effects. Existing examples focus on correctness rather than reproducible performance measurement.
Constraints
- Benchmarks built on top of liboqs implementations.
- Support for both stateful (XMSS) and stateless (SPHINCS+) schemes.
- Ability to simulate realistic workloads including cache eviction.
- Designed as a lightweight harness without modifying liboqs itself.
Architecture
- C-based benchmarking harness that invokes liboqs signing and verification primitives.
- Thread pool workers execute signing and verification tasks concurrently.
- Configurable parameters for thread count, cache flush interval, and message size.
- Separate benchmark executables for XMSS and SPHINCS+ test cycles.
- Build system integrates with an existing liboqs source tree.
Key decisions
Thread Pool for Parallel Benchmarking
Used a lightweight C thread pool to simulate concurrent signing workloads and measure scaling behavior across multiple worker threads.
Cache Eviction Support
Added optional cache flushing between operations to avoid unrealistic warm-cache performance measurements.
Algorithm-Specific Test Cycles
Separated XMSS and SPHINCS+ benchmark flows to reflect differences between stateful and stateless signature schemes.
Risks and mitigations
Benchmark results may vary across hardware platforms
Benchmarks expose configurable parameters and encourage reporting of hardware context for reproducibility.
Stateful signature schemes like XMSS require careful key usage during repeated tests
Benchmark cycles regenerate keys as part of each full run to avoid state reuse errors.
Next steps
- Add support for additional post-quantum signature schemes as they are standardized
- Add scripting for repeatable cross-platform experiment runs and result collection