49f1827633
## Objective Reduce the time spent on ECS benchmarks without significantly compromising coverage. ## Background A `cargo bench -p benches --bench ecs` takes about 45 minutes. I'm guessing this bench is mainly used to check for regressions after ECS changes, and requiring 2x45 minute tests means that most people will skip benchmarking entirely. I noticed that some benches are repeated with sizes from long linear progressions (10, 20, ..., 100). This might be nice for detailed profiling, but seems too much for a overall regression check. ## Solution The PR follows the principles of "three or four different sizes is fine" and "powers of ten where it fits". The number of benches is reduced from 394 to 238 (-40%), and time from 46.2 minutes to 32.8 (-30%). While some coverage is lost, I think it's reasonable for anyone doing detailed profiling of a particular feature to temporarily add more benches. There's a couple of changes to avoid leading zeroes. I felt that `0010, 0100, 1000` is harder to read than `10, 100, 1000`. ## Is That Enough? 32 minutes is still too much. Possible future options: - Reduce measurement and warmup times. I suspect the current times (mostly 4-5 seconds total) are too conservative, and 1 second would be fine for spotting significant regressions. - Split the bench into quick and detailed variants. ## Testing ``` cargo bench -p benches --bench ecs ```
57 lines
1.9 KiB
Rust
57 lines
1.9 KiB
Rust
mod iter;
|
|
mod send;
|
|
|
|
use criterion::{criterion_group, Criterion};
|
|
|
|
criterion_group!(benches, send, iter);
|
|
|
|
fn send(c: &mut Criterion) {
|
|
let mut group = c.benchmark_group("events_send");
|
|
group.warm_up_time(core::time::Duration::from_millis(500));
|
|
group.measurement_time(core::time::Duration::from_secs(4));
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_4_events_{}", count), |b| {
|
|
let mut bench = send::Benchmark::<4>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_16_events_{}", count), |b| {
|
|
let mut bench = send::Benchmark::<16>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_512_events_{}", count), |b| {
|
|
let mut bench = send::Benchmark::<512>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
group.finish();
|
|
}
|
|
|
|
fn iter(c: &mut Criterion) {
|
|
let mut group = c.benchmark_group("events_iter");
|
|
group.warm_up_time(core::time::Duration::from_millis(500));
|
|
group.measurement_time(core::time::Duration::from_secs(4));
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_4_events_{}", count), |b| {
|
|
let mut bench = iter::Benchmark::<4>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_16_events_{}", count), |b| {
|
|
let mut bench = iter::Benchmark::<4>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
for count in [100, 1_000, 10_000] {
|
|
group.bench_function(format!("size_512_events_{}", count), |b| {
|
|
let mut bench = iter::Benchmark::<512>::new(count);
|
|
b.iter(move || bench.run());
|
|
});
|
|
}
|
|
group.finish();
|
|
}
|