
# Objective - Our benchmarks and `compile_fail` tests lag behind the rest of the engine because they are not in the Cargo workspace, so not checked by CI. - Fixes #16801, please see it for further context! ## Solution - Add benchmarks and `compile_fail` tests to the Cargo workspace. - Fix any leftover formatting issues and documentation. ## Testing - I think CI should catch most things! ## Questions <details> <summary>Outdated issue I was having with function reflection being optional</summary> The `reflection_types` example is failing in Rust-Analyzer for me, but not a normal check. ```rust error[E0004]: non-exhaustive patterns: `ReflectRef::Function(_)` not covered --> examples/reflection/reflection_types.rs:81:11 | 81 | match value.reflect_ref() { | ^^^^^^^^^^^^^^^^^^^ pattern `ReflectRef::Function(_)` not covered | note: `ReflectRef<'_>` defined here --> /Users/bdeep/dev/bevy/bevy/crates/bevy_reflect/src/kind.rs:178:1 | 178 | pub enum ReflectRef<'a> { | ^^^^^^^^^^^^^^^^^^^^^^^ ... 188 | Function(&'a dyn Function), | -------- not covered = note: the matched value is of type `ReflectRef<'_>` help: ensure that all possible cases are being handled by adding a match arm with a wildcard pattern or an explicit pattern as shown | 126 ~ ReflectRef::Opaque(_) => {}, 127 + ReflectRef::Function(_) => todo!() | ``` I think it is because the following line is feature-gated:cc0f6a8db4/examples/reflection/reflection_types.rs (L117-L122)
My theory for why this is happening is because the benchmarks enabled `bevy_reflect`'s `function` feature, which gets merged with the rest of the features when RA checks the workspace, but the `#[cfg(...)]` gate in the example isn't detecting it:cc0f6a8db4/benches/Cargo.toml (L19)
Any thoughts on how to fix this? It's not blocking, since the example still compiles as normal, but it's just RA and the command `cargo check --workspace --all-targets` appears to fail. </summary>
134 lines
4.2 KiB
Rust
134 lines
4.2 KiB
Rust
use std::{
|
|
env,
|
|
path::{Path, PathBuf},
|
|
};
|
|
|
|
// Re-export ui_test so all the tests use the same version.
|
|
pub use ui_test;
|
|
|
|
use ui_test::{
|
|
color_eyre::eyre::eyre,
|
|
default_file_filter, default_per_file_config,
|
|
dependencies::DependencyBuilder,
|
|
run_tests_generic,
|
|
spanned::Spanned,
|
|
status_emitter::{Gha, StatusEmitter, Text},
|
|
Args, Config, OutputConflictHandling,
|
|
};
|
|
|
|
/// Use this instead of hand rolling configs.
|
|
///
|
|
/// `root_dir` is the directory your tests are contained in. Needs to be a path from crate root.
|
|
/// This config will build dependencies and will assume that the cargo manifest is placed at the
|
|
/// current working directory.
|
|
fn basic_config(root_dir: impl Into<PathBuf>, args: &Args) -> ui_test::Result<Config> {
|
|
let root_dir = root_dir.into();
|
|
|
|
match root_dir.try_exists() {
|
|
Ok(true) => { /* success */ }
|
|
Ok(false) => {
|
|
return Err(eyre!("path does not exist: {:?}", root_dir));
|
|
}
|
|
Err(error) => {
|
|
return Err(eyre!("failed to read path: {:?} ({:?})", root_dir, error));
|
|
}
|
|
}
|
|
|
|
let mut config = Config {
|
|
bless_command: Some(
|
|
"`cargo test` with the BLESS environment variable set to any non empty value"
|
|
.to_string(),
|
|
),
|
|
output_conflict_handling: if env::var_os("BLESS").is_some() {
|
|
OutputConflictHandling::Bless
|
|
} else {
|
|
// stderr output changes between rust versions so we just rely on annotations
|
|
OutputConflictHandling::Ignore
|
|
},
|
|
..Config::rustc(root_dir)
|
|
};
|
|
|
|
config.with_args(args);
|
|
|
|
let bevy_root = "..";
|
|
|
|
// Don't leak contributor filesystem paths
|
|
config.path_stderr_filter(Path::new(bevy_root), b"$BEVY_ROOT");
|
|
config.path_stderr_filter(Path::new(env!("RUSTUP_HOME")), b"$RUSTUP_HOME");
|
|
|
|
// ui_test doesn't compile regex with perl character classes.
|
|
// \pL = unicode class for letters, \pN = unicode class for numbers
|
|
config.stderr_filter(r"\/home\/[\pL\pN_@#\-\. ]+", "$HOME");
|
|
// Paths in .stderr seem to always be normalized to use /. Handle both anyway.
|
|
config.stderr_filter(
|
|
r"[a-zA-Z]:(?:\\|\/)users(?:\\|\/)[\pL\pN_@#\-\. ]+", // NOTE: [\pL\pN_@#\-\. ] is a poor attempt at handling usernames
|
|
"$HOME",
|
|
);
|
|
|
|
// Manually insert @aux-build:<dep> comments into test files. This needs to
|
|
// be done to build and link dependencies. Dependencies will be pulled from a
|
|
// Cargo.toml file.
|
|
config.comment_defaults.base().custom.insert(
|
|
"dependencies",
|
|
Spanned::dummy(vec![Box::new(DependencyBuilder::default())]),
|
|
);
|
|
|
|
Ok(config)
|
|
}
|
|
|
|
/// Runs ui tests for a single directory.
|
|
///
|
|
/// `root_dir` is the directory your tests are contained in. Needs to be a path from crate root.
|
|
pub fn test(test_name: impl Into<String>, test_root: impl Into<PathBuf>) -> ui_test::Result<()> {
|
|
test_multiple(test_name, [test_root])
|
|
}
|
|
|
|
/// Run ui tests with the given config
|
|
pub fn test_with_config(test_name: impl Into<String>, config: Config) -> ui_test::Result<()> {
|
|
test_with_multiple_configs(test_name, [Ok(config)])
|
|
}
|
|
|
|
/// Runs ui tests for a multiple directories.
|
|
///
|
|
/// `root_dirs` paths need to come from crate root.
|
|
pub fn test_multiple(
|
|
test_name: impl Into<String>,
|
|
test_roots: impl IntoIterator<Item = impl Into<PathBuf>>,
|
|
) -> ui_test::Result<()> {
|
|
let args = Args::test()?;
|
|
|
|
let configs = test_roots.into_iter().map(|root| basic_config(root, &args));
|
|
|
|
test_with_multiple_configs(test_name, configs)
|
|
}
|
|
|
|
/// Run ui test with the given configs.
|
|
///
|
|
/// Tests for configs are run in parallel.
|
|
pub fn test_with_multiple_configs(
|
|
test_name: impl Into<String>,
|
|
configs: impl IntoIterator<Item = ui_test::Result<Config>>,
|
|
) -> ui_test::Result<()> {
|
|
let configs = configs
|
|
.into_iter()
|
|
.collect::<ui_test::Result<Vec<Config>>>()?;
|
|
|
|
let emitter: Box<dyn StatusEmitter + Send> = if env::var_os("CI").is_some() {
|
|
Box::new((
|
|
Text::verbose(),
|
|
Gha::<true> {
|
|
name: test_name.into(),
|
|
},
|
|
))
|
|
} else {
|
|
Box::new(Text::quiet())
|
|
};
|
|
|
|
run_tests_generic(
|
|
configs,
|
|
default_file_filter,
|
|
default_per_file_config,
|
|
emitter,
|
|
)
|
|
}
|