bevy/tools/compile_fail_utils/src/lib.rs
Gino Valente f8ef767ab0
compile_fail_utils: Verify path exists (#14827)
# Objective

It looks like running `compile_fail_utils::test` on an invalid path just
skips the test. I'm not sure why `ui_test` doesn't fail the test, but it
seems pretty easy to accidentally cause a test to be skipped (I
experienced
[this](https://github.com/bevyengine/bevy/pull/14813#discussion_r1721880973)
while doing some refactoring on `bevy_reflect`).

## Solution

Check to make sure the given path exists before continuing on with the
tests.

Alternatively, we could look into seeing why this doesn't work properly
upstream. But I figured this solution was simple enough just to
implement directly without having to worry about updating `ui_test`.

## Testing

To verify that this works as expected `cd` into
`crates/bevy_reflect/compile_fail`. Then run the following:

```
cargo test --target-dir ../../../target
```

All compile fail tests should pass. Now edit the path used in
`crates/bevy_reflect/compile_fail/tests/derive.rs`. For example:

```diff
fn main() -> compile_fail_utils::ui_test::Result<()> {
-    compile_fail_utils::test("reflect_derive", "tests/reflect_derive")
+    compile_fail_utils::test("reflect_derive", "tests/does_not_exist")
}
```

Run the tests again:

```
cargo test --target-dir ../../../target
```

Verify the test fails with an error like:

```
Error: path does not exist: "tests/does_not_exist"
```
2024-08-22 16:54:26 +00:00

127 lines
4.1 KiB
Rust

use std::{
env,
path::{Path, PathBuf},
};
// Re-export ui_test so all the tests use the same version.
pub use ui_test;
use ui_test::{
color_eyre::eyre::eyre,
default_file_filter, default_per_file_config,
dependencies::DependencyBuilder,
run_tests_generic,
spanned::Spanned,
status_emitter::{Gha, StatusEmitter, Text},
Args, Config, OutputConflictHandling,
};
/// Use this instead of hand rolling configs.
///
/// `root_dir` is the directory your tests are contained in. Needs to be a path from crate root.
/// This config will build dependencies and will assume that the cargo manifest is placed at the
/// current working directory.
fn basic_config(root_dir: impl Into<PathBuf>, args: &Args) -> ui_test::Result<Config> {
let root_dir = root_dir.into();
match root_dir.try_exists() {
Ok(true) => { /* success */ }
Ok(false) => {
return Err(eyre!("path does not exist: {:?}", root_dir));
}
Err(error) => {
return Err(eyre!("failed to read path: {:?} ({:?})", root_dir, error));
}
}
let mut config = Config {
bless_command: Some(
"`cargo test` with the BLESS environment variable set to any non empty value"
.to_string(),
),
output_conflict_handling: if env::var_os("BLESS").is_some() {
OutputConflictHandling::Bless
} else {
// stderr output changes between rust versions so we just rely on annotations
OutputConflictHandling::Ignore
},
..Config::rustc(root_dir)
};
config.with_args(args);
let bevy_root = "..";
// Don't leak contributor filesystem paths
config.path_stderr_filter(Path::new(bevy_root), b"$BEVY_ROOT");
config.path_stderr_filter(Path::new(env!("RUSTUP_HOME")), b"$RUSTUP_HOME");
// ui_test doesn't compile regex with perl character classes.
// \pL = unicode class for letters, \pN = unicode class for numbers
config.stderr_filter(r"\/home\/[\pL\pN_@#\-\. ]+", "$HOME");
// Paths in .stderr seem to always be normalized to use /. Handle both anyway.
config.stderr_filter(
r"[a-zA-Z]:(?:\\|\/)users(?:\\|\/)[\pL\pN_@#\-\. ]+", // NOTE: [\pL\pN_@#\-\. ] is a poor attempt at handling usernames
"$HOME",
);
// Manually insert @aux-build:<dep> comments into test files. This needs to
// be done to build and link dependencies. Dependencies will be pulled from a
// Cargo.toml file.
config.comment_defaults.base().custom.insert(
"dependencies",
Spanned::dummy(vec![Box::new(DependencyBuilder::default())]),
);
Ok(config)
}
/// Runs ui tests for a single directory.
///
/// `root_dir` is the directory your tests are contained in. Needs to be a path from crate root.
pub fn test(test_name: impl Into<String>, test_root: impl Into<PathBuf>) -> ui_test::Result<()> {
test_multiple(test_name, [test_root])
}
/// Run ui tests with the given config
pub fn test_with_config(test_name: impl Into<String>, config: Config) -> ui_test::Result<()> {
test_with_multiple_configs(test_name, [Ok(config)])
}
/// Runs ui tests for a multiple directories.
///
/// `root_dirs` paths need to come from crate root.
pub fn test_multiple(
test_name: impl Into<String>,
test_roots: impl IntoIterator<Item = impl Into<PathBuf>>,
) -> ui_test::Result<()> {
let args = Args::test()?;
let configs = test_roots.into_iter().map(|root| basic_config(root, &args));
test_with_multiple_configs(test_name, configs)
}
/// Run ui test with the given configs.
///
/// Tests for configs are run in parallel.
pub fn test_with_multiple_configs(
test_name: impl Into<String>,
configs: impl IntoIterator<Item = ui_test::Result<Config>>,
) -> ui_test::Result<()> {
let configs = configs.into_iter().collect::<ui_test::Result<Vec<Config>>>()?;
let emitter: Box<dyn StatusEmitter + Send> = if env::var_os("CI").is_some() {
Box::new((Text::verbose(), Gha::<true> { name: test_name.into() }))
} else {
Box::new(Text::quiet())
};
run_tests_generic(
configs,
default_file_filter,
default_per_file_config,
emitter,
)
}