# Objective
- Our benchmarks and `compile_fail` tests lag behind the rest of the
engine because they are not in the Cargo workspace, so not checked by
CI.
- Fixes#16801, please see it for further context!
## Solution
- Add benchmarks and `compile_fail` tests to the Cargo workspace.
- Fix any leftover formatting issues and documentation.
## Testing
- I think CI should catch most things!
## Questions
<details>
<summary>Outdated issue I was having with function reflection being
optional</summary>
The `reflection_types` example is failing in Rust-Analyzer for me, but
not a normal check.
```rust
error[E0004]: non-exhaustive patterns: `ReflectRef::Function(_)` not covered
--> examples/reflection/reflection_types.rs:81:11
|
81 | match value.reflect_ref() {
| ^^^^^^^^^^^^^^^^^^^ pattern `ReflectRef::Function(_)` not covered
|
note: `ReflectRef<'_>` defined here
--> /Users/bdeep/dev/bevy/bevy/crates/bevy_reflect/src/kind.rs:178:1
|
178 | pub enum ReflectRef<'a> {
| ^^^^^^^^^^^^^^^^^^^^^^^
...
188 | Function(&'a dyn Function),
| -------- not covered
= note: the matched value is of type `ReflectRef<'_>`
help: ensure that all possible cases are being handled by adding a match arm with a wildcard pattern or an explicit pattern as shown
|
126 ~ ReflectRef::Opaque(_) => {},
127 + ReflectRef::Function(_) => todo!()
|
```
I think it is because the following line is feature-gated:
cc0f6a8db4/examples/reflection/reflection_types.rs (L117-L122)
My theory for why this is happening is because the benchmarks enabled
`bevy_reflect`'s `function` feature, which gets merged with the rest of
the features when RA checks the workspace, but the `#[cfg(...)]` gate in
the example isn't detecting it:
cc0f6a8db4/benches/Cargo.toml (L19)
Any thoughts on how to fix this? It's not blocking, since the example
still compiles as normal, but it's just RA and the command `cargo check
--workspace --all-targets` appears to fail.
</summary>
# Objective
It looks like running `compile_fail_utils::test` on an invalid path just
skips the test. I'm not sure why `ui_test` doesn't fail the test, but it
seems pretty easy to accidentally cause a test to be skipped (I
experienced
[this](https://github.com/bevyengine/bevy/pull/14813#discussion_r1721880973)
while doing some refactoring on `bevy_reflect`).
## Solution
Check to make sure the given path exists before continuing on with the
tests.
Alternatively, we could look into seeing why this doesn't work properly
upstream. But I figured this solution was simple enough just to
implement directly without having to worry about updating `ui_test`.
## Testing
To verify that this works as expected `cd` into
`crates/bevy_reflect/compile_fail`. Then run the following:
```
cargo test --target-dir ../../../target
```
All compile fail tests should pass. Now edit the path used in
`crates/bevy_reflect/compile_fail/tests/derive.rs`. For example:
```diff
fn main() -> compile_fail_utils::ui_test::Result<()> {
- compile_fail_utils::test("reflect_derive", "tests/reflect_derive")
+ compile_fail_utils::test("reflect_derive", "tests/does_not_exist")
}
```
Run the tests again:
```
cargo test --target-dir ../../../target
```
Verify the test fails with an error like:
```
Error: path does not exist: "tests/does_not_exist"
```
# Objective
The github action summary titles every compile test group as
`compile_fail_utils`.
## Solution
Manually specify group names for compile fail tests.
## Testing
- Wait for compile fail tests to run.
- Observe the generated summary.
# Objective
- Follow-up of #13184 :)
- We use `ui_test` to test compiler errors for our custom macros.
- There are four crates related to compile fail tests
- `bevy_ecs_compile_fail_tests`, `bevy_macros_compile_fail_tests`, and
`bevy_reflect_compile_fail_tests`, which actually test the macros.
-
[`bevy_compile_test_utils`](64c1c65783/crates/bevy_compile_test_utils),
which provides helpers and common patterns for these tests.
- All of these crates reside within the `crates` directory.
- This can be confusing, especially for newcomers. All of the other
folders in `crates` are actual published libraries, except for these 4.
## Solution
- Move all compile fail tests to a `compile_fail` folder under their
corresponding crate.
- E.g. `crates/bevy_ecs_compile_fail_tests` would be moved to
`crates/bevy_ecs/compile_fail`.
- Move `bevy_compile_test_utils` to `tools/compile_fail_utils`.
There are a few benefits to this approach:
1. An internal testing detail is less intrusive (and confusing) for
those who just want to browse the public Bevy interface.
2. Follows a pre-existing approach of organizing related crates inside a
larger crate's folder.
- See `bevy_gizmos/macros` for an example.
4. Makes consistent the terms `compile_test`, `compile_fail`, and
`compile_fail_test` in code. It's all just `compile_fail` now, because
we are specifically testing the error messages on compiler failures.
- To be clear it can still be referred to by these terms in comments and
speech, just the names of the crates and the CI command are now
consistent.
## Testing
Run the compile fail CI command:
```shell
cargo run -p ci -- compile-fail
```
If it still passes, then my refactor was successful.
