bevy/crates/bevy_ecs/macros/src/lib.rs
TheRawMeatball 73c78c3667 Use lifetimed, type erased pointers in bevy_ecs (#3001)
# Objective

`bevy_ecs` has large amounts of unsafe code which is hard to get right and makes it difficult to audit for soundness.

## Solution

Introduce lifetimed, type-erased pointers: `Ptr<'a>` `PtrMut<'a>` `OwningPtr<'a>'` and `ThinSlicePtr<'a, T>` which are newtypes around a raw pointer with a lifetime and conceptually representing strong invariants about the pointee and validity of the pointer.

The process of converting bevy_ecs to use these has already caught multiple cases of unsound behavior.

## Changelog

TL;DR for release notes: `bevy_ecs` now uses lifetimed, type-erased pointers internally, significantly improving safety and legibility without sacrificing performance. This should have approximately no end user impact, unless you were meddling with the (unfortunately public) internals of `bevy_ecs`.

- `Fetch`, `FilterFetch` and `ReadOnlyFetch` trait no longer have a `'state` lifetime
    - this was unneeded
- `ReadOnly/Fetch` associated types on `WorldQuery` are now on a new `WorldQueryGats<'world>` trait
    - was required to work around lack of Generic Associated Types (we wish to express `type Fetch<'a>: Fetch<'a>`)
- `derive(WorldQuery)` no longer requires `'w` lifetime on struct
    - this was unneeded, and improves the end user experience
- `EntityMut::get_unchecked_mut` returns `&'_ mut T` not `&'w mut T`
    - allows easier use of unsafe API with less footguns, and can be worked around via lifetime transmutery as a user
- `Bundle::from_components` now takes a `ctx` parameter to pass to the `FnMut` closure
    - required because closure return types can't borrow from captures
- `Fetch::init` takes `&'world World`, `Fetch::set_archetype` takes `&'world Archetype` and `&'world Tables`, `Fetch::set_table` takes `&'world Table`
    - allows types implementing `Fetch` to store borrows into world
- `WorldQuery` trait now has a `shrink` fn to shorten the lifetime in `Fetch::<'a>::Item`
    - this works around lack of subtyping of assoc types, rust doesnt allow you to turn `<T as Fetch<'static>>::Item'` into `<T as Fetch<'a>>::Item'`
    - `QueryCombinationsIter` requires this
- Most types implementing `Fetch` now have a lifetime `'w`
    - allows the fetches to store borrows of world data instead of using raw pointers

## Migration guide

- `EntityMut::get_unchecked_mut` returns a more restricted lifetime, there is no general way to migrate this as it depends on your code
- `Bundle::from_components` implementations must pass the `ctx` arg to `func`
- `Bundle::from_components` callers have to use a fn arg instead of closure captures for borrowing from world
- Remove lifetime args on `derive(WorldQuery)` structs as it is nonsensical
- `<Q as WorldQuery>::ReadOnly/Fetch` should be changed to either `RO/QueryFetch<'world>` or `<Q as WorldQueryGats<'world>>::ReadOnly/Fetch`
- `<F as Fetch<'w, 's>>` should be changed to `<F as Fetch<'w>>`
- Change the fn sigs of `Fetch::init/set_archetype/set_table` to match respective trait fn sigs
- Implement the required `fn shrink` on any `WorldQuery` implementations
- Move assoc types `Fetch` and `ReadOnlyFetch` on `WorldQuery` impls to `WorldQueryGats` impls
- Pass an appropriate `'world` lifetime to whatever fetch struct you are for some reason using

### Type inference regression

in some cases rustc may give spurrious errors when attempting to infer the `F` parameter on a query/querystate this can be fixed by manually specifying the type, i.e. `QueryState:🆕:<_, ()>(world)`. The error is rather confusing:

```rust=
error[E0271]: type mismatch resolving `<() as Fetch<'_>>::Item == bool`
    --> crates/bevy_pbr/src/render/light.rs:1413:30
     |
1413 |             main_view_query: QueryState::new(world),
     |                              ^^^^^^^^^^^^^^^ expected `bool`, found `()`
     |
     = note: required because of the requirements on the impl of `for<'x> FilterFetch<'x>` for `<() as WorldQueryGats<'x>>::Fetch`
note: required by a bound in `bevy_ecs::query::QueryState::<Q, F>::new`
    --> crates/bevy_ecs/src/query/state.rs:49:32
     |
49   |     for<'x> QueryFetch<'x, F>: FilterFetch<'x>,
     |                                ^^^^^^^^^^^^^^^ required by this bound in `bevy_ecs::query::QueryState::<Q, F>::new`
```

---

Made with help from @BoxyUwU and @alice-i-cecile 

Co-authored-by: Boxy <supbscripter@gmail.com>
2022-04-27 23:44:06 +00:00

476 lines
18 KiB
Rust

extern crate proc_macro;
mod component;
mod fetch;
use crate::fetch::derive_world_query_impl;
use bevy_macro_utils::{derive_label, get_named_struct_fields, BevyManifest};
use proc_macro::TokenStream;
use proc_macro2::Span;
use quote::{format_ident, quote};
use syn::{
parse::{Parse, ParseStream},
parse_macro_input,
punctuated::Punctuated,
token::Comma,
DeriveInput, Field, GenericParam, Ident, Index, LitInt, Result, Token, TypeParam,
};
struct AllTuples {
macro_ident: Ident,
start: usize,
end: usize,
idents: Vec<Ident>,
}
impl Parse for AllTuples {
fn parse(input: ParseStream) -> Result<Self> {
let macro_ident = input.parse::<Ident>()?;
input.parse::<Comma>()?;
let start = input.parse::<LitInt>()?.base10_parse()?;
input.parse::<Comma>()?;
let end = input.parse::<LitInt>()?.base10_parse()?;
input.parse::<Comma>()?;
let mut idents = vec![input.parse::<Ident>()?];
while input.parse::<Comma>().is_ok() {
idents.push(input.parse::<Ident>()?);
}
Ok(AllTuples {
macro_ident,
start,
end,
idents,
})
}
}
#[proc_macro]
pub fn all_tuples(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as AllTuples);
let len = input.end - input.start;
let mut ident_tuples = Vec::with_capacity(len);
for i in input.start..=input.end {
let idents = input
.idents
.iter()
.map(|ident| format_ident!("{}{}", ident, i));
if input.idents.len() < 2 {
ident_tuples.push(quote! {
#(#idents)*
});
} else {
ident_tuples.push(quote! {
(#(#idents),*)
});
}
}
let macro_ident = &input.macro_ident;
let invocations = (input.start..=input.end).map(|i| {
let ident_tuples = &ident_tuples[0..i - input.start];
quote! {
#macro_ident!(#(#ident_tuples),*);
}
});
TokenStream::from(quote! {
#(
#invocations
)*
})
}
static BUNDLE_ATTRIBUTE_NAME: &str = "bundle";
#[proc_macro_derive(Bundle, attributes(bundle))]
pub fn derive_bundle(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
let ecs_path = bevy_ecs_path();
let named_fields = match get_named_struct_fields(&ast.data) {
Ok(fields) => &fields.named,
Err(e) => return e.into_compile_error().into(),
};
let is_bundle = named_fields
.iter()
.map(|field| {
field
.attrs
.iter()
.any(|a| *a.path.get_ident().as_ref().unwrap() == BUNDLE_ATTRIBUTE_NAME)
})
.collect::<Vec<bool>>();
let field = named_fields
.iter()
.map(|field| field.ident.as_ref().unwrap())
.collect::<Vec<_>>();
let field_type = named_fields
.iter()
.map(|field| &field.ty)
.collect::<Vec<_>>();
let mut field_component_ids = Vec::new();
let mut field_get_components = Vec::new();
let mut field_from_components = Vec::new();
for ((field_type, is_bundle), field) in
field_type.iter().zip(is_bundle.iter()).zip(field.iter())
{
if *is_bundle {
field_component_ids.push(quote! {
component_ids.extend(<#field_type as #ecs_path::bundle::Bundle>::component_ids(components, storages));
});
field_get_components.push(quote! {
self.#field.get_components(&mut func);
});
field_from_components.push(quote! {
#field: <#field_type as #ecs_path::bundle::Bundle>::from_components(ctx, &mut func),
});
} else {
field_component_ids.push(quote! {
component_ids.push(components.init_component::<#field_type>(storages));
});
field_get_components.push(quote! {
#ecs_path::ptr::OwningPtr::make(self.#field, &mut func);
});
field_from_components.push(quote! {
#field: func(ctx).inner().as_ptr().cast::<#field_type>().read(),
});
}
}
let field_len = field.len();
let generics = ast.generics;
let (impl_generics, ty_generics, where_clause) = generics.split_for_impl();
let struct_name = &ast.ident;
TokenStream::from(quote! {
/// SAFE: ComponentId is returned in field-definition-order. [from_components] and [get_components] use field-definition-order
unsafe impl #impl_generics #ecs_path::bundle::Bundle for #struct_name #ty_generics #where_clause {
fn component_ids(
components: &mut #ecs_path::component::Components,
storages: &mut #ecs_path::storage::Storages,
) -> Vec<#ecs_path::component::ComponentId> {
let mut component_ids = Vec::with_capacity(#field_len);
#(#field_component_ids)*
component_ids
}
#[allow(unused_variables, unused_mut, non_snake_case)]
unsafe fn from_components<T, F>(ctx: &mut T, mut func: F) -> Self
where
F: FnMut(&mut T) -> #ecs_path::ptr::OwningPtr<'_>
{
Self {
#(#field_from_components)*
}
}
#[allow(unused_variables, unused_mut, forget_copy, forget_ref)]
fn get_components(self, mut func: impl FnMut(#ecs_path::ptr::OwningPtr<'_>)) {
#(#field_get_components)*
}
}
})
}
fn get_idents(fmt_string: fn(usize) -> String, count: usize) -> Vec<Ident> {
(0..count)
.map(|i| Ident::new(&fmt_string(i), Span::call_site()))
.collect::<Vec<Ident>>()
}
#[proc_macro]
pub fn impl_param_set(_input: TokenStream) -> TokenStream {
let mut tokens = TokenStream::new();
let max_params = 8;
let params = get_idents(|i| format!("P{}", i), max_params);
let params_fetch = get_idents(|i| format!("PF{}", i), max_params);
let metas = get_idents(|i| format!("m{}", i), max_params);
let mut param_fn_muts = Vec::new();
for (i, param) in params.iter().enumerate() {
let fn_name = Ident::new(&format!("p{}", i), Span::call_site());
let index = Index::from(i);
param_fn_muts.push(quote! {
pub fn #fn_name<'a>(&'a mut self) -> <#param::Fetch as SystemParamFetch<'a, 'a>>::Item {
// SAFE: systems run without conflicts with other systems.
// Conflicting params in ParamSet are not accessible at the same time
// ParamSets are guaranteed to not conflict with other SystemParams
unsafe {
<#param::Fetch as SystemParamFetch<'a, 'a>>::get_param(&mut self.param_states.#index, &self.system_meta, self.world, self.change_tick)
}
}
});
}
for param_count in 1..=max_params {
let param = &params[0..param_count];
let param_fetch = &params_fetch[0..param_count];
let meta = &metas[0..param_count];
let param_fn_mut = &param_fn_muts[0..param_count];
tokens.extend(TokenStream::from(quote! {
impl<'w, 's, #(#param: SystemParam,)*> SystemParam for ParamSet<'w, 's, (#(#param,)*)>
{
type Fetch = ParamSetState<(#(#param::Fetch,)*)>;
}
// SAFE: All parameters are constrained to ReadOnlyFetch, so World is only read
unsafe impl<#(#param_fetch: for<'w1, 's1> SystemParamFetch<'w1, 's1>,)*> ReadOnlySystemParamFetch for ParamSetState<(#(#param_fetch,)*)>
where #(#param_fetch: ReadOnlySystemParamFetch,)*
{ }
// SAFE: Relevant parameter ComponentId and ArchetypeComponentId access is applied to SystemMeta. If any ParamState conflicts
// with any prior access, a panic will occur.
unsafe impl<#(#param_fetch: for<'w1, 's1> SystemParamFetch<'w1, 's1>,)*> SystemParamState for ParamSetState<(#(#param_fetch,)*)>
{
fn init(world: &mut World, system_meta: &mut SystemMeta) -> Self {
#(
// Pretend to add each param to the system alone, see if it conflicts
let mut #meta = system_meta.clone();
#meta.component_access_set.clear();
#meta.archetype_component_access.clear();
#param_fetch::init(world, &mut #meta);
let #param = #param_fetch::init(world, &mut system_meta.clone());
)*
#(
system_meta
.component_access_set
.extend(#meta.component_access_set);
system_meta
.archetype_component_access
.extend(&#meta.archetype_component_access);
)*
ParamSetState((#(#param,)*))
}
fn new_archetype(&mut self, archetype: &Archetype, system_meta: &mut SystemMeta) {
let (#(#param,)*) = &mut self.0;
#(
#param.new_archetype(archetype, system_meta);
)*
}
}
impl<'w, 's, #(#param_fetch: for<'w1, 's1> SystemParamFetch<'w1, 's1>,)*> SystemParamFetch<'w, 's> for ParamSetState<(#(#param_fetch,)*)>
{
type Item = ParamSet<'w, 's, (#(<#param_fetch as SystemParamFetch<'w, 's>>::Item,)*)>;
#[inline]
unsafe fn get_param(
state: &'s mut Self,
system_meta: &SystemMeta,
world: &'w World,
change_tick: u32,
) -> Self::Item {
ParamSet {
param_states: &mut state.0,
system_meta: system_meta.clone(),
world,
change_tick,
}
}
}
impl<'w, 's, #(#param: SystemParam,)*> ParamSet<'w, 's, (#(#param,)*)>
{
#(#param_fn_mut)*
}
}));
}
tokens
}
#[derive(Default)]
struct SystemParamFieldAttributes {
pub ignore: bool,
}
static SYSTEM_PARAM_ATTRIBUTE_NAME: &str = "system_param";
/// Implement `SystemParam` to use a struct as a parameter in a system
#[proc_macro_derive(SystemParam, attributes(system_param))]
pub fn derive_system_param(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
let fields = match get_named_struct_fields(&ast.data) {
Ok(fields) => &fields.named,
Err(e) => return e.into_compile_error().into(),
};
let path = bevy_ecs_path();
let field_attributes = fields
.iter()
.map(|field| {
(
field,
field
.attrs
.iter()
.find(|a| *a.path.get_ident().as_ref().unwrap() == SYSTEM_PARAM_ATTRIBUTE_NAME)
.map_or_else(SystemParamFieldAttributes::default, |a| {
syn::custom_keyword!(ignore);
let mut attributes = SystemParamFieldAttributes::default();
a.parse_args_with(|input: ParseStream| {
if input.parse::<Option<ignore>>()?.is_some() {
attributes.ignore = true;
}
Ok(())
})
.expect("Invalid 'render_resources' attribute format.");
attributes
}),
)
})
.collect::<Vec<(&Field, SystemParamFieldAttributes)>>();
let mut fields = Vec::new();
let mut field_indices = Vec::new();
let mut field_types = Vec::new();
let mut ignored_fields = Vec::new();
let mut ignored_field_types = Vec::new();
for (i, (field, attrs)) in field_attributes.iter().enumerate() {
if attrs.ignore {
ignored_fields.push(field.ident.as_ref().unwrap());
ignored_field_types.push(&field.ty);
} else {
fields.push(field.ident.as_ref().unwrap());
field_types.push(&field.ty);
field_indices.push(Index::from(i));
}
}
let generics = ast.generics;
let (impl_generics, ty_generics, where_clause) = generics.split_for_impl();
let lifetimeless_generics: Vec<_> = generics
.params
.iter()
.filter(|g| matches!(g, GenericParam::Type(_)))
.collect();
let mut punctuated_generics = Punctuated::<_, Token![,]>::new();
punctuated_generics.extend(lifetimeless_generics.iter().map(|g| match g {
GenericParam::Type(g) => GenericParam::Type(TypeParam {
default: None,
..g.clone()
}),
_ => unreachable!(),
}));
let mut punctuated_generic_idents = Punctuated::<_, Token![,]>::new();
punctuated_generic_idents.extend(lifetimeless_generics.iter().map(|g| match g {
GenericParam::Type(g) => &g.ident,
_ => unreachable!(),
}));
let struct_name = &ast.ident;
let fetch_struct_name = Ident::new(&format!("{}State", struct_name), Span::call_site());
let fetch_struct_visibility = &ast.vis;
TokenStream::from(quote! {
impl #impl_generics #path::system::SystemParam for #struct_name #ty_generics #where_clause {
type Fetch = #fetch_struct_name <(#(<#field_types as #path::system::SystemParam>::Fetch,)*), #punctuated_generic_idents>;
}
#[doc(hidden)]
#fetch_struct_visibility struct #fetch_struct_name<TSystemParamState, #punctuated_generic_idents> {
state: TSystemParamState,
marker: std::marker::PhantomData<fn()->(#punctuated_generic_idents)>
}
unsafe impl<TSystemParamState: #path::system::SystemParamState, #punctuated_generics> #path::system::SystemParamState for #fetch_struct_name<TSystemParamState, #punctuated_generic_idents> #where_clause {
fn init(world: &mut #path::world::World, system_meta: &mut #path::system::SystemMeta) -> Self {
Self {
state: TSystemParamState::init(world, system_meta),
marker: std::marker::PhantomData,
}
}
fn new_archetype(&mut self, archetype: &#path::archetype::Archetype, system_meta: &mut #path::system::SystemMeta) {
self.state.new_archetype(archetype, system_meta)
}
fn apply(&mut self, world: &mut #path::world::World) {
self.state.apply(world)
}
}
impl #impl_generics #path::system::SystemParamFetch<'w, 's> for #fetch_struct_name <(#(<#field_types as #path::system::SystemParam>::Fetch,)*), #punctuated_generic_idents> #where_clause {
type Item = #struct_name #ty_generics;
unsafe fn get_param(
state: &'s mut Self,
system_meta: &#path::system::SystemMeta,
world: &'w #path::world::World,
change_tick: u32,
) -> Self::Item {
#struct_name {
#(#fields: <<#field_types as #path::system::SystemParam>::Fetch as #path::system::SystemParamFetch>::get_param(&mut state.state.#field_indices, system_meta, world, change_tick),)*
#(#ignored_fields: <#ignored_field_types>::default(),)*
}
}
}
})
}
/// Implement `WorldQuery` to use a struct as a parameter in a query
#[proc_macro_derive(WorldQuery, attributes(world_query))]
pub fn derive_world_query(input: TokenStream) -> TokenStream {
let ast = parse_macro_input!(input as DeriveInput);
derive_world_query_impl(ast)
}
#[proc_macro_derive(SystemLabel)]
pub fn derive_system_label(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let mut trait_path = bevy_ecs_path();
trait_path.segments.push(format_ident!("schedule").into());
trait_path
.segments
.push(format_ident!("SystemLabel").into());
derive_label(input, &trait_path)
}
#[proc_macro_derive(StageLabel)]
pub fn derive_stage_label(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let mut trait_path = bevy_ecs_path();
trait_path.segments.push(format_ident!("schedule").into());
trait_path.segments.push(format_ident!("StageLabel").into());
derive_label(input, &trait_path)
}
#[proc_macro_derive(AmbiguitySetLabel)]
pub fn derive_ambiguity_set_label(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let mut trait_path = bevy_ecs_path();
trait_path.segments.push(format_ident!("schedule").into());
trait_path
.segments
.push(format_ident!("AmbiguitySetLabel").into());
derive_label(input, &trait_path)
}
#[proc_macro_derive(RunCriteriaLabel)]
pub fn derive_run_criteria_label(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
let mut trait_path = bevy_ecs_path();
trait_path.segments.push(format_ident!("schedule").into());
trait_path
.segments
.push(format_ident!("RunCriteriaLabel").into());
derive_label(input, &trait_path)
}
pub(crate) fn bevy_ecs_path() -> syn::Path {
BevyManifest::default().get_path("bevy_ecs")
}
#[proc_macro_derive(Component, attributes(component))]
pub fn derive_component(input: TokenStream) -> TokenStream {
component::derive_component(input)
}