
# Objective Add specialized UI transform `Component`s and fix some related problems: * Animating UI elements by modifying the `Transform` component of UI nodes doesn't work very well because `ui_layout_system` overwrites the translations each frame. The `overflow_debug` example uses a horrible hack where it copies the transform into the position that'll likely cause a panic if any users naively copy it. * Picking ignores rotation and scaling and assumes UI nodes are always axis aligned. * The clipping geometry stored in `CalculatedClip` is wrong for rotated and scaled elements. * Transform propagation is unnecessary for the UI, the transforms can be updated during layout updates. * The UI internals use both object-centered and top-left-corner-based coordinates systems for UI nodes. Depending on the context you have to add or subtract the half-size sometimes before transforming between coordinate spaces. We should just use one system consistantly so that the transform can always be directly applied. * `Transform` doesn't support responsive coordinates. ## Solution * Unrequire `Transform` from `Node`. * New components `UiTransform`, `UiGlobalTransform`: - `Node` requires `UiTransform`, `UiTransform` requires `UiGlobalTransform` - `UiTransform` is a 2d-only equivalent of `Transform` with a translation in `Val`s. - `UiGlobalTransform` newtypes `Affine2` and is updated in `ui_layout_system`. * New helper functions on `ComputedNode` for mapping between viewport and local node space. * The cursor position is transformed to local node space during picking so that it respects rotations and scalings. * To check if the cursor hovers a node recursively walk up the tree to the root checking if any of the ancestor nodes clip the point at the cursor. If the point is clipped the interaction is ignored. * Use object-centered coordinates for UI nodes. * `RelativeCursorPosition`'s coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5). * Replaced the `normalized_visible_node_rect: Rect` field of `RelativeCursorPosition` with `cursor_over: bool`, which is set to true when the cursor is over an unclipped point on the node. The visible area of the node is not necessarily a rectangle, so the previous implementation didn't work. This should fix all the logical bugs with non-axis aligned interactions and clipping. Rendering still needs changes but they are far outside the scope of this PR. Tried and abandoned two other approaches: * New `transform` field on `Node`, require `GlobalTransform` on `Node`, and unrequire `Transform` on `Node`. Unrequiring `Transform` opts out of transform propagation so there is then no conflict with updating the `GlobalTransform` in `ui_layout_system`. This was a nice change in its simplicity but potentially confusing for users I think, all the `GlobalTransform` docs mention `Transform` and having special rules for how it's updated just for the UI is unpleasently surprising. * New `transform` field on `Node`. Unrequire `Transform` on `Node`. New `transform: Affine2` field on `ComputedNode`. This was okay but I think most users want a separate specialized UI transform components. The fat `ComputedNode` doesn't work well with change detection. Fixes #18929, #18930 ## Testing There is an example you can look at: ``` cargo run --example ui_transform ``` Sometimes in the example if you press the rotate button couple of times the first glyph from the top label disappears , I'm not sure what's causing it yet but I don't think it's related to this PR. ## Migration Guide New specialized 2D UI transform components `UiTransform` and `UiGlobalTransform`. `UiTransform` is a 2d-only equivalent of `Transform` with a translation in `Val`s. `UiGlobalTransform` newtypes `Affine2` and is updated in `ui_layout_system`. `Node` now requires `UiTransform` instead of `Transform`. `UiTransform` requires `UiGlobalTransform`. In previous versions of Bevy `ui_layout_system` would overwrite UI node's `Transform::translation` each frame. `UiTransform`s aren't overwritten and there is no longer any need for systems that cache and rewrite the transform for translated UI elements. `RelativeCursorPosition`'s coordinates are now object-centered with (0,0) at the the center of the node and the corners at (±0.5, ±0.5). Its `normalized_visible_node_rect` field has been removed and replaced with a new `cursor_over: bool` field which is set to true when the cursor is hovering an unclipped area of the UI node. --------- Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
325 lines
12 KiB
Rust
325 lines
12 KiB
Rust
use crate::{
|
|
picking_backend::clip_check_recursive, ui_transform::UiGlobalTransform, ComputedNode,
|
|
ComputedNodeTarget, Node, UiStack,
|
|
};
|
|
use bevy_ecs::{
|
|
change_detection::DetectChangesMut,
|
|
entity::{ContainsEntity, Entity},
|
|
hierarchy::ChildOf,
|
|
prelude::{Component, With},
|
|
query::QueryData,
|
|
reflect::ReflectComponent,
|
|
system::{Local, Query, Res},
|
|
};
|
|
use bevy_input::{mouse::MouseButton, touch::Touches, ButtonInput};
|
|
use bevy_math::Vec2;
|
|
use bevy_platform::collections::HashMap;
|
|
use bevy_reflect::{std_traits::ReflectDefault, Reflect};
|
|
use bevy_render::{camera::NormalizedRenderTarget, prelude::Camera, view::InheritedVisibility};
|
|
use bevy_window::{PrimaryWindow, Window};
|
|
|
|
use smallvec::SmallVec;
|
|
|
|
#[cfg(feature = "serialize")]
|
|
use bevy_reflect::{ReflectDeserialize, ReflectSerialize};
|
|
|
|
/// Describes what type of input interaction has occurred for a UI node.
|
|
///
|
|
/// This is commonly queried with a `Changed<Interaction>` filter.
|
|
///
|
|
/// Updated in [`ui_focus_system`].
|
|
///
|
|
/// If a UI node has both [`Interaction`] and [`InheritedVisibility`] components,
|
|
/// [`Interaction`] will always be [`Interaction::None`]
|
|
/// when [`InheritedVisibility::get()`] is false.
|
|
/// This ensures that hidden UI nodes are not interactable,
|
|
/// and do not end up stuck in an active state if hidden at the wrong time.
|
|
///
|
|
/// Note that you can also control the visibility of a node using the [`Display`](crate::ui_node::Display) property,
|
|
/// which fully collapses it during layout calculations.
|
|
///
|
|
/// # See also
|
|
///
|
|
/// - [`Button`](crate::widget::Button) which requires this component
|
|
/// - [`RelativeCursorPosition`] to obtain the position of the cursor relative to current node
|
|
#[derive(Component, Copy, Clone, Eq, PartialEq, Debug, Reflect)]
|
|
#[reflect(Component, Default, PartialEq, Debug, Clone)]
|
|
#[cfg_attr(
|
|
feature = "serialize",
|
|
derive(serde::Serialize, serde::Deserialize),
|
|
reflect(Serialize, Deserialize)
|
|
)]
|
|
pub enum Interaction {
|
|
/// The node has been pressed.
|
|
///
|
|
/// Note: This does not capture click/press-release action.
|
|
Pressed,
|
|
/// The node has been hovered over
|
|
Hovered,
|
|
/// Nothing has happened
|
|
None,
|
|
}
|
|
|
|
impl Interaction {
|
|
const DEFAULT: Self = Self::None;
|
|
}
|
|
|
|
impl Default for Interaction {
|
|
fn default() -> Self {
|
|
Self::DEFAULT
|
|
}
|
|
}
|
|
|
|
/// A component storing the position of the mouse relative to the node, (0., 0.) being the center and (0.5, 0.5) being the bottom-right
|
|
/// If the mouse is not over the node, the value will go beyond the range of (-0.5, -0.5) to (0.5, 0.5)
|
|
///
|
|
/// It can be used alongside [`Interaction`] to get the position of the press.
|
|
///
|
|
/// The component is updated when it is in the same entity with [`Node`].
|
|
#[derive(Component, Copy, Clone, Default, PartialEq, Debug, Reflect)]
|
|
#[reflect(Component, Default, PartialEq, Debug, Clone)]
|
|
#[cfg_attr(
|
|
feature = "serialize",
|
|
derive(serde::Serialize, serde::Deserialize),
|
|
reflect(Serialize, Deserialize)
|
|
)]
|
|
pub struct RelativeCursorPosition {
|
|
/// True if the cursor position is over an unclipped area of the Node.
|
|
pub cursor_over: bool,
|
|
/// Cursor position relative to the size and position of the Node.
|
|
/// A None value indicates that the cursor position is unknown.
|
|
pub normalized: Option<Vec2>,
|
|
}
|
|
|
|
impl RelativeCursorPosition {
|
|
/// A helper function to check if the mouse is over the node
|
|
pub fn cursor_over(&self) -> bool {
|
|
self.cursor_over
|
|
}
|
|
}
|
|
|
|
/// Describes whether the node should block interactions with lower nodes
|
|
#[derive(Component, Copy, Clone, Eq, PartialEq, Debug, Reflect)]
|
|
#[reflect(Component, Default, PartialEq, Debug, Clone)]
|
|
#[cfg_attr(
|
|
feature = "serialize",
|
|
derive(serde::Serialize, serde::Deserialize),
|
|
reflect(Serialize, Deserialize)
|
|
)]
|
|
pub enum FocusPolicy {
|
|
/// Blocks interaction
|
|
Block,
|
|
/// Lets interaction pass through
|
|
Pass,
|
|
}
|
|
|
|
impl FocusPolicy {
|
|
const DEFAULT: Self = Self::Pass;
|
|
}
|
|
|
|
impl Default for FocusPolicy {
|
|
fn default() -> Self {
|
|
Self::DEFAULT
|
|
}
|
|
}
|
|
|
|
/// Contains entities whose Interaction should be set to None
|
|
#[derive(Default)]
|
|
pub struct State {
|
|
entities_to_reset: SmallVec<[Entity; 1]>,
|
|
}
|
|
|
|
/// Main query for [`ui_focus_system`]
|
|
#[derive(QueryData)]
|
|
#[query_data(mutable)]
|
|
pub struct NodeQuery {
|
|
entity: Entity,
|
|
node: &'static ComputedNode,
|
|
transform: &'static UiGlobalTransform,
|
|
interaction: Option<&'static mut Interaction>,
|
|
relative_cursor_position: Option<&'static mut RelativeCursorPosition>,
|
|
focus_policy: Option<&'static FocusPolicy>,
|
|
inherited_visibility: Option<&'static InheritedVisibility>,
|
|
target_camera: &'static ComputedNodeTarget,
|
|
}
|
|
|
|
/// The system that sets Interaction for all UI elements based on the mouse cursor activity
|
|
///
|
|
/// Entities with a hidden [`InheritedVisibility`] are always treated as released.
|
|
pub fn ui_focus_system(
|
|
mut state: Local<State>,
|
|
camera_query: Query<(Entity, &Camera)>,
|
|
primary_window: Query<Entity, With<PrimaryWindow>>,
|
|
windows: Query<&Window>,
|
|
mouse_button_input: Res<ButtonInput<MouseButton>>,
|
|
touches_input: Res<Touches>,
|
|
ui_stack: Res<UiStack>,
|
|
mut node_query: Query<NodeQuery>,
|
|
clipping_query: Query<(&ComputedNode, &UiGlobalTransform, &Node)>,
|
|
child_of_query: Query<&ChildOf>,
|
|
) {
|
|
let primary_window = primary_window.iter().next();
|
|
|
|
// reset entities that were both clicked and released in the last frame
|
|
for entity in state.entities_to_reset.drain(..) {
|
|
if let Ok(NodeQueryItem {
|
|
interaction: Some(mut interaction),
|
|
..
|
|
}) = node_query.get_mut(entity)
|
|
{
|
|
*interaction = Interaction::None;
|
|
}
|
|
}
|
|
|
|
let mouse_released =
|
|
mouse_button_input.just_released(MouseButton::Left) || touches_input.any_just_released();
|
|
if mouse_released {
|
|
for node in &mut node_query {
|
|
if let Some(mut interaction) = node.interaction {
|
|
if *interaction == Interaction::Pressed {
|
|
*interaction = Interaction::None;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
let mouse_clicked =
|
|
mouse_button_input.just_pressed(MouseButton::Left) || touches_input.any_just_pressed();
|
|
|
|
let camera_cursor_positions: HashMap<Entity, Vec2> = camera_query
|
|
.iter()
|
|
.filter_map(|(entity, camera)| {
|
|
// Interactions are only supported for cameras rendering to a window.
|
|
let Some(NormalizedRenderTarget::Window(window_ref)) =
|
|
camera.target.normalize(primary_window)
|
|
else {
|
|
return None;
|
|
};
|
|
let window = windows.get(window_ref.entity()).ok()?;
|
|
|
|
let viewport_position = camera
|
|
.physical_viewport_rect()
|
|
.map(|rect| rect.min.as_vec2())
|
|
.unwrap_or_default();
|
|
window
|
|
.physical_cursor_position()
|
|
.or_else(|| {
|
|
touches_input
|
|
.first_pressed_position()
|
|
.map(|pos| pos * window.scale_factor())
|
|
})
|
|
.map(|cursor_position| (entity, cursor_position - viewport_position))
|
|
})
|
|
.collect();
|
|
|
|
// prepare an iterator that contains all the nodes that have the cursor in their rect,
|
|
// from the top node to the bottom one. this will also reset the interaction to `None`
|
|
// for all nodes encountered that are no longer hovered.
|
|
let mut hovered_nodes = ui_stack
|
|
.uinodes
|
|
.iter()
|
|
// reverse the iterator to traverse the tree from closest nodes to furthest
|
|
.rev()
|
|
.filter_map(|entity| {
|
|
let Ok(node) = node_query.get_mut(*entity) else {
|
|
return None;
|
|
};
|
|
|
|
let inherited_visibility = node.inherited_visibility?;
|
|
// Nodes that are not rendered should not be interactable
|
|
if !inherited_visibility.get() {
|
|
// Reset their interaction to None to avoid strange stuck state
|
|
if let Some(mut interaction) = node.interaction {
|
|
// We cannot simply set the interaction to None, as that will trigger change detection repeatedly
|
|
interaction.set_if_neq(Interaction::None);
|
|
}
|
|
return None;
|
|
}
|
|
let camera_entity = node.target_camera.camera()?;
|
|
|
|
let cursor_position = camera_cursor_positions.get(&camera_entity);
|
|
|
|
let contains_cursor = cursor_position.is_some_and(|point| {
|
|
node.node.contains_point(*node.transform, *point)
|
|
&& clip_check_recursive(*point, *entity, &clipping_query, &child_of_query)
|
|
});
|
|
|
|
// The mouse position relative to the node
|
|
// (-0.5, -0.5) is the top-left corner, (0.5, 0.5) is the bottom-right corner
|
|
// Coordinates are relative to the entire node, not just the visible region.
|
|
let normalized_cursor_position = cursor_position.and_then(|cursor_position| {
|
|
// ensure node size is non-zero in all dimensions, otherwise relative position will be
|
|
// +/-inf. if the node is hidden, the visible rect min/max will also be -inf leading to
|
|
// false positives for mouse_over (#12395)
|
|
node.node.normalize_point(*node.transform, *cursor_position)
|
|
});
|
|
|
|
// If the current cursor position is within the bounds of the node's visible area, consider it for
|
|
// clicking
|
|
let relative_cursor_position_component = RelativeCursorPosition {
|
|
cursor_over: contains_cursor,
|
|
normalized: normalized_cursor_position,
|
|
};
|
|
|
|
// Save the relative cursor position to the correct component
|
|
if let Some(mut node_relative_cursor_position_component) = node.relative_cursor_position
|
|
{
|
|
*node_relative_cursor_position_component = relative_cursor_position_component;
|
|
}
|
|
|
|
if contains_cursor {
|
|
Some(*entity)
|
|
} else {
|
|
if let Some(mut interaction) = node.interaction {
|
|
if *interaction == Interaction::Hovered
|
|
|| (normalized_cursor_position.is_none())
|
|
{
|
|
interaction.set_if_neq(Interaction::None);
|
|
}
|
|
}
|
|
None
|
|
}
|
|
})
|
|
.collect::<Vec<Entity>>()
|
|
.into_iter();
|
|
|
|
// set Pressed or Hovered on top nodes. as soon as a node with a `Block` focus policy is detected,
|
|
// the iteration will stop on it because it "captures" the interaction.
|
|
let mut iter = node_query.iter_many_mut(hovered_nodes.by_ref());
|
|
while let Some(node) = iter.fetch_next() {
|
|
if let Some(mut interaction) = node.interaction {
|
|
if mouse_clicked {
|
|
// only consider nodes with Interaction "pressed"
|
|
if *interaction != Interaction::Pressed {
|
|
*interaction = Interaction::Pressed;
|
|
// if the mouse was simultaneously released, reset this Interaction in the next
|
|
// frame
|
|
if mouse_released {
|
|
state.entities_to_reset.push(node.entity);
|
|
}
|
|
}
|
|
} else if *interaction == Interaction::None {
|
|
*interaction = Interaction::Hovered;
|
|
}
|
|
}
|
|
|
|
match node.focus_policy.unwrap_or(&FocusPolicy::Block) {
|
|
FocusPolicy::Block => {
|
|
break;
|
|
}
|
|
FocusPolicy::Pass => { /* allow the next node to be hovered/pressed */ }
|
|
}
|
|
}
|
|
// reset `Interaction` for the remaining lower nodes to `None`. those are the nodes that remain in
|
|
// `moused_over_nodes` after the previous loop is exited.
|
|
let mut iter = node_query.iter_many_mut(hovered_nodes);
|
|
while let Some(node) = iter.fetch_next() {
|
|
if let Some(mut interaction) = node.interaction {
|
|
// don't reset pressed nodes because they're handled separately
|
|
if *interaction != Interaction::Pressed {
|
|
interaction.set_if_neq(Interaction::None);
|
|
}
|
|
}
|
|
}
|
|
}
|