bevy/crates/bevy_picking/src/events.rs
Miles Silberling-Cook 82128d778a
Picking event ordering (#14862)
# Objective

Correctly order picking events. Resolves
https://github.com/bevyengine/bevy/issues/5984.

## Solution

Event ordering [very long standing
problem](https://github.com/aevyrie/bevy_mod_picking/issues/294) with
mod picking, stemming from two related issues. The first problem was
that `Pointer<T>` events of different types couldn't be ordered, but we
have already gotten around that in the upstream by switching to
observers. Since observers run in the order they are triggered, this
isn't an issue.

The second problem was that the underlying event streams that picking
uses to create it's pointer interaction events *also* lacked ordering,
and the systems that generated the points couldn't interleave events.
This PR fixes that by unifying the event streams and integrating the
various interaction systems.

The concrete changes are as follows:
+ `bevy_winit::WinitEvent` has been moved to `bevy_window::WindowEvent`.
This provides a unified (and more importantly, *ordered*) input stream
for both `bevy_window` and `bevy_input` events.
+ Replaces `InputMove` and `InputPress` with `PointerInput`, a new
unified input event which drives picking and interaction. This event is
built to have drop-in forward compatibility with [winit's upcoming
pointer abstraction](https://github.com/rust-windowing/winit/pull/3876).
I have added code to emulate it using the current winit input
abstractions, but this entire thing will be much more robust when it
lands.
+ Rolls `pointer_events` `send_click_and_drag_events` and
`send_drag_over_events` into a single system, which operates directly on
`PointerEvent` and triggers observers as output.

The PR also improves docs and takes the opportunity to
refactor/streamline the pointer event dispatch logic.

## Status & Testing

This PR is now feature complete and documented. While it is
theoretically possible to add unit tests for the ordering, building the
picking mocking for that will take a little while.

Feedback on the chosen ordering of events is within-scope.

## Migration Guide

For users switching from `bevy_mod_picking` to `bevy_picking`:
+ Instead of adding an `On<T>` component, use `.observe(|trigger:
Trigger<T>|)`. You may now apply multiple handlers to the same entity
using this command.
+ Pointer interaction events now have semi-deterministic ordering which
(more or less) aligns with the order of the raw input stream. Consult
the docs on `bevy_picking::event::pointer_events` for current
information. You may need to adjust your event handling logic
accordingly.
+ `PointerCancel` has been replaced with `Pointer<Cancled>`, which now
has the semantics of an OS touch pointer cancel event.
+ `InputMove` and `InputPress` have been merged into `PointerInput`. The
use remains exactly the same.
+ Picking interaction events are now only accessible through observers,
and no `EventReader`. This functionality may be re-implemented later.

For users of `bevy_winit`:
+ The event `bevy_winit::WinitEvent` has moved to
`bevy_window::WindowEvent`. If this was the only thing you depended on
`bevy_winit` for, you should switch your dependency to `bevy_window`.
+ `bevy_window` now depends on `bevy_input`. The dependencies of
`bevy_input` are a subset of the existing dependencies for `bevy_window`
so this should be non-breaking.
2024-09-04 19:41:06 +00:00

623 lines
25 KiB
Rust

//! This module defines a stateful set of interaction events driven by the `PointerInput` stream
//! and the hover state of each Pointer.
//!
//! # Usage
//!
//! To receive events from this module, you must use an [`Observer`]
//! The simplest example, registering a callback when an entity is hovered over by a pointer, looks like this:
//!
//! ```rust
//! # use bevy_ecs::prelude::*;
//! # use bevy_picking::prelude::*;
//! # let mut world = World::default();
//! world.spawn_empty()
//! .observe(|trigger: Trigger<Pointer<Over>>| {
//! println!("I am being hovered over");
//! });
//! ```
//!
//! Observers give us three important properties:
//! 1. They allow for attaching event handlers to specific entities,
//! 2. they allow events to bubble up the entity hierarchy,
//! 3. and they allow events of different types to be called in a specific order.
//!
//! The order in which interaction events are received is extremely important, and you can read more
//! about it on the docs for the dispatcher system: [`pointer_events`]. This system runs in
//! [`PreUpdate`](bevy_app::PreUpdate) in [`PickSet::Focus`](crate::PickSet::Focus). All pointer-event
//! observers resolve during the sync point between [`pointer_events`] and
//! [`update_interactions`](crate::focus::update_interactions).
//!
//! # Events Types
//!
//! The events this module defines fall into a few broad categories:
//! + Hovering and movement: [`Over`], [`Move`], and [`Out`].
//! + Clicking and pressing: [`Down`], [`Up`], and [`Click`].
//! + Dragging and dropping: [`DragStart`], [`Drag`], [`DragEnd`], [`DragEnter`], [`DragOver`], [`DragDrop`], [`DragLeave`].
//!
//! When received by an observer, these events will always be wrapped by the [`Pointer`] type, which contains
//! general metadata about the pointer and it's location.
use std::fmt::Debug;
use bevy_ecs::prelude::*;
use bevy_hierarchy::Parent;
use bevy_math::Vec2;
use bevy_reflect::prelude::*;
use bevy_utils::{tracing::debug, Duration, HashMap, Instant};
use crate::{
backend::{prelude::PointerLocation, HitData},
focus::{HoverMap, PreviousHoverMap},
pointer::{
Location, PointerAction, PointerButton, PointerId, PointerInput, PointerMap, PressDirection,
},
};
/// Stores the common data needed for all pointer events.
///
/// The documentation for the [`pointer_events`] explains the events this module exposes and
/// the order in which they fire.
#[derive(Clone, PartialEq, Debug, Reflect, Component)]
pub struct Pointer<E: Debug + Clone + Reflect> {
/// The pointer that triggered this event
pub pointer_id: PointerId,
/// The location of the pointer during this event
pub pointer_location: Location,
/// Additional event-specific data. [`DragDrop`] for example, has an additional field to describe
/// the `Entity` that is being dropped on the target.
pub event: E,
}
impl<E> Event for Pointer<E>
where
E: Debug + Clone + Reflect,
{
type Traversal = Parent;
const AUTO_PROPAGATE: bool = true;
}
impl<E: Debug + Clone + Reflect> std::fmt::Display for Pointer<E> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_fmt(format_args!(
"{:?}, {:.1?}, {:.1?}",
self.pointer_id, self.pointer_location.position, self.event
))
}
}
impl<E: Debug + Clone + Reflect> std::ops::Deref for Pointer<E> {
type Target = E;
fn deref(&self) -> &Self::Target {
&self.event
}
}
impl<E: Debug + Clone + Reflect> Pointer<E> {
/// Construct a new `Pointer<E>` event.
pub fn new(id: PointerId, location: Location, event: E) -> Self {
Self {
pointer_id: id,
pointer_location: location,
event,
}
}
}
/// Fires when a pointer is canceled, and it's current interaction state is dropped.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Cancel {
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires when a the pointer crosses into the bounds of the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Over {
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires when a the pointer crosses out of the bounds of the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Out {
/// Information about the latest prior picking intersection.
pub hit: HitData,
}
/// Fires when a pointer button is pressed over the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Down {
/// Pointer button pressed to trigger this event.
pub button: PointerButton,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires when a pointer button is released over the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Up {
/// Pointer button lifted to trigger this event.
pub button: PointerButton,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires when a pointer sends a pointer down event followed by a pointer up event, with the same
/// `target` entity for both events.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Click {
/// Pointer button pressed and lifted to trigger this event.
pub button: PointerButton,
/// Information about the picking intersection.
pub hit: HitData,
/// Duration between the pointer pressed and lifted for this click
pub duration: Duration,
}
/// Fires while a pointer is moving over the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Move {
/// Information about the picking intersection.
pub hit: HitData,
/// The change in position since the last move event.
pub delta: Vec2,
}
/// Fires when the `target` entity receives a pointer down event followed by a pointer move event.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragStart {
/// Pointer button pressed and moved to trigger this event.
pub button: PointerButton,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires while the `target` entity is being dragged.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct Drag {
/// Pointer button pressed and moved to trigger this event.
pub button: PointerButton,
/// The total distance vector of a drag, measured from drag start to the current position.
pub distance: Vec2,
/// The change in position since the last drag event.
pub delta: Vec2,
}
/// Fires when a pointer is dragging the `target` entity and a pointer up event is received.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragEnd {
/// Pointer button pressed, moved, and lifted to trigger this event.
pub button: PointerButton,
/// The vector of drag movement measured from start to final pointer position.
pub distance: Vec2,
}
/// Fires when a pointer dragging the `dragged` entity enters the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragEnter {
/// Pointer button pressed to enter drag.
pub button: PointerButton,
/// The entity that was being dragged when the pointer entered the `target` entity.
pub dragged: Entity,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires while the `dragged` entity is being dragged over the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragOver {
/// Pointer button pressed while dragging over.
pub button: PointerButton,
/// The entity that was being dragged when the pointer was over the `target` entity.
pub dragged: Entity,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Fires when a pointer dragging the `dragged` entity leaves the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragLeave {
/// Pointer button pressed while leaving drag.
pub button: PointerButton,
/// The entity that was being dragged when the pointer left the `target` entity.
pub dragged: Entity,
/// Information about the latest prior picking intersection.
pub hit: HitData,
}
/// Fires when a pointer drops the `dropped` entity onto the `target` entity.
#[derive(Clone, PartialEq, Debug, Reflect)]
pub struct DragDrop {
/// Pointer button lifted to drop.
pub button: PointerButton,
/// The entity that was dropped onto the `target` entity.
pub dropped: Entity,
/// Information about the picking intersection.
pub hit: HitData,
}
/// Dragging state.
#[derive(Debug, Clone)]
pub struct DragEntry {
/// The position of the pointer at drag start.
pub start_pos: Vec2,
/// The latest position of the pointer during this drag, used to compute deltas.
pub latest_pos: Vec2,
}
/// An entry in the cache that drives the `pointer_events` system, storing additional data
/// about pointer button presses.
#[derive(Debug, Clone, Default)]
pub struct PointerState {
/// Stores the press location and start time for each button currently being pressed by the pointer.
pub pressing: HashMap<Entity, (Location, Instant)>,
/// Stores the the starting and current locations for each entity currently being dragged by the pointer.
pub dragging: HashMap<Entity, DragEntry>,
/// Stores the hit data for each entity currently being dragged over by the pointer.
pub dragging_over: HashMap<Entity, HitData>,
}
/// Dispatches interaction events to the target entities.
///
/// Within a single frame, events are dispatched in the following order:
/// + The sequence [`DragEnter`], [`Over`].
/// + Any number of any of the following:
/// + For each movement: The sequence [`DragStart`], [`Drag`], [`DragOver`], [`Move`].
/// + For each button press: Either [`Down`], or the sequence [`DragDrop`], [`DragEnd`], [`DragLeave`], [`Click`], [`Up`].
/// + For each pointer cancellation: Simply [`Cancel`].
/// + Finally the sequence [`DragLeave`], [`Out`].
///
/// Only the last event in a given sequence is garenteed to be present.
///
/// Additionally, across multiple frames, the following are also strictly ordered by the interaction state machine:
/// + When a pointer moves over the target: [`Over`], [`Move`], [`Out`].
/// + When a pointer presses buttons on the target: [`Down`], [`Up`], [`Click`].
/// + When a pointer drags the target: [`DragStart`], [`Drag`], [`DragEnd`].
/// + When a pointer drags something over the target: [`DragEnter`], [`DragOver`], [`DragDrop`], [`DragLeave`].
/// + When a pointer is canceled: No other events will follow the [`Cancel`] event for that pointer.
///
/// Two events -- [`Over`] and [`Out`] -- are driven only by the [`HoverMap`]. The rest rely on additional data from the
/// [`PointerInput`] event stream. To receive these events for a custom pointer, you must add [`PointerInput`] events.
///
/// Note: Though it is common for the [`PointerInput`] stream may contain multiple pointer movements and presses each frame,
/// the hover state is determined only by the pointer's *final position*. Since the hover state ultimately determines which
/// entities receive events, this may mean that an entity can receive events which occurred before it was actually hovered.
#[allow(clippy::too_many_arguments)]
pub fn pointer_events(
// Input
mut input_events: EventReader<PointerInput>,
// ECS State
pointers: Query<&PointerLocation>,
pointer_map: Res<PointerMap>,
hover_map: Res<HoverMap>,
previous_hover_map: Res<PreviousHoverMap>,
// Local state
mut pointer_state: Local<HashMap<(PointerId, PointerButton), PointerState>>,
// Output
mut commands: Commands,
) {
// Setup utilities
let now = Instant::now();
let pointer_location = |pointer_id: PointerId| {
pointer_map
.get_entity(pointer_id)
.and_then(|entity| pointers.get(entity).ok())
.and_then(|pointer| pointer.location.clone())
};
// If the entity is hovered...
for (pointer_id, hovered_entity, hit) in hover_map
.iter()
.flat_map(|(id, hashmap)| hashmap.iter().map(|data| (*id, *data.0, data.1.clone())))
{
// ...but was not hovered last frame...
if !previous_hover_map
.get(&pointer_id)
.iter()
.any(|e| e.contains_key(&hovered_entity))
{
let Some(location) = pointer_location(pointer_id) else {
debug!(
"Unable to get location for pointer {:?} during pointer over",
pointer_id
);
continue;
};
// Possibly send DragEnter events
for button in PointerButton::iter() {
let state = pointer_state.entry((pointer_id, button)).or_default();
for drag_target in state
.dragging
.keys()
.filter(|&&drag_target| hovered_entity != drag_target)
{
state.dragging_over.insert(hovered_entity, hit.clone());
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragEnter {
button,
dragged: *drag_target,
hit: hit.clone(),
},
),
hovered_entity,
);
}
}
// Always send Over events
commands.trigger_targets(
Pointer::new(pointer_id, location.clone(), Over { hit: hit.clone() }),
hovered_entity,
);
}
}
// Dispatch input events...
for PointerInput {
pointer_id,
location,
action,
} in input_events.read().cloned()
{
match action {
// Pressed Button
PointerAction::Pressed { direction, button } => {
let state = pointer_state.entry((pointer_id, button)).or_default();
// Possibly emit DragEnd, DragDrop, DragLeave on button releases
if direction == PressDirection::Up {
// For each currently dragged entity
for (drag_target, drag) in state.dragging.drain() {
// Emit DragDrop
for (dragged_over, hit) in state.dragging_over.iter() {
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragDrop {
button,
dropped: drag_target,
hit: hit.clone(),
},
),
*dragged_over,
);
}
// Emit DragEnd
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragEnd {
button,
distance: drag.latest_pos - drag.start_pos,
},
),
drag_target,
);
// Emit DragLeave
for (dragged_over, hit) in state.dragging_over.iter() {
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragLeave {
button,
dragged: drag_target,
hit: hit.clone(),
},
),
*dragged_over,
);
}
}
}
// Send a Down or possibly a Click and an Up button events
for (hovered_entity, hit) in previous_hover_map
.get(&pointer_id)
.iter()
.flat_map(|h| h.iter().map(|(entity, data)| (*entity, data.clone())))
{
match direction {
PressDirection::Down => {
// Send the Down event first
let event =
Pointer::new(pointer_id, location.clone(), Down { button, hit });
commands.trigger_targets(event.clone(), hovered_entity);
// Also insert the press into the state
state
.pressing
.insert(hovered_entity, (event.pointer_location, now));
}
PressDirection::Up => {
// If this pointer previously pressed the hovered entity, first send a Click event
if let Some((_location, press_instant)) =
state.pressing.get(&hovered_entity)
{
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
Click {
button,
hit: hit.clone(),
duration: now - *press_instant,
},
),
hovered_entity,
);
}
// Always send the Up event
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
Up {
button,
hit: hit.clone(),
},
),
hovered_entity,
);
// Also clear the state
state.pressing.clear();
state.dragging.clear();
state.dragging_over.clear();
}
};
}
}
// Moved
PointerAction::Moved { delta } => {
for (hovered_entity, hit) in hover_map
.get(&pointer_id)
.iter()
.flat_map(|h| h.iter().map(|(entity, data)| (*entity, data.to_owned())))
{
// Send drag events to entities being dragged or dragged over
for button in PointerButton::iter() {
let state = pointer_state.entry((pointer_id, button)).or_default();
// Emit a DragStart the first time the pointer moves while pressing an entity
for (location, _instant) in state.pressing.values() {
if state.dragging.contains_key(&hovered_entity) {
continue; // this entity is already logged as being dragged
}
state.dragging.insert(
hovered_entity,
DragEntry {
start_pos: location.position,
latest_pos: location.position,
},
);
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragStart {
button,
hit: hit.clone(),
},
),
hovered_entity,
);
}
// Emit a Drag event to the dragged entity when it is dragged over another entity.
for (dragged_entity, drag) in state.dragging.iter_mut() {
let drag_event = Drag {
button,
distance: location.position - drag.start_pos,
delta: location.position - drag.latest_pos,
};
drag.latest_pos = location.position;
let target = *dragged_entity;
let event = Pointer::new(pointer_id, location.clone(), drag_event);
commands.trigger_targets(event, target);
}
// Emit a DragOver to the hovered entity when dragging a different entity over it.
for drag_target in state.dragging.keys()
.filter(
|&&drag_target| hovered_entity != drag_target, /* can't drag over itself */
)
{
commands.trigger_targets(
Pointer::new(pointer_id, location.clone(), DragOver { button, dragged: *drag_target, hit: hit.clone() }),
hovered_entity,
);
}
}
// Always send Move event
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
Move {
hit: hit.clone(),
delta,
},
),
hovered_entity,
);
}
}
// Canceled
PointerAction::Canceled => {
// Emit a Cancel to the hovered entity.
for (hovered_entity, hit) in hover_map
.get(&pointer_id)
.iter()
.flat_map(|h| h.iter().map(|(entity, data)| (*entity, data.to_owned())))
{
commands.trigger_targets(
Pointer::new(pointer_id, location.clone(), Cancel { hit }),
hovered_entity,
);
}
// Clear the local state for the canceled pointer
for button in PointerButton::iter() {
if let Some(state) = pointer_state.get_mut(&(pointer_id, button)) {
state.pressing.clear();
state.dragging.clear();
state.dragging_over.clear();
}
}
}
}
}
// If the entity was hovered by a specific pointer last frame...
for (pointer_id, hovered_entity, hit) in previous_hover_map
.iter()
.flat_map(|(id, hashmap)| hashmap.iter().map(|data| (*id, *data.0, data.1.clone())))
{
// ...but is now not being hovered by that same pointer...
if !hover_map
.get(&pointer_id)
.iter()
.any(|e| e.contains_key(&hovered_entity))
{
let Some(location) = pointer_location(pointer_id) else {
debug!(
"Unable to get location for pointer {:?} during pointer out",
pointer_id
);
continue;
};
// Possibly send DragLeave events
for button in PointerButton::iter() {
let state = pointer_state.entry((pointer_id, button)).or_default();
state.dragging_over.remove(&hovered_entity);
for drag_target in state.dragging.keys() {
commands.trigger_targets(
Pointer::new(
pointer_id,
location.clone(),
DragLeave {
button,
dragged: *drag_target,
hit: hit.clone(),
},
),
hovered_entity,
);
}
}
// Always send Out events
commands.trigger_targets(
Pointer::new(pointer_id, location.clone(), Out { hit: hit.clone() }),
hovered_entity,
);
}
}
}