bevy/viewport-node.md at bf42cb3532b168fae222616b4351aeeb0c75587c

# Objective

Add a viewport widget.

## Solution

- Add a new `ViewportNode` component to turn a UI node into a viewport.
- Add `viewport_picking` to pass pointer inputs from other pointers to
the viewport's pointer.
- Notably, this is somewhat functionally different from the viewport
widget in [the editor
prototype](https://github.com/bevyengine/bevy_editor_prototypes/pull/110/files#L124),
which just moves the pointer's location onto the render target. Viewport
widgets have their own pointers.
  - Care is taken to handle dragging in and out of viewports.
- Add `update_viewport_render_target_size` to update the viewport node's
render target's size if the node size changes.
- Feature gate picking-related viewport items behind
`bevy_ui_picking_backend`.

## Testing

I've been using an example I made to test the widget (and added it as
`viewport_node`):

<details><summary>Code</summary>

```rust
//! A simple scene to demonstrate spawning a viewport widget. The example will demonstrate how to
//! pick entities visible in the widget's view.

use bevy::picking::pointer::PointerInteraction;
use bevy::prelude::*;

use bevy::ui::widget::ViewportNode;
use bevy::{
    image::{TextureFormatPixelInfo, Volume},
    window::PrimaryWindow,
};
use bevy_render::{
    camera::RenderTarget,
    render_resource::{
        Extent3d, TextureDescriptor, TextureDimension, TextureFormat, TextureUsages,
    },
};

fn main() {
    App::new()
        .add_plugins((DefaultPlugins, MeshPickingPlugin))
        .add_systems(Startup, test)
        .add_systems(Update, draw_mesh_intersections)
        .run();
}

#[derive(Component, Reflect, Debug)]
#[reflect(Component)]
struct Shape;

fn test(
    mut commands: Commands,
    window: Query<&Window, With<PrimaryWindow>>,
    mut images: ResMut<Assets<Image>>,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    // Spawn a UI camera
    commands.spawn(Camera3d::default());

    // Set up an texture for the 3D camera to render to
    let window = window.get_single().unwrap();
    let window_size = window.physical_size();
    let size = Extent3d {
        width: window_size.x,
        height: window_size.y,
        ..default()
    };
    let format = TextureFormat::Bgra8UnormSrgb;
    let image = Image {
        data: Some(vec![0; size.volume() * format.pixel_size()]),
        texture_descriptor: TextureDescriptor {
            label: None,
            size,
            dimension: TextureDimension::D2,
            format,
            mip_level_count: 1,
            sample_count: 1,
            usage: TextureUsages::TEXTURE_BINDING
                | TextureUsages::COPY_DST
                | TextureUsages::RENDER_ATTACHMENT,
            view_formats: &[],
        },
        ..default()
    };
    let image_handle = images.add(image);

    // Spawn the 3D camera
    let camera = commands
        .spawn((
            Camera3d::default(),
            Camera {
                // Render this camera before our UI camera
                order: -1,
                target: RenderTarget::Image(image_handle.clone().into()),
                ..default()
            },
        ))
        .id();

    // Spawn something for the 3D camera to look at
    commands
        .spawn((
            Mesh3d(meshes.add(Cuboid::new(5.0, 5.0, 5.0))),
            MeshMaterial3d(materials.add(Color::WHITE)),
            Transform::from_xyz(0.0, 0.0, -10.0),
            Shape,
        ))
        // We can observe pointer events on our objects as normal, the
        // `bevy::ui::widgets::viewport_picking` system will take care of ensuring our viewport
        // clicks pass through
        .observe(on_drag_cuboid);

    // Spawn our viewport widget
    commands
        .spawn((
            Node {
                position_type: PositionType::Absolute,
                top: Val::Px(50.0),
                left: Val::Px(50.0),
                width: Val::Px(200.0),
                height: Val::Px(200.0),
                border: UiRect::all(Val::Px(5.0)),
                ..default()
            },
            BorderColor(Color::WHITE),
            ViewportNode::new(camera),
        ))
        .observe(on_drag_viewport);
}

fn on_drag_viewport(drag: Trigger<Pointer<Drag>>, mut node_query: Query<&mut Node>) {
    if matches!(drag.button, PointerButton::Secondary) {
        let mut node = node_query.get_mut(drag.target()).unwrap();

        if let (Val::Px(top), Val::Px(left)) = (node.top, node.left) {
            node.left = Val::Px(left + drag.delta.x);
            node.top = Val::Px(top + drag.delta.y);
        };
    }
}

fn on_drag_cuboid(drag: Trigger<Pointer<Drag>>, mut transform_query: Query<&mut Transform>) {
    if matches!(drag.button, PointerButton::Primary) {
        let mut transform = transform_query.get_mut(drag.target()).unwrap();
        transform.rotate_y(drag.delta.x * 0.02);
        transform.rotate_x(drag.delta.y * 0.02);
    }
}

fn draw_mesh_intersections(
    pointers: Query<&PointerInteraction>,
    untargetable: Query<Entity, Without<Shape>>,
    mut gizmos: Gizmos,
) {
    for (point, normal) in pointers
        .iter()
        .flat_map(|interaction| interaction.iter())
        .filter_map(|(entity, hit)| {
            if !untargetable.contains(*entity) {
                hit.position.zip(hit.normal)
            } else {
                None
            }
        })
    {
        gizmos.arrow(point, point + normal.normalize() * 0.5, Color::WHITE);
    }
}
```

</details>

## Showcase


https://github.com/user-attachments/assets/39f44eac-2c2a-4fd9-a606-04171f806dc1

## Open Questions

- <del>Not sure whether the entire widget should be feature gated behind
`bevy_ui_picking_backend` or not? I chose a partial approach since maybe
someone will want to use the widget without any picking being
involved.</del>
- <del>Is `PickSet::Last` the expected set for `viewport_picking`?
Perhaps `PickSet::Input` is more suited.</del>
- <del>Can `dragged_last_frame` be removed in favor of a better dragging
check? Another option that comes to mind is reading `Drag` and `DragEnd`
events, but this seems messier.</del>

---------

Co-authored-by: ickshonpe <david.curthoys@googlemail.com>
Co-authored-by: François Mockers <mockersf@gmail.com>

2025-05-05 22:57:37 +00:00

1.7 KiB

Raw Blame History

title: ViewportNode authors: ["@chompaa", "@ickshonpe"] pull_requests: [17253]

Bevy UI now has a ViewportNode component, which lets you render camera output directly to a UI node. Furthermore, if the bevy_ui_picking_backend feature is enabled, you can pick using the rendered target. That is, you can use any picking backend through the viewport node, as per normal. In terms of UI, the API usage is really straightforward:

commands.spawn((
  // `ViewportNode` requires `Node`, so we just need this component!
  ViewportNode::new(camera)
  // To disable picking "through" the viewport, just disable picking for the node.
  // Pickable::IGNORE
));

The referenced camera here does require its target to be a RenderTarget::Image. See the new viewport_node for more implementation details.

Showcase

https://private-user-images.githubusercontent.com/26204416/402285264-39f44eac-2c2a-4fd9-a606-04171f806dc1.mp4?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDU4NTY4MDgsIm5iZiI6MTc0NTg1NjUwOCwicGF0aCI6Ii8yNjIwNDQxNi80MDIyODUyNjQtMzlmNDRlYWMtMmMyYS00ZmQ5LWE2MDYtMDQxNzFmODA2ZGMxLm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA0MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNDI4VDE2MDgyOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg0ZDU0OGFmM2Q3NTJmOWJkNDYzODMxNjkyOTBlYzFmNmQ2YWUzMGMzMjJjMjFiZWI0ZmY3ZjZkMjNiMzA5NzkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.DXec6l2SYDIpSCRssEB4o3er7ib3jUQ9t9fvjdY3hYw

1.7 KiB Raw Blame History

Showcase

1.7 KiB

Raw Blame History