forestia/bevy
2
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Rework ViewTarget to better support post processing (#6415)

Browse Source 
# Objective

Post processing effects cannot read and write to the same texture. Currently they must own their own intermediate texture and redundantly copy from that back to the main texture. This is very inefficient.

Additionally, working with ViewTarget is more complicated than it needs to be, especially when working with HDR textures. 

## Solution

`ViewTarget` now stores two copies of the "main texture". It uses an atomic value to track which is currently the "main texture" (this interior mutability is necessary to accommodate read-only RenderGraph execution). 

`ViewTarget` now has a `post_process_write` method, which will return a source and destination texture. Each call to this method will flip between the two copies of the "main texture".

```rust
let post_process = render_target.post_process_write();
let source_texture = post_process.source;
let destination_texture = post_process.destination;
```
The caller _must_ read from the source texture and write to the destination texture, as it is assumed that the destination texture will become the new "main texture".


For simplicity / understandability `ViewTarget` is now a flat type. "hdr-ness" is a property of the `TextureFormat`. The internals are fully private in the interest of providing simple / consistent apis. Developers can now easily access the main texture by calling `view_target.main_texture()`.

HDR ViewTargets no longer have an "ldr texture" with `TextureFormat::bevy_default`. They _only_ have their two "hdr" textures. This simplifies the mental model.  All we have is the "currently active hdr texture" and the "other hdr texture", which we flip between for post processing effects.

The tonemapping node has been rephrased to use this "post processing pattern". The blit pass has been removed, and it now only runs a pass when HDR is enabled. Notably, both the input and output texture are assumed to be HDR. This means that tonemapping behaves just like any other "post processing effect". It could theoretically be moved anywhere in the "effect chain" and continue to work.

In general, I think these changes will make the lives of people making post processing effects much easier. And they better position us to start building higher level / more structured  "post processing effect stacks".

---

## Changelog

- `ViewTarget` now stores two copies of the "main texture". Calling `ViewTarget::post_process_write` will flip between copies of the main texture.
This commit is contained in:
Carter Anderson 2022-10-31 20:22:18 +00:00
parent 89b3422f62
commit b6e46a73cd
7 changed files with 182 additions and 200 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
crates/bevy_core_pipeline/src/tonemapping/blit.wgsl
View File

@ -1,11 +0,0 @@
#import bevy_core_pipeline::fullscreen_vertex_shader
@group(0) @binding(0)
var texture: texture_2d<f32>;
@group(0) @binding(1)
var texture_sampler: sampler;
@fragment
fn fs_main(in: FullscreenVertexOutput) -> @location(0) vec4<f32> {
return textureSample(texture, texture_sampler, in.uv);
}

48
crates/bevy_core_pipeline/src/tonemapping/mod.rs
View File

@ -1,20 +1,18 @@
mod node;
use bevy_ecs::query::QueryItem;
use bevy_render::camera::Camera;
use bevy_render::extract_component::{ExtractComponent, ExtractComponentPlugin};
pub use node::TonemappingNode;
use crate::fullscreen_vertex_shader::fullscreen_shader_vertex_state;
use bevy_app::prelude::*;
use bevy_asset::{load_internal_asset, HandleUntyped};
use bevy_ecs::prelude::*;
use bevy_render::renderer::RenderDevice;
use bevy_render::texture::BevyDefault;
use bevy_render::{render_resource::*, RenderApp};
use bevy_ecs::query::QueryItem;
use bevy_reflect::TypeUuid;
use crate::fullscreen_vertex_shader::fullscreen_shader_vertex_state;
use bevy_render::camera::Camera;
use bevy_render::extract_component::{ExtractComponent, ExtractComponentPlugin};
use bevy_render::renderer::RenderDevice;
use bevy_render::view::ViewTarget;
use bevy_render::{render_resource::*, RenderApp};
const TONEMAPPING_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 17015368199668024512);
@ -22,9 +20,6 @@ const TONEMAPPING_SHADER_HANDLE: HandleUntyped =
const TONEMAPPING_SHARED_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2499430578245347910);
const BLIT_SHADER_HANDLE: HandleUntyped =
HandleUntyped::weak_from_u64(Shader::TYPE_UUID, 2982361071241723543);
pub struct TonemappingPlugin;
impl Plugin for TonemappingPlugin {
@ -41,7 +36,6 @@ impl Plugin for TonemappingPlugin {
"tonemapping_shared.wgsl",
Shader::from_wgsl
);
load_internal_asset!(app, BLIT_SHADER_HANDLE, "blit.wgsl", Shader::from_wgsl);
app.add_plugin(ExtractComponentPlugin::<Tonemapping>::default());
@ -53,9 +47,8 @@ impl Plugin for TonemappingPlugin {
#[derive(Resource)]
pub struct TonemappingPipeline {
hdr_texture_bind_group: BindGroupLayout,
texture_bind_group: BindGroupLayout,
tonemapping_pipeline_id: CachedRenderPipelineId,
blit_pipeline_id: CachedRenderPipelineId,
}
impl FromWorld for TonemappingPipeline {
@ -91,9 +84,9 @@ impl FromWorld for TonemappingPipeline {
fragment: Some(FragmentState {
shader: TONEMAPPING_SHADER_HANDLE.typed(),
shader_defs: vec![],
entry_point: "fs_main".into(),
entry_point: "fragment".into(),
targets: vec![Some(ColorTargetState {
format: TextureFormat::bevy_default(),
format: ViewTarget::TEXTURE_FORMAT_HDR,
blend: None,
write_mask: ColorWrites::ALL,
})],
@ -103,29 +96,10 @@ impl FromWorld for TonemappingPipeline {
multisample: MultisampleState::default(),
};
let blit_descriptor = RenderPipelineDescriptor {
label: Some("blit pipeline".into()),
layout: Some(vec![tonemap_texture_bind_group.clone()]),
vertex: fullscreen_shader_vertex_state(),
fragment: Some(FragmentState {
shader: BLIT_SHADER_HANDLE.typed(),
shader_defs: vec![],
entry_point: "fs_main".into(),
targets: vec![Some(ColorTargetState {
format: TextureFormat::bevy_default(),
blend: None,
write_mask: ColorWrites::ALL,
})],
}),
primitive: PrimitiveState::default(),
depth_stencil: None,
multisample: MultisampleState::default(),
};
let mut cache = render_world.resource_mut::<PipelineCache>();
TonemappingPipeline {
hdr_texture_bind_group: tonemap_texture_bind_group,
texture_bind_group: tonemap_texture_bind_group,
tonemapping_pipeline_id: cache.queue_render_pipeline(tonemap_descriptor),
blit_pipeline_id: cache.queue_render_pipeline(blit_descriptor),
}
}
}

36
crates/bevy_core_pipeline/src/tonemapping/node.rs
View File

@ -11,7 +11,7 @@ use bevy_render::{
TextureViewId,
},
renderer::RenderContext,
view::{ExtractedView, ViewMainTexture, ViewTarget},
view::{ExtractedView, ViewTarget},
};
pub struct TonemappingNode {
@ -54,31 +54,25 @@ impl Node for TonemappingNode {
Err(_) => return Ok(()),
};
let ldr_texture = match &target.main_texture {
ViewMainTexture::Hdr { ldr_texture, .. } => ldr_texture,
ViewMainTexture::Sdr { .. } => {
// non-hdr does tone mapping in the main pass node
return Ok(());
}
};
let tonemapping_enabled = tonemapping.map_or(false, |t| t.is_enabled);
let pipeline_id = if tonemapping_enabled {
tonemapping_pipeline.tonemapping_pipeline_id
} else {
tonemapping_pipeline.blit_pipeline_id
};
if !tonemapping_enabled || !target.is_hdr() {
return Ok(());
}
let pipeline = match pipeline_cache.get_render_pipeline(pipeline_id) {
let pipeline = match pipeline_cache
.get_render_pipeline(tonemapping_pipeline.tonemapping_pipeline_id)
{
Some(pipeline) => pipeline,
None => return Ok(()),
};
let main_texture = target.main_texture.texture();
let post_process = target.post_process_write();
let source = post_process.source;
let destination = post_process.destination;
let mut cached_bind_group = self.cached_texture_bind_group.lock().unwrap();
let bind_group = match &mut *cached_bind_group {
Some((id, bind_group)) if main_texture.id() == *id => bind_group,
Some((id, bind_group)) if source.id() == *id => bind_group,
cached_bind_group => {
let sampler = render_context
.render_device
@ -89,11 +83,11 @@ impl Node for TonemappingNode {
.render_device
.create_bind_group(&BindGroupDescriptor {
label: None,
layout: &tonemapping_pipeline.hdr_texture_bind_group,
layout: &tonemapping_pipeline.texture_bind_group,
entries: &[
BindGroupEntry {
binding: 0,
resource: BindingResource::TextureView(main_texture),
resource: BindingResource::TextureView(source),
},
BindGroupEntry {
binding: 1,
@ -102,7 +96,7 @@ impl Node for TonemappingNode {
],
});
let (_, bind_group) = cached_bind_group.insert((main_texture.id(), bind_group));
let (_, bind_group) = cached_bind_group.insert((source.id(), bind_group));
bind_group
}
};
@ -110,7 +104,7 @@ impl Node for TonemappingNode {
let pass_descriptor = RenderPassDescriptor {
label: Some("tonemapping_pass"),
color_attachments: &[Some(RenderPassColorAttachment {
view: ldr_texture,
view: destination,
resolve_target: None,
ops: Operations {
load: LoadOp::Clear(Default::default()), // TODO shouldn't need to be cleared

2
crates/bevy_core_pipeline/src/tonemapping/tonemapping.wgsl
View File

@ -7,7 +7,7 @@ var hdr_texture: texture_2d<f32>;
var hdr_sampler: sampler;
@fragment
fn fs_main(in: FullscreenVertexOutput) -> @location(0) vec4<f32> {
fn fragment(in: FullscreenVertexOutput) -> @location(0) vec4<f32> {
let hdr_color = textureSample(hdr_texture, hdr_sampler, in.uv);
return vec4<f32>(reinhard_luminance(hdr_color.rgb), hdr_color.a);

14
crates/bevy_core_pipeline/src/upscaling/mod.rs
View File

@ -38,16 +38,16 @@ impl Plugin for UpscalingPlugin {
#[derive(Resource)]
pub struct UpscalingPipeline {
ldr_texture_bind_group: BindGroupLayout,
texture_bind_group: BindGroupLayout,
}
impl FromWorld for UpscalingPipeline {
fn from_world(render_world: &mut World) -> Self {
let render_device = render_world.resource::<RenderDevice>();
let ldr_texture_bind_group =
let texture_bind_group =
render_device.create_bind_group_layout(&BindGroupLayoutDescriptor {
label: Some("upscaling_ldr_texture_bind_group_layout"),
label: Some("upscaling_texture_bind_group_layout"),
entries: &[
BindGroupLayoutEntry {
binding: 0,
@ -68,9 +68,7 @@ impl FromWorld for UpscalingPipeline {
],
});
UpscalingPipeline {
ldr_texture_bind_group,
}
UpscalingPipeline { texture_bind_group }
}
}
@ -92,7 +90,7 @@ impl SpecializedRenderPipeline for UpscalingPipeline {
fn specialize(&self, key: Self::Key) -> RenderPipelineDescriptor {
RenderPipelineDescriptor {
label: Some("upscaling pipeline".into()),
layout: Some(vec![self.ldr_texture_bind_group.clone()]),
layout: Some(vec![self.texture_bind_group.clone()]),
vertex: fullscreen_shader_vertex_state(),
fragment: Some(FragmentState {
shader: UPSCALING_SHADER_HANDLE.typed(),
@ -126,7 +124,7 @@ fn queue_upscaling_bind_groups(
for (entity, view_target) in view_targets.iter() {
let key = UpscalingPipelineKey {
upscaling_mode: UpscalingMode::Filtering,
texture_format: view_target.out_texture_format,
texture_format: view_target.out_texture_format(),
};
let pipeline = pipelines.specialize(&mut pipeline_cache, &upscaling_pipeline, key);

11
crates/bevy_core_pipeline/src/upscaling/node.rs
View File

@ -56,10 +56,7 @@ impl Node for UpscalingNode {
Err(_) => return Ok(()),
};
let upscaled_texture = match &target.main_texture {
bevy_render::view::ViewMainTexture::Hdr { ldr_texture, .. } => ldr_texture,
bevy_render::view::ViewMainTexture::Sdr { texture, .. } => texture,
};
let upscaled_texture = target.main_texture();
let mut cached_bind_group = self.cached_texture_bind_group.lock().unwrap();
let bind_group = match &mut *cached_bind_group {
@ -74,7 +71,7 @@ impl Node for UpscalingNode {
.render_device
.create_bind_group(&BindGroupDescriptor {
label: None,
layout: &upscaling_pipeline.ldr_texture_bind_group,
layout: &upscaling_pipeline.texture_bind_group,
entries: &[
BindGroupEntry {
binding: 0,
@ -100,10 +97,10 @@ impl Node for UpscalingNode {
let pass_descriptor = RenderPassDescriptor {
label: Some("upscaling_pass"),
color_attachments: &[Some(RenderPassColorAttachment {
view: &target.out_texture,
view: target.out_texture(),
resolve_target: None,
ops: Operations {
load: LoadOp::Clear(Default::default()), // TODO dont_care
load: LoadOp::Clear(Default::default()),
store: true,
},
})],

260
crates/bevy_render/src/view/mod.rs
View File

@ -1,12 +1,7 @@
pub mod visibility;
pub mod window;
use bevy_utils::HashMap;
pub use visibility::*;
use wgpu::{
Color, Extent3d, Operations, RenderPassColorAttachment, TextureDescriptor, TextureDimension,
TextureFormat, TextureUsages,
};
pub use window::*;
use crate::{
@ -25,6 +20,12 @@ use bevy_ecs::prelude::*;
use bevy_math::{Mat4, UVec4, Vec3, Vec4};
use bevy_reflect::Reflect;
use bevy_transform::components::GlobalTransform;
use bevy_utils::HashMap;
use std::sync::atomic::{AtomicUsize, Ordering};
use wgpu::{
Color, Extent3d, Operations, RenderPassColorAttachment, TextureDescriptor, TextureDimension,
TextureFormat, TextureUsages,
};
pub struct ViewPlugin;
@ -116,78 +117,108 @@ pub struct ViewUniformOffset {
pub offset: u32,
}
#[derive(Clone)]
pub enum ViewMainTexture {
Hdr {
hdr_texture: TextureView,
sampled_hdr_texture: Option<TextureView>,
ldr_texture: TextureView,
},
Sdr {
texture: TextureView,
sampled_texture: Option<TextureView>,
},
}
impl ViewMainTexture {
pub fn texture(&self) -> &TextureView {
match self {
ViewMainTexture::Hdr { hdr_texture, .. } => hdr_texture,
ViewMainTexture::Sdr { texture, .. } => texture,
}
}
}
#[derive(Component)]
pub struct ViewTarget {
pub main_texture: ViewMainTexture,
pub out_texture: TextureView,
pub out_texture_format: TextureFormat,
main_textures: MainTargetTextures,
main_texture_format: TextureFormat,
/// 0 represents `main_textures.a`, 1 represents `main_textures.b`
main_texture: AtomicUsize,
out_texture: TextureView,
out_texture_format: TextureFormat,
}
pub struct PostProcessWrite<'a> {
pub source: &'a TextureView,
pub destination: &'a TextureView,
}
impl ViewTarget {
pub const TEXTURE_FORMAT_HDR: TextureFormat = TextureFormat::Rgba16Float;
/// Retrieve this target's color attachment. This will use [`Self::sampled_main_texture`] and resolve to [`Self::main_texture`] if
/// the target has sampling enabled. Otherwise it will use [`Self::main_texture`] directly.
pub fn get_color_attachment(&self, ops: Operations<Color>) -> RenderPassColorAttachment {
let (target, sampled) = match &self.main_texture {
ViewMainTexture::Hdr {
hdr_texture,
sampled_hdr_texture,
..
} => (hdr_texture, sampled_hdr_texture),
ViewMainTexture::Sdr {
texture,
sampled_texture,
} => (texture, sampled_texture),
};
match sampled {
Some(sampled_target) => RenderPassColorAttachment {
view: sampled_target,
resolve_target: Some(target),
ops,
},
None => RenderPassColorAttachment {
view: target,
resolve_target: None,
match &self.main_textures.sampled {
Some(sampled_texture) => RenderPassColorAttachment {
view: sampled_texture,
resolve_target: Some(self.main_texture()),
ops,
},
None => self.get_unsampled_color_attachment(ops),
}
}
/// Retrieve an "unsampled" color attachment using [`Self::main_texture`].
pub fn get_unsampled_color_attachment(
&self,
ops: Operations<Color>,
) -> RenderPassColorAttachment {
RenderPassColorAttachment {
view: match &self.main_texture {
ViewMainTexture::Hdr { hdr_texture, .. } => hdr_texture,
ViewMainTexture::Sdr { texture, .. } => texture,
},
view: self.main_texture(),
resolve_target: None,
ops,
}
}
/// The "main" unsampled texture.
pub fn main_texture(&self) -> &TextureView {
if self.main_texture.load(Ordering::SeqCst) == 0 {
&self.main_textures.a
} else {
&self.main_textures.b
}
}
/// The "main" sampled texture.
pub fn sampled_main_texture(&self) -> Option<&TextureView> {
self.main_textures.sampled.as_ref()
}
#[inline]
pub fn main_texture_format(&self) -> TextureFormat {
self.main_texture_format
}
/// Returns `true` if and only if the main texture is [`Self::TEXTURE_FORMAT_HDR`]
#[inline]
pub fn is_hdr(&self) -> bool {
self.main_texture_format == ViewTarget::TEXTURE_FORMAT_HDR
}
/// The final texture this view will render to.
#[inline]
pub fn out_texture(&self) -> &TextureView {
&self.out_texture
}
/// The format of the final texture this view will render to
#[inline]
pub fn out_texture_format(&self) -> TextureFormat {
self.out_texture_format
}
/// This will start a new "post process write", which assumes that the caller
/// will write the [`PostProcessWrite`]'s `source` to the `destination`.
///
/// `source` is the "current" main texture. This will internally flip this
/// [`ViewTarget`]'s main texture to the `destination` texture, so the caller
/// _must_ ensure `source` is copied to `destination`, with or without modifications.
/// Failing to do so will cause the current main texture information to be lost.
pub fn post_process_write(&self) -> PostProcessWrite {
let old_is_a_main_texture = self.main_texture.fetch_xor(1, Ordering::SeqCst);
// if the old main texture is a, then the post processing must write from a to b
if old_is_a_main_texture == 0 {
PostProcessWrite {
source: &self.main_textures.a,
destination: &self.main_textures.b,
}
} else {
PostProcessWrite {
source: &self.main_textures.b,
destination: &self.main_textures.a,
}
}
}
}
#[derive(Component)]
@ -230,6 +261,13 @@ fn prepare_view_uniforms(
.write_buffer(&render_device, &render_queue);
}
#[derive(Clone)]
struct MainTargetTextures {
a: TextureView,
b: TextureView,
sampled: Option<TextureView>,
}
#[allow(clippy::too_many_arguments)]
fn prepare_view_targets(
mut commands: Commands,
@ -243,7 +281,7 @@ fn prepare_view_targets(
let mut textures = HashMap::default();
for (entity, camera, view) in cameras.iter() {
if let Some(target_size) = camera.physical_target_size {
if let (Some(texture_view), Some(texture_format)) = (
if let (Some(out_texture_view), Some(out_texture_format)) = (
camera.target.get_texture_view(&windows, &images),
camera.target.get_texture_format(&windows, &images),
) {
@ -253,77 +291,69 @@ fn prepare_view_targets(
depth_or_array_layers: 1,
};
let main_texture = textures
let main_texture_format = if view.hdr {
ViewTarget::TEXTURE_FORMAT_HDR
} else {
TextureFormat::bevy_default()
};
let main_textures = textures
.entry((camera.target.clone(), view.hdr))
.or_insert_with(|| {
let main_texture_format = if view.hdr {
ViewTarget::TEXTURE_FORMAT_HDR
} else {
TextureFormat::bevy_default()
let descriptor = TextureDescriptor {
label: None,
size,
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format: main_texture_format,
usage: TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
};
let main_texture = texture_cache.get(
&render_device,
TextureDescriptor {
label: Some("main_texture"),
size,
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format: main_texture_format,
usage: TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
},
);
let sampled_main_texture = (msaa.samples > 1).then(|| {
texture_cache
MainTargetTextures {
a: texture_cache
.get(
&render_device,
TextureDescriptor {
label: Some("main_texture_sampled"),
size,
mip_level_count: 1,
sample_count: msaa.samples,
dimension: TextureDimension::D2,
format: main_texture_format,
usage: TextureUsages::RENDER_ATTACHMENT,
label: Some("main_texture_a"),
..descriptor
},
)
.default_view
});
if view.hdr {
let ldr_texture = texture_cache.get(
&render_device,
TextureDescriptor {
label: Some("ldr_texture"),
size,
mip_level_count: 1,
sample_count: 1,
dimension: TextureDimension::D2,
format: TextureFormat::bevy_default(),
usage: TextureUsages::RENDER_ATTACHMENT
| TextureUsages::TEXTURE_BINDING,
},
);
ViewMainTexture::Hdr {
hdr_texture: main_texture.default_view,
sampled_hdr_texture: sampled_main_texture,
ldr_texture: ldr_texture.default_view,
}
} else {
ViewMainTexture::Sdr {
texture: main_texture.default_view,
sampled_texture: sampled_main_texture,
}
.default_view,
b: texture_cache
.get(
&render_device,
TextureDescriptor {
label: Some("main_texture_b"),
..descriptor
},
)
.default_view,
sampled: (msaa.samples > 1).then(|| {
texture_cache
.get(
&render_device,
TextureDescriptor {
label: Some("main_texture_sampled"),
size,
mip_level_count: 1,
sample_count: msaa.samples,
dimension: TextureDimension::D2,
format: main_texture_format,
usage: TextureUsages::RENDER_ATTACHMENT,
},
)
.default_view
}),
}
});
commands.entity(entity).insert(ViewTarget {
main_texture: main_texture.clone(),
out_texture: texture_view.clone(),
out_texture_format: texture_format,
main_textures: main_textures.clone(),
main_texture_format,
main_texture: AtomicUsize::new(0),
out_texture: out_texture_view.clone(),
out_texture_format,
});
}
}