bevy/examples/3d/meshlet.rs
JMS55 aa626e4f0b
Per-meshlet compressed vertex data (#15643)
# Objective
- Prepare for streaming by storing vertex data per-meshlet, rather than
per-mesh (this means duplicating vertices per-meshlet)
- Compress vertex data to reduce the cost of this

## Solution
The important parts are in from_mesh.rs, the changes to the Meshlet type
in asset.rs, and the changes in meshlet_bindings.wgsl. Everything else
is pretty secondary/boilerplate/straightforward changes.

- Positions are quantized in centimeters with a user-provided power of 2
factor (ideally auto-determined, but that's a TODO for the future),
encoded as an offset relative to the minimum value within the meshlet,
and then stored as a packed list of bits using the minimum number of
bits needed for each vertex position channel for that meshlet
- E.g. quantize positions (lossly, throws away precision that's not
needed leading to using less bits in the bitstream encoding)
- Get the min/max quantized value of each X/Y/Z channel of the quantized
positions within a meshlet
- Encode values relative to the min value of the meshlet. E.g. convert
from [min, max] to [0, max - min]
- The new max value in the meshlet is (max - min), which only takes N
bits, so we only need N bits to store each channel within the meshlet
(lossless)
- We can store the min value and that it takes N bits per channel in the
meshlet metadata, and reconstruct the position from the bitstream
- Normals are octahedral encoded and than snorm2x16 packed and stored as
a single u32.
- Would be better to implement the precise variant of octhedral encoding
for extra precision (no extra decode cost), but decided to keep it
simple for now and leave that as a followup
- Tried doing a quantizing and bitstream encoding scheme like I did for
positions, but struggled to get it smaller. Decided to go with this for
simplicity for now
- UVs are uncompressed and take a full 64bits per vertex which is
expensive
  - In the future this should be improved
- Tangents, as of the previous PR, are not explicitly stored and are
instead derived from screen space gradients
- While I'm here, split up MeshletMeshSaverLoader into two separate
types

Other future changes include implementing a smaller encoding of triangle
data (3 u8 indices = 24 bits per triangle currently), and more
disk-oriented compression schemes.

References:
* "A Deep Dive into UE5's Nanite Virtualized Geometry"
https://advances.realtimerendering.com/s2021/Karis_Nanite_SIGGRAPH_Advances_2021_final.pdf#page=128
(also available on youtube)
* "Towards Practical Meshlet Compression"
https://arxiv.org/pdf/2404.06359
* "Vertex quantization in Omniforce Game Engine"
https://daniilvinn.github.io/2024/05/04/omniforce-vertex-quantization.html

## Testing

- Did you test these changes? If so, how?
- Converted the stanford bunny, and rendered it with a debug material
showing normals, and confirmed that it's identical to what's on main.
EDIT: See additional testing in the comments below.
- Are there any parts that need more testing?
- Could use some more size comparisons on various meshes, and testing
different quantization factors. Not sure if 4 is a good default. EDIT:
See additional testing in the comments below.
- Also did not test runtime performance of the shaders. EDIT: See
additional testing in the comments below.
- How can other people (reviewers) test your changes? Is there anything
specific they need to know?
- Use my unholy script, replacing the meshlet example
https://paste.rs/7xQHk.rs (must make MeshletMesh fields pub instead of
pub crate, must add lz4_flex as a dev-dependency) (must compile with
meshlet and meshlet_processor features, mesh must have only positions,
normals, and UVs, no vertex colors or tangents)

---

## Migration Guide
- TBD by JMS55 at the end of the release
2024-10-08 18:42:55 +00:00

134 lines
4.7 KiB
Rust

//! Meshlet rendering for dense high-poly scenes (experimental).
// Note: This example showcases the meshlet API, but is not the type of scene that would benefit from using meshlets.
#[path = "../helpers/camera_controller.rs"]
mod camera_controller;
use bevy::{
pbr::{
experimental::meshlet::{MaterialMeshletMeshBundle, MeshletPlugin},
CascadeShadowConfigBuilder, DirectionalLightShadowMap,
},
prelude::*,
render::render_resource::AsBindGroup,
};
use camera_controller::{CameraController, CameraControllerPlugin};
use std::{f32::consts::PI, path::Path, process::ExitCode};
const ASSET_URL: &str =
"https://raw.githubusercontent.com/JMS55/bevy_meshlet_asset/8443bbdee0bf517e6c297dede7f6a46ab712ee4c/bunny.meshlet_mesh";
fn main() -> ExitCode {
if !Path::new("./assets/models/bunny.meshlet_mesh").exists() {
eprintln!("ERROR: Asset at path <bevy>/assets/models/bunny.meshlet_mesh is missing. Please download it from {ASSET_URL}");
return ExitCode::FAILURE;
}
App::new()
.insert_resource(DirectionalLightShadowMap { size: 4096 })
.add_plugins((
DefaultPlugins,
MeshletPlugin {
cluster_buffer_slots: 8192,
},
MaterialPlugin::<MeshletDebugMaterial>::default(),
CameraControllerPlugin,
))
.add_systems(Startup, setup)
.run();
ExitCode::SUCCESS
}
fn setup(
mut commands: Commands,
asset_server: Res<AssetServer>,
mut standard_materials: ResMut<Assets<StandardMaterial>>,
mut debug_materials: ResMut<Assets<MeshletDebugMaterial>>,
mut meshes: ResMut<Assets<Mesh>>,
) {
commands.spawn((
Camera3d::default(),
Transform::from_translation(Vec3::new(1.8, 0.4, -0.1)).looking_at(Vec3::ZERO, Vec3::Y),
Msaa::Off,
EnvironmentMapLight {
diffuse_map: asset_server.load("environment_maps/pisa_diffuse_rgb9e5_zstd.ktx2"),
specular_map: asset_server.load("environment_maps/pisa_specular_rgb9e5_zstd.ktx2"),
intensity: 150.0,
..default()
},
CameraController::default(),
));
commands.spawn((
DirectionalLight {
illuminance: light_consts::lux::FULL_DAYLIGHT,
shadows_enabled: true,
..default()
},
CascadeShadowConfigBuilder {
num_cascades: 1,
maximum_distance: 15.0,
..default()
}
.build(),
Transform::from_rotation(Quat::from_euler(EulerRot::ZYX, 0.0, PI * -0.15, PI * -0.15)),
));
// A custom file format storing a [`bevy_render::mesh::Mesh`]
// that has been converted to a [`bevy_pbr::meshlet::MeshletMesh`]
// using [`bevy_pbr::meshlet::MeshletMesh::from_mesh`], which is
// a function only available when the `meshlet_processor` cargo feature is enabled.
let meshlet_mesh_handle = asset_server.load("models/bunny.meshlet_mesh");
let debug_material = debug_materials.add(MeshletDebugMaterial::default());
for x in -2..=2 {
commands.spawn(MaterialMeshletMeshBundle {
meshlet_mesh: meshlet_mesh_handle.clone(),
material: standard_materials.add(StandardMaterial {
base_color: match x {
-2 => Srgba::hex("#dc2626").unwrap().into(),
-1 => Srgba::hex("#ea580c").unwrap().into(),
0 => Srgba::hex("#facc15").unwrap().into(),
1 => Srgba::hex("#16a34a").unwrap().into(),
2 => Srgba::hex("#0284c7").unwrap().into(),
_ => unreachable!(),
},
perceptual_roughness: (x + 2) as f32 / 4.0,
..default()
}),
transform: Transform::default()
.with_scale(Vec3::splat(0.2))
.with_translation(Vec3::new(x as f32 / 2.0, 0.0, -0.3)),
..default()
});
}
for x in -2..=2 {
commands.spawn(MaterialMeshletMeshBundle {
meshlet_mesh: meshlet_mesh_handle.clone(),
material: debug_material.clone(),
transform: Transform::default()
.with_scale(Vec3::splat(0.2))
.with_rotation(Quat::from_rotation_y(PI))
.with_translation(Vec3::new(x as f32 / 2.0, 0.0, 0.3)),
..default()
});
}
commands.spawn((
Mesh3d(meshes.add(Plane3d::default().mesh().size(5.0, 5.0))),
MeshMaterial3d(standard_materials.add(StandardMaterial {
base_color: Color::WHITE,
perceptual_roughness: 1.0,
..default()
})),
));
}
#[derive(Asset, TypePath, AsBindGroup, Clone, Default)]
struct MeshletDebugMaterial {
_dummy: (),
}
impl Material for MeshletDebugMaterial {}