Skip to content

Commit

Permalink
feat: add splat mesh (#48)
Browse files Browse the repository at this point in the history
* feat: add splat mesh
* fix shaderMaterial
  • Loading branch information
vis-prime authored Dec 5, 2023
1 parent 54e2c9e commit 9ace203
Show file tree
Hide file tree
Showing 5 changed files with 774 additions and 4 deletions.
73 changes: 73 additions & 0 deletions .storybook/stories/Splat.stories.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import * as THREE from 'three'
import { Setup } from '../Setup'
import { Meta } from '@storybook/html'
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls'
import { GUI } from 'lil-gui'
import { Splat, SplatLoader } from '../../src/core/Splat'

export default {
title: 'Abstractions/Splat',
} as Meta

let gui: GUI

export const SplatStory = async () => {
gui = new GUI({ title: 'Splat Story', closeFolders: true })
const { renderer, scene, camera } = Setup()

const controls = new OrbitControls(camera, renderer.domElement)
controls.target.set(0, 1, 0)
controls.update()

scene.background = new THREE.Color('white')

camera.position.set(10, 10, 10)

loadSplats(renderer, camera, scene)
}

async function loadSplats(renderer: THREE.WebGLRenderer, camera: THREE.PerspectiveCamera, scene: THREE.Scene) {
const cakewalk = 'https://huggingface.co/cakewalk/splat-data/resolve/main'
const dylanebert = 'https://huggingface.co/datasets/dylanebert/3dgs/resolve/main/kitchen'

const loader = new SplatLoader(renderer)
const [shoeSplat, plushSplat, kitchenSplat] = await Promise.all([
loader.loadAsync(`${cakewalk}/nike.splat`),
loader.loadAsync(`${cakewalk}/plush.splat`),
loader.loadAsync(`${dylanebert}/kitchen-7k.splat`),
])

const shoe1 = new Splat(shoeSplat, camera, { alphaTest: 0.1 })
shoe1.scale.setScalar(0.5)
shoe1.position.set(0, 1.6, 2)
scene.add(shoe1)

// This will re-use the same data, only one load, one parse, one worker, one buffer
const shoe2 = new Splat(shoeSplat, camera, { alphaTest: 0.1 })
shoe2.scale.setScalar(0.5)
shoe2.position.set(0, 1.6, -1.5)
shoe2.rotation.set(Math.PI, 0, Math.PI)
scene.add(shoe2)

const plush = new Splat(plushSplat, camera, { alphaTest: 0.1 })
plush.scale.setScalar(0.5)
plush.position.set(-1.5, 1.6, 1)
scene.add(plush)

const kitchen = new Splat(kitchenSplat, camera)
kitchen.position.set(0, 0.25, 0)
scene.add(kitchen)

// add gui
const folder = gui.addFolder('SPLAT')

folder.add(shoe1, 'visible').name('Shoe 1 visible')

folder.add(shoe2, 'visible').name('Shoe 2 visible')

folder.add(plush, 'visible').name('Plush visible')

folder.add(kitchen, 'visible').name('Kitchen visible')
}

SplatStory.storyName = 'Default'
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ import { pcss, ... } from '@pmndrs/vanilla'
<li><a href="#outlines">Outlines</a></li>
<li><a href="#billboard">Billboard</a></li>
<li><a href="#text">Text</a></li>
<li><a href="#splat">Splat</a></li>
</ul>
<li><a href="#gizmos">Gizmos</a></li>
<ul>
Expand Down Expand Up @@ -607,6 +608,46 @@ const preloadRelatedParams = {
},
```
#### Splat
[![storybook](https://img.shields.io/badge/-storybook-%23ff69b4)](https://pmndrs.github.io/drei-vanilla/?path=/story/abstractions-splat--splat-story)
[drei counterpart](https://github.com/pmndrs/drei#splat)
A declarative abstraction around [antimatter15/splat](https://github.com/antimatter15/splat). It supports re-use, multiple splats with correct depth sorting, splats can move and behave as a regular object3d's, supports alphahash & alphatest, and stream-loading.
```js
const loader = new SplatLoader(renderer)

const [shoeSplat, plushSplat, kitchenSplat] = await Promise.all([
loader.loadAsync(`shoe.splat`),
loader.loadAsync(`plush.splat`),
loader.loadAsync(`kitchen.splat`),
])

const shoe1 = new Splat(shoeSplat, camera, { alphaTest: 0.1 })
shoe1.position.set(0, 1.6, 2)
scene.add(shoe1)

// This will re-use the same data, only one load, one parse, one worker, one buffer
const shoe2 = new Splat(shoeSplat, camera, { alphaTest: 0.1 })
scene.add(shoe2)

const plush = new Splat(plushSplat, camera, { alphaTest: 0.1 })
scene.add(plush)

const kitchen = new Splat(kitchenSplat, camera)
scene.add(kitchen)
```

In order to depth sort multiple splats correctly you can either use alphaTest, for instance with a low value. But keep in mind that this can show a slight outline under some viewing conditions.

You can also use alphaHash, but this can be slower and create some noise, you would typically get rid of the noise in postprocessing with a TAA pass. You don't have to use alphaHash on all splats.

```js
const plush = new Splat(plushSplat, camera, { alphaHash: true })
```

#### Sprite Animator

[![storybook](https://img.shields.io/badge/-storybook-%23ff69b4)](https://pmndrs.github.io/drei-vanilla/?path=/story/misc-spriteanimator--sprite-animator-story)
Expand Down
Loading

0 comments on commit 9ace203

Please sign in to comment.