We may earn a commission from links on this page.
My question about virtual reality has always been, “But what is it? forI finally have an answer: Guassian Splatting. We’ve always tried to capture our past, whether it’s through physical photographs, VHS tapes or every picture you’ve stored in the cloud, but we’re limited to viewing our personal history in flat media, usually from behind a screen and always from the same angle. But this technology of Gaussian splatting allows you to modify the object 3. People, or places, so instead of a picture of your child’s favorite toy, you can scan an actual one that instead of a snapshot of Thanksgiving dinner, you can have a photorealistic diorama of the dining room that you can walk around.
What is Gaussian splating?
Gaussian splicing is a technological nascent. It was first theoretically introduced in 2023 Research paper By Bernhard Kerble, Georgios Kopanas, Thomas Leimkuhler, and George Dratakis. The paper details a new rendering technique that creates 3D models from millions of semi-transparent blobs called “Gaussians” instead of the solid triangles used in traditional computer graphics. Once calculated, the Gaussians are “splatted” onto a 2D plane by your computer, and are arranged and layered within the splat based on how it should look from any point of view. Because the blobs are semi-transparent, they don’t block each other. They blend like brushstrokes in a painting.
Another bonus: Splatting offers much higher detail for its file size than traditional methods of scanning. Older scans work on the geometric principle of stretching a virtual skin made up of triangles over an object. For detailed scans, it can be billions of triangles, resulting in PC-choking file sizes. Splatting is based on mathematical probability rather than strict geometry. Instead of a solid edge, each “blob” is a small cloud that tells the computer how likely a color is to exist in that location. It simply stores the position, color and transparency of millions of related areas in space, as well as how light should be reflected from different angles. The result is files that are large compared to Word documents, but not so large that you can’t work with them on the phone.
Gaussian splatting quickly progressed from theory to practice, and now splats can be created and rendered with just a decent smartphone, making them more accessible than older methods that sometimes required laser scanners or specialized equipment.
Why You Should Start Splatting
3D scanning is already in commercial use for things like mapping real estate for virtual tours and creating photorealistic assets for video games, but Gaussian splatting is accessible enough that anyone can future-proof their nostalgia.
Splatting gives your future self (or your children) the ability to “meet” your current life with a level of reality that is fascinating. It allows you to digitally “bottle” and preserve the exact layout and volume of a moment in time. If your parents have one, you’ll be able to walk around your childhood bedroom, or check out every nook and cranny of the first car you ever bought.
“Digital preservation” and “3D modeling” may sound clinical, but the results of Gaussian splotches are anything but sterile. While photography captures one angle of light in a room, a Gaussian splat captures the behavior of light from all angles, so the result is not what the past looks like, but how the past is. feels It’s hard to describe, but capturing the quality of light on an object or location puts you in touch with it in a way you didn’t think possible. It combines with spats and the vagueness of your own memories to add an ethereal, dreamlike experience unlike anything else. (I love splats.)
How to start splatting
The barrier to entry for splatting is some time to understand how it works. You don’t need a special LiDAR scanner or a high-powered PC, just a relatively recent smartphone. Here’s how to get started:
Choose an application: Although the technology is new, some apps are making it very user-friendly. Here are two I tried:
-
Scaniverse: Great for iPhone users, Scaniverse is free, and it processes Splats completely on your device in just a minute or two.
-
Luma 3D Capture: Available on both Android And iPhoneLuma is great for beginners, with a scanning process that walks you through creating your first splat.
Capture one: Here are some things to consider when making your captures.
-
Before you start scanning locations or large objects, choose something small and simple so you can get the concepts down. But not pets: Your subject has to remain completely still during the procedure. (Make an exception for your child. They still may not catch on enough, but having your child’s role model is important for your future.)
-
Place your subject in an evenly lit room where he has enough room to walk around.
-
Hit record and walk in a slow, steady circle around your subject with your camera centered on it.
-
Make two passes, one looking down from a high angle, the other from a low angle, looking up.
-
Gaussian scatter hates uniformity. They clash with plain white walls, so think in terms of texture. Also, avoid clear glass and mirrors that confuse depth calculations.
Take a banana: Now that you’ve captured your splat, take a break so the computer can do its work. How long it takes depends on the app you’re using, your phone, and how detailed your scan is. Scaniverse processes the splat right on your phone. For something as simple as the guitar below, it took about two minutes to render on the iPhone 17 Pro. Luma 3D capture processes files in the cloud, so how long it takes depends on how many people are in front of you in the queue. It could be a few minutes. It could be a few hours—the app sends an alert when your image is finished cooking. The video below took several hours.
Enjoy your creation: Once the math math is done, you can view your creation on your smartphone screen or computer. Pinch to zoom, drag to rotate and marvel at how the scan perfectly captures the vibe of an object or space.
Share your creation: These apps give you some easy ways to share your volumetric memory:
-
video: You can plot the camera path through your splat to export a simple, 2D “fly-through” video. Below is my first scan using Scaniverse on YouTube (it’s sloppy; I was new), and my second attempt with Luma.
-
Web Link: You can generate a simple web link and text your friends or family through both apps. When they tap it, it opens an interactive 3D viewer in their browser—no special apps, accounts, or heavy downloads required.
How to step inside your splats
It’s kind of cool to see a 3D scan on your phone or PC, but you don’t really understand what these things are like until you check them out in a virtual reality device, where you can physically walk around the Thanksgiving table or lean over the couch to inspect the texture. Here’s how you can do it on the two biggest headsets right now.
What do you think so far?
Apple Vision Pro
There was a powerful Apple Vision Pro built up To do this. Apple incorporated “spatial scenes” into the OS itself. It gives 2D photos a little 3D pop, but you can take it a bit further with apps like Splat Studio which will generate a deeper 3D scene from a 2D photo and let you change settings to improve it. But you can get deeper with it Spatial Media Toolkit. It lets you convert 2D videos into stereoscopic 3D videos. But the final boss is watching full splats like the ones you made yourself with apps Luma 3D Capture Or Polychem.
If you follow the above steps, you will be able to export the Splat file (.ply or .spz) you created right from your phone to your Vision Pro and step inside the Splat or walk around the object you scanned. You can also check Splats uploaded by other users.
Meta Quest 3 and 3S
Meta accepts a Gaussian splat revolution. Apps like Aerovis (Also on Vision Prof) lets you view splats you’ve created on your phone, and 4D splats are also available on Quest (more on that below). Meta is also taking the first steps towards removing the middleman from your phone entirely. Hyperscape Capture is a still-in-beta app that uses Quest’s existing camera to scan your room, then saves a 3D version of your space. Meta promises that soon you’ll be able to send a link to a friend with a headset so they can “visit.”
The future of 4D splatting
As famous as I am for Gaussian splatting, the technology is in its “version 1.0 era”. Capturing the right splat takes time and patience and the subject needs to be perfectly still, and the result isn’t always perfect, but technology is evolving so fast that the next thing is already emerging. Cutting-Gaussian-edge is 4D splicing – the fourth dimension is time. 4D Splat is a 3D volumetric video, moving scene you can view from any point inside or outside the scene. Unlike stereoscopic 3D movies that let you see from one point, these are true holographs. At least they are inside a VR rig.
The technology is already being used commercially, most notably in A$AP Rocky’s music video “Helicopter”, in which the performers were captured by 56 cameras and the footage converted into 4D splats, allowing the use of any angle or impossible camera movement. Check it out:
And there are some 4D splats that you can check out in your headset as well. Quest 3 app Gracia There are few volumetric videos that are very impressive. Gracia lets you stream or download 4D splats of people, and place them in augmented reality wherever you want. You can then hit “play” and view them from any angle, or move them all the way around. To see what I mean, check out this video showing my view from inside a Quest 3 headset as singer Amy May performs the song on my front lawn (with a cameo from my no doubt confused neighbor).
You probably don’t have 20 or so GoPros to create content like Gracia, but there are some practical tools for creating 4D splats for consumers. Kiri Engine Uses Apple’s open-source ML-Sharp tool to convert standard single-lens video into 4D splat. It doesn’t create an AI-assisted projection of stereoscopic 3D like Splat Studio, but converts each individual frame into a separate splat. It’s too technical for me to really mess with and 3D isn’t real 3D is guess work, but I’d be surprised if there isn’t a way to take volumetric video with just a few smart phone angles in the works somewhere.
The Gaussian splat is as much of a revelation as I imagine the snapshot developing immediately in the 1960s. Like early Polaroids, it’s a bit painful, and the results are sometimes grainy, “dreamy” and reminiscent of pointillism, but the emotional impact of a new way of looking at the past is so strong. So start splatting now; Your future self will thank you.
