We may earn commission from the links on this page.
My question about virtual reality has always been, “But what is it For?” I finally have an answer: Gaussian Splatting. We’ve always tried to capture our past, whether it’s through physical photographs, VHS tapes, or every photo you have stored in the cloud, but we’ve been limited to viewing our personal histories in flat media, usually from behind a screen, and always from the same angle. But Gaussian Splatting changes that. This technique allows you to create volumetric 3D models of objects, people, or places. Instead of a picture of your child’s favorite toy, you can take a photorealistic diorama of the dining room that you can examine from every angle.
What is Gaussian Splatting?
Gaussian splatting is a technological neophyte. It was first theoretically introduced in 2023 research paper By Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuhler and George Dratakis. The paper details a new rendering technique that creates 3D models from millions of semi-transparent blobs, called “Gaussians”, instead of the solid triangles used in traditional computer graphics. Once calculated, the Gaussians are “splatted” by your computer on a 2D plane, and arranged and layered based on how they should look from any viewpoint within the splat. Since the drops are semi-transparent, they do not block each other. They blend together like brushstrokes in a painting.
Another bonus: Splatting provides a very high level of detail for its file size compared to traditional methods of scanning. Older scans work on the geometric principle of drawing a virtual skin made of triangles over an object. For a detailed scan, this could be billions of triangles, resulting in a PC-choking file size. Splatting is based on mathematical probability rather than rigorous geometry. Instead of a solid edge, each “drop” is a tiny cloud that tells the computer how likely the color is to be present at that location. It simply stores the position, color, and transparency of millions of relevant areas in space, as well as how they should reflect light from different angles. The result is files that are larger than Word documents, but not so large that you can’t work with them on the phone.
Gaussian splatting moved quickly from theory to practice, and now splats can be created and presented with just a good smartphone, making it more accessible than older methods that sometimes required laser scanners or specialized equipment.
Why should you start splatting?
3D scanning is already in use professionally in things like mapping real estate for virtual tours and creating photorealistic assets for video games, but Gaussian splatting is so accessible that anyone can future-proof their old memories.
Sleaze gives your future self (or your children) the ability to “visit” your present life with a level of breathtaking realism. This lets you digitally “bottle” the exact layout and volume of a moment in time and preserve it. If your parents have it, you’ll be able to wander around their childhood bedroom, or examine every angle and detail of the first car you bought.
“Digital preservation” and “3D modeling” sound clinical, but the results of Gaussian splats are anything but sterile. Whereas photography captures one angle of light in a room, Gaussian splats capture the behavior of light from all angles, so the result is not what the past looks like, but what the past looks like. feels like. It’s hard to describe, but capturing the quality of light on an object or place brings you in touch with it in a way you didn’t think possible. This combined with the blur of the spats and your own memories creates an ethereal, dreamlike experience that is unlike anything else. (I love splats.)
how to start splatting
The barrier to entry for splatting is just a little time to figure out how it works. You don’t need a special LiDAR scanner or an extremely powerful PC, just a relatively recent smartphone. Here’s how to get started:
choose an app: Although the technology is new, some apps are making it very user friendly. Here are the two I’ve tried:
-
Scaniverse: Excellent for iPhone users, Scaniverse is free, and it completely processes the splat on your device in just a minute or two.
-
luma 3d capture: Available on both Android and iPhone, Luma is great for beginners, with a scanning process that helps you create your first splat.
capture:Here are some things to consider when creating your capture.
-
Before you start scanning places or large objects, choose something small and simple so you can understand the concepts. But not pets: your subject must remain completely still during the process. (Make an exception for your child. They won’t stand still, but even a blurry model of your child is important for your future.)
-
Place your subject in an evenly lit room with plenty of room to move around.
-
Hit record and walk in slow, steady circles around your subject, keeping your camera focused on it.
-
Make two passes, one from a higher angle looking downwards, the other from a lower angle, looking upwards.
-
Gaussian splats hate uniformity. They clash with plain white walls, so think in terms of texture. Also, avoid clear glass and mirrors that confuse depth calculations.
Take a banana: Now that you’ve captured your splat, take a break so the computer can do its thing. How long this takes depends on the app you’re using, your phone, and how detailed your scan is. Scaniverse processes Splats directly on your phone. For something as simple as the guitar below, rendering on the iPhone 17 Pro took about two minutes. Luma 3D Capture processes the files in the cloud, so how long it takes depends on how many people are in front of you in the queue. This may take a few minutes. This may take a few hours—the app sends an alert when your image is cooked. The video below took several hours.
enjoy your creation: Once the math is finished, you can view your creation directly on your smartphone screen or computer. Pinch to zoom, drag to rotate and marvel at how well the scan captured the vibe of the object or location.
share your creation:These apps give you some easy ways to share your volumetric memory:
-
Video: You can plot a camera path through your splat to export a seamless, 2D “fly-through” video. Below is my first scan on YouTube using Scaniverse (it’s sloppy; I was new), and my second attempt with Luma.
-
Web link: You can generate a simple web link and text it to your friends or family through both apps. When they tap it, it opens an interactive 3D viewer in their browser — no special apps, accounts, or bulky downloads required.
How to step inside your splats
Viewing 3D scans on your phone or PC is kind of cool, but you can’t really understand how adorable these things are until you see them in a virtual reality device, where you can physically walk around that Thanksgiving table or bend over on the couch to inspect the textures. Here’s how you can do it on two of the biggest headsets out there right now.
What do you think so far?
apple vision pro
The powerful Apple Vision Pro was Make To do this. Apple included “spatial views” directly into the OS. This gives a slight 3D pop to a 2D photo, but you can take it a little further with apps like Splat Studio that will generate a deeper 3D scene from a 2D photo and let you change settings to make it better. But you can go deeper with the Spatial Media Toolkit. It lets you convert 2D videos into stereoscopic 3D videos. But the final boss is seeing the perfect splat that you’ve created yourself with apps like Luma 3D Capture or Polycam.
If you follow the steps above, you should be able to export the splat file (.ply or .spz) you created directly from your phone to your Vision Pro and step inside the splat or move it around the object you scanned. You can also view splats uploaded by other users.
Meta Quest 3 and 3S
The meta has embraced the Gaussian Splat revolution. Apps like Airvis (also on Vision Pro) let you check out splats created on your phone, and there’s even 4D splats available on the Quest (more on that below). Meta is also taking the first steps towards eliminating the middleman from your phone completely. Hyperscape Capture is a still-in-beta app that uses the Quest’s existing cameras to scan your room, then save a 3D version of your space. Meta promises that soon you’ll be able to send a link to a friend with a headset so they can “come visit.”
The future of 4D splatting
As excited as I am for Gaussian Splatting, the technology is in its “version 1.0 era.” Capturing a good splat takes time and patience and requires the subject to remain absolutely still, and the result is not always perfect, but technology is evolving so rapidly that the next thing is already emerging. The state-of-the-art is Gaussian edge 4D splatting – the fourth dimension is time. 4D splats are 3D volumetric videos, dynamic scenes that you can view from any point inside or outside the scene. Unlike stereoscopic 3D movies, which let you view from a single point, these are true holographs. At least they’re inside a VR rig.
The technology is already in use commercially, most notably in A$AP Rocky’s music video “Helicopter”, in which the cast was captured by 56 cameras and the footage was converted into 4D splats, allowing any angle or impossible camera movements to be used. Check it out:
And there are some 4D splats that you can see in your headset as well. The Quest 3 app Gracia has some volumetric videos that are pretty impressive. Gracia lets you stream or download 4D splats of people and place them anywhere you like in augmented reality. You can then press “Play” and view them from any angle, or even move around them. To see what I mean, check out this video I made showing my view from within the Quest 3 headset, with singer Amy May performing a song on my front lawn (with a cameo from my undoubtedly confused neighbor).
You probably don’t have a series of 20 or more GoPros to create content like Garcia, but there are some experimental devices out there for consumers to create 4D splats. kiri engine Uses Apple’s open-source ML-Sharp tool to transform a standard single-lens video into a 4D splat. It doesn’t create an AI-assisted approximation of stereoscopic 3D like Splat Studio does, but rather converts each individual frame into a separate splat. It’s too technical for me to really mess with this and the 3D is projections and not actual 3D, but I’d be surprised if there wasn’t a way somewhere to take volumetric video with just a few smart phone angles.
Gaussian splats are as much of a revelation as I imagine instant-developing snapshots were in the 1960s. Like early Polaroids, it takes a little pains, and the results are sometimes grainy, “dreamy” and reminiscent of pointillism, but the emotional impact of a new way of looking at the past is so strong. So start splatting now; Your future self will thank you.
