LabHub

Blog

3D Gaussian Splatting & Photogrammetry 2026 Deep Dive - Nerfstudio · Postshot · Luma Genie · Polycam · Apple Object Capture · RealityCapture Field Guide

2026-05-16 · 19 min read · #english#3d-gaussian-splatting#photogrammetry#nerfstudio#postshot#luma-ai#polycam#apple-object-capture#realitycapture#3d-reconstruction

한국어English日本語

Prologue — The Polygon Era Is Ending

Through the 2010s, "3D" meant polygon meshes. Vertices, faces, UVs, textures. Blender, Maya, ZBrush. Turning photos into meshes was the job of photogrammetry, and RealityCapture and Metashape sat at its peak.

In July 2023, a single INRIA paper changed the landscape. "3D Gaussian Splatting for Real-Time Radiance Field Rendering" — Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis. NeRF's per-frame inference, which took one to five minutes, was replaced by alpha compositing of millions of ellipsoid primitives. The same image quality dropped to under 10ms.

The May 2026 landscape looks like this.

This article walks the May 2026 3DGS and photogrammetry stack end to end. Capture, SfM with COLMAP and Hloc, training, export, web viewer — and between the lines, real Korean and Japanese deployments, academic baselines, and the license traps.

Chapter 1 · Two Paradigms — Mesh and Splat

Start with a picture.

[100 photos]
     |
     v
 [SfM: COLMAP / Hloc / Glomap]
     |
     +-> camera poses
     +-> sparse point cloud
     |
     v
 +---------+        +-------------------+
 |  MVS    |        |  3D Gaussians     |
 | (dense) |        |  (millions of     |
 +---------+        |   ellipsoids)     |
     |              +-------------------+
     v                       |
 [mesh .obj/.glb]            v
     |              [.ply / .splat / .ksplat]
     v                       |
 [polygon render]            v
                    [splat raster ~10ms]

The left branch is photogrammetry — photos to mesh. It started with aerial survey in the 1990s and matured into RealityCapture, Metashape, and Meshroom. The output is polygons and textures, ready for game engines, CAD, and 3D printers.

The right branch is Gaussian splatting — photos to 3D Gaussian ellipsoids. Instead of meshes, millions of soft points with position, covariance, color, and opacity describe the scene. Hair, glass, smoke — anything polygons struggled with — comes through naturally.

One line to remember: mesh is 3D you touch and cut. Splat is 3D you look at and orbit.

Chapter 2 · How 3DGS Works — Ellipsoid Alpha Compositing

A single 3D Gaussian is defined by five things.

  1. Mean — a 3D position (x, y, z).
  2. Covariance — a 3 by 3 matrix. In practice it factors into a rotation quaternion and a scale vector.
  3. Opacity — an alpha value.
  4. SH coefficients — spherical harmonics that encode view-dependent color. Degree 3 means 48 values per Gaussian.
  5. View-dependent color — synthesized from SH given the camera direction.

Rendering goes:

1. Project every Gaussian into camera space.
2. Compress each into a screen-space 2D Gaussian (Jacobian approximation).
3. Sort by depth (front-to-back).
4. Alpha-blend per pixel.

Where NeRF integrates dozens to hundreds of samples per pixel via volumetric ray marching, 3DGS composites pre-sorted Gaussians in a single pass. That is the 100x speed gap.

Training starts from the sparse SfM points, re-renders the photos, and updates Gaussian position, covariance, SH, and alpha with differentiable operators. About 30,000 iterations stabilize results for a typical 100-photo 1080p scene.

Chapter 3 · The INRIA Reference Implementation

The original authors' code (graphdeco-inria/gaussian-splatting) is still the baseline every later paper compares against. CUDA 11.8 or newer, 12 GB of VRAM minimum.

git clone https://github.com/graphdeco-inria/gaussian-splatting --recursive
cd gaussian-splatting
conda env create --file environment.yml
conda activate gaussian_splatting

# Photos to camera poses via COLMAP
python convert.py -s data/my_scene

# Train (30k iter, about 30-60 min on V100/3090)
python train.py -s data/my_scene -m output/my_scene

# Viewer
./SIBR_viewers/install/bin/SIBR_gaussianViewer_app -m output/my_scene

Strengths: clean, reproducible reference results. Weaknesses: brittle Windows builds, limited training options, and a non-commercial license. Production work has to move downstream.

Chapter 4 · Nerfstudio 1.4 — The De Facto Standard

UC Berkeley's KAIR Lab kicked off Nerfstudio in 2022. By 2024 it became less a NeRF toolkit and more the standard 3DGS pipeline. As of May 2026 we are on version 1.4.

Install is one line.

pip install nerfstudio
ns-install-cli

The shortest workflow.

# 1. Photos or video to camera poses (COLMAP-driven)
ns-process-data images --data /path/photos --output-dir /path/processed

# 2. Train (Splatfacto = Nerfstudio's 3DGS implementation)
ns-train splatfacto --data /path/processed

# 3. A viewer auto-launches in the browser at http://localhost:7007

# 4. Export a .ply
ns-export gaussian-splat --load-config outputs/.../config.yml --output-dir exports/

Three key variants.

Training knobs. ns-train splatfacto --max-num-iterations 30000 --pipeline.model.cull-alpha-thresh 0.005. About 20 minutes on a 3090 (24 GB) for 100 1080p photos.

Chapter 5 · gsplat — The Backend Nerfstudio Built

Nerfstudio 1.4 is fast because the same team built a separate library, gsplat. It rewrites INRIA's CUDA kernels and adds:

You can drop it into your own project.

import torch
from gsplat import rasterization

# means: (N, 3), quats: (N, 4), scales: (N, 3),
# opacities: (N,), colors: (N, K, 3)  where K = number of SH coeffs
renders, alphas, info = rasterization(
    means=means,
    quats=quats,
    scales=scales,
    opacities=opacities,
    colors=colors,
    viewmats=viewmats,
    Ks=intrinsics,
    width=W,
    height=H,
    render_mode='RGB',
)

This interface has become the standard, and most follow-up work (SuGaR, 2DGS, SpotlessSplats) builds on top of gsplat.

Chapter 6 · Postshot 2.0 — The Windows GUI Champion

Jawset's Postshot is a Windows-only GUI. It is the name students, artists, and VFX professionals reach for first, and 2.0 shipped in 2026.

Key features.

Recommended specs: RTX 3060 (12 GB) or better, Windows 10/11. 12 GB of VRAM holds about 5 to 6 million Gaussians comfortably.

License: free for personal and education, USD 99/year Postshot Plus for commercial.

[GUI workflow]
photo folder -> Add Project
-> Tracking (COLMAP, automatic)
-> Train (Quality preset, about 60 to 90 min)
-> Refine (clean and shrink the Gaussian set)
-> Export -> .ksplat

Chapter 7 · Luma AI Genie 2 — Text and Video to 3D

Luma AI launched as a NeRF SDK in 2022 and joined the text-to-3D race in 2024 with Genie. As of May 2026 we are at Genie 2.

Three entry points.

  1. Capture (video) — orbit a subject with an iPhone for 30 to 60 seconds; the cloud bakes both 3DGS and NeRF.
  2. Genie 2 (text-to-3D) — a prompt like "1970s Volkswagen Beetle in studio light" yields a mesh with PBR textures. Roughly 30 seconds to 2 minutes.
  3. Interactive Scenes — a single photo or short video becomes a free-orbit scene. WebGL embed included.

Capture API.

# Upload video -> get a 3DGS .ply back
curl -X POST https://api.lumalabs.ai/dream-machine/v1/captures \
  -H "Authorization: Bearer $LUMA_API_KEY" \
  -F "title=My Capture" \
  -F "media=@capture.mp4"

License: free personal, USD 30/month Pro, metered API. Commercial use is allowed, with Luma branding only on the free tier.

Chapter 8 · Polycam — When the Phone Became the Capture Device

Polycam launched in 2020 as the first mobile app to take serious advantage of the iPhone 12 Pro's LiDAR. By 2026 it ships all five capture modes.

  1. LiDAR Mode — direct mesh capture using LiDAR on iPhone/iPad Pro.
  2. Photo Mode — photogrammetry in the cloud from photos only.
  3. Room Mode — automatic indoor floor plans with USD/USDZ export.
  4. Gaussian Splat Mode — record a video, cloud trains a .ply and .splat.
  5. Capture API — upload external camera output (drone, DSLR) for cloud training.

Polycam's web viewer plays 100 MB-class .splat files smoothly on Safari and Chrome. The embed is a single iframe.

License: free tier has export restrictions. Pro is USD 17/month and Team is USD 1,200/year. Real estate, museums, and construction sites in Korea and Japan adopted it quickly.

Notable case: Tokai University's architecture department in Japan built a campus digital twin with Polycam Room Mode in 2025.

Chapter 9 · Apple Object Capture and RealityKit — The macOS 15 Sequoia API

Apple introduced the Object Capture API at WWDC 2021 (macOS Monterey). macOS 15 Sequoia (2024) sharpened it, and macOS 26 (2026) lets iPhone Photogrammetry build meshes directly on the phone.

The shortest Swift example.

import RealityKit
import Foundation

let inputFolder = URL(fileURLWithPath: "/path/photos")
let outputURL = URL(fileURLWithPath: "/path/model.usdz")

let session = try PhotogrammetrySession(input: inputFolder)
try session.process(requests: [
    .modelFile(url: outputURL, detail: .full)
])

for try await output in session.outputs {
    switch output {
    case .processingComplete:
        print("done")
    case .requestProgress(_, let fraction):
        print("\(fraction * 100)%")
    default: break
    }
}

Key traits.

License cost is zero. Hardware: M1 or newer macOS, iPhone 12 Pro or newer.

Chapter 10 · RealityCapture 1.5 — Industrial Photogrammetry Went Free

Capturing Reality in Prague built RealityCapture, the long-running industrial benchmark. Film VFX, AAA game assets, and cultural heritage digitization all relied on it — Vindictus, Black Myth Wukong, and the Petra digitization project among them.

Epic Games acquired the company in 2021, and as of June 2024 the tool is fully free. The May 2026 1.5 release added:

Recommended specs: Windows 11, RTX 3080 or better, 32 GB RAM. 1,000-photo projects are routine.

License: free, but an Epic Games account is required. Output meshes have no watermark.

Chapter 11 · Scaniverse 4 — Niantic's Mobile Capture App

Scaniverse launched in 2020 as an iOS LiDAR capture app and was acquired by Niantic in 2021. The Android version arrived in 2024, and version 4 in 2026 placed Gaussian Splat mode at the center.

Highlights.

Capture takes 1 to 3 minutes, training another 3 to 10. Quality matches Polycam and Luma or sits slightly below, but the local plus free combo is decisive.

Chapter 12 · Meshroom 2024.x — The AliceVision Open Source Path

The most complete open source photogrammetry tool is AliceVision Meshroom. It has been in development since 2018; the 2024 build (GPL v3) runs on Linux, Windows, and macOS.

Highlights.

The shortest workflow.

CameraInit -> FeatureExtraction -> ImageMatching
   -> FeatureMatching -> StructureFromMotion
   -> PrepareDenseScene -> DepthMap -> DepthMapFilter
   -> Meshing -> MeshFiltering -> Texturing

Commercial use allowed, GPL v3. Downsides: a heavy GUI and quality that lags RealityCapture or Metashape by 5 to 10 percent.

Chapter 13 · Follow-up Research — 2DGS, SuGaR, Mip-Splatting, SpotlessSplats

3DGS spawned hundreds of follow-up papers in two years. As of May 2026, five have become real tools.

Combining all three is likely to be the next baseline.

Chapter 14 · Export Formats — .ply, .splat, .ksplat

Three formats dominate.

Conversion is usually:

# Nerfstudio .ply -> .splat (Antimatter15 converter)
git clone https://github.com/antimatter15/splat
node splat/convert.js outputs/.../point_cloud/iteration_30000/point_cloud.ply

# .splat -> .ksplat (Mark Kellogg gaussian-splats-3d)
npm install -g @mkkellogg/gaussian-splats-3d
gaussian-splats-3d convert input.splat output.ksplat

Size comparison.

Scene.ply.splat.ksplat
Living room (100 photos, 1080p)380 MB52 MB28 MB
Small subject (50 photos, 720p)110 MB15 MB8 MB
Campus (500 photos, 4K)1.4 GB230 MB140 MB

Chapter 15 · Web Viewers — Three.js, Babylon.js, Antimatter15

Three options for orbiting a Gaussian splat in the browser in 2026.

Three.js plus gaussian-splats-3d, shortest possible:

import * as THREE from 'three'
import { GaussianSplats3D } from '@mkkellogg/gaussian-splats-3d'

const viewer = new GaussianSplats3D.Viewer({
  selfDrivenMode: true,
  threeScene: new THREE.Scene(),
  useWorkers: true,
})

await viewer.addSplatScene('/scenes/room.ksplat', {
  splatAlphaRemovalThreshold: 5,
  position: [0, 0, 0],
})

viewer.start()

On the WebGPU path, Babylon.js 8 (February 2026) is fastest. Around 60fps on an iPhone 15 at 3 million Gaussians.

Chapter 16 · Capture Best Practices — The 100-Photo Rule

About 70 percent of 3DGS and photogrammetry quality comes from the capture. Five principles.

  1. Overlap over 70 percent — adjacent photos must share more than 70 percent of their frame for SfM stability.
  2. Three heights — eye level, knee level, above head. Every subject needs top and bottom angles.
  3. Light on a cloudy day — direct sun creates hard shadows that hurt training. An overcast noon is ideal.
  4. Avoid reflective and transparent surfaces — mirrors and glass break SfM. Mask partially when needed.
  5. Remove moving objects when possible — people and cars require SpotlessSplats or Splatfacto-W masking.

Recommended counts. Small object 50 photos, single room 100 to 200, building exterior 300 to 500, campus 1,000 or more.

Capture patterns.

object   -> 360 orbit plus high and low tiers (three orbits total)
room     -> one lap along the walls plus a single ceiling pass
building -> four exterior sides plus a top shot (drone)
garden   -> grid pattern, six to ten parallel rows

Chapter 17 · Korean and Japanese Deployments

Five notable adoptions between 2024 and 2025.

The common thread: re-baking existing photo and video assets through the new pipeline drove adoption faster than fresh captures.

Chapter 18 · Apple Vision Pro, Immersive Scenes, Reality Composer Pro 2.0

After Apple Vision Pro shipped in 2024, visionOS 2 in 2025 stabilized the Immersive Scenes API. Reality Composer Pro 2.0 on macOS 15 Sequoia edits USDZ directly and partially imports Gaussian splatting.

The shortest workflow.

1. RealityCapture / Object Capture -> .usdz mesh
2. Edit materials and anchors in Reality Composer Pro 2.0
3. Drop into the RealityKit scope of a visionOS Xcode project
4. Present in an ImmersiveSpace for 6DoF viewing

Native 3DGS import is still pending, but running Antimatter15's WebGL viewer inside Vision Pro Safari is the de facto workaround. Native integration is expected in visionOS 3 in late 2026.

Chapter 19 · License Matrix

ToolLicenseCommercial UseWatermark
INRIA gaussian-splattingNon-commercialNo-
NerfstudioApache 2.0YesNone
gsplatApache 2.0YesNone
PostshotFree personal, USD 99/yr commercialYes (paid)None
Luma AI GenieFree or USD 30/mo ProYesLuma branding on free
PolycamFree or USD 17/mo ProSome free, export limitsNone on Pro
Apple Object CaptureApple LicenseYesNone
RealityCapture 1.5Free (Epic Account)YesNone
Scaniverse 4FreeYesNone
MeshroomGPL v3Yes (source disclosure)None

Decision branches for production. Use INRIA original for academic baselines, Nerfstudio plus gsplat for self-hosted services, paid Postshot for fast GUI work, Polycam Pro or Scaniverse for mobile and field capture.

Chapter 20 · One-page Decision Matrix

ScenarioFirst ChoiceSecond ChoiceNotes
Fast iPhone captureScaniverse 4PolycamBoth on-device
100 photos, desktopPostshot 2.0NerfstudioGUI vs CLI
Academic baselineINRIA originalNerfstudioNon-commercial caution
Self-hosted serviceNerfstudio plus .ksplatLuma Capture APILicense and cost
AR Quick Look or Vision ProApple Object CaptureRealityCaptureUSDZ direct
Film VFX or game assetsRealityCapture 1.5MetashapeUnreal integration
Cultural heritageMetashape plus NerfstudioRealityCapturePrecision
Instant generation from textLuma Genie 2Meshy or TripoMesh output
Open source obligationMeshroom plus NerfstudiogsplatGPL or Apache

Chapter 21 · The Next Six Months in One Line

The next two quarters of 3DGS narrow to three threads.

  1. Dynamic Gaussian Splatting — adds a time axis (4DGS). Gaussians move with the video. The 2024 4DGS paper was the starting point; Nerfstudio is expected to ship a native integration in 2026.
  2. AI alignment — pulls semantic labels out of Gaussians, allowing "select this chair" interactions. LangSplat and Feature 3DGS lead this front.
  3. Mobile training — Scaniverse 4 took the first step. As phone-side training stabilizes, zero-cloud-cost capture-to-view becomes default.

One line: 2026's 3D is "shoot it and touch it" 3D. People who do not know polygons or textures shoot a few photos, get 3D, and add detail with a prompt. 3DGS is the fastest path there.

Chapter 22 · Next Step — A One-Week Learning Plan

If you finished this article, here is one good way to spend the next seven days.

Capture is not a tool skill but eye training. After one week, you will start to see how a given scene will look once it is baked.

References

Comments

No comments yet.

Sign in to leave a comment