Monocular Dynamic Gaussian Splatting Overfits

Abstract

Dynamic 3D Gaussian Splatting achieves impressive novel-view synthesis on monocular video by coupling a deformable point cloud with Adaptive Density Control (ADC), but exhibits a severe train–test generalization gap. On the D-NeRF benchmark (8 synthetic scenes) we measure an average gap of 6.18 dB (up to 11 dB per scene) and, through a systematic ablation of every ADC sub-operation (split, clone, prune, frequency, threshold, schedule), identify splitting as the dominant pathway.

Our central finding is that Elastic Energy Regularization (EER)—an isotropic k-NN penalty on the relative deformation of neighboring Gaussians—breaks the log-linear count–gap correlation observed across ablations. This reframes overfitting from a capacity problem to an incoherent deformation problem. We evaluate 48 configurations spanning four axes of control (capacity, deformation complexity, view-dependent encoding, stochastic regularization); capacity control and coherence regularization compound, and GAD+LogiGrow+PTDrop+EER closes 57.4% of the baseline gap.

All findings are on synthetic D-NeRF scenes; real-world validation (HyperNeRF, Deformable-3DGS cross-architecture) is partial and still in progress — see the cross-architecture section.

Method Taxonomy

We organize 8 mitigation methods along 3 axes of control, plus stochastic regularization:

GAD: a BIC-motivated threshold schedule

We adapt the per-iteration gradient threshold as

where K(t) is the current count, N is the number of training pixels, and Δℓ_ema is an EMA of the per-iteration loss improvement. λ is the single tunable knob. The mapping from BIC to this formula is a heuristic (see paper, §6.2); the empirical diminishing-returns exponent we measure (α ≈ 0.04) is too mild to justify the often-quoted O((N/λ)^1/4) growth bound, so we present the bound qualitatively as "sublinear in N".

EER: k-NN elastic strain energy

For a subset of Gaussians i and their k=8 canonical neighbors j, we penalize

where u(x, t) is the deformation offset at time t. This is the discrete elastic strain — physically the correct choice for linear elasticity (Hooke's law penalizes ∂u/∂x, not ∂u). In canonical space the k-NN graph is stable; we rebuild it every 500 iterations and apply a cosine ramp from iteration 3K to 10K.

Cross-Architecture Validation (Deformable-3DGS)

Main experiments are on 4DGS (HexPlane deformation). We ported EER and GAD to Deformable-3DGS (MLP deformation) and ran baseline + EER on three D-NeRF scenes for 20K iterations.

Phase 1: direct-transfer test at D-NeRF-tuned λ=0.05

Direct transfer at λ=0.05 is poor (mean −6% reduction). Why? Deformable-3DGS trains with L1+0.2·(1−SSIM) vs.\ 4DGS's pure L1 — the loss magnitude is roughly 3× larger and λ=0.05 is therefore under-regularized. Our dimensional-analysis note (paper §6.2) predicts the correct λ for Deformable-3DGS is ≈ 0.15–0.30. Testing this directly:

Phase 2: λ sweep on Deformable-3DGS Lego (dimensional-analysis test)

Clean monotonic dose-response. At every λ ≥ 0.15, EER simultaneously reduces the gap by 22–41% AND slightly improves test PSNR — a rare regularizer that gives you both. The coherence mechanism transfers across deformation architectures; the hyperparameter requires per-architecture calibration, exactly as the dimensional-analysis note predicted.

BibTeX

@article{droby2026monodygs,
  author  = {Ahmad Droby},
  title   = {Monocular Dynamic Gaussian Splatting Overfits:
             A Diagnostic Study of Densification in 4D Gaussian Fields},
  journal = {arXiv preprint},
  year    = {2026}
}