Overview
In this project, I implemented and refined a complete pipeline for 3DGS-based scenes' reconstruction, relighting and rendering in Unity, which is based on 3DGS, GS-IR and UnityGaussianSplatting. The goal was to create a interactive, relightable, realistic and highly configurable scene asset in Unity from 3D construction images.
Key Features
- Various Input Support: In addition to providing real scene datasets, users can also use virtual scene by the RGBD sampler implemented in this project.
-
Extended framework: implemented a new real-time relighting and rendering framework developed in Unity, based on the open-source 3DGS renderer UnityGaussianSplatting, supporting extended point cloud data and its conversion.
-
Realistic rendering: RUGS utilizes a custom rendering pipeline and compute shaders to achieve fast and realistic real-time rendering with image-based lighting (IBL) and physically based rendering (PBR).
- Hybrid rendering: RUGS supports hybrid rendering with virtual objects.
- Virtual lighting: RUGS supports rendering virtual lights with configurable params and shadows.
- Curves editing: RUGS provides curve-based scene editing features to further enhance practicality and demonstrate extensibility.
Conclusion
In this project, I implemented a comprehensive, modular framework for 3D scene reconstruction and relighting. By leveraging the strengths of 3DGS and GS-IR, and integrating them into a flexible Unity-based engine, combining photorealistic reconstruction with physically-based relighting and creative editing tools, this system not only supports faithful reproduction of real-world environments but also enables creative reimagining of their visual appearance under different lighting scenarios, and bridges the gap between neural rendering research and real-time simulation environments. This contributes to ongoing efforts in neural rendering, immersive simulation, and intelligent content generation—paving the way for future research in physically-grounded, editable, and interactive 3D visual computing. Future work will explore improved illumination models, better decomposition methods, more practical functions, etc. to further enhance realism and performance and practicality.
In the last, I used RUGS to reconstruct my dorm room and used it to commemorate.
