# UniSHARP: monocular novel-view synthesis extended from perspective cameras to fisheye and panorama

> UniSHARP is the official code for an Insta360 Research paper that predicts a 3D Gaussian representation from a single image and renders novel views. Its contribution is handling universal camera systems, wide-FoV, fisheye and panoramic, not just perspective.

**Insta360-Research-Team/UniSHARP** — Official implementation of UniSHARP: Universal Sharp Monocular View Synthesis

- Repository: https://github.com/Insta360-Research-Team/UniSHARP
- Stars: 350 · Forks: 21
- Language: Python
- License: NOASSERTION
- Published: 2026-09-18 · Updated: 2026-09-18 · Language: en
- Canonical page: https://hysenlabs.com/projects/insta360-research-team-unisharp

## One image in, novel views out, across camera types

UniSHARP is the official implementation of Universal Sharp Monocular View Synthesis, from Insta360 Research and academic collaborators. The README states its scope directly: given a single image from a perspective, wide-FoV, fisheye or panoramic camera, UniSHARP predicts a 3D Gaussian representation and renders high-quality novel views.

The user is a researcher or engineer in 3D reconstruction and view synthesis who needs to work with camera systems beyond the standard perspective pinhole. The README frames UniSHARP as extending SHARP-style photorealistic monocular view synthesis to universal camera systems, which is the specific gap it fills: much prior work assumes a perspective camera, and this handles the wide and distorted optics that real capture devices, especially 360 cameras, actually produce.

That focus is unsurprising given the authors: Insta360 makes panoramic and action cameras, so universal-camera view synthesis is directly relevant to their hardware. The project ships code, an arXiv paper, a Hugging Face demo, and a dataset called OmniRooms, so it is a full research release rather than a method description, though its license is unresolved on GitHub.

## Gaussians from a single image, and the OmniRooms dataset

The method predicts a 3D Gaussian representation from one image and renders novel views from it. Working from a single image is the hard constraint: there is no second view to triangulate depth, so the model must infer 3D structure, which is why the README emphasizes universal camera ray and feature prediction as a dependency.

The dataset is a substantial part of the contribution. The README describes OmniRooms as a panoramic simulation dataset suited to 3D reconstruction, especially 3D Gaussian splatting tasks, consisting of 16 large indoor scenes with multiple rooms and 300k RGB images with depth, collected via AirSim. An OmniRooms-Wide variant projects the panoramas into 130-degree fisheye views. The capture design is specific: for each anchor on a 0.5m voxel grid, one central camera plus 29 cameras randomly sampled within a 30cm cube, all sharing a fixed orientation to isolate translation-induced synthesis, each frame a 1024x2048 equirectangular image.

That fixed-orientation, translation-only design is a deliberate choice to isolate the hard part of view synthesis, moving the camera position rather than just rotating it, which is exactly where monocular methods struggle. The code also lists support for many data sources including RealEstate10K, HM3D, WildRGB-D, DL3DV and ScanNet++ Fisheye, so it is not tied to OmniRooms alone.

## Installing UniSHARP and its external dependencies

UniSHARP is a research codebase with a clear environment setup. The README starts with a clone and a fresh Conda environment:

```bash
git clone https://github.com/Insta360-Research-Team/UniSHARP.git
cd Unisharp
conda create -n unisharp python=3.12 -y
conda activate unisharp
```

PyTorch is installed for your CUDA version, and the README records the versions it was smoke-tested with:

```bash
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0
pip install -r requirements.txt
```

The important detail is the external dependencies, which are not optional. The README says UniSHARP uses UniK3D for universal camera ray and feature prediction, cloned into the project:

```bash
git clone https://github.com/lpiccinelli-eth/UniK3D.git UniK3D
```

and that fisheye rendering depends on the GEER CUDA rasterizer from 3DGEER, cloned into a `3dgeer` directory. So a working setup is UniSHARP plus UniK3D plus 3DGEER, and the CUDA rasterizer means a compatible NVIDIA GPU and CUDA toolchain are required. The README also notes training starts the UniSHARP heads from scratch while loading the original pretrained UniK3D weights through its loader.

## The limitations: monocular ambiguity, heavy dependencies, unclear license

The honest limitations begin with the problem itself. Single-image novel-view synthesis is fundamentally under-constrained: from one image the true 3D structure is ambiguous, so the model infers plausible geometry rather than measures it. UniSHARP produces high-quality results within its trained distribution, but a single image cannot recover what it never saw, and large viewpoint changes into occluded regions are where any monocular method is weakest.

The dependency stack is the practical barrier. UniSHARP needs UniK3D for camera-ray prediction and the 3DGEER GEER CUDA rasterizer for fisheye rendering, both cloned separately and the latter requiring CUDA compilation. That is more moving parts than a self-contained model, and each is a place setup can fail. A compatible NVIDIA GPU is effectively mandatory.

The most consequential ambiguity is legal: GitHub reports the license as NOASSERTION, meaning the repository's license terms are not cleanly resolved. Anyone intending to use UniSHARP beyond reading the code should determine the exact terms, and check the separate licenses of UniK3D, 3DGEER and the OmniRooms dataset, before building on it commercially, since the released artifacts each carry their own conditions.

## Against multi-view Gaussian splatting or a perspective-only method

Two alternatives frame UniSHARP. One is standard multi-view 3D Gaussian splatting, which reconstructs a scene from many images. The other is a perspective-only monocular synthesis method.

Against multi-view splatting, UniSHARP's difference is that it works from a single image, which is far more convenient but far more constrained: multi-view methods measure geometry from overlapping views and produce more reliable reconstructions, while UniSHARP infers it from one shot, trading accuracy for the ability to work when you only have one image. Against a perspective-only monocular method, UniSHARP's difference is universal camera support: it handles wide-FoV, fisheye and panoramic inputs that perspective-only methods cannot, which is the whole reason an Insta360 team built it. Choose multi-view splatting when you have many images and want accurate reconstruction. Choose a perspective-only method if all your input is standard pinhole and you want a simpler pipeline. Choose UniSHARP when your input is a single image from a wide, fisheye or panoramic camera and universal-camera support is the capability you need.

## An unresolved license, research maintenance, and what to verify first

UniSHARP's maintenance is paper-driven: it ships with an arXiv paper, a project page, a Hugging Face demo and the OmniRooms dataset, which is a complete release, but updates will follow research rather than a product cadence. The last push was on 2026-06-11, so the code is recent.

The license question is the thing to resolve before any serious use. Because GitHub reports NOASSERTION, the terms are not automatically clear, and the external dependencies and dataset each have their own licenses. Treat the whole stack, UniSHARP, UniK3D, 3DGEER and OmniRooms, as separate license questions rather than assuming one permissive umbrella.

The concrete first step is to try the hosted Hugging Face demo before building anything locally, since it lets you judge output quality on your kind of input, perspective, fisheye or panoramic, without the multi-repository setup. If the results fit your need, then set up the full stack, confirming a compatible CUDA GPU for the 3DGEER rasterizer, and verify the license terms for your intended use before committing to it.

## Conclusion

UniSHARP is worth adopting if you need novel-view synthesis from a single image taken by a wide-FoV, fisheye or panoramic camera, where perspective-only methods do not apply: predicting a 3D Gaussian representation across universal camera systems, backed by the OmniRooms dataset, is its real contribution. It is the wrong tool when you have many images, where multi-view splatting reconstructs more accurately, or when you cannot supply a CUDA GPU for the 3DGEER rasterizer and the UniK3D dependency. Before committing, try the Hugging Face demo to judge quality on your input type, and resolve the licenses, since GitHub reports the repository as NOASSERTION and UniK3D, 3DGEER and OmniRooms each carry their own terms.

## FAQ

### What camera types does UniSHARP support?

The README says UniSHARP takes a single image from a perspective, wide-FoV, fisheye or panoramic camera and predicts a 3D Gaussian representation to render novel views. Extending monocular view synthesis to these universal camera systems is its main contribution.

### What external dependencies does UniSHARP need?

The README requires UniK3D for universal camera ray and feature prediction and the GEER CUDA rasterizer from 3DGEER for fisheye rendering, both cloned separately. It also needs PyTorch (smoke-tested with 2.8.0) and a CUDA-capable GPU for the rasterizer.

### What is the OmniRooms dataset?

The README describes OmniRooms as a panoramic simulation dataset for 3D reconstruction: 16 large indoor scenes with 300k RGB images and depth, collected via AirSim, with an OmniRooms-Wide variant projecting the panoramas into 130-degree fisheye views.

## Sources

- [Insta360-Research-Team/UniSHARP on GitHub](https://github.com/Insta360-Research-Team/UniSHARP)
- [Issues](https://github.com/Insta360-Research-Team/UniSHARP/issues)
- [README](https://github.com/Insta360-Research-Team/UniSHARP/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/insta360-research-team-unisharp