Mentions légales du service

Skip to content
Snippets Groups Projects
Commit 8e87318e authored by KOPANAS Georgios's avatar KOPANAS Georgios
Browse files

readme

parent 79f874a3
No related branches found
No related tags found
No related merge requests found
## Point Based Neural Rendering with Per-View Optimization
Georgios Kopanas, Julien Phillip, Thomas Leimkhuler, George Drettakis<br>
[Full Paper](http://www-sop.inria.fr/reves/Basilic/2021/KPLD21/KopanasPointBasedNeuralRenderingPerViewOptimization.pdf)<br>
![Teaser image](./docs/teaser.png)
Abstract: *There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with
Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural
representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS,
but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel-view synthesis. A key element of our approach is our new differentiable point-based pipeline, based
on bi-directional Elliptical Weighted Average splatting, a probabilistic depth test and effective camera selection. We use these
elements together in our neural renderer, that outperforms all previous methods both in quality and speed in almost all scenes
we tested. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel-view synthesis*
## Content
Our implementation consistis of:
1. A python module used for training
2. A plugin project on-top of [sIBR platform](https://sibr.gitlabpages.inria.fr/?page=index.html) that pre-processes the scene and renders interactively the results of the training.
## Recommended Setup
* This is tested only in Windows 10 platform
* A GPU with at least 12GB of memory for small scenes and 24GB for larger scenes, drivers, cuda 10.0 toolkit and cuDNN 7.5
* Python 3.8 64-bit
* Anaconda3 virtual environment
* Cmake at least 3.18.0
* Visual Studio 2019 Community
## Install sIBR
This is a short step-by-step guide to install sIBR with all the necessary projects. For the full sibr documentation please visit [here](https://sibr.gitlabpages.inria.fr/?page=index.html)
```.bash
git clone https://gitlab.inria.fr/sibr/sibr_core.git
cd sibr_core/src/projects/
git clone https://gitlab.inria.fr/gkopanas/pointbased_neural_rendering.git
git clone https://gitlab.inria.fr/jphilip/torchgl_interop.git
cd torchgl_interop
git checkout origin/pbnr -b pbnr
```
## Prepared Scenes
## Training a Scene
### Prepare you own scene
### Run Training script
## Running the Interactive Renderer
docs/hugo_thumb.jpg

282 KiB

docs/museum_thumb.jpg

267 KiB

docs/overview.gif

177 KiB

docs/ponche_thumb.jpg

226 KiB

docs/stairs_thumb.jpg

265 KiB

docs/street_thumb.jpg

279 KiB

docs/teaser.PNG

1.65 MiB

docs/tree_thumb.jpg

249 KiB

docs/truck_thumb.jpg

222 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment