SpikeNeRF

Codes for CVPR 2024 paper "SpikeNeRF: Learning Neural Radiance Fields from Continuous Spike Stream"

Contents

• Demo
• Code
• Requirements
• Data
• Usage
  • trian
  • test
• Acknowledgement

Demo

Demo.mp4

YouTube demo video link.

Code

The code is based on nerf-pytorch.

Requirements

git clone https://github.com/BIT-Vision/SpikeNeRF.git
cd SpikeNeRF
pip install -r requirements.txt

Data

You can download the training data we processed from Google Drive. Place the downloaded dataset according to the following directory structure:

├── configs                                                                                                       
│   ├── ...                                                                                     
│                                                                                               
├── data                                                                                                                                                                                                       
│   ├── nerf_llff_data                                                                                                                                                                                                                              
│   │   └── toys  # downloaded llff dataset                                                                                  
|   |   └── ...
|   ├── nerf_synthetic
|   |   └── lego
|   |   └── chair # downloaded synthetic dataset
|   |   └── ...

Usage

train

To train SpikeNeRF on different datasets:

python run_spikenerf.py --config configs/{DataType}/{DataSet}.txt

replace {DataType} with nerf_synthetic or nerf_llff_data , and replace {DataSet} with toys | dolls | lego | chair | etc.

test

To test SpikeNeRF trained on different datasets: render the test data：

python run_spikenerf.py --config configs/{DataType}/{DataSet}.txt --render_only --render_test

render the novel view:

python run_spikenerf.py --config configs/{DataType}/{DataSet}.txt --render_only

replace {DataType} with nerf_synthetic or nerf_llff_data , and replace {DataSet} with toys | dolls | lego | chair | etc.

Acknowledgement

This codebase is built upon nerf-pytorch, and thanks to the above open source projects for their help and inspiration.