3d reconstruction example

docs/demo/sparse_tensor_reconstruction.rst

@@ -0,0 +1,73 @@
+3D Sparsity Pattern Reconstruction
+==================================
+
+In this page, we will go over a simple demo example that trains a 3D
+convolutional neural network that reconstructs a 3D sparsity pattern from an
+one-hot vector. This is similar to the `Octree Generating Networks, ICCV'17
+<https://arxiv.org/abs/1703.09438>`_. The input one-hot vector indicates a 3D
+Computer Aided Design (CAD) chairs from the ModelNet40 dataset. Here, we use a
+small subset.
+
+We use :attr:`MinkowskiEngine.MinkowskiConvolutionTranspose` along with
+:attr:`MinkowskiEngine.MinkowskiPruning` to sequentially upsample a voxel by a
+factor of 2 and then remove some of the upsampled voxels to generate target
+shapes.  The general network architecture looks similar to the following
+diagram, but the details might differ.
+
+.. image:: ../images/generative_3d_net.png
+
+
+Before we proceed, please go over `the training and data loading tutorial
+<https://stanfordvl.github.io/MinkowskiEngine/demo/training.html>`_ first.
+
+
+Making a Sparsity Pattern Reconstruction Network
+------------------------------------------------
+
+To create a sparse tensor defined in a 3D grid world from a vector, we need to upsample sequentially from a :math:`1 \times 1 \times 1` resolution voxel. Here, we use a block that consists of :attr:`MinkowskiEngine.MinkowskiConvolutionTranspose`, :attr:`MinkowskiEngine.MinkowskiConvolution`, and :attr:`MinkowskiEngine.MinkowskiPruning`.
+
+During a forward pass, we create two paths for 1) the main features and 2) a sparse voxel classification to remove unnecessary voxels.
+
+.. code-block:: python
+
+   out = upsample_block(z)
+   out_cls = classification(out).F
+   out = pruning(out, out_cls > 0)
+
+
+After multiple steps of upsampling and carving out unnecessary voxels, we have a target sparse tensor. The final reconstruction captures the geometry very accurately. Here, we visualized the hierarchical reconstruction result at each step: upsampling, pruning.
+
+.. image:: ../images/generative_3d_results.gif
+
+
+Running the Example
+-------------------
+
+To train a network, go to the Minkowski Engine root directory, and type
+
+
+.. code-block::
+
+   python -m examples.reconstruction --train
+
+
+To visualize the network prediction after you finished training, type
+
+.. code-block::
+
+   python -m examples.reconstruction
+
+
+.. image:: ../images/demo_reconstruction.png
+
+The program will visualize two 3D shapes. One one the left is the target 3D
+shape, one on the right is the reconstructed network prediction.
+
+
+Using Pretrained Weights
+------------------------
+
+You can also download a pretrained model from here: `modelnet_reconstruction.pth <>`_.
+
+The entire code can be found at `example/reconstruction.py
+<https://github.com/StanfordVL/MinkowskiEngine/blob/master/examples/reconstruction.py>`_.

docs/index.rst

@@ -28,6 +28,7 @@ Table of Contents
    demo/training
    demo/modelnet40_classification
    demo/segmentation
+   demo/sparse_tensor_reconstruction
    demo/interop
    demo/multigpu
    demo/pointnet