Problems with dense zero tensors

[PMMon]

Describe the bug
When working with zero tensors (e.g. empty event-based images) two problems arise:

• The conversion from a dense zero tensor (as of torch.zeros()) to a sparse tensor and then back to a dense tensor either results in a) an error (e.g. when using MinkowskiEngine.MinkowskiOps.to_sparse) or b) in an inefficiently large sparse tensor (e.g. when using MinkowskiEngine.MinkowskiOps.to_sparse_all.
• The convolution of a sparse representation of a zero tensor either results in a) an error or b) in an inefficiently large sparse tensor.

---

To Reproduce - A toy example

import torch
import MinkowskiEngine as ME

# Create empty toy image data of batch size 4 and 1 channel
x = torch.zeros(4, 1, 34, 34)

# ========== Conversion issue ==========
to_dense = ME.MinkowskiToDenseTensor(x.shape)

# Convert to sparse data
sparse_data = ME.to_sparse(x)

# a) - Try to convert back to dense data
dense_data = to_dense(sparse_data)      # raises error

# Convert data to_sparse_all representation
sparse_data_all = ME.to_sparse_all(x)       # creates large sparse tensor
print(sparse_data_all.shape)

# b) - Try to convert back to dense data
dense_data = to_dense(sparse_data_all) # raises no error

# ========== Convolution issue ==========

# Define Convolution
conv1 = ME.MinkowskiConvolution(1, 32, 3, dimension=2)

# a) Try convolution on empty image with to_dense representation
out = conv1(sparse_data)        # causes python kernel to fail

# b) Try convolution on empty image with to_dense_all representation
out = conv1(sparse_data_all)        # results in large sparse tensor
print(out.shape)

---

Expected behavior
An efficient and functional handling of zero tensors. Ideally, it is possible to use a sparse representation of zero tensors, similar to the output of MinkowskiEngine.MinkowskiOps.to_sparse, that can be converted back to a dense representation, i.e. information about the original shape of the dense tensor is saved somewhere.

---

Desktop (please complete the following information):

• OS: Ubuntu 20.04
• Python version: 3.8.5
• Pytorch version: 1.9.0
• CUDA version: 10.2
• NVIDIA Driver version: 460.32.03
• Minkowski Engine version: 0.5.4
• Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)

wget -q https://raw.githubusercontent.com/NVIDIA/MinkowskiEngine/master/MinkowskiEngine/diagnostics.py ; python diagnostics.py

==========System==========
Linux-5.4.0-70-generic-x86_64-with-glibc2.29
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"
3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0]
==========Pytorch==========
1.9.0+cu102
torch.cuda.is_available(): True
==========NVIDIA-SMI==========
/usr/bin/nvidia-smi
Driver Version 460.32.03
CUDA Version 11.2
VBIOS Version 84.00.53.00.01
Image Version G001.0000.01.03
==========NVCC==========
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_21:12:58_PST_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0
==========CC==========
/usr/bin/c++
c++ (Ubuntu 8.4.0-3ubuntu2) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

==========MinkowskiEngine==========
0.5.4
MinkowskiEngine compiled with CUDA Support: True
NVCC version MinkowskiEngine is compiled: 11020
CUDART version MinkowskiEngine is compiled: 11020