Classification using resnet cannot run through cifar10 dataset

[SouthMemory]

I have generate all image files as 28x28x3 .nrrd files and label files as 1x1x1 .nrrd files

There seem to be something wrong with the way I prepare the labels
Please help me if anyone knows how to fix this.

error:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1576, in _create_c_op
c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 2 but is rank 1 for 'worker_0/ResNet/fc/fc' (op: 'MatMul') with input shapes: [1], [1,10].

config:
[img]
csv_file = ./cifar10/model/img_partial.csv
path_to_search = ./cifar10/train_cifar10/
filename_contains = .nrrd
filename_not_contains = label_
spatial_window_size = (32, 32)
interp_order = -1
loader = simpleitk

[label]
csv_file = ./cifar10/model/label_partial.csv
path_to_search = ./cifar10/train_cifar10/
filename_contains = label_, .nrrd
spatial_window_size = (1, 1)
interp_order = -1
loader = simpleitk

############################ custom configuration sections
[CLASSIFICATION]
image = img
label = label
output_prob = False
num_classes = 10
label_normalisation = False

Please some share some demos using NiftyNet to perform Classification tasks,Thanks very much!

[kennethaweberii]

I am having a similar issue using resnet for classification. My input data are 2D nifti images. Any help is appreciated! Thank you, Ken.

python /home/kenweber/NiftyNet-0.5.0/net_classify.py train -c SCI_Ambulation_Classification_config_training.ini

Configuration File:
[image]
path_to_search = /home/kenweber/SCI_Ambulation_Classification/data/training/
filename_contains = subject_,_image
filename_not_contains = _label
spatial_window_size = (512, 512,1)
interp_order = 3

[label]
path_to_search = /home/kenweber/SCI_Ambulation_Classification/data/training/
filename_contains = subject_,_label
filename_not_contains = _image
interp_order = -1
spatial_window_size = (1, 1, 1)

############################## system configuration sections
[SYSTEM]
cuda_devices = 0
num_gpus = 1
num_threads = 1
model_dir = /home/kenweber/SCI_Ambulation_Classification/model/

[NETWORK]
name = resnet
activation_function = relu
batch_size = 4
queue_length = 20
window_sampling = resize

histogram normalisation

histogram_ref_file = /home/kenweber/SCI_Ambulation_Classification/model/standardisation_models.txt
norm_type = percentile
cutoff = (0.001, 1.000)
normalisation = True
whitening = True
normalise_foreground_only = True
foreground_type = threshold_plus
multimod_foreground_type = and

[TRAINING]
lr = 0.0003
loss_type = CrossEntropy
starting_iter = 0
save_every_n = 500
validation_every_n = 10
validation_max_iter = 1

do_elastic_deformation = False
#deformation_sigma = 50
#num_ctrl_points = 6
#proportion_to_deform=0.9
exclude_fraction_for_validation=0.1
exclude_fraction_for_inference=0.1

[INFERENCE]
#inference_iter = -1
save_seg_dir = /home/kenweber/SCI_Ambulation_Classification/data/output
output_interp_order = -1
spatial_window_size = (512,512)

############################ custom configuration sections
[CLASSIFICATION]
image = image
label = label
output_prob = False
num_classes = 2
label_normalisation = False

Output and Error:
python /home/kenweber/NiftyNet-0.5.0/net_classify.py train -c SCI_Ambulation_Classification_config_training.ini
NiftyNet version 0.5.0+3.g2fda8676.dirty
[CUSTOM]
-- num_classes: 2
-- output_prob: False
-- label_normalisation: False
-- image: ('image',)
-- sampler: ()
-- label: ('label',)
-- inferred: ()
-- name: net_classify
[CONFIG_FILE]
-- path: /home/kenweber/SCI_Ambulation_Classification/scripts/config/SCI_Ambulation_Classification_config_training.ini
[IMAGE]
-- csv_file:
-- path_to_search: /home/kenweber/SCI_Ambulation_Classification/data/training/
-- filename_contains: ('subject_', '_image')
-- filename_not_contains: ('label',)
-- filename_removefromid:
-- interp_order: 3
-- loader: None
-- pixdim: ()
-- axcodes: ()
-- spatial_window_size: (512, 512, 1)
[LABEL]
-- csv_file:
-- path_to_search: /home/kenweber/SCI_Ambulation_Classification/data/training/
-- filename_contains: ('subject', '_label')
-- filename_not_contains: ('_image',)
-- filename_removefromid:
-- interp_order: -1
-- loader: None
-- pixdim: ()
-- axcodes: ()
-- spatial_window_size: (1, 1, 1)
[SYSTEM]
-- cuda_devices: 0
-- num_threads: 1
-- num_gpus: 1
-- model_dir: /home/kenweber/SCI_Ambulation_Classification/model
-- dataset_split_file: ./dataset_split.csv
-- event_handler: ('model_saver', 'model_restorer', 'sampler_threading', 'apply_gradients', 'output_interpreter', 'console_logger', 'tensorboard_logger')
-- iteration_generator: iteration_generator
-- action: training
[NETWORK]
-- name: resnet
-- activation_function: relu
-- batch_size: 4
-- smaller_final_batch_mode: pad
-- decay: 0.0
-- reg_type: L2
-- volume_padding_size: (0, 0, 0)
-- volume_padding_mode: minimum
-- window_sampling: resize
-- queue_length: 20
-- multimod_foreground_type: and
-- histogram_ref_file: /home/kenweber/SCI_Ambulation_Classification/model/standardisation_models.txt
-- norm_type: percentile
-- cutoff: (0.001, 1.0)
-- foreground_type: threshold_plus
-- normalisation: True
-- whitening: True
-- normalise_foreground_only: True
-- weight_initializer: he_normal
-- bias_initializer: zeros
-- keep_prob: 1.0
-- weight_initializer_args: {}
-- bias_initializer_args: {}
[TRAINING]
-- optimiser: adam
-- sample_per_volume: 1
-- rotation_angle: ()
-- rotation_angle_x: ()
-- rotation_angle_y: ()
-- rotation_angle_z: ()
-- scaling_percentage: ()
-- antialiasing: True
-- bias_field_range: ()
-- bf_order: 3
-- random_flipping_axes: -1
-- do_elastic_deformation: False
-- num_ctrl_points: 4
-- deformation_sigma: 15
-- proportion_to_deform: 0.5
-- lr: 0.0003
-- loss_type: CrossEntropy
-- starting_iter: 0
-- save_every_n: 500
-- tensorboard_every_n: 20
-- max_iter: 10000
-- max_checkpoints: 100
-- validation_every_n: 10
-- validation_max_iter: 1
-- exclude_fraction_for_validation: 0.1
-- exclude_fraction_for_inference: 0.1
-- vars_to_restore:
-- vars_to_freeze:
[INFERENCE]
-- spatial_window_size: (512, 512, 1)
-- inference_iter: -1
-- dataset_to_infer:
-- save_seg_dir: /home/kenweber/SCI_Ambulation_Classification/data/output
-- output_postfix: _niftynet_out
-- output_interp_order: -1
-- border: (0, 0, 0)
INFO:niftynet: set CUDA_VISIBLE_DEVICES to 0
INFO:niftynet: starting classification application
INFO:niftynet: csv_file = not found, writing to "/home/kenweber/SCI_Ambulation_Classification/model/image.csv" instead.
INFO:niftynet: Overwriting existing: "/home/kenweber/SCI_Ambulation_Classification/model/image.csv".
INFO:niftynet: [image] search file folders, writing csv file /home/kenweber/SCI_Ambulation_Classification/model/image.csv
INFO:niftynet: csv_file = not found, writing to "/home/kenweber/SCI_Ambulation_Classification/model/label.csv" instead.
INFO:niftynet: Overwriting existing: "/home/kenweber/SCI_Ambulation_Classification/model/label.csv".
INFO:niftynet: [label] search file folders, writing csv file /home/kenweber/SCI_Ambulation_Classification/model/label.csv
WARNING:niftynet: Loading from existing partitioning file /home/kenweber/SCI_Ambulation_Classification/model/dataset_split.csv, ignoring partitioning ratios.
INFO:niftynet:

Number of subjects 143, input section names: ['subject_id', 'image', 'label']
Dataset partitioning:
-- training 113 cases (79.02%),
-- validation 15 cases (10.49%),
-- inference 15 cases (10.49%).

INFO:niftynet: Image reader: loading 113 subjects from sections ('image',) as input [image]
INFO:niftynet: Image reader: loading 113 subjects from sections ('label',) as input [label]
INFO:niftynet: Image reader: loading 15 subjects from sections ('image',) as input [image]
INFO:niftynet: Image reader: loading 15 subjects from sections ('label',) as input [label]
INFO:niftynet: normalisation histogram reference models ready for image:('image',)
INFO:niftynet: normalisation histogram reference models ready for image:('image',)
2019-08-23 17:10:11.049319: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-08-23 17:10:11.049790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce RTX 2070 major: 7 minor: 5 memoryClockRate(GHz): 1.62
pciBusID: 0000:42:00.0
totalMemory: 7.79GiB freeMemory: 7.60GiB
2019-08-23 17:10:11.049812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-08-23 17:10:11.378434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-23 17:10:11.378485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-08-23 17:10:11.378491: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-08-23 17:10:11.378613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/device:GPU:0 with 7320 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070, pci bus id: 0000:42:00.0, compute capability: 7.5)
INFO:niftynet: reading size of preprocessed images
INFO:niftynet: initialised resize sampler {'image': (1, 512, 512, 1, 1, 1), 'image_location': (1, 7), 'label': (1, 1, 1, 1, 1, 1), 'label_location': (1, 7)}
INFO:niftynet: reading size of preprocessed images
INFO:niftynet: initialised resize sampler {'image': (1, 512, 512, 1, 1, 1), 'image_location': (1, 7), 'label': (1, 1, 1, 1, 1, 1), 'label_location': (1, 7)}
WARNING:niftynet: From /home/kenweber/NiftyNet-0.5.0/niftynet/engine/application_initializer.py:106: calling VarianceScaling.init (from tensorflow.python.ops.init_ops) with distribution=normal is deprecated and will be removed in a future version.
Instructions for updating:
normal is a deprecated alias for truncated_normal
INFO:niftynet: using ResNet
2019-08-23 17:10:11.446274: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-08-23 17:10:11.446322: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-23 17:10:11.446355: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-08-23 17:10:11.446368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-08-23 17:10:11.446472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/device:GPU:0 with 7320 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2070, pci bus id: 0000:42:00.0, compute capability: 7.5)
INFO:niftynet: Initialising Dataset from 113 subjects...
INFO:niftynet: Initialising Dataset from 15 subjects...
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 512, 512, 64)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 256, 256, 128)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
(4, 128, 128, 256)
Traceback (most recent call last):
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1628, in _create_c_op
c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 2 but is rank 1 for 'worker_0/ResNet/fc/fc' (op: 'MatMul') with input shapes: [4], [4,2].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kenweber/NiftyNet-0.5.0/net_classify.py", line 8, in
sys.exit(main())
File "/home/kenweber/NiftyNet-0.5.0/niftynet/init.py", line 142, in main
app_driver.run(app_driver.app)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/engine/application_driver.py", line 189, in run
is_training_action=self.is_training_action)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/engine/application_driver.py", line 270, in create_graph
outputs_collector, gradients_collector)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/application/classification_application.py", line 270, in connect_data_and_network
net_out = self.net(image, **net_args)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 34, in call
return self._op(*args, **kwargs)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 360, in call
return self._call_func(args, kwargs)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
result = self._func(*args, **kwargs)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/network/resnet.py", line 72, in layer_op
return layers.fc(out)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 34, in call
return self._op(*args, **kwargs)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 360, in call
return self._call_func(args, kwargs)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 311, in _call_func
result = self._func(*args, **kwargs)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/fully_connected.py", line 72, in layer_op
name='fc')
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 2057, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4560, in mat_mul
name=name)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in init
control_input_ops)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op
raise ValueError(str(e))
ValueError: Shape must be rank 2 but is rank 1 for 'worker_0/ResNet/fc/fc' (op: 'MatMul') with input shapes: [4], [4,2].

originally defined at:
File "/home/kenweber/NiftyNet-0.5.0/niftynet/network/resnet.py", line 67, in layer_op
layers = self.create()
File "/home/kenweber/NiftyNet-0.5.0/niftynet/network/resnet.py", line 58, in create
fc=FCLayer(self.num_classes)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/fully_connected.py", line 46, in init
super(FCLayer, self).init(name=name)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 58, in init
super(TrainableLayer, self).init(name=name)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 26, in init
self.op = tf.make_template(name, self.layer_op, create_scope_now=True)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 154, in make_template
**kwargs)

originally defined at:
File "/home/kenweber/NiftyNet-0.5.0/niftynet/engine/application_driver.py", line 258, in create_graph
application.initialise_network()
File "/home/kenweber/NiftyNet-0.5.0/niftynet/application/classification_application.py", line 205, in initialise_network
acti_func=self.net_param.activation_function)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/network/resnet.py", line 44, in init
name=name)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/network/base_net.py", line 21, in init
super(BaseNet, self).init(name=name)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 58, in init
super(TrainableLayer, self).init(name=name)
File "/home/kenweber/NiftyNet-0.5.0/niftynet/layer/base_layer.py", line 26, in init
self.op = tf.make_template(name, self.layer_op, create_scope_now=True)
File "/home/kenweber/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/template.py", line 154, in make_template
**kwargs)

[kennethaweberii]

Hello NiftyNet, I am just following up on this. Do you have any suggestions or other examples of how to use NiftyNet for a classification task? Thanks, Ken

[shenao1994]

Hello NiftyNet, I am just following up on this. Do you have any suggestions or other examples of how to use NiftyNet for a classification task? Thanks, Ken

Hello.Did you solve the problem?

[kennethaweberii]

I have not solved the problem. I am still waiting for a response.

[rachidf]

Hi kenneth,
I ran into the same problem as you. I had to change the size of my input images from 2d (64x64x1) to 'real' 3d images of size 64x64x2 where the second slice is an exact copy of the first one. I don't know how much this would affect the classification performance, but this helped me get rid of a similar 'rank' error to what you are getting. Labels can still be of size 1x1x1. I hope this helps.

[shenao1994]

I have not solved the problem. I am still waiting for a response.

I am so so so excited,I have solved the problem with the help of my mentor.You just need to modify the code in the resnet.py file. Specifically, in line 109, out = tf.reduce_mean (tf.nn.relu (layers.bn (out, is_training)), axis = [1, 2, 3] is modified to axis = [1, 2].This can be perfect Solve this problem.I hope this helps.

[shenao1994]

Hi kenneth,
I ran into the same problem as you. I had to change the size of my input images from 2d (64x64x1) to 'real' 3d images of size 64x64x2 where the second slice is an exact copy of the first one. I don't know how much this would affect the classification performance, but this helped me get rid of a similar 'rank' error to what you are getting. Labels can still be of size 1x1x1. I hope this helps.

I did the same in the beginning, but after all, the way to handle 3d images is different from 2d images. You can refer to my answer to solve this problem.

[rachidf]

Hi everyone,
Any ideas how to generate attention/activation for an image classification model?
Thank you and best regards,
Rachid

[rachidf]

I meant activation maps in my previous question ...