Quantitative Fractography Semantic Segmentation

This repository is created to present the training process and the predictions results that are published on the scientific research work: " Toward quantitative fractography using convolutional neural networks "(https://arxiv.org/abs/1908.02242).

The source code is a modification of the code published at image-segmentation-keras developed by https://divamgupta.com, with the addition of some extra tools needed for the training process.

The main objective of publishing this work is to propose a new method for the topographic charactirization of fracture surfaces based on Convolutional Neural Networks and attract the interest of the Fractography research community in order to built on this basis and develop tools that optimize the Quantitative Fractogrphy techniques.

More specifically, the Convolutional Neural Network (CNN) model after being trained in Scanning Electron Microscopy (SEM) images of fracture surfaces is able to identify the intergranular or transgranular fracture modes for any brittle material.

Annotation of the training and validation datasets

The first part of the training of every Convolutional Neural Network (CNN) model involveds the annotation of the images. In our case the dataset is composed by SEM images of the fracture surfaces.

The annotation for the SEM fracture images has been performed with the online open source VGG Image Annotator (http://www.robots.ox.ac.uk/~vgg/software/via/via.html). Using the polygon tool it becomes possible to label the different areas of the SEM images as intergranular or transgranular, while the areas that were more ambiguous or between the borders of adjucent areas were classified as background. Furthermore, the image annotation is a very time consuming task and the introduction of the background label was necessary.

After annotating around 1000 images (with size 640x640), the next step is to convert the annotations into a format that is suitable for the training program. This is done using the Export_annotations.py script in the Convert_VGG_Annotations folder.

Training the network

The code for training the network and performing the predictions is using Keras with Tensorflow as a backend. The train.py code is used to train the network and the following command line arguments need to be defined:

• save_weights_path : directory to save the .hdf5 file of the trained weights
• train_images : directory of the training dataset (/Convert_VGG_Annotations/fracture_images/train)
• val_images : directory of the validation dataset (/Convert_VGG_Annotations/fracture_images/val)
• train_annotations : directory of the annotations of the training dataset (/Convert_VGG_Annotations/annotations/train)
• val_annotations : directory of the annotations of the validation dataset (/Convert_VGG_Annotations/annotations/val)
• logs : directory of Tensorboard log files
• n_classes : number of classes (including the background class)
• input_height : height in pixels of the train and val images(default value: 640)
• input_width : width in pixels of the train and val images(default value: 640)
• start_epoch : initial epoch to start the training - if it is a new training, use the default value zero
• end_epoch : final training epoch
• epoch_steps : number of iterations per epoch
• batch_size : depending on the GPU memory of your computer, define the batch size for training
• val_batch_size : depending on the GPU memory of your computer, define the batch size for training
• init_learning_rate : learning rate of training
• optimizer_name : choose optimizer - _options: rmsprop, adadelta, sgd. Default parameter: adam
• load_weights : directory of the pre-trained weights, in case of continuing the training from previous pre-trained stage. No need to define this in case of starting a new training.

An example of execution command is:

python train.py --save_weights_path="weights/" --logs="logs/" --start_epoch=0 \
                --train_images="Convert_VGG_Annotations/fracture_images/train/" \
                --train_annotations="Convert_VGG_Annotations/annotations/train/" \
                --val_images="Convert_VGG_Annotations/fracture_images/val/" \
                --val_annotations="Convert_VGG_Annotations/annotations/val/" \
                --n_classes=3  --optimizer_name="adadelta" \
                --init_learning_rate=0.00008 --input_height=640 --input_width=640

It is important to note that the training and the validation accuracy should not be very different. High training accuracy with low validation accuracy can be an indication of overfitting and this can result into low accuracy predictions on a new dataset. Our maximum training accuracy was 72.5%, while the maximum validation accuracy reached to 71.3%.

Using Tensorboard it is possible to monitor the training and validation accuracies ( or Losses) and fine-tune the training parameters to acheive the best training of the model.

Predictions

Once the training is completed, the trained parameters of the each layer have been stored on the .hdf5 weights file.

Importing the weights to the predict.py code it becomes possible to classify every pixel of any SEM fracture image of a brittle material as intergranular or transgranular. The pixels that are not classified are considered as background, as it is done in the training process.

To run the predict.py script, it is neccesary to provide the path for the trained weights (save_weights_path), the number of classes (nClasses), the dimensions of the test images that you wish to classify(input_height and input_width) and the directory of the test images. Note that the dimensions of the test image can be different than the dimensions of the images that are used during the training(640x640) and interestringly the predictions on larger images (1280x1280) where equal or even more accurate than the images of the same size as the training images.

So, you can run the following command:

python  predict.py --save_weights_path="weights/trained_weights.hdf5" --test_images="test_dataset/" --n_classes=3 --input_height=640 --input_width=640

And you have classified your fracture images !!!

Prerequisites

• Keras 2.0
• Opencv for Python
• Tensorflow
• Pillow