From 6c62650e6bbcdde9e0ce8ade13bccafaeebc36f0 Mon Sep 17 00:00:00 2001 From: Alexander Suslov Date: Tue, 13 Jul 2021 14:11:59 +0300 Subject: [PATCH] Update documentation (#830) Updated Readme.md --- README.md | 356 +++++--- docs/compression_algorithms/Quantization.md | 1 + examples/tensorflow/classification/README.md | 42 +- .../tensorflow/object_detection/README.md | 20 +- examples/tensorflow/segmentation/README.md | 12 +- examples/torch/classification/README.md | 46 +- examples/torch/object_detection/README.md | 12 +- .../torch/semantic_segmentation/README.md | 14 +- nncf/tensorflow/README.md | 125 --- tests/torch/test_sanity_third_party.py | 125 --- .../0001-Modifications-for-NNCF-usage.patch | 763 ------------------ third_party_integration/mmdetection/README.md | 39 - 12 files changed, 312 insertions(+), 1243 deletions(-) delete mode 100644 nncf/tensorflow/README.md delete mode 100644 third_party_integration/mmdetection/0001-Modifications-for-NNCF-usage.patch delete mode 100644 third_party_integration/mmdetection/README.md diff --git a/README.md b/README.md index 3859c7ba77a..a93a2d15603 100644 --- a/README.md +++ b/README.md @@ -1,38 +1,52 @@ # Neural Network Compression Framework (NNCF) -This repository contains a PyTorch\*-based framework and samples for neural networks compression. +NNCF provides a suite of advanced algorithms for Neural Networks inference optimization in [OpenVINO™](https://github.com/openvinotoolkit/openvino) with minimal accuracy drop. -The framework is organized as a Python\* package that can be built and used in a standalone mode. -The framework architecture is unified to make it easy to add different compression methods. +NNCF is designed to work with models from [PyTorch](https://pytorch.org/) and [TensorFlow](https://www.tensorflow.org/). -The samples demonstrate the usage of compression algorithms for three different use cases on public models and datasets: Image Classification, Object Detection and Semantic Segmentation. -[Compression results](#nncf-compressed-model-zoo) achievable with the NNCF-powered samples can be found in a table at the end of this document. +NNCF provides samples that demonstrate the usage of compression algorithms for three different use cases on public PyTorch and +TensorFlow models and datasets: Image Classification, Object Detection and Semantic Segmentation. +[Compression results](#nncf-compressed-model-zoo) achievable with the NNCF-powered samples can be found in a table at +the end of this document. + +The framework is organized as a Python\* package that can be built and used in a standalone mode. The framework +architecture is unified to make it easy to add different compression algorithms for both PyTorch and TensorFlow deep +learning frameworks. ## Key Features -- Support of various compression algorithms, applied during a model fine-tuning process to achieve best compression parameters and accuracy: - - [Quantization](./docs/compression_algorithms/Quantization.md) - - [Binarization](./docs/compression_algorithms/Binarization.md) - - [Sparsity](./docs/compression_algorithms/Sparsity.md) - - [Filter pruning](./docs/compression_algorithms/Pruning.md) -- Automatic, configurable model graph transformation to obtain the compressed model. The source model is wrapped by the custom class and additional compression-specific layers are inserted in the graph. -- Common interface for compression methods -- GPU-accelerated layers for faster compressed model fine-tuning -- Distributed training support +- Support of various compression algorithms, applied during a model fine-tuning process to achieve a better performance-accuracy trade-off: + + |Compression algorithm|PyTorch|TensorFlow| + | :--- | :---: | :---: | + |[Quantization](./docs/compression_algorithms/Quantization.md) | Supported | Supported | + |[Mixed-Precision Quantization](./docs/compression_algorithms/Quantization.md#mixed_precision_quantization) | Supported | Not supported | + |[Binarization](./docs/compression_algorithms/Binarization.md) | Supported | Not supported | + |[Sparsity](./docs/compression_algorithms/Sparsity.md) | Supported | Supported | + |[Filter pruning](./docs/compression_algorithms/Pruning.md) | Supported | Supported | + +- Automatic, configurable model graph transformation to obtain the compressed model. + > **NOTE**: Limited support for TensorFlow models. The models created using Sequential or Keras Functional API are only supported. +- Common interface for compression methods. +- GPU-accelerated layers for faster compressed model fine-tuning. +- Distributed training support. - Configuration file examples for each supported compression algorithm. -- Git patches for prominent third-party repositories ([mmdetection](https://github.com/open-mmlab/mmdetection), [huggingface-transformers](https://github.com/huggingface/transformers)) demonstrating the process of integrating NNCF into custom training pipelines -- Exporting compressed models to ONNX\* checkpoints ready for usage with [OpenVINO™ toolkit](https://github.com/openvinotoolkit/). +- Git patches for prominent third-party repository ([huggingface-transformers](https://github.com/huggingface/transformers)) demonstrating the process of integrating NNCF into custom training pipelines. +- Exporting PyTorch compressed models to ONNX\* checkpoints and TensorFlow compressed models to SavedModel or Frozen Graph format, ready to use with [OpenVINO™ toolkit](https://github.com/openvinotoolkit/). ## Usage The NNCF is organized as a regular Python package that can be imported in your target training pipeline script. -The basic workflow is loading a JSON configuration script containing NNCF-specific parameters determining the compression to be applied to your model, and then passing your model along with the configuration script to the `nncf.create_compressed_model` function. -This function returns a wrapped model ready for compression fine-tuning, and handle to the object allowing you to control the compression during the training process: +The basic workflow is loading a JSON configuration script containing NNCF-specific parameters determining the compression to be applied to your model, and then passing your model along with the configuration script to the `create_compressed_model` function. +This function returns a model with additional modifications necessary to enable algorithm-specific compression during fine-tuning and handle to the object allowing you to control the compression during the training process: + +### Usage example with PyTorch ```python import torch import nncf # Important - should be imported directly after torch + from nncf import NNCFConfig -from nncf.torch import create_compressed_model, NNCFConfig, register_default_init_args +from nncf.torch import create_compressed_model, register_default_init_args # Instantiate your uncompressed model from torchvision.models.resnet import resnet50 @@ -42,23 +56,56 @@ model = resnet50() nncf_config = NNCFConfig.from_json("resnet50_int8.json") # Provide data loaders for compression algorithm initialization, if necessary -nncf_config = register_default_init_args(nncf_config, train_loader, criterion=loss_criterion) +import torchvision.datasets as datasets +representative_dataset = datasets.ImageFolder("/path") +init_loader = torch.utils.data.DataLoader(representative_dataset) +nncf_config = register_default_init_args(nncf_config, init_loader) # Apply the specified compression algorithms to the model -comp_ctrl, compressed_model = create_compressed_model(model, nncf_config) +compression_ctrl, compressed_model = create_compressed_model(model, nncf_config) -# Now use compressed_model as a usual torch.nn.Module to fine-tune compression parameters along with the model weights +# Now use compressed_model as a usual torch.nn.Module +# to fine-tune compression parameters along with the model weights # ... the rest of the usual PyTorch-powered training pipeline # Export to ONNX or .pth when done fine-tuning -comp_ctrl.export_model("compressed_model.onnx") +compression_ctrl.export_model("compressed_model.onnx") torch.save(compressed_model.state_dict(), "compressed_model.pth") ``` -For a more detailed description of NNCF usage in your training code, see [Usage.md](./docs/Usage.md). For in-depth examples of NNCF integration, browse the [sample scripts](#model-compression-samples) code, or the [example patches](#third-party-repository-integration) to third-party repositories. +### Usage example with TensorFlow +```python +import tensorflow as tf + +from nncf import NNCFConfig +from nncf.tensorflow import create_compressed_model, register_default_init_args + +# Instantiate your uncompressed model +from tensorflow.keras.applications import ResNet50 +model = ResNet50() + +# Load a configuration file to specify compression +nncf_config = NNCFConfig.from_json("resnet50_int8.json") + +# Provide dataset for compression algorithm initialization +representative_dataset = tf.data.Dataset.list_files("/path/*.jpeg") +nncf_config = register_default_init_args(nncf_config, representative_dataset, batch_size=1) + +# Apply the specified compression algorithms to the model +compression_ctrl, compressed_model = create_compressed_model(model, nncf_config) + +# Now use compressed_model as a usual Keras model +# to fine-tune compression parameters along with the model weights + +# ... the rest of the usual TensorFlow-powered training pipeline + +# Export to Frozen Graph, TensorFlow SavedModel or .h5 when done fine-tuning +compression_ctrl.export_model("compressed_model.pb", save_format='frozen_graph') +``` -For more details about the framework architecture, refer to the [NNCFArchitecture.md](./docs/NNCFArchitecture.md). +For a more detailed description of NNCF usage in your training code, see [this tutorial](docs/Usage.md). +For in-depth examples of NNCF integration, browse the [sample scripts](#model-compression-samples) code, or the [example patches](#third-party-repository-integration) to third-party repositories. ## Model Compression Samples @@ -66,157 +113,206 @@ For more details about the framework architecture, refer to the [NNCFArchitectur For a quicker start with NNCF-powered compression, you can also try the sample scripts, each of which provides a basic training pipeline for classification, semantic segmentation and object detection neural network training correspondingly. To run the samples please refer to the corresponding tutorials: -- [Image Classification sample](examples/torch/classification/README.md) -- [Object Detection sample](examples/torch/object_detection/README.md) -- [Semantic Segmentation sample](examples/torch/semantic_segmentation/README.md) + +- PyTorch samples: + - [Image Classification sample](examples/torch/classification/README.md) + - [Object Detection sample](examples/torch/object_detection/README.md) + - [Semantic Segmentation sample](examples/torch/semantic_segmentation/README.md) +- TensorFlow samples: + - [Image Classification sample](examples/tensorflow/classification/README.md) + - [Object Detection sample](examples/tensorflow/object_detection/README.md) + - [Instance Segmentation sample](examples/tensorflow/semantic_segmentation/README.md) ## Third-party repository integration NNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories. -See [third_party_integration](./third_party_integration) for examples of code modifications (Git patches and base commit IDs are provided) that are necessary to integrate NNCF into select repositories. +### Used by + +- [OpenVINO Training Extensions](https://github.com/openvinotoolkit/training_extensions) + + NNCF is integrated into OpenVINO Training Extensions as model optimization backend. So you can train, optimize and export new models based on the available model templates as well as run exported models with OpenVINO. + +### Git patches for third-party repository +See [third_party_integration](./third_party_integration) for examples of code modifications (Git patches and base commit IDs are provided) that are necessary to integrate NNCF into the following repositories: + - [huggingface-transformers](https://github.com/huggingface/transformers) ## System requirements - Ubuntu\* 18.04 or later (64-bit) - Python\* 3.6.2 or later -- NVidia CUDA\* Toolkit 10.2 or later^ -- PyTorch\* 1.5 or later (1.8.0 not supported, 1.8.1 supported) +- Supported frameworks: + - PyTorch\* >=1.5.0, <=1.8.1 (1.8.0 not supported) + - TensorFlow\* 2.4.2 -*NOTE:* The best known PyTorch version for the current NNCF is 1.8.1, and it is highly recommended to use it. - -^ If a torch package built for specific CUDA version is already present in the environment into which NNCF is being installed, -and if it has a matching base version, then the CUDA version for which the present torch package is targeted will be used. - -Otherwise NNCF will install the latest available torch version from pip, which is targeted to the CUDA version of PyTorch packaging strategy's choosing. -For PyTorch 1.8.1, the default CUDA is 10.2. +This repository is tested on Python* 3.6.2+, PyTorch* 1.8.1 (NVidia CUDA\* Toolkit 10.2) and TensorFlow* 2.4.2 (NVidia CUDA\* Toolkit 11.0). ## Installation We suggest to install or use the package in the [Python virtual environment](https://docs.python.org/3/tutorial/venv.html). +If you want to optimize a model from PyTorch, install PyTorch by following [PyTorch installation guide](https://pytorch.org/get-started/locally/#start-locally). +If you want to optimize a model from TensorFlow, install TensorFlow by following [TensorFlow installation guide](https://www.tensorflow.org/install/). #### As a package built from a checked-out repository: -1) Install the following system dependencies: - -`sudo apt-get install python3-dev` - -2) Install the package and its dependencies by running the following in the repository root directory: - -`python setup.py install` +Install the package and its dependencies by running the following in the repository root directory: +``` +python setup.py install +``` +Alternatively, If you don't install any backend you can install NNCF and PyTorch in one line with: +``` +python setup.py install --torch +``` +Install NNCF and TensorFlow in one line: +``` +python setup.py install --tf +``` _NB_: For launching example scripts in this repository, we recommend replacing the `install` option above with `develop` and setting the `PYTHONPATH` variable to the root of the checked-out repository. #### As a PyPI package: + NNCF can be installed as a regular PyPI package via pip: ``` -sudo apt install python3-dev pip install nncf ``` +Alternatively, If you don't install any backend you can install NNCF and PyTorch in one line with: +``` +pip install nncf[torch] +``` +Install NNCF and TensorFlow in one line: +``` +pip install nncf[tf] +``` #### As a Docker image Use one of the Dockerfiles in the [docker](./docker) directory to build an image with an environment already set up and ready for running NNCF [sample scripts](#model-compression-samples). -**NOTE**: If you want to use sample training scripts provided in the NNCF repository under `examples`, you should install the corresponding Python package dependencies: -``` -pip install -r examples/torch/requirements.txt -``` - ## Contributing Refer to the [CONTRIBUTING.md](./CONTRIBUTING.md) file for guidelines on contributions to the NNCF repository. ## NNCF Compressed Model Zoo -Results achieved using sample scripts and NNCF configuration files provided with this repository. See README.md files for [sample scripts](#model-compression-samples) for links to exact configuration files and final PyTorch checkpoints. - -Quick jump to sample type: - -[Classification](#classification) - -[Object detection](#object-detection) - -[Semantic segmentation](#semantic-segmentation) - -[Natural language processing (3rd-party training pipelines)](#nlp) - -[Object detection (3rd-party training pipelines)](#object-detection-mmdetection) - -[Instance Segmentation (3rd-party training pipelines)](#instance-segmentation-mmdetection) - - +Results achieved using sample scripts, example patches to third-party repositories and NNCF configuration files provided +with this repository. See README.md files for [sample scripts](#model-compression-samples) and [example patches](#third-party-repository-integration) +to find instruction and links to exact configuration files and final checkpoints. +- [PyTorch models](#pytorch-models) + * [Classification](#pytorch_classification) + * [Object detection](#pytorch_object_detection) + * [Semantic segmentation](#pytorch_semantic_segmentation) + * [Natural language processing (3rd-party training pipelines)](#pytorch_nlp) +- [TensorFlow models](#tensorflow-models) + * [Classification](#tensorflow_classification) + * [Object detection](#tensorflow_object_detection) + * [Instance segmentation](#tensorflow_instance_segmentation) + +### PyTorch models + + #### Classification -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|ResNet-50|INT8|ImageNet|76.16|76.42| -|ResNet-50|INT8 (per-tensor only)|ImageNet|76.16|76.37| -|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.16|76.2| -|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|76.16|75.43| -|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|76.16|75.55| -|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|76.16|75.62| -|Inception V3|INT8|ImageNet|77.34|78.25| -|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.34|77.58| -|MobileNet V2|INT8|ImageNet|71.93|71.35| -|MobileNet V2|INT8 (per-tensor only)|ImageNet|71.93|71.3| -|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|71.93|70.92| -|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.93|71.11| -|SqueezeNet V1.1|INT8|ImageNet|58.24|58.28| -|SqueezeNet V1.1|INT8 (per-tensor only)|ImageNet|58.24|58.26| -|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.24|58.9| -|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|69.8|61.63| -|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|69.8|61.61| -|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.8|69.26| -|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.8|69.32| -|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|73.3|72.73| -|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|69.75|68.82| - +|PyTorch Model|Compression algorithm|Dataset|Accuracy (Drop) %| +| :---: | :---: | :---: | :---: | +|ResNet-50|INT8|ImageNet|76.42 (-0.26)| +|ResNet-50|INT8 (per-tensor for weights)|ImageNet|76.37 (-0.21)| +|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.2 (-0.04)| +|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|75.43 (0.73)| +|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|75.55 (0.61)| +|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|75.62 (0.54)| +|Inception V3|INT8|ImageNet|78.25 (-0.91)| +|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.58 (-0.24)| +|MobileNet V2|INT8|ImageNet|71.35 (0.58)| +|MobileNet V2|INT8 (per-tensor for weights)|ImageNet|71.3 (0.63)| +|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|70.92 (1.01)| +|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.11 (0.82)| +|SqueezeNet V1.1|INT8|ImageNet|58.28 (-0.04)| +|SqueezeNet V1.1|INT8 (per-tensor for weights)|ImageNet|58.26 (-0.02)| +|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.9 (-0.66)| +|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|61.63 (8.17)| +|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|61.61 (8.19)| +|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.26 (0.54)| +|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.32 (0.48)| +|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|72.73 (0.57)| +|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|68.82 (0.93)| + + #### Object detection -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|62.23|62.94| -|SSD300-VGG-BN|INT8|VOC12+07 train, VOC07 eval|78.28|77.96| -|SSD300-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|78.28|77.59| -|SSD300-VGG-BN|Filter pruning, 40%, geometric median criterion|VOC12+07 train, VOC07 eval|78.28|77.72| -|SSD512-VGG-BN|INT8|VOC12+07 train, VOC07 eval|80.26|80.12| -|SSD512-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|80.26|79.67| +|PyTorch Model|Compression algorithm|Dataset|mAP (drop) %| +| :---: | :---: | :---: | :---: | +|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|62.94 (-0.71)| +|SSD300-VGG-BN|INT8|VOC12+07 train, VOC07 eval|77.96 (0.32)| +|SSD300-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|77.59 (0.69)| +|SSD300-VGG-BN|Filter pruning, 40%, geometric median criterion|VOC12+07 train, VOC07 eval|77.72 (0.56)| +|SSD512-VGG-BN|INT8|VOC12+07 train, VOC07 eval|80.12 (0.14)| +|SSD512-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|79.67 (0.59)| + #### Semantic segmentation -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|UNet|INT8|CamVid|71.95|71.8| -|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|71.95|72.03| -|ICNet|INT8|CamVid|67.89|67.86| -|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.89|67.18| -|UNet|INT8|Mapillary|56.23|55.87| -|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|56.23|55.65| -|UNet|Filter pruning, 25%, geometric median criterion|Mapillary|56.23|55.62| - - +|PyTorch Model|Compression algorithm|Dataset|Accuracy (Drop) %| +| :---: | :---: | :---: | :---: | +|UNet|INT8|CamVid|71.8 (0.15)| +|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|72.03 (-0.08)| +|ICNet|INT8|CamVid|67.86 (0.03)| +|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.18 (0.71)| +|UNet|INT8|Mapillary|55.87 (0.36)| +|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|55.65 (0.58)| +|UNet|Filter pruning, 25%, geometric median criterion|Mapillary|55.62 (0.61)| + + #### NLP -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|BERT-base-chinese|INT8|XNLI|77.68|77.22| -|BERT-large (Whole Word Masking)|INT8|SQuAD v1.1|93.21 (F1)|92.68 (F1)| -|RoBERTa-large|INT8|MNLI|90.6 (matched)|89.25 (matched)| -|DistilBERT-base|INT8|SST-2|91.1|90.3| -|MobileBERT|INT8|SQuAD v1.1|89.98 (F1)|89.4 (F1)| -|GPT-2|INT8|WikiText-2 (raw)|19.73 (perplexity) | 20.9 (perplexity)| +|PyTorch Model|Compression algorithm|Dataset|Accuracy (Drop) %| +| :---: | :---: | :---: | :---: | +|BERT-base-chinese|INT8|XNLI|77.22 (0.46)| +|BERT-large (Whole Word Masking)|INT8|SQuAD v1.1|F1: 92.68 (0.53)| +|RoBERTa-large|INT8|MNLI|matched: 89.25 (1.35)| +|DistilBERT-base|INT8|SST-2|90.3 (0.8)| +|MobileBERT|INT8|SQuAD v1.1|F1: 89.4 (0.58)| +|GPT-2|INT8|WikiText-2 (raw)|perplexity: 20.9 (-1.17)| +### TensorFlow models -#### Object detection (MMDetection) - -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|RetinaNet-ResNet50-FPN|INT8|COCO2017|35.6 (avg bbox mAP)|35.3 (avg bbox mAP)| -|RetinaNet-ResNet50-FPN|INT8 + Sparsity 50%|COCO2017|35.6 (avg bbox mAP)|34.7 (avg bbox mAP)| -|RetinaNet-ResNeXt101-64x4d-FPN|INT8|COCO2017|39.6 (avg bbox mAP)|39.1 (avg bbox mAP)| + +#### Classification -#### Instance Segmentation (MMDetection) +|Tensorflow Model|Compression algorithm|Dataset|Accuracy (Drop) %| +| :---: | :---: | :---: | :---: | +|Inception V3|INT8 (per-tensor for weights)|ImageNet|78.35 (-0.45)| +|Inception V3|Sparsity 54% (Magnitude)|ImageNet|77.87 (0.03)| +|Inception V3|INT8 (per-tensor for weights) + Sparsity 61% (RB)|ImageNet|77.58 (0.32)| +|MobileNet V2|INT8 (per-tensor for weights)|ImageNet|71.66 (0.19)| +|MobileNet V2|Sparsity 50% (RB)|ImageNet|71.34 (0.51)| +|MobileNet V2|INT8 (per-tensor for weights) + Sparsity 52% (RB)|ImageNet|71.0 (0.85)| +|MobileNet V3 small|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) |ImageNet|67.7 (0.68)| +|MobileNet V3 small|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) + Sparsity 42% (RB)|ImageNet|67.7 (0.68)| +|MobileNet V3 large|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) |ImageNet|75.0 (0.81)| +|MobileNet V3 large|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) + Sparsity 42% (RB)|ImageNet|75.15 (0.66)| +|ResNet50|INT8 (per-tensor for weights)|ImageNet|75.0 (0.04)| +|ResNet50|Sparsity 80% (RB)|ImageNet|74.36 (0.68)| +|ResNet50|INT8 (per-tensor for weightsy) + Sparsity 65% (RB)|ImageNet|74.3 (0.74)| +|ResNet50|Filter Pruning 40%, geometric_median criterion|ImageNet|74.98 (0.06)| +|ResNet50|Filter Pruning 40%, geometric_median criterion + INT8 (per-tensor for weights)|ImageNet|75.08 (-0.04)| +|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.90 (-0.06)| + + +#### Object detection -|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy| -| :---: | :---: | :---: | :---: | :---: | -|Mask-RCNN-ResNet50-FPN|INT8|COCO2017|40.8 (avg bbox mAP), 37.0 (avg segm mAP)|40.6 (avg bbox mAP), 36.5 (avg segm mAP)| +|TensorFlow Model|Compression algorithm|Dataset|mAP (drop) %| +| :---: | :---: | :---: | :---: | +|RetinaNet|INT8 (per-tensor for weights)|COCO2017|33.22 (0.22)| +|RetinaNet|Sparsity 50% (Magnitude)|COCO2017|33.13 (0.31)| +|RetinaNet|Filter Pruning 40%, geometric_median criterion|COCO2017|32.7 (0.74)| +|RetinaNet|Filter Pruning 40%, geometric_median criterion + INT8 (per-tensor for weights)|COCO2017|32.82 (0.62)| +|YOLOv4|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations)|COCO2017|46.15 (0.89)| +|YOLOv4|Sparsity 50% (Magnitude)|COCO2017|46.54 (0.50)| + + +#### Instance segmentation + +|TensorFlow Model|Compression algorithm|Dataset|mAP (drop) %| +| :---: | :---: | :---: | :---: | +|MaskRCNN|INT8 (per-tensor for weights)|COCO2017|bbox: 37.12 (0.21)
segm: 33.52 (0.04)| +|MaskRCNN|Sparsity 50% (Magnitude)|COCO2017|bbox: 36.93 (0.40)
segm: 33.23 (0.33)| ## Citing diff --git a/docs/compression_algorithms/Quantization.md b/docs/compression_algorithms/Quantization.md index 2bf8160a271..20b3b6322ea 100644 --- a/docs/compression_algorithms/Quantization.md +++ b/docs/compression_algorithms/Quantization.md @@ -136,6 +136,7 @@ This regime is used when `"target_device": "CPU"` or `"target_device": "ANY"` se --- + #### Mixed-Precision Quantization Quantization to lower precisions (e.g. 6, 4, 2 bits) is an efficient way to accelerate inference of neural networks. diff --git a/examples/tensorflow/classification/README.md b/examples/tensorflow/classification/README.md index 8e020d652c0..5a7c34572f1 100644 --- a/examples/tensorflow/classification/README.md +++ b/examples/tensorflow/classification/README.md @@ -9,6 +9,14 @@ This sample demonstrates a DL model compression in case of the Image Classificat - Export to Frozen Graph or TensorFlow SavedModel that is supported by the OpenVINO™ toolkit. - Distributed training on multiple GPUs on one machine is supported using [tf.distribute.MirroredStrategy](https://www.tensorflow.org/api_docs/python/tf/distribute/MirroredStrategy). +## Installation + +To work with the sample you should install the corresponding Python package dependencies + +``` +pip install -r examples/tensorflow/requirements.txt +``` + ## Quantize Pretrained Model This scenario demonstrates quantization with fine-tuning of MobileNetV2 on the ImageNet dataset. @@ -102,30 +110,30 @@ To export a model to the OpenVINO IR and run it using the Intel® Deep Learning |**Model**|**Compression algorithm**|**Dataset**|**Accuracy (Drop) %**|**NNCF config file**|**TensorFlow checkpoint**| | :---: | :---: | :---: | :---: | :---: | :---: | -|Inception V3|INT8 w:sym,per-tensor a:sym,per-tensor |ImageNet|78.35 (-0.45)|[inception_v3_imagenet_int8.json](configs/quantization/inception_v3_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_int8_w_sym_t_half_a_sym_t.tar.gz)| -|Inception V3|Sparsity 54% (Magnitude)|ImageNet|77.87 (-0.03)|[inception_v3_imagenet_magnitude_sparsity.json](configs/sparsity/inception_v3_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_sparsity_54.tar.gz)| -|Inception V3|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 61% (RB)|ImageNet|77.58 (-0.32)|[inception_v3_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/inception_v3_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_int8_w_sym_t_half_a_sym_t_rb_sparsity_61.tar.gz)| -|MobileNet V2|INT8 w:sym,per-tensor a:sym,per-tensor |ImageNet|71.66 (-0.19)|[mobilenet_v2_imagenet_int8.json](configs/quantization/mobilenet_v2_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_int8_w_sym_t_half_a_sym_t.tar.gz)| -|MobileNet V2|Sparsity 50% (RB)|ImageNet|71.34 (-0.51)|[mobilenet_v2_imagenet_rb_sparsity.json](configs/sparsity/mobilenet_v2_imagenet_rb_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_rb_sparsity_50.tar.gz)| -|MobileNet V2|int8 w:sym,per-tensor a:sym,per-tensor + sparsity 52% (RB)|ImageNet|71.0 (-0.85)|[mobilenet_v2_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v2_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_int8_w_sym_t_half_a_sym_t_rb_sparsity_52.tar.gz)| -|MobileNet V3 small|INT8 w:sym,per-channel a:asym,per-tensor |ImageNet|67.7 (-0.68)|[mobilenet_v3_small_imagenet_int8.json](configs/quantization/mobilenet_v3_small_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_small_int8_w_sym_ch_half_a_asym_t.tar.gz)| -|MobileNet V3 small|INT8 w:sym,per-channel a:asym,per-tensor + Sparsity 42% (RB)|ImageNet|67.7 (-0.68)|[mobilenet_v3_small_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v3_small_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_small_int8_w_sym_ch_half_a_asym_t_rb_sparsity_42.tar.gz)| -|MobileNet V3 large|INT8 w:sym,per-channel a:asym,per-tensor |ImageNet|75.0 (-0.81)|[mobilenet_v3_large_imagenet_int8.json](configs/quantization/mobilenet_v3_large_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_large_int8_w_sym_ch_half_a_asym_t.tar.gz)| -|MobileNet V3 large|INT8 w:sym,per-channel a:asym,per-tensor + Sparsity 42% (RB)|ImageNet|75.15 (-0.66)|[mobilenet_v3_large_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v3_large_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_large_int8_w_sym_ch_half_a_asym_t_sparsity_42.tar.gz)| -|ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor|ImageNet|75.0 (-0.04)|[resnet50_imagenet_int8.json](configs/quantization/resnet50_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t.tar.gz)| -|ResNet50|Sparsity 80% (RB)|ImageNet|74.36 (-0.68)|[resnet50_imagenet_rb_sparsity.json](configs/sparsity/resnet50_imagenet_rb_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_rb_sparsity_80.tar.gz)| -|ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 65% (RB)|ImageNet|74.3 (-0.74)|[resnet50_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t_rb_sparsity_65.tar.gz)| -|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.90 (+0.06)|[mobilenet_v2_hub_imagenet_magnitude_sparsity.json](configs/sparsity/mobilenet_v2_hub_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/tf1_mobilenet_v2_1.0_224_sparsity_35.tar.gz)| +|Inception V3|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations) |ImageNet|78.35 (-0.45)|[inception_v3_imagenet_int8.json](configs/quantization/inception_v3_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_int8_w_sym_t_half_a_sym_t.tar.gz)| +|Inception V3|Sparsity 54% (Magnitude)|ImageNet|77.87 (0.03)|[inception_v3_imagenet_magnitude_sparsity.json](configs/sparsity/inception_v3_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_sparsity_54.tar.gz)| +|Inception V3|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations) + Sparsity 61% (RB)|ImageNet|77.58 (0.32)|[inception_v3_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/inception_v3_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/inception_v3_int8_w_sym_t_half_a_sym_t_rb_sparsity_61.tar.gz)| +|MobileNet V2|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations) |ImageNet|71.66 (0.19)|[mobilenet_v2_imagenet_int8.json](configs/quantization/mobilenet_v2_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_int8_w_sym_t_half_a_sym_t.tar.gz)| +|MobileNet V2|Sparsity 50% (RB)|ImageNet|71.34 (0.51)|[mobilenet_v2_imagenet_rb_sparsity.json](configs/sparsity/mobilenet_v2_imagenet_rb_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_rb_sparsity_50.tar.gz)| +|MobileNet V2|int8(per-tensor, symmetric for weights; per-tensor, symmetric for activations) + sparsity 52% (RB)|ImageNet|71.0 (0.85)|[mobilenet_v2_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v2_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v2_int8_w_sym_t_half_a_sym_t_rb_sparsity_52.tar.gz)| +|MobileNet V3 small|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) |ImageNet|67.7 (0.68)|[mobilenet_v3_small_imagenet_int8.json](configs/quantization/mobilenet_v3_small_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_small_int8_w_sym_ch_half_a_asym_t.tar.gz)| +|MobileNet V3 small|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) + Sparsity 42% (RB)|ImageNet|67.7 (0.68)|[mobilenet_v3_small_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v3_small_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_small_int8_w_sym_ch_half_a_asym_t_rb_sparsity_42.tar.gz)| +|MobileNet V3 large|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) |ImageNet|75.0 (0.81)|[mobilenet_v3_large_imagenet_int8.json](configs/quantization/mobilenet_v3_large_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_large_int8_w_sym_ch_half_a_asym_t.tar.gz)| +|MobileNet V3 large|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations) + Sparsity 42% (RB)|ImageNet|75.15 (0.66)|[mobilenet_v3_large_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v3_large_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mobilenet_v3_large_int8_w_sym_ch_half_a_asym_t_sparsity_42.tar.gz)| +|ResNet50|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations)|ImageNet|75.0 (0.04)|[resnet50_imagenet_int8.json](configs/quantization/resnet50_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t.tar.gz)| +|ResNet50|Sparsity 80% (RB)|ImageNet|74.36 (0.68)|[resnet50_imagenet_rb_sparsity.json](configs/sparsity/resnet50_imagenet_rb_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_rb_sparsity_80.tar.gz)| +|ResNet50|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations) + Sparsity 65% (RB)|ImageNet|74.3 (0.74)|[resnet50_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t_rb_sparsity_65.tar.gz)| +|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.90 (-0.06)|[mobilenet_v2_hub_imagenet_magnitude_sparsity.json](configs/sparsity/mobilenet_v2_hub_imagenet_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/tf1_mobilenet_v2_1.0_224_sparsity_35.tar.gz)| #### Results for filter pruning |**Model**|**Compression algorithm**|**Dataset**|**Accuracy (Drop) %**|**GFLOPS**|**MParams**|**NNCF config file**|**TensorFlow checkpoint**| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | |ResNet50|None|ImageNet|75.04|7.75 (100%)|25.5 (100%)|-|-| -|ResNet50|Filter Pruning 40%|ImageNet|74.98 (-0.06)|4.29 (55.35%)|15.8 (61.96%)|[Link](configs/pruning/resnet50_imagenet_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_pruning_40.tar.gz)| -|ResNet50|Filter Pruning 40% + INT8 w:sym,per-tensor a:sym,per-tensor|ImageNet|75.08 (+0.04)|4.27 (55.10%)|15.8 (61.96%)|[Link](configs/pruning_quantization/resnet50_imagenet_pruning_geometric_median_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t_pruning_40.tar.gz)| +|ResNet50|Filter Pruning 40%, geometric_median criterion|ImageNet|74.98 (0.06)|4.29 (55.35%)|15.8 (61.96%)|[Link](configs/pruning/resnet50_imagenet_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_pruning_40.tar.gz)| +|ResNet50|Filter Pruning 40%, geometric_median criterion + INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations)|ImageNet|75.08 (-0.04)|4.27 (55.10%)|15.8 (61.96%)|[Link](configs/pruning_quantization/resnet50_imagenet_pruning_geometric_median_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/resnet50_int8_w_sym_t_half_a_sym_t_pruning_40.tar.gz)| #### Results for accuracy-aware compressed training |**Model**|**Compression algorithm**|**Dataset**|**Accuracy (Drop) %**|**NNCF config file**| | :---: | :---: | :---: | :---: | :---: | -|ResNet50|Sparsity 75% (magnitude)|ImageNet|74.42 (-0.62)|[resnet50_imagenet_magnitude_sparsity_accuracy_aware.json](configs/sparsity/resnet50_imagenet_magnitude_sparsity_accuracy_aware.json)| +|ResNet50|Sparsity 75% (magnitude)|ImageNet|74.42 (0.62)|[resnet50_imagenet_magnitude_sparsity_accuracy_aware.json](configs/sparsity/resnet50_imagenet_magnitude_sparsity_accuracy_aware.json)| diff --git a/examples/tensorflow/object_detection/README.md b/examples/tensorflow/object_detection/README.md index a18f5bba3d9..e237aa19d2d 100644 --- a/examples/tensorflow/object_detection/README.md +++ b/examples/tensorflow/object_detection/README.md @@ -12,6 +12,14 @@ The sample receives a configuration file where the training schedule, hyper-para - Export to Frozen Graph or TensorFlow SavedModel that is supported by the OpenVINO™ toolkit. - Distributed training on multiple GPUs on one machine is supported using [tf.distribute.MirroredStrategy](https://www.tensorflow.org/api_docs/python/tf/distribute/MirroredStrategy). +## Installation + +To work with the sample you should install the corresponding Python package dependencies + +``` +pip install -r examples/tensorflow/requirements.txt +``` + ## Quantize Pretrained Model This scenario demonstrates quantization with fine-tuning of RetinaNet with ResNet-50 backbone on the COCO2017 dataset. @@ -147,15 +155,15 @@ To export a model to the OpenVINO IR and run it using the Intel® Deep Learning |**Model**|**Compression algorithm**|**Dataset**|**mAP (drop) %**|**NNCF config file**|**TensorFlow checkpoint**| | :---: | :---: | :---: | :---: | :---: | :---: | |RetinaNet|None|COCO2017|33.44|[retinanet_coco.json](configs/retinanet_coco.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet.tar.gz)| -|RetinaNet|INT8 w:sym,per-tensor a:sym,per-tensor |COCO2017|33.22 (-0.22)|[retinanet_coco_int8.json](configs/quantization/retinanet_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_int8_w_sym_t_half_a_sym_t.tar.gz)| -|RetinaNet|Sparsity 50% (Magnitude)|COCO2017|33.13 (-0.31)|[retinanet_coco_magnitude_sparsity.json](configs/sparsity/retinanet_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_sparsity_50.tar.gz)| +|RetinaNet|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations)|COCO2017|33.22 (0.22)|[retinanet_coco_int8.json](configs/quantization/retinanet_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_int8_w_sym_t_half_a_sym_t.tar.gz)| +|RetinaNet|Sparsity 50% (Magnitude)|COCO2017|33.13 (0.31)|[retinanet_coco_magnitude_sparsity.json](configs/sparsity/retinanet_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_sparsity_50.tar.gz)| |YOLOv4|None|COCO2017|47.04|[yolo_v4_coco.json](configs/yolo_v4_coco.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/yolo_v4.tar.gz)| -|YOLOv4|INT8 w:sym,per-channel a:asym,per-tensor |COCO2017|46.15 (-0.89)|[yolo_v4_coco_int8.json](configs/quantization/yolo_v4_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/yolo_v4_int8_w_sym_ch_half_a_asym_t.tar.gz)| -|YOLOv4|Sparsity 50% (Magnitude)|COCO2017|46.54 (-0.50)|[yolo_v4_coco_magnitude_sparsity.json](configs/sparsity/yolo_v4_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/yolo_v4_sparsity_50.tar.gz)| +|YOLOv4|INT8 (per-channel, symmetric for weights; per-tensor, asymmetric for activations)|COCO2017|46.15 (0.89)|[yolo_v4_coco_int8.json](configs/quantization/yolo_v4_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/yolo_v4_int8_w_sym_ch_half_a_asym_t.tar.gz)| +|YOLOv4|Sparsity 50% (Magnitude)|COCO2017|46.54 (0.50)|[yolo_v4_coco_magnitude_sparsity.json](configs/sparsity/yolo_v4_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/yolo_v4_sparsity_50.tar.gz)| #### Results for filter pruning |**Model**|**Compression algorithm**|**Dataset**|**mAP (drop) %**|**GFLOPS**|**MParams**|**NNCF config file**|**TensorFlow checkpoint**| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | |RetinaNet|None|COCO2017|33.44|194.1 (100%)|60.8 (100%)|[retinanet_coco.json](configs/retinanet_coco.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet.tar.gz)| -|RetinaNet|Filter Pruning 40%|COCO2017|32.7 (-0.74)|107.7 (55.49%)|34.7 (57.07%)|[retinanet_coco_pruning.json](configs/pruning/retinanet_coco_pruning.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_pruning_40.tar.gz)| -|RetinaNet|Filter Pruning 40% + INT8 w:sym,per-tensor a:sym,per-tensor|COCO2017|32.82 (-0.62)|107.6 (55.44%)|34.7 (57.07%)|[retinanet_coco_pruning_int8.json](configs/pruning_quantization/retinanet_coco_pruning_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_int8_w_sym_t_half_a_sym_t_pruning_40.tar.gz)| +|RetinaNet|Filter Pruning 40%, geometric_median criterion|COCO2017|32.7 (0.74)|107.7 (55.49%)|34.7 (57.07%)|[retinanet_coco_pruning.json](configs/pruning/retinanet_coco_pruning.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_pruning_40.tar.gz)| +|RetinaNet|Filter Pruning 40%, geometric_median criterion + INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations)|COCO2017|32.82 (0.62)|107.6 (55.44%)|34.7 (57.07%)|[retinanet_coco_pruning_int8.json](configs/pruning_quantization/retinanet_coco_pruning_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/retinanet_int8_w_sym_t_half_a_sym_t_pruning_40.tar.gz)| diff --git a/examples/tensorflow/segmentation/README.md b/examples/tensorflow/segmentation/README.md index c7517da5e74..f0d93be6974 100644 --- a/examples/tensorflow/segmentation/README.md +++ b/examples/tensorflow/segmentation/README.md @@ -12,6 +12,14 @@ The sample receives a configuration file where the training schedule, hyper-para - Export to Frozen Graph or TensorFlow SavedModel that is supported by the OpenVINO™ toolkit. - Distributed training on multiple GPUs on one machine is supported using [tf.distribute.MirroredStrategy](https://www.tensorflow.org/api_docs/python/tf/distribute/MirroredStrategy). +## Installation + +To work with the sample you should install the corresponding Python package dependencies + +``` +pip install -r examples/tensorflow/requirements.txt +``` + ## Quantize Pretrained Model This scenario demonstrates quantization with fine-tuning of Mask R-CNN with ResNet-50 backbone on the COCO2017 dataset. @@ -129,5 +137,5 @@ To export a model to the OpenVINO IR and run it using the Intel® Deep Learning |**Model**|**Compression algorithm**|**Dataset**|**mAP (drop) %**|**NNCF config file**|**TensorFlow checkpoint**| | :---: | :---: | :---: | :---: | :---: | :---: | |MaskRCNN|None|COCO2017|bbox: 37.33
segm: 33.56|[Link](configs/mask_rcnn_coco.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mask_rcnn_baseline.tar.gz)| -|MaskRCNN|INT8 w:sym,per-tensor a:sym,per-tensor|COCO2017|bbox: 37.12 (-0.21)
segm: 33.52 (-0.04)|[Link](configs/quantization/mask_rcnn_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mask_rcnn_int8_w_sym_t_half_a_sym_t.tar.gz)| -|MaskRCNN|Sparsity 50% (Magnitude)|COCO2017|bbox: 36.93 (-0.40)
segm: 33.23 (-0.33)|[Link](configs/sparsity/mask_rcnn_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mask_rcnn_sparsity_50.tar.gz)| +|MaskRCNN|INT8 (per-tensor, symmetric for weights; per-tensor, symmetric for activations)|COCO2017|bbox: 37.12 (0.21)
segm: 33.52 (0.04)|[Link](configs/quantization/mask_rcnn_coco_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mask_rcnn_int8_w_sym_t_half_a_sym_t.tar.gz)| +|MaskRCNN|Sparsity 50% (Magnitude)|COCO2017|bbox: 36.93 (0.40)
segm: 33.23 (0.33)|[Link](configs/sparsity/mask_rcnn_coco_magnitude_sparsity.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/tensorflow/models/develop/mask_rcnn_sparsity_50.tar.gz)| diff --git a/examples/torch/classification/README.md b/examples/torch/classification/README.md index 929b82a6fa9..e057c07a3da 100644 --- a/examples/torch/classification/README.md +++ b/examples/torch/classification/README.md @@ -67,23 +67,23 @@ To export a model to the OpenVINO IR and run it using the Intel® Deep Learning |Model|Compression algorithm|Dataset|Accuracy (Drop) %|NNCF config file|PyTorch checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | |ResNet-50|None|ImageNet|76.16|[resnet50_imagenet.json](configs/quantization/resnet50_imagenet.json)|-| -|ResNet-50|INT8|ImageNet|76.42 (+0.26)|[resnet50_imagenet_int8.json](configs/quantization/resnet50_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int8.pth)| -|ResNet-50|INT8 (per-tensor only)|ImageNet|76.37 (+0.21)|[resnet50_imagenet_int8_per_tensor.json](configs/quantization/resnet50_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int8_per_tensor.pth)| -|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.2 (+0.04)|[resnet50_imagenet_mixed_int_manual.json](configs/mixed_precision/resnet50_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int4_int8.pth)| -|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|75.43 (-0.73)|[resnet50_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_rb_sparsity_int8.pth)| -|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|75.55 (-0.61)|[resnet50_imagenet_rb_sparsity50_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity50_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_rb_sparsity50_int8.pth)| +|ResNet-50|INT8|ImageNet|76.42 (-0.26)|[resnet50_imagenet_int8.json](configs/quantization/resnet50_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int8.pth)| +|ResNet-50|INT8 (per-tensor only)|ImageNet|76.37 (-0.21)|[resnet50_imagenet_int8_per_tensor.json](configs/quantization/resnet50_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int8_per_tensor.pth)| +|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.2 (-0.04)|[resnet50_imagenet_mixed_int_manual.json](configs/mixed_precision/resnet50_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_int4_int8.pth)| +|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|75.43 (0.73)|[resnet50_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_rb_sparsity_int8.pth)| +|ResNet-50|INT8 + Sparsity 50% (RB)|ImageNet|75.55 (0.61)|[resnet50_imagenet_rb_sparsity50_int8.json](configs/sparsity_quantization/resnet50_imagenet_rb_sparsity50_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_rb_sparsity50_int8.pth)| |Inception V3|None|ImageNet|77.34|[inception_v3_imagenet.json](configs/quantization/inception_v3_imagenet.json)|-| -|Inception V3|INT8|ImageNet|78.25 (+0.91)|[inception_v3_imagenet_int8.json](configs/quantization/inception_v3_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/inception_v3_imagenet_int8.pth)| -|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.58 (+0.24)|[inception_v3_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/inception_v3_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/inception_v3_imagenet_rb_sparsity_int8.pth)| +|Inception V3|INT8|ImageNet|78.25 (-0.91)|[inception_v3_imagenet_int8.json](configs/quantization/inception_v3_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/inception_v3_imagenet_int8.pth)| +|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.58 (-0.24)|[inception_v3_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/inception_v3_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/inception_v3_imagenet_rb_sparsity_int8.pth)| |MobileNet V2|None|ImageNet|71.93|[mobilenet_v2_imagenet.json](configs/quantization/mobilenet_v2_imagenet.json)|-| -|MobileNet V2|INT8|ImageNet|71.35 (-0.58)|[mobilenet_v2_imagenet_int8.json](configs/quantization/mobilenet_v2_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int8.pth)| -|MobileNet V2|INT8 (per-tensor only)|ImageNet|71.3 (-0.63)|[mobilenet_v2_imagenet_int8_per_tensor.json](configs/quantization/mobilenet_v2_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int8_per_tensor.pth)| -|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|70.92 (-1.01)|[mobilenet_v2_imagenet_mixed_int_manual.json](configs/mixed_precision/mobilenet_v2_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int4_int8.pth)| -|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.11 (-0.82)|[mobilenet_v2_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v2_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_rb_sparsity_int8.pth)| +|MobileNet V2|INT8|ImageNet|71.35 (0.58)|[mobilenet_v2_imagenet_int8.json](configs/quantization/mobilenet_v2_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int8.pth)| +|MobileNet V2|INT8 (per-tensor only)|ImageNet|71.3 (0.63)|[mobilenet_v2_imagenet_int8_per_tensor.json](configs/quantization/mobilenet_v2_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int8_per_tensor.pth)| +|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|70.92 (1.01)|[mobilenet_v2_imagenet_mixed_int_manual.json](configs/mixed_precision/mobilenet_v2_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_int4_int8.pth)| +|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.11 (0.82)|[mobilenet_v2_imagenet_rb_sparsity_int8.json](configs/sparsity_quantization/mobilenet_v2_imagenet_rb_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/mobilenet_v2_imagenet_rb_sparsity_int8.pth)| |SqueezeNet V1.1|None|ImageNet|58.24|[squeezenet1_1_imagenet.json](configs/quantization/squeezenet1_1_imagenet.json)|-| -|SqueezeNet V1.1|INT8|ImageNet|58.28 (+0.04)|[squeezenet1_1_imagenet_int8.json](configs/quantization/squeezenet1_1_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int8.pth)| -|SqueezeNet V1.1|INT8 (per-tensor only)|ImageNet|58.26 (+0.02)|[squeezenet1_1_imagenet_int8_per_tensor.json](configs/quantization/squeezenet1_1_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int8_per_tensor.pth)| -|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.9 (+0.66)|[squeezenet1_1_imagenet_mixed_int_manual.json](configs/mixed_precision/squeezenet1_1_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int4_int8.pth)| +|SqueezeNet V1.1|INT8|ImageNet|58.28 (-0.04)|[squeezenet1_1_imagenet_int8.json](configs/quantization/squeezenet1_1_imagenet_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int8.pth)| +|SqueezeNet V1.1|INT8 (per-tensor only)|ImageNet|58.26 (-0.02)|[squeezenet1_1_imagenet_int8_per_tensor.json](configs/quantization/squeezenet1_1_imagenet_int8_per_tensor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int8_per_tensor.pth)| +|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.9 (-0.66)|[squeezenet1_1_imagenet_mixed_int_manual.json](configs/mixed_precision/squeezenet1_1_imagenet_mixed_int_manual.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/squeezenet1_1_imagenet_int4_int8.pth)| #### Binarization @@ -94,28 +94,28 @@ As an example of NNCF convolution binarization capabilities, you may use the con |Model|Compression algorithm|Dataset|Accuracy (Drop) %|NNCF config file|PyTorch Checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | |ResNet-18|None|ImageNet|69.8|[resnet18_imagenet.json](configs/binarization/resnet18_imagenet.json)|-| -|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|61.63 (-8.17)|[resnet18_imagenet_binarization_xnor.json](configs/binarization/resnet18_imagenet_binarization_xnor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_binarization_xnor.pth)| -|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|61.61 (-8.19)|[resnet18_imagenet_binarization_dorefa.json](configs/binarization/resnet18_imagenet_binarization_dorefa.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_binarization_dorefa.pth)| +|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|61.63 (8.17)|[resnet18_imagenet_binarization_xnor.json](configs/binarization/resnet18_imagenet_binarization_xnor.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_binarization_xnor.pth)| +|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|61.61 (8.19)|[resnet18_imagenet_binarization_dorefa.json](configs/binarization/resnet18_imagenet_binarization_dorefa.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_binarization_dorefa.pth)| ### Results for filter pruning |Model|Compression algorithm|Dataset|Accuracy (Drop) %|GFLOPS|MParams|NNCF config file|PyTorch Checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | |ResNet-50|None|ImageNet|76.16|8.18 (100%)|25.50 (100%)|[Link](configs/quantization/resnet50_imagenet.json)|-| -|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|75.62 (-0.54)|4.58 (56.00%)|16.06 (62.98%)|[Link](configs/pruning/resnet50_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_filter_pruning_geomean.pth)| +|ResNet-50|Filter pruning, 40%, geometric median criterion|ImageNet|75.62 (0.54)|4.58 (56.00%)|16.06 (62.98%)|[Link](configs/pruning/resnet50_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet50_imagenet_filter_pruning_geomean.pth)| |ResNet-18|None|ImageNet|69.8|3.63 (100%)|11.68 (100%)|[Link](configs/binarization/resnet18_imagenet.json)|-| -|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.26 (-0.54)|2.75 (75.75%)|9.23 (79.02%)|[Link](configs/pruning/resnet18_pruning_magnitude.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_filter_pruning_magnitude.pth)| -|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.32 (-0.48)|2.75 (75.75%)|9.23 (79.02%)|[Link](configs/pruning/resnet18_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_filter_pruning_geomean.pth)| +|ResNet-18|Filter pruning, 40%, magnitude criterion|ImageNet|69.26 (0.54)|2.75 (75.75%)|9.23 (79.02%)|[Link](configs/pruning/resnet18_pruning_magnitude.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_filter_pruning_magnitude.pth)| +|ResNet-18|Filter pruning, 40%, geometric median criterion|ImageNet|69.32 (0.48)|2.75 (75.75%)|9.23 (79.02%)|[Link](configs/pruning/resnet18_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet18_imagenet_filter_pruning_geomean.pth)| |ResNet-34|None|ImageNet|73.26|7.33 (100%)|21.78 (100%)|[Link](configs/pruning/resnet34_imagenet.json)|-| -|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|72.72 (-0.54)|5.06 (69.03%)|15.47 (71.03%)|[Link](configs/pruning/resnet34_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet34_imagenet_filter_pruning_geomean.pth)| +|ResNet-34|Filter pruning, 40%, geometric median criterion|ImageNet|72.72 (0.54)|5.06 (69.03%)|15.47 (71.03%)|[Link](configs/pruning/resnet34_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/resnet34_imagenet_filter_pruning_geomean.pth)| |GoogLeNet|None|ImageNet|69.72|2.99 (100%)|6.61 (100%)|[Link](configs/pruning/googlenet_imagenet.json)|-| -|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|68.89 (-0.83)|1.36 (45.48%)|3.47 (52.50%)|[Link](configs/pruning/googlenet_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/googlenet_imagenet_filter_pruning_geomean.pth)| +|GoogLeNet|Filter pruning, 40%, geometric median criterion|ImageNet|68.89 (0.83)|1.36 (45.48%)|3.47 (52.50%)|[Link](configs/pruning/googlenet_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/googlenet_imagenet_filter_pruning_geomean.pth)| ### Results for accuracy-aware compressed training |Model|Compression algorithm|Dataset|Accuracy (Drop) %|NNCF config file| | :---: | :---: | :---: | :---: | :---: | |ResNet-50|None|ImageNet|76.16|[resnet50_imagenet.json](configs/quantization/resnet50_imagenet.json)| -|ResNet-50|Filter pruning, 52.5%, geometric median criterion|ImageNet|75.23 (-0.93)|[resnet50_imagenet_accuracy_aware.json](configs/pruning/resnet50_imagenet_accuracy_aware.json)| +|ResNet-50|Filter pruning, 52.5%, geometric median criterion|ImageNet|75.23 (0.93)|[resnet50_imagenet_accuracy_aware.json](configs/pruning/resnet50_imagenet_accuracy_aware.json)| |ResNet-18|None|ImageNet|69.8|[resnet18_imagenet.json](configs/binarization/resnet18_imagenet.json)| -|ResNet-18|Filter pruning, 50%, geometric median criterion|ImageNet|69.92 (+0.12)|[resnet18_imagenet_accuracy_aware.json](configs/pruning/resnet18_imagenet_accuracy_aware.json)| +|ResNet-18|Filter pruning, 50%, geometric median criterion|ImageNet|69.92 (-0.12)|[resnet18_imagenet_accuracy_aware.json](configs/pruning/resnet18_imagenet_accuracy_aware.json)| diff --git a/examples/torch/object_detection/README.md b/examples/torch/object_detection/README.md index e079bfc4f30..0c3d3006a5b 100644 --- a/examples/torch/object_detection/README.md +++ b/examples/torch/object_detection/README.md @@ -55,16 +55,16 @@ To export a model to OpenVINO IR and run it using Intel Deep Learning Deployment |Model|Compression algorithm|Dataset|mAP (drop) %|NNCF config file|PyTorch checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | |SSD300-MobileNet|None|VOC12+07 train, VOC07 eval|62.23|[ssd300_mobilenet_voc.json](configs/ssd300_mobilenet_voc.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_mobilenet_voc.pth)| -|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|62.94 (+0.71)|[ssd300_mobilenet_voc_magnitude_int8.json](configs/ssd300_mobilenet_voc_magnitude_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_mobilenet_voc_magnitude_sparsity_int8.pth)| +|SSD300-MobileNet|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|62.94 (-0.71)|[ssd300_mobilenet_voc_magnitude_int8.json](configs/ssd300_mobilenet_voc_magnitude_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_mobilenet_voc_magnitude_sparsity_int8.pth)| |SSD300-VGG-BN|None|VOC12+07 train, VOC07 eval|78.28|[ssd300_vgg_voc.json](configs/ssd300_vgg_voc.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc.pth)| -|SSD300-VGG-BN|INT8|VOC12+07 train, VOC07 eval|77.96 (-0.32)|[ssd300_vgg_voc_int8.json](configs/ssd300_vgg_voc_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_int8.pth)| -|SSD300-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|77.59 (-0.69)|[ssd300_vgg_voc_magnitude_sparsity_int8.json](configs/ssd300_vgg_voc_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_magnitude_sparsity_int8.pth)| +|SSD300-VGG-BN|INT8|VOC12+07 train, VOC07 eval|77.96 (0.32)|[ssd300_vgg_voc_int8.json](configs/ssd300_vgg_voc_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_int8.pth)| +|SSD300-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|77.59 (0.69)|[ssd300_vgg_voc_magnitude_sparsity_int8.json](configs/ssd300_vgg_voc_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_magnitude_sparsity_int8.pth)| |SSD512-VGG-BN|None|VOC12+07 train, VOC07 eval|80.26|[ssd512_vgg_voc.json](configs/ssd512_vgg_voc.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd512_vgg_voc.pth)| -|SSD512-VGG-BN|INT8|VOC12+07 train, VOC07 eval|80.12 (-0.14)|[ssd512_vgg_voc_int8.json](configs/ssd512_vgg_voc_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd512_vgg_voc_int8.pth)| -|SSD512-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|79.67 (-0.59)|[ssd512_vgg_voc_magnitude_sparsity_int8.json](configs/ssd512_vgg_voc_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd512_vgg_voc_magnitude_sparsity_int8.pth)| +|SSD512-VGG-BN|INT8|VOC12+07 train, VOC07 eval|80.12 (0.14)|[ssd512_vgg_voc_int8.json](configs/ssd512_vgg_voc_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd512_vgg_voc_int8.pth)| +|SSD512-VGG-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07 train, VOC07 eval|79.67 (0.59)|[ssd512_vgg_voc_magnitude_sparsity_int8.json](configs/ssd512_vgg_voc_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd512_vgg_voc_magnitude_sparsity_int8.pth)| #### Results for filter pruning |Model|Compression algorithm|Dataset|mAP (drop) %|GFLOPS|MParams|NNCF config file|PyTorch checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | |SSD300-VGG-BN|None|VOC12+07 train, VOC07 eval|78.28|61.1 (100%)|26.3 (100%)|[Link](configs/ssd300_vgg_voc.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc.pth)| -|SSD300-VGG-BN|Filter pruning 40%,
geometric median criterion|VOC12+07 train, VOC07 eval|77.72 (-0.56)|25.8 (42.23%)|11.4 (43.35%)|[Link](configs/ssd300_vgg_voc_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_pruning_geometric_median.pth)| +|SSD300-VGG-BN|Filter pruning 40%,
geometric median criterion|VOC12+07 train, VOC07 eval|77.72 (0.56)|25.8 (42.23%)|11.4 (43.35%)|[Link](configs/ssd300_vgg_voc_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/ssd300_vgg_voc_pruning_geometric_median.pth)| diff --git a/examples/torch/semantic_segmentation/README.md b/examples/torch/semantic_segmentation/README.md index 7e33aac5cd8..17931e11885 100644 --- a/examples/torch/semantic_segmentation/README.md +++ b/examples/torch/semantic_segmentation/README.md @@ -58,17 +58,17 @@ To export a model to OpenVINO IR and run it using Intel Deep Learning Deployment |Model|Compression algorithm|Dataset|Accuracy (Drop) %|NNCF config file|PyTorch checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | |UNet|None|CamVid|71.95|[unet_camvid.json](configs/unet_camvid.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_camvid.pth)| -|UNet|INT8|CamVid|71.8 (-0.15)|[unet_camvid_int8.json](configs/unet_camvid_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_camvid_int8.pth)| -|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|72.03 (+0.08)|[unet_camvid_magnitude_sparsity_int8.json](configs/unet_camvid_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_camvid_magnitude_sparsity_int8.pth)| +|UNet|INT8|CamVid|71.8 (0.15)|[unet_camvid_int8.json](configs/unet_camvid_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_camvid_int8.pth)| +|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|72.03 (-0.08)|[unet_camvid_magnitude_sparsity_int8.json](configs/unet_camvid_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_camvid_magnitude_sparsity_int8.pth)| |ICNet|None|CamVid|67.89|[icnet_camvid.json](configs/icnet_camvid.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/icnet_camvid.pth)| -|ICNet|INT8|CamVid|67.86 (-0.03)|[icnet_camvid_int8.json](configs/icnet_camvid_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/icnet_camvid_int8.pth)| -|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.18 (-0.71)|[icnet_camvid_magnitude_sparsity_int8.json](configs/icnet_camvid_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/icnet_camvid_magnitude_sparsity_int8.pth)| +|ICNet|INT8|CamVid|67.86 (0.03)|[icnet_camvid_int8.json](configs/icnet_camvid_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/icnet_camvid_int8.pth)| +|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.18 (0.71)|[icnet_camvid_magnitude_sparsity_int8.json](configs/icnet_camvid_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/icnet_camvid_magnitude_sparsity_int8.pth)| |UNet|None|Mapillary|56.23|[unet_mapillary.json](configs/unet_mapillary.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary.pth)| -|UNet|INT8|Mapillary|55.87 (-0.36)|[unet_mapillary_int8.json](configs/unet_mapillary_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_int8.pth)| -|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|55.65 (-0.58)|[unet_mapillary_magnitude_sparsity_int8.json](configs/unet_mapillary_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_magnitude_sparsity_int8.pth)| +|UNet|INT8|Mapillary|55.87 (0.36)|[unet_mapillary_int8.json](configs/unet_mapillary_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_int8.pth)| +|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|55.65 (0.58)|[unet_mapillary_magnitude_sparsity_int8.json](configs/unet_mapillary_magnitude_sparsity_int8.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_magnitude_sparsity_int8.pth)| #### Results for filter pruning |Model|Compression algorithm|Dataset|Accuracy (Drop) %|GFLOPS|MParams|NNCF config file|PyTorch checkpoint| | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | |UNet|None|Mapillary|56.23|875.0 (100%)|31.0 (100%)|[Link](configs/unet_mapillary.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary.pth)| -|UNet|Filter pruning 25%,
geometric median criterion|Mapillary|55.62 (-0.61)|589.4 (67.36%)|18.6 (60.00%)|[Link](configs/unet_mapillary_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_pruning_geometric_median.pth)| +|UNet|Filter pruning 25%,
geometric median criterion|Mapillary|55.62 (0.61)|589.4 (67.36%)|18.6 (60.00%)|[Link](configs/unet_mapillary_pruning_geometric_median.json)|[Link](https://storage.openvinotoolkit.org/repositories/nncf/models/develop/unet_mapillary_pruning_geometric_median.pth)| diff --git a/nncf/tensorflow/README.md b/nncf/tensorflow/README.md deleted file mode 100644 index a07cf980563..00000000000 --- a/nncf/tensorflow/README.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -> #### This directory contains the experimental implementation of the Neural Network Compression Framework for TensorFlow. The implementation is subject to change without notice and offers no guarantees, including for the public API. Stay tuned. - ---- - -# Neural Network Compression Framework for TensorFlow (NNCF TF) - -This repository contains a TensorFlow*-based framework and samples for neural networks compression. - -The framework is the implementaiton of the [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf_pytorch) for TensorFlow\*. - -The framework is organized as a Python\* package that can be built and used in a standalone mode. -The framework architecture is unified to make it easy to add different compression methods. - -The samples demonstrate the usage of compression algorithms for two different use cases on public models and datasets: Image Classification, Object Detection. -[Compression results](#nncf-tf-compression-results) achievable with the NNCF-powered samples can be found in a table at the end of this document. - -## Key Features - -- Support of various compression algorithms, applied during a model fine-tuning process to achieve best compression parameters and accuracy: - - Quantization - - Sparsity - - Filter pruning -- Automatic, configurable model graph transformation to obtain the compressed model. The model is wrapped by the custom class and additional compression-specific layers are inserted in the graph. - > **NOTE**: Only Keras models created using Sequential or Keras Functional API are supported. -- Common interface for compression methods. -- Distributed training support. -- Configuration file examples for each supported compression algorithm. -- Exporting compressed models to Frozen Graph or TensorFlow\* SavedModel ready for usage with [OpenVINO™ toolkit](https://github.com/openvinotoolkit/). - -## Usage -The NNCF TF is organized as a regular Python package that can be imported in an arbitrary training script. -The basic workflow is loading a JSON configuration script containing NNCF-specific parameters determining the compression to be applied to your model, and then passing your model along with the configuration script to the `nncf.tensorflow.create_compressed_model` function. -This function returns a transformed model ready for compression fine-tuning, and handle to the object allowing you to control the compression during the training process: - -```python -import tensorflow as tf - -from nncf import NNCFConfig -from nncf.tensorflow import create_compressed_model -from nncf.tensorflow.initialization import register_default_init_args - -# Instantiate your uncompressed model -from tensorflow.keras.applications import ResNet50 -model = ResNet50() - -# Load a configuration file to specify compression -nncf_config = NNCFConfig.from_json("resnet50_imagenet_int8.json") - -# Provide dataset for compression algorithm initialization -dataset = tf.data.Dataset.list_files("/path/*.txt") -nncf_config = register_default_init_args(nncf_config, dataset, batch_size=1) - -# Apply the specified compression algorithms to the model -compression_ctrl, compressed_model = create_compressed_model(model, nncf_config) - -# Now use compressed_model as a usual Keras model - -# ... the rest of the usual TensorFlow-powered training pipeline - -# Export to Frozen Graph, TensorFlow SavedModel or .h5 when done fine-tuning -compression_ctrl.export_model("compressed_model.pb", save_format='frozen_graph') -``` - -## Model Compression Samples - -For a quick start with NNCF-powered compression, you can also try the sample scripts, each of them provides a basic training pipeline for Image Classification and Object Detection correspondingly. - -To run the samples please refer to the corresponding tutorials: -- [Image Classification sample](../../examples/tensorflow/classification/README.md) -- [Object Detection sample](../../examples/tensorflow/object_detection/README.md) -- [Instance Segmentation sample](../../examples/tensorflow/segmentation/README.md) - -## System requirements -- Ubuntu\* 16.04 or later (64-bit) -- Python\* 3.6.2 or later -- NVIDIA CUDA\* Toolkit 11.0 -- TensorFlow\* 2.4.2 - -## NNCF TF compression results - -Achieved using sample scripts and NNCF TF configuration files provided with this repository. See README files for [sample scripts](#model-compression-samples) for links to exact configuration files and pre-trained models. - -Quick jump to the samples: -- [Classification](#classification) -- [Object Detection](#object-detection) -- [Instance Segmentation](#instance-segmentation) - -#### Classification - -|**Model**|**Compression algorithm**|**Dataset**|**TensorFlow FP32 baseline**|**TensorFlow compressed accuracy**| -| :---: | :---: | :---: | :---: | :---: | -|Inception V3|INT8 w:sym,per-tensor a:sym,per-tensor |ImageNet|77.9|78.35| -|Inception V3|Sparsity 54% (Magnitude)|ImageNet|77.9|77.87| -|Inception V3|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 61% (RB)|ImageNet|77.9|77.58| -|MobileNet V2|INT8 w:sym,per-tensor a:sym,per-tensor |ImageNet|71.85|71.66| -|MobileNet V2|Sparsity 50% (RB)|ImageNet|71.85|71.34| -|MobileNet V2|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 52% (RB)|ImageNet|71.85|71.0| -|MobileNet V3 small|INT8 w:sym,per-channel a:asym,per-tensor |ImageNet|68.38|67.7| -|MobileNet V3 small|INT8 w:sym,per-channel a:asym,per-tensor + Sparsity 42% (RB)|ImageNet|68.38|67.7| -|MobileNet V3 large|INT8 w:sym,per-channel a:asym,per-tensor |ImageNet|75.81|75.0| -|MobileNet V3 large|INT8 w:sym,per-channel a:asym,per-tensor + Sparsity 42% (RB)|ImageNet|75.81|75.15| -|ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor|ImageNet|75.04|75.0| -|ResNet50|Sparsity 80% (RB)|ImageNet|75.04|74.36| -|ResNet50|INT8 w:sym,per-tensor a:sym,per-tensor + Sparsity 65% (RB)|ImageNet|75.04|74.3| -|ResNet50|Filter Pruning 40%|ImageNet|75.04|74.98| -|TensorFlow Hub MobileNet V2|Sparsity 35% (Magnitude)|ImageNet|71.84|71.90| - -#### Object detection - -|**Model**|**Compression algorithm**|**Dataset**|**TensorFlow FP32 baseline mAP**|**TensorFlow compressed mAP**| -| :---: | :---: | :---: | :---: | :---: | -|RetinaNet|INT8 w:sym,per-tensor a:sym,per-tensor |COCO2017|33.44|33.22| -|RetinaNet|Sparsity 50% (Magnitude)|COCO2017|33.44|33.13| -|RetinaNet|Filter Pruning 40%|COCO2017|33.44|32.7| - -#### Instance Segmentation - -|**Model**|**Compression algorithm**|**Dataset**|**TensorFlow FP32 baseline mAP**|**TensorFlow compressed mAP**| -| :---: | :---: | :---: | :---: | :---: | -|MaskRCNN|INT8 w:sym,per-tensor a:sym,per-tensor |COCO2017|bbox: 37.33
segm: 33.56|bbox: 37.12
segm: 33.39| -|MaskRCNN|Sparsity 50% (Magnitude)|COCO2017|bbox: 37.33
segm: 33.56|bbox: 36.93
segm: 33.23| - -## Legal Information -[*] Other names and brands may be claimed as the property of others. diff --git a/tests/torch/test_sanity_third_party.py b/tests/torch/test_sanity_third_party.py index d87fddf7740..ca80c380576 100644 --- a/tests/torch/test_sanity_third_party.py +++ b/tests/torch/test_sanity_third_party.py @@ -19,12 +19,10 @@ import pytest from nncf.torch import BKC_TORCH_VERSION -from nncf.torch import BKC_TORCHVISION_VERSION from tests.torch.helpers import Command from tests.common.helpers import PROJECT_ROOT TRANSFORMERS_COMMIT = "b0892fa0e8df02d683e05e625b3903209bff362d" -MMDETECTION_COMMIT = "3e902c3afc62693a71d672edab9b22e35f7d4776" INSTALL_PATH = PROJECT_ROOT.parent DATASET_PATH = os.path.join(PROJECT_ROOT, "tests", "torch", "data", "mock_datasets") @@ -227,126 +225,3 @@ def test_convert_to_onnx(self, temp_folder): self.cuda_visible_string), self.TRANS_PATH) runner.run() assert os.path.exists(os.path.join(temp_folder["models"], "squad", "model.onnx")) - - -# pylint:disable=line-too-long -class TestMmdetection: - @pytest.fixture(autouse=True) - def setup(self, temp_folder): - self.PATH_TO_PATCH = str(os.path.join(PROJECT_ROOT, "third_party_integration", "mmdetection", - "0001-Modifications-for-NNCF-usage.patch")) - self.VENV_MMDET_PATH = str(os.path.join(temp_folder["venv"], "mmdet")) - self.activate_venv = str("CUDA_HOME=/usr/local/cuda-10.2 . {}/bin/activate".format(self.VENV_MMDET_PATH)) - self.mmdet_python = str("{}/bin/python".format(self.VENV_MMDET_PATH)) - self.MMDET_PATH = str(os.path.join(self.VENV_MMDET_PATH, "mmdetection")) - - @pytest.mark.dependency(name='install_mmdet') - def test_install_mmdet(self, pip_cache_dir): - version_string = "{}.{}".format(sys.version_info[0], sys.version_info[1]) - subprocess.call("virtualenv -ppython{} {}".format(version_string, self.VENV_MMDET_PATH), shell=True) - pip_runner = CachedPipRunner(self.activate_venv, pip_cache_dir) - - pip_runner.run_pip("install --upgrade pip") - subprocess.run( - "{} && {}/bin/python setup.py --torch develop".format(self.activate_venv, self.VENV_MMDET_PATH), check=True, - shell=True, cwd=PROJECT_ROOT) - pip_runner.run_pip("uninstall setuptools -y") - pip_runner.run_pip("install setuptools") - subprocess.run("{}; git clone https://github.com/open-mmlab/mmdetection.git".format(self.activate_venv), - check=True, shell=True, cwd=self.VENV_MMDET_PATH) - subprocess.run("{}; git checkout {}".format(self.activate_venv, MMDETECTION_COMMIT), check=True, shell=True, - cwd=self.MMDET_PATH) - subprocess.run("{}; cp {} .".format(self.activate_venv, self.PATH_TO_PATCH), check=True, shell=True, - cwd=self.MMDET_PATH) - subprocess.run("{}; git apply 0001-Modifications-for-NNCF-usage.patch".format(self.activate_venv), - check=True, shell=True, cwd=self.MMDET_PATH) - pip_runner.run_pip("install mmcv-full==1.2.0 " - "-f https://download.openmmlab.com/mmcv/dist/cu102/torch{}/index.html".format( - BKC_TORCH_VERSION), - cwd=self.MMDET_PATH, - use_cache=False) - pip_runner.run_pip("install onnx onnxruntime", cwd=self.MMDET_PATH) - pip_runner.run_pip("install torchvision=={}".format(BKC_TORCHVISION_VERSION), cwd=self.MMDET_PATH) - pip_runner.run_pip("install -r requirements/build.txt", cwd=self.MMDET_PATH) - pip_runner.run_pip("install -v -e .", cwd=self.MMDET_PATH) - pip_runner.run_pip("install -U \"git+https://github.com/open-mmlab/cocoapi.git#subdirectory=pycocotools\"", - cwd=self.MMDET_PATH) - subprocess.run("{}; mkdir {}".format(self.activate_venv, self.MMDET_PATH + "/data"), check=True, shell=True, - cwd=self.MMDET_PATH) - subprocess.run("{}; ln -s {}/voc data/VOCdevkit".format(self.activate_venv, DATASET_PATH), check=True, - shell=True, - cwd=self.MMDET_PATH) - subprocess.run("{}; ln -s {}/coco data/coco".format(self.activate_venv, DATASET_PATH), check=True, - shell=True, - cwd=self.MMDET_PATH) - - @pytest.mark.dependency(depends=['install_mmdet'], name='ssd300_train') - def test_ssd300_train(self): - subprocess.run( - "wget https://s3.ap-northeast-2.amazonaws.com/open-mmlab/mmdetection/models/ssd300_voc_vgg16_caffe_240e_20190501-7160d09a.pth", - check=True, shell=True, cwd=self.MMDET_PATH) - comm_line = "tools/train.py configs/pascal_voc/ssd300_voc_int8.py" - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - assert os.path.exists(os.path.join(self.MMDET_PATH, "work_dirs", "ssd300_voc_int8", "latest.pth")) - - @pytest.mark.dependency(depends=['install_mmdet', 'ssd300_train']) - def test_ssd300_eval(self): - checkpoint = os.path.join(self.MMDET_PATH, "work_dirs", "ssd300_voc_int8", "latest.pth") - comm_line = "tools/test.py configs/pascal_voc/ssd300_voc_int8.py {} --eval mAP".format(checkpoint) - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - - @pytest.mark.dependency(depends=['install_mmdet', 'ssd300_train']) - def test_ssd300_export2onnx(self): - checkpoint = os.path.join(self.MMDET_PATH, "work_dirs", "ssd300_voc_int8", "latest.pth") - comm_line = "tools/pytorch2onnx.py configs/pascal_voc/ssd300_voc_int8.py {} --output-file ssd300_voc_int8.onnx".format( - checkpoint) - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - assert os.path.exists(os.path.join(self.MMDET_PATH, "ssd300_voc_int8.onnx")) - - @pytest.mark.dependency(depends=['install_mmdet'], name='retinanet_train') - def test_retinanet_train(self): - subprocess.run( - "wget https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmdetection/models/retinanet_r50_fpn_2x_20190616-75574209.pth", - check=True, shell=True, cwd=self.MMDET_PATH) - comm_line = "tools/train.py configs/retinanet/retinanet_r50_fpn_1x_int8.py" - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - assert os.path.exists(os.path.join(self.MMDET_PATH, "work_dirs", "retinanet_r50_fpn_1x_int8", "latest.pth")) - - @pytest.mark.dependency(depends=['install_mmdet', 'retinanet_train']) - def test_retinanet_eval(self): - checkpoint = os.path.join(self.MMDET_PATH, "work_dirs", "retinanet_r50_fpn_1x_int8", "latest.pth") - comm_line = "tools/test.py configs/retinanet/retinanet_r50_fpn_1x_int8.py {} --eval bbox".format(checkpoint) - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - - @pytest.mark.dependency(depends=['install_mmdet', 'retinanet_train']) - def test_retinanet_export2onnx(self): - checkpoint = os.path.join(self.MMDET_PATH, "work_dirs", "retinanet_r50_fpn_1x_int8", "latest.pth") - comm_line = "tools/pytorch2onnx.py configs/retinanet/retinanet_r50_fpn_1x_int8.py {} --output-file retinanet_r50_fpn_1x_int8.onnx".format( - checkpoint) - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - assert os.path.exists(os.path.join(self.MMDET_PATH, "retinanet_r50_fpn_1x_int8.onnx")) - - @pytest.mark.dependency(depends=['install_mmdet'], name='maskrcnn_train') - def test_maskrcnn_train(self): - subprocess.run( - "wget http://download.openmmlab.com/mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth", - check=True, shell=True, cwd=self.MMDET_PATH) - comm_line = "tools/train.py configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py" - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() - assert os.path.exists( - os.path.join(self.MMDET_PATH, "work_dirs", "mask_rcnn_r50_caffe_fpn_1x_coco_int8", "latest.pth")) - - @pytest.mark.dependency(depends=['install_mmdet', 'maskrcnn_train']) - def test_maskrcnn_eval(self): - checkpoint = os.path.join(self.MMDET_PATH, "work_dirs", "mask_rcnn_r50_caffe_fpn_1x_coco_int8", "latest.pth") - comm_line = "tools/test.py configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py {} --eval bbox".format( - checkpoint) - runner = Command(create_command_line(comm_line, self.activate_venv, self.mmdet_python), self.MMDET_PATH) - runner.run() diff --git a/third_party_integration/mmdetection/0001-Modifications-for-NNCF-usage.patch b/third_party_integration/mmdetection/0001-Modifications-for-NNCF-usage.patch deleted file mode 100644 index 71ee5d490a4..00000000000 --- a/third_party_integration/mmdetection/0001-Modifications-for-NNCF-usage.patch +++ /dev/null @@ -1,763 +0,0 @@ -From f75bec2a9b6bc5772e55648c5855decf2f0b57c0 Mon Sep 17 00:00:00 2001 -From: Aleksei Kashapov -Date: Mon, 14 Dec 2020 20:41:54 +0300 -Subject: [PATCH] Modifications for NNCF usage - ---- - .../mask_rcnn_r50_caffe_fpn_1x_coco_int8.py | 63 +++++++++++ - configs/pascal_voc/ssd300_voc_int8.py | 103 +++++++++++++++++ - .../retinanet/retinanet_r50_fpn_1x_int8.py | 49 ++++++++ - mmdet/apis/train.py | 29 ++++- - mmdet/core/__init__.py | 1 + - mmdet/core/export/pytorch2onnx.py | 8 ++ - mmdet/core/nncf/__init__.py | 9 ++ - mmdet/core/nncf/hooks.py | 24 ++++ - mmdet/core/nncf/utils.py | 105 ++++++++++++++++++ - mmdet/models/dense_heads/anchor_head.py | 1 + - mmdet/models/dense_heads/base_dense_head.py | 5 +- - mmdet/models/detectors/base.py | 17 ++- - .../roi_heads/mask_heads/fcn_mask_head.py | 13 ++- - mmdet/models/roi_heads/standard_roi_head.py | 8 +- - tools/pytorch2onnx.py | 24 ++-- - tools/test.py | 20 +++- - tools/train.py | 6 + - 17 files changed, 454 insertions(+), 31 deletions(-) - create mode 100644 configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py - create mode 100644 configs/pascal_voc/ssd300_voc_int8.py - create mode 100644 configs/retinanet/retinanet_r50_fpn_1x_int8.py - create mode 100644 mmdet/core/nncf/__init__.py - create mode 100644 mmdet/core/nncf/hooks.py - create mode 100644 mmdet/core/nncf/utils.py - -diff --git a/configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py b/configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py -new file mode 100644 -index 0000000..a0e77fe ---- /dev/null -+++ b/configs/mask_rcnn/mask_rcnn_r50_caffe_fpn_1x_coco_int8.py -@@ -0,0 +1,63 @@ -+_base_ = './mask_rcnn_r50_fpn_1x_coco.py' -+model = dict( -+ pretrained='open-mmlab://detectron2/resnet50_caffe', -+ backbone=dict(norm_cfg=dict(requires_grad=False), style='caffe')) -+# use caffe img_norm -+img_norm_cfg = dict( -+ mean=[103.530, 116.280, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False) -+train_pipeline = [ -+ dict(type='LoadImageFromFile'), -+ dict(type='LoadAnnotations', with_bbox=True, with_mask=True), -+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True), -+ dict(type='RandomFlip', flip_ratio=0.5), -+ dict(type='Normalize', **img_norm_cfg), -+ dict(type='Pad', size_divisor=32), -+ dict(type='DefaultFormatBundle'), -+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']), -+] -+test_pipeline = [ -+ dict(type='LoadImageFromFile'), -+ dict( -+ type='MultiScaleFlipAug', -+ img_scale=(1333, 800), -+ flip=False, -+ transforms=[ -+ dict(type='Resize', keep_ratio=True), -+ dict(type='RandomFlip'), -+ dict(type='Normalize', **img_norm_cfg), -+ dict(type='Pad', size_divisor=32), -+ dict(type='ImageToTensor', keys=['img']), -+ dict(type='Collect', keys=['img']), -+ ]) -+] -+data = dict( -+ train=dict(pipeline=train_pipeline), -+ val=dict(pipeline=test_pipeline), -+ test=dict(pipeline=test_pipeline)) -+ -+# overload some training params: -+optimizer = dict(type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001) -+optimizer_config = dict(grad_clip=None) -+lr_config = dict( -+ policy='step', -+ warmup='linear', -+ warmup_iters=500, -+ warmup_ratio=0.001, -+ step=[3, 5]) -+total_epochs = 6 -+work_dir = './work_dirs/mask_rcnn_r50_caffe_fpn_1x_coco_int8' -+ -+# we start from the pre-trained checkpoint available from the mmdet github repo -+# download from here: http://download.openmmlab.com/mmdetection/v2.0/mask_rcnn/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco/mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth -+load_from = './mask_rcnn_r50_caffe_fpn_mstrain-poly_3x_coco_bbox_mAP-0.408__segm_mAP-0.37_20200504_163245-42aa3d00.pth' -+ -+find_unused_parameters=True -+ -+# nncf config: -+input_size = (1333, 800) -+nncf_config = dict(input_info=dict(sample_size=[1,3,input_size[0],input_size[1]]), -+ compression=[dict(algorithm="quantization", -+ weights=dict(per_channel=True), -+ initializer=dict(range=dict(num_init_samples=5, -+ type='min_max')))], -+ log_dir=work_dir) -diff --git a/configs/pascal_voc/ssd300_voc_int8.py b/configs/pascal_voc/ssd300_voc_int8.py -new file mode 100644 -index 0000000..a3bcaab ---- /dev/null -+++ b/configs/pascal_voc/ssd300_voc_int8.py -@@ -0,0 +1,103 @@ -+_base_ = [ -+ '../_base_/models/ssd300.py', '../_base_/datasets/voc0712.py', -+ '../_base_/default_runtime.py' -+] -+model = dict( -+ bbox_head=dict( -+ num_classes=20, anchor_generator=dict(basesize_ratio_range=(0.2, -+ 0.9)))) -+ -+test_cfg = dict( -+ nms=dict(type='nms', iou_threshold=0.45, split_thr=10000000000), -+ min_bbox_size=0, -+ score_thr=0.02, -+ max_per_img=200) -+ -+# dataset settings -+dataset_type = 'VOCDataset' -+img_norm_cfg = dict(mean=[123.675, 116.28, 103.53], std=[1, 1, 1], to_rgb=True) -+train_pipeline = [ -+ dict(type='LoadImageFromFile', to_float32=True), -+ dict(type='LoadAnnotations', with_bbox=True), -+ dict( -+ type='PhotoMetricDistortion', -+ brightness_delta=32, -+ contrast_range=(0.5, 1.5), -+ saturation_range=(0.5, 1.5), -+ hue_delta=18), -+ dict( -+ type='Expand', -+ mean=img_norm_cfg['mean'], -+ to_rgb=img_norm_cfg['to_rgb'], -+ ratio_range=(1, 4)), -+ dict( -+ type='MinIoURandomCrop', -+ min_ious=(0.1, 0.3, 0.5, 0.7, 0.9), -+ min_crop_size=0.3), -+ dict(type='Resize', img_scale=(300, 300), keep_ratio=False), -+ dict(type='Normalize', **img_norm_cfg), -+ dict(type='RandomFlip', flip_ratio=0.5), -+ dict(type='DefaultFormatBundle'), -+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']), -+] -+test_pipeline = [ -+ dict(type='LoadImageFromFile'), -+ dict( -+ type='MultiScaleFlipAug', -+ img_scale=(300, 300), -+ flip=False, -+ transforms=[ -+ dict(type='Resize', keep_ratio=False), -+ dict(type='Normalize', **img_norm_cfg), -+ dict(type='ImageToTensor', keys=['img']), -+ dict(type='Collect', keys=['img']), -+ ]) -+] -+data = dict( -+ samples_per_gpu=4, -+ workers_per_gpu=4, -+ train=dict( -+ type='RepeatDataset', times=10, dataset=dict(pipeline=train_pipeline)), -+ val=dict(pipeline=test_pipeline), -+ test=dict(pipeline=test_pipeline)) -+# optimizer -+optimizer = dict(type='SGD', lr=1e-4, momentum=0.9, weight_decay=5e-4) -+optimizer_config = dict() -+# learning policy -+lr_config = dict( -+ policy='step', -+ warmup='linear', -+ warmup_iters=500, -+ warmup_ratio=0.1, -+ step=[16, 20]) -+checkpoint_config = dict(interval=1) -+ -+# yapf:disable -+log_config = dict( -+ interval=50, -+ hooks=[ -+ dict(type='TextLoggerHook'), -+ # dict(type='TensorboardLoggerHook') -+ ]) -+# yapf:enable -+ -+# runtime settings -+total_epochs = 24 -+ -+dist_params = dict(backend='nccl') -+log_level = 'INFO' -+work_dir = './work_dirs/ssd300_voc_int8' -+workflow = [('train', 1)] -+ -+# we start from the pre-trained checkpoint available from the mmdet github repo -+# download from here: https://s3.ap-northeast-2.amazonaws.com/open-mmlab/mmdetection/models/ssd300_voc_vgg16_caffe_240e_20190501-7160d09a.pth -+load_from = './ssd300_voc_vgg16_caffe_240e_20190501-7160d09a.pth' -+resume_from = None -+ -+# nncf config -+input_size = (300, 300) -+nncf_config = dict(input_info=dict(sample_size=[1,3,input_size[0],input_size[1]]) -+ ,compression=[dict(algorithm='quantization', -+ initializer=dict(range=dict(num_init_samples=10, -+ type="threesigma")))], -+ log_dir=work_dir) -diff --git a/configs/retinanet/retinanet_r50_fpn_1x_int8.py b/configs/retinanet/retinanet_r50_fpn_1x_int8.py -new file mode 100644 -index 0000000..60ae180 ---- /dev/null -+++ b/configs/retinanet/retinanet_r50_fpn_1x_int8.py -@@ -0,0 +1,49 @@ -+_base_ = [ -+ '../_base_/models/retinanet_r50_fpn.py', -+ '../_base_/datasets/coco_detection.py', -+ '../_base_/schedules/schedule_1x.py', '../_base_/default_runtime.py' -+] -+ -+test_cfg = dict( -+ nms_pre=1000, -+ min_bbox_size=0, -+ score_thr=0.05, -+ nms=dict(type='nms', iou_threshold=0.5, split_thr=10000000000), -+ max_per_img=100) -+ -+# optimizer -+optimizer = dict(type='SGD', lr=0.001, momentum=0.9, weight_decay=0.0001) -+optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=35, norm_type=2)) -+ -+# learning policy -+lr_config = dict( -+ policy='step', -+ warmup='linear', -+ warmup_iters=500, -+ warmup_ratio=1.0 / 10, -+ step=[8, 11]) -+checkpoint_config = dict(interval=1) -+# yapf:disable -+log_config = dict( -+ interval=50, -+ hooks=[ -+ dict(type='TextLoggerHook'), -+ # dict(type='TensorboardLoggerHook') -+ ]) -+# yapf:enable -+# runtime settings -+total_epochs = 3 -+dist_params = dict(backend='nccl') -+log_level = 'INFO' -+work_dir = './work_dirs/retinanet_r50_fpn_1x_int8' -+# we start from the pre-trained checkpoint available from the mmdet github repo -+# download from here: https://open-mmlab.s3.ap-northeast-2.amazonaws.com/mmdetection/models/retinanet_r50_fpn_2x_20190616-75574209.pth -+load_from = './retinanet_r50_fpn_2x_20190616-75574209.pth' -+resume_from = None -+workflow = [('train', 1)] -+# nncf config -+input_size = (1333, 800) -+nncf_config = dict(input_info=dict(sample_size=[1,3,input_size[0],input_size[1]]), -+ compression=[dict(algorithm='quantization', -+ initializer=dict(range=dict(num_init_samples=10)))], -+ log_dir=work_dir) -diff --git a/mmdet/apis/train.py b/mmdet/apis/train.py -index ad17a53..719e9bc 100644 ---- a/mmdet/apis/train.py -+++ b/mmdet/apis/train.py -@@ -4,14 +4,17 @@ import numpy as np - import torch - from mmcv.parallel import MMDataParallel, MMDistributedDataParallel - from mmcv.runner import (HOOKS, DistSamplerSeedHook, EpochBasedRunner, -- Fp16OptimizerHook, OptimizerHook, build_optimizer) -+ Fp16OptimizerHook, OptimizerHook, build_optimizer, load_checkpoint) - from mmcv.utils import build_from_cfg - --from mmdet.core import DistEvalHook, EvalHook -+from mmdet.core import DistEvalHook, EvalHook, Fp16OptimizerHook, CompressionHook - from mmdet.datasets import (build_dataloader, build_dataset, - replace_ImageToTensor) - from mmdet.utils import get_root_logger - -+from nncf.torch.utils import get_all_modules -+from mmdet.core.nncf import wrap_nncf_model -+ - - def set_random_seed(seed, deterministic=False): - """Set random seed. -@@ -68,13 +71,25 @@ def train_detector(model, - seed=cfg.seed) for ds in dataset - ] - -+ if cfg.load_from: -+ load_checkpoint(model=model, filename=cfg.load_from) -+ - # put model on gpus -+ model = model.cuda() -+ -+ # nncf model wrapper -+ if cfg.ENABLE_COMPRESSION: -+ model, compression_ctrl = wrap_nncf_model(model, cfg, data_loaders[0]) -+ print(*get_all_modules(model).keys(), sep="\n") -+ else: -+ compression_ctrl = None -+ - if distributed: - find_unused_parameters = cfg.get('find_unused_parameters', False) - # Sets the `find_unused_parameters` parameter in - # torch.nn.parallel.DistributedDataParallel - model = MMDistributedDataParallel( -- model.cuda(), -+ model, - device_ids=[torch.cuda.current_device()], - broadcast_buffers=False, - find_unused_parameters=find_unused_parameters) -@@ -82,6 +97,9 @@ def train_detector(model, - model = MMDataParallel( - model.cuda(cfg.gpu_ids[0]), device_ids=cfg.gpu_ids) - -+ if cfg.ENABLE_COMPRESSION and distributed: -+ compression_ctrl.distributed() -+ - # build runner - optimizer = build_optimizer(model, cfg.optimizer) - runner = EpochBasedRunner( -@@ -109,6 +127,8 @@ def train_detector(model, - cfg.get('momentum_config', None)) - if distributed: - runner.register_hook(DistSamplerSeedHook()) -+ if cfg.ENABLE_COMPRESSION: -+ runner.register_hook(CompressionHook(compression_ctrl=compression_ctrl)) - - # register eval hooks - if validate: -@@ -143,6 +163,9 @@ def train_detector(model, - hook = build_from_cfg(hook_cfg, HOOKS) - runner.register_hook(hook, priority=priority) - -+ if cfg.ENABLE_COMPRESSION: -+ runner.register_hook(CompressionHook(compression_ctrl=compression_ctrl)) -+ - if cfg.resume_from: - runner.resume(cfg.resume_from) - elif cfg.load_from: -diff --git a/mmdet/core/__init__.py b/mmdet/core/__init__.py -index b075369..d8fc903 100644 ---- a/mmdet/core/__init__.py -+++ b/mmdet/core/__init__.py -@@ -6,3 +6,4 @@ from .fp16 import * # noqa: F401, F403 - from .mask import * # noqa: F401, F403 - from .post_processing import * # noqa: F401, F403 - from .utils import * # noqa: F401, F403 -+from .nncf import * -diff --git a/mmdet/core/export/pytorch2onnx.py b/mmdet/core/export/pytorch2onnx.py -index 8f9309d..91efe16 100644 ---- a/mmdet/core/export/pytorch2onnx.py -+++ b/mmdet/core/export/pytorch2onnx.py -@@ -5,6 +5,7 @@ import numpy as np - import torch - from mmcv.runner import load_checkpoint - -+from ..nncf.utils import wrap_nncf_model - - def generate_inputs_and_wrap_model(config_path, checkpoint_path, input_config): - """Prepare sample input and wrap model for ONNX export. -@@ -81,6 +82,13 @@ def build_model_from_cfg(config_path, checkpoint_path): - # build the model - model = build_detector(cfg.model, train_cfg=None, test_cfg=cfg.test_cfg) - load_checkpoint(model, checkpoint_path, map_location='cpu') -+ -+ # nncf model wrapper -+ if 'nncf_config' in cfg: -+ print('NNCF config: {}'.format(cfg.nncf_config)) -+ cfg.nncf_load_from = checkpoint_path -+ model, compression_ctrl = wrap_nncf_model(model, cfg, None) -+ - model.cpu().eval() - return model - -diff --git a/mmdet/core/nncf/__init__.py b/mmdet/core/nncf/__init__.py -new file mode 100644 -index 0000000..bc743b9 ---- /dev/null -+++ b/mmdet/core/nncf/__init__.py -@@ -0,0 +1,9 @@ -+from .hooks import CompressionHook -+from .utils import wrap_nncf_model -+from .utils import load_checkpoint -+ -+__all__ = [ -+ 'CompressionHook', -+ 'wrap_nncf_model', -+ 'load_checkpoint', -+] -diff --git a/mmdet/core/nncf/hooks.py b/mmdet/core/nncf/hooks.py -new file mode 100644 -index 0000000..d6b3887 ---- /dev/null -+++ b/mmdet/core/nncf/hooks.py -@@ -0,0 +1,16 @@ -+from texttable import Texttable -+from mmcv.runner.hooks.hook import Hook -+ -+ -+class CompressionHook(Hook): -+ def __init__(self, compression_ctrl=None): -+ self.compression_ctrl = compression_ctrl -+ -+ def after_train_iter(self, runner): -+ self.compression_ctrl.scheduler.step() -+ -+ def after_train_epoch(self, runner): -+ self.compression_ctrl.scheduler.epoch_step() -+ -+ def before_run(self, runner): -+ runner.logger.info(self.compression_ctrl.statistics().to_str()) -diff --git a/mmdet/core/nncf/utils.py b/mmdet/core/nncf/utils.py -new file mode 100644 -index 0000000..c903a55 ---- /dev/null -+++ b/mmdet/core/nncf/utils.py -@@ -0,0 +1,109 @@ -+import pathlib -+from collections import OrderedDict -+ -+import torch -+from nncf.torch.initialization import PTInitializingDataLoader -+from nncf.config.structures import QuantizationRangeInitArgs -+from nncf.config.structures import BNAdaptationInitArgs -+ -+from nncf import NNCFConfig -+from nncf.torch import load_state -+from nncf.torch import create_compressed_model -+from nncf.torch import nncf_model_input -+ -+ -+def wrap_nncf_model(model, cfg, data_loader_for_init=None): -+ pathlib.Path(cfg.work_dir).mkdir(parents=True, exist_ok=True) -+ nncf_config = NNCFConfig.from_dict(cfg.nncf_config) -+ -+ if data_loader_for_init is not None: -+ wrapped_loader = MMInitializeDataLoader(data_loader_for_init) -+ -+ nncf_config.register_extra_structs([ -+ QuantizationRangeInitArgs(wrapped_loader), -+ BNAdaptationInitArgs(wrapped_loader), -+ ]) -+ -+ input_size = nncf_config.get( -+ 'sample_size', (1, 3, cfg.input_size[0], cfg.input_size[1]) -+ ) -+ -+ def wrap_inputs(args, kwargs): -+ # during model's forward -+ if 'img' in kwargs: -+ img = kwargs['img'] -+ if isinstance(img, list): -+ assert len(img) == 1, 'Input list must have a length 1' -+ assert torch.is_tensor(img[0]), 'Input for a model must be a tensor' -+ img[0] = nncf_model_input(img[0]) -+ else: -+ assert torch.is_tensor(img), 'Input for a model must be a tensor' -+ img = nncf_model_input(img) -+ kwargs['img'] = img -+ return args, kwargs -+ -+ # during model's export -+ if isinstance(args, tuple): -+ assert torch.is_tensor(args[0][0]), 'Input for a model must be a tensor' -+ for img in args[0]: -+ img = nncf_model_input(img) -+ -+ return args, kwargs -+ -+ def dummy_forward(model): -+ device = next(model.parameters()).device -+ input_args = ([nncf_model_input(torch.randn(input_size).to(device)), ],) -+ input_kwargs = dict(return_loss=False, dummy_forward=True) -+ model(*input_args, **input_kwargs) -+ -+ model.dummy_forward_fn = dummy_forward -+ -+ checkpoint = None -+ if cfg.get('nncf_load_from', None): -+ checkpoint = load_checkpoint(model, cfg.nncf_load_from) -+ -+ compression_ctrl, model = create_compressed_model( -+ model, -+ nncf_config, -+ compression_state=checkpoint, -+ dummy_forward_fn=dummy_forward, -+ wrap_inputs_fn=wrap_inputs, -+ ) -+ -+ return model, compression_ctrl -+ -+ -+def load_checkpoint(model, filename, map_location=None, strict=False): -+ """Load checkpoint from a file or URI. -+ -+ Args: -+ model (Module): Module to load checkpoint. -+ filename (str): Either a filepath or URL or modelzoo://xxxxxxx. -+ map_location (str): Same as :func:`torch.load`. -+ strict (bool): Whether to allow different params for the model and -+ checkpoint. -+ -+ Returns: -+ dict or OrderedDict: The loaded checkpoint. -+ """ -+ # load checkpoint from modelzoo or file or url -+ checkpoint = torch.load(filename, map_location=map_location) -+ # get state_dict from checkpoint -+ if isinstance(checkpoint, OrderedDict): -+ state_dict = checkpoint -+ elif isinstance(checkpoint, dict) and 'state_dict' in checkpoint: -+ state_dict = checkpoint['state_dict'] -+ else: -+ raise RuntimeError('No state_dict found in checkpoint file {}'.format(filename)) -+ _ = load_state(model, state_dict, strict) -+ return checkpoint -+ -+ -+class MMInitializeDataLoader(PTInitializingDataLoader): -+ def get_inputs(self, dataloader_output): -+ # redefined PTInitializingDataLoader because -+ # of DataContainer format in mmdet -+ kwargs = {k: v.data[0] for k, v in dataloader_output.items()} -+ return (), kwargs -+ -+ # get_targets TBD -diff --git a/mmdet/models/dense_heads/anchor_head.py b/mmdet/models/dense_heads/anchor_head.py -index a5bb413..48332a8 100644 ---- a/mmdet/models/dense_heads/anchor_head.py -+++ b/mmdet/models/dense_heads/anchor_head.py -@@ -2,6 +2,7 @@ import torch - import torch.nn as nn - from mmcv.cnn import normal_init - from mmcv.runner import force_fp32 -+from nncf.torch.dynamic_graph.context import no_nncf_trace - - from mmdet.core import (anchor_inside_flags, build_anchor_generator, - build_assigner, build_bbox_coder, build_sampler, -diff --git a/mmdet/models/dense_heads/base_dense_head.py b/mmdet/models/dense_heads/base_dense_head.py -index de11e4a..0fcdc4d 100644 ---- a/mmdet/models/dense_heads/base_dense_head.py -+++ b/mmdet/models/dense_heads/base_dense_head.py -@@ -1,6 +1,7 @@ - from abc import ABCMeta, abstractmethod - - import torch.nn as nn -+from nncf.torch.dynamic_graph.context import no_nncf_trace - - - class BaseDenseHead(nn.Module, metaclass=ABCMeta): -@@ -51,7 +52,9 @@ class BaseDenseHead(nn.Module, metaclass=ABCMeta): - loss_inputs = outs + (gt_bboxes, img_metas) - else: - loss_inputs = outs + (gt_bboxes, gt_labels, img_metas) -- losses = self.loss(*loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore) -+ -+ with no_nncf_trace(): -+ losses = self.loss(*loss_inputs, gt_bboxes_ignore=gt_bboxes_ignore) - if proposal_cfg is None: - return losses - else: -diff --git a/mmdet/models/detectors/base.py b/mmdet/models/detectors/base.py -index 7c6d5e9..488650e 100644 ---- a/mmdet/models/detectors/base.py -+++ b/mmdet/models/detectors/base.py -@@ -135,6 +135,9 @@ class BaseDetector(nn.Module, metaclass=ABCMeta): - if not isinstance(var, list): - raise TypeError(f'{name} must be a list, but got {type(var)}') - -+ if 'dummy_forward' in kwargs: -+ return self.forward_dummy(imgs[0]) -+ - num_augs = len(imgs) - if num_augs != len(img_metas): - raise ValueError(f'num of augmentations ({len(imgs)}) ' -@@ -165,8 +168,12 @@ class BaseDetector(nn.Module, metaclass=ABCMeta): - assert 'proposals' not in kwargs - return self.aug_test(imgs, img_metas, **kwargs) - -- @auto_fp16(apply_to=('img', )) -- def forward(self, img, img_metas, return_loss=True, **kwargs): -+ @abstractmethod -+ def forward_dummy(self, img, **kwargs): -+ pass -+ -+ @auto_fp16(apply_to=('img',)) -+ def forward(self, img, img_metas=[], return_loss=True, **kwargs): - """Calls either :func:`forward_train` or :func:`forward_test` depending - on whether ``return_loss`` is ``True``. - -@@ -216,7 +223,7 @@ class BaseDetector(nn.Module, metaclass=ABCMeta): - - return loss, log_vars - -- def train_step(self, data, optimizer): -+ def train_step(self, data, optimizer, compression_ctrl=None): - """The iteration step during training. - - This method defines an iteration step during training, except for the -@@ -246,6 +253,10 @@ class BaseDetector(nn.Module, metaclass=ABCMeta): - losses = self(**data) - loss, log_vars = self._parse_losses(losses) - -+ if compression_ctrl is not None: -+ compression_loss = compression_ctrl.loss() -+ loss += compression_loss -+ - outputs = dict( - loss=loss, log_vars=log_vars, num_samples=len(data['img_metas'])) - -diff --git a/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py b/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py -index 0cba3cd..d41b158 100644 ---- a/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py -+++ b/mmdet/models/roi_heads/mask_heads/fcn_mask_head.py -@@ -10,6 +10,8 @@ from torch.nn.modules.utils import _pair - from mmdet.core import mask_target - from mmdet.models.builder import HEADS, build_loss - -+from nncf.torch.dynamic_graph.context import no_nncf_trace -+ - BYTES_PER_FLOAT = 4 - # TODO: This memory limit may be too much or too little. It would be better to - # determine it based on available resources. -@@ -140,11 +142,12 @@ class FCNMaskHead(nn.Module): - if mask_pred.size(0) == 0: - loss_mask = mask_pred.sum() - else: -- if self.class_agnostic: -- loss_mask = self.loss_mask(mask_pred, mask_targets, -- torch.zeros_like(labels)) -- else: -- loss_mask = self.loss_mask(mask_pred, mask_targets, labels) -+ with no_nncf_trace(): -+ if self.class_agnostic: -+ loss_mask = self.loss_mask(mask_pred, mask_targets, -+ torch.zeros_like(labels)) -+ else: -+ loss_mask = self.loss_mask(mask_pred, mask_targets, labels) - loss['loss_mask'] = loss_mask - return loss - -diff --git a/mmdet/models/roi_heads/standard_roi_head.py b/mmdet/models/roi_heads/standard_roi_head.py -index c530f2a..30d98a4 100644 ---- a/mmdet/models/roi_heads/standard_roi_head.py -+++ b/mmdet/models/roi_heads/standard_roi_head.py -@@ -1,4 +1,5 @@ - import torch -+from nncf.torch.dynamic_graph.context import no_nncf_trace - - from mmdet.core import bbox2result, bbox2roi, build_assigner, build_sampler - from ..builder import HEADS, build_head, build_roi_extractor -@@ -151,9 +152,10 @@ class StandardRoIHead(BaseRoIHead, BBoxTestMixin, MaskTestMixin): - - bbox_targets = self.bbox_head.get_targets(sampling_results, gt_bboxes, - gt_labels, self.train_cfg) -- loss_bbox = self.bbox_head.loss(bbox_results['cls_score'], -- bbox_results['bbox_pred'], rois, -- *bbox_targets) -+ with no_nncf_trace(): -+ loss_bbox = self.bbox_head.loss(bbox_results['cls_score'], -+ bbox_results['bbox_pred'], rois, -+ *bbox_targets) - - bbox_results.update(loss_bbox=loss_bbox) - return bbox_results -diff --git a/tools/pytorch2onnx.py b/tools/pytorch2onnx.py -index a8e7487..41f59c4 100644 ---- a/tools/pytorch2onnx.py -+++ b/tools/pytorch2onnx.py -@@ -39,17 +39,19 @@ def pytorch2onnx(config_path, - if model.with_mask: - output_names.append('masks') - -- torch.onnx.export( -- model, -- tensor_data, -- output_file, -- input_names=['input'], -- output_names=output_names, -- export_params=True, -- keep_initializers_as_inputs=True, -- do_constant_folding=True, -- verbose=show, -- opset_version=opset_version) -+ with torch.no_grad(): -+ torch.onnx.export( -+ model, -+ tensor_data, -+ output_file, -+ input_names=['input'], -+ output_names=output_names, -+ export_params=True, -+ keep_initializers_as_inputs=True, -+ do_constant_folding=True, -+ verbose=show, -+ opset_version=opset_version, -+ enable_onnx_checker=False) - - model.forward = orig_model.forward - print(f'Successfully exported ONNX model: {output_file}') -diff --git a/tools/test.py b/tools/test.py -index 8dcd305..0333d30 100644 ---- a/tools/test.py -+++ b/tools/test.py -@@ -15,6 +15,8 @@ from mmdet.datasets import (build_dataloader, build_dataset, - replace_ImageToTensor) - from mmdet.models import build_detector - -+from mmdet.core.nncf import wrap_nncf_model -+ - - def parse_args(): - parser = argparse.ArgumentParser( -@@ -160,11 +162,19 @@ def main(): - # build the model and load checkpoint - model = build_detector(cfg.model, train_cfg=None, test_cfg=cfg.test_cfg) - fp16_cfg = cfg.get('fp16', None) -- if fp16_cfg is not None: -- wrap_fp16_model(model) -- checkpoint = load_checkpoint(model, args.checkpoint, map_location='cpu') -- if args.fuse_conv_bn: -- model = fuse_conv_bn(model) -+ # nncf model wrapper -+ if 'nncf_config' in cfg: -+ cfg.nncf_load_from = args.checkpoint -+ model.cuda() # for wrap_nncf_model -+ model, compression_ctrl = wrap_nncf_model(model, cfg, None) -+ checkpoint = torch.load(args.checkpoint, map_location=None) -+ else: -+ if fp16_cfg is not None: -+ wrap_fp16_model(model) -+ checkpoint = load_checkpoint(model, args.checkpoint, map_location='cpu') -+ if args.fuse_conv_bn: -+ model = fuse_conv_bn(model) -+ - # old versions did not save class info in checkpoints, this walkaround is - # for backward compatibility - if 'CLASSES' in checkpoint['meta']: -diff --git a/tools/train.py b/tools/train.py -index 91a59b8..458ad26 100644 ---- a/tools/train.py -+++ b/tools/train.py -@@ -142,6 +142,12 @@ def main(): - logger.info(f'Distributed training: {distributed}') - logger.info(f'Config:\n{cfg.pretty_text}') - -+ if 'nncf_config' in cfg: -+ logger.info('NNCF config: {}'.format(cfg.nncf_config)) -+ cfg.ENABLE_COMPRESSION = True -+ else: -+ cfg.ENABLE_COMPRESSION = False -+ - # set random seeds - if args.seed is not None: - logger.info(f'Set random seed to {args.seed}, ' --- -2.28.0.windows.1 - diff --git a/third_party_integration/mmdetection/README.md b/third_party_integration/mmdetection/README.md deleted file mode 100644 index 3dd92b8b80b..00000000000 --- a/third_party_integration/mmdetection/README.md +++ /dev/null @@ -1,39 +0,0 @@ -## Integrating NNCF into MMLab mmdetection repository -https://github.com/open-mmlab/mmdetection - -This folder contains a git patch to enable NNCF-based quantization-aware training for the object detection models on the COCO/Pascal VOC datasets from the *mmdetection* repository. Tested on SSD (with a VGG backbone) and RetinaNet (ResNet50 & ResNeXt101-64x4d backbones) models only (both quantization and quantization+sparsity pipelines). - -Instructions: -1. Apply the `0001-Modifications-for-NNCF-usage.patch` file to the mmdetection repository checked out at commit id: `3e902c3afc62693a71d672edab9b22e35f7d4776` (v2.7.0) - -2. To start quantization-aware fine-tuning of a model on the COCO dataset, you can use the regular configuration scripts provided in the repository for the specific model and dataset of interest. The only modification that is needed to be done is the addition of the `nncf_config` part to the full mmdetection config script. Examples of such configs are provided within the supplied patch (`configs/pascal_voc/ssd300_voc_int8.py` and `configs/retinanet/retinanet_r50_fpn_1x_int8.py` files). -Make sure to download the uncompressed model checkpoint files to start from using the links specified in each .py config file. -Model fine-tuning can be run using the common mmdetection command lines. For instance, the following command line, while run from the mmdetection repository root, will launch SSD-VGG quantization fine-tuning on the VOC dataset (provided you set proper paths to the dataset in the config file): -`python tools/train.py configs/pascal_voc/ssd300_voc_int8.py` - - Distributed multiprocessing is also supported, simply use the corresponding version of the command line in the mmdetection repository: -`./tools/dist_train.sh configs/pascal_voc/ssd300_voc_int8.py ${GPU_NUMBER}` - - Note that in all cases the training hyperparameters might have to be adjusted to accomodate the hardware you have available. - -### Current best results: - -**RetinaNet-ResNet50-FPN**: - -_Full-precision FP32 baseline model_ - 35.6 average bounding box mAP on the `coco_2017_val` dataset. - -_INT8 model (symmetrically quantized)_ - 35.3 average bounding box mAP on the `coco_2017_val` dataset. - -_INT8+sparse model (symmetrically quantized, 50% sparsity rate)_ - 34.7 average box mAP on the `coco_2017_val` dataset. - -**RetinaNet-ResNeXt101-64x4d-FPN**: - -_Full-precision FP32 baseline model_ - 39.6 average bounding box mAP on the `coco_2017_val` dataset. - -_INT8 model (symmetrically quantized)_ - 39.1 average bounding box mAP on the `coco_2017_val` dataset. - -**Mask-RCNN-ResNet50-FPN**: - -_Full-precision FP32 baseline model_ - 40.8 average bounding box mAP on the `coco_2017_val` dataset, 37.0 average segmentation mAP on the `coco_2017_val` dataset. - -_INT8 model (symmetrically quantized)_ - 40.6 average bounding box mAP on the `coco_2017_val` dataset, 36.5 average segmentation mAP on the `coco_2017_val` dataset.