This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample #10229

Merged

nswamy merged 6 commits into apache:master from lanking520:scala-package

Mar 24, 2018

Member

lanking520 commented Mar 23, 2018 •

edited

Loading

Description

Add Object Detection Class for MXNet Inference API, also add test and example to it. This code is fully tested with SSD model.
@Roshrini @nswamy @aaronmarkham

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

lanking520 requested a review from yzhliu as a code owner

March 23, 2018 20:57

lanking520 mentioned this pull request

[MXNET-116] object detector class added #10179

Closed

6 tasks

nswamy suggested changes

View reviewed changes

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md Outdated

		@@ -0,0 +1,116 @@
		# Single Shot Multi Object Detection using Scala Inference API

		In this example, you will learn how to use Scala Inference API to import pre-trained Single Shot Multi Object Detection (SSD) MXNet model.

Member

nswamy Mar 23, 2018

In this example, you will learn how to use Scala Inference API to run Inference on pre-trained Single Shot Multi Object Detection (SSD) MXNet model.

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md Outdated


		In this example, you will learn how to use Scala Inference API to import pre-trained Single Shot Multi Object Detection (SSD) MXNet model.

		The pre-trained model is trained on the [Pascal VOC 2012 dataset](http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html). The network is a SSD model built on Resnet50 as base network to extract image features. The model is trained to detect the following entities (classes): ['aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus', 'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike', 'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tvmonitor']. For more details about the model, you can refer to the [MXNet SSD example](https://github.com/apache/incubator-mxnet/tree/master/example/ssd).

Member

nswamy Mar 23, 2018

The model is trained on the [Pascal VOC 2012 dataset]

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md

+              ./get_resnet_data.sh
+              ```
+              **Note**: You may need to run `chmod +x get_resnet_data.sh` before running this script.

Member

nswamy Mar 23, 2018

Do you still need this if the file already is set to be a executable?
Also you do not need to chmod+x if you running the script as bash ./get_resnet_data.sh

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md Outdated

+              In the pre-trained model, the `input_name` is `data` and shape is `(1, 3, 512, 512)`.
+              This shape translates to: a batch of `1` image, the image has color and uses `3` channels (RGB), and the image has the dimensions of `512` pixels in height by `512` pixels in width.
+              The signature also specifies `image/jpeg` as the expected input type, since this example's image pre-processor only supports the handling of binary JPEG images.

Member

nswamy Mar 23, 2018

what is this? copy paste error?

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md Outdated


		The signature also specifies `image/jpeg` as the expected input type, since this example's image pre-processor only supports the handling of binary JPEG images.

		The signature specifies the output shape is `(1, 6132, 6)`. As with the input, the `1` is the number of images. `6132` is the number of prediction results, and `6` is for the size of each prediction. Each prediction contains the following components:

Member

nswamy Mar 23, 2018

what is signature?

...-package/examples/src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/README.md Outdated



		## Infer API Details
		This example uses ObjectDetector class provided by MXNet's scala package Infer APIs. It provides methods to load the images, create NDArray out of BufferedImage and run prediction using Classifier and Predictor APIs.

Member

nswamy Mar 23, 2018

BufferedImage-> Java BufferedImage

.../src/main/scala/ml/dmlc/mxnetexamples/inferexample/objectdetector/SSDClassifierExample.scala Outdated

+              import java.nio.file.{Files, Paths}
+              class SSDClassifierExample {
+                @Option(name = "--model-dir", usage = "the input model directory and prefix of the model")

Member

nswamy Mar 23, 2018

--model-path-prefix

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/ObjectDetector.scala Outdated

+              class ObjectDetector(modelPathPrefix: String,
+                                   inputDescriptors: IndexedSeq[DataDesc]) {
+                val classifier: Classifier = getClassifier(modelPathPrefix, inputDescriptors)

Member

nswamy Mar 23, 2018

use ImageClassifier instead?

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/ObjectDetector.scala Outdated

+                // Considering 'NCHW' as default layout when not provided
+                // Else get axis according to the layout
+                val height = inputShape(if (inputLayout.indexOf('H')<0) 2 else inputLayout.indexOf('H'))

Member

nswamy Mar 23, 2018

use these variables from the ImageClassifier.

scala-package/infer/src/main/scala/ml/dmlc/mxnet/infer/ObjectDetector.scala Outdated

+                  val result = objectDetectWithNDArray(IndexedSeq(op), topK)
+                  handler.execute(op.dispose())
+                  for (ele <- imageBatch) {

Member

nswamy Mar 23, 2018

imageBatch.foreach(_.dispose())

lanking520 added 2 commits

March 23, 2018 15:18


          Inference - ObjectDetector and SSDObjectDetectorExample


          Using ImageClassifier instead of Classifier, add space after Script

916d1ab

lanking520 force-pushed the scala-package branch from 342a3b1 to 916d1ab Compare

March 23, 2018 22:22

lanking520 added 2 commits

March 23, 2018 15:56


          Update ReadMe.md

0d6d3a0


          Update OD

344c14d

nswamy approved these changes

View reviewed changes

lanking520 added 2 commits

March 23, 2018 18:39


          Fixed CI Issues

ffc75f9


          Second patch for CI test

d4ebbaf

nswamy merged commit 4950f66 into apache:master

ashokei pushed a commit to ashokei/incubator-mxnet that referenced this pull request


          [MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample (a…

9e748b3

…pache#10229)

* Inference - ObjectDetector and SSDObjectDetectorExample

jinhuang415 pushed a commit to jinhuang415/incubator-mxnet that referenced this pull request


          [MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample (a…

…pache#10229)

* Inference - ObjectDetector and SSDObjectDetectorExample

lanking520 deleted the scala-package branch

April 4, 2018 18:26

rahul003 pushed a commit to rahul003/mxnet that referenced this pull request


          [MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample (a…

007c9e6

…pache#10229)

* Inference - ObjectDetector and SSDObjectDetectorExample

zheng-da pushed a commit to zheng-da/incubator-mxnet that referenced this pull request


          [MXNET-116] Inference - ObjectDetector and SSDObjectDetectorExample (a…

b5e477e

…pache#10229)

* Inference - ObjectDetector and SSDObjectDetectorExample

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet