E3_image_recognition_clothing_detector.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Learning Episode 3: Convolutional Neural Networks\n",
    "\n",
    "@sunilmallya\n",
    "\n",
    "@jrhunt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data\n",
    "\n",
    "We first fetch the [FMNIST](http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/) dataset. \n",
    "\n",
    "\n",
    "Each image in this dataset has been resized into 28x28 with grayscale value between 0 and 254. The following codes download and load the images and the according labels into `numpy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# copy helper code from: https://github.com/dmlc/mxnet-notebooks/blob/master/python/tutorials/mnist.ipynb\n",
    "\n",
    "import numpy as np\n",
    "import os\n",
    "import urllib\n",
    "import gzip\n",
    "import struct\n",
    "\n",
    "def download_data(url, force_download=True): \n",
    "    fname = url.split(\"/\")[-1]\n",
    "    if force_download or not os.path.exists(fname):\n",
    "        urllib.urlretrieve(url, fname)\n",
    "    return fname\n",
    "\n",
    "def read_data(label_url, image_url):\n",
    "    with gzip.open(download_data(label_url)) as flbl:\n",
    "        magic, num = struct.unpack(\">II\", flbl.read(8))\n",
    "        label = np.fromstring(flbl.read(), dtype=np.int8)\n",
    "    with gzip.open(download_data(image_url), 'rb') as fimg:\n",
    "        magic, num, rows, cols = struct.unpack(\">IIII\", fimg.read(16))\n",
    "        image = np.fromstring(fimg.read(), dtype=np.uint8).reshape(len(label), rows, cols)\n",
    "    return (label, image)\n",
    "\n",
    "path='http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/'\n",
    "\n",
    "(train_lbl, train_img) = read_data(\n",
    "    path+'train-labels-idx1-ubyte.gz', path+'train-images-idx3-ubyte.gz')\n",
    "(val_lbl, val_img) = read_data(\n",
    "    path+'t10k-labels-idx1-ubyte.gz', path+'t10k-images-idx3-ubyte.gz')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Labels\n",
    "Each training and test example is assigned to one of the following labels:\n",
    "\n",
    "| Label | Description |\n",
    "| --- | --- |\n",
    "| 0 | T-shirt/top |\n",
    "| 1 | Trouser |\n",
    "| 2 | Pullover |\n",
    "| 3 | Dress |\n",
    "| 4 | Coat |\n",
    "| 5 | Sandal |\n",
    "| 6 | Shirt |\n",
    "| 7 | Sneaker |\n",
    "| 8 | Bag |\n",
    "| 9 | Ankle boot |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAA/CAYAAADwizNIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl0VOX5x79zZ9+yTBayGJIQdgIBWcJSUTiigIAUURCX\nWrUVxaVwLFq3WjylKFKlilJEjkUCPZyIhXJKKagoxQIiEsAACQkhCVlnssw+c2fm/v7I73m5M5kk\nk2REzLmfc+ZkMnPnvu997/s+77O975UJggAJCQkJiZ8+3I9dAQkJCQmJ6CAJdAkJCYk+giTQJSQk\nJPoIkkCXkJCQ6CNIAl1CQkKijyAJdAkJCYk+giTQJSQkJPoIkkCXkJCQ6CNIAl1CQkKij6C4loXJ\nZLJrvixVEARZJPWQyWQIXTU7ZswYFBQUoKCgAMeOHYPL5QIAeDweTJgwAUuWLMH58+excuVKNDU1\nRaUeoaSmpmLFihV4++23ceXKlbDHTJkyBaNHj8amTZvA83zU6zF48GDMnTsXixYtQlNTEzZt2oTP\nP/8c48aNAwA88MADmDZtGtxuNz766COsXr26q8vqcXtEQv/+/VFZWRnRsZHUI1zfSE1NxYIFC7Bs\n2TJYrVacOXMGbrcbAJCQkICJEyfi+PHjePjhh+F0Ojs9V6T1uBZ0Z7z8//FBn8+fPx+nTp1CRUVF\n0OeDBw/G9OnTsXHjxqjW44fmeq5HRwdesxcA4Vq/uqrH/98c9rrpppuEjz76SLBYLEJDQ4NgtVoF\np9MphKOxsVFoaGgQAoGAYLPZhM8++0zIz8/vUT3CvYxGo/Dcc88JtbW1QmVlpXDmzBnh5MmTwtGj\nR4WjR48KZ8+eFUpKSoSmpiZhx44dwtKlS3vdHuLXfffdJ5w9e1Y4ceKEUFRUJBw/flwoLi4W7Ha7\nIAiC0NzcLDQ3NwsWi0UoKSkRTp48KZw9e1aoq6sTCgsLo1YPep07d06oqqoSvv/+e2HQoEHtvu/f\nv7/Q2NgotLa2ChcvXhQMBkNU+0e/fv2ETZs2CeXl5UJVVZVw8eJFoaKiQnC73UH9wufzCc3NzUJZ\nWZlQVVUlFBcXC++++66QkJAQ1fb4McYLvTiOY++zsrKEd999t107hOL3+wWe54XXXnutw3P9VNvj\nx6hHuJdMuIZ7uVwvM1u4esTFxeHAgQMYOHAgZDIZXC4XvF4vfD4fBEGAXC6HXq8HALjdbgQCgaDf\nK5VKqNVqKBQKFBUV4eabb+5RPUJZunQpnE4nXn31VZhMJuj1eigUClYPr9eLY8eOYfPmzTAYDNiy\nZUtU2mPYsGF4//33ceXKFRgMBnAch0AgAJ7n0b9/fwBgbSAIAlpbW8HzPHw+H+rq6pCdnQ2LxYL7\n7ruvV/UQc+HCBaSkpEChUEChUMDr9eI///kPZs+eTb+H3++Hy+WCx+NBenp6p20RST1Iox42bBgO\nHTqE5uZmeDwe+Hw++P1+eL1eWCwWxMTEwO/3AwBcLhc0Gg0SEhIgl8uhUqmgUCjgdruxfv16bN68\nOSrt8UMQST2oLwBAVVUVkpOToVAowPM8eJ6HQqGAy+WC3W4HAMTHx0OhUMDn80GhUEClUsHlcuHb\nb79l40R8zkjr0RUymSzovCTrOrIuZs2ahX379iEvLw+nT58mIRmV+0JlhitXzJdffomXX34ZX331\nFTQaDbP6wtUjbDnXs0APNU9jY2MxZ84cFBQUsO/lcjl8Pl+73wHo1g0pLi5Gv379YLVaEQgEIJfL\n4ff72bk4jmPliD8LV9/4+HhMmzYN3333Hfuupx3j6aefRn19Pfr164cVK1bAYDAwge5wOFBcXIx3\n3nkHAwYMQENDA7Zv397p+SKtx+7du+F0OhEIBBATEwOdTgee52G328HzPJqamqDT6QAAfr8fWq0W\nAODz+cDzPFwuF2688UZs3LgRH3/8cY/rIebo0aMYOnQovF4vdDoddDodOI5DSUkJACAjIwN2ux1y\nuRz19fUYNmxYp23RnXp8/fXXiI+PR2trKxQKBQRBgM/nQyAQgNfrhdvtZoNPrVYjNjYWCoUiqK8o\nlUoolUrk5+fDZrP1uj1+CCKd4ACgvLwcmZmZcDgckMvlEAQBHMdBEAQolUp27TzPs+9oDHEcB51O\nhxMnTmD8+PHtzh1NgU4TbUcsWLAAeXl5GDFiBEaOHAmZTIZRo0bB7Xb3qB7i6wh9T9BnSqUSPM8j\nPz8fe/fuRWJiImsTlUoFr9dLx0ck0K+pD7270M0YPnw4AGDlypVwOByw2+1wOBw4ePBgkJDlOA4y\nmYx9JpfLIyrnlltuQb9+/dDa2gq5XA6O46BSqRAXF8cGZSAQYAI8EAgEzf5erxfNzc1sAhAEAc8+\n+2yH2ml3sNlsSEpKQkVFBZYtW4bMzEwkJycDAMrKytDQ0IB+/foFCY9o8M477+D3v/896uvrUV1d\njbi4ONa5PB4P0tLS2LEWiwUOh4P97/F4YDKZUFFREVaY95SLFy9i+PDh8Pv9QZMtxRYGDRqE1tZW\nqFQqaDSaqJWbkZGBxMREOBwOKJVK+Hw+aLVaNqEEAgFWJwDQaDTQaDRBnzudTng8Hmi1Wjz00EN4\n5513ola/awkJokceeQRZWVmw2WxsLIiVL+orQLCCRZNhIBBAa2sr8vLycN9996GgoKBTzTVSQiaF\ndsJ8+fLlOHjwIGbNmoVnnnkGZrMZ2dnZqK2txcmTJ/Hwww/jyJEjvaqDIAhB10xyiOSHUqmE0+kE\nx3HgeR533nknduzYAb/fj8bGRixcuBAAuoyHheO6FuikJc+fPx8AMH36dNTX10Oj0cBoNGLu3Ll4\n/fXXUVNTE3TzjEYjAoFAkJDpjFmzZkGhUECpVDJNgud5/OEPf8CVK1dQVlaG7OxsFuiRy+VscMbF\nxWHChAl4/PHH2YQQCARwxx13RKUNaHJKTEwEADQ2NqKmpgYAoNfr0b9/f/j9frF/LyocPHgQU6dO\nxcKFC/H1119DqVTCYDCgvr4eHo8HNTU1LEis1+uhVCrR0tKC1NRUAIDBYMAvfvGLqNUHAIqKilhf\n8Hg88Pv9mDhxIvue53kmXCK995GQnJwMk8mEQCAApVIJjUYDv98PpVLJJnfxZEqKBX0XCAQQFxeH\n1tZWKJVKzJs37ycp0Gk8AsDmzZvhdDqhUqngdruZIkTCLFzwVyxoybr2eDzYtm0bDh06hCtXrjDX\nTDQZM2YMAEClUmH27NmYMmUKkpKSsHXrVuzfvx9ffvklpk2bhilTpsDtdmP48OEoLi7uVZnia6c2\no78kqAOBALKzs7Fz5054vV5wHIcVK1bg8uXLHQbPu+K6Fug0y990000A2oQax3HgOA579uzBpEmT\nsG3bNhw+fBhFRUU4evQoZsyYgcmTJ+PLL7/Evn37IirnnnvuYRq4z+eDSqWCw+HAmjVrsHjxYkyZ\nMgVr167Fiy++CAA4deoUUlJSIJfLUV1djRdeeAGPPfYY85d6PB4kJydj5MiROHPmTK/agAaJ3++H\nXC5HXFxcu2Po5pMrJlq88sorePLJJ5GZmYmamhrYbDY4HA60trYCaNM0AKC+vp75RltaWmAymbBr\n1y60tLREtT5VVVVMSMrlctjtdhw5coT1E5vNxrSgUJdGb5g4cSI4joPJZGIThtfrhdlsRnl5OUpK\nSmC329kE53A44PV6odFocOONN2LmzJnwer0wGAxMs/8pQgLpxIkTLH4TGxtLrgnmq+7MUiQhRfeR\n/O7z58/Hhg0bunSPdIVYCOr1etxxxx1MEWtsbMTbb7+Nl19+GdXV1fjd736H1NRUCIKAb7/9FpMm\nTcL8+fPhdrt7LdBDYwJpaWno168fez9x4kSkpaVBqVTC7XajubkZer0en332Wa/KvW4FOgmpRYsW\nYeTIkQDatDKNRoOMjAwsW7YMxcXF+P7772E0GnHTTTdhyZIl8Hq9OHLkCJ5++ukgs68zbrjhBrS2\ntjI/JwA26LZv3w6e55lpCADHjx/HuHHjwHEcqqqqmJZMZjYFCWfMmNFrgW40GqFWq+FyuZj2H2rC\nkXYUTTcDaUp5eXks0Er+c51OB6fTyeqh0+ngdruD6vO3v/0tanUhLl26FOTW8nq9bDIB2voMuUWi\n6X7asGED9u7diyeffBLjxo3D8uXLcerUKfa9Xq+HXq9nfSY2NhZ6vR4tLS1ME798+TJiY2Ph9Xpx\nww03RK1uPwYUm5DL5cwCAToP9okhDZ3cEQqFApMnT8aGDRt6bWXSGBEEATExMXC73cxHP3PmTMyZ\nMwdLlizBjh07AAC1tbUA2oSsxWLBDTfcgF/96lc4cOAAjh8/3ut6DBs2DAUFBTAajcxqzMrKgsVi\nQXp6Os6ePQuLxcKCyiR/Qs8VKdedQA8diGvWrAnSSsnM9fv9yMvLw/DhwxEIBHDhwgWcPXsWPp8P\ny5cvR3JyMgoLC7ssLz8/H06nE36/n5nKKpUKVqsVADB+/Hg4nU7k5OTg6aefZnUkE2nWrFkA2vzI\nCQkJCAQCzK9+22234c9//nOv2oN846SVkoYIXDXteZ4Hx3FR1dDJ7K2qqkJxcTGGDBkCp9OJ1tZW\nBAKBIAFus9nQr18/+Hw+cByHsrKyqNVDTHV1NWJjY9HY2Aiv18vaRFxnEjA98T92xNatWxEIBLB/\n/36cOHECiYmJaGhogEwmY21itVpZm5HAMhqNyM7ORk1NDebMmYPW1lY0NDSw4Gk0CPXVkjAD0M59\nEao1AmAxge4IUp7noVarg4J9ZEGS5k11CldPqovP54NcLgfP85g3b153Lz0s4ut3Op2QyWTM/fnX\nv/4Vc+fODfu79PR0xMTE4Pjx4yzoTrGqnkD979y5c5g5cyYaGhrCHudwOKBQKFBdXY2dO3eisrIy\nSHETxwQj4boT6KEdy2q1Ij4+HkCbC0apVDLXBrlHBEFAXl4ei1Dr9XqcPHkyovLeeOMNJggDgQCL\nLAcCAUybNg0pKSlISEiAUqlk/mGe5+F2u6FSqRAfH48nnngCBoMBXq8Xer0eHo8HSqWSWRa9geM4\npg1zHBfkxxS3l8fj6XVZndWB4hIajQZWq5X5ToGrrjGqF2k90YaCnzKZLCgITG1AbcRxXJcLvbrD\nP//5T8ycORMPPPAAJk+ejB07duCJJ55AbGwshgwZAqPRyDI7ADC3m9/vx+bNm2G1WrFu3Tq43W40\nNjZi7ty5GD58OMxmc6/rFio0xf+LBcErr7yCZ599FjExMUG/7+7EN3nyZOh0Ong8HigUCvj9fjZx\niJMHwk0Q4mAhCX6KR4ROND1FXG5rayt2796N3bt3s890Oh1cLle7LJSMjAw0NTXBbDbj73//O3Jy\ncjB06NCo1KmhoYGNXSC4zYuKipCfnw+r1Ypbb70VQHufe3cmlutOoIei1WqDNFKPxwOHwwGr1YqU\nlJSgIIxarWYzdEpKSkTn/+yzz5CYmIi0tDSo1WoolUqYzWb4/X4cPHgwOGlflIImDnqR6a9Wq5kG\nbbFYIrIQuoKuXaFQsFk7NF1SqVTC4/EwH120II2usrKSuZgoB1+r1bJBqNVq4XQ64Xa7kZSUhOrq\nalbnaAe4qJOLzXaxX5buSXNzc9TKfPPNN+Hz+dDY2IjLly9jzpw5WLp0KatPqEAioUbZUm63G6Wl\npWhsbMS+fftw/vz5qAhzgtohtK2feuopjB8/HnPnzoXX64XNZsPhw4dZTApom3zefPNNZn12hUql\nYuNAHBQmbTI0QCyuo/g9uc7E7pEfAtJ2AQTJkVBffXJyMux2O7uW2NjYqFl5dH/E56OxsXHjRkya\nNAkcx+GGG25g7kwAGDt2LAoKClBfXx9xWdedQBfnjhqNRiQkJLCGIB+Tz+eD0+mETqeD3W5nCzdc\nLhd0Oh2qq6uh0Wgwbdq0LstbtWoVVq1ahYSEBIwZMwbPP/88xo4dC5vNhrq6OtjtdigUinZClOpK\nAl2v16OkpKTdgqLeQAtTqEOEq0MgEGDXTkE3Cs5Fi0uXLjFtKiEhAeXl5eB5HklJSQCApqYm+Hw+\neDweyOXyiGMXPUGsgYknWYLaKppB0V27duH222/H0KFD8dVXX6GwsBApKSm4dOkSsxZ1Ol079w8F\nR2NiYjBw4EAsWrQI6enpGDJkCI4ePYrDhw/3ql6hE1pubi4eeeQR1gdHjRoFi8UCs9kMu92O1NRU\n5k8mli1bhp/97GcRlzlx4kSWc06Cmed5qFSqdr70cAt5aKIjga5UKlk7zZgxAwcOHOhVm4QiFtyU\nPkr3SWzRGAwGPProo9i9ezc2bNiA1tbWqPWhcNYKtdXWrVuxceNGGAwGOBwO3HXXXSgrK8PevXsB\ntClLM2bMiHgri+tOoJMv0O/34/HHH4fBYGAzFrlXNBoNUlJSWOoY+eJUKhW0Wi0+/fRT5OfnQ6VS\nRVyuxWLBwYMH4XQ68a9//YtljWg0mrAmJHUGKl+lUuHzzz+Palu43W54PJ52QkxcB3EgsqWlJerC\nHGjzkYu1YI7jgjT0pqYmJCcnw2g0AkBUg7OhhGp6Yr+wWMCJ8+R7y5gxY1gmwqFDh3DrrbdiyJAh\nAK5qfaEuA3FuttlsxpYtW/Czn/0MlZWVLOe5O9B1hiw2AQCYTCZs2bIFt912G3ieZ9bJpUuXoFAo\noFarcfHiRSQnJ2P9+vUA2vaiWbp0KV566SXk5ubilltuwaFDhyKqB7WzWFiScKc889D7RND4Jpcl\naelyuRyvvPJKjwV6d9L8yN9P74G2DJiioiJMmTIFq1evxpAhQ7B///4e1aWjelE7hNa1ubkZOp0O\ner0eW7duhSAILK53+vRpVFVVRVzedSfQaUk3AHz33XdBjU+dRa/Xw+/3M+1ZqVRCpVKxANW9996L\nN954I+IbQpoC+c5dLhcbQJF0EhrUNJBCg1M9RbwooavjfgghSkLK6/WyYJ7FYgHQJsTpPmm1WtTW\n1rKVttcCso7E5jOZy36/H9nZ2VErKyMjA3K5HCaTCbW1tbDb7fD7/XA4HKx88aRPAk2j0cDn88Fk\nMsHhcECn0yE1NRVGoxEZGRk4f/58t64VCF6wc/fdd+P+++/HbbfdBpfLherqavj9frZFRUxMDDwe\nDzweD0aOHImmpiYsWbIEy5Ytg8/nQ3l5OZRKJbRabcQppqS1kiAmTTuSrKJQHzvlodNkoFarI6pD\nOLo71sST0ZQpU3Ds2DFs2LABDz74IBYvXgyVShWV4H5n2jmRnp6OmpoaGI1GLFu2DDExMXjxxReh\nUqmYph4pP4pAp4FH5hcJUiA4kLN7927wPM86MQUvKauDAqM0iMmkS0tL61YOtNi/dfbsWWaW0Wdi\nf634Guh3JHQpPzuS5caRQOcVr1Kl/6kcgrIGwmUy9BQ6V0JCAkwmE+x2O4tN1NTUwGAwAGjTECko\nK5fLkZmZCQBR958DwVqO+D1w1aQPBAIsgB2tMr1eLwRBQEtLCwwGAxNI4swjsaVA90ypVILjOOYH\njYmJgVwux8CBAyMW6KHa8KpVq/DYY4/BaDSiubkZV65cgd/vZ8kDYtcHCc+WlhYm6IuLizFhwgQA\nwF/+8hf88pe/hNlsjmgx3AsvvBAUM6D4SSSIfebkxqREB5/Ph/Hjx/d4QU13ECcWvPnmm0hKSsIf\n//hH/OY3v4HZbEZBQQHbYiJaiK+LgskkV0jGiSfF559/HjKZDJs2bepWOdJ+6BISEhJ9hGuuodPs\n2JH2duedd+L+++9Hfn4+fD4fbDYb08zJleHxeJj2Q3510qbJZfPAAw90a3ELadVOp5MFX0nLCqed\n04tycslsjCY6nY6twKN2E6c/ietDvmNahBQNxDvqnTx5EhUVFdDr9XC5XEhNTWVaeVlZGVwuF2Jj\nY1FTU4OMjIyolB/KyJEjgzaBAtpr6nQfyHqIBlSWIAioq6uDTqdrl54nDk5S7rA4l/jSpUvMEuU4\nDrGxsRGVPXXqVMybNw95eXlQq9Xo168ftFot7HY7zGYz23ZBr9dDEAQWmKZ6UB0UCgU8Hg+8Xi+G\nDh0Km80GpVKJ1tZW1NbWQq1W47nnnuuyPgkJCcxXTnEbceZLV9D9UalU7bYKaGlp+cG1c6DN3TJo\n0CBs3LgRCoUCNTU1eOSRR3D+/HkolUpkZWV1e60AWWnitMyOLGWxO/by5cs4d+5cUIoztafZbO5W\nhgvwIwh0sdBLTEzEgAEDMGrUKKSnp2Px4sVIT09nLhWPxwOj0chyisk8o61KlUoliouLodfrMXjw\nYNahA4EARo0a1a16haa+0WfiwSx2cYiDXtQpu7tirivEfuLQwEq4Y4H2O0BGg9tvvx2lpaUoKyuD\nw+FAU1MTTCYTy4+12WxwuVzM1ZKWlobU1FTU1tZG1QU0btw4toBHvDo0NADH8zz0ej3bDjUa0ERS\nWVkZFK+g+y7OhCJXonhVLwkIsZDtildffRWLFi1iCgNtTet2uyGTyaDVaiEIAtuqluooTgYgl49a\nrYZKpWIBSfJ9k3Cl3TI7IzMzk20sRW4DSlsMTSMVCzcAQeOE/OWBQAA+n4+9727qYuiajI6OoXqo\n1Wo4HA6MGTMGW7Zswfnz55GVlYUlS5aw8TVlyhTk5uZ22W/CXXOkCh2Nh6NHj6KkpITln1OfoV05\nv//++4jOJ+aaC/SZM2di3bp1iIuLg16vZxqLw+Fge1mT9u71elFWVoapU6cCACoqKpj2Q6tHc3Jy\noFarYbVa4fF4oFKpoFarw+55EimJiYlsX5CO0gUJ8l1TXCCahDtfuAwC8RLqcEuHewJ1ruzsbIwZ\nMwbnz5+HyWRCUlISLly4AL1ej0GDBgFoC5CK29tqtWLZsmV46aWXoibMATAfb+jEKp5A6Z41Nzfj\nueeei4pAF5+flmkDVxfHhAuKAmAxDZmsbbEbbUsARJYJtH79ehw5cgTTp0/H2LFjWZ6ywWBg1ipp\n+2TNiS04sVD1er3geZ4JYbJkaS8Rnuexa9cuPPjggx3WZ86cOez6qHyaPLuzQRylOpJFKY7/dAdx\nMJzqFe4Yag+Hw4HMzEysXr0a//73vzF16lRMmjQp6Hi6NtrLvSPCxdUAYPTo0VixYgVWrlyJuro6\nAFfHEk2aLpcLH3zwAXuyFUFjhfoV7ScTqfUDXGOBLpfLsXHjRrZzHblPgKuBAvpfr9fDZDLhtdde\nY3sgzJs3DzabDT6fD+fOnUNFRQUGDBiAmJgYFhylxos0UEOIbwx1FFrMAwRrgmJNg4Kn4pWC0dLQ\nyQ0UuqAoNI1R7H6Ij4+PyqZYdN333HMPvvvuO5YJkZOTg6qqKowYMYIdU1lZiaSkJFy5cgXJycmw\nWCzIzMxEbm4uzp492+u6EOPGjQvaoiFcHjoFy71eb1RW6oaDNEqynsLdF/GiM71eD6PRiIqKCgwd\nOjRi4SWTyfC///2PpfJpNBoMHToUo0aNwoABA5CRkQGdTheU3VNfX88yjcxmM5qbm9HS0sI+J0FF\nLg8AbL1FV/2WgoQ+n49NSHq9vp22KnZ90XXQX/qcHhCi1WqDrNue0FG9w6VXbty4ERUVFZgwYQKm\nTJnS7jd+vx9JSUmdBkRpryCyMD744APce++9LCkiISEBs2bNYus0qC+QKzQ7Oxvz589niiotJiLZ\nRTurUhr0dSvQV6xYgcTERNTX10Or1UKr1bLIu1wuR0JCAlpbW2GxWKDRaGC32/Hee++xTbF27tyJ\nnJwcGAwGTJ48Gfn5+UxjVyqVQZoTx3E9Tl2jzBUS2GIzkr4nTYjcPABYlkG0IHNZLLA76rykIVF7\nRovx48fjm2++YWY7aRnifWMofc3tdiMnJwctLS1obm7G8OHDoyrQk5KS2P4c1A7h1ghQfMVgMAQ9\n9aWnOJ1OaDSaINOd9j8Ru9nEgovqRdYbx3EoLS3FiBEjgjKjOqOpqQkGgwEZGRns3PX19fj000+h\n0+mY0KF+KJfLodVq2bk1Gg3UajUMBgPi4+ORkZEBo9HIHqqgUCig1+thtVrB8zx7UEhH7Nmzh2Vd\nhLoaQx8GQ1o3tQcdK15UJHZN9QQ6r8lkQkZGBnJycrBr1y72fWi/eP/998HzPMaOHcuyfICrfZlc\neV0ttQ9dQTpu3DgWw5DJ2jaIo+2jKY4nrsuBAwdw6NAhnDt3DgDaKZ9ZWVngeb5H1uU1Feh1dXXs\nSTc8z6OxsREajYblwjocDtTX17O9IrxeL/x+P7Zt2wagbXOm5ORkaLVaJkDES69JoyaNhR5i3F06\n6mDh0uTEWgDtthctDZ2CIzRgO0LsourObN4VgwcPRmVlJbRaLQtO026LQPC+ziTo7XY70tPTUVlZ\n2avNjUJJSEiAVqtl+8gQYqFKGqBSqcTXX3+N6dOnY9q0ab1yu1DQneM4pmGRNUnl0//ithf7mMmv\nXVZWxlwVkbrG7HZ7kPmv0+lYum5MTAy0Wi3L36ZApVjgUHlmsxklJSWQyWTMLUkCzOfzwWq1drka\nkRQr0q51Oh1zlZKApjYRt4dYa6c6kZXSnfUeodBv8vPzkZ2djdbWVubaEkOxnalTp0KlUjFXofg8\n4ok5Jyen03LvvPNOZGVlYdOmTXC5XEwTd7lcsNlsbHXwW2+9FZSYceLECQBteeeTJ0/u8PyhFsJ1\nq6GXl5dDEAQ0NDRAq9UiJiaG7a9NnZECN7ScWiaTsRksPT0dXq8XLS0tsNvtLEpPvkEKsCQkJEAm\nk2Hs2LE9qmdH5rBYUybovXhBR7Qgs5Y6XGdmOvkzexM7CGXYsGFsolSr1WyhDGk01JEpK0ihUODC\nhQvIzc3FlStXEBcXh8TExKjsWzJt2rR2pjnHcez5osBVrdHv97Nnw44cObJXAp3OKZfL0djYyMoN\nN5GIMxzEk30gEIDJZMK5c+eCls33BKfTycYDLfK6Vvz85z8HcFWgA8AzzzyD999/n235QO0fbtdF\nmvzkcjk0Gg1KSkowePDgIAsqLS2NPcAllHDZZoIgdHl///GPfwBokx+33357u+/Fk4/f7+9yU66B\nAwfi9ddfx6uvvgq32822RSZ3DW1kx3Ectm7dirVr1+LDDz9Ebm4ugLbnKXS0+yK1gVhr785kd03z\n0I8cOYILzUQ2AAAIFklEQVRdu3ax/aBramrQ0NDANBZ6EpFGo2HJ9m63G1arFVarlR3f1NTEVurR\nAwNo8YTdbkdDQwPkcnm3dv0LZ7aLNQ7xAAz1mXYnc6E7aDQalpLZmVkqdgVF8hzNSCGXD+2bo9Fo\nWBYRZSVQ/IKyXEpLS5GSkgKj0ciCqNHgrrvugtPpZK4OSp0DwBb4kFuEFtkIgtBjK00M3X9xkIti\nJjTRqVQqlp1Aj6Cj90DbQ8jLy8uZ/z+altS1gjaBo34JAB9++CG2b9/OrAjKvCHBLV58RZaKQqFA\nRUUF3njjDQDBGnJnQdnQMSoef0VFRaipqcFTTz0VdMy7776LvLw85OXlYd++fTh27Fin16hQKNgW\nFh2xbt06XL58GRqNhilQdrudubion1L69OnTpzFixAj4fD74fD4888wzADpWHOPj44Ossu64pK55\nlsvy5cvx7bff4sUXX0RqairbS5qW25OmJ+4E4oGr0WiCUpFkMhl72kdsbCwEoe0hzRcvXsT69evx\n9ttvR1Qv8ezP83xQFgKZ3OIgT6imIBbo0XK5DBgwAMBVs5nKCM3qIG2A3FjRIjExEWq1GvX19Rg9\nejQLjKpUKvA8zzo+bbswduxYfPLJJ7BardBoNEhKSurWfjqdMWjQIPac11A/5datWwFctRTIXFUq\nlRgxYkSvy6b7X1paysqxWCyw2WzMvUFBeSBYYyeXCAVFgasbqv0UUavV7bZqfuihh/DQQw+x/7Va\nLWJjY4M0dIo1hQbsN2/ezNw1PM/j7rvvxpo1a8KWvWDBArhcLjQ1NaGurg42mw1utxsulwsejweJ\niYn47W9/i8LCQtTW1mLx4sV4+OGHceHCBQDo9JGINKZCXVYdUV1djezsbFitVnAcx4LDFB8hJcDl\ncjG5QUFTmlRC5YRGo4HT6URcXBwbx93dbO+a9ioSPtu2bcO2bduwcOFCrF27FoMGDWK5tqF51w6H\ng723WCzgeZ5FhAGwwBNNCHv27MGZM2d6na5GQlo8OEN9gqHvoz1I3W43W+BE+3+L07CANr85+WuN\nRmNUHy6RnJwMjuNgNpthMpmgVCpRXV0NtVqNpqYmZhaK69Pc3My2H7bZbMjJyUFRUVGv61JYWIjc\n3FzW1jRRhGoyNHgoA+H06dO9Kld83yl7hBa0abVapKWlBbkgxL/jOI5pbunp6eypU3K5vFf7lvxY\nzJ49GxaLJSjLIxwulytiIdTS0gKdTscsri+++KLDY7Ozs5GTk4O0tDTExcWB53k0NDTA7/ejsrIS\nZrMZx48fx8KFCzF9+nSMGDECxcXFTOunxwJ2FiS32WwRyY6VK1fi6NGjMBqNLJuJ9sAnBUBsjalU\nKqaZA+EfNkL9jPZGouO6wzUV6KEXUFhYyPYMHz16NPr374+6ujoMHDgQ586dg9vtZpHgHxrxbNnY\n2Bj08GUSqHSMOF9UPLFEW0Pfv38/hg4divj4eDZAKKtHXA+gzT8YCAS6vYtfZ9CWniaTCUBbR6MJ\nROzr1Ov1SEtLg8lkYk+QoiyhrszXSFmzZg3+9Kc/AUDQAwrEgpSyUWjhilqtxksvvdSrcikQLPaz\nFhYWIj4+HnV1dUGLW0I1dJpUmpqacPToUQBXF679EAvAfmiampqQk5ODU6dOQaVShd3+lwRZaPaR\neOyIFyMdPnwYd9xxB1QqFYqKivDss892WP66deuC/k9OTsbAgQORmprKMoEGDBiA6dOnw2g0Ys+e\nPdiwYQMuXbrEftNVxpPdbsejjz7aznUTyrFjxyCTybBkyRKsXr0amZmZnU7SpaWl+OSTTzo9J43n\nqVOnshWi3ZUl143dd+rUKfacxt48yy8axMTEMNeOwWAIshzE0MBsaWmBWq3G8OHDAYSffXuCw+HA\nW2+9hQULFiApKYktKBFvm0D/l5WVYdeuXVF92n1ubi7Onz/PMljkcjl74ssXX3yBX//61wDahN7e\nvXtZKmdCQgKsVitKS0uD0sh6y8SJE5lgJCEqfqgHLYOXydoednLvvff22mLR6XRM06KJbcWKFb06\np/hcPzXKy8uZhUEbtRkMBmYpdbbkXQz122PHjmHmzJlQqVRYu3Ztt+rS0NDQaXCxJ5SWluLjjz+O\n+Pjt27dj+/btANq2prj55psxduxYZGVlIS4uDoIg4MqVK0GPvuvIuqHJZtWqVfjmm28AdP9JZNeN\nQP+xEfvFT548CY1GA4vFwkx7juOYyU1uFgpW8jwPk8mEI0eOAOheEKOrOrlcLhQUFABo82lnZmay\nQKMgCCgvL0d1dXWQBh8tC2HevHksvey9997DsGHDUFpaigEDBqCkpKSdefzBBx8AaFu88UNAWtG8\nefOQm5uL2bNnB5nHO3fuRFpaGrZs2cJSXXuL2WzGuXPnYDAY8N///hdA5ysTu6KwsBBDhgz50ZWW\nniKTyfDyyy/DYrGwNMee5PlT29XX1zOXVTRXFfcGUlS6y5kzZyJ6KHxH/YY+37lzJ/usu/tDyaI1\n+CUkJCQkflx+eo48CQkJCYmwSAJdQkJCoo8gCXQJCQmJPoIk0CUkJCT6CJJAl5CQkOgjSAJdQkJC\noo8gCXQJCQmJPoIk0CUkJCT6CJJAl5CQkOgjSAJdQkJCoo8gCXQJCQmJPoIk0CUkJCT6CJJAl5CQ\nkOgjSAJdQkJCoo8gCXQJCQmJPoIk0CUkJCT6CJJAl5CQkOgjSAJdQkJCoo8gCXQJCQmJPoIk0CUk\nJCT6CJJAl5CQkOgjSAJdQkJCoo8gCXQJCQmJPsL/AQ9oGn/gv8NeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc423981910>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "for i in range(10):\n",
    "    plt.subplot(1,10,i+1)\n",
    "    plt.imshow(train_img[i], cmap='Greys_r')\n",
    "    plt.axis('off')\n",
    "    \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_img[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'batch_size': 200,\n",
       " 'cursor': -200,\n",
       " 'data': [('data', <NDArray 60000x1x28x28 @cpu(0)>)],\n",
       " 'data_list': [<NDArray 60000x1x28x28 @cpu(0)>, <NDArray 60000 @cpu(0)>],\n",
       " 'label': [('softmax_label', <NDArray 60000 @cpu(0)>)],\n",
       " 'last_batch_handle': 'pad',\n",
       " 'num_data': 60000L,\n",
       " 'num_source': 2}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transform the data\n",
    "\n",
    "import mxnet as mx\n",
    "\n",
    "# 4D (batch_size, num_channels, width, height) ==> (bsize, 1, 28, 28)\n",
    "\n",
    "def to_4d(img):\n",
    "    return img.reshape(img.shape[0], 1, 28, 28).astype(np.float32)/255\n",
    "\n",
    "batch_size = 200\n",
    "\n",
    "train_iter = mx.io.NDArrayIter(to_4d(train_img), train_lbl, batch_size, shuffle=True)\n",
    "val_iter = mx.io.NDArrayIter(to_4d(val_img), val_lbl, batch_size)\n",
    "\n",
    "train_iter.__dict__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.36.0 (20140111.2315)\n",
       " -->\n",
       "<!-- Title: plot Pages: 1 -->\n",
       "<svg width=\"214pt\" height=\"1606pt\"\n",
       " viewBox=\"0.00 0.00 214.00 1606.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 1602)\">\n",
       "<title>plot</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-1602 210,-1602 210,4 -4,4\"/>\n",
       "<!-- data -->\n",
       "<g id=\"node1\" class=\"node\"><title>data</title>\n",
       "<ellipse fill=\"#8dd3c7\" stroke=\"black\" cx=\"47\" cy=\"-29\" rx=\"47\" ry=\"29\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-25.3\" font-family=\"Times,serif\" font-size=\"14.00\">data</text>\n",
       "</g>\n",
       "<!-- convolution7 -->\n",
       "<g id=\"node2\" class=\"node\"><title>convolution7</title>\n",
       "<polygon fill=\"#fb8072\" stroke=\"black\" points=\"94,-168 -7.10543e-15,-168 -7.10543e-15,-110 94,-110 94,-168\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-142.8\" font-family=\"Times,serif\" font-size=\"14.00\">Convolution</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-127.8\" font-family=\"Times,serif\" font-size=\"14.00\">5x5/1, 30</text>\n",
       "</g>\n",
       "<!-- convolution7&#45;&gt;data -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>convolution7&#45;&gt;data</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-99.8131C47,-86.1516 47,-71.0092 47,-58.3283\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-109.906 42.5001,-99.9062 47,-104.906 47.0001,-99.9062 47.0001,-99.9062 47.0001,-99.9062 47,-104.906 51.5001,-99.9062 47,-109.906 47,-109.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"71\" y=\"-80.3\" font-family=\"Times,serif\" font-size=\"14.00\">1x28x28</text>\n",
       "</g>\n",
       "<!-- activation9 -->\n",
       "<g id=\"node3\" class=\"node\"><title>activation9</title>\n",
       "<polygon fill=\"#ffffb3\" stroke=\"black\" points=\"94,-278 -7.10543e-15,-278 -7.10543e-15,-220 94,-220 94,-278\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-252.8\" font-family=\"Times,serif\" font-size=\"14.00\">Activation</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-237.8\" font-family=\"Times,serif\" font-size=\"14.00\">relu</text>\n",
       "</g>\n",
       "<!-- activation9&#45;&gt;convolution7 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>activation9&#45;&gt;convolution7</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-209.813C47,-196.152 47,-181.009 47,-168.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-219.906 42.5001,-209.906 47,-214.906 47.0001,-209.906 47.0001,-209.906 47.0001,-209.906 47,-214.906 51.5001,-209.906 47,-219.906 47,-219.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-190.3\" font-family=\"Times,serif\" font-size=\"14.00\">30x24x24</text>\n",
       "</g>\n",
       "<!-- pooling7 -->\n",
       "<g id=\"node4\" class=\"node\"><title>pooling7</title>\n",
       "<polygon fill=\"#80b1d3\" stroke=\"black\" points=\"94,-388 -7.10543e-15,-388 -7.10543e-15,-330 94,-330 94,-388\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-362.8\" font-family=\"Times,serif\" font-size=\"14.00\">Pooling</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-347.8\" font-family=\"Times,serif\" font-size=\"14.00\">max, 2x2/1</text>\n",
       "</g>\n",
       "<!-- pooling7&#45;&gt;activation9 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>pooling7&#45;&gt;activation9</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-319.813C47,-306.152 47,-291.009 47,-278.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-329.906 42.5001,-319.906 47,-324.906 47.0001,-319.906 47.0001,-319.906 47.0001,-319.906 47,-324.906 51.5001,-319.906 47,-329.906 47,-329.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-300.3\" font-family=\"Times,serif\" font-size=\"14.00\">30x24x24</text>\n",
       "</g>\n",
       "<!-- convolution8 -->\n",
       "<g id=\"node5\" class=\"node\"><title>convolution8</title>\n",
       "<polygon fill=\"#fb8072\" stroke=\"black\" points=\"94,-498 -7.10543e-15,-498 -7.10543e-15,-440 94,-440 94,-498\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-472.8\" font-family=\"Times,serif\" font-size=\"14.00\">Convolution</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-457.8\" font-family=\"Times,serif\" font-size=\"14.00\">5x5/1, 50</text>\n",
       "</g>\n",
       "<!-- convolution8&#45;&gt;pooling7 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>convolution8&#45;&gt;pooling7</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-429.813C47,-416.152 47,-401.009 47,-388.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-439.906 42.5001,-429.906 47,-434.906 47.0001,-429.906 47.0001,-429.906 47.0001,-429.906 47,-434.906 51.5001,-429.906 47,-439.906 47,-439.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-410.3\" font-family=\"Times,serif\" font-size=\"14.00\">30x23x23</text>\n",
       "</g>\n",
       "<!-- activation10 -->\n",
       "<g id=\"node6\" class=\"node\"><title>activation10</title>\n",
       "<polygon fill=\"#ffffb3\" stroke=\"black\" points=\"94,-608 -7.10543e-15,-608 -7.10543e-15,-550 94,-550 94,-608\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-582.8\" font-family=\"Times,serif\" font-size=\"14.00\">Activation</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-567.8\" font-family=\"Times,serif\" font-size=\"14.00\">relu</text>\n",
       "</g>\n",
       "<!-- activation10&#45;&gt;convolution8 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>activation10&#45;&gt;convolution8</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-539.813C47,-526.152 47,-511.009 47,-498.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-549.906 42.5001,-539.906 47,-544.906 47.0001,-539.906 47.0001,-539.906 47.0001,-539.906 47,-544.906 51.5001,-539.906 47,-549.906 47,-549.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-520.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x19x19</text>\n",
       "</g>\n",
       "<!-- pooling8 -->\n",
       "<g id=\"node7\" class=\"node\"><title>pooling8</title>\n",
       "<polygon fill=\"#80b1d3\" stroke=\"black\" points=\"94,-718 -7.10543e-15,-718 -7.10543e-15,-660 94,-660 94,-718\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-692.8\" font-family=\"Times,serif\" font-size=\"14.00\">Pooling</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-677.8\" font-family=\"Times,serif\" font-size=\"14.00\">max, 2x2/1</text>\n",
       "</g>\n",
       "<!-- pooling8&#45;&gt;activation10 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>pooling8&#45;&gt;activation10</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-649.813C47,-636.152 47,-621.009 47,-608.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-659.906 42.5001,-649.906 47,-654.906 47.0001,-649.906 47.0001,-649.906 47.0001,-649.906 47,-654.906 51.5001,-649.906 47,-659.906 47,-659.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-630.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x19x19</text>\n",
       "</g>\n",
       "<!-- convolution9 -->\n",
       "<g id=\"node8\" class=\"node\"><title>convolution9</title>\n",
       "<polygon fill=\"#fb8072\" stroke=\"black\" points=\"94,-828 -7.10543e-15,-828 -7.10543e-15,-770 94,-770 94,-828\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-802.8\" font-family=\"Times,serif\" font-size=\"14.00\">Convolution</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-787.8\" font-family=\"Times,serif\" font-size=\"14.00\">5x5/1, 50</text>\n",
       "</g>\n",
       "<!-- convolution9&#45;&gt;pooling8 -->\n",
       "<g id=\"edge7\" class=\"edge\"><title>convolution9&#45;&gt;pooling8</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-759.813C47,-746.152 47,-731.009 47,-718.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-769.906 42.5001,-759.906 47,-764.906 47.0001,-759.906 47.0001,-759.906 47.0001,-759.906 47,-764.906 51.5001,-759.906 47,-769.906 47,-769.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-740.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x18x18</text>\n",
       "</g>\n",
       "<!-- activation11 -->\n",
       "<g id=\"node9\" class=\"node\"><title>activation11</title>\n",
       "<polygon fill=\"#ffffb3\" stroke=\"black\" points=\"94,-938 -7.10543e-15,-938 -7.10543e-15,-880 94,-880 94,-938\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-912.8\" font-family=\"Times,serif\" font-size=\"14.00\">Activation</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-897.8\" font-family=\"Times,serif\" font-size=\"14.00\">relu</text>\n",
       "</g>\n",
       "<!-- activation11&#45;&gt;convolution9 -->\n",
       "<g id=\"edge8\" class=\"edge\"><title>activation11&#45;&gt;convolution9</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-869.813C47,-856.152 47,-841.009 47,-828.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-879.906 42.5001,-869.906 47,-874.906 47.0001,-869.906 47.0001,-869.906 47.0001,-869.906 47,-874.906 51.5001,-869.906 47,-879.906 47,-879.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-850.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x14x14</text>\n",
       "</g>\n",
       "<!-- pooling9 -->\n",
       "<g id=\"node10\" class=\"node\"><title>pooling9</title>\n",
       "<polygon fill=\"#80b1d3\" stroke=\"black\" points=\"94,-1048 -7.10543e-15,-1048 -7.10543e-15,-990 94,-990 94,-1048\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1022.8\" font-family=\"Times,serif\" font-size=\"14.00\">Pooling</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1007.8\" font-family=\"Times,serif\" font-size=\"14.00\">max, 2x2/1</text>\n",
       "</g>\n",
       "<!-- pooling9&#45;&gt;activation11 -->\n",
       "<g id=\"edge9\" class=\"edge\"><title>pooling9&#45;&gt;activation11</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-979.813C47,-966.152 47,-951.009 47,-938.328\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-989.906 42.5001,-979.906 47,-984.906 47.0001,-979.906 47.0001,-979.906 47.0001,-979.906 47,-984.906 51.5001,-979.906 47,-989.906 47,-989.906\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-960.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x14x14</text>\n",
       "</g>\n",
       "<!-- flatten2 -->\n",
       "<g id=\"node11\" class=\"node\"><title>flatten2</title>\n",
       "<polygon fill=\"#fdb462\" stroke=\"black\" points=\"94,-1158 -7.10543e-15,-1158 -7.10543e-15,-1100 94,-1100 94,-1158\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1125.3\" font-family=\"Times,serif\" font-size=\"14.00\">flatten2</text>\n",
       "</g>\n",
       "<!-- flatten2&#45;&gt;pooling9 -->\n",
       "<g id=\"edge10\" class=\"edge\"><title>flatten2&#45;&gt;pooling9</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-1089.81C47,-1076.15 47,-1061.01 47,-1048.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-1099.91 42.5001,-1089.91 47,-1094.91 47.0001,-1089.91 47.0001,-1089.91 47.0001,-1089.91 47,-1094.91 51.5001,-1089.91 47,-1099.91 47,-1099.91\"/>\n",
       "<text text-anchor=\"middle\" x=\"74.5\" y=\"-1070.3\" font-family=\"Times,serif\" font-size=\"14.00\">50x13x13</text>\n",
       "</g>\n",
       "<!-- fullyconnected4 -->\n",
       "<g id=\"node12\" class=\"node\"><title>fullyconnected4</title>\n",
       "<polygon fill=\"#fb8072\" stroke=\"black\" points=\"94,-1268 -7.10543e-15,-1268 -7.10543e-15,-1210 94,-1210 94,-1268\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1242.8\" font-family=\"Times,serif\" font-size=\"14.00\">FullyConnected</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1227.8\" font-family=\"Times,serif\" font-size=\"14.00\">500</text>\n",
       "</g>\n",
       "<!-- fullyconnected4&#45;&gt;flatten2 -->\n",
       "<g id=\"edge11\" class=\"edge\"><title>fullyconnected4&#45;&gt;flatten2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-1199.81C47,-1186.15 47,-1171.01 47,-1158.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-1209.91 42.5001,-1199.91 47,-1204.91 47.0001,-1199.91 47.0001,-1199.91 47.0001,-1199.91 47,-1204.91 51.5001,-1199.91 47,-1209.91 47,-1209.91\"/>\n",
       "<text text-anchor=\"middle\" x=\"61\" y=\"-1180.3\" font-family=\"Times,serif\" font-size=\"14.00\">8450</text>\n",
       "</g>\n",
       "<!-- activation12 -->\n",
       "<g id=\"node13\" class=\"node\"><title>activation12</title>\n",
       "<polygon fill=\"#ffffb3\" stroke=\"black\" points=\"94,-1378 -7.10543e-15,-1378 -7.10543e-15,-1320 94,-1320 94,-1378\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1352.8\" font-family=\"Times,serif\" font-size=\"14.00\">Activation</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1337.8\" font-family=\"Times,serif\" font-size=\"14.00\">relu</text>\n",
       "</g>\n",
       "<!-- activation12&#45;&gt;fullyconnected4 -->\n",
       "<g id=\"edge12\" class=\"edge\"><title>activation12&#45;&gt;fullyconnected4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-1309.81C47,-1296.15 47,-1281.01 47,-1268.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-1319.91 42.5001,-1309.91 47,-1314.91 47.0001,-1309.91 47.0001,-1309.91 47.0001,-1309.91 47,-1314.91 51.5001,-1309.91 47,-1319.91 47,-1319.91\"/>\n",
       "<text text-anchor=\"middle\" x=\"57.5\" y=\"-1290.3\" font-family=\"Times,serif\" font-size=\"14.00\">500</text>\n",
       "</g>\n",
       "<!-- fullyconnected5 -->\n",
       "<g id=\"node14\" class=\"node\"><title>fullyconnected5</title>\n",
       "<polygon fill=\"#fb8072\" stroke=\"black\" points=\"94,-1488 -7.10543e-15,-1488 -7.10543e-15,-1430 94,-1430 94,-1488\"/>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1462.8\" font-family=\"Times,serif\" font-size=\"14.00\">FullyConnected</text>\n",
       "<text text-anchor=\"middle\" x=\"47\" y=\"-1447.8\" font-family=\"Times,serif\" font-size=\"14.00\">10</text>\n",
       "</g>\n",
       "<!-- fullyconnected5&#45;&gt;activation12 -->\n",
       "<g id=\"edge13\" class=\"edge\"><title>fullyconnected5&#45;&gt;activation12</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M47,-1419.81C47,-1406.15 47,-1391.01 47,-1378.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"47,-1429.91 42.5001,-1419.91 47,-1424.91 47.0001,-1419.91 47.0001,-1419.91 47.0001,-1419.91 47,-1424.91 51.5001,-1419.91 47,-1429.91 47,-1429.91\"/>\n",
       "<text text-anchor=\"middle\" x=\"57.5\" y=\"-1400.3\" font-family=\"Times,serif\" font-size=\"14.00\">500</text>\n",
       "</g>\n",
       "<!-- softmax_label -->\n",
       "<g id=\"node15\" class=\"node\"><title>softmax_label</title>\n",
       "<ellipse fill=\"#8dd3c7\" stroke=\"black\" cx=\"159\" cy=\"-1459\" rx=\"47\" ry=\"29\"/>\n",
       "<text text-anchor=\"middle\" x=\"159\" y=\"-1455.3\" font-family=\"Times,serif\" font-size=\"14.00\">softmax_label</text>\n",
       "</g>\n",
       "<!-- softmax -->\n",
       "<g id=\"node16\" class=\"node\"><title>softmax</title>\n",
       "<polygon fill=\"#fccde5\" stroke=\"black\" points=\"170,-1598 76,-1598 76,-1540 170,-1540 170,-1598\"/>\n",
       "<text text-anchor=\"middle\" x=\"123\" y=\"-1565.3\" font-family=\"Times,serif\" font-size=\"14.00\">softmax</text>\n",
       "</g>\n",
       "<!-- softmax&#45;&gt;fullyconnected5 -->\n",
       "<g id=\"edge14\" class=\"edge\"><title>softmax&#45;&gt;fullyconnected5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M97.2538,-1531.41C87.3525,-1517.34 76.2083,-1501.51 66.9347,-1488.33\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"103.23,-1539.91 93.7952,-1534.32 100.353,-1535.82 97.4753,-1531.73 97.4753,-1531.73 97.4753,-1531.73 100.353,-1535.82 101.155,-1529.14 103.23,-1539.91 103.23,-1539.91\"/>\n",
       "<text text-anchor=\"middle\" x=\"97\" y=\"-1510.3\" font-family=\"Times,serif\" font-size=\"14.00\">10</text>\n",
       "</g>\n",
       "<!-- softmax&#45;&gt;softmax_label -->\n",
       "<g id=\"edge15\" class=\"edge\"><title>softmax&#45;&gt;softmax_label</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M135.648,-1530.06C140.313,-1516.06 145.505,-1500.48 149.807,-1487.58\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"132.365,-1539.91 131.258,-1529 133.946,-1535.16 135.527,-1530.42 135.527,-1530.42 135.527,-1530.42 133.946,-1535.16 139.796,-1531.84 132.365,-1539.91 132.365,-1539.91\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fc423829bd0>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Model\n",
    "\n",
    "data = mx.sym.Variable('data')\n",
    "#mxnet.symbol.Convolution(data=None, weight=None, bias=None, kernel=_Null, stride=_Null, dilate=_Null, pad=_Null, num_filter=_Null, num_group=_Null, workspace=_Null, no_bias=_Null, cudnn_tune=_Null, cudnn_off=_Null, layout=_Null, name=None, attr=None, out=None, **kwargs)¶\n",
    "\n",
    "# conv 1 layer\n",
    "conv1 = mx.sym.Convolution(data=data, kernel=(5,5), num_filter=30)\n",
    "act1 = mx.sym.Activation(data=conv1, act_type='relu')\n",
    "pool1 = mx.sym.Pooling(data=act1, pool_type=\"max\", kernel=(2,2))\n",
    "\n",
    "# conv 2 layer\n",
    "conv2 = mx.sym.Convolution(data=pool1, kernel=(5,5), num_filter=50)\n",
    "act2 = mx.sym.Activation(data=conv2, act_type='relu')\n",
    "pool2 = mx.sym.Pooling(data=act2, pool_type=\"max\", kernel=(2,2))\n",
    "\n",
    "# conv 3 layer\n",
    "conv3 = mx.sym.Convolution(data=pool2, kernel=(5,5), num_filter=50)\n",
    "act3 = mx.sym.Activation(data=conv3, act_type='relu')\n",
    "pool3 = mx.sym.Pooling(data=act3, pool_type=\"max\", kernel=(2,2))\n",
    "\n",
    "\n",
    "# Fully connected layer\n",
    "flatten = mx.sym.Flatten(pool3)\n",
    "fc1 = mx.sym.FullyConnected(data=flatten, num_hidden=500)\n",
    "act3 = mx.sym.Activation(data=fc1, act_type='relu')\n",
    "\n",
    "fc2 = mx.sym.FullyConnected(data=act3, num_hidden=10)\n",
    "\n",
    "lenet = mx.sym.SoftmaxOutput(data=fc2, name=\"softmax\")\n",
    "\n",
    "# Output layer\n",
    "\n",
    "shape = {\"data\": (batch_size, 1, 28, 28)}\n",
    "mx.viz.plot_network(lenet, shape=shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Already bound, ignoring bind()\n",
      "INFO:root:Epoch[0] Batch [200]\tSpeed: 9048.97 samples/sec\tTrain-accuracy=0.100299\n",
      "INFO:root:Epoch[0] Train-accuracy=0.097172\n",
      "INFO:root:Epoch[0] Time cost=6.640\n",
      "INFO:root:Epoch[0] Validation-accuracy=0.100000\n",
      "INFO:root:Epoch[1] Batch [200]\tSpeed: 9177.58 samples/sec\tTrain-accuracy=0.106294\n",
      "INFO:root:Epoch[1] Train-accuracy=0.167475\n",
      "INFO:root:Epoch[1] Time cost=6.534\n",
      "INFO:root:Epoch[1] Validation-accuracy=0.359800\n",
      "INFO:root:Epoch[2] Batch [200]\tSpeed: 9134.87 samples/sec\tTrain-accuracy=0.332289\n",
      "INFO:root:Epoch[2] Train-accuracy=0.640758\n",
      "INFO:root:Epoch[2] Time cost=6.542\n",
      "INFO:root:Epoch[2] Validation-accuracy=0.606100\n",
      "INFO:root:Epoch[3] Batch [200]\tSpeed: 9189.22 samples/sec\tTrain-accuracy=0.722687\n",
      "INFO:root:Epoch[3] Train-accuracy=0.758788\n",
      "INFO:root:Epoch[3] Time cost=6.521\n",
      "INFO:root:Epoch[3] Validation-accuracy=0.748600\n",
      "INFO:root:Epoch[4] Batch [200]\tSpeed: 9182.11 samples/sec\tTrain-accuracy=0.781667\n",
      "INFO:root:Epoch[4] Train-accuracy=0.798333\n",
      "INFO:root:Epoch[4] Time cost=6.512\n",
      "INFO:root:Epoch[4] Validation-accuracy=0.757200\n",
      "INFO:root:Epoch[5] Batch [200]\tSpeed: 9010.95 samples/sec\tTrain-accuracy=0.811269\n",
      "INFO:root:Epoch[5] Train-accuracy=0.823283\n",
      "INFO:root:Epoch[5] Time cost=6.678\n",
      "INFO:root:Epoch[5] Validation-accuracy=0.790300\n",
      "INFO:root:Epoch[6] Batch [200]\tSpeed: 8960.94 samples/sec\tTrain-accuracy=0.835373\n",
      "INFO:root:Epoch[6] Train-accuracy=0.843788\n",
      "INFO:root:Epoch[6] Time cost=6.716\n",
      "INFO:root:Epoch[6] Validation-accuracy=0.804100\n",
      "INFO:root:Epoch[7] Batch [200]\tSpeed: 8701.52 samples/sec\tTrain-accuracy=0.852040\n",
      "INFO:root:Epoch[7] Train-accuracy=0.858384\n",
      "INFO:root:Epoch[7] Time cost=6.769\n",
      "INFO:root:Epoch[7] Validation-accuracy=0.806800\n",
      "INFO:root:Epoch[8] Batch [200]\tSpeed: 9196.72 samples/sec\tTrain-accuracy=0.865672\n",
      "INFO:root:Epoch[8] Train-accuracy=0.869040\n",
      "INFO:root:Epoch[8] Time cost=6.521\n",
      "INFO:root:Epoch[8] Validation-accuracy=0.816200\n",
      "INFO:root:Epoch[9] Batch [200]\tSpeed: 9101.51 samples/sec\tTrain-accuracy=0.876194\n",
      "INFO:root:Epoch[9] Train-accuracy=0.878636\n",
      "INFO:root:Epoch[9] Time cost=6.569\n",
      "INFO:root:Epoch[9] Validation-accuracy=0.813000\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "\n",
    "import logging\n",
    "logging.getLogger().setLevel(logging.DEBUG)\n",
    "\n",
    "ctx = [mx.gpu(i) for i in range(2)]\n",
    "num_epoch = 10\n",
    "\n",
    "net = mx.mod.Module(symbol=lenet, context=ctx)\n",
    "net.bind(data_shapes=[train_iter.provide_data[0]], label_shapes=[train_iter.provide_label[0]])\n",
    "\n",
    "net.fit(train_iter,\n",
    "       val_iter,\n",
    "       optimizer=\"sgd\",\n",
    "       optimizer_params={'learning_rate' : 0.1},\n",
    "       eval_metric='acc',\n",
    "       batch_end_callback=mx.callback.Speedometer(batch_size, 200),\n",
    "       num_epoch=num_epoch\n",
    "       )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACuhJREFUeJzt3U1sTW0bxfF9VFut9lFSbUq0lKRIGgMTKlJBmBMxNhTz\nJow6wMiI0oGxCCPMpSGCoBMxE0kT8Zn46PcX7Tvy5k28e13n6e20p9b/N132PrunXfbg2ve9C/Pz\n8xkAPyuW+gIALA3KD5ii/IApyg+YovyAKcoPmKL8gCnKD5ii/ICplYv5YYVCgccJy0xVVZXMZ2Zm\nFulKfldTUyPzycnJRbqS5WV+fr5QzL/jzg+YovyAKcoPmKL8gCnKD5ii/IApyg+YKizmTj7M+Utj\nxYr8/8Pn5uaSzr1p0yaZnzhxQubqOYG+vr4FXdMv6ufOsixL+dtezjtcMecHIFF+wBTlB0xRfsAU\n5QdMUX7AFOUHTDHnXwYKBT22Tfkd3r59W+bd3d0yHxoaknllZWVu1tzcLI/t7e2V+fXr12Veyucf\nyhlzfgAS5QdMUX7AFOUHTFF+wBTlB0wt6tbdWJjq6mqZT01N5WbRKG/79u0yj8ZxKfbs2SPz6No/\nf/4s87t37+ZmK1fqP/0fP37I/G/AnR8wRfkBU5QfMEX5AVOUHzBF+QFTlB8wxZLeMlBRUSHznz9/\nyvz48eO52aVLl+SxW7ZskflSOnbsmMx7enpkrp4jUEuNsyzLZmdnZV7OWNILQKL8gCnKD5ii/IAp\nyg+YovyAKcoPmGLOXwbq6upkPjY2JvMbN27kZo8ePZLH9vf3yzx6DfZSboF9//59mZ8/fz43GxgY\nkMcu5/X+zPkBSJQfMEX5AVOUHzBF+QFTlB8wRfkBU+zbXwaiOX5k48aNuVk0x4+Uco6f+gzBixcv\nZN7Z2ZmbRXP+qqoqmZfznL9Y3PkBU5QfMEX5AVOUHzBF+QFTlB8w9deM+goFvYoxWqKZsrQ5dVl0\ntDV3JPrZS3nulM+Oxmnq1eNZlmWTk5My7+rqys0uX74sj52enpZ5JOV7Waxl9tz5AVOUHzBF+QFT\nlB8wRfkBU5QfMEX5AVN/zZw/mo0u51cuR/bu3Vuyc0ffa8pMOprjR6Itz9esWbPgc6c+e7GYW+Iv\nFHd+wBTlB0xRfsAU5QdMUX7AFOUHTFF+wFRZzfmjNdAps1O1tjvLsuzJkycyr6ioyM1KvY3z1q1b\nZV5ZWVnSzy9XbW1tMt+9e/ciXcnyxJ0fMEX5AVOUHzBF+QFTlB8wRfkBU5QfMFVYzHXHhUKhZB92\n8uRJmV+7dk3m4+PjC85Xr16ddO6hoSGZr127VuZNTU252eDgoDx2165dMo/WtUfPOKjnI2pra+Wx\nr1+/lvmqVatkrt4L8Pz5c3ns0aNHZT4yMiLzT58+yXxiYiI3i57r6O3tlfmdO3eKemkAd37AFOUH\nTFF+wBTlB0xRfsAU5QdMUX7AVFmt54/U19cvKMuyLPv8+bPMoz3e1bx6eHhYHqtmulmWZa2trTJv\naGiQudoHobOzUx4bzfGjvfWj49VeA9H30tLSIvPp6WmZq++tvb1dHhvtkRD9TqJr//btW2727Nkz\neezTp09lXizu/IApyg+YovyAKcoPmKL8gCnKD5haVqO+c+fO5WYXL16Ux546dUrm0ehGLX1euVJ/\njdHS08iHDx9kvmJF/v/hKssyPcIsRrTdulpWGy11bmxslHk0jhsYGMjNoteaRyPMubk5mads5/7u\n3TuZR8uFi8WdHzBF+QFTlB8wRfkBU5QfMEX5AVOUHzC1rLbuVks4oyW5r169knm0Pfbo6GhuNjs7\nK48dGxuT+eTkpMxrampkrmbp0bVFs/JoW/KI+tmia4tEzwH09fXlZqdPn5bHqt93lunvPMvi5wDU\n8xUfP36Uxx45ckTmY2NjbN0NIB/lB0xRfsAU5QdMUX7AFOUHTFF+wFRZrec/c+aMzB88eJCbRVtM\nr1+/XubR8w5qTX60Jj5a2/3161eZr1u3TuZqVh+tmY9+7mgvgWg9v3rddLQPQvQMQvSK74MHDy74\nsyPRHD/aR0H9Pf3zzz9J5y4Wd37AFOUHTFF+wBTlB0xRfsAU5QdMUX7AVFnN+ffv3y/zs2fPLvjc\n0frr6DkBNVuN1ttHM+Xo2qL9ANQe801NTfLYaE4frZmPfjZ1/mivgOh77e/vl7mapUevVT9w4IDM\no9eDR89+qO8t+s6j5wCKxZ0fMEX5AVOUHzBF+QFTlB8wRfkBU5QfMFVWc/6Ojg6ZDw4OLvjc0drw\naN27kvKO+mKOj65dPaMQvWc+mtNH6/2jWb3aqyA69v379zKvrq6WeXd3d2725s2bpM/etGmTzCcm\nJmSuvve6ujp5bPT3VCzu/IApyg+YovyAKcoPmKL8gCnKD5gqq1FfNLpJES2xTNmKORqHReeO8mjU\npz4/Onc0NopeHx6NSNX3Hl1btN36hg0bZP7w4cPcLHo9eFdXl8yjEapaTpxl+ncW/b7b2tpkXizu\n/IApyg+YovyAKcoPmKL8gCnKD5ii/ICpsprzp9i8eXPS8dHc9k+9Fvn/SV1Wq46Pfq7UeXV9fb3M\no6WtSvSdR9trv337NjdrbW1N+uzotevR8xPqGYfoeZfUv/VfuPMDpig/YIryA6YoP2CK8gOmKD9g\nivIDpspqzh+t71b27du3ZJ+dKprjR9ScP3VeHT0HEL2qWu0HEL1qOnpterQmv729PTeLrntmZkbm\n0Xbr0fMN6u+toaFBHtvc3CzzYnHnB0xRfsAU5QdMUX7AFOUHTFF+wBTlB0yV1Zw/Zd69bdu2P3gl\nv1Nz2WjmG4mOj/JS7jUQ7ds/PDy84HOPjIzIPPp7iPa3Hx0dzc2i9zik5pHoGQUl2ougWNz5AVOU\nHzBF+QFTlB8wRfkBU5QfMFVWo76UsVFjY2PSZ0ejF7WFdbTsNfX14NHIK+UV3dG24dGy2+hnV0uG\no+XE0SgvGnGmjECj7yU6d+pr15UtW7Ys+Nj/xZ0fMEX5AVOUHzBF+QFTlB8wRfkBU5QfMFVWc/4r\nV67I/OrVq7lZNHeNlqambN2dOvNN3bpbLfmNnjEo5bbhWaZfVR0tVS7lduqpy7BTz5/yvf+pa+fO\nD5ii/IApyg+YovyAKcoPmKL8gCnKD5gqqzn/rVu3ZL5z587crKenRx4brdeP5uHq+NS136lbc5dy\nZl3KeXUk+l6XUup3Hu2DoNTW1iZ99i/c+QFTlB8wRfkBU5QfMEX5AVOUHzBF+QFThVLOaX/7sEKh\nZB/28uVLme/YsUPmU1NTMo/2mFdK/Yptte49uu7UZwhS3ikQHVvKV5+nnruUvzO1B0KWZdnjx49l\nfvjw4aJ+OO78gCnKD5ii/IApyg+YovyAKcoPmKL8gKnyXTD9L3379k3m0frp1PfYp5y7lHvnRzPj\nUl9biqX87Og5gFI+J1BZWSmP7ejoSPrs/17DHzkLgGWH8gOmKD9givIDpig/YIryA6b+miW94+Pj\nMo9GdTMzMzKvq6v719e0WNQY88uXL0nnXsptwaPlyCnLiaMludES7+jvLRotq1fGr169Wh578+ZN\nmV+4cIElvQDyUX7AFOUHTFF+wBTlB0xRfsAU5QdM/TVz/miOf+jQIZm3tLTIXM1eo2Wz0TME379/\nl/no6KjM7927J3N4mZ+fZ84PIB/lB0xRfsAU5QdMUX7AFOUHTFF+wNSizvkBlA/u/IApyg+YovyA\nKcoPmKL8gCnKD5ii/IApyg+YovyAKcoPmKL8gCnKD5ii/IApyg+YovyAKcoPmKL8gCnKD5ii/IAp\nyg+YovyAKcoPmKL8gKn/ACt29ccpmY9pAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc42368ded0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted: 8 , probability: 0.930137 \n"
     ]
    }
   ],
   "source": [
    "\n",
    "# prediction function\n",
    "\n",
    "idx = 30; \n",
    "plt.imshow(val_img[idx], cmap='Greys_r')\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "\n",
    "preds = net.predict(val_iter)\n",
    "prob = preds[idx].asnumpy()\n",
    "\n",
    "print \"predicted: %d , probability: %f \" % (prob.argmax(), max(prob))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}