02_PH_photos_multi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparing a computer vision training set for labeling photos \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "This notebook covers the steps involved in preparing a labeled dataset of images derived from the Newspaper Navigator dataset. Specifically this training set is primarily intended for use in a Programming Historian lesson on computer vision. \n",
    "\n",
    "## Aims \n",
    "This notebook cover the steps to creating a dataset based on the news-navigator dataset. This includes:\n",
    "- creating a subsample of the news navigator training set\n",
    "- setting up the annotation process\n",
    "- processing the annotations \n",
    "- uploading the dataset to Zenodo\n",
    "\n",
    "It doesn't cover training a model based on this data. This process is covered in a the programming historian lesson for which this dataset was created. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by importing the `sample` module. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nnanno.sample import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create an nnSampler instance which we can use to create a sample. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sampler = nnSampler()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can access some information about the 'population' from the sampler object. For example we can see how the number of photos in the collection changes over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD4CAYAAAAO9oqkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAqR0lEQVR4nO3deZhcZZ328e+vtl7TS5LO2p0F0gFC2MMioiBrQEdwxgWYgSgor4IIo74zoDPqDDojo+8w4DbDCIqjIyKDA0IQEAiLypJAdhLSWbuz9L7vVfW8f9SpTiXpTjrp7lSdrvtzXX119XPOqXoORequZzvHnHOIiEh2C6S7AiIikn4KAxERURiIiIjCQEREUBiIiAgQSncFjtTkyZPdnDlz0l0NERHfWLFiRYNzrmywbb4Ngzlz5rB8+fJ0V0NExDfMbPtQ29RNJCIiCgMREVEYiIgICgMREUFhICIiKAxERASFgYiIoDAQyXrxuOPhN3bQ2tWf7qpIGikMRLLcU2t2c8dja3jg1S3proqkkcJAJIvF4477nt8EwNNr96S5NpJOCgORLLZ07W421XVwzjET2VTXwaba9nRXSdJEYSCSpeJxx72/30TllELu+cSpgFoH2UxhIJKlkq2CL1xUyfTiPBbNLmXpmt3prpakicJAJAslWwXzphRyxUnTAbj8pOls2NPOlvqONNdO0kFhIJKFNta2s6mug8+8by7BgAGweOE0QF1F2UphIJKFdrd2A1A5dcJA2cySPE6tKOHpteoqykYKA5EstKe1F4BpRbn7lF9x0jTW7mxjR2NXOqolaaQwEMlCtW09mEHZhJx9yhefmBg/eHFjXTqqJWmkMBDJQrVtPUwqyCEc3PcjoGJiHsV5YTZqvUHWURiIZKE9bT1MK845oNzMqJxSSFWtZhRlG4WBSBba09pzwHhBUuXUQqo0vTTrKAxEslBdey9ThwiDeVMm0NTZR2NH71GulaSTwkAky/RGYzR19h0kDAoB2FSn1kE2URiIZJm6tsGnlSZVKgyy0rDCwMz+2szWmdlaM/ulmeWa2Vwze93MqszsV2YW8fbN8f6u8rbPSXmeO73yjWZ2WUr5Yq+syszuGPWzFJEBtW09AEwtHjwMphfnUhAJsllhkFUOGQZmNhP4ArDIObcQCAJXA3cD9zjn5gHNwI3eITcCzV75Pd5+mNkC77gTgcXAD80saGZB4AfA5cAC4BpvXxEZA3uSYVB04GwiSMwomjd1ApvqNL00mwy3mygE5JlZCMgHdgMXAo962x8CrvIeX+n9jbf9IjMzr/xh51yvc24rUAWc5f1UOee2OOf6gIe9fUVkDOxpTYTBUN1EkOgq2qTppVnlkGHgnNsJfBfYQSIEWoEVQItzLurtVgPM9B7PBKq9Y6Pe/pNSy/c7ZqjyA5jZTWa23MyW19fXD+f8RGQ/tW095IQCFOeFh9xn3pRC6tp7ae3WfZGzxXC6iUpJfFOfC8wACkh08xx1zrn7nXOLnHOLysrK0lEFEd+rbetlWnEuiQb74JKDyFUaN8gaw+kmuhjY6pyrd871A48B7wVKvG4jgHJgp/d4J1AB4G0vBhpTy/c7ZqhyERkDe9p6mDph6C4igMopiauZVmncIGsMJwx2AOeYWb7X938RsB54Efiot88S4HHv8RPe33jbX3DOOa/8am+20VygEngDeBOo9GYnRUgMMj8x8lMTkcHUtvUMOZMoaWZpHrnhgMYNskjoUDs45143s0eBt4Ao8DZwP/AU8LCZfdMre8A75AHgv8ysCmgi8eGOc26dmT1CIkiiwC3OuRiAmX0eeIbETKUHnXPrRu8URSTJOcee1h4uXTD4TKKkYMA4tqxQaw2yyCHDAMA593Xg6/sVbyExE2j/fXuAjw3xPN8CvjVI+VJg6XDqIiJHrq07Sm80PuTq41TzphSyfFvzUaiVZAKtQBbJInvXGBw6DCqnFLKzpZvO3ugh9xX/UxiIZJFkGEw7xJgBJC5YB7BZVzDNCgoDkSxSO4wFZ0mVU71rFGkQOSsoDESySPK6RPvf7nIwsyfmEwqYWgZZQmEgkkX2tPVQmh8mNxw85L6hYICKiflsb+w6CjWTdFMYiGSR2raeYQ0eJ82ZlM/Whs4xrJFkCoWBSBapbRv6DmeDmTO5gG2NnSTWjcp4pjAQySJ72oa+9/Fg5k4uoKsvRn27boE53ikMRLJEfyxOQ0fvIS9FkWrOpAIAdRVlAYWBSJaob+/FueFNK02aOzkRBtsaFQbj3bAuRyEi/vbWjmbuenI9AMeUFQz7uBkleUSCAbY2aEbReKeWgcg4sLOlmy8+spKe/tg+5X3ROLc//DZ//sM/UtPczXc+ejJnz5047OcNBoyKiXlsUzfRuKcwEBkHXninlsfe2snqmtZ9yl/ZVM//rtzFjefNZdmXL+BjiyoOelObwcz1ZhTJ+KYwEBkHqpu7gQP79pMDv7deOI+CnCPrFZ49KREG8biml45nCgORcWCHt0p4/+6crQ2dlOSHKcmPHPFzz5lcQE9/nNr2nhHVUTKbwkBkHKhuToTB/peO2NbYyexJwx8wHsxcTS/NCgoDkXFgR5PXMtivm2hbQxdzJ+WP6LnnTM4feC4ZvxQGIj7X2tVPe0+USDDAtoa9l47o6Y+xq7WbOZNH1jKYUZxHJBRg+2EOIte199Da1T+i15ajR2Eg4nPJLqIzZpfS2RejoaMvUd7UhXN7VxEfqUDAmD3x8C5Y19Eb5c++9yrf+K1uZ+4XCgMRn0t2Eb1v/mSAgW/wyQ/vkbYMks9xONNLv/f8Jmrbemnp6hvxa8vRoTAQ8bnqZBjMKwP2hkByMHnuCFsGkFhrsL2xi3jc0djRy8f//U88uqJm0H2r6jp44NWtAPTHNB3VL3Q5ChGf29HURUl+mOOnTyAYsIEQ2NqYmFZanB8e8WvMmVRAbzTO1sZOvvTIKlZWt9Abi/PRM8r32c85xz/8dh15kSDTi3Ppi8VH/NpydKhlIOJz1c3dVJTmEw4GKC/NG+jO2dbQOeLxgqTkjKLPPLScVTUtnD13IqtrWmjs2PfS1s+ur+WVTQ188ZL5TC3KpV9h4BsKAxGfq27qYtbExIf1nEkF+4TB3FEYL4C9Vy/d0tDJ339wAV+54gScg1c2NQzsE487vv30Bo6bOoHrzplNJBhQGPiIwkDEx+Jxx87mbson5gGJ21Rub+jyppX2jFrLYOqEXI4pK+Cz5x/LDefN5aSZxUwqiPDixrqBfV7b2sjWhk4+d8GxhIIBwsEA/VGNGfiFxgxEfKy2vYe+WHygZTB7UgHtvVHe3tEC7O3eGalAwHj+i+cPXOQuEDDOn1/GixvriMUdwYDxqzerKcoNsXjhNADCIbUM/EQtAxEfS16TqKI08aGf7M5Z9m7iG/totQyAA652ev5xZTR39bO6poWWrj6eXruHj5w2k9xwEIBw0DSA7CNqGYj4WPJqpRUDLYPE72Ub6oHRDYP9vb+yjIDBixvrWVXdQl80zifOnDWwXWMG/qIwEPGxHU1dmMHMksSYQXlpPgGDjbXtlI7StNKhlBZEOKWihJc21tEbjXNyeTELZhQNbA8HA1pn4CPqJhLxsZqmLqYX5RIJJf4pR0IByr0uo9FYeXwoHzhuCqtqWtmwp52PL6rYZ1tYLQNfURiI+NiOpq6BLqKkZFfRaKw8PpQLjkuses4NB/jwqTP22RYOmcLARxQGIj5W3XxgGCTHCUZ6H4PhWDijmJkleXzktJkU5e7bJRVRN5GvaMxAxKd6+mPUtvUOzCRKSnYPjda00oMJBIylt72PPG8GUapwMEAs7gamnkpmUxiI+FSNN5No1qS8fcpPLi8mYHDijOKjUo/ivMEHqcPBRMdDfyxOMHBgWEhmURiI+FTyPgb7twzOnDORFX93CaUFR37f49EQDiZaA/2x+MDaA8lcGjMQ8amdXstgZmneAdvSHQTAwAwnjRv4w7DCwMxKzOxRM9tgZu+Y2XvMbKKZPWdmm7zfpd6+Zmb3mVmVma02s9NTnmeJt/8mM1uSUn6Gma3xjrnP9l/qKCIHqG/vxQwmF+akuyqDSu0mksw33JbBvcDvnHPHA6cA7wB3AM875yqB572/AS4HKr2fm4AfAZjZRODrwNnAWcDXkwHi7fOZlOMWj+y0RMa/+o5eJuZHBj50M03IGzTuiyoM/OCQ/xeZWTHwfuABAOdcn3OuBbgSeMjb7SHgKu/xlcDPXMJrQImZTQcuA55zzjU555qB54DF3rYi59xrLnEn75+lPJeIDKG+vZeyCZnZKoDUbiKFgR8M5yvFXKAe+ImZvW1mPzazAmCqc263t88eYKr3eCZQnXJ8jVd2sPKaQcoPYGY3mdlyM1teX18/jKqLjF+ZHgZ7u4k0ZuAHwwmDEHA68CPn3GlAJ3u7hADwvtGP+TvunLvfObfIObeorKxsrF9OJKPVt/dSlqHjBaAxA78ZThjUADXOude9vx8lEQ61XhcP3u/kXS52AqkXKSn3yg5WXj5IuYgMwTlHQ0emtwy8MQOFgS8cMgycc3uAajM7ziu6CFgPPAEkZwQtAR73Hj8BXO/NKjoHaPW6k54BLjWzUm/g+FLgGW9bm5md480iuj7luURkEO29UXqj8YydSQSJy1EA9GsA2ReGu+jsVuAXZhYBtgCfIhEkj5jZjcB24OPevkuBK4AqoMvbF+dck5ndBbzp7fePzrkm7/HNwE+BPOBp70dEhlDfnrgRfUa3DLTOwFeGFQbOuZXAokE2XTTIvg64ZYjneRB4cJDy5cDC4dRFRHwSBhoz8JXMnKAsIgfljzDQmIGfKAxEfGggDPwwZqAw8AWFgYgP1Xf0Eg7akFcMzQTqJvIXhYGID9W39zK5MIdABt8nYGAAOaoBZD9QGIj4UKavPgaNGfiNwkDEhzJ99TFozMBvFAYiPlSf4auPQWMGfqMwEPGZWNzR6Ksw0JiBHygMRHymuauPuMvsNQaQMmagy1H4gsJAxGf8sMYAwMwIB03dRD6hMBDxGT+sPk4KBwMKA59QGIj4jP/CQGMGfqAwEPGZ+o5EGGTy5auTwsGA1hn4hMJAxGfq23vJjwQpyBnuFejTJxI03c/AJxQGIj7jh9XHSeGQxgz8QmEg4jN+WH2cpDED/1AYiPiMH1YfJ2nMwD8UBiI+46duoojWGfiGwkDER3qjMVq7+33WTaQw8AOFgYiPNHb0Af5YYwBeGOh+Br6gMBDxET8tOIPEbCKNGfiDwkDER/wWBhoz8A+FgYiP1LX7Z/UxaMzATxQGIj6yqa6dvHCQqUW56a7KsGidgX8oDER8ZN3ONhbMKCIYsHRXZVjCwYDuZ+ATCgMRn4jHHet2tbJwRlG6qzJskZDGDPxCYSDiE9ubuujsi3HijOJ0V2XYNGbgHwoDEZ9Yu7MVgBNn+qdloDED/1AYiPjE2l2tRIIBKqdMSHdVhk3XJvIPhYGIT6zf1cb8aYVEQv75Z6t1Bv7hn/+rRLKYc461O1tZ6KPxAki0DJyDWFxdRZlOYSDiA7tae2ju6ufEmT4LA68Vo9ZB5lMYiPjAuuTgsY+mlUKiZQBo3MAHFAYiPrB2VxsBgxOm+SsMIsHE4jjdBznzKQxEfGDdzlbmTSkkLxJMd1UOS7JloOmlmW/YYWBmQTN728ye9P6ea2avm1mVmf3KzCJeeY73d5W3fU7Kc9zplW80s8tSyhd7ZVVmdsconp/IuLBuV5uvFpsl7Q0DtQwy3eG0DG4D3kn5+27gHufcPKAZuNErvxFo9srv8fbDzBYAVwMnAouBH3oBEwR+AFwOLACu8fYVERKXrd7T1uO78QLYO4CsMYPMN6wwMLNy4IPAj72/DbgQeNTb5SHgKu/xld7feNsv8va/EnjYOdfrnNsKVAFneT9Vzrktzrk+4GFvXxEB1u1KDB4v9NlMIkgZM1AYZLzQMPf7N+BvgOTSx0lAi3Mu6v1dA8z0Hs8EqgGcc1Eza/X2nwm8lvKcqcdU71d+9vBPQWT8icbiPLu+lqVrdvPihjpyQgEW+LFlkOwm0q0vM94hw8DMPgTUOedWmNkFY16jg9flJuAmgFmzZqWzKiJj6nsvVHHv85uYVBDhw6fO4OOLKijKDae7WodNU0v9Yzgtg/cCHzazK4BcoAi4Fygxs5DXOigHdnr77wQqgBozCwHFQGNKeVLqMUOV78M5dz9wP8CiRYv0VUPGJeccT6zaxXuOmcTPP322b+5dMBgNIPvHIccMnHN3OufKnXNzSAwAv+Cc+0vgReCj3m5LgMe9x094f+Ntf8E557zyq73ZRnOBSuAN4E2g0pudFPFe44lROTsRH9pc38HWhk6uOGmar4MAEvczAIWBHwx3zGAwfws8bGbfBN4GHvDKHwD+y8yqgCYSH+4459aZ2SPAeiAK3OKciwGY2eeBZ4Ag8KBzbt0I6iXia8+urwXg4gVT01yTkVPLwD8OKwycc8uAZd7jLSRmAu2/Tw/wsSGO/xbwrUHKlwJLD6cuIuPVs+tqObm8mOnFeemuyogNjBloADnjaQWySAapa+thZXULl46DVgGoZeAnCgORDPLcO4kuoksWTEtzTUZHRGHgGwoDkQzy7LpaZk/KZ/7UwnRXZVSENYDsGwoDkQzR3tPPnzY3cumCqSQW7fvf3nUGGjPIdAoDkQzx0rv19MXi46aLCFJXIKtlkOkUBiIZoLath39euoHpxbmcMbs03dUZNRoz8A+FgUiatXb3s+TBN2jp6uP+6xb5fqFZqrAuVOcbI1l0JiIj1NMf4zM/W87m+g4e/OSZnFTuvyuTHkwy2DRmkPkUBiJpdM9z7/LG1ibuu+Y03ldZlu7qjDozIxIMqGXgA+omEkkT5xxPrt7NxSdM4cOnzEh3dcZMOGgaQPYBhYFImmyq62BnSzcXnTA+VhsPJRxSy8APFAYiafL8O3UAfOC4KWmuydgKBwMaM/ABhYFImrywoZYTZxQxrTg33VUZUxoz8AeFgUgaNHf2sWJ7MxcdP75bBeCNGSgMMp7CQCQNXnq3nriDC8f5eAEkuokUBplPYSCSBs9vqGNyYYSTZ46vdQWDCQcDup+BDygMRI6y/liclzbW8YHjphAYR6uNh6LZRP6gMBA5ylZsb6atJ8pFJ4z/8QKAiMYMfEFhIHIUVdW184MXqwgHjfPG4YrjwWjMwB90OQqRo+Dd2nbuenI9r2xqIBIK8IULKynMyY5/fuFggM6+WLqrIYeQHf83iqTZPy19h5XVLXzpkvlce/YsJhXmpLtKR004GNDlKHxAYSByFGxt6OT8+WXcelFluqty1EVCGjPwA40ZiIyxaCzOzuZuZk3MT3dV0kJjBv6gMBAZY7tbe4jGHbMnZXMYaJ1BplMYiIyxHU1dAFRkccugTy2DjKcwEBljyTDI1m4irTPwB4WByBjb0dRFOGhML85Ld1XSQrOJ/EFhIDLGdjR2UV6aP65udH84Epej0JhBplMYiIyxHU1dWTteAHvHDJxTIGQyhYHIGNvR1MXsLA6DSDDRIorGFQaZTGEgMoZau/pp7e7P2sFjSLQMAA0iZziFgcgYyvZppZASBrqnQUZTGIiMoWQYZOuCM0gMIAP0x9UyyGQKA5ExtL2pE8julkFyzEDdRJlNYSAyhqqbuphUEMmay1UPRt1E/qAwEBlDO5q6mJXFXUSwNwx0SYrMdsgwMLMKM3vRzNab2Tozu80rn2hmz5nZJu93qVduZnafmVWZ2WozOz3luZZ4+28ysyUp5WeY2RrvmPvMLDtX58i4s72xK6tnEoFmE/nFcFoGUeBLzrkFwDnALWa2ALgDeN45Vwk87/0NcDlQ6f3cBPwIEuEBfB04GzgL+HoyQLx9PpNy3OKRn5pIevXH4uxqyd5LVydFQhoz8INDhoFzbrdz7i3vcTvwDjATuBJ4yNvtIeAq7/GVwM9cwmtAiZlNBy4DnnPONTnnmoHngMXetiLn3GsusUTxZynPJeJbu1q6ibvsvUBdkloG/nBYYwZmNgc4DXgdmOqc2+1t2gNM9R7PBKpTDqvxyg5WXjNI+WCvf5OZLTez5fX19YdTdZGjbntjdl+tNGlgzEADyBlt2GFgZoXA/wC3O+faUrd53+jH/J12zt3vnFvknFtUVlY21i8nMiIDl67WADKglkGmG1YYmFmYRBD8wjn3mFdc63Xx4P2u88p3AhUph5d7ZQcrLx+kXMTXqpu6iIQCTJ2Qm+6qpFVEYeALw5lNZMADwDvOuX9N2fQEkJwRtAR4PKX8em9W0TlAq9ed9AxwqZmVegPHlwLPeNvazOwc77WuT3kuEd/a3thFRWkegSy9dHVSWAPIvjCclTDvBa4D1pjZSq/sK8C3gUfM7EZgO/Bxb9tS4AqgCugCPgXgnGsys7uAN739/tE51+Q9vhn4KZAHPO39iPiWc463q5tZNGdiuquSdnvXGWjMIJMdMgycc68CQ321uWiQ/R1wyxDP9SDw4CDly4GFh6qLiF9U1XVQ29bL++ZNTndV0m6gm0h3O8toWoEsMgZerWoA4LxKhYEGkP1BYSAyBl7d1MCcSfmUl2b3TCKAsC5U5wsKA5FR1h+L89qWRrUKPMlLWGvMILMpDERG2crqFjr7Ypyn8QJAU0v9QmEgMspe3dRAwOA9xygMIPUS1gqDTKYwEBllr1Y1cFJ5CcX54XRXJSMEA0bA1DLIdAoDkVHU3tPPyuoWTSndTzgYSPuYQWt3P/c89y5NnX1prUemUhiIjKLXtzQRizveqzDYRyQYSHvL4L9f38G9z2/iugdep7WrP611yUQKA5FR9GpVA3nhIKfPLkl3VTJKOJT+MHhqzS6mF+fybm07S37yBh290bTWJ9MoDERG0WtbGlk0p5ScUDDdVcko4aClNQy2NXSydmcbN7x3Lt+/9nTW7Gzlhp+8SU9/LG11yjQKA5FR0tUX5d3adk6bVXronbNMOBhI6/0MnlqTuPXK5SdN47ITp/GvHz+FN7Y18dM/bktbnTKNwkBklKzd2UbcwakVxemuSsZJ95jBU6t3c9qskoEV4VeeOpP3zy/jP17arO4ij8JAZJSsrmkB4OTykrTWIxOF0xgGW+o7WL+7jQ+eNH2f8i9eMp/mrn5++oetaalXplEYiIySldUtzCzJY3JhTrqrknHCofSNGSz1uog+ePK+YXBqRQkXnzCF+1/eQmu3ZhcpDERGyeqaVk5RF9GgIsEAL7/bwIXfXcaf//APPPDq8L6Nx+OOu3+3gWv/8zXaeo7sA/vJ1btZNLuU6cV5B2y7/eL5tPVEh12f8UxhIDIKmjr72NHUpS6iIdx+8Xw+fmY5J8wooqc/zl1PrueFDbUHPaanP8atD7/Nj5Zt5o+bG7ntl28Tix98EDoWd9z7+02c+a3fc9k9L7PkwTfYsKf9gFZB0sKZxSw+cRoPvrqV5ixfjKYwEBkFyfGCUxQGg3r//DK+edVJ/ODa03ns5nM5YXoRX/71auraegbdv7Wrn+sfeIOnVu/mK1ccz7c+spAXN9Zz9+82DPkade09XPfA69zz+3dZML2Iion51Lb1cExZAR86ecaQx9160Tw6eqM8uXrXiM/Tz4Zz20sROYRV1a2YwUnl6iY6lNxwkO9dcyof+t6rfPGRVfzshrP2uU90fXsv1z3wOlvqO/neNafxZ6ckPsg37mnn/pe3UJQbojgvzPbGLurae4nFHXHneHNbMx29/fzLX5zMxxaVk7il+qEtmF7EMZMLeHZ9Lde9Z85YnLIvKAxERsHqmhbmlRVSmKN/UsMxb8oEvvahE/nKb9Zw9zMbuPXCSgpzQuxp7eEvf/wau1p6+Mmnztznsh5//6EFbK7v4LvPvgtAbjjA1KJcQgEjYEbllEK+8eETOW7ahMOqi5lxyYlTeeCVrbR291Ocl50XGNT/uSIj5JxjVU0r588vS3dVfOWasyp4Y2sj//HSFn7+p+18+NSZ/KGqgabOPh664SzOmjtxn/3DwQA/+eRZrN/dxvTiXMoKc/ZpUYzEpQum8R8vbWHZxjquPHXmqDyn32jMQGSEdrX20NDRq5lEh8nMuOcTp/LYzedy+UnT+c3bNbR09fHzT599QBAkRUIBTq0oYWpR7qgFAcBpFSVMLszh2fUHH9Qez9QyEBmh1dUtgAaPj4SZcfqsUk6fVcrX/mwB/dE4k9KwTiMQMC5ZMIXfrtpNbzQ27GtL9fTH+MIv32belEI+f+E88iP+/UhVy0BkhFbWtBAOGsdPP7y+atlXUW44LUGQdOmCaXT0RvnT5sZhH/OdZzby7PpafrhsMxf/v5d4es1unPPnvZ4VBiIj9PaOFhZML9KVSn3uPcdOoiASHHZX0Z82N/LgH7Zy3TmzefSz76EoL8znfvEWl9/7Cve/vJk9rYNPm81U/m3TiGSAjXvaeWNrE7ddVJnuqsgI5YaDXHDcFJ5bX8s3r1x40DGJ9p5+vvzrVcyemM+dVxxPfiTEk7eexyPLa/jV8mr+aekG/vnpDcwsyWNSQYSJBRFKCyLe4xzKJuQwoziXacW5zJ5UQPAgr9XW0099ey/HTC4Y9nTZI6EwEBmBHy6rIj8S5JPnzkl3VWQUXLJgKk+t2c2z62tZvHDakPvd9eR6drd28+vPnjswThAKBrj27Flce/YsttR38NtVu9nW2ElTZx8NHX28W9tBY2cvPf37XqPpzDml/PzTZw/asuzpj3HN/a+xblcbM4pzOf+4Ms6fX8YlC6YdNECOhMJA5Ahtb+zkt6t28en3HUNpQSTd1ZFRcPGCqVROKeTmX6zgS5cex+fOP/aAFsIjb1bzyPIabr7gWM6YPfi9K44pK+S2iwdvLXb1Ralr62V3aw8rq1u4+3cb+OaT73DXVQv32c85x1d/s5b1u9u49cJ5bKrt4MlVu3lhQx2XnTh0UB0phYHIEfr3lzYTCgb49Hlz010VGSWFOSH+95b3csdja/jOMxt5a3szd3/05IEr0S7f1sRX/3cN582bzBcvmX9Er5EfCTFncog5kwt4z7GTaO7q4/6Xt3D67BI+clr5wH4/f30H//NWDbdfXMntFydeqz8Wp6a5e0y6ixQGIkdgd2s3j66o4RNnVjClKDfd1ZFRVJAT4r6rT2XR7FK++dR6LvjOMm75wDwWL5zGZ3++gpkleXz/2tMIBUdn/s3fXHYcK6tbuPOxNRTlhomEAlQ3dfOPv13HhcdP4QsX7m1hhIMB5k4uGJXX3Z/5dRrUokWL3PLly9NdDclSX3t8Lb94fQfLvnwBFRPz010dGSNVdR18++l3+P07dZhBYSTEb245l3lTRncacV17Dx+871Xq23sHyo6ZXMBvbn4vxfmjd3kMM1vhnFs02Da1DEQOQ2t3P197fC2Pr9zFNWfNUhCMc/OmFPLjJWfyx6oG/vOVLdxw3txRDwKAKRNyefLW81hd00pJfpjivDCzJuaTGz5605UVBiLD9KfNjXzpkZXUtvfyxUvmc/MFx6a7SnKUnDtvMuemXDRvLEwtyuWSBenrclQYiAzDG1ubWPLgG5SX5vHY587llIqSdFdJZFQpDEQOYWtDJzf913LKJyaCoCRf00hl/NHlKEQOormzj0/95A0CZvzkk2cqCGTcUstAxgXnHJvrO4nFHcGAkRMKUF6ad1jzsVu7+tne1Mm2xi6qm7rY0djFm9ua2NXSw39/5mxmTxqbKX0imSBjwsDMFgP3AkHgx865b6e5SpJB+mNxOnqidPRG6Y3GKMwJU5QXorsvxmNv7eSXb+5gS33nPsfMLMnj4hOmcMHxUwia0dzVR2t3P33RONG4oy8ap7qpiy0NnWyp76C5q3+f48sm5DBrYj53XnECi+YMfn19kfEiI9YZmFkQeBe4BKgB3gSucc6tH+qYsV5nEI3F2d3aQ01zN+GgMXdyARMLIiNa+eecG9MLTY2VeNzR0ReloydKOBigJD9MOGXBTTzuiDlHKGD7nJ9zbuBDty8apy+W+N0bjeFc4sJg+ZHE1Ln6jl7q2nqpa++loaOX+vZeatt6qG7uZmdzFw0dfQet4xmzS/nz02dSkhch5hxt3f0s21jPK5vq6Y3GhzyubEIOx0wu4JiyQo6ZXMDsSfnMnlRAxcQ8X1+bXmQwflhncBZQ5ZzbAmBmDwNXAkOGwZG67J6X6YnGDih3DuLOEY05ovE4LV39ROP7BmVRbojSggj90Th9MUd/LE4snvgNEAkGiIQChIKJe7IGzIjFHV19Ubr7Y0TjjjzvAzAnFCQ1F4IBI2hGIGDEvRt8x5wjFkt8oMa9IEleJiX1AzZgRji493UNMNt7To7EB7Zz4GBgezBgxB3E4olzTn4vSK2Xc9AX27staUJuiFDA6OqL7fNhmwgEvJuUH9l7BFAQCTKlKJfy0jwWLJjKtKI8ivJCFOaEyAkH6eiJ0trdTzQW57KF05g/9cC53391zmy6+qK8vaOFSChAaX6Y4rwIOeEAoYARCiTeLxHJnDCYCVSn/F0DnL3/TmZ2E3ATwKxZs47ohU6pKKbP+/BKfjAmBQJGKGAEA4kPjoqJ+VSU5hONx9lS38mWhg7ae6JEggHCoQDhgBEKJj5YgIFvvtGYw5H4MDQSy9vzIkFCAaO7L0ZXf4ye/pRAciQ++L0P7OQHdcAS9QkFE9+4nUt823Yucfu/nFCiHvG4o98Lsbjb+6EfMDASH84BS/w274zj3usFAzbwY5aoy/7/XXLCQSbkhCjMDRGNO5o7+2jq7CMWd+RHguSGgwQDRjTuiMXjxB0DwRYOGJFQgLAXlDmhADnhIAaJ/xZ9URyJb+hTixL3tS2bkEPBKN1YPj8S2uem6iIyuEwJg2Fxzt0P3A+JbqIjeY5/+egpR/TaFxx3RIeJiPhCprSRdwIVKX+Xe2UiInIUZEoYvAlUmtlcM4sAVwNPpLlOIiJZIyO6iZxzUTP7PPAMiamlDzrn1qW5WiIiWSMjwgDAObcUWJrueoiIZKNM6SYSEZE0UhiIiIjCQEREFAYiIkKGXJvoSJhZPbA93fU4TJOBhnRX4ijTOWcHnbM/zHbOlQ22wbdh4Edmtnyoi0SNVzrn7KBz9j91E4mIiMJAREQUBkfb/emuQBronLODztnnNGYgIiJqGYiIiMJARERQGIyYmT1oZnVmtjal7FQze83MVprZcjM7K2XbBV75OjN7KaV8sZltNLMqM7vjaJ/H4TicczazYjP7rZmt8s75UynHLDGzTd7PknScy3AMcb6nmNmfzGyNd35FKdvu9N7HjWZ2WUq539/jQc/ZzC4xsxVe+QozuzDlmDO88iozu88y+Cbgh/s+e9tnmVmHmX05pcw37/M+ErdR1M+R/gDvB04H1qaUPQtc7j2+AljmPS4hcV/nWd7fU7zfQWAzcAwQAVYBC9J9bqN0zl8B7vYelwFN3jlOBLZ4v0u9x6XpPrfDON83gfO9xzcAd3mPF3jvXw4w13tfg+PkPR7qnE8DZniPFwI7U455AziHxJ1Un07+P5KJP4dzzinbHwV+DXzZ+9tX73Pqj1oGI+Sce5nEB9w+xUDyG0QxsMt7fC3wmHNuh3dsnVd+FlDlnNvinOsDHgauHNOKj8BhnrMDJnjfCAu946LAZcBzzrkm51wz8ByweKzrfiSGON/5wMve4+eAv/AeXwk87Jzrdc5tBapIvL/j4T0e9Jydc28755Lv9zogz8xyzGw6UOSce80lPil/Blw15pU/Qof5PmNmVwFbSZxzkq/e51QKg7FxO/AdM6sGvgvc6ZXPB0rNbJnXnL7eK58JVKccX+OV+cntDH7O3wdOIBEOa4DbnHNx/H/O69j7j/xj7L1t61Dn5ffzhaHPOdVfAG8553pJnF9NyrZxc85mVgj8LfAP++3v2/dZYTA2Pgf8tXOuAvhr4AGvPAScAXyQxDfjvzez+emp4qgb6pwvA1YCM4BTge/v3+/qUzcAN5vZCmAC0Jfm+hwNBz1nMzsRuBv4P2mo21gZ6py/AdzjnOtIV8VGW8bc6WycWQLc5j3+NfBj73EN0Oic6wQ6zexl4BSvPPVbVjmw8yjVdbQMdc6fAr7tdRNUmdlW4HgS53dByvHlwLKjUtNR4JzbAFwK4AX6B71NOxn6vfT1e3yQc8bMyoHfANc75zZ7xTtJnGfSeDrns4GPmtm/kBgLjJtZD7ACn77PahmMjV3A+d7jC4FN3uPHgfPMLGRm+ST+h3qHxCBVpZnNNbMIcDXwxFGu80gNdc47gIsAzGwqcByJweJngEvNrNTMSkn8g3vmqNZ4BMxsivc7APwd8O/epieAq70+87lAJYlBVN+/x0Ods5mVAE8Bdzjn/pDc3zm3G2gzs3O8MaPrSfwb8I2hztk59z7n3Bzn3Bzg34B/cs59Hz+/z+kewfb7D/BLYDfQT+Ib/o3AeSS+IawCXgfOSNn//5KYUbQWuD2l/ArgXRIzEb6a7vMarXMm0T30LInxgrXAX6U8zw0kBlirgE+l+7wO83xv896vd4Fv463m9/b/qvc+biRl9sw4eI8HPWcSH5KdJLoDkz/JmXKLvPd9M4nxI0vXOY32+5xy3DfwZhP57X1O/dHlKERERN1EIiKiMBARERQGIiKCwkBERFAYiIgICgMREUFhICIiwP8HFdQicEPCWwoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sampler.population['photos_count'].plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should be careful when looking at a chart like this of not mistaking this as a full representation. In this case we need to know some context about collections like the Chronicling America corpus from which the Newspaper Navigator data was draw since these collections are neither 'neutral' nor entirely representative. For example the sharp drop ~1920 in the number of photographs for each year could be mistaken as a sudden drop in the number of newspapers publishing photos. Evan a cursory knowledge of photojournalism would challenge this assumption, with the 1930s-1950s considered a [golden age](https://en.wikipedia.org/wiki/Photojournalism#Golden_age) of photojournalism. The drop is in this case more easily explained by the context in which this collection was made and specifically in this case the influence of copyright on digitization policies. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a sample we define a few variables for a sample. `sample_size` is the size we want for our sample. If `year_sample` is true then this will be the sample size for each year. In this case since we want to create a sample size we want to annotate we set `year_sample` to `False`. `kind` refers to the predicted label from Newspaper Navigator. Because photography in newspapers appeared later in the 19th Century, we choose a later `start_year` and stop at 1920 when the number of images in the data drops off massively. `step` can be used to change the step between sample years. In this case we sample every year but we could change this to ever `n` years. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_photos_sample = sampler.create_sample(sample_size=2000, \n",
    "                                         kind='photos',\n",
    "                                         start_year=1895, \n",
    "                                         end_year=1920, \n",
    "                                         step=1, \n",
    "                                         year_sample=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get back pandas DataFrame containing the newspaper navigator dataset. This includes metadata about the newspaper form which the image came from, the `box` which contains the predicted bounding box coordinates for the predicted visual content from newspaper navigator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>filepath</th>\n",
       "      <th>pub_date</th>\n",
       "      <th>page_seq_num</th>\n",
       "      <th>edition_seq_num</th>\n",
       "      <th>batch</th>\n",
       "      <th>lccn</th>\n",
       "      <th>box</th>\n",
       "      <th>score</th>\n",
       "      <th>ocr</th>\n",
       "      <th>place_of_publication</th>\n",
       "      <th>geographic_coverage</th>\n",
       "      <th>name</th>\n",
       "      <th>publisher</th>\n",
       "      <th>url</th>\n",
       "      <th>page_url</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mthi_kestrel_ver02/data/sn85053323/print/18951...</td>\n",
       "      <td>1895-11-28</td>\n",
       "      <td>696</td>\n",
       "      <td>1</td>\n",
       "      <td>mthi_kestrel_ver02</td>\n",
       "      <td>sn85053323</td>\n",
       "      <td>[0.6757705539920897, 0.13269955294949193, 0.88...</td>\n",
       "      <td>0.989924</td>\n",
       "      <td>[AVM., SCHEMA, V,]</td>\n",
       "      <td>Neihart, Mont.</td>\n",
       "      <td>[Montana--Cascade--Neihart]</td>\n",
       "      <td>The Neihart herald. [volume]</td>\n",
       "      <td>Herald Pub. Co.</td>\n",
       "      <td>https://news-navigator.labs.loc.gov/data/mthi_...</td>\n",
       "      <td>https://chroniclingamerica.loc.gov/data/batche...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            filepath    pub_date  \\\n",
       "0  mthi_kestrel_ver02/data/sn85053323/print/18951...  1895-11-28   \n",
       "\n",
       "   page_seq_num edition_seq_num               batch        lccn  \\\n",
       "0           696               1  mthi_kestrel_ver02  sn85053323   \n",
       "\n",
       "                                                 box     score  \\\n",
       "0  [0.6757705539920897, 0.13269955294949193, 0.88...  0.989924   \n",
       "\n",
       "                  ocr place_of_publication          geographic_coverage  \\\n",
       "0  [AVM., SCHEMA, V,]       Neihart, Mont.  [Montana--Cascade--Neihart]   \n",
       "\n",
       "                           name        publisher  \\\n",
       "0  The Neihart herald. [volume]  Herald Pub. Co.   \n",
       "\n",
       "                                                 url  \\\n",
       "0  https://news-navigator.labs.loc.gov/data/mthi_...   \n",
       "\n",
       "                                            page_url  \n",
       "0  https://chroniclingamerica.loc.gov/data/batche...  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_photos_sample.head(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have our sample we want to annotate this sample with labels which we'll then use to train a computer vision model on. In this case we will annotate by creating `IIIF` urls which we then load into the [label studio](https://labelstud.io) annotation tool. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nnanno.annotate import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll create a folder to store our annotations "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Path('ph/photos_multi').mkdir()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use a small function to convert our sample DataFrame into a JSON file which we'll use to load our 'tasks' i.e. images into label studio and save this into our folder. The `size` variable is used to determine the size we request via IIIF. In this case we use a relatively small size to annotate since this will make the loading of the images a bit quicker. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_label_studio_json(sampler, 'ph/photos_multi/photo_tasks.json', size=(500,500))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have our sample in a format that label studio can take as input we'll setup the annotations. There are various ways we can do this, one way is via the GUI app. To setup a new label-studio annotation environment we'll use the `init` command and give a name for our project. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`label-studio init photo-annotations`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will launch label-studio. From here we can load our tasks by uploading our photo_tasks.json file. \n",
    "\n",
    "We then create our labels. The label studio documentation goes into detail on how to setup labels. For simple tasks it's often easiest to use one of the templates and then adjust the labels as required. For this example dataset the following is used:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```xml\n",
    "<View>\n",
    "  <Choices name=\"choice\" toName=\"image\" showInLine=\"true\" choice=\"multiple\">\n",
    "        <Choice value=\"human\"/>\n",
    "    \t<Choice value=\"animal\"/>\n",
    "    \t<Choice value=\"human-structure\"/>\n",
    "     \t<Choice value=\"landscape\"/>\n",
    "  </Choices>\n",
    "<Image name=\"image\" value=\"$image\"/>\n",
    "</View>\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have completed the annotations we can export them to a CSV file using the export functionality within label studio. This will contain our original sample plus our new data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = load_annotations_csv('ph/photos_multi/results.csv', kind='label')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we'll want the images locally for training models we'll now download the images associated with our labels. We can do this using the `download_sample` method from `nnsampler` and passing our DataFrame. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb41ed3ef23f447aade46c084bce5d50",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(HTML(value=''), FloatProgress(value=0.0, max=2000.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "sampler.download_sample('ph/photos_multi/images', df=df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This download method will try and download all of the images associated with the annotations DataFrame but there may be some that are not available. We can quickly compare the downloaded image folder and DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of DataFrame 2000 and number of images in ph/photos_multi/images 2000 match 😀\n"
     ]
    }
   ],
   "source": [
    "check_download_df_match('ph/photos_multi/images', df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll save the annotations to a new CSV file that contains the processed 'labels' column we'll use when training our model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('ph/photos_multi/annotations.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll also create a version with just the labels and images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_json('ph/photos_multi/images/2021_01_02_16_38_2000_sample.json')[['download_image_path','label']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('ph/photos_multi/multi_label.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload to Zenodo\n",
    "\n",
    "*This section is obviously optional. This is shown here only since this was the process used for this data. Most of these instructions are taken directly from the Zenodo docs.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we zip our image directory "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!zip -j -r -q ph/photos_multi/images.zip ph/photos_multi/images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We import a few more libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "import os\n",
    "import requests \n",
    "from pathlib import Path\n",
    "from tqdm.notebook import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`dotenv` is used to load the credenitals for the zenodo api. These credentials shouldn't be shared so we don't want them included in a notebook being shared publicly. There are different ways of managing these kinds of credentials with one simple way is to put them in a `.env` file. This can be access by dotenv but won't be versioned by Git so avoids sharing our credentials with the wider world. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ACCESS_TOKEN = os.getenv(\"ACCESS_TOKEN\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Zenodo repository allows for uploads via an API. In the below cells I have just followed the steps outlined in the zenodo [documentation](https://developers.zenodo.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "r = requests.get('https://zenodo.org/api/deposit/depositions',params={'access_token': ACCESS_TOKEN})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "201"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "headers = {\"Content-Type\": \"application/json\"}\n",
    "params={'access_token': ACCESS_TOKEN}\n",
    "r = requests.post('https://zenodo.org/api/deposit/depositions',\n",
    "                   params=params,\n",
    "                   json={},\n",
    "                   headers=headers)\n",
    "r.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'conceptrecid': '4487140',\n",
       " 'created': '2021-02-01T17:03:26.087698+00:00',\n",
       " 'files': [],\n",
       " 'id': 4487141,\n",
       " 'links': {'bucket': 'https://zenodo.org/api/files/d35b386f-98eb-404a-ba1e-c119f9f23b40',\n",
       "  'discard': 'https://zenodo.org/api/deposit/depositions/4487141/actions/discard',\n",
       "  'edit': 'https://zenodo.org/api/deposit/depositions/4487141/actions/edit',\n",
       "  'files': 'https://zenodo.org/api/deposit/depositions/4487141/files',\n",
       "  'html': 'https://zenodo.org/deposit/4487141',\n",
       "  'latest_draft': 'https://zenodo.org/api/deposit/depositions/4487141',\n",
       "  'latest_draft_html': 'https://zenodo.org/deposit/4487141',\n",
       "  'publish': 'https://zenodo.org/api/deposit/depositions/4487141/actions/publish',\n",
       "  'self': 'https://zenodo.org/api/deposit/depositions/4487141'},\n",
       " 'metadata': {'prereserve_doi': {'doi': '10.5281/zenodo.4487141',\n",
       "   'recid': 4487141}},\n",
       " 'modified': '2021-02-01T17:03:26.087708+00:00',\n",
       " 'owner': 32477,\n",
       " 'record_id': 4487141,\n",
       " 'state': 'unsubmitted',\n",
       " 'submitted': False,\n",
       " 'title': ''}"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket_url = r.json()[\"links\"][\"bucket\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('ph/photos_multi/images.zip'),\n",
       " PosixPath('ph/photos_multi/multi_label.csv'),\n",
       " PosixPath('ph/photos_multi/annotations.csv'),\n",
       " PosixPath('ph/photos_multi/results.csv'),\n",
       " PosixPath('ph/photos_multi/photo_tasks.json')]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "files = [f for f in Path('ph/photos_multi').iterdir() if f.is_file() and not f.name.startswith('.')]\n",
    "files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cf33e67534eb4baa99b62edc02be2968",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(HTML(value=''), FloatProgress(value=0.0, max=5.0), HTML(value='')))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mimetype': 'application/json',\n",
       " 'updated': '2021-02-01T17:12:24.999647+00:00',\n",
       " 'links': {'self': 'https://zenodo.org/api/files/d35b386f-98eb-404a-ba1e-c119f9f23b40/ph/photos_multi/photo_tasks.json',\n",
       "  'version': 'https://zenodo.org/api/files/d35b386f-98eb-404a-ba1e-c119f9f23b40/ph/photos_multi/photo_tasks.json?versionId=e4eb4982-50f4-490f-99b1-d1060de83805',\n",
       "  'uploads': 'https://zenodo.org/api/files/d35b386f-98eb-404a-ba1e-c119f9f23b40/ph/photos_multi/photo_tasks.json?uploads'},\n",
       " 'is_head': True,\n",
       " 'created': '2021-02-01T17:12:24.994665+00:00',\n",
       " 'checksum': 'md5:d79ed84baafb5c8f089ba37614be9335',\n",
       " 'version_id': 'e4eb4982-50f4-490f-99b1-d1060de83805',\n",
       " 'delete_marker': False,\n",
       " 'key': 'ph/photos_multi/photo_tasks.json',\n",
       " 'size': 2646679}"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for file in tqdm(files):\n",
    "    with open(file,\"rb\") as fp:\n",
    "        r = requests.put(\n",
    "            \"%s/%s\" % (bucket_url, file),\n",
    "            data=fp,\n",
    "            params=params)\n",
    "r.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Metadata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once this has been done we should make sure we create an additional metadata related to the upload within Zenodo. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datasheets for Datasets\n",
    "\n",
    "Beyond making sure that we complete the 'standard' metadata field, when creating a dataset intended for training a deep learning model you may want to consider giving a broader context to the data, how it was gathered and some of the potential limitations of with the data. These limitations may become apparent to you but may be less obvious to new users of the data particularly since these users may be divorced from the immediate context under which this data was gathered. \n",
    "\n",
    "One approach to documenting this kind of information is outlined in [Datasheets for Datasets](https://arxiv.org/abs/1803.09010) by Gebru *et al*. The paper abstract gives some background to the motivations and suggested solutions:\n",
    "\n",
    "> The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains. To address this gap, we propose datasheets for datasets. In the electronics industry, every component, no matter how simple or complex, is accompanied with a datasheet that describes its operating characteristics, test results, recommended uses, and other information. By analogy, we propose that every dataset be accompanied with a datasheet that documents its motivation, composition, collection process, recommended uses, and so on. Datasheets for datasets will facilitate better communication between dataset creators and dataset consumers, and encourage the machine learning community to prioritize transparency and accountability. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the record for this data on Zenodo ([10.5281/zenodo.4487141]()) which attempts to put (some) of this paper into practice. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### fin"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}