notebooks/04-onnx-export.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jBasof3bv1LB"
   },
   "source": [
    "<h1><center>How to export 🤗 Transformers Models to ONNX ?<h1><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[ONNX](http://onnx.ai/) is open format for machine learning models. It allows to save your neural network's computation graph in a framework agnostic way, which might be particulary helpful when deploying deep learning models.\n",
    "\n",
    "Indeed, businesses might have other requirements _(languages, hardware, ...)_ for which the training framework might not be the best suited in inference scenarios. In that context, having a representation of the actual computation graph that can be shared accross various business units and logics across an organization might be a desirable component.\n",
    "\n",
    "Along with the serialization format, ONNX also provides a runtime library which allows efficient and hardware specific execution of the ONNX graph. This is done through the [onnxruntime](https://microsoft.github.io/onnxruntime/) project and already includes collaborations with many hardware vendors to seamlessly deploy models on various platforms.\n",
    "\n",
    "Through this notebook we'll walk you through the process to convert a PyTorch or TensorFlow transformers model to the [ONNX](http://onnx.ai/) and leverage [onnxruntime](https://microsoft.github.io/onnxruntime/) to run inference tasks on models from  🤗 __transformers__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yNnbrSg-5e1s"
   },
   "source": [
    "## Exporting 🤗 transformers model to ONNX\n",
    "\n",
    "---\n",
    "\n",
    "Exporting models _(either PyTorch or TensorFlow)_ is easily achieved through the conversion tool provided as part of 🤗 __transformers__ repository. \n",
    "\n",
    "Under the hood the process is sensibly the following: \n",
    "\n",
    "1. Allocate the model from transformers (**PyTorch or TensorFlow**)\n",
    "2. Forward dummy inputs through the model this way **ONNX** can record the set of operations executed\n",
    "3. Optionally define dynamic axes on input and output tensors\n",
    "4. Save the graph along with the network parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting git+https://github.com/huggingface/transformers\n",
      "  Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-9rvbp9p8\n",
      "  Running command git clone -q https://github.com/huggingface/transformers /tmp/pip-req-build-9rvbp9p8\n",
      "Requirement already satisfied, skipping upgrade: numpy in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (1.18.1)\n",
      "Requirement already satisfied, skipping upgrade: tokenizers==0.8.1.rc2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (0.8.1rc2)\n",
      "Requirement already satisfied, skipping upgrade: packaging in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (20.4)\n",
      "Requirement already satisfied, skipping upgrade: filelock in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (3.0.12)\n",
      "Requirement already satisfied, skipping upgrade: requests in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (2.23.0)\n",
      "Requirement already satisfied, skipping upgrade: tqdm>=4.27 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (4.46.1)\n",
      "Requirement already satisfied, skipping upgrade: regex!=2019.12.17 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (2020.6.8)\n",
      "Requirement already satisfied, skipping upgrade: sentencepiece!=0.1.92 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (0.1.91)\n",
      "Requirement already satisfied, skipping upgrade: sacremoses in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers==3.0.2) (0.0.43)\n",
      "Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from packaging->transformers==3.0.2) (2.4.7)\n",
      "Requirement already satisfied, skipping upgrade: six in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from packaging->transformers==3.0.2) (1.15.0)\n",
      "Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers==3.0.2) (3.0.4)\n",
      "Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers==3.0.2) (2.9)\n",
      "Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers==3.0.2) (1.25.9)\n",
      "Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers==3.0.2) (2020.6.20)\n",
      "Requirement already satisfied, skipping upgrade: click in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from sacremoses->transformers==3.0.2) (7.1.2)\n",
      "Requirement already satisfied, skipping upgrade: joblib in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from sacremoses->transformers==3.0.2) (0.15.1)\n",
      "Building wheels for collected packages: transformers\n",
      "  Building wheel for transformers (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for transformers: filename=transformers-3.0.2-py3-none-any.whl size=883063 sha256=5f2caef76450921ae2e5b10abbbaab436e9c87c83486114fa08d305e4396d4cd\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-kftypcjz/wheels/42/68/45/c63edff61c292f2dfd4df4ef6522dcbecc603e7af82813c1d7\n",
      "Successfully built transformers\n",
      "Installing collected packages: transformers\n",
      "  Attempting uninstall: transformers\n",
      "    Found existing installation: transformers 3.0.2\n",
      "    Uninstalling transformers-3.0.2:\n",
      "      Successfully uninstalled transformers-3.0.2\n",
      "Successfully installed transformers-3.0.2\n",
      "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
      "Requirement already up-to-date: torch==1.6.0+cpu in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.6.0+cpu)\n",
      "Requirement already up-to-date: torchvision==0.7.0+cpu in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (0.7.0+cpu)\n",
      "Requirement already satisfied, skipping upgrade: numpy in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from torch==1.6.0+cpu) (1.18.1)\n",
      "Requirement already satisfied, skipping upgrade: future in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from torch==1.6.0+cpu) (0.18.2)\n",
      "Requirement already satisfied, skipping upgrade: pillow>=4.1.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from torchvision==0.7.0+cpu) (7.2.0)\n",
      "Requirement already up-to-date: onnxruntime==1.4.0 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.4.0)\n",
      "Requirement already satisfied, skipping upgrade: protobuf in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime==1.4.0) (3.12.2)\n",
      "Requirement already satisfied, skipping upgrade: numpy>=1.16.6 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime==1.4.0) (1.18.1)\n",
      "Requirement already satisfied, skipping upgrade: setuptools in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->onnxruntime==1.4.0) (47.1.1.post20200604)\n",
      "Requirement already satisfied, skipping upgrade: six>=1.9 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->onnxruntime==1.4.0) (1.15.0)\n",
      "Looking in indexes: https://test.pypi.org/simple/\n",
      "Requirement already satisfied: ort-nightly in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.4.0.dev202008262)\n",
      "Requirement already satisfied: protobuf in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from ort-nightly) (3.12.2)\n",
      "Requirement already satisfied: numpy>=1.16.6 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from ort-nightly) (1.18.1)\n",
      "Requirement already satisfied: setuptools in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->ort-nightly) (47.1.1.post20200604)\n",
      "Requirement already satisfied: six>=1.9 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->ort-nightly) (1.15.0)\n",
      "Requirement already up-to-date: onnxruntime-tools in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.4.2)\n",
      "Requirement already satisfied, skipping upgrade: numpy in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (1.18.1)\n",
      "Requirement already satisfied, skipping upgrade: coloredlogs in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (14.0)\n",
      "Requirement already satisfied, skipping upgrade: py3nvml in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (0.2.6)\n",
      "Requirement already satisfied, skipping upgrade: psutil in /home/mfuntowicz/.local/lib/python3.8/site-packages/psutil-5.7.0-py3.8-linux-x86_64.egg (from onnxruntime-tools) (5.7.0)\n",
      "Requirement already satisfied, skipping upgrade: packaging in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (20.4)\n",
      "Requirement already satisfied, skipping upgrade: py-cpuinfo in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (5.0.0)\n",
      "Requirement already satisfied, skipping upgrade: onnx in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-tools) (1.7.0)\n",
      "Requirement already satisfied, skipping upgrade: humanfriendly>=7.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from coloredlogs->onnxruntime-tools) (8.2)\n",
      "Requirement already satisfied, skipping upgrade: xmltodict in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from py3nvml->onnxruntime-tools) (0.12.0)\n",
      "Requirement already satisfied, skipping upgrade: six in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from packaging->onnxruntime-tools) (1.15.0)\n",
      "Requirement already satisfied, skipping upgrade: pyparsing>=2.0.2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from packaging->onnxruntime-tools) (2.4.7)\n",
      "Requirement already satisfied, skipping upgrade: typing-extensions>=3.6.2.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnx->onnxruntime-tools) (3.7.4.2)\n",
      "Requirement already satisfied, skipping upgrade: protobuf in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnx->onnxruntime-tools) (3.12.2)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied, skipping upgrade: setuptools in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->onnx->onnxruntime-tools) (47.1.1.post20200604)\r\n"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install --upgrade git+https://github.com/huggingface/transformers\n",
    "!{sys.executable} -m pip install --upgrade torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html\n",
    "!{sys.executable} -m pip install --upgrade onnxruntime==1.4.0\n",
    "!{sys.executable} -m pip install -i https://test.pypi.org/simple/ ort-nightly\n",
    "!{sys.executable} -m pip install --upgrade onnxruntime-tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PwAaOchY4N2-"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /home/mfuntowicz/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391\n",
      "Model config BertConfig {\n",
      "  \"architectures\": [\n",
      "    \"BertForMaskedLM\"\n",
      "  ],\n",
      "  \"attention_probs_dropout_prob\": 0.1,\n",
      "  \"gradient_checkpointing\": false,\n",
      "  \"hidden_act\": \"gelu\",\n",
      "  \"hidden_dropout_prob\": 0.1,\n",
      "  \"hidden_size\": 768,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 3072,\n",
      "  \"layer_norm_eps\": 1e-12,\n",
      "  \"max_position_embeddings\": 512,\n",
      "  \"model_type\": \"bert\",\n",
      "  \"num_attention_heads\": 12,\n",
      "  \"num_hidden_layers\": 12,\n",
      "  \"pad_token_id\": 0,\n",
      "  \"type_vocab_size\": 2,\n",
      "  \"vocab_size\": 28996\n",
      "}\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ONNX opset version set to: 11\n",
      "Loading pipeline (model: bert-base-cased, tokenizer: bert-base-cased)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt from cache at /home/mfuntowicz/.cache/torch/transformers/5e8a2b4893d13790ed4150ca1906be5f7a03d6c4ddf62296c383f6db42814db2.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1\n",
      "loading model card file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-modelcard.json from cache at /home/mfuntowicz/.cache/torch/transformers/72b46f187c40a666d54782e06684c2870e109350a3efe9aa5027253dec2e671d.455d944f3d1572ab55ed579849f751cf37f303e3388980a42d94f7cd57a4e331\n",
      "Model card: {\n",
      "  \"caveats_and_recommendations\": {},\n",
      "  \"ethical_considerations\": {},\n",
      "  \"evaluation_data\": {},\n",
      "  \"factors\": {},\n",
      "  \"intended_use\": {},\n",
      "  \"metrics\": {},\n",
      "  \"model_details\": {},\n",
      "  \"quantitative_analyses\": {},\n",
      "  \"training_data\": {}\n",
      "}\n",
      "\n",
      "loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /home/mfuntowicz/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391\n",
      "Model config BertConfig {\n",
      "  \"architectures\": [\n",
      "    \"BertForMaskedLM\"\n",
      "  ],\n",
      "  \"attention_probs_dropout_prob\": 0.1,\n",
      "  \"gradient_checkpointing\": false,\n",
      "  \"hidden_act\": \"gelu\",\n",
      "  \"hidden_dropout_prob\": 0.1,\n",
      "  \"hidden_size\": 768,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 3072,\n",
      "  \"layer_norm_eps\": 1e-12,\n",
      "  \"max_position_embeddings\": 512,\n",
      "  \"model_type\": \"bert\",\n",
      "  \"num_attention_heads\": 12,\n",
      "  \"num_hidden_layers\": 12,\n",
      "  \"pad_token_id\": 0,\n",
      "  \"type_vocab_size\": 2,\n",
      "  \"vocab_size\": 28996\n",
      "}\n",
      "\n",
      "loading weights file https://cdn.huggingface.co/bert-base-cased-pytorch_model.bin from cache at /home/mfuntowicz/.cache/torch/transformers/d8f11f061e407be64c4d5d7867ee61d1465263e24085cfa26abf183fdc830569.3fadbea36527ae472139fe84cddaa65454d7429f12d543d80bfc3ad70de55ac2\n",
      "All model checkpoint weights were used when initializing BertModel.\n",
      "\n",
      "All the weights of BertModel were initialized from the model checkpoint at bert-base-cased.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use BertModel for predictions without further training.\n",
      "/home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/modeling_bert.py:201: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!\n",
      "  position_ids = self.position_ids[:, :seq_length]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating folder onnx\n",
      "Using framework PyTorch: 1.6.0\n",
      "Found input input_ids with shape: {0: 'batch', 1: 'sequence'}\n",
      "Found input token_type_ids with shape: {0: 'batch', 1: 'sequence'}\n",
      "Found input attention_mask with shape: {0: 'batch', 1: 'sequence'}\n",
      "Found output output_0 with shape: {0: 'batch', 1: 'sequence'}\n",
      "Found output output_1 with shape: {0: 'batch'}\n",
      "Ensuring inputs are in correct order\n",
      "position_ids is not present in the generated input list.\n",
      "Generated inputs order: ['input_ids', 'attention_mask', 'token_type_ids']\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages/transformers/modeling_utils.py:1570: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!\n",
      "  input_tensor.shape == tensor_shape for input_tensor in input_tensors\n"
     ]
    }
   ],
   "source": [
    "!rm -rf onnx/\n",
    "from pathlib import Path\n",
    "from transformers.convert_graph_to_onnx import convert\n",
    "\n",
    "# Handles all the above steps for you\n",
    "convert(framework=\"pt\", model=\"bert-base-cased\", output=Path(\"onnx/bert-base-cased.onnx\"), opset=11)\n",
    "\n",
    "# Tensorflow \n",
    "# convert(framework=\"tf\", model=\"bert-base-cased\", output=\"onnx/bert-base-cased.onnx\", opset=11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to leverage runtime for inference over an ONNX graph\n",
    "\n",
    "---\n",
    "\n",
    "As mentionned in the introduction, **ONNX** is a serialization format and many side projects can load the saved graph and run the actual computations from it. Here, we'll focus on the official [onnxruntime](https://microsoft.github.io/onnxruntime/). The runtime is implemented in C++ for performance reasons and provides API/Bindings for C++, C, C#, Java and Python.\n",
    "\n",
    "In the case of this notebook, we will use the Python API to highlight how to load a serialized **ONNX** graph and run inference workload on various backends through **onnxruntime**.\n",
    "\n",
    "**onnxruntime** is available on pypi:\n",
    "\n",
    "- onnxruntime: ONNX + MLAS (Microsoft Linear Algebra Subprograms)\n",
    "- onnxruntime-gpu: ONNX + MLAS + CUDA\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (3.0.2)\n",
      "Requirement already satisfied: onnxruntime-gpu in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.3.0)\n",
      "Requirement already satisfied: onnx in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (1.7.0)\n",
      "Requirement already satisfied: psutil in /home/mfuntowicz/.local/lib/python3.8/site-packages/psutil-5.7.0-py3.8-linux-x86_64.egg (5.7.0)\n",
      "Requirement already satisfied: matplotlib in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (3.3.1)\n",
      "Requirement already satisfied: tqdm>=4.27 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (4.46.1)\n",
      "Requirement already satisfied: numpy in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (1.18.1)\n",
      "Requirement already satisfied: sacremoses in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (0.0.43)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (2020.6.8)\n",
      "Requirement already satisfied: filelock in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (3.0.12)\n",
      "Requirement already satisfied: sentencepiece!=0.1.92 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (0.1.91)\n",
      "Requirement already satisfied: requests in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (2.23.0)\n",
      "Requirement already satisfied: packaging in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (20.4)\n",
      "Requirement already satisfied: tokenizers==0.8.1.rc2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from transformers) (0.8.1rc2)\n",
      "Requirement already satisfied: protobuf in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnxruntime-gpu) (3.12.2)\n",
      "Requirement already satisfied: six in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnx) (1.15.0)\n",
      "Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from onnx) (3.7.4.2)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (2.4.7)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (1.2.0)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (2.8.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (0.10.0)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (7.2.0)\n",
      "Requirement already satisfied: certifi>=2020.06.20 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from matplotlib) (2020.6.20)\n",
      "Requirement already satisfied: click in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from sacremoses->transformers) (7.1.2)\n",
      "Requirement already satisfied: joblib in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from sacremoses->transformers) (0.15.1)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers) (1.25.9)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers) (3.0.4)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from requests->transformers) (2.9)\n",
      "Requirement already satisfied: setuptools in /home/mfuntowicz/miniconda3/envs/pytorch/lib/python3.8/site-packages (from protobuf->onnxruntime-gpu) (47.1.1.post20200604)\n"
     ]
    }
   ],
   "source": [
    "!pip install transformers onnxruntime-gpu onnx psutil matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-gP08tHfBvgY"
   },
   "source": [
    "## Preparing for an Inference Session\n",
    "\n",
    "---\n",
    "\n",
    "Inference is done using a specific backend definition which turns on hardware specific optimizations of the graph. \n",
    "\n",
    "Optimizations are basically of three kinds: \n",
    "\n",
    "- **Constant Folding**: Convert static variables to constants in the graph \n",
    "- **Deadcode Elimination**: Remove nodes never accessed in the graph\n",
    "- **Operator Fusing**: Merge multiple instruction into one (Linear -> ReLU can be fused to be LinearReLU)\n",
    "\n",
    "ONNX Runtime automatically applies most optimizations by setting specific `SessionOptions`.\n",
    "\n",
    "Note:Some of the latest optimizations that are not yet integrated into ONNX Runtime are available in [optimization script](https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/transformers) that tunes models for the best performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# # An optional step unless\n",
    "# # you want to get a model with mixed precision for perf accelartion on newer GPU\n",
    "# # or you are working with Tensorflow(tf.keras) models or pytorch models other than bert\n",
    "\n",
    "# !pip install onnxruntime-tools\n",
    "# from onnxruntime_tools import optimizer\n",
    "\n",
    "# # Mixed precision conversion for bert-base-cased model converted from Pytorch\n",
    "# optimized_model = optimizer.optimize_model(\"bert-base-cased.onnx\", model_type='bert', num_heads=12, hidden_size=768)\n",
    "# optimized_model.convert_model_float32_to_float16()\n",
    "# optimized_model.save_model_to_file(\"bert-base-cased.onnx\")\n",
    "\n",
    "# # optimizations for bert-base-cased model converted from Tensorflow(tf.keras)\n",
    "# optimized_model = optimizer.optimize_model(\"bert-base-cased.onnx\", model_type='bert_keras', num_heads=12, hidden_size=768)\n",
    "# optimized_model.save_model_to_file(\"bert-base-cased.onnx\")\n",
    "\n",
    "\n",
    "# optimize transformer-based models with onnxruntime-tools\n",
    "from onnxruntime_tools import optimizer\n",
    "from onnxruntime_tools.transformers.onnx_model_bert import BertOptimizationOptions\n",
    "\n",
    "# disable embedding layer norm optimization for better model size reduction\n",
    "opt_options = BertOptimizationOptions('bert')\n",
    "opt_options.enable_embed_layer_norm = False\n",
    "\n",
    "opt_model = optimizer.optimize_model(\n",
    "    'onnx/bert-base-cased.onnx',\n",
    "    'bert', \n",
    "    num_heads=12,\n",
    "    hidden_size=768,\n",
    "    optimization_options=opt_options)\n",
    "opt_model.save_model_to_file('bert.opt.onnx')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from os import environ\n",
    "from psutil import cpu_count\n",
    "\n",
    "# Constants from the performance optimization available in onnxruntime\n",
    "# It needs to be done before importing onnxruntime\n",
    "environ[\"OMP_NUM_THREADS\"] = str(cpu_count(logical=True))\n",
    "environ[\"OMP_WAIT_POLICY\"] = 'ACTIVE'\n",
    "\n",
    "from onnxruntime import GraphOptimizationLevel, InferenceSession, SessionOptions, get_all_providers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2k-jHLfdcTFS"
   },
   "outputs": [],
   "source": [
    "from contextlib import contextmanager\n",
    "from dataclasses import dataclass\n",
    "from time import time\n",
    "from tqdm import trange\n",
    "\n",
    "def create_model_for_provider(model_path: str, provider: str) -> InferenceSession: \n",
    "  \n",
    "  assert provider in get_all_providers(), f\"provider {provider} not found, {get_all_providers()}\"\n",
    "\n",
    "  # Few properties that might have an impact on performances (provided by MS)\n",
    "  options = SessionOptions()\n",
    "  options.intra_op_num_threads = 1\n",
    "  options.graph_optimization_level = GraphOptimizationLevel.ORT_ENABLE_ALL\n",
    "\n",
    "  # Load the model as a graph and prepare the CPU backend \n",
    "  session = InferenceSession(model_path, options, providers=[provider])\n",
    "  session.disable_fallback()\n",
    "    \n",
    "  return session\n",
    "\n",
    "\n",
    "@contextmanager\n",
    "def track_infer_time(buffer: [int]):\n",
    "    start = time()\n",
    "    yield\n",
    "    end = time()\n",
    "\n",
    "    buffer.append(end - start)\n",
    "\n",
    "\n",
    "@dataclass\n",
    "class OnnxInferenceResult:\n",
    "  model_inference_time: [int]  \n",
    "  optimized_model_path: str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "teJdG3amE-hR"
   },
   "source": [
    "## Forwarding through our optimized ONNX model running on CPU\n",
    "\n",
    "---\n",
    "\n",
    "When the model is loaded for inference over a specific provider, for instance **CPUExecutionProvider** as above, an optimized graph can be saved. This graph will might include various optimizations, and you might be able to see some **higher-level** operations in the graph _(through [Netron](https://github.com/lutzroeder/Netron) for instance)_ such as:\n",
    "- **EmbedLayerNormalization**\n",
    "- **Attention**\n",
    "- **FastGeLU**\n",
    "\n",
    "These operations are an example of the kind of optimization **onnxruntime** is doing, for instance here gathering multiple operations into bigger one _(Operator Fusing)_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "dmC22kJfVGYe",
    "outputId": "f3aba5dc-15c0-4f82-b38c-1bbae1bf112e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt from cache at /home/mfuntowicz/.cache/torch/transformers/5e8a2b4893d13790ed4150ca1906be5f7a03d6c4ddf62296c383f6db42814db2.e13dbb970cb325137104fb2e5f36fe865f27746c6b526f6352861b1980eb80b1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequence output: (1, 6, 768), Pooled output: (1, 768)\n"
     ]
    }
   ],
   "source": [
    "from transformers import BertTokenizerFast\n",
    "\n",
    "tokenizer = BertTokenizerFast.from_pretrained(\"bert-base-cased\")\n",
    "cpu_model = create_model_for_provider(\"onnx/bert-base-cased.onnx\", \"CPUExecutionProvider\")\n",
    "\n",
    "# Inputs are provided through numpy array\n",
    "model_inputs = tokenizer(\"My name is Bert\", return_tensors=\"pt\")\n",
    "inputs_onnx = {k: v.cpu().detach().numpy() for k, v in model_inputs.items()}\n",
    "\n",
    "# Run the model (None = get all the outputs)\n",
    "sequence, pooled = cpu_model.run(None, inputs_onnx)\n",
    "\n",
    "# Print information about outputs\n",
    "\n",
    "print(f\"Sequence output: {sequence.shape}, Pooled output: {pooled.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmarking PyTorch model\n",
    "\n",
    "_Note: PyTorch model benchmark is run on CPU_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "PS_49goe197g",
    "outputId": "0ef0f70c-f5a7-46a0-949a-1a93f231d193"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json from cache at /home/mfuntowicz/.cache/torch/transformers/b945b69218e98b3e2c95acf911789741307dec43c698d35fad11c1ae28bda352.9da767be51e1327499df13488672789394e2ca38b877837e52618a67d7002391\n",
      "Model config BertConfig {\n",
      "  \"architectures\": [\n",
      "    \"BertForMaskedLM\"\n",
      "  ],\n",
      "  \"attention_probs_dropout_prob\": 0.1,\n",
      "  \"gradient_checkpointing\": false,\n",
      "  \"hidden_act\": \"gelu\",\n",
      "  \"hidden_dropout_prob\": 0.1,\n",
      "  \"hidden_size\": 768,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 3072,\n",
      "  \"layer_norm_eps\": 1e-12,\n",
      "  \"max_position_embeddings\": 512,\n",
      "  \"model_type\": \"bert\",\n",
      "  \"num_attention_heads\": 12,\n",
      "  \"num_hidden_layers\": 12,\n",
      "  \"pad_token_id\": 0,\n",
      "  \"type_vocab_size\": 2,\n",
      "  \"vocab_size\": 28996\n",
      "}\n",
      "\n",
      "loading weights file https://cdn.huggingface.co/bert-base-cased-pytorch_model.bin from cache at /home/mfuntowicz/.cache/torch/transformers/d8f11f061e407be64c4d5d7867ee61d1465263e24085cfa26abf183fdc830569.3fadbea36527ae472139fe84cddaa65454d7429f12d543d80bfc3ad70de55ac2\n",
      "All model checkpoint weights were used when initializing BertModel.\n",
      "\n",
      "All the weights of BertModel were initialized from the model checkpoint at bert-base-cased.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use BertModel for predictions without further training.\n",
      "Warming up: 100%|██████████| 10/10 [00:00<00:00, 39.30it/s]\n",
      "Tracking inference time on PyTorch: 100%|██████████| 100/100 [00:02<00:00, 41.09it/s]\n"
     ]
    }
   ],
   "source": [
    "from transformers import BertModel\n",
    "\n",
    "PROVIDERS = {\n",
    "    (\"cpu\", \"PyTorch CPU\"),\n",
    "#  Uncomment this line to enable GPU benchmarking\n",
    "#    (\"cuda:0\", \"PyTorch GPU\")\n",
    "}\n",
    "\n",
    "results = {}\n",
    "\n",
    "for device, label in PROVIDERS:\n",
    "    \n",
    "    # Move inputs to the correct device\n",
    "    model_inputs_on_device = {\n",
    "        arg_name: tensor.to(device)\n",
    "        for arg_name, tensor in model_inputs.items()\n",
    "    }\n",
    "\n",
    "    # Add PyTorch to the providers\n",
    "    model_pt = BertModel.from_pretrained(\"bert-base-cased\").to(device)\n",
    "    for _ in trange(10, desc=\"Warming up\"):\n",
    "      model_pt(**model_inputs_on_device)\n",
    "\n",
    "    # Compute \n",
    "    time_buffer = []\n",
    "    for _ in trange(100, desc=f\"Tracking inference time on PyTorch\"):\n",
    "      with track_infer_time(time_buffer):\n",
    "        model_pt(**model_inputs_on_device)\n",
    "\n",
    "    # Store the result\n",
    "    results[label] = OnnxInferenceResult(\n",
    "        time_buffer, \n",
    "        None\n",
    "    ) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Kda1e7TkEqNR"
   },
   "source": [
    "## Benchmarking PyTorch & ONNX on CPU\n",
    "\n",
    "_**Disclamer: results may vary from the actual hardware used to run the model**_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 170
    },
    "colab_type": "code",
    "id": "WcdFZCvImVig",
    "outputId": "bfd779a1-0bc7-42db-8587-e52a485ec5e3"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Tracking inference time on CPUExecutionProvider: 100%|██████████| 100/100 [00:01<00:00, 63.62it/s]\n"
     ]
    }
   ],
   "source": [
    "PROVIDERS = {\n",
    "    (\"CPUExecutionProvider\", \"ONNX CPU\"),\n",
    "#  Uncomment this line to enable GPU benchmarking\n",
    "#     (\"CUDAExecutionProvider\", \"ONNX GPU\")\n",
    "}\n",
    "\n",
    "\n",
    "for provider, label in PROVIDERS:\n",
    "    # Create the model with the specified provider\n",
    "    model = create_model_for_provider(\"onnx/bert-base-cased.onnx\", provider)\n",
    "\n",
    "    # Keep track of the inference time\n",
    "    time_buffer = []\n",
    "\n",
    "    # Warm up the model\n",
    "    model.run(None, inputs_onnx)\n",
    "\n",
    "    # Compute \n",
    "    for _ in trange(100, desc=f\"Tracking inference time on {provider}\"):\n",
    "      with track_infer_time(time_buffer):\n",
    "          model.run(None, inputs_onnx)\n",
    "\n",
    "    # Store the result\n",
    "    results[label] = OnnxInferenceResult(\n",
    "      time_buffer,\n",
    "      model.get_session_options().optimized_model_filepath\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABRoAAAPeCAYAAABjjKazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAA9hAAAPYQGoP6dpAABezklEQVR4nOzdeZjd8/3//8fJNklkEyIJQmxFbPWh1dDGFiL2rYQuUVsX1Fof+WgtpVJUpVW7fkI1GruqfuxLKtaKWtuqqBBrbEkkYZA5vz/6zfyMLDJeM2aG2+26znXlvM/7nPfznFmucfdeKtVqtRoAAAAAgALtWnoAAAAAAKDtExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAYA2ZeDAgdlnn30+8fNPP/30rLzyymnfvn2++MUvNtlcnyV33XVXKpVK7rrrrpYepYFZs2ZlmWWWybhx4z7V7R5zzDHZaKONGvWcz/r32bzvkauuuqqlR2kyF198cSqVSqZMmfKx65b+HgKAzyqhEQBaoXPOOSeVSqXRcYNFu+WWW3L00Udnk002ydixY3PKKae09Egt6pxzzsnFF1/c0mMstl/96lfp3r17RowY8alu97DDDsujjz6a66+/frHW930GAHxedWjpAQCA+Y0bNy4DBw7Mgw8+mMmTJ2fVVVdt6ZFajaeeeirt2n2y/1d6xx13pF27dvntb3+bTp06NfFkbc8555yTpZdeer49s4YMGZJ33nmnVX1G77//fn71q1/l8MMPT/v27T/Vbffr1y877bRTfvGLX2THHXf82PV9n7VN3/rWtzJixIjU1NS09CgA0GbZoxEAWplnn3029957b375y1+mT58+n/phoklSV1eXd99991Pf7uKoqalJx44dP9Fzp02bli5dujRp/JkzZ06TvVZr0a5du3Tu3PkTB93mcMMNN+S1117LHnvs0SLb32OPPTJx4sT8+9///th1m/r7rFqt5p133mmS1/osaK7fT+3bt0/nzp1TqVSa/LUX5YMPPsh77733qW4TAJpL6/nrEQBI8p+9GZdccslst9122X333RuExvfffz+9e/fOd77znfmeN3PmzHTu3DlHHXVU/bLa2tocf/zxWXXVVVNTU5MBAwbk6KOPTm1tbYPnViqVHHzwwRk3blzWWmut1NTU5KabbkqS/OIXv8jGG2+cpZZaKl26dMkGG2ywwPOyvfPOO/nhD3+YpZdeOt27d8+OO+6YF198MZVKJSeccEKDdV988cXsu+++6du3b2pqarLWWmvlf//3fxfr8/noudHmnVftnnvuyRFHHJE+ffpkiSWWyC677JLXXnutwXscO3ZsZs+enUqlkkql0uCw4d///vfZYIMN0qVLl/Tu3TsjRozI1KlTG2x7s802y9prr51JkyZlyJAh6dq1a/7nf/7nE33W1113XdZee+369z/v8/7o57Tffvtl2WWXTU1NTVZaaaV8//vfbxAlpk+fnsMOOywDBgxITU1NVl111Zx66qmpq6v72M/xySefzIQJE+o/j8022yzJgs/ROO+9P/bYY9l0003TtWvXrLrqqvXfCxMmTMhGG22ULl26ZPXVV89tt922wPfzSb/u1113XQYOHJhVVlmlwfJ99tkn3bp1y/PPP5/tt98+3bp1y3LLLZezzz47SfL4449niy22yBJLLJEVV1wxl112WYPnv//++znxxBOz2mqrpXPnzllqqaXy1a9+NbfeemuD9YYOHZok+eMf/7jIORf1ffbBBx/kpJNOyiqrrJKampoMHDgw//M//zPf98jAgQOz/fbb5+abb86GG26YLl265Pzzz1/kdh944IFss8026dmzZ7p27ZpNN90099xzT4N1nnvuufzgBz/I6quvni5dumSppZbK17/+9QWek3D69Ok5/PDDM3DgwNTU1GT55ZfPt7/97bz++usN1qurq8vPfvazLL/88uncuXO23HLLTJ48eZGzJskJJ5yQSqWSf/7zn9ljjz3So0ePLLXUUjn00EPni4iL+v30t7/9LcOHD0+PHj3SrVu3bLnllrn//vvrn/vQQw+lUqnkkksumW+Gm2++OZVKJTfccEOSBZ+jsVqt5uSTT87yyy+frl27ZvPNN8+TTz65wPe0OD+LU6ZMSaVSyS9+8YuMGTOm/nvh73//+8d+ZgDQFjh0GgBamXHjxmXXXXdNp06dstdee+Xcc8/NX//613zpS19Kx44ds8suu+Saa67J+eef32CPqeuuuy61tbX156+rq6vLjjvumIkTJ+bAAw/Mmmuumccffzxnnnlm/vWvf+W6665rsN077rgjV1xxRQ4++OAsvfTSGThwYJL/nBdvxx13zDe+8Y289957GT9+fL7+9a/nhhtuyHbbbVf//H322SdXXHFFvvWtb+UrX/lKJkyY0ODxeV599dV85StfqY8Hffr0yY033pj99tsvM2fOzGGHHfaJPrdDDjkkSy65ZI4//vhMmTIlY8aMycEHH5zLL788SXLppZfmggsuyIMPPpiLLrooSbLxxhsnSX72s5/lJz/5SfbYY4/sv//+ee2113LWWWdlyJAh+dvf/pZevXrVb+eNN97I8OHDM2LEiHzzm99M3759G/1ZT5w4Mddcc01+8IMfpHv37vn1r3+d3XbbLc8//3yWWmqpJMlLL72UL3/5y5k+fXoOPPDArLHGGnnxxRdz1VVXZc6cOenUqVPmzJmTTTfdNC+++GK++93vZoUVVsi9996bUaNG5eWXX86YMWMW+nmNGTMmhxxySLp165Zjjz02SdK3b99FfsZvvfVWtt9++4wYMSJf//rXc+6552bEiBEZN25cDjvssHzve9/L3nvvndNPPz277757pk6dmu7duycp/7rfe++9+a//+q8FPjZ37twMHz48Q4YMyWmnnZZx48bl4IMPzhJLLJFjjz023/jGN7LrrrvmvPPOy7e//e0MHjw4K620UpL/BK/Ro0dn//33z5e//OXMnDkzDz30UB5++OFstdVW9dvo2bNnVlllldxzzz05/PDDFzrnor7P9t9//1xyySXZfffdc+SRR+aBBx7I6NGj849//CPXXnttg9d56qmnstdee+W73/1uDjjggKy++uoL3eYdd9yR4cOHZ4MNNsjxxx+fdu3aZezYsdliiy1y991358tf/nKS5K9//WvuvffejBgxIssvv3ymTJmSc889N5tttln+/ve/p2vXrkn+c9Gdr33ta/nHP/6RfffdN//1X/+V119/Pddff31eeOGFLL300vXb/vnPf5527drlqKOOyowZM3LaaaflG9/4Rh544IGFzvthe+yxRwYOHJjRo0fn/vvvz69//eu89dZb+d3vfjffe/zo76cnn3wyX/va19KjR48cffTR6dixY84///xsttlm9eF7ww03zMorr5wrrrgiI0eObPCal19+eZZccskMGzZsofMdd9xxOfnkk7Pttttm2223zcMPP5ytt956vj0QG/uzOHbs2Lz77rs58MADU1NTk969ey/W5wUArV4VAGg1HnrooWqS6q233lqtVqvVurq66vLLL1899NBD69e5+eabq0mqf/rTnxo8d9ttt62uvPLK9fcvvfTSart27ap33313g/XOO++8apLqPffcU78sSbVdu3bVJ598cr6Z5syZ0+D+e++9V1177bWrW2yxRf2ySZMmVZNUDzvssAbr7rPPPtUk1eOPP75+2X777Vft379/9fXXX2+w7ogRI6o9e/acb3sfteKKK1ZHjhxZf3/s2LHVJNWhQ4dW6+rq6pcffvjh1fbt21enT59ev2zkyJHVJZZYosHrTZkypdq+ffvqz372swbLH3/88WqHDh0aLN90002rSarnnXdeg3Ub+1l36tSpOnny5Ppljz76aDVJ9ayzzqpf9u1vf7varl276l//+tf5PoN57/Okk06qLrHEEtV//etfDR4/5phjqu3bt68+//zz8z33w9Zaa63qpptuOt/yO++8s5qkeuedd8733i+77LL6Zf/85z/rv3fuv//++uXzvkfHjh1bv6zk6/7+++9XK5VK9cgjj5zvsZEjR1aTVE855ZT6ZW+99Va1S5cu1UqlUh0/fvx88374+3G99darbrfddgvd9odtvfXW1TXXXPNj11vQ99kjjzxSTVLdf//9Gyw/6qijqkmqd9xxR/2yFVdcsZqketNNN33sturq6qqrrbZaddiwYQ2+/+fMmVNdaaWVqltttVWDZR913333VZNUf/e739UvO+6446pJqtdcc80Ct1et/v/fI2uuuWa1tra2/vFf/epX1STVxx9/fJFzH3/88dUk1R133LHB8h/84AfVJNVHH320ftnCfj/tvPPO1U6dOlWfeeaZ+mUvvfRStXv37tUhQ4bULxs1alS1Y8eO1TfffLN+WW1tbbVXr17Vfffdt37ZvN8lzz77bLVarVanTZtW7dSpU3W77bZr8Nn+z//8TzVJg99Di/uz+Oyzz1aTVHv06FGdNm3aIj8jAGiLHDoNAK3IuHHj0rdv32y++eZJ/nPI4J577pnx48dn7ty5SZItttgiSy+9dP2eesl/9jS79dZbs+eee9Yvu/LKK7PmmmtmjTXWyOuvv15/22KLLZIkd955Z4Ntb7rpphk0aNB8M3Xp0qXBdmbMmJGvfe1refjhh+uXzzuM8Qc/+EGD5x5yyCEN7ler1Vx99dXZYYcdUq1WG8w1bNiwzJgxo8HrNsaBBx7Y4NxqX/va1zJ37tw899xzi3zeNddck7q6uuyxxx4N5unXr19WW221+T6nmpqa+Q5db+xnPXTo0AaHAK+77rrp0aNH/fn/6urqct1112WHHXbIhhtuON/M897nlVdema997WtZcsklG2x36NChmTt3bv7yl7983MfWKN26dWtwxefVV189vXr1ypprrtngCunz/j3v/ZR+3d98881Uq9UsueSSC11n//33r/93r169svrqq2eJJZZocE7HefN++DyLvXr1ypNPPpmnn376Y9//vM/5k/i///u/JMkRRxzRYPmRRx6ZJPnzn//cYPlKK620yD3t5nnkkUfy9NNPZ++9984bb7xR/7nOnj07W265Zf7yl7/UH7r74Z/l999/P2+88UZWXXXV9OrVq8Hnf/XVV2e99dbLLrvsMt/2Pnr+wu985zsN9qz+2te+liSLdS7LJDnooIMa3J/3O2Pe5zXPR38/zZ07N7fcckt23nnnrLzyyvXL+/fvn7333jsTJ07MzJkzkyR77rln3n///VxzzTX1691yyy2ZPn16g9+ZH3XbbbflvffeyyGHHNLgfS9o79vG/izutttu6dOnz0K3DQBtlUOnAaCVmDt3bsaPH5/NN988zz77bP3yjTbaKGeccUZuv/32bL311unQoUN22223XHbZZamtrU1NTU2uueaavP/++w3+o/npp5/OP/7xj4X+x+y0adMa3J93KOlH3XDDDTn55JPzyCOPNDiX3If/w/u5555Lu3bt5nuNj14t+7XXXsv06dNzwQUX5IILLlisuRbXCius0OD+vCj11ltvLfJ5Tz/9dKrValZbbbUFPv7RC88st9xy813ko7Gf9UdnnTfvvFlfe+21zJw5M2uvvfbHzv7YY48t9nZLLb/88vOFpp49e2bAgAHzLUvS4P00xde9Wq0ucHnnzp3n+wx69uy50Hk//D3x05/+NDvttFO+8IUvZO21184222yTb33rW1l33XUXuP1PeqGQeT8jH/2Z6NevX3r16jVfEF/Yz+NHzQukHz0s+MNmzJiRJZdcMu+8805Gjx6dsWPH5sUXX2zwec6YMaP+388880x22223xdr+J/25m+ejP3errLJK2rVrN995Iz/6ebz22muZM2fOAg8pX3PNNVNXV5epU6dmrbXWynrrrZc11lgjl19+efbbb78k/zlseumll67/nwELMu9r8tEZ+/TpM1/0buzP4uJ+fQGgrREaAaCVuOOOO/Lyyy9n/PjxGT9+/HyPjxs3LltvvXWSZMSIETn//PNz4403Zuedd84VV1yRNdZYI+utt179+nV1dVlnnXXyy1/+coHb+2gc+vDeTvPcfffd2XHHHTNkyJCcc8456d+/fzp27JixY8fOd1GNxTFvz6pvfvObCw0jCwo8i6N9+/YLXL6wOPXhmSqVSm688cYFvka3bt0a3F/Q59TYz/qTzrqg7W611VY5+uijF/j4F77whUa93sdZ2Nwf935Kv+69e/dOpVJZaLz6pHMlyZAhQ/LMM8/kj3/8Y2655ZZcdNFFOfPMM3Peeec12Esy+U88+/D5CT+JxQ2VC/o+W5B5n+3pp5+eL37xiwtcZ9738CGHHJKxY8fmsMMOy+DBg9OzZ89UKpWMGDHiYy8etDBN9b08z8I+n8X9PBZmzz33zM9+9rO8/vrr6d69e66//vrstdde6dChaf5zqLE/i6XvBwBaK6ERAFqJcePGZZlllqm/Wu6HXXPNNbn22mtz3nnnpUuXLhkyZEj69++fyy+/PF/96ldzxx131F/QY55VVlkljz76aLbccstPvBfW1Vdfnc6dO+fmm29OTU1N/fKxY8c2WG/FFVdMXV1dnn322QZ7/3z06rN9+vRJ9+7dM3fu3Pqr+La0VVZZJdVqNSuttNInDnNN8Vl/WJ8+fdKjR4888cQTH7vdWbNmfeLPsilmXRylX/cOHTpklVVWabCnb1OadyX373znO5k1a1aGDBmSE044Yb7Q+OyzzzaI+Y0x72fk6aefzpprrlm//NVXX8306dOz4oorfqLXnXcIfo8ePT72s73qqqsycuTInHHGGfXL3n333UyfPn2+1/y4772m8vTTTzfYu2/y5Mmpq6urvxjVwvTp0yddu3bNU089Nd9j//znP9OuXbsGgX/PPffMiSeemKuvvjp9+/bNzJkzG5wGYEHmfU2efvrpBodnv/baa/NF79KfRQD4rHCORgBoBd55551cc8012X777bP77rvPdzv44IPz9ttv5/rrr0+StGvXLrvvvnv+9Kc/5dJLL80HH3ww37nG9thjj7z44ou58MILF7i92bNnf+xc7du3T6VSqT8/ZJJMmTJlvqsozzuX3DnnnNNg+VlnnTXf6+222265+uqrFxgyXnvttY+dqantuuuuad++fU488cT59sKqVqt54403PvY1muKz/rB27dpl5513zp/+9Kc89NBD8z0+b8499tgj9913X26++eb51pk+fXo++OCDRW5niSWWmC8yNYem+LoPHjx4gZ9FqY9+fbt165ZVV121wWkCkv8cWvzMM8/UX0G6sbbddtskme/qw/P2gl3QFdoXxwYbbJBVVlklv/jFLzJr1qz5Hv/wZ9u+ffv5vsfPOuusBj/fyX/OH/joo4/OdyXs5JPvqbgwH/0fK/N+ZwwfPnyRz2vfvn223nrr/PGPf2xwmPWrr76ayy67LF/96lfTo0eP+uVrrrlm1llnnVx++eW5/PLL079//wwZMmSR2xg6dGg6duyYs846q8H7XtDV3Et/FgHgs8IejQDQClx//fV5++23s+OOOy7w8a985Svp06dPxo0bVx8U99xzz5x11lk5/vjjs8466zTYSypJvvWtb+WKK67I9773vdx5553ZZJNNMnfu3Pzzn//MFVdckZtvvnmBFxr5sO222y6//OUvs80222TvvffOtGnTcvbZZ2fVVVfNY489Vr/eBhtskN122y1jxozJG2+8ka985SuZMGFC/vWvfyVpuOfcz3/+89x5553ZaKONcsABB2TQoEF588038/DDD+e2227Lm2+++Yk+w09qlVVWycknn5xRo0ZlypQp2XnnndO9e/c8++yzufbaa3PggQfmqKOOWuRrNMVn/VGnnHJKbrnllmy66aY58MADs+aaa+bll1/OlVdemYkTJ6ZXr1750Y9+lOuvvz7bb7999tlnn2ywwQaZPXt2Hn/88Vx11VWZMmXKIg/13WCDDXLuuefm5JNPzqqrrppllllmkeesK1H6dd9pp51y6aWX5l//+leTHhI+aNCgbLbZZtlggw3Su3fvPPTQQ7nqqqty8MEHN1jvtttuS7VazU477fSJtrPeeutl5MiRueCCCzJ9+vRsuummefDBB3PJJZdk5513rr8AVGO1a9cuF110UYYPH5611lor3/nOd7LccsvlxRdfzJ133pkePXrkT3/6U5Jk++23z6WXXpqePXtm0KBBue+++3LbbbdlqaWWavCaP/rRj3LVVVfl61//evbdd99ssMEGefPNN3P99dfnvPPO+8R7dS7Is88+mx133DHbbLNN7rvvvvz+97/P3nvvvVjbOPnkk3Prrbfmq1/9an7wgx+kQ4cOOf/881NbW5vTTjttvvX33HPPHHfccencuXP222+/tGu36H0u+vTpk6OOOiqjR4/O9ttvn2233TZ/+9vfcuONN873c1X6swgAnxVCIwC0AuPGjUvnzp2z1VZbLfDxdu3aZbvttsu4cePyxhtvZKmllsrGG2+cAQMGZOrUqQu8cmq7du1y3XXX5cwzz8zvfve7XHvttenatWtWXnnlHHrooYsVa7bYYov89re/zc9//vMcdthhWWmllXLqqadmypQpDUJjkvzud79Lv3798oc//CHXXntthg4dmssvvzyrr756OnfuXL9e37598+CDD+anP/1prrnmmpxzzjlZaqmlstZaa+XUU09t5CfXNI455ph84QtfyJlnnpkTTzwxyX/Oq7j11lsvNP5+WFN81h+13HLL5YEHHshPfvKTjBs3LjNnzsxyyy2X4cOHp2vXrkmSrl27ZsKECTnllFNy5ZVX5ne/+1169OiRL3zhCznxxBPrL8qyMMcdd1yee+65nHbaaXn77bez6aabNltoLP2677DDDll66aVzxRVX5Mc//nGTzfXDH/4w119/fW655ZbU1tZmxRVXzMknn5wf/ehHDda78sor89WvfrXB1cIb66KLLsrKK6+ciy++ONdee2369euXUaNG5fjjjy96D5tttlnuu+++nHTSSfnNb36TWbNmpV+/ftloo43y3e9+t369X/3qV2nfvn3GjRuXd999N5tsskluu+22+a5u3a1bt9x99905/vjjc+211+aSSy7JMsssky233DLLL7980awfdfnll+e4447LMccckw4dOuTggw/O6aefvljPXWuttXL33Xdn1KhRGT16dOrq6rLRRhvl97//fYOroM+z55575sc//nHmzJmzyKtNf9jJJ5+czp0757zzzqsP5bfccst8e6CW/iwCwGdFpdrUxz8AAPw/jzzySNZff/38/ve/zze+8Y2WHoc27qSTTsrYsWPz9NNPL/QiJM3hlVdeyUorrZTx48d/4j0aaeiEE07IiSeemNdee82efgDwGeIcjQBAk3jnnXfmWzZmzJi0a9fuY8+FBovj8MMPz6xZsxZ4VfbmNGbMmKyzzjoiIwDAx3DoNADQJE477bRMmjQpm2++eTp06JAbb7wxN954Yw488MAGV3+FT6pbt26ZNm3ap77dn//855/6NgEA2iKhEQBoEhtvvHFuvfXWnHTSSZk1a1ZWWGGFnHDCCTn22GNbejQAAOBT0KLnaDz33HNz7rnnZsqUKUn+c0Ln4447LsOHD0+SvPvuuznyyCMzfvz41NbWZtiwYTnnnHPSt2/flhoZAAAAAFiAFg2Nf/rTn9K+ffusttpqqVarueSSS3L66afnb3/7W9Zaa618//vfz5///OdcfPHF6dmzZw4++OC0a9cu99xzT0uNDAAAAAAsQKu76nTv3r1z+umnZ/fdd0+fPn1y2WWXZffdd0+S/POf/8yaa66Z++67L1/5yldaeFIAAAAAYJ5Wc47GuXPn5sorr8zs2bMzePDgTJo0Ke+//36GDh1av84aa6yRFVZYoVGhsa6uLi+99FK6d++eSqXSXOMDAAAAwGdStVrN22+/nWWXXTbt2rVb6HotHhoff/zxDB48OO+++266deuWa6+9NoMGDcojjzySTp06pVevXg3W79u3b1555ZWFvl5tbW1qa2vr77/44osZNGhQc40PAAAAAJ8LU6dOzfLLL7/Qx1s8NK6++up55JFHMmPGjFx11VUZOXJkJkyY8Ilfb/To0TnxxBPnWz516tT06NGjZFQAAAAA+NyZOXNmBgwYkO7duy9yvVZ3jsahQ4dmlVVWyZ577pktt9wyb731VoO9GldcccUcdthhOfzwwxf4/I/u0Tjvg5gxY4bQCAAAAACNNHPmzPTs2fNj+9rCD6puIXV1damtrc0GG2yQjh075vbbb69/7Kmnnsrzzz+fwYMHL/T5NTU16dGjR4MbAAAAANC8WvTQ6VGjRmX48OFZYYUV8vbbb+eyyy7LXXfdlZtvvjk9e/bMfvvtlyOOOCK9e/dOjx49csghh2Tw4MGuOA0AAAAArUyLhsZp06bl29/+dl5++eX07Nkz6667bm6++eZstdVWSZIzzzwz7dq1y2677Zba2toMGzYs55xzTkuODAAAAAAsQKs7R2NTW9xjyAEAAACA+bXZczQCAAAAAG2P0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAK3c7NmzU6lUUqlUMnv27JYeBwAAABZIaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIq1aGgcPXp0vvSlL6V79+5ZZpllsvPOO+epp55qsM5mm22WSqXS4Pa9732vhSYGAAAAABakRUPjhAkTctBBB+X+++/Prbfemvfffz9bb711Zs+e3WC9Aw44IC+//HL97bTTTmuhiQEAAACABenQkhu/6aabGty/+OKLs8wyy2TSpEkZMmRI/fKuXbumX79+n/Z4AAAAAMBialXnaJwxY0aSpHfv3g2Wjxs3LksvvXTWXnvtjBo1KnPmzGmJ8QAAAACAhWjRPRo/rK6uLocddlg22WSTrL322vXL995776y44opZdtll89hjj+W///u/89RTT+Waa65Z4OvU1tamtra2/v7MmTObfXYAAAAA+LxrNaHxoIMOyhNPPJGJEyc2WH7ggQfW/3udddZJ//79s+WWW+aZZ57JKqusMt/rjB49OieeeGKzzwsAAAAA/P9axaHTBx98cG644YbceeedWX755Re57kYbbZQkmTx58gIfHzVqVGbMmFF/mzp1apPPCwAAAAA01KJ7NFar1RxyyCG59tprc9ddd2WllVb62Oc88sgjSZL+/fsv8PGamprU1NQ05ZgAAAAAwMdo0dB40EEH5bLLLssf//jHdO/ePa+88kqSpGfPnunSpUueeeaZXHbZZdl2222z1FJL5bHHHsvhhx+eIUOGZN11123J0QEAAACAD6lUq9Vqi228Ulng8rFjx2afffbJ1KlT881vfjNPPPFEZs+enQEDBmSXXXbJj3/84/To0WOxtjFz5sz07NkzM2bMWOznALQms2fPTrdu3ZIks2bNyhJLLNHCEwEAAPB5srh9rcUPnV6UAQMGZMKECZ/SNAAAAADAJ9UqLgYDAAAAALRtQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQrENLD0C5gcf8uaVHAJpR3Xvv1v97zZ/clHadOrfgNEBzmvLz7Vp6BAAA+MTs0QgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKtWhoHD16dL70pS+le/fuWWaZZbLzzjvnqaeearDOu+++m4MOOihLLbVUunXrlt122y2vvvpqC00MAAAAACxIi4bGCRMm5KCDDsr999+fW2+9Ne+//3623nrrzJ49u36dww8/PH/6059y5ZVXZsKECXnppZey6667tuDUAAAAAMBHdWjJjd90000N7l988cVZZpllMmnSpAwZMiQzZszIb3/721x22WXZYostkiRjx47Nmmuumfvvvz9f+cpXWmJsAAAAAOAjWtU5GmfMmJEk6d27d5Jk0qRJef/99zN06ND6ddZYY42ssMIKue+++1pkRgAAAABgfi26R+OH1dXV5bDDDssmm2yStddeO0nyyiuvpFOnTunVq1eDdfv27ZtXXnllga9TW1ub2tra+vszZ85stpkBAAAAgP9oNXs0HnTQQXniiScyfvz4otcZPXp0evbsWX8bMGBAE00IAAAAi2/27NmpVCqpVCoNrkUA8FnVKkLjwQcfnBtuuCF33nlnll9++frl/fr1y3vvvZfp06c3WP/VV19Nv379Fvhao0aNyowZM+pvU6dObc7RAQAAAIC0cGisVqs5+OCDc+211+aOO+7ISiut1ODxDTbYIB07dsztt99ev+ypp57K888/n8GDBy/wNWtqatKjR48GNwAAAACgebXoORoPOuigXHbZZfnjH/+Y7t271593sWfPnunSpUt69uyZ/fbbL0cccUR69+6dHj165JBDDsngwYNdcRoAAAAAWpEWDY3nnntukmSzzTZrsHzs2LHZZ599kiRnnnlm2rVrl9122y21tbUZNmxYzjnnnE95UoCW065T56z43ze09BgAAACwSC0aGqvV6seu07lz55x99tk5++yzP4WJAAAAAIBPolVcDAYAAAAAaNuERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKBYh8asPH369Fx77bW5++6789xzz2XOnDnp06dP1l9//QwbNiwbb7xxc80JAAAAALRii7VH40svvZT9998//fv3z8knn5x33nknX/ziF7Pllltm+eWXz5133pmtttoqgwYNyuWXX97cMwMAAAAArcxi7dG4/vrrZ+TIkZk0aVIGDRq0wHXeeeedXHfddRkzZkymTp2ao446qkkHBQAAAABar8UKjX//+9+z1FJLLXKdLl26ZK+99spee+2VN954o0mGAwAAAADahsU6dPrjImPp+gAAAABA29boq05fcskl+fOf/1x//+ijj06vXr2y8cYb57nnnmvS4QAAAACAtqHRofGUU05Jly5dkiT33Xdfzj777Jx22mlZeumlc/jhhzf5gAAAAABA67dY52j8sKlTp2bVVVdNklx33XXZbbfdcuCBB2aTTTbJZptt1tTzAQAAAABtQKP3aOzWrVv9xV5uueWWbLXVVkmSzp0755133mna6QAAAACANqHRezRutdVW2X///bP++uvnX//6V7bddtskyZNPPpmBAwc29XwAAAAAQBvQ6D0azz777AwePDivvfZarr766vorTE+aNCl77bVXkw8IAAAAALR+jd6jsVevXvnNb34z3/ITTzyxSQYCAAAAANqeRofGJHn33Xfz2GOPZdq0aamrq6tfXqlUssMOOzTZcAAAAABA29Do0HjTTTflW9/6Vv0FYT6sUqlk7ty5TTIYAAAAANB2NPocjYccckj22GOPvPzyy6mrq2twExkBAAAA4POp0aHx1VdfzRFHHJG+ffs2xzwAAAAAQBvU6NC4++6756677mqGUQAAAACAtqrR52j8zW9+k69//eu5++67s84666Rjx44NHv/hD3/YZMMBAAAAAG1Do0PjH/7wh9xyyy3p3Llz7rrrrlQqlfrHKpWK0AgAAAAAn0ONDo3HHntsTjzxxBxzzDFp167RR14DAAAAAJ9BjS6F7733Xvbcc0+REQAAAACo1+haOHLkyFx++eXNMQsAAAAA0EY1+tDpuXPn5rTTTsvNN9+cddddd76Lwfzyl79ssuEAAAAAgLah0aHx8ccfz/rrr58keeKJJxo89uELwwAAAAAAnx+NDo133nlnc8wBAAAAALRhrugCAAAAABRbrND4ve99Ly+88MJiveDll1+ecePGFQ0FAAAAALQti3XodJ8+fbLWWmtlk002yQ477JANN9wwyy67bDp37py33norf//73zNx4sSMHz8+yy67bC644ILmnhsAAAAAaEUWKzSedNJJOfjgg3PRRRflnHPOyd///vcGj3fv3j1Dhw7NBRdckG222aZZBgUAAAAAWq/FvhhM3759c+yxx+bYY4/NW2+9leeffz7vvPNOll566ayyyiquOA0AAAAAn2ONvup0kiy55JJZcsklm3oWAAAAAKCNctVpAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUOwThcYPPvggt912W84///y8/fbbSZKXXnops2bNatLhAAAAAIC2odFXnX7uueeyzTbb5Pnnn09tbW222mqrdO/ePaeeempqa2tz3nnnNcecAAAAAEAr1ug9Gg899NBsuOGGeeutt9KlS5f65bvssktuv/32Jh0OAAAAAGgbGr1H491335177703nTp1arB84MCBefHFF5tsMAAAAACg7Wj0Ho11dXWZO3fufMtfeOGFdO/evUmGAgAAAADalkaHxq233jpjxoypv1+pVDJr1qwcf/zx2XbbbZtyNgAAAACgjWj0odNnnHFGhg0blkGDBuXdd9/N3nvvnaeffjpLL710/vCHPzTHjAAAAABAK9fo0Lj88svn0Ucfzfjx4/PYY49l1qxZ2W+//fKNb3yjwcVhAAAAAIDPj0aHxiTp0KFDvvnNbzb1LAAAAABAG/WJQuNLL72UiRMnZtq0aamrq2vw2A9/+MMmGQwAAAAAaDsaHRovvvjifPe7302nTp2y1FJLpVKp1D9WqVSERgAAAAD4HGp0aPzJT36S4447LqNGjUq7do2+aDUAAAAA8BnU6FI4Z86cjBgxQmQEAAAAAOo1uhbut99+ufLKK5tjFgAAAACgjWr0odOjR4/O9ttvn5tuuinrrLNOOnbs2ODxX/7yl002HAAAAADQNnyi0HjzzTdn9dVXT5L5LgYDAAAAAHz+NDo0nnHGGfnf//3f7LPPPs0wDgAAAADQFjX6HI01NTXZZJNNmmMWAAAAAKCNanRoPPTQQ3PWWWc1xywAAAAAQBvV6EOnH3zwwdxxxx254YYbstZaa813MZhrrrmmyYYDAAAAANqGRofGXr16Zdddd22OWQAAAACANqrRoXHs2LHNMQcAAAAA0IY1+hyNAAAAAAAftVh7NP7Xf/1Xbr/99iy55JJZf/31U6lUFrruww8/3GTDAQAAAABtw2KFxp122ik1NTX1/15UaAQAAAAAPn8WKzQef/zx9f8+4YQTmmsWAAAAAKCNavQ5GldeeeW88cYb8y2fPn16Vl555SYZCgAAAABoWxodGqdMmZK5c+fOt7y2tjYvvPBCkwwFAAAAALQti3XodJJcf/319f+++eab07Nnz/r7c+fOze23356VVlqpaacDAIDPuIHH/LmlRwCaSd1779b/e82f3JR2nTq34DRAc5vy8+1aeoQWt9ihceedd06SVCqVjBw5ssFjHTt2zMCBA3PGGWc06XAAAAAAQNuw2KGxrq4uSbLSSivlr3/9a5ZeeulmGwoAAAAAaFsWOzTO8+yzzzbHHAAAAABAG9boi8EAAAAAAHyU0AgAAAAAFBMaAQAAAIBiQiMAAAAAUOwThcZnnnkmP/7xj7PXXntl2rRpSZIbb7wxTz75ZJMOBwAAAAC0DY0OjRMmTMg666yTBx54INdcc01mzZqVJHn00Udz/PHHN/mAAAAAAEDr1+jQeMwxx+Tkk0/Orbfemk6dOtUv32KLLXL//fc36XAAAAAAQNvQ6ND4+OOPZ5dddplv+TLLLJPXX3+9SYYCAAAAANqWRofGXr165eWXX55v+d/+9rcst9xyTTIUAAAAANC2NDo0jhgxIv/93/+dV155JZVKJXV1dbnnnnty1FFH5dvf/nZzzAgAAAAAtHKNDo2nnHJK1lhjjQwYMCCzZs3KoEGDMmTIkGy88cb58Y9/3KjX+stf/pIddtghyy67bCqVSq677roGj++zzz6pVCoNbttss01jRwYAAAAAmlmHxj6hU6dOufDCC3Pcccfl8ccfz6xZs7L++utntdVWa/TGZ8+enfXWWy/77rtvdt111wWus80222Ts2LH192tqahq9HQAAAACgeTU6NM4zYMCADBgwoGjjw4cPz/Dhwxe5Tk1NTfr161e0HQAAAACgeTX60Onddtstp5566nzLTzvttHz9619vkqE+7K677soyyyyT1VdfPd///vfzxhtvNPk2AAAAAIAyjQ6Nf/nLX7LtttvOt3z48OH5y1/+0iRDzbPNNtvkd7/7XW6//faceuqpmTBhQoYPH565c+cu9Dm1tbWZOXNmgxsAAAAA0Lwafej0rFmz0qlTp/mWd+zYscmj3ogRI+r/vc4662TdddfNKquskrvuuitbbrnlAp8zevTonHjiiU06BwAAAACwaI3eo3GdddbJ5ZdfPt/y8ePHZ9CgQU0y1MKsvPLKWXrppTN58uSFrjNq1KjMmDGj/jZ16tRmnQkAAAAA+AR7NP7kJz/JrrvummeeeSZbbLFFkuT222/PH/7wh1x55ZVNPuCHvfDCC3njjTfSv3//ha5TU1PjytQAAAAA8ClrdGjcYYcdct111+WUU07JVVddlS5dumTdddfNbbfdlk033bRRrzVr1qwGeyc+++yzeeSRR9K7d+/07t07J554Ynbbbbf069cvzzzzTI4++uisuuqqGTZsWGPHBgAAAACaUaNDY5Jst9122W677Yo3/tBDD2XzzTevv3/EEUckSUaOHJlzzz03jz32WC655JJMnz49yy67bLbeeuucdNJJ9lgEAAAAgFbmE4XGJHnvvfcybdq01NXVNVi+wgorLPZrbLbZZqlWqwt9/Oabb/6k4wEAAAAAn6JGh8ann346++67b+69994Gy6vVaiqVSubOndtkwwEAAAAAbUOjQ+M+++yTDh065IYbbkj//v1TqVSaYy4AAAAAoA1pdGh85JFHMmnSpKyxxhrNMQ8AAAAA0Aa1a+wTBg0alNdff705ZgEAAAAA2qhGh8ZTTz01Rx99dO6666688cYbmTlzZoMbAAAAAPD50+hDp4cOHZok2XLLLRssdzEYAAAAAPj8anRovPPOO5tjDgAAAACgDWt0aNx0002bYw4AAAAAoA1r9Dkak+Tuu+/ON7/5zWy88cZ58cUXkySXXnppJk6c2KTDAQAAAABtQ6ND49VXX51hw4alS5cuefjhh1NbW5skmTFjRk455ZQmHxAAAAAAaP0aHRpPPvnknHfeebnwwgvTsWPH+uWbbLJJHn744SYdDgAAAABoGxodGp966qkMGTJkvuU9e/bM9OnTm2ImAAAAAKCNaXRo7NevXyZPnjzf8okTJ2bllVdukqEAAAAAgLal0aHxgAMOyKGHHpoHHngglUolL730UsaNG5ejjjoq3//+95tjRgAAAACglevQ2Cccc8wxqaury5Zbbpk5c+ZkyJAhqampyVFHHZVDDjmkOWYEAAAAAFq5RoXGuXPn5p577slBBx2UH/3oR5k8eXJmzZqVQYMGpVu3bs01IwAAAADQyjUqNLZv3z5bb711/vGPf6RXr14ZNGhQc80FAAAAALQhjT5H49prr51///vfzTELAAAAANBGNTo0nnzyyTnqqKNyww035OWXX87MmTMb3AAAAACAz59GXwxm2223TZLsuOOOqVQq9cur1WoqlUrmzp3bdNMBAAAAAG1Co0PjnXfe2RxzAAAAAABtWKND46abbtoccwAAAAAAbVijz9GYJHfffXe++c1vZuONN86LL76YJLn00kszceLEJh0OAAAAAGgbGh0ar7766gwbNixdunTJww8/nNra2iTJjBkzcsoppzT5gAAAAABA6/eJrjp93nnn5cILL0zHjh3rl2+yySZ5+OGHm3Q4AAAAAKBtaHRofOqppzJkyJD5lvfs2TPTp09vipkAAAAAgDam0aGxX79+mTx58nzLJ06cmJVXXrlJhgIAAAAA2pZGh8YDDjgghx56aB544IFUKpW89NJLGTduXI466qh8//vfb44ZAQAAAIBWrkNjn3DMMcekrq4uW265ZebMmZMhQ4akpqYmRx11VA455JDmmBEAAAAAaOUWKzQ+9thjWXvttdOuXbtUKpUce+yx+dGPfpTJkydn1qxZGTRoULp169bcswIAAAAArdRiHTq9/vrr5/XXX0+SrLzyynnjjTfSqVOnDBo0KF/+8pdFRgAAAAD4nFus0NirV688++yzSZIpU6akrq6uWYcCAAAAANqWxTp0erfddsumm26a/v37p1KpZMMNN0z79u0XuO6///3vJh0QAAAAAGj9Fis0XnDBBdl1110zefLk/PCHP8wBBxyQ7t27N/dsAAAAAEAbsdhXnd5mm22SJJMmTcqhhx4qNAIAAAAA9RY7NM4zduzY5pgDAAAAAGjDGh0aZ8+enZ///Oe5/fbbM23atPkuDOMcjQAAAADw+dPo0Lj//vtnwoQJ+da3vlV/cRgAAAAA4POt0aHxxhtvzJ///OdssskmzTEPAAAAANAGtWvsE5Zccsn07t27OWYBAAAAANqoRofGk046Kccdd1zmzJnTHPMAAAAAAG1Qow+dPuOMM/LMM8+kb9++GThwYDp27Njg8YcffrjJhgMAAAAA2oZGh8add965GcYAAAAAANqyRofG448/vjnmAAAAAADasEafoxEAAAAA4KMWe4/GJZdcMpVK5WPXe/PNN4sGAgAAAADansUOjWPGjGnGMQAAAACAtmyxQ+PIkSObcw4AAAAAoA1zjkYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIot9lWn5zniiCMWuLxSqaRz585ZddVVs9NOO6V3797FwwEAAAAAbUOjQ+Pf/va3PPzww5k7d25WX331JMm//vWvtG/fPmussUbOOeecHHnkkZk4cWIGDRrU5AMDAAAAAK1Pow+d3mmnnTJ06NC89NJLmTRpUiZNmpQXXnghW221Vfbaa6+8+OKLGTJkSA4//PDmmBcAAAAAaIUaHRpPP/30nHTSSenRo0f9sp49e+aEE07Iaaedlq5du+a4447LpEmTmnRQAAAAAKD1anRonDFjRqZNmzbf8tdeey0zZ85MkvTq1Svvvfde+XQAAAAAQJvwiQ6d3nfffXPttdfmhRdeyAsvvJBrr702++23X3beeeckyYMPPpgvfOELTT0rAAAAANBKNfpiMOeff34OP/zwjBgxIh988MF/XqRDh4wcOTJnnnlmkmSNNdbIRRdd1LSTAgAAAACtVqNDY7du3XLhhRfmzDPPzL///e8kycorr5xu3brVr/PFL36xyQYEAAAAAFq/Rh86/fvf/z5z5sxJt27dsu6662bddddtEBkBAAAAgM+fRofGww8/PMsss0z23nvv/N///V/mzp3bHHMBAAAAAG1Io0Pjyy+/nPHjx6dSqWSPPfZI//79c9BBB+Xee+9tjvkAAAAAgDag0aGxQ4cO2X777TNu3LhMmzYtZ555ZqZMmZLNN988q6yySnPMCAAAAAC0co2+GMyHde3aNcOGDctbb72V5557Lv/4xz+aai4AAAAAoA1p9B6NSTJnzpyMGzcu2267bZZbbrmMGTMmu+yyS5588smmng8AAAAAaAMavUfjiBEjcsMNN6Rr167ZY4898pOf/CSDBw9ujtkAAAAAgDai0aGxffv2ueKKKzJs2LC0b9++wWNPPPFE1l577SYbDgAAAABoGxodGseNG9fg/ttvv50//OEPueiiizJp0qTMnTu3yYYDAAAAANqGT3SOxiT5y1/+kpEjR6Z///75xS9+kS222CL3339/U84GAAAAALQRjdqj8ZVXXsnFF1+c3/72t5k5c2b22GOP1NbW5rrrrsugQYOaa0YAAAAAoJVb7D0ad9hhh6y++up57LHHMmbMmLz00ks566yzmnM2AAAAAKCNWOw9Gm+88cb88Ic/zPe///2sttpqzTkTAAAAANDGLPYejRMnTszbb7+dDTbYIBtttFF+85vf5PXXX2/O2QAAAACANmKxQ+NXvvKVXHjhhXn55Zfz3e9+N+PHj8+yyy6burq63HrrrXn77bebc04AAAAAoBVr9FWnl1hiiey7776ZOHFiHn/88Rx55JH5+c9/nmWWWSY77rhjc8wIAAAAALRyjQ6NH7b66qvntNNOywsvvJA//OEPTTUTAAAAANDGFIXGedq3b5+dd945119/fVO8HAAAAADQxjRJaAQAAAAAPt+ERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAinVo6QEAAADgs6hdp85Z8b9vaOkxAD419mgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKtWho/Mtf/pIddtghyy67bCqVSq677roGj1er1Rx33HHp379/unTpkqFDh+bpp59umWEBAAAAgIVq0dA4e/bsrLfeejn77LMX+Phpp52WX//61znvvPPywAMPZIkllsiwYcPy7rvvfsqTAgAAAACL0qElNz58+PAMHz58gY9Vq9WMGTMmP/7xj7PTTjslSX73u9+lb9++ue666zJixIhPc1QAAAAAYBFa7Tkan3322bzyyisZOnRo/bKePXtmo402yn333bfQ59XW1mbmzJkNbgAAAABA82q1ofGVV15JkvTt27fB8r59+9Y/tiCjR49Oz549628DBgxo1jkBAAAAgFYcGj+pUaNGZcaMGfW3qVOntvRIAAAAAPCZ12pDY79+/ZIkr776aoPlr776av1jC1JTU5MePXo0uAEAAAAAzavVhsaVVlop/fr1y+23316/bObMmXnggQcyePDgFpwMAAAAAPioFr3q9KxZszJ58uT6+88++2weeeSR9O7dOyussEIOO+ywnHzyyVlttdWy0kor5Sc/+UmWXXbZ7Lzzzi03NAAAAAAwnxYNjQ899FA233zz+vtHHHFEkmTkyJG5+OKLc/TRR2f27Nk58MADM3369Hz1q1/NTTfdlM6dO7fUyAAAAADAArRoaNxss81SrVYX+nilUslPf/rT/PSnP/0UpwIAAAAAGqvVnqMRAAAAAGg7hEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAECxVh0aTzjhhFQqlQa3NdZYo6XHAgAAAAA+okNLD/Bx1lprrdx222319zt0aPUjAwAAAMDnTquvdh06dEi/fv1aegwAAAAAYBFa9aHTSfL0009n2WWXzcorr5xvfOMbef755xe5fm1tbWbOnNngBgAAAAA0r1YdGjfaaKNcfPHFuemmm3Luuefm2Wefzde+9rW8/fbbC33O6NGj07Nnz/rbgAEDPsWJAQAAAODzqVWHxuHDh+frX/961l133QwbNiz/93//l+nTp+eKK65Y6HNGjRqVGTNm1N+mTp36KU4MAAAAAJ9Prf4cjR/Wq1evfOELX8jkyZMXuk5NTU1qamo+xakAAAAAgFa9R+NHzZo1K88880z69+/f0qMAAAAAAB/SqkPjUUcdlQkTJmTKlCm59957s8suu6R9+/bZa6+9Wno0AAAAAOBDWvWh0y+88EL22muvvPHGG+nTp0+++tWv5v7770+fPn1aejQAAAAA4ENadWgcP358S48AAAAAACyGVn3oNAAAAADQNgiNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYkIjAAAAAFBMaAQAAAAAigmNAAAAAEAxoREAAAAAKCY0AgAAAADFhEYAAAAAoJjQCAAAAAAUExoBAAAAgGJCIwAAAABQTGgEAAAAAIoJjQAAAABAMaERAAAAACgmNAIAAAAAxYRGAAAAAKCY0AgAAAAAFBMaAQAAAIBiQiMAAAAAUExoBAAAAACKCY0AAAAAQDGhEQAAAAAoJjQCAAAAAMWERgAAAACgmNAIAAAAABQTGgEAAACAYm0iNJ599tkZOHBgOnfunI022igPPvhgS48EAAAAAHxIqw+Nl19+eY444ogcf/zxefjhh7Peeutl2LBhmTZtWkuPBgAAAAD8P60+NP7yl7/MAQcckO985zsZNGhQzjvvvHTt2jX/+7//29KjAQAAAAD/T4eWHmBR3nvvvUyaNCmjRo2qX9auXbsMHTo099133wKfU1tbm9ra2vr7M2bMSJLMnDmzeYdtQXW1c1p6BACgCXyW/15h4fwtBwCfDZ/lv+XmvbdqtbrI9Vp1aHz99dczd+7c9O3bt8Hyvn375p///OcCnzN69OiceOKJ8y0fMGBAs8wIANBUeo5p6QkAAPikPg9/y7399tvp2bPnQh9v1aHxkxg1alSOOOKI+vt1dXV58803s9RSS6VSqbTgZACf3MyZMzNgwIBMnTo1PXr0aOlxAABYTP6OAz4LqtVq3n777Sy77LKLXK9Vh8all1467du3z6uvvtpg+auvvpp+/fot8Dk1NTWpqalpsKxXr17NNSLAp6pHjx7+QAUAaIP8HQe0dYvak3GeVn0xmE6dOmWDDTbI7bffXr+srq4ut99+ewYPHtyCkwEAAAAAH9aq92hMkiOOOCIjR47MhhtumC9/+csZM2ZMZs+ene985zstPRoAAAAA8P+0+tC455575rXXXstxxx2XV155JV/84hdz0003zXeBGIDPspqamhx//PHznRoCAIDWzd9xwOdJpfpx16UGAAAAAPgYrfocjQAAAABA2yA0AgAAAADFhEYAAAAAoJjQCNDGXHzxxenVq1dLjwEAAAANCI3A594+++yTSqWSSqWSTp06ZdVVV81Pf/rTfPDBBx/73Isvvrj+uQu7TZkypfnfxGK6+uqrs9lmm6Vnz57p1q1b1l133fz0pz/Nm2++maTh+2nXrl2WX375fOc738m0adOSJFOmTEmlUskjjzwy32tvttlmOeywwz7FdwMAsHBTp07Nvvvum2WXXTadOnXKiiuumEMPPTRvvPFGg/U222yzVCqVjB8/vsHyMWPGZODAgfX35/2dtM022zRYb/r06alUKrnrrruSJI8++mg6deqU66+/vsF6V199dTp37pwnnnhikXP7ew1oy4RGgCTbbLNNXn755Tz99NM58sgjc8IJJ+T000//2Oftueeeefnll+tvgwcPzgEHHNBg2YABAxZ7jvfee6/kbSzSsccemz333DNf+tKXcuONN+aJJ57IGWeckUcffTSXXnpp/Xo9evTIyy+/nBdeeCEXXnhhbrzxxnzrW99qtrkAAJrav//972y44YZ5+umn84c//CGTJ0/Oeeedl9tvvz2DBw+uj3bzdO7cOT/+8Y/z/vvvL/J1O3TokNtuuy133nnnQtdZb731ctxxx+XAAw+sj5rTpk3L9773vZx44olZe+21F/pcf68BbZ3QCJCkpqYm/fr1y4orrpjvf//7GTp0aK6//vrMnj07PXr0yFVXXdVg/euuuy5LLLFEPvjgg/Tr16/+1qlTp3Tt2rX+/nvvvZddd9013bp1S48ePbLHHnvk1VdfrX+dE044IV/84hdz0UUXZaWVVkrnzp2T/Of/jH/3u99N375907lz56y99tq54YYbGsxw8803Z80110y3bt3qQ+nCPPjggznllFNyxhln5PTTT8/GG2+cgQMHZquttsrVV1+dkSNH1q9bqVTSr1+/LLvsshk+fHh++MMf5rbbbss777zTFB81AECzO+igg9KpU6fccsst2XTTTbPCCitk+PDhue222/Liiy/m2GOPbbD+XnvtlenTp+fCCy9c5OsuscQS2XfffXPMMccscr1Ro0ZlhRVWyEEHHZQk+e53v5vVVlstRx111EKf4+814LNAaARYgC5duuS9997LEksskREjRmTs2LENHh87dmx23333dO/efaGvUVdXl5122ilvvvlmJkyYkFtvvTX//ve/s+eeezZYb/Lkybn66v+vvTsLifJt4zj+G61wy60kK0sYzVLJqDTPFEMTqSOlJJWiFAMpUkNDPEjFSkOCJKwgyj08SBCiHNQioyRbEAyFbDOjIqNM1MpI34NoyNTRsqh/7/dzNHPfz73MHF1c9/KcV21trdra2jQyMqKoqChdv35dlZWV6ujoUEFBgaytrc1thoaGVFRUpIqKCjU3N+vp06cWA9eqqio5ODgoJSVlwnpLdz7a2tpqZGRkWkfJAQAA/rQ3b97IZDIpJSVFtra2Y+rc3d0VHx+vmpoajY6OmssdHR2VnZ2tvLw8DQ4OWuw/JydH7e3t4xaiv2Vtba2ysjLV1dUpLi5OJpNJpaWlY+K57xGvAfgXkGgEgG+Mjo6qsbFRJpNJ69evlyQlJSXJZDKZdwy+evVKFy9e1M6dOy321dTUpPb2dlVXV2vt2rUKDg5WeXm5rl69qlu3bpmfGx4eVnl5uVavXq2AgAA1NjaqtbVVtbW1ioiIkNFo1KZNmxQVFWVu8+nTJ508eVKBgYFas2aNdu/eraampknn0tXVJaPRqNmzZ//Q/9HV1WUex1JSFQAA4G/R1dWl0dFR+fr6Tljv6+urt2/fqre3d0x5SkqKbGxsdPToUYv9L1q0SHv37lV2drbFxJ6vr69SU1N17tw55eTkyMfHZ8p5E68B+K8j0QgAki5cuCAHBwfZ2NgoKipKsbGxysnJkSStW7dO/v7+KisrkyRVVlbK09NTISEhFvvs7OzUkiVLxtzR6OfnJ2dnZ3V2dprLPD095ebmZv7e1tYmDw8Pi8GonZ2dvLy8zN8XLlxovgB8It+u2E/l3bt3cnBwkJ2dnZYvX64FCxaoqqpq2u0BAAD+Bj8S/0hfrtLJy8tTUVGRXr9+bfHZ/fv3q7e3V2fOnJn0mYGBAdXU1MjOzk7Xrl37pfMlXgPwtyLRCACSwsLC1NbWpq6uLr1//15lZWWyt7c31yclJam0tFTSl2PTO3bskMFg+CVjfzuOpHFHfCby/Uq3wWCwGJz6+Pjo0aNHU15wLklz585VW1ub7t27p8HBQTU3N5uTno6OjpK+BLff6+vrk5OT05T9AwAA/E7e3t4yGAxjFna/1dnZKRcXlzELvV8lJCTI09NT+fn5FsdwdnZWVlaWcnNzNTQ0NOEzGRkZsrGx0Y0bN9TY2Kjy8nKLfRKvAfgXkGgEAH1J9nl7e2vp0qWaNWvWuPqEhAR1d3eruLhYHR0dYy7jnoyvr696enrU09NjLuvo6FBfX5/8/PwmbRcQEKBnz57p/v37P/djJhAXF6eBgQGVlJRMWN/X12f+bGVlJW9vbxmNxnFJT1dXV82fP1937twZU97f368HDx5MeSQIAADgd5s3b54iIiJUUlIy7uUoL1++VFVVlWJjYydcNLaystLhw4d14sQJPXnyxOI4e/bskZWVlY4dOzaurqGhQadPn1ZZWZlWrVql/Px8paamWnx5H/EagH8BiUYAmAYXFxdFR0crIyNDGzZskIeHx5RtwsPDtXLlSsXHx+vu3btqbW3Vtm3bFBoaqsDAwEnbhYaGKiQkRDExMWpoaNDjx4916dIl1dfX//T8g4ODlZmZqX379ikzM1MtLS3q7u5WU1OTNm/ebD4WPh3p6ek6dOiQqqqq9PDhQ7W2tio+Pl5ubm6Kjo7+6TkCAAD8KsePH9fHjx8VGRmp5uZm9fT0qL6+XhEREVq8eLEOHjw4aduNGzcqODhYp06dsjiGjY2NcnNzVVxcPKa8v79fiYmJysjIUFBQkCQpLS1Nfn5+Sk5OnrQ/4jUA/wISjQAwTYmJiRoeHp7yJTBfGQwG1dXVycXFRSEhIQoPD5fRaFRNTc2Ubc+fP6+goCBt3bpVfn5+yszM1OfPn2c0/8LCQlVXV+vmzZuKjIyUv7+/0tPTFRAQMK0dml9lZmbqwIEDKiwsVEBAgGJiYmRvb68rV65M69g3AADA77Zs2TLdvn1bRqNRW7ZskZeXl5KTkxUWFqaWlha5urpabF9YWKgPHz5MOc727dtlNBrHlKWmpsrJycl837f0ZQfi2bNndfnyZYtHqInXAPzXGUZ/9IZcAPg/VVFRobS0ND1//lxz5sz509MBAAAAAOCvMv4iMgDAGENDQ3rx4oUKCgq0a9cukowAAAAAAEyAo9MAMIUjR45oxYoVcnd3V1ZW1p+eDgAAAAAAfyWOTgMAAAAAAACYMXY0AgAAAAAAAJgxEo0AAAAAAAAAZoxEIwAAAAAAAIAZI9EIAAAAAAAAYMZINAIAAAAAAACYMRKNAAAAAAAAAGaMRCMAAAAAAACAGSPRCAAAAAAAAGDGSDQCAAAAAAAAmLH/AdVxUJyGIWHGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1600x1200 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "\n",
    "\n",
    "# Compute average inference time + std\n",
    "time_results = {k: np.mean(v.model_inference_time) * 1e3 for k, v in results.items()}\n",
    "time_results_std = np.std([v.model_inference_time for v in results.values()]) * 1000\n",
    "\n",
    "plt.rcdefaults()\n",
    "fig, ax = plt.subplots(figsize=(16, 12))\n",
    "ax.set_ylabel(\"Avg Inference time (ms)\")\n",
    "ax.set_title(\"Average inference time (ms) for each provider\")\n",
    "ax.bar(time_results.keys(), time_results.values(), yerr=time_results_std)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Quantization support from transformers\n",
    "\n",
    "Quantization enables the use of integers (_instead of floatting point_) arithmetic to run neural networks models faster. From a high-level point of view, quantization works as mapping the float32 ranges of values as int8 with the less loss in the performances of the model.\n",
    "\n",
    "Hugging Face provides a conversion tool as part of the transformers repository to easily export quantized models to ONNX Runtime. For more information, please refer to the following: \n",
    "\n",
    "- [Hugging Face Documentation on ONNX Runtime quantization supports](https://huggingface.co/transformers/master/serialization.html#quantization)\n",
    "- [Intel's Explanation of Quantization](https://nervanasystems.github.io/distiller/quantization.html)\n",
    "\n",
    "With this method, the accuracy of the model remains at the same level than the full-precision model. If you want to see benchmarks on model performances, we recommand reading the [ONNX Runtime notebook](https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/quantization/notebooks/Bert-GLUE_OnnxRuntime_quantization.ipynb) on the subject."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmarking PyTorch quantized model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:01<00:00, 90.15it/s]\n"
     ]
    }
   ],
   "source": [
    "import torch \n",
    "\n",
    "# Quantize\n",
    "model_pt_quantized = torch.quantization.quantize_dynamic(\n",
    "    model_pt.to(\"cpu\"), {torch.nn.Linear}, dtype=torch.qint8\n",
    ")\n",
    "\n",
    "# Warm up \n",
    "model_pt_quantized(**model_inputs)\n",
    "\n",
    "# Benchmark PyTorch quantized model\n",
    "time_buffer = []\n",
    "for _ in trange(100):\n",
    "    with track_infer_time(time_buffer):\n",
    "        model_pt_quantized(**model_inputs)\n",
    "    \n",
    "results[\"PyTorch CPU Quantized\"] = OnnxInferenceResult(\n",
    "    time_buffer,\n",
    "    None\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Benchmarking ONNX quantized model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "As of onnxruntime 1.4.0, models larger than 2GB will fail to quantize due to protobuf constraint.\n",
      "This limitation will be removed in the next release of onnxruntime.\n",
      "Quantized model has been written at bert.onnx: ✔\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Tracking inference time on CPUExecutionProvider with quantized model: 100%|██████████| 100/100 [00:00<00:00, 237.49it/s]\n"
     ]
    }
   ],
   "source": [
    "from transformers.convert_graph_to_onnx import quantize\n",
    "\n",
    "# Transformers allow you to easily convert float32 model to quantized int8 with ONNX Runtime\n",
    "quantized_model_path = quantize(Path(\"bert.opt.onnx\"))\n",
    "\n",
    "# Then you just have to load through ONNX runtime as you would normally do\n",
    "quantized_model = create_model_for_provider(quantized_model_path.as_posix(), \"CPUExecutionProvider\")\n",
    "\n",
    "# Warm up the overall model to have a fair comparaison\n",
    "outputs = quantized_model.run(None, inputs_onnx)\n",
    "\n",
    "# Evaluate performances\n",
    "time_buffer = []\n",
    "for _ in trange(100, desc=f\"Tracking inference time on CPUExecutionProvider with quantized model\"):\n",
    "    with track_infer_time(time_buffer):\n",
    "        outputs = quantized_model.run(None, inputs_onnx)\n",
    "\n",
    "# Store the result\n",
    "results[\"ONNX CPU Quantized\"] = OnnxInferenceResult(\n",
    "    time_buffer, \n",
    "    quantized_model_path\n",
    ") "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the inference performance of each providers "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 676
    },
    "colab_type": "code",
    "id": "dj-rS8AcqRZQ",
    "outputId": "b4bf07d1-a7b4-4eff-e6bd-d5d424fd17fb"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABRoAAAPeCAYAAABjjKazAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAA9hAAAPYQGoP6dpAABnA0lEQVR4nOzdd3iV9fn48fuEEWaCDBmCshwgai22igNQUcSJooKruFvrVmqlDkSpOKrSWhVHi6MouKBoiwsRxVmx7lZBQVFQFIVA0KDk/P7ol/yMAUz4JCbR1+u6znVxnvOc89w5SY7y5hmZbDabDQAAAACABDnVPQAAAAAAUPsJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AQK3SsWPHOProo9f7+VdeeWV07tw56tSpEz/5yU8qba4fkieeeCIymUw88cQT1T1KKcuXL48NN9wwxo8f/71u99xzz43tt9++Qs/5of+crf4Zuffee6t7lEpz6623RiaTiXnz5n3nuqmfQwDwQyU0AkANdP3110cmk6lw3GDdHnnkkTjnnHNip512inHjxsWll15a3SNVq+uvvz5uvfXW6h6j3P74xz9G06ZNY8iQId/rds8444x45ZVXYsqUKeVa388ZAPBjVbe6BwAAyho/fnx07NgxXnjhhZgzZ0507dq1ukeqMd56663IyVm/fyt9/PHHIycnJ/7yl79E/fr1K3my2uf666+Pli1bltkzq3fv3vHFF1/UqPfoq6++ij/+8Y9x5plnRp06db7Xbbdp0yYOOOCA+MMf/hD777//d67v56x2Ouqoo2LIkCGRm5tb3aMAQK1lj0YAqGHmzp0bzzzzTFx99dXRqlWr7/0w0YiI4uLi+PLLL7/37ZZHbm5u1KtXb72eu2jRomjYsGGlxp8VK1ZU2mvVFDk5OdGgQYP1DrpV4cEHH4xPPvkkDj300GrZ/qGHHhozZ86Md9999zvXreyfs2w2G1988UWlvNYPQVV9PtWpUycaNGgQmUym0l97Xb7++utYuXLl97pNAKgqNef/HgGAiPjf3owbbLBB7LPPPnHwwQeXCo1fffVVNG/ePI455pgyzysoKIgGDRrEsGHDSpYVFRXFiBEjomvXrpGbmxsdOnSIc845J4qKiko9N5PJxCmnnBLjx4+PLbfcMnJzc+Ohhx6KiIg//OEPseOOO0aLFi2iYcOG0bNnzzWel+2LL76I0047LVq2bBlNmzaN/fffPz788MPIZDJx0UUXlVr3ww8/jGOPPTZat24dubm5seWWW8Zf//rXcr0/3z432urzqj399NNx1llnRatWraJx48Zx4IEHxieffFLqaxw3blwUFhZGJpOJTCZT6rDhv/3tb9GzZ89o2LBhNG/ePIYMGRLz588vte2+fftGjx49YtasWdG7d+9o1KhR/O53v1uv93ry5MnRo0ePkq9/9fv97ffpuOOOi3bt2kVubm506tQpTjrppFJRYsmSJXHGGWdEhw4dIjc3N7p27RqXX355FBcXf+f7+MYbb8SMGTNK3o++fftGxJrP0bj6a3/11VejT58+0ahRo+jatWvJz8KMGTNi++23j4YNG8bmm28ejz322Bq/nvX9vk+ePDk6duwYXbp0KbX86KOPjiZNmsT7778f++67bzRp0iQ22mijuO666yIi4rXXXovddtstGjduHJtssknceeedpZ7/1VdfxciRI2PTTTeNBg0aRIsWLWLnnXeORx99tNR6/fr1i4iIv//97+ucc10/Z19//XVccskl0aVLl8jNzY2OHTvG7373uzI/Ix07dox99903Hn744dhuu+2iYcOGceONN65zu88//3zstddekZ+fH40aNYo+ffrE008/XWqd9957L37961/H5ptvHg0bNowWLVrEIYccssZzEi5ZsiTOPPPM6NixY+Tm5kb79u3jF7/4RXz66ael1isuLo7f//730b59+2jQoEHsvvvuMWfOnHXOGhFx0UUXRSaTif/+979x6KGHRl5eXrRo0SJOP/30MhFxXZ9P//73v2PAgAGRl5cXTZo0id133z2ee+65kue++OKLkclk4rbbbiszw8MPPxyZTCYefPDBiFjzORqz2WyMGjUq2rdvH40aNYpdd9013njjjTV+TeX5XZw3b15kMpn4wx/+EGPGjCn5WXjzzTe/8z0DgNrAodMAUMOMHz8+DjrooKhfv34cdthhccMNN8S//vWv+NnPfhb16tWLAw88MO6///648cYbS+0xNXny5CgqKio5f11xcXHsv//+MXPmzDjxxBOjW7du8dprr8U111wTb7/9dkyePLnUdh9//PG4++6745RTTomWLVtGx44dI+J/58Xbf//944gjjoiVK1fGhAkT4pBDDokHH3ww9tlnn5LnH3300XH33XfHUUcdFTvssEPMmDGj1OOrffzxx7HDDjuUxINWrVrF1KlT47jjjouCgoI444wz1ut9O/XUU2ODDTaIESNGxLx582LMmDFxyimnxMSJEyMi4o477oibbropXnjhhbjlllsiImLHHXeMiIjf//73ccEFF8Shhx4axx9/fHzyySdx7bXXRu/evePf//53NGvWrGQ7ixcvjgEDBsSQIUPiyCOPjNatW1f4vZ45c2bcf//98etf/zqaNm0af/rTn2LQoEHx/vvvR4sWLSIiYsGCBfHzn/88lixZEieeeGJsscUW8eGHH8a9994bK1asiPr168eKFSuiT58+8eGHH8Yvf/nL2HjjjeOZZ56J4cOHx8KFC2PMmDFrfb/GjBkTp556ajRp0iTOO++8iIho3br1Ot/jzz//PPbdd98YMmRIHHLIIXHDDTfEkCFDYvz48XHGGWfEr371qzj88MPjyiuvjIMPPjjmz58fTZs2jYj07/szzzwTP/3pT9f42KpVq2LAgAHRu3fvuOKKK2L8+PFxyimnROPGjeO8886LI444Ig466KAYO3Zs/OIXv4hevXpFp06dIuJ/wWv06NFx/PHHx89//vMoKCiIF198MV566aXYY489SraRn58fXbp0iaeffjrOPPPMtc65rp+z448/Pm677bY4+OCD4+yzz47nn38+Ro8eHf/5z39i0qRJpV7nrbfeisMOOyx++ctfxgknnBCbb775Wrf5+OOPx4ABA6Jnz54xYsSIyMnJiXHjxsVuu+0WTz31VPz85z+PiIh//etf8cwzz8SQIUOiffv2MW/evLjhhhuib9++8eabb0ajRo0i4n8X3dlll13iP//5Txx77LHx05/+ND799NOYMmVKfPDBB9GyZcuSbV922WWRk5MTw4YNi6VLl8YVV1wRRxxxRDz//PNrnfebDj300OjYsWOMHj06nnvuufjTn/4Un3/+edx+++1lvsZvfz698cYbscsuu0ReXl6cc845Ua9evbjxxhujb9++JeF7u+22i86dO8fdd98dQ4cOLfWaEydOjA022CD69++/1vkuvPDCGDVqVOy9996x9957x0svvRR77rlnmT0QK/q7OG7cuPjyyy/jxBNPjNzc3GjevHm53i8AqPGyAECN8eKLL2YjIvvoo49ms9lstri4ONu+ffvs6aefXrLOww8/nI2I7AMPPFDquXvvvXe2c+fOJffvuOOObE5OTvapp54qtd7YsWOzEZF9+umnS5ZFRDYnJyf7xhtvlJlpxYoVpe6vXLky26NHj+xuu+1WsmzWrFnZiMieccYZpdY9+uijsxGRHTFiRMmy4447Ltu2bdvsp59+WmrdIUOGZPPz88ts79s22WST7NChQ0vujxs3LhsR2X79+mWLi4tLlp955pnZOnXqZJcsWVKybOjQodnGjRuXer158+Zl69Spk/39739favlrr72WrVu3bqnlffr0yUZEduzYsaXWreh7Xb9+/eycOXNKlr3yyivZiMhee+21Jct+8YtfZHNycrL/+te/yrwHq7/OSy65JNu4cePs22+/Xerxc889N1unTp3s+++/X+a537Tllltm+/TpU2b59OnTsxGRnT59epmv/c477yxZ9t///rfkZ+e5554rWb76Z3TcuHEly1K+71999VU2k8lkzz777DKPDR06NBsR2UsvvbRk2eeff55t2LBhNpPJZCdMmFBm3m/+PG6zzTbZffbZZ63b/qY999wz261bt+9cb00/Zy+//HI2IrLHH398qeXDhg3LRkT28ccfL1m2ySabZCMi+9BDD33ntoqLi7Obbrpptn///qV+/lesWJHt1KlTdo899ii17NueffbZbERkb7/99pJlF154YTYisvfff/8at5fN/v+fkW7dumWLiopKHv/jH/+YjYjsa6+9ts65R4wYkY2I7P77719q+a9//etsRGRfeeWVkmVr+3waOHBgtn79+tl33nmnZNmCBQuyTZs2zfbu3btk2fDhw7P16tXLfvbZZyXLioqKss2aNcsee+yxJctWf5bMnTs3m81ms4sWLcrWr18/u88++5R6b3/3u99lI6LU51B5fxfnzp2bjYhsXl5edtGiRet8jwCgNnLoNADUIOPHj4/WrVvHrrvuGhH/O2Rw8ODBMWHChFi1alVEROy2227RsmXLkj31Iv63p9mjjz4agwcPLll2zz33RLdu3WKLLbaITz/9tOS22267RUTE9OnTS227T58+0b179zIzNWzYsNR2li5dGrvssku89NJLJctXH8b461//utRzTz311FL3s9ls3HfffbHffvtFNpstNVf//v1j6dKlpV63Ik488cRS51bbZZddYtWqVfHee++t83n3339/FBcXx6GHHlpqnjZt2sSmm25a5n3Kzc0tc+h6Rd/rfv36lToEeOutt468vLyS8/8VFxfH5MmTY7/99ovtttuuzMyrv8577rkndtlll9hggw1Kbbdfv36xatWqePLJJ7/rbauQJk2alLri8+abbx7NmjWLbt26lbpC+uo/r/56Ur/vn332WWSz2dhggw3Wus7xxx9f8udmzZrF5ptvHo0bNy51TsfV837zPIvNmjWLN954I2bPnv2dX//q93l9/POf/4yIiLPOOqvU8rPPPjsiIv7xj3+UWt6pU6d17mm32ssvvxyzZ8+Oww8/PBYvXlzyvhYWFsbuu+8eTz75ZMmhu9/8Xf7qq69i8eLF0bVr12jWrFmp9/++++6LbbbZJg488MAy2/v2+QuPOeaYUntW77LLLhER5TqXZUTEySefXOr+6s+M1e/Xat/+fFq1alU88sgjMXDgwOjcuXPJ8rZt28bhhx8eM2fOjIKCgoiIGDx4cHz11Vdx//33l6z3yCOPxJIlS0p9Zn7bY489FitXroxTTz211Ne9pr1vK/q7OGjQoGjVqtVatw0AtZVDpwGghli1alVMmDAhdt1115g7d27J8u233z6uuuqqmDZtWuy5555Rt27dGDRoUNx5551RVFQUubm5cf/998dXX31V6i/Ns2fPjv/85z9r/cvsokWLSt1ffSjptz344IMxatSoePnll0udS+6bf/F+7733Iicnp8xrfPtq2Z988kksWbIkbrrpprjpppvKNVd5bbzxxqXur45Sn3/++TqfN3v27Mhms7Hpppuu8fFvX3hmo402KnORj4q+19+edfW8q2f95JNPoqCgIHr06PGds7/66qvl3m6q9u3blwlN+fn50aFDhzLLIqLU11MZ3/dsNrvG5Q0aNCjzHuTn56913m/+TFx88cVxwAEHxGabbRY9evSIvfbaK4466qjYeuut17j99b1QyOrfkW//TrRp0yaaNWtWJoiv7ffx21YH0m8fFvxNS5cujQ022CC++OKLGD16dIwbNy4+/PDDUu/n0qVLS/78zjvvxKBBg8q1/fX9vVvt2793Xbp0iZycnDLnjfz2+/HJJ5/EihUr1nhIebdu3aK4uDjmz58fW265ZWyzzTaxxRZbxMSJE+O4446LiP8dNt2yZcuSfwxYk9Xfk2/P2KpVqzLRu6K/i+X9/gJAbSM0AkAN8fjjj8fChQtjwoQJMWHChDKPjx8/Pvbcc8+IiBgyZEjceOONMXXq1Bg4cGDcfffdscUWW8Q222xTsn5xcXFstdVWcfXVV69xe9+OQ9/c22m1p556Kvbff//o3bt3XH/99dG2bduoV69ejBs3rsxFNcpj9Z5VRx555FrDyJoCT3nUqVNnjcvXFqe+OVMmk4mpU6eu8TWaNGlS6v6a3qeKvtfrO+uatrvHHnvEOeecs8bHN9tsswq93ndZ29zf9fWkft+bN28emUxmrfFqfeeKiOjdu3e888478fe//z0eeeSRuOWWW+Kaa66JsWPHltpLMuJ/8eyb5ydcH+UNlWv6OVuT1e/tlVdeGT/5yU/WuM7qn+FTTz01xo0bF2eccUb06tUr8vPzI5PJxJAhQ77z4kFrU1k/y6ut7f0p7/uxNoMHD47f//738emnn0bTpk1jypQpcdhhh0XdupXz16GK/i6mfj0AUFMJjQBQQ4wfPz423HDDkqvlftP9998fkyZNirFjx0bDhg2jd+/e0bZt25g4cWLsvPPO8fjjj5dc0GO1Ll26xCuvvBK77777eu+Fdd9990WDBg3i4Ycfjtzc3JLl48aNK7XeJptsEsXFxTF37txSe/98++qzrVq1iqZNm8aqVatKruJb3bp06RLZbDY6deq03mGuMt7rb2rVqlXk5eXF66+//p3bXb58+Xq/l5Uxa3mkft/r1q0bXbp0KbWnb2VafSX3Y445JpYvXx69e/eOiy66qExonDt3bqmYXxGrf0dmz54d3bp1K1n+8ccfx5IlS2KTTTZZr9ddfQh+Xl7ed7639957bwwdOjSuuuqqkmVffvllLFmypMxrftfPXmWZPXt2qb375syZE8XFxSUXo1qbVq1aRaNGjeKtt94q89h///vfyMnJKRX4Bw8eHCNHjoz77rsvWrduHQUFBaVOA7Amq78ns2fPLnV49ieffFImeqf+LgLAD4VzNAJADfDFF1/E/fffH/vuu28cfPDBZW6nnHJKLFu2LKZMmRIRETk5OXHwwQfHAw88EHfccUd8/fXXZc41duihh8aHH34YN9988xq3V1hY+J1z1alTJzKZTMn5ISMi5s2bV+YqyqvPJXf99deXWn7ttdeWeb1BgwbFfffdt8aQ8cknn3znTJXtoIMOijp16sTIkSPL7IWVzWZj8eLF3/kalfFef1NOTk4MHDgwHnjggXjxxRfLPL56zkMPPTSeffbZePjhh8uss2TJkvj666/XuZ3GjRuXiUxVoTK+77169Vrje5Hq29/fJk2aRNeuXUudJiDif4cWv/POOyVXkK6ovffeOyKizNWHV+8Fu6YrtJdHz549o0uXLvGHP/whli9fXubxb763derUKfMzfu2115b6/Y743/kDX3nllTJXwo5Y/z0V1+bb/7Cy+jNjwIAB63xenTp1Ys8994y///3vpQ6z/vjjj+POO++MnXfeOfLy8kqWd+vWLbbaaquYOHFiTJw4Mdq2bRu9e/de5zb69esX9erVi2uvvbbU172mq7mn/i4CwA+FPRoBoAaYMmVKLFu2LPbff/81Pr7DDjtEq1atYvz48SVBcfDgwXHttdfGiBEjYquttiq1l1RExFFHHRV33313/OpXv4rp06fHTjvtFKtWrYr//ve/cffdd8fDDz+8xguNfNM+++wTV199dey1115x+OGHx6JFi+K6666Lrl27xquvvlqyXs+ePWPQoEExZsyYWLx4ceywww4xY8aMePvttyOi9J5zl112WUyfPj223377OOGEE6J79+7x2WefxUsvvRSPPfZYfPbZZ+v1Hq6vLl26xKhRo2L48OExb968GDhwYDRt2jTmzp0bkyZNihNPPDGGDRu2zteojPf62y699NJ45JFHok+fPnHiiSdGt27dYuHChXHPPffEzJkzo1mzZvGb3/wmpkyZEvvuu28cffTR0bNnzygsLIzXXnst7r333pg3b946D/Xt2bNn3HDDDTFq1Kjo2rVrbLjhhus8Z12K1O/7AQccEHfccUe8/fbblXpIePfu3aNv377Rs2fPaN68ebz44otx7733ximnnFJqvcceeyyy2WwccMAB67WdbbbZJoYOHRo33XRTLFmyJPr06RMvvPBC3HbbbTFw4MCSC0BVVE5OTtxyyy0xYMCA2HLLLeOYY46JjTbaKD788MOYPn165OXlxQMPPBAREfvuu2/ccccdkZ+fH927d49nn302HnvssWjRokWp1/zNb34T9957bxxyyCFx7LHHRs+ePeOzzz6LKVOmxNixY9d7r841mTt3buy///6x1157xbPPPht/+9vf4vDDDy/XNkaNGhWPPvpo7LzzzvHrX/866tatGzfeeGMUFRXFFVdcUWb9wYMHx4UXXhgNGjSI4447LnJy1r3PRatWrWLYsGExevTo2HfffWPvvfeOf//73zF16tQyv1epv4sA8EMhNAJADTB+/Pho0KBB7LHHHmt8PCcnJ/bZZ58YP358LF68OFq0aBE77rhjdOjQIebPn7/GK6fm5OTE5MmT45prronbb789Jk2aFI0aNYrOnTvH6aefXq5Ys9tuu8Vf/vKXuOyyy+KMM86ITp06xeWXXx7z5s0rFRojIm6//fZo06ZN3HXXXTFp0qTo169fTJw4MTbffPNo0KBByXqtW7eOF154IS6++OK4//774/rrr48WLVrElltuGZdffnkF37nKce6558Zmm20W11xzTYwcOTIi/ndexT333HOt8febKuO9/raNNtoonn/++bjgggti/PjxUVBQEBtttFEMGDAgGjVqFBERjRo1ihkzZsSll14a99xzT9x+++2Rl5cXm222WYwcObLkoixrc+GFF8Z7770XV1xxRSxbtiz69OlTZaEx9fu+3377RcuWLePuu++O888/v9LmOu2002LKlCnxyCOPRFFRUWyyySYxatSo+M1vflNqvXvuuSd23nnnUlcLr6hbbrklOnfuHLfeemtMmjQp2rRpE8OHD48RI0YkfQ19+/aNZ599Ni655JL485//HMuXL482bdrE9ttvH7/85S9L1vvjH/8YderUifHjx8eXX34ZO+20Uzz22GNlrm7dpEmTeOqpp2LEiBExadKkuO2222LDDTeM3XffPdq3b58067dNnDgxLrzwwjj33HOjbt26ccopp8SVV15ZruduueWW8dRTT8Xw4cNj9OjRUVxcHNtvv3387W9/K3UV9NUGDx4c559/fqxYsWKdV5v+plGjRkWDBg1i7NixJaH8kUceKbMHaurvIgD8UGSylX38AwDA/3n55Zdj2223jb/97W9xxBFHVPc41HKXXHJJjBs3LmbPnr3Wi5BUhY8++ig6deoUEyZMWO89GintoosuipEjR8Ynn3xiTz8A+AFxjkYAoFJ88cUXZZaNGTMmcnJyvvNcaFAeZ555ZixfvnyNV2WvSmPGjImtttpKZAQA+A4OnQYAKsUVV1wRs2bNil133TXq1q0bU6dOjalTp8aJJ55Y6uqvsL6aNGkSixYt+t63e9lll33v2wQAqI2ERgCgUuy4447x6KOPxiWXXBLLly+PjTfeOC666KI477zzqns0AADge+AcjQAAAABAMudoBAAAAACSCY0AAAAAQLIf/Dkai4uLY8GCBdG0adPIZDLVPQ4AAAAA1CrZbDaWLVsW7dq1i5ycte+3+IMPjQsWLHClSwAAAABINH/+/Gjfvv1aH//Bh8amTZtGxP/eiLy8vGqeBgAAAABql4KCgujQoUNJZ1ubH3xoXH24dF5entAIAAAAAOvpu05L6GIwAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExohBqosLAwMplMZDKZKCwsrO5xAAAAAL6T0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgWbWGxhtuuCG23nrryMvLi7y8vOjVq1dMnTq15PEvv/wyTj755GjRokU0adIkBg0aFB9//HE1TgwAAAAArEm1hsb27dvHZZddFrNmzYoXX3wxdttttzjggAPijTfeiIiIM888Mx544IG45557YsaMGbFgwYI46KCDqnNkAAAAAGANMtlsNlvdQ3xT8+bN48orr4yDDz44WrVqFXfeeWccfPDBERHx3//+N7p16xbPPvts7LDDDuV6vYKCgsjPz4+lS5dGXl5eVY4OlaawsDCaNGkSERHLly+Pxo0bV/NEAAAAwI9VeftajTlH46pVq2LChAlRWFgYvXr1ilmzZsVXX30V/fr1K1lniy22iI033jieffbZapwUAAAAAPi2utU9wGuvvRa9evWKL7/8Mpo0aRKTJk2K7t27x8svvxz169ePZs2alVq/devW8dFHH6319YqKiqKoqKjkfkFBQVWNDgAAAAD8n2rfo3HzzTePl19+OZ5//vk46aSTYujQofHmm2+u9+uNHj068vPzS24dOnSoxGkBAAAAgDWp9tBYv3796Nq1a/Ts2TNGjx4d22yzTfzxj3+MNm3axMqVK2PJkiWl1v/444+jTZs2a3294cOHx9KlS0tu8+fPr+KvAAAAAACo9tD4bcXFxVFUVBQ9e/aMevXqxbRp00oee+utt+L999+PXr16rfX5ubm5kZeXV+oGAAAAAFStaj1H4/Dhw2PAgAGx8cYbx7Jly+LOO++MJ554Ih5++OHIz8+P4447Ls4666xo3rx55OXlxamnnhq9evUq9xWnAQAAAIDvR7WGxkWLFsUvfvGLWLhwYeTn58fWW28dDz/8cOyxxx4REXHNNddETk5ODBo0KIqKiqJ///5x/fXXV+fIAAAAAMAaZLLZbLa6h6hKBQUFkZ+fH0uXLnUYNbVGYWFhNGnSJCIili9fHo0bN67miQAAAIAfq/L2tRp3jkYAAAAAoPYRGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEhWraFx9OjR8bOf/SyaNm0aG264YQwcODDeeuutUuv07ds3MplMqduvfvWrapoYAAAAAFiTag2NM2bMiJNPPjmee+65ePTRR+Orr76KPffcMwoLC0utd8IJJ8TChQtLbldccUU1TQwAAAAArEnd6tz4Qw89VOr+rbfeGhtuuGHMmjUrevfuXbK8UaNG0aZNm+97PAAAAACgnGrUORqXLl0aERHNmzcvtXz8+PHRsmXL6NGjRwwfPjxWrFhRHeMBAAAAAGtRrXs0flNxcXGcccYZsdNOO0WPHj1Klh9++OGxySabRLt27eLVV1+N3/72t/HWW2/F/fffv8bXKSoqiqKiopL7BQUFVT47AAAAAPzY1ZjQePLJJ8frr78eM2fOLLX8xBNPLPnzVlttFW3bto3dd9893nnnnejSpUuZ1xk9enSMHDmyyucFAAAAAP6/GnHo9CmnnBIPPvhgTJ8+Pdq3b7/OdbfffvuIiJgzZ84aHx8+fHgsXbq05DZ//vxKnxcAAAAAKK1a92jMZrNx6qmnxqRJk+KJJ56ITp06fedzXn755YiIaNu27Rofz83Njdzc3MocEwAAAAD4DtUaGk8++eS488474+9//3s0bdo0Pvroo4iIyM/Pj4YNG8Y777wTd955Z+y9997RokWLePXVV+PMM8+M3r17x9Zbb12dowMAAAAA35DJZrPZatt4JrPG5ePGjYujjz465s+fH0ceeWS8/vrrUVhYGB06dIgDDzwwzj///MjLyyvXNgoKCiI/Pz+WLl1a7udAdSssLIwmTZpERMTy5cujcePG1TwRAAAA8GNV3r5W7YdOr0uHDh1ixowZ39M0AAAAAMD6qhEXgwEAAAAAajehEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkq1vdA5Cu47n/qO4RqGTFK78s+XO3Cx6KnPoNqnEaqsK8y/ap7hEAAACgUtmjEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAFAhhYWFkclkIpPJRGFhYXWPAwAA1BBCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAklVraBw9enT87Gc/i6ZNm8aGG24YAwcOjLfeeqvUOl9++WWcfPLJ0aJFi2jSpEkMGjQoPv7442qaGAAAAABYk2oNjTNmzIiTTz45nnvuuXj00Ufjq6++ij333DMKCwtL1jnzzDPjgQceiHvuuSdmzJgRCxYsiIMOOqgapwYAAAAAvq1udW78oYceKnX/1ltvjQ033DBmzZoVvXv3jqVLl8Zf/vKXuPPOO2O33XaLiIhx48ZFt27d4rnnnosddtihOsYGAAAAAL6lRp2jcenSpRER0bx584iImDVrVnz11VfRr1+/knW22GKL2HjjjePZZ5+tlhkBAAAAgLKqdY/GbyouLo4zzjgjdtppp+jRo0dERHz00UdRv379aNasWal1W7duHR999NEaX6eoqCiKiopK7hcUFFTZzAAAAADA/9SYPRpPPvnkeP3112PChAlJrzN69OjIz88vuXXo0KGSJgQAAAAA1qZGhMZTTjklHnzwwZg+fXq0b9++ZHmbNm1i5cqVsWTJklLrf/zxx9GmTZs1vtbw4cNj6dKlJbf58+dX5egAAAAAQFRzaMxms3HKKafEpEmT4vHHH49OnTqVerxnz55Rr169mDZtWsmyt956K95///3o1avXGl8zNzc38vLySt0AAAAAgKpVredoPPnkk+POO++Mv//979G0adOS8y7m5+dHw4YNIz8/P4477rg466yzonnz5pGXlxennnpq9OrVyxWnAQAAAKAGqdbQeMMNN0RERN++fUstHzduXBx99NEREXHNNddETk5ODBo0KIqKiqJ///5x/fXXf8+TAgAAAADrUq2hMZvNfuc6DRo0iOuuuy6uu+6672EiAAAAAGB91IiLwQAAAAAAtZvQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQCAH4TCwsLIZDKRyWSisLCwuscBgB8doREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMnqVmTlJUuWxKRJk+Kpp56K9957L1asWBGtWrWKbbfdNvr37x877rhjVc0JAAAAANRg5dqjccGCBXH88cdH27ZtY9SoUfHFF1/ET37yk9h9992jffv2MX369Nhjjz2ie/fuMXHixKqeGQAAAACoYcq1R+O2224bQ4cOjVmzZkX37t3XuM4XX3wRkydPjjFjxsT8+fNj2LBhlTooAAAAAFBzlSs0vvnmm9GiRYt1rtOwYcM47LDD4rDDDovFixdXynAAAAAAQO1QrkOnvysypq4PAAAAANRuFb7q9G233Rb/+Mc/Su6fc8450axZs9hxxx3jvffeq9ThAAAAAIDaocKh8dJLL42GDRtGRMSzzz4b1113XVxxxRXRsmXLOPPMMyt9QAAAAACg5ivXORq/af78+dG1a9eIiJg8eXIMGjQoTjzxxNhpp52ib9++lT0fAAAAAFALVHiPxiZNmpRc7OWRRx6JPfbYIyIiGjRoEF988UXlTgcAAAAA1AoV3qNxjz32iOOPPz623XbbePvtt2PvvfeOiIg33ngjOnbsWNnzAQAAAAC1QIX3aLzuuuuiV69e8cknn8R9991XcoXpWbNmxWGHHVbpAwIAAAAANV+F92hs1qxZ/PnPfy6zfOTIkZUyEAAAAABQ+1Q4NEZEfPnll/Hqq6/GokWLori4uGR5JpOJ/fbbr9KGAwAAAABqhwqHxoceeiiOOuqokgvCfFMmk4lVq1ZVymAAAAAAQO1R4XM0nnrqqXHooYfGwoULo7i4uNRNZAQAAACAH6cKh8aPP/44zjrrrGjdunVVzAMAAAAA1EIVPnT64IMPjieeeCK6dOlSFfMAEZFTv0Fs8tsHq3sMAAAAgHKrcGj885//HIccckg89dRTsdVWW0W9evVKPX7aaadV2nAAAAAAQO1Q4dB41113xSOPPBINGjSIJ554IjKZTMljmUxGaAQAAACAH6EKh8bzzjsvRo4cGeeee27k5FT4FI8AAAAAwA9QhUvhypUrY/DgwSIjAAAAAFCiwrVw6NChMXHixKqYBQAAAACopSp86PSqVaviiiuuiIcffji23nrrMheDufrqqyttOAAAAACgdqhwaHzttddi2223jYiI119/vdRj37wwDAAAAADw41Hh0Dh9+vSqmAMAAAAAqMVc0QUAAAAASFau0PirX/0qPvjgg3K94MSJE2P8+PFJQwEAAAAAtUu5Dp1u1apVbLnllrHTTjvFfvvtF9ttt120a9cuGjRoEJ9//nm8+eabMXPmzJgwYUK0a9cubrrppqqeGwAAAACoQcoVGi+55JI45ZRT4pZbbonrr78+3nzzzVKPN23aNPr16xc33XRT7LXXXlUyKAAAAABQc5X7YjCtW7eO8847L84777z4/PPP4/33348vvvgiWrZsGV26dHHFaQAAAAD4EavwVacjIjbYYIPYYIMNKnsWAAAAAKCWctVpAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkGy9QuPXX38djz32WNx4442xbNmyiIhYsGBBLF++vFKHAwAAAABqhwpfdfq9996LvfbaK95///0oKiqKPfbYI5o2bRqXX355FBUVxdixY6tiTgAAAACgBqvwHo2nn356bLfddvH5559Hw4YNS5YfeOCBMW3atEodDgAAAACoHSq8R+NTTz0VzzzzTNSvX7/U8o4dO8aHH35YaYMBAAAAALVHhfdoLC4ujlWrVpVZ/sEHH0TTpk0rZSgAAAAAoHapcGjcc889Y8yYMSX3M5lMLF++PEaMGBF77713Zc4GAAAAANQSFT50+qqrror+/ftH9+7d48svv4zDDz88Zs+eHS1btoy77rqrKmYEAAAAAGq4CofG9u3bxyuvvBITJkyIV199NZYvXx7HHXdcHHHEEaUuDgMAAAAA/HhUODRGRNStWzeOPPLIyp4FAAAAAKil1is0LliwIGbOnBmLFi2K4uLiUo+ddtpplTIYAAAAAFB7VDg03nrrrfHLX/4y6tevHy1atIhMJlPyWCaTERoBAAAA4EeowqHxggsuiAsvvDCGDx8eOTkVvmg1AAAAAPADVOHQuGLFihgyZIjICEC5dDz3H9U9ApWseOWXJX/udsFDkVO/QTVOQ2Wbd9k+1T0CAAC1VIVr4XHHHRf33HNPVcwCAAAAANRSFd6jcfTo0bHvvvvGQw89FFtttVXUq1ev1ONXX311pQ0HAAAAANQO6xUaH3744dh8880jIspcDAYAAAAA+PGpcGi86qqr4q9//WscffTRVTAOAAAAAFAbVfgcjbm5ubHTTjtVxSwAAAAAQC1V4dB4+umnx7XXXlsVswAAAAAAtVSFD51+4YUX4vHHH48HH3wwttxyyzIXg7n//vsrbTgAAAAAoHaocGhs1qxZHHTQQVUxCwAAAABQS1U4NI4bN64q5gAAAAAAarEKn6OxMj355JOx3377Rbt27SKTycTkyZNLPX700UdHJpMpddtrr72qZ1gAAAAAYK3KtUfjT3/605g2bVpssMEGse2220Ymk1nrui+99FK5N15YWBjbbLNNHHvssWs9HHuvvfYqtRdlbm5uuV8fAAAAAPh+lCs0HnDAASWB74ADDlhnaKyIAQMGxIABA9a5Tm5ubrRp06ZStgcAAAAAVI1yhcYRI0aU/Pmiiy6qqlnW6IknnogNN9wwNthgg9htt91i1KhR0aJFi+91BgAAAABg3Sp8jsbOnTvH4sWLyyxfsmRJdO7cuVKGWm2vvfaK22+/PaZNmxaXX355zJgxIwYMGBCrVq1a63OKioqioKCg1A0AAAAAqFoVvur0vHnz1hj6ioqK4oMPPqiUoVYbMmRIyZ+32mqr2HrrraNLly7xxBNPxO67777G54wePTpGjhxZqXMAAAAAAOtW7tA4ZcqUkj8//PDDkZ+fX3J/1apVMW3atOjUqVPlTvctnTt3jpYtW8acOXPWGhqHDx8eZ511Vsn9goKC6NChQ5XOBQAAAAA/duUOjQMHDoyIiEwmE0OHDi31WL169aJjx45x1VVXVepw3/bBBx/E4sWLo23btmtdJzc315WpAQAAAOB7Vu7QWFxcHBERnTp1in/961/RsmXL5I0vX7485syZU3J/7ty58fLLL0fz5s2jefPmMXLkyBg0aFC0adMm3nnnnTjnnHOia9eu0b9//+RtAwAAAACVp8LnaJw7d26lbfzFF1+MXXfdteT+6kOehw4dGjfccEO8+uqrcdttt8WSJUuiXbt2seeee8Yll1xij0UAAAAAqGEqHBorU9++fSObza718Ycffvh7nAYAAAAAWF851T0AAAAAAFD7CY0AAAAAQDKhEQAAAABItl6h8Z133onzzz8/DjvssFi0aFFEREydOjXeeOONSh0OAAAAAKgdKhwaZ8yYEVtttVU8//zzcf/998fy5csjIuKVV16JESNGVPqAAAAAAEDNV+HQeO6558aoUaPi0Ucfjfr165cs32233eK5556r1OEAAAAAgNqhwqHxtddeiwMPPLDM8g033DA+/fTTShkKAAAAAKhdKhwamzVrFgsXLiyz/N///ndstNFGlTIUAAAAAFC7VDg0DhkyJH7729/GRx99FJlMJoqLi+Ppp5+OYcOGxS9+8YuqmBEAAAAAqOEqHBovvfTS2GKLLaJDhw6xfPny6N69e/Tu3Tt23HHHOP/886tiRgAAAACghqtb0SfUr18/br755rjwwgvjtddei+XLl8e2224bm266aVXMBwAAAADUAhUOjat16NAhOnToUJmzAAAAAAC1VIUPnR40aFBcfvnlZZZfccUVccghh1TKUAAAAABA7VLh0Pjkk0/G3nvvXWb5gAED4sknn6yUoQAAAACA2qXCoXH58uVRv379Msvr1asXBQUFlTIUAAAAAFC7VDg0brXVVjFx4sQyyydMmBDdu3evlKEAAAAAgNqlwheDueCCC+Kggw6Kd955J3bbbbeIiJg2bVrcddddcc8991T6gAAAAABAzVfh0LjffvvF5MmT49JLL4177703GjZsGFtvvXU89thj0adPn6qYEQAAAACo4SocGiMi9tlnn9hnn30qexYAAAAAoJZar9AYEbFy5cpYtGhRFBcXl1q+8cYbJw8FAAAAANQuFQ6Ns2fPjmOPPTaeeeaZUsuz2WxkMplYtWpVpQ0HAAAAANQOFQ6NRx99dNStWzcefPDBaNu2bWQymaqYCwAAAACoRSocGl9++eWYNWtWbLHFFlUxDwAAAABQC+VU9Andu3ePTz/9tCpmAQAAAABqqQqHxssvvzzOOeeceOKJJ2Lx4sVRUFBQ6gYAAAAA/PhU+NDpfv36RUTE7rvvXmq5i8EAAAAAwI9XhUPj9OnTq2IOAAAAAKAWq3Bo7NOnT1XMAQAAAADUYhU+R2NExFNPPRVHHnlk7LjjjvHhhx9GRMQdd9wRM2fOrNThAAAAAIDaocKh8b777ov+/ftHw4YN46WXXoqioqKIiFi6dGlceumllT4gAAAAAFDzVTg0jho1KsaOHRs333xz1KtXr2T5TjvtFC+99FKlDgcAAAAA1A4VDo1vvfVW9O7du8zy/Pz8WLJkSWXMBAAAAADUMhUOjW3atIk5c+aUWT5z5szo3LlzpQwFAAAAANQuFQ6NJ5xwQpx++unx/PPPRyaTiQULFsT48eNj2LBhcdJJJ1XFjAAAAABADVe3ok8499xzo7i4OHbfffdYsWJF9O7dO3Jzc2PYsGFx6qmnVsWMAAAAAEANV6HQuGrVqnj66afj5JNPjt/85jcxZ86cWL58eXTv3j2aNGlSVTMCAAAAADVchUJjnTp1Ys8994z//Oc/0axZs+jevXtVzQUAAAAA1CIVPkdjjx494t13362KWQAAAACAWqrCoXHUqFExbNiwePDBB2PhwoVRUFBQ6gYAAAAA/PhU+GIwe++9d0RE7L///pHJZEqWZ7PZyGQysWrVqsqbDgAAAACoFSocGqdPn14VcwAAAAAAtViFQ2OfPn2qYg4AAAAAoBar8DkaIyKeeuqpOPLII2PHHXeMDz/8MCIi7rjjjpg5c2alDgcAAAAA1A4VDo333Xdf9O/fPxo2bBgvvfRSFBUVRUTE0qVL49JLL630AQEAAACAmm+9rjo9duzYuPnmm6NevXoly3faaad46aWXKnU4AAAAAKB2qHBofOutt6J3795llufn58eSJUsqYyYAAAAAoJapcGhs06ZNzJkzp8zymTNnRufOnStlKAAAAACgdqlwaDzhhBPi9NNPj+effz4ymUwsWLAgxo8fH8OGDYuTTjqpKmYEAAAAAGq4uhV9wrnnnhvFxcWx++67x4oVK6J3796Rm5sbw4YNi1NPPbUqZgQAAAAAarhyhcZXX301evToETk5OZHJZOK8886L3/zmNzFnzpxYvnx5dO/ePZo0aVLVswIAAAAANVS5Dp3edttt49NPP42IiM6dO8fixYujfv360b179/j5z38uMgIAAADAj1y5QmOzZs1i7ty5ERExb968KC4urtKhAAAAAIDapVyHTg8aNCj69OkTbdu2jUwmE9ttt13UqVNnjeu+++67lTogAAAAAFDzlSs03nTTTXHQQQfFnDlz4rTTTosTTjghmjZtWtWzAQAAAAC1RLmvOr3XXntFRMSsWbPi9NNPFxoBAAAAgBLlDo2rjRs3rirmAAAAAABqsQqHxsLCwrjsssti2rRpsWjRojIXhnGORgAAAAD48alwaDz++ONjxowZcdRRR5VcHAYAAAAA+HGrcGicOnVq/OMf/4iddtqpKuYBAAAAAGqhnIo+YYMNNojmzZtXxSwAAAAAQC1V4dB4ySWXxIUXXhgrVqyoinkAAAAAgFqowodOX3XVVfHOO+9E69ato2PHjlGvXr1Sj7/00kuVNhwAAAAAUDtUODQOHDiwCsYAAAAAAGqzCofGESNGVMUcAAAAAEAtVuFzNAIAAAAAfFu592jcYIMNIpPJfOd6n332WdJAAAAAAEDtU+7QOGbMmCocAwAAAACozcodGocOHVqVcwAAAAAAtZhzNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkKzcV51e7ayzzlrj8kwmEw0aNIiuXbvGAQccEM2bN08eDgAAAACoHSocGv/973/HSy+9FKtWrYrNN988IiLefvvtqFOnTmyxxRZx/fXXx9lnnx0zZ86M7t27V/rAAAAAAEDNU+FDpw844IDo169fLFiwIGbNmhWzZs2KDz74IPbYY4847LDD4sMPP4zevXvHmWeeWRXzAgAAAAA1UIVD45VXXhmXXHJJ5OXllSzLz8+Piy66KK644opo1KhRXHjhhTFr1qxKHRQAAAAAqLkqHBqXLl0aixYtKrP8k08+iYKCgoiIaNasWaxcuTJ9OgAAAACgVlivQ6ePPfbYmDRpUnzwwQfxwQcfxKRJk+K4446LgQMHRkTECy+8EJtttlllzwoAAABQKxUWFkYmk4lMJhOFhYXVPQ5UiQpfDObGG2+MM888M4YMGRJff/31/16kbt0YOnRoXHPNNRERscUWW8Qtt9xSuZMCAAAAADVWhUNjkyZN4uabb45rrrkm3n333YiI6Ny5czRp0qRknZ/85CeVNiAAAAAAUPNV+NDpv/3tb7FixYpo0qRJbL311rH11luXiowAAAAAwI9PhUPjmWeeGRtuuGEcfvjh8c9//jNWrVpVFXMBAAAAALVIhUPjwoULY8KECZHJZOLQQw+Ntm3bxsknnxzPPPNMVcwHAAAAANQCFQ6NdevWjX333TfGjx8fixYtimuuuSbmzZsXu+66a3Tp0qUqZgQAAAAAargKXwzmmxo1ahT9+/ePzz//PN577734z3/+U1lzAQAAAAC1SIX3aIyIWLFiRYwfPz723nvv2GijjWLMmDFx4IEHxhtvvFHZ8wEAAAAAtUCF92gcMmRIPPjgg9GoUaM49NBD44ILLohevXpVxWwAAAAAQC1R4dBYp06duPvuu6N///5Rp06dUo+9/vrr0aNHj0obDgAAAACoHSocGsePH1/q/rJly+Kuu+6KW265JWbNmhWrVq2qtOEAAAAAgNphvS8G8+STT8Zf/vKXuO+++6Jdu3Zx0EEHxXXXXVeZswEAQJXqeO4/qnsEKlHxyi9L/tztgocip36DapyGqjDvsn2qewQA1qFCofGjjz6KW2+9Nf7yl79EQUFBHHrooVFUVBSTJ0+O7t27V9WMAAAAAEANV+6rTu+3336x+eabx6uvvhpjxoyJBQsWxLXXXluVswEAAAAAtUS592icOnVqnHbaaXHSSSfFpptuWpUzAQAAAAC1TLn3aJw5c2YsW7YsevbsGdtvv338+c9/jk8//bQqZwMAAAAAaolyh8Yddtghbr755li4cGH88pe/jAkTJkS7du2iuLg4Hn300Vi2bFlVzgkAAAAA1GDlDo2rNW7cOI499tiYOXNmvPbaa3H22WfHZZddFhtuuGHsv//+VTEjAAAAAFDDVTg0ftPmm28eV1xxRXzwwQdx1113VdZMAAAAAEAtkxQaV6tTp04MHDgwpkyZUhkvBwAAAADUMpUSGgEAAACAHzehEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJqjU0Pvnkk7HffvtFu3btIpPJxOTJk0s9ns1m48ILL4y2bdtGw4YNo1+/fjF79uzqGRYAAAAAWKtqDY2FhYWxzTbbxHXXXbfGx6+44or405/+FGPHjo3nn38+GjduHP37948vv/zye54UAAAAAFiXutW58QEDBsSAAQPW+Fg2m40xY8bE+eefHwcccEBERNx+++3RunXrmDx5cgwZMuT7HBUAAAAAWIcae47GuXPnxkcffRT9+vUrWZafnx/bb799PPvss9U4GQAAAADwbdW6R+O6fPTRRxER0bp161LLW7duXfLYmhQVFUVRUVHJ/YKCgqoZEAAAAAAoUWP3aFxfo0ePjvz8/JJbhw4dqnskAAAAAPjBq7GhsU2bNhER8fHHH5da/vHHH5c8tibDhw+PpUuXltzmz59fpXMCAAAAADU4NHbq1CnatGkT06ZNK1lWUFAQzz//fPTq1Wutz8vNzY28vLxSNwAAAACgalXrORqXL18ec+bMKbk/d+7cePnll6N58+ax8cYbxxlnnBGjRo2KTTfdNDp16hQXXHBBtGvXLgYOHFh9QwMAAAAAZVRraHzxxRdj1113Lbl/1llnRUTE0KFD49Zbb41zzjknCgsL48QTT4wlS5bEzjvvHA899FA0aNCgukYGAAAAANagWkNj3759I5vNrvXxTCYTF198cVx88cXf41QAAAAAQEXV2HM0AgAAAAC1h9AIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGR1q3sAAKB2yanfIDb57YPVPQYAAFDD2KMRAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyWp0aLzooosik8mUum2xxRbVPRYAAAAA8C11q3uA77LlllvGY489VnK/bt0aPzIAAAAA/OjU+GpXt27daNOmTXWPAQAAAACsQ40+dDoiYvbs2dGuXbvo3LlzHHHEEfH++++vc/2ioqIoKCgodQMAAAAAqlaNDo3bb7993HrrrfHQQw/FDTfcEHPnzo1ddtklli1bttbnjB49OvLz80tuHTp0+B4nBgAAAIAfpxodGgcMGBCHHHJIbL311tG/f//45z//GUuWLIm77757rc8ZPnx4LF26tOQ2f/7873FiAAAAAPhxqvHnaPymZs2axWabbRZz5sxZ6zq5ubmRm5v7PU4FAAAAANToPRq/bfny5fHOO+9E27Ztq3sUAAAAAOAbanRoHDZsWMyYMSPmzZsXzzzzTBx44IFRp06dOOyww6p7NAAAAADgG2r0odMffPBBHHbYYbF48eJo1apV7LzzzvHcc89Fq1atqns0AAAAAOAbanRonDBhQnWPAAAAAACUQ40+dBoAAAAAqB2ERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJLVre4BAAAAgNI6nvuP6h6BSla88suSP3e74KHIqd+gGqehKsy7bJ/qHqHa2aMRAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMnqVvcAAAAAlSGnfoPY5LcPVvcYAPCjZY9GAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJCsVoTG6667Ljp27BgNGjSI7bffPl544YXqHgkAAAAA+IYaHxonTpwYZ511VowYMSJeeuml2GabbaJ///6xaNGi6h4NAAAAAPg/NT40Xn311XHCCSfEMcccE927d4+xY8dGo0aN4q9//Wt1jwYAAAAA/J8aHRpXrlwZs2bNin79+pUsy8nJiX79+sWzzz67xucUFRVFQUFBqRsAAAAAULUy2Ww2W91DrM2CBQtio402imeeeSZ69epVsvycc86JGTNmxPPPP1/mORdddFGMHDmyzPKlS5dGXl5elc4LAAAAsCaFhYXRpEmTiIhYvnx5NG7cuJongvIrKCiI/Pz87+xrNXqPxvUxfPjwWLp0aclt/vz51T0SAAAAAPzg1a3uAdalZcuWUadOnfj4449LLf/444+jTZs2a3xObm5u5Obmfh/jAQAAAAD/p0bv0Vi/fv3o2bNnTJs2rWRZcXFxTJs2rdSh1AAAAABA9arRezRGRJx11lkxdOjQ2G677eLnP/95jBkzJgoLC+OYY46p7tEAAAAAgP9T40Pj4MGD45NPPokLL7wwPvroo/jJT34SDz30ULRu3bq6RwMAAAAA/k+Nvup0ZSjvVXEAAAAAqoqrTlOb/WivOg0AAAAAfP+ERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACS1a3uAQAAAAB+6Bo3bhzZbLa6x4AqZY9GAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJhEYAAAAAIJnQCAAAAAAkExoBAAAAgGRCIwAAAACQTGgEAAAAAJIJjQAAAABAMqERAAAAAEgmNAIAAAAAyYRGAAAAACCZ0AgAAAAAJBMaAQAAAIBkQiMAAAAAkExoBAAAAACSCY0AAAAAQDKhEQAAAABIJjQCAAAAAMmERgAAAAAgmdAIAAAAACQTGgEAAACAZEIjAAAAAJBMaAQAAAAAkgmNAAAAAEAyoREAAAAASCY0AgAAAADJ6lb3AFUtm81GRERBQUE1TwIAAAAAtc/qrra6s63NDz40Llu2LCIiOnToUM2TAAAAAEDttWzZssjPz1/r45nsd6XIWq64uDgWLFgQTZs2jUwmU93jQLkVFBREhw4dYv78+ZGXl1fd4wCU4jMKqKl8PgE1mc8oaqtsNhvLli2Ldu3aRU7O2s/E+IPfozEnJyfat29f3WPAesvLy/MfIKDG8hkF1FQ+n4CazGcUtdG69mRczcVgAAAAAIBkQiMAAAAAkExohBoqNzc3RowYEbm5udU9CkAZPqOAmsrnE1CT+Yzih+4HfzEYAAAAAKDq2aMRAAAAAEgmNAIAAAAAyYRGAAAAACCZ0Ai1wK233hrNmjWr7jEAANbK/69UrXnz5kUmk4mXX365yrbhewhAKqGRH6Wjjz46MplMZDKZqF+/fnTt2jUuvvji+Prrr7/zubfeemvJc9d2mzdvXtV/EeV03333Rd++fSM/Pz+aNGkSW2+9dVx88cXx2WefRUTprycnJyfat28fxxxzTCxatCgi1v0/tX379o0zzjjje/xqgPnz58exxx4b7dq1i/r168cmm2wSp59+eixevLjUen379o1MJhMTJkwotXzMmDHRsWPHkvurPwP22muvUustWbIkMplMPPHEExER8corr0T9+vVjypQppda77777okGDBvH666+vc26fRVBx/n/l+/uM+Oyzz+KMM86ITTbZJOrXrx/t2rWLY489Nt5///3K/lLL5eijj46BAweWWtahQ4dYuHBh9OjRo1pmAvAPHpSH0MiP1l577RULFy6M2bNnx9lnnx0XXXRRXHnlld/5vMGDB8fChQtLbr169YoTTjih1LIOHTqUe46VK1emfBnrdN5558XgwYPjZz/7WUydOjVef/31uOqqq+KVV16JO+64o2S9vLy8WLhwYXzwwQdx8803x9SpU+Ooo46qsrmA9fPuu+/GdtttF7Nnz4677ror5syZE2PHjo1p06ZFr169Sv5CvlqDBg3i/PPPj6+++mqdr1u3bt147LHHYvr06WtdZ5tttokLL7wwTjzxxJKouWjRovjVr34VI0eOXOdffH0Wwfrz/ytV/xnx2WefxQ477BCPPfZYjB07NubMmRMTJkyIOXPmxM9+9rN49913k7dRGerUqRNt2rSJunXrVvcosE7+UfTlMq/tHzz4MREa+dHKzc2NNm3axCabbBInnXRS9OvXL6ZMmRKFhYWRl5cX9/6/9u49KKq6jQP4FwhmYRfUkAS5rYokhKhlURowWoRM5C2VEpXxMjmDNyogHWYSHcFmREfRTCcNQTGxF8wZU8mlAicMGnIHCgwQEErJ9YIOYqLwvH/4emK5LAJab/r9zOwMe87v/M5vD/Ds7zzP2bP/+Y9R+y+//BJqtRp37tyBo6Oj8rCysoKNjY3yvLm5GdOnT4dGo4GdnR1mzZqFP/74Q+knPj4eo0ePxq5duzBkyBCoVCoAd98oFy9ejEGDBkGlUsHHxwdHjhwxGkN2dja8vLyg0WiUE4+uFBYWIjExERs3bsSGDRswbtw4aLVaBAUFITMzExEREUpbMzMzODo6YvDgwQgJCcHy5cuh0+lw8+bNB3GoiegBWbJkCaysrPD1118jMDAQbm5uCAkJgU6nw++//464uDij9m+//TYaGhrw6aefmuxXrVZjwYIFWLlypcl2q1atgpubG5YsWQIAWLx4MYYPH47o6Ogut2EsIuobzlcefoyIi4vD+fPnodPpEBISAjc3NwQEBCA7OxuWlpZKzAMArVaLzZs3G20/evRoxMfHK883bdqEkSNHQq1Ww9XVFZGRkWhsbFTW37tap6vjFB8fj9TUVBw+fFhJanz33Xcdkhhtr3ht+7iXdLl16xaio6Ph7OwMtVoNPz8/ZV3bsbi5ucHGxgbTpk3rkAgi6ikWRXuHBQ96lDDRSPQ/1tbWaG5uhlqtxltvvYWUlBSj9SkpKZgxYwZsbW277KO1tRVTpkzBlStXkJubixMnTqCqqgphYWFG7SorK5GZmYmsrCzo9Xq0trYiJCQE33//Pfbt24fS0lJ89NFHsLCwULZpampCUlIS9u7di7y8PNTW1po8uU9PT4dGo0FkZGSn601djm5tbY3W1tb7+mgWEf09rly5guzsbERGRsLa2tponaOjI8LDw5GRkQERUZbb2dkhLi4Oa9euxY0bN0z2Hx8fj5KSkg5Ji7YsLCyUk9/Zs2cjOzsbe/bsMYpV7TEWET1YnK8YH4u+xojW1lYcOHAA4eHhcHR07NB/ZGQksrOzOyRHTDE3N0dycjJ++eUXpKam4ptvvkFsbKxRG1PHKTo6GrNmzVKSjxcuXMC4ceM67GfLli1GV6iuWLECTz31FEaMGAEAWLp0KU6dOoUDBw6guLgYM2fOxKRJk1BRUQEAKCgowMKFC7F06VLo9XpMmDAB69at69HxI2qPRdHeYcGDBY9HCRON9NgTEeh0OmRnZ2PixIkAgEWLFiE7O1sJtBcvXsTRo0exYMECk33l5OSgpKQE+/fvx3PPPQc/Pz+kpaUhNzcXP/74o9KuubkZaWlpGDNmDHx9faHT6VBYWIisrCwEBQVh6NChCA0NRUhIiLLN7du3sWPHDowdOxbPPvssli5dipycnC7HUlFRgaFDh8LS0rJHx6OiokLZj6mTFCL6e1VUVEBE4OXl1el6Ly8vXL16FQaDwWh5ZGQkVCoVNm3aZLL/wYMHY8WKFYiLizN50u7l5YWoqCh8/vnniI+Ph6enZ7fjZiwi6jvOVzpu9yBihMFgQENDg8nYKiKorKy87z6joqIwYcIEaLVaTJw4EevWrcPBgweN2pg6ThqNBtbW1srVrPeuSG2vX79+yvr8/Hzs3LkTWVlZcHR0RG1tLVJSUvDFF1/A398fw4YNQ3R0NF5++WUlOb1lyxZMmjQJsbGx8PT0xPLlyxEcHHzfr5OoPRZFe4cFDxY8HjVMNNJj68iRI9BoNFCpVAgJCUFYWJhSBXrhhRfwzDPPIDU1FQCwb98+uLu7IyAgwGSfZWVlcHV1Nbrnkbe3N/r374+ysjJlmbu7OxwcHJTner0eLi4uJk/YbWxsMGzYMOW5k5OTcj+QzrR9A+/OtWvXoNFoYGNjg6effhqDBg1Cenr6fW9PRH+fnvxvA3c/drl27VokJSXh0qVLJtt+8MEHMBgM+Oyzz7ps09jYiIyMDNjY2ODkyZMPdLyMRUQdcb7yl4cZI7obR2eJvq7odDq88sorcHZ2hq2tLebOnYvLly+jqalJadPT42TK6dOnMXfuXGzbtg3jx48HAJSUlKClpQWenp7QaDTKIzc3F2fPngVw9+/Az8/PqK+XXnqpV2MgAlgU7S0WPFjweNQw0UiPrQkTJkCv16OiogI3b95Eamoq1Gq1sn7RokXYs2cPgLsfQ5o/fz7MzMweyL7b7gdAh4pfZ9q/8ZmZmZmcFHt6eqKqqqrb+50AgK2tLfR6PX7++WfcuHEDeXl5yhuynZ0dgLuT+/YaGhrQr1+/bvsnor7z8PCAmZmZURKgrbKyMgwYMMAoKXDPnDlz4O7u3m2FuH///li1ahXWrFljdELcVkxMDFQqFfLz86HT6ZCWlmayT8Yior7hfOUvDyNGODg4dEiwtlVWVoYnnngCQ4YMAXD3KqH2r6ft2GtqahAaGgpfX19kZmaiqKgIH3/8MQDjL9Tp6XHqSn19PSZPnoxFixZh4cKFyvLGxkZYWFigqKgIer1eeZSVlWHLli093g9RT7Ao2jsseNzFgse/HxON9NhSq9Xw8PCAm5tbpzeznTNnDs6dO4fk5GSUlpYa3ZujK15eXqirq0NdXZ2yrLS0FA0NDfD29u5yO19fX/z2228oLy/v3YvpxOzZs9HY2Ijt27d3ur6hoUH52dzcHB4eHhg6dGiHk4gnn3wSAwcORFFRkdHy69evo7KystsKIRE9GPb29ggKCsL27ds73Aeovr4e6enpCAsL6zTBYG5ujvXr1+OTTz5BTU2Nyf0sW7YM5ubmnZ6InjhxArt27UJqaipGjRqFdevWISoqyuQXPTAWEfUN5ysNys8PI0aYm5tj1qxZ2L9/P+rr643W3bx5E9u3b8e0adOURKWDg4NRzLt+/Tqqq6uV50VFRWhtbcXGjRvx4osvwtPTE+fPn+/+QLRjZWWFlpYWk23+/PNPTJkyBSNGjOhwJdiYMWPQ0tKCixcvwsPDw+hx76OZXl5eKCgoMNruhx9+6PFYie5hUZQFDxY8CGCikahLAwYMwPTp0xETE4PXXnsNLi4u3W7z6quvYuTIkQgPD8dPP/2EwsJCzJs3D4GBgRg7dmyX2wUGBiIgIABvvvkmTpw4gerqahw7dgzHjx/v9fj9/PwQGxuL999/H7GxsTh16hTOnTuHnJwczJw5U/mY1f147733kJiYiPT0dJw9exaFhYUIDw+Hg4MDpk+f3usxElHPbNu2Dbdu3UJwcDDy8vJQV1eH48ePIygoCM7OzkhISOhy29dffx1+fn7YuXOnyX2oVCqsWbMGycnJRsuvX7+OhQsXIiYmBs8//zwA4N1334W3tzfeeeedLvtjLCJ6uDhf+UtvY0RCQgIcHR0RFBSEY8eOoa6uDnl5eQgODu5QeJk4cSL27t2LkydPoqSkBBEREUb3fvPw8MDt27exdetWVFVVYe/evdixY0ePj4tWq0VxcTF+/fVXXLp0qdMEyOLFi1FXV4fk5GQYDAbU19ejvr4ezc3N8PT0RHh4OObNm4esrCxUV1ejsLAQ69evx1dffQUAWL58OY4fP46kpCRUVFRg27ZtffpdErEoyoJHWyx4PMaE6DEUEREhU6ZM6bZdTk6OAJCDBw922SYwMFBWrFihPD937pxMnjxZ1Gq12NraysyZM6W+vl5Zv3r1ahk1alSHfi5fvizz588Xe3t7UalU4uPjI0eOHBERkZSUFOnXr59R+0OHDsn9/AtnZGRIQECA2NrailqtFl9fX1m7dq1cvXq1y77bu3PnjiQnJ8vIkSPFxsZGXFxcJCwsTKqrq7vdPxE9WDU1NRIRESGDBg0SS0tLcXV1lWXLlsmlS5eM2rWPTSIi+fn5AkDc3d2VZZ3FgDt37oi3t7cAkG+//VZERObPny8+Pj5y69Yto7bl5eViY2MjqampJsfNWETUc5yv/H0xwmAwyLJly8TV1VUsLCwEgIwbN04uX75s1O7atWsSFhYmdnZ24urqKnv27JFRo0bJ6tWrlTabNm0SJycnsba2luDgYElLSxMAJl9L++N08eJFCQoKEo1Go8Ti6upqASCnT58WERF3d3cB0OFxL243NzfLhx9+KFqtViwtLcXJyUmmTZsmxcXFyn52794tLi4uYm1tLW+88YYkJSV1e5yJTCkvL5eBAweKv7+/5ObmSm1trRw7dkx8fHxk+PDhRv9Tnc1V/P39RaVSdTtX2b17t6hUKqO/+WvXromrq6usWrVKadfS0iLjx4+X0NBQk+OOjY0VCwsLiYmJkfz8fKmpqRGdTiczZsyQzZs3dzmO9hITE8Xe3l727dsnlZWVUlBQIKGhoaLVaqWpqanL7QwGgwwbNkx8fHzk6NGjUltbK7m5ueLv7y+urq5y/vx5pe3KlSvF0dFR8vLypLi4WKZOnSoajUaJQ3q9XgDI5s2b5ezZs5KWlibOzs49jkMJCQni5uYmZ86cEYPBIM3NzR3i0Lx588TJyUlKS0vlwoULyuPefDE8PFy0Wq1kZmZKVVWVFBQUSGJiovK+cerUKTE3N5cNGzZIeXm5bN26Vfr378849C/HRCORCWlpaWJvb9/hxJqIiIjo/wXnKw/erl27xMrKSg4dOvRPD4XoX4dFURY8WPB4vJmJ9OJD+ESPuKamJly4cAGTJ0/G1KlTTX4ckYiIiOifwPnKw3Xo0CGcOXMGUVFR9/VFOERED9Lu3bsRGRmJjIwMTJ069Z8eDtF9Y6KRqBPx8fFISEhAQEAADh8+DI1G808PiYiIiMgI5ytERI82Fjzo34iJRiIiIiIiIiIiIuozfus0ERERERERERER9RkTjURERERERERERNRnTDQSERERERERERFRnzHRSERERERERERERH3GRCMRERERERERERH1GRONRERERERERERE1GdMNBIREREREREREVGfMdFIREREREREREREfcZEIxEREREREREREfXZfwH+3hof9wmK1wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1600x1200 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "\n",
    "\n",
    "# Compute average inference time + std\n",
    "time_results = {k: np.mean(v.model_inference_time) * 1e3 for k, v in results.items()}\n",
    "time_results_std = np.std([v.model_inference_time for v in results.values()]) * 1000\n",
    "\n",
    "plt.rcdefaults()\n",
    "fig, ax = plt.subplots(figsize=(16, 12))\n",
    "ax.set_ylabel(\"Avg Inference time (ms)\")\n",
    "ax.set_title(\"Average inference time (ms) for each provider\")\n",
    "ax.bar(time_results.keys(), time_results.values(), yerr=time_results_std)\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "ONNX Overview",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}