03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get a good answer for the user query.\n",
    "\n",
    "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from typing import List\n",
    "from operator import itemgetter\n",
    "\n",
    "# LangChain Imports needed\n",
    "from langchain_openai import AzureChatOpenAI\n",
    "from langchain_openai import AzureOpenAIEmbeddings\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.retrievers import BaseRetriever\n",
    "from langchain_core.callbacks import CallbackManagerForRetrieverRun\n",
    "from langchain_core.documents import Document\n",
    "from langchain_core.runnables import ConfigurableField\n",
    "\n",
    "\n",
    "# Our own libraries needed\n",
    "from common.prompts import DOCSEARCH_PROMPT\n",
    "from common.utils import get_search_results\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"srch-index-files\"\n",
    "index2_name = \"srch-index-csv\"\n",
    "indexes = [index2_name, index1_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "The idea is that the answers using Azure OpenAI only looks at the information contained on these publications.\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- What is CLP?\n",
    "- How Markov chains work?\n",
    "- What are some examples of reinforcement learning?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"What medicine reduces inflamation in the lungs?\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 6 mandatory fields present on each index**: `id, title, name, location, chunk, chunkVector`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**.\n",
    "\n",
    "We are going to use Hybrid Queries: Text + Vector Search combined for optimal results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: srch-index-csv Results Found: 70150, Results Returned: 10\n",
      "200\n",
      "Index: srch-index-files Results Found: 118846, Results Returned: 10\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "k = 10\n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION, # Text query\n",
    "        \"select\": \"id, title, name, location, chunk\",\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"vectorQueries\": [{\"text\": QUESTION, \"fields\": \"chunkVector\", \"kind\": \"text\", \"k\": k, \n",
    "                           \"threshold\": { \n",
    "                                 \"kind\": \"vectorSimilarity\", \n",
    "                                 \"value\": 0.8 \n",
    "                              }\n",
    "                          }], # Vector query\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"count\":\"true\",\n",
    "        \"top\": k\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f33018be-350d-4c54-b491-a86bc1cfffb6",
   "metadata": {},
   "source": [
    "#### Important Note: \n",
    "You may encounter errors when attempting to search for results IF the indexer is still processing documents. This occurs because the embedding model is heavily utilized by the indexer, hitting its TPM quota. If you experience search errors, please try again or wait until the indexing is complete, which may take several hours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "255c40f5-d836-480c-8c68-06a2282c8146",
   "metadata": {},
   "outputs": [],
   "source": [
    "# agg_search_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5>Answer - score: 0.92</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000 This cytokine is strongly regulated by the soluble receptor IL-22-binding protein (IL-22BP), ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Targeting the IL-22/IL-22BP axis enhances tight junctions and reduces inflammation during influenza infection</a> - score: 3.42</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Importantly, we show that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung Taken together, our results demonstrate the IL-22/IL-22BP axis is a potential targetable pathway for reducing influenza-induced pneumonia..\u0000 This cytokine is strongly regulated by the soluble receptor IL-22-binding protein (IL-22BP), ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Severe Acute Lung Injury Related to COVID-19 Infection: A Review and the Possible Role for Escin.</a> - score: 3.07</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "or moderate effect on the lung injury In fact, ALI seems to be related to the inflammatory burst and release of proinflammatory mediators that induce intra-alveolar fibrin accumulation that reduces the gas exchange Therefore, an add-on therapy with drugs able to reduce inflammation, edema, and cell activation has been proposed as well as a treatmen..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1055/s-0033-1351119; https://www.ncbi.nlm.nih.gov/pubmed/23934717/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Treatment of acute lung injury: current and emerging pharmacological therapies.</a> - score: 3.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Treatment of acute lung injury: current and emerging pharmacological therapies.. As a syndrome of injurious pulmonary inflammation resulting in deranged respiratory physiology, acute lung injury affords numerous potential therapeutic targets. Two main pharmacological treatment strategies have arisen-the attempted inhibition of excessive inflammation or the manipulation of the resulting physiological derangement causing respiratory failure. Additionally, such interventions may allow the delivery of less injurious mechanical ventilation. An emerging approach is the use of cell-based therapy, which, rather than inhibiting the inflammatory process, seeks to convert it from an injurious process to a reparative one. This review outlines previous, current, and emerging pharmacological therapies for acute lung injury.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Low doses of radiation therapy increase the immunosuppressive profile of lung macrophages via IL-10 production and IFNγ/IL-6 suppression: a therapeutic strategy to counteract lung inflammation?</a> - score: 2.85</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The objective of this study was to determine the mechanism by which low doses of radiation therapy (RT) protect the lung from inflammation Nerve- and airway-associated macrophages (NAMs) and the pro-versus anti-inflammatory cytokine balance (IL6-IFNγ/IL-10) were recently shown to regulate lung inflammation Here, we used Toll-like receptor 3 (TLR3) ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Therapeutic Applications</a> - score: 2.62</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "For instance, glucocorticoids are considered the most effective anti-inflammatory therapies for chronic inflammatory and immune diseases, such as asthma But they are relatively ineffective in asthmatic smokers, and patients with chronic obstructive pulmonary disease (COPD) or cystic fibrosis (CF) As such, the pharmaceutical industry is exploring ne..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1021/mp070048k; https://www.ncbi.nlm.nih.gov/pubmed/18491918/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Inhalable siRNA: potential as a therapeutic agent in the lungs.</a> - score: 2.42</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "RNAi is a naturally occurring gene regulatory mechanism, which has a number of advantages over other gene/antisense therapies including specificity of inhibition, potency, the small size of the molecules and the diminished risk of toxic effects, e.g., immune responses Targeted, local delivery of RNAi to the lungs via inhalation offers a unique oppo..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2892358/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Using Complementary and Alternative Medicines to Target the Host Response during Severe Influenza</a> - score: 2.18</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Biochanin A, a ligand of the peroxisome proliferator activated receptors (PPAR) alpha and gamma and the active isoflavone in Trifolium pratense (red clover), has anti-inflammatory properties, and thus could be used as an influenza treatment This is of great interest since we have recently shown that gemfibrozil, a drug used to treat hyperlipidemia ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1002/ddr.21676; https://www.ncbi.nlm.nih.gov/pubmed/32337769/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Noscapine, a possible drug candidate for attenuation of cytokine release associated with SARS-CoV-2.</a> - score: 1.98</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Noscapine, a medication used for the treatment of cough, has been shown to inhibit bradykinin enhanced cough response in man As it is already marketed in a number of countries as a cough medicine, even for children, a suitable formulation with all the required licenses is available that can be rapidly utilized in preliminary trials..\u0000 Inactivation ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/15339574/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">[Development of the integrated TCM hepatology with western medicine].</a> - score: 1.97</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The treatment for severe acute respiratory syndrome (SARS) that happened epidemically this spring confirmed again that the integrated traditional Chinese and western medicine had its clinical advantage and vitality, and indicated that this integration could benefit the pathologic understanding and therapeutic strategies for the diseases Based on th..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pubmed/18830453/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">An Evaluation of the Additive Effect of Natural Herbal Medicine on SARS or SARS-like Infectious Diseases in 2003: A Randomized, Double-blind, and Controlled Pilot Study.</a> - score: 1.85</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "to the severity of infiltration in each lung field (three lung fields in both right and left lungs) The main outcome measurements were the improving chest radiographic scores (IRS) and the duration (days) till improvement (DI) One patient from the placebo group passed away Patients from NHM A took less days before showing improvement (6.7 +/- 1.8) ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0701/0701074v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">Microsoft Word - H-robustness3.doc</a> - score: 1.32</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "American Journal of   Respiratory And Critical Care Medicine, 171, 417-418 Monastersky, R. (2005) The Number That's Devouring Science The Chronicle of Higher   Education 52, 8, A12, http://chronicle.com/free/v52/i08/08a01201.htm   Oppenheim, C. (2006) Using the h-index to rank influential British researchers in information   science and librariansh..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0207/0207088v2.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">arXiv:cs/0207088v2  [cs.LO]  6 Aug 2002</a> - score: 1.05</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "We extend the analysis developed by Villadsen [40] Three experts in medicine provided information related to the diagnosis of two diseases: disease-1 and disease-2 The information concerning John and Mary can be paraphrased as follows:  — Expert I (a clinician):  Symptom-1 and symptom-2 together imply disease-1 Symptom-1 and symptom-3 together impl..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0207/0207088v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">arXiv:cs/0207088v1  [cs.LO]  25 Jul 2002</a> - score: 1.05</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "We extend the analysis developed by Villadsen [25] Three experts in medicine provided information related to the diagnosis of two diseases: disease-1 and disease-2 The information concerning John and Mary can be paraphrased as follows:  — Expert I (a clinician):  Symptom-1 and symptom-2 together imply disease-1 Symptom-1 and symptom-3 together impl..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0204/0204041v1.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">arXiv:cs/0204041v1  [cs.CR]  18 Apr 2002</a> - score: 1.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "An example of an ontology in a field out-  side of medicine might be the Library of Congress cataloguing system Within  medicine the classification of diseases by the World Health Organisation is such  a universally recognised classification system Other classifications such as those  developed for hospital administration — the HL7 attribute model ..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://datasetsgptsmartsearch.blob.core.windows.net/arxivcs/pdf/0207/0207088v3.pdf?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D\">0207088v3.pdf</a> - score: 1.02</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "than an alternative to works on paraconsistency Higher order logic is not really needed for the case study — first order logic is enough — but the purpose of the case study is mainly to illustrate the working of the paraconsistency Even though the standard foundation of mathematics, pure    4 Jørgen Villadsen  axiomatic set theory, can be stated in..."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunk\": result['chunk'], \n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "### Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic re-ranking feature of Azure AI Search service is good. It gives answers (sometimes) and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**. This is what RAG (Retreival Augmented Generation) is about.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/en/latest/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal:\n",
    "\n",
    "- text-embedding-ada-002 (or newer)\n",
    "- gpt-35-turbo (1106 or newer)\n",
    "- gpt-4-turbo (1106 or newer)\n",
    "\n",
    "Reference for Azure OpenAI models (regions, limits, dimensions, etc): [HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains refer to sequences of calls - whether to an LLM, a tool, or a data preprocessing step.\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {},
   "outputs": [],
   "source": [
    "COMPLETION_TOKENS = 2000\n",
    "llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                      temperature=0, \n",
    "                      max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a3b55adb-6f98-4f15-b67a-9fbba5820560",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_parser = StrOutputParser()\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"You are an assistant that give thorough responses to users.\"),\n",
    "    (\"user\", \"{input}. Give your response in {language}\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6417d052-0035-4635-93e8-2bd3ec50d796",
   "metadata": {},
   "source": [
    "The | symbol is similar to a unix pipe operator, which chains together the different components feeds the output from one component as input into the next component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "77a37e60-a1ef-4750-a1ec-9e4fe5ba07fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "6be6b4df-ee2c-4a0c-8ad3-a672d70f4f8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "El medicamento que se utiliza comúnmente para reducir la inflamación en los pulmones es un corticoesteroide inhalado. Este tipo de medicamento ayuda a disminuir la inflamación en las vías respiratorias y puede ser recetado para tratar condiciones como el asma y la enfermedad pulmonar obstructiva crónica (EPOC). Es importante seguir las indicaciones de un médico para el uso adecuado de este tipo de medicamentos."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 30.1 ms, sys: 7.68 ms, total: 37.7 ms\n",
      "Wall time: 2.03 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "display(Markdown(chain.invoke({\"input\": QUESTION, \"language\": \"Spanish\"})))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the environmental variable set above `os.environ[\"GPT35_DEPLOYMENT_NAME\"]`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many building blocks to help solve a specific task. For example, LangChain supports some end-to-end chains (such as `create_retrieval_chain` for QnA Doc retrieval, Summarization, etc).\n",
    "\n",
    "We will build our own specific chain in this workshop for digging deeper and solve our use case of enhancing the results of Azure AI Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, let's first review the concept of Embeddings and Vector Search and RAG. \n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "### Why Do We Need Vectors?\n",
    "\n",
    "Vectors are essential for several reasons:\n",
    "\n",
    "- **Semantic Richness**: They convert the semantic meaning of text into mathematical vectors, capturing nuances that simple keyword searches miss. This makes them incredibly powerful for understanding and processing language.\n",
    "- **Human-like Searching**: Searching using vector distances mimics the human approach to finding information based on context and meaning, rather than relying solely on exact word matches.\n",
    "- **Efficiency in Scale**: Vector representations allow for efficient handling and searching of large datasets. By reducing complex text to numerical vectors, algorithms can quickly sift through vast amounts of information.\n",
    "\n",
    "### Understanding LLM Tokens' Context Limitation\n",
    "\n",
    "Large Language Models (LLMs) like GPT come with a token limit for each input, which poses a challenge when dealing with lengthy documents or extensive data sets. This limitation restricts the model's ability to understand and generate responses based on the full context of the information provided. It becomes crucial, therefore, to devise strategies that can effectively manage and circumvent this limitation to leverage the full power of LLMs.\n",
    "\n",
    "To address this challenge, the solution incorporates several key steps:\n",
    "\n",
    "1. **Segmenting Documents**: Breaking down large documents into smaller, manageable segments.\n",
    "2. **Vectorization of Chunks**: Converting these segments into vectors, making them compatible with vector-based search techniques.\n",
    "3. **Hybrid Search**: Employing both vector and text search methods to pinpoint the most relevant segments in relation to the query.\n",
    "4. **Optimal Context Provision**: Presenting the LLM with the most pertinent segments, ensuring a balance between detail and brevity to stay within token limits.\n",
    "\n",
    "\n",
    "Our ultimate goal is to rely solely on vector indexes and hybrid searchs (vector + text). While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure AI Search has automated chunking strategies and vectorization**.\n",
    "\n",
    "It's important to note that **document segmentation and vectorization have already been completed in AI Azure Search**, as seen in the `ordered_content` dictionary. This pre-processing step simplifies subsequent operations, ensuring rapid response times and adherence to the token limits of the chosen OpenAI model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "So really, our only job now is to make sure that the results from the Azure AI Search queries fit on the LLM context size, and then let it do its magic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_name = \"srch-index-files\"\n",
    "index2_name = \"srch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later.\n",
    "\n",
    "`get_search_results()` do the multi-index search and returns the combined ordered list of documents/chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 20\n"
     ]
    }
   ],
   "source": [
    "k = 20  # play with this parameter and see the quality of the final answer\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=k, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "235d4238-df6e-40eb-ab38-4fe6db614acd",
   "metadata": {},
   "source": [
    "Now let's create a Prompt Template that will ground the response only in the chunks retrieve by our hybrid AI Search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f86ed786-aca0-4e25-947b-d9cf3a82665c",
   "metadata": {},
   "outputs": [],
   "source": [
    "template = \"\"\"Answer the question thoroughly, based **ONLY** on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "25cba3d1-b5ab-4e28-96b3-ef923d99dc9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on the provided context, the medicine that reduces inflammation in the lungs is IL-22. The context specifically mentions that administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung. IL-22 is part of the IL-22/IL-22BP axis, which is a potential targetable pathway for reducing influenza-induced pneumonia and promoting tight junction formation in the lungs. Therefore, IL-22 is the medicine that has been shown to reduce inflammation in the lungs during influenza infection."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 15.5 ms, sys: 0 ns, total: 15.5 ms\n",
      "Wall time: 2.19 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "# Creation of our custom chain\n",
    "chain = prompt | llm | output_parser\n",
    "\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "406c2b91-6752-4ea2-b95c-d1a52dbdd62b",
   "metadata": {},
   "source": [
    "### From GPT-3.5 to GPT-4\n",
    "\n",
    "Now let's see how the response changes if we change to GPT-4\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f01705f1-7194-452b-a170-c09ba7752b1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_2 = AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], temperature=0.5, max_tokens=COMPLETION_TOKENS)\n",
    "chain = prompt | llm_2 | output_parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "4c83bb17-36d3-4eb6-ae6f-9c68f3033d2b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Based on the provided context, several medicines and treatments have been identified to reduce inflammation in the lungs:\n",
       "\n",
       "1. **Interleukin-22 (IL-22):** Administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lung, demonstrating that the IL-22/IL-22BP axis is a potential pathway for reducing influenza-induced pneumonia.\n",
       "\n",
       "2. **Chloroquine:** In human lung parenchymal explants, a clinically achievable concentration of chloroquine inhibited the lipopolysaccharide-induced release of inflammatory cytokines (TNF-α, IL-6, CCL2, and CCL3) significantly, suggesting its role in mitigating the cytokine storm associated with severe pneumonia caused by coronaviruses.\n",
       "\n",
       "3. **Escin:** Suggested for its potent anti-inflammatory and anti-viral effects in lung injury, indicating it could be a therapeutic opportunity as add-on therapy in Acute Lung Injury (ALI) related to COVID-19 infection.\n",
       "\n",
       "4. **Corticosteroids:** Known for their anti-inflammatory effects, corticosteroids are considered effective in chronic inflammatory and immune diseases like asthma, though their effectiveness varies among different patient groups.\n",
       "\n",
       "5. **Tanshinone IIA (TIIA):** Demonstrated to notably attenuate Bleomycin-induced Pulmonary Fibrosis (PF) and inflammation in rats, indicating it may exert protective effects through the ACE-2/ANG-(1-7) axis.\n",
       "\n",
       "6. **Low doses of radiation therapy (RT):** Shown to increase the immunosuppressive profile of lung macrophages via IL-10 production and IFNγ/IL-6 suppression, suggesting a therapeutic strategy to counteract lung inflammation.\n",
       "\n",
       "7. **Tocilizumab (Interleukin-6 blockade):** Demonstrated to curb the \"cytokine storm,\" prevent ICU admission, and the requirement for mechanical ventilation in highly selected patients with severe COVID-19.\n",
       "\n",
       "8. **Antibiotics (Macrolides and Ketolides):** Indicated for their anti-inflammatory effects on the chronically inflamed airways in addition to their anti-infective action, especially in the context of asthma exacerbated by atypical respiratory pathogens.\n",
       "\n",
       "9. **Noscapine:** Used for the treatment of cough, has been shown to inhibit bradykinin enhanced cough response, indicating a potential role in attenuating cytokine release associated with SARS-CoV-2.\n",
       "\n",
       "These treatments vary in their mechanism of action, from directly targeting cytokines involved in inflammation (like IL-22 and IL-6 blockade) to more broad anti-inflammatory effects (such as corticosteroids and chloroquine). Some are specific to certain conditions (e.g., Escin for COVID-19 related ALI, TIIA for pulmonary fibrosis), while others have a more general application in reducing lung inflammation."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 19.3 ms, sys: 533 µs, total: 19.9 ms\n",
      "Wall time: 15.9 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(chain.invoke({\"question\": QUESTION, \"context\": ordered_results})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92a28869-13c1-463e-8285-9870e6ac1946",
   "metadata": {},
   "source": [
    "#### As we can see, the model selection MATTERS!\n",
    "\n",
    "We will dive deeper into this later, but for now, **look at the diference between GPT3.5 and GPT4, in quality and in response time**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "417925af-486a-40bc-a290-28c35968c581",
   "metadata": {},
   "source": [
    "# Improving the Prompt and adding citations\n",
    "\n",
    "We could see above that the answer given by GPT3.5 was very simple compared to GPT4, even when the prompt says \"thorough responses to users\". We also could see that there is no citations or references. **How do we know if the answer is grounded on the context or not?**\n",
    "\n",
    "Let's see if these two issues can be improved by Prompt Engineering.<br>\n",
    "On `common/prompts.py` we created a prompt called `DOCSEARCH_PROMPT` check it out!\n",
    "\n",
    "**Let's also create a custom Retriever class** so we can plug it in easily within the chain building. \n",
    "Note: we can also use the Azure AI Search retriever class [HERE](https://python.langchain.com/docs/integrations/vectorstores/azuresearch), however we want to create a custom Retriever for the following reasons:\n",
    "\n",
    "1) We want to do multi-index searches in one call\n",
    "2) Easier to teach complex concepts of LangChain in this notebook\n",
    "3) We want to use the REST API vs the Python Azure Search SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "bdf31f99-0dfb-423a-81f5-03018e61d9a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomRetriever(BaseRetriever):\n",
    "    \n",
    "    topK : int\n",
    "    reranker_threshold : int\n",
    "    indexes: List\n",
    "    sas_token: str = None\n",
    "    \n",
    "    def _get_relevant_documents(self, query: str) -> List[Document]:\n",
    "        \n",
    "        ordered_results = get_search_results(query, self.indexes, k=self.topK, \n",
    "                                             reranker_threshold=self.reranker_threshold, \n",
    "                                             sas_token=self.sas_token)\n",
    "        top_docs = []\n",
    "        for key,value in ordered_results.items():\n",
    "            location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "            top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location, \"score\":value[\"score\"]}))\n",
    "\n",
    "        return top_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "19b39c79-c827-4437-b58b-6a6fae53b968",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the retriever\n",
    "retriever = CustomRetriever(indexes=indexes, topK=k, reranker_threshold=1, sas_token=os.environ['BLOB_SAS_TOKEN'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c7aa4f58-4791-40a0-80c5-6582e0574579",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test retreiver\n",
    "results = retriever.invoke(QUESTION)\n",
    "len(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "11b6546f-b5c5-4168-97fc-2636c50e41c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can create now a dynamically configurable llm object that can change the model at runtime\n",
    "dynamic_llm = AzureChatOpenAI(deployment_name=os.environ[\"GPT35_DEPLOYMENT_NAME\"], \n",
    "                              temperature=0.5, max_tokens=COMPLETION_TOKENS).configurable_alternatives(\n",
    "    # This gives this field an id\n",
    "    # When configuring the end runnable, we can then use this id to configure this field\n",
    "    ConfigurableField(id=\"model\"),\n",
    "    # This sets a default_key.\n",
    "    # If we specify this key, the default LLM  (initialized above) will be used\n",
    "    default_key=\"gpt35\",\n",
    "    # This adds a new option, with name `gpt4`\n",
    "    gpt4=AzureChatOpenAI(deployment_name=os.environ[\"GPT4_DEPLOYMENT_NAME\"], \n",
    "                         temperature=0.5, max_tokens=COMPLETION_TOKENS),\n",
    "    gpt4o=AzureChatOpenAI(deployment_name=os.environ[\"GPT4o_DEPLOYMENT_NAME\"], \n",
    "                         temperature=0.5, max_tokens=COMPLETION_TOKENS),\n",
    "    # You can add more configuration options here\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "d7da2f31-cf5d-4f3a-aad5-67b50b56968e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declaration of the chain with the dynamic llm and the new prompt\n",
    "configurable_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | dynamic_llm   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b67200e5-d3ae-4c86-9f69-bc7b964ab532",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Chloroquine has been identified as a medication that can mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses, in addition to its antiviral activity [[1]](https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z). Furthermore, glucocorticoids are considered the most effective anti-inflammatory therapies for chronic inflammatory and immune diseases affecting the lungs, such as asthma [[2]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7120595/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z). These medications can help in reducing inflammation in the lungs and managing conditions associated with inflammatory lung diseases."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 112 ms, sys: 13.2 ms, total: 125 ms\n",
      "Wall time: 8.69 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt35\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8661b48d-1e57-4a70-9b0a-cc59f9093267",
   "metadata": {},
   "source": [
    "As seen above, we were able to improve the quality and breath of the answer and add citations with only prompt engineering!\n",
    "\n",
    "#### Let's try again, but with GPT-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "efcfac6b-bac2-40c6-9ded-e4ee38e3093f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Several studies have highlighted different medicines and therapeutic approaches that can reduce inflammation in the lungs:\n",
       "\n",
       "1. **Interleukin-22 (IL-22):** A study demonstrated that a pro-IL-22 environment reduces pulmonary inflammation during H1N1 infection and protects the lung by promoting tight junction formation. The administration of recombinant IL-22 in vivo was shown to reduce inflammation and fluid leak into the lung, suggesting that the IL-22/IL-22BP axis could be a potential target for reducing influenza-induced pneumonia[[1]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "2. **Chloroquine:** In human lung parenchymal explants, chloroquine at clinically achievable concentrations inhibited the release of several inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3. This suggests that, besides its antiviral activity, chloroquine might also mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses[[2]](https://doi.org/10.1093/cid/ciaa546; https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "3. **Escin:** An article reviewing literature data related to the use of escin, which has potent anti-inflammatory and anti-viral effects in lung injury, suggested that it could represent a therapeutic opportunity as add-on therapy in acute lung injury (ALI) related to COVID-19 infection[[3]](https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "4. **Low doses of radiation therapy (RT):** This therapy was shown to protect the lung from inflammation by increasing IL-10 secretion and decreasing IFNγ production in human lung macrophages. This suggests that low doses of RT could mitigate lung inflammatory processes in situations such as COVID-19-induced ARDS[[4]](https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "5. **Tocilizumab:** A study related the short-term experience regarding IL6 blockade with tocilizumab in selected patients with severe COVID-19-related pneumonia. It was demonstrated that IL6 blockade could curb the \"cytokine storm\", prevent ICU admission, and the requirement for mechanical ventilation[[5]](https://doi.org/10.1101/2020.04.20.20061861?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "These findings highlight a variety of approaches, including cytokine therapy, antiviral and anti-inflammatory drugs, radiation therapy, and immunomodulatory drugs, as potential strategies to reduce lung inflammation in the context of various pulmonary diseases and infections."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 117 ms, sys: 180 µs, total: 117 ms\n",
      "Wall time: 26.5 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b54345b-7539-4110-9163-3565fa90d4b3",
   "metadata": {},
   "source": [
    "#### And GPT-4o"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e79d8270-eebc-4d31-8afa-8263262764f9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Several medicines have been shown to reduce inflammation in the lungs:\n",
       "\n",
       "1. **Interleukin-22 (IL-22)**: Administering recombinant IL-22 in vivo reduces inflammation and fluid leak into the lungs, which can be beneficial in reducing influenza-induced pneumonia [[1]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "2. **Chloroquine**: This drug, at clinically achievable concentrations in the lung, has been shown to inhibit the release of several pro-inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3, thereby potentially mitigating the cytokine storm associated with severe pneumonia caused by coronaviruses [[2]](https://doi.org/10.1093/cid/ciaa546; https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "3. **Escin**: Known for its potent anti-inflammatory and antiviral effects, escin could be used as an add-on therapy for acute lung injury (ALI) related to COVID-19 infection [[3]](https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "4. **Low-dose radiation therapy (RT)**: Low doses of RT (0.5-1 Gy) have been shown to regulate lung inflammation by increasing IL-10 secretion and decreasing IFNγ production, which helps protect the lung from inflammation [[4]](https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "5. **Tocilizumab**: This IL6 blockade has shown promise in curbing the \"cytokine storm\" associated with severe COVID-19, potentially preventing ICU admission and the requirement for mechanical ventilation [[5]](https://doi.org/10.1101/2020.04.20.20061861?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "6. **Glucocorticoids**: Although their effectiveness and potential immunosuppressive effects are debated, glucocorticoids like dexamethasone have been used to reduce inflammation in the lungs in conditions like SARS [[6]](https://doi.org/10.1128/jvi.02190-07; https://www.ncbi.nlm.nih.gov/pubmed/18287230/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
       "\n",
       "These treatments highlight various approaches to mitigating lung inflammation, ranging from cytokine modulation to antiviral and anti-inflammatory therapies."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 109 ms, sys: 3.94 ms, total: 113 ms\n",
      "Wall time: 22.8 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "try:\n",
    "    display(Markdown(configurable_chain.with_config(configurable={\"model\": \"gpt4o\"}).invoke({\"question\": QUESTION})))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c93b000a-8d0a-4574-be7c-48d26dfb4c70",
   "metadata": {},
   "source": [
    "#### As you can see the answer from GPT4 is richer, and includes all the relevant chunks. GPT3.5 tends to focus in the first and last chunks only"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6690453b-a9b1-4907-bd43-8c6b3ecba26e",
   "metadata": {},
   "source": [
    "## Adding Streaming to improve user experience and performance\n",
    "\n",
    "It is obvious by now that **GPT4 answers are better quality than GPT3.5**. None are incorrect, but GPT4 is better at understanding the context, following the prompt instructions and on giving a comprehensive answer.\n",
    "\n",
    "One way to make GPT4 look faster is to stream the answer, so the user can see the response as it is typed. To do this, we just simply need to call the method `stream` instead of `invoke`. More on Streaming and Callbacks in later notebooks, but for now, this is one simple way to do it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6d250c88-5984-438f-8390-1d93756048ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Several medicines have been identified to reduce inflammation in the lungs:\n",
      "\n",
      "1. **Interleukin-22 (IL-22)**: This cytokine is critical in protecting the lung during infections such as influenza. It promotes tight junction formation in the lungs, reducing pulmonary inflammation. Administering recombinant IL-22 has been shown to reduce inflammation and fluid leak into the lung during H1N1 infection [[1]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "2. **Chloroquine**: Clinically achievable concentrations of chloroquine in the lung can significantly inhibit the release of pro-inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3, thereby mitigating the cytokine storm associated with severe pneumonia caused by coronaviruses [[2]](https://doi.org/10.1093/cid/ciaa546; https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "3. **Escin**: This agent has potent anti-inflammatory and antiviral effects and is suggested as a potential add-on therapy for acute lung injury (ALI) related to COVID-19 infection [[3]](https://doi.org/10.1002/jcph.1644; https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "4. **Low-dose radiation therapy (RT)**: Low doses of chest irradiation (below 1 Gy) have been shown to protect the lung from inflammation by increasing IL-10 secretion and decreasing IFNγ production, thus favoring an anti-inflammatory cytokine environment [[4]](https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "5. **Tocilizumab**: An IL-6 blocker, tocilizumab, has shown promise in reducing the cytokine storm and preventing ICU admission and mechanical ventilation in severe COVID-19-related pneumonia [[5]](https://doi.org/10.1101/2020.04.20.20061861?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "These treatments highlight the variety of approaches being explored to manage lung inflammation, especially in the context of severe infections and chronic conditions."
     ]
    }
   ],
   "source": [
    "for chunk in configurable_chain.with_config(configurable={\"model\": \"gpt4o\"}).stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d1fe9e3-6772-42d3-aa8a-dff103768451",
   "metadata": {},
   "source": [
    "# Using Other models from the Azure AI Catalog (optional)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a90536f-b5d8-4b82-8109-a46ec46dad72",
   "metadata": {},
   "source": [
    "Besides OpenAI models, Azure AI Studio provides a wide selection of models from an extensive catalog featuring providers such as Mistral, Cohere, Meta, HuggingFace, Nvidia, Databricks, and Microsoft's own offerings.\n",
    "\n",
    "Several models are available as serverless, pay-as-you-go endpoints. Examples include Mistral-Large and Cohere Command R+.\n",
    "\n",
    "To use the following code, visit https://ai.azure.com and deploy [Mistral-Large](https://ai.azure.com/explore/models/Mistral-large/version/1/registry/azureml-mistral) and [Cohere-Command-R-plus](https://ai.azure.com/explore/models/Cohere-command-r-plus/version/3/registry/azureml-cohere) on a pay-as-you-go basis, then copy the endpoint URLs and keys into the cells below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "fc06a00a-011e-489c-ba3f-8d1aaad3d3fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_mistralai import ChatMistralAI\n",
    "from langchain_cohere.chat_models import ChatCohere"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "50c1855f-8c05-4824-ae71-6a0ee7ed8695",
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"MISTRAL_API_ENDPOINT\"] = \"https://<YOUR_MISTRAL_DEPLOYMENT_NAME>.<YOUR_REGION>.inference.ai.azure.com\"\n",
    "os.environ[\"MISTRAL_API_KEY\"] = \"ENTER HERE YOUR MISTRAL ENDPOINT API KEY\"\n",
    "os.environ[\"COHERE_API_ENDPOINT\"] = \"https://<YOUR_COHERE_DEPLOYMENT_NAME>.<YOUR_REGION>.inference.ai.azure.com/v1\"\n",
    "os.environ[\"COHERE_API_KEY\"] = \"ENTER HERE YOUR COHERE ENDPOINT API KEY\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "37e5ce45-ab84-439f-bd14-8fd31b377469",
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_mistral = ChatMistralAI(endpoint=os.environ[\"MISTRAL_API_ENDPOINT\"],\n",
    "                    api_key=os.environ[\"MISTRAL_API_KEY\"],\n",
    "                    temperature=0, max_tokens=COMPLETION_TOKENS,\n",
    "                    streaming=True)\n",
    "\n",
    "llm_cohere = ChatCohere(\n",
    "    base_url=os.environ[\"COHERE_API_ENDPOINT\"],\n",
    "    cohere_api_key=os.environ[\"COHERE_API_KEY\"],\n",
    "    streaming=True,\n",
    "    max_tokens=COMPLETION_TOKENS,\n",
    "    temperature=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cc6c49a-9046-4416-aaf2-75c0c80136d7",
   "metadata": {},
   "source": [
    "Let's call the invoke method and see if they work"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "50f570f9-86aa-40f4-9ad0-b7ddf40c5434",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'I am a language model trained by the Mistral AI team. I am designed to understand and generate human-like text based on the input I receive. I can assist with a wide range of tasks, from answering questions and providing information to helping with brainstorming and creativity.'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_mistral.invoke(\"who are you?\").content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "78726c37-7ecb-40fe-8645-6de89416c171",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'I am Coral, a brilliant, sophisticated AI-assistant chatbot trained to assist human users by providing thorough responses. I am powered by Command, a large language model built by the company Cohere.'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_cohere.invoke(\"who are you?\").content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5d19cd5-b044-4eef-ad6c-057f7f174378",
   "metadata": {},
   "source": [
    "Great, now let's add the same RAG chain that we used before with the Azure OpenAI models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "e441c1d3-5133-427a-9120-2208afd16f9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "mistral_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | llm_mistral   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")\n",
    "\n",
    "cohere_chain = (\n",
    "    {\n",
    "        \"context\": itemgetter(\"question\") | retriever, # Passes the question to the retriever and the results are assign to context\n",
    "        \"question\": itemgetter(\"question\")\n",
    "    }\n",
    "    | DOCSEARCH_PROMPT  # Passes the input variables above to the prompt template\n",
    "    | llm_cohere   # Passes the finished prompt to the LLM\n",
    "    | StrOutputParser()  # converts the output (Runnable object) to the desired output (string)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6452aec2-ad0d-4575-9e86-f951a2cb109c",
   "metadata": {},
   "source": [
    "Since they also both support streaming, we can use the `stream` method from langchain as well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "81b5695d-30f5-4ee0-88c1-3ac622733f5b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Several documents suggest different medications that can reduce inflammation in the lungs.\n",
      "\n",
      "1. Interleukin-22 (IL-22) is a cytokine that can protect the lung during infection by reducing pulmonary inflammation and promoting tight junction formation. Administering recombinant IL-22 in vivo has been shown to reduce inflammation and fluid leak into the lung [[1]](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6917921/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "2. Chloroquine, at clinically achievable concentrations in the lung (100 μM), has been found to inhibit the lipopolysaccharide-induced release of pro-inflammatory cytokines such as TNF-α, IL-6, CCL2, and CCL3 in human lung parenchymal explants. This suggests that chloroquine may mitigate the cytokine storm associated with severe pneumonia caused by coronaviruses [[2]](https://doi.org/10.1093/cid/ciaa546;%20https://www.ncbi.nlm.nih.gov/pubmed/32382733/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "3. Escin, an agent with potent anti-inflammatory and anti-viral effects, has been suggested as a therapeutic opportunity for add-on therapy in acute lung injury (ALI) related to COVID-19 infection [[3]](https://doi.org/10.1002/jcph.1644;%20https://www.ncbi.nlm.nih.gov/pubmed/32441805/?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "4. Low doses of radiation therapy (RT) have been shown to protect the lung from inflammation by increasing the percentage of nerve- and airway-associated macrophages (NAMs) producing the anti-inflammatory cytokine IL-10, leading to a more favorable pro- versus anti-inflammatory cytokine balance (IL6-IFNγ/IL-10) [[4]](https://doi.org/10.1101/2020.05.11.077651?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "5. Tocilizumab, an IL-6 blocker, has been demonstrated to curb the \"cytokine storm\" in selected patients with severe COVID-19-related pneumonia, preventing ICU admission and the requirement for mechanical ventilation [[5]](https://doi.org/10.1101/2020.04.20.20061861?sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-07-09T13:52:04Z&st=2024-07-09T05:52:04Z&spr=https&sig=9Nx31tWOzf6CWylUZnGaciT9VDWVJSJ9vQulMcshm7Q%3D).\n",
      "\n",
      "These findings suggest that various medications, including cytokines, antiviral agents, anti-inflammatory drugs, and even radiation therapy, may have potential roles in reducing lung inflammation. However, further research is needed to fully understand their mechanisms of action, efficacy, and safety in different clinical contexts."
     ]
    }
   ],
   "source": [
    "for chunk in mistral_chain.stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "5ff34dfd-52e6-45b0-a990-46a59947b8c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "- Chloroquine\n",
      "- Escin\n",
      "- Interleukin-22 (IL-22)\n",
      "- Low doses of radiation therapy (RT)\n",
      "- Tocilizumab\n",
      "- Tanshinone IIA (TIIA)\n",
      "- Dexamethasone\n",
      "- Natural herbal medicine (NHM)\n",
      "- Corticosteroids"
     ]
    }
   ],
   "source": [
    "for chunk in cohere_chain.stream({\"question\": QUESTION}):\n",
    "    print(chunk, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a51d5e2-ef4e-4355-b933-a03c7decb550",
   "metadata": {},
   "source": [
    "The two models, Mistral Large and Command R+, as demonstrated, are capable of capturing context more effectively than GPT-3.5, resulting in superior responses. In terms of speed and breath of response, they are comparable to GPT-4.\n",
    "\n",
    "As we will see in the next notebooks, we will need a bit more than just understanding all the context in order to build a multi-agentic bot architecture (parallel function calling for example), but for now it is good to understand the options available in terms of: \n",
    "- Context understanding\n",
    "- Speed of response\n",
    "- Quality of response\n",
    "- Price\n",
    "\n",
    "Check out these Model Benchmark charts provided by Azure AI Studio: https://ai.azure.com/explore/benchmarks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### By using OpenAI, the answers to user questions are way better than taking just the results from Azure AI Search. So the summary is:\n",
    "- Utilizing Azure AI Search, we conduct a multi-index hybrid search that identifies the top chunks of documents from each index.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!\n",
    "\n",
    "##### Important observations on this notebook:\n",
    "\n",
    "1) Answers with GPT-3.5 are less quality but fast\n",
    "2) Answers with GPT-3.5 sometimes failed on provinding citations in the right format\n",
    "3) Answers with GPT-4 are great quality but slower\n",
    "4) Answers with GPT-4 always provide good and diverse citations in the right format\n",
    "5) Models like Mistral Large or Cohere Command R+ provide, so far, a similar experience to GPT-4\n",
    "5) Streaming the answers improves the user experience big time!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}