redis-openAI-doc-chat.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0b692c73",
   "metadata": {},
   "source": [
    "# Using Redis and OpenAI to chat with PDF documents\n",
    "\n",
    "This notebook demonstrates how to use RedisAI and (Azure) OpenAI to chat with PDF documents. The PDF included is\n",
    "a informational brochure about the Chevy Colorado pickup truck.\n",
    "\n",
    "In this notebook, we will use LLamaIndex to chunk, vectorize, and store the PDF document in Redis as vectors\n",
    "alongside associated text. The query interface provided by LLamaIndex will be used to search for relevant\n",
    "information given queries from the user."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "949e6cf1",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pypdf in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (3.17.0)\n",
      "Requirement already satisfied: PyPDF2 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (3.0.1)\n",
      "Requirement already satisfied: transformers in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (4.35.0)\n",
      "Requirement already satisfied: tiktoken in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (0.5.1)\n",
      "Requirement already satisfied: ipywidgets in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (8.1.1)\n",
      "Requirement already satisfied: nltk in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (3.8.1)\n",
      "Requirement already satisfied: redis>=5.0.0 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (5.0.1)\n",
      "Requirement already satisfied: llama_index==0.8.58 in /opt/conda/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (0.8.58)\n",
      "Collecting openai==0.28.1 (from -r requirements.txt (line 10))\n",
      "  Downloading openai-0.28.1-py3-none-any.whl.metadata (11 kB)\n",
      "Requirement already satisfied: SQLAlchemy>=1.4.49 in /opt/conda/lib/python3.11/site-packages (from SQLAlchemy[asyncio]>=1.4.49->llama_index==0.8.58->-r requirements.txt (line 9)) (2.0.22)\n",
      "Requirement already satisfied: aiostream<0.6.0,>=0.5.2 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.5.2)\n",
      "Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.5.14)\n",
      "Requirement already satisfied: deprecated>=1.2.9.3 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.2.14)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (2023.10.0)\n",
      "Requirement already satisfied: langchain>=0.0.303 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.0.331)\n",
      "Requirement already satisfied: nest-asyncio<2.0.0,>=1.5.8 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.5.8)\n",
      "Requirement already satisfied: numpy in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.26.1)\n",
      "Requirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (2.1.2)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (8.2.3)\n",
      "Requirement already satisfied: typing-extensions>=4.5.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (4.8.0)\n",
      "Requirement already satisfied: typing-inspect>=0.8.0 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (0.9.0)\n",
      "Requirement already satisfied: urllib3<2 in /opt/conda/lib/python3.11/site-packages (from llama_index==0.8.58->-r requirements.txt (line 9)) (1.26.18)\n",
      "Requirement already satisfied: requests>=2.20 in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (2.31.0)\n",
      "Requirement already satisfied: tqdm in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (4.66.1)\n",
      "Requirement already satisfied: aiohttp in /opt/conda/lib/python3.11/site-packages (from openai==0.28.1->-r requirements.txt (line 10)) (3.8.6)\n",
      "Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (3.13.1)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.17.3)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (23.2)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (6.0.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (2023.10.3)\n",
      "Requirement already satisfied: tokenizers<0.15,>=0.14 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.14.1)\n",
      "Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.11/site-packages (from transformers->-r requirements.txt (line 3)) (0.4.0)\n",
      "Requirement already satisfied: comm>=0.1.3 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (0.1.4)\n",
      "Requirement already satisfied: ipython>=6.1.0 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (8.16.1)\n",
      "Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (5.11.2)\n",
      "Requirement already satisfied: widgetsnbextension~=4.0.9 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (4.0.9)\n",
      "Requirement already satisfied: jupyterlab-widgets~=3.0.9 in /opt/conda/lib/python3.11/site-packages (from ipywidgets->-r requirements.txt (line 5)) (3.0.9)\n",
      "Requirement already satisfied: click in /opt/conda/lib/python3.11/site-packages (from nltk->-r requirements.txt (line 7)) (8.1.7)\n",
      "Requirement already satisfied: joblib in /opt/conda/lib/python3.11/site-packages (from nltk->-r requirements.txt (line 7)) (1.3.2)\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /opt/conda/lib/python3.11/site-packages (from dataclasses-json<0.6.0,>=0.5.7->llama_index==0.8.58->-r requirements.txt (line 9)) (3.20.1)\n",
      "Requirement already satisfied: wrapt<2,>=1.10 in /opt/conda/lib/python3.11/site-packages (from deprecated>=1.2.9.3->llama_index==0.8.58->-r requirements.txt (line 9)) (1.15.0)\n",
      "Requirement already satisfied: backcall in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.0)\n",
      "Requirement already satisfied: decorator in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (5.1.1)\n",
      "Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.19.1)\n",
      "Requirement already satisfied: matplotlib-inline in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.1.6)\n",
      "Requirement already satisfied: pickleshare in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.7.5)\n",
      "Requirement already satisfied: prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (3.0.39)\n",
      "Requirement already satisfied: pygments>=2.4.0 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (2.16.1)\n",
      "Requirement already satisfied: stack-data in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.6.2)\n",
      "Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (4.8.0)\n",
      "Requirement already satisfied: anyio<4.0 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (3.7.1)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (1.33)\n",
      "Requirement already satisfied: langsmith<0.1.0,>=0.0.52 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (0.0.57)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /opt/conda/lib/python3.11/site-packages (from langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.4.2)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (23.1.0)\n",
      "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (3.3.0)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (6.0.4)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (4.0.3)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.9.2)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.4.0)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.11/site-packages (from aiohttp->openai==0.28.1->-r requirements.txt (line 10)) (1.3.1)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1->-r requirements.txt (line 10)) (3.4)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1->-r requirements.txt (line 10)) (2023.7.22)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/lib/python3.11/site-packages (from SQLAlchemy>=1.4.49->SQLAlchemy[asyncio]>=1.4.49->llama_index==0.8.58->-r requirements.txt (line 9)) (3.0.0)\n",
      "Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.11/site-packages (from typing-inspect>=0.8.0->llama_index==0.8.58->-r requirements.txt (line 9)) (1.0.0)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2023.3.post1)\n",
      "Requirement already satisfied: tzdata>=2022.1 in /opt/conda/lib/python3.11/site-packages (from pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (2023.3)\n",
      "Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.11/site-packages (from anyio<4.0->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (1.3.0)\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /opt/conda/lib/python3.11/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.8.3)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /opt/conda/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.4)\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.11/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.7.0)\n",
      "Requirement already satisfied: wcwidth in /opt/conda/lib/python3.11/site-packages (from prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.8)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /opt/conda/lib/python3.11/site-packages (from pydantic<3,>=1->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.10.1 in /opt/conda/lib/python3.11/site-packages (from pydantic<3,>=1->langchain>=0.0.303->llama_index==0.8.58->-r requirements.txt (line 9)) (2.10.1)\n",
      "Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->llama_index==0.8.58->-r requirements.txt (line 9)) (1.16.0)\n",
      "Requirement already satisfied: executing>=1.2.0 in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (1.2.0)\n",
      "Requirement already satisfied: asttokens>=2.1.0 in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (2.4.0)\n",
      "Requirement already satisfied: pure-eval in /opt/conda/lib/python3.11/site-packages (from stack-data->ipython>=6.1.0->ipywidgets->-r requirements.txt (line 5)) (0.2.2)\n",
      "Downloading openai-0.28.1-py3-none-any.whl (76 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.0/77.0 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: openai\n",
      "  Attempting uninstall: openai\n",
      "    Found existing installation: openai 1.0.1\n",
      "    Uninstalling openai-1.0.1:\n",
      "      Successfully uninstalled openai-1.0.1\n",
      "Successfully installed openai-0.28.1\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "# Install the Python requirements\n",
    "%pip install -r requirements.txt --no-cache-dir"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "47264e32",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-10T12:20:23.988789Z",
     "start_time": "2023-02-10T12:20:23.967877Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import logging\n",
    "\n",
    "logging.basicConfig(\n",
    "    stream=sys.stdout, level=logging.INFO\n",
    ") # logging.DEBUG for more verbose output\n",
    "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n",
    "\n",
    "import textwrap\n",
    "import openai\n",
    "\n",
    "from langchain.llms import AzureOpenAI, OpenAI\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from llama_index.vector_stores import RedisVectorStore\n",
    "from llama_index import LangchainEmbedding\n",
    "from llama_index import (\n",
    "    GPTVectorStoreIndex,\n",
    "    SimpleDirectoryReader,\n",
    "    LLMPredictor,\n",
    "    PromptHelper,\n",
    "    ServiceContext,\n",
    "    StorageContext\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0ad91218",
   "metadata": {},
   "source": [
    "# Azure OpenAI | OpenAI (direct)\n",
    "\n",
    "The notebook allows the user two choose between using the OpenAI and Azure OpenAI endpoints. Make sure to follow the instructions in the README and set the .env correctly according to whichever API you are using. \n",
    "\n",
    "NOTE: ONLY ONE API CAN BE USED AT A TIME.\n",
    "\n",
    "- **[Use Azure OpenAI](#Azure-OpenAI)**\n",
    "- **[Use OpenAI](#OpenAI) (direct)**"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0023333d",
   "metadata": {},
   "source": [
    "## Azure OpenAI \n",
    "\n",
    "Here we setup the AzureOpenAI models and API keys that we set by reading from the environment above. The ``PromptHelper`` sets the parameters for the OpenAI model. The classes defined here are used together to provide a QnA interface between the user and the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "32a77108",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using OpenAI models: text-embedding-ada-002 and gpt-35-turbo\n",
      "Using Azure deployments: <your deployment name here> and <your deployment name here>\n"
     ]
    }
   ],
   "source": [
    "# setup Llama Index to use Azure OpenAI\n",
    "openai.api_type = \"azure\"\n",
    "openai.api_base = os.getenv(\"AZURE_OPENAI_API_BASE\")\n",
    "openai.api_version = os.getenv(\"OPENAI_API_VERSION\")\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "\n",
    "# Get the OpenAI model names ex. \"text-embedding-ada-002\"\n",
    "embedding_model = os.getenv(\"OPENAI_EMBEDDING_MODEL\")\n",
    "text_model = os.getenv(\"OPENAI_TEXT_MODEL\")\n",
    "# get the Azure Deployment name for the model\n",
    "embedding_model_deployment = os.getenv(\"AZURE_EMBED_MODEL_DEPLOYMENT_NAME\")\n",
    "text_model_deployment = os.getenv(\"AZURE_TEXT_MODEL_DEPLOYMENT_NAME\")\n",
    "\n",
    "print(f\"Using OpenAI models: {embedding_model} and {text_model}\")\n",
    "print(f\"Using Azure deployments: {embedding_model_deployment} and {text_model_deployment}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c67d58db",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "llm = AzureOpenAI(deployment_name=text_model_deployment, model_kwargs={\n",
    "    \"api_key\": openai.api_key,\n",
    "    \"api_base\": openai.api_base,\n",
    "    \"api_type\": openai.api_type,\n",
    "    \"api_version\": openai.api_version,\n",
    "})\n",
    "llm_predictor = LLMPredictor(llm=llm)\n",
    "\n",
    "embedding_llm = LangchainEmbedding(\n",
    "    OpenAIEmbeddings(\n",
    "        model=embedding_model,\n",
    "        deployment=embedding_model_deployment,\n",
    "        openai_api_key= openai.api_key,\n",
    "        openai_api_base=openai.api_base,\n",
    "        openai_api_type=openai.api_type,\n",
    "        openai_api_version=openai.api_version,\n",
    "    ),\n",
    "    embed_batch_size=1,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a31b8dae",
   "metadata": {},
   "source": [
    "## OpenAI\n",
    "\n",
    "The ``OpenAI`` class provides a simple interface to the OpenAI API.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4ce97716",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using OpenAI models: text-embedding-ada-002 and gpt-35-turbo\n"
     ]
    }
   ],
   "source": [
    "# setup Llama Index to use OpenAI direct\n",
    "openai.api_type = \"openai\"\n",
    "openai.api_version = os.getenv(\"OPENAI_API_VERSION\")\n",
    "openai.api_base = os.getenv(\"OPENAI_API_BASE\")\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "# Get the OpenAI model names ex. \"text-embedding-ada-002\"\n",
    "embedding_model = os.getenv(\"OPENAI_EMBEDDING_MODEL\")\n",
    "text_model = os.getenv(\"OPENAI_TEXT_MODEL\")\n",
    "\n",
    "print(f\"Using OpenAI models: {embedding_model} and {text_model}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "076cf33f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up LLM\n",
    "llm = OpenAI(model_kwargs={\n",
    "    \"api_key\": openai.api_key,\n",
    "    \"api_base\": openai.api_base,\n",
    "    \"api_type\": openai.api_type,\n",
    "    \"api_version\" : openai.api_version,\n",
    "})\n",
    "llm_predictor = LLMPredictor(llm=llm)\n",
    "\n",
    "# Set up Embedding model\n",
    "embedding_llm = LangchainEmbedding(\n",
    "    OpenAIEmbeddings(\n",
    "        model=embedding_model,\n",
    "        openai_api_version=openai.api_version,\n",
    "        openai_api_key= openai.api_key,\n",
    "        openai_api_base=openai.api_base,\n",
    "        openai_api_type=openai.api_type,\n",
    "    ),\n",
    "    embed_batch_size=1,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59ff935d",
   "metadata": {},
   "source": [
    "### LLamaIndex\n",
    "\n",
    "[LlamaIndex](https://github.com/jerryjliu/llama_index) (GPT Index) is a project that provides a central interface to connect your LLM's with external data sources. It provides a simple interface to vectorize and store embeddings in Redis, create search indices using Redis, and perform vector search to find context for generative models like GPT.\n",
    "\n",
    "Here we will use it to load in the documents (Chevy Colorado Brochure)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "68cbd239-880e-41a3-98d8-dbb3fab55431",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-10T12:20:30.175678Z",
     "start_time": "2023-02-10T12:20:30.172456Z"
    },
    "pycharm": {
     "is_executing": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Document ID: c325c82f-9408-42fe-9a5a-c58e196ef5a3\n"
     ]
    }
   ],
   "source": [
    "# load documents\n",
    "documents = SimpleDirectoryReader('./docs').load_data()\n",
    "print('Document ID:', documents[0].doc_id)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "697a59d2",
   "metadata": {},
   "source": [
    "Llamaindex also works with frameworks like langchain to make prompting and other aspects of a chat based application easier. Here we can use the ``PromptHelper`` class to help us generate prompts for the (Azure) OpenAI model. The will be off by default as it can be tricky to setup correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "147e7678",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set number of output tokens\n",
    "num_output = int(os.getenv(\"OPENAI_MAX_TOKENS\"))\n",
    "# max LLM token input size\n",
    "max_input_size = int(os.getenv(\"CHUNK_SIZE\"))\n",
    "# set maximum chunk overlap\n",
    "max_chunk_overlap = float(os.getenv(\"CHUNK_OVERLAP\"))\n",
    "\n",
    "prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "132b7b01",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /tmp/llama_index...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    }
   ],
   "source": [
    "# define the service we will use to answer questions\n",
    "# if you executive the Azure OpenAI code above, your Azure Models and creds will be used and the same for OpenAI\n",
    "service_context = ServiceContext.from_defaults(\n",
    "    llm_predictor=llm_predictor,\n",
    "    embed_model=embedding_llm,\n",
    "    prompt_helper=prompt_helper # uncomment to use prompt_helper.\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "dd270925",
   "metadata": {},
   "source": [
    "## Initialize Redis as a Vector Database\n",
    "\n",
    "Now we have our documents read in, we can initialize the ``RedisVectorStore``. This will allow us to store our vectors in Redis and create an index.\n",
    "\n",
    "The ``GPTVectorStoreIndex`` will then create the embeddings from the text chunks by calling out to OpenAI's API. The embeddings will be stored in Redis and an index will be created.\n",
    "\n",
    "NOTE: If you didn't set the ``OPENAI_API_KEY`` environment variable, you will get an error here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "fd85f591-334a-492a-ba80-89fe7c79288e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Redis address: redis://:0kYQxoG2RfkL0zVzQRXm0MSwBmVJ3D89+csfCH8E4fo=@sgn-llm.eastus2.redisenterprise.cache.azure.net:10000\n"
     ]
    }
   ],
   "source": [
    "def format_redis_conn_from_env(using_ssl=False):\n",
    "    start = \"rediss://\" if using_ssl else \"redis://\"\n",
    "    # if using RBAC\n",
    "    password = os.getenv(\"REDIS_PASSWORD\", None)\n",
    "    username = os.getenv(\"REDIS_USERNAME\", \"\")\n",
    "    if password != None:\n",
    "        start += f\"{username}:{password}@\"\n",
    "\n",
    "    return start + f\"{os.getenv('REDIS_HOST')}:{os.getenv('REDIS_PORT')}\"\n",
    "\n",
    "# make using_ssl=True to use SSL with ACRE\n",
    "redis_url = format_redis_conn_from_env(using_ssl=False)\n",
    "print(f\"Using Redis address: {redis_url}\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "30b0d59f-a7ac-413e-ab1f-3e46628d6e95",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create VectorStore\n",
    "vector_store = RedisVectorStore(\n",
    "    index_name=\"chevy_docs\",\n",
    "    index_prefix=\"blog\",\n",
    "    redis_url=redis_url,\n",
    "    overwrite=True\n",
    ")\n",
    "\n",
    "# access the underlying client in the RedisVectorStore implementation to ping the redis instance\n",
    "vector_store.client.ping()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ba1558b3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-10T12:20:33.735897Z",
     "start_time": "2023-02-10T12:20:30.404245Z"
    },
    "pycharm": {
     "is_executing": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.vector_stores.redis:Creating index chevy_docs\n",
      "Creating index chevy_docs\n",
      "INFO:llama_index.vector_stores.redis:Added 27 documents to index chevy_docs\n",
      "Added 27 documents to index chevy_docs\n"
     ]
    }
   ],
   "source": [
    "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n",
    "index = GPTVectorStoreIndex.from_documents(\n",
    "    documents,\n",
    "    storage_context=storage_context,\n",
    "    service_context=service_context\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04304299-fc3e-40a0-8600-f50c3292767e",
   "metadata": {},
   "source": [
    "## Test the RAG pipeline!\n",
    "\n",
    "Now that we have our document stored in the index, we can ask questions against the index. The index will use the data stored in itself as the knowledge base for the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "35369eda",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-10T12:20:51.328762Z",
     "start_time": "2023-02-10T12:20:33.822688Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.vector_stores.redis:Using filters: *\n",
      "Using filters: *\n",
      "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n",
      "Querying index chevy_docs\n",
      "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_bd662262-87bf-4428-9e3e-155d20e99622', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "Found 2 results for query with id ['blog/vector_bd662262-87bf-4428-9e3e-155d20e99622', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "\n",
      "  The Chevrolet Colorado is available in Extended Cab, Crew Cab Short Box, and Crew Cab Long Box\n",
      "variants. It includes a 2.5L DOHC 4-cylinder with Variable Valve Timing (VVT) and Direct Injection,\n",
      "and a 3.6L DOHC V6 with Variable Valve Timing (VVT) and Direct Injection (Crew Cab 4x4 and Crew Cab\n",
      "Long Box 2WD models). The transmission is either a 6-speed automatic, electronically controlled with\n",
      "overdrive or an 8-speed automatic, electronically controlled with overdrive, Tow/Haul mode and Hitch\n",
      "Guidance 1 (Crew Cab 4x4 and Crew Cab Long Box 2WD models). It also includes 17\" Blade Silver\n",
      "Metallic-Painted Aluminum Wheels (standard on LT), 18\" Dark Argent Metallic-Painted Aluminum Wheels\n",
      "(available on LT), and 18\" Black-Painted Aluminum Wheels with Red Accents (available on LT with\n",
      "Redline Edition). Additional features available for the Chevrolet Colorado include Five AEV hot-\n",
      "stamped boron steel skid plates, AEV fender flares, Fog lamps, Front and rear floor liners with AEV\n",
      "logo, AEV embroidered head restraints, recovery points, and the compatibility with Apple CarPlay.\n"
     ]
    }
   ],
   "source": [
    "query_engine = index.as_query_engine()\n",
    "response = query_engine.query(\"What types of variants are available for the Chevrolet Colorado?\")\n",
    "print(\"\\n\", textwrap.fill(str(response), 100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "99212d33",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-10T12:21:10.337294Z",
     "start_time": "2023-02-10T12:20:51.338718Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.vector_stores.redis:Using filters: *\n",
      "Using filters: *\n",
      "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n",
      "Querying index chevy_docs\n",
      "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_efcf4237-6811-4d27-b6b3-49f0062880f9', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "Found 2 results for query with id ['blog/vector_efcf4237-6811-4d27-b6b3-49f0062880f9', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "\n",
      "  The Chevy Colorado's maximum towing capacity is 7,700 lbs with the available GM-exclusive Duramax®\n",
      "2.8L Turbo-Diesel engine and the Trailering Package, LT Convenience Package and Safety Package. The\n",
      "Colorado Crew Cab ZR2 also features an AEV-designed aluminum wheels, bumper with winch provisions,\n",
      "rear bumper with recovery points, five hot-stamped boron steel skid plates, fender flares, fog\n",
      "lamps, front and rear floor liners with AEV logo, and AEV embroidered head restraints. However, due\n",
      "to current supply chain shortages, certain features shown may have limited or late availability, or\n",
      "may no longer be available. The trailering capacity of your specific vehicle may vary depending on\n",
      "the weight of passengers, cargo and options or accessories. You can check which smartphones are\n",
      "compatible with the Vehicle User Interface at Google Play or Apple's App Store.\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\"What is the maximum towing capacity of the chevy colorado?\")\n",
    "print(\"\\n\", textwrap.fill(str(response), 100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4a028452",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:llama_index.vector_stores.redis:Using filters: *\n",
      "Using filters: *\n",
      "INFO:llama_index.vector_stores.redis:Querying index chevy_docs\n",
      "Querying index chevy_docs\n",
      "INFO:llama_index.vector_stores.redis:Found 2 results for query with id ['blog/vector_a8ee72a5-fbef-4cea-acc7-633809a5777c', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "Found 2 results for query with id ['blog/vector_a8ee72a5-fbef-4cea-acc7-633809a5777c', 'blog/vector_ef90fb76-37d3-40d6-b059-ce31151a1071']\n",
      "\n",
      "  The main differences between the three engine types available for the Chevy Colorado are the power\n",
      "output, torque output, displacement, bore and stroke, compression ratio, block and cylinder head\n",
      "material, valvetrain, fuel delivery, maximum payload rating, maximum trailering weight rating, and\n",
      "estimated fuel economy. The 2.5L DOHC I-4 engine has 200 hp @ 6300 rpm and 191 lb.-ft. of torque @\n",
      "4400 rpm with a displacement of 2460 cc (150 cu. in.), a bore and stroke of 88 mm x 101 mm (3.46 in.\n",
      "x 3.97 in.), a compression ratio of 11.3:1, a cast-aluminum block and cylinder head, dual-overhead\n",
      "camshafts with 4 valves per cylinder and Variable Valve Timing (VVT), direct injection, a maximum\n",
      "payload rating of 1,420 lbs. and a maximum trailering weight rating of 3,500 lbs., with an EPA-\n",
      "estimated fuel economy of 19 MPG city/25 highway for 2WD and 19 MPG city/24 highway for 4x4. The\n",
      "Chevy Colorado can also be equipped with an AEV rear bumper with recovery points, five AEV hot-\n",
      "stamped boron steel skid plates, front and rear\n"
     ]
    }
   ],
   "source": [
    "response = query_engine.query(\"What are the main differences between the three engine types available for the Chevy Colorado?\")\n",
    "print(\"\\n\", textwrap.fill(str(response), 100))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}