rag_with_weaviate.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple Rag Implementation Using LangChain\n",
    "\n",
    "<img align=\"left\" width=\"450\" alt=\"RagChart\" src=\"https://miro.medium.com/v2/resize:fit:1400/format:webp/1*EF7wzF_mTbT-CK0OwEGzyQ.png\" style=\"margin: 0px 15px 0px 0px;\"/>\n",
    "\n",
    "Retrieval-Augmented Generation (RAG) is the concept to provide LLMs with additional information from an external knowledge source. This allows them to generate more accurate and contextual answers while reducing hallucinations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "import weaviate\n",
    "from langchain.vectorstores import Weaviate\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from weaviate.embedded import EmbeddedOptions\n",
    "from langchain.prompts import ChatPromptTemplate\n",
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.schema.runnable import RunnablePassthrough\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "from langchain.schema.output_parser import StrOutputParser"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data collected from President Biden's State of Union Address from 2022. The raw text document is available in [LangChain's Github](https://raw.githubusercontent.com/langchain-ai/langchain/master/docs/docs/modules/state_of_the_union.txt) repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "url = \"https://raw.githubusercontent.com/langchain-ai/langchain/master/docs/docs/modules/state_of_the_union.txt\"\n",
    "res = requests.get(url)\n",
    "with open(\"state_of_the_union.txt\", \"w\") as f:\n",
    "    f.write(res.text)\n",
    "'''\n",
    "loader = TextLoader('./state_of_the_union.txt')\n",
    "documents = loader.load()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, chunk your documents — Because the Document, in its original state, is too long to fit into the LLM’s context window"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)\n",
    "chunks = text_splitter.split_documents(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Embed and store the chunks to enable semantic search across the text chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/weaviate/warnings.py:158: DeprecationWarning: Dep016: You are using the Weaviate v3 client, which is deprecated.\n",
      "            Consider upgrading to the new and improved v4 client instead!\n",
      "            See here for usage: https://weaviate.io/developers/weaviate/client-libraries/python\n",
      "            \n",
      "  warnings.warn(\n",
      "{\"action\":\"startup\",\"default_vectorizer_module\":\"none\",\"level\":\"info\",\"msg\":\"the default vectorizer modules is set to \\\"none\\\", as a result all new schema classes without an explicit vectorizer setting, will use this vectorizer\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "{\"action\":\"startup\",\"auto_schema_enabled\":true,\"level\":\"info\",\"msg\":\"auto schema enabled setting is set to \\\"true\\\"\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "{\"level\":\"info\",\"msg\":\"No resource limits set, weaviate will use all available memory and CPU. To limit resources, set LIMIT_RESOURCES=true\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Started /Users/ktuna/.cache/weaviate-embedded: process ID 10150\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "{\"level\":\"warning\",\"msg\":\"Multiple vector spaces are present, GraphQL Explore and REST API list objects endpoint module include params has been disabled as a result.\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "{\"action\":\"grpc_startup\",\"level\":\"info\",\"msg\":\"grpc server listening at [::]:50060\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "{\"action\":\"restapi_management\",\"level\":\"info\",\"msg\":\"Serving weaviate at http://127.0.0.1:8079\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class `OpenAIEmbeddings` was deprecated in LangChain 0.0.9 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import OpenAIEmbeddings`.\n",
      "  warn_deprecated(\n",
      "{\"level\":\"info\",\"msg\":\"Created shard langchain_ddf295adc9d1426abed18004f2a75b55_H8w1tVyBTrYA in 4.350083ms\",\"time\":\"2024-04-29T08:42:27+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":1000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:27+03:00\",\"took\":96625}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_10d6217c2e384668ac218fd7fd818c87_jwimPNVul6RU in 1.716167ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_529ce1f424cb43459f36cc0a78127585_gE8bc3hcw0CI in 1.886334ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":50083}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":73958}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_97e9a153e75f49f586cddd66353fcd68_nkfS0OjMGfGE in 2.088375ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":57708}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_e08648b5a8114645b52a841d38514bf5_0ivQpP8r3Umc in 1.788792ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":57667}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_b5457816a6b74e2fb0ca42ef4ba85440_lsEao5lygrM6 in 2.565834ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":36250}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_3ee005ba1e3247ba94df6a0b5ec33c0c_TFkvumxUwsVF in 2.284792ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":36000}\n",
      "{\"level\":\"info\",\"msg\":\"Completed loading shard langchain_05192ad181314f60bf9fce48a0e2512c_ZoVExI59XU1W in 4.307959ms\",\"time\":\"2024-04-29T08:42:28+03:00\"}\n",
      "{\"action\":\"hnsw_vector_cache_prefill\",\"count\":3000,\"index_id\":\"main\",\"level\":\"info\",\"limit\":1000000000000,\"msg\":\"prefilled vector cache\",\"time\":\"2024-04-29T08:42:28+03:00\",\"took\":1993417}\n",
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/pydantic/main.py:1070: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/\n",
      "  warnings.warn('The `dict` method is deprecated; use `model_dump` instead.', category=PydanticDeprecatedSince20)\n"
     ]
    }
   ],
   "source": [
    "client = weaviate.Client(\n",
    "  embedded_options = EmbeddedOptions()\n",
    ")\n",
    "\n",
    "vectorstore = Weaviate.from_documents(\n",
    "    client = client,    \n",
    "    documents = chunks,\n",
    "    embedding = OpenAIEmbeddings(),\n",
    "    by_text = False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "retriever = vectorstore.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Augment the promt with the additional context"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_variables=['context', 'question'] messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context', 'question'], template=\"You are an assistant for question-answering tasks. \\nUse the following pieces of retrieved context to answer the question. \\nIf you don't know the answer, just say that you don't know. \\nUse three sentences maximum and keep the answer concise.\\nQuestion: {question} \\nContext: {context} \\nAnswer:\\n\"))]\n"
     ]
    }
   ],
   "source": [
    "template = \"\"\"You are an assistant for question-answering tasks. \n",
    "Use the following pieces of retrieved context to answer the question. \n",
    "If you don't know the answer, just say that you don't know. \n",
    "Use three sentences maximum and keep the answer concise.\n",
    "Question: {question} \n",
    "Context: {context} \n",
    "Answer:\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)\n",
    "\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's generate!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/langchain_core/_api/deprecation.py:119: LangChainDeprecationWarning: The class `ChatOpenAI` was deprecated in LangChain 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import ChatOpenAI`.\n",
      "  warn_deprecated(\n",
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/pydantic/main.py:1070: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/\n",
      "  warnings.warn('The `dict` method is deprecated; use `model_dump` instead.', category=PydanticDeprecatedSince20)\n",
      "/Users/ktuna/projects/github_repos/simple_rag_implementation/env/lib/python3.9/site-packages/pydantic/main.py:1070: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/\n",
      "  warnings.warn('The `dict` method is deprecated; use `model_dump` instead.', category=PydanticDeprecatedSince20)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"The president honored Justice Breyer for his service and dedication to the country. He mentioned nominating Judge Ketanji Brown Jackson to continue Justice Breyer's legacy of excellence. The president did not provide specific comments about Justice Breyer's work or character.\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
    "\n",
    "rag_chain = (\n",
    "    {\"context\": retriever,  \"question\": RunnablePassthrough()} \n",
    "    | prompt \n",
    "    | llm\n",
    "    | StrOutputParser() \n",
    ")\n",
    "\n",
    "query = \"What did the president say about Justice Breyer\"\n",
    "rag_chain.invoke(query)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain_env",
   "language": "python",
   "name": "langchain_env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}