{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Lab 3 for Week 1 Day 4\n", "\n", "We're going to build a simple agent that chats with my linkedin profile.\n", "\n", "In the folder `me` I've put my resume `Profile.pdf` - it's a PDF download of my LinkedIn profile.\n", "\n", "I've also made a file called `summary.txt` containing a summary of my career." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

Looking up packages

\n", " In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n", " and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n", " ChatGPT or Claude, and you find all open-source packages on the repository https://pypi.org.\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Importing necessary packages\n", "# Gradio is used to create simple user interfaces to interact with what is being built.\n", "# pypdf used to load pdf files\n", "\n", "import os\n", "import boto3\n", "import gradio as gr\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "from pypdf import PdfReader" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Loading environment variables and initializing openai client\n", "load_dotenv(override=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Importing amazon bedrock and deepseek api keys for authentication\n", "amazon_bedrock_bedrock_api_key = os.getenv('AMAZON_BEDROCK_API_KEY')\n", "deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Amazon Bedrock Client\n", "\n", "bedrock_client = boto3.client(\n", " service_name=\"bedrock-runtime\",\n", " region_name=\"us-east-1\"\n", ")\n", "\n", "# Deepseek Client\n", "\n", "deepseek_client = OpenAI(\n", " api_key=deepseek_api_key, \n", " base_url=\"https://api.deepseek.com\"\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "reader = PdfReader(\"me/Profile.pdf\")\n", "linkedin = \"\"\n", "for page in reader.pages:\n", " text = page.extract_text()\n", " if text:\n", " linkedin += text" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(linkedin)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n", " summary = f.read()\n", "print(summary)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "name = \"Ian Kisali\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This code constructs a system prompt for an AI agent to role-play as a specific person (defined by `name`).\n", "The prompt guides the AI to answer questions as if it were that person, using their career summary,\n", "LinkedIn profile, and project information for context. The final prompt ensures that the AI stays\n", "in character and responds professionally and helpfully to visitors on the user's website.\n", "\"\"\"\n", "\n", "profile_background_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n", "particularly questions related to {name}'s career, background, skills and experience. \\\n", "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n", "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n", "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "If you don't know the answer, say so.\"\n", "\n", "profile_background_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "profile_background_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "profile_background_prompt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This function handles a chat interaction with the Amazon Bedrock API.\n", "\n", "It takes the user's latest message and conversation history,\n", "prepends a system prompt to define the AI's role and context,\n", "and sends the full message list to the Anthropic Claude 3.5 Sonnet model.\n", "\n", "The function returns the AI's response text from the API's output.\n", "\"\"\"\n", "def chat(message, history):\n", " messages = (\n", " [{\"role\": \"assistant\", \"content\": [{\"text\": profile_background_prompt}]}] +\n", " [{\"role\": m[\"role\"], \"content\": [{\"text\": m[\"content\"]}]} for m in history] +\n", " [{\"role\": \"user\", \"content\": [{\"text\": message}]}]\n", " )\n", " response = bedrock_client.converse(\n", " modelId=\"anthropic.claude-3-5-sonnet-20240620-v1:0\",\n", " messages=messages\n", " )\n", " return response['output']['message']['content'][0]['text']" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This line launches a Gradio chat interface using the `chat` function to handle user input.\n", "\n", "- `gr.ChatInterface(chat, type=\"messages\")` creates a UI that supports message-style chat interactions.\n", "- `launch(share=True)` starts the web app and generates a public shareable link so others can access it.\n", "\"\"\"\n", "\n", "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### LLM Response Evaluation\n", "\n", "1. Be able to ask an LLM to evaluate an answer\n", "2. Be able to rerun if the answer fails evaluation\n", "3. Put this together into 1 workflow\n", "\n", "All without any Agentic framework!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Create a Pydantic model for the Evaluation\n", "\"\"\"\n", "This code defines a Pydantic model named 'Evaluation' to structure evaluation data.\n", "\n", "The model includes:\n", "- is_acceptable (bool): Indicates whether the submission meets the criteria.\n", "- feedback (str): Provides written feedback or suggestions for improvement.\n", "\n", "Pydantic ensures type validation and data consistency.\n", "\"\"\"\n", "\n", "from pydantic import BaseModel\n", "\n", "class Evaluation(BaseModel):\n", " is_acceptable: bool\n", " feedback: str" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This code builds a system prompt for an AI evaluator agent.\n", "\n", "The evaluator's role is to assess the quality of an Agent's response in a simulated conversation,\n", "where the Agent is acting as {name} on their personal/professional website.\n", "\n", "The evaluator receives context including {name}'s summary and LinkedIn profile,\n", "and is instructed to determine whether the Agent's latest reply is acceptable,\n", "while providing constructive feedback.\n", "\"\"\"\n", "\n", "evaluator_profile_background_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n", "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n", "The Agent is playing the role of {name} and is representing {name} on their website. \\\n", "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n", "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n", "\n", "evaluator_profile_background_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n", "evaluator_profile_background_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This function generates a user prompt for the evaluator agent.\n", "\n", "It organizes the full conversation context by including:\n", "- the full chat history,\n", "- the most recent user message,\n", "- and the most recent agent reply.\n", "\n", "The final prompt instructs the evaluator to assess the quality of the agent’s response,\n", "and return both an acceptability judgment and constructive feedback.\n", "\"\"\"\n", "\n", "def evaluator_user_prompt(reply, message, history):\n", " user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n", " user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n", " user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n", " user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n", " return user_prompt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This script tests whether the Google Generative AI API key is working correctly.\n", "\n", "- It loads the API key using `getenv`.\n", "- Attempts to generate a simple response using the \"gemini-2.5-flash\" model.\n", "- Prints confirmation if the key is valid, or shows an error message if the request fails.\n", "\"\"\"\n", "\"\"\"\n", "This line initializes an OpenAI-compatible client for accessing Google's Generative Language API.\n", "\n", "- `api_key` is retrieved from environment variables.\n", "- `base_url` points to Google's OpenAI-compatible endpoint.\n", "\n", "This setup allows you to use OpenAI-style syntax to interact with Google's Gemini models.\n", "\"\"\"\n", "gemini_client = OpenAI(\n", " api_key=os.getenv(\"GEMINI_API_KEY\"), \n", " base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", ")\n", "\n", "try:\n", " response = gemini_client.chat.completions.create(\n", " model=\"gemini-2.5-flash\",\n", " messages=[\n", " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", " {\n", " \"role\": \"user\",\n", " \"content\": \"Explain to me how AI works\"\n", " }\n", " ]\n", ")\n", " print(\"✅ API key is working!\")\n", " print(f\"Response: {response}\")\n", "except Exception as e:\n", " print(f\"❌ API key test failed: {e}\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This function sends a structured evaluation request to the Gemini API and returns a parsed `Evaluation` object.\n", "\n", "- It constructs the message list using:\n", " - a system prompt defining the evaluator's role and context\n", " - a user prompt containing the conversation history, user message, and agent reply\n", "\n", "- It uses Gemini's OpenAI-compatible API to process the evaluation request,\n", " specifying `response_format=Evaluation` to get a structured response.\n", "\n", "- The function returns the parsed evaluation result (acceptability and feedback).\n", "\"\"\"\n", "\n", "def evaluate(reply, message, history) -> Evaluation:\n", " messages = [{\"role\": \"system\", \"content\": evaluator_profile_background_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n", " response = gemini_client.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n", " return response.choices[0].message.parsed" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This code sends a test question to the AI agent and evaluates its response.\n", "\n", "1. It builds a message list including:\n", " - the system prompt that defines the agent’s role\n", " - a user question: \"do you hold a certification?\"\n", "\n", "2. The message list is sent to Deepseek `deepseek-chat` model to generate a response.\n", "\n", "3. The reply is extracted from the API response.\n", "\n", "4. The `evaluate()` function is then called with:\n", " - the agent’s reply\n", " - the original user message\n", " - and just the system prompt as history (no prior user/agent exchange)\n", "\n", "This allows automated evaluation of how well the agent answers the question.\n", "\"\"\"\n", "\n", "messages = [{\"role\": \"system\", \"content\": profile_background_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a certification?\"}]\n", "response = deepseek_client.chat.completions.create(model=\"deepseek-chat\", messages=messages)\n", "reply = response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "reply" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "evaluate(reply, \"do you hold a certification?\", messages[:1])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This function re-generates a response after a previous reply was rejected during evaluation.\n", "\n", "It:\n", "1. Appends rejection feedback to the original system prompt to inform the agent of:\n", " - its previous answer,\n", " - and the reason it was rejected.\n", "\n", "2. Reconstructs the full message list including:\n", " - the updated system prompt,\n", " - the prior conversation history,\n", " - and the original user message.\n", "\n", "3. Sends the updated prompt to Deepseek `deepseek-chat` model.\n", "\n", "4. Returns a revised response from the model that ideally addresses the feedback.\n", "\"\"\"\n", "\n", "def rerun(reply, message, history, feedback):\n", " updated_profile_background_prompt = profile_background_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n", " updated_profile_background_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n", " updated_profile_background_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n", " messages = [{\"role\": \"system\", \"content\": updated_profile_background_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = deepseek_client.chat.completions.create(model=\"deepseek-chat\", messages=messages)\n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This function handles a chat interaction with conditional behavior and automatic quality control.\n", "\n", "Steps:\n", "1. If the user's message contains the word \"certification\", the agent is instructed to respond entirely in Pig Latin by appending an instruction to the system prompt.\n", "2. Constructs the full message history including the updated system prompt, prior conversation, and the new user message.\n", "3. Sends the request to OpenAI's GPT-4o-mini model and receives a reply.\n", "4. Evaluates the reply using a separate evaluator agent to determine if the response meets quality standards.\n", "5. If the evaluation passes, the reply is returned.\n", "6. If the evaluation fails, the function logs the feedback and calls `rerun()` to generate a corrected reply based on the feedback.\n", "\"\"\"\n", "\n", "def chat(message, history):\n", " if \"certification\" in message:\n", " system = profile_background_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n", " it is mandatory that you respond only and entirely in pig latin\"\n", " else:\n", " system = profile_background_prompt\n", " messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = deepseek_client.chat.completions.create(model=\"deepseek-chat\", messages=messages)\n", " reply =response.choices[0].message.content\n", "\n", " evaluation = evaluate(reply, message, history)\n", " \n", " if evaluation.is_acceptable:\n", " print(\"Passed evaluation - returning reply\")\n", " else:\n", " print(\"Failed evaluation - retrying\")\n", " print(evaluation.feedback)\n", " reply = rerun(reply, message, history, evaluation.feedback) \n", " return reply" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "This launches a Gradio chat interface using the `chat` function.\n", "\n", "- `type=\"messages\"` enables multi-turn chat with message bubbles.\n", "- `share=True` generates a public link so others can interact with the app.\n", "\"\"\"\n", "\n", "gr.ChatInterface(chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.1" } }, "nbformat": 4, "nbformat_minor": 2 }