{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ImagineUI RAG Evaluation\n", "\n", "This notebook evaluates the RAG system's ability to match user design requirements with appropriate examples." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since many pre-built evaluators are built to evaluate question-answer pairs, we're going to need a custom evaluator that can score how well the provided designs match the user's intention." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This evaluator will analyze the responses based on five criteria:\n", "\n", "1. Visual Style Match (0-10): How well does the design's visual style align with what was requested?\n", "2. Layout Requirements (0-10): How well does the design's layout meet the specified needs?\n", "3. Color & Mood (0-10): How well do the colors and overall mood match the requirements?\n", "4. Specific Features (0-10): How well does the design include specifically requested features?\n", "5. Overall Match (0-10): Overall, how well does this design satisfy the user's needs?\n", "\n", "The overall scores are then averaged to assess the application's performance." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "from langchain.evaluation import StringEvaluator\n", "from langchain_openai import ChatOpenAI\n", "from langchain.smith import RunEvalConfig\n", "from typing import Dict, Any\n", "import json\n", "\n", "class DesignMatchEvaluator(StringEvaluator):\n", " \"\"\"Evaluates how well a retrieved design matches the user's requirements.\"\"\"\n", " \n", " def __init__(self):\n", " self.llm = ChatOpenAI(model=\"gpt-4\", temperature=0)\n", " \n", " def _evaluate_strings(\n", " self, prediction: str, reference: str, input: str, **kwargs\n", " ) -> Dict[str, Any]:\n", " \"\"\"Synchronous evaluation - required by StringEvaluator\"\"\"\n", " raise NotImplementedError(\n", " \"This evaluator only supports async evaluation. Use aevaluate_strings instead.\"\n", " )\n", " \n", " async def aevaluate_strings(\n", " self, prediction: str, reference: str, input: str, **kwargs\n", " ) -> Dict[str, Any]:\n", " \"\"\"Evaluate the match between user requirements and retrieved design.\"\"\"\n", " \n", " # Fix the JSON template in the prompt\n", " prompt = f\"\"\"You are evaluating a design recommendation system.\n", " \n", " USER REQUIREMENTS:\n", " {input}\n", " \n", " RECOMMENDED DESIGN:\n", " {prediction}\n", " \n", " Score how well the recommended design matches the user's requirements in these categories:\n", " 1. Visual Style Match (0-10): How well does the design's visual style align with what was requested?\n", " 2. Layout Requirements (0-10): How well does the design's layout meet the specified needs?\n", " 3. Color & Mood (0-10): How well do the colors and overall mood match the requirements?\n", " 4. Specific Features (0-10): How well does the design include specifically requested features?\n", " 5. Overall Match (0-10): Overall, how well does this design satisfy the user's needs?\n", " \n", " Provide scores and brief explanations in JSON format exactly like this:\n", " {{\n", " \"visual_style\": {{\"score\": 7, \"reason\": \"The design aligns with...\"}},\n", " \"layout\": {{\"score\": 8, \"reason\": \"The layout provides...\"}},\n", " \"color_mood\": {{\"score\": 6, \"reason\": \"The colors match...\"}},\n", " \"features\": {{\"score\": 7, \"reason\": \"The design includes...\"}},\n", " \"overall\": {{\"score\": 7, \"reason\": \"Overall, this design...\"}}\n", " }}\n", " \n", " Return only valid JSON, nothing else.\n", " \"\"\"\n", " \n", " response = await self.llm.ainvoke(prompt)\n", " try:\n", " scores = json.loads(response.content)\n", " # Calculate normalized score (0-1)\n", " normalized_score = scores[\"overall\"][\"score\"] / 10.0\n", " return {\n", " \"score\": normalized_score,\n", " \"visual_style_score\": scores[\"visual_style\"][\"score\"] / 10.0,\n", " \"layout_score\": scores[\"layout\"][\"score\"] / 10.0,\n", " \"color_mood_score\": scores[\"color_mood\"][\"score\"] / 10.0,\n", " \"features_score\": scores[\"features\"][\"score\"] / 10.0,\n", " \"reasoning\": scores\n", " }\n", " except Exception as e:\n", " print(f\"Error parsing evaluation response: {e}\")\n", " print(f\"Response content: {response.content}\")\n", " return {\"score\": 0, \"error\": str(e)}" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "# Create test dataset\n", "test_queries = [\n", " \"I need a minimalist design with lots of whitespace, using a monochromatic color scheme\",\n", " \"Looking for a playful, colorful design with rounded elements and a friendly feel\",\n", " \"Need a professional business design with a dark theme and sharp angles\",\n", " \"Want a nature-inspired design with organic shapes and earthy colors\",\n", " \"Looking for a tech-focused design with a futuristic feel and neon accents\",\n", " \"I want the craziest design you can find!\",\n", " \"I'd like an eye-catching design for a small business with some quirky personality. Make us stand out from the crowd!\",\n", " \"I want something clinical and informative. It should focus on clear presentation of information and reader usability.\"\n", "]\n", "\n", "# Initialize evaluator\n", "evaluator = DesignMatchEvaluator()\n", "\n", "# Configure evaluation\n", "eval_config = RunEvalConfig(\n", " evaluators=[evaluator]\n", ")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loaded 141 design documents\n", "Created dataset: design_evaluation_1740511422\n", "Evaluating query 1/5: I need a minimalist design with lots of whitespace...\n", "Generated query: minimalist design whitespace monochromatic color scheme aesthetic clean simple modern design category theme key visual characteristics visual impact desired specific preferences constraints minimalistic design whitespace monochromatic color scheme.\n", " LangSmith error: 'NoneType' object has no attribute 'id'\n", " Score: 0.90\n", "Evaluating query 2/5: Looking for a playful, colorful design with rounde...\n", "Generated query: playful colorful design rounded elements friendly feel\n", " LangSmith error: 'NoneType' object has no attribute 'id'\n", " Score: 0.80\n", "Evaluating query 3/5: Need a professional business design with a dark th...\n", "Generated query: \"professional business design dark theme sharp angles\"\n", " LangSmith error: 'NoneType' object has no attribute 'id'\n", " Score: 0.80\n", "Evaluating query 4/5: Want a nature-inspired design with organic shapes ...\n", "Generated query: nature-inspired design organic shapes earthy colors organic design aesthetic natural theme earth tones earthy color palette organic shapes nature-inspired visuals serene mood organic design elements nature-inspired color scheme\n", " LangSmith error: 'NoneType' object has no attribute 'id'\n", " Score: 0.75\n", "Evaluating query 5/5: Looking for a tech-focused design with a futuristi...\n", "Generated query: \"tech-focused design futuristic neon accents\"\n", " LangSmith error: 'NoneType' object has no attribute 'id'\n", " Score: 0.80\n", "Saved results to evaluation_results_1740511422.csv\n", "\n", "--- EVALUATION SUMMARY ---\n", "Number of queries evaluated: 5\n", "\n", "Average Scores:\n", "Overall: 0.81\n", "Visual Style: 0.84\n", "Layout: 0.80\n", "Color & Mood: 0.88\n", "Features: 0.70\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAJOCAYAAACqS2TfAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAdIJJREFUeJzt3QncTOX///GPXcqa7EIpS9Zs0YISlYpWrSRpkVIqpSyhUsnSYkmRNpEWLUQSbZSQSqKkIlkruxDn/3hfv/+Z78yY+3bjPvfMPfN6Ph7jNmfOzH3OmXOf63yu5XPl8DzPMwAAAAAAkOlyZv5HAgAAAAAAIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIehGtvHggw9ajhw5suR3NWvWzD18s2fPdr/7jTfeyJLff91111nFihUtkW3bts1uuOEGK1WqlDs2d9xxhyW76PPit99+c/s+bty4uG4XAABZcQ+2cePGeG8KkC0RdCMuFKTo4u0/8ufPb2XKlLFWrVrZU089ZVu3bs2U3/Pnn3+6gmLRokWWaBJ52zLikUcecd/jLbfcYi+//LJde+216a6/Z88e9902aNDAChYsaEcddZT7v5bptWSnAL1jx452/PHHu/NdlRVnnHGG9e3bN96bBgAxjRgxwpXRjRo1ivemJJzdu3fbk08+aXXr1rVChQpZkSJF7KSTTrIbb7zRli5dGu/Ny/befvttO/fcc6148eKWN29ed494+eWX28cff5xy91tIDrnjvQFIbf3797dKlSq5oGvt2rWuRVktpkOGDLF3333XatWqFVq3V69edt999x30hbZfv36u1bhOnToZft+HH35oQUtv25577jnbt2+fJTIVfKecckqGgsbt27db69at7ZNPPrHzzz/fteTnzJnTpk2bZt26dbO33nrLpkyZYkceeaQlo+XLl7sKhiOOOMKuv/56952vWbPGFi5caI899pg7DwAg0bz66qvuejVv3jx3HatcuXK8NylhXHLJJfbBBx/YlVdeaZ07d3b3MQq233//fWvSpIlVrVo13puYLXme58pJVeqrQqN79+6uklplpgLxs846y7744gt3jIO+FwQyE0E34kq1mPXr1w8979mzpwvmFJhdeOGF9uOPP7pARXLnzu0eQdqxY4cVKFDA1arGU548eSzRrV+/3qpXr56hdVVoKuB++umnrWvXrqHlaiUfPny4W3b33XfbyJEjLSsL9n///Td0fgVp6NChrju+atkrVKiw33HMSqoASdbKDQCZ59dff7U5c+a4StGbbrrJBeBZ3TNHlc9qUVbvoETy9ddfu+D64Ycftvvvvz/itWeeecY2bdqUZduickz3LKrITgaDBw92AbffABM+rPCBBx5wPeuCvheMl6y8L0HWS46/UCSVM88803r37m2///67vfLKK+mO6Z4xY4addtpprluXuitXqVIlVACq1Vyti6JuvX5Xdn/8rcbm1qhRwxYsWOC6+SrY9t8bPXbXt3fvXreOal0VuKhiYNWqVRHrqCZVLbnRwj/zQNsWa0y3gqW77rrLypcvb/ny5XP7+sQTT7iLdDh9joLYyZMnu/3TuuryplbljFAQ2KlTJytZsqS70aldu7a9+OKL+41v1w2ZWqf9bVf36Vj++OMPGzNmjPtewwNu36233mrNmze3559/3q0r2m4ti3UDVrZsWbv00ksjlg0bNszto7ZX260bxH/++SfivTqeqsyZPn26q+hRofbss8+611544QW3fSVKlHDHS5UJmVkB8Msvv1i5cuX2C7hFvzOaWk+aNm3quuGr26LOlfHjx0esM2nSJKtXr57bD3W/u+aaa2z16tUR6+g80t+Ffv95553nPu/qq68+qOM2f/58N+xDv0O/Sz1T1AoBILkpyC5atKjrpaRrrp771KpbrFgxV35F27Jli7umqCLVt2vXLhewq6Vc11iVYz169HDLY5Vf+l26Nmldv+xSeafWzaOPPtpdi3T9i5VnZefOnXb77be7a5aueSqndW3UZ+s+IpyW63qm659fVo4dO/aAx0bXVDn11FP3ey1XrlxuG6N/j8pVdZHW79F1VJXOqlDwrVixwi677DJ3XHU/op5kKmPD+eXvhAkTXO8/lYdaV8dcvvrqKzvnnHOscOHCbrnKEbUKh9PwPQW0KhO1LSqDzj77bNfzKiM0plvdvFU2aT/VW02Bok+/U/cNsei+ReVJWvTdDRw40PUS0PcdK4+PhrI1bNjQ/f/vv/9251nNmjVdWadtUmPOt99+G3HM0rvfyuhx8z9L9w86vzVUTPcQse5N//vvPxswYIBbR8dYx1r3jtHne1r3JYdzDJHAPCAOXnjhBUWK3tdffx3z9VWrVrnXL7300tCyvn37umW+xYsXe3nz5vXq16/vPfnkk96oUaO8u+++2zvjjDPc62vXrvX69+/v3nPjjTd6L7/8snv88ssv7vWmTZt6pUqV8o455hjvtttu85599llv8uTJodf08M2aNct9Ts2aNb1atWp5Q4YM8e677z4vf/783oknnujt2LEjtG6FChW8Dh067LdP4Z95oG3T+/U5vn379nlnnnmmlyNHDu+GG27wnnnmGe+CCy5w77/jjjsifo+W1a5d2ytdurQ3YMAAb9iwYd5xxx3nFShQwNu4cWO634v2o1q1al6ePHm8O++803vqqae8008/3X2mPsffdm1r8eLFvTp16oS2fdu2bTE/c/To0e7948aNO+D58Nxzz7nnOjY5c+b01qxZE7HeJ5984tabNGlSaJmOR+7cub3OnTu7c+Dee+/1jjzySK9Bgwbe7t27I76XypUre0WLFnXfndbV9ypa97rrrvOGDh3qPf30017Lli3d79FxTus7lF9//dWtp+1Pj77jXLlyeTNnzkx3Pf9Y6HuuUaOG9/DDD3vDhw93+3jttdfud7y03dpm7c8RRxzhVaxY0fvnn39C6+k8ypcvn3f88ce7/2ufX3rppQwft3Xr1rnjpXN80KBB7vt54IEH3DkCILlVrVrV69Spk/v/p59+6q458+bNC71+/fXXe0WKFPF27doV8b4XX3wxonzfu3evu6aqDFJ5pbK2a9eu7vrTpk2biPfqfbq+qFzu16+fu/5988037rVy5cp5Xbp0cddllcENGzZ067///vsRn3H55Ze75bpm6v16rjJRy3Qf4VNZps8sX768K3NGjhzpXXjhhW49XVfTM2fOHLeerp979uxJd93Vq1d7ZcqUCe2/rre9e/d2++lfr7UtJUuW9AoWLOiusdo/bbPKwbfeemu/e5Hq1au78lfrDRw40Nu+fbsrX3RP1LhxY2/w4MFuH3S/omVfffVV6DOuuuoqt6x79+7e888/7z322GPufuKVV15Jdz/8ezDdB2l9fQ/XXHNN6Fj7VE5o2ffffx/xfp07Wu6XQbF8+OGHbh19Hxmhc0zlm8pAnVd6X9myZb3ChQu7456R+62MHreFCxe68lTl7KOPPurKZ32v/rkVTuWtfw+rc7B9+/buedu2bSPWS+u+5HCOIRIXQTcSMugWXTTr1q2bZtCtC6Oeb9iwIc3P0OenFRQpeNJrusjFei1W0K2L+ZYtW0LLX3/9dbdcQf/BBN0H2rbooFuVAVr3oYceilhPF3QFaMuXLw8t03oqLMKXffvtt265Asr0KLDWeuGFrwIwFUZHHXVUxL5r+1q3bu0diG4y9Jn+jVMsKsy0jm4CZNmyZTG3Vzdc2g6/kuOzzz5z67366qsR602bNm2/5dpeLdNr0cIrTXytWrVylRWZEXSrgkhBsdbVjVK3bt3cd6obpXCbNm1yN12NGjXydu7cGfGaKl7876NEiRIuKA9fRzee+vw+ffrsV/CrMA+X0eP29ttvH/DvFEDymT9/vvvbnzFjRuj6owBV1y7f9OnT3TrvvfdexHvPO++8iGunAhwFj7ruhFPZq/d/8cUXoWV6rnV/+OGHA16ndS3UdVAV0r4FCxbErIxWpWp00K0KBVVOR1dGX3HFFe7+I1a54NPx8O8hFCxfeeWVLrj6/fff91tXAZf2KdZ11L+u++Vk+DHaunWrV6lSJRfkqeIi/F5Exzd8+/Q5J5xwgiu3/M/0j5k+4+yzzw4t077deuut3sHy78FUMRFdLmu57jP8ckwNEqrIDXf77be7it20KuhF91L6LJU9GfHvv/+Gjk14uazgODxwT+t+62COmyoaVHHiB/Py888/u8qj8HvTRYsWueeq2A6nRiEt//jjjw94X3I4xxCJi+7lSFjqKpReFnN1KZd33nnnkJOOqdtPrO5xaWnfvr3rruZTl7vSpUvb1KlTLUj6fHVZU5e5cOpurvsUdUcO16JFC9etyaeEdOp2pe5rB/o96jqvxDDh48v1ezUmWeOyD5b/HYYft2j+a34XuRNPPNElO5k4cWJE1351JbzgggtC453UxVrdwdQ1Tl3e/Ie6Her8mTVrVsTvUZe+WN2ywsdPbd682X2GunfpeOn54VKXRY3nVhdwdcNXxtu2bdu6Lo1Kmhc+XELHSwkDo8cw+t3X1N1bQwC6dOkSsY66gKpLXnR3RFE3xnAZPW7+35jGLqZChnkA/0fdu3V98of56PrTrl07161Z12LRkBx14Q6/Tmt4iq5jWjf8elOtWjV3fQq/3uj9En2d1rU3Vr6Q8Ou0fo+uzaeffnpEt2i/K7quj+Fuu+22iOcqN998801Xnuj/4dulMkKfnV53ax0PdQl+6KGHXBf81157zQ2V0hAi7bs/plv3Jhrqpd8Tnr8m/HP8slddpjVczqdrsTKhq8xYsmRJxPs6dOgQcTxUvvz888921VVX2V9//RXaFw1LU+KxTz/9NHSfpOu6ulMrudih0H7GOrb+fZDKljZt2rhj4g9/0zmj80TlXno5Rfx7gPTuF6Lv4fyx7Pod2nd/qGFGustn9Ljpsz/66CO3/Roi4NNwCXVnD+cfB+Wyib5fk+gyOtZ9yeEcQyQugm4kLAV56V14VbBpPJXmitbNwRVXXGGvv/76QQXgGg91MEnTTjjhhP0KTF100xrPnFk0vl0X+ujjoRsZ//Vwxx577H6foRuD6PG6sX6P9jE6IUtavycj/G1OrwIlVmCu71djqvxxyhpLpWAz/GZOhaVujjQm7Zhjjol46PyJTlKmwi0W/R5VVKgg0w2J3u+P78+MoNuvSFACGBXo3333nZtyTclgdFOlwjx8nKDGtKfF/w50UxFNN7XR35F+h8aTh8vocdPNrzL0Kuurbq51E6Dx79Hj0gAkD93cK7hWwK3cHcparoemDVu3bp3NnDkzdG3R9UEV3/41QUnXVEEXfZ3+4Ycf9rvW6JooGb1Oq/JP45xV2ahxz/oM5d4Iv0br+qfyK/ozorOub9iwwQXGo0eP3m+7/Ir4AyW5VMCnxF5K+KoAVgGStk/3IX7+Ev0eBZLpXdP97Y51TU+r7I3ePx1jPxiP3h/lS9H34x+nxx9/3BYvXuzG1SvQ15jkA1XIp3cfpAp+HfPw+yA1UKxcudI+++wz91xlnM6dA00tqsYByei0sbrfU6JSbZO+D5VT2meVsRkpuzN63HQuaLx5rOz90cv8czB6uRo0dH9xoO/ycI8hEldypv9DtqeEWrrQpTc9iWp5VQupWnLVHKqGW7WAqj3XlF9qGT6QIDJExkr84d/IZGSbMkNavyc66VpW8G8aVAimNVWHXpPw1g3dtCmbvVpJlPRFNzKq/VWyk/ACV4FjeIKfcCo4D/R9K9BVjbYCVmVK1Y2IKmJUW63CPLOnbtN3o6QvejRu3Njd2Gr7FfQHIbwl4GCPm85l9S748ssv7b333nMtO0o6pOyyWqYWBQDJRTOIaHomBd56RNN1o2XLlu7/quxW4if1tlILnK7TupaGJ4HS9UbXO11fY9E190DXaQUeSoimpKeaO1w9zNQLS5WA0UkmM8K/rqv3kQKuWMKnLD0QbY+OhSoh1LNJxyE8UVdmiz5G/v4MGjQozXLWv14rCZp6CGj6Ld0r6T2aulIVJtGttod6z6OWWzWGKBmuvjP9VNB5oHLOn2bt+++/d+fTgajyWol3VS4pcZkqY1Te6Z4hI2V3Ro9beKK4w70XzOh96KEeQyQugm4kJLUIyoEyNOriqoBJDxXougCr5lmBuC5MGb3oZZRfKxoexKoFILxwVotyrOlCVLt53HHHhZ4fzLapy5pqOVX7G94arDlB/dczgz5HAbAKovBA7XB+jwpxBZr6TlVzG8tLL73kWk3CA2rV/qoWXhUpajXQDYEKYQWR4TXsOi7q8XCoFSgKJlWbrXnhw3sIRHd5DILf3VA3uOIPCVArRFoVTv53sGzZslD3TJ+WZeQ7OtjjptYbPTQ9jm5wlQFdN+PqZQIguSioVqWcpnOMpuuwgrVRo0a5a4eCAQWcuk6ra7QCdpXB0dcbZZNWOX2oZbK6gquFWxV/4WWAgu5wuv6p/FILfXiLrMrp6IpFlaWqDM/MIEYVAbof0L2CejXpOKr1Vtf09Gi7df2OltGy1y879Lsysj/6ztQFXw+14p588snu+p6RoFv7Ft46q2OrYx4+44rKfHXZVsWDAnp1sddc5gdqeNA55HfXV2+zA62vSmFVXGuGlHC6B1Orty+t8y6jx03fo86/6PNIopf556COk9/oIGql1nZl9D7qUI8hEhfdy5FwVGirxlIXdX96o1g0VUQ0v6bS7+rmj3vJrDkzFRyGd3vSBV8BU3hBpYu4WgHDpwJRt7joqcUOZts03ZNuDjT/Zzi1xKowOZTa6bR+z9q1ayPG6GnqC82vrdpedTc+WGrFUHc9BXmxpuHSzZu+c02nEt0NWq3dOpaawkU3MOFdFv0aex0XnS/RtN0ZObZ+ARbeC0C9LKJv5g6HWmlijYn2x3753QrVeqQbQU2ZEl2z7m+fAnXdAOi4hXfzVkuTujlqbPeBZPS4aThCdO+I6L8xAMlDXWgVWGsaI+UsiX6oAlRloCopRZWzWq7KS1Ws6voR6zqtYULh+SvCf5/Gz2bkOq2yzh9PLurOrEAknF9Rr9bwcCrDoj9PrdIK5mMFxOoWnh4FVOr6G03Xzrlz57rAUYG9jo8qi3V8lI8jmn99Vdk7b948916fjou6vyuYjTXGPZzycejeQ9NsaYhQWvuj4xfd7VrliYavZfSaHl0Z4x/b6PsQdYNWGaKpKLVN6lVwIJqu695773VlmX7G6p2nFl8dK/97jF5HveOip89M634ro8dNv0dBuc638LHwCrijc+rouxRNyRnO7+mRkTL6cI4hEhct3YgrXaxUk6uCWrWACr6UhEU1gSrUo5NJhevfv7/rXq4LmNZXba0KWgVufjISXUw1hkYBioIZXXg1Li2tMTQHoq5L+mwFkdpeXVTVIqnaR59a/xSMq9VWNxvqvqxCIjyx2cFum5KwqDZXLQi60VDXPXUL01g6daOK/uxDpfHF6iqo+Z01f7kKe+2LxjxrXzOa3CSaKgf0PatWXcMA/BZttVpoHxTMq8tyNB0/zcGph459dE203qfCSEGqEqIoaFVLg26IVPAqYVn4nN6x6D3qTq5j7BdsujnUjYjfAn24VEut43nxxReHekUoyYsqcbRf+g792nYdK51DmldUtdy6eVMr0Y4dO9x86do/fZ7OQe2/kt7pXNS+6vu68847D7g9GT1u+n36m7rooovcOaabbR0bbad/YwEgeajc1d+5unLHoh4vCibVGu4H1/qpwEvzcKsbeXjrnh84qLv1zTff7HoQqYeNgj+VCVruz1GcHpXzClpUdui6qPJewZ/KX394kh9EKZhWeaXEWNpeJQD96aef9mvxfPTRR932qNxVGa7AVpX5ujarkjhWxb5P12RthwJNddXWdVyBnq6ZCsr0+/0KXfXAU3mt667KWB0flS261n7++efuPkDJM9W6q89T4lJ9nj5LLfaqGIgeIhRNr2sMst6v7u0qH5SzRtukfdQ1W4G/vlvdI+n6rvsIVaZrX7/++uuYZXAs2iadH/ouVEmg+xsdi+h5pevWrevGsvuJ9NSanhH33HOPywGg7dG2a1vVrVoNAgp6FXDPmTPHravKId0Lan81h7u6pevcDO9VeKD7rYwcN9HYd32POn+VnNRvCNE+qhz16ThoyIIqTBTk63vXNuv7VAWMn5wwIw71GCJBxTt9OlJ7yjD/oSmuNGe2pmfQlBHhU1OlNWWY5lbUHJ+aJ1Hv109N2/HTTz9FvO+dd95xc1r60zr4U0Zouo+TTjop5valNWXYa6+95vXs2dNN2aQpoDRlVqwpQjTXo6YX07QVp556qpt+Jfoz09u26CnD/OlDNHe29lPzaGuaC82dHD7NhehzYk0HktZUZtE0N3PHjh3dPNw6rpqTM9aUWBmdMsynuVw1zVu9evXclBeaeuPkk09205SFz6cdTccv1vQb0XOB63P1nWjKLW1zjx49vD///DND2/vuu++6eTk1RYemZ9G8pWPHjnW/V9OPHO6UYZoSR9+JprfRdC36/o499lg3jY0/V2j09jRp0sTtT6FChdx8tDr3wk2cONFNqadzrFixYt7VV1/t/fHHHxHr6PvWsT7U46ap3PQ3pW3V79F5f/7557vzGUDy0bRIug5GT2cYTtctXcP8qbZUBmmu61jTWvp0jdd1VWWuriWal1jXHs3FvXnz5gOWXzJmzBhX7un9mkNc193o+wLRtuszdF3UFJOaG9mfhlLzK0eXd1pX26990n3IWWed5a6N6dH79FkqDzTtmMpw7ZOmL3vjjTf2W1/3CZo6TPOPa/s15Zd+b/gc5yoLNA2o5j7Xd6DrfvQc5P69yKRJk2Jul6bmvPjii72jjz7a/R6Ve5qnXPdLot93zz33uLmldc1X+aD/jxgxwjsQ/1gvWbLEbafer33WnOvRU1z6Hn/8cfeeRx55xDtYOo6a313fo46vjnO7du282bNnR0wZdtddd7nXVI7pfmHu3LkHdb+VkePm03OVu7o30vzgmudcv1/fVzjN3a5zW9OO6bzS+aV7R23vwd5HHc4xRGLJoX/iHfgDAAAAQVBLpFoN1Sqb3rA1ZC71mlLvK/XQizWrSjJQ67Va5qNz/mSWVDiGqYIx3QAAAEgKGiceTd291QVbid+QNdSmpwRn6l6dLMFi9LmlQFu5WZo1axbI70vGY5jKGNMNAACApKB5qJVDQ2NnNSuGcsfoofHU0dOTIfMpAZxyA2hMtMZYK29LstBYceW80U/NSKPksMoJ06NHj0z9Pcl8DFMZ3csBAACQFJSMtV+/frZkyRKXGFMthErmpkSkCsIRLHWDVoIyJS5T8lRNRZYslGhNgbCSumnqusaNG7tEeZmd4CyZj2EqI+gGAABuNohBgwa5VkJlV9Z8zBqvmJ7Zs2db9+7d3ZhGtSL26tXLtQQBAID/YUw3AABwXRo13U30PLzpTR2kqZzUjVeJqjT1naa70xRQAADgf2jpBgAAETSf8YFauu+9916bMmWKLV68OLTsiiuucHPTTps2LYu2FACAxJdyg1v27dtnf/75pxUsWNDdVAAAkJ2ornzr1q1WpkwZl5E5XubOnWstWrSIWNaqVSvX4p2WXbt2uUd4mfz333/b0UcfTZkMAEjaMjnlgm4F3GSvBABkd6tWrbJy5crF7fcrmVDJkiUjlun5li1b3NQ6RxxxxH7vGThwoEtyBQBAKpXJKRd0q4XbPzCFChWK9+YAAHBQFNSq8tgvz7KTnj17usRrvs2bN7vs0pTJAIBkLpNTLuj2u6+pcKeABwBkV/Hujl2qVClbt25dxDI9V9kaq5VbNM2OHtEokwEAyVwmk70cAAAcNM1RO3PmzP3mSNZyAADwPwTdAADAtm3b5qb+0sOfEkz/X7lyZahrePv27UPr33zzzbZixQrr0aOHLV261EaMGGGvv/663XnnnXHbBwAAEhFBNwAAsPnz51vdunXdQzT2Wv/v06ePe75mzZpQAC6VKlVyU4apdVvzew8ePNief/55l8EcAACk8DzdGuxeuHBhl7yF8WMAgOwmmcqxZNoXAEDq2ZLBcoyWbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAABIxqB75MiRVqtWLStUqJB7NG7c2D744IN03zNp0iSrWrWq5c+f32rWrGlTp07Nsu0FAAAAACDbBN3lypWzRx991BYsWGDz58+3M88809q0aWM//PBDzPXnzJljV155pXXq1Mm++eYba9u2rXssXrw4y7cdAAAAAIADyeF5nmcJpFixYjZo0CAXWEdr166dbd++3d5///3QslNOOcXq1Kljo0aNytDnb9myxQoXLmybN292resAAGQnyVSOJdO+AABSz5YMlmMJM6Z77969NmHCBBdUq5t5LHPnzrUWLVpELGvVqpVbnpZdu3a5gxH+AAAAAAAgK+S2OPv+++9dkP3vv//aUUcdZW+//bZVr1495rpr1661kiVLRizTcy1Py8CBA61fv36Zvt1Ifp3GfW2JbMx1DeK9CQAAAAAOIO4t3VWqVLFFixbZV199Zbfccot16NDBlixZkmmf37NnT9fc7z9WrVqVaZ8NAAAAAEBCt3TnzZvXKleu7P5fr149+/rrr+3JJ5+0Z599dr91S5UqZevWrYtYpudanpZ8+fK5BwAAAAAAKdfSHW3fvn1uHHYs6oY+c+bMiGUzZsxIcww4AAAAAAAp29Ktrt/nnnuuHXvssbZ161YbP368zZ4926ZPn+5eb9++vZUtW9aNy5Zu3bpZ06ZNbfDgwda6dWuXeE1TjY0ePTqeuwEAAAAAQOIF3evXr3eB9Zo1a1yq9Vq1armA++yzz3avr1y50nLm/F9jfJMmTVxg3qtXL7v//vvthBNOsMmTJ1uNGjXiuBcAAAAAACRg0D1mzJh0X1erd7TLLrvMPQAAAAAASHQJN6YbAAAAAIBkQdANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAhI7qA+GAAAAAAOV8X7psR7ExLCb4+2jvcm4BDR0g0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEByB/XBAAAAyJ4q3jcl3puQEH57tHW8NwFAEqClGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAhI7qA+GAAAZC/Dhw+3QYMG2dq1a6127dr29NNPW8OGDdNcf9iwYTZy5EhbuXKlFS9e3C699FIbOHCg5c+f3+Kl4n1TLNX99mjreG8CACAMLd0AAMAmTpxo3bt3t759+9rChQtd0N2qVStbv359zPXHjx9v9913n1v/xx9/tDFjxrjPuP/++7N82wEASGQE3QAAwIYMGWKdO3e2jh07WvXq1W3UqFFWoEABGzt2bMz158yZY6eeeqpdddVVVrFiRWvZsqVdeeWVNm/evCzfdgAAEllcg251QWvQoIEVLFjQSpQoYW3btrVly5al+55x48ZZjhw5Ih7x7MYGAEB2t3v3bluwYIG1aNEitCxnzpzu+dy5c2O+p0mTJu49fpC9YsUKmzp1qp133nlZtt0AAGQHcR3T/cknn9itt97qAu///vvPdUlTTfmSJUvsyCOPTPN9hQoVigjOFXgDAIBDs3HjRtu7d6+VLFkyYrmeL126NOZ71MKt95122mnmeZ4rx2+++eZ0u5fv2rXLPXxbtmzJxL0AACAxxTXonjZt2n6t2GrxVs35GWeckeb7FGSXKlUqC7YQAADEMnv2bHvkkUdsxIgR1qhRI1u+fLl169bNBgwYYL17906zh1u/fv2yfFuBeCGx3/8huR9SXUKN6d68ebP7WaxYsXTX27Ztm1WoUMHKly9vbdq0sR9++CGLthAAgOSjzOO5cuWydevWRSzX87QquRVYX3vttXbDDTdYzZo17aKLLnJBuALrffv2xXxPz549XVnvP1atWhXI/gAAkEgSJuhWAX3HHXe4pCw1atRIc70qVaq4pC7vvPOOvfLKK+59Glf2xx9/xFxf3djUfS38AQAA/idv3rxWr149mzlzZmiZylc9b9y4ccz37Nixw437DqfAXdTdPJZ8+fK5IWLhDwAAkl3CzNOtsd2LFy+2zz//PN31VPiH3wAo4K5WrZo9++yzrktbNLqyAQBwYJourEOHDla/fn03N7fm4N6+fbvLZi7t27e3smXLunJVLrjgApfxvG7duqHu5Wr91nI/+AYAAAkSdHft2tXef/99+/TTT61cuXIH9d48efK4Al+FfVpd2XQj4VNLt7qlAwCA/2nXrp1t2LDB+vTpY2vXrrU6deq43Ct+crWVK1dGtGz36tXL5VjRz9WrV9sxxxzjAu6HH344jnsBAEDiiWvQre5nt912m7399tsuIUulSpUO+jOUbfX7779Pc4oSdWXTAwAAHLgSXI9YVE6Hy507t/Xt29c9AABAggbd6lI+fvx4Nz5bc3WrZl0KFy5sRxxxRMzubP3797dTTjnFKleubJs2bbJBgwbZ77//7hK5AAAAAACQSOIadI8cOdL9bNasWcTyF154wa677rqY3dn++ecf69y5swvQixYt6hK/zJkzx6pXr57FWw8AAAAAQIJ3Lz+Q6O5sQ4cOdQ8AAAAAABJdwkwZBgAAAABAsiHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABCQ3EF9MAAAAAAgMVS8b0q8NyEh/PZo6yz/nbR0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAAEh6AYAAAAAICAE3QAAAAAABISgGwAAAACAZAy6Bw4caA0aNLCCBQtaiRIlrG3btrZs2bIDvm/SpElWtWpVy58/v9WsWdOmTp2aJdsLAAAAAEC2Cbo/+eQTu/XWW+3LL7+0GTNm2J49e6xly5a2ffv2NN8zZ84cu/LKK61Tp072zTffuEBdj8WLF2fptgMAAAAAcCC5LY6mTZsW8XzcuHGuxXvBggV2xhlnxHzPk08+aeecc47dc8897vmAAQNcwP7MM8/YqFGjsmS7AQAAAADIdmO6N2/e7H4WK1YszXXmzp1rLVq0iFjWqlUrtxwAAAAAgEQS15bucPv27bM77rjDTj31VKtRo0aa661du9ZKliwZsUzPtTyWXbt2uYdvy5YtmbjVAAAAAABkg5Zuje3WuOwJEyZkerK2woULhx7ly5fP1M8HAAAAACChg+6uXbva+++/b7NmzbJy5cqlu26pUqVs3bp1Ecv0XMtj6dmzp+u27j9WrVqVqdsOAAAAAEBCBt2e57mA++2337aPP/7YKlWqdMD3NG7c2GbOnBmxTInUtDyWfPnyWaFChSIeAAAAAAAk/ZhudSkfP368vfPOO26ubn9ctrqBH3HEEe7/7du3t7Jly7pu4tKtWzdr2rSpDR482Fq3bu26o8+fP99Gjx4dz10BAAAAACCxWrpHjhzpunw3a9bMSpcuHXpMnDgxtM7KlSttzZo1oedNmjRxgbqC7Nq1a9sbb7xhkydPTjf5GgAAAAAAKdfSre7lBzJ79uz9ll122WXuAQAAAABAIkuIRGoAAAAAACQjgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAADAGT58uFWsWNHy589vjRo1snnz5qW7/qZNm+zWW2+10qVLW758+ezEE0+0qVOnZtn2AgCQHcR1nm4AAJAYJk6caN27d7dRo0a5gHvYsGHWqlUrW7ZsmZUoUWK/9Xfv3m1nn322e+2NN96wsmXL2u+//25FihSJy/YDAJCoCLoBAIANGTLEOnfubB07dnTPFXxPmTLFxo4da/fdd99+62v533//bXPmzLE8efK4ZWolBwAAkeheDgBAilOr9YIFC6xFixahZTlz5nTP586dG/M97777rjVu3Nh1Ly9ZsqTVqFHDHnnkEdu7d2+av2fXrl22ZcuWiAcAAMmOoBsAgBS3ceNGFywreA6n52vXro35nhUrVrhu5XqfxnH37t3bBg8ebA899FCav2fgwIFWuHDh0KN8+fKZvi8AACQagm4AAHDQ9u3b58Zzjx492urVq2ft2rWzBx54wHVLT0vPnj1t8+bNoceqVauydJsBAIgHxnQDAJDiihcvbrly5bJ169ZFLNfzUqVKxXyPMpZrLLfe56tWrZprGVd39bx58+73HmU41wMAgFRCSzcAAClOAbJaq2fOnBnRkq3nGrcdy6mnnmrLly936/l++uknF4zHCrgBAEhVBN0AAMBNF/bcc8/Ziy++aD/++KPdcssttn379lA28/bt27vu4T69ruzl3bp1c8G2Mp0rkZoSqwEAgP+hezkAAHBjsjds2GB9+vRxXcTr1Klj06ZNCyVXW7lypcto7lMStOnTp9udd95ptWrVcvN0KwC/995747gXAAAkHoJuAADgdO3a1T1imT179n7L1PX8yy+/zIItAwAg+6J7OQAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAAJBIQfdxxx1nf/31137LN23a5F4DAAAAAACHGHT/9ttvtnfv3v2W79q1y1avXp0Z2wUAAAAAQLaX+2BWfvfdd0P/nz59uhUuXDj0XEH4zJkzrWLFipm7hQAAAAAApELQ3bZtW/czR44c1qFDh4jX8uTJ4wLuwYMHZ+4WAgAAAACQCkH3vn373M9KlSrZ119/bcWLFw9quwAAAAAASK2g2/frr79m/pYAAAAAAJBkDinoFo3f1mP9+vWhFnDf2LFjM2PbAAAAAABIvaC7X79+1r9/f6tfv76VLl3ajfEGAAAAAACZEHSPGjXKxo0bZ9dee+2hvB0AAAAAgJRwSPN0796925o0aZL5WwMAAAAAQKoH3TfccIONHz8+87cGAAAAAIBU717+77//2ujRo+2jjz6yWrVquTm6ww0ZMiSztg8AAAAAgNQKur/77jurU6eO+//ixYsjXiOpGgAAAAAAhxF0z5o161DeBgAAAABASjmkMd0AAAAAACCglu7mzZun2438448/PpSPBQAAAAAgqRxS0O2P5/bt2bPHFi1a5MZ3d+jQIbO2DQAAAACA1Au6hw4dGnP5gw8+aNu2bTvcbQIAAAAAIClk6pjua665xsaOHZuZHwkAAAAAQLaVqUH33LlzLX/+/Jn5kQAAAAAApFb38osvvjjiued5tmbNGps/f7717t07s7YNAAAAAIDUC7oLFy4c8TxnzpxWpUoV69+/v7Vs2TKztg0AAAAAgNQLul944YXM3xIAAAAAAJLMIQXdvgULFtiPP/7o/n/SSSdZ3bp1M2u7AAAAAABIzaB7/fr1dsUVV9js2bOtSJEibtmmTZusefPmNmHCBDvmmGMyezsBAAAAAEiN7OW33Xabbd261X744Qf7+++/3WPx4sW2ZcsWu/322zP8OZ9++qldcMEFVqZMGcuRI4dNnjw53fUV5Gu96MfatWsPZTcAAAAAAEi8lu5p06bZRx99ZNWqVQstq169ug0fPvygEqlt377dateubddff/1+GdHTs2zZMitUqFDoeYkSJQ5i6wEAAAAASOCge9++fZYnT579lmuZXsuoc8891z0OloJsv1s7AAAAAABJ1b38zDPPtG7dutmff/4ZWrZ69Wq788477ayzzrKg1alTx0qXLm1nn322ffHFF+muu2vXLtftPfwBAAAAAEDCBt3PPPOMC14rVqxoxx9/vHtUqlTJLXv66actKAq0R40aZW+++aZ7lC9f3po1a2YLFy5M8z0DBw5084r7D70HAAAAAICE7V6uwFWBrsZ1L1261C3T+O4WLVpYkKpUqeIeviZNmtgvv/xiQ4cOtZdffjnme3r27Gndu3cPPVfFAIE3AAAAACDhWro//vhjlzBNgauyhqt7tzKZ69GgQQM3V/dnn31mWalhw4a2fPnyNF/Ply+fS7oW/gAAAAAAIOGC7mHDhlnnzp1jBq7qun3TTTfZkCFDLCstWrTIdTsHAAAAACBbdy//9ttv7bHHHkvzdU0X9sQTT2T487Zt2xbRSv3rr7+6ILpYsWJ27LHHuq7hStD20ksvhYJ+jR1Xi/q///5rzz//vGt9//DDDw9mNwAAAAAASLyge926dTGnCgt9WO7ctmHDhgx/3vz586158+ah5/7Y6w4dOti4ceNszZo1tnLlytDru3fvtrvuussF4gUKFLBatWq5ceXhnwEAAAAAQLYMusuWLWuLFy+2ypUrx3z9u+++O6iu3so87nlemq8r8A7Xo0cP9wAAAAAAIOnGdJ933nnWu3dv17U72s6dO61v3752/vnnZ+b2AQAAAACQGi3dvXr1srfeestOPPFE69q1a2j6Lk0bNnz4cNu7d6898MADQW0rAAAAAADJG3SXLFnS5syZY7fccotLcuZ3Ddf0Ya1atXKBt9YBAAAAAAAHGXRLhQoVbOrUqfbPP/+4zOMKvE844QQrWrRoMFsIAAAAAECqBN0+BdkNGjTI3K0BAAAAACBVE6kBAAAAAICMI+gGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAADjDhw+3ihUrWv78+a1Ro0Y2b968DL1vwoQJliNHDmvbtm3g2wgAQHZD0A0AAGzixInWvXt369u3ry1cuNBq165trVq1svXr16f7vt9++83uvvtuO/3007NsWwEAyE4IugEAgA0ZMsQ6d+5sHTt2tOrVq9uoUaOsQIECNnbs2DTfs3fvXrv66qutX79+dtxxx2Xp9gIAkF0QdAMAkOJ2795tCxYssBYtWoSW5cyZ0z2fO3dumu/r37+/lShRwjp16pSh37Nr1y7bsmVLxAMAgGRH0A0AQIrbuHGja7UuWbJkxHI9X7t2bcz3fP755zZmzBh77rnnMvx7Bg4caIULFw49ypcvf9jbDgBAoiPoBgAAB2Xr1q127bXXuoC7ePHiGX5fz549bfPmzaHHqlWrAt1OAAASQe54bwAAAIgvBc65cuWydevWRSzX81KlSu23/i+//OISqF1wwQWhZfv27XM/c+fObcuWLbPjjz9+v/fly5fPPQAASCW0dAMAkOLy5s1r9erVs5kzZ0YE0XreuHHj/davWrWqff/997Zo0aLQ48ILL7TmzZu7/9NtHACA/6GlGwAAuOnCOnToYPXr17eGDRvasGHDbPv27S6bubRv397Kli3rxmVrHu8aNWpEvL9IkSLuZ/RyAABSHUE3AACwdu3a2YYNG6xPnz4ueVqdOnVs2rRpoeRqK1eudBnNAQDAwSHoBgAATteuXd0jltmzZ6f73nHjxgW0VQAAZG9UWQMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAQAi6AQAAAAAICEE3AAAAAAABIegGAAAAACAgBN0AAAAAAASEoBsAAAAAgIAQdAMAAAAAEBCCbgAAAAAAAkLQDQAAAABAMgbdn376qV1wwQVWpkwZy5Ejh02ePPmA75k9e7adfPLJli9fPqtcubKNGzcuS7YVAAAAAIBsFXRv377dateubcOHD8/Q+r/++qu1bt3amjdvbosWLbI77rjDbrjhBps+fXrg2woAAAAAwMHKbXF07rnnukdGjRo1yipVqmSDBw92z6tVq2aff/65DR061Fq1ahXglgIAAAAAkORjuufOnWstWrSIWKZgW8vTsmvXLtuyZUvEAwAAAACApG/pPlhr1661kiVLRizTcwXSO3futCOOOGK/9wwcOND69esX6HZ1Gve1Jaox1zWI9yYAQJZI5GuxcD0GACA1ZauW7kPRs2dP27x5c+ixatWqeG8SAAAAACBFZKuW7lKlStm6desilul5oUKFYrZyi7Kc6wEAAAAAQFbLVi3djRs3tpkzZ0YsmzFjhlsOAAAAAECiiWvQvW3bNjf1lx7+lGD6/8qVK0Ndw9u3bx9a/+abb7YVK1ZYjx49bOnSpTZixAh7/fXX7c4774zbPgAAAAAAkJBB9/z5861u3bruId27d3f/79Onj3u+Zs2aUAAumi5sypQprnVb83tr6rDnn3+e6cIAAAAAAAkprmO6mzVrZp7npfn6uHHjYr7nm2++CXjLAAAAAABIsTHdAAAAAABkJwTdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAABn+PDhVrFiRcufP781atTI5s2bl+a6zz33nJ1++ulWtGhR92jRokW66wMAkKoIugEAgE2cONG6d+9uffv2tYULF1rt2rWtVatWtn79+pjrz54926688kqbNWuWzZ0718qXL28tW7a01atXZ/m2AwCQyAi6AQCADRkyxDp37mwdO3a06tWr26hRo6xAgQI2duzYmOu/+uqr1qVLF6tTp45VrVrVnn/+edu3b5/NnDkzy7cdAIBERtANAECK2717ty1YsMB1EfflzJnTPVcrdkbs2LHD9uzZY8WKFQtwSwEAyH5yx3sDAABAfG3cuNH27t1rJUuWjFiu50uXLs3QZ9x7771WpkyZiMA92q5du9zDt2XLlsPYagAAsgdaugEAwGF59NFHbcKECfb222+7JGxpGThwoBUuXDj00DhwAACSHUE3AAAprnjx4pYrVy5bt25dxHI9L1WqVLrvfeKJJ1zQ/eGHH1qtWrXSXbdnz562efPm0GPVqlWZsv0AACQygm4AAFJc3rx5rV69ehFJ0PykaI0bN07zfY8//rgNGDDApk2bZvXr1z/g78mXL58VKlQo4gEAQLJjTDcAAHDThXXo0MEFzw0bNrRhw4bZ9u3bXTZzad++vZUtW9Z1EZfHHnvM+vTpY+PHj3dze69du9YtP+qoo9wDAAD8H4JuAABg7dq1sw0bNrhAWgG0pgJTC7afXG3lypUuo7lv5MiRLuv5pZdeGvE5muf7wQcfzPLtBwAgURF0AwAAp2vXru4Ry+zZsyOe//bbb1m0VQAAZG+M6QYAAAAAICAE3QAAAAAABISgGwAAAACAgBB0AwAAAAAQEIJuAAAAAAACQtANAAAAAEBACLoBAAAAAAgIQTcAAAAAAMkcdA8fPtwqVqxo+fPnt0aNGtm8efPSXHfcuHGWI0eOiIfeBwAAAABAool70D1x4kTr3r279e3b1xYuXGi1a9e2Vq1a2fr169N8T6FChWzNmjWhx++//56l2wwAAAAAQLYIuocMGWKdO3e2jh07WvXq1W3UqFFWoEABGzt2bJrvUet2qVKlQo+SJUtm6TYDAAAAAJDwQffu3bttwYIF1qJFi/9tUM6c7vncuXPTfN+2bdusQoUKVr58eWvTpo398MMPWbTFAAAAAABkk6B748aNtnfv3v1aqvV87dq1Md9TpUoV1wr+zjvv2CuvvGL79u2zJk2a2B9//BFz/V27dtmWLVsiHgAAAAAApET38oPVuHFja9++vdWpU8eaNm1qb731lh1zzDH27LPPxlx/4MCBVrhw4dBDreMAAAAAACR90F28eHHLlSuXrVu3LmK5nmusdkbkyZPH6tata8uXL4/5es+ePW3z5s2hx6pVqzJl2wEAAAAASOigO2/evFavXj2bOXNmaJm6i+u5WrQzQt3Tv//+eytdunTM1/Ply+eynYc/AAAAAADICrktzjRdWIcOHax+/frWsGFDGzZsmG3fvt1lMxd1JS9btqzrJi79+/e3U045xSpXrmybNm2yQYMGuSnDbrjhhjjvCQAAAAAACRZ0t2vXzjZs2GB9+vRxydM0VnvatGmh5GorV650Gc19//zzj5tiTOsWLVrUtZTPmTPHTTcGAAAAAEAiiXvQLV27dnWPWGbPnh3xfOjQoe4BAAAAAECiy3bZywEAAAAAyC4IugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAAJDMQffw4cOtYsWKlj9/fmvUqJHNmzcv3fUnTZpkVatWdevXrFnTpk6dmmXbCgBAsqI8BgAgCYPuiRMnWvfu3a1v3762cOFCq127trVq1crWr18fc/05c+bYlVdeaZ06dbJvvvnG2rZt6x6LFy/O8m0HACBZUB4DAJCkQfeQIUOsc+fO1rFjR6tevbqNGjXKChQoYGPHjo25/pNPPmnnnHOO3XPPPVatWjUbMGCAnXzyyfbMM89k+bYDAJAsKI8BAAhGbouj3bt324IFC6xnz56hZTlz5rQWLVrY3LlzY75Hy1UTH0418ZMnT465/q5du9zDt3nzZvdzy5YtmbQXZrt3brNElZn7mWoS+XsVvlsgNf9m/c/xPM+yU3mcVWXyvl07LNVlxvHkOGbOseQ4/h+OY+bgOGaOzCxzMlomxzXo3rhxo+3du9dKliwZsVzPly5dGvM9a9eujbm+lscycOBA69ev337Ly5cvb6nglS7x3gIEhe8WSO2/2a1bt1rhwoWzTXksqV4mZ5XCw+K9BcmDY5k5OI6Zg+OYuMfxQGVyXIPurKBa+/Ca+H379tnff/9tRx99tOXIkcOyC9Wi6KZk1apVVqhQIUsG7FPiS7b9ScZ9Srb9EfYpfapNV+FepkwZy26SpUxOpXM3XjiWmYPjmDk4jpljSxIex4yWyXENuosXL265cuWydevWRSzX81KlSsV8j5YfzPr58uVzj3BFihSx7EonaLKcpD72KfEl2/4k4z4l2/4I+5S2zGrhzsryOBnL5FQ6d+OFY5k5OI6Zg+OYOQol2XHMSJkc10RqefPmtXr16tnMmTMjar31vHHjxjHfo+Xh68uMGTPSXB8AAKSP8hgAgODEvXu5upl16NDB6tevbw0bNrRhw4bZ9u3bXfZUad++vZUtW9aNA5Nu3bpZ06ZNbfDgwda6dWubMGGCzZ8/30aPHh3nPQEAIPuiPAYAIEmD7nbt2tmGDRusT58+LvlKnTp1bNq0aaHkLCtXrnQZVH1NmjSx8ePHW69evez++++3E044wWVKrVGjhiUzdcfT3KnR3fKyM/Yp8SXb/iTjPiXb/gj7FB+Ux6nxPWcXHMvMwXHMHBzHzJEvhY9jDi8z5xwBAAAAAACJMaYbAAAAAIBkRtANAAAAAEBACLoBAAAAAAgIQTcAAAAAJIi9e/fGexOQyQi642j48OFWsWJFy58/vzVq1MjmzZuX5rrNmjWzHDly7PfQNC3hfvzxR7vwwgvdJO1HHnmkNWjQwGWczY77s23bNuvatauVK1fOjjjiCKtevbqNGjXKstLB7JNoip0qVaq47S1fvrzdeeed9u+//x7WZyb6Pmn6IJ1nBQsWtBIlSljbtm1t2bJllp2/I9+jjz7qzss77rjDslIQ+7R69Wq75ppr7Oijj3br1axZ003vlB33RzcjvXv3tkqVKrl1jj/+eBswYIBlZV7Qg9mnPXv2WP/+/d12av3atWu7rOCH85lIDlu3bo33JgBIMM8//7y9/PLLad6bIJtS9nJkvQkTJnh58+b1xo4d6/3www9e586dvSJFinjr1q2Luf5ff/3lrVmzJvRYvHixlytXLu+FF14IrbN8+XKvWLFi3j333OMtXLjQPX/nnXfS/MxE3x99xvHHH+/NmjXL+/XXX71nn33WraN9ygoHu0+vvvqqly9fPvdT2zt9+nSvdOnS3p133nnIn5kd9qlVq1bue9N3uGjRIu+8887zjj32WG/btm3Zcn988+bN8ypWrOjVqlXL69atm5dVgtinv//+26tQoYJ33XXXeV999ZW3YsUKt56uEdlxfx5++GHv6KOP9t5//323zqRJk7yjjjrKe/LJJwPfn0PZpx49enhlypTxpkyZ4v3yyy/eiBEjvPz587vr9KF+JrK/p556yl1bfv/993hvSkrat29fvDchIe3duzd0fDhG8dG8eXOvWrVqrlzYuXNnvDcHmYSgO04aNmzo3XrrrREXOd2UDRw4MEPvHzp0qFewYMGIwKZdu3beNddc4yXL/px00kle//79I9Y7+eSTvQceeMBLxH3SumeeeWbEsu7du3unnnrqIX9mdtinaOvXr1dzo/fJJ5942XV/tm7d6p1wwgnejBkzvKZNm2Zp0B3EPt17773eaaed5sVDEPvTunVr7/rrr49Y5+KLL/auvvpqLxH3SZUGzzzzTLrbG+9rA7KeKo9KlCjh9erVi8A7iwNKiQ4oCTC9iHuwlStXxnVbUpF/DurnJZdc4tWuXdtVQBN4x5bd/mbpXh4Hu3fvtgULFliLFi1Cy3LmzOmez507N0OfMWbMGLviiitcF3LZt2+fTZkyxU488URr1aqV6+ar7omTJ0+27Lg/0qRJE3v33Xddt1hVEM2aNct++ukna9mypSXiPml79R6/S+iKFSts6tSpdt555x3yZyb6PsWyefNm97NYsWKWXffn1ltvdUMdwj87KwS1T/o7ql+/vl122WXu2lC3bl177rnnsu3+aJ2ZM2e664F8++239vnnn9u5556bkPu0a9cu12U8nLrFa5sP9TORff3www/u5/333+8eL7zwgj377LNZNhQsVek+SX9XMnr0aOvUqZO1b9/eDSMSDSVKZe+9954NHjzYHacuXbpYw4YNbefOnfHerJSic1DlgX6qi3mRIkXs6aefdvfyWp7qvP8/hEzXyk2bNmW/v9l4R/2paPXq1a4lcM6cORHL1S1crR0Hou6her9++tRFW8sKFCjgDRkyxPvmm29cC0mOHDm82bNne9ltf+Tff//12rdv717LnTu363r54osvelnhUPdJ3Vvz5Mnjtlfvv/nmmw/7MxN5n2K1IqgVMr2W8ETfn9dee82rUaNGqGY5K1u6g9onddfWo2fPnq5Ls4ZqqHvzuHHjvOy4PzrP1Hqv65vW0c9HHnnEywqHsk9XXnmlV716de+nn35y2/7hhx96RxxxhLumHepnIntSd1H12HrppZdCy1Rmly1b1rv//vtp8c4CGu5RqlQpdz0cMGCAu3507NjRS3UPPvigd8wxx7heUcWLF3fDXBCfllvdh6g3VLNmzdx9fbly5dwy3Ren+rGZPHmyu8fUMK0dO3Z42Qkt3dmQWoWVBEm1kD7VTEqbNm1c0qE6derYfffdZ+eff36WJx/LjP0R1e59+eWXrpVOrUCqgVUL5EcffWSJaPbs2fbII4/YiBEjbOHChfbWW2+53gdK8JRdHew+6ftZvHixTZgwwbLj/qxatcq6detmr7766n4tk9n5O9L14eSTT3brqZX7xhtvtM6dOyfktSEj+/P666+772j8+PFunRdffNGeeOIJ9zMRPfnkk3bCCSdY1apVLW/evC5BZMeOHUOtbkgdKpuLFy/ukiTpHBaV2XfddZc7f2nxznz//fdf6P/qOaJryhtvvOGuMzVq1HC9TpQMNNVbD/v27euOxxdffOF6Hh577LHx3rSUo5Zb9fK64YYbXFJk9cj45ZdfXPnRp0+flG7xzpEjh73zzjvu3Lz00ktdfKO/XV9WJlI9ZPGO+lPRrl27XEKwt99+O2K5WnUvvPDCA463KVSokDds2LD9PlMtPqq1ja7RbdKkiZfd9ke1V2rpUqKkcJ06dXKJu4J2KPuk2uG77747YtnLL7/sWrTUunU4xylR9ymcxqOqNlZJurJCEPujz9JlUZ/rP/RcLSH6/3///Zft9kmU2E5/O+FUS6wxw9lxf3SeRY+R1rWvSpUqXtAO5+9YvSf++OMPV2Ova7Navw/3M5F9+OevEhiee+65LnfBK6+8EnqdFu/Mddddd3mbNm1y/9+zZ4/7+cYbb7ieBqK/NyVgHDVqlHu+ZcsWl+wwlcfE6trbpUsXV2Yop86ff/6533rZbRxtdqNeMFWrVvX++eefiOXqdaeyT71lslsLb2ZYvXq1V69evVDZr1Z/HSO1fC9ZssQti74vTTRUs8eBWjrq1avnxiSGt0TpeePGjdN976RJk9z4QE39E/2ZqqmNnqpJYx4rVKhg2W1/NMWOHtEtQbly5Qq16ifaPu3YsSPm9vo1cIdznBJ1n/yfarl7++237eOPP3bTOGWFIPbnrLPOsu+//94WLVoUemgs9NVXX+3+76+bnfZJTj311GxzbcjI/qS1TqJeG3zqPVG2bFnX8vbmm2+6nkmH+5nIPnTO6nvV1HHqyZUvXz4bO3ZszBZvjef87bff4r3J2daSJUvc39iZZ55pW7Zssdy5c7vl+vsrU6aM61Fw7bXXuh4yN910k3tNUyiql5bySKQKf0zsSy+9ZBMnTrRBgwa5qQvVE0etrOqJuHbt2tB6P//8c/YbR5tN+OWbWrI1ll7XB7+8E10T/v77b9cjQb2/Uk3+/PldvKDpaXWMNF2tWrvVc089+ZQjJeF7j8U76k9VqqnSGEuNqVQNzY033uimh1m7dq17/dprr/Xuu+++mK1AylIey1tvveVah0ePHu39/PPP3tNPP+1aTz777LNsuT+q1VMGc00ZptZTTUulcahqocsKB7tPffv2dRnYNe5G26txm5ry7PLLL8/wZ2bHfbrlllu8woULu9wB4dPAZUVNbBD7Ey2rs5cHsU+a/kw9YZQtWdcGZUPVOLHwVrbstD8dOnRwLYL+lGG69mkMolqPs8LB7tOXX37pvfnmm266sE8//dS1cFaqVCmiJSPe1wZkPY3xj9XirZwGKuvUe8NvocXBUYuXZtA45ZRTvLp163qbN292yzW1paZiUu+lxx57LLS+yit9F5oBJtVacjWF69lnn+3GyYZP26ox3ppqUtn1VYa0bNnSq1mzpnst1Y5RVtI1v2jRovvlMlGuJk3J2rZt2yzrUZhI/vnnH++KK65wf8/qIdumTRs385FawDX2XWVmoiPojiMFxerCo2Q6SpajG7PwG33dWIZbunSp6+qqm9C0jBkzxqtcubIrsDXVgLpdZNf9UeCmeYXVBVb7o66jgwcPztKL/cHsk26OVEgpQND2li9f3nXTiu4ilN5nZsd90ncY6xFeeGen/Yl30B3UPr333nsuQZwCO3VdU+Vcdt0fdQPVd6LP1DrHHXecm0pQ3bQTcZ9UIaUbfR17zS+uoFw3Cgfzmci+/DJLQwu+//57b+PGjaFKyWXLlsUMvFW5rKAcB8+vqNBwoC+++MJ1J69Tp06oq7muhWqgUPI03TO988473llnneUCSv+9yRxUxto3VfSpYlPBy9ixY0PLVVGrey/dVzZu3DhLr7HJzv8evvvuO2/SpEne9OnT3fVBVOl85JFHukByw4YN7n64d+/e3lVXXZUSXcv3/f9jozhBMYIqq//++283Ja2Oje5fdB/gUwAePbw2EeXQP/FubQcAAEg2usVSd1wlQHrggQfcNDelSpWyc845xw3LKV26tBv6oa7le/futXbt2tn1118f781OCupuetppp7nEYDq+6pKqRI2ahkmJ1NR9WsOGlOBQ34m6+efJk8d9D0EPJUoEv/76a8RwsKVLl1qvXr3sr7/+ct3LNZ2aKFmlhvtpqJWOi4bH+N31cXg0BOKWW25xU3lqGISuF48//rhdeeWVbvilktNq+JG6mms61unTp7uhSKlwzXzrrbese/fuVrhwYde1XN3sR44c6f6mfepuP2TIEPe3/Nlnn1mVKlUsocU76gcAAEhWH3zwgRsyoURpaq3RMIjSpUu7ltZVq1aFWryV9FQtNn5XaBw6Tb9XsWJFb9GiRa6ruYapqfdIrVq1Qr1m9F2oK69awP2WtVTpzq9kXWeccYY7N8NpmjAtV0/J8ePH7/e+oJOJphKdmxpCpER+arX99ttv3bCknDlzutZcv+u/kv+p16qGUqWKuXPnumGL/nDSd9991w0J6devX2gdHRf1KFPvME2Fmh3Q0g0AABAAtcRoipvmzZtbz549bePGja6lSgm9tm3b5loPH374YdfivXz5cteiVb58+XhvdlK04jZt2tRNAankdGq9Vst3jx49XIv3J598YoUKFYrZwpYKPv30UzetbMmSJe3mm2+2Vq1ahV6bOnWqa2nVlGGPPvqotW7dOq7bmqzU2+Kxxx5z34U/9ZVas3U9UGJafQ+aKiwVPf300zZnzhx77bXX3FSuat1W0jQl+RNdO9etW+da/tVr6LjjjrPsIMHTvAEAAGQffluGso+ru7KyY2vO3Q0bNtjpp59u5557rruhVGZ6dS9VN/M//vjDKleuTMB9CPxZC8Jn1FC3aQXYunlXJnN1i9axV3ZuBTjVq1cPZYX2JWvAHWtWhzPOOMMdC3Ulf+aZZ2zatGkRx0EzeWguZJ2rCIbOSZ2b+tv3z1t1pb7ooots69atroIuVXj//29Xs6ps377dVYzp71TX0CZNmrjAWn/LokBb56xmX1GFUXYJuIWgGwAAIJMoaFEwrRZCtbgqcDnppJPs5ZdfdsHgI4884tarU6eOlStXzt18J/xUNwnMP3bqKRAePGuqRI3V1jSQ/npqMevXr5/7TvwpmZKZAm7/+CivgMbEPvXUU/bnn3+646Mp0zSWWEHMc889Z6tXr3br6Nzs06dPaJo7ZL5q1aq5fAKaIlDfh3/eKogsVqyYa81NFTly5HCt+6rs+fHHH93+KxeDAm5dRzXFn38u6jz+5ZdfXGCe3a6bdC8HAAA4TH73ZLVSXX311daiRQu7/fbbQ6+re/mHH35oH3/8sWvRuueee6x48eLWuXNnd5OJQw8o1XVcrbdKAKYb9U6dOrnl6l7+wQcfuBa0WN3IUyVpmrrYq6uuEnZpDug1a9bYuHHj7OKLL7avv/7ahg4d6s7No446yp2Tc+fOdb00UqnLfVD8Y/jdd9+5Oc//+ecf1/NFLbmDBw9282+3adPGXTM0zEQVIePHj7cvv/zSzSmfCsdm27ZtLmlc3bp17Y477nCvXXbZZa7yUkNBateu7ZZpbu6xY8e6hIiqtMhuSD8IAABwmHTzqIBa42CV3fnss8+OCA6PP/54t1xZoRXcvPPOOy4zNAH3wVNLl46nKHCpWbOmGwM7YcIE69+/v40ZM8Z1PdV4erWcvfLKK3bNNddEBOqSCgG3smC/9NJLNmPGDDeEQfusrNDXXnutq/xR66ICPXU1V1B45plnkqU8k68LGr+tLOUaQ79ixQr3Pdx///2uMkTn48SJE913oPNYY5WnTJmS9AG3f2yUdVxDbIoWLWpdunQxn3oAaEiO8gvoPFQvIfVm0VCI7BhwC39NAAAAmUA3jt9++627WdRNtfhBnqYC03K9ri69ak088cQT47zF2Y+6i6vlSy23GhuvgFo/NV2QWrt1bHXstVythQoelbhOQXd26456KKJbp9V1WcFcrVq13GsKqNWFXD0ylG9A06YpwNND6/k9AAi4M8c333zjAm4F1RqbfOSRR7prgSqL9F1oOrtLLrnE9cbQ+anzWOdrqjjuuOPceamEcro+ip4XKFDAtWhr6jD1zNAxOfnkk12Cv+yK7uUAAACZRN1IdXOtAEbdRI8++uj9Wlh37dqVEmOKg6IApnfv3q4LtLqW+wGlHv5xVouhehMo+FZgqXm4L7jgAks1AwYMcGO2dTzCzz0dN7Uiqvt9jRo14r2Z2Z7/Nx49ZOH11193eQTUoqs54rWOvgMde+V8UG+XVOzCv2LFClcBodZ/5RLQ2G39/epvtmLFivtdM5NBcu0NgMOmmkbVyqo2UQWzEtFoOhEltQAA/B+/zULdl9Ud9KOPPrKff/7ZBYDvv/++a+FSi5bGcPo3j/57CLgPnjIZq3zSMdSNugIXZSBXy7ffuusHPaJ1brzxRnvhhRdcGTZv3jy3PBXamjRG+/LLL3f/v+qqq+yYY45xY2bDK3vy58/vxhWnYsCX2fzKHv39P/7447Zy5crQa+pRoIzcOtZaR2Pq9R2MGjXKZS8PzxyfKsfq999/d9dJvzJIrdi6hu7Zs8edtzp+yRZwS/LtEYDDom5OulnUeBp1d3r33XetWbNmbrxXEJSBEgCya5ZyBXTK9KyWVyVI0thidYNUYip1b77hhhvcfN3+e3Dw1Fqo4FHlkgJtjUfWVEsaP9+hQwc3hlv87tPhNPZb86ErO/K///6bEt+B5iBX4KKWVGXI1zFSa78qgdTCqNbVBx980CXuyq7jYxOJzindIzVv3txNxaaKHj/w1vVBf/8PPPCAe+7Pya1g3M9UnmrHqkKFCu54jBgxwmXN9wNv5R1QUjXlYlAlW9JR93IAkH/++UdNAN7s2bPTXefGG2/0SpQo4eXLl8876aSTvPfeey/0+htvvOFVr17dy5s3r1ehQgXviSeeiHi/lvXv39+79tprvYIFC3odOnRwyz/77DPvtNNO8/Lnz++VK1fOu+2227xt27YFuLcAcOi++uorr0iRIt6IESPc8w8++MDLkSOH16dPn9A633zzjZczZ07vqquu8vbu3RvHrc2+nn/+ea9QoULe4MGDvS+//HK/1x988EEvV65c3gsvvBBadvvtt3tz584NPb///vu9+vXre1u2bPGSzb59+/ZbtmTJEq906dLeU0895Z5rv0eOHOmOQe7cub1q1ap5p556qrd79273Oufm4du8ebO7HzriiCO8Jk2aeL169fJ+++0399qLL77oFShQwJ2X69ev91avXu2uE8cee6z3xx9/eMnOP0f3Rp1nuj/U3/aAAQO8devWuWU6NiVLlvTOPPNMb8+ePV4yIegGEKIL3FFHHeXdcccd3r///rvf67pgnnLKKa5g+fDDD71ffvnFBdxTp051r8+fP9/dYCqoXrZsmbsJUgEUfjOkoFsXWV1sly9fHnoceeSR3tChQ72ffvrJ++KLL7y6det61113XZbuPwAciH/jOGrUKO+SSy5x///999/dDXSXLl1C6/k309999527HuLgzZw50ytTpoz35ptvphtsKoBRhcdNN93kAp6qVauGbtjXrl3rbuAXLFjgJbPoAOWxxx7zqlSp4srp8PNW5auCcv95sgU28fDff/+5n7oXuvzyy71OnTq5Y6/Ae82aNe61V1991StWrJhXtmxZ78QTT3SNC8l+ToabPn2699xzz3nbt2+PWK57QTXS6L7RP1Z//vmnuy9MNgTdACKopbpo0aKuxVk3Lz179vS+/fbb0EVTQXVaN5BqzTn77LMjlt1zzz2u5Ts86G7btm3EOiqg1HoeTi3f+l07d+7MxL0DgMxprVEl4TXXXOOCGt1A6xrmvzZjxgzXerNp06a4bnN29+ijj3oXXXRRxI26WrsHDRrkeks98sgjoQpiteS2bt3a69ixY6gF1w8od+zY4SWzhx9+2J1/EydODC1TZU/NmjVDleL+MQlHC/eh8Y9b9PFbuHCh60Ggc1Q9NI4//viIwFsVQG+99ZY3bdo0b+XKlV6yCz/nHnjgAVcxpkaY6L/Hbt26eUcffbTXu3dvd4ySFfMBANhvTLeySCrTpsYjKrOpEoM8//zztn79ejc+LK1pbpRQqE2bNhHLTj31VBs2bFhERk+NrwunKXSU8VfZZX2qFFT2So1JY8wZgEQYi6hpvpYuXWodO3Z0ibqUYLJJkybWtm1blxjJp7HeSlrFtEuHZ/HixW4KME0fJBoHqmOuabCUHEz/19y9zz77rJuXW2O9lRFZwueZVtKwZKZzUYniNP/2uHHj3HFQduxTTjnFjd0+99xzXab3aMmYrCqrkqbpvNP8202bNrXGjRu71+rWreuuB7fffrt99dVXbpy3n2+gc+fOLkHtRRddZMlO49lLlCjh/u6UF0iJ0x566CGXw0fT1OneTuO2/b9rjfHW3+3LL7/sjl2y4q8NwH50oTz77LNdYiDNf3rddddZ3759QwlADpd/U+RT4gx/vlD/oUBcmUCVBAcA4snPeK0kSQrwREHNaaed5iojVVmpZEnKVN6zZ08XdN9zzz37XetwcDp16mSzZs1yCao0f7GmYFPQojl9FXBfffXVriLEn9/XP976vsIrPJIpeZoClljHScG2ElEdddRR9uSTT7rKaiXpUkIqZYZOlcztQdO5pL95NShoPnhVaPTo0cMl+xPNu60EdV9//bVbrnN08uTJrvFBlUXJTpnaVemjCh/9vapCsnjx4u41NeDcdtttroJswoQJ7jiK/n5VablgwYLQusmIKlgAB1S9enVXaGiKB2WMVVbzWK3dKuSjpxbTc60bnVE2nDL9auqMypUrB7L9AJAZdNN4xhlnuMBbFYXKUqxgu3379i7Iq1SpkmvVmT59Oj10MoFmztCxVItivXr17O6773aBZN68ed3ryv6sIDN6CrZkCrLDhc9drGnp/Lm3FdhpDmg9Jk6c6OY91jmq6agU0Kj8Vg+2ZD0uWa1w4cIuK7l6vejvXD3yfvjhBxsyZIhrrFCGeAWRDRo0cDMbqIVXFSKqkEt2+ltU5eRll13msuUrO7kqKHfu3Okabp544gl3Dt97772uokgVZepZqUA92TO551Af83hvBIDEoK5Q/oVSAXbBggVt/vz5rmZSBba6SanFYePGja5wUZCsQkcF+TnnnOOmIVEho+5s7dq1cy0QmvNb00KotVwqVqxod9xxh3v41LVctaL6vZpeRxdhBeEqpDSPIwBkNX/uZ5+GyGjKKV271HVcQ2784E8tiWvXrrVSpUpZ7dq13TAcBEtTh6m8UjdzVX4ke0AZfj7ed999rmVVwZ+CHAU0M2fO3K+VUJXkWq5zVt18Tz/99DhtffJVfOhaoBZbVWqoS7mmZVNr9po1a9x0gZoSTPcxqhjy76+OPvpoSwWqeNQ9nZxwwgnuXi5fvnzumPlDPRRwq1ejejpqWIQad5JevAeVA0gcSkhz3333eSeffLJXuHBhN8WFn4HTT3zx119/uUQ1SnqhZGs1atTw3n///f2mDMuTJ4/L5quEN+GUSE0JiKLNmzfPJWFT9nRlMq9Vq5ZLDgMA8UqapoRIzz77bMRrut7p+jZr1qw4bV1yueyyy7zRo0dneH1NJbl06VLvnHPO8erUqRNKlhZr6qxkpPJT04GpzPSz6CtBlTJir1q1ar8EVpqKqXHjxt7YsWPjts3Jxk+gpvsi3Q81bNjQJfITTQn22muveRMmTIjIbJ5KlPjw559/dglxdS+n82/n/0+KG50cN5WS+dHSDQAAUprfeqUWQ/1Ua5W6Qj766KM2duxYN35TSSLVY0ctNjfeeKNr2VZrTbJ3iQxa165dXa8B9aRSN+n0qIeBxsyqhUw9sdTFWgnCwhN1JjOdc/44YrXya/91zNTtXi3ZGk+rlm2NKQ7viq5EX2qNHTx4cLx3IWn455xab3UO65xUF2ldGzIr/01264WhXpDad/2d6rqon5988okb237UUUe5/Az6e33qqafce3TcdI4mey8VH4nUAABAyvKDE82+cM0117jZFdQlUkGNbqzVRVTddjU28aSTTrK3337bypYt67o3a+wsDo+GEN11110uI3z4DBax6OZc40O7dOliU6dOdTfwylKeCgG3aPiCzksF0d98840b+qWKIY0jVvdmDfdSN111efYD7o8//thlk/aHeCFz6JzT9UHdpXUO16lTxyUOGz16tAvEUy3g1hAbJZRU5c6FF17outirglJDEjWOe8eOHS4xrpL+abjDWWed5Y5hqgTcQks3AABISf4N4/fff+/Guyro1rQ/Skil6Wv8lsSnn37a3TQqEZICGwU0ylmhZGojR46M924kBbXg6uZc47MP1OLtS+YW7ugxwNE5BnTe6fxUFmi1+iuBmsbO6tx97LHHQsdl1apV7nxVRREyJvpYZ7TFu1u3bjZ79mwXVCqfTap47733XGWYKn+UWE69LtQLaNKkSW62AVWMLVmyxJ2zmgJQ19EaNWpYyol3/3YAAIB40RjMunXrunwW0cufeeYZl7viyiuvDC3/9NNP3fjNypUre4sWLYrDFmd/4eOvw/+v70Dj5V955RUvlekca9asmffJJ5+kuc7999/vFS9e3B0/jXNv06aNW+bzx7rj4PhjjDUW/qeffspQvgB/3LbGeN92223er7/+6qUK7evpp5/uPfXUU+756tWrvYoVK7p8QDlz5vRef/31iPV37drlpSq6lwMAgJSlDM979uxxratqtfK7nCsrtlq+e/XqZe+8846be1vUIq6sxZoiSJnKcXB0bP1WRI35VPZi38CBA10m44x0NU9mJUqUcK2tmqIuehpOn7qLa72iRYtaw4YN3VSe/fr1C70ePk85Dm6oiXq+nH/++a61VnNr++erWnTfeuutNLuaazyzxitrlpZUoeN12mmnuWuljtWZZ55pZ599thvL3bJlSzedonpj+PwZH1IRQTcAAEhZ3377rS1fvtx1d9TNs4IdfzyspmS66qqr3NhhzcUbTstwcMKTe6kruZLTKUmdEoGpO7VojLICb00hqTGyqahKlSouh4ACuQEDBkQE3jqGorwDmn/7iiuucHMea+pNBdp+xREOns5NXQs0P7wCxltvvdXKlCnjXvv777/dOasu/RI9OjdZhzkcyLHHHusSoqnyZ+jQoXbiiSe6hH0lS5Z052iBAgXccdyyZYulOoJuAACQsipXrux++i3Z0WM5K1Wq5ObajU6alkoJgDKLH3Br/PaQIUOsadOm1qdPH5eI6p577nHj5f3AW8nV1HqmhEypSAGLWk11ninw/vzzzyOO4bp161wiNQU1ak30A+5UDf4ygwLpN954w1XA9e/f3wXcfiWHsnE/9NBDbn70OXPmuO/FD7z9dZKZ9tXf319++cW++uor+/LLL10vIR0njWlfvHixa+VXjgHf8OHDXUVGoUKFLNXR9wQAAKQs3STqhvCll15ymcsrVKgQ0Sr7zz//uG6j9erVi/emJgW1FKqCQ8GNMh2rFVc382rV1lRtw4YNcy29jzzyiGtFU3fVVOUH3rfffrsL+JSoSj0DFHAri7kqgtTlWXQMCbgPjwJp/b1r+i8dWyWlUwvtkUceabt373aB96WXXuoCTn03mqJN1w+/IiQVjo+61z/wwAMuOZoS/em4qGJMMzycfPLJ9uSTT7pjoy76Gpajc1et4CB7OQAASHG6kVT23Xbt2rmuupoazKdA55VXXnFZif2AHAefCVo/VZGhDNsrVqxw035NmzbNdd9XS3etWrXc2GQFk+puXrNmzdBn6AY/lcco//zzzy540XFUVmxl01cuAg2N8KdNS+Xjk5lU8XPxxRfb+vXr3RRt6j2wefNmF2Dr+GvcspapNVeVQrouhGeZTzaatUG9KeTTTz+11q1bu+7jyimgYFvTg6lbuTK3//77766yTMdEOTH8qdTwfwi6AQBAStNN9PPPP+/GJmouWbUmli5d2o3j/uCDD2zmzJluKjEcuk2bNrmWQ/3cunWr612gZFXnnnuu626uMd067koIduedd7obe0QG3pqKSudj1apVCbgDpmR+asHWUAe1dqvCSD1ebrzxRhdQ6qcS2alVN1ktWLDAVUR+9NFHrkVfif0057uCaU1FpwRqF1xwgXse3jtIPTEUqId3MwdjugEAQIpTt1zNua1xs2rlVvdRtdYoSNT4TQLuw6NM5Lo511hQHdPy5cu74FuBtt8Spu9AAbiCSd3cI5KCO1VEqGJISdMIuIOhYypq2VambQXYqojT8S9Xrpx7rnNWFUTJHHDr77B58+bu79bPxq6x2Rq7rdZ+DQ0555xzXK8L0Zzc6lquwFtJ1Ai498dfKgAAgJk1atTIJUpSa426koZn28ah07hPHU9lJdeYbSWnU7C4du1aN75bXVjHjBnjAnElWNO6BJT7Uwu3xngLxyeYYRD+MVULr4Y6nHLKKS5DvFpvR40a5YZEaMhJMlOljoJq9ax4+OGHQ8s1Zlu9fpT7Qt3Mn332WXfclExt1qxZli9fPve3nj9//rhuf6KiezkAAEDUzXf0/5ExaVVUaK7e0aNHu26nCryVNV5jQjWWXtmPlaRK3VjVgstxR9D8c2zjxo0uCVg0DYFQ8jr1LlDrtrqSK6BUwr9k7vmibuMKrpXAcOLEiaHl+ttV7x9lKFdlmZKkNW7c2LZt22YDBw60F154wQXeSoKI2Ai6AQAAkKmmTp3qWsQUrPgUsGjsvLJBq1uquq0qYZU/7RAt3MhKyqCvyiDNv+13oQ6nrtQLFy50w00UTNauXdvKli1ryey3336zyy+/3OW06NGjh+tGr6BaFRDz5s1zw0M0lluVZAq41fVe2d6nTJmS1JURmYGgGwAAAJlGN+cdOnRwXXMHDRoU0ZL43HPPucRpp59+uktSdeKJJ4Zeozs/sqqFW93F1ZqrMfLKCJ/euqmaLV9j2jU+W63aL7/8srVs2dK9rpZutXqrG7rmNFeLt4aMIH0E3QAAADhksYJljc2ePHmyC6rVUqYuuqIuug0aNHBJ1DRWVq2MQFbSsAYlTVQmbo2RVyZ9RNIsAqqQ0HEaMGCA3XXXXW45PVEOHdWJAAAAOOyAW13GlVxJU7ApadpFF11kP/74o/Xs2dMlSRM/Y7lawMlSjnjQuGR1l54+fXrovEQkVZaNHDnS9UhR8jQF36KAm/baQ0PQDQAAgIPi33j7AbfGfz7yyCNuHOyGDRvcMs23rQzQS5YssQsvvNAF5Ndcc417Xa3ceq+CdiArqUJI56K6mKvbNOdgbBqvrTm49beuSoovvvjCLU/FLveZge7lAAAAOGRqEevbt6/LPl6rVi23bNeuXW4KIVGSJU23pCRNupHXnL7KUs4YbgTNH5etFu2dO3e6BGH+Mg1tuPfee23o0KFuDDPSHuOtigpletexUq4GHDw65QMAACBDNMXXJZdcYpdeeql7rgBm+fLldtVVV7mAWzfo6oqqsbLly5d38x1fffXVbl5ftXArqRpZypEV/OBaicCUV+CPP/5wlT5nnXWWCyLvvvtut456ZKjyR2OYsb8TTjjBDQfR/OSaZQCHhqsdAAAADkhBi6YFatOmTWiZghqN0/7ggw/ctEuagklJ05o3b27Lli1z8/tecMEFLlmVn0xNLdwE3Mhs0dnG9X8lTVPFT69evaxatWruuXpeqHJIvS/uuecedy6qpVu9L2666aa47kOiqlq1qr366qsuozkODd3LAQAAcEArVqxwUwMpmBk+fLjLRK5WQmnbtq2tWrXKjdXW1EKa01hzdffv398FOUcffXS8Nx9Jbs2aNaHu43qoN8WNN95oBQsWdEn+/AqfMWPGuKnrlG9AQbeMGDHCVRQpMAeCwEAaAAAAHHAKocqVK9v777/vni9YsMAlWVJroWh6sBkzZrggRgH3nj17XKBTrlw5K1asWJy3HsnujTfesFNPPdW++uorVymk7uJqldVYbj+xn2h5586dXXZuna++Ll26EHAjUATdAAAASJdaENWC/d5777nnCq7VzXzYsGEukZoouN6yZYuNHTvWvbZ69Wp77bXXXBBEhmgEScMXatSoYbfddpt9/fXXbpkqfjTkQT0wdC6Gn4Nq1f7zzz9t8+bNcdxqpBKCbgAAAKRLXXSbNm1qb775pq1fv961CqrrbqtWrezJJ590Y7f9rOU//PCDC8AXLlzoxsmqmy9ZyhEkVQjdddddVrZsWTcu+8svv3TnnpYpt0C3bt1cTgKfAnMlBfMz7ANBY0w3AAAAYial8m8T/dZqZShXN17Nc+x3O9f4biWoUjboTp06ubm6FczoPXv37rVcuXLFeW+QKgnUZs+e7TLna3o6DW/Qufrtt99aixYt7LjjjrMiRYpY0aJFXb6Bzz77zA2FALIC1Y4AAACI4He7VTDjT/Gl4ObCCy+07777ztatW+de19hYTbV0zjnnuKBb0zPlz58/FLATcCNo4RnLmzVr5s7HY4891nU1nzNnjgusFXife+65riVcQyU09puAG1mJlm4AAACETJs2zU2x1LFjR5fhuUSJEqHXfvnlF9farazk6rrrW7JkiUtMpYCHQBtZ2cKt7uNbt251FUWag1u++OILe/zxx914br/F2+91Qe8LxANBNwAAAEI0Jnvo0KFuXt6aNWtagwYNrHfv3nbUUUe5x7333muff/65m5Nb2cnDWxqFoAZZFXAra7mmrVOmcmUpr1Onjg0aNMgaNWrkAm/9X1OJDR482E477bSI9wJZiaAbAAAA+9F47XHjxtlbb71l27dvd8mq1JKt4KZDhw42adIkF8horDeJ0pDVlCzNT+SnIDt37txunnhRzoH69evbrFmz7KGHHnIVQerBoaEPQDwQdAMAACAmBSsaz60Ww08++cQ+/vhjN1ZWyaoUcE+fPt2OOOKIeG8mklh0y7T/XFPVvfTSSy55mjKVq+JH2fMVgBcvXtw++ugjt756ZWjqMPXKAOKFakkAAADEpEBGmcg1xvv99993Xcp///13151XaDlEkNSLQgG2elfMnz/fFixYEArA165d68Zx6/zUebpz5073/xdeeMGtO2/ePLeeKocIuBFvtHQDAAAgwy2NCnQ0TvaEE05wY7fpXo4g+OeVkvRpTnjNFV+gQAGbOHGi60qujORKkNavX7+IpH4Ktq+66io3LZiy6wOJIHe8NwAAAACJKzrpVOHChd1DSJqGoCp6FHArqZ9aqrt06WI33XSTa7HWcr2uSp8ePXrYiBEj3HNNWbdlyxabMmWKC8o1JzeQKGjpBgAAAJBQ/v77b2vTpo2dfPLJLlmaL7xnhYY6vPLKKzZw4EA75phjXGWQemF88MEH7n1AoiDoBgAAAJBQ1K38wgsvtLFjx7rW7ughDP6whz179tiKFStcUj/NKa9EapUqVYrbdgOx0L0cAAAAQEJZtGiRa8k+/fTTXXAdnTtAy3bs2GGLFy+2hg0bWpUqVeK6vUB6yHoBAAAAIKFomi+NzdY88RIrWZ9awZVZf/fu3XHYQiDjCLoBAAAAJJQKFSpYoUKF3FzcavH2hY+M/e2336xevXpunm4gkRF0AwAAAEgoZcuWtZEjR7qx2r1793ZjvMO7ld9///32xhtvWMeOHffLsA8kGhKpAQAAAEg4Gsf93HPPWdeuXa1y5crWuHFjy58/v61evdq+/PJLmzZtmtWtWzfemwkcEEE3AAAAgIQ1b948GzRokC1fvtwKFixoTZo0sU6dOrm5uoHsgKAbAAAAQELbu3ev5cqVK96bARwSxnQDAAAASGjh2ctpM0R2Q0s3AAAAAAABoaUbAAAAAICAEHQDAAAAABAQgm4AAAAAAAJC0A0AAAAAQEAIugEAAAAACAhBNwAAAAAAASHoBgAAAAAgIATdAAAAAAAEhKAbAAAAAICAEHQDAAAAABAQgm4AAAAAACwY/w84SNWVpEBhEwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
queryscorereasoning
0I need a minimalist design with lots of whites...0.90{'visual_style': {'score': 9, 'reason': 'The d...
1Looking for a playful, colorful design with ro...0.80{'visual_style': {'score': 8, 'reason': 'The d...
2Need a professional business design with a dar...0.80{'visual_style': {'score': 8, 'reason': 'The d...
3Want a nature-inspired design with organic sha...0.75{'visual_style': {'score': 8, 'reason': 'The d...
4Looking for a tech-focused design with a futur...0.80{'visual_style': {'score': 9, 'reason': 'The d...
\n", "
" ], "text/plain": [ " query score \\\n", "0 I need a minimalist design with lots of whites... 0.90 \n", "1 Looking for a playful, colorful design with ro... 0.80 \n", "2 Need a professional business design with a dar... 0.80 \n", "3 Want a nature-inspired design with organic sha... 0.75 \n", "4 Looking for a tech-focused design with a futur... 0.80 \n", "\n", " reasoning \n", "0 {'visual_style': {'score': 9, 'reason': 'The d... \n", "1 {'visual_style': {'score': 8, 'reason': 'The d... \n", "2 {'visual_style': {'score': 8, 'reason': 'The d... \n", "3 {'visual_style': {'score': 8, 'reason': 'The d... \n", "4 {'visual_style': {'score': 9, 'reason': 'The d... " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "View detailed results in LangSmith:\n", "https://smith.langchain.com/projects/imagineui\n" ] } ], "source": [ "# Run evaluation using correct LangSmith API\n", "from chains.design_rag import DesignRAG\n", "import asyncio\n", "from langsmith import Client\n", "import uuid\n", "import json\n", "\n", "# Extract design IDs from the responses\n", "def extract_design_id(response):\n", " \"\"\"Extract the design ID from the response URL\"\"\"\n", " try:\n", " # Find the URL line in the response\n", " if isinstance(response, str) and \"URL:\" in response:\n", " url_line = [line for line in response.split('\\n') if \"URL:\" in line][0]\n", " # Extract the ID from the URL\n", " return url_line.split('/')[-1]\n", " return \"unknown\"\n", " except Exception:\n", " return \"unknown\"\n", "\n", "async def evaluate_rag():\n", " # Create LangSmith client\n", " client = Client()\n", " \n", " # Initialize the RAG system\n", " rag = DesignRAG()\n", " results = []\n", " \n", " # Create a unique dataset name using a timestamp\n", " import time\n", " timestamp = int(time.time())\n", " dataset_name = f\"design_evaluation_{timestamp}\"\n", " \n", " # Create dataset in LangSmith - API has changed, no longer accepts examples directly\n", " try:\n", " dataset = client.create_dataset(\n", " dataset_name=dataset_name,\n", " description=\"Design RAG system evaluation\"\n", " )\n", " print(f\"Created dataset: {dataset_name}\")\n", " \n", " # Now add examples to the dataset\n", " for query in test_queries:\n", " client.create_example(\n", " dataset_id=dataset.id,\n", " inputs={\"query\": query},\n", " outputs={} # No outputs yet\n", " )\n", " except Exception as e:\n", " print(f\"Error with dataset: {e}\")\n", " # Continue without dataset\n", " \n", " # Now evaluate each query\n", " for i, query in enumerate(test_queries):\n", " print(f\"Evaluating query {i+1}/{len(test_queries)}: {query[:50]}...\")\n", " \n", " try:\n", " # Execute the query without LangSmith integration first\n", " response = await rag.query_similar_designs([query])\n", " \n", " # Evaluate response\n", " eval_result = await evaluator.aevaluate_strings(\n", " prediction=response,\n", " reference=\"\", # No ground truth in RAG\n", " input=query\n", " )\n", " \n", " # Manual logging to LangSmith - simpler approach\n", " try:\n", " # Create a run with the required run_type parameter\n", " run = client.create_run(\n", " project_name=\"imagineui\",\n", " run_type=\"chain\", # Required parameter\n", " name=f\"design_query_{i}\",\n", " inputs={\"query\": query},\n", " outputs={\"response\": response}\n", " )\n", " \n", " # Add evaluation as feedback\n", " client.create_feedback(\n", " run_id=run.id,\n", " key=\"design_match\",\n", " score=eval_result.get(\"score\", 0),\n", " comment=json.dumps(eval_result.get(\"reasoning\", {}))\n", " )\n", " \n", " run_id = run.id\n", " except Exception as e:\n", " print(f\" LangSmith error: {e}\")\n", " run_id = \"error\"\n", " \n", " # Add to our local results\n", " results.append({\n", " \"query\": query,\n", " \"response\": response,\n", " **eval_result,\n", " \"run_id\": run_id\n", " })\n", " \n", " print(f\" Score: {eval_result.get('score', 0):.2f}\")\n", " \n", " except Exception as e:\n", " print(f\"Error evaluating query {i}: {str(e)}\")\n", " results.append({\n", " \"query\": query,\n", " \"error\": str(e)\n", " })\n", " \n", " # Save results locally as a backup\n", " try:\n", " import pandas as pd\n", " results_df = pd.DataFrame(results)\n", " results_df.to_csv(f\"evaluation_results_{timestamp}.csv\", index=False)\n", " print(f\"Saved results to evaluation_results_{timestamp}.csv\")\n", " except Exception as e:\n", " print(f\"Error saving results: {e}\")\n", " \n", " return results\n", "\n", "# Run the evaluation\n", "results = await evaluate_rag()\n", "\n", "# Display results\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "df = pd.DataFrame(results)\n", "\n", "#grab the design id from the response\n", "if 'response' in df.columns:\n", " df['design_id'] = df['response'].apply(extract_design_id)\n", "\n", "# Check if we have score data\n", "if 'score' in df.columns and not df['score'].isnull().all():\n", " # Calculate average scores\n", " print(\"\\n--- EVALUATION SUMMARY ---\")\n", " print(f\"Number of queries evaluated: {len(df)}\")\n", " print(\"\\nAverage Scores:\")\n", " print(f\"Overall: {df['score'].mean():.2f}\")\n", " print(f\"Visual Style: {df['visual_style_score'].mean():.2f}\")\n", " print(f\"Layout: {df['layout_score'].mean():.2f}\")\n", " print(f\"Color & Mood: {df['color_mood_score'].mean():.2f}\")\n", " print(f\"Features: {df['features_score'].mean():.2f}\")\n", "\n", " # Create a summary visualization\n", " plt.figure(figsize=(10, 6))\n", "\n", " # Score distribution\n", " plt.subplot(1, 2, 1)\n", " plt.hist(df['score'], bins=10, alpha=0.7)\n", " plt.title('Distribution of Overall Scores')\n", " plt.xlabel('Score')\n", " plt.ylabel('Count')\n", "\n", " # Category comparison\n", " plt.subplot(1, 2, 2)\n", " categories = ['Overall', 'Visual Style', 'Layout', 'Color & Mood', 'Features']\n", " scores = [\n", " df['score'].mean(), \n", " df['visual_style_score'].mean(), \n", " df['layout_score'].mean(),\n", " df['color_mood_score'].mean(), \n", " df['features_score'].mean()\n", " ]\n", "\n", " plt.bar(categories, scores)\n", " plt.title('Average Scores by Category')\n", " plt.ylim(0, 1.0)\n", " plt.xticks(rotation=45)\n", "\n", " plt.tight_layout()\n", " plt.show()\n", "\n", " # Show detailed results\n", " display(df[['query', 'score', 'design_id', 'reasoning']])\n", "else:\n", " # Show error information\n", " print(\"Evaluation failed to produce scores. See errors:\")\n", " display(df[['query', 'error'] if 'error' in df.columns else ['query']])\n", "\n", "# Print URL to view results in LangSmith\n", "print(\"\\nView detailed results in LangSmith:\")\n", "print(\"https://smith.langchain.com/projects/imagineui\")" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 2 }