Spaces:
Sleeping
Sleeping
File size: 131,979 Bytes
d48d4f3 |
1 |
{"cells":[{"cell_type":"markdown","metadata":{"id":"sDv-N0TTz0VE"},"source":["# What is Pytorch?\n","PyTorch is an open-source library used in machine learning library developed using Torch library for python program. It is developed by Facebook’s AI Research lab and released in January 2016 as a free and open-source library mainly used in computer vision, deep learning, and natural language processing applications. Programmer can build a complex neural network with ease using PyTorch as it has a core data structure, Tensor, multi-dimensional array like Numpy arrays. PyTorch use is increasing in current industries and in the research community as it is flexible, faster, easy to get the project up and running, due to which PyTorch is one of the top deep learning tools."]},{"cell_type":"markdown","metadata":{"id":"irVOTYdzyNJJ"},"source":["# How to install Pytorch\n","In order to install Pytorch, you could do it using different package managers such as `conda` and `pip`. Visit [this link](https://pytorch.org/get-started/locally/) for information about the installation on your device."]},{"cell_type":"markdown","metadata":{"id":"YLQylD11zw4t"},"source":["# Tensors\n","Tensors are a specialized data structure that are very similar to arrays and matrices. In PyTorch, we use tensors to encode the inputs and outputs of a model, as well as the model’s parameters. Tensors are just like Numpy arrays, except that they can run on GPUs and other hardware accelerators. Tensors are also optimized for automatic differentiation (which is referred to autograd from now on). Tensor API is really similar with Numpy array API, so if you are familiar with Numpy arrays, you are probably not going to face any problem with Tensors.\n"]},{"cell_type":"code","execution_count":1,"metadata":{"id":"FnosP4Qg1BuV","executionInfo":{"status":"ok","timestamp":1678603823386,"user_tz":-210,"elapsed":6653,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["# imporing packages\n","import torch\n","import numpy as np"]},{"cell_type":"markdown","metadata":{"id":"wnoHqqlj1iGO"},"source":["## Initializing Tensors"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"lDAnwOLG1hIu","outputId":"5c192952-9185-4040-e272-06794a8366f3","executionInfo":{"status":"ok","timestamp":1678603849212,"user_tz":-210,"elapsed":752,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[{"output_type":"stream","name":"stdout","text":["Tensor created directly from data: \n"," tensor([[ 1, 4],\n"," [ 6, 10]]) \n","\n","Tensor created from a numpy array: \n"," tensor([[ 1, 4],\n"," [ 6, 10]]) \n"," \n","Ones Tensor: \n"," tensor([[1, 1],\n"," [1, 1]]) \n","\n","Random Tensor: \n"," tensor([[0.9627, 0.5478],\n"," [0.6493, 0.4798]]) \n","\n","Random Tensor: \n"," tensor([[0.5167, 0.9801, 0.7066, 0.4006],\n"," [0.0555, 0.3062, 0.1756, 0.7882],\n"," [0.9939, 0.6109, 0.0108, 0.3264]]) \n","\n","Ones Tensor: \n"," tensor([[1., 1., 1., 1.],\n"," [1., 1., 1., 1.],\n"," [1., 1., 1., 1.]]) \n","\n","Zeros Tensor: \n"," tensor([[0., 0., 0., 0.],\n"," [0., 0., 0., 0.],\n"," [0., 0., 0., 0.]])\n"]}],"source":["# initializing directly from data\n","data = [[1, 4],[6, 10]]\n","x_data = torch.tensor(data)\n","print(f\"Tensor created directly from data: \\n {x_data} \\n\")\n","\n","# from numpy arrays\n","np_array = np.array(data)\n","x_np = torch.from_numpy(np_array)\n","print(f\"Tensor created from a numpy array: \\n {x_np} \\n \")\n","\n","# from another tensor\n","x_ones = torch.ones_like(x_data) # retains the properties of x_data\n","print(f\"Ones Tensor: \\n {x_ones} \\n\")\n","\n","x_rand = torch.rand_like(x_data, dtype=torch.float) # overrides the datatype of x_data\n","print(f\"Random Tensor: \\n {x_rand} \\n\")\n","\n","shape = (3,4,)\n","rand_tensor = torch.rand(shape)\n","ones_tensor = torch.ones(shape)\n","zeros_tensor = torch.zeros(shape)\n","\n","print(f\"Random Tensor: \\n {rand_tensor} \\n\")\n","print(f\"Ones Tensor: \\n {ones_tensor} \\n\")\n","print(f\"Zeros Tensor: \\n {zeros_tensor}\")"]},{"cell_type":"markdown","metadata":{"id":"1pS6uApz4gie"},"source":["## Attributes of a Tensor"]},{"cell_type":"code","execution_count":3,"metadata":{"id":"udW5cVKlyIR7","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1678603868147,"user_tz":-210,"elapsed":621,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"ec568db2-7fc7-4126-b0d1-6d1c77d18a96"},"outputs":[{"output_type":"stream","name":"stdout","text":["Shape of tensor: torch.Size([3, 4])\n","Datatype of tensor: torch.float32\n","Device tensor is stored on: cpu\n"]}],"source":["tensor = torch.rand(3,4)\n","\n","print(f\"Shape of tensor: {tensor.shape}\")\n","print(f\"Datatype of tensor: {tensor.dtype}\")\n","print(f\"Device tensor is stored on: {tensor.device}\")"]},{"cell_type":"markdown","metadata":{"id":"y39VfFN55e_4"},"source":["## Operations on Tensors\n","Torch Tensors contain various operations including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), etc.\n","\n","By default, tensors are created on CPU. In order to get the most out of tensors, we can move these tensors to GPU:"]},{"cell_type":"code","execution_count":4,"metadata":{"id":"Xu0Ltb464p14","executionInfo":{"status":"ok","timestamp":1678603875074,"user_tz":-210,"elapsed":530,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["# We move our tensor to the GPU if available\n","if torch.cuda.is_available():\n"," tensor = tensor.to(\"cuda\")"]},{"cell_type":"code","execution_count":10,"metadata":{"id":"Fd9ZbCgP6qcs","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1678603932773,"user_tz":-210,"elapsed":10,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"abae3c5d-e24c-426f-e666-48ad18e876de"},"outputs":[{"output_type":"stream","name":"stdout","text":["First row: tensor([1., 1., 1.])\n","First column: tensor([1., 1., 1.])\n","Last column: tensor([1., 1., 1.])\n","tensor([[1., 1., 0.],\n"," [1., 1., 0.],\n"," [1., 1., 0.]])\n"]}],"source":["# Standard numpy-like indexing and slicing\n","tensor = torch.ones(3, 3)\n","print(f\"First row: {tensor[0]}\")\n","print(f\"First column: {tensor[:, 0]}\")\n","print(f\"Last column: {tensor[..., -1]}\")\n","tensor[:,2] = 0\n","print(tensor)"]},{"cell_type":"code","execution_count":11,"metadata":{"id":"ruFKbyCR6xfk","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1678603935040,"user_tz":-210,"elapsed":7,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"9cebb62a-83ca-47fe-b72b-6ac537075e15"},"outputs":[{"output_type":"stream","name":"stdout","text":["tensor([[1., 1., 0., 1., 1., 0., 1., 1., 0.],\n"," [1., 1., 0., 1., 1., 0., 1., 1., 0.],\n"," [1., 1., 0., 1., 1., 0., 1., 1., 0.]])\n"]}],"source":["# Joining tensors\n","t1 = torch.cat([tensor, tensor, tensor], dim=1)\n","print(t1)"]},{"cell_type":"code","execution_count":12,"metadata":{"id":"8MWGwqPI67RU","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1678603943054,"user_tz":-210,"elapsed":556,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"2c93cc3e-32ab-46eb-ff1a-21dbb5cfd5b4"},"outputs":[{"output_type":"stream","name":"stdout","text":["Matrix Multiplication: \n"," y1: tensor([[2., 2., 2.],\n"," [2., 2., 2.],\n"," [2., 2., 2.]]) \n","\n"," y2: tensor([[2., 2., 2.],\n"," [2., 2., 2.],\n"," [2., 2., 2.]]) \n","\n"," y3: tensor([[2., 2., 2.],\n"," [2., 2., 2.],\n"," [2., 2., 2.]]) \n","\n","Element-wise Product: \n"," z1: tensor([[1., 1., 0.],\n"," [1., 1., 0.],\n"," [1., 1., 0.]]) \n","\n"," z2: tensor([[1., 1., 0.],\n"," [1., 1., 0.],\n"," [1., 1., 0.]]) \n","\n"," z3: tensor([[1., 1., 0.],\n"," [1., 1., 0.],\n"," [1., 1., 0.]])\n"]}],"source":["# Arithmatic operations\n","# This computes the matrix multiplication between two tensors. y1, y2, y3 will have the same value\n","y1 = tensor @ tensor.T\n","y2 = tensor.matmul(tensor.T)\n","\n","y3 = torch.rand_like(y1)\n","torch.matmul(tensor, tensor.T, out=y3)\n","\n","\n","# This computes the element-wise product. z1, z2, z3 will have the same value\n","z1 = tensor * tensor\n","z2 = tensor.mul(tensor)\n","\n","z3 = torch.rand_like(tensor)\n","torch.mul(tensor, tensor, out=z3)\n","\n","print(f\"Matrix Multiplication: \\n y1: {y1} \\n\\n y2: {y2} \\n\\n y3: {y3} \\n\")\n","print(f\"Element-wise Product: \\n z1: {z1} \\n\\n z2: {z2} \\n\\n z3: {z3}\")"]},{"cell_type":"code","execution_count":13,"metadata":{"id":"XZvbDzkM7ayG","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1678603947924,"user_tz":-210,"elapsed":510,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"6493ba6d-380b-4b55-da5c-e14f3d5a2f85"},"outputs":[{"output_type":"stream","name":"stdout","text":["tensor(6.) <class 'torch.Tensor'>\n","6.0 <class 'float'>\n"]}],"source":["# Single-element tensors\n","agg = tensor.sum()\n","print(agg, type(agg))\n","agg_item = agg.item()\n","print(agg_item, type(agg_item))"]},{"cell_type":"markdown","metadata":{"id":"Nj0HjxOLk7Kb"},"source":["# Datasets and Dataloaders\n","PyTorch provides two data primitives: `torch.utils.data.DataLoader` and `torch.utils.data.Dataset` that allow you to use pre-loaded datasets as well as your own data. `Dataset` stores the samples and their corresponding labels, and `DataLoader` wraps an iterable around the Dataset to enable easy access to the samples.\n","\n","Pytorch contains various pre-loaded subclasses of `Dataset` which you can load such as CIFAR10, MNIST, etc. Here is an example of loading MNIST test and train datasets."]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":423,"referenced_widgets":["d413d51bccd44eaeb62a5886c1779388","45b25e785f5f47e3a00721b884818faa","e49a0fdc81484f6085c97dc7d2673de3","e7f61788756c4cf793827cce1e5a9757","03c1adf12a7d429d8c917f29bf708236","9a9cf2cfd7f24cf78fd610db69200038","663e6dc5bd204e35856e6ec5f8511585","b2b41eb2871749309e6409467e7728c7","74e79bb356bb424bbadd404df5393202","c9c5e9c1bd48407ca1e0593c247edd72","566a0aec3a9941f5aaf2d10faada0480","3b9a38876a944ac2bf9a9d48f96532d5","d5e088bc30c848bbb6ee0dd83a7788f8","0bce3184553c427db39f9ef313477306","fe79558b71744a789dbd8ecef771b40c","0e974bded8f44afab292eace71c1a0e2","8caef2ea7007438f990a912915eee696","7cfe9445e7524d72b4c7c4df9f4c515d","0aedc2afd99b46639cddea273f12fa67","93625f34347d4610a04dce294d3b4f1e","932f11b1d0014e23b8b13993aa161b23","b2242d35002643d2b298ac5a714f0219","1fd97732ca884679a71b142c3d9f6780","322dee672a0f4c45979ed1f16d0e5bed","c0f8ff6470654b1cbac1c8d7e9b702ff","ca9a89ff6b824024895846f6e77ab9b8","3ea06647a49a417a80088e8ed9a46404","c63e32998f2a4349a2e8f3967e7ace75","29ed5f7b443446f0958e3f62ed689d3d","79a5ba729a5e406a8b4ead0a75668561","034507653f3f4cac9f419adfff976e0b","86cf3a19466d4278829cba5d3e3ee59e","1577d0e00ac64d9c9f25bfd710915c89","6c461b09af05473aa2cb2e116963d1af","010e7be4778b41a78a53fb101d06f272","e2a3ed2047904419a75b9c94dffd236c","8a8e8c276f774c0cb306b0a4fcf6f62d","3b1b11b7be0242df957021058280cf52","521694b3e81f41449b468cf7fb952573","bcc8347c4b4049a0a3ca8f279684d7e2","a5e698b7c50a48aeab26df687f6fb7da","e4281dc403874993bbc196c91be3ec76","96c99b3110a6423081f6b061db82b54b","88b8adaf05a24940bf8711f49f14fc52"]},"id":"hqvnvRK9k7Kb","executionInfo":{"status":"ok","timestamp":1678603955816,"user_tz":-210,"elapsed":2422,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"7c497b91-76b8-4f03-8029-561411faa31b"},"outputs":[{"output_type":"stream","name":"stdout","text":["Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n","Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz\n"]},{"output_type":"display_data","data":{"text/plain":[" 0%| | 0/9912422 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d413d51bccd44eaeb62a5886c1779388"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n","Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz\n"]},{"output_type":"display_data","data":{"text/plain":[" 0%| | 0/28881 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"3b9a38876a944ac2bf9a9d48f96532d5"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"]},{"output_type":"display_data","data":{"text/plain":[" 0%| | 0/1648877 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"1fd97732ca884679a71b142c3d9f6780"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw\n","\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n","Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"]},{"output_type":"display_data","data":{"text/plain":[" 0%| | 0/4542 [00:00<?, ?it/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6c461b09af05473aa2cb2e116963d1af"}},"metadata":{}},{"output_type":"stream","name":"stdout","text":["Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw\n","\n"]}],"source":["from torch.utils.data import Dataset\n","from torchvision import datasets\n","from torchvision.transforms import ToTensor\n","import matplotlib.pyplot as plt\n","\n","\n","training_data = datasets.MNIST(\n"," root=\"data\", # the root directory to save the downloaded dataset\n"," train=True, # choose whether it is test or train\n"," download=True, # choose whether to download the dataset or not \n"," transform=ToTensor() # what transforms should be applied to the dataset, in this case it just \n"," # converts the images to torch.tensor\n",")\n","\n","test_data = datasets.MNIST(\n"," root=\"data\",\n"," train=False,\n"," download=True,\n"," transform=ToTensor()\n",")"]},{"cell_type":"code","execution_count":18,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":299},"id":"-pfFAGlwk7Kb","executionInfo":{"status":"ok","timestamp":1678604009657,"user_tz":-210,"elapsed":16,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"63b7e6d6-eba4-4d76-a0d6-4f15d5584d1d"},"outputs":[{"output_type":"stream","name":"stdout","text":["Shape of the image of the first data: torch.Size([1, 28, 28])\n","Label of the first data: 2\n"]},{"output_type":"display_data","data":{"text/plain":["<Figure size 432x288 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAOcAAAD3CAYAAADmIkO7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAJl0lEQVR4nO3dcYjX9R3H8ddbvejUq+U85NjWioSVKCwJnI42/5hoRlmCi7wMDBWCcRRTlMHwlFzoH20t/9j+yUHb8I8ZBpuNK3IE2ZhjutHAkW233agwhzKncs7zsz/uYlL3ff/0d/7u9/p5zwf8wO59n9/3W/b0o/fh60UpRQD8TGr2DQAYHXECpogTMEWcgCniBEwRJ2CKOE1FxG8iYt14r4UP4mywiOiPiG80+z7GKiI2RcQ7EXEmIv4WEZuafU/XuynNvgG0jJD0uKQ/SbpDUl9EDJRS9jb3tq5f7JxNEhG3RMQvI+KjiDg18uPPf+LT7oiI30XEvyPilYiYcdn6r0TEoYg4HRF/jIjFjbzfUsquUsofSikXSyl/kfSKpK828poTHXE2zyRJeyR9UdKtks5L2v2Jz3lc0hOSuiRdlPRDSYqIz0n6laRnJM2QtFHSvojorHXRiFg9EnTV69YreI+QdK+kP1/hvyvqQJxNUkr5VyllXynlXCnljKQdkr7+iU97qZTyTinlrKTvSvpmREyW9JikA6WUA6WUS6WU1yT9XtLyK7juz0spn0le/7iC2+/V/39xQYPwZ84miYipkr4vaZmkW0Y+3BERk0spQyP/PHDZkr9LapM0U8O77aqIeOCyeZukg429aykivqXhHf3eUspgo683kRFn83xb0pckLSilfBgRX5Z0RMNfePnYFy778a2S/ivppIajfamUsv5qLxoR3ZJ+nHzKnKrdMyKekLRF0tdKKf+82mvj6vDb2vHRFhE3XvaaIqlDw3/OPD3yhZ6to6x7LCLmjOyy2yX9YmRX/amkByJiaURMHnnPxaN8QelTSik/K6VMT15VYXZL+p6kJaWUv9b53wFXgTjHxwENh/jxq1fSDyS1a3gn/K2kX4+y7iVJP5H0oaQbJfVIUillQNIKSd+R9JGGd9JNauzP5zOSPivpcET8Z+T1owZeb8ILHrYGPLFzAqaIEzBFnIAp4gRMpeecEcFXi4AGK6XEaB9n5wRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqamNPsGML46OjrS+fTp0ytn999/f7q2s7MznT/33HPpfHBwMJ1PNOycgCniBEwRJ2CKOAFTxAmYIk7AFHECpjjnbDG33XZbOt+8eXM6X7hwYTqfO3fu1d7SFevq6krnPT09Dbt2K2LnBEwRJ2CKOAFTxAmYIk7AFHECpqKUUj2MqB6ibnfeeWfl7KmnnkrXdnd3p/P29vZ0HhHpfGBgoHJ25syZdO1dd92Vzk+ePJnOFy9eXDk7duxYuraVlVJG/Ulh5wRMESdgijgBU8QJmCJOwBRxAqaIEzDFI2N1uPnmm9P5zp070/kjjzxSOav1V1eO1bvvvpvOly5dWjlra2tL19Y6i5w5c+aY5hMNOydgijgBU8QJmCJOwBRxAqaIEzBFnIApzjnr8PDDD6fzdevWjdOdfNp7772XzpcsWZLOs+c5Z8+eXdc9oT7snIAp4gRMESdgijgBU8QJmCJOwBRxAqY456zDqlWrGvbe/f396fzw4cPpvNa3AMzOMWup9ffS4tpi5wRMESdgijgBU8QJmCJOwBRxAqaIEzDFOWcd1q9fn843bNiQzvv6+ipnx48fT9eeOHEinTfSrFmzmnbtiYidEzBFnIAp4gRMESdgijgBU8QJmOIopQ7vv/9+Ou/t7R2fGxlnCxcubPYtTCjsnIAp4gRMESdgijgBU8QJmCJOwBRxAqY452wxPT096XzatGkNu/a8efPGtP7QoUPp/O233x7T+19v2DkBU8QJmCJOwBRxAqaIEzBFnIAp4gRMcc7ZAFOnTk3nc+bMqZxt3bo1Xbt8+fK67uljkyblvx5funSp7veu9Zzr2rVr0/nQ0FDd174esXMCpogTMEWcgCniBEwRJ2CKOAFTxAmY4pxzFG1tben87rvvTuf79u1L511dXZWz8+fPp2trnSXWeiZy2bJl6bzWGW1mypT8f6eVK1em8+eff75yduHChbruqZWxcwKmiBMwRZyAKeIETBEnYIo4AVPECZiKUkr1MKJ62MJuuOGGdF7rLPDll18e0/W3bdtWOXvjjTfStW+99VY6nzFjRjqv9f5z585N543U3d1dOdu/f3+6dnBw8BrfzfgppcRoH2fnBEwRJ2CKOAFTxAmYIk7AFHECpq7bo5Tssa/t27enazdt2jSma7/66qvpfM2aNZWz06dPp2s7OzvT+YEDB9L5/Pnz03n2aNauXbvStbWOYVasWJHOM6+//no637lzZzo/depU3deWpKNHj45pfYajFKDFECdgijgBU8QJmCJOwBRxAqaIEzDVsueckydPTuc7duyonG3cuDFde/bs2XS+ZcuWdL537950np253XPPPena3bt3p/Na648fP57On3zyycrZwYMH07U33XRTOl+0aFE6zx4Ze/DBB9O106ZNS+e1DAwMpPPbb799TO+f4ZwTaDHECZgiTsAUcQKmiBMwRZyAKeIETLXsOWd2HidJL7zwQuXs3Llz6doNGzak876+vnS+YMGCdL527drK2X333ZeubW9vT+e1nlXds2dPOq913tcsjz76aDpfvXr1mN7/6aefTue1zofHgnNOoMUQJ2CKOAFTxAmYIk7AFHECpogTMNWy55wffPBBOs/+ftda3y7u2LFj6bzWs4OzZ89O52PR29ubzp999tl0PjQ0dA3vBtcC55xAiyFOwBRxAqaIEzBFnIAp4gRMtexRypEjR9L5vHnzxulOPq3Wt+F78803K2f79+9P1/b396fzixcvpnP44SgFaDHECZgiTsAUcQKmiBMwRZyAKeIETLXsOWdHR0c6f+ihhypn8+fPT9eeOHEinb/44ovpPPsWf5J04cKFdI6JhXNOoMUQJ2CKOAFTxAmYIk7AFHECpogTMNWy55zA9YJzTqDFECdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqaIEzBFnIAp4gRMESdgijgBU8QJmCJOwBRxAqailNLsewAwCnZOwBRxAqaIEzBFnIAp4gRMESdg6n+jnCI8sIh0qAAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}}],"source":["# Choosing one data and visualizing it\n","print(f\"Shape of the image of the first data: {training_data[5][0].shape}\")\n","print(f\"Label of the first data: {training_data[5][1]}\")\n","\n","# Visualizing it\n","img, label = training_data[5]\n","plt.title(f\"Label = {label}\")\n","plt.imshow(img.squeeze(), cmap=\"gray\")\n","plt.axis(False)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"OVDD6eSWk7Kc"},"source":["### Creating a Custom Dataset\n","Most of the time, you might now work with pre-loaded datasets (such as some of the questions of your second homework:)). In these case you must create your own custom dataset. In order to do so you should create a child from the class `Dataset` as shown below:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"EoZcm6SSk7Kc"},"outputs":[],"source":["import os\n","import pandas as pd\n","from torchvision.io import read_image\n","\n","class CustomImageDataset(Dataset):\n"," def __init__(self, annotations_file, img_dir, transform=None, target_transform=None):\n"," self.img_labels = pd.read_csv(annotations_file)\n"," self.img_dir = img_dir\n"," self.transform = transform\n"," self.target_transform = target_transform\n","\n"," def __len__(self):\n"," return len(self.img_labels)\n","\n"," def __getitem__(self, idx):\n"," img_path = os.path.join(self.img_dir, self.img_labels.iloc[idx, 0])\n"," image = read_image(img_path)\n"," label = self.img_labels.iloc[idx, 1]\n"," if self.transform:\n"," image = self.transform(image)\n"," if self.target_transform:\n"," label = self.target_transform(label)\n"," return image, label"]},{"cell_type":"markdown","metadata":{"id":"V4k18mWAk7Kc"},"source":["### Preparing your data for training with DataLoader\n","using `DataLoader`, you can shuffle data, created minibatches from it, and iterate through does minibatches just by one command:"]},{"cell_type":"code","execution_count":19,"metadata":{"id":"hv4nwzYKk7Kd","executionInfo":{"status":"ok","timestamp":1678604031704,"user_tz":-210,"elapsed":697,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["from torch.utils.data import DataLoader\n","\n","train_dataloader = DataLoader(training_data, batch_size=64, shuffle=True)\n","test_dataloader = DataLoader(test_data, batch_size=64, shuffle=True)"]},{"cell_type":"code","execution_count":20,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"7k4zN4--k7Kd","executionInfo":{"status":"ok","timestamp":1678604035270,"user_tz":-210,"elapsed":525,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"a26c3758-fca7-4fcc-f433-622892be338a"},"outputs":[{"output_type":"stream","name":"stdout","text":["First iter: \n"," tensor([5, 4, 0, 5, 4, 2, 8, 7, 7, 2, 0, 6, 7, 6, 9, 2, 3, 3, 3, 1, 8, 4, 6, 3,\n"," 9, 1, 1, 4, 4, 8, 9, 1, 7, 4, 1, 5, 8, 0, 6, 8, 0, 8, 2, 7, 0, 8, 4, 4,\n"," 3, 8, 6, 2, 6, 2, 7, 6, 3, 1, 7, 8, 9, 1, 4, 9]) \n"," \n","Second iter: \n"," tensor([4, 1, 1, 7, 9, 8, 5, 8, 6, 4, 1, 3, 3, 7, 2, 9, 8, 1, 1, 8, 1, 3, 1, 1,\n"," 8, 7, 0, 8, 1, 4, 8, 6, 9, 5, 4, 4, 8, 4, 5, 9, 7, 8, 4, 9, 1, 0, 4, 2,\n"," 5, 9, 9, 4, 6, 9, 0, 1, 3, 8, 8, 5, 6, 2, 2, 8])\n","Features batch shape: \n"," torch.Size([64, 1, 28, 28]) \n","\n","Labels batch shape: \n"," torch.Size([64]) \n","\n"]}],"source":["# iterating on train_dataloader and printing just the labels in order to see the difference\n","first_batch = next(iter(train_dataloader))\n","second_batch = next(iter(train_dataloader))\n","print(f\"First iter: \\n {first_batch[1]} \\n \")\n","print(f\"Second iter: \\n {second_batch[1]}\")\n","\n","print(f\"Features batch shape: \\n {first_batch[0].shape} \\n\")\n","print(f\"Labels batch shape: \\n {first_batch[1].shape} \\n\")"]},{"cell_type":"markdown","metadata":{"id":"vAVcSxfBk7Kd"},"source":["# Transforms\n","Data does not always come in its final processed form that is required for training machine learning algorithms. We use transforms to perform some manipulation of the data and make it suitable for training.\n","\n","All TorchVision datasets have two parameters -`transform` to modify the features and `target_transform` to modify the labels - that accept callables containing the transformation logic. For more information about transform you can visit [torchvision.transform](https://pytorch.org/vision/stable/transforms.html).\n","\n","Here is an example of using transfomrs on `torchvision.datasets.MNIST`. MNIST datasets contains PIL images which should be converted tensors normalized between 0 to 1(using `ToTensor()`)and labels which are integers and should be converted to one-hot encoded values as shown below.\n","\n","In the example below, we also use `Lambda` transforms which apply any user-defined lambda function. "]},{"cell_type":"code","execution_count":21,"metadata":{"id":"rPhGnTUSk7Kd","executionInfo":{"status":"ok","timestamp":1678604045417,"user_tz":-210,"elapsed":1462,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["from torchvision.transforms import ToTensor, Lambda\n","ds = datasets.MNIST(\n"," root=\"data\",\n"," train=True,\n"," download=True,\n"," transform=ToTensor(),\n"," target_transform = Lambda(lambda y: torch.zeros(10, dtype=torch.float).scatter_(dim=0, index=torch.tensor(y),\n"," value=1))\n",")"]},{"cell_type":"code","execution_count":22,"metadata":{"scrolled":true,"colab":{"base_uri":"https://localhost:8080/"},"id":"gYFDYMB8k7Ke","executionInfo":{"status":"ok","timestamp":1678604048032,"user_tz":-210,"elapsed":4,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"ffec8aca-d3c4-4e8c-a74f-2f515bcbd52d"},"outputs":[{"output_type":"stream","name":"stdout","text":["Features tensor: \n"," tensor([[[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0118, 0.0706, 0.0706, 0.0706,\n"," 0.4941, 0.5333, 0.6863, 0.1020, 0.6510, 1.0000, 0.9686, 0.4980,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.1176, 0.1412, 0.3686, 0.6039, 0.6667, 0.9922, 0.9922, 0.9922,\n"," 0.9922, 0.9922, 0.8824, 0.6745, 0.9922, 0.9490, 0.7647, 0.2510,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1922,\n"," 0.9333, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922,\n"," 0.9922, 0.9843, 0.3647, 0.3216, 0.3216, 0.2196, 0.1529, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706,\n"," 0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.9922, 0.7765, 0.7137,\n"," 0.9686, 0.9451, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.3137, 0.6118, 0.4196, 0.9922, 0.9922, 0.8039, 0.0431, 0.0000,\n"," 0.1686, 0.6039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0549, 0.0039, 0.6039, 0.9922, 0.3529, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.5451, 0.9922, 0.7451, 0.0078, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0431, 0.7451, 0.9922, 0.2745, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.1373, 0.9451, 0.8824, 0.6275,\n"," 0.4235, 0.0039, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.3176, 0.9412, 0.9922,\n"," 0.9922, 0.4667, 0.0980, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1765, 0.7294,\n"," 0.9922, 0.9922, 0.5882, 0.1059, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0627,\n"," 0.3647, 0.9882, 0.9922, 0.7333, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.9765, 0.9922, 0.9765, 0.2510, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1804, 0.5098,\n"," 0.7176, 0.9922, 0.9922, 0.8118, 0.0078, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.1529, 0.5804, 0.8980, 0.9922,\n"," 0.9922, 0.9922, 0.9804, 0.7137, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0941, 0.4471, 0.8667, 0.9922, 0.9922, 0.9922,\n"," 0.9922, 0.7882, 0.3059, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0902, 0.2588, 0.8353, 0.9922, 0.9922, 0.9922, 0.9922, 0.7765,\n"," 0.3176, 0.0078, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.6706,\n"," 0.8588, 0.9922, 0.9922, 0.9922, 0.9922, 0.7647, 0.3137, 0.0353,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.2157, 0.6745, 0.8863, 0.9922,\n"," 0.9922, 0.9922, 0.9922, 0.9569, 0.5216, 0.0431, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.5333, 0.9922, 0.9922, 0.9922,\n"," 0.8314, 0.5294, 0.5176, 0.0627, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000],\n"," [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n"," 0.0000, 0.0000, 0.0000, 0.0000]]]) \n","\n","Labels tensor: \n"," tensor([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.])\n"]}],"source":["print(f\"Features tensor: \\n {ds[0][0]} \\n\")\n","print(f\"Labels tensor: \\n {ds[0][1]}\")"]},{"cell_type":"markdown","metadata":{"id":"6GGD2h_Pk7Ke"},"source":["# Build the neural network"]},{"cell_type":"markdown","metadata":{"id":"hgJ1N9OVk7Ke"},"source":["Neural networks comprise of layers/modules that perform operations on data. The `torch.nn` namespace provides all the building blocks you need to build your own neural network. Every module in PyTorch subclasses the `nn.Module`. A neural network is a module itself that consists of other modules (layers). This nested structure allows for building and managing complex architectures easily."]},{"cell_type":"code","execution_count":23,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"jpJKvmlUk7Ke","executionInfo":{"status":"ok","timestamp":1678604056763,"user_tz":-210,"elapsed":891,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"a2b7f758-2a27-447a-a05a-76cfd83cf5d3"},"outputs":[{"output_type":"stream","name":"stdout","text":["Using cpu device\n"]}],"source":["import torch.nn as nn\n","\n","# setting the device to cuda if available\n","device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n","print(f\"Using {device} device\")"]},{"cell_type":"markdown","metadata":{"id":"V9BAk0K9k7Kf"},"source":["## Define the class\n","We define our neural network by subclassing `nn.Module`, and initialize the neural network layers in `__init__`. Every `nn.Module` subclass implements the operations on input data in the `forward` method."]},{"cell_type":"code","execution_count":24,"metadata":{"id":"ODWz5CHPk7Kf","executionInfo":{"status":"ok","timestamp":1678604075391,"user_tz":-210,"elapsed":666,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["class NeuralNetwork(nn.Module):\n"," def __init__(self):\n"," super(NeuralNetwork, self).__init__()\n"," self.flatten = nn.Flatten()\n"," self.linear_relu_stack = nn.Sequential(\n"," nn.Linear(28*28, 512),\n"," nn.ReLU(),\n"," nn.Linear(512, 512),\n"," nn.ReLU(),\n"," nn.Linear(512, 10),\n"," )\n","\n"," def forward(self, x):\n"," x = self.flatten(x)\n"," logits = self.linear_relu_stack(x)\n"," return logits\n"," \n","class NeuralNetwork1(nn.Module):\n"," def __init__(self):\n"," super(NeuralNetwork, self).__init__()\n"," self.flatten = nn.Flatten()\n"," self.linear1 = nn.Linear(28*28, 512)\n"," self.relu1 = nn.ReLU()\n"," self.linear2 = nn.Linear(512, 512)\n"," self.relu2 = nn.ReLU()\n"," self.linear3 = nn.Linear(512, 10)\n","\n"," def forward(self, x):\n"," x = self.flatten(x)\n"," x = self.linear1(x)\n"," x = self.relu1(x)\n"," x = self.linear2(x)\n"," x = self.relu2(x)\n"," logits = self.linear3(x)\n"," return logits"]},{"cell_type":"code","execution_count":25,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4yrbvZj0k7Kf","executionInfo":{"status":"ok","timestamp":1678604078007,"user_tz":-210,"elapsed":886,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"4801d8a3-348d-4d48-9dae-94b1f347556f"},"outputs":[{"output_type":"stream","name":"stdout","text":["NeuralNetwork(\n"," (flatten): Flatten(start_dim=1, end_dim=-1)\n"," (linear_relu_stack): Sequential(\n"," (0): Linear(in_features=784, out_features=512, bias=True)\n"," (1): ReLU()\n"," (2): Linear(in_features=512, out_features=512, bias=True)\n"," (3): ReLU()\n"," (4): Linear(in_features=512, out_features=10, bias=True)\n"," )\n",")\n"]}],"source":["# Creating an instance of the class `NeuralNetwork` and moving it to device\n","model = NeuralNetwork().to(device)\n","print(model)"]},{"cell_type":"markdown","metadata":{"id":"NCqs07uMk7Kf"},"source":["whenever we want to get the model's output on an input data `X`, we pass the data to the model directly using `model(X)`. This automatically runs `forward` method with some other background calculations. **Do not run `model.forward()` directly.**\n","\n","Note that the device of the data passed to the model should be the same device in which the model is saved."]},{"cell_type":"code","execution_count":26,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"hJrI6vSsk7Kf","executionInfo":{"status":"ok","timestamp":1678604082193,"user_tz":-210,"elapsed":8,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"9b63ee93-6cb4-412e-b8a0-50d9edb73687"},"outputs":[{"output_type":"stream","name":"stdout","text":["Model's output: \n"," tensor([[ 0.0169, 0.0447, 0.1313, -0.1328, 0.0067, -0.0359, 0.1599, 0.0207,\n"," -0.0513, -0.0066]], grad_fn=<AddmmBackward0>) \n","\n","Prediction probabilities: \n"," tensor([[0.0998, 0.1026, 0.1119, 0.0860, 0.0988, 0.0947, 0.1152, 0.1002, 0.0932,\n"," 0.0975]], grad_fn=<SoftmaxBackward0>) \n","\n","Predicted label: \n"," 6 \n","\n"]}],"source":["X = torch.rand(1, 28, 28, device=device) # creating a random input with the same device\n","\n","# passing the input to the model\n","logits = model(X)\n","print(f\"Model's output: \\n {logits} \\n\")\n","\n","# calculating model predictions by applying a softmax\n","probs = nn.Softmax(dim=1)(logits) # finding the probabilities\n","print(f\"Prediction probabilities: \\n {probs} \\n\")\n","\n","y_pred = probs.argmax(dim = 1)\n","print(f\"Predicted label: \\n {y_pred.item()} \\n\")"]},{"cell_type":"markdown","metadata":{"id":"F9y5HZ0Tk7Kg"},"source":["Each Layer in the previous models works as explained below:\n","- `nn.Flatten()`: Flattens the data except for dim=0. For example a data with the shape of (1, 10, 10) is converted to a data with the shape of (1, 100)\n","- `nn.Linear(in_features, out_features)`: A single linear layer with `in_features` inputs and `out_features` outputs\n","- `nn.ReLU()`: Simply applies relu function on each element of its input\n","- `nn.Sequential(module1, module2, ...)`: It is a container of modules. It simply passes the data through the modules with the given order\n","- `nn.Softmax(dim)`: Applies a softmax function on the given dimension of the data"]},{"cell_type":"markdown","metadata":{"id":"bvxlPH9Dk7Kg"},"source":["### Model Parameters\n","Many layers inside a neural network are parameterized, i.e. have associated weights and biases that are optimized during training. Subclassing `nn.Module` automatically tracks all fields defined inside your model object, and makes all parameters accessible using your model’s `parameters()` or `named_parameters()` methods."]},{"cell_type":"code","execution_count":27,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"_RBktBoDk7Kg","executionInfo":{"status":"ok","timestamp":1678604092259,"user_tz":-210,"elapsed":512,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"5e79bfbd-bf7f-4861-8439-395c8482d0e5"},"outputs":[{"output_type":"stream","name":"stdout","text":["Model structure: NeuralNetwork(\n"," (flatten): Flatten(start_dim=1, end_dim=-1)\n"," (linear_relu_stack): Sequential(\n"," (0): Linear(in_features=784, out_features=512, bias=True)\n"," (1): ReLU()\n"," (2): Linear(in_features=512, out_features=512, bias=True)\n"," (3): ReLU()\n"," (4): Linear(in_features=512, out_features=10, bias=True)\n"," )\n",")\n","\n","\n","Layer: linear_relu_stack.0.weight | Size: torch.Size([512, 784]) | Values : tensor([[-0.0349, 0.0039, -0.0200, ..., -0.0022, -0.0352, 0.0247],\n"," [ 0.0182, 0.0185, 0.0222, ..., 0.0111, 0.0070, -0.0148]],\n"," grad_fn=<SliceBackward0>) \n","\n","Layer: linear_relu_stack.0.bias | Size: torch.Size([512]) | Values : tensor([-0.0111, 0.0309], grad_fn=<SliceBackward0>) \n","\n","Layer: linear_relu_stack.2.weight | Size: torch.Size([512, 512]) | Values : tensor([[-0.0397, 0.0313, 0.0340, ..., 0.0367, -0.0354, 0.0040],\n"," [ 0.0168, -0.0172, -0.0326, ..., -0.0151, -0.0182, -0.0417]],\n"," grad_fn=<SliceBackward0>) \n","\n","Layer: linear_relu_stack.2.bias | Size: torch.Size([512]) | Values : tensor([0.0109, 0.0195], grad_fn=<SliceBackward0>) \n","\n","Layer: linear_relu_stack.4.weight | Size: torch.Size([10, 512]) | Values : tensor([[-0.0098, 0.0043, 0.0033, ..., -0.0353, 0.0218, -0.0093],\n"," [-0.0122, 0.0195, 0.0276, ..., 0.0206, 0.0142, 0.0162]],\n"," grad_fn=<SliceBackward0>) \n","\n","Layer: linear_relu_stack.4.bias | Size: torch.Size([10]) | Values : tensor([-0.0353, 0.0053], grad_fn=<SliceBackward0>) \n","\n"]}],"source":["print(f\"Model structure: {model}\\n\\n\")\n","\n","for name, param in model.named_parameters():\n"," print(f\"Layer: {name} | Size: {param.size()} | Values : {param[:2]} \\n\")"]},{"cell_type":"markdown","metadata":{"id":"_AOX8xMdk7Kg"},"source":["# Automatic Differentiation with `torch.autograd`\n","The most frequently used algorithm in neural networks is `back propagation`. One of the fundumental benefits of `Pytorch` and other deep learning frameworks is the implementation of automatic differetiaion using back propagation. This means that in order to find the gradients of a model, you can simply call `backward()` method and torch will automatically calculate the gradients of the pararmeters for you.\n","\n","For example, we try to calculate the gradients of a given function $y = exp(x^Tw)$ where $x \\in \\mathcal{R}^{10}$ is a constant vector and $w \\in \\mathcal{R}^{10}$ is our variable. Using back propagation, we can take $z = x^Tw$. Then $\\frac{d\\exp(z)}{dz} = \\exp(z)$, $\\frac{dz}{dw_i} = x_i$, and consequently $\\frac{dy}{dw_i} = \\frac{dy}{dz} \\frac{dz}{dw_i} = x_i . \\exp(x^Tw)= x_i . y $"]},{"cell_type":"code","execution_count":28,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"MaT_c2x1k7Kg","executionInfo":{"status":"ok","timestamp":1678604139191,"user_tz":-210,"elapsed":654,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"4bb6bcc5-d75f-4c20-8cf1-343f8e5bdc2c"},"outputs":[{"output_type":"stream","name":"stdout","text":["Expected gradients: \n"," tensor([ 244.6919, 978.7677, -244.6919, 734.0758, 0.0000]) \n","\n","Calculated gradients: \n"," tensor([ 244.6919, 978.7677, -244.6919, 734.0758, 0.0000])\n"]}],"source":["x = torch.tensor([1.,4.,-1.,3.,0.])\n","w = torch.tensor([0.1,1.5,1.2,0.2,1.], requires_grad=True)\n","z = torch.matmul(x,w)\n","y = torch.exp(z)\n","y.backward()\n","\n","\n","expected_grads = (y * x).detach() # setting requires_grad to False\n","\n","print(f\"Expected gradients: \\n {expected_grads} \\n\")\n","print(f\"Calculated gradients: \\n {w.grad}\")"]},{"cell_type":"code","execution_count":29,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"yKyszkkAk7Kh","executionInfo":{"status":"ok","timestamp":1678604145717,"user_tz":-210,"elapsed":5,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"836e531e-86d8-41bd-9967-e4810ed08881"},"outputs":[{"output_type":"stream","name":"stdout","text":["First method's requires_grad before: True\n","First method's requires_grad after: False\n","\n","Second method's requires_grad before: True\n","Second method's requires_grad after: False\n"]}],"source":["# how to set requires grad to False\n","# there are two ways to do so\n","# 1\n","z1 = x * w\n","print(\"First method's requires_grad before:\", z1.requires_grad)\n","with torch.no_grad():\n"," z1 = x * w\n","print(\"First method's requires_grad after:\", z1.requires_grad)\n","\n","# 2\n","print()\n","z2 = x * w\n","print(\"Second method's requires_grad before:\", z2.requires_grad)\n","\n","z2 = z2.detach()\n","print(\"Second method's requires_grad after:\", z2.requires_grad)\n"]},{"cell_type":"markdown","metadata":{"id":"WY4OdA-Ok7Kh"},"source":["# Optimization\n","Now that we have the model, dataset and parameters, we should optimize our model on the given dataset."]},{"cell_type":"code","execution_count":30,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"j2W525QFk7Kh","executionInfo":{"status":"ok","timestamp":1678604148818,"user_tz":-210,"elapsed":580,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"c10d6b97-abbe-4199-a111-516a1dd53c7d"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["NeuralNetwork(\n"," (flatten): Flatten(start_dim=1, end_dim=-1)\n"," (linear_relu_stack): Sequential(\n"," (0): Linear(in_features=784, out_features=512, bias=True)\n"," (1): ReLU()\n"," (2): Linear(in_features=512, out_features=512, bias=True)\n"," (3): ReLU()\n"," (4): Linear(in_features=512, out_features=10, bias=True)\n"," )\n",")"]},"metadata":{},"execution_count":30}],"source":["# creating an instance of our model\n","model = NeuralNetwork().to(device)\n","\n","# setting models mode to train mode\n","model.train()"]},{"cell_type":"markdown","metadata":{"id":"K_NtEjJqk7Kh"},"source":["## Hyperparameters\n","We define the following hyperparameters for training:\n","- **Number of Epochs** - the number times to iterate over the dataset\n","- **Batch Size** - the number of data samples propagated through the network before the parameters are updated\n","- **Learning Rate** - how much to update models parameters at each batch/epoch. Smaller values yield slow learning speed, while large values may result in unpredictable behavior during training.\n","\n"]},{"cell_type":"code","execution_count":31,"metadata":{"id":"N6WUgpg5k7Kh","executionInfo":{"status":"ok","timestamp":1678604151259,"user_tz":-210,"elapsed":8,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["learning_rate = 1e-3\n","batch_size = 64\n","epochs = 5"]},{"cell_type":"markdown","metadata":{"id":"IMjLy9d6k7Ki"},"source":["## Optimization loop\n","Each iteration of the optimization loop is called **epoch**. In each epoch these two main parts should be implemented:\n","- **The Train Loop** - Iterate over the batches and try to converge to optimal parameters\n","- **The Validation/Test Loop** - Iterate over test/validation dataset to see whether the model is improving or not\n","\n","We also need a loss function. There are various loss functions implemented in `torch.nn`. In this example, we use cross entropy."]},{"cell_type":"code","execution_count":32,"metadata":{"id":"2x3hlA84k7Ki","executionInfo":{"status":"ok","timestamp":1678604152899,"user_tz":-210,"elapsed":7,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["# Initialize the loss function\n","loss_fn = nn.CrossEntropyLoss()"]},{"cell_type":"markdown","metadata":{"id":"trHaQ4NOk7Ki"},"source":["We also have to add an optimizer to the loop. The objective of an optimizer is to use the gradien of the parameters and optimize them based on the selected optimization algorithm such as SGD, Adam, etc."]},{"cell_type":"code","execution_count":34,"metadata":{"id":"FlnIZm7lk7Ki","executionInfo":{"status":"ok","timestamp":1678604190902,"user_tz":-210,"elapsed":559,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, momentum=0.9)"]},{"cell_type":"markdown","metadata":{"id":"Kv_X6IFIk7Ki"},"source":["Inside the training loop for each batch, we first call `optimizer.zero_grad()` to make all gradients equal to zero. Then we call `loss.backward()` to calculate the gradients (using the autograd which was explained before), and finally, we call `optimizer.step()` to adjust the parameters based on their gradients.\n","\n","The final implementation of an optimization loop should be something like this:"]},{"cell_type":"code","execution_count":35,"metadata":{"id":"AifqNuNrk7Ki","executionInfo":{"status":"ok","timestamp":1678604193209,"user_tz":-210,"elapsed":5,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"outputs":[],"source":["def train_loop(dataloader, model, loss_fn, optimizer):\n"," size = len(dataloader.dataset)\n"," for batch, (X, y) in enumerate(dataloader):\n"," X = X.to(device)\n"," y = y.to(device)\n"," # Compute prediction and loss\n"," pred = model(X)\n"," loss = loss_fn(pred, y)\n","\n"," # Backpropagation\n"," optimizer.zero_grad()\n"," loss.backward()\n"," optimizer.step()\n","\n"," if batch % 100 == 0:\n"," loss, current = loss.item(), batch * len(X)\n"," print(f\"loss: {loss:>7f} [{current:>5d}/{size:>5d}]\")\n","\n","\n","def test_loop(dataloader, model, loss_fn):\n"," size = len(dataloader.dataset)\n"," num_batches = len(dataloader)\n"," test_loss, correct = 0, 0\n","\n"," with torch.no_grad():\n"," for X, y in dataloader:\n"," X = X.to(device)\n"," y = y.to(device)\n"," pred = model(X)\n"," test_loss += loss_fn(pred, y).item()\n"," correct += (pred.argmax(1) == y).type(torch.float).sum().item()\n","\n"," test_loss /= num_batches\n"," correct /= size\n"," print(f\"Test Error: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")"]},{"cell_type":"code","execution_count":36,"metadata":{"scrolled":false,"colab":{"base_uri":"https://localhost:8080/"},"id":"z9iz34FMk7Kj","executionInfo":{"status":"ok","timestamp":1678604397518,"user_tz":-210,"elapsed":201513,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"02629757-ecee-4733-a980-d055cc6dd8a5"},"outputs":[{"output_type":"stream","name":"stdout","text":["Epoch 1\n","-------------------------------\n","loss: 2.303223 [ 0/60000]\n","loss: 2.304196 [ 6400/60000]\n","loss: 2.300118 [12800/60000]\n","loss: 2.293629 [19200/60000]\n","loss: 2.279165 [25600/60000]\n","loss: 2.278445 [32000/60000]\n","loss: 2.275583 [38400/60000]\n","loss: 2.272434 [44800/60000]\n","loss: 2.265806 [51200/60000]\n","loss: 2.263628 [57600/60000]\n","Test Error: \n"," Accuracy: 47.2%, Avg loss: 2.258419 \n","\n","Epoch 2\n","-------------------------------\n","loss: 2.263481 [ 0/60000]\n","loss: 2.255519 [ 6400/60000]\n","loss: 2.246176 [12800/60000]\n","loss: 2.243089 [19200/60000]\n","loss: 2.226432 [25600/60000]\n","loss: 2.237672 [32000/60000]\n","loss: 2.225709 [38400/60000]\n","loss: 2.207955 [44800/60000]\n","loss: 2.209689 [51200/60000]\n","loss: 2.192782 [57600/60000]\n","Test Error: \n"," Accuracy: 67.3%, Avg loss: 2.193746 \n","\n","Epoch 3\n","-------------------------------\n","loss: 2.207658 [ 0/60000]\n","loss: 2.188000 [ 6400/60000]\n","loss: 2.191008 [12800/60000]\n","loss: 2.178785 [19200/60000]\n","loss: 2.173828 [25600/60000]\n","loss: 2.130807 [32000/60000]\n","loss: 2.146893 [38400/60000]\n","loss: 2.117692 [44800/60000]\n","loss: 2.098394 [51200/60000]\n","loss: 2.083458 [57600/60000]\n","Test Error: \n"," Accuracy: 68.4%, Avg loss: 2.084905 \n","\n","Epoch 4\n","-------------------------------\n","loss: 2.081283 [ 0/60000]\n","loss: 2.042619 [ 6400/60000]\n","loss: 2.072763 [12800/60000]\n","loss: 2.054853 [19200/60000]\n","loss: 2.000119 [25600/60000]\n","loss: 2.057152 [32000/60000]\n","loss: 2.017565 [38400/60000]\n","loss: 1.986733 [44800/60000]\n","loss: 1.920048 [51200/60000]\n","loss: 1.900524 [57600/60000]\n","Test Error: \n"," Accuracy: 70.2%, Avg loss: 1.896177 \n","\n","Epoch 5\n","-------------------------------\n","loss: 1.891385 [ 0/60000]\n","loss: 1.844790 [ 6400/60000]\n","loss: 1.848859 [12800/60000]\n","loss: 1.848612 [19200/60000]\n","loss: 1.780923 [25600/60000]\n","loss: 1.748496 [32000/60000]\n","loss: 1.759260 [38400/60000]\n","loss: 1.703711 [44800/60000]\n","loss: 1.651100 [51200/60000]\n","loss: 1.561859 [57600/60000]\n","Test Error: \n"," Accuracy: 73.2%, Avg loss: 1.605770 \n","\n","Epoch 6\n","-------------------------------\n","loss: 1.680934 [ 0/60000]\n","loss: 1.616143 [ 6400/60000]\n","loss: 1.564987 [12800/60000]\n","loss: 1.524909 [19200/60000]\n","loss: 1.550407 [25600/60000]\n","loss: 1.491756 [32000/60000]\n","loss: 1.327466 [38400/60000]\n","loss: 1.331278 [44800/60000]\n","loss: 1.262214 [51200/60000]\n","loss: 1.254692 [57600/60000]\n","Test Error: \n"," Accuracy: 76.9%, Avg loss: 1.279823 \n","\n","Epoch 7\n","-------------------------------\n","loss: 1.341795 [ 0/60000]\n","loss: 1.265673 [ 6400/60000]\n","loss: 1.236625 [12800/60000]\n","loss: 1.101788 [19200/60000]\n","loss: 1.221997 [25600/60000]\n","loss: 1.113819 [32000/60000]\n","loss: 1.072886 [38400/60000]\n","loss: 0.982836 [44800/60000]\n","loss: 1.109988 [51200/60000]\n","loss: 0.963634 [57600/60000]\n","Test Error: \n"," Accuracy: 80.4%, Avg loss: 1.019816 \n","\n","Epoch 8\n","-------------------------------\n","loss: 0.945164 [ 0/60000]\n","loss: 0.757766 [ 6400/60000]\n","loss: 1.010496 [12800/60000]\n","loss: 0.935692 [19200/60000]\n","loss: 0.920248 [25600/60000]\n","loss: 0.998523 [32000/60000]\n","loss: 0.911712 [38400/60000]\n","loss: 0.920831 [44800/60000]\n","loss: 0.925198 [51200/60000]\n","loss: 0.910937 [57600/60000]\n","Test Error: \n"," Accuracy: 82.1%, Avg loss: 0.842116 \n","\n","Epoch 9\n","-------------------------------\n","loss: 0.924726 [ 0/60000]\n","loss: 1.015149 [ 6400/60000]\n","loss: 0.795692 [12800/60000]\n","loss: 0.872419 [19200/60000]\n","loss: 0.810613 [25600/60000]\n","loss: 0.701530 [32000/60000]\n","loss: 0.795524 [38400/60000]\n","loss: 0.773257 [44800/60000]\n","loss: 0.723039 [51200/60000]\n","loss: 0.720438 [57600/60000]\n","Test Error: \n"," Accuracy: 83.4%, Avg loss: 0.723503 \n","\n","Epoch 10\n","-------------------------------\n","loss: 0.820295 [ 0/60000]\n","loss: 0.814022 [ 6400/60000]\n","loss: 0.784014 [12800/60000]\n","loss: 0.782202 [19200/60000]\n","loss: 0.734190 [25600/60000]\n","loss: 0.671185 [32000/60000]\n","loss: 0.665508 [38400/60000]\n","loss: 0.627052 [44800/60000]\n","loss: 0.809398 [51200/60000]\n","loss: 0.707665 [57600/60000]\n","Test Error: \n"," Accuracy: 84.6%, Avg loss: 0.641895 \n","\n","Done!\n"]}],"source":["loss_fn = nn.CrossEntropyLoss()\n","optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n","\n","epochs = 10\n","for t in range(epochs):\n"," print(f\"Epoch {t+1}\\n-------------------------------\")\n"," train_loop(train_dataloader, model, loss_fn, optimizer)\n"," test_loop(test_dataloader, model, loss_fn)\n","print(\"Done!\")"]},{"cell_type":"markdown","metadata":{"id":"tbDGabpZk7Kj"},"source":["# Saving and loading the model\n","After training the model, we should save it to be able to use it afterwards. There are two ways to do this:\n","- Saving just the weights of the model\n","- Saving the weights and the structure "]},{"cell_type":"markdown","metadata":{"id":"LDF_e4HZk7Kj"},"source":["### Saving & Loading the weigths"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"POMbMm4rk7Kj","outputId":"048684d3-1f82-48e7-9baf-4c1139b6f872"},"outputs":[{"data":{"text/plain":["NeuralNetwork(\n"," (flatten): Flatten(start_dim=1, end_dim=-1)\n"," (linear_relu_stack): Sequential(\n"," (0): Linear(in_features=784, out_features=512, bias=True)\n"," (1): ReLU()\n"," (2): Linear(in_features=512, out_features=512, bias=True)\n"," (3): ReLU()\n"," (4): Linear(in_features=512, out_features=10, bias=True)\n"," )\n",")"]},"execution_count":30,"metadata":{},"output_type":"execute_result"}],"source":["# saving\n","torch.save(model.state_dict(), 'model_weights.pth')\n","\n","# loading\n","model.load_state_dict(torch.load('model_weights.pth'))\n","model.eval()"]},{"cell_type":"markdown","metadata":{"id":"CqQ3pEvQk7Kj"},"source":["### Saving & Loading the model and its weigths"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"X56fengBk7Kj","outputId":"96c99022-c187-44cd-c7da-c4a13bd26d2d"},"outputs":[{"data":{"text/plain":["NeuralNetwork(\n"," (flatten): Flatten(start_dim=1, end_dim=-1)\n"," (linear_relu_stack): Sequential(\n"," (0): Linear(in_features=784, out_features=512, bias=True)\n"," (1): ReLU()\n"," (2): Linear(in_features=512, out_features=512, bias=True)\n"," (3): ReLU()\n"," (4): Linear(in_features=512, out_features=10, bias=True)\n"," )\n",")"]},"execution_count":31,"metadata":{},"output_type":"execute_result"}],"source":["# saving\n","torch.save(model, 'model.pth')\n","\n","# loading\n","model = torch.load('model.pth')\n","model.eval()"]},{"cell_type":"markdown","source":["Load model directly from drive with gdown lib."],"metadata":{"id":"nBwWkUWMnL6x"}},{"cell_type":"code","source":["# Upgrade gdown to work properly.\n","!pip install --upgrade --no-cache-dir gdown"],"metadata":{"id":"eNnHOthznKLj"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["import gdown\n","# https://drive.google.com/file/d/1kAhqypQ3h97dYzRaKRAfWWEnUMtkYrdq/view?usp=sharing\n","!gdown 1kAhqypQ3h97dYzRaKRAfWWEnUMtkYrdq\n","\n","# Check if CUDA is available. \n","if torch.cuda.is_available():\n"," Model = torch.load(\"./CNN3.h5\")\n","else:\n"," Model = torch.load(\"./CNN3.h5\", map_location=torch.device('cpu'))"],"metadata":{"id":"YtJ7KDyUnXCn"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["# Model Evaluation\n","After training and saving the model, we should evaluate the model."],"metadata":{"id":"7p5Kj7ooohaF"}},{"cell_type":"code","source":["def Evaluation_func(dataloader, model):\n"," Pred_label = []\n"," True_label = []\n"," model.eval()\n"," with torch.no_grad():\n"," for X, y in dataloader:\n"," X = X.to(device)\n"," y = y.to(device)\n"," # Compute prediction\n"," pred = model(X)\n","\n"," # get true label and predicate label\n"," label = y.cpu().numpy()\n"," True_label.extend(label)\n"," pred_label = pred.argmax(1).cpu().numpy()\n"," Pred_label.extend(pred_label)\n","\n"," return True_label, Pred_label"],"metadata":{"id":"--2gGSvKo5FW","executionInfo":{"status":"ok","timestamp":1678605046855,"user_tz":-210,"elapsed":3,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}}},"execution_count":37,"outputs":[]},{"cell_type":"code","source":["from sklearn.metrics import (accuracy_score, confusion_matrix, f1_score,\n"," precision_score, recall_score)\n","import seaborn as sns\n","True_label, Pred_label = Evaluation_func(test_dataloader, model)\n","\n","test_acc = accuracy_score(True_label, Pred_label)\n","test_prc = precision_score(True_label, Pred_label, average='macro')\n","test_rcl = recall_score(True_label, Pred_label, average='macro')\n","test_F1 = f1_score(True_label, Pred_label, average='macro')\n","\n","test_cf = confusion_matrix(True_label, Pred_label)\n","\n","classes=[\"0\", \"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\"]\n","\n","plt.figure(figsize=(8, 8))\n","sns.heatmap(test_cf, xticklabels=classes, yticklabels=classes, annot=True, fmt='.0f')\n","plt.title('Confusion Matrix for Test Data')\n","plt.show()\n","print(\"\\n\")\n","print(f\"Test Accuracy: {test_acc*100:.2f} %\")\n","print(f\"Test Precision: {test_prc*100:.2f} %\")\n","print(f\"Test Recall: {test_rcl*100:.2f} %\")\n","print(f\"Test F1-score: {test_F1*100:.2f} %\")"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":603},"id":"rEIxL1ssplOU","executionInfo":{"status":"ok","timestamp":1678605090432,"user_tz":-210,"elapsed":5787,"user":{"displayName":"Mohammadreza Estarki","userId":"02628779610032382505"}},"outputId":"0b7554fe-b82c-4682-dcf2-6627b96e4e58"},"execution_count":39,"outputs":[{"output_type":"display_data","data":{"text/plain":["<Figure size 576x576 with 2 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAdAAAAHiCAYAAABY752mAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABw5ElEQVR4nO3dd3xT1f/H8ddJ080sZbVFQEBUVFbZe+/hAmUoCqIsceIWAfXrABEcCIKyZDrYe4MyWmjZq2W2pVD2piPn90fS2iJQCEnu5dfP00ceJufe5Ly5uc3JOffcG6W1RgghhBB3xmJ0ACGEEOJeJA2oEEII4QRpQIUQQggnSAMqhBBCOEEaUCGEEMIJ0oAKIYQQTpAGVNyUUspfKTVXKXVOKTXzLl6ns1JqiSuzGUEptVAp9byTz/1UKXVSKZXo6lxCCGNIA/r/gFKqk1IqUil1USl1zPFBX9sFL/0UUBgooLV+2tkX0Vr/prVu6oI8WSil6iultFLqr+vKyzvKV93m63yilJqc3Xpa6xZa6wlO5LwPeBN4WGtd5E6ff/1rOd7n9JtWSl3K9LiOE695SCnV+BbL6yulbJnqiFNKzVBKVbmDOm5rGwtxL5EG9B6nlHoD+Bb4HHtjdx/wI9DOBS9fHNintU51wWu5SxJQQylVIFPZ88A+V1Wg7O7mb+U+4JTW+oQTdVszP9ZaH9Fa50q/OYrLZypbexc5byXBUV9uoDqwB1irlGrkpvqEMD+ttdzu0RuQF7gIPH2LdXyxN7AJjtu3gK9jWX0gDnvv6ARwDHjBsWwQkAykOOroDnwCTM702iUADVgdj7sBB4ALwEGgc6bydZmeVxOIAM45/l8z07JVwBDgb8frLAGCb/JvS8//E9DHUeYFxAMfA6syrTsCOAqcBzYDdRzlza/7d27NlOMzR44rQGlHWQ/H8lHAH5le/0tgOaCuy9jY8Xyb4/XHO8rbAjuBs47XfSjTcw4B7wDbgGvp2/cm20ADpTO910OBI8Bxx3bxdywLBuY56jsNrMX+BXqSI9sVR74BN9vONyj/Hoi8i238ArDb8T4fAF42+m9KbnK7k5vhAeR2F2+e/YMpNZsP2MHABqAQUBD4BxjiWFbf8fzBgDfQErgM5Hcs/4SsDeb1j0s4PsCtQKDjg7OsY1lRoJzjfjccDSgQBJwBujqe96zjcQHH8lVALPAA4O94/MVN/m31sTegNYGNjrKWwGKgB1kb0C5AAUedbwKJgN+N/l2ZchwByjme403WBjQAey+3G1AHOAmE3SpnpscPAJeAJo7XHQDEAD6O5YeAaKAYjgbwFu9v5gZ0ODDHsY1zA3OB/zmW/Q97g+rtuNXB0dg76mt8izqy5M9U3hB74xvo5DZuBZQCFFAP+75Xyei/K7nJ7XZvMoR7bysAnNS3HmLtDAzWWp/QWidh71l2zbQ8xbE8RWu9AHsPoayTeWzAI0opf631Ma31zhus0wrYr7WepLVO1VpPxT4c2CbTOr9qrfdpra8AM4AKt6pUa/0PEKSUKgs8B0y8wTqTtdanHHUOw95by+7fOV5rvdPxnJTrXu8y9u34DTAZ6Ke1jsvm9dJ1BOZrrZc6Xnco9i8LNTOtM1JrfdSxDbKllFJAT+B1rfVprfUF7MP6zzhWScH+paa4471eq7W+2wthJ2Bv/PLBnW9jrfV8rXWstluNfbThjo/hCmEUaUDvbaeA4OuPk10nBDic6fFhR1nGa1zXAF8GcnGHtNaXsDcMrwDHlFLzlVIP3kae9EyhmR5nnql6u3kmAX2BBsBf1y9USr2llNrtmFF8Fvvwd3A2r3n0Vgu11huxDz0q7A397cqyDbTWNkddmbfBLeu+gYLYe8WblVJnHf/GRY5ygK+x93KXKKUOKKXevcPXv5FQ7D3gs3Dn21gp1UIptUEpddqxfstbrS+E2UgDem9bj/0YWftbrJOAfTJQuvscZc64hP1DOl2WGaVa68Va6ybYezp7gJ9vI096pngnM6WbBPQGFjh6hxkcM1MHAB2wD0/nw378VaVHv8lr3rKHppTqg72XleB4/duVZRs4eo/FyLoN7rR3eBL7ccxyWut8jlte7ZhopLW+oLV+U2t9P/bjr29kmgDkbE/0cWCL1vrSnW5jpZQv8Af23ndhx/oLMq0vhOlJA3oP01qfwz5Z5gelVHulVIBSytvxzf4rx2pTgQ+VUgWVUsGO9Z09nSAaqOs4lSIv8F76AqVUYaVUO6VUIPZG/SL2Id3rLQAecJx6Y1VKdQQexj7BxWla64PYj6N9cIPFubEf600CrEqpj4E8mZYfB0rcyUxbpdQDwKfYj/t1BQYopSrc5tNnAK2UUo2UUt7Yjxdew3582imOXuzPwHClVCFHxlClVDPH/dZKqdKOxvockMa/789x4P7bqccxIzlUKTUQ+3Hm9x2L7nQb+2D/8pEEpCqlWgAuP9VJCHeSBvQe5zjW9AbwIfYPo6PYhzJnOVb5FIjEPqNzO7DFUeZMXUuB6Y7X2kzWRs/iyJGAfZZnPaDXDV7jFNAae6NxCnuvpbXW+qQzma577XVa6xv1rhdjH87ch33o9CpZh0jTLxJxSim1Jbt6HEPmk4EvtdZbtdb7sTckkxw9q+xy7sXe8H6HvefYBmijtU7O7rnZeAf7MO0GpdR5YBn/HoMs43h8EfvIxY9a65WOZf/D/iXrrFLqrZu8dohS6qLj+RHAo0B9rXX6BTLuaBs7jtG+iv3LxBmgE/YJUELcM9Jn4QkhhBDiDkgPVAghhHCCNKBCCCGEE6QBFUIIIZwgDagQQgjhBGlAhRBCCCfc6go2LnFl4UhTTfPN3e5LoyOIe5TV4mV0hAw2faNTbI1jk9n894zU5Hi3Xawi5eQBl+8I3sH3m/biGtIDFUIIIZzg9h6oEEKIHMKWZnQCj5IGVAghhGuY7NCCu8kQrhBCCOEE6YEKIYRwDZv0QIUQQgiRDemBCiGEcAmdw46BSgMqhBDCNWQIVwghhBDZkR6oEEII18hhQ7jSAxVCCCGcID1QIYQQrpHDrkQkPVAhhBDCCdIDFUII4Ro57BioNKBCCCFcQ05jEUIIIUR2TNOA/rZ6K09+MZUnvpjC5FVbsyybuDKKCq/9wJmLV7KU7zhynMpv/MjS6BiP5WzWtD47d6xhz651DHi7j8fqvZGwsBCWLZnJtq0r2Rq9gn59u0ueTIx+r8LCirJ48TSiopazZcsy+vR5EYD8+fMyf/5v7NixmvnzfyNfvrwezwbQt293orYsIzpqOf36Gfte/TxmGAlxW4mOWm5oDjDffgzG78u3S2uby29mZooGNObYKf5cv4vJbzzFjLefYe2uQxxJOgtA4pkLrN9zlKL5c2V5TprNxoi566letpjHclosFkaO+IzWbbrwaPkGdOzYnoceKuOx+q+XmprK2wMG8Vj5BtSq3YZevbpJHgczvFepqWm8886nVKzYiLp12/HKK8/x4INleOutPqxc+TePPFKPlSv/5q23ens0F0C5h8vS/cVnqVmrNZXDm9KyZWNKlSrh8RzpJk6cQavWnQ2rPzMz7cdgjn1Z3JgpGtADx8/waPHC+Pt4Y/WyULlUCMu3HQBg6Ky/ea1tTUBlec7UNdtp9FgpgnIFeCxn1SoViY09xMGDR0hJSWHGjNm0bdPMY/VfLzHxBFHROwC4ePESe/bsJzSkiOTBHO9VYuIJorNsjxhCQ4vQpk0TJk/+HYDJk3+nbdumHs0F8OCDpdm0KZorV66SlpbG2jUbaN++hcdzpFu7biOnz5w1rP7MzLQfgzn25dtms7n+ZmLZNqBKqQeVUu8opUY6bu8opR5yZYjSRYLYciCBs5euciU5hXW7DnP87EVWbj9AwbyBlA0NzrJ++rIOtR5xZYxshYQW4WhcQsbjuPhjhBj4h5VZ8eJhVCj/CBs3RRkdBTA+j9neq+LFw6hQoRybNkVRqFAwiYknAPuHdaFCwdk82/V27tpL7dpVCQrKh7+/H82bNyQsLMTjOczO6P0YzLcv35K2uf5mYrechauUegd4FpgGbHIUhwFTlVLTtNZfuCLE/UWCeKFRJXqNmoO/j5WyocEkp6YxbulmRvVq+5/1v/5rHf3b1MBiUTd4tZwnMDCAGdN/5o23BnLhwkWj45guj9ECAwOYOnU0b7016IbbQ2vPZ9qzJ4avh/7IgvlTuHTpMlu37SQtLWedBJ8d2Y9FdrI7jaU7UE5rnZK5UCn1DbATuGEDqpTqCfQE+K5fR7q3qJltkMerP8zj1R8GYOS89RTIHcDK7Qfp8NV0AE6cu8izQ2cw+Y2n2HX0BO9MWALA2UtXWLf7MF4WCw0fuz/beu5GQnwixTJ9Sw8LLUpCQqJb68yO1Wpl5vSfmTr1L2bNWmhoFjPlMct7ZbVamTZtNNOm/cXs2YsAOHHiJEWKFCIx8QRFihQiKemkx3MBjB8/jfHjpwEwZPA7xMUfMySHGZllPwbz7Mu3JYddiSi7BtQGhACHrysv6lh2Q1rrMcAYgCsLR97W9+vTFy4TlDuAY2cusGLbASa+9hSd65XPWN5i0ESmvPk0+XP5s+Dj5zLKP/ptOXXLFXd74wkQERlN6dIlKVGiGPHxiXTo0I6uzxk7I+7nMcPYvSeGb0eMMTRHOrPkMct7NXr01+zZE8PIkWMzyubNW0qXLk8xdOiPdOnyFHPnLvV4LoCCBQuQlHSKYsVCaN++BbXr/He0J6cyy34M5tmXxX9l14C+BixXSu0HjjrK7gNKA31dGeTNXxdx7tJVrF4W3nuqLnkCfF358i6RlpZG/9c+ZMH8KXhZLIyfMJ1du/YZlqdWzSp07fIU27bvIjLC3iP/6KMvWLhoRY7PY4b3qmbNKnTu/CTbt+9m40Z7L+bjj79i6NAf+e23UXTr1pEjR+Lp3LmXR3Olmz5tDAUK5CclJZVX+3/AuXPnDckBMHnSD9SrW4Pg4CAOHYhk0OCh/OroHXuamfZjMMe+fNtMfszS1ZTO5gCMUsoCVAVCHUXxQITW+rb66rfbA/WU3O2+NDqCuEdZLV5GR8hgM9kHlc2IA7nCKanJ8W6bPHJt53KX7wi+5RqZdrJLtpfy0/YzWTd4IIsQQghxz5Br4QohhHANk42MuJspLqQghBBC3GukByqEEMI1TH7lIFeTBlQIIYRL3Obc0v83ZAhXCCGEcIL0QIUQQriGTCISQgghRHakARVCCOEaBv2cmVLqF6XUCaXUjkxlQUqppUqp/Y7/53eUK8cvi8UopbYppSples7zjvX3K6Wez65eaUCFEELc68YDza8rexdYrrUuAyx3PAZoAZRx3HoCo8De4AIDgWrYr743ML3RvRlpQIUQQriGQb8HqrVeA5y+rrgdMMFxfwLQPlP5RG23AcinlCoKNAOWaq1Pa63PAEv5b6OchUwiEkII4Rrm+jmzwlrr9N/oSwQKO+6H8u+PowDEOcpuVn5T0gMVQghhWkqpnkqpyEy3nnf6Gtr+qykuv9C99ECFEEK4hhtOY8n8+9J36LhSqqjW+phjiPaEozweKJZpvTBHWTxQ/7ryVbeqQHqgQggh/j+aA6TPpH0emJ2p/DnHbNzqwDnHUO9ioKlSKr9j8lBTR9lNSQ9UCCGEaxh0LVyl1FTsvcdgpVQc9tm0XwAzlFLdgcNAB8fqC4CWQAxwGXgBQGt9Wik1BIhwrDdYa339xKQspAEVQgjhGgZdiUhr/exNFjW6wboa6HOT1/kF+OV263V7A5q73ZfuruKOXElYa3SELPxD6hgdIYPZfvbd5Uf871KquWYYilvwtXobHSGLa6kpRkcQbiA9UCGEEK6Rw37OTCYRCSGEEE6QHqgQQgjXyGE9UGlAhRBCuIT8oLYQQgghsiU9UCGEEK6Rw4ZwpQcqhBBCOEF6oEIIIVzDoAspGEV6oEIIIYQTpAcqhBDCNXLYMVBpQIUQQriGDOEKIYQQIjvSAxVCCOEaOWwIV3qgQgghhBOkByqEEMI1ctgxUGlAhRBCuIYM4ZpPs6b12bljDXt2rWPA2zf8IfG79uHn31C31TO07/JKRtm58xfo0f99WnbsTo/+73Pu/IWM8lffG8zjz/XimR792X/gUMZzmj75PI937cWTz/ehw4uvuiVrurCwEJYtmcm2rSvZGr2Cfn27u7W+27F/3waitiwjMmIJG9YvMDSLJ/abezWPr68v6/+ex+bIpWyNXsHAj980NM/PY4aRELeV6KjlhtQfGlqUBQunErl5KRGRS+jd+wUAHn30IVas/JNNmxYx8/ex5M6dy5B8Ztp3xL+U1tqtFVh9Qu+qAovFwu6da2ne8lni4o6xYf0CunTtze7d+516vSsJa29YHhm9nQB/f94fMpRZk38CYNgP48ibJzc9unZg7KQZnL9wgTd6d2fo92MJCPCn94udOXD4KJ8N+4FxI78A7A3o9HEjyZ8v723l8Q+p49S/A6BIkUIULVKIqOgd5MoVyKaNi3jyqRed3jbK6ST/2r9vA9VrtODUqTN3/Vp3s+O4er+5W2bLAxAYGMClS5exWq2sWfUXr78xkI2bthiSpU7taly8eIlffx1BhYqN7vr1fK3ed7R+kSIFKVKkENHRO8mVK5B1f8/lmY49GfPzMN5/73PWrdvIc889TfESxRgy+Js7znMtNeWOn5PO1ftOanK8K/7Ub+jK/G9d3qD4t3rNbXnvlul7oFWrVCQ29hAHDx4hJSWFGTNm07ZNM5fXE17hUfLmyZ2lbOXa9bRr0RiAdi0as2LNegBiDx2hWqXyANxfvBjxx45z8vTdNxh3KjHxBFHROwC4ePESe/bsJzSkiMdzmJGn9pt7NQ/ApUuXAfD2tmL19sbdX6ZvZe26jZw+c9aw+hMTk4iO3gnY/5b27o0lJKQIpUuXZN26jQAsX76Odu1aeDybGfcdYWf6BjQktAhH4xIyHsfFHyPEQ43EqTNnKRgcBEBwgfyccvyBly19P8tW/w3A9l17OXb8BMdPnARAKUXP1z+gw4v9mDnbc0OYxYuHUaH8I2zcFOWxOm9Ea83CBVPZuGEhPbp3NiyHkfvNvZAH7D2byIglHIvfxvLla9gUYey+Yxb33RdG+fIPExERze7d+2ndpikATzzRkrCwoh7PY8Z956a0zfU3EzN9A2oWSimUso8k9Oj6NBcuXuLJ5/vw2+9zeLBMKbws9k05cdRQZv76PaOGDWHqn/OIjN7u9myBgQHMmP4zb7w1kAsXLrq9vlup3+BxqlZrTus2XejVqxu1a1czNI+4OZvNRniVphQvGU6V8IqUK1fW6EiGCwwMYMrUUQwYMJgLFy7S65UB9HypC+v+nkuu3LlITnZ+KDZHsNlcfzMxp2fhKqVe0Fr/epNlPYGeAMorLxZLoLPVkBCfSLGwkIzHYaFFSUhIdPr17kSB/PlIOnmagsFBJJ08TZDjuGauwEA+/eANwN7javZUN8JC7d8ICxcMznhuo7o12b5rL+EVHnVbRqvVyszpPzN16l/MmrXQbfXcrvT3JinpFLNmL6RKlQoZQ2AezWHgfnMv5Mns3LnzrFr9t32iys69RscxjNVqZcqUn5g+bRZzZi8GYN++WNq2fQ6A0qVL0rx5A4/nMvO+k9PdTQ900M0WaK3HaK3Dtdbhd9N4AkRERlO6dElKlCiGt7c3HTq0Y+68JXf1mrerfu3qzF64DIDZC5fRoE4NAM5fuEhKiv2b6B9zF1G5wqPkCgzk8pWrGceVLl+5yj+btlDm/hJuzfjzmGHs3hPDtyPGuLWe2xEQ4E+uXIEZ95s0rmfYB7KR+829kCc4OIi8efMA4OfnR+NGddm7N9awPGYwatSX7N0bw3ffjcsoK1iwAGAfgXrnnb6MG/ubx3OZbd+5pRw2hHvLHqhSatvNFgGFXR/nv9LS0uj/2ocsmD8FL4uF8ROms2vXPpfX8/bAL4iI2sbZs+dp1L4Lvbt3pUfXDrz50ef8OW8xIUUKMWzI+wAcOHyUDz4dhgJKlSzO4PdeA+DU6TP0f3+IPXdqGi2b1qd29XCXZ01Xq2YVunZ5im3bdxEZYf+D+uijL1i4aIXb6ryVwoUL8vtM+4ePl9WLadNmsWTJKkOyeGq/uVfzFC1amF/GfYuXlwWLxcLvv89l/oJlhuWZPOkH6tWtQXBwEIcORDJo8FB+HT/NY/XXqBFOp85PsmP7btZvsM9d+GTgV5QqVZKeL3cFYM7sxUycONNjmdKZbd8R/7rlaSxKqeNAM+D6KaYK+EdrHfLfZ2V1t6exuNrNTmMxyt2cxuJqZpsrbqodR9xT7vQ0Fne7m9NYXM2tp7H89YXrT2N5/F2zfTRlyO4Y6Dwgl9Y6+voFSqlV7ggkhBBC3Atu2YBqrW96aRutdSfXxxFCCHHPMvkxS1eTa+EKIYRwDZOfduJqch6oEEII4QTpgQohhHAN6YEKIYQQIjvSAxVCCOEaBv4ggRGkARVCCOEaMoQrhBBCiOxID1QIIYRrSA9UCCGEENmRHqgQQgjXkCsRCSGEEE6QIVwhhBBCZEd6oEIIIVwjh50HKj1QIYQQwgnSAxVCCOEacgxUCCGEENlxew/UavFydxV3JN99DY2OkMX5r1obHSFDofeXGB0hC425jqdcS00xOkIGX6u30RGySLWlGR0hC7N97lzDPPuOW+WwHqgM4QohhHCNHHYeqAzhCiGEEE6QHqgQQgiX0DZzHXZxN+mBCiGEEE6QHqgQQgjXkElEQgghhBNkEpEQQgghsiM9UCGEEK4hk4iEEEIIkR3pgQohhHANmUQkhBBCOCGHNaAyhCuEEEI4QXqgQgghXEN+UFsIIYQQ2ZEeqBBCCNeQY6BCCCGEyI4pe6CjR39NixaNSEo6ReXKTQB49NGH+O67z8mVK5DDh+Po1u1VLly46PYsoaFF+XnsNxQqFIzWml9/mcqPP/7KhInf88AD9wOQN28ezp07T43qLd2Ww1qxEdZHaoPW2E7Fk7xkAqSlAuBdryPWcjW58mN/x7qNsT5SC2w29JWLJC+dgL5w2i25fH19WLx0Br4+PlitXsyatZDPPv2Wcb8Mp2Klx0hNSSFy81Ze7fsBqampbsmQzizv1c38PGYYrVo25kTSSSpUbOTx+s22fUaPHkpLx995pcqNAZg86cd/s+TLw7mz56larbnbs6SzWCysXjuLhITjdHz6JYoXD+OX8SMICspPdPQOevZ4k5QUz/84drOm9fnmm8F4WSz88utUvvr6B49nuC1yIQXjTZo0k7Ztn8tSNmrUV3z00ReEhzdlzpxFvPHGyx7JkpaWyvvvfUp45SY0qP84PV/uyoMPlub55/pSo3pLalRvyexZC5k9e5HbMqjAfFgrNOTqlM+5OnkwKAteZasAYClUHOUXkGV9W9IRrk79nKu/DSEtZjPedZ50W7Zr15Jp1aKTY1u0onGTelSpUoHp02dTqUIjqlZpjr+fH91e6Oi2DOnM8F7dysSJM2jVurMhdYP5ts+kSTNp07ZrlrIuXXtTtVpzqlZrzqy/FjJr9kKPZEnXq3c39u6NzXg8aMgAfvzhVyqWb8jZs+d47vmnPZoH7I36yBGf0bpNFx4t34COHdvz0ENlPJ7jtmib628mZsoGdN26TZw5czZLWZkyJVm7diMAy5evpX17z/QgEhOTiI7eCcDFi5fYuzeWkJAiWdZ54slWzJwxx71BLBaweoOyoKw+6ItnQSm86zxJ8to/sqxqi9sHqfZvyWnHDqJy5XNrtEuXLgPg7W3F29uKBpYsXpWxPDJyK6GhRd2aAUz0Xt3E2nUbOX3dfu1JZts+69Zt/M/feWZPPtWaGdNneyQLQEhIEZo1b8DECTMyyurWq8Gsv+yN+JTf/qRV6yYey5OuapWKxMYe4uDBI6SkpDBjxmzatmnm8Rziv7JtQJVSDyqlGimlcl1X7rlxFWDXrn20adMUgCeeaEVYmPs/kK93331hlC//MBER0RlltWpV5cSJk8TGHnJbvfrSWVI3L8W/+//wf+krdPIVbEd2Yy3fgLQDW+Hy+Zs+11quFmmHdrotG9i/If+zYT4HD0eyYvk6IjNtH6vVyrOdHmfpktVuzXA9o96re4XZt0/t2tU4cfwkMR7M8sVXH/Lxh19ic0yECSqQn3NnL5CWlgZAQnwiRa/7wuEJIaFFOBqXkPE4Lv7Yf774mIZNu/5mYrdsQJVSrwKzgX7ADqVUu0yLP3dnsOu9/PLbvPzyc/zzz3xy585FcrJnj0MEBgYwZeooBgwYnOXY69Md2rr/G7tvAF6lynPl1w+4MnYAePvi9VB1vMpUJjV65U2f5vVgNSyFi5O6eYlb49lsNmpWb0XZMjUIDy/Pww8/kLFs+Igh/L1uE//8E+HWDJkZ+l7dA+6F7dOxQztmzPBc77NZ8wYkJZ0iOnqHx+oU977sJhG9BFTWWl9USpUAfldKldBajwDUzZ6klOoJ9ASwWvPj5ZXrZqvetn37YmndugsApUuXpHnzhnf9mrfLarUyZcpPTJ82izmzF2eUe3l50a5tM2rVbuPW+r3uexB97iRcsX/YpcVE4V29Dcrqjd8LQ+wrefvg120IV8d/BICl2IN4V23B1ZnDMiYbudu5cxdYs2Y9jZvUY9eufbz3/qsEBwfRqe/7HqkfjH+vzO5e2D5eXl60a9ecGjU9N9GrevXKtGjZiCZN6+Pn50vu3Ln48quPyJsvN15eXqSlpRESWoRjCYkey5QuIT6RYmEhGY/DQouSYECO26HlNJasy7XWFwG01oeA+kALpdQ33KIB1VqP0VqHa63DXdF4AhQsWAAApRTvvfcqY8dOdsnr3o5Ro75k794YvvtuXJbyhg1rs3ffARLi3bsz6wunsRS9334MFPAq9iCpW5Zx5ecBXP3lA67+8gGkJGc0nqpgMXwadeHanB/hygW3ZgsODiJv3twA+Pn50rBhHfbti+X5bh1p1LguLzz/KtqDVycx+r0yu3th+zRqWIe9+2KJ92CWQZ8M5eGytXmsXD1e7NafNavX81L3N1i7ZgPtH28BQKfOT7Bg/jKPZUoXERlN6dIlKVGiGN7e3nTo0I6589w7quQ0GcLN4rhSqkL6A0dj2hoIBh51V6iJE79j1apZPPDA/cTEbKRbt4506NCO7dtXsW3bShISjjMh04F+d6pRI5xOnZ+kXr0arN+wgPUbFtCsWX0AnnqqDTNnun/Iy5Z4iLT9W/Dr9CF+XT4GpUjdsfam6/vUeRLl7Ytvq574df4Qnza93ZatcJFCLFg0lQ0bF7Jm7WxWrFjLooUrGDHyUwoVCmbFqj/5Z8N83n2vn9sypDPDe3Urkyf9wLo1cyj7QCkOHYjkhW7PeLR+s22fiRO/Z7Xj7zw2ZhPdutlnaj/doa1HJw/dysCPvqJP3xeJ2rqCoKB8TJww0+MZ0tLS6P/ahyyYP4Ud21bx++9z2bVrn8dziP9St+odKKXCgFSt9X++Ciqlammt/86uAj+/+0z1FcLLYq6Jx0mfm2c2XaH3zfWtVmOqXYdrqZ4//+9mfB2jEWaRakszOkIWflYfoyNkcSn5qtERMqQmx9909PBuXfq0i8v/aAM/nOy2vHfrlq2J1jruRo2nY1m2jacQQgjhbkqp15VSO5VSO5RSU5VSfkqpkkqpjUqpGKXUdKWUj2NdX8fjGMfyEs7Wa67umBBCiHuXAcdAlVKhwKtAuNb6EcALeAb4EhiutS4NnAG6O57SHTjjKB/uWM8p0oAKIYRwDZvN9bfbYwX8lVJWIAA4BjQEfncsnwC0d9xv53iMY3kjpZRTw8TSgAohhLhnaa3jgaHAEewN5zlgM3BWa51+Dl8cEOq4HwocdTw31bF+AWfqlgZUCCGEa7hhCFcp1VMpFZnp1jNzlUqp/Nh7lSWBECAQ8MiV8kz5ayxCCCEE2K8rAIy5xSqNgYNa6yQApdSfQC0gn1LK6uhlhgHxjvXjgWJAnGPINy9wypls0gMVQgjhGsb8GssRoLpSKsBxLLMRsAtYCTzlWOd57JelBZjjeIxj+Qrt5NVepAEVQghxz9Jab8Q+GWgLsB17uzYGeAd4QykVg/0YZ/rlt8YBBRzlbwDvOlu3DOEKIYRwDYMuvae1HggMvK74AFD1ButeBVzyw67SgAohhHAJuZi8EEIIIbIlPVAhhBCuYfJfT3E16YEKIYQQTpAeqBBCCNfIYT1QaUCFEEK4xu2dt/n/hgzhCiGEEE6QHqgQQgjXkCFc10oz2S/Vmy1PwfcXGx0hw8mRTxgdIYvcvacbHSELP6uP0REyXEtNNjpCFl4WL6MjZJFmsvMRnfqtLGF60gMVQgjhElp6oEIIIYQTclgDKpOIhBBCCCdID1QIIYRrmOzYs7tJD1QIIYRwgvRAhRBCuIYcAxVCCCFEdqQHKoQQwjVyWA9UGlAhhBAuoXXOakBlCFcIIYRwgvRAhRBCuEYOG8KVHqgQQgjhBOmBCiGEcI0c1gOVBlQIIYRL5LSLycsQrhBCCOEE6YEKIYRwDemBms/+fRuI2rKMyIglbFi/wLAcDzxQisiIJRm3Uyf38Gq/Hh7NEBpalAULpxK5eSkRkUvo3fuFjGWvvPI8W6KWExG5hE8/fddtGSZFHuCJX1bz5K+reXduFNdS09Ba893aPbQdu4rHx61iyuaDAJy/msLrf0Xy9K9r6DxpHTFJF9yW63rNmtZn54417Nm1jgFv9/FYvel8fX1YtWYW6zcsICJyMR98+FqW5V8PHUjiiR0ez5Uub948TJs2hu3bV7Nt2yqqV6vssbrDwoqyePE0oqKWs2XLMvr0eRGAJ55oxZYty7h8+RCVKj3msTxmf6/6v/oS0dEriIpazqRJP+Dr62tYFvGve6YH2rjJ05w6dcbQDPv2xRJepSkAFouFw4c2M2v2Qo9mSEtL5f33PiU6eie5cgWy7u+5rFixlkKFCtK6dROqV2tBcnIyBQsWcEv9xy9cZeqWQ/z5Qj38vL14e84WFu1JQGs4fv4qs7rXw6IUpy9dA2DshhjKFsrD8MfDOXjqIv9btoMxHau7JVtmFouFkSM+o3nLZ4mLO8aG9QuYO28Ju3fvd3vd6a5dS6ZVi05cunQZq9XK0uUzWbJ4FRER0VSs9Cj58uX1WJYbGf7NYJYsXskzz/TE29ubgAB/j9WdmprGO+98SnT0DnLlCmT9+vksX76WnTv30rFjT3744X8eywLmfq9CQorQp8+LPFa+AVevXmXKlJ/o2KEdEyfNMCzTTeWsH2O5N3qgZtSwYW0OHDjMkSPxHq03MTGJ6OidAFy8eIm9e2MJCSlCj5c6M2zYKJKTkwFISjrltgxpNs211DRSbTaupqRRMNCPmdGH6VmzDBalAAgKtH9DPnDqAlWLBwNQskAuEs5d4ZSjcXWnqlUqEht7iIMHj5CSksKMGbNp26aZ2+u93qVLlwHw9rbi7W1FY2/cP/vsPT780LONRGZ58uSmdu1q/PLrVABSUlI4d+68x+pPTDxBdLS9R3fx4iX27IkhNLQIe/fGsH//AY/lyMys7xWA1WrF398PLy8vAvz9STiWaGiem9E27fKbmd0TDajWmoULprJxw0J6dO9sdBwAOnZox/TpswzNcN99YZQv/zAREdGUKXM/NWtVZdXqWSxaPJ1Kld0z/FU4tx/PVbmf5qNX0OTH5eTytVKzZEHizl5m8Z4EOk1cR5/fN3H4zCUAHiiYh+X77H/s24+d5dj5Kxy/cNUt2TILCS3C0biEjMdx8ccICSni9nqvZ7FY+GfDfA4ejmTF8nVERkTzyivPMX/+Mo4nJnk8T7qSJe/j5MlTjBs7nIhNixn909ce7YFmVrx4GBUqlGPTpihD6k9n1vcqISGR4cN/4kDsJo4eieL8+fMsW7bGsDziX9k2oEqpqkqpKo77Dyul3lBKtXR/tH/Vb/A4Vas1p3WbLvTq1Y3atat5svr/8Pb2pnXrpvz+xzzDMgQGBjBl6igGDBjMhQsXsXp5kT9/XurXa88HH3zOpEk/uKXe81dTWBVznPk9G7CkVyOupKQxf2ccyWk2fK1eTHmuNk88VoxPFm4F4MVqpbhwLYUO49cybcshyhbOg0W5JZop2Ww2alZvRdkyNQgPL0+tWlVp/0RLfho1wdBcVi8vKlZ8lNGjJ1KlajMuXbrMgAF9PZ4jMDCAqVNH89Zbg7hw4aLH68/MrO9Vvnx5adOmGWUeqM59xSsREBhAp05PGJrppmza9TcTu2UDqpQaCIwERiml/gd8DwQC7yqlPrjF83oqpSKVUpE226W7DpmQYO/BJCWdYtbshVSpUuGuX/NuNG/egKio7Zw4cdKQ+q1WK1Om/MT0abOYM3sxAPEJiRn3N0duxWazERwc5PK6Nxw+SWhef4ICfPH2stCoTBGiE85QOLcfjcrYe3gNyxRhv2OyUC5fbwa3KM+MbnX4tGV5zlxOJixfgMtzXS8hPpFiYSEZj8NCi2bsR0Y4d+4Ca9asp2696pQqVYJtO1axc/daAgL82bp9pcfzxMUfIy7uGJsi7L2+P/6cT8UKj3o0g9VqZdq00Uyb9hezZy/yaN23Yrb3qlGjOhw6dISTJ0+TmprKrFkLqVE93OM5xH9l1wN9CqgF1AX6AO211kOAZkDHmz1Jaz1Gax2utQ63WALvKmBAgD+5cgVm3G/SuB47d+69q9e8Wx07tjd0+HbUqC/ZuzeG774bl1E2d+4S6tazT84pXbokPj7enDx52uV1F83tx7aEs1xJsc+83XjkJPcXyEWD0kWIOGo/7hp59DT3Bdnfs/NXU0hJs88s+HPbUSqHBZHL19vlua4XERlN6dIlKVGiGN7e3nTo0I6585a4vd7MgoODyJs3NwB+fr40bFiHqKgdlCpZlXIP1aHcQ3W4fPkK5R9t4NFcAMePJxEXl8ADD5QC7Mf0d+/e59EMo0d/zZ49MYwcOdaj9d6Imd+ro0fiqVqtEv7+fgA0bFCbPXs8NxnujtjccDOx7Gbhpmqt04DLSqlYrfV5AK31FaWUR/5phQsX5PeZ9obCy+rFtGmzWLJklSeqvqGAAH8aN6pL797vGFJ/jRrhdOr8JDu272b9BvspPZ8M/IqJE2bw009fERGxmOSUFHq+9KZb6n80JD+NHyjKsxPX4mVRPFgoL08+dh9XU228Pz+KyZEHCfD2YmAz+zHYg6cu8tHCrSigVHAuPmle3i25rpeWlkb/1z5kwfwpeFksjJ8wnV27PNtAFC5SiDE/D8XL4oXFovjzz/ksWrjCoxlu5bXXP2LihO/w8fHmwMEj9OjxhsfqrlmzCp07P8n27bvZuNE+k/3jj7/C19eHb74ZTMGCQfz1169s27aLNm26uj2Pmd+rTRFR/PnnfDZtWkxqaipbo3fy89jfjI4lAHWr329TSm0EGmitLyulLFprm6M8L7BSa10puwq8fULNPYhtMB+r+3tjt+vkSHMdV8nde7rREbLws/oYHSHDtdRkoyNk4WXxMjpCFlaT5THT+5WSHO+2WQhnnq7v8s/7/DNXmXbWRHY90Lpa62sA6Y2ngzfwvNtSCSGEuPeYfMjV1W7ZgKY3njcoPwkYM4NGCCGEMIF75kpEQgghzM3sFz5wtXviQgpCCCGE2UgPVAghhGvIMVAhhBDizukc1oDKEK4QQgjhBOmBCiGEcA3pgQohhBAiO9IDFUII4RI57RioNKBCCCFcI4c1oDKEK4QQQjhBeqBCCCFcIqcN4UoPVAghhHCC9ECFEEK4RE7rgUoDKoQQwiVyWgMqQ7hCCCGEE6QHKoQQwjW0MjqBR7m9ATXbr8OZ7e29lppidIQMuXtPNzpCFue/aGl0hCzyvLvA6AgZvCzmGjxKtaUZHSELm8nGEkvkLWJ0BOEG0gMVQgjhEib73uJ25voaK4QQQtwjpAcqhBDCJbTNbAfJ3EsaUCGEEC4hQ7hCCCGEyJb0QIUQQriEzmGnsUgPVAghhHCC9ECFEEK4RE47BioNqBBCCJfIabNwZQhXCCGEcIL0QIUQQriENtu1W91MeqBCCCGEE6QHKoQQwiVy2jFQaUCFEEK4RE5rQGUIVwghhHCC9ECFEEK4hEwiMhlfX1/W/z2PzZFL2Rq9goEfv2lYlgceKEVkxJKM26mTe3i1Xw/D8php26Rr1rQ+O3esYc+udQx4u49H6rRWaozfc4Pwe+4TfFq+BF5WfJo+j1/Xj/HrOhCf1q+At2/G+l4PhOP3/CD8nhuET0vPvX9GbJvMRo8eytEjUWzZvCxLee9e3di2dSVRW5bx+WfvezwXwM9jhpEQt5XoqOWG1A8wZvRQ4o5GE7Xl3+3z5BOtiI5aztUrR6hU6TG31v+/ER+zYddS5q/594ft8+bLw/iZP7B041+Mn/kDefLmBqBqzcpsiV3NnJVTmLNyCn3ffMmt2cSNmb4BvXbtGo2bdqByeBMqhzelWdP6VKtayZAs+/bFEl6lKeFVmlK1WnMuX77CrNkLDckC5to2ABaLhZEjPqN1my48Wr4BHTu256GHyri1TpUrH9aKjbg65VOuTvwElAWvslVJXjWdq5MGc3XSIPSF01grNLCvn68Q3lVbcHXal1ydOJDkldNv+fquYsS2ud6kSTNp07ZrlrJ69WrQpk1Twqs0o2Klxgz/drRHM6WbOHEGrVp3NqTujAyTZtK6TZcsZTt37aVDx5dYu3aj2+v/c9pcXnymX5ayl1/txj9rI2hS7XH+WRvBy692y1gWuSGKtg060bZBJ74f9rPb890ObVMuv5mZ6RtQgEuXLgPg7W3F6u2NNsE4QcOGtTlw4DBHjsQbmsNM26ZqlYrExh7i4MEjpKSkMGPGbNq2aeb+ii0WsHqDsqC8fdCXzkLy1X+XW73/vftoHVKiV8I1+3bjygX358PAbZPJunUbOXPmbJayni915euhP5KcnAxAUtIpj2ZKt3bdRk5fl83TbrR99uyJYd++Ax6pP2J9FOfOnMtS1qhFPf6aPg+Av6bPo3HL+h7Jcq9RSuVTSv2ulNqjlNqtlKqhlApSSi1VSu13/D+/Y12llBqplIpRSm1TSjnd67jjBlQpNdHZypxlsViIjFjCsfhtLF++hk0RUZ6O8B8dO7Rj+vRZRscw1bYJCS3C0biEjMdx8ccICSni1jr1xbOkRi7Bv8eX+L88FH3tCrbDuwDwadoN/5eHYQkqQmrUCgBU/sJY8hfGt+M7+D77HpYS5dyaL50R2+Z2lClzP7VqVWXtmjksXTqTypXLGx1JZBJcsABJx08CkHT8JMEFC2QsqxD+KHNWTmXstJGULnu/URGz0Fq5/HabRgCLtNYPAuWB3cC7wHKtdRlgueMxQAugjOPWExjl7L/3lg2oUmrOdbe5wBPpj52t9E7ZbDbCqzSleMlwqoRXpFy5sp6q+oa8vb1p3bopv/8xz9AcYL5t43G+AXiVqsCVce9xZczb4O2D10PVAEheMp4rY97CduoYXmXDAVAWL1T+wlybOZTk+T/j0+Q58PU38l9gKKvVSlD+fNSp25b33vuMKb/9aHQkcQvpI0y7tu2hfqXWtG3wLJPGTmfUxGEGJ7PTNtffsqOUygvUBcYBaK2TtdZngXbABMdqE4D2jvvtgInabgOQTylV1Jl/b3Y90DDgPPANMMxxu5Dp/g0ppXoqpSKVUpE22yVnct3QuXPnWbX6b5o1re+y13RG8+YNiIrazokTJw3NkZkZtk1CfCLFwkIyHoeFFiUhIdGtdXrd9xD6/Em4chFsaaTtj8JStNS/K2hN2t4IrGUqA2C7cIa02GiwpaHPn0SfOY4lX2G3ZgRjts3tiI8/lnEcPzIyGptNExwcZHAqke5k0ikKFg4GoGDhYE6dPA3AxYuXuHzpCgCrl/2N1Wolf1A+o2IarSSQBPyqlIpSSo1VSgUChbXWxxzrJALpf+ihwNFMz49zlN2x7BrQcGAz8AFwTmu9CriitV6ttV59sydprcdorcO11uEWS6AzuTIEBweRN28eAPz8/GjcqC5798be1WverY4d25ti+NZs2yYiMprSpUtSokQxvL296dChHXPnLXFrnfrCaSxF7gerDwBe9z2IPp2IylcwYx2vUuWxnbb/HaXFRuEV5uil++VC5S+M7VySWzOCMdvmdsyZs5h69WoCUKZ0Sbx9vDnp+JAWxluxaA2Pd2wNwOMdW7N8of1jN7jQv0O5j1Ush8Vi4czps0ZEzMKmlctvmTtkjlvP66q1ApWAUVrrisAl/h2uBUDbu+4unyByy/NAtdY2YLhSaqbj/8eze46rFS1amF/GfYuXlwWLxcLvv89l/oJl2T/RTQIC/GncqC69e79jWIZ0Zts2aWlp9H/tQxbMn4KXxcL4CdPZtWufW+u0JR4kbf9m/Lp8CDYbthNHSN2+Bt+n3kT5+gEKW1Icycsn29c/tBNdvBx+zw8CbSNlze9w1XWjJDdjxLa53sSJ31O3TnWCg4OIjdnEkE+HMX7CdMaMGcqWzctITk6mR4/XPZop3eRJP1Cvbg2Cg4M4dCCSQYOH8uv4aR7NMGni99R1ZDgQG8HgIcM4c/osw4cPoWDBIGbPmsDWbTtp3bpL9i/mhOGjP6NqrXDyB+Vj7dYFjPhqNKNHjmfE2C94unM74o8eo38Pe7vQvE0jOnV7itTUNK5dvcZrPd9zSyYz0FqPAcbcYpU4IE5rnT5V+nfsDehxpVRRrfUxxxDtCcfyeKBYpueHOcrumLqTWZtKqVZALa31bZ8sZvUJNX7KbCZmmxRtqo1jMue/aGl0hCzyvLvA6AgZvCzmmkCfZjPXLylblLn+0ovncf9hgtu1P2mz2zbO3gdbuPwjreyehdnmVUqtBXporfcqpT4B0oc+T2mtv1BKvQsEaa0HONqxvkBLoBowUmtd1Zlsd9Sb1FrPB+Y7U5EQQoj/3ww8b7Mf8JtSygc4ALyA/RDlDKVUd+Aw0MGx7gLsjWcMcNmxrlPkUn5CCCHuaVrraOxzdq7X6AbrasAllwKTBlQIIYRLmOAaNx5lrgMpQgghxD1CeqBCCCFcwuzXrnU1aUCFEEK4hO32L733/4IM4QohhBBOkB6oEEIIl7iDi7//vyA9UCGEEMIJ0gMVQgjhEnIaixBCCCGyJT1QIYQQLpHTZuFKAyqEEMIlZBKREEIIIbIlPVAhhBAuIZOIhBBCCJEt6YEKIYRwCZlE5GK+Vm93V3FHUtJSjY6QhZl2tzL5Qo2OkEX+9xcZHSGLqNBKRkfIUOP4DqMjXMdcf1f5fAONjpBF3MWTRkfwCJlEJIQQQohsyRCuEEIIl8hpQ7jSAxVCCCGcID1QIYQQLpHDzmKRBlQIIYRryBCuEEIIIbIlPVAhhBAuIaexCCGEECJb0gMVQgjhEjajA3iY9ECFEEIIJ0gPVAghhEtoU12c1P2kARVCCOESthx2IqgM4QohhBBOkB6oEEIIl7DlsCFc6YEKIYQQTpAeqBBCCJfIaZOITNcDDQ0tyoKFU4ncvJSIyCX07v0CAI8++hArVv7Jpk2LmPn7WHLnzmVIvr59uxO1ZRnRUcvp16+7x+sfM3oocUejidqyLKPsySdaER21nKtXjlCp0mNuzzDk2w9Zs3Mhs1ZPySh78+N+zF03nT9XTmbEr1+SO0/W96doaGEiDqykW6/Obs02evRQjh6JYsvmf7fP5Ek/smnjIjZtXMTevf+waaN7f6i77NqxlFn4HaXnj6DU7G8AKPbdAErPH0Hp+SMou3YspeePACBX7QqUnjPcvv6c4QTWcN/75+vrw6o1s1i/YQERkYv54MPXsiz/euhAEk947oe6b/RepXutf0+uXT1KgQL5PZYnT97cjJ04gnURC1i7aT7hVSqQL39eZswax/oti5gxaxx58+XxSJawsKIsWjSNLVuWsXnzUvr0sX8Ofv75+0RHL2fTpkVMnz6avHk9k+d22dxwMzPTNaBpaam8/96nhFduQoP6j9Pz5a48+GBpfvjxCz7+6EuqVm3O3DmLee31nh7PVu7hsnR/8Vlq1mpN5fCmtGzZmFKlSng0w8RJM2ndpkuWsp279tKh40usXbvRIxlmTZvHy8+8lqVs/epNtK/XiScadOFw7BFeevX5LMsHDHqNtcvXuz3bpEkzadO2a5ayLl17U7Vac6pWa86svxYya/ZCt+c40OkDYlr1J7bdGwAc7fcVMa36E9OqP+cW/cP5RfZtkXr6PId6DGF/i34cfWs4xb55w22Zrl1LplWLTtSo3pIa1VvRuEk9qlSpAEDFSo+SL19et9V9Izd6r8DeeDRuXJfDR+I8mufTLz5g5bK11K7Skoa12rNvXyz9Xn+Jtas3UKNSc9au3kC/11/ySJbU1DTeffdTKlVqTL167Xn55ed48MEyLF++lsqVm1K1anP27z/I22/39kgecWOma0ATE5OIjt4JwMWLl9i7N5aQkCKULl2SdevsDcTy5eto166Fx7M9+GBpNm2K5sqVq6SlpbF2zQbat/dsjnXrNnLmzNksZXv2xLBv3wGPZdi8IZpzZ89nKftn9UbS0tIA2Lp5B4VDCmUsa9iiLnFHEojZ6/6MN9o+mT35VGtmTJ/t9hy3krdlbc7OXQ3A1V0HSD1xGoBr+46g/HxQPu47snLp0mUAvL2teHtb0YDFYuGzz97jww//57Z6b+Rm79XXXw3kvfc/Q2vPnRORO08uatQK57eJvwOQkpLC+XMXaN6yEdOnzAJg+pRZtGjV2CN5EhNPEB1tHw24ePESe/bEEBJSmOXL12b8nW3aFEVoaFGP5LldGuXym5mZrgHN7L77wihf/mEiIqLZvXs/rds0BeCJJ1oSFub5HWfnrr3Url2VoKB8+Pv70bx5Q8LCQjyew+ye6NQmo7cZEOBP977PMWroWINTQe3a1Thx/CQxsYfcW5GGkhMHU3rOcPI/2yzLooCq5Ug9eZbkQ8f+87Q8LWpydUcsOjnVbdEsFgv/bJjPwcORrFi+jsiIaF555Tnmz1/G8cQkt9V7u9q0bkpCQiLbt+/2aL33FQ/j1MnTjPjxfyxb+yfffDeEgAB/ChYswInj9u1y4ngSBQsW8GgusH8OVqhQjoiI6Czlzz3XgcWLV3k8j/jXHTWgSqnaSqk3lFJN3RUoXWBgAFOmjmLAgMFcuHCRXq8MoOdLXVj391xy5c5FcnKKuyP8x549MXw99EcWzJ/CvLmT2bptZ8a3QWHX87VupKamMe8P+3HG3m+/xMTRU7l8+YrByaBjh3bMmOH+3mfs0wOIafMaB1/4hAJdWxFQtVzGsnxt6nJu7pr/PMe3zH0Ueacb8R/84NZsNpuNmtVbUbZMDcLDy1OrVlXaP9GSn0ZNcGu9t8Pf348BA/oyaPAwj9dttVp5tPzDTBg3lcZ1nuDypSs3HK7VHv7J6MDAAKZO/Ym337Z/DqYbMKAvaWmpTJv2l0fzZCenHQO95ViRUmqT1rqq4/5LQB/gL2CgUqqS1vqLmzyvJ9ATwMc7CKs1952FslqZMuUnpk+bxZzZiwHYty+Wtm2fA6B06ZI0b97gjl7TVcaPn8b48dMAGDL4HeLi/9uTyKnad2xFvSa16f5Un4yyxyqVo2nrBrz5UV9y582NttlIvnaNKb/87tFsXl5etGvXnBo1W7q9rtTj9iHZtFPnOL94PQHlH+Dypp3gZSFP8xrEtHk9y/rWIgUoPvp94t4cTvKRRLfnAzh37gJr1qynbr3qlCpVgm07VgH2EYOt21dS/lHP/33df38JSpQoRkSE/W8+LLQoGzYspHbtNhw/7t7ecUJ8Ignxx9myeRsAc2cvpt/rL5GUdIpChQty4ngShQoX5GTSabfmyMxqtTJ16k9Mnz6L2bP/nfjWpctTtGzZiBYtnvVYlttl9gbP1bI72OKd6X5PoInWOkkpNRTYANywAdVajwHGAAQGlLjjr2yjRn3J3r0xfPfduIyyggULkJR0CqUU77zTl3Fjf7vTl3WJ9BzFioXQvn0Latdpa0gOs6ndoDov9unK84+/wtUr1zLKn2v3csb93m/14PKlKx5vPAEaNazD3n2xxMe7t4FS/r4oiwXbpSsof19y1anIiZH2L1y5alXgWmw8qYmnMta35A6kxC8DSfxyApc3u3fYMjg4iJSUFM6du4Cfny8NG9bhm29+olTJqhnrJJ7YYUjjCbBz5x6K3Vcx4/Hevf9Qs2YrTp064/a6k06cJCH+GKVKlyQ25iB16tVg395Y9u2NpWOn9nw3/Gc6dmrPogXL3Z4l3U8/fcXevTGMHPnv4Y8mTerxxhuv0LRpB65cueqxLOLGsmtALUqp/NiHepXWOglAa31JKeWWAzU1aoTTqfOT7Ni+m/UbFgDwycCvKFWqJD1fts/YmzN7MRMnznRH9dmaPm0MBQrkJyUllVf7f8C5c+ezf5ILTZr4PXXr1iA4OIgDsREMHjKMM6fPMnz4EAoWDGL2rAls3baT1q27ZP9iTvr6pyFUqVmJfEH5WB41lx++HsNLrz6Pt48PY2d8B9gnEg0e8KXbMtzMxInfU7dOdYKDg4iN2cSQT4cxfvx0nu7Q1iOTh6zB+Sg++gMAlJcXZ+es5uKaLYBj+HbO6izrF3i+Fb7Fi1Lo1Wco9OozABx87mPSTp1zebbCRQox5ueheFm8sFgUf/45n0ULV7i8ntt1s/fKKO8P+JQfx36Nj7c3hw8dpX+f97EoCz9PGE6nrk8SdzSBl7q9nv0LuUDNmuF07vwk27fvZoPjc3DgwK8ZNuwTfH19mDdvMmCfSPTqqx94JNPtMPukH1dTt5rpppQ6hL1XrgAN1NJaH1NK5QLWaa0rZFeBMz1Qd0pJc98EjXtdmXyhRkfIIuZcgtERsogsWsHoCBlqHPfc+Zq3I8Vmrr+rfL6BRkfI4nyy8XMA0l25cthtrdz8ws+6/PO+1fGppm2Vb9kD1VqXuMkiG/C4y9MIIYS4Z9lM29S5h1MnnGmtLwMHXZxFCCGEuGfItXCFEEK4RE77NRZpQIUQQriEqSa8eICpr0QkhBBCmJX0QIUQQrhETruQgvRAhRBCCCdID1QIIYRL2JRMIhJCCCHumEwiEkIIIUS2pAcqhBDCJWQSkRBCCCGyJT1QIYQQLiHXwhVCCCGckNMu5SdDuEIIIYQTpAcqhBDCJeQ0FiGEEEJkS3qgQgghXEImEbmYTZurU69NlsdiMc8gwN4zcUZHyMLLRNsGIPxYtNERMpwZ+ZTREbLI03eG0RGyOHP1otERsjDb56BwDemBCiGEcImcdiEFaUCFEEK4RE7rZ5trjEwIIYS4R0gPVAghhEvktElE0gMVQgghnCA9UCGEEC4hk4iEEEIIJ+S0BlSGcIUQQggnSA9UCCGES2iZRCSEEEKI7EgDKoQQwiVsbrjdLqWUl1IqSik1z/G4pFJqo1IqRik1XSnl4yj3dTyOcSwv4ey/VxpQIYQQLmFkAwr0B3ZnevwlMFxrXRo4A3R3lHcHzjjKhzvWc4o0oEIIIe5pSqkwoBUw1vFYAQ2B3x2rTADaO+63czzGsbyRY/07Jg2oEEIIl9BuuN2mb4EB/NtpLQCc1VqnOh7HAaGO+6HAUQDH8nOO9e+YNKBCCCFMSynVUykVmenW87rlrYETWuvNns4mp7EIIYRwCXdcC1drPQYYc4tVagFtlVItAT8gDzACyKeUsjp6mWFAvGP9eKAYEKeUsgJ5gVPOZDNdDzQsrCiLFk1jy5ZlbN68lD59XgDggw9eIzZ2Ixs2LGDDhgU0a9bAkHz7920gassyIiOWsGH9Ao/XP3r0UI4eiWLL5mVZynv36sa2rSuJ2rKMzz973+O5AH4eM4yEuK1ERy03pH648faZPOlHNm1cxKaNi9i79x82bVxkWBbw7Hs1efNBnhy/hqfGr+HdeVFcS01Da8336/bS7pdVPPHraqZsOQTAhIgDdJy4lo4T1/LU+DVU/mYB564kuzVfZs2a1mfnjjXs2bWOAW/38Vi96caMHkrc0Wiitvz7fj35RCuio5Zz9coRKlV6zOOZ0hm9bcxMa/2e1jpMa10CeAZYobXuDKwE0n95/nlgtuP+HMdjHMtXaO3cL56brgeamprGu+9+SnT0DnLlCuSff+axfPk6AL77bhzffnurLyKe0bjJ05w6dcaQuidNmsmoUeP5Zdy3GWX16tWgTZumhFdpRnJyMgULOjWcf9cmTpzBjz/+yq+/jjCkfrjx9unStXfG/S+/+Ihz588blsWT79WJC1eZuuUQf3Sri5+3FwPmbmHxnmNoNIkXrvLXC/WwKMXpy9cAeL7K/Txf5X4AVsce57fNh8jr7+O2fJlZLBZGjviM5i2fJS7uGBvWL2DuvCXs3r3fI/UDTJw0kx9HjefXX77NKNu5ay8dOr7ED987PVHzrplh29wuk13K7x1gmlLqUyAKGOcoHwdMUkrFAKexN7pOMV0Dmph4gsTEEwBcvHiJPXtiCAkpbHAq81i3biPFi4dlKev5Ule+Hvojycn23kJSklOjEXdt7Q2yedqNtk9mTz7VmubNOhqWxdPvVZpNcy01DauX4mpqGgVz+fLD3/v4vGUFLI6Jh0EBvv953qI9CTR/sKhbs2VWtUpFYmMPcfDgEQBmzJhN2zbNPNpI3Oj92rMnxmP134wZts3tMroB1VqvAlY57h8Aqt5gnavA066o75ZDuEqpakqpPI77/kqpQUqpuUqpL5VSeV0R4Fbuuy+MChXKERERDcArrzzHpk2L+Omnr8mXL4+7q78hrTULF0xl44aF9Oje2ZAM1ytT5n5q1arK2jVzWLp0JpUrlzc6kinVrl2NE8dPEhN7yLAMnnyvCuX247kqJWnx80qa/LSCXD7e1ChRkLizl1my9xidJq+jzx8RHD5zKcvzrqSk8c+hkzQqU8Rt2a4XElqEo3EJGY/j4o8REuK5+s1Mto15ZXcM9BfgsuP+COwHW790lP3qxlwEBgYwdepPvP32YC5cuMjPP0/m4YfrUq1aCxITT/DFFx+5s/qbqt/gcapWa07rNl3o1asbtWtXMyRHZlarlaD8+ahTty3vvfcZU3770ehIptSxQztmzJid/Ypu5Mn36vzVFFbFnGBej/osebkhV1LSmL8rnuQ0Gz5WC1O61OaJx4oxaPG2LM9bE3ucCiH5PTZ8K/7/MPA0FkNk14BaMp1HE661fk1rvU5rPQi4/2ZPyjztODX14h2HslqtTJ36E9Onz2L2bPuEjxMnTmKz2dBa88svUwkPN6aXlZCQCNiH3mbNXkiVKhUMyZFZfPwxZs1eCEBkZDQ2myY4OMjgVObi5eVFu3bNmfn7HENzePK92nj4JCF5/QkK8MXby0LDMoXZmnCGwrn8MnqXDUsXZn/ShSzPW7z3mEeHbwES4hMpFhaS8TgstGjG31pOJ9vGvLJrQHcopV5w3N+qlAoHUEo9AKTc7Ela6zFa63CtdbjVmuuOQ/3001fs3RvDyJFjM8qKFCmUcb9du2bs2rX3jl/3bgUE+JMrV2DG/SaN67Fzp+dzXG/OnMXUq1cTgDKlS+Lt483Jk6cNTmUujRrWYe++WOLjjf3g8eR7VSSPP9uPneVKin3m7aYjpygZlIv6pQsTccR+7HVz3Gnuyx+Y8ZwL11LYHHea+qU9O+8gIjKa0qVLUqJEMby9venQoR1z5y3xaAazupe2jU25/mZm2U0i6gGMUEp9CJwE1iuljmK/ikMPdwSqWTOczp2fZPv23WzYYD9NZODAr+nQoS2PPfYwWmsOH46jXz/Pn6pRuHBBfp9pn8jlZfVi2rRZLFmyyqMZJk78nrp1qhMcHERszCaGfDqM8ROmM2bMULZsXkZycjI9erzu0UzpJk/6gXp1axAcHMShA5EMGjyUX8dP82iGG26f8dN5ukNbZkz37PCt0e/Vo0Xz0bhMETpNWoeXRfFgoTw8+VgxrqXaeH9BNL9tOYi/t5WPmz6a8ZyV+49TvXgw/t6enV+YlpZG/9c+ZMH8KXhZLIyfMJ1du/Z5NMOkid9T17H/HoiNYPCQYZw5fZbhw4dQsGAQs2dNYOu2nbRu3cWjucywbW6X0ZOIPE3dzukvjolEJbE3uHFa6+O3W4G/f3FTDWOnpqVmv5IHWSzmORU3zWau3d/LRNvGbM6MfCr7lTwoT98ZRkfIwuLcpU3dxubcaYZukZoc77aN80XxLi7/h757eLK53sxMbutrptb6PLDVzVmEEELcw8zzNcEz5Cu+EEII4QTTXUhBCCHEvcmWw/qg0oAKIYRwCXPNonA/GcIVQgghnCA9UCGEEC6RswZwpQcqhBBCOEV6oEIIIVxCjoEKIYQQIlvSAxVCCOESZr92ratJAyqEEMIlctp5oDKEK4QQQjhBeqBCCCFcImf1P6UHKoQQQjhFeqBCCCFcIqedxiINqBBCCJeQSURCCCGEyJbbe6ApaanuruKOBPr4GR0hi2upKUZHMC2tzfVt1maiPHn6zjA6QhYXJvQwOkIWebuNMzpCFmb73HEX8/yFeIb0QIUQQggnyDFQIYQQLiGTiIQQQggnyCQiIYQQQmRLeqBCCCFcImf1P6UHKoQQQjhFeqBCCCFcQiYRCSGEEE7QOWwQV4ZwhRBCCCdID1QIIYRL5LQhXOmBCiGEEE6QHqgQQgiXkAspCCGEECJb0gMVQgjhEjmr/ykNqBBCCBeRIVwhhBBCZOueaEAtFgsRmxYz+68JHq/b19eHFav+ZN36eWyIWMh7H/QHoG69GqxZN5v1mxYyavTXeHl5eSRPWFhRFi+eRlTUcrZsWUafPi8CkD9/XubP/40dO1Yzf/5v5MuX1yN5rtesaX127ljDnl3rGPB2H4/XP2b0UOKORhO1ZVlGWf78+ViwYAo7d65lwYIphmybsLAQli2ZybatK9kavYJ+fbt7PMP1jHivJq3fyxM/LODJHxby7u//cC0ljU0HjvPMT4t58oeFfPjXBlLT7CdDnL+SzOvT1vL0jwvpPGYJMcfPujXbjfad//3vQ7ZvW8XmyKXMnDGWvHnzuDXD9SwWC2v/nsP0mT8D8PO4b4jcspT1mxby/Y9fYLWaaxDR5oabmd0TDeir/XqwZ89+Q+q+di2ZNq26ULtGa2rXaEPjxnWpWq0So0Z/zQvd+lOjaguOHo2nU+cnPJInNTWNd975lIoVG1G3bjteeeU5HnywDG+91YeVK//mkUfqsXLl37z1Vm+P5MnMYrEwcsRntG7ThUfLN6Bjx/Y89FAZj2aYOGkmrdt0yVI24O0+rFzxN+XK1WHlir8NadhTU1N5e8AgHivfgFq129CrVzePb5vMjHivjp+/zNSN+5jSsyl/9GlBmk2zcPthPpq1gS+fqsEffVoQkjeQuVsPAjB27S7KFsnPzN4t+PTx6ny1aItb891o31m+fA0VKjaicngT9u8/wDsD+ro1w/V69e7G3r2xGY9nTJ9DeKUm1KjaAn9/P57v1sGjeURWpm9AQ0OL0rJFI375ZaphGS5dugyAt7cVb28raWlppCQnExtzCICVK9bRtl1zj2RJTDxBdPQOAC5evMSePTGEhhahTZsmTJ78OwCTJ/9O27ZNPZIns6pVKhIbe4iDB4+QkpLCjBmzadummUczrFu3kTNnzmYpa9OmKZMmzwRg0uSZtG3r2Uxgf9+isrxv+wkNKeLxHOmMeq/SbDaupaSRmmbjakoa/j5WvL0sFA+29+yqlyrMsl1xABxIOkfVkoUAKFkwDwlnL3Hq4lW3ZbvRvrNs2RrS0tIA2LhxC6GhRd1W//VCQorQrHkDJk6YkVG2dMmqjPubI7cS4sE8t0O74T8zM30D+s2wQbz73qfYbMZ15i0WC2v/mUvMwU2sXPE3myO34mW1UrHiowC0a9+C0DDP78jFi4dRoUI5Nm2KolChYBITTwD2D+tChYI9nicktAhH4xIyHsfFHyPEwEYinRm2TWbFi4dRofwjbNwUZVgGI96rwnkCeK7mgzQfPpcmw2aTy8+bpuWKkWbT7Iw/DcDSXXEcP2//wvpA4Xws321vTLfHneLY2csZy4zQrVtHFi9e6bH6vvjqQz7+8MsbfvZZrVaeebY9y5au9lie2yFDuJkopV5VShXzVJjrtWrZmBMnTrIlartREQCw2WzUqdmGh8vWolJ4eR56+AFe7Nafz7/8gBWr/uTixUsZ31I9JTAwgKlTR/PWW4O4cOHif5Zrc39xM5Q2cOMEBgYwY/rPvPHWwBu+b/+fnb+SzKo98cx/rTVL3mzHleRUFmw7zBdP1WTo4i10HrOEQB8rFqUAeLH2w1y4mkKHUYuYtmkfZYvmz1jmae++04/U1DSmTP3TI/U1a96ApKRTGaNN1/tm+GD+/juC9f9EeiSPuLHsjkAPAd5VSsUCU4GZWuuk7F5UKdUT6AmgvPJisQQ6Fa5mzXDatG5Ki+YN8fPzJU+e3EwYP5Lnu73q1OvdrXPnLrB2zXoaN67LdyPH0qLpMwA0bFib0qVLeCyH1Wpl2rTRTJv2F7NnLwLgxImTFClSiMTEExQpUoikpJMey5MuIT6RYmEhGY/DQouSkJDo8RzX+++2OWVIDqvVyszpPzN16l/MmrXQkAzpjHivNhxIJDR/IEGBfgA0eiiM6KMnaVW+BL++2BiAf2KOcfjUBQBy+XkzuH01wP6lp+W3cwnLn8utGW+ka9enadmyMc2ad/RYndWrV6ZFy0Y0aVofPz9fcufOxZixw+jZ403eea8fBYKD6N/pA4/luV1mH3J1teyGcA8AYdgb0srALqXUIqXU80qp3Dd7ktZ6jNY6XGsd7mzjCfDBh19Q4v5wSj9Qnc5derNy5d8ebzwLBAeRN6/9n+rn50uDhrXZty+W4IIFAPDx8eG1N17ml3GeO0Y7evTX7NkTw8iRYzPK5s1bSpcuTwHQpctTzJ271GN50kVERlO6dElKlCiGt7c3HTq0Y+68JR7Pcb2585bStcvTAHTt8jRz5xqT6ecxw9i9J4ZvR4wxpP7MjHiviuYNZFvcKa4kp6K1ZuPB49xfMA+nHcc1k1PTGP/3bp4OLw3Ye6wpqfaRnT+3HKBy8ULk8vN2a8brNW1an7fe7MUTT77AlSvuO/56vUGfDOXhsrV5rFw9XuzWnzWr19Ozx5s893wHGjWqS/cX+hs6kiLssuuBaq21DVgCLFFKeQMtgGeBoUBBN+czXJHCBflpzNdYvLywWCz89ed8Fi9ayZBP36VZiwZYlIVxY39jzer1HslTs2YVOnd+ku3bd7Nxo70X8/HHXzF06I/89tsounXryJEj8XTu3MsjeTJLS0uj/2sfsmD+FLwsFsZPmM6uXfs8mmHSxO+pW7cGwcFBHIiNYPCQYXz99fdMmfIT3V54hiNH4ujUyfPbplbNKnTt8hTbtu8iMsLeUH300RcsXLTC41nAmPfq0bACNH64GM+OXoyXxcKDRfPxZOVSfL9iO2v3JWDTmqfDS1P1/sIAHDx5no/+2ohSUKpgXj5pV9Wt+W607wwY0BdfHx8WLrB/Qd64aQt9+77n1hy3MnzEEI4eiWfpCvuEwblzFvPVF98blud6Zj9m6WrqVt9ilFJRWuuKN1kWoLXO9oi+1SfUVF+TAn38jI6QxbXUFKMjZEi1efY4bnaMOt51Mzb5xn9TFyb0MDpCFnm7jTM6Qhb+3r5GR8hw7mKs2/6wuhZ/wuV/JJMO/2muD4JMshvCvemg/+00nkIIIcT/V7ccwtVae3b8TQghxD0rp43RmP48UCGEEMKMzHUhRSGEEPcs+TUWIYQQQmRLeqBCCCFcIqddSEEaUCGEEC6R084DlSFcIYQQwgnSAxVCCOESMolICCGEENmSHqgQQgiXkElEQgghhBNkEpEQQgghsiU9UCGEEC6R036jVHqgQgghhBOkByqEEMIlctppLNKACiGEcAmZRCSEEEKIbLm9B2pRyt1V3JGrqclGRzCt/P65jI6Qxflrl42OkEWgt6/RETKYbT/O222c0RGyOPVyBaMjZFF07E6jI3hETjsPVHqgQgghhBPkGKgQQgiXyGmTiKQHKoQQQjhBGlAhhBAuobV2+S07SqliSqmVSqldSqmdSqn+jvIgpdRSpdR+x//zO8qVUmqkUipGKbVNKVXJ2X+vNKBCCCFcwuaG221IBd7UWj8MVAf6KKUeBt4FlmutywDLHY8BWgBlHLeewChn/73SgAohhLhnaa2Paa23OO5fAHYDoUA7YIJjtQlAe8f9dsBEbbcByKeUKupM3dKACiGEcAnthv+UUj2VUpGZbj1vVr9SqgRQEdgIFNZaH3MsSgQKO+6HAkczPS3OUXbHZBauEEII09JajwHGZLeeUioX8Afwmtb6vMp0DQKttVZKuXyKsDSgQgghXMKo01iUUt7YG8/ftNZ/OoqPK6WKaq2POYZoTzjK44FimZ4e5ii7YzKEK4QQwiUMmoWrgHHAbq31N5kWzQGed9x/Hpidqfw5x2zc6sC5TEO9d0R6oEIIIe5ltYCuwHalVLSj7H3gC2CGUqo7cBjo4Fi2AGgJxACXgRecrVgaUCGEEC5hxBCu1nodcLOLrje6wfoa6OOKumUIVwghhHCC9ECFEEK4RE77NRZpQIUQQriE7TYm/fx/IkO4QgghhBNM34D27dudqC3LiI5aTr9+3Q3JMHr0UI4eiWLL5mVZynv36sa2rSuJ2rKMzz9737Askyf9yKaNi9i0cRF79/7Dpo2LPJIl3eZty1n9zxxWrp3F0lV/ZJT36NmFfyIWsnbDPD4e/LZHsphx+1gsFtb+PYfpM38G4KWXuxK1dQXnLsYSVCC/x3KYaT8GGDN6KHFHo4na8m+e//3vQ7ZvW8XmyKXMnDGWvHnzuK1+VSiUgAEjM265vpqBd/22AHjXbU3AB6MIeO8HfNv+O0nTp8nTBH40hsAPfsLrQaevQZ4tX18fVq2ZxfoNC4iIXMwHH74GQPHiYaxc/Rdbt69kwsTv8Pb2dlsGZ2g33MzM1A1ouYfL0v3FZ6lZqzWVw5vSsmVjSpUq4fEckybNpE3brlnK6tWrQZs2TQmv0oyKlRoz/NvRhmXp0rU3Vas1p2q15sz6ayGzZi/0SJbMHm/9PA3qtKdJ/ScBqFWnGs1bNaJ+rbbUqd6aH0eO80gOM26fXr27sXdvbMbjjes3065NVw4fjvNoDjPtxwATJ82kdZsuWcqWL19DhYqNqBzehP37D/DOgL5uq1+fiOfyV6/ab1+/hk6+RurW9XiVeRTro9W5/GU/Lv+vD8kr7OflW4oUw1qpLpf+15vLowbi16EXKPd8hF67lkyrFp2oUb0lNaq3onGTelSpUoEhn77LD9+No/yjDTh79hzPd+uQ/YsJtzF1A/rgg6XZtCmaK1eukpaWxto1G2jfvoXHc6xbt5EzZ85mKev5Ule+HvojycnJACQlnTIsS2ZPPtWaGdNn33S5p7zQ/VlGDh9DcnIKACdPnvZIvWbbPiEhRWjWvAETJ8zIKNu2bRdHjjh14ZO7Yqb9+GZ5li1bQ1paGgAbN24hNNSpa3zfMa+y5dEnj6HPJOFduyXJS2dCaioA+uI5AKyPVid1yxpITUWfPo4t6RiW4g+4LdOlS5cB8Pa24u1tRWP/wvPXX/YvgL9N/oPWrZu6rX5n2NAuv5nZLRtQpZSPUuo5pVRjx+NOSqnvlVJ9HJdOcqudu/ZSu3ZVgoLy4e/vR/PmDQkLC3F3tbelTJn7qVWrKmvXzGHp0plUrlze6EjUrl2NE8dPEhN7yKP1amDmrHEsW/0HXR3fiEuVKkH1GuEsWj6D2fMnUaHSox7NdCNGbJ8vvvqQjz/8EpvtNn+YycPMuB+n69atI4sXr/RIXd6V6pKyeQ0AloKheJUqR8Abw/B/9X9Y7isDgMpbANuZpIzn2M6exJKvgNsyWSwW/tkwn4OHI1mxfB0HDxzm7LnzGV8w4uMTCQkpnM2rCHfKbhbur451ApRSzwO5gD+xn5xalX8vk+QWe/bE8PXQH1kwfwqXLl1m67adGTuP0axWK0H581GnblvCwysw5bcfKftgLUMzdezQjhkzPN/7bN3sWRKPnSA4OIiZs34lZt8BvKxe5M+fl+aNOlCx0qOMHf8t4Y/955xmj/L09mnWvAFJSaeIjt5B7TrVPFbvnTDjfgzw7jv9SE1NY8rUP7Nf+W55WfF6pCrX5jp++crihQrIzeVv3sRy3wP4v/AOlwb1cH+O69hsNmpWb0XevLmZOm00DzxQyuMZ7pTZe4yull0D+qjW+jGllBX7xXZDtNZpSqnJwNabPcnxczM9Aby88mHxCnQ64Pjx0xg/fhoAQwa/Q1y8U5csdLn4+GMZx9IiI6Ox2TTBwUEeG6q8npeXF+3aNadGzZYerzvxmP0azSdPnmbBvKVUrPwYxxKOM2/uUgCitmzHZrNRoEB+Tp064/F8YMz2qV69Mi1aNqJJ0/r4+fmSO3cuxowdRs8eb3osQ3bMth8DdO36NC1bNqZZ844eqc/6cGVscbHoC2cB0OdOkrrtHwBsR/aB1qhcedDnTmHJXzDjeZZ8waScdf+Q97lzF1izZj1Vq1UiX948eHl5kZaWRmhoERISjru9/jtxO9eu/f8ku2OgFqWUD5AbCADyOsp9gZsO4Wqtx2itw7XW4XfTeAIULGgfIilWLIT27Vswbdqsu3o9V5kzZzH16tUEoEzpknj7eBv6odOoYR327oslPj7Ro/UGBPgTmCsw4379hrXYs2s/C+Yvy+h13V+qBD7e3oY1nmDM9hn0yVAeLlubx8rV48Vu/Vmzer2pGk8w337ctGl93nqzF088+QJXrlz1SJ3WSvUyhm8BUrdtwKvMYwCogiHgZUVfPE/q9o1YK9UFqxUVVBhLwRBsh/e5JVNwcBB58+YGwM/Pl4YN67B3bwxr1mzg8cft80A6d3mS+fOXuqV+cXuy64GOA/YAXsAHwEyl1AGgOjDNzdkAmD5tDAUK5CclJZVX+3/AuXPnPVFtFhMnfk/dOtUJDg4iNmYTQz4dxvgJ0xkzZihbNi8jOTmZHj1eNy7L+Ok83aGtIZOHChYqwPjJPwBgtXrx5+/zWLF8Ld7e3oz44XPWrJ9LSkoKfXu965E8Zts+N/Jyr+fp/9pLFC5ckH82zGfp4lX06+v+00fMtB8DTJr4PXXr1iA4OIgDsREMHjKMAQP64uvjw8IFUwHYuGkLffu+574QPr5YH6zA1enfZxSlbFiKX6f+BLz7A6SlcHXycABsiUdIjVpL4PujIC2NqzNHgXbPse3CRQox5ueheFm8sFgUf/45n0ULV7Bn937GT/yOjwa+ybatu5gwfkb2L+ZBOW0IV2XX5VZKhQBorROUUvmAxsARrfWm26nAxzfMVFs084+siqzy+AYYHSGL89cuGx0hCz+rj9ERMlxNTTY6QhZmG7o79XIFoyNkUXTsTqMjZLh4+aDbPgSrhtRz+Y6wKWG1aT+0s72Un9Y6IdP9s8Dv7gwkhBDi3iTXwhVCCCGcYLaRCHcz9YUUhBBCCLOSHqgQQgiXyGmTiKQHKoQQQjhBeqBCCCFcIqcdA5UGVAghhEvIEK4QQgghsiU9UCGEEC6R084DlR6oEEII4QTpgQohhHAJWw6bRCQ9UCGEEMIJ0gMVQgjhEjntGKg0oEIIIVxChnCFEEIIkS3pgQohhHCJnDaEKz1QIYQQwglu74EqZa4fEzfbtRrNdMzg3NVLRkfIoky+UKMjZLH/bLzRETKYab8ByO3jb3SELILHbDU6Qhan+lcxOoJHmG2/dDcZwhVCCOESMoQrhBBCiGxJD1QIIYRL5LQhXOmBCiGEEE6QHqgQQgiXyGnHQKUBFUII4RJa24yO4FEyhCuEEEI4QXqgQgghXMKWw4ZwpQcqhBBCOEF6oEIIIVzCbFd6czfpgQohhBBOkB6oEEIIl8hpx0ClARVCCOESMoQrhBBCiGxJD1QIIYRLyLVwhRBCCJEtUzago0cP5eiRKLZsXpZRNnDgW0RGLGHTxkXMn/cbRYsW9lieMaOHEnc0mqgt/+bJnz8fCxZMYefOtSxYMIV8+fJ6LM/1LBYLEZsWM/uvCR6v2+htM+TbD1mzcyGzVk/JKHvz437MXTedP1dOZsSvX5I7Ty4AatStyowlE/hr1W/MWDKBarUruy3XjfTt252oLcuIjlpOv37dPVr39cLCQli2ZCbbtq5ka/QK+vU1Jo/FYmH133OYNnNMlvIvvv6Io4me/VHsG33uAPTu1Y1tW1cStWUZn3/2vtvqV8Eh+PcflnELHDQZ79qt8e30ZkZZwDs/4d9/mP0JXlZ8n+6L/2vD8e//DV73l3Nbttul3fCfmZmyAZ00aSZt2nbNUvbNNz8RXqUpVas1Z8GCZXzwfn+P5Zk4aSat23TJUjbg7T6sXPE35crVYeWKvxnwdh+P5bneq/16sGfPfkPqNnrbzJo2j5efeS1L2frVm2hfrxNPNOjC4dgjvPTq8wCcOX2WPl3f5PH6nXn/1UH87/tP3JbreuUeLkv3F5+lZq3WVA5vSsuWjSlVqoTH6r9eamoqbw8YxGPlG1Crdht69erGQw+V8XiOV3p3Y9/emCxlFSo+YsgX0ht97tSrV4M2bZoSXqUZFSs1Zvi3o91Wvz6ZwJURb9pvI99Gp1wjdcdGrk0ZllGeumMDqTs2AOBdtTEAV759natjB+HTqhso5bZ8t0Nr7fKbmZmyAV23biNnzpzNUnbhwsWM+wGBAXhyu94oT5s2TZk0eSYAkybPpG3bZp4LlEloaFFatmjEL79MNaR+o7fN5g3RnDt7PkvZP6s3kpaWBsDWzTsoHFIIgD079pF0/CQAMXsO4Ofni7ePt9uyZfbgg6XZtCmaK1eukpaWxto1G2jfvoVH6r6RxMQTREXvAODixUvs2bOf0JAiHs0QElKEps3rM3HCjIwyi8XC4M/eZeCHX3o0C9x4X+75Ule+HvojycnJACQlnfJIFq/Sj6JPHUefTcpSbn2sJqnR6wBQhYqRFrMdAH3pHPrqJSyhpTyST9iZsgG9mUGDBhATs5Fnn3mcQYOHGpqlUKFgEhNPAPYPo0KFgg3J8c2wQbz73qfYbOb5FQSzbBuAJzq1Ye3y9f8pb9q6Ibu27yUlOcUjOXbu2kvt2lUJCsqHv78fzZs3JCwsxCN1Z6d48TAqlH+EjZuiPFrv5199yMAPv8Rm+/fb8EuvdGXh/OUcP550i2d6Tpky91OrVlXWrpnD0qUzqVy5vEfqtZavTWr02ixllpIPoy+eRZ86BoDt2CGsD1cBiwWVvxBeoaVQ+Yz7WwP7eaCuvplZtg2oUup+pdRbSqkRSqlvlFKvKKXyeCLc9QYO/IrSpasxddpf9OrVzYgIN2XEUEOrlo05ceIkW6K2e7zuO2HUMEzP17qRmprGvD8WZSkvVbYkr3/Uh0FvfeGxLHv2xPD10B9ZMH8K8+ZOZuu2nRm9ZCMFBgYwY/rPvPHWwCyjPO7WrHkDTiadYmv0zoyyIkUK0b59C8b8NNFjObJjtVoJyp+POnXb8t57nzHltx/dX6mXFevDVUjd/k+WYu/ytTN6nwCpkcuxnTuFf7+v8W3zImmH94CJvkjnBLdsQJVSrwI/AX5AFcAXKAZsUErVv8XzeiqlIpVSkWlprv+jnDbtLx5v39Llr3snTpw4SZEi9qHBIkUKeWxoJ7OaNcNp07opMfs28NvkH2nQoBYTxo/0eI7rmWHbtO/YinpNavNO74+zlBcuWoiRv37F+30HcfRwvEczjR8/jeo1WtKo8VOcPXOO/fsPeLT+61mtVmZO/5mpU/9i1qyFHq27WvXKNG/ZiK07VzFu/LfUqVeD9RELKVmqOFu2LWfrzlUEBPizeetyj+a6Xnz8MWbNtm+byMhobDZNcHCQW+v0KluRtPgD6Ivn/i20WPB6pDqp2/7+t8xmI3ner1wZ8SZXJ36B8g/EdjLBrdmyI8dAs3oJaKG1/hRoDJTTWn8ANAeG3+xJWusxWutwrXW4l1culwQtnWnCRZvWTdl73cQDT5s7bylduzwNQNcuTzN37hKPZ/jgwy8ocX84pR+oTucuvVm58m+e7/aqx3Ncz+htU7tBdV7s05W+z73F1SvXMspz58nFqN++YfinPxAVsc2jmQAKFiwAQLFiIbRv34Jp02Z5PENmP48Zxu49MXw7Ykz2K7vY4E+G8kjZ2pQvV5/u3V5j7er1lCxWmQdL1aB8ufqUL1efy5evULl8I49ny2zOnMXUq1cTgDKlS+Lt483Jk6fdWqe1Qh1St67LUuZVujw6KR59LtOXUW8f8Pa1Ly9THtLS0Cfi3JotOzatXX4zs9u5kIIVSMPe+8wFoLU+opRy2+yLiRO/p26d6gQHBxEbs4khnw6jebOGPPBAKWw2G0eOxNG3n/umk19v0sTvqVu3BsHBQRyIjWDwkGF8/fX3TJnyE91eeIYjR+Lo1KmXx/KYidHb5uufhlClZiXyBeVjedRcfvh6DC+9+jzePj6MnfEdYJ9INHjAl3Tq/jTFSobR683u9HrTftrGSx1f5fTJM27Ll9n0aWMoUCA/KSmpvNr/A86dO5/9k9ykVs0qdO3yFNu27yIywv4F56OPvmDhohWGZTLajT53xk+YzpgxQ9myeRnJycn06PG6e0N4+2ItXZ5rf/6UpdhavhYp1x0TVbny4t/9Y9Aa27lTXJ1u/OhTTqNu1UVWSvUHugMbgTrAl1rrX5VSBYE/tNZ1s6vA16+Yqb5CmG1IwEzfsCwGT4G/Xpl8oUZHyGL/Wc8O+d6KmfYbgNw+/kZHyOJy6rXsV/KgU/2rGB0hQ64v/3TbH3r+XKVdvmOeuRhjrg+mTG7ZA9Vaj1BKLQMeAoZprfc4ypOAbBtPIYQQ4v+rbIdwtdY7gZ3ZrSeEECJnM/tpJ652T50HKoQQQpiF/BqLEEIIlzDbHBN3kwZUCCGES5htcpu7yRCuEEII4QTpgQohhHAJs//8mKtJD1QIIYRwgvRAhRBCuEROOwYqDagQQgiXyGmzcGUIVwghhHCC9ECFEEK4hEwiEkIIIUS2pAcqhBDCJeQYqBBCCOEErbXLb7dDKdVcKbVXKRWjlHrXzf/MDNKACiGEuGcppbyAH4AWwMPAs0qphz1RtzSgQgghXEK74XYbqgIxWusDWutkYBrQzkX/pFuSBlQIIcS9LBQ4mulxnKPM7dw+ieja1aPKFa+jlOqptR7jitdyBTPlMVMWkDzZMVMeM2UByXMrZspyM6nJ8S75vM9MKdUT6JmpaIxZtsO91APtmf0qHmWmPGbKApInO2bKY6YsIHluxUxZPEZrPUZrHZ7pdn3jGQ8Uy/Q4zFHmdvdSAyqEEEJcLwIoo5QqqZTyAZ4B5niiYjkPVAghxD1La52qlOoLLAa8gF+01js9Ufe91ICaYsw7EzPlMVMWkDzZMVMeM2UByXMrZspiKlrrBcACT9erctqVI4QQQghXkGOgQgghhBPuiQbUqMs03STLL0qpE0qpHUbmcGQpppRaqZTapZTaqZTqb3AeP6XUJqXUVkeeQUbmcWTyUkpFKaXmmSDLIaXUdqVUtFIq0gR58imlfldK7VFK7VZK1TAwS1nHdkm/nVdKvWZgntcd+/AOpdRUpZSfUVkcefo7suw0cruIrEw/hOu4TNM+oAn2E2QjgGe11rsMylMXuAhM1Fo/YkSGTFmKAkW11luUUrmBzUB7A7eNAgK11heVUt7AOqC/1nqDEXkcmd4AwoE8WuvWRuVwZDkEhGutTxqZI51SagKwVms91jF7MUBrfdbgWOl/8/FANa31YQPqD8W+7z6stb6ilJoBLNBaj/d0FkeeR7BfXacqkAwsAl7RWscYkUf8617ogRp2maYb0VqvAU4bVX9mWutjWustjvsXgN146AocN8mjtdYXHQ+9HTfDvqEppcKAVsBYozKYlVIqL1AXGAegtU42Q+Pp0AiINaLxzMQK+CulrEAAkGBgloeAjVrry1rrVGA18ISBeYTDvdCAGnaZpnuJUqoEUBHYaHAOL6VUNHACWKq1NjLPt8AAwGZghsw0sEQptdlxdRUjlQSSgF8dQ9xjlVKBBmdK9www1ajKtdbxwFDgCHAMOKe1XmJUHmAHUEcpVUApFQC0JOuFA4RB7oUGVGRDKZUL+AN4TWt93sgsWus0rXUF7FcDqeoYfvI4pVRr4ITWerMR9d9Eba11Jey/GtHHcTjAKFagEjBKa10RuAQYOr8AwDGU3BaYaWCG/NhHuUoCIUCgUqqLUXm01ruBL4El2Idvo4E0o/KIf90LDahhl2m6FziONf4B/Ka1/tPoPOkcw4ErgeYGRagFtHUcd5wGNFRKTTYoC5DRs0FrfQL4C/vhCaPEAXGZRgh+x96gGq0FsEVrfdzADI2Bg1rrJK11CvAnUNPAPGitx2mtK2ut6wJnsM8LEQa7FxpQwy7TZHaOSTvjgN1a629MkKegUiqf474/9olfe4zIorV+T2sdprUugX2fWaG1NqwXoZQKdEz0wjFU2hT70JwhtNaJwFGlVFlHUSPAkMln13kWA4dvHY4A1ZVSAY6/sUbY5xcYRilVyPH/+7Af/5xiZB5hZ/orERl5maYbUUpNBeoDwUqpOGCg1nqcQXFqAV2B7Y7jjgDvO67KYYSiwATHLEoLMENrbfjpIyZRGPjL/nmMFZiitV5kbCT6Ab85vpgeAF4wMozji0UT4GUjc2itNyqlfge2AKlAFMZfBegPpVQBIAXoY6IJXzma6U9jEUIIIczoXhjCFUIIIUxHGlAhhBDCCdKACiGEEE6QBlQIIYRwgjSgQgghhBOkARVCCCGcIA2oEEII4QRpQIUQQggn/B8S4m5kcKSWBAAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}},{"output_type":"stream","name":"stdout","text":["\n","\n","Test Accuracy: 84.63 %\n","Test Precision: 84.50 %\n","Test Recall: 84.29 %\n","Test F1-score: 84.21 %\n"]}]}],"metadata":{"colab":{"provenance":[]},"kernelspec":{"display_name":"Python 3.9.13 ('base')","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.13"},"vscode":{"interpreter":{"hash":"304d98a18a6597b5074573a35a99b631fbbf66bbb48b57fc984d20c778d1912e"}},"widgets":{"application/vnd.jupyter.widget-state+json":{"d413d51bccd44eaeb62a5886c1779388":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_45b25e785f5f47e3a00721b884818faa","IPY_MODEL_e49a0fdc81484f6085c97dc7d2673de3","IPY_MODEL_e7f61788756c4cf793827cce1e5a9757"],"layout":"IPY_MODEL_03c1adf12a7d429d8c917f29bf708236"}},"45b25e785f5f47e3a00721b884818faa":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9a9cf2cfd7f24cf78fd610db69200038","placeholder":"","style":"IPY_MODEL_663e6dc5bd204e35856e6ec5f8511585","value":"100%"}},"e49a0fdc81484f6085c97dc7d2673de3":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_b2b41eb2871749309e6409467e7728c7","max":9912422,"min":0,"orientation":"horizontal","style":"IPY_MODEL_74e79bb356bb424bbadd404df5393202","value":9912422}},"e7f61788756c4cf793827cce1e5a9757":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c9c5e9c1bd48407ca1e0593c247edd72","placeholder":"","style":"IPY_MODEL_566a0aec3a9941f5aaf2d10faada0480","value":" 9912422/9912422 [00:00<00:00, 22527122.44it/s]"}},"03c1adf12a7d429d8c917f29bf708236":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9a9cf2cfd7f24cf78fd610db69200038":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"663e6dc5bd204e35856e6ec5f8511585":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"b2b41eb2871749309e6409467e7728c7":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"74e79bb356bb424bbadd404df5393202":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c9c5e9c1bd48407ca1e0593c247edd72":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"566a0aec3a9941f5aaf2d10faada0480":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3b9a38876a944ac2bf9a9d48f96532d5":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_d5e088bc30c848bbb6ee0dd83a7788f8","IPY_MODEL_0bce3184553c427db39f9ef313477306","IPY_MODEL_fe79558b71744a789dbd8ecef771b40c"],"layout":"IPY_MODEL_0e974bded8f44afab292eace71c1a0e2"}},"d5e088bc30c848bbb6ee0dd83a7788f8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8caef2ea7007438f990a912915eee696","placeholder":"","style":"IPY_MODEL_7cfe9445e7524d72b4c7c4df9f4c515d","value":"100%"}},"0bce3184553c427db39f9ef313477306":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_0aedc2afd99b46639cddea273f12fa67","max":28881,"min":0,"orientation":"horizontal","style":"IPY_MODEL_93625f34347d4610a04dce294d3b4f1e","value":28881}},"fe79558b71744a789dbd8ecef771b40c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_932f11b1d0014e23b8b13993aa161b23","placeholder":"","style":"IPY_MODEL_b2242d35002643d2b298ac5a714f0219","value":" 28881/28881 [00:00<00:00, 1620501.04it/s]"}},"0e974bded8f44afab292eace71c1a0e2":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8caef2ea7007438f990a912915eee696":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7cfe9445e7524d72b4c7c4df9f4c515d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"0aedc2afd99b46639cddea273f12fa67":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"93625f34347d4610a04dce294d3b4f1e":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"932f11b1d0014e23b8b13993aa161b23":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b2242d35002643d2b298ac5a714f0219":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1fd97732ca884679a71b142c3d9f6780":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_322dee672a0f4c45979ed1f16d0e5bed","IPY_MODEL_c0f8ff6470654b1cbac1c8d7e9b702ff","IPY_MODEL_ca9a89ff6b824024895846f6e77ab9b8"],"layout":"IPY_MODEL_3ea06647a49a417a80088e8ed9a46404"}},"322dee672a0f4c45979ed1f16d0e5bed":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c63e32998f2a4349a2e8f3967e7ace75","placeholder":"","style":"IPY_MODEL_29ed5f7b443446f0958e3f62ed689d3d","value":"100%"}},"c0f8ff6470654b1cbac1c8d7e9b702ff":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_79a5ba729a5e406a8b4ead0a75668561","max":1648877,"min":0,"orientation":"horizontal","style":"IPY_MODEL_034507653f3f4cac9f419adfff976e0b","value":1648877}},"ca9a89ff6b824024895846f6e77ab9b8":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_86cf3a19466d4278829cba5d3e3ee59e","placeholder":"","style":"IPY_MODEL_1577d0e00ac64d9c9f25bfd710915c89","value":" 1648877/1648877 [00:00<00:00, 14354516.92it/s]"}},"3ea06647a49a417a80088e8ed9a46404":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c63e32998f2a4349a2e8f3967e7ace75":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"29ed5f7b443446f0958e3f62ed689d3d":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"79a5ba729a5e406a8b4ead0a75668561":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"034507653f3f4cac9f419adfff976e0b":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"86cf3a19466d4278829cba5d3e3ee59e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1577d0e00ac64d9c9f25bfd710915c89":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6c461b09af05473aa2cb2e116963d1af":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_010e7be4778b41a78a53fb101d06f272","IPY_MODEL_e2a3ed2047904419a75b9c94dffd236c","IPY_MODEL_8a8e8c276f774c0cb306b0a4fcf6f62d"],"layout":"IPY_MODEL_3b1b11b7be0242df957021058280cf52"}},"010e7be4778b41a78a53fb101d06f272":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_521694b3e81f41449b468cf7fb952573","placeholder":"","style":"IPY_MODEL_bcc8347c4b4049a0a3ca8f279684d7e2","value":"100%"}},"e2a3ed2047904419a75b9c94dffd236c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a5e698b7c50a48aeab26df687f6fb7da","max":4542,"min":0,"orientation":"horizontal","style":"IPY_MODEL_e4281dc403874993bbc196c91be3ec76","value":4542}},"8a8e8c276f774c0cb306b0a4fcf6f62d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_96c99b3110a6423081f6b061db82b54b","placeholder":"","style":"IPY_MODEL_88b8adaf05a24940bf8711f49f14fc52","value":" 4542/4542 [00:00<00:00, 183190.49it/s]"}},"3b1b11b7be0242df957021058280cf52":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"521694b3e81f41449b468cf7fb952573":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"bcc8347c4b4049a0a3ca8f279684d7e2":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a5e698b7c50a48aeab26df687f6fb7da":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e4281dc403874993bbc196c91be3ec76":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"96c99b3110a6423081f6b061db82b54b":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"88b8adaf05a24940bf8711f49f14fc52":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":0} |