logistic_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d170e3ec",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/david-adewoyin/machine_learning_from_scatch/blob/main/logistic_regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "747b92cc",
   "metadata": {},
   "source": [
    "## Logistic Regression\n",
    "### Logistic Regression in Plain Python\n",
    "In *Logistic Regression* we try to predict the outcome of a **binary** variable from one or more independent variables. Logistic Regression is an example of a binary classification problem where for example we might try to predict whether an email is a *spam/not spam* or whether a student will *pass/ or fail* an exam from one or more independent variables such as hours spent reading and how much class they attended.\n",
    "\n",
    "### Notation and Definitions:  \n",
    "- $y:   $    target value $\\in \\{{0,1}\\}$. \n",
    "- $\\boldsymbol{x}:   $    independent (explanatory) variable also known as the input variable or feature.\n",
    "- $\\alpha:$ is the learning rate which determmines the step size at each iteration\n",
    "- $\\boldsymbol{w},b :   $ are the parameters of the algorithm\n",
    "- $f_{\\boldsymbol{w},b}(x^{(i)}) = g(z):  $ is our prediction for example $i$ using parameters $\\boldsymbol{w},b$.\n",
    "- $\\boldsymbol{D} = \\{{\\boldsymbol{(x^{(1)}},y^{(1)})},...,\\boldsymbol{(x^{(n)}},y^{(n)})\\}$: is the set of n training examples, where:\n",
    "    - $\\boldsymbol{(x^{(i)}}) $ is the $d$ dimensional input vector ;being the feature vector consisting of$ (x_1,x_2, ...x_d)$ of the training example $\\boldsymbol{i}$ \n",
    "    - $y^{(i)}$ is the desired output value for that input vector\n",
    "- $(\\boldsymbol{x^{(i)}},y^{(i)})$: an instance of a training example.\n",
    "\n",
    "### Intuition\n",
    "<figure style=\"margin-left:auto;margin-right:auto;text-align:center\">\n",
    "    <img src= \"images/sigmoidUnit.png\" width=\"600\" style=\" margin-left=auto;margin-right=auto;display=block;\"/> <br>\n",
    "       <cite style='text-align:center'><a href =\"https://machinelearningmastery.com/a-gentle-introduction-to-sigmoid-function/\">machine learning mastery</a></cite><br><br>\n",
    "    </figure>\n",
    "A logistic function output values in between 0 and 1 corresponding to the probabilities of y given the independent variable. The output of a linear function which is continuous in nature is fed into a sigmoid function which has the range of values of [0,1].  \n",
    "\n",
    "\n",
    "### Training \n",
    "Unlike the linear model, the logistic model does not have a closed form solution but can only be trained using gradient descent\n",
    "\n",
    "#### Gradient Descent Algorithm\n",
    "\n",
    "- Given a set of training examples ${(x^{(n)},y^{(n)})}$: the goal of gradient descent is to minimize the cost function  below through an iterative process.  \n",
    "\n",
    "$$ J(\\mathbf{w},b) = - \\frac{1}{m} \\sum_{i=1}^{m} \\left[ loss(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}), y^{(i)}) \\right] \\tag{1} $$\n",
    "\n",
    "where the **loss** of a single training data is given as :  \n",
    "$$  L((f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}),y^{(i)}) = \\begin{cases} \n",
    " \\text{- } log(f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})  &   \\text{if $y^{(i)} = 1$ }    \\\\\n",
    "\\text{- }log(1 - f_{\\mathbf{w},b}(\\mathbf{x}^{(i)})) & \\text{if $y^{(i)} = 0$}\n",
    "\\end{cases}$$ \n",
    "\n",
    "**Steps:**   \n",
    "Step 0: Initialize all weights $\\boldsymbol{w}$, and b at random or zero   \n",
    "\n",
    "\n",
    "**Repeat until convergence {**   \n",
    "Step 1 : calculate the linear combination of the **feature vectors** $\\boldsymbol{x_{i}}$ and weights $\\boldsymbol{w},  b$   \n",
    "Step 2: pass the output of the previous step into the sigmoid function $$ g(z) = \\frac{1}{{1+e^{-z}}} \\tag{1} $$\n",
    "where: $0 < g(z) <1$   \n",
    "Step 3:  calculate the partial derivates of **w** and *b* :\n",
    "\n",
    "\n",
    "$$  \n",
    "\\begin{align}\n",
    "\\frac{\\partial}{\\partial w_j} J(\\boldsymbol{w},b) = \\frac{1}{m}\\sum\\limits_{i=1}^{m}(g(z) - y^{(i)})x_j^{(i)} \\tag{2} \\\\\n",
    "\\frac{\\partial}{\\partial b} J(\\boldsymbol{w},b) = \\frac{1}{m}\\sum\\limits_{i=1}^{m}(g(z) - y^{(i)}) \\tag{3}\n",
    "\\end{align}\n",
    "$$ \n",
    "Step 4: update **w** and *b* :   \n",
    "\n",
    "$$  \n",
    "\\begin{align}\n",
    "w_{j} = w_{j} - \\alpha \\frac{\\partial}{\\partial w_j} J(\\boldsymbol{w},b)  \\tag{4} \\\\ \n",
    "b = b - \\alpha \\frac{\\partial}{\\partial b} J(\\boldsymbol{w},b)  \\tag{5}\n",
    "\\end{align}\n",
    "$$  \n",
    "**}**  \n",
    "Below we implement The logistic regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a9ff69dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Importing required modules\n",
    "import numpy as np\n",
    "from sklearn.datasets import make_blobs\n",
    "from sklearn.model_selection import train_test_split\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "np.random.seed(126)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8c1f9b40",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generating the dataset\n",
    "X,y=make_blobs(n_samples=1000,centers=2,random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "238dea65",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the shape of X_train is (750, 2)\n"
     ]
    }
   ],
   "source": [
    "# Spliting our dataset into train and test set\n",
    "X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=123)\n",
    "\n",
    "# X has two features and  1000 examples\n",
    "print(f\"the shape of X_train is {X_train.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "39c03fa1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Second Against First Feature')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABYlElEQVR4nO3dd3gU1dfA8e/Zvml0RIoCNsSCIoIVEBs27L2AvSuKvWIvrx0sWH9i76goVhTFgqKCglgQEOk1kLZ1zvvHTELK7mZTNyH38zx5yM7szJxNwr0zt5wrqophGIbR8rgyHYBhGIaRGaYCMAzDaKFMBWAYhtFCmQrAMAyjhTIVgGEYRgtlKgDDMIwWylQARpMjIiNEZGqGrj1JRIZn4trO9WeLyOBMXd9oWUwF0AKJyF4i8q2IrBORNSLyjYjsmum4akJEskWkUEQ+rM/zqupBqvp8Xc6RTgUmIl+KSMj5DKVfu6vqdqr6ZS2u2V1EVEQ8Kd4zWkSila55VU2vleCcL9blHEbmJP1jMTZOIpIHTATOB14HfMDeQDiTcdXCMdgxHyAim6rq0kwHVAsXqerT6b5ZRDyqGqvjNV9T1VPqeI56U0+fyagl8wTQ8mwNoKqvqGpcVUtU9RNV/bX0DSJyhojMEZG1IvKxiGxebt92IvKp8+SwXESuc7b7ReQhEVnifD0kIn5n32ARWSQio0RkhYgsFZHTy52znYi8JyLrReQHYIs0Psdw4AngV+Dk8jtEpK+I/CIiBSLyhoi8JiK3O/vaiMhEEVnpfL6JItK13LFfishZzvcjRGSqiNznvHe+iBxU7r0jRGSec535InKyiGzrxLW7c4edn+4vxjnnAhHZz/l+tIi8KSIvish6YISI9BeR6c7ParmIPOAc+pXzb37p00QNr5vqd/6wiPznXPMnEdnb2T4UuA443rnmzMqfodzneNH5vvRJ5UwRWQhMru76RsMxFUDL8xcQF5HnReQgEWlTfqeIHIH9n/oooAPwNfCKsy8X+Az4COgMbAl87hx6PbAbsBPQB+gP3FDu1J2AVkAX4Ezg0XLXfhQIAZsCZzhfSYnIZsBg4CXn67Ry+3zAO8D/gLZO7EeWO9wFPAdsDmwGlABjU1xuAPAn0B64F3hGbNnAI8BBqpoL7AHMUNU5wHnAd6qao6qtU32WNBwOvAm0dj7rw8DDqpqHXVG+7rxvoPNva+e636V7gVS/c8eP2L/XtsDLwBsiElDVj4A7sZ8qclS1Tw0+1yBgW+DANK5vNBRVNV8t7Av7P97/gEVADHgP2MTZNwk4s9x7XUAxdoF5IvBLknP+Axxc7vWBwALn+8HYBa2n3P4V2BWGG4gCvcrtuxOYmiL+G7ALW7Arojiws/N6ILAYkHLvnwrcnuRcOwFry73+EjjL+X4EMLfcvixAsSuzbCAfOBoIVjrniFTxl7tOsXOOfOBnZ/sCYD/n+9HAV5WO+wq4BWhfaXt3JzZPimuOBiLlrpnv/PyS/s6TnGct0KfcOV+stL/sM1R+T7k4e5bbX6Prm6/6+zJPAC2Qqs5R1RGq2hXYHrsQeMjZvTnwsIjkO80XawDBvnPvhl3QJ9IZ+Lfc63+dbaVWa8W23mIgB/uOzwP8V+nYVE7DvhtGVZcAU7CbhErjWKxOSeIoO7eIZInIOBH512lW+QpoLSLuJNdaVvqNqhY73+aoahFwPPbd/lIR+UBEelUTd2WXqGpr56tvkvf8V+n1mdjNeH+IyI8icmgNr/l6uWu2dn5+qX7nOE13c8QeNJCP/STXvobXraz850p5faPhmAqghVPVP7CfBrZ3Nv0HnFupkAiq6rfOvmTt86UFSanNnG3VWYn9FNKt0rEJicgewFbAtSKyTESWYTfTnCj2CJilQBcRkXKHlT/3KGAbYIDazSilTSfl358WVf1YVffHbrr6A3iqdFdNz5XqMpWu+beqngh0BO4B3nSao+pyzaS/c6e9/2rgOKCN2k1a69jw80p03SLsp6VSnar5XKn+5owGZCqAFkZEejl3dF2d192wm3a+d97yBHbhup2zv5WIHOvsmwh0EpGRYnf65orIAGffK8ANItJBRNoDNwHVDg9U1TjwNjDauTvvzYa7+USGA58CvbGbb3bCrryygIOA77CbhC4SEY+IHI7dH1EqF7s5Kl9E2gI3VxdjIiKyiYgMcwrfMFDoXBdgOdDV6Y+oVyJyioh0UFULuwkH57orAQvoWYvTpvqd52JX0CsBj4jcBOSVO3Y50F1EypclM4ATRMQrIv2wR2zV9vpGAzIVQMtTgH3HPE1EirAL/lnYd8ao6jvYd5avOk0ks7ALVlS1ANgfOAy7aeRvYB/nvLcD07FH5fwG/OxsS8dF2M1By7CfRp5L9CYRCWDfiY5R1WXlvuYDLwDDVTWC3Zl4JnYBeQp2xVU6zPUhIAiscj77R2nGWJkL+2e2BLvJYhBwgbNvMjAbWCYiq2p5/mSGArNFpBC7Q/gEVQ05zVN3AN84TSm7pXvCVL9z4GPsNvq/sJvmQlRsvnnD+Xe1iPzsfH8j9pPiWuz+ipfrcH2jAUnFplLD2PiIyDTgCVVNWLEYRktlngCMjY6IDBKRTk4T0HBgR2p/p28YGy0zE9jYGG2DPT4+B3vU0jHaPGcKG0aDMk1AhmEYLZRpAjIMw2ihmlUTUPv27bV79+6ZDsMwDKNZ+emnn1apaofK25tVBdC9e3emT5+e6TAMwzCaFRFJOLveNAEZhmG0UKYCMAzDaKFMBWAYhtFCZbwCEBG32It3TMx0LIZhGC1JxisA4FJgTqaDaGnUWoNGfkTjizMdimEYGZLRCsDJSHkIkPa6qEbdqFpY629DVwxE156HrhyKteZs1Cqu/mDDMDYqmX4CeAi4CjuNbUIico7Ya6BOX7lyZaMFtrHS4leg+E0gAloAhCHyHbq+VlmRDcNoxjJWATgrGa1Q1Z9SvU9Vn1TVfqrar0OHKvMYjJoqfhY7HX55EQhNQjWc6AjDMDZSmXwC2BMYJiILgFeBISJS7QIiRh1Z65PsUFDTDGQYLUnGKgBVvVZVu6pqd+AEYLKqnpKpeFoM324k/LW7NwFp3djRGIaRQZnuAzAameReAZLNhiwgLiCA5N1KxWV0G55G/8JacwbW8p2wVgzCKvof9kqHhmE0hiaRC0hVvwS+zHAYLYJ4Nof2E9GiZyHyC3h6INlnIt5ejRqHxhaia47b0OykxVDwIBpfjORd36ixGEZL1SQqACMxjUxHC8dCbAF4t0NyLq6Xglrcm2a8kNWicVCl07kEil9Fcy5CXK0yEpdhtCSmCaiJ0tBkdM0ZEPkWrCUQ/gxdfTwa/TXTodWP6EwgXnW7+OwKzzCMBmcqgCZIVdH1twGh8luBEnT9PRmKqp65tyDhn59GwN2l0cMxjJbIVABNkZaAtSzxvuisxo0lBQ19irX6OKwV+2Ctux6NL0n7WMk5F/BV2uqHwL6Iu329xmkYRmKmAmiKxG9/JeJuV6dTa3gq1upTsFbuZxfasUW1Oo9V+DS67gqIzgBrMZS8ja46HI0nqbgqEW9vpM3j4N4ccAN+CB6NtLq3VvEYhlFzphO4CRJxo8ETofglKjYDBSHr3Fqf1yp+DdbfSdlM4JLFaOgjaDcB8XRL+zxqFUPhGCrOKI6DFqGFTyKtbkrrPOLfE+nwKWoVgfgRMX+OhtGYzBNAEyW5oyB4FOAHyQKCkHM2knVcrc6nGoGCe0hYaBc9VrNzRb7F7pOoLAaR72scm7iyTeFvGBlg/tc1ItW4XUDGl4F3R8S7VdL3iniQVqPR3CvAWgHuTREJ1v7i8UUkzrlnQXhaWqdQjaH5l0N4MhBJ/CZ359pGaBhGIzMVQCPR+BJ0zclg5YNagKL+wUjrB1Le/YorB1w5dQ/A1QY0lnifu2Nap9CipyH8JUkLfwJIzjm1ic4wjAwwFUAj0fxLIb6UCnfh4S/R4peR7NMa5poatrN8RmaApzv49rDnFVQowINIdvJ+BbWKIPwZWOug+Hkq9kmUlwt5NyC+/vUWv2EYDctUAI1A4yshOoeqTTAhKH4FGqACUCsfXX0sWCuddAs++/quTewmJXGGYOZcjgT2QdVu0y+fD0gjM9G1p2NnCo2S/M7fDR0+wVXHEUqGYTQuUwE0ighJ+9sbKAe/FoyB+BIgWi4G7CGb+CB4CpJ7MWgJVv6VEJoExFHf3kirm8HVCc0/H7Sw+ou5e5jC3zCaITMKqDG4OoMrUQHpheDQhrlmeBIbCv/KIlD8HBpfY/dLhD60txGHyFfo6mPQyI/2hLSESp8SvCBZSKs76y1s1QiqyZqZDMOoT6YCaAQigrS+zxnO6TS9SBa4uyDZ5zXQVat5uBM3FD/njA4qX1FYYBVD+CvQBLl6AFxdwD8Eskcg7T9AfDuV7VKrEI3MQGOLUI2j0T/Q2NyyJqZkNL4Ca81Z6PKd0OU7Y60+ETU5gQyjQZkmoEYivl2g/cdo8esQX4T4BkDwECTZjN+6Ch4LRU8CSZqYVMBak6SQL4Ho7yTu8PUjORciWUdX2WMVPgmFY0E8TtOWgnjtKQPuDtD68YRDX1Xj6JoTnE5yJ57oz+jq46DDZHsklGEY9c48ATQicW+CK/diXK3vQbKOarDCX+OLwbcreHekar6dUhHwH2A/CVQRhKRZR30QHFb1mqHPoehRIOT0G0SBmNOMVALxheiaU+wJaZWFvwJrLRWzg6pdiYQmJv2chmHUjakANiJqFWKtOQNdeSCsPROi00n6KxY/+AeDuxvgLbfDZd/BU5TkKgXo2jPtSqb8tYueSdFnUCoC4S+qbo4vdEYZVVaCxuZVc07DMGrLVAAbEc2/HCJTsTt0o9htL0k6VNVCrMVI25cgeCjgx07KJs6EsRRt9pHv0VVHV7ybt1alEWA08fu82zqVTiWShXi3r/68hmHUiqkANhJqFUBkSg2OiIMrD3Hl4Wp1D9LxO2duQJyK+YKSXXANWvL+hte+vam+SykO3n5VN3t3BfeWVGyu8oCrLQQaaJSUYRimAmhq1CrGKngEa8VArOUDsdbfi2UlSeFQ/rjIT6S8a6/ADb7+iKvthk3hz+yO4ZoIfYoVX4tV8BCEf9hw7qQCiHcbe8EbDZUtAC8iSNvnIeskkNYguRA8HGn3JiLJ+jAMw6grMwqoCVGNoKsPh/i/GzYWP23n2u/4BRCoMFO34sEl2OPz06kELPDugBaNh8CBiHsT0CISJ4tLIfY7rNyt0jVTVACe3mj4S3T9rfYkNQmgwZOQ3MvtjKB510HedTWLwTCMWjMVQFMSmmh3iFama9DlA4AQ6u4GOVfjCh5Q4S3i3Q7FDVT/tAAKReNQvFBwD+rdyakAkqV6SMJaTtUKJ45dEVWOJQjBIejaSyjrl9BiKH4GDX8KrR807f2G0chME1AToqFPSH4H7xSa8f9g3RVoqOJoGvFsBv79ST7sszILe45AFKI/2nfzNY848WZnkpt9f+GsZZA7EsLfULVTWiH+L7r6JKyST2oRg2EYtWUqgKZE0kvLDCG08MGqh7e+H3IuAtemQJ6z8HpNWNh377mkbsuvhhbbax6IH4hC8HDwHwqxP1McFIJ1V9lrJhiG0ShMBdCESM6Z6b+5fD9B6fHiQbLPQHKvgOBB4B1ciygUcEPW6bU4tvw5wk6zUhRKXoVVe9qZSVMqRtecXYfrGoZRE6YPoBqqas+KjXwHrtYQOAhxtWqYi7lywdsfoj+k8d4uVTapVWinT4gvJq2hnEkVQMlL2PcHNewYrqvoVKzoAlze7o17XcNogcwTQAqqFpp/Gbr2NLTwYXT9XejKQWgkjQK6ptcKTUZXDIboL+kdEF+AteoINPbfhnMUPeV0Itel8Ad7reASGr3wLxX+ODPXNYwWJmNPACLSDRgPdMIuaZ5U1YczFU9CoQ8h8kW5FAcldoqatRdBx2/rbSFztQrR/MtIvtpWIjGI/Y6u2hclAIFDIFw6C7iZk9xMR2AYLUImnwBiwChV3RbYDbhQRHpnMJ4qtOTtJPltohCdUX8XinzDhhz7ieSBq1OK/SEIvQW6vP5iSouQOu5UknUyuyEwKHHSOMMw6lXGKgBVXaqqPzvfFwBzgKoN2xmVbEhmuhOuKp0tvgxr7QVYy7bDWrYDVv6VqJWP/QCU6nwxexnHpkaC4NmRmv8ZecE3BHx7VtrushfPWbkfunwnrLXno9aaegrWMIzKmkQfgIh0B3YGpiXYd46ITBeR6StXVjeKpJ7jCh5tF3JVuMG7c43OpVqCrj4GwpOxE7WFIfQhuvok1Ls7FVMhV1ZMxtrjU1G1VwNrP8X5eST6WVXmhrxb7CaryE/ltrsABes/7J9FDMJT0DWnVruYjGEYtZPxCkBEcoC3gJGqur7yflV9UlX7qWq/Dh06NG5wgYPANxC7YHMBAZAg0npMzdv/SyaBVUjFgjwK1lIkNgta3U3i5pSM/4pS0sLHYfWBzgIyYaptEhI/eLeD9ddTsc8j0VNQzB7RFJ1enyEbhuHI6DBQEfFiF/4vqerbmYwlERE3tH7EHpkT+Q6kFQQPrphELU0a+wP7Tr7yjijE5uLKHoHl7gZrz7InUuHGLhTTSe1QXwLUrCO6BMI1XLBF4xBbXKPraHQ2WvQShD8HcUHgYCT3moYbjmsYLUQmRwEJ8AwwR1UfyFQc1RER8PW1v+pyHs82KFlUqQTECx57xq7L1wftOA0NT4X4PLAKoOhp6j6sM10NvRi703ey7mJSN3mVoxYUPgmaj73CGFDyHhr9Fdq9j0jTfkIyjKYsk08AewKnAr+JyAxn23Wq+mHmQmpAwYOh8AGwQmxoBvKA5KDFb6KhDyFwjN3cUfSEsyiLRdoFZbOg1GyYqh/cXcFaQsUnoaidTTTyDfj3rt8QDaMFyVgFoKpTqf0YwmZHJAjt3kLX3wLhKYCAqw3oeghPsl+XvIddSDZms08TI12BdfbiNMFj7Aqz5Pmq79MIxP42FYBh1IFJBdGIxN0JafM4ABr6Al13Wbl5Boo9Oqjas1CbIajNhrc70uYpu/8F0OK30ZJETWc+cPdo/PgMYyNiGlAzRMOfOp29NSTtSG+4ZTMV/cleF6FU8CBwZVPxT9UDrvbgH9jY0RnGRsVUAJkitUy5rMU0n5YzwX7I9IH/cNKquLQELX5rwxkkiLR7w1lz2G2fz78v0u61sqcEwzBqxzQB1TPViN1B6WqPuHKSvk+CR6PFL1PzTt5aPDVkiv9g8A0A8SPWctR1FIQ+AV1pr/2rxSTuFK7YxCXuzkjbp5w1hCX5spiGYdSIqQDqkVX0LBSOwc4YF0ODRyB5NyVe2NzdFTzbQOzXRo+z0YQ/sGf76lqUKEgAJA/afYG4N0FX7Gl35lYQRLKOTng6M+TTMOqXqQDqiZZMhIKHqTBmv+Rd1FoH/t3sHDf+vRHxYBW/DetvYaPI3FkdXVbu+2LQEKy/CbJPBd9uEP4Uu2knas8S9u0JgcMyFa1htCimAqgnWvgYVSdshSH8MRr+EsQDkofm3Q7rR9Pwk66aKgsiX6OR6dg/L5+9LXAEknUsePuaJh7DaCTmmbq+pFzu0Fke0VoO66+hRdz5V6u0sowAUTtJnrePKfwNoxGZCqC+ePuk8SYLrFU0ycyeGReF6OxMB2EYLYqpAOqJ5I5yUkdX9yN1sooalaidF6ny1ugfWGtHYa08AGvVMVhFr6IazkB8hrHxMX0A9US820LbN9HCR+xF5DUOuoYqaR3cPeyJTdG/qD7JW+mi7D42+mYjyQPPthU2WUXPQcH/UeFnWDAbLXkF2r2GiKlIDaMuzBNAmqzid7FW7IO1rBfWyn2xSqrmrBPvVrjajMHVcQrS4QN7qKdkOXsDIDlI6/uQti8hedeDq2Pqi3p2gHYTIfvs+v9ATYIHJBskD2nzeIX2f40vh4L7qJoXKQ6xf9DiCY0ZqGFslMwTQBqs4ndh/Y2UjdyJ/wfrrsECXMGDEx4jrlbQfiKEPkIjP4N7cyTrCMTVxn5D1nFo4UOpLxz7E9FCCA5Di8ax0SWJ8w9FAkMgsK+dLK8cLZlA8r6SCIQ/guwTGjpCw9iomQogHYX3U3XYZshO75ykAgDsCWDBYUhwWJV9qgrVLnUYQvNHQt49bHSFP0DuKMRTdRloq/BRKHyUlLOkXa0bLCzDaClME1A1VBWsZYl3xhfX/rzrb3P6CKph5cO6kbW+TpPl6obL0wW11qGRH9HYQgA0OgcKq3va8SNZJzdKmIaxMTNPANUQEdS1iT2GvzJ351qdU2OLoOR10kvrrOVSRm9EWt2LVfAQFD1jp3bWCOrbGTy9SN3h7YHckYhv10YK1DA2XqYCSEfOSCd1Q/lmoADkXF6780V/BnGnV/672tpPARsVF6y7xPlcUSgd1hn5GWJLkhzjhsDhSN61Zi1gw6gnpgkoDa6soyFvNLg2cTZsAnm34woeUssTtiP1j16c0TGtoPUTG2F7t+XMnK68AE7EWf6x6nwA8CC5o0zhbxj1yFQA6bJW28s3kgXWWoh8WfsJSb7dAH+SnV7IOg3JuxXp+BUu37aQe30tg24KaprawQvBo7Any7nt1/gh7wbE3aHeozOMlsw0AaVBSz6AwrFUmLgV+hQVH9Lq7lqcb6KdG6gy1+ZI63sR384VNktgCLquFbCuxtfKPA/pLXXpkCCSdxNknQzhz+3jA0MRT7eGCtAwWizzBJAGLXqChJk+SyaiVs0WaFGNQMEt9vEVuMC3K3h3QqN/oeHvUasQABFPE1/+MNVdfrqFvwAByLsVETfi3RrJOR/JOdsU/obRQMwTQDqsVUl2uDY0C6UrNpfEvb8WhN5CI1PAKnQ6iaNo7kgIngThiQmOaSpS9WZ7SGsOg7c/ZJ0G4U+xCu4Ad0ck+1wksF99BWkYRiWmAkiHdxdn4ZJKBZ1kgauG7dKuVqDJCkTdkFa69FIFj0BsedVrNxfu7hBfROr1DzwQPBbWXY295KUF1lJ03Sg0fhmu7BGNEalhtDjVNgGJyNYi8rmIzHJe7ygiNzR8aE2H5F7m5PQp/+MKQN71NV6YXNxdwNub9DtHSyAyuUbXaDoCds6j4NGkvteIwfqrgUIqpH/QEih8CMsqREs+xMofhbXuVjT6R8OGbRgtRDp9AE8B1+I05qrqr0CLSsIini2Qdu/YSxW6u4Fvd6TNk7iCtVu6UFo/CtIu/QPUovk8rAngB2kLuTci/j1xtboZ3JtWc1yStA8KrDkVXXcthN6HkhfR1Udg5d9kz9I2DKPW0qkAslT1h0rbNsLENKmJpzuu1v+Hq8PnuNo+j/h3q/253O2RduOx0zxXxw/Bw+2mo+bAd5Cd1lnXQ8HNWKtPRmP/2XMaaiUMsX+o2AlvQehVdF2LehA1jHqXTgWwSkS2wGmEFpFjgKUNGlULIJ4tIfcK7PkAQeyO5NJx76XNQ35wd0CyT0favg6e3hmKtgYiH0JsJvY9QgyiP6FrjofA0difsyb8IG1I2n8QeseuXAzDqJV0KoALgXFALxFZDIwEzquPi4vIUBH5U0Tmisg19XHO5sSVPQLp8Kmd3iDvRqTjt9DmWXBvSdlsWE8vsNYjnm642k+Atq+Bb5C9loB780yGn0L5phkLtBhcORAYgj3BK82FXHyDwNcvxRvcEJ1e+zANo4VL2bAsdg/n+aq6n4hkAy5VLaiPCzvnfhTYH1gE/Cgi76nq7/Vx/lJqrYPQJ6CF4N/bvvNuQsTdCbI2dKloyev2egOl4+fDk9HIdOjwEeJqi8u3M7R9yn6vtQZdUfumqEajJYi1GGn9IBr9C6I/o/HVUDSO5KOD/EirGyG2AA1/RuJWR7eTVsMwjNpI+QSgqnFgF+f7ovoq/B39gbmqOk9VI8CrwOH1eH40/C26ciBacDtacD+66ih7FEkT6TxUVTS+HLXW269ji+zKqkKhaIGWoEUvVTleXG1B2ic5e81GJzUoCZYt9yjerZGsE3DlXoh0/ArcmyU5xgOuNoh/AGSdkvg9rizw7dFAQRvGxi+doSW/iMh7wBtAWf4CVX27jtfuApRvwF0EDKj8JhE5BzgHYLPNkhQWCaiG0fyLqqZSDr0FgX3Av3ctQq4/Gv4eXXeNnWMIC/XtDoFh9sLoVXIMhSH6S9VzRH6BpHVyisVU6k2A1OP7S/lRyQHVCss+iqs15N2Brj270nmCkH22vaAO4Mq7DsvTG9bfjD1MVMC9CdLmCXuWtGEYtZLO/562wGpgSLltCtS1Akg0EL7KrbmqPgk8CdCvX7/0b90j0xJv1xK05C0kgxWAxhaga8+lwsiWyLcQXwKaKHWCFzw9K55DFc2/nKopJRpL6YL1adC1sPYsNDAUWt1dsRLwD4A2Y9D1d0F8vt3pm3MekjW84tWyjkCDh0BsDhAAz1YVzmMYRs1VWwGo6ukNdO1FQPkkL12BZMnga05T3AGn2tcItPgFqubIiUF8np1q2lpVcb94kKzTNhyvcbTgXrBqvyJZ3VkguaCr03x/CYQ+guChVZ6+xD8I6TAIrfSEUJmIF7w71iFmwzDKq7YCEJHnSHxnfkYdr/0jsJWI9AAWY08uO6mO59zAvxsJm0EkCwnWa1dDzcX+JXGnpmUX/tIKdB2g4O6OtLod8Wxo/tL1t0JJXR/A6kiyIPs0KHyc9JqBAErQkolJn77MHb1hNK50moDKZyELAEdSD3fqqhoTkYuAj7F7LJ9V1dl1PW8pkSCadx+sG4XdVBEFAuDfB/xDqjm6gfl2dZqoEjXfRIFiaD0W8fVDXLkV9qq11in8G7Ppx43d5BPHvvO3O18l+1zwbGPP0tW1aZxH7M5dwzCahHSagN4q/1pEXgE+q4+Lq+qHwIf1ca5EXMH9Ud8naMn7oAWIfxB4+2b8TlOyTkCLnwcrSsJ2dC2GyA9IYJ+q+2ILk3QUN6Q4IBA4CiSO+IeCfxAiLntsv+9zdMUQ7PUKUnXTKISnYhVPQIKHZ/z3YBgtXW3WA9gKSH84ToaJuxOunLNx5V6O+HZpEoWOuFoh7SbYTwIJ+ey1gBNxdwVNtWh6eS4755CnD+mlnQBkkyQ7YhD+CMk+GwnsYxf+pYe4cpD2bzlDMl2AB7y7Ys9yrjQc1VoKBTejRU+l+RkMw2go6fQBFFDxtm4ZcHWDRdRCiLsjtB6HrtzLnqRWgRsJDktyXDs0cBCEJpJ8qGcQWj+IeHs76xjH0eJXoOR1iOUDy5MHpqn2FaJrL0U6fFA1Lk83pO1zqDrDNK016OojwUpwPi2BosfQ7OGIJFsa0zCMhlbtE4Cq5qpqXrmvrSs3Cxm1I64spM1z4GrvLAKfDZKHtBmLuJPdiYO0uhN8eyXZ64FWj+AKDEHcnRARRDy4sk9F2jwJ7kQLrtdAfCEaTz76SMSFiKCFDzijmVI0CcWX1S0WwzDqJJ31AD5PZ5tRO+Lrg3SYirT5H9LmaaTjd9XOURDxIm0edXIBVf4VxqHgLlSrjszR/MsgXtc8fgIat+chWOvsJS4TCSVL31AaTNykcTCMDEtaAYhIQETaAu1FpI2ItHW+ugOdGy3CFkDEZVcEvl3sse5pHeOD1mMS7FG7nb3k/Ypb46sgOos6zxB2d0RjC9FV+6Ir9kCX98Vadw1aecZ1ys/hh+ARiCunbrEYhlEnqZ4AzgV+Ano5/5Z+vYudxM3IMInNs/PsVFGChqdU3KQh0l+FLOHVgGzIvgDyL3CWeYwCESiZgK4+CS2/1GXwWOxO4ATnyToBybupDrEYhlEfknYCq+rDwMMicrGqJrrVNBqRxuahhePsu3jPlkjOueBOlgjOA+4ND2ka/QMNTa7j8FEFimB9oqzdFsRmo2tOgrYv2U1UOeej0V8g4uQwEpfd5NNmPC6PeYA0jKYgnXkAY0Rke6A35RK5q+r4hgzM2ECjv9uFq4aBOMT/QcNfQOsnQFo7Ce/KzyfwIE6KaWv9PVD8ElVTTzSA6BwIfQzBQxHxIW3/h0ZnQ/R3e/iqb0CF4aOGYWRWOsNAbwYGY1cAHwIHAVMBUwE0El1/pz05rIwFhKDgNqTteDT/fGeCmBvwIK3uRjw90chMKH6Z9FM11FUYDX2MBA8t2yLe7cC7XSNd3zCMmkhnXv4xQB/gF1U9XUQ2AZ5u2LCMCqIzE2+Pzwf3JrjaT0RjC+xKwrN1WYpkDX1A4sLfg90ZnGrWrgt78lhNKg8XVEpdURf2chQxM1fAMBpIOhVAiapaIhITkTxgBdCzuoOMeuRqBdaKqtvFT+mvUDzdEx2I3XlbuaC3Emwrz+Psr+mTgw+CR6GhSXZqa+8O4N21xrOv1SpCC26DkolADPX0Qlrdhnh3qGE8hmGkkk6D7HQRaQ08hT0K6Gfgh4YMyqgkawRVF1QPQPCE1G3qgYNJvuxCgu2SBa1fccbnVzNcVLraWUtLJ7Dht+PMvwRdd529Atvac9A1JyWck5CK5p/vFP4R7A7m39E1p9orphmGUW/SmQl8garmq+oT2Ov3Dm/ANQKMBCT7DMg6BvCD5AA+CByI5I5KfRwxElcALsoWnS+/zdUevFs7M3iroUsBD+Tdafc5dPzKXtTGWg1aBMTsJqnoLLTwyTQ+pXPa2FyIzMAu/MvviKLFL6Z9HsMwqpfOTGARkVNE5CZVXQDki0j/hg/NKCXiwpV3I9JxKtL2f0jHr3C1/r+yJROT0fCXJL6Td4P/IOfuPQcIgrunPRvZleU0LVUnbi9HGZuLBA5ENQqx36jatBSu2doFsYVJUkZHIfZn+ucxDKNa6TQBPQbsDpzovC7ATATLCHG1Qrw72ovBp3VANgm7ecSN+PsjHachbZ5F2r+Fq8OHiKcrIm7IOo1yI35TiNh3/QDFb5C8XyHNpSMBPFsmWRbTB94+6Z/HMIxqpVMBDFDVC3F6BFV1LWnnFjYySYKHkfBXrEDgAHusvm8nxLNlxeNyLk2Rqro8lz2+HyD8UfK31WD9ZfFsZi/aU6ECcoEEkKyT0z6PYRjVS6cCiIqIG+f2TkQ6UKNbOiNTxN0ZWt0NBOymHsmxl8RsMxZxtUp+nLhJL22ED8ku7Q5KdvcvEDwxyb4kR7S+H7LPAmlrp7rwD0bavYm4O9ToPIZhpJbOMNBHgHeAjiJyB/a8gBsaNCqj3riCh6D+QU5TjQf8uyMJ8wdVItkpdvrtiqTV7faaA2CvFlb4EFWGjro625PBakDEi+ReArmX1Og4wzBqJp1UEC+JyE/Avti3hUeo6pwGj8yoN+LKgcABNTsm6wSnE7lyls+2dr4fT3fnScHZnH0KGp4MsVlOaoqA3dfQZkyTWIXNMIyqklYAIvKJqpaWGkeq6l2NFJNRDVWF+DxAwN2jQQpY8e+O5pwFheMAr9Mi5EXaPot4t6j6fvFB2xcg8h1EfwZXBwgcXGVRe8Mwmo5UTwDlG1yPBUwF0ARo9Fd07SVgrbU3uNtD6zEbmmLqkSvnYjR4HER+BFcO+PZMuV6BiIB/D/vLMIwmL1UncKpcAUYGqLUOXTMcrCXYTTMlEP8PXXMaahU1yDXFvQkSPBTxD057sRrDaC5Ulbcemshxm57Fgd7jOW/nK5nxxaxMh9VoUlUAPUXkPRF5v9z3ZV+NFaBRTuhDeynFyjQG4Y8bPx7DaObG3/I6/7vhVdYuX4cVt/hn5gJuOPQu5kz7O9OhNYpUTUCHl/v+voYOxKiexleSOEFbBOJppG8wDKNMJBThjfveJ1xccZGkcEmE529+jbs/2vgHO6ZaEWxKsn1GZohvF7Q4q9LaANgrffn6ZiYow2imVi9ZS7LxEwtmLWzcYDLELM/UnPh2B892VJwlGwDvLvaXYRhpa7tpazRJT+dm23Zp3GAyxFQAzYiIC2n7HOReDp5twNMLcq9E2owzY+0No4b8QT9HXnIQ/ix/pe0+Tht9fIaialzpzASudyLyf8Bh2Dl//wFOV9X8TMTS3Ij4kOwRkD0i06EYRrN3+u0nkpWXxRv3vUfBmkK69erCBQ+dzvZ79sp0aI1CNMkzkDP6J+lQUFUdVuuLihwATFbVmIjc45zv6uqO69evn06fPr22lzUMw0jKsixcro2zUUREflLVfpW3p3oCKB35cxTQCShdjeNEYEFdglHVT8q9/B47v5BhGEbGbKyFfyrVjgISkdtUdWC5Xe+LyFf1GMMZwGvJdorIOcA5AJtttlk9XtYwjKaiYG0hyxespOPm7clra9KHNJZ0+gA6iEhPVZ0HICI9qJgmIiER+Qz7yaGy61X1Xec91wMx4KVk51HVJ4EnwW4CSiNewzCaiXg8zmMjn2PS05Px+j1EwzEOGD6Ii8eehdvjrv4ERp2kUwFcBnwpIvOc192Bc6s7SFX3S7VfRIYDhwL7arKOCMMwNmqv3j2Bj5/7kmg4SjRsrwT32Ytf0aZTa4anMRJnzbK1jLtiPN+9Px23x81+pwzkjDtOJJiTOuW5ZVm888iHTHjkQwryi+i9+zacfN1RbNdCOn9LJe0ErvAmET9Q+pP5Q1XDqd6fxvmGAg8Ag1R1ZbrHmU5gw9i4HN3hDNavLqiyPbt1FhPWPJ/y2FBxmNN7XcLaZfnEY/YaVV6/ly137s7D39yRcmj0g+eO4/OXvq4yC3izbbsy+u0r6LbNxjUPoDadwOXtgn3n7wH6iAiqOr4O8YwF/MCnzi/pe1U9rw7nMwyjGSrMT5zEsGhdMapapRCPx+NM/2gGi/9exn9/La5Q+ANEw1Hm/baQ2d/8wfZ7bZvw3KuXruXT8VPKnjjKWzhnEZcPvImXFj6Bz7/xJz+stgIQkReALYAZQGkmMgVqXQGo6pbVv8swjI3dVn178uePc6ts36JP97LCf/5v//LOmEks/nspC2YtJBqOEY3EiEViCc8ZKYnwz8x/k1YAC2YtxBfwJqwAAMKhCN9M+IEOXdsRCUXZbo+t8Qf9Cd/b3KXzBNAP6G3a6Q3DqG8XPHw6V+13K9FQBMtSxCX4Al4ufPgMAL5+exr3nPYI0XAMK57eUuRqKaHiREkTbZ16dCSapPIAKCkMcdfJDyMieP1exCVc8fT5DDqu+nUu4vE48WgcX8CXVqyZls7A11kkHs1jGIZRrX/nLOL+Mx/jogHXMvaSZ1j+74Zuv967bc2Y7+9k4LG7061XFwYesxsPf3MHO+y9LfFYnAfPeYJwcSTtwr9UNJT47h6gy5abst0eW+P2JCn+1K5ErLhFuDhMqDDE/53+KIv+Xpr0nKHiMA+c/QTDck/lsNxTOafPKGZ/+2eNYs6EajuBReQLYCfgB6Csx6QuM4Fry3QCG0bz8tvXv3P1AbcTi8RQVdweF/4sPw9/cwfdt+uW8th5v/3LxbtdS6QkeWGeiD/Lz0WPnMHQM4YAdtrncEmEnNbZZc1KxQUlPHDOOKa8/k1aS1+5vW6OHTWMM+88KeH+6w65k5lfzCJSruIJZPt5/Of/o+tWm9Yo/oaQrBM4nSeA0cARwJ3A/eW+DMMwkrIsi+sPvZtoOErpjWY8ZlG8voRxV6TuQvxn5gKuO/jOGhf+AC6XMPDY3SkuKOHOkx7i8NbDOa7TWZze61JmTpkNQFZukBteGclbK57h+KsPp/OWnfAFknf6xqNx1q1an3Df0nnLqxT+YHdIv/3gxBrH35iqrQCcGcF/ALnO1xyzVoBhGNX57IUplBSUJNz361ezkx5XUljCFfuMZvXiNbW6brvObYhFYlzY/xqmvPEdsUiMWDTO4r+Xcs0Bt/H9Bz+VvTevXR5n3XUKz/81hl0P2jnpOf1ZPnY7NHHK9cVzl+FNMGIoHrOY38TXFai2AhCR47Cbf44FjgOmiYjJ3WMYRkqfvfh10n3+rEDSfV+/NY1YLMHSpw6X28Xg4/dIOlN40d9LOa7z2Sz6c0mVvoNYNM6Nw+7m8sE3kb9yXYV9R488FH9W1c5bcQm9+m/FgEMSL7rUfbuuCUcUeXweeg3YKunnaArSaQK6HthVVYer6mlAf+DGhg3LMIymrLighLcemshV+9/KvSPG8uf0f6q8J9VErIHHDKiyLRKK8MKtb/D4Zf8jVJh8FI/H6+ac/zuNnDbZid+gdpNNUgq/fTWH4zqfzeWDb2Lahz8DsMPe23LeAyPwZ/nwB324XEJWXpDzHxjO3R/fgNuduMJp36UdA4/bo0LlIQK+gJejLj0keRxNQDqdwL+p6g7lXruAmeW3NRbTCWwYmVe0rogL+l3N6qVrCRdHyoZujnziHPY7ZVDZ+ya/MpUHz36CUKXZtl6/hwn5z+PzbygwVZVR+4zmzx/nEimJJL22x+tm16E7c+u7V/PCrW/w2r0TCBcnf386/EEfR486jNNvPQGwR/TM/20hee1y6LJleh248Vic1+6dwISxH1G8voQ+g3tz3v3Dm8yM4rp0An8kIh+LyAgRGQF8AEyq7wANw2geJoydxKrFa8oKXrWUcHGEMRc+Q6RcU8jg4/eg/8F9CWT7cbldeP1e/Fk+7pp0Q4XCH2D2t3/y90//pCz8RYRBx+3BtS9fCsBJ1x1FvwN3gjouhhcuifDyHW+VdRAHsvxsO2CrtAt/ALfHzUnXHc3Y7+/k0HP3J1wc4YOnPmPFwrQz3WREOp3AVwLjgB2BPsCTqnpVQwdmGEbj+O/Pxdx92hiGb30x1x1yJ7O++SPl+6e+80OVES8ACPwzY0HZS5fLxQ2vXcY9n97EgSP2oU2nVqil3H/W43w6vuI4kr9+/Id4knb/QI6fU28+hteWPcXOQ3bghVveYNIznzPvt4V89970tIZxVkvhhsPupqQoedNTdf6ZuYCzdricd8d+xMwvZ/Pu2I84a4dRzPv133oIsGGkkwqiB/Chqr7tvA6KSHdVXdDQwRmG0bDmz1rIpXtcT7jEnmy1ZO4yfp0ym2tfvJQ9j+if8Jhk+frjsTi5ldrlRQQrbjH55a8JO3f3S+ct5+ELnqJwXRFHXnwwAG06tUJciW/lY5E4C2Yv4sQu55ZVEi63C1CseP0lKLDiFlNe/xa3282kZz9HLeXAEfuw/2mD0kpNPeaiZygp2FCBxJx0FWMvfoYHptxab3HWp3SagN4Aynelx51thmE0c09f8yKholCF0TLh4ghjL36G4sISEvURHnXpwQSyK+bGcblddNlqU7pu3bnK+5+97uWywn/DNcKMv/l14rE4U974jgfOfiLxUwV2Qfr1m99XeEKw4lZahb/b4yIrL8jtE6+hU4+OyWf/YucQevP+93nkwqf47as5zJr6B49e8iw3Hn5Pwp9DZb9/l3jmb1OeEZxOLiCPqpb99lQ1IiLNI9GFYRgp/f7tXyQq21YtXsMRbYaTnZfFKTcew1GXHlI2qmfAIbtw3JWH8+rd7+DxebDiFh26tee2964pO96yLH76ZCb/zFjA3z/Pq3oB7IyfYy56hk/Gf5kydUNtdOvVmW69urDzvjuw/6mDcLmEY0cNY+o701i2YAVL/1le5Rhf0MeSecsrxBIqDvPbV7/z65Tf6TN4uyrHxGNxfv7sV9Ysy8cX8FVJLw12n0JTlU4FsFJEhqnqewAicjiwqmHDMgyjMbTqkJc0JbPGlcK1Rfzvhldxu10c4TTXAJx607EcfuFQ/vhhLq075rFV354V0iyMGnwzi/9eSqg4jFqJ755VlQ+f+Qytx2acUqsXr2X14rXM+e4vNt+2Kw+c9QT5K9YRKg7j8XoQlyAuwXJSSfuCPlp3yGP1krVVzhUqCjNzyuwqFcDiuUsZNfhmSgpCWJZFNBzF5XZVeJryBX0cfHbKtbEyKp1hoFtgL9nYBbu7ZRFwmqpWzeHawMwwUMOoXx89O5mxlzyb8M61vFbtc3lj+TMpx/aXemLU/3jvsU+SpltubF6/B1WqpI/OygvSqn0e4hIOHDGYnNbZjLtifMKmqK379WTstLsrfP6zdrichb8vqtA85HK7EJcQyPITDUfpd+BOXP/qZRlfW6DWC8Ko6j/AbiKSg11hVF2+xzCMZunA0/dh5aLVvHbvu7g9LorXJ07dsH51AfFYHI+3+kaDz1/8uskU/mB3Iie60Y3H4tzzyY1s2nMTFsz+j2uH3p60H+Kv6fO446SHuOGVywD77n/ZvOVVzmvFLXpu350z7zyJbtt0ZtOem9T/B6pH6aSC2EREngHeUNUCEektImc2QmyGYTQwEeHUm47ljeVP89DXt9Ftm6qduADtu7ZLq/AHEvYp1JY3RYK2dGmScaJWXPFn+YjH4ly13y2sqib30JTXvmXtCjt9RFF+EbFo4jUF1LLof9DOTb7wh/RGAf0P+Bgo/cv4CxjZQPEYhpEBwewAPXbY3E6FEKw4xsOf5eOse05J+1yDT9gDjy/d1WaTENi8d1f67t+nbueBhPMEXG4XW/frSdtObZjxxay0ZxO/O+ZDACaM/ajCUpSlxCVEIzFeuuPNpNlDm5J0KoD2qvo6zlBQVY2xYWlIwzA2Iv0P2pmb37qCLfp0x5/lY/PeXbn2xUsZcsJeaZ9jryMH2Ll03MmLF7fHRSAnxegYhX9/X8S09+u/z8/ldtFmk1Zc+6I9o7hwbeJO8ET++fVfCtcV8dkLXyXcr5ay6M8lvHzH25y+7aUsnV91tFFTkk41XSQi7XDqURHZDViX+hDDMJqrXYfuzK5Dk6dGLjVr6hw+enYyoZII+xy/J7sP68en46cw5qKn7XH/Ci63JByvH49ZuKIWg0/Yk6/f+j518rZ6ZsUtCtYWMvqo/+Ohb25n+723TblEZHk9d9icSU99Xu28gEgoSiwSY9wV4xn91pX1EXaDSGcUUF9gDLA99vKQHYBjVPXXhg+vIjMKyDCahudHv84b971HpCSMqp2uoc/g7Zn55eyUmTybmkC2n3s/u4kfJs3gpdvfTDpkFexVwV6c9yj3jhjLL5/PSvv87xe8WF/h1lpdRgH9LCKDgG2w0y79qapNp4vfMIxGM2fa3zx6yTP8+WPF9M+hwjC/fPZb0nQOTVWoKMwle1xPn8HbsVXfnvyVIK012G37J117JO27tEuaCiORpjwJDFJUACKyK/Cfqi5T1ZiI7AIcDfwrIqNVtXbL9RiG0Sz99dM/XLnv6KQdppFQJGWqhSZLYeYXyVcoExe069yW4646AoDDzj+QqRN+qLbZyhf0ccg5TXcSGKTuBB4HRABEZCBwNzAeu/3/yYYPzTCMpuR/N76acrSM2+Mmp3U2rmb2FFAdl8vFoz/cVXY332fwdgw9Y5+k7/cFvfiCPnbZf0dOvrFpL56YqgnIXe4u/3jsNNBvAW+JyIwGj8wwjCZl7i/zU+73eN1c98pljLnwaVYvWYOIUJxkTeDmJB6zKCkMMe2Dn/jk+S+xLIv9ThnEPzMWMPfn+cScJwGvz0ObTdtw9l0ns2XfHgkT4yVStK6Iya98w9J5y9hm163Y4/B+eH2NM3M4aSewiMwCdnKaf/4AzlHVr0r3qer2jRJhOaYT2DAyZ+TeNzD7m8SZLb0BL5c8ehZDTx+CqvLXT/N499FJTH55aqOO8GkoOa2ziISjRErs7s9Atp8Bh+5C5y024dPnpxAqCqMoJYUhNtm8AyNuPYEhJ1Y/dPbf3/9j5N43EovECBWFCeYEaN+1HY98ewc5rZMseVkLtVkR7BVgioi8C5QAXzsn2hIzDNQwWpzTbj6uyqLpXr+H3Q7dhTeWPsXQ04cA9uziLfpszvfvTU9Y+Hv9nrRyCjUlhfnFZYU/2J3H0yb+xO6H7coxlx9G0boiivKLsWIWS/9Zzj3Dx3D2jpdz1ymPpFxg557TxlCUX0SoyM7FVFIYYum85Yy/5fUG/0yQogJQ1TuAUdgzgffSDY8KLuDi+ri4iFwhIioi7evjfIZhNJy+++3Ilc9dRPsubXF73ASy/Rw98lBGv3Ml2a0q3q1Oef07igsSDweNhmN4/XWcKdwEhIsjfPHqVMZdOb5K+gsrZrFg1n988crXXHPgbbz98MQqxxesLWT+rP+qHBuLxPjytW8bMPINUv4WVPX7BNv+qo8Li0g3YH9gYX2czzCMhjfo2N0ZeMxulBSG8Gf5cLsTr5T11ZvfJV3iEShLuubyuMpSMjc3qsonz3+Zcu6Aql1RjLviBVTt5Htutwt/lt/uLE/SBO9OMYu6PmWyGn4QuAp4N4MxGIZRQyJCVm6wwraFfyzmySvH8+uU38nKC9ojZoRq1+u1YhadenRg+YJVaa261dQU5Ren9T4rbvHkVS8wbtR4xCUEcwIcf9Xh9BqwFb9/91fFNQQCXg4YPriBIq4oIxWAiAwDFqvqzOraAkXkHOAcgM0226wRojMMoyZWLFzJxbtdS0lBCFW7I7Qmls1f2UCRNS2lTzoaV4rWFfPyHW9z6PkHsGz+CorWlxANR/F43fTs050TrzuqUWJqsApARD4DOiXYdT1wHXBAOudR1Sdx5h3069ev+d0iGMZG7s0HJhIpiTTLO/hMChWH+fCpz3h92VP8OGkGyxespOPm7Znz/d9cc8BtdOvVhWNGHcbm23ZtsBgarAJQ1YRT4ERkB6AHUHr33xX4WUT6q+qyhorHMIyGMef7v8rGwteFL+AjEkovLfPGIlISIRqKsdeRA1jyzzIu7H8NoaIwsUiMOdP+5otXv2H021eyee+utOqQV+8rizV6E5Cq/gZ0LH0tIguAfqpq1hk2jGZo895d+euneRXasWtj72MHsHD2Iv7+ZX61fQcbi+zW2WTl2f0pT1/zIsXrirGcTmUrbhEuDnPdQXfgC3hBhKNHHsLwW4/H5aqfTuJmmLjDMIym5NgrhuFNc7WwVL5+cxoHDB9M163Sm0GbUU7J6fG5cXsTj4Sqjj/Lx5l3nlRWmP8yeVZZ4V+eqhIuiRAuDvPWQx/w6t3v1DrsyjJeAahqd3P3bxjNk6rywZOfEbfiaWUC9WX5cCVJGBcpifD0NS9RuK6wvsOsf87DTiwStye7lfvoHp+nwutSLreLtpu2JpDtp9s2nbny2Qs56Mx9y/Znt8qq9rLh4jBv3Pd+vfW3NP/ZGIZhZMwXr37DpGc+Jxap2AeQbCEYj8fNq4vGcc6OV7Bq0eoq+y3LYt3KggaLt8Eo5LbNweV2Mfi4PYiEo3w6fgoxZ6EZt8dNbtscnpx5P63a5yU8xZGXHMxzN7xKuDic8lJF64qIx+Jpr9GciqkADMOotXfHTipLY1BeaeHvcrvK+gZ8QR9n3X0yua1z2GbXLRJWANFweitzNUWFa4sI5gb48OnP7A3OXboIBHMCPPDl6KSFP9gVwH9/LOaT56fgC3gpWl+csC+kU49N6qXwhybQBGQYRvNVtC75RCiXW+i6zaa07dQaj9dNPBZn3BXjeeKK5xl24YH4m/hiKTWlqhSvLyEajhENx8pGRqlCuCTMxCc/S3m8y+Vi5BPn8uL8R7npzSu49oVL8Acr5l7yZ/k4/8ER9RazqQAMw6i1vY/ZLeni71ZcKVhTRNH6YmJRu608XBxh4hOf8P17P3HqzU07V359ioZjvP/4J/z1U+IVx8pr26kNfffdgSEn7c09n97ETvtsT+uOeWy/Vy9ue+8adj+sSlLPWqt2TeCmxKSDNoymZfXSNZzQ5dyk+8UlCXPl+IM+Xvz3cU7ofDbxZpoLqDb8WT5ueuMK+h+0c6NetzbpoA3DMFKa+vYPKfcnTZQmQqQkwi4H9MHtqd0wyvrm9rqTPs3Ul3BxhIcveLLJzJo2FYBhGCn9+/t/vP3QB3z07GQK1lYcojl/Vu2S+Xq8btp2as0Vz15I1202JZgTsIdPZlC/A/rQqUfH6t9YR2uW5rNg9kIevfRZztr+Mq4Zejs/f/5bg183EdMEZBhGQqrKo5c+y6RnJqOWhdvjRhVunXAVfffbEYDX7p3AM9e+XKM7WrfHRZ/B23HhI2eyWa8uqCqzv/mDJfOW89vXc/jmnR8oWNP4cwFcbhcul9Q5rUVum2zun3ILV+9/G2uXV107y+PzkJUboHh9Sdm1/Fl+LnhoBAef1TCLyJsmIMNogdYuz2fuL/MpKaz52rw/fforHz/3hZ2vJmwvWRguDnPLMfeV5ezZ95SBeGq4uEs8ZvHL5Fmcv8tVTHr2c0SErfttwUu3vclHz0yudeHvDXjq1IRjxS1i0bh9jlouWBbI9vPo9Hvosf3mnDb6uLKF5Ev5Al427dGRonUlFSqacHGYcaPGEwlHK5+yQZkKwDA2QiGnoD65+wWM2udmjt3kLF66/c0a3al//L8vEo7xL15fwjk7XcHv3/1J+85t2axXlxrHp5YSKYkw9qJnKMwv4qZh97Dkn+U1Pk950VAMTy3TMlSObc8j+nPGnSfVuEJRVWZNtZeAPOSc/TnsggPxBrxk5QXxBbzsOnRnrLiVcLEcRVn815I6x18TZiKYYWyEHjp3HD98+DPRcJSoc1f56t0T2LTnJgw5ae+0zmGlWNFr8V9LuWr/27j2pUv4Z8aChO8pXfikdJ2ARNxeD1++9g0zpsxOK6bqlK40VheluXeOufxQXhj9eo2S3IWLI7z5wPt8884PWJbF/qcO4virD2fJ3OV06NaO9p3bMmqfm1k8t2ri41gkTqsOySeKNQTzBGAYG5mSwhK+evP7KoVhqDjMq/dMSPs8+54ykEB28slakVCEZ697Kel+f9DPvZ/eRCAn1YQvpXBtEd4MdwBXltcmh58/+w1Laz5EdcGs//hmwg9899507h0xljEXPk2v/lvSvnNbAI678vAqTUMen4edBm9H205t6iX+dJkKwDA2MkXrikm20F6iTslkdjt0F3Yf1s9ORZyAWsraFeuTHt+uSxu22XVLNkuxoImIMOTkvYmleNrIhMmvTOWGw+4iHq15BVD+iSFUFGbaBz8z+9s/y7YNOLgvp995IoFsf1nTUJ9BvbnulZH1EXqNmArAMDYybTdtQ1Ze1cyS4hJ2HNQ77fO4XC6uffFSbn77yoQZPEWErXfZIml2zzPvPAmAIy8+KOk1Rr99Je27tKVzz0SLB2ZYut0l4qSF9rgTZkQNFYf56dOZFbYddckhvLH8Ge6bPJrn/x7D3R/fSE7r7HoIumZMBWAYGxmXy8VFY87En7Uhj4zb4yKYE+D0208s2zZ/1kK+fut7/p2zKOm5RIT+Q3fm4DP3rXA+2JDc7ZQbjqmQE19cwqHnHcDeR+0GwL+/L0o42SuYE8DldvH+4x+z/N/muS6wP8vHYecdyHn3j+DkG4+pkrsHwOf3ktOqauEeyPKzVd+etO/SrjFCTahpNbwZhlEvBh27O+02bc0rd09g6bzlbL9XL0689kg27bEJJUUhbhx2N39M+xu3x008GmfHQb0Z/faV+AJVCzCAi8aeSeuOrXhnzIcUrSum546bc+HDZ7DlTj3Ycqce7HXUAL568ztEhEHH7s7mvbuVHbt03vKEo15i0TirFq3hzQcmVpsCORFxCYi9yHqmiEvYZtctOHDEPhSsLeT1eydUfY8I+5y4Z+MHlwYzEcwwWpgHzxvHp89PKRsdBPb49EPPO4DzHxhR7fGqiiTrZEjgnTEf8tilzyXcN+zCA3nv0Y/TPlddiVvqtcLwBby8uOBx2nRsBcCML2Yx+qj/Kxv1pKpc/8plDDi4b71dszaSTQQzTwCG0YKoKp+98FWFwh/s4ZMfP/dFWhVATQp/gLy2OYhIwqGgH4xLnSK5vtVX4e/22EtBnv/A8LLCH2CnfbbnjeVP89vXf6CWxQ57b5v0qaopMBWAYbQgqlql8C8VLo40yDWjkTi+oDfh+RM1DTUlbo8bt8eFx+chFokx7IIDcXs9+IJehpywF123rrp+sdfnpe++O2Qg2pozFYBhtCAul4veu2/D7G/+qLBdRNhpn+0AKMwv4qs3vmPt8nXsOKg32+/Vq8Z3/eX13W8He93cZqjjZu05auQhdOreke336pWRkToNyfQBGEYLM3/WQkbudYMzSziGL+DFF/Ax5vs7KVhbxDUH3IZlWYRLIviDPnYY2Jvb3r26TmmbL9z1av76aV49forGE8jyM27mfXTeogkOVU2T6QMwDAOAHttvxrNzHub9xz/mnxkL2LrfFhx67v606pDHSZudR3HBhsRxoaIwv075nUnPTObQc/ev9TV3PWhn/v5lfvL1ATLEH/TRqkMeqxavSZryIVQSZsKYSex36kAmjJ3E6sVrGHDILhx05hCCOcFGjrh+mScAwzAA+GfmAkbufSOhwlCVfb36b8mY7++q9bmXzlvO8K0ubtCFUDbbtguL/lpao9w93oCXWydcxX1nPMb61QVJF6XPa5dDuCRCJBRFLcUf9NGucxsem34P2QnG+Dc1Jh20YRgZs2nPTQjmNezdcqgozIT85/HWID11NBTlicuf5/m/xzD09CFJ37d+dSHh4kjZE0y4JMKqxWt499GP6hx3JpkKwDAMAHrssBnZCQppf5afA1MUjunqO6RhR8asWLiKqW9P4+JHz8If9KXdcb103nLWry5k5313SJjKIZlIKMrXb02rbbhNgqkADMMA7BFCN791JVm5QQLZflwuIZDtZ8dBvTnozLpXACNuP4FgTiB5IVv7gUZl3npoIn9N/4esVkGCuQFy2mSnfd7t9+qFqwYVAEBeu9xaRNl0mE5gwzDKbDtgK15e+DhTXq+/YaClNt+2K2Om3cX40a/zy2e/UphfjMstqNoLsseisZQTtdxuF3HLSpmkbf6vC1nw238J5xd4vG4762i540Wg8xad6NDVzsdz+EUH8fZDHyQ8t7ikQid2INvPkZccXM2nbtoy1gksIhcDFwEx4ANVvaq6Y0wnsGFsPFYsXMmU178jEooy/7d/mfrONOKxSh24Yq/V6/G42e+UgRx+0UG8cNsbTE3Q9FK6elfSTmCBXv234t/Z/xEqDuPP8uP1eXhgyq10325D7qKHL3iSD8Z9VqHD2uv3kNcul+L1JbjcQiQU48Rrj+TUm46t+w+iESTrBM5IBSAi+wDXA4eoalhEOqrqiuqOMxWAYTR9oeIwHz07malvTyO3bQ7DLjiQnatp/79iyGhmfll1VbCsvCCjnrmAPYb146dPZjL24mdZvnAlbq8bK2ahqvaonCw/alnVrgh27n2n0XPHzZkz7W86dG3H3sfsRjA7UOE9sWiM47ucw/pVBRW2+4M+rnnxErx+L736b0mr9o27elddNLV5AOcDd6tqGCCdwt8wjKYvXBLm0j2uZ/HcpWWpH36Y9AsHnTmEIy4+mK5bbZrwuB0Gbsvv3/1VJU1FNByj925b8ce0v7n12PvLCviYM1yzU8+O9Nxxc/rt34ffvp7DlNe/xUox12D9mgL67rcjfffbMel7fv7st7LzlxePxflj2t+cdfcpqX8IzUimOoG3BvYWkWkiMkVEds1QHIZh1KPPXviKJXOXVcj7EymJ8O7Yjzi3zyiGb30xc77/s8pxh184lGBuoMIi7P4sP/udMpD2Xdrxwm1vJry7XzZvBUePPJTDzj+Q4648HG+S1ctK7XfqoGo/w7qV6xPOV4hF46xeurba45uTBqsAROQzEZmV4Otw7CePNsBuwJXA65Kkl0lEzhGR6SIyfeXK5rlohGG0FN++N51Qktz+kVCUJXOXccmeN3DdIXdUmHHcukMrHp9+D/ucuBet2ufReYtNOPPOExk57hwAFqZYtOalO94CYMude3DdSyPJbZeT8H0ut4uFv1c8T2F+EZNfmcrkl7+mYG0hANvv3SthJ3Igx8+uQ3dO8embnwZrAlLV/ZLtE5HzgbfVrmZ/EBELaA9UKeFV9UngSbD7ABooXMMw6kFWbhqTvdRuZrn/zMe48fVRZZs7btaBa8ZfnPCQzlt0YtWiNQn3zft1IRfvfh2rl6yhZ5/unH7bCYy74oUqi8xYcYuP//cFex05AIApb3zH/40Yay9pqRCPWVz21Lnsd/JADjprXz5+7gtCRfY5/EEf3bbpwt5HD0jnx1DBsgUrmPzy15QUhhlwSF+222ObehlVVR8y1QcwARgCfCkiWwM+YFWGYjEMo5506Jbe8obxaJzv3p9O0fpishOsX1zZ6bedwGUDb0q4L395PvnL8wFYuWg1P3/6Ky534gJ29RK7CWfNsrXcO2IskZKKKaofPGccO+69LRc+fAZ9Bm3He49/TFF+MfucuCfDLjgQry91E1Nlk1/5mvvPegIrFicWizNhzIfseeQArn7+oiZRCWSqD+BZoKeIzAJeBYZrc0pKZBhGQrltcki3XHO5XRSuLUrrvdvvtS1DTtqr+pm6ip3lNMlooAW/LeTXr35POoNXLYspb3yPiBAJR1kw6z8W/P4f/7vpNZ666kWikdSjjMorWlfE/Wc9QaQkQixqzz8IFYX55p1p/PjRjLTP05AyUgGoakRVT1HV7VW1r6pOzkQchmHUr12H7oQvwcLoiQRzgmk/MQBcPf5izrt/OB26tsPjc6ec4SvuxEVbNBLj5TvfJhKKYiVo54/HLCIlEX76dCYPnvME+SvWEQ1FiZRE+Oi5yTx6ybNpx/vz57PweKum0A4Vhfni1alpn6chmVQQhmHUmy137sG+J+9NINtfts3lcYqZcgW2P8vPRWPOxOVKvwhyuVwcdekhvLzwCU67+fiUaRvy2uYSqDS+v9SSucsYcEjfhOsbeP0eBhzalxdve7PKCmbh4gifjp9SofM6FXeSSkgEPHVYW6E+mVQQhmHUq5FPnMueRwzg0xemgMJ+pw6k+/bdeOO+9/jtqzl06tGR468+gt67bV3ra+w4qDcutxsrXnW8vriEIy8Zykt3vFNln8sl9Oq/JZv16sIRlxzMhDGTnH4AxR/0M/TMIWy5Uw+WzU88NcnldrFu5fq0Orv77r9jwvUP/EE/+51W/XDUxmDWAzAMo9kpLizhyNYjsKzEaR+223Mbdh26M6/e/U7ZSB6AQE6AsdPuYvNtuwLw+3d/8vnLX6OWMuTEvdh+r20BGH30//HthB+rzAfIyg3y5spn0u4M/vHjGdxy9H2I2M1LInDExQdx9j2n1uZj11pTmwlsGIZRa/N/XYg/20dJQdXFawDm/jKfSx47mw5d2/HKXe+wdkU+vXfbhrPvObms8Afovfs29N59myrHD7/leH76ZCbh4jCldUAg28+po4+r0UigXQ/ciVf+e4Jv3vmBUFGYfkN3SjobOhNMBWAYRpM288vZPDryWRbM+o/cNjkcM+owdh/WL+XKXx6Pm2XzV3DA8MEcMHxwja/ZY/vNeGjq7Txz7cv88cPftOvchpOvP5rBx+9Z43Pltslh6Bl1T6fdEEwFYBhGk/Xnj3O5/tA7yzpk168u4KXb36JgTQFdt+7MPzMXJEwPHY3E6LHDZnW69hZ9unPnh9fV6RxNnRkFZBhGk/X8za8lGI0T5t1HP+aG1y6je+9uVY7xB33seUR/Nu2xSWOF2WyZJwDDMJqs2d9WTRwH9lBKtZSnfnuA6Z/M5JW73mbuLwsI5gQYdsEBHH/VEY0baDNlKgDDMJqkksISQoWJO3kj4SjtnVW8+h3Qh34H9GnM0DYapgnIMIwmadn8FfgCiWcVZ+cGqyzkYtSceQIwDKNJat+1HfEkI31679mLGV/MYuo70+yJVacOpMf2dev0bYnME4BhGE1Sbpsc9jtlb/yVcgv5gz6suMWNw+7mvUc/4s0H3ueiAdfy7qOTMhRp82UqAMMwmqxLHjubQ88/gEC2H5fbxaY9N+HkG45m1tdzCBXZk7SsuJ3A7ckrX2DtinWZDrlZMRWAYRhNlsfr4bz7hjMh/3neXTee8XPHsvzflRXSO5RyedxM/3hG4wfZjJk+AMMwmjy32407y86g6Qv4cLmkyuLvIoLPX7MFW1o68wRgGEazsv9pg/AmKOjVsuh/8Ma1Zm9DMxWAYRjNylZ9e3Lq6OPwBbwEsv0EcwMEsvzc9OYVBHPSWJPYKGPSQRuG0SytWrya6R/PxBfwMuDQXdJaW7ilMumgDcPYqLTv0q7JZtlsLkwTkGEYRgtlKgDDMIwWylQAhmEYLZSpAAzDMFooUwEYhmG0UM1qGKiIrASKgFWZjiWB9pi4aqqpxmbiqhkTV81kIq7NVbVD5Y3NqgIAEJHpicazZpqJq+aaamwmrpoxcdVMU4rLNAEZhmG0UKYCMAzDaKGaYwXwZKYDSMLEVXNNNTYTV82YuGqmycTV7PoADMMwjPrRHJ8ADMMwjHpgKgDDMIwWqtlUACJyrIjMFhFLRPpV2netiMwVkT9F5MAMxriTiHwvIjNEZLqI9M9ULJWJyMXOz2e2iNyb6XjKE5ErRERFpH2mYwEQkf8TkT9E5FcReUdEWmc4nqHO726uiFyTyVjKE5FuIvKFiMxx/q4uzXRMpUTELSK/iMjETMdSnoi0FpE3nb+vOSKyeybjaTYVADALOAr4qvxGEekNnABsBwwFHhMRd+OHB8C9wC2quhNwk/M640RkH+BwYEdV3Q64L8MhlRGRbsD+wMJMx1LOp8D2qroj8BdwbaYCcf6WHwUOAnoDJzp/801BDBilqtsCuwEXNqHYLgXmZDqIBB4GPlLVXkAfMhxjs6kAVHWOqv6ZYNfhwKuqGlbV+cBcIFN33grkOd+3ApZkKI7KzgfuVtUwgKquyHA85T0IXIX9s2sSVPUTVY05L78HumYwnP7AXFWdp6oR4FXsv/mMU9Wlqvqz830BdmHWJbNRgYh0BQ4Bns50LOWJSB4wEHgGQFUjqpqfyZiaTQWQQhfgv3KvF5G5P8KRwP+JyH/Yd9kZu3OsZGtgbxGZJiJTRGTXTAcEICLDgMWqOjPTsaRwBjApg9dvSn/fSYlId2BnYFqGQwF4CPumwspwHJX1BFYCzznNU0+LSHYmA2pSK4KJyGdApwS7rlfVd5MdlmBbg91NpooR2Be4TFXfEpHjsGv6/RoqlhrE5QHaYD+m7wq8LiI9tRHGAFcT13XAAQ0dQyLp/K2JyPXYzRwvNWZslTTq33dtiEgO8BYwUlXXZziWQ4EVqvqTiAzOZCwJeIC+wMWqOk1EHgauAW7MZEBNhqrWprBcBHQr97orDdj0kipGERmP3fYI8AaN+AhaTVznA287Bf4PImJhJ6Ramam4RGQHoAcwU0TA/r39LCL9VXVZpuIqF99w4FBg38aoKFNo1L/vmhIRL3bh/5Kqvp3peIA9gWEicjAQAPJE5EVVPSXDcYH9u1ykqqVPSW9iVwAZszE0Ab0HnCAifhHpAWwF/JChWJYAg5zvhwB/ZyiOyiZgx4OIbA34yHCWRFX9TVU7qmp3Ve2O/Z+jb2MU/tURkaHA1cAwVS3OcDg/AluJSA8R8WEPeHgvwzEBIHbN/QwwR1UfyHQ8AKp6rap2df6mTgAmN5HCH+dv+z8R2cbZtC/wewZDalpPAKmIyJHAGKAD8IGIzFDVA1V1toi8jv2DjAEXqmo8Q2GeDTwsIh4gBJyToTgqexZ4VkRmARFgeIbvapu6sYAf+NR5OvleVc/LRCCqGhORi4CPATfwrKrOzkQsCewJnAr8JiIznG3XqeqHmQupybsYeMmpzOcBp2cyGJMKwjAMo4XaGJqADMMwjFowFYBhGEYLZSoAwzCMFspUAIZhGC2UqQAMwzBaKFMBGBsVEYk72VhLv7qLyLc1PMdIEclKsu9LJzNn6fmPqUWMOzkTlQwjo8wwUGOjIiKFqpqTxvvcyeaLiMgCoJ+qVpksJyJfAleo6vQ6xDjCOf9FNThGsP+/NrX8NkYzZp4AjI2eiBQ6/w528te/jD15KVtEPhCRmSIyS0SOF5FLgM7AFyLyRZrn7yAib4nIj87Xns72/iLyrZP461sR2caZAHQrcLzzBHG8iIwWkSvKnW+W8+TS3ckZ/xjwM9BNRK50rvGriNxSzz8qo4VpNjOBDSNNwXKzUuer6pGV9vfHzvU/X0SOBpao6iEAItJKVdeJyOXAPomeABwviUiJ8/2+2DneH1TVqSKyGfas3W2BP4CBzmze/YA7VfVoEbmJck8AIjI6xefZBjhdVS8QkQOwU530x04S956IDFTVr1IcbxhJmQrA2NiUOAvyJPODs24EwG/AfSJyDzBRVb9O8xonl28Ccgr33k7aCLATkOVirwnxvIhshZ3B01uDz1HqX1X93vn+AOfrF+d1DnaFYCoAo1ZMBWC0NEWl36jqXyKyC3AwcJeIfKKqt9binC5gd1UtKb9RRMYAX6jqkU6+/C+THB+jYnNsIFG82Hf9d6nquFrEaBhVmD4Ao8USkc5Asaq+iL2AT19nVwGQW4NTfQKUdeiKyE7Ot62Axc73I8q9v/L5F5ReW0T6YqfJTuRj4Awn/z4i0kVEOtYgTsOowFQARku2A/b6CDOwF6i53dn+JDAp3U5g4BKgn9Mx+ztQmjn0Xuwni2+wM3mW+gK7yWiGiByPnU+/rRPH+djrEFehqp8ALwPfichv2Pnka1JRGUYFZhioYRhGC2WeAAzDMFooUwEYhmG0UKYCMAzDaKFMBWAYhtFCmQrAMAyjhTIVgGEYRgtlKgDDMIwW6v8BEDvH+i5+xz0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(X_train[:,0],X_train[:,1],c=y_train)\n",
    "plt.xlabel(\"First Feature\")\n",
    "plt.ylabel(\"Second Feature\")\n",
    "plt.title(\"Second Against First Feature\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78934bea",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e96e31b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LogisticRegression:\n",
    "    \n",
    "    def __init__(self,learning_rate= 0.001,n_iter=10000,verbose=False):\n",
    "        self.learning_rate = learning_rate\n",
    "        self.n_iter = n_iter\n",
    "        self.verbose = False\n",
    "    \n",
    "    def predict(self,X):\n",
    "        \"\"\"\"\n",
    "        predict output an hard prediction namely 0 or 1\n",
    "        \"\"\" \n",
    "        z = np.dot(self.w,X.T) + self.b\n",
    "        return self._sigmoid(z) > 0.5\n",
    "    \n",
    "    def predict_proba(self,X):\n",
    "        \"\"\"\"\n",
    "        predict_proba output soft predictions of probabilities between 0 and 1\n",
    "        \"\"\" \n",
    "        z = np.dot(self.w,X.T) + self.b\n",
    "        return self._sigmoid(z) \n",
    "        \n",
    "    def fit(self,X,y):\n",
    "        \"\"\"\n",
    "        Trains the logistic regression model\n",
    "        \"\"\"\n",
    "        m_rows,n_features  = X.shape # (number of examples, number of features )\n",
    "\n",
    "        ## initialize  the set of weights and bias to some random value\n",
    "        self.w =np.zeros((n_features,))\n",
    "        self.b = 0\n",
    "        costs= []\n",
    "        \n",
    "        for i in range(self.n_iter):\n",
    "            dj_dw,dj_db = self.gradient(X,y)\n",
    "            self.w = self.w -  self.learning_rate * dj_dw\n",
    "            self.b = self.b -  self.learning_rate * dj_db\n",
    "            \n",
    "            z = np.dot(self.w,X.T) + self.b\n",
    "            pred = self._sigmoid(z)\n",
    "            cost =self._cost_function(pred,y)\n",
    "            costs.append(cost)\n",
    "            \n",
    "            if self.verbose and i % 500 == 0: # print out the value of our costs function at every 500 iterations\n",
    "                print(f\"Cost as iteration {i}: {cost}\")\n",
    "        \n",
    "    def gradient(self,X,y):\n",
    "        m_rows,n_features  = X.shape # (number of examples, number of features )\n",
    "        dj_dw = np.zeros((n_features,))\n",
    "        dj_db = 0\n",
    "        \n",
    "        \n",
    "        pred = self.predict_proba(X)\n",
    "        err = pred - y\n",
    "        \n",
    "        for i in range(n_features):\n",
    "            dj_dw[i] = np.sum( err * X[:,i])\n",
    "        dj_db = np.sum(err)\n",
    "        \n",
    "        dj_dw = dj_dw/ m_rows\n",
    "        dj_db = dj_db/ m_rows\n",
    "        \n",
    "        return (dj_dw,dj_db)\n",
    "    \n",
    "    def _sigmoid(self,z):\n",
    "        \"\"\"\"\n",
    "        Function to calculate the sigmoid function\n",
    "        \"\"\" \n",
    "        return 1/(1+np.exp(-z)) \n",
    "           \n",
    "    def _cost_function(self,y_hat,y):\n",
    "        \"\"\"\"\n",
    "        Function to calculate the cost \n",
    "        \"\"\" \n",
    "        m_rows = y_hat.shape\n",
    "        first_part = y * np.log(y_hat)\n",
    "        second_part = (1 -y) * np.log(1 - y_hat)\n",
    "       \n",
    "        cost = -1 * np.sum(first_part  - second_part) / m_rows\n",
    "        return cost\n",
    "    \n",
    "    def score(self,pred,y):\n",
    "        return np.sum(pred==y)/len(y) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7255a211",
   "metadata": {},
   "source": [
    "## Training our model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2a629fa4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy on the training set: 1.00\n"
     ]
    }
   ],
   "source": [
    "## Instantializing our model\n",
    "model = LogisticRegression()\n",
    "model.fit(X_train,y_train)\n",
    "y_pred_train = model.predict(X_train)\n",
    "\n",
    "##calculating the accuracy of our model on the training set\n",
    "acc_train=model.score(y_pred_train,y_train)\n",
    "\n",
    "print(f\"accuracy on the training set: {acc_train :.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2119b95e",
   "metadata": {},
   "source": [
    "## Testing our model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e8a8b863",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy on the testing set: 1.0\n"
     ]
    }
   ],
   "source": [
    "## calling predict on the test set\n",
    "y_pred_test=model.predict(X_test)\n",
    "\n",
    "## calculating the accuracy of our model \n",
    "acc_test = model.score(y_pred_test,y_test)\n",
    "\n",
    "print(f\"accuracy on the testing set: {acc_test}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42a10244",
   "metadata": {},
   "source": [
    "Our model has an accuracy of 1.0 on both the training and test set because the dataset is linearly seperable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df81bd61",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}