\n",
"\n",
"In this exercise we will implement linear regression and see it work on data\n",
"\n",
"Files included with this exercise:\n",
"\n",
" - ex1data1.txt - Dataset for linear regression with one variable\n",
" - ex1data2.txt - Dataset for linear regression with multiple variables\n",
" \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# used for manipulating directory paths\n",
"import os\n",
"\n",
"# Scientific and vector computation for python\n",
"import numpy as np\n",
"\n",
"# Plotting library\n",
"from matplotlib import pyplot\n",
"from mpl_toolkits.mplot3d import Axes3D # needed to plot 3-D surfaces\n",
"import matplotlib.patches as mpatches \n",
"import matplotlib.lines as mlines # for creating a legend\n",
"\n",
"# tells matplotlib to embed plots within the notebook\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
1 Simple Python function
\n",
"\n",
"We will warmup by creating a function which returns an n x n identity matrix"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def warmupexercise(x):\n",
" A = np.identity(x)\n",
" return A"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can run the function with an input of 5 to create a 5 x 5 identity matrix"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[1., 0., 0., 0., 0.],\n",
" [0., 1., 0., 0., 0.],\n",
" [0., 0., 1., 0., 0.],\n",
" [0., 0., 0., 1., 0.],\n",
" [0., 0., 0., 0., 1.]])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"warmupexercise(5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
2 Linear Regression with One Variable
\n",
"\n",
"In this part of this exercise, we will implement linear regression with one variable to predict profits for a food truck. Suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet. The chain already has trucks in various cities and we have data for profits and populations from the cities.\n",
"\n",
"We would like to use this data to help you select which city to expand to next. The file ex1data1.txt contains the dataset for our linear regression problem. The first column is the population of a city and the second column is the profit of a food truck in that city. A negative value for profit indicates a loss. \n",
"\n",
"We now load the data"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# Read comma separated data\n",
"data = np.loadtxt(os.path.join('Data', 'ex1data1.txt'), delimiter=',')\n",
"X, y = data[:, 0], data[:, 1]\n",
"\n",
"m = y.size # number of training examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
2.1 Plotting the Data
\n",
"\n",
"Before starting on the task it is useful to visualize the data. Since we are dealing with only two variables, we can do this with a scatterplot."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"def plotData(X,y):\n",
" red_x = pyplot.plot(X, y, 'ro', ms=10, mec='k')\n",
" pyplot.title('Figure 1: Scatter Plot of Training Data')\n",
" pyplot.xlabel('Population of City in 10,000s')\n",
" pyplot.ylabel('Profit in $10,000s')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEWCAYAAABv+EDhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2de5xVVdn4v8/MHGWm4SAwQF5CasouKnmZFKOLXX7llKIVZaHiBcS4+AZiAl3t7c3qLbQLailTXgjTyIqMqYyyLKQaLEcN0zm95Q0VL+EghAd4fn+sdeDM4ex99pk59/N8P5/1mXP2XmuvZ++zZz1rPetZzxJVxTAMw6g/GsotgGEYhlEeTAEYhmHUKaYADMMw6hRTAIZhGHWKKQDDMIw6xRSAYRhGnWIKoAIRkfEislVEGssti5EdETlHRH5forpeLSJ/EZF+EfmvEtT3dxF5c6HzGpWHKYAyIiL/FJHtvrFPpYNU9WFVbVXVXRUg434issrLqiJyYp7lDxeRX4rIcyLybxHZICLvGaJMJ4rIoxnHLhWRFUO5bpZ6LhWRpP9d/i0i60TkhEFc5w4RmTkEUS4B7lDV4ar6jYxr35/27uwSkf+kff/EYCpT1Ver6p2FzpsPIjLT30/qXv5PRL4jIq/K4xorROTSQstWS5gCKD+n+MY+lR4vZmUi0jSIYr8HzgSeGETZnwK3A+OAscB/Ac8P4jpFJeS53KyqrcAY3HO4VUSkdJIBcChwf7YTqnp46t0B7gTmpb1Ll2XmH+TvXy7u9Pc1AngnkAR6ROS15RWrhlBVS2VKwD+Bd2Y5PgFQoMl/fznwO6Af+BVwJbDCnzsReDTousClwCpgBa7hnYlT/IuBBPAMcAswKoK8jwIn5nF/bf4+DgjJcyrwVy9bAjjJHz8X2Ojv+R/ABf74S4DtwG5gq0/TgBdxDcRW4B6fdwTQBWwCHgP+B2j0584B/gBcATwL/E8W2S5NPWf//XB/P22+/O/Tzr0R+DOwxf99oz/+BWAX8B8v27KA5zAF18j/G7gDeK0//uuM8oeFPMs7gJkZx2b6d+cb/j4vBV4F/Mb/9k8DNwIjsv3O/pnd5N+ffuA+4JhB5u3wv3U/8H3gB8ClAfcyEzfqyTz+c+D7/nMD7t1+Istzm+Pfhxf9c/uRP/4p3PvU75/3lHK3A+VMNgKoDlYCfwJG4/6Bz8qz/Km4f5QDgO/heuGnAW8FDgKewymVvBGRaSLSG3D6GaAPWCEip4nIuIyyxwE3AB/3sr0Fp7wAngJOBuI4ZXCFiByjqi8AncDjurenuxK4DN9bV9XX+2tcD+wEXgkcDbwL17CkOB7XGIzFNdRh97k/rtF/VFWfzjg3CvgZrpEdDVwO/ExERqvqJxnYM5+X5dqH4RrO+biRxhrgpyKyn6q+PaP8g2FyBvBGnDIdA3wZEFxjfSDwOuAVwKdDyp+GUxIHAN3+PvPK65/fj4HlwCjghz5vvtwKpM853IZTaC/FKZwbAVT1KuBm4DL/3N7n8z8ITMZ1Dr4ArMx8L+sJUwDl58fevvxvEflx5kkRGQ+8AfiMqr6oqr8HVudZx12q+mNV3a2q24ELgE+q6qOqugOnVKYOxjygqitVdWLAOQXehmvUlwKbROR3aXbcGcB3VPV2L9tjqvqAL/szVU2o47fALxn4jx+K/6fuBOar6guq+hSut//htGyPq+o3VXWnfy7Z+JCI/Bt4BDiW7I3We4GHVPVGf62bgAeAUyKKezrwM/8cksBXgWZcw10IHlbVq1V1l6puV9UHVXWtf59Sz+WtIeV/q6q/UDcndSNw1CDyTgZ2q+oyVU2q6g+ADYO4l8dxCgT/zlynqv2q+h/ce3ysiLwkqLCq3qKqm3zZlbh3s2MQctQE1WQPrFVOU9VfhZw/CHhWVbelHXsEeFkedTyS8f1Q4Ecisjvt2C6cnf6xPK6bE1V9FJgHICIvA67B9fpPwN3DmmzlRKQT+CxwGK6j0gLcm0fVhwIxnNJJHWtg4LPIfC7ZuEVVz8yR5yDgXxnH/gUcHOH6+5RX1d0i8kge5XMx4D5F5KW4nvlkYDjuuWwOKZ8+97MNZ4bLN+9BOHNRoFwRORhnysJ7yX0RmIozy6Xe5zbghWyFReQcYAHu/QBo9fnrEhsBVD6bgFEi0pJ2LL3xfwHXOAJ7/inGZFwjM+TrI0Cnqh6QloapakEb/0xU9RGcqemINDnaM/N5c8EPcT3hcap6AE5RpFrybCFss93jDqAt7R7jqnp4SJnB8jh7G5QU49mrTHPVM6C8n2R+GYVTxpn1fxn3bI5U1TjOtFXsie1NwCEZx/LpxKQ4DWcSA5gOvAd4O86k80p/POt7IiKvAK4GZgOj/Xv1AMW/94rFFECFo6r/AnqAS71L5gkMNC08CAwTkfeKSAw3ybV/jst+C/iCiBwKICJjROTUoMwisr+IDPNf9xORYVE8YURkpIh8TkReKSINItIGnAes91m6gHNF5B3+/MEi8hpgP38Pm4GdfjTwrrRLPwmMFpERGccmiEgDgKpuwpmNlopI3F+/XUTCTB2DZQ1wmJ8PaRKR03G29dvSZHtFSPlbgPf65xADFuIa6HVFkBVcr/8FYIsflV1cpHrS+T3QKCKz/TP6AM6klhMRaRSRV4jIVcCbgM/7U8Nxz+kZXCcocx4n87m34pTCZndZmQm8ZrA3VAuYAqgOzsCZTJ7BTd7djHvxUdUtOI+H5bge4wvsO9TO5Ou4eYRfikg/rkE+PiT/33GeNwcDv/CfU8rjDBHJ6qKI88CYgPNceh43SbcD1+NEVf+En+DFec/8FjhUVftxE9W34Caop5E27+HnCW4C/uHnTg7CeZQAPCMid/vP03HK5G/+OqtwE58FRVWfwU1YL8T9RpcAJ6dNFn8dN8fynIjsM4Gqqn/Hudl+E+eVcwrOPfjFQsvq+SxwHO6Zr8aNtoqKn2t6H/BR3G/xIZzi3BFS7M0ishX37vwa18h3qGrqffsubvT0OM6jJ1NhLgde75/7KlXtxZm+/oQbkbwG+GMBbq9qETdPZ1QTInIz8ICqfrbcshjGYBGRDcDXVPXGcstSr9gIoAoQkTd480WDiJyEc+vcx2PIMCoZcSu4x3kT0AxcD/yX5ZarnjEvoOrgpTj/59E4885sVf1LeUUyjLx5Lc58+RLcor8PqOqT5RWpvjETkGEYRp1iJiDDMIw6pSpMQG1tbTphwoRyi2EYhlFVbNiw4WlVzVwXtIeiKQDvX3wDzn69G7hGVb/uw7Oez96Vh59Q1ayrQVNMmDCBnp6eYolqGIZRk4hI5gr1ARRzBLATWKiqd4vIcGCDiNzuz12hql8tYt2GYRhGDoqmAPxKzE3+c7+IbKRwsU0MwzCMIVKSSWARmYALx5tadTdPRHr9Dj8jA8rMEpEeEenZvDksTpVhGIYxGIquAESkFbfUfL6qPo8LxtSOCxO7CRcmeB9U9RpV7VDVjjFjAucwDMMwao5EIsGCOXMYF4/T2NDAuHicBXPmkEgkClpPURWAD2z1Q+B7qnorgKo+6eOS7wauxcUkMQzDMIDu7m4mTZxI8/LlrOvvZ4cq6/r7aV6+nEkTJ9Ld3V2wuoqmAHy0yC5go6pennY8PRjX+3ABwgzDMOqeRCLB9KlTWb1tG5clk7TjJmrbgcuSSVZv28b0qVMLNhIo5ghgMm7rwreLyF99eg/wvyJyr99G8G24zRkMwzDqnmVLl3J+MskJAedPAGYmk1x5xRUFqa8qQkF0dHSorQMwDKPWGRePs66/f99dktJIAJPjcZ7YsiXn9URkg6oGbnlpoSAMwzAqhKe3bt1na7lMxvt8hcAUgGEYRoXQ1tq6z+bSmTzs8xUCUwCGYRgVwrQzz6QrFgvNszwWY9pZZxWkPlMAhmEYFcK8hQu5NhbjroDzd+EUwNwFhfGdMQVgGIZRIbS3t3PDqlVMaWlhSSxGAkjiJn6XxGJMaWnhhlWraG8PmyaOjikAwzCMCqKzs5P1vb3smDWLyfE4zQ0NTI7H2TFrFut7e+ns7CxYXeYGahhG1ZBIJFi2dCkrV6zg6a1baWttZdqZZzJv4cKC9YprCXMDNQyjJihliIR6wUYAhmFUPIlEgkkTJ7J627asq2TvAqa0tLC+t9dGAmnYCMAwjKqn1CES6gVTAIZhVDwrV6xgRjIZmmdmMsnKG28skUS1gSkAwzAqnlKHSKgXTAEYhlHxlDpEQr1gCsAwjIqnFCESSrULVyVhCsAwjIqn2CES6tXF1BSAYRgVTzFDJJR6F65KwhSAYRhVQbFCJNSzi6ktBDMMo64p9C5clYQtBDMMwwgh08U0gduofBzQ6P9+A9jc318G6YqLKQDDMOqadBfTbmAS0AysA3b4vy3AMNWamww2BWAYRl2TcjFNANOB1cBlMGAy+IvAr6DmJoNNARiGUdekXEw/BZwPdTUZbArAMIy6JuVi+hNgRo68tRZvyBSAYRh1T2dnJztE6i7ekCkAwzAM6jPekCkAwzAMShNvqNIwBWAYhkHx4w1VIqYADMMwKG68oUqlaApARF4mIr8RkY0icr+IfMwfHyUit4vIQ/7vyGLJYBiGkQ/FijdUqRQtFpCIHAgcqKp3i8hwYANwGnAO8KyqfklEFgMjVXVR2LUsFpBhGEb+lC0WkKpuUtW7/ed+YCNwMHAqcL3Pdj1OKRiGYRglpiRzACIyATga+CMwTlU3gVMSwNiAMrNEpEdEejZv3lwKMQ3DMOqKoisAEWkFfgjMV9Xno5ZT1WtUtUNVO8aMGVM8AQ3DMOqUoioAEYnhGv/vqeqt/vCTfn4gNU/wVDFlMAzDMLJTTC8gAbqAjap6edqp1cDZ/vPZwE+KJYNhGIYRTFMRrz0ZOAu4V0T+6o99AvgScIuIzMCtrP5gEWUwDMMwAiiaAlDV3wMScPodxarXMAzDiIatBDYMw6hTTAEYhmHUKaYADMMw6hRTAIZhGHWKKQDDMIwCkkgkWDBnDuPicRobGhgXj7NgzpyK3EzeFIBhGEaB6O7uZtLEiTQvX866/n52qLKuv5/m5cuZNHEi3d3d5RZxAEWLBlpILBqoYRiVTiKRYNLEiazeto0Tspy/C5jS0sL63t6S7SlQtmigtUI1DecMwygfy5Yu5fxkMmvjD3ACMDOZ5MorriilWKGYAgih2oZzhmGUj5UrVjAjmQzNMzOZZOWNN5ZIotyYCSiAShzOGYZRuTQ2NLBDNTS8QhJobmhg565dJZHJTECDpBqHc4ZhlI+21lb+lSPPwz5fpWAKIIBqHM4ZRq1TyXNy0848k65YLDTP8liMaWedVSKJcmMKIICnt27l0Bx5xvt8hmEUn0qfk5u3cCHXxmLcFXD+LpwCmLtgQSnFCsUUQADVOJwzjFolkUgwfepUVm/bxmXJJO24UMbtwGXJJKu3bWP61KllHQm0t7dzw6pVTGlpYUksRgJn808AS2IxprS0cMOqVRU1Z2gKIIBqHM4ZRq1SLXNynZ2drO/tZcesWUyOx2luaGByPM6OWbNY39tLZ2dnWeXLxLyAAjAvIMOoHMbF46zr7yfsPy0BTI7HeWLLllKJVfGYF9AgqcbhnGHUKjYnVxxMAYRQbcM5w6hVbE6uOJgCyEF7ezuXL1vGE1u2sHPXLp7YsoXLly2znr9hlBCbkysOpgAMw6h4qtHFshqIpABE5IMiMtx//pSI3CoixxRXNMMwDIfNyRWHqCOAT6tqv4i8CXg3cD1wdfHEMgzDGIjNyRWeqAogFbnovcDVqvoTYL/iiGQYtUElhy2oVmxOrrBEVQCPici3gQ8Ba0Rk/zzKGkbdUelhCwwDIi4EE5EW4CTgXlV9SEQOBI5U1V8WW0CwHcGM6sIWERqVwpAXgomIAEf6r0eIyPHAE6Vq/A2j2qiWsAWGEToCEJF3AVcBDwGP+cOHAK8E5tgIwDD2xcIWGJVCrhFA2OY1AF8H3qmq/8y46MuBNcBrQyr+DnAy8JSqHuGPXQqcD2z22T6hqmtyyGAYVYWFLTCqhVwmoCbg0SzHHwPCl+XBdbh5g0yuUNWjfLLG36hIhuLBY2ELjGohlwL4DvBnEVkkItN8WgT8EegKK6iqvwOeLZCchlEyhurBY2ELjGohpxeQiLwOmAIcDAhuRLBaVf+W8+IiE4DbMkxA5wDPAz3AQlV9Ltd1bA7AKBWF8OAxLyCjUhiyF5Cq/k1VvwR8Frci+EtRGv8ArsZt4nMUsAlYGpRRRGaJSI+I9GzevDkom2EUlEJ48FjYAqNaCFUAIjJeRL4vIk/hzD5/EpGn/LEJ+Vamqk+q6i5V3Q1cCxwXkvcaVe1Q1Y4xY8bkW5VhDIqVK1YwI5kMzTMzmWTljTeG5rGwBUY1kMsN9C7ga8AqVd3ljzUCHwTmq+qk0IvvawI6UFU3+c8LgONV9cO5hDQTkFEqGhsa2KEa6h6XBJobGti5a1dILsMoP0M1AbWp6s2pxh/A9+C/D4zOUfFNOHPnq0XkURGZAfyviNwrIr3A2wCL3WpUFObBY9QTuRTABhG5SkSOF5GDfDpeRK4C/hJWUFU/oqoHqmpMVQ9R1S5VPUtVj1TViao6JTUaMAqDBR8bOubBY9QTuRTAdOBe4HPAL4Bf+s/3AfYfUEFY8LHCYBuPGPVEpGBw5cbmAMIxt8PBkUgkWLZ0KStXrODprVtpa21l2plncmRHB4suvJCZySQzk0nG48w+y2Mxlsdi3LBqlU3iGlXBkOYARKRJRC4QkW4R6RWRe/znj4pIrpXARomw4GP5EzZiWnThhXz5m980Dx6j5snlBXQT8G/cDmCpkBCHAGcDo1T19KJLiI0AcmHBx/LDRkxGvTBUL6BjVHW2qq5X1Ud9Wq+qs4GjCyuqMVgs+Fh+2IjJMBy5FMBzfkP4PflEpEFETgdyhnAwSoO5LuZHoRZ7GUa1k0sBfBiYCjwpIg+KyIPAE8D7/TmjAjDXxfywEZNhOEL3A/D7AJwOICKjcXMGT5dALiMP5i1cyKTrr+eUALNGynVxvbkuAn7ElGPOxEZMRj0QeWN3VX0m1fiLSIeIHFw8sYx8sOBj+WEjJsNwRFYAGVwI3CYiNxdSGGPwWPCx6NhiL8NwDGkhmIgMV9X+AsqTFXMDNQpNd3c306dOtcVeRk0z5P0ARGSEiJwuIheJyAL/+QCAUjT+RnGp1/hBlTpiqtffwygTqhqYcLGAEriNXD7l07f8selhZQuZjj32WDWi0dfXp/Nnz9axw4drg4iOHT5c58+erX19ffvkXbNmjba1tOiSWEz7QJOgfaBLYjFta2nRNWvWlOEO6hf7PYxCA/RoWBsfehL+DhyQ5fhI4MGwsoVM1awA8mmQh1rHiOZmbQG92DccYQ1IX1+ftrW06Dpwr0FGWgfa1tJSUDmNYOz3MIpBLgWQywQkQLZJgt3+nBFCKSJ0pur4z7XX0rR9O78CvoLbd7PJ/70smWT1tm1Mnzp1jynBVsNWFvZ7GOUgVyygs4HP4MJAP+IPjwf+H/B5Vb2u2AJCdU4ClyLeTHodtwDNwGUh+ZfEYuyYNYvLly2z+EEVhv0eRjEY0iSwql4PdAC/BXYALwJ3AB2lavyrlVL06NLrWAnMyJE/PbyBrYatLOz3MMqB7QdQJErRo0uvoxGnoaPuZWs9zsrCfg+jGAzZDTTkwvcOtmw9UIoeXXodbZBXQDhbDVtZ2O9hlINcG8K8PyB9AHhpiWSsSkoRoTO9jmlAV4786Q2IrYatLOz3MMpBrhHAzcAU4JSMdDIwrLiiVTel6NGl1zEPuBYiNyAWP6iysN/DKAthPqLABuCIgHOPhJUtZKrGdQCl8OvOrGMNaBvoYu///6L/u6ipKXAhUV9fny6YO1fHxePa2NCg4+JxXTB3rvmblwn7PYxCwhAXgr0ZGB9wriOsbCFTNSoA1b0rOxf7lZ2pBnlxAVd2ZtaxEfQ80BGgDaCjW1qsATGMOiWXAsjlBnqnqj4ccK663HLKQCnizWTWcURDAz+Lxzlv7lwe7Ovj6Rde4PJly8x0YBjGPuR0AxWRscALqvqCiDQDFwHDga+r6qYSyFiVbqCGYRjlphBuoN8HRvvPnwNeidsPeOXQxTMMwzDKRS430LNx4WRO9J9PB3pw+wIfKiLTRWRi8cU0jOJg4ZeNeibXCOAOYDuwEXgMeBL4qT/+jP+by93dMCqSUgTrM4xKJtck8L+ArwO3AbcA/+0nhRV4WlUfVtWs69JF5Dsi8pSI3Jd2bJSI3C4iD/m/Iwt3K4YRnUQiwfSpU1m9bRuXJZM5o6caRi2Scw5AVa/G/V8coqq3+cPPAB/JUfQ64KSMY4uBtar6KmCt/24YJcfCLxtGkYPBicgE4DZVPcJ//ztwoqpuEpEDgTtU9dW5rmNeQEahseBrRj1QtGBwg2RcynXU/x0blFFEZolIj4j0bN68uWQCGvWBhV82jNIrgMio6jWq2qGqHWPGjCm3OEaNUYpgfYZR6ZRaATzpTT/4v0+VuP49mPtffZNPsD57V4xaJZIC8CGgHxKRLSLyvIj0i8jzg6hvNXC2/3w28JNBXGPImPufETX88hHHHGPvilG7hAUKSiWgD3htlLxpZW4CNuGi2j6K27FwNM775yH/d1SUaxUyGFwponQa1UGuYH1dXV32rhhVDUMJBpfGk6q6MU/F8hFVPVBVY6p6iKp2qeozqvoOVX2V//tsPtcsBOb+Z6TIFazv3p4ee1eMmiaSG6iIfB23A9iPcVvPAqCqtxZPtL0U0g3U3P+MqNi7YlQ7udxAw/YQTycObAPelXZMgZIogEJi7n9GVOxdMWqdSApAVc8ttiCloq21lX/l6NWZ+58B9q4YtU+uaKCX+L/fFJFvZKbSiFhYSrFXr1Eb2Lti1Dq5JoFTE789uP2BM1PVEdX9L7V5ulE5lNof394Vo+YJcxGqlFToPYFLsVevUVhSv9kS/5sl/W+2pMi/mb0rRjVDgdxAa4pS7NVbTxS7Z17O0M32rhi1TFGjgRYKiwZauXR3dzN96lTOTyaZkUxyKG6HoK5YjGtjMW5YtWrIjeSCOXNoXr6cy5LJwDxLYjF2zJrF5cuWDakuw6glChINVEQmRzlm1CZBPfxf//rXJemZr1yxghkhjT+4BVkrb7xxSPUYRr0R1QT0zYjHjBojLG7S+979bt6+Y0fRV8qaP75hFIdcbqAniMhCYIyIXJSWLgUaSyJhianGyI/FkjmX7f3nO3eydtcuwmopRM/cQjcbRnHINQLYD2jF/d8PT0vPA1OLK1rpqcYoocWUOVLcJODKkGsE9czzUVrmj28YRSLMRSiVgEOj5CtWKrQbaDaqMUposWUeO3y49gVcO5X6QMflOh+PD7huvi6d1fjbGEYlQA430FwN/9f835/iYvkPSGFlC5lKoQDmz56tS2Kx0MZucSymC+bOLUr9fX19On/2bB07fLg2iOjY4cN1/uzZoY3aUGXOVWeDiCZzKIAXQRt9Az4fdCxog/87H/SCpqYB9Q+2MTd/fMPIn6EqgGP837dmS2FlC5lKoQAi93YzerOFIKhHvLipSeNNTTqiuTlrAz0UmaP0wqNevxW0BXSh/5661iJ/vKura0+9Q1FafX19umDuXB0Xj2tjQ4OOi8d1wdy51vM3jACGqgDW+r9fDstX7FQKBRC5tyuyp8xgeu2ZROkRjwZ9IKOB7urq0v2z9LgzG+wXQRsbGvKus62lRc+ZNi1nY30R6Et8mSg9+nIqWsOoN3IpgFyTwAeKyFuBKSJytIgck54KNQ9RCbQ2NUXyNGn1k5GFmnyNMtF6PvBtBvrXXzhjBmcB63AbNKwDmoFJQHrN2bxjom6K0yCSMxbOtcBZvkzYtVKuoObSaRgVRJh2wHn6dAP9wG8y0q/DyhYylWIEMCIW08U5eqaLQEfEYgWdlBzsROsloAuC6k4bCWQzp+TTCw+yvV8soiNAR2QZdYT16G0EYBilg6GYgPZkgk9HyVesVAoFIL7hDG3UQRtECjphnM9Eay6lsKdurxyCFFHkOr3pKJvtfUQspmtx5qd8rlXuyXbDqCcKogDcdZgCfNWnk6OWK0Qq1SRwl2/kF/sGdo+niT/e5Xummb3YbB4w54KObm2NVO9gRgDZlEJ6/hG+8c/mHVOIXnhKiYwlvxGAuXQaRunIpQCixgL6IvAx4G8+fcwfqxmmnXkmfbEY63E29ck4m/pk/3098JBfbJRux+7G2d2bGWiPHwds37o151xApEVOwLSMYw8DbQH5x+NsdkHRKguxsCq1Onca0BV6pYHXam9v54ZVq5jS0sKSWIwEkMTtrbskFmNKSws3rFpFe3vYPlyGYRSEMO2QSkAv0JD2vRHojVK2ECnfEcBgvHPy6ZmmetB9RDAb5ejNRqo3Sy97sR9lZPO9X5uj916IXnjKlDPYZ2AunYZRfCjQHEAvMCrt+6hKVQBD2Tgk6mKjVOM3H3RJDvNHFHt2UL2X+MZ1TZZGNY5zD13CQN/7Jd78c9rJJw+qzqgLq9KVyBqym84Wgo5ubrZFWoZRJgqlAD6CC/N+HXA98H/Ah6OULUSKqgAK0bON0jNdu3atxhsbtZlwH/yU/XtMa2vOEUlmvaNbWjTe2KgXNDUNaFQXNTXpMN/Ih64daG7O2ZvOdq/nnXGGnjNtWqTRU7oSWQv6MdAx/pm0gL7/5JML3qMvxNoLw6gXhqwAAAFeBhyImwg+FXhprnKFTFEVQCk8TNasWaOjm5v1YhHtwy3QOtc3yAI6KkMZvOgbxMGMSIKU0btPPFEvLsDII9u95Tt6KqUpp1zbQhpGtVKoEcCGKPmKlaIqgGL7mPf19ekB++23p+edMn1kmmFSXkNr2DtPUEiPl3zuM2qPudK9cypdPsOoRAqlAK4E3hAlbzFSVAUwmHAO+fDuE0/UhWkNbJR1AzPJvmArW089amOdz31G7TFXun9+pctnGJVIoRTA34BdOG+9XuDeSpwEjtozboG8e4p9fX3azF7TTpQJ4EtAhzNwbiBzzUAbbnVxV1dX5MY6n/uslRg9lS6fYVQihVIAh2ZLUcoGXO+fXon8NZeAmocCmD97tn5cJLSRWAw6SSTvnuL82bMHrHqNugBqdN9vwhMAABeFSURBVNr3IJPRQt9YL43YWEfpDV8sopNyPYu0HnO+q4NLTaXLZxiVyJAUADAMmA8sAy4AmsLyR01eAbRFzZ+PF1DOXi8D/eSjml3GDh8+wB8/cgiENGUQxWQUpFQyTUW57OEt/j6j9pgrvYdd6fIZRiWSSwHkWgl8PdDhe+udwNIc+ctKe3s723GuSktg4CpTf/wG4M24aJP5RPR8euvWAate2yBS9NDh/vMyXFTPwW6vODOZ5NtXXkljQwNvPPpoJp94Iqc0Nweupt0OvCWHfOlRNyt928VKl88wqpIw7QDcm/a5Cbg7LH/UhFtHcDewAZgVkGcW0AP0jB8/PrLGGzt8uK7FTbyO8z3wcf57X1pPcXRra15eJanrpnrxUeYAFjU1abyxUdeRR8ycHKOJ9LmBkcOG6ftPPjmrC2a+PeZK97KpdPkMoxJhiCagu8O+DzYBB/m/Y4F7gLeE5c9nJXAU+/iipiY9eOTIvHzp58+erYubmvbY8S/A2fdzNUipyd0oJqONELjJSzblkNnoDTBn4dYmBC1Qy7w/1crfdrHS5TOMSmOoCmAX8LxP/cDOtM/Ph5WNmoBLgYvD8uSjAKLax+NRe+RZesh9uBHFAQzcCjGoQerr63P7CITUtcYrlIvZN7RDG+gH2TuKSfciGgH6hiOOCPQiWkRwOIlqjNFT6fIZRiUxJAVQjAS8BBie9nkdcFJYmXyDwQX1FBc1Ne3xtsk3jn36dRelhWdYC3ocaDPO7z7VIK1du3bA5PKIWGyAh1J6Qy7kdtlsAf0i2b2IZkYoPxo3wghSUBZewTBqj0pUAK/wZp97gPuBT+YqM5j9ANauXasdhx+uLWkN7MEjR+oFjY2q5B/HPkWUHmi2kAVr2Ru/J9Md9L98Tz1Mlo/jRi3ZGvn5EcovxJmXMuW18AqGUbtUnAIYTBrsCCCzUUvfvnCokTyDes1r164NNEGt8TJkNuSjIiqjAwLODUWZ2cSqYdQudacAwhq1dLPPUGL5h/Wa442NusiPMrKlc2HA5HMfboQymG0hs91XVHOWqoVXMIxap+4UQFCjtgYGhHJQ3BaPcfadxL2I4O0Uc/WaR+fojWf21ueTx8bqEa8ZdQRgi6sMo7bJpQAibQlZTaxcsYIZyeSAYwlgOi6OdWohVzewCLel4XPs3QLyWOBbwFFveAOHHXbYPtdftnQp5yeTgQu6noM920VmYzPwDdyWkY3ANcD7yb2t4lXAewPO5bstY4r0rS2DSF8sZhhGjRGmHSol5TMCyBYzJmXvT5l9biG3+acFtLWhQbu6ulR1r82/hfBNYMJ642v8dVM7ZyX9tR6IKM/MkF76YMxZNgIwjNqGehsBpDYrT2clMANox4WCmAmcS3hYhrnA4bt3c+GMGSxYsGBPyIhe9m783ozbED592/dpuE3cM0mNQn4FfNHL0oQLKdHk5QoKYXEK0DRsGD9uaeGuLNdu9/neCSxuaoq80bqFVzCMOidMO1RKGswcQKaffXqPPZedPtXzHeNHC1ECzA0IM5Elf5DXUfrx1AKzzBAWFzQ16YK5c3OuhO3q6sprkZR5ARlGbUO9TQL39fVpfP/9dRTZN0xvI7rXTYNvoD+eI+9iBm768sHGRo03NQ1oqIMifeZrvin0SlgLr2AYtUtdKoCR++9fkFDJLeQfxC3VYK9du3ZAQx2mdFILwxYRHlKiWFh4BcOoTepOAUTxbb8IdFKWRjwzxk5zxmghM0/KrLQRZ7IJa7BzTbj2gZ7nlY41woZhFIJcCqDmJoGzuYFmMge3r2VqQrUbN5nbjJvc3YGLUz3HH7shIE9qIvhNuIncHbNmsb63l87Ozn3qzDXh2g6MjcW4YO5cdu7axRNbtnD5smX7TNwahmEUCnFKorLp6OjQnp6eSHkbGxrYoUpTSJ4kbquzUcAHgFXAT8nuFXQX8C5g/xx5Tmpq4u4HHghssBOJBJMmTmT1tm2B15jS0sL63l5r9A3DKAgiskFVO4LO19wIIJsbaCYP4xTAVbhdaXK5hL4WOC9HntnAlVdcEVhne3s7N6xaxZSWlsBdvLK5ahqGYRSLmlMAQaaWBLAAtwL3MEAaGpjZ2MhG4KM5rvl/uA2Rwzh/505W3nhjaJ7Ozk7W9/ayY9YsJsfjNDc0MDkeDzUdGYZhFIuaMwElEgmOP/JIfrp9+54eezduEdb5uAVhh+L28/2WCFep8jXgb8AK4Fnc6GAXbrOC6bjQDTsgp1mpuaGBnbt25Xt7hmEYRaHuTEDt7e286W1voxO3OvbXuEZ8NXAZe1fgtgNfUeVXwHxgG7Ae19D34kYLAjyOs/9HMSu1tbYW/H4MwzCKRc0pAIC77ryTW3GN+fuBswm338/D9fbTlcMXcZO+vwbeDlydo04LmWAYRrVRkwrg6a1beQtwOa73PjtH/lm4eEGZnICLGzQOF98nWxwe/PFrm5qYu2DB4AQ2DMMoAzWpANI9gZ4mPDwz+JDHAedmAt8H+oF3AMfhRgUpD57FQCeQ3L2bBx98cIiSG4ZhlI6aVADpnkBtRLTfB5wbjzMl7QDuBU7ERecchttD4EXcorGf79jB9KlTSSQSWa+TSCRYMGcO4+JxGhsaGBePs2DOnMD8hmEYxaYmFcC8hQu5NhbjLiJuluLzZeNhYD/gYGAZzh30V7hFZH/AmZna8eaiZDLrWoDu7u494aTX9fezQ5V1/f00L1/OpIkT6e7u3qeMYRhGsalJBZC+6GorbtetMPv9clz8/2xci5sjSI///2+caejKjLwzk8l91gIkEgmmT53K6m3buCyZHDDRfFkyyept20JHDoZhGMWiJhUA7F10tfuMM9iK2yzlYgZutrLYH1+Ca5AzuQs3erjQn78M5056FvAQ8G3cto7jcG6jSfbdPjHXFpJhIwfDMIxiUrMKIEU8HmdYczO7gTuBo4DhwETgKzgvoc/hVgNn7sQ1BRcILl05nACcAzwG++wO9iZg+P77D6g/SnC6bCMHwzCMYlOzCiDd7r5h+3b+CuwEduN666nG+8+4qJ834ZTC/rhGfgduYVi24AyzceEhBphzcOsGdieTA8w5tvG6YRiVSk0qgGx298eAB9h3T97Uoq+fA6kIQo+zd3I3G0FuoyfgYgJ9+NRT9yiBqMHpbBWxYRilpiYVQLrdPRUE7lRczz3MFn8+0MrQ3EZnA33337/Hu6eYG6+ba6lhGEOhJhVAyu6evonLMHKvCP4ozkz0iRz5wtxGx+MWjaW8e06ZOnWPS2o27sIpgHxXEZtrqWEYQyZsu7BKSflsCamq2iCiD2Rstt4Qsidv+kbwjX5bxlvCNmkP2Sc4fX/gxbGYLpg7t+Abr/f19WlbS0vkjeQNw6hPqMQtIUXkJBH5u4j0icjiQl+/rbWVL+NMOimTTz4rgufh/PyXMNAz6OMidLKvZ1A66aODlHdPofcBMNdSwzAKQph2KEbCuc4ngFfgFtneA7wurEy+I4D5s2friIxe+nzQJTlGAItBF/hyY/zncWmjgvPOOENHDhsW3vNOq/dFv8F7ocm1wfyekUg8XvC6DcOoHqjAEcBxQJ+q/kNVX8TFWju1kBXMW7iQ5xkYBG4eblVvlBXB43Ebw1wOPAF83G/W3rViBd+79VamtLTss6gs27qBYnn3mGupYRiFoBwK4GDgkbTvj/pjAxCRWSLSIyI9mzdvzquC9vZ2RjY3DzD5tOMa53fiVgCHNd7pXj6Zk7Qpc85vDz+cDtwE82Syrxso1h4B5lpqGEYhKIcCkCzH9tmXUlWvUdUOVe0YM2ZM3pVMP+ccljcN3MSxE/gQ8Ftcox3UeF8LvJfgzdrb29v5/k9+QlNLC3fiRgmZ6wYG690ThWK6lhqGUUeE2YeKkXBzlL9I+74EWBJWJt85AFXnKTNy//33sdf3ZXgHZbPjt4CObm3VBXPnhnrSFNq7J597My8gwzByQQXOAfwZeJWIvFxE9gM+jIuxVnB2AScz0JsH4G04U9BCMkxBvse/as0anu7v5/Jlywb0/DMptHdPVNKjnS6JxbLeQ+aoxTAMI5OSKwBV3Ymbk/0FsBG4RVXvL3Q9y5YuZe7u3fwJZ+JJN/kcgovb8wcRjt1vvyE13O3t7Vy+bBlPbNnCzl27eGLLlpyKoxCUS/kYhlE7iBslVDYdHR3a09OTV5lx8Tjr+vsD/fXB9ZiPb2nh6RdeGJJ8hmEYlYiIbFDVjqDzNRkKAqK7Sj63bVvesXMsBo9hGLVAzSqAqK6SwyGvFbMWg8cwjFqhZhXAtDPP5Fs58iwHPgCRN2Ox7R0Nw6glalYBzFu4kKvIvfL340RfMWsxeAzDqCVqVgG0t7cTa27mFPYN6pa+8jdG9BWztr2jYRi1RM0qAIBzzzmHDzY17eMGmr7yN58VsxaDxzCMWqKmFcC8hQtZtd9+fBAXrmEnA8M25BuuwWLwGIZRS9S0Aij0ilmLwWMYRi1R0woACrtidt7ChUXZ3tEwDKMc1KQCyFyo9cajj0Z37+YPd989pHANFoPHMIxaouYUQLEXalkMHsMwaoWaigWUSCSYNHEiq7dty+qrfxcwpaWF9b291ks3DKPmqatYQLZQyzAMIzo1pQBsoZZhGEZ0akoB2EItwzCM6NSUArCFWoZhGNGpKQVgC7UMwzCiU1MKwBZqGYZhRKemFIAt1DIMw4hOTSkAsIVahmEYUamphWCGYRjGXupqIZhhGIYRHVMAhmEYdYopAMMwjDqlKuYARGQz5FzjFUQb8HQBxSk2Jm/xqTaZTd7iUm3yQnSZD1XVMUEnq0IBDAUR6QmbBKk0TN7iU20ym7zFpdrkhcLJbCYgwzCMOsUUgGEYRp1SDwrgmnILkCcmb/GpNplN3uJSbfJCgWSu+TkAwzAMIzv1MAIwDMMwsmAKwDAMo06pGQUgIv8UkXtF5K8isk/gIHF8Q0T6RKRXRI4ph5xelld7OVPpeRGZn5HnRBHZkpbnMyWW8Tsi8pSI3Jd2bJSI3C4iD/m/IwPKnu3zPCQiZ5dZ5q+IyAP+N/+RiBwQUDb0/SmhvJeKyGNpv/t7AsqeJCJ/9+/z4jLKe3OarP8Ukb8GlC3H832ZiPxGRDaKyP0i8jF/vCLf4xB5i/cOq2pNJOCfQFvI+fcA3YAAk4A/lltmL1cj8ARuwUb68ROB28oo11uAY4D70o79L7DYf14MfDlLuVHAP/zfkf7zyDLK/C6gyX/+cjaZo7w/JZT3UuDiCO9MAngFsB9wD/C6csibcX4p8JkKer4HAsf4z8OBB4HXVep7HCJv0d7hmhkBROBU4AZ1rAcOEJEDyy0U8A4goaqDXelcFFT1d8CzGYdPBa73n68HTstS9N3A7ar6rKo+B9wOnFQ0QdPIJrOq/lJVd/qv64FDSiFLFAKecRSOA/pU9R+q+iLwfdxvU1TC5BURAT4E3FRsOaKiqptU9W7/uR/YCBxMhb7HQfIW8x2uJQWgwC9FZIOIzMpy/mDgkbTvj/pj5ebDBP/TnCAi94hIt4gcXkqhAhinqpvAvazA2Cx5KvU5A5yHGwVmI9f7U0rm+eH+dwLME5X4jN8MPKmqDwWcL+vzFZEJwNHAH6mC9zhD3nQK+g43DVbACmSyqj4uImOB20XkAd9jSSFZypTVB1ZE9gOmAEuynL4bZxba6u3APwZeVUr5BknFPWcAEfkksBP4XkCWXO9Pqbga+DzumX0eZ1Y5LyNPJT7jjxDe+y/b8xWRVuCHwHxVfd4NVnIXy3KsJM84U9604wV/h2tmBKCqj/u/TwE/wg2T03kUeFna90OAx0sjXSCdwN2q+mTmCVV9XlW3+s9rgJiItJVawAyeTJnN/N+nsuSpuOfsJ/BOBs5QbyzNJML7UxJU9UlV3aWqu4FrA+SoqGcsIk3A+4Gbg/KU6/mKSAzXmH5PVW/1hyv2PQ6Qt2jvcE0oABF5iYgMT33GTZrcl5FtNTBdHJOALalhYBkJ7DWJyEu9XRUROQ73Wz1TQtmysRpIeUOcDfwkS55fAO8SkZHefPEuf6wsiMhJwCJgiqpuC8gT5f0pCRnzUu8LkOPPwKtE5OV+FPlh3G9TLt4JPKCqj2Y7Wa7n6/9/uoCNqnp52qmKfI+D5C3qO1zMWe1SJZw3xD0+3Q980h//KPBR/1mAK3HeE/cCHWWWuQXXoI9IO5Yu7zx/L/fgJn7eWGL5bgI2AUlcb2gGMBpYCzzk/47yeTuA5WllzwP6fDq3zDL34Wy5f/XpWz7vQcCasPenTPLe6N/PXlxDdWCmvP77e3BeIolyyuuPX5d6b9PyVsLzfRPObNOb9vu/p1Lf4xB5i/YOWygIwzCMOqUmTECGYRhG/pgCMAzDqFNMARiGYdQppgAMwzDqFFMAhmEYdYopACMSIrLLRxm8T0R+ICItBb7+OSKyLEeeE0XkjWnfPyoi0wspR5Y6v+IjM34ly7lOEenx0RsfEJGvZsrl7+ugPOtcLiKvyyP/a0TkLhHZISIXZ5zLGTVUAqJj+jUzWSPoSpkivhoFphT+uJaqPwFb0z5/D7iowNc/B1iWI8+l5IiUWYT7fh7YP8vxI3A++K/x35uAOVny3UGR15zgYtm8AfhC+vMhYtRQAqJjEhBBlzJGfLVU2GQjAGMw3Am8EkBELvKjgvvE72kgIhN8j/h633NclRoxiItZ3uY/d4jIHZkXF5FTROSPIvIXEfmViIwTFxzro8ACPxJ5s7jY+Rf7MkeJyHrZGzM91Yu9Q0S+LCJ/EpEHReTNWeoT39O/T1w89dP98dXAS4A/po6lcQnwBVV9AEBVd6rqVb7cpSJysYhMxS0u+p6X+b0i8qO0ev+fiNyacd2UzB3+81YR+YK4oIDrRWRcZn5VfUpV/4xboJVO1KihQdExgyLoZo2UKSKNInJd2nNckKUuo4IwBWDkhbi4L53AvSJyLHAucDyuh3i+iBzts74auEZVJ+J60XPyqOb3wCRVPRrXaF2iqv8EvgVcoapHqeqdGWVuABb5+u4FPpt2rklVjwPmZxxP8X7gKOD1uLAGXxGRA1V1CrDd15cZ5+YIYEPYTajqKqAHF7/lKGAN8FoRGeOznAt8N+waOAW0XlVfD/wOOD9H/nSiRrQMio4ZVD7o+FG48MVHqOqR5L43o8yYAjCi0ixut6ce4GFczJI3AT9S1RfUBa67FRcWGOARVf2D/7zC543KIcAvRORe4ONAaChsERkBHKCqv/WHrsdtXpIi1cveAEzIcok3ATepC8L2JPBbnEmloKiq4kI9nCluV6cTCA7tm+JF4Db/OUj+IIYa0TKofNDxfwCvEJFviotf83yWfEYFYQrAiEqqJ3yUql7oTQphcXUzG5rU953sfe+GBZT9Jm4+4EjggpB8Udnh/+4iewj0SPGBM7gfOHYQ5b4LnIkLBPgD3bvRRxBJrzggWP4goka0DIqOGVQ+63FvDno9bt5jLrA8D1mNMmAKwBgKvwNOE5EWcREI34ebHwAYLyIn+M8fwZl1wG1bl2o4PxBw3RHAY/5zuodJP26rvAGo6hbguTT7/lm4Xnw+93G6t2GPwY0e/pSjzFeAT4jIYQAi0iAiF2XJN0BmdSF7Hwc+hQuiVkwCo4aKyBdF5H0+X1B0zKAIulkjZfq5nQZV/SHwadz2kUYFU0sbwhglRlXvFpHr2NtYLlfVv/gJ243A2SLybVzUxat9ns8BXSLyCfbd7SjFpcAPROQxXCTUl/vjPwVWicipwIUZZc4GvuUnm/+Bs69H5Uc4c8w9uJHKJar6RFgBVe31k943+ToV+FmWrNd5ubYDJ6jqdpwX1RhV/VseMgYiIi/FmebiwG4v1+vUbX4yD9dgNwLfUdX7fbEj2RtC+kvALSIyA2fe+6A/voa90Si34Z+pqj4rIp/HKRiA//bHXg98V0RSHctsGx0ZFYRFAzUKjlcAt6nqEWUWpSIRt97hL6raVUYZfqGq7y5X/UZlYCMAwyghIrIBeAFYWE45rPE3wEYAhmEYdYtNAhuGYdQppgAMwzDqFFMAhmEYdYopAMMwjDrFFIBhGEad8v8BMmVtTnuIW5MAAAAASUVORK5CYII=\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plotData(X,y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
2.2 Gradient Descent
\n",
"\n",
"Here we will fit theta to our data using Gradient Descent."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"m = y.size # number of samples\n",
"X = np.stack([np.ones(m), X], axis=1) # add collumn of ones to data for theta0 intercept term\n",
" \n",
"# NOTE: If ValueError: all input arrays must have the same shape appears then you may have run this cel multiple times\n",
"# which will have added multiple collumns of ones to the matrix X\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"# Here we set the number of iterations as well as learning rate alpha\n",
"iterations = 1500\n",
"alpha = 0.01"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we are properly setup we can begin implementing Gradient Descent. We do this by subtracting from theta our scaled derivative of the cost function. We will also keep track of the cost function to check accuracy. Relevant formulas are as follows:\n",
"\n",
"$$J(\\Theta ) = 1/(2m)\\sum_{i = 1}^{m} (h_\\theta (x^i) - y^i)$$\n",
"\n",
"$$h_\\theta(x) = \\theta^Tx = \\theta_0 + \\theta_1x_1$$\n",
"\n",
"$$\\theta_j = \\theta_j - (\\alpha/m)\\sum_{i = 1}^{m}(h_\\theta(x^i) - y^i)x_j^i$$\n",
"\n",
"We begin by creating a function which can return the cost J given training set X and y and an intitial theta\n"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"def computeCost(X,y,theta):\n",
" # initialize some useful values\n",
" m = y.size # number of training examples\n",
" J = 0 # initialize zero cost\n",
" \n",
" # Vectorized implementation of cost function J(theta)\n",
" H = X.dot(theta)\n",
" J = np.subtract(H,y)\n",
" J = np.square(J)\n",
" J = np.sum(J)\n",
" J = J*(1/(2*m))\n",
" # ===========================================================\n",
" return J"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can run the function with a few different theta initializations "
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcomputeCost\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0.0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m0.0\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'With theta = [0, 0] \\nCost computed = %.2f'\u001b[0m \u001b[1;33m%\u001b[0m \u001b[0mJ\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[1;31m# further testing of the cost function\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcomputeCost\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0marray\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m2\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m\u001b[0m in \u001b[0;36mcomputeCost\u001b[1;34m(X, y, theta)\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;31m# Vectorized implementation of cost function J(theta)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m \u001b[0mH\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 8\u001b[0m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msubtract\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mH\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msquare\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mJ\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;31mValueError\u001b[0m: shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)"
]
}
],
"source": [
"J = computeCost(X, y, theta=np.array([0.0, 0.0]))\n",
"print('With theta = [0, 0] \\nCost computed = %.2f' % J)\n",
"\n",
"# further testing of the cost function\n",
"J = computeCost(X, y, theta=np.array([-1, 2]))\n",
"print('With theta = [-1, 2]\\nCost computed = %.2f' % J)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have a working cost function, we can implement Gradient Descent"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"def gradientDescent(X, y, theta, alpha, num_iters):\n",
" # Initialize useful values\n",
" m = y.size\n",
" n = theta.size\n",
" J_history = []\n",
" \n",
" # make a copy of theta, to avoid changing the original array, since numpy arrays\n",
" # are passed by reference to functions\n",
" theta = theta.copy()\n",
" \n",
" for i in range(num_iters):\n",
" hypothesis = X.dot(theta)\n",
" errors = np.subtract(hypothesis,y)\n",
" Xtrans = X.transpose()\n",
" gradient = alpha*(1/m)*Xtrans.dot(errors)\n",
" theta = theta - gradient\n",
" J_history.append(computeCost(X,y,theta))\n",
" return(theta, J_history)\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 8\u001b[0m \u001b[0malpha\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;36m0.01\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 10\u001b[1;33m \u001b[0mtheta\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mJ_history\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgradientDescent\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m \u001b[1;33m,\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0malpha\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0miterations\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 11\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'Theta found by gradient descent: {:.4f}, {:.4f}'\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0mtheta\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m\u001b[0m in \u001b[0;36mgradientDescent\u001b[1;34m(X, y, theta, alpha, num_iters)\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 11\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnum_iters\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 12\u001b[1;33m \u001b[0mhypothesis\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 13\u001b[0m \u001b[0merrors\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msubtract\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mhypothesis\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 14\u001b[0m \u001b[0mXtrans\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtranspose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;31mValueError\u001b[0m: shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)"
]
}
],
"source": [
"# Test Case\n",
"\n",
"# initialize fitting parameters\n",
"theta = np.zeros(2)\n",
"\n",
"# some gradient descent settings\n",
"iterations = 1500\n",
"alpha = 0.01\n",
"\n",
"theta, J_history = gradientDescent(X ,y, theta, alpha, iterations)\n",
"print('Theta found by gradient descent: {:.4f}, {:.4f}'.format(*theta))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we have a theta we can fit our data to a line"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[0mplotData\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mpyplot\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mtheta\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;34m'-'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 3\u001b[0m \u001b[0mpyplot\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mlegend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Training Data'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'Linear Regression'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 4\u001b[0m \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;31mValueError\u001b[0m: shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEWCAYAAACqitpwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7xVdZ3/8dcb2ApHOF44SCZe8mgXLTJhFIaaabqYRwutOY03hEyiAGeGE11gmn5qTdZUShleUpgSkNQoiwoix8kug2ighZYXznZS8QqpePAYbvDz+2N9t2w2+372fX+ej8d6nH2++7vW+q7FYX/297K+X5kZzjnnXDkNqnUBnHPONR8PLs4558rOg4tzzrmy8+DinHOu7Dy4OOecKzsPLs4558rOg0uLknS4pO2SBte6LC4zSR+R9NsqnesNku6R1CfpX6pwvgclvaPceV398ODS5CT9WdJLIZAkt9ea2aNmNtzMdtVBGfeRtCKU1SS9s8j9j5P0C0nPSXpe0gZJpw6wTO+UtDkt7WJJywZy3AznuVhSIvy7PC9praSJJRzndknTB1CUzwC3m9kIM7si7dh/TPnb2SXprym//1spJzOzN5jZb8qdtxiSpofrSV7L/0n6L0nHFHGMZZIuLnfZmoEHl9bwgRBIktsTlTyZpCEl7PZbYArwVAn7/gS4FRgNHAz8C/BCCcepqBz35SYzGw6MIroPP5Sk6pUMgCOAP2Z6w8yOS/7tAL8BLkz5W7o0PX+J//618ptwXfsD7wESwHpJb6ptsZqAmfnWxBvwZ+A9GdKPBAwYEn5/HfBroA/4b+BKYFl4753A5mzHBS4GVgDLiD7UpxN9cZkHxIG/ADcDBxVQ3s3AO4u4vo5wHQfkyHM68PtQtjhwSkg/H7g/XPPDwMdD+n7AS8ArwPawnQO8TPThsx34Q8i7P7AYeBJ4HPgPYHB47yPA/wILgGeB/8hQtouT9zn8fly4no6w/29T3vtb4HfAtvDzb0P6l4BdwF9D2RZmuQ+TiQLI88DtwJtC+v+k7f/6HPfydmB6Wtr08LdzRbjOi4FjgF+Gf/utwFJg/0z/zuGefS/8/fQB9wEnlJh3fPi37gNuBL4PXJzlWqYT1dbS038O3BheDyL6234qw32bFf4eXg737ZaQ/u9Ef0994X5PrvXnQC02r7m4pOXAXcBIog+H84rc/3Si/4QHADcQ1R7OAP4eeC3wHFHAKpqkcyRtzPL2X4BeYJmkMySNTtv3RGAJ8OlQtr8jCowAzwDvB9qJAs0CSSeY2YtAF/CE7f6Gvhy4lFDLMLO3hmNcD+wEjgbeBpxM9KGVdBLRB83BREEg13XuSxRQNpvZ1rT3DgJ+RvQBPhK4HPiZpJFm9jn2rFFcmOHYryf6UJ5DVENaBfxE0j5m9q60/R/KVc4s/pYoUI8C/hMQUSA4BDgWOAr4fI79zyAKQAcAq8N1FpU33L8fAYuAg4AfhLzF+iGQ2sfzU6Jg+RqiYLYUwMyuAm4CLg337YMh/0PAJKIvHl8Clqf/XbYCDy6t4UehPf95ST9Kf1PS4cDfAP/PzF42s98CK4s8xx1m9iMze8XMXgI+DnzOzDab2Q6igNVdSpOJmS03s7FZ3jPgH4gCxmXAk5J+ndJufgHwX2Z2ayjb42b2QNj3Z2YWt8ivgF+w54dKTuEDowuYY2YvmtkzRLWUs1KyPWFm3zKzneG+ZPJPkp4HHgPGkfkD8TRgk5ktDcf6HvAA8IECi3sm8LNwHxLA14FhREGhHB41s6vNbJeZvWRmD5nZbeHvKXlf/j7H/r8yszUW9QEuBY4vIe8k4BUzW2hmCTP7PrChhGt5gig4Ef5mvmtmfWb2V6K/43GS9su2s5ndbGZPhn2XE/1tji+hHA2tkdpGXenOMLP/zvH+a4Fnzaw/Je0x4LAizvFY2u9HALdIeiUlbRdRv8jjRRw3LzPbDFwIIOkw4Fqi2spEomtYlWk/SV3ARcDrib5otQH3FnHqI4AYUUBLpg1iz3uRfl8yudnMpuTJ81rgkbS0R4BDCzj+Xvub2SuSHiti/3z2uE5JryGqUUwCRhDdly059k/ta+snaposNu9riZrQsparQIcSNe8RRlN+GegmaqpM/j13AC9m2lnSR4Aeor8PgOEhf0vxmouDqL/gIEltKWmpgeVFog9e4NX/cKPSjpE+vfZjQJeZHZCyDTWzsgaWdGb2GFHz25tTytGZni80ofyA6Bv8aDM7gCgIJaNEpunCM13jDqAj5Rrbzey4HPuU6gl2f1glHc7uQJ3vPHvsHwYMHEb5An36+f+T6N68xczaiZr7Kj1I4UlgTFpaMV+Qks4gaiYEmAqcCryLqJnr6JCe8e9E0lHA1cBMYGT4u3qAyl973fHg4jCzR4D1wMVhWPBE9mxueQgYKuk0STGiDst98xz2GuBLko4AkDRK0unZMkvaV9LQ8Os+koYWMmJK0oGSLpF0tKRBkjqAjwLrQpbFwPmS3h3eP1TSG4F9wjVsAXaGWszJKYd+Ghgpaf+0tCMlDQIwsyeJmtIuk9Qejt8pKVfzT6lWAa8P/U9DJJ1J1Jfx05SyHZVj/5uB08J9iAFziT7811agrBDVVl4EtoXa5KcqdJ5UvwUGS5oZ7tE/EjUz5iVpsKSjJF0FvB34YnhrBNF9+gvRF6z0frP0+z6cKOBsiQ6r6cAbS72gRubBxSWdS9SM9BeijtibiP5TYWbbiEbGLCL6pvsiezc/pPsmUb/NLyT1EX3Yn5Qj/4NEI7QOBdaE18nAdK6kjMNkiUbqHEk0wu0Fog7XHUTflDGzuwid9USjrH4FHGFmfUSDDm4mGmxwDin9TKFf5nvAw6Gv6rVEI48A/iLp7vB6KlGg+lM4zgqiTuyyMrO/EA0+mEv0b/QZ4P0pHf/fJOrTek7SXp3hZvYg0VDvbxGN3voA0RD1l8td1uAi4ESie76SqJZYUaFv74PAJ4j+Lf6JKCjvyLHbOyRtJ/rb+R+iADLezJJ/b98hqvU9QTTyKz0YLwLeGu77CjPbSNQceBdRTeqNwJ1luLyGo6g/1Lk9SboJeMDMLqp1WZwrlaQNwDfMbGmty9JqvObiAJD0N6FJZ5CkU4iGFu81ssy5eqZoZoXRoVnsAqKawy9qXa5W5KPFXNJriMb3jyRq8pppZvfUtkjOFe1NRE26+xE9MPuPZvZ0bYvUmrxZzDnnXNl5s5hzzrmy82axoKOjw4488shaF8M55xrKhg0btppZ+nNvHlySjjzySNavX1/rYjjnXEORlD5zBODNYs455yrAg4tzzrmy8+DinHOu7Dy4OOdci4nH4/TMmsXo9nYGDxrE6PZ2embNIh6Pl+0cHlycc66FrF69mgljxzJs0SLW9vWxw4y1fX0MW7SICWPHsnr16rKcp2LBRdIbJP0+ZXtB0hxJB0m6VdKm8PPAkF+SrpDUK2mjpBNSjjUt5N8kaVpK+jhJ94Z9rkjOopvtHM4518ri8ThTu7tZ2d/PpYkEnURDhjuBSxMJVvb3M7W7uyw1mIoFFzN70MyON7Pjiaa97gduIVpX/TYzOwa4LfwO0Yp+x4RtBtGaCMnlXS8imlH3ROCilGBxdcib3O+UkJ7tHM4517IWXnYZH0skmJjl/YnA9ESCKxcsGPC5qtUs9m4gHtYNOZ1o3XHCz+SSrqcDS8KSs+uAAyQdArwPuNXMnjWz54BbgVPCe+1mdkdY6nZJ2rEyncM551rW8mXLuCCRyJlneiLB8qUDn0S6WsHlLKK1MSBa9e9JeHWxpYND+qHsuSTp5pCWK31zhvRc59iDpBmS1ktav2VLrhVYnXOu8W3dvn2v5UzTHR7yDVTFg4ukfYDJ7F5oKWvWDGlWQnrBzOxaMxtvZuNHjdpr9gLnnGsqHcOHk/Fx+hSPhnwDVY2aSxdwd8q010+HJi3Cz2dC+mb2XO96DNHqb7nSx2RIz3UO55xrWedMmcLiWCxnnkWxGOecd96Az1WN4HI2u5vEIFryNDniaxrw45T0qWHU2ARgW2jSWgOcHNZKP5BonfM14b0+SRPCKLGpacfKdA7nnGtZF86dy3WxGHdkef8OouAyu6dnwOeqaHCR1Aa8l2gRqqSvAO+VtCm895WQvgp4GOgFriNasx0zexb4IvC7sH0hpAHMJFrDupdoYaDVec7hnHMtq7OzkyUrVjC5rY35sRhxIEH04Tk/FmNyWxtLVqygs7NzwOfyxcKC8ePHm8+K7JxrBfF4nCsXLGD50qVs3b6djuHDOee885jd01N0YJG0wczG75XuwSXiwcU554qXLbj49C/OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pxzruw8uDjnnCs7Dy7OOefKzoOLc865svPg4pzbQzwep2fWLEa3tzN40CBGt7fTM2sW8Xi81kVzDaSiwUXSAZJWSHpA0v2SJko6SNKtkjaFnweGvJJ0haReSRslnZBynGkh/yZJ01LSx0m6N+xzhSSF9IzncM7ltnr1aiaMHcuwRYtY29fHDjPW9vUxbNEiJowdy+rVq2tdRNcgKl1z+SbwczN7I/BW4H5gHnCbmR0D3BZ+B+gCjgnbDOBqiAIFcBFwEnAicFFKsLg65E3ud0pIz3YO51wW8Xicqd3drOzv59JEgk5gCNAJXJpIsLK/n6nd3V6DcQWpWHCR1A78HbAYwMxeNrPngdOB60O264EzwuvTgSUWWQccIOkQ4H3ArWb2rJk9B9wKnBLeazezO8zMgCVpx8p0DudcFgsvu4yPJRJMzPL+RGB6IsGVCxZUs1iuQVWy5nIUsAX4jqR7JC2StB8w2syeBAg/Dw75DwUeS9l/c0jLlb45Qzo5zuGcy2L5smVckEjkzDM9kWD50qVVKpFrZJUMLkOAE4CrzextwIvkbp5ShjQrIb1gkmZIWi9p/ZYtW4rZ1bmms3X7do7Ik+fwkM+5fCoZXDYDm83szvD7CqJg83Ro0iL8fCYl/2Ep+48BnsiTPiZDOjnOsQczu9bMxpvZ+FGjRpV0kc41i47hw3kkT55HQz7n8qlYcDGzp4DHJL0hJL0b+BOwEkiO+JoG/Di8XglMDaPGJgDbQpPWGuBkSQeGjvyTgTXhvT5JE8Iosalpx8p0DudcFudMmcLiWCxnnkWxGOecd54PV3b5mVnFNuB4YD2wEfgRcCAwkmgE16bw86CQV8CVQBy4FxifcpyPAr1hOz8lfTxwX9hnIaCQnvEcubZx48aZc62st7fXOtrabC2YZdjWgnW0tdnixYuto63N5sdi1guWAOsFmx+LWUdbm61atarWl+KqCFhvmT7/MyW24ubBxTmzVatWWUdbm80LgePlEDjmhcCRDCz5AlBvb2+tL6Vient7bc7MmXbwiBE2SLKDR4ywOTNnNvU155ItuPgT+s65V3V1dbFu40Z2zJjBpPZ2hg0axKT2dnbMmMG6jRu5d/36lh6u7A+ZFi7ZjNTyxo8fb+vXr691MZyra6Pb21nb10dnjjxxYFJ7O09t21atYlVFPB5nwtixrOzvzxhc7wAmt7WxbuNGOjtz3aHmImmDmY1PT/eai3OuYK08XNkfMi2OBxfnXMFaebiyP2RaHA8uzrmCFTNcudm0cq2tFB5cnHMFu3DuXK6Lxbgjy/t3EAWX2T091SxWVbRyra0UHlyccwXr7OxkyYoVTG5rY34sRhxIEHXiz4/FmNzWxpIVK5qyQ7uVa22l8ODinCtKvuHKXV1dtS5iRbRyra0UPhQ58KHIzrl8Vq9ezdTubqYnEkxPJDicqClsUSzGoliMJStWNG1wzcaHIjvn3ABVu9bWyHO4ec0l8JqLc66eJGtJH0skuCCR4AjgEWBxLMZ1dVRLylZz8eASeHBxztWLRpoNwJvFnHNZNXLzSzNqhtkAPLg41+J8Msb60wyzARQUXCR9WNKI8PrfJf1Q0gmVLZpzrtLi8ThTu7tZ2d/PpYkEnUTrk3cClyYSrOzvZ2p3t9dgqqwZZgMotObyeTPrk/R24H3A9cDVlSuWc64amqH5pRk1w2wAhQaXXeHnacDVZvZjYJ/KFMk5Vy3N0PzSjJphNoBCg8vjkr4N/BOwStK+RezrnKtTzdD80oyaYTaAQgPEPwFrgFPM7HngIODTFSuVc64qmqH5pRk1wxxueYOLJAFvCb++WdJJwFNm9ouKlsw5V3HN0PzSrBp9DrecD1FKOhm4CtgEPB6SxwBHA7OaKcD4Q5SuFTXSw3quPmV7iHJInv2+CbzHzP6cdrDXAauAN5WthM65qnu1+SXPZIweWFyx8jWLDQE2Z0h/HMhdl3bONYRGb35x9Slfs9h8os78G4HHQvJhwFnAzWb25YqXsEq8Wcw554pXUrOYmX1Z0o+ByUTPU4moJnOumf2pIiV1zjnX8PL1uRCCyJ8kHRT9as9VvljOOecaWc4+F0mHS7pR0jPAncBdkp4JaUdWo4DOOecaT74O/ZuAW4BDzOwYMzsGOAT4EVE/jHOuDHzKe9ds8gWXDjO7ycySc4thZrvM7EZgZGWL5lxr8CnvXTPKF1w2SLpK0kmSXhu2kyRdBdyT7+CS/izpXkm/l7Q+pB0k6VZJm8LPA0O6JF0hqVfSxtQp/SVNC/k3SZqWkj4uHL837Ktc53Cu3viU965Z5QsuU4F7gUuI5hb7RXh9H1DofBD/YGbHpwxVmwfcFprYbgu/A3QBx4RtBmFK/zCQ4CLgJOBE4KKUYHF1yJvc75Q853CurviU965Z5XzOZcAHl/4MjDezrSlpDwLvNLMnJR0C3G5mbwizLt9uZt9LzZfczOzjIf3bwO1h+6WZvTGkn53Ml+0cucrqz7m4Whjd3s7avj5yPf8eBya1t/PUtm3VKpZzBcv2nEu+0WJDJH1c0urQVPWH8PoTkgp5Qt+AX0jaIGlGSBttZk8ChJ8Hh/RD2f2gJkTP0xyaJ31zhvRc50i/vhmS1ktav2XLlgIux7ny8invXbPK95zLUuB5oqaw5Af5GGAasAw4M8/+k8zsCUkHA7dKeiBHXmVIsxLSC2Zm1wLXQlRzKWZf58qhY/hwHslTc/Ep710jytfncoKZzTSzdWa2OWzrzGwm8LZ8BzezJ8LPZ4iGNJ8IPB2aqgg/nwnZNxNNLZM0BngiT/qYDOnkOIdzdcWnvHfNKl9weU7ShyW9mk/SIElnAjmf1Je0n6QRydfAyUQDAVYS1XwIP38cXq8EpoZRYxOAbaFJaw1wsqQDQ0f+ycCa8F6fpAlhlNjUtGNlOodzdaUZVhx0LpN8weUsoJuoJvCQpIeAp4APhfdyGQ38VtIfgLuAn5nZz4GvAO+VtAl4b/gdoin8HwZ6geuAWQBm9izwReB3YftCSAOYCSwK+8SB5AMB2c7h3Kvq4cHFZlhx0LlMCh4tJmlkyL81b+YG5KPFWsvq1auZ2t3NxxIJLkgkOAJ4BFgci3FdWMOkmlPNx+NxrlywgOVLl7J1+3Y6hg/nnPPOY3ZPjwcWV9eyjRYraSiypPHAk2b2eN7MDcKDS+uo1eqL8XichZddxvJly3YHkClTuHDuXA8grmGVNBQ5h38GfirppoEVy7nqq8WDiz7Fi2s1A3qIUtIIM+srY3lqppVrLq32jbraDy76OvWumZVcc5G0v6QzJX1SUk94fQBAswSWVtaK36ir/eCiT/HiWlG+ZY6nEs3r9Qsg2b8yhmgE1iVmtqTiJaySVqy5tOo36mrXXHyKF9fMSq25fA4YFx6k/I+wfQIYD/x7JQrqqqdVv1FX+8FFn+LFtaJ8wUVknlLlFTJPv+IayPJly7ggkciZZ3oiwfKlS6tUouqo9oOLHcOH80iePD7Fi2s2+YLLl4C7JV0t6d/Cdg1wd3jPNbBW/UZdzIOL5XjQ0qd4ca0oZ3Axs+uJmsB+BewAXiaa6n68mX230oVzldXK36i7urpYt3EjO2bMYFJ7O8MGDWJSezs7Zsxg3caNdHV1lW2wg0/x4lqSmflmxrhx46zVzJk50+bHYmaQdZsXi1nP7Nm1LmrV9fb2Wkdbm63Ncl/WgnW0tVlvb29Bx1u1apV1tLXZvFjMesFeBusN97ejrc1WrVpV4StyrjKA9ZbhM7XUhyiRdG8ZY5yrAf9GnV25BzsUUlNyrpnkG4r8oWxvAdeY2aiKlKoGWnEoMuyeY2t6IsH0RILDiZrCFsViLKrBHFv1wocPO1eYbEOR8y0WdhNwA5lHjA0tR8FcbSW/UV+5YAGT0iZNXNfCkya26mAH58olX81lAzDNzO7L8N5jZnZYht0aUqvWXFxmXnNxrjClPkQ5B3ghy3sfHHCpnKtTPnzYuYHJNxT5N2b2aJb3/Gu+a1o+2MG5gSlk4sqDwzLFSBom6XOSvpJco965ZuQrRDo3MIUMRb4RGBleXwIcDTwHLK9UoZyrBz582LnS5evQnwZ8Afg80fDjLwBfBbYTzZZ8MfB7M9tY8ZJWmHfoO+dc8Uodinw78BJwP7A/8DTwE6JAc2F434fKOOec20PO4GJmj0j6JvBTIAZMNbNHJR0ObM3W2e+cc6615au5YGZXS1oKvGJm/SH5L8DZFS2Zc865hpU3uACY2fa031+sTHGcc841g5InrnTOOeey8eDinHOu7Dy4OOecK7uCgoukD0naJGmbpBck9UnKNueYc3WhHEsUO+dKU2jN5avAZDPb38zazWyEmbVXsmDODUS5lih2zpWm0ODytJndX8oJJA2WdI+kn4bfXyfpzlATuknSPiF93/B7b3j/yJRjzA/pD0p6X0r6KSGtV9K8lPSM53CtIR6PM7W7m5X9/VyaSNBJNCyyE7g0kWBlfz9Tu7u9BuNcBRUaXNaHD+mzQxPZh3KsUpnuX4me8E/6T2CBmR1DNEfZBSH9AuA5MzsaWBDyIelY4CzgOOAU4KoQsAYDVwJdwLHA2SFvrnO4FlDuJYqdc8UrNLi0A/3AycAHwvb+fDtJGgOcBiwKvwt4F7AiZLkeOCO8Pj38Tnj/3SH/6cCNZrbDzP4P6AVODFuvmT1sZi8TTbB5ep5zuBawfNkyLkgkcuaZnkiwfOnSKpXIudZT6EOU55d4/G8AnwFGhN9HAs+b2c7w+2bg0PD6UOCxcL6dkraF/IcC61KOmbrPY2npJ+U5xx4kzQBmABx++OElXJ6rR75EsXO1lzO4SPqMmX1V0reAvaZPNrN/ybHv+4FnzGyDpHcmkzNktTzvZUvPVOvKlX/vRLNrgWshmhU5Ux7XeDqGD+eRPEsUPxryOecqI1/NJdlXUspc9JOAyZJOBYYSNa19AzhA0pBQsxgDPBHybwYOAzZLGkI0C/OzKelJqftkSt+a4xyuBZwzZQqLFy3i0hxNY75EsXMVZmYV34B3Aj8Nr78PnBVeXwPMCq9nA9eE12cBN4fXxwF/APYFXgc8DAwmCowPh7R9Qp7jcp0j1zZu3DhzzaG3t9c62tpsLZhl2NaCdbS1WW9vb62L6lzDA9Zbhs/UWjyh/1ngk5J6ifpHFof0xcDIkP5JYB6Amf0RuBn4E/BzYLaZ7bKoVnIhsIaohnVzyJvrHK4F+BLFztVezpUoW4mvRNl84vE4Vy5YwPKlS9m6fTsdw4dzznnnMbunxwOLc2WSbSXKgoKLpElm9r/50hqZBxfnnCtetuBSaLPYtwpMcy2qVefxatXrdi6fnMFF0kRJc4FRkj6Zsl1M1KnuXMvO49Wq1+1cIXI2i0n6e6KRXp8gGnWV1Af8xMw2VbR0VeTNYqWJx+NMGDuWlf39GadbuQOY3NbGuo0bm6qfo1Wv27l0JTWLmdmvzOwSYIKZXZKyXd5MgcWVrlXn8WrV63auUPlqLt8wszmSfkLmJ/QnV7Jw1eQ1l9KMbm9nbZ6n4ePApPZ2ntq2rVrFqrhWvW7n0mWrueR7Qn9J+Pn18hfJNYNWncerVa/buULlCy5fA94NnGpmn61CeVyDadV5vFr1up0rVL6hyIeETv3Jkt4m6YTUrRoFdPXtnClTWByL5cyTbx6vRhzOW47rdq6pZZoTJrkB3cBqotFhv0zb/ifXvo22+dxipRnoPF6rVq2yjrY2mx+LWS9YAqwXbH4sZh1tbbZq1aqs550zc6YdPGKEDZLs4BEjbM7MmVWbL8znL3MuQpa5xQqdePLzheRr5M2DS+mSAWJeCBAvhwAxr4AAUcoHdLEBqVKBqNTrdq6ZDCi4RPszmahj/+vA+wvdr1E2Dy4D09vbaz2zZ9vo9nYbPGiQjW5vt57Zs3N+gM+ZOdPmx2IZA0tymxeLWc/s2Xucp5iAVGrNqJLX7VwzyRZcCp1b7MtEywrfEJLODgecX67muVrzocjVV8pw3p5ZsxiWZ62W+bEYO2bMYHZPjz/o6FyFlToUOek04HgzeyUc7HrgHqBpgourvlKG8y5ftoy1OQILRA8vTlq6FHvllYIfdLx84cJiiu6cy6OY9VwOSHm9f7kL4lpPx/DhPJInT/pw3mIC0vJly7iggEC0fOnSQorrnCtCocHly8A9kr4bai0bgEsrVyxXz8o1dLiQ4bxXAX996aVXj19MQComEDXicGjn6lne4CJJwG+BCcAPwzbRzG6scNlcHSrnTMAXzp3LdbEYd2R5/w6iKSJ+mEi8evyJ73hHwc+XFBqIRuy7r89u7Fy5ZerlT9+ADYXka+TNR4vlV4lnO7IO5wXrAFuVdvyRw4bZgUOHFlSGQkajfXzIEGsfMsSfV3GuRGQZLVZos9g6SX9TuRDnGkElZgLu6upi3caN7Jgxg3GxGEOBScAOYB3QlXb8j+3cyT+85z1MbmtjfixGHEgQjSqbH4sxua2NJStW0NnZWVDN6HtmzArHLtc1OecouObyJ2AX0f/hjcC9wMZC9m2UzWsu+R08YoT15qgFWKh1jG5vr/jxC32+JN+DjgcNG1bRa3Ku2THA51wy9ouaWb4m7Ybhz7nkN3jQIHaY5Ry/ngCGDW2WVKcAABURSURBVBrEzl276ub48XicKxcsYPnSpWzdvp2O4cM557zzmN3Tw+uPOaai1+RcsytpsTBJQyXNAT4NnAI8bmaPJLcKldXVifQRVEPNmEFUfc1mIDMBlzI0OVtZU0d7dXZ2cvnChTy1bRs7d+3iqW3buHzhQjo7Owd0zoHw0Wmu2eXrc7keGE/UDNYFXFbxErm6kGlU2Eagg2jYYLbxUwOZCbjUmYYHMoKtFrMbl3PEnXN1K1NbWXID7k15PQS4O1f+Rt68z2W3gkaFhb6Ico6sKmU02kBHsFV7dmOfTdk1G0ocLfbq481mtrNiEc7VlUJGhX2E6CnabCO1StHZ2cmSFSsKGglWTFlzjfYq5ZwDUYkRd87VpUwRJ7kRjRB7IWx9wM6U1y/k2rfRNq+57FboqK02qMhMwMXMNFyuEWzVmt240iPunKs2BjrlfrNvzRRcBrp+ySDJEnk+AF8OgaUWZU7NK6h6WQei0Hs7CGqyCJpzxcoWXIqZuNLVWCEjjMrRWVztEVTFlDk97yioyWivUhV6b0eCd/S7xpYp4pRjA4YCdwF/AP4IXBLSXwfcCWwCbgL2Cen7ht97w/tHphxrfkh/EHhfSvopIa0XmJeSnvEcubZ6r7kUsuhVuTqLS1nEq1TFlDlT3jlg8/PUBMpV1nIo6N6C9XhHv2sQVLtZDBAwPLyOhQ/7CcDNwFkh/RpgZng9C7gmvD4LuCm8PjYEqH1D0IgDg8MWB44C9gl5jg37ZDxHrq2eg0uhH8AfOeecsgSFao5oKiaQZcrbSzRyrVFGX5U6Eq/egqRzSVUPLnucBNqAu4GTgK3AkJA+EVgTXq8hmm0ZomHPW0OAmg/MTznWmrDfq/uG9PlhU7Zz5NrqObgU+gG8f6jV5MpXaGdxtdaHL6aDO1veVeEDeV7IW+9r2We7t59l78k6S/m3c66asgWXiva5SBos6ffAM8CtRDWN5233sObNwKHh9aHAYwDh/W1ETc+vpqftky19ZI5zpJdvhqT1ktZv2bJlIJdaUYUuevVyIlH0yo7ZpE4oOam9nWGDBjGpvZ0dM2awbuNGurq68h6jEMWsuZItbxfRJJc7iCa93BeylrUenozPdG/fAvyVvSfrTFXov51zdSFTxCn3RrSK5S+BdwC9KemHER7UJOqXGZPyXpwoUFwJTElJXwz8I/BhYFFK+nnAt4BR2c6Ra6vnmksxI4waaZhrb2+vtRdR2xroMN5C+q1qxYcou0ZFLUeLmdnzwO1EfS4HSErOEzgGeCK83hwCAeH9/YFnU9PT9smWvjXHORpSwYtexWJVn8qkVMlRX8fu3MmiPHmTZR7IVC3xeJyp3d2s7O/n0kSCTqK2107g0kSClf39TO3urtncXrWYhsa5isoUccqxEdUgDgivhwG/Ad4PfJ89O9tnhdez2bND/+bw+jj27NB/mKgzf0h4/Tp2d+gfF/bJeI5cWz3XXArtc/nouec2xNQiqZ3axXTID2SgQTVHwJXCp4VxjYoajBYbC9xDtP7LfcD/C+lHEQ1R7g1BYN+QPjT83hvePyrlWJ8jaiZ7EOhKST8VeCi897mU9IznyLXVc3Ap5oOnWh3xA5H+QZ+tQ/5T0l5lLvX6GqHZqRH+7ZxLV/Xg0mhbPQcXs+I+eKo1lUmpMn3Q9xI92zEabDDYKIhGv2V5Qr/Y66vFrAOlqPd/O+fSZQsuBS0W1goaYbGwXItelWtixWqo9KJjmYxub2dtXx+57lKcaJTZU9u2leWczrWCkhYLc/Ul16JXjaQWC3R5h7lz1eXBxVVdLT7oL5w7l+tiMe7I8v4d4Zyze3rKdk7nWpkHF1d1tfigr/a6Lc61Og8uTaAenjovRqkf9AO9zmrNOuCcw0eLJbd6Hy2WTT0/dZ5PMSOjGvk6nWtm+Gix3BphtFi6eDzOhLFjWdnfn3HZ3DuAyW1trNu4saGae+LxOAsvu4zly5axdft2DmhrY+eOHfx8586muk7nmoGPFmtCjbYee6mLnZ3+4ot8PEtggfq7Tucc3iyW3OqhWazY5YkH+tT5QJdDLsZAFjs7mMaakNO5VoI/oV/fwaWUPoWBPHVezT6MQqevOf/cczPO/zUolK/en653rhV5cKnj4FLqpIWl1lyqPUniQBc785qLc/UrW3DxPpc6UGrfSakPIxZyvmn9/Rx79NEZ+0WKHRJc6GJnL2RZ7OwcokV8cvGn652rM5kiTitutay5FFMDSe8nGQZ2VpZv9gOu8WRoKqtk893QLNdRzLT8zrnqwpvF6je4FLzSZJiCPv2D/VNg+4MtprBp2gvuq0n7AD9w6FAbOWxYxZrv9o/FsjafJafl/xR7Tsv/mcGD/TkX52ooW3DxZrE6UMhEjr8BhpplXEnxa8Bq4J+BoVLep84Lnjgy5feJwOt37OCjf/1rxZrvPnTmmVmnhekCrgKuDecZRrSs6TXATT/5iT9d71y9yRRxWnGrZc2lkA7vkyT7tJQzT6ErKZ5/7rn5j0W0vkpqWkeWZqtyDiDIum5NOP+qEq/ZOVcZeLNY/QaXQj58hxX4wZ5tga2kVatW2YFDh9r+5OnDyHC+gQwJLmWxszaiprnRIdBl64/xUWLO1Y4HlzoOLmb5P3wL7pcJtYBMfRCpQSzb0sLZaggDqbmknr+YVRYbZfVI51pZtuDifS51It+MvYX2k4wCVvb3M7W7e6+hwalDkLuAdcAOYBIwFBgffl8X3k/XKfFtKWcZcg0JLnaxs1osKuacK5NMEacVt1rXXPIp6EFEdveTZOqLyDVqq5DhvqWOFqvoNXufi3M1hTeLNXZwKahTPKXZKlPzVL5mpmRT2SfTm8oyPOdSSN9JVa7Zn29xrqayBRdvFmsQqQtszYU9F9gCJgNLiIYmAxwObN2+fY9j5Gtm6gJuAhbHYlkX06rmglu+eqRzDSxTxGnFrd5rLkm9vb22fyxmo8g9kiq95tLb22vjjzvO9g+d/geDzcmwXz02MxU7EMA5Vz34YmG5NdJiYT2zZjFs0SIuzTFf1/xYjB0zZnD5woWsXr2aqd3dTH/5Zabv3MkRwCNE83VdR1Tj6cIX3XLOFS/bYmEeXIJGCi7FrEAJ5M37AaB7yBB+sM8+LFmxwp92d84VzFeibCLF9EUUMgPyR4C73/jGsveZOOdalweXBlVox3oh093PBB599FFvCnPOlY0HlwYVj8dZeNll3LBkCVv6+hi5336cfe65zO7p2SNIbN2+PeMaKakOB7a88ELONVmcc64YHlwa0OrVq5kwdizDFi1ibV8fO8xY29fHsEWLmDB2LKtXr341b6FPuY+EjPs751wpKhZcJB0m6ZeS7pf0R0n/GtIPknSrpE3h54EhXZKukNQraaOkE1KONS3k3yRpWkr6OEn3hn2ukKK5SbKdoxnE43GmdndnnHr/0kRir6lfCpruHpiSZX/nnCtFJWsuO4G5ZvYmoqU3Zks6FpgH3GZmxwC3hd8hGg17TNhmAFdDFCiAi4CTgBOBi1KCxdUhb3K/U0J6tnM0vGKXRL5w7tysa6RANFpsETA7y/7OOVeSTA+/VGIDfgy8F3gQOCSkHQI8GF5/Gzg7Jf+D4f2zgW+npH87pB0CPJCS/mq+bOfItTXKQ5TFLImcVOwaKT6NvXOuUNRy+hdJRwJvA+4ERpvZkwDh58Eh26HAYym7bQ5pudI3Z0gnxznSyzVD0npJ67ds2VLq5VVVoR30qVO/pI4sewvRKo6TyD4DcqapY5xzrhgVDy6ShgM/AOaY2Qu5smZIsxLSC2Zm15rZeDMbP2rUqGJ2rZlSp6FPTnc/YsQIHgSeAi5n91xk+fZ3zrliVDS4SIoRBZYbzOyHIflpSYeE9w8Bngnpm4HDUnYfAzyRJ31MhvRc5yibeDxOz6xZjG5vZ/CgQYxub6/KUN5C16PPtqbKQPd3zrmCZGorK8dGVLNYAnwjLf1rwLzweh7w1fD6NGB12G8CcFdIPwj4P+DAsP0fcFB473chr8K+p+Y6R66tmD6XZB/G/NCHkQj9FPMrMO18uoFOQ+/T2Dvnyolqr+cCvJ2omWoj8PuwnUr0SMVtwKbwMxkoBFxJNIvJvcD4lGN9FOgN2/kp6eOB+8I+C9k9V1rGc+TaCg0u9fDhPNA1Vaq5JotzrrlVPbg02lZocKmX1REHOg29T2PvnCuHbMHFZ0UOCp0VeXR7O2v7+jJ2hCfFgUnt7Ty1bdvutDBdy/Jly9i6fTsdw4dzzpQpXDh3rs/p5ZxrWD4rcpmUMhS4mOlanHOuGXhwKVKxQ4GLna7FOeeagQeXIhU7lLfY6Vqcc64ZeJ9LUGifSzGrQHZ2dpbcR+Occ43A+1zKpJhVIKG0PhrnnGt0HlxKUOgqkFD6dC3OOdfIPLiUKDlX11PbtrFz1y6e2raNyxcu3GtYsU+34pxrRR5cKqyg9VRiMWb39FSzWM45V1EeXCqs2D4a55xrBh5cqqCYPhrnnGsGPhQ5KHQosnPOud18KLJzzrmq8eDinHOu7Dy4OOecKzvvcwkkbYG8zzvWmw5ga60LUQf8PkT8PkT8PkSqdR+OMLNR6YkeXBqYpPWZOtJajd+HiN+HiN+HSK3vgzeLOeecKzsPLs4558rOg0tju7bWBagTfh8ifh8ifh8iNb0P3ufinHOu7Lzm4pxzruw8uDjnnCs7Dy4NTtKHJf1R0iuSWm74paRTJD0oqVfSvFqXpxYk/ZekZyTdV+uy1JKkwyT9UtL94f/Ev9a6TLUgaaikuyT9IdyHS2pRDg8uje8+4EPAr2tdkGqTNBi4EugCjgXOlnRsbUtVE98FTql1IerATmCumb0JmADMbtG/hx3Au8zsrcDxwCmSJlS7EB5cGpyZ3W9mD9a6HDVyItBrZg+b2cvAjcDpNS5T1ZnZr4Fna12OWjOzJ83s7vC6D7gfOLS2pao+i2wPv8bCVvWRWx5cXCM7FHgs5ffNtOCHidubpCOBtwF31rYktSFpsKTfA88At5pZ1e/DkGqf0BVP0n8Dr8nw1ufM7MfVLk8dUYY0H1vf4iQNB34AzDGzF2pdnlows13A8ZIOAG6R9GYzq2qfnAeXBmBm76l1GerUZuCwlN/HAE/UqCyuDkiKEQWWG8zsh7UuT62Z2fOSbifqk6tqcPFmMdfIfgccI+l1kvYBzgJW1rhMrkYkCVgM3G9ml9e6PLUiaVSosSBpGPAe4IFql8ODS4OT9EFJm4GJwM8kral1marFzHYCFwJriDpvbzazP9a2VNUn6XvAHcAbJG2WdEGty1Qjk4DzgHdJ+n3YTq11oWrgEOCXkjYSfQG71cx+Wu1C+PQvzjnnys5rLs4558rOg4tzzrmy8+DinHOu7Dy4OOecKzsPLs4558rOg4urG5J2heGj90n6vqS2Mh//I5IW5snzTkl/m/L7JyRNLWc5Mpzza2H22q9leK9L0vow0+8Dkr6eXq5wXa8t8pyLipnUUdIbJd0haYekT6W9l3dmakkHSbpV0qbw88CQLklXhH03SjohZZ9pIf8mSdOKuT5Xez4U2dUNSdvNbHh4fQOwoZwPw0n6CDDezC7MkediYLuZfb1c5y2gXC8Ao8xsR1r6m4EfA6eZ2QOShgAzzOyqtHy3A58ys/UVLOPBwBHAGcBzyfsTZqZ+CHgv0YwJvwPONrM/pe3/VeBZM/tKCEAHmtlnw3Mo/wycCpwEfNPMTpJ0ELAeGE80pc8GYJyZPVepa3Tl5TUXV69+AxwNIOmToTZzn6Q5Ie3I8E3++vCNd0WypiPpz5I6wuvx4cN3D5I+IOlOSfdI+m9Jo8Nkh58AekIN6h2SLk5+U5d0vKR14Xy3pHz7vl3SfypaQ+MhSe/IcD6FGsp9ku6VdGZIXwnsB9yZTEvxGeBLZvYARA+NJgNLslySuok+gG8IZT5N0i0p532vpL2mQQllHh9eb5f0JUXrf6yTNDo9v5k9Y2a/AxJpbxU6M/XpwPXh9fVEQSqZviTM5LsOOEDSIcD7iB7+ezYElFuJpo4fLOm7KfexJ8O5XB3w4OLqTviG3gXcK2kccD7Rt9oJwMckvS1kfQNwrZmNBV4AZhVxmt8CE8zsbUQfiJ8xsz8D1wALzOx4M/tN2j5LgM+G890LXJTy3hAzOxGYk5ae9CGitTXeSjQdx9ckHWJmk4GXwvluStvnzUTf2LMysxVE3/DPNbPjgVXAmySNClnOB76T6xhEwW1dWP/j18DH8uRPVejM1KPN7MlQ5ieBg/Psny39eOBQM3uzmb2F/NfmasSDi6snwxRNE74eeJRonqi3A7eY2YthjYofAsmawWNm9r/h9bKQt1BjgDWS7gU+DRyXK7Ok/YEDzOxXIel64O9SsiRrBxuAIzMc4u3A98xsl5k9DfwK+JsiylsQi9q5lwJTFM0vNRFYnWe3l4Hk9CDZyp/NQGemzrZ/tvSHgaMkfUvSKURfKlwd8uDi6knyG/zxZvbPoZkl04dMUvqHWPL3nez+2x6aZd9vAQvDt9+P58hXqGR/yS4yzzae6zqy+SMwroT9vgNMAc4Gvh/mYMslYbs7X7OVP5tCZ6Z+OjR3EX4+k2f/jOmhieytwO3AbGBREWV1VeTBxdW7XwNnSGqTtB/wQaL+GIDDJU0Mr88mauoC+DO7P5T/Mctx9wceD69TRyL1ASPSM5vZNuC5lP6U84hqH8Vcx5mhz2AUUa3nrjz7fA34N0mvB5A0SNInM+Tbo8xm9gTRB/S/Ey2BXElZZ6aW9GVJHwz5VrL7Pk8jGqiQTJ8a+qQmANtCs9ka4GRJB4a+rZOJapodwCAz+wHweeDV0WWuvvh6Lq6umdndkr7L7g/iRWZ2T+h8vx+YJunbwCbg6pDnEmCxpH8j+0qEFwPfl/Q4sA54XUj/CbBC0ulEo5hSTQOuCQMHHibqzyjULURNVH8gqmF9xsyeyrWDmW0MAxi+F85pwM8yZP1uKNdLwEQzewm4gWgE2p8y5C+apNcQNVe2A6+Ech1rZi9ISs5MPRj4r5SZqd/C7iUQvgLcrGjG5keBD4f0VUQjxXqBfsI9NbNnJX2RKHgBfCGkvRX4jqTkF+P55bg+V34+FNk1pBBcfmpmb65xUeqSoud57jGzxTUswxoze1+tzu9qy2suzjUZSRuAF4G5tSyHB5bW5jUX55xzZecd+s4558rOg4tzzrmy8+DinHOu7Dy4OOecKzsPLs4558ru/wMCGOgOadljGgAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plotData(X[:,1], y)\n",
"pyplot.plot(X[:,1],np.dot(X,theta),'-')\n",
"pyplot.legend(['Training Data', 'Linear Regression'])\n",
"pass"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"For population = 35,000, we predict a profit of 0.00\n",
"\n",
"For population = 70,000, we predict a profit of 0.00\n",
"\n"
]
}
],
"source": [
"# Predict values for population sizes of 35,000 and 70,000\n",
"predict1 = np.dot([1, 3.5], theta)\n",
"print('For population = 35,000, we predict a profit of {:.2f}\\n'.format(predict1*10000))\n",
"\n",
"predict2 = np.dot([1, 7], theta)\n",
"print('For population = 70,000, we predict a profit of {:.2f}\\n'.format(predict2*10000))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
2.4 Visualizing J(theta)
\n",
"\n",
"To better understand our cost function, we will now plot the cost over a 2-d grid of theta0 and theta1 values."
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta0\u001b[0m \u001b[1;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta0_vals\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 10\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mj\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta1\u001b[0m \u001b[1;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta1_vals\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 11\u001b[1;33m \u001b[0mJ_vals\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mj\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mcomputeCost\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mtheta0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtheta1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 12\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 13\u001b[0m \u001b[1;31m# Because of the way meshgrids work in the surf command, we need to\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;32m\u001b[0m in \u001b[0;36mcomputeCost\u001b[1;34m(X, y, theta)\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[1;31m# Vectorized implementation of cost function J(theta)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 7\u001b[1;33m \u001b[0mH\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mX\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtheta\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 8\u001b[0m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msubtract\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mH\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0my\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 9\u001b[0m \u001b[0mJ\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msquare\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mJ\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
"\u001b[1;31mValueError\u001b[0m: shapes (47,3) and (2,) not aligned: 3 (dim 1) != 2 (dim 0)"
]
}
],
"source": [
"# grid over which we will calculate J\n",
"theta0_vals = np.linspace(-10, 10, 100)\n",
"theta1_vals = np.linspace(-1, 4, 100)\n",
"\n",
"# initialize J_vals to a matrix of 0's\n",
"J_vals = np.zeros((theta0_vals.shape[0], theta1_vals.shape[0]))\n",
"\n",
"# Fill out J_vals\n",
"for i, theta0 in enumerate(theta0_vals):\n",
" for j, theta1 in enumerate(theta1_vals):\n",
" J_vals[i, j] = computeCost(X, y, [theta0, theta1])\n",
" \n",
"# Because of the way meshgrids work in the surf command, we need to\n",
"# transpose J_vals before calling surf, or else the axes will be flipped\n",
"J_vals = J_vals.T\n",
"\n",
"# surface plot\n",
"fig = pyplot.figure(figsize=(12, 5))\n",
"ax = fig.add_subplot(121, projection='3d')\n",
"ax.plot_surface(theta0_vals, theta1_vals, J_vals, cmap='viridis')\n",
"pyplot.xlabel('theta0')\n",
"pyplot.ylabel('theta1')\n",
"pyplot.title('Surface')\n",
"\n",
"# contour plot\n",
"# Plot J_vals as 15 contours spaced logarithmically between 0.01 and 100\n",
"ax = pyplot.subplot(122)\n",
"pyplot.contour(theta0_vals, theta1_vals, J_vals, linewidths=2, cmap='viridis', levels=np.logspace(-2, 3, 20))\n",
"pyplot.xlabel('theta0')\n",
"pyplot.ylabel('theta1')\n",
"pyplot.plot(theta[0], theta[1], 'ro', ms=10, lw=2)\n",
"pyplot.title('Contour, showing minimum')\n",
"pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
3 Linear Regression with Multiple Variables
\n",
"\n",
"Here we implement linear regression with multiple variable to predict the price of houses\n",
"\n",
"
3.1 Feature Normalization
\n",
"\n",
"We begin by creating a function to normalize our features by setting the mean to zero and standard deviation to 1"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" X[:,0] X[:, 1] y\n",
"--------------------------\n",
" 2104 3 399900\n",
" 1600 3 329900\n",
" 2400 3 369000\n",
" 1416 2 232000\n",
" 3000 4 539900\n",
" 1985 4 299900\n",
" 1534 3 314900\n",
" 1427 3 198999\n",
" 1380 3 212000\n",
" 1494 3 242500\n"
]
}
],
"source": [
"# Load data\n",
"data = np.loadtxt(os.path.join('Data', 'ex1data2.txt'), delimiter=',')\n",
"X = data[:, :2]\n",
"y = data[:, 2]\n",
"m = y.size\n",
"\n",
"# print out some data points\n",
"print('{:>8s}{:>8s}{:>10s}'.format('X[:,0]', 'X[:, 1]', 'y'))\n",
"print('-'*26)\n",
"for i in range(10):\n",
" print('{:8.0f}{:8.0f}{:10.0f}'.format(X[i, 0], X[i, 1], y[i]))"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
"def featureNormalize(X):\n",
" # Normalize features in x returning normalized version of X where \n",
" # mean value of each feature is zero and the standard deviation is 1\n",
" \n",
" # You need to set these values correctly\n",
" X_norm = X.copy()\n",
" mu = np.zeros(X.shape[1])\n",
" sigma = np.zeros(X.shape[1])\n",
" m = X.shape[0]\n",
" n = X.shape[1]\n",
"\n",
" # =========================== YOUR CODE HERE =====================\n",
" mu = np.mean(X, axis = 0)\n",
" sigma = np.std(X, axis = 0)\n",
" tempMu = np.zeros(X.shape)\n",
" for i in range(m):\n",
" tempMu[i,:] = mu\n",
" X_norm = np.subtract(X_norm,tempMu)\n",
" for i in range(n):\n",
" X_norm[:,i] = np.divide(X_norm[:,i],sigma[i])\n",
" \n",
" \n",
" # ================================================================\n",
" return X_norm, mu, sigma"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Computed mean is [2000.68085106 3.17021277]\n",
"Computed sigma is [7.86202619e+02 7.52842809e-01]\n"
]
}
],
"source": [
"X_norm, mu, sigma =featureNormalize(X);\n",
"print(\"Computed mean is \", mu)\n",
"print(\"Computed sigma is \", sigma)"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [],
"source": [
"# Add intercept term to X\n",
"X = np.concatenate([np.ones((m, 1)), X_norm], axis=1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
3.2 Gradient Descent
\n",
"We can now apply gradient descent to our normalized, multivariate data set and plot the cost relative to the number of iterations"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"theta computed from gradient descent: [334302.06399328 99411.44947359 3267.01285407]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAERCAYAAABxZrw0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxddZ3/8dfn5mZpkyYhbbokbUkXoC2le1lEdoZNFHRAVBQYF1QcfzrO/FTGnzOO/nRG+emojAsIiAs6IooiDpvsIBbS0kIXWmpp6d50S7plvZ/fH+eEhpCkKc255+bc9/PxuI97tnvOJ6fp+558zznfY+6OiIgkTyruAkREJBoKeBGRhFLAi4gklAJeRCShFPAiIgmlgBcRSaicC3gzu83MtpnZ0n4se7qZLTKzdjO7rNu8q83s5fB1dXQVi4jkppwLeOB24IJ+LvsqcA3wi64TzawK+FfgJOBE4F/N7KiBK1FEJPflXMC7+xPAzq7TzGySmd1vZgvN7EkzmxIuu9bdXwAy3VZzPvCQu+90913AQ/T/S0NEJBHScRfQTzcDH3P3l83sJOD7wNl9LF8LrO8yviGcJiKSN3I+4M2sDHgL8Gsz65xcfKiP9TBNfTKISF7J+YAnaEba7e6zDuMzG4Azu4yPBR4bwJpERHJezrXBd+fuTcArZnY5gAVmHuJjDwDnmdlR4cnV88JpIiJ5I+cC3sx+CTwDHGdmG8zsQ8CVwIfMbAmwDLgkXHa+mW0ALgduMrNlAO6+E/gK8Fz4+nI4TUQkb5i6CxYRSaacO4IXEZGBkVMnWUeMGOF1dXVxlyEiMmgsXLhwu7tX9zQvpwK+rq6O+vr6uMsQERk0zGxdb/PURCMiklAKeBGRhFLAi4gklAJeRCShFPAiIgmlgBcRSSgFvIhIQg36gG/ryPCDx/7KE6sa4i5FRCSnDPqAT6eMm5/4K//z4ua4SxERySmDPuDNjOm1FSzd1Bh3KSIiOWXQBzzA8TUVrNyyh9b27o9mFRHJX4kI+Om15bR1OKu27om7FBGRnJGMgK+pAGCZmmlERF6TiIAfXzWUYcVplm5sirsUEZGckYiAT6WMaTXlOtEqItJFIgIeYHptBSs2N9HeoROtIiKQqIAvp7ktw5rt++IuRUQkJyQn4MMTrUs3qplGRAQSFPATq8soKUzpRKuISCjSgDezSjO7y8xeMrMVZnZKVNsqSBnTxuhEq4hIp6iP4L8D3O/uU4CZwIooNza9toLlm5rIZDzKzYiIDAqRBbyZlQOnA7cCuHuru++OansQtMPvbWln3c79UW5GRGRQiPIIfiLQAPzYzJ43s1vMrLT7QmZ2rZnVm1l9Q8ORdfl7fG05oBOtIiIQbcCngTnAD9x9NrAP+Hz3hdz9Znef5+7zqqurj2iDx4wcRlFBSgEvIkK0Ab8B2ODuC8LxuwgCPzJF6RRTa8pZsiHSliARkUEhsoB39y3AejM7Lpx0DrA8qu11mjW2ghc3NNKhE60ikueivormk8AdZvYCMAv4WsTbY+a4Sva1drB6296oNyUiktPSUa7c3RcD86LcRnezxlUCsHj9Lo4bPSybmxYRySmJuZO1U93wUspL0ixerxOtIpLfEhfwqZQxc1wli9frRKuI5LfEBTwEzTSrtu5hf2t73KWIiMQmsQHfkXF1PCYieS2RAT9jbHCidYmaaUQkjyUy4KuHFVNbOUTt8CKS1xIZ8ACzxutEq4jkt+QG/NhKNu4+QMOelrhLERGJRXIDfrza4UUkvyU24KfXVFCQMjXTiEjeSmzADykqYMroYSx6dVfcpYiIxCKxAQ8w9+ijWLx+N+0dmbhLERHJukQH/Ly6Kva3drBi8564SxERybpkB/zRRwFQv25nzJWIiGRfogO+pnIINRUl1K9TO7yI5J9EBzzA3Loq6tfuxF1PeBKR/JL4gJ9fdxRbm1rYsOtA3KWIiGRV4gN+btgOv1DNNCKSZxIf8FNGl1NWnNaJVhHJO4kP+IKUMXt8JfVrdQQvIvkl8QEPQTPNyq17aGpui7sUEZGsyYuAn19XhTs8/6r6pRGR/JEXAT9rXCUFKaN+rdrhRSR/pKNcuZmtBfYAHUC7u8+Lcnu9KS1OM21MOc++ooAXkfyRjSP4s9x9Vlzh3umkCVU8v343zW0dcZYhIpI1edFEA3DKpOG0tmfUDi8ieSPqgHfgQTNbaGbX9rSAmV1rZvVmVt/Q0BBZIfMnVJEy+MuaHZFtQ0Qkl0Qd8Ke6+xzgQuATZnZ69wXc/WZ3n+fu86qrqyMrpLykkOm1FTyjgBeRPBFpwLv7pvB9G3A3cGKU2zuUkycOZ/GraocXkfwQWcCbWamZDescBs4Dlka1vf44eWIVrR0ZPcZPRPJClEfwo4CnzGwJ8CzwR3e/P8LtHdL8urAd/q9qphGR5IvsOnh3XwPMjGr9b8awkkJOqK3gL2t0PbyIJF/eXCbZ6eRJw1m8fjcHWtUOLyLJln8BP3G42uFFJC/kXcDPr6uiIGW6Hl5EEi/vAr6sOM0JtRX8WSdaRSTh8i7gAU47ZgSL1+9W//Aikmh5GvDVdGScZ3QULyIJlpcBP3t8JaVFBTz5cnR934iIxC0vA76wIMUpk0bw5Mvb4y5FRCQyeRnwAKcfO4J1O/azbse+uEsREYlE3gb8accEPVfqKF5EkipvA75u+FDGHjVE7fAiklh5G/BmxmnHVPPn1Tto78jEXY6IyIDL24CH4Hr4PS3tLNmgx/iJSPLkdcC/ZdJwUgZPrFI7vIgkT14HfOXQImaMreTxVWqHF5HkyeuABzjruJEs2bCbHXtb4i5FRGRA5X3AnzN1JO7w6EodxYtIsuR9wB9fU86o8mIeeWlr3KWIiAyovA94M+PsKSN5YtV2Wtt1uaSIJEfeBzzA2VNGsbelnefW6lmtIpIcCnjg1MnDKUqneHjFtrhLEREZMAp4YGhRmrdMGs7DL23F3eMuR0RkQEQe8GZWYGbPm9m9UW/rSJwzdRTrduxnzXb1LikiyZCNI/hPASuysJ0jcvaUkQA8omYaEUmISAPezMYCbwNuiXI7A6G2cghTRg/jTyt0uaSIJEPUR/DfBj4L9Hr9oZlda2b1Zlbf0BDvzUZ/M20Uz63dqbtaRSQRIgt4M7sY2ObuC/tazt1vdvd57j6vuro6qnL65fzjR5NxeGi5juJFZPCL8gj+VOAdZrYW+G/gbDP7eYTbO2LH15QzrmoI9y3dEncpIiJHLLKAd/fr3X2su9cB7wEecff3R7W9gWBmXDh9DH/+63YaD7TFXY6IyBHRdfDdXDB9NG0drr5pRGTQy0rAu/tj7n5xNrZ1pGaNrWRUeTH3vahmGhEZ3HQE300qZVxw/GgeX9XA/tb2uMsREXnTFPA9uGD6GFraMzymPuJFZBBTwPfgxAlVDC8t0tU0IjKoKeB7UJAyzjt+FI+s2MqB1o64yxEReVN6DXgzq+rjVZrNIuPw9pk17Gvt4GFdTSMig1S6j3kLAQesp8+ZGcDn3f2OKAqL20kThjOqvJh7Fm/i4hk1cZcjInLYeg14d5/Q1wfNrBp4HEhkwBekjLfPqOGnz6yjcX8bFUML4y5JROSwvOk2eHdvAD43gLXknHfMqqG1I8P9yzbHXYqIyGE7opOs7v6HgSokF51QW8GEEaX8fvGmuEsRETlsuoqmD2bGO2bW8MyaHWxtao67HBGRw3LIgDezn/VnWlK9Y1YN7vCHJTqKF5HBpT9H8Md3HTGzAmBuNOXknknVZZxQW8E9CngRGWT6ug7+ejPbA8wws6bwtQfYBvw+axXmgEtm1fDChkZe3ron7lJERPqt14B3939392HADe5eHr6Guftwd78+izXG7p2za0mnjF8v3BB3KSIi/dafJpp7O+9cNbP3m9m3zOzoiOvKKcPLijln6kh+u2gDbR29Pl5WRCSn9CfgfwDsN7OZBA/QXgf8NNKqctC7541j+95WHn1pW9yliIj0S38Cvt3dHbgE+I67fwcYFm1ZueeMY6upHlbMnfVqphGRwaE/Ab/HzK4HPgD8MbyKJu/u208XpHjXnFoeXbmNbXt0TbyI5L7+BPwVQAvwQXffAtQCN0RaVY66fO44OjLO757fGHcpIiKHdMiAD0P9DqDCzC4Gmt0979rgASaPLGPO+ErurN9A0GolIpK7+nMn67uBZ4HLgXcDC8zssqgLy1VXzB/H6m17eW7trrhLERHpU3+aaL4AzHf3q939KuBE4IvRlpW73jGzlmElaX72l3VxlyIi0qf+BHzK3bteG7ijn59LpCFFBVw+dxz3L92sk60iktP6E9T3m9kDZnaNmV0D/BG471AfMrMSM3vWzJaY2TIz+7cjLTZXXHnyeNo6nF89uz7uUkREetWfk6z/G7gJmAHMBG5298/2Y90twNnuPhOYBVxgZicfSbG5YlJ1GW+dPIJfPPsq7bqzVURyVF+djU02s1MB3P237v4Zd/8HYIeZTTrUij2wNxwtDF+JufTk/ScfzebGZh7Wna0ikqP6OoL/NtBT94n7w3mHZGYFZraYoAfKh9x9QQ/LXGtm9WZW39DQ0J/V5oRzp45kdHkJP9fJVhHJUX0FfJ27v9B9orvXA3X9Wbm7d7j7LGAscKKZTe9hmZvdfZ67z6uuru5n2fFLF6R430njefLl7azepm6ERST39BXwJX3MG3I4G3H33cBjwAWH87lcd+VJ4ylOp7j1qVfiLkVE5A36CvjnzOwj3Sea2YeAhYdasZlVm1llODwEOBd46c0WmouGlxXzrjlj+c2ijTTsaYm7HBGR1+kr4D8N/J2ZPWZm3wxfjwMfBj7Vj3WPAR41sxeA5wja4O898pJzy4dPm0Bre0Y3PolIzkn3NsPdtwJvMbOzgM628z+6+yP9WXHYfj/7yEvMbZOqyzh36kh+/pd1XHfmJEoKC+IuSUQE6N918I+6+43hq1/hnm8+fNpEdu5r5TeL1Fe8iOSOvO1yYCCdNKGKE2oruPXJV8hkEnOpv4gMcgr4AWBmXHv6RNZs38cDy7bEXY6ICKCAHzAXnTCGiSNKufGR1eorXkRyggJ+gBSkjOvOmszyzU08vELdF4hI/BTwA+iSWTWMqxrCjY+8rKN4EYmdAn4AFRakuO7MySzZ0MgTL2+PuxwRyXMK+AH2t3PGUlNRwncf1lG8iMRLAT/AitIpPnbmJBau28WTOooXkRgp4CNwxfxx1FYO4YYHVuq6eBGJjQI+AsXpAj7zN8fy4sZG7luq6+JFJB4K+IhcOruWY0eV8f8eXEmbHusnIjFQwEekIGX87/On8Mr2fdy1UH3UiEj2KeAjdO7UkcwZX8m3/7SK5raOuMsRkTyjgI+QmfG5C6awtamFHz2xJu5yRCTPKOAjdtLE4Vw4fTTff+yvbGlsjrscEckjCvgs+OeLptLhztfvT9QTC0Ukxyngs2Bc1VA+ctoE7n5+I4te3RV3OSKSJxTwWXLdmZMZOayYL/9huW5+EpGsUMBnSWlxms9eMIXF63frskkRyQoFfBa9a3YtJ9ZV8bX7VrBjb0vc5YhIwingsyiVMr76zunsa2nnq/+zIu5yRCThFPBZdsyoYXzsjEn8dtFGnl6t3iZFJDqRBbyZjTOzR81shZktM7NPRbWtweYTZ02mbvhQvnD3i7rDVUQiE+URfDvwj+4+FTgZ+ISZTYtwe4NGSWEB//fSE1i7Yz//+adVcZcjIgkVWcC7+2Z3XxQO7wFWALVRbW+weesxI3jvieP40RNrWLhuZ9zliEgCZaUN3szqgNnAgh7mXWtm9WZW39DQkI1ycsYX3jaNMRVD+Mc7l7C/tT3uckQkYSIPeDMrA34DfNrdm7rPd/eb3X2eu8+rrq6OupycUlac5obLZ7B2x36+cf/KuMsRkYSJNODNrJAg3O9w999Gua3B6i2TRnDNW+q4/c9reUrPcBWRARTlVTQG3AqscPdvRbWdJPjcBVOYVF3KP9y5mO26AUpEBkiUR/CnAh8AzjazxeHrogi3N2gNKSrgv943h8YDbXzmziXqq0ZEBkSUV9E85e7m7jPcfVb4+p+otjfYTR1Tzr9cPI0nVjXwoyf1cBAROXK6kzWHXHnSeC46YTQ3PLCShevUrbCIHBkFfA4xM/79XTOoqRzCdXcsZNsePQFKRN48BXyOqRhSyE0fmEvTgXau+/kiWtszcZckIoOUAj4HTR1Tzg2Xz6B+3S6+cu/yuMsRkUEqHXcB0rOLZ9Tw4sZGbnp8DVPHlPO+k8bHXZKIDDI6gs9hnz1/CmceV80Xf7+Ux1flVzcOInLkFPA5rCBl/Nf75nDsqGF84o5FLN/0hp4eRER6pYDPcWXFaX58zXzKitN88Pbn2NKoK2tEpH8U8IPA6IoSbrtmPnua2/jArQvYua817pJEZBBQwA8S02rKueXq+by6cz9X3baApua2uEsSkRyngB9ETpk0nB++fy4rt+zhgz9+Tn3Ii0ifFPCDzFlTRvKd98xm0au7uPanC/VMVxHplQJ+ELrohDF847KZPLV6Ox/+Sb2O5EWkRwr4QeqyuWO54bIZ/Pmv27nq1mfVJi8ib6CAH8QunzeOG987h8Xrd3PljxawS1fXiEgXCvhB7m0zxnDzVXNZuXUPV9z8DJsbD8RdkojkCAV8Apw9ZRS3XzOfTbubufR7T7N0Y2PcJYlIDlDAJ8RbJo/gro+fQoEZ777pGR5esTXukkQkZgr4BJkyupzffeJUJlWX8ZGf1nP706/grue7iuQrBXzCjCwv4VcfPZlzpo7iS39Yzj/+egkHWnWtvEg+UsAn0NCiNDe9fy6fPvcY7n5+I+/8/tOs27Ev7rJEJMsU8AmVShmfPvdYfnzNfLY0NXPxjU/x4LItcZclIlmkgE+4M48byR/+/q3UDS/l2p8t5J/vflF3vorkicgC3sxuM7NtZrY0qm1I/4yrGspdHz+Fj54xkV8++ypv++5TLFm/O+6yRCRiUR7B3w5cEOH65TAUpwu4/sKp/OLDJ9PS1sHf/uDPfOuhVbS06wSsSFJFFvDu/gSwM6r1y5tzyqTh3Pfp03n7zBq++/DLXPSdJ3lurf6ZRJIo9jZ4M7vWzOrNrL6hQQ+WzoaKIYX85xWzuP3v5tPcluHyHz7DP9/9Io371WGZSJJYlDfCmFkdcK+7T+/P8vPmzfP6+vrI6pE32t/azrceXMVtT79CxZBCPnPecbx3/jjSBbF/94tIP5jZQnef19M8/S/Oc0OL0vyfi6dx7ydP47jRw/ji75bytu8+xdOrt8ddmogcIQW8AMEzX3/5kZP5wZVz2NfazpW3LOADty5gsa62ERm0orxM8pfAM8BxZrbBzD4U1bZkYJgZF54whj995gy+cNFUlm1q4tLvPc2Hf1LP8k1NcZcnIocp0jb4w6U2+Nyyt6Wd259+hZufWENTczsXHD+aj54xkdnjj4q7NBEJ9dUGr4CXQ2o80MatT67h9j+vpam5nRMnVHHtaRM5e8pIUimLuzyRvKaAlwGxt6WdXz23ntueeoWNuw8weWQZV59yNJfOrmVYSWHc5YnkJQW8DKi2jgx/fGEztzy1hqUbmxhaVMAls2p434lHc8LYirjLE8krCniJhLvzwoZG7liwjnuWbKK5LcP02nLeOXssb585hpHDSuIuUSTxFPASucYDbdy9aAN3LdrA0o1NpAxOnTyCS2fVcv700ZQVp+MuUSSRFPCSVau37eF3z2/id4s3smHXAUoKU5x2TDXnTRvFOVNHUVVaFHeJIomhgJdYuDsL1+3iniWbeGj5VjY3NpMymFdXxXnTRnH2lJFMGFGKma7EEXmzFPASO3dn6cYmHlq+hQeXb+WlLXsAqKko4a3HjODUycFrRFlxzJWKDC4KeMk5r+7YzxMvN/D06u08vXo7Tc3BU6amjinnlInDmVd3FPOOPoqR5TpRK9IXBbzktI6Ms3RjI0+t3s6TLzfw/Ku7aWnPADCuaghzxx/F3LoqZo+r5NhRwyhKqwslkU4KeBlUWtszLNvUyMJ1u1i4bhf163bRsKcFgMIC47jRwzh+TAXTa8uZVlPB1DHDGFqkq3QkPyngZVBzdzbsOsCSDbtZurGJZZsaWbapiZ37WgFIGdQNL2XSyDImjyzjmPB9UnUZpbo8UxKur4DXb7/kPDNjXNVQxlUN5eIZNUAQ+psbm1m2qYmlGxtZtXUPq7ft5dGXttGeOXjQUlNRwqSRZdQNL2V81VDGVQ15bV3l6l5BEk4BL4OSmVFTOYSayiH8zbRRr01v68iwbsd+Vm/by+ptQeivbtjLkvW7XzuR26lyaGEY+kOprRzCqPISRpeXMLqimFHlJYwcVqL2fhnUFPCSKIUFKSaHTTQw+nXzGve3sX7Xftbv3M+r4Wv9rgMs39TEQ8u30hqe2O1qRFkRo8pLwlcxVaVFVJUWM6KsKBwuYkRZMUcNLdKXgeQcBbzkjYqhhVQMrWB67Rs7RHN3du9vY0tTM1uamtnaGL43NbOlsZnNjc28sGE3O/e1kunltNWwkjQjyoqpHFpIeUkhFUMKKR+S7jIcvpcE0yuGFDKspJDS4gKKClK64UsGnAJehKDJ56jSIo4qLWLqmPJel8tknMYDbezY18rOfa3s2Nvy2vDOfa1s39tC44E2du9vZd2OfTQ1t9N4oI2O3r4VQumUMbSogNLi9Ovfi9IMLU5TWlTA0KI0pcXB+5DCFCWFBRQXpihJB+/F6QJKwvfidDg/HY4XpihO60sk3yjgRQ5DKnXwi6C/3J39rR00NbfRdCAI/KYDbeF4G/taO9jf2s6+lvC9tYP9LcH71j3N7N/ewb7Wdva3BO+H+K7oU1E6RUk6RXFh8FdDYYFRWJAi3XU4ZRSlg/d0QYqighTpcF5hgZFOpQ4OvzY9GE+ZUZAKXl2HC8Lh1GvDvH5+OD3V7bPp163n4Gc6382CaWZgGCkLvqxT4bilwDi4TCr8gktZt2UT+sWngBeJmJlRWpymtDjNmCPsLt/daWnPsK+lnZb2TPjqoLktQ0tbBy3tGZrD99cPh8u0d9DS+d6eob3Dac9kaG0P3ts6MrR1OPta2mnr8HA8Q3vGae9wWjsytHdkDg5n/JB/nQwWwZdEly8Mszd8Oby2TMoOuexr6wy/O4yDn+9cnnB8eGkxd37slAH/mRTwIoOImVFSWEBJYUHcpbwmk3HaMsEXQ0fGyWScDj/43pE5+Mq405Ghy7DT3mW462cOLpsJ3jvX2WX9DmTccQ++/DysJ5geTnNwPBzvXN7D4YPzcH/deM/rPLitHpcN1985j/C7L1i9h++vH8ehfEg0UayAF5EjkkoZxakCdE9Z7tF1XSIiCRVpwJvZBWa20sxWm9nno9yWiIi8XmQBb2YFwPeAC4FpwHvNbFpU2xMRkdeL8gj+RGC1u69x91bgv4FLItyeiIh0EWXA1wLru4xvCKeJiEgWRBnwPd058IYLZs3sWjOrN7P6hoaGCMsREckvUQb8BmBcl/GxwKbuC7n7ze4+z93nVVdXR1iOiEh+iTLgnwOOMbMJZlYEvAe4J8LtiYhIF5E+0cnMLgK+DRQAt7n7Vw+xfAOw7k1ubgSw/U1+Nkqq6/CorsOTq3VB7taWtLqOdvcemz9y6pF9R8LM6nt7bFWcVNfhUV2HJ1frgtytLZ/q0p2sIiIJpYAXEUmoJAX8zXEX0AvVdXhU1+HJ1bogd2vLm7oS0wYvIiKvl6QjeBER6UIBLyKSUIM+4HOpS2IzW2tmL5rZYjOrD6dVmdlDZvZy+H5Ulmq5zcy2mdnSLtN6rMUC3w334QtmNifLdX3JzDaG+21xeP9E57zrw7pWmtn5EdY1zsweNbMVZrbMzD4VTo91n/VRV6z7zMxKzOxZM1sS1vVv4fQJZrYg3F+/Cm9yxMyKw/HV4fy6LNd1u5m90mV/zQqnZ+13P9xegZk9b2b3huPR7i9/7fFVg+9FcAPVX4GJQBGwBJgWYz1rgRHdpn0D+Hw4/Hng61mq5XRgDrD0ULUAFwH3EfQfdDKwIMt1fQn4px6WnRb+mxYDE8J/64KI6hoDzAmHhwGrwu3Hus/6qCvWfRb+3GXhcCGwINwPdwLvCaf/EPh4OHwd8MNw+D3AryLaX73VdTtwWQ/LZ+13P9zeZ4BfAPeG45Hur8F+BD8YuiS+BPhJOPwT4NJsbNTdnwB29rOWS4CfeuAvQKWZjcliXb25BPhvd29x91eA1QT/5lHUtdndF4XDe4AVBL2fxrrP+qirN1nZZ+HPvTccLQxfDpwN3BVO776/OvfjXcA5ZtZTh4RR1dWbrP3um9lY4G3ALeG4EfH+GuwBn2tdEjvwoJktNLNrw2mj3H0zBP9ZgZGxVdd7LbmwH/8+/BP5ti7NWLHUFf45PJvg6C9n9lm3uiDmfRY2NywGtgEPEfy1sNvd23vY9mt1hfMbgeHZqMvdO/fXV8P99Z9mVty9rh5qHmjfBj4LZMLx4US8vwZ7wPerS+IsOtXd5xA8xeoTZnZ6jLUcjrj34w+AScAsYDPwzXB61usyszLgN8Cn3b2pr0V7mBZZbT3UFfs+c/cOd59F0FPsicDUPrYdW11mNh24HpgCzAeqgM9lsy4zuxjY5u4Lu07uY9sDUtdgD/h+dUmcLe6+KXzfBtxN8Eu/tfNPvvB9W1z19VFLrPvR3beG/ykzwI842KSQ1brMrJAgRO9w99+Gk2PfZz3VlSv7LKxlN/AYQRt2pZmle9j2a3WF8yvof1PdkdZ1QdjU5e7eAvyY7O+vU4F3mNlagqbkswmO6CPdX4M94HOmS2IzKzWzYZ3DwHnA0rCeq8PFrgZ+H0d9od5quQe4Kryi4GSgsbNZIhu6tXm+k2C/ddb1nvCKggnAMcCzEdVgwK3ACnf/VpdZse6z3uqKe5+ZWbWZVYbDQ4BzCc4PPApcFi7WfX917sfLgEc8PIOYhbpe6vIlbQTt3F33V+T/ju5+vbuPdfc6gpx6xN2vJOr9FdXZ4my9CM6CryJo//tCjHVMJLh6YQmwrLMWgnazh4GXw/eqLNXzS4I/3dsIjgY+1FstBH8Ofi/chy8C87Jc18/C7b4Q/mKP6bL8F8K6VgIXRljXWwn+BH4BWL9lBPoAAAR/SURBVBy+Lop7n/VRV6z7DJgBPB9ufynwL13+HzxLcHL310BxOL0kHF8dzp+Y5boeCffXUuDnHLzSJmu/+11qPJODV9FEur/UVYGISEIN9iYaERHphQJeRCShFPAiIgmlgBcRSSgFvIhIQingJTJm5mb2zS7j/2RmXxqgdd9uZpcdeskj3s7lFvTk+Gi36XUW9ohpZrOsS2+OA7DNSjO7rst4jZnd1ddnRHqigJcotQDvMrMRcRfSlZkVHMbiHwKuc/ez+lhmFsG16YdTQ7qP2ZUEvQkCwR3S7h75l5kkjwJeotRO8JzJf+g+o/sRuJntDd/PNLPHzexOM1tlZv9hZlda0Mf3i2Y2qctqzjWzJ8PlLg4/X2BmN5jZc2HHUh/tst5HzewXBDe0dK/nveH6l5rZ18Np/0Jwo9EPzeyGnn7A8A7qLwNXWNDP+BXhXc23hTU8b2aXhMteY2a/NrM/EHRKV2ZmD5vZonDbnT2h/gcwKVzfDd3+Wigxsx+Hyz9vZmd1Wfdvzex+C/oW/0aX/XF7+HO9aGZv+LeQ5OrrKEJkIHwPeKEzcPppJkHHVTuBNcAt7n6iBQ+7+CTw6XC5OuAMgk63HjWzycBVBLebz7egx8CnzezBcPkTgekedKP7GjOrAb4OzAV2EYTvpe7+ZTM7m6Df9fqeCnX31vCLYJ67/324vq8R3Fr+wfC2+WfN7E/hR04BZrj7zvAo/p3u3hT+lfMXM7uHoN/56R50mNXZi2SnT4TbPcHMpoS1HhvOm0XQ22QLsNLMbiTo/bLW3aeH66rse9dLkugIXiLlQc+HPwX+12F87DkPOodqIbiFvDOgXyQI9U53unvG3V8m+CKYQtAH0FUWdBe7gKCrgWPC5Z/tHu6h+cBj7t7gQdesdxA8mOTNOg/4fFjDYwS3nY8P5z3k7p2dRhnwNTN7AfgTQRexow6x7rcSdFOAu78ErAM6A/5hd29092ZgOXA0wX6ZaGY3mtkFQF89ZErC6AhesuHbwCKCXvw6tRMeYIQdQBV1mdfSZTjTZTzD639nu/ez4QSh+Ul3f6DrDDM7E9jXS30D/eAJA/7W3Vd2q+GkbjVcCVQDc929zYKeBkv6se7edN1vHUDa3XeZ2UzgfIKj/3cDH+zXTyGDno7gJXLhEeudBCcsO60laBKB4Ok1hW9i1ZebWSpsl59I0LnWA8DHLehiFzM71oLePfuyADjDzEaEJ2DfCzx+GHXsIXicXqcHgE+GX1yY2exePldB0Ed4W9iWfnQv6+vqCYIvBsKmmfEEP3ePwqaflLv/BvgiweMSJU8o4CVbvgl0vZrmRwSh+izQ/ci2v1YSBPF9wMfCpolbCJonFoUnJm/iEH+petA97PUEXbcuARa5++F06/woMK3zJCvwFYIvrBfCGr7Sy+fuAOZZ8ID2K4GXwnp2EJw7WNrDyd3vAwVm9iLwK+CasCmrN7XAY2Fz0e3hzyl5Qr1JiogklI7gRUQSSgEvIpJQCngRkYRSwIuIJJQCXkQkoRTwIiIJpYAXEUmo/w+aU9iboE99qAAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Initialize (Adjusting these values can change how the effectiveness\n",
"# of our minimization as seen on our graph)\n",
"alpha = 0.01\n",
"num_iters = 400\n",
"\n",
"# initialize theta and run Gradient Descent\n",
"theta = np.zeros(3)\n",
"theta, J_history = gradientDescent(X,y,theta,alpha,num_iters)\n",
"\n",
"# Graph it\n",
"pyplot.plot(np.arange(len(J_history)), J_history)\n",
"pyplot.xlabel(\"Number of Iterations\")\n",
"pyplot.ylabel(\"Cost J\")\n",
"\n",
"# Resulting theta\n",
"print('theta computed from gradient descent: {:s}'.format(str(theta)))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"