4.1 Movie Recommendations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Movie Recommendations\n",
    "\n",
    "In this case study we will build multiple model for making recommendations, validate and compare them.\n",
    "\n",
    "## Identification Information\n",
    "\n",
    "Name: Alex Perusse\n",
    "\n",
    "MIT xPro Username: aperusse\n",
    "\n",
    "MIT xPro Email: aperusse@cityyear.org\n",
    "\n",
    "## Data Exporation\n",
    "\n",
    "To begin we import the data and perform some cursory data exploration.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "from surprise import Dataset, SVD, NormalPredictor, BaselineOnly, KNNBasic, NMF\n",
    "from surprise.model_selection import cross_validate, KFold"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:3: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\\t'.\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n"
     ]
    }
   ],
   "source": [
    "ratingsdata = Dataset.load_builtin('ml-100k')\n",
    "col_names = ['user_id', 'item_id', 'rating', 'timestamp']\n",
    "rawdata = pd.read_table(ratingsdata.ratings_file, names=col_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>100000.00000</td>\n",
       "      <td>100000.000000</td>\n",
       "      <td>100000.000000</td>\n",
       "      <td>1.000000e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>462.48475</td>\n",
       "      <td>425.530130</td>\n",
       "      <td>3.529860</td>\n",
       "      <td>8.835289e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>266.61442</td>\n",
       "      <td>330.798356</td>\n",
       "      <td>1.125674</td>\n",
       "      <td>5.343856e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.747247e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>254.00000</td>\n",
       "      <td>175.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>8.794487e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>447.00000</td>\n",
       "      <td>322.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.828269e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>682.00000</td>\n",
       "      <td>631.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>8.882600e+08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>943.00000</td>\n",
       "      <td>1682.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>8.932866e+08</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            user_id        item_id         rating     timestamp\n",
       "count  100000.00000  100000.000000  100000.000000  1.000000e+05\n",
       "mean      462.48475     425.530130       3.529860  8.835289e+08\n",
       "std       266.61442     330.798356       1.125674  5.343856e+06\n",
       "min         1.00000       1.000000       1.000000  8.747247e+08\n",
       "25%       254.00000     175.000000       3.000000  8.794487e+08\n",
       "50%       447.00000     322.000000       4.000000  8.828269e+08\n",
       "75%       682.00000     631.000000       4.000000  8.882600e+08\n",
       "max       943.00000    1682.000000       5.000000  8.932866e+08"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rawdata.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEWCAYAAACNJFuYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG6ZJREFUeJzt3X20XXV95/H3xwBKiwqayNAEDG2zOqKrRoyQLmdaFAcC2kJX1WJbiQ6dVAtTndppwT6gKB07nWqHqeLCkgK2iAytJWWCMVWotaOBKM9SSwYRUlKIhkex2OB3/ti/q6fZ5+aee5PccyHv11pn3X2++7f3+Z4NN5+7H84+qSokSRr0tHE3IEmaewwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1GA560kvy80k+NcbX/3CS395N6zosyaNJ5rXn1yb5xd2x7ra+q5Os3F3r01NX/JyDZluSu4CDgSeAR4FPAmdU1aMjLLsY+Cqwb1Vt33Ndfvf17qLrdTtdv18GLgEuqKrvzGBdv1hVfz2NZa4F/rSq/ng6r9WWfRfww1X1C9NdVnLPQePyk1V1ALAUeAlw1pj72ZmfrKpnAs8H3gf8BnDh7n6RJPvs7nVKM2U4aKyq6p+AdXQhAUCSVye5IcnDSe5pfwFP+Gz7+WA7/PJjSd6U5HMDy1eStyS5I8kDST6YJG3evCR/kOTrSb6a5Iw2fsp/mKvqoapaA/wssDLJi9o6L0ry3jY9P8lVSR5Msi3J3yZ5WpKPAocBf9X6/vUki9trn5bkbuAzA7XBfn4oyXVJHkpyZZLntNc6JsnmwR6T3JXkVUlWAO8Efra93k1t/ncPU7W+fivJ15Lcn+SSJM9u8yb6WJnk7ra9fnOqbaSnDsNBY5VkEXACsGmg/E3gVOBA4NXAW5Oc3Ob9ePt5YFUdUFWfn2TVrwFeBrwYeD1wfKv/p/Z6S4EjgZOHLr0TVXUdsBn490Nmv6PNW0B3OOqd3SL1RuBu2h5TVf33gWV+AnjBQI87OhX4j8AP0B3eOm+EHj8J/C7w8fZ6Lx4y7E3t8QrgB4EDgD/aYcy/A34EOBb4nSQvmOq19dRgOGhc/jLJI8A9wP3A2RMzquraqrqlqr5TVTcDH6P7B3Q63ldVD1bV3cA1fG/P5PXA/6yqzVX1AN1hopm4F3jOkPq/AIcAz6+qf6mqv62pT+y9q6q+WVXfmmT+R6vq1qr6JvDbwOsnTljvop8H3l9Vd7bzPWcBp+yw1/LuqvpWVd0E3EQXttoLGA4al5PbcfxjgH8LzJ+YkeToJNck2ZrkIeAtg/NH9E8D04/R/VUM3V/f9wzMG5yejoXAtiH136fbC/pUkjuTnDnCuqbqYXD+14B9mf72GOYH2voG170P3R7PhMm2o57iDAeNVVX9DXAR8D8GypcCa4BDq+rZwIeBTCyyiy+5BVg08PzQ6a4gycvowuFzO86rqkeq6h1V9YPATwK/muTYidmTrHKq9zTY42F0eydfpzv89n0Dfc2jO5w16nrvpTvJPrju7cB9UyynvYDhoLngD4H/kGTi0M8zgW1V9c9JjgJ+bmDsVuA7dMfIZ+Jy4G1JFiY5kO7Ko5EkeVaS1wCX0V1eesuQMa9J8sPtBPjDdJe/PtFm3zfDvn8hyRFJvg84B7iiqp4A/gF4RjuBvy/wW8DTB5a7D1icZLLf848B/yXJ4UkO4HvnKPb4JcKa+wwHjV1VbaX77MDEB8l+GTinnZP4Hbp/0CfGPgacC/xduyJo+TRf7iPAp4CbgRuAtXzvMwyT+auB8yO/CbwfePMkY5cAf033+Y3PAx+qqmvbvP8G/Fbr+9em0fNH6fau/gl4BvAr0F09Rbet/hj4R7o9icGrl/53+/mNJF8ast7Vbd2fpfvsyD8D/3kafekpzA/Baa+W5ATgw1X1/CkHS3sR9xy0V0myf5ITk+yTZCHdVVKfGHdf0lzjnoP2Ku24/d/QXSH1LeD/AG+rqofH2pg0xxgOkqQeDytJknqetDf6mj9/fi1evHjcbUjSk8b8+fNZt27duqpaMdXYJ204LF68mI0bN467DUl6Ukky0qfrPawkSeqZMhySPKPdLvimJLcleXerX9RueXxjeyxt9SQ5L8mmJDcnOXJgXSvbbZTvyMC3USV5aZJb2jLnTdxeWZI0HqMcVnoceGVVPdo+ov+5JFe3ef+1qq7YYfwJdJ8SXQIcDZwPHN3uQX82sIzuni9fTLKm3RnzfGAV8AW6T6yuAK5GkjQWU+45VGfi6xv3bY+dXf96EnBJW+4LwIFJDqG7V/36qtrWAmE9sKLNe1ZVfb7d2vgSZnCPfUnS7jPSOYd03551I91999dX1YY269x26OgDSSZu+LWQf32L4c2ttrP65iH1YX2sSrIxycatW7eO0rokaQZGCoeqeqKqltLd6viodF+PeBbdp0xfRvelJxN3txx2vqBmUB/WxwVVtayqli1YsGDYEEnSbjCtq5Wq6kHgWmBFVW1ph44eB/4EOKoN28y/vv/8Irr7xu+svmhIXZI0JqNcrbSg3feeJPsDrwL+vp0roF1ZdDJwa1tkDXBqu2ppOfBQVW2h+xL545IclOQg4DhgXZv3SJLlbV2nAlfu3rcpSZqOUa5WOgS4uH3L1NOAy6vqqiSfSbKA7rDQjXRf5Qjd1UYn0n1V4mO0+95X1bYk7wGub+POqaqJr1l8K9396venu0rJK5UkaYyetDfeW7ZsWfkJaelfu3TD3eNuYVb93NGHjbuFJ50kX6yqZVON8xPSkqQew0GS1GM4SJJ6DAdJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1GA6SpB7DQZLUYzhIknoMB0lSj+EgSeqZMhySPCPJdUluSnJbkne3+uFJNiS5I8nHk+zX6k9vzze1+YsH1nVWq38lyfED9RWttinJmbv/bUqSpmOUPYfHgVdW1YuBpcCKJMuB3wM+UFVLgAeA09r404AHquqHgQ+0cSQ5AjgFeCGwAvhQknlJ5gEfBE4AjgDe0MZKksZkynCozqPt6b7tUcArgSta/WLg5DZ9UntOm39skrT6ZVX1eFV9FdgEHNUem6rqzqr6NnBZGytJGpORzjm0v/BvBO4H1gP/D3iwqra3IZuBhW16IXAPQJv/EPDcwfoOy0xWH9bHqiQbk2zcunXrKK1LkmZgpHCoqieqaimwiO4v/RcMG9Z+ZpJ5060P6+OCqlpWVcsWLFgwdeOSpBmZ1tVKVfUgcC2wHDgwyT5t1iLg3ja9GTgUoM1/NrBtsL7DMpPVJUljMsrVSguSHNim9wdeBdwOXAO8tg1bCVzZpte057T5n6mqavVT2tVMhwNLgOuA64El7eqn/ehOWq/ZHW9OkjQz+0w9hEOAi9tVRU8DLq+qq5J8GbgsyXuBG4AL2/gLgY8m2US3x3AKQFXdluRy4MvAduD0qnoCIMkZwDpgHrC6qm7bbe9QkjRtU4ZDVd0MvGRI/U668w871v8ZeN0k6zoXOHdIfS2wdoR+JUmzwE9IS5J6DAdJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6hnlxnvSk9KlG+4edwvSk5Z7DpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1GA6SpJ4pwyHJoUmuSXJ7ktuSvK3V35XkH5Pc2B4nDixzVpJNSb6S5PiB+opW25TkzIH64Uk2JLkjyceT7Le736gkaXSj7DlsB95RVS8AlgOnJzmizftAVS1tj7UAbd4pwAuBFcCHksxLMg/4IHACcATwhoH1/F5b1xLgAeC03fT+JEkzMGU4VNWWqvpSm34EuB1YuJNFTgIuq6rHq+qrwCbgqPbYVFV3VtW3gcuAk5IEeCVwRVv+YuDkmb4hSdKum9Y5hySLgZcAG1rpjCQ3J1md5KBWWwjcM7DY5labrP5c4MGq2r5Dfdjrr0qyMcnGrVu3Tqd1SdI0jBwOSQ4A/hx4e1U9DJwP/BCwFNgC/MHE0CGL1wzq/WLVBVW1rKqWLViwYNTWJUnTNNItu5PsSxcMf1ZVfwFQVfcNzP8IcFV7uhk4dGDxRcC9bXpY/evAgUn2aXsPg+MlSWMwytVKAS4Ebq+q9w/UDxkY9tPArW16DXBKkqcnORxYAlwHXA8saVcm7Ud30npNVRVwDfDatvxK4Mpde1uSpF0xyp7Dy4E3ArckubHV3kl3tdFSukNAdwG/BFBVtyW5HPgy3ZVOp1fVEwBJzgDWAfOA1VV1W1vfbwCXJXkvcANdGEmSxmTKcKiqzzH8vMDanSxzLnDukPraYctV1Z10VzNJkuYAPyEtSeoxHCRJPYaDJKnHcJAk9RgOkqQew0GS1GM4SJJ6DAdJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6hnla0IlaU66dMPd425h1v3c0YfNyuu45yBJ6jEcJEk9hoMkqWfKcEhyaJJrktye5LYkb2v15yRZn+SO9vOgVk+S85JsSnJzkiMH1rWyjb8jycqB+kuT3NKWOS9J9sSblSSNZpQ9h+3AO6rqBcBy4PQkRwBnAp+uqiXAp9tzgBOAJe2xCjgfujABzgaOBo4Czp4IlDZm1cByK3b9rUmSZmrKcKiqLVX1pTb9CHA7sBA4Cbi4DbsYOLlNnwRcUp0vAAcmOQQ4HlhfVduq6gFgPbCizXtWVX2+qgq4ZGBdkqQxmNY5hySLgZcAG4CDq2oLdAECPK8NWwjcM7DY5lbbWX3zkLokaUxGDockBwB/Dry9qh7e2dAhtZpBfVgPq5JsTLJx69atU7UsSZqhkcIhyb50wfBnVfUXrXxfOyRE+3l/q28GDh1YfBFw7xT1RUPqPVV1QVUtq6plCxYsGKV1SdIMjHK1UoALgdur6v0Ds9YAE1ccrQSuHKif2q5aWg481A47rQOOS3JQOxF9HLCuzXskyfL2WqcOrEuSNAaj3D7j5cAbgVuS3Nhq7wTeB1ye5DTgbuB1bd5a4ERgE/AY8GaAqtqW5D3A9W3cOVW1rU2/FbgI2B+4uj0kSWMyZThU1ecYfl4A4Ngh4ws4fZJ1rQZWD6lvBF40VS+SpNnhJ6QlST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1GA6SpB7DQZLUYzhIknoMB0lSj+EgSeoxHCRJPYaDJKnHcJAk9RgOkqQew0GS1GM4SJJ6pgyHJKuT3J/k1oHau5L8Y5Ib2+PEgXlnJdmU5CtJjh+or2i1TUnOHKgfnmRDkjuSfDzJfrvzDUqSpm+UPYeLgBVD6h+oqqXtsRYgyRHAKcAL2zIfSjIvyTzgg8AJwBHAG9pYgN9r61oCPACctitvSJK066YMh6r6LLBtxPWdBFxWVY9X1VeBTcBR7bGpqu6sqm8DlwEnJQnwSuCKtvzFwMnTfA+SpN1sV845nJHk5nbY6aBWWwjcMzBmc6tNVn8u8GBVbd+hPlSSVUk2Jtm4devWXWhdkrQzMw2H84EfApYCW4A/aPUMGVszqA9VVRdU1bKqWrZgwYLpdSxJGtk+M1moqu6bmE7yEeCq9nQzcOjA0EXAvW16WP3rwIFJ9ml7D4PjJUljMqM9hySHDDz9aWDiSqY1wClJnp7kcGAJcB1wPbCkXZm0H91J6zVVVcA1wGvb8iuBK2fSkyRp95lyzyHJx4BjgPlJNgNnA8ckWUp3COgu4JcAquq2JJcDXwa2A6dX1RNtPWcA64B5wOqquq29xG8AlyV5L3ADcOFue3eSpBmZMhyq6g1DypP+A15V5wLnDqmvBdYOqd9JdzWTJGmO8BPSkqQew0GS1GM4SJJ6DAdJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1zOg7pPXkc+mGu8fdgqQnEfccJEk9hoMkqcdwkCT1GA6SpJ4pwyHJ6iT3J7l1oPacJOuT3NF+HtTqSXJekk1Jbk5y5MAyK9v4O5KsHKi/NMktbZnzkmR3v0lJ0vSMsudwEbBih9qZwKeragnw6fYc4ARgSXusAs6HLkyAs4GjgaOAsycCpY1ZNbDcjq8lSZplU4ZDVX0W2LZD+STg4jZ9MXDyQP2S6nwBODDJIcDxwPqq2lZVDwDrgRVt3rOq6vNVVcAlA+uSJI3JTM85HFxVWwDaz+e1+kLgnoFxm1ttZ/XNQ+pDJVmVZGOSjVu3bp1h65KkqezuE9LDzhfUDOpDVdUFVbWsqpYtWLBghi1KkqYy03C4rx0Sov28v9U3A4cOjFsE3DtFfdGQuiRpjGYaDmuAiSuOVgJXDtRPbVctLQceaoed1gHHJTmonYg+DljX5j2SZHm7SunUgXVJksZkynsrJfkYcAwwP8lmuquO3gdcnuQ04G7gdW34WuBEYBPwGPBmgKraluQ9wPVt3DlVNXGS+610V0TtD1zdHpKkMZoyHKrqDZPMOnbI2AJOn2Q9q4HVQ+obgRdN1Yckafb4CWlJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1GA6SpB7DQZLUYzhIknoMB0lSj+EgSeqZ8mtCn4ou3XD3uFuQpDnNPQdJUo/hIEnq2aVwSHJXkluS3JhkY6s9J8n6JHe0nwe1epKcl2RTkpuTHDmwnpVt/B1JVu7aW5Ik7ardsefwiqpaWlXL2vMzgU9X1RLg0+05wAnAkvZYBZwPXZgAZwNHA0cBZ08EiiRpPPbEYaWTgIvb9MXAyQP1S6rzBeDAJIcAxwPrq2pbVT0ArAdW7IG+JEkj2tVwKOBTSb6YZFWrHVxVWwDaz+e1+kLgnoFlN7faZHVJ0pjs6qWsL6+qe5M8D1if5O93MjZDarWTen8FXQCtAjjssMOm26skaUS7tOdQVfe2n/cDn6A7Z3BfO1xE+3l/G74ZOHRg8UXAvTupD3u9C6pqWVUtW7Bgwa60LknaiRmHQ5LvT/LMiWngOOBWYA0wccXRSuDKNr0GOLVdtbQceKgddloHHJfkoHYi+rhWkySNya4cVjoY+ESSifVcWlWfTHI9cHmS04C7gde18WuBE4FNwGPAmwGqaluS9wDXt3HnVNW2XehLkrSLZhwOVXUn8OIh9W8Axw6pF3D6JOtaDayeaS+SpN3LT0hLknoMB0lSj+EgSeoxHCRJPYaDJKnHcJAk9RgOkqQew0GS1GM4SJJ6DAdJUo/hIEnqMRwkST2GgySpx3CQJPUYDpKkHsNBktRjOEiSegwHSVKP4SBJ6jEcJEk9hoMkqcdwkCT1zJlwSLIiyVeSbEpy5rj7kaS92ZwIhyTzgA8CJwBHAG9IcsR4u5KkvdecCAfgKGBTVd1ZVd8GLgNOGnNPkrTX2mfcDTQLgXsGnm8Gjt5xUJJVwKr29NEkX5nh680Hvj7DZfck+5oe+5oe+5qeOdnXz+9aXyMvN1fCIUNq1StUXQBcsMsvlmysqmW7up7dzb6mx76mx76mZ2/va64cVtoMHDrwfBFw75h6kaS93lwJh+uBJUkOT7IfcAqwZsw9SdJea04cVqqq7UnOANYB84DVVXXbHnzJXT40tYfY1/TY1/TY1/Ts1X2lqndoX5K0l5srh5UkSXOI4SBJ6nnKhkOS1UnuT3LrJPOT5Lx2u46bkxw5R/o6JslDSW5sj9+Zpb4OTXJNktuT3JbkbUPGzPo2G7GvWd9mSZ6R5LokN7W+3j1kzNOTfLxtrw1JFs+Rvt6UZOvA9vrFPd3XwGvPS3JDkquGzJv17TViX2PZXknuSnJLe82NQ+bv2d/HqnpKPoAfB44Ebp1k/onA1XSfsVgObJgjfR0DXDWG7XUIcGSbfibwD8AR495mI/Y169usbYMD2vS+wAZg+Q5jfhn4cJs+Bfj4HOnrTcAfzfb/Y+21fxW4dNh/r3FsrxH7Gsv2Au4C5u9k/h79fXzK7jlU1WeBbTsZchJwSXW+AByY5JA50NdYVNWWqvpSm34EuJ3uk+uDZn2bjdjXrGvb4NH2dN/22PHqjpOAi9v0FcCxSYZ94HO2+xqLJIuAVwN/PMmQWd9eI/Y1V+3R38enbDiMYNgtO8b+j07zY+2wwNVJXjjbL952519C91fnoLFus530BWPYZu1QxI3A/cD6qpp0e1XVduAh4LlzoC+An2mHIq5IcuiQ+XvCHwK/Dnxnkvlj2V4j9AXj2V4FfCrJF9PdOmhHe/T3cW8Oh5Fu2TEGXwKeX1UvBv4X8Jez+eJJDgD+HHh7VT284+whi8zKNpuir7Fss6p6oqqW0n2i/6gkL9phyFi21wh9/RWwuKp+FPhrvvfX+h6T5DXA/VX1xZ0NG1Lbo9trxL5mfXs1L6+qI+nuVn16kh/fYf4e3V57czjMyVt2VNXDE4cFqmotsG+S+bPx2kn2pfsH+M+q6i+GDBnLNpuqr3Fus/aaDwLXAit2mPXd7ZVkH+DZzOIhxcn6qqpvVNXj7elHgJfOQjsvB34qyV10d11+ZZI/3WHMOLbXlH2NaXtRVfe2n/cDn6C7e/WgPfr7uDeHwxrg1HbGfznwUFVtGXdTSf7NxHHWJEfR/Tf6xiy8boALgdur6v2TDJv1bTZKX+PYZkkWJDmwTe8PvAr4+x2GrQFWtunXAp+pdiZxnH3tcFz6p+jO4+xRVXVWVS2qqsV0J5s/U1W/sMOwWd9eo/Q1ju2V5PuTPHNiGjgO2PEKxz36+zgnbp+xJyT5GN1VLPOTbAbOpjs5R1V9GFhLd7Z/E/AY8OY50tdrgbcm2Q58CzhlT/+CNC8H3gjc0o5XA7wTOGygt3Fss1H6Gsc2OwS4ON0XVT0NuLyqrkpyDrCxqtbQhdpHk2yi+wv4lD3c06h9/UqSnwK2t77eNAt9DTUHttcofY1jex0MfKL9zbMPcGlVfTLJW2B2fh+9fYYkqWdvPqwkSZqE4SBJ6jEcJEk9hoMkqcdwkCT1GA7SLkry9iTfN/B87cRnDaQnKy9llUbQPmSXqurdf6d9unZZVX191huT9hD3HKRJJFmc7nskPkR3/6YLk2zMwPckJPkV4AeAa5Jc02p3JZk/sPxH2jKfap9aJsnL2o3cPp/k9zPJ93tI42I4SDv3I3S3RX4J8I6qWgb8KPATSX60qs6ju5/NK6rqFUOWXwJ8sKpeCDwI/Eyr/wnwlqr6MeCJPf4upGkyHKSd+1q7Vz7A65N8CbgBeCFwxAjLf7WqJm778UVgcTsf8cyq+r+tfulu7VjaDZ6y91aSdpNvAiQ5HPg14GVV9UCSi4BnjLD84wPTTwD7M/xWy9Kc4p6DNJpn0QXFQ0kOprvH/oRH6L7CdCRV9QDwSLuTJozxBnPSZNxzkEZQVTcluQG4DbgT+LuB2RcAVyfZMsl5h2FOAz6S5Jt037nw0O7sV9pVXsoqjUGSAya+oCjJmcAhVfW2MbclfZd7DtJ4vDrJWXS/g19jjN+pIA3jnoMkqccT0pKkHsNBktRjOEiSegwHSVKP4SBJ6vn/kEUq8GTQcwUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(rawdata['rating'], bins=5, kde=False)\n",
    "plt.title(\"Rating Distribution\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 1: Data Analysis\n",
    "\n",
    "**Describe the dataset. How many ratings are in the dataset? How would you describe the distribution of ratings? Is there anything else we should observe? Make sure the histogram is visible in the notebook.**\n",
    "\n",
    "In this dataset, we are given 100000 User/Item/Rating pairings, from which we hope to make recommendations.  Ratings are given on a scale from 1 to 5, where 5 is the most positive rating. As you might expect, ratings are distributed such that a 4 is the most likely response followed by a 3 or a 5.  1's or 2's are rarer and thus probably carry a little more information than a 4.  \n",
    "\n",
    "## Baseline Model - Random\n",
    "\n",
    "As our baseline we will use `NormalPredictor`, which simply predicts a random rating based on the distribution of the training set, which is assumed to be normal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE Over 5 Folds: 1.5252159899507298 +/- 0.05840238755785964\n"
     ]
    }
   ],
   "source": [
    "model_baseline = NormalPredictor()\n",
    "results_baseline = cross_validate(model_baseline, ratingsdata, measures=['RMSE'], cv=100, verbose=False)\n",
    "m, s = results_baseline['test_rmse'].mean(), 2*results_baseline['test_rmse'].std()\n",
    "print(f\"RMSE Over 5 Folds: {m} +/- {s}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## User-Based Collaborative Filtering\n",
    "\n",
    "User-based Collaborative Filtering works on the premise that people who rated things similarly in the past will do so in the future.  It makes predictions by averaging the ratings that similar users would have given the item."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE Over 100 Folds: 0.9670531814311453 +/- 0.050096632817287025\n"
     ]
    }
   ],
   "source": [
    "model_user = KNNBasic(sim_options={'user_based': True}, verbose=False)\n",
    "results_user = cross_validate(model_user, ratingsdata, measures=['RMSE'], cv=100, verbose=False)\n",
    "m, s = results_user['test_rmse'].mean(), 2*results_user['test_rmse'].std()\n",
    "print(f\"RMSE Over 100 Folds: {m} +/- {s}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Item-Based Collaborative Filtering\n",
    "\n",
    "Similarly, item-based collaborative filtering makes predictions about items using items which have receieved similar ratings. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE Over 100 Folds: 0.9607477776787791 +/- 0.0458838205226042\n"
     ]
    }
   ],
   "source": [
    "model_item = KNNBasic(sim_options={'user_based': False}, verbose=False)\n",
    "results_item = cross_validate(model_item, ratingsdata, measures=['RMSE'], cv=100, verbose=False)\n",
    "m, s = results_item['test_rmse'].mean(), 2*results_item['test_rmse'].std()\n",
    "print(f\"RMSE Over 100 Folds: {m} +/- {s}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 2: Collaborative Filtering Models\n",
    "\n",
    "**Compare the results from the user-user and item-item models. How do they compare to each other? How do they compare to our original \"random\" model? Can you provide any intuition as to why the results came out the way they did?**\n",
    "\n",
    "Atleast in our run, the results weren't that different with user-based and item-based collaborative filtering with item-based only barely having an advantage, $0.961 < 0.966$. However, both were significantly better than the random model whose RMSE was considerably higher, $0.966 < 1.519$, than either of the collaborative filtering based approahces.  This makes sense of course, because the random results are... random. It gives a nice sense of what would happen if we simply rolled a die to make recommendations as opposed to using the data we have.\n",
    "\n",
    "## Model 4: Matrix Factorization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE Over 50 Folds: 0.9242514125226332 +/- 0.031770023605667906\n"
     ]
    }
   ],
   "source": [
    "model_svd = SVD()\n",
    "folds = 50\n",
    "results_item = cross_validate(model_svd, ratingsdata, measures=['RMSE'], cv=folds, verbose=False)\n",
    "m, s = results_item['test_rmse'].mean(), 2*results_item['test_rmse'].std()\n",
    "print(f\"RMSE Over {folds} Folds: {m} +/- {s}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 3: Matrix Factorization Model\n",
    "\n",
    "**The matrix factorization model is different from the collaborative filtering models. Briefly describe this difference. Also, compare the RMSE again. Does it improve? Can you offer any reasoning as to why that might be?**\n",
    "\n",
    "Where Collaborative filtering takes advantage of nearest neighbor to try to make recomendations to user based on similar users or items based on similar items, matrix factorization is useful when you believe there are latent factors, that is features not directly represented in the data, that are truly the levers in someones preferences.  For example, perhaps there are certain actors or actressses who appears in movies which cause them to be popular.  Matrix Factorization allows one to model this possiblity and try to derive that matrix of latent factors.  \n",
    "\n",
    "We in fact did see a improvement in performance by using matrix factorication by a small margin, $0.924 < 0.961$.\n",
    "\n",
    "## Taking Model Criticism Further: Precision and Recall\n",
    "\n",
    "Going beyond the RMSE, we can evaluate the precision and recall of our recommender as well in a similar fachion to how one does in a classification problem. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def precision_recall_at_k(predictions, k=10, threshold=3.5):\n",
    "    '''Return precision and recall at k metrics for each user.'''\n",
    "\n",
    "    # First map the predictions to each user.\n",
    "    user_est_true = dict()\n",
    "    for uid, _, true_r, est, _ in predictions:\n",
    "        current = user_est_true.get(uid, list())\n",
    "        current.append((est, true_r))\n",
    "        user_est_true[uid] = current\n",
    "\n",
    "    precisions = dict()\n",
    "    recalls = dict()\n",
    "    for uid, user_ratings in user_est_true.items():\n",
    "\n",
    "        # Sort user ratings by estimated value\n",
    "        user_ratings.sort(key=lambda x: x[0], reverse=True)\n",
    "\n",
    "        # Number of relevant items\n",
    "        n_rel = sum((true_r >= threshold) for (_, true_r) in user_ratings)\n",
    "\n",
    "        # Number of recommended items in top k\n",
    "        n_rec_k = sum((est >= threshold) for (est, _) in user_ratings[:k])\n",
    "\n",
    "        # Number of relevant and recommended items in top k\n",
    "        n_rel_and_rec_k = sum(((true_r >= threshold) and (est >= threshold))\n",
    "                              for (est, true_r) in user_ratings[:k])\n",
    "\n",
    "        # Precision@K: Proportion of recommended items that are relevant\n",
    "        precisions[uid] = n_rel_and_rec_k / n_rec_k if n_rec_k != 0 else 1\n",
    "\n",
    "        # Recall@K: Proportion of relevant items that are recommended\n",
    "        recalls[uid] = n_rel_and_rec_k / n_rel if n_rel != 0 else 1\n",
    "\n",
    "    return precisions, recalls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "baseline\n",
      "Over 5-folds\n",
      "Average Precision: 0.6012145022718249\n",
      "Average Recall: 0.2707458905346869\n",
      "Over 10-folds\n",
      "Average Precision: 0.602457203177272\n",
      "Average Recall: 0.2709060915008587\n",
      "user\n",
      "Over 5-folds\n",
      "Average Precision: 0.8443762436192029\n",
      "Average Recall: 0.27138036262948206\n",
      "Over 10-folds\n",
      "Average Precision: 0.8457641624272216\n",
      "Average Recall: 0.27219540289037625\n",
      "item\n",
      "Over 5-folds\n",
      "Average Precision: 0.9425407266620075\n",
      "Average Recall: 0.17057555577493896\n",
      "Over 10-folds\n",
      "Average Precision: 0.9428893250490159\n",
      "Average Recall: 0.17027643050263264\n",
      "SVD\n",
      "Over 5-folds\n",
      "Average Precision: 0.8733167399522237\n",
      "Average Recall: 0.26416978079592945\n",
      "Over 10-folds\n",
      "Average Precision: 0.8723540686805373\n",
      "Average Recall: 0.2611656280625076\n"
     ]
    }
   ],
   "source": [
    "algos = {'baseline': model_baseline, 'user': model_user, 'item': model_item, 'SVD': model_svd}\n",
    "\n",
    "for name, algo in algos.items():\n",
    "    print(name)\n",
    "    p_arr = []\n",
    "    r_arr = []\n",
    "    for folds in [5, 10]:\n",
    "        print(f\"Over {folds}-folds\")\n",
    "        kf = KFold(n_splits=5)\n",
    "        fold = 0\n",
    "        for trainset, testset in kf.split(ratingsdata):\n",
    "            algo.fit(trainset)\n",
    "            predictions = algo.test(testset)\n",
    "            precisions, recalls = precision_recall_at_k(predictions, k=5, threshold=4)\n",
    "            fold += 1\n",
    "\n",
    "            # Precision and recall can then be averaged over all users\n",
    "            #print(f\"Fold {fold}\")\n",
    "            p = sum(prec for prec in precisions.values()) / len(precisions)\n",
    "            r = sum(rec for rec in recalls.values()) / len(recalls)\n",
    "            p_arr.append(p)\n",
    "            r_arr.append(r)\n",
    "        print(f\"Average Precision: {np.mean(p_arr)}\")\n",
    "        print(f\"Average Recall: {np.mean(r_arr)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 4: Precision/Recall\n",
    "\n",
    "**Compute the precision and recall, for each of the 4 models, at k = 5 and 10. This is 2 x 2 x 4 = 16 numerical values. Do you note anything interesting about these values? Anything differerent from the RMSE values you computed above?**\n",
    "\n",
    "As you would expect, the random model performs the worst in terms of precision of all of the models, although not worse in Recall.  That brings us to the user and item collaborative filters.  Here we see a class tradeoff in precision and recall.  Item-based collaborative filtering gets the higher precision but at the cost of having the worst recall.  This is likely because the item-based model returns a lot of negative recommendations and few positives for any user.  As a result, there are fewer false positives because there are also fewer positives identified by the model at all.  You might interpret this as more conservative behavior, only recommending what you like.\n",
    "\n",
    "The SVD plays the balance the best, with a higher precision than user-based and with nearly the same recall. \n",
    "\n",
    "From our analysis thus far, I would continue to conlude that matrix factorization seems to be the best model that we've tried."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_top_n(predictions, n=5):\n",
    "    '''Return the top-N recommendation for each user from a set of predictions.\n",
    "\n",
    "    Args:\n",
    "        predictions(list of Prediction objects): The list of predictions, as\n",
    "            returned by the test method of an algorithm.\n",
    "        n(int): The number of recommendation to output for each user. Default\n",
    "            is 10.\n",
    "\n",
    "    Returns:\n",
    "    A dict where keys are user (raw) ids and values are lists of tuples:\n",
    "        [(raw item id, rating estimation), ...] of size n.\n",
    "    '''\n",
    "\n",
    "    # First map the predictions to each user.\n",
    "    top_n = dict()\n",
    "    for uid, iid, true_r, est, _ in predictions:\n",
    "        current = top_n.get(uid, [])\n",
    "        current.append((iid, est))\n",
    "        top_n[uid] = current\n",
    "\n",
    "    # Then sort the predictions for each user and retrieve the k highest ones.\n",
    "    for uid, user_ratings in top_n.items():\n",
    "        user_ratings.sort(key=lambda x: x[1], reverse=True)\n",
    "        top_n[uid] = user_ratings[:n]\n",
    "\n",
    "    return top_n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainset = ratingsdata.build_full_trainset()\n",
    "model_baseline.fit(trainset)\n",
    "predictions = model_baseline.test(trainset.build_anti_testset())\n",
    "topn = get_top_n(predictions, n=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('768', 5), ('277', 5), ('526', 5), ('512', 5), ('480', 5)]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Predictions for user `1`\n",
    "topn['1']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question 5: Top N Predicions\n",
    "\n",
    "**Do the top n predictions that you received make sense? What is the rating value (1-5) of these predictions? How could you use these predictions in the real-world if you were trying to build a generic content recommender system for a company?**\n",
    "\n",
    "Scanning through the predictions our model is returning 5 recommended items all with a predicted rating of 5. This is exactly what I would expect, since it should be the five highest rated items for that individual. \n",
    "\n",
    "In order to leverage this type of system in production for a real-world company my strategy would be to generate a much large set of potential recommendation for each user, say the top 100, and then randomly select a subset of those to actually recommend to the user.  This would help ensure that user gets different recommendations each time they come to the site, which helps support exploration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}