AV ML Hikeathon Predicting Links - Prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](hike.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ML Hikeathon\n",
    "\n",
    "## Problem Statement\n",
    "\n",
    "**Link Prediction on Hike’s Social Network**\n",
    "Social networks are highly dynamic; they grow and change quickly over time through the addition of new edges, signifying the appearance of new interactions in the underlying social structure. The fundamental computational problem underlying social-network evolution is the Link Prediction problem:\n",
    " \n",
    "\n",
    "Hike is a social platform and predicting links in their network forms the basis for recommending new friends to our users with whom they can possibly start a chat. High-quality recommendations help in strengthening the network by aiding the creation of new social connections between existing users. It also helps in the retention of new users by helping them find friends as they join the platform.\n",
    "\n",
    "Can you develop an algorithm to predict whether a Hike user will chat another Hike user who is part of his/her phone contact book?\n",
    "\n",
    " \n",
    "![title](pic.png)\n",
    "\n",
    "\n",
    "## Data Description\n",
    "The data for this competition i\n",
    "s a subset of the Hike’s social graph and the anonymised features of users. Explicitly, the training data is of the following form:\n",
    "train.zip contains 2 files namely **train.csv, user_features.csv**\n",
    "\n",
    "**train.csv**\n",
    "**node1_id, node2_id, is_chat**\n",
    "Where node1_id and node2_id are anonymised identifiers for users who are in each other’s phone address book. is_chat signifies their chat relationship. is_chat is 1, if the first user sends a chat message with the second user, and 0 otherwise.\n",
    "\n",
    " \n",
    "\n",
    "**user_features.csv**\n",
    "**node_id, f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12, f13**\n",
    "This file contains some anonymised features for all nodes/users. Here node_id (corresponding to node1_id and node2_id in train/test files) represents the user for whom we have features from f1 to f13\n",
    "\n",
    "Mostly these features convey information around how active the users are in the app for the given time period - different slices of user engagement metrics. f13 is a categorical feature, f1-f12 are ordinal features each representing no. of days a user did some specific activity on the app in the last 31 days.\n",
    "\n",
    "**test.csv (contained in test.zip)**\n",
    "Build a model that can learn to predict probability of a node-pair in the test set to have a chat relation. The test set contains an id and a pairs of nodes\n",
    "**id, node1_id, node2_id**\n",
    "\n",
    "for which participants are required to predict is_chat on the test set. (Note that id is just here to identify a unique row in test set and is used in the submission format section)\n",
    "\n",
    " \n",
    "## Submission Format\n",
    "Submission file must be in a zipped format containing only one csv corresponding to the submission. The format to be followed is:\n",
    " \n",
    "\n",
    "|id|is_chat|\n",
    "|---|---|\n",
    "|1|0.25|\n",
    "|2|0.26|\n",
    "|3|0.69|\n",
    "|4|0.27|\n",
    "|5|0.22|\n",
    "\n",
    "## Evaluation Metric\n",
    "The submitted output will be evaluated by the **AUC-ROC score**\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load in \n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the \"../input/\" directory.\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n",
    "\n",
    "import os\n",
    "print(os.listdir(\"../input/av-ml-hikeathon-predicting-links\"))\n",
    "\n",
    "# Any results you write to the current directory are saved as output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a",
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "user=pd.read_csv('../input/av-ml-hikeathon/train/user_features.csv')\n",
    "test=pd.read_csv('../input/av-ml-hikeathon/test/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "import gc\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "63"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dftest = pd.merge(pd.merge(test, user.rename(columns={'node_id':'node1_id'}), how='left',on='node1_id'),\n",
    "                    user.rename(columns={'node_id':'node2_id'}), how='left',on='node2_id')\n",
    "\n",
    "del user\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "filename='../input/av-ml-hikeathon-predicting-links/lgbm_model4.sav'\n",
    "loaded_model = pickle.load(open(filename, 'rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p=loaded_model.predict_proba(dftest.drop('id',axis=1)[:dftest.shape[0]//2])[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p2=loaded_model.predict_proba(dftest.drop('id',axis=1)[dftest.shape[0]//2:])[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s=pd.DataFrame({'id':test['id'],'is_chat':np.append(p,p2)})\n",
    "s.to_csv('s7.csv',index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}