re-init

Author: tdpetrou

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+*.ipynb_checkpoints

Introduction to Statistical Learning/Chapter 01 and 02.ipynb

@@ -0,0 +1,101 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Introduction to Statistical Learning \n",
+    "Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani is considered a canonical text in the field of statistical/machine learning and is an absolutely fantastic way to move forward in your analytics career. [The text is free to download](http://www-bcf.usc.edu/~gareth/ISL/) and an [online course by the authors themselves](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about) is currently available in self-pace mode, meaning you can complete it any time. Make sure to **[REGISTER FOR THE STANDFORD COURSE!](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about)** The videos have also been [archived here on youtube](http://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Chapter 10: Unsupervised Learning\n",
+    "Book has been about supervised learning until now. No response variable. Will try and find interesting things in explanatory variables. Chapter will focus on principal components analysis and clustering. No always an exact goal.\n",
+    "\n",
+    "## Principal Components Analysis\n",
+    "Chapter 6 covered principal component regression where the original features were mapped to a smaller feature space that are then used as inputs into linear regression solved normally through least squares.\n",
+    "\n",
+    "PCA can be used to visualize high dimensional data in 2 or 3 dimensions. The first component is a weighted linear combination of all the original features where the sum of the squared weights equals 1. These weights are the loading factors. The loading factors of the first principal component maximize the weighted sum of the features for each observation.\n",
+    "\n",
+    "The second principal component is uncorrelated with the first which makes it orthogonal to it.\n",
+    "\n",
+    "The first PC can also be interpreted as the line closest to the data.\n",
+    "\n",
+    "Very important to scale the data first - 0 mean, 1 std. The variances won't make sense otherwise.\n",
+    "\n",
+    "### Proportion of variance explained\n",
+    "Each principal component explains some of the variance of the original data. We can find the proportion that each principal component explains by dividing each components variance by the total raw variance. Summing all the variances for each component equals 1.\n",
+    "\n",
+    "Examine a scree plot (for an elbow) to choose the number of principal components to use. Or can use cross validation to choose.\n",
+    "\n",
+    "## Clustering\n",
+    "Finding groups within data that are similar.\n",
+    "\n",
+    "Can cluster by using the features or by the observation (transposing data matrix)\n",
+    "\n",
+    "### K-Means\n",
+    "Clustering where you define the number of clusters ahead of time. Algorithm works iteratively by first randomly choosing assigning each point to a cluster and computing cluster centers. All points are then reassigned based on euclidean distance to centroids. A new centroid is found by averaging the points in each cluster. Process stops after centroids stop moving or some max number of iterations.\n",
+    "\n",
+    "Can do initial assignment multiple times and choose clustering assignment with least total variance.\n",
+    "\n",
+    "### Hierarchical Clustering\n",
+    "No need to pre-specify number of clusters. Most common type is bottom-up or Agglomerative. \n",
+    "\n",
+    "#### Interpreting Dendogram\n",
+    "Similarity of points should be determined by the vertical axis not the horizontal axis. The lower on the dendogram that they are connected, the closer they are.\n",
+    "\n",
+    "Hierarchical clustering works by putting each point in its own cluster. Then each pairwise dissimilarity is computed and the least dissimilar clusters are fused. This dissimilarity is the height of the dendogram. Dissimilarity is calculated through a type of linkage and distance metric (usually euclidean).\n",
+    "\n",
+    "## Expectation Maximization\n",
+    "K-means and hieracrchical cluserting are 'hard' clustering meaning that each observation belongs to exactly one cluster. There are other clustering algorithms that 'soft' cluster meaning that observations can belong to multiple clusters.\n",
+    "\n",
+    "One way to perform soft clustering is by assuming that each cluster is modeled by a normal (gaussian) distribution. And so the whole data set is a mixture of gaussian also called Gaussian Mixture Model.\n",
+    "\n",
+    "The goal here is to find the parameters for the multivariate gaussian and assign each observation a probability of being in a certain cluster.\n",
+    "\n",
+    "### EM algorithm\n",
+    "In K-means, we start the algorithm by randomly assigning each point a cluster to find the first centroid. Somewhat similarly, the EM algorithm randomly assigns the parameters of gaussian distribution for each cluster. Then using bayes theorem (with initial priors as all equal), we can determine the probability of each point being a part of each cluster. This is expectation step.\n",
+    "\n",
+    "The maximization step is to recalculate the parameters(mean and covariance) of the multivariate gaussian distribution using the now weighted (by probabilities) observations.\n",
+    "\n",
+    "\n",
+    "\n",
+    "[Excellent video explanation of EM](https://www.youtube.com/watch?v=REypj2sy_5U)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "anaconda-cloud": {},
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}