updaetd all notebooks

Author: tdpetrou

Introduction to Statistical Learning/Chapter 01 and 02.ipynb

@@ -5,54 +5,7 @@
    "metadata": {},
    "source": [
     "# Introduction to Statistical Learning \n",
-    "Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani is considered a canonical text in the field of statistical/machine learning and is an absolutely fantastic way to move forward in your analytics career. [The text is free to download](http://www-bcf.usc.edu/~gareth/ISL/) and an [online course by the authors themselves](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about) is currently available in self-pace mode, meaning you can complete it any time. Make sure to **[REGISTER FOR THE STANDFORD COURSE!](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about)** The videos have also been [archived here on youtube](http://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/).\n",
-    "\n",
-    "# How will Houston Data Science cover the course?\n",
-    "The Stanford online course covers the entire book in 9 weeks and with the R programming language. The pace that we cover the book is yet to be determined as there are many unknown variables such as interest from members, availability of a venue and general level of skills of those participating. That said, a meeting once per week to discuss the current chapter or previous chapter solutions is the target.\n",
-    "\n",
-    "\n",
-    "# Python in place of R\n",
-    "Although R is a fantastic programming language and is the language that all the ISLR labs are written in, the Python programming language, except for rare exceptions, contains analgous libraries that contain the same statistical functionality as those in R.\n",
-    "\n",
-    "# Notes, Exercises and Programming Assignments all in the Jupyter Notebok\n",
-    "ISLR has both end of chapter problems and programming assignments. All chapter problems and programming assignments will be answered in the notebook.\n",
-    "\n",
-    "# Replicating Plots\n",
-    "The plots in ISLR are created in R. Many of them will be replicated here in the notebook when they appear in the text\n",
-    "\n",
-    "# Book Data\n",
-    "The data from the books was downloaded using R. All the datasets are found in either the MASS or ISLR packages. They are now in the data directory. See below"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "metadata": {
-    "collapsed": false
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\u001b[31mAdvertising.csv\u001b[m\u001b[m* carseats.csv     khan_xtrain.csv  portfolio.csv\r\n",
-      "Credit.csv       college.csv      khan_ytest.csv   smarket.csv\r\n",
-      "auto.csv         default.csv      khan_ytrain.csv  usarrests.csv\r\n",
-      "boston.csv       hitters.csv      nci60_data.csv   \u001b[31mwage.csv\u001b[m\u001b[m*\r\n",
-      "caravan.csv      khan_xtest.csv   nci60_labs.csv   weekly.csv\r\n"
-     ]
-    }
-   ],
-   "source": [
-    "ls data"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# ISLR Videos\n",
-    "[All Old Videos](https://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/)"
+    "Introduction to Statistical Learning by Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani is considered a canonical text in the field of statistical/machine learning and is an absolutely fantastic way to move forward in your analytics career. [The text is free to download](http://www-bcf.usc.edu/~gareth/ISL/) and an [online course by the authors themselves](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about) is currently available in self-pace mode, meaning you can complete it any time. Make sure to **[REGISTER FOR THE STANDFORD COURSE!](https://lagunita.stanford.edu/courses/HumanitiesSciences/StatLearning/Winter2016/about)** The videos have also been [archived here on youtube](http://www.r-bloggers.com/in-depth-introduction-to-machine-learning-in-15-hours-of-expert-videos/)."
    ]
   },
   {
@@ -69,7 +22,48 @@
     "\n",
     "The second principal component is uncorrelated with the first which makes it orthogonal to it.\n",
     "\n",
-    "The first pc can also be interpretted as the line closest to the data."
+    "The first PC can also be interpreted as the line closest to the data.\n",
+    "\n",
+    "Very important to scale the data first - 0 mean, 1 std. The variances won't make sense otherwise.\n",
+    "\n",
+    "### Proportion of variance explained\n",
+    "Each principal component explains some of the variance of the original data. We can find the proportion that each principal component explains by dividing each components variance by the total raw variance. Summing all the variances for each component equals 1.\n",
+    "\n",
+    "Examine a scree plot (for an elbow) to choose the number of principal components to use. Or can use cross validation to choose.\n",
+    "\n",
+    "## Clustering\n",
+    "Finding groups within data that are similar.\n",
+    "\n",
+    "Can cluster by using the features or by the observation (transposing data matrix)\n",
+    "\n",
+    "### K-Means\n",
+    "Clustering where you define the number of clusters ahead of time. Algorithm works iteratively by first randomly choosing assigning each point to a cluster and computing cluster centers. All points are then reassigned based on euclidean distance to centroids. A new centroid is found by averaging the points in each cluster. Process stops after centroids stop moving or some max number of iterations.\n",
+    "\n",
+    "Can do initial assignment multiple times and choose clustering assignment with least total variance.\n",
+    "\n",
+    "### Hierarchical Clustering\n",
+    "No need to pre-specify number of clusters. Most common type is bottom-up or Agglomerative. \n",
+    "\n",
+    "#### Interpreting Dendogram\n",
+    "Similarity of points should be determined by the vertical axis not the horizontal axis. The lower on the dendogram that they are connected, the closer they are.\n",
+    "\n",
+    "Hierarchical clustering works by putting each point in its own cluster. Then each pairwise dissimilarity is computed and the least dissimilar clusters are fused. This dissimilarity is the height of the dendogram. Dissimilarity is calculated through a type of linkage and distance metric (usually euclidean).\n",
+    "\n",
+    "## Expectation Maximization\n",
+    "K-means and hieracrchical cluserting are 'hard' clustering meaning that each observation belongs to exactly one cluster. There are other clustering algorithms that 'soft' cluster meaning that observations can belong to multiple clusters.\n",
+    "\n",
+    "One way to perform soft clustering is by assuming that each cluster is modeled by a normal (gaussian) distribution. And so the whole data set is a mixture of gaussian also called Gaussian Mixture Model.\n",
+    "\n",
+    "The goal here is to find the parameters for the multivariate gaussian and assign each observation a probability of being in a certain cluster.\n",
+    "\n",
+    "### EM algorithm\n",
+    "In K-means, we start the algorithm by randomly assigning each point a cluster to find the first centroid. Somewhat similarly, the EM algorithm randomly assigns the parameters of gaussian distribution for each cluster. Then using bayes theorem (with initial priors as all equal), we can determine the probability of each point being a part of each cluster. This is expectation step.\n",
+    "\n",
+    "The maximization step is to recalculate the parameters(mean and covariance) of the multivariate gaussian distribution using the now weighted (by probabilities) observations.\n",
+    "\n",
+    "\n",
+    "\n",
+    "[Excellent video explanation of EM](https://www.youtube.com/watch?v=REypj2sy_5U)"
    ]
   },
   {
@@ -85,9 +79,9 @@
  "metadata": {
   "anaconda-cloud": {},
   "kernelspec": {
-   "display_name": "Python [Root]",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "Python [Root]"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -99,9 +93,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.5.2"
+   "version": "3.6.1"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 0
+ "nbformat_minor": 1
 }