diff --git a/README.md b/README.md index ab92eb7..07c9532 100644 --- a/README.md +++ b/README.md @@ -1,653 +1,1222 @@ -# Machine Learning & Deep Learning Tutorials [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) - -- This repository contains a topic-wise curated list of Machine Learning and Deep Learning tutorials, articles and other resources. Other awesome lists can be found in this [list](https://github.com/sindresorhus/awesome). - -- If you want to contribute to this list, please read [Contributing Guidelines](https://github.com/ujjwalkarn/Machine-Learning-Tutorials/blob/master/contributing.md). - -- [Curated list of R tutorials for Data Science, NLP and Machine Learning](https://github.com/ujjwalkarn/DataScienceR). - -- [Curated list of Python tutorials for Data Science, NLP and Machine Learning](https://github.com/ujjwalkarn/DataSciencePython). - - -##Table of Contents -- [Miscellaneous](#general) -- [Interview Resources](#interview) -- [Artificial Intelligence](#ai) -- [Genetic Algorithms](#ga) -- [Statistics](#stat) -- [Useful Blogs](#blogs) -- [Resources on Quora](#quora) -- [Resources on Kaggle](#kaggle) -- [Cheat Sheets](#cs) -- [Classification](#classification) -- [Linear Regression](#linear) -- [Logistic Regression](#logistic) -- [Model Validation using Resampling](#validation) - - [Cross Validation](#cross) - - [Bootstraping](#boot) -- [Deep Learning](#deep) - - [Frameworks](#frame) - - [Feed Forward Networks](#feed) - - [Recurrent Neural Nets, LSTM, GRU](#rnn) - - [Restricted Boltzmann Machine, DBNs](#rbm) - - [Autoencoders](#auto) - - [Convolutional Neural Nets](#cnn) -- [Natural Language Processing](#nlp) - - [Topic Modeling, LDA](#topic) - - [Word2Vec](#word2vec) -- [Computer Vision](#vision) -- [Support Vector Machine](#svm) -- [Reinforcement Learning](#rl) -- [Decision Trees](#dt) -- [Random Forest / Bagging](#rf) -- [Boosting](#gbm) -- [Ensembles](#ensem) -- [Stacking Models](#stack) -- [VC Dimension](#vc) -- [Bayesian Machine Learning](#bayes) -- [Semi Supervised Learning](#semi) -- [Optimizations](#opt) -- [Other Useful Tutorials](#other) - - -##Miscellaneous -- [Machine Learning for Software Engineers](https://github.com/ZuzooVn/machine-learning-for-software-engineers) -- [Dive into Machine Learning](https://github.com/hangtwenty/dive-into-machine-learning) -- [A curated list of awesome Machine Learning frameworks, libraries and software](https://github.com/josephmisiti/awesome-machine-learning) -- [A curated list of awesome data visualization libraries and resources.](https://github.com/fasouto/awesome-dataviz) -- [An awesome Data Science repository to learn and apply for real world problems](https://github.com/okulbilisim/awesome-datascience) -- [The Open Source Data Science Masters](http://datasciencemasters.org/) -- [Machine Learning FAQs on Cross Validated](http://stats.stackexchange.com/questions/tagged/machine-learning) -- [List of Machine Learning University Courses](https://github.com/prakhar1989/awesome-courses#machine-learning) -- [Machine Learning algorithms that you should always have a strong understanding of](https://www.quora.com/What-are-some-Machine-Learning-algorithms-that-you-should-always-have-a-strong-understanding-of-and-why) -- [Difference between Linearly Independent, Orthogonal, and Uncorrelated Variables](http://terpconnect.umd.edu/~bmomen/BIOM621/LineardepCorrOrthogonal.pdf) -- [List of Machine Learning Concepts](https://en.wikipedia.org/wiki/List_of_machine_learning_concepts) -- [Slides on Several Machine Learning Topics](http://www.slideshare.net/pierluca.lanzi/presentations) -- [MIT Machine Learning Lecture Slides](http://www.ai.mit.edu/courses/6.867-f04/lectures.html) -- [Comparison Supervised Learning Algorithms](http://www.dataschool.io/comparing-supervised-learning-algorithms/) -- [Learning Data Science Fundamentals](http://www.dataschool.io/learning-data-science-fundamentals/) -- [Machine Learning mistakes to avoid](https://medium.com/@nomadic_mind/new-to-machine-learning-avoid-these-three-mistakes-73258b3848a4#.lih061l3l) -- [Statistical Machine Learning Course](http://www.stat.cmu.edu/~larry/=sml/) -- [TheAnalyticsEdge edX Notes and Codes](https://github.com/pedrosan/TheAnalyticsEdge) -- [In-depth introduction to machine learning in 15 hours of expert videos](http://www.dataschool.io/15-hours-of-expert-machine-learning-videos/) - - -##Interview Resources -- [41 Essential Machine Learning Interview Questions (with answers)](https://www.springboard.com/blog/machine-learning-interview-questions/) -- [How can a computer science graduate student prepare himself for data scientist interviews?](https://www.quora.com/How-can-a-computer-science-graduate-student-prepare-himself-for-data-scientist-machine-learning-intern-interviews) -- [How do I learn Machine Learning?](https://www.quora.com/How-do-I-learn-machine-learning-1) -- [FAQs about Data Science Interviews](https://www.quora.com/topic/Data-Science-Interviews/faq) -- [What are the key skills of a data scientist?](https://www.quora.com/What-are-the-key-skills-of-a-data-scientist) - - -##Artificial Intelligence -- [Awesome Artificial Intelligence (GitHub Repo)](https://github.com/owainlewis/awesome-artificial-intelligence) -- [UC Berkeley CS188 Intro to AI](http://ai.berkeley.edu/home.html), [Lecture Videos](http://ai.berkeley.edu/lecture_videos.html), [2](https://www.youtube.com/watch?v=W1S-HSakPTM) -- [MIT 6.034 Artificial Intelligence Lecture Videos](https://www.youtube.com/playlist?list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi), [Complete Course](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/) -- [edX course | Klein & Abbeel](https://courses.edx.org/courses/BerkeleyX/CS188x_1/1T2013/info) -- [Udacity Course | Norvig & Thrun](https://www.udacity.com/course/intro-to-artificial-intelligence--cs271) -- [TED talks on AI](http://www.ted.com/playlists/310/talks_on_artificial_intelligen) - - -##Genetic Algorithms -- [Genetic Algorithms Wikipedia Page](https://en.wikipedia.org/wiki/Genetic_algorithm) -- [Simple Implementation of Genetic Algorithms in Python (Part 1)](http://outlace.com/Simple-Genetic-Algorithm-in-15-lines-of-Python/), [Part 2](http://outlace.com/Simple-Genetic-Algorithm-Python-Addendum/) -- [Genetic Algorithms vs Artificial Neural Networks](http://stackoverflow.com/questions/1402370/when-to-use-genetic-algorithms-vs-when-to-use-neural-networks) -- [Genetic Algorithms Explained in Plain English](http://www.ai-junkie.com/ga/intro/gat1.html) -- [Genetic Programming](https://en.wikipedia.org/wiki/Genetic_programming) - - [Genetic Programming in Python (GitHub)](https://github.com/trevorstephens/gplearn) - - [Genetic Alogorithms vs Genetic Programming (Quora)](https://www.quora.com/Whats-the-difference-between-Genetic-Algorithms-and-Genetic-Programming), [StackOverflow](http://stackoverflow.com/questions/3819977/what-are-the-differences-between-genetic-algorithms-and-genetic-programming) - - -##Statistics -- [Stat Trek Website](http://stattrek.com/) - A dedicated website to teach yourselves Statistics -- [Learn Statistics Using Python](https://github.com/rouseguy/intro2stats) - Learn Statistics using an application-centric programming approach -- [Statistics for Hackers | Slides | @jakevdp](https://speakerdeck.com/jakevdp/statistics-for-hackers) - Slides by Jake VanderPlas -- [Online Statistics Book](http://onlinestatbook.com/2/index.html) - An Interactive Multimedia Course for Studying Statistics -- [What is a Sampling Distribution?](http://stattrek.com/sampling/sampling-distribution.aspx) -- Tutorials - - [AP Statistics Tutorial](http://stattrek.com/tutorials/ap-statistics-tutorial.aspx) - - [Statistics and Probability Tutorial](http://stattrek.com/tutorials/statistics-tutorial.aspx) - - [Matrix Algebra Tutorial](http://stattrek.com/tutorials/matrix-algebra-tutorial.aspx) -- [What is an Unbiased Estimator?](https://www.physicsforums.com/threads/what-is-an-unbiased-estimator.547728/) -- [Goodness of Fit Explained](https://en.wikipedia.org/wiki/Goodness_of_fit) -- [What are QQ Plots?](http://onlinestatbook.com/2/advanced_graphs/q-q_plots.html) -- [OpenIntro Statistics](https://www.openintro.org/stat/textbook.php?stat_book=os) - Free PDF textbook - - -##Useful Blogs -- [Edwin Chen's Blog](http://blog.echen.me/) - A blog about Math, stats, ML, crowdsourcing, data science -- [The Data School Blog](http://www.dataschool.io/) - Data science for beginners! -- [ML Wave](http://mlwave.com/) - A blog for Learning Machine Learning -- [Andrej Karpathy](http://karpathy.github.io/) - A blog about Deep Learning and Data Science in general -- [Colah's Blog](http://colah.github.io/) - Awesome Neural Networks Blog -- [Alex Minnaar's Blog](http://alexminnaar.com/) - A blog about Machine Learning and Software Engineering -- [Statistically Significant](http://andland.github.io/) - Andrew Landgraf's Data Science Blog -- [Simply Statistics](http://simplystatistics.org/) - A blog by three biostatistics professors -- [Yanir Seroussi's Blog](https://yanirseroussi.com/) - A blog about Data Science and beyond -- [fastML](http://fastml.com/) - Machine learning made easy -- [Trevor Stephens Blog](http://trevorstephens.com/) - Trevor Stephens Personal Page -- [no free hunch | kaggle](http://blog.kaggle.com/) - The Kaggle Blog about all things Data Science -- [A Quantitative Journey | outlace](http://outlace.com/) - learning quantitative applications -- [r4stats](http://r4stats.com/) - analyze the world of data science, and to help people learn to use R -- [Variance Explained](http://varianceexplained.org/) - David Robinson's Blog -- [AI Junkie](http://www.ai-junkie.com/) - a blog about Artificial Intellingence -- [Deep Learning Blog by Tim Dettmers](http://timdettmers.com/)- Making deep learning accessible -- [J Alammar's Blog](http://jalammar.github.io/)- Blog posts about Machine Learning and Neural Nets -- [Adam Geitgey](https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471#.f7vwrtfne) - Easiest Introduction to machine learning - - -##Resources on Quora -- [Most Viewed Machine Learning writers](https://www.quora.com/topic/Machine-Learning/writers) -- [Data Science Topic on Quora](https://www.quora.com/Data-Science) -- [William Chen's Answers](https://www.quora.com/William-Chen-6/answers) -- [Michael Hochster's Answers](https://www.quora.com/Michael-Hochster/answers) -- [Ricardo Vladimiro's Answers](https://www.quora.com/Ricardo-Vladimiro-1/answers) -- [Storytelling with Statistics](https://datastories.quora.com/) -- [Data Science FAQs on Quora](https://www.quora.com/topic/Data-Science/faq) -- [Machine Learning FAQs on Quora](https://www.quora.com/topic/Machine-Learning/faq) - - -##Kaggle Competitions WriteUp -- [How to almost win Kaggle Competitions](https://yanirseroussi.com/2014/08/24/how-to-almost-win-kaggle-competitions/) -- [Convolution Neural Networks for EEG detection](http://blog.kaggle.com/2015/10/05/grasp-and-lift-eeg-detection-winners-interview-3rd-place-team-hedj/) -- [Facebook Recruiting III Explained](http://alexminnaar.com/tag/kaggle-competitions.html) -- [Predicting CTR with Online ML](http://mlwave.com/predicting-click-through-rates-with-online-machine-learning/) -- [How to Rank 10% in Your First Kaggle Competition](https://dnc1994.com/2016/05/rank-10-percent-in-first-kaggle-competition-en/) - - -##Cheat Sheets -- [Probability Cheat Sheet](http://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf), [Source](http://www.wzchen.com/probability-cheatsheet/) -- [Machine Learning Cheat Sheet](https://github.com/soulmachine/machine-learning-cheat-sheet) - - -##Classification -- [Does Balancing Classes Improve Classifier Performance?](http://www.win-vector.com/blog/2015/02/does-balancing-classes-improve-classifier-performance/) -- [What is Deviance?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart) -- [When to choose which machine learning classifier?](http://stackoverflow.com/questions/2595176/when-to-choose-which-machine-learning-classifier) -- [What are the advantages of different classification algorithms?](https://www.quora.com/What-are-the-advantages-of-different-classification-algorithms) -- [ROC and AUC Explained](http://www.dataschool.io/roc-curves-and-auc-explained/) ([related video](https://youtu.be/OAl6eAyP-yo)) -- [An introduction to ROC analysis](https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf) -- [Simple guide to confusion matrix terminology](http://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/) - - - -##Linear Regression -- [General](#general-) - - [Assumptions of Linear Regression](http://pareonline.net/getvn.asp?n=2&v=8), [Stack Exchange](http://stats.stackexchange.com/questions/16381/what-is-a-complete-list-of-the-usual-assumptions-for-linear-regression) - - [Linear Regression Comprehensive Resource](http://people.duke.edu/~rnau/regintro.htm) - - [Applying and Interpreting Linear Regression](http://www.dataschool.io/applying-and-interpreting-linear-regression/) - - [What does having constant variance in a linear regression model mean?](http://stats.stackexchange.com/questions/52089/what-does-having-constant-variance-in-a-linear-regression-model-mean/52107?stw=2#52107) - - [Difference between linear regression on y with x and x with y](http://stats.stackexchange.com/questions/22718/what-is-the-difference-between-linear-regression-on-y-with-x-and-x-with-y?lq=1) - - [Is linear regression valid when the dependant variable is not normally distributed?](https://www.researchgate.net/post/Is_linear_regression_valid_when_the_outcome_dependant_variable_not_normally_distributed) -- Multicollinearity and VIF - - [Dummy Variable Trap | Multicollinearity](https://en.wikipedia.org/wiki/Multicollinearity) - - [Dealing with multicollinearity using VIFs](https://jonlefcheck.net/2012/12/28/dealing-with-multicollinearity-using-variance-inflation-factors/) - -- [Residual Analysis](#residuals-) - - [Interpreting plot.lm() in R](http://stats.stackexchange.com/questions/58141/interpreting-plot-lm) - - [How to interpret a QQ plot?](http://stats.stackexchange.com/questions/101274/how-to-interpret-a-qq-plot?lq=1) - - [Interpreting Residuals vs Fitted Plot](http://stats.stackexchange.com/questions/76226/interpreting-the-residuals-vs-fitted-values-plot-for-verifying-the-assumptions) - -- [Outliers](#outliers-) - - [How should outliers be dealt with?](http://stats.stackexchange.com/questions/175/how-should-outliers-be-dealt-with-in-linear-regression-analysis) - -- [Elastic Net](https://en.wikipedia.org/wiki/Elastic_net_regularization) - - [Regularization and Variable Selection via the -Elastic Net](https://web.stanford.edu/~hastie/Papers/elasticnet.pdf) - - -##Logistic Regression -- [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression) -- [Geometric Intuition of Logistic Regression](http://florianhartl.com/logistic-regression-geometric-intuition.html) -- [Obtaining predicted categories (choosing threshold)](http://stats.stackexchange.com/questions/25389/obtaining-predicted-values-y-1-or-0-from-a-logistic-regression-model-fit) -- [Residuals in logistic regression](http://stats.stackexchange.com/questions/1432/what-do-the-residuals-in-a-logistic-regression-mean) -- [Difference between logit and probit models](http://stats.stackexchange.com/questions/20523/difference-between-logit-and-probit-models#30909), [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression), [Probit Model Wiki](https://en.wikipedia.org/wiki/Probit_model) -- [Pseudo R2 for Logistic Regression](http://stats.stackexchange.com/questions/3559/which-pseudo-r2-measure-is-the-one-to-report-for-logistic-regression-cox-s), [How to calculate](http://stats.stackexchange.com/questions/8511/how-to-calculate-pseudo-r2-from-rs-logistic-regression), [Other Details](http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm) -- [Guide to an in-depth understanding of logistic regression](http://www.dataschool.io/guide-to-logistic-regression/) - - -##Model Validation using Resampling - -- [Resampling Explained](https://en.wikipedia.org/wiki/Resampling_(statistics)) -- [Partioning data set in R](http://stackoverflow.com/questions/13536537/partitioning-data-set-in-r-based-on-multiple-classes-of-observations) -- [Implementing hold-out Validaion in R](http://stackoverflow.com/questions/22972854/how-to-implement-a-hold-out-validation-in-r), [2](http://www.gettinggeneticsdone.com/2011/02/split-data-frame-into-testing-and.html) - - -- [Cross Validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)) - - [Training with Full dataset after CV?](http://stats.stackexchange.com/questions/11602/training-with-the-full-dataset-after-cross-validation) - - [Which CV method is best?](http://stats.stackexchange.com/questions/103459/how-do-i-know-which-method-of-cross-validation-is-best) - - [Variance Estimates in k-fold CV](http://stats.stackexchange.com/questions/31190/variance-estimates-in-k-fold-cross-validation) - - [Is CV a subsitute for Validation Set?](http://stats.stackexchange.com/questions/18856/is-cross-validation-a-proper-substitute-for-validation-set) - - [Choice of k in k-fold CV](http://stats.stackexchange.com/questions/27730/choice-of-k-in-k-fold-cross-validation) - - [CV for ensemble learning](http://stats.stackexchange.com/questions/102631/k-fold-cross-validation-of-ensemble-learning) - - [k-fold CV in R](http://stackoverflow.com/questions/22909197/creating-folds-for-k-fold-cv-in-r-using-caret) - - [Good Resources](http://www.chioka.in/tag/cross-validation/) - - Overfitting and Cross Validation - - [Preventing Overfitting the Cross Validation Data | Andrew Ng](http://ai.stanford.edu/~ang/papers/cv-final.pdf) - - [Over-fitting in Model Selection and Subsequent Selection Bias in -Performance Evaluation](http://www.jmlr.org/papers/volume11/cawley10a/cawley10a.pdf) - - [CV for detecting and preventing Overfitting](http://www.autonlab.org/tutorials/overfit10.pdf) - - [How does CV overcome the Overfitting Problem](http://stats.stackexchange.com/questions/9053/how-does-cross-validation-overcome-the-overfitting-problem) - - - - -- [Bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(statistics)) - - [Why Bootstrapping Works?](http://stats.stackexchange.com/questions/26088/explaining-to-laypeople-why-bootstrapping-works) - - [Good Animation](https://www.stat.auckland.ac.nz/~wild/BootAnim/) - - [Example of Bootstapping](http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm) - - [Understanding Bootstapping for Validation and Model Selection](http://stats.stackexchange.com/questions/14516/understanding-bootstrapping-for-validation-and-model-selection?rq=1) - - [Cross Validation vs Bootstrap to estimate prediction error](http://stats.stackexchange.com/questions/18348/differences-between-cross-validation-and-bootstrapping-to-estimate-the-predictio), [Cross-validation vs .632 bootstrapping to evaluate classification performance](http://stats.stackexchange.com/questions/71184/cross-validation-or-bootstrapping-to-evaluate-classification-performance) - - - -##Deep Learning -- [A curated list of awesome Deep Learning tutorials, projects and communities](https://github.com/ChristosChristofidis/awesome-deep-learning) -- [Lots of Deep Learning Resources](http://deeplearning4j.org/documentation.html) -- [Interesting Deep Learning and NLP Projects (Stanford)](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/) -- [Core Concepts of Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/) -- [Understanding Natural Language with Deep Neural Networks Using Torch](https://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) -- [Stanford Deep Learning Tutorial](http://ufldl.stanford.edu/tutorial/) -- [Deep Learning FAQs on Quora](https://www.quora.com/topic/Deep-Learning/faq) -- [Google+ Deep Learning Page](https://plus.google.com/communities/112866381580457264725) -- [Recent Reddit AMAs related to Deep Learning](http://deeplearning.net/2014/11/22/recent-reddit-amas-about-deep-learning/), [Another AMA](https://www.reddit.com/r/IAmA/comments/3mdk9v/we_are_google_researchers_working_on_deep/) -- [Where to Learn Deep Learning?](http://www.kdnuggets.com/2014/05/learn-deep-learning-courses-tutorials-overviews.html) -- [Deep Learning nvidia concepts](http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/) -- [Introduction to Deep Learning Using Python (GitHub)](https://github.com/rouseguy/intro2deeplearning), [Good Introduction Slides](https://speakerdeck.com/bargava/introduction-to-deep-learning) -- [Video Lectures Oxford 2015](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu), [Video Lectures Summer School Montreal](http://videolectures.net/deeplearning2015_montreal/) -- [Deep Learning Software List](http://deeplearning.net/software_links/) -- [Hacker's guide to Neural Nets](http://karpathy.github.io/neuralnets/) -- [Top arxiv Deep Learning Papers explained](http://www.kdnuggets.com/2015/10/top-arxiv-deep-learning-papers-explained.html) -- [Geoff Hinton Youtube Vidoes on Deep Learning](https://www.youtube.com/watch?v=IcOMKXAw5VA) -- [Awesome Deep Learning Reading List](http://deeplearning.net/reading-list/) -- [Deep Learning Comprehensive Website](http://deeplearning.net/), [Software](http://deeplearning.net/software_links/) -- [deeplearning Tutorials](http://deeplearning4j.org/) -- [AWESOME! Deep Learning Tutorial](https://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks) -- [Deep Learning Basics](http://alexminnaar.com/deep-learning-basics-neural-networks-backpropagation-and-stochastic-gradient-descent.html) -- [Stanford Tutorials](http://ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/) -- [Train, Validation & Test in Artificial Neural Networks](http://stackoverflow.com/questions/2976452/whats-is-the-difference-between-train-validation-and-test-set-in-neural-networ) -- [Artificial Neural Networks Tutorials](http://stackoverflow.com/questions/478947/what-are-some-good-resources-for-learning-about-artificial-neural-networks) -- [Neural Networks FAQs on Stack Overflow](http://stackoverflow.com/questions/tagged/neural-network?sort=votes&pageSize=50) -- [Deep Learning Tutorials on deeplearning.net](http://deeplearning.net/tutorial/index.html) -- [Neural Networks and Deep Learning Online Book](http://neuralnetworksanddeeplearning.com/) - -- Neural Machine Translation - - [Introduction to Neural Machine Translation with GPUs (part 1)](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-with-gpus/), [Part 2](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-2/), [Part 3](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/) - - [Deep Speech: Accurate Speech Recognition with GPU-Accelerated Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-speech-accurate-speech-recognition-gpu-accelerated-deep-learning/) - - -- Deep Learning Frameworks - - [Torch vs. Theano](http://fastml.com/torch-vs-theano/) - - [dl4j vs. torch7 vs. theano](http://deeplearning4j.org/compare-dl4j-torch7-pylearn.html) - - [Deep Learning Libraries by Language](http://www.teglor.com/b/deep-learning-libraries-language-cm569/) - - - [Theano](https://en.wikipedia.org/wiki/Theano_(software)) - - [Website](http://deeplearning.net/software/theano/) - - [Theano Introduction](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/) - - [Theano Tutorial](http://outlace.com/Beginner-Tutorial-Theano/) - - [Good Theano Tutorial](http://deeplearning.net/software/theano/tutorial/) - - [Logistic Regression using Theano for classifying digits](http://deeplearning.net/tutorial/logreg.html#logreg) - - [MLP using Theano](http://deeplearning.net/tutorial/mlp.html#mlp) - - [CNN using Theano](http://deeplearning.net/tutorial/lenet.html#lenet) - - [RNNs using Theano](http://deeplearning.net/tutorial/rnnslu.html#rnnslu) - - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm) - - [RBM using Theano](http://deeplearning.net/tutorial/rbm.html#rbm) - - [DBNs using Theano](http://deeplearning.net/tutorial/DBN.html#dbn) - - [All Codes](https://github.com/lisa-lab/DeepLearningTutorials) - - [Deep Learning Implementation Tutorials - Keras and Lasagne](https://github.com/vict0rsch/deep_learning/) - - - [Torch](http://torch.ch/) - - [Torch ML Tutorial](http://code.madbits.com/wiki/doku.php), [Code](https://github.com/torch/tutorials) - - [Intro to Torch](http://ml.informatik.uni-freiburg.de/_media/teaching/ws1415/presentation_dl_lect3.pdf) - - [Learning Torch GitHub Repo](https://github.com/chetannaik/learning_torch) - - [Awesome-Torch (Repository on GitHub)](https://github.com/carpedm20/awesome-torch) - - [Machine Learning using Torch Oxford Univ](https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/), [Code](https://github.com/oxford-cs-ml-2015) - - [Torch Internals Overview](https://apaszke.github.io/torch-internals.html) - - [Torch Cheatsheet](https://github.com/torch/torch7/wiki/Cheatsheet) - - [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) - - - Caffe - - [Deep Learning for Computer Vision with Caffe and cuDNN](https://devblogs.nvidia.com/parallelforall/deep-learning-computer-vision-caffe-cudnn/) - - - TensorFlow - - [Website](http://tensorflow.org/) - - [TensorFlow Examples for Beginners](https://github.com/aymericdamien/TensorFlow-Examples) - - [Stanford Tensorflow for Deep Learning Research Course](https://web.stanford.edu/class/cs20si/syllabus.html) - - [GitHub Repo](https://github.com/chiphuyen/tf-stanford-tutorials) - - [Simplified Scikit-learn Style Interface to TensorFlow](https://github.com/tensorflow/skflow) - - [Learning TensorFlow GitHub Repo](https://github.com/chetannaik/learning_tensorflow) - - [Benchmark TensorFlow GitHub](https://github.com/soumith/convnet-benchmarks/issues/66) - - [Awesome TensorFlow List](https://github.com/jtoy/awesome-tensorflow) - - [TensorFlow Book](https://github.com/BinRoot/TensorFlow-Book) - - - -- Feed Forward Networks - - [A Quick Introduction to Neural Networks](https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/) - - [Implementing a Neural Network from scratch](http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/), [Code](https://github.com/dennybritz/nn-from-scratch) - - [Speeding up your Neural Network with Theano and the gpu](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/), [Code](https://github.com/dennybritz/nn-theano) - - [Basic ANN Theory](https://takinginitiative.wordpress.com/2008/04/03/basic-neural-network-tutorial-theory/) - - [Role of Bias in Neural Networks](http://stackoverflow.com/questions/2480650/role-of-bias-in-neural-networks) - - [Choosing number of hidden layers and nodes](http://stackoverflow.com/questions/3345079/estimating-the-number-of-neurons-and-number-of-layers-of-an-artificial-neural-ne),[2](http://stackoverflow.com/questions/10565868/multi-layer-perceptron-mlp-architecture-criteria-for-choosing-number-of-hidde?lq=1),[3](http://stackoverflow.com/questions/9436209/how-to-choose-number-of-hidden-layers-and-nodes-in-neural-network/2#) - - [Backpropagation in Matrix Form](http://sudeepraja.github.io/Neural/) - - [ANN implemented in C++ | AI Junkie](http://www.ai-junkie.com/ann/evolved/nnt6.html) - - [Simple Implementation](http://stackoverflow.com/questions/15395835/simple-multi-layer-neural-network-implementation) - - [NN for Beginners](http://www.codeproject.com/Articles/16419/AI-Neural-Network-for-beginners-Part-of) - - [Regression and Classification with NNs (Slides)](http://www.autonlab.org/tutorials/neural13.pdf) - - [Another Intro](http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html) - - -- Recurrent and LSTM Networks - - [awesome-rnn: list of resources (GitHub Repo)](https://github.com/kjw0612/awesome-rnn) - - [Recurrent Neural Net Tutorial Part 1](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/), [Part 2] (http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-2-implementing-a-language-model-rnn-with-python-numpy-and-theano/), [Part 3] (http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/), [Code](https://github.com/dennybritz/rnn-tutorial-rnnlm/) - - [NLP RNN Representations](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/) - - [The Unreasonable effectiveness of RNNs](http://karpathy.github.io/2015/05/21/rnn-effectiveness/), [Torch Code](https://github.com/karpathy/char-rnn), [Python Code](https://gist.github.com/karpathy/d4dee566867f8291f086) - - [Intro to RNN](http://deeplearning4j.org/recurrentnetwork.html), [LSTM](http://deeplearning4j.org/lstm.html) - - [An application of RNN](http://hackaday.com/2015/10/15/73-computer-scientists-created-a-neural-net-and-you-wont-believe-what-happened-next/) - - [Optimizing RNN Performance](http://svail.github.io/) - - [Simple RNN](http://outlace.com/Simple-Recurrent-Neural-Network/) - - [Auto-Generating Clickbait with RNN](https://larseidnes.com/2015/10/13/auto-generating-clickbait-with-recurrent-neural-networks/) - - [Sequence Learning using RNN (Slides)](http://www.slideshare.net/indicods/general-sequence-learning-with-recurrent-neural-networks-for-next-ml) - - [Machine Translation using RNN (Paper)](http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf) - - [Music generation using RNNs (Keras)](https://github.com/MattVitelli/GRUV) - - [Using RNN to create on-the-fly dialogue (Keras)](http://neuralniche.com/post/tutorial/) - - Long Short Term Memory (LSTM) - - [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) - - [LSTM explained](https://apaszke.github.io/lstm-explained.html) - - [Beginner’s Guide to LSTM](http://deeplearning4j.org/lstm.html) - - [Implementing LSTM from scratch](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/), [Python/Theano code](https://github.com/dennybritz/rnn-tutorial-gru-lstm) - - [Torch Code for character-level language models using LSTM](https://github.com/karpathy/char-rnn) - - [LSTM for Kaggle EEG Detection competition (Torch Code)](https://github.com/apaszke/kaggle-grasp-and-lift) - - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm) - - [Deep Learning for Visual Q&A | LSTM | CNN](http://avisingh599.github.io/deeplearning/visual-qa/), [Code](https://github.com/avisingh599/visual-qa) - - [Computer Responds to email using LSTM | Google](http://googleresearch.blogspot.in/2015/11/computer-respond-to-this-email.html) - - [LSTM dramatically improves Google Voice Search](http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html), [Another Article](http://deeplearning.net/2015/09/30/long-short-term-memory-dramatically-improves-google-voice-etc-now-available-to-a-billion-users/) - - [Understanding Natural Language with LSTM Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) - - [Torch code for Visual Question Answering using a CNN+LSTM model](https://github.com/abhshkdz/neural-vqa) - - Gated Recurrent Units (GRU) - - [LSTM vs GRU](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/) - - - -- [Recursive Neural Network (not Recurrent)](https://en.wikipedia.org/wiki/Recursive_neural_network) - - [Recursive Neural Tensor Network (RNTN)](http://deeplearning4j.org/recursiveneuraltensornetwork.html) - - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) - - -- Restricted Boltzmann Machine - - [Beginner's Guide about RBMs](http://deeplearning4j.org/restrictedboltzmannmachine.html) - - [Another Good Tutorial](http://deeplearning.net/tutorial/rbm.html) - - [Introduction to RBMs](http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/) - - [Hinton's Guide to Training RBMs](https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf) - - [RBMs in R](https://github.com/zachmayer/rbm) - - [Deep Belief Networks Tutorial](http://deeplearning4j.org/deepbeliefnetwork.html) - - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) - - -- Autoencoders: Unsupervised (applies BackProp after setting target = input) - - [Andrew Ng Sparse Autoencoders pdf](https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf) - - [Deep Autoencoders Tutorial](http://deeplearning4j.org/deepautoencoder.html) - - [Denoising Autoencoders](http://deeplearning.net/tutorial/dA.html), [Theano Code](http://deeplearning.net/tutorial/code/dA.py) - - [Stacked Denoising Autoencoders](http://deeplearning.net/tutorial/SdA.html#sda) - - - -- Convolutional Neural Networks - - [An Intuitive Explanation of Convolutional Neural Networks](https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/) - - [Awesome Deep Vision: List of Resources (GitHub)](https://github.com/kjw0612/awesome-deep-vision) - - [Intro to CNNs](http://deeplearning4j.org/convolutionalnets.html) - - [Understanding CNN for NLP](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/) - - [Stanford Notes](http://vision.stanford.edu/teaching/cs231n/), [Codes](http://cs231n.github.io/), [GitHub](https://github.com/cs231n/cs231n.github.io) - - [JavaScript Library (Browser Based) for CNNs](http://cs.stanford.edu/people/karpathy/convnetjs/) - - [Using CNNs to detect facial keypoints](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/) - - [Deep learning to classify business photos at Yelp](http://engineeringblog.yelp.com/2015/10/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html) - - [Interview with Yann LeCun | Kaggle](http://blog.kaggle.com/2014/12/22/convolutional-nets-and-cifar-10-an-interview-with-yan-lecun/) - - [Visualising and Understanding CNNs](https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf) - - - -##Natural Language Processing -- [A curated list of speech and natural language processing resources](https://github.com/edobashira/speech-language-processing) -- [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) -- [tf-idf explained](http://michaelerasm.us/tf-idf-in-10-minutes/) -- [Interesting Deep Learning NLP Projects Stanford](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/) -- [NLP from Scratch | Google Paper](https://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/35671.pdf) -- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf) -- [Bag of Words](https://en.wikipedia.org/wiki/Bag-of-words_model) - - [Classification text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/) - -- [Topic Modeling](https://en.wikipedia.org/wiki/Topic_model) - - [LDA](https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation), [LSA](https://en.wikipedia.org/wiki/Latent_semantic_analysis), [Probabilistic LSA](https://en.wikipedia.org/wiki/Probabilistic_latent_semantic_analysis) - - [What is a good explanation of Latent Dirichlet Allocation?](https://www.quora.com/What-is-a-good-explanation-of-Latent-Dirichlet-Allocation) - - [Awesome LDA Explanation!](http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/). [Another good explanation](http://confusedlanguagetech.blogspot.in/2012/07/jordan-boyd-graber-and-philip-resnik.html) - - [The LDA Buffet- Intuitive Explanation](http://www.matthewjockers.net/2011/09/29/the-lda-buffet-is-now-open-or-latent-dirichlet-allocation-for-english-majors/) - - [Difference between LSI and LDA](https://www.quora.com/Whats-the-difference-between-Latent-Semantic-Indexing-LSI-and-Latent-Dirichlet-Allocation-LDA) - - [Original LDA Paper](https://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf) - - [alpha and beta in LDA](http://datascience.stackexchange.com/questions/199/what-does-the-alpha-and-beta-hyperparameters-contribute-to-in-latent-dirichlet-a) - - [Intuitive explanation of the Dirichlet distribution](https://www.quora.com/What-is-an-intuitive-explanation-of-the-Dirichlet-distribution) - - [Topic modeling made just simple enough](https://tedunderwood.com/2012/04/07/topic-modeling-made-just-simple-enough/) - - [Online LDA](http://alexminnaar.com/online-latent-dirichlet-allocation-the-best-option-for-topic-modeling-with-large-data-sets.html), [Online LDA with Spark](http://alexminnaar.com/distributed-online-latent-dirichlet-allocation-with-apache-spark.html) - - [LDA in Scala](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-i-the-theory.html), [Part 2](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-ii-the-code.html) - - [Segmentation of Twitter Timelines via Topic Modeling](http://alexperrier.github.io/jekyll/update/2015/09/16/segmentation_twitter_timelines_lda_vs_lsa.html) - - [Topic Modeling of Twitter Followers](http://alexperrier.github.io/jekyll/update/2015/09/04/topic-modeling-of-twitter-followers.html) - - -- word2vec - - [Google word2vec](https://code.google.com/archive/p/word2vec) - - [Bag of Words Model Wiki](https://en.wikipedia.org/wiki/Bag-of-words_model) - - [word2vec Tutorial](https://rare-technologies.com/word2vec-tutorial/) - - [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf) - - [Skip Gram Model Tutorial](http://alexminnaar.com/word2vec-tutorial-part-i-the-skip-gram-model.html), [CBoW Model](http://alexminnaar.com/word2vec-tutorial-part-ii-the-continuous-bag-of-words-model.html) - - [Word Vectors Kaggle Tutorial Python](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors) - - [Making sense of word2vec](http://rare-technologies.com/making-sense-of-word2vec/) - - [word2vec explained on deeplearning4j](http://deeplearning4j.org/word2vec.html) - - [Quora word2vec](https://www.quora.com/How-does-word2vec-work) - - [Other Quora Resources](https://www.quora.com/What-are-the-continuous-bag-of-words-and-skip-gram-architectures-in-laymans-terms), [2](https://www.quora.com/What-is-the-difference-between-the-Bag-of-Words-model-and-the-Continuous-Bag-of-Words-model), [3](https://www.quora.com/Is-skip-gram-negative-sampling-better-than-CBOW-NS-for-word2vec-If-so-why) - - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) - -- Text Clustering - - [How string clustering works](http://stackoverflow.com/questions/8196371/how-clustering-works-especially-string-clustering) - - [Levenshtein distance for measuring the difference between two sequences](https://en.wikipedia.org/wiki/Levenshtein_distance) - - [Text clustering with Levenshtein distances](http://stackoverflow.com/questions/21511801/text-clustering-with-levenshtein-distances) - -- Text Classification - - [Classification Text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/) - -- [Language learning with NLP and reinforcement learning](http://blog.dennybritz.com/2015/09/11/reimagining-language-learning-with-nlp-and-reinforcement-learning/) -- [Kaggle Tutorial Bag of Words and Word vectors](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 3](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors) -- [What would Shakespeare say (NLP Tutorial)](https://gigadom.wordpress.com/2015/10/02/natural-language-processing-what-would-shakespeare-say/) -- [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf) - - -##Computer Vision -- [Awesome computer vision (github)](https://github.com/jbhuang0604/awesome-computer-vision) -- [Awesome deep vision (github)](https://github.com/kjw0612/awesome-deep-vision) - - - -##Support Vector Machine -- [Highest Voted Questions about SVMs on Cross Validated](http://stats.stackexchange.com/questions/tagged/svm) -- [Help me Understand SVMs!](http://stats.stackexchange.com/questions/3947/help-me-understand-support-vector-machines) -- [SVM in Layman's terms](https://www.quora.com/What-does-support-vector-machine-SVM-mean-in-laymans-terms) -- [How does SVM Work | Comparisons](http://stats.stackexchange.com/questions/23391/how-does-a-support-vector-machine-svm-work) -- [A tutorial on SVMs](http://alex.smola.org/papers/2003/SmoSch03b.pdf) -- [Practical Guide to SVC](http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf), [Slides](http://www.csie.ntu.edu.tw/~cjlin/talks/freiburg.pdf) -- [Introductory Overview of SVMs](http://www.statsoft.com/Textbook/Support-Vector-Machines) -- Comparisons - - [SVMs > ANNs](http://stackoverflow.com/questions/6699222/support-vector-machines-better-than-artificial-neural-networks-in-which-learn?rq=1), [ANNs > SVMs](http://stackoverflow.com/questions/11632516/what-are-advantages-of-artificial-neural-networks-over-support-vector-machines), [Another Comparison](http://www.svms.org/anns.html) - - [Trees > SVMs](http://stats.stackexchange.com/questions/57438/why-is-svm-not-so-good-as-decision-tree-on-the-same-data) - - [Kernel Logistic Regression vs SVM](http://stats.stackexchange.com/questions/43996/kernel-logistic-regression-vs-svm) - - [Logistic Regression vs SVM](http://stats.stackexchange.com/questions/58684/regularized-logistic-regression-and-support-vector-machine), [2](http://stats.stackexchange.com/questions/95340/svm-v-s-logistic-regression), [3](https://www.quora.com/Support-Vector-Machines/What-is-the-difference-between-Linear-SVMs-and-Logistic-Regression) -- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf) -- [Variable Importance from SVM](http://stats.stackexchange.com/questions/2179/variable-importance-from-svm) -- Software - - [LIBSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) - - [Intro to SVM in R](http://cbio.ensmp.fr/~jvert/svn/tutorials/practical/svmbasic/svmbasic_notes.pdf) -- Kernels - - [What are Kernels in ML and SVM?](https://www.quora.com/What-are-Kernels-in-Machine-Learning-and-SVM) - - [Intuition Behind Gaussian Kernel in SVMs?](https://www.quora.com/Support-Vector-Machines/What-is-the-intuition-behind-Gaussian-kernel-in-SVM) -- Probabilities post SVM - - [Platt's Probabilistic Outputs for SVM](http://www.csie.ntu.edu.tw/~htlin/paper/doc/plattprob.pdf) - - [Platt Calibration Wiki](https://en.wikipedia.org/wiki/Platt_scaling) - - [Why use Platts Scaling](http://stats.stackexchange.com/questions/5196/why-use-platts-scaling) - - [Classifier Classification with Platt's Scaling](http://fastml.com/classifier-calibration-with-platts-scaling-and-isotonic-regression/) - - - -##Reinforcement Learning -- [Awesome Reinforcement Learning (GitHub)](https://github.com/aikorea/awesome-rl) -- [RL Tutorial Part 1](http://outlace.com/Reinforcement-Learning-Part-1/), [Part 2](http://outlace.com/Reinforcement-Learning-Part-2/) - - -##Decision Trees -- [Wikipedia Page - Lots of Good Info](https://en.wikipedia.org/wiki/Decision_tree_learning) -- [FAQs about Decision Trees](http://stats.stackexchange.com/questions/tagged/cart) -- [Brief Tour of Trees and Forests](http://statistical-research.com/a-brief-tour-of-the-trees-and-forests/) -- [Tree Based Models in R](http://www.statmethods.net/advstats/cart.html) -- [How Decision Trees work?](http://www.aihorizon.com/essays/generalai/decision_trees.htm) -- [Weak side of Decision Trees](http://stats.stackexchange.com/questions/1292/what-is-the-weak-side-of-decision-trees) -- [Thorough Explanation and different algorithms](http://www.ise.bgu.ac.il/faculty/liorr/hbchap9.pdf) -- [What is entropy and information gain in the context of building decision trees?](http://stackoverflow.com/questions/1859554/what-is-entropy-and-information-gain) -- [Slides Related to Decision Trees](http://www.slideshare.net/pierluca.lanzi/machine-learning-and-data-mining-11-decision-trees) -- [How do decision tree learning algorithms deal with missing values?](http://stats.stackexchange.com/questions/96025/how-do-decision-tree-learning-algorithms-deal-with-missing-values-under-the-hoo) -- [Using Surrogates to Improve Datasets with Missing Values](https://www.salford-systems.com/videos/tutorials/tips-and-tricks/using-surrogates-to-improve-datasets-with-missing-values) -- [Good Article](https://www.mindtools.com/dectree.html) -- [Are decision trees almost always binary trees?](http://stats.stackexchange.com/questions/12187/are-decision-trees-almost-always-binary-trees) -- [Pruning Decision Trees](https://en.wikipedia.org/wiki/Pruning_(decision_trees)), [Grafting of Decision Trees](https://en.wikipedia.org/wiki/Grafting_(decision_trees)) -- [What is Deviance in context of Decision Trees?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart) -- Comparison of Different Algorithms - - [CART vs CTREE](http://stats.stackexchange.com/questions/12140/conditional-inference-trees-vs-traditional-decision-trees) - - [Comparison of complexity or performance](https://stackoverflow.com/questions/9979461/different-decision-tree-algorithms-with-comparison-of-complexity-or-performance) - - [CHAID vs CART](http://stats.stackexchange.com/questions/61230/chaid-vs-crt-or-cart) , [CART vs CHAID](http://www.bzst.com/2006/10/classification-trees-cart-vs-chaid.html) - - [Good Article on comparison](http://www.ftpress.com/articles/article.aspx?p=2248639&seqNum=11) -- CART - - [Recursive Partitioning Wikipedia](https://en.wikipedia.org/wiki/Recursive_partitioning) - - [CART Explained](http://documents.software.dell.com/Statistics/Textbook/Classification-and-Regression-Trees) - - [How to measure/rank “variable importance” when using CART?](http://stats.stackexchange.com/questions/6478/how-to-measure-rank-variable-importance-when-using-cart-specifically-using) - - [Pruning a Tree in R](http://stackoverflow.com/questions/15318409/how-to-prune-a-tree-in-r) - - [Does rpart use multivariate splits by default?](http://stats.stackexchange.com/questions/4356/does-rpart-use-multivariate-splits-by-default) - - [FAQs about Recursive Partitioning](http://stats.stackexchange.com/questions/tagged/rpart) -- CTREE - - [party package in R](https://cran.r-project.org/web/packages/party/party.pdf) - - [Show volumne in each node using ctree in R](http://stackoverflow.com/questions/13772715/show-volume-in-each-node-using-ctree-plot-in-r) - - [How to extract tree structure from ctree function?](http://stackoverflow.com/questions/8675664/how-to-extract-tree-structure-from-ctree-function) -- CHAID - - [Wikipedia Artice on CHAID](https://en.wikipedia.org/wiki/CHAID) - - [Basic Introduction to CHAID](https://smartdrill.com/Introduction-to-CHAID.html) - - [Good Tutorial on CHAID](http://www.statsoft.com/Textbook/CHAID-Analysis) -- MARS - - [Wikipedia Article on MARS](https://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines) -- Probabilistic Decision Trees - - [Bayesian Learning in Probabilistic Decision Trees](http://www.stats.org.uk/bayesian/Jordan.pdf) - - [Probabilistic Trees Research Paper](http://people.stern.nyu.edu/adamodar/pdfiles/papers/probabilistic.pdf) - - -##Random Forest / Bagging -- [Awesome Random Forest (GitHub)**](https://github.com/kjw0612/awesome-random-forest) -- [How to tune RF parameters in practice?](https://www.kaggle.com/forums/f/15/kaggle-forum/t/4092/how-to-tune-rf-parameters-in-practice) -- [Measures of variable importance in random forests](http://stats.stackexchange.com/questions/12605/measures-of-variable-importance-in-random-forests) -- [Compare R-squared from two different Random Forest models](http://stats.stackexchange.com/questions/13869/compare-r-squared-from-two-different-random-forest-models) -- [OOB Estimate Explained | RF vs LDA](https://stat.ethz.ch/education/semesters/ss2012/ams/slides/v10.2.pdf) -- [Evaluating Random Forests for Survival Analysis Using Prediction Error Curve](https://www.jstatsoft.org/index.php/jss/article/view/v050i11) -- [Why doesn't Random Forest handle missing values in predictors?](http://stats.stackexchange.com/questions/98953/why-doesnt-random-forest-handle-missing-values-in-predictors) -- [How to build random forests in R with missing (NA) values?](http://stackoverflow.com/questions/8370455/how-to-build-random-forests-in-r-with-missing-na-values) -- [FAQs about Random Forest](http://stats.stackexchange.com/questions/tagged/random-forest), [More FAQs](http://stackoverflow.com/questions/tagged/random-forest) -- [Obtaining knowledge from a random forest](http://stats.stackexchange.com/questions/21152/obtaining-knowledge-from-a-random-forest) -- [Some Questions for R implementation](http://stackoverflow.com/questions/20537186/getting-predictions-after-rfimpute), [2](http://stats.stackexchange.com/questions/81609/whether-preprocessing-is-needed-before-prediction-using-finalmodel-of-randomfore), [3](http://stackoverflow.com/questions/17059432/random-forest-package-in-r-shows-error-during-prediction-if-there-are-new-fact) - - -##Boosting -- [Boosting for Better Predictions](http://www.datasciencecentral.com/profiles/blogs/boosting-algorithms-for-better-predictions) -- [Boosting Wikipedia Page](https://en.wikipedia.org/wiki/Boosting_(machine_learning)) -- [Introduction to Boosted Trees | Tianqi Chen](https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf) -- Gradient Boosting Machine - - [Gradiet Boosting Wiki](https://en.wikipedia.org/wiki/Gradient_boosting) - - [Guidelines for GBM parameters in R](http://stats.stackexchange.com/questions/25748/what-are-some-useful-guidelines-for-gbm-parameters), [Strategy to set parameters](http://stats.stackexchange.com/questions/35984/strategy-to-set-the-gbm-parameters) - - [Meaning of Interaction Depth](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm), [2](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm) - - [Role of n.minobsinnode parameter of GBM in R](http://stats.stackexchange.com/questions/30645/role-of-n-minobsinnode-parameter-of-gbm-in-r) - - [GBM in R](http://www.slideshare.net/mark_landry/gbm-package-in-r) - - [FAQs about GBM](http://stats.stackexchange.com/tags/gbm/hot) - - [GBM vs xgboost](https://www.kaggle.com/c/higgs-boson/forums/t/9497/r-s-gbm-vs-python-s-xgboost) - -- xgboost - - [xgboost tuning kaggle](https://www.kaggle.com/khozzy/rossmann-store-sales/xgboost-parameter-tuning-template/log) - - [xgboost vs gbm](https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/13012/question-to-experienced-kagglers-and-anyone-who-wants-to-take-a-shot/68296#post68296) - - [xgboost survey](https://www.kaggle.com/c/higgs-boson/forums/t/10335/xgboost-post-competition-survey) - - [Practical XGBoost in Python online course (free)](http://education.parrotprediction.teachable.com/courses/practical-xgboost-in-python) -- AdaBoost - - [AdaBoost Wiki](https://en.wikipedia.org/wiki/AdaBoost), [Python Code](https://gist.github.com/tristanwietsma/5486024) - - [AdaBoost Sparse Input Support](http://hamzehal.blogspot.com/2014/06/adaboost-sparse-input-support.html) - - [adaBag R package](https://cran.r-project.org/web/packages/adabag/adabag.pdf) - - [Tutorial](http://math.mit.edu/~rothvoss/18.304.3PM/Presentations/1-Eric-Boosting304FinalRpdf.pdf) - - -##Ensembles -- [Wikipedia Article on Ensemble Learning](https://en.wikipedia.org/wiki/Ensemble_learning) -- [Kaggle Ensembling Guide](http://mlwave.com/kaggle-ensembling-guide/) -- [The Power of Simple Ensembles](http://www.overkillanalytics.net/more-is-always-better-the-power-of-simple-ensembles/) -- [Ensemble Learning Intro](http://machine-learning.martinsewell.com/ensembles/) -- [Ensemble Learning Paper](http://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/springerEBR09.pdf) -- [Ensembling models with R](http://amunategui.github.io/blending-models/), [Ensembling Regression Models in R](http://stats.stackexchange.com/questions/26790/ensembling-regression-models), [Intro to Ensembles in R](http://www.vikparuchuri.com/blog/intro-to-ensemble-learning-in-r/) -- [Ensembling Models with caret](http://stats.stackexchange.com/questions/27361/stacking-ensembling-models-with-caret) -- [Bagging vs Boosting vs Stacking](http://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning) -- [Good Resources | Kaggle Africa Soil Property Prediction](https://www.kaggle.com/c/afsis-soil-properties/forums/t/10391/best-ensemble-references) -- [Boosting vs Bagging](http://www.chioka.in/which-is-better-boosting-or-bagging/) -- [Resources for learning how to implement ensemble methods](http://stats.stackexchange.com/questions/32703/resources-for-learning-how-to-implement-ensemble-methods) -- [How are classifications merged in an ensemble classifier?](http://stats.stackexchange.com/questions/21502/how-are-classifications-merged-in-an-ensemble-classifier) - - -##Stacking Models -- [Stacking, Blending and Stacked Generalization](http://www.chioka.in/stacking-blending-and-stacked-generalization/) -- [Stacked Generalization (Stacking)](http://machine-learning.martinsewell.com/ensembles/stacking/) -- [Stacked Generalization: when does it work?](http://www.ijcai.org/Proceedings/97-2/011.pdf) -- [Stacked Generalization Paper](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.56.1533&rep=rep1&type=pdf) - - -##Vapnik–Chervonenkis Dimension -- [Wikipedia article on VC Dimension](https://en.wikipedia.org/wiki/VC_dimension) -- [Intuitive Explanantion of VC Dimension](https://www.quora.com/Explain-VC-dimension-and-shattering-in-lucid-Way) -- [Video explaining VC Dimension](https://www.youtube.com/watch?v=puDzy2XmR5c) -- [Introduction to VC Dimension](http://www.svms.org/vc-dimension/) -- [FAQs about VC Dimension](http://stats.stackexchange.com/questions/tagged/vc-dimension) -- [Do ensemble techniques increase VC-dimension?](http://stats.stackexchange.com/questions/78076/do-ensemble-techniques-increase-vc-dimension) - - - -##Bayesian Machine Learning -- [Bayesian Methods for Hackers (using pyMC)](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers) -- [Should all Machine Learning be Bayesian?](http://videolectures.net/bark08_ghahramani_samlbb/) -- [Tutorial on Bayesian Optimisation for Machine Learning](http://www.iro.umontreal.ca/~bengioy/cifar/NCAP2014-summerschool/slides/Ryan_adams_140814_bayesopt_ncap.pdf) -- [Bayesian Reasoning and Deep Learning](http://blog.shakirm.com/2015/10/bayesian-reasoning-and-deep-learning/), [Slides](http://blog.shakirm.com/wp-content/uploads/2015/10/Bayes_Deep.pdf) -- [Bayesian Statistics Made Simple](http://greenteapress.com/wp/think-bayes/) -- [Kalman & Bayesian Filters in Python](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python) -- [Markov Chain Wikipedia Page](https://en.wikipedia.org/wiki/Markov_chain) - - - -##Semi Supervised Learning -- [Wikipedia article on Semi Supervised Learning](https://en.wikipedia.org/wiki/Semi-supervised_learning) -- [Tutorial on Semi Supervised Learning](http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf) -- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf) -- [Taxonomy](http://is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/taxo_[0].pdf) -- [Video Tutorial Weka](https://www.youtube.com/watch?v=sWxcIjZFGNM) -- [Unsupervised, Supervised and Semi Supervised learning](http://stats.stackexchange.com/questions/517/unsupervised-supervised-and-semi-supervised-learning) -- [Research Papers 1](http://mlg.eng.cam.ac.uk/zoubin/papers/zglactive.pdf), [2](http://mlg.eng.cam.ac.uk/zoubin/papers/zgl.pdf), [3](http://icml.cc/2012/papers/616.pdf) - - - - -##Optimization -- [Mean Variance Portfolio Optimization with R and Quadratic Programming](http://www.wdiam.com/2012/06/10/mean-variance-portfolio-optimization-with-r-and-quadratic-programming/?utm_content=buffer04c12&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer) -- [Algorithms for Sparse Optimization and Machine -Learning](http://www.ima.umn.edu/2011-2012/W3.26-30.12/activities/Wright-Steve/sjw-ima12) -- [Optimization Algorithms in Machine Learning](http://pages.cs.wisc.edu/~swright/nips2010/sjw-nips10.pdf), [Video Lecture](http://videolectures.net/nips2010_wright_oaml/) -- [Optimization Algorithms for Data Analysis](http://www.birs.ca/workshops/2011/11w2035/files/Wright.pdf) -- [Video Lectures on Optimization](http://videolectures.net/stephen_j_wright/) -- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf) -- [The Interplay of Optimization and Machine Learning Research](http://jmlr.org/papers/volume7/MLOPT-intro06a/MLOPT-intro06a.pdf) - - -##Other Tutorials -- For a collection of Data Science Tutorials using R, please refer to [this list](https://github.com/ujjwalkarn/DataScienceR). -- For a collection of Data Science Tutorials using Python, please refer to [this list](https://github.com/ujjwalkarn/DataSciencePython). + +# Machine Learning & Deep Learning Tutorials [![Awesome](https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg)](https://github.com/sindresorhus/awesome) + +- This repository contains a topic-wise curated list of Machine Learning and Deep Learning tutorials, articles and other resources. Other awesome lists can be found in this [list](https://github.com/sindresorhus/awesome). + +- If you want to contribute to this list, please read [Contributing Guidelines](https://github.com/ujjwalkarn/Machine-Learning-Tutorials/blob/master/contributing.md). + +- [Curated list of R tutorials for Data Science, NLP and Machine Learning](https://github.com/ujjwalkarn/DataScienceR). + +- [Curated list of Python tutorials for Data Science, NLP and Machine Learning](https://github.com/ujjwalkarn/DataSciencePython). + + +## Contents +- [Introduction](#general) +- [Interview Resources](#interview) +- [Artificial Intelligence](#ai) +- [Genetic Algorithms](#ga) +- [Statistics](#stat) +- [Useful Blogs](#blogs) +- [Resources on Quora](#quora) +- [Resources on Kaggle](#kaggle) +- [Cheat Sheets](#cs) +- [Classification](#classification) +- [Linear Regression](#linear) +- [Logistic Regression](#logistic) +- [Model Validation using Resampling](#validation) + - [Cross Validation](#cross) + - [Bootstraping](#boot) +- [Deep Learning](#deep) + - [Frameworks](#frame) + - [Feed Forward Networks](#feed) + - [Recurrent Neural Nets, LSTM, GRU](#rnn) + - [Restricted Boltzmann Machine, DBNs](#rbm) + - [Autoencoders](#auto) + - [Convolutional Neural Nets](#cnn) + - [Graph Representation Learning](#nrl) +- [Natural Language Processing](#nlp) + - [Topic Modeling, LDA](#topic) + - [Word2Vec](#word2vec) +- [Computer Vision](#vision) +- [Support Vector Machine](#svm) +- [Reinforcement Learning](#rl) +- [Decision Trees](#dt) +- [Random Forest / Bagging](#rf) +- [Boosting](#gbm) +- [Ensembles](#ensem) +- [Stacking Models](#stack) +- [VC Dimension](#vc) +- [Bayesian Machine Learning](#bayes) +- [Semi Supervised Learning](#semi) +- [Optimizations](#opt) +- [Other Useful Tutorials](#other) + + + +## Introduction + +- [Machine Learning Course by Andrew Ng (Stanford University)](https://www.coursera.org/learn/machine-learning) + +- [AI/ML YouTube Courses](https://github.com/dair-ai/ML-YouTube-Courses) + +- [Curated List of Machine Learning Resources](https://hackr.io/tutorials/learn-machine-learning-ml) + +- [In-depth introduction to machine learning in 15 hours of expert videos](http://www.dataschool.io/15-hours-of-expert-machine-learning-videos/) + +- [An Introduction to Statistical Learning](http://www-bcf.usc.edu/~gareth/ISL/) + +- [List of Machine Learning University Courses](https://github.com/prakhar1989/awesome-courses#machine-learning) + +- [Machine Learning for Software Engineers](https://github.com/ZuzooVn/machine-learning-for-software-engineers) + +- [Dive into Machine Learning](https://github.com/hangtwenty/dive-into-machine-learning) + +- [A curated list of awesome Machine Learning frameworks, libraries and software](https://github.com/josephmisiti/awesome-machine-learning) + +- [A curated list of awesome data visualization libraries and resources.](https://github.com/fasouto/awesome-dataviz) + +- [An awesome Data Science repository to learn and apply for real world problems](https://github.com/okulbilisim/awesome-datascience) + +- [The Open Source Data Science Masters](http://datasciencemasters.org/) + +- [Machine Learning FAQs on Cross Validated](http://stats.stackexchange.com/questions/tagged/machine-learning) + +- [Machine Learning algorithms that you should always have a strong understanding of](https://www.quora.com/What-are-some-Machine-Learning-algorithms-that-you-should-always-have-a-strong-understanding-of-and-why) + +- [Difference between Linearly Independent, Orthogonal, and Uncorrelated Variables](http://terpconnect.umd.edu/~bmomen/BIOM621/LineardepCorrOrthogonal.pdf) + +- [List of Machine Learning Concepts](https://en.wikipedia.org/wiki/List_of_machine_learning_concepts) + +- [Slides on Several Machine Learning Topics](http://www.slideshare.net/pierluca.lanzi/presentations) + +- [MIT Machine Learning Lecture Slides](http://www.ai.mit.edu/courses/6.867-f04/lectures.html) + +- [Comparison Supervised Learning Algorithms](http://www.dataschool.io/comparing-supervised-learning-algorithms/) + +- [Learning Data Science Fundamentals](http://www.dataschool.io/learning-data-science-fundamentals/) + +- [Machine Learning mistakes to avoid](https://medium.com/@nomadic_mind/new-to-machine-learning-avoid-these-three-mistakes-73258b3848a4#.lih061l3l) + +- [Statistical Machine Learning Course](http://www.stat.cmu.edu/~larry/=sml/) + +- [TheAnalyticsEdge edX Notes and Codes](https://github.com/pedrosan/TheAnalyticsEdge) + +- [Have Fun With Machine Learning](https://github.com/humphd/have-fun-with-machine-learning) + +- [Twitter's Most Shared #machineLearning Content From The Past 7 Days](http://theherdlocker.com/tweet/popularity/machinelearning) + +- [Grokking Machine Learning](https://www.manning.com/books/grokking-machine-learning) + + + +## Interview Resources + +- [41 Essential Machine Learning Interview Questions (with answers)](https://www.springboard.com/blog/machine-learning-interview-questions/) + +- [How can a computer science graduate student prepare himself for data scientist interviews?](https://www.quora.com/How-can-a-computer-science-graduate-student-prepare-himself-for-data-scientist-machine-learning-intern-interviews) + +- [How do I learn Machine Learning?](https://www.quora.com/How-do-I-learn-machine-learning-1) + +- [FAQs about Data Science Interviews](https://www.quora.com/topic/Data-Science-Interviews/faq) + +- [What are the key skills of a data scientist?](https://www.quora.com/What-are-the-key-skills-of-a-data-scientist) + +- [The Big List of DS/ML Interview Resources](https://towardsdatascience.com/the-big-list-of-ds-ml-interview-resources-2db4f651bd63) + + + +## Artificial Intelligence + +- [Awesome Artificial Intelligence (GitHub Repo)](https://github.com/owainlewis/awesome-artificial-intelligence) + +- [UC Berkeley CS188 Intro to AI](http://ai.berkeley.edu/home.html), [Lecture Videos](http://ai.berkeley.edu/lecture_videos.html), [2](https://www.youtube.com/watch?v=W1S-HSakPTM) + +- [Programming Community Curated Resources for learning Artificial Intelligence](https://hackr.io/tutorials/learn-artificial-intelligence-ai) + +- [MIT 6.034 Artificial Intelligence Lecture Videos](https://www.youtube.com/playlist?list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi), [Complete Course](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/) + +- [edX course | Klein & Abbeel](https://courses.edx.org/courses/BerkeleyX/CS188x_1/1T2013/info) + +- [Udacity Course | Norvig & Thrun](https://www.udacity.com/course/intro-to-artificial-intelligence--cs271) + +- [TED talks on AI](http://www.ted.com/playlists/310/talks_on_artificial_intelligen) + + + +## Genetic Algorithms + +- [Genetic Algorithms Wikipedia Page](https://en.wikipedia.org/wiki/Genetic_algorithm) + +- [Simple Implementation of Genetic Algorithms in Python (Part 1)](http://outlace.com/miniga.html), [Part 2](http://outlace.com/miniga_addendum.html) + +- [Genetic Algorithms vs Artificial Neural Networks](http://stackoverflow.com/questions/1402370/when-to-use-genetic-algorithms-vs-when-to-use-neural-networks) + +- [Genetic Algorithms Explained in Plain English](http://www.ai-junkie.com/ga/intro/gat1.html) + +- [Genetic Programming](https://en.wikipedia.org/wiki/Genetic_programming) + + - [Genetic Programming in Python (GitHub)](https://github.com/trevorstephens/gplearn) + + - [Genetic Alogorithms vs Genetic Programming (Quora)](https://www.quora.com/Whats-the-difference-between-Genetic-Algorithms-and-Genetic-Programming), [StackOverflow](http://stackoverflow.com/questions/3819977/what-are-the-differences-between-genetic-algorithms-and-genetic-programming) + + + +## Statistics + +- [Stat Trek Website](http://stattrek.com/) - A dedicated website to teach yourselves Statistics + +- [Learn Statistics Using Python](https://github.com/rouseguy/intro2stats) - Learn Statistics using an application-centric programming approach + +- [Statistics for Hackers | Slides | @jakevdp](https://speakerdeck.com/jakevdp/statistics-for-hackers) - Slides by Jake VanderPlas + +- [Online Statistics Book](http://onlinestatbook.com/2/index.html) - An Interactive Multimedia Course for Studying Statistics + +- [What is a Sampling Distribution?](http://stattrek.com/sampling/sampling-distribution.aspx) + +- Tutorials + + - [AP Statistics Tutorial](http://stattrek.com/tutorials/ap-statistics-tutorial.aspx) + + - [Statistics and Probability Tutorial](http://stattrek.com/tutorials/statistics-tutorial.aspx) + + - [Matrix Algebra Tutorial](http://stattrek.com/tutorials/matrix-algebra-tutorial.aspx) + +- [What is an Unbiased Estimator?](https://www.physicsforums.com/threads/what-is-an-unbiased-estimator.547728/) + +- [Goodness of Fit Explained](https://en.wikipedia.org/wiki/Goodness_of_fit) + +- [What are QQ Plots?](http://onlinestatbook.com/2/advanced_graphs/q-q_plots.html) + +- [OpenIntro Statistics](https://www.openintro.org/stat/textbook.php?stat_book=os) - Free PDF textbook + + + +## Useful Blogs + +- [Edwin Chen's Blog](http://blog.echen.me/) - A blog about Math, stats, ML, crowdsourcing, data science + +- [The Data School Blog](http://www.dataschool.io/) - Data science for beginners! + +- [ML Wave](http://mlwave.com/) - A blog for Learning Machine Learning + +- [Andrej Karpathy](http://karpathy.github.io/) - A blog about Deep Learning and Data Science in general + +- [Colah's Blog](http://colah.github.io/) - Awesome Neural Networks Blog + +- [Alex Minnaar's Blog](http://alexminnaar.com/) - A blog about Machine Learning and Software Engineering + +- [Statistically Significant](http://andland.github.io/) - Andrew Landgraf's Data Science Blog + +- [Simply Statistics](http://simplystatistics.org/) - A blog by three biostatistics professors + +- [Yanir Seroussi's Blog](https://yanirseroussi.com/) - A blog about Data Science and beyond + +- [fastML](http://fastml.com/) - Machine learning made easy + +- [Trevor Stephens Blog](http://trevorstephens.com/) - Trevor Stephens Personal Page + +- [no free hunch | kaggle](http://blog.kaggle.com/) - The Kaggle Blog about all things Data Science + +- [A Quantitative Journey | outlace](http://outlace.com/) - learning quantitative applications + +- [r4stats](http://r4stats.com/) - analyze the world of data science, and to help people learn to use R + +- [Variance Explained](http://varianceexplained.org/) - David Robinson's Blog + +- [AI Junkie](http://www.ai-junkie.com/) - a blog about Artificial Intellingence + +- [Deep Learning Blog by Tim Dettmers](http://timdettmers.com/) - Making deep learning accessible + +- [J Alammar's Blog](http://jalammar.github.io/)- Blog posts about Machine Learning and Neural Nets + +- [Adam Geitgey](https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471#.f7vwrtfne) - Easiest Introduction to machine learning + +- [Ethen's Notebook Collection](https://github.com/ethen8181/machine-learning) - Continuously updated machine learning documentations (mainly in Python3). Contents include educational implementation of machine learning algorithms from scratch and open-source library usage + + + +## Resources on Quora + +- [Most Viewed Machine Learning writers](https://www.quora.com/topic/Machine-Learning/writers) + +- [Data Science Topic on Quora](https://www.quora.com/Data-Science) + +- [William Chen's Answers](https://www.quora.com/William-Chen-6/answers) + +- [Michael Hochster's Answers](https://www.quora.com/Michael-Hochster/answers) + +- [Ricardo Vladimiro's Answers](https://www.quora.com/Ricardo-Vladimiro-1/answers) + +- [Storytelling with Statistics](https://datastories.quora.com/) + +- [Data Science FAQs on Quora](https://www.quora.com/topic/Data-Science/faq) + +- [Machine Learning FAQs on Quora](https://www.quora.com/topic/Machine-Learning/faq) + + + +## Kaggle Competitions WriteUp + +- [How to almost win Kaggle Competitions](https://yanirseroussi.com/2014/08/24/how-to-almost-win-kaggle-competitions/) + +- [Convolution Neural Networks for EEG detection](http://blog.kaggle.com/2015/10/05/grasp-and-lift-eeg-detection-winners-interview-3rd-place-team-hedj/) + +- [Facebook Recruiting III Explained](http://alexminnaar.com/tag/kaggle-competitions.html) + +- [Predicting CTR with Online ML](http://mlwave.com/predicting-click-through-rates-with-online-machine-learning/) + +- [How to Rank 10% in Your First Kaggle Competition](https://dnc1994.com/2016/05/rank-10-percent-in-first-kaggle-competition-en/) + + + +## Cheat Sheets + +- [Probability Cheat Sheet](http://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf), +[Source](http://www.wzchen.com/probability-cheatsheet/) + +- [Machine Learning Cheat Sheet](https://github.com/soulmachine/machine-learning-cheat-sheet) + +- [ML Compiled](https://ml-compiled.readthedocs.io/en/latest/) + + + +## Classification + +- [Does Balancing Classes Improve Classifier Performance?](http://www.win-vector.com/blog/2015/02/does-balancing-classes-improve-classifier-performance/) + +- [What is Deviance?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart) + +- [When to choose which machine learning classifier?](http://stackoverflow.com/questions/2595176/when-to-choose-which-machine-learning-classifier) + +- [What are the advantages of different classification algorithms?](https://www.quora.com/What-are-the-advantages-of-different-classification-algorithms) + +- [ROC and AUC Explained](http://www.dataschool.io/roc-curves-and-auc-explained/) ([related video](https://youtu.be/OAl6eAyP-yo)) + +- [An introduction to ROC analysis](https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf) + +- [Simple guide to confusion matrix terminology](http://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/) + + + + +## Linear Regression + +- [General](#general-) + + - [Assumptions of Linear Regression](http://pareonline.net/getvn.asp?n=2&v=8), [Stack Exchange](http://stats.stackexchange.com/questions/16381/what-is-a-complete-list-of-the-usual-assumptions-for-linear-regression) + + - [Linear Regression Comprehensive Resource](http://people.duke.edu/~rnau/regintro.htm) + + - [Applying and Interpreting Linear Regression](http://www.dataschool.io/applying-and-interpreting-linear-regression/) + + - [What does having constant variance in a linear regression model mean?](http://stats.stackexchange.com/questions/52089/what-does-having-constant-variance-in-a-linear-regression-model-mean/52107?stw=2#52107) + + - [Difference between linear regression on y with x and x with y](http://stats.stackexchange.com/questions/22718/what-is-the-difference-between-linear-regression-on-y-with-x-and-x-with-y?lq=1) + + - [Is linear regression valid when the dependant variable is not normally distributed?](https://www.researchgate.net/post/Is_linear_regression_valid_when_the_outcome_dependant_variable_not_normally_distributed) +- Multicollinearity and VIF + + - [Dummy Variable Trap | Multicollinearity](https://en.wikipedia.org/wiki/Multicollinearity) + + - [Dealing with multicollinearity using VIFs](https://jonlefcheck.net/2012/12/28/dealing-with-multicollinearity-using-variance-inflation-factors/) + +- [Residual Analysis](#residuals-) + + - [Interpreting plot.lm() in R](http://stats.stackexchange.com/questions/58141/interpreting-plot-lm) + + - [How to interpret a QQ plot?](http://stats.stackexchange.com/questions/101274/how-to-interpret-a-qq-plot?lq=1) + + - [Interpreting Residuals vs Fitted Plot](http://stats.stackexchange.com/questions/76226/interpreting-the-residuals-vs-fitted-values-plot-for-verifying-the-assumptions) + +- [Outliers](#outliers-) + + - [How should outliers be dealt with?](http://stats.stackexchange.com/questions/175/how-should-outliers-be-dealt-with-in-linear-regression-analysis) + +- [Elastic Net](https://en.wikipedia.org/wiki/Elastic_net_regularization) + - [Regularization and Variable Selection via the +Elastic Net](https://web.stanford.edu/~hastie/Papers/elasticnet.pdf) + + + +## Logistic Regression + +- [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression) + +- [Geometric Intuition of Logistic Regression](http://florianhartl.com/logistic-regression-geometric-intuition.html) + +- [Obtaining predicted categories (choosing threshold)](http://stats.stackexchange.com/questions/25389/obtaining-predicted-values-y-1-or-0-from-a-logistic-regression-model-fit) + +- [Residuals in logistic regression](http://stats.stackexchange.com/questions/1432/what-do-the-residuals-in-a-logistic-regression-mean) + +- [Difference between logit and probit models](http://stats.stackexchange.com/questions/20523/difference-between-logit-and-probit-models#30909), [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression), [Probit Model Wiki](https://en.wikipedia.org/wiki/Probit_model) + +- [Pseudo R2 for Logistic Regression](http://stats.stackexchange.com/questions/3559/which-pseudo-r2-measure-is-the-one-to-report-for-logistic-regression-cox-s), [How to calculate](http://stats.stackexchange.com/questions/8511/how-to-calculate-pseudo-r2-from-rs-logistic-regression), [Other Details](http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm) + +- [Guide to an in-depth understanding of logistic regression](http://www.dataschool.io/guide-to-logistic-regression/) + + + +## Model Validation using Resampling + +- [Resampling Explained](https://en.wikipedia.org/wiki/Resampling_(statistics)) + +- [Partioning data set in R](http://stackoverflow.com/questions/13536537/partitioning-data-set-in-r-based-on-multiple-classes-of-observations) + +- [Implementing hold-out Validaion in R](http://stackoverflow.com/questions/22972854/how-to-implement-a-hold-out-validation-in-r), [2](http://www.gettinggeneticsdone.com/2011/02/split-data-frame-into-testing-and.html) + + + +- [Cross Validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics)) + - [How to use cross-validation in predictive modeling](http://stuartlacy.co.uk/2016/02/04/how-to-correctly-use-cross-validation-in-predictive-modelling/) + - [Training with Full dataset after CV?](http://stats.stackexchange.com/questions/11602/training-with-the-full-dataset-after-cross-validation) + + - [Which CV method is best?](http://stats.stackexchange.com/questions/103459/how-do-i-know-which-method-of-cross-validation-is-best) + + - [Variance Estimates in k-fold CV](http://stats.stackexchange.com/questions/31190/variance-estimates-in-k-fold-cross-validation) + + - [Is CV a subsitute for Validation Set?](http://stats.stackexchange.com/questions/18856/is-cross-validation-a-proper-substitute-for-validation-set) + + - [Choice of k in k-fold CV](http://stats.stackexchange.com/questions/27730/choice-of-k-in-k-fold-cross-validation) + + - [CV for ensemble learning](http://stats.stackexchange.com/questions/102631/k-fold-cross-validation-of-ensemble-learning) + + - [k-fold CV in R](http://stackoverflow.com/questions/22909197/creating-folds-for-k-fold-cv-in-r-using-caret) + + - [Good Resources](http://www.chioka.in/tag/cross-validation/) + + - Overfitting and Cross Validation + + - [Preventing Overfitting the Cross Validation Data | Andrew Ng](http://ai.stanford.edu/~ang/papers/cv-final.pdf) + + - [Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation](http://www.jmlr.org/papers/volume11/cawley10a/cawley10a.pdf) + + - [CV for detecting and preventing Overfitting](http://www.autonlab.org/tutorials/overfit10.pdf) + + - [How does CV overcome the Overfitting Problem](http://stats.stackexchange.com/questions/9053/how-does-cross-validation-overcome-the-overfitting-problem) + + + + +- [Bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(statistics)) + + - [Why Bootstrapping Works?](http://stats.stackexchange.com/questions/26088/explaining-to-laypeople-why-bootstrapping-works) + + - [Good Animation](https://www.stat.auckland.ac.nz/~wild/BootAnim/) + + - [Example of Bootstapping](http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm) + + - [Understanding Bootstapping for Validation and Model Selection](http://stats.stackexchange.com/questions/14516/understanding-bootstrapping-for-validation-and-model-selection?rq=1) + + - [Cross Validation vs Bootstrap to estimate prediction error](http://stats.stackexchange.com/questions/18348/differences-between-cross-validation-and-bootstrapping-to-estimate-the-predictio), [Cross-validation vs .632 bootstrapping to evaluate classification performance](http://stats.stackexchange.com/questions/71184/cross-validation-or-bootstrapping-to-evaluate-classification-performance) + + + + +## Deep Learning + +- [fast.ai - Practical Deep Learning For Coders](http://course.fast.ai/) + +- [fast.ai - Cutting Edge Deep Learning For Coders](http://course.fast.ai/part2.html) + +- [A curated list of awesome Deep Learning tutorials, projects and communities](https://github.com/ChristosChristofidis/awesome-deep-learning) + +- **[Deep Learning Papers Reading Roadmap](https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap/blob/master/README.md)** + +- [Lots of Deep Learning Resources](http://deeplearning4j.org/documentation.html) + +- [Interesting Deep Learning and NLP Projects (Stanford)](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/) + +- [Core Concepts of Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/) + +- [Understanding Natural Language with Deep Neural Networks Using Torch](https://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) + +- [Stanford Deep Learning Tutorial](http://ufldl.stanford.edu/tutorial/) + +- [Deep Learning FAQs on Quora](https://www.quora.com/topic/Deep-Learning/faq) + +- [Google+ Deep Learning Page](https://plus.google.com/communities/112866381580457264725) + +- [Recent Reddit AMAs related to Deep Learning](http://deeplearning.net/2014/11/22/recent-reddit-amas-about-deep-learning/), [Another AMA](https://www.reddit.com/r/IAmA/comments/3mdk9v/we_are_google_researchers_working_on_deep/) + +- [Where to Learn Deep Learning?](http://www.kdnuggets.com/2014/05/learn-deep-learning-courses-tutorials-overviews.html) + +- [Deep Learning nvidia concepts](http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/) + +- [Introduction to Deep Learning Using Python (GitHub)](https://github.com/rouseguy/intro2deeplearning), [Good Introduction Slides](https://speakerdeck.com/bargava/introduction-to-deep-learning) + +- [Video Lectures Oxford 2015](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu), [Video Lectures Summer School Montreal](http://videolectures.net/deeplearning2015_montreal/) + +- [Deep Learning Software List](http://deeplearning.net/software_links/) + +- [Hacker's guide to Neural Nets](http://karpathy.github.io/neuralnets/) + +- [Top arxiv Deep Learning Papers explained](http://www.kdnuggets.com/2015/10/top-arxiv-deep-learning-papers-explained.html) + +- [Geoff Hinton Youtube Vidoes on Deep Learning](https://www.youtube.com/watch?v=IcOMKXAw5VA) + +- [Awesome Deep Learning Reading List](http://deeplearning.net/reading-list/) + +- [Deep Learning Comprehensive Website](http://deeplearning.net/), [Software](http://deeplearning.net/software_links/) + +- [deeplearning Tutorials](http://deeplearning4j.org/) + +- [AWESOME! Deep Learning Tutorial](https://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks) + +- [Deep Learning Basics](http://alexminnaar.com/deep-learning-basics-neural-networks-backpropagation-and-stochastic-gradient-descent.html) + +- [Intuition Behind Backpropagation](https://medium.com/spidernitt/breaking-down-neural-networks-an-intuitive-approach-to-backpropagation-3b2ff958794c) + +- [Stanford Tutorials](http://ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/) + +- [Train, Validation & Test in Artificial Neural Networks](http://stackoverflow.com/questions/2976452/whats-is-the-difference-between-train-validation-and-test-set-in-neural-networ) + +- [Artificial Neural Networks Tutorials](http://stackoverflow.com/questions/478947/what-are-some-good-resources-for-learning-about-artificial-neural-networks) + +- [Neural Networks FAQs on Stack Overflow](http://stackoverflow.com/questions/tagged/neural-network?sort=votes&pageSize=50) + +- [Deep Learning Tutorials on deeplearning.net](http://deeplearning.net/tutorial/index.html) + +- [Neural Networks and Deep Learning Online Book](http://neuralnetworksanddeeplearning.com/) + +- Neural Machine Translation + + - **[Machine Translation Reading List](https://github.com/THUNLP-MT/MT-Reading-List#machine-translation-reading-list)** + + - [Introduction to Neural Machine Translation with GPUs (part 1)](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-with-gpus/), [Part 2](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-2/), [Part 3](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/) + + - [Deep Speech: Accurate Speech Recognition with GPU-Accelerated Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-speech-accurate-speech-recognition-gpu-accelerated-deep-learning/) + + + +- Deep Learning Frameworks + + - [Torch vs. Theano](http://fastml.com/torch-vs-theano/) + + - [dl4j vs. torch7 vs. theano](http://deeplearning4j.org/compare-dl4j-torch7-pylearn.html) + + - [Deep Learning Libraries by Language](http://www.teglor.com/b/deep-learning-libraries-language-cm569/) + + + - [Theano](https://en.wikipedia.org/wiki/Theano_(software)) + + - [Website](http://deeplearning.net/software/theano/) + + - [Theano Introduction](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/) + + - [Theano Tutorial](http://outlace.com/Beginner-Tutorial-Theano/) + + - [Good Theano Tutorial](http://deeplearning.net/software/theano/tutorial/) + + - [Logistic Regression using Theano for classifying digits](http://deeplearning.net/tutorial/logreg.html#logreg) + + - [MLP using Theano](http://deeplearning.net/tutorial/mlp.html#mlp) + + - [CNN using Theano](http://deeplearning.net/tutorial/lenet.html#lenet) + + - [RNNs using Theano](http://deeplearning.net/tutorial/rnnslu.html#rnnslu) + + - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm) + + - [RBM using Theano](http://deeplearning.net/tutorial/rbm.html#rbm) + + - [DBNs using Theano](http://deeplearning.net/tutorial/DBN.html#dbn) + + - [All Codes](https://github.com/lisa-lab/DeepLearningTutorials) + + - [Deep Learning Implementation Tutorials - Keras and Lasagne](https://github.com/vict0rsch/deep_learning/) + + - [Torch](http://torch.ch/) + + - [Torch ML Tutorial](http://code.madbits.com/wiki/doku.php), [Code](https://github.com/torch/tutorials) + + - [Intro to Torch](http://ml.informatik.uni-freiburg.de/_media/teaching/ws1415/presentation_dl_lect3.pdf) + + - [Learning Torch GitHub Repo](https://github.com/chetannaik/learning_torch) + + - [Awesome-Torch (Repository on GitHub)](https://github.com/carpedm20/awesome-torch) + + - [Machine Learning using Torch Oxford Univ](https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/), [Code](https://github.com/oxford-cs-ml-2015) + + - [Torch Internals Overview](https://apaszke.github.io/torch-internals.html) + + - [Torch Cheatsheet](https://github.com/torch/torch7/wiki/Cheatsheet) + + - [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) + + - Caffe + - [Deep Learning for Computer Vision with Caffe and cuDNN](https://devblogs.nvidia.com/parallelforall/deep-learning-computer-vision-caffe-cudnn/) + + - TensorFlow + - [Website](http://tensorflow.org/) + + - [TensorFlow Examples for Beginners](https://github.com/aymericdamien/TensorFlow-Examples) + + - [Stanford Tensorflow for Deep Learning Research Course](https://web.stanford.edu/class/cs20si/syllabus.html) + + - [GitHub Repo](https://github.com/chiphuyen/tf-stanford-tutorials) + + - [Simplified Scikit-learn Style Interface to TensorFlow](https://github.com/tensorflow/skflow) + + - [Learning TensorFlow GitHub Repo](https://github.com/chetannaik/learning_tensorflow) + + - [Benchmark TensorFlow GitHub](https://github.com/soumith/convnet-benchmarks/issues/66) + + - [Awesome TensorFlow List](https://github.com/jtoy/awesome-tensorflow) + + - [TensorFlow Book](https://github.com/BinRoot/TensorFlow-Book) + + - [Android TensorFlow Machine Learning Example](https://blog.mindorks.com/android-tensorflow-machine-learning-example-ff0e9b2654cc) + + - [GitHub Repo](https://github.com/MindorksOpenSource/AndroidTensorFlowMachineLearningExample) + - [Creating Custom Model For Android Using TensorFlow](https://blog.mindorks.com/creating-custom-model-for-android-using-tensorflow-3f963d270bfb) + - [GitHub Repo](https://github.com/MindorksOpenSource/AndroidTensorFlowMNISTExample) + + + +- Feed Forward Networks + + - [A Quick Introduction to Neural Networks](https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/) + + - [Implementing a Neural Network from scratch](http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/), [Code](https://github.com/dennybritz/nn-from-scratch) + + - [Speeding up your Neural Network with Theano and the gpu](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/), [Code](https://github.com/dennybritz/nn-theano) + + - [Basic ANN Theory](https://takinginitiative.wordpress.com/2008/04/03/basic-neural-network-tutorial-theory/) + + - [Role of Bias in Neural Networks](http://stackoverflow.com/questions/2480650/role-of-bias-in-neural-networks) + + - [Choosing number of hidden layers and nodes](http://stackoverflow.com/questions/3345079/estimating-the-number-of-neurons-and-number-of-layers-of-an-artificial-neural-ne),[2](http://stackoverflow.com/questions/10565868/multi-layer-perceptron-mlp-architecture-criteria-for-choosing-number-of-hidde?lq=1),[3](http://stackoverflow.com/questions/9436209/how-to-choose-number-of-hidden-layers-and-nodes-in-neural-network/2#) + + - [Backpropagation in Matrix Form](http://sudeepraja.github.io/Neural/) + + - [ANN implemented in C++ | AI Junkie](http://www.ai-junkie.com/ann/evolved/nnt6.html) + + - [Simple Implementation](http://stackoverflow.com/questions/15395835/simple-multi-layer-neural-network-implementation) + + - [NN for Beginners](http://www.codeproject.com/Articles/16419/AI-Neural-Network-for-beginners-Part-of) + + - [Regression and Classification with NNs (Slides)](http://www.autonlab.org/tutorials/neural13.pdf) + + - [Another Intro](http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html) + + + +- Recurrent and LSTM Networks + - [awesome-rnn: list of resources (GitHub Repo)](https://github.com/kjw0612/awesome-rnn) + + - [Recurrent Neural Net Tutorial Part 1](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/), [Part 2](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-2-implementing-a-language-model-rnn-with-python-numpy-and-theano/), [Part 3](http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/), [Code](https://github.com/dennybritz/rnn-tutorial-rnnlm/) + + - [NLP RNN Representations](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/) + + - [The Unreasonable effectiveness of RNNs](http://karpathy.github.io/2015/05/21/rnn-effectiveness/), [Torch Code](https://github.com/karpathy/char-rnn), [Python Code](https://gist.github.com/karpathy/d4dee566867f8291f086) + + - [Intro to RNN](http://deeplearning4j.org/recurrentnetwork.html), [LSTM](http://deeplearning4j.org/lstm.html) + + - [An application of RNN](http://hackaday.com/2015/10/15/73-computer-scientists-created-a-neural-net-and-you-wont-believe-what-happened-next/) + + - [Optimizing RNN Performance](http://svail.github.io/) + + - [Simple RNN](http://outlace.com/Simple-Recurrent-Neural-Network/) + + - [Auto-Generating Clickbait with RNN](https://larseidnes.com/2015/10/13/auto-generating-clickbait-with-recurrent-neural-networks/) + + - [Sequence Learning using RNN (Slides)](http://www.slideshare.net/indicods/general-sequence-learning-with-recurrent-neural-networks-for-next-ml) + + - [Machine Translation using RNN (Paper)](http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf) + + - [Music generation using RNNs (Keras)](https://github.com/MattVitelli/GRUV) + + - [Using RNN to create on-the-fly dialogue (Keras)](http://neuralniche.com/post/tutorial/) + + - Long Short Term Memory (LSTM) + + - [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/) + + - [LSTM explained](https://apaszke.github.io/lstm-explained.html) + + - [Beginner’s Guide to LSTM](http://deeplearning4j.org/lstm.html) + + - [Implementing LSTM from scratch](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/), [Python/Theano code](https://github.com/dennybritz/rnn-tutorial-gru-lstm) + + - [Torch Code for character-level language models using LSTM](https://github.com/karpathy/char-rnn) + + - [LSTM for Kaggle EEG Detection competition (Torch Code)](https://github.com/apaszke/kaggle-grasp-and-lift) + + - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm) + + - [Deep Learning for Visual Q&A | LSTM | CNN](http://avisingh599.github.io/deeplearning/visual-qa/), [Code](https://github.com/avisingh599/visual-qa) + + - [Computer Responds to email using LSTM | Google](http://googleresearch.blogspot.in/2015/11/computer-respond-to-this-email.html) + + - [LSTM dramatically improves Google Voice Search](http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html), [Another Article](http://deeplearning.net/2015/09/30/long-short-term-memory-dramatically-improves-google-voice-etc-now-available-to-a-billion-users/) + + - [Understanding Natural Language with LSTM Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) + + - [Torch code for Visual Question Answering using a CNN+LSTM model](https://github.com/abhshkdz/neural-vqa) + + - [LSTM for Human Activity Recognition](https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition/) + + - Gated Recurrent Units (GRU) + + - [LSTM vs GRU](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/) + + - [Time series forecasting with Sequence-to-Sequence (seq2seq) rnn models](https://github.com/guillaume-chevalier/seq2seq-signal-prediction) + + + + +- [Recursive Neural Network (not Recurrent)](https://en.wikipedia.org/wiki/Recursive_neural_network) + + - [Recursive Neural Tensor Network (RNTN)](http://deeplearning4j.org/recursiveneuraltensornetwork.html) + + - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) + + + +- Restricted Boltzmann Machine + + - [Beginner's Guide about RBMs](http://deeplearning4j.org/restrictedboltzmannmachine.html) + + - [Another Good Tutorial](http://deeplearning.net/tutorial/rbm.html) + + - [Introduction to RBMs](http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/) + + - [Hinton's Guide to Training RBMs](https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf) + + - [RBMs in R](https://github.com/zachmayer/rbm) + + - [Deep Belief Networks Tutorial](http://deeplearning4j.org/deepbeliefnetwork.html) + + - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) + + + +- Autoencoders: Unsupervised (applies BackProp after setting target = input) + + - [Andrew Ng Sparse Autoencoders pdf](https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf) + + - [Deep Autoencoders Tutorial](http://deeplearning4j.org/deepautoencoder.html) + + - [Denoising Autoencoders](http://deeplearning.net/tutorial/dA.html), [Theano Code](http://deeplearning.net/tutorial/code/dA.py) + + - [Stacked Denoising Autoencoders](http://deeplearning.net/tutorial/SdA.html#sda) + + + + +- Convolutional Neural Networks + + - [An Intuitive Explanation of Convolutional Neural Networks](https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/) + + - [Awesome Deep Vision: List of Resources (GitHub)](https://github.com/kjw0612/awesome-deep-vision) + + - [Intro to CNNs](http://deeplearning4j.org/convolutionalnets.html) + + - [Understanding CNN for NLP](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/) + + - [Stanford Notes](http://vision.stanford.edu/teaching/cs231n/), [Codes](http://cs231n.github.io/), [GitHub](https://github.com/cs231n/cs231n.github.io) + + - [JavaScript Library (Browser Based) for CNNs](http://cs.stanford.edu/people/karpathy/convnetjs/) + + - [Using CNNs to detect facial keypoints](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/) + + - [Deep learning to classify business photos at Yelp](http://engineeringblog.yelp.com/2015/10/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html) + + - [Interview with Yann LeCun | Kaggle](http://blog.kaggle.com/2014/12/22/convolutional-nets-and-cifar-10-an-interview-with-yan-lecun/) + + - [Visualising and Understanding CNNs](https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf) + + + +- Network Representation Learning + + - [Awesome Graph Embedding](https://github.com/benedekrozemberczki/awesome-graph-embedding) + + - [Awesome Network Embedding](https://github.com/chihming/awesome-network-embedding) + + - [Network Representation Learning Papers](https://github.com/thunlp) + + - [Knowledge Representation Learning Papers](https://github.com/thunlp/KRLPapers) + + - [Graph Based Deep Learning Literature](https://github.com/naganandy/graph-based-deep-learning-literature) + + + +## Natural Language Processing + +- [A curated list of speech and natural language processing resources](https://github.com/edobashira/speech-language-processing) + +- [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/) + +- [tf-idf explained](http://michaelerasm.us/post/tf-idf-in-10-minutes/) + +- [Interesting Deep Learning NLP Projects Stanford](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/) + +- [The Stanford NLP Group](https://nlp.stanford.edu/) + +- [NLP from Scratch | Google Paper](https://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/35671.pdf) + +- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf) + +- [Bag of Words](https://en.wikipedia.org/wiki/Bag-of-words_model) + + - [Classification text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/) + + + +- Topic Modeling + - [Topic Modeling Wikipedia](https://en.wikipedia.org/wiki/Topic_model) + - [**Probabilistic Topic Models Princeton PDF**](http://www.cs.columbia.edu/~blei/papers/Blei2012.pdf) + + - [LDA Wikipedia](https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation), [LSA Wikipedia](https://en.wikipedia.org/wiki/Latent_semantic_analysis), [Probabilistic LSA Wikipedia](https://en.wikipedia.org/wiki/Probabilistic_latent_semantic_analysis) + + - [What is a good explanation of Latent Dirichlet Allocation (LDA)?](https://www.quora.com/What-is-a-good-explanation-of-Latent-Dirichlet-Allocation) + + - [**Introduction to LDA**](http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/), [Another good explanation](http://confusedlanguagetech.blogspot.in/2012/07/jordan-boyd-graber-and-philip-resnik.html) + + - [The LDA Buffet - Intuitive Explanation](http://www.matthewjockers.net/2011/09/29/the-lda-buffet-is-now-open-or-latent-dirichlet-allocation-for-english-majors/) + + - [Your Guide to Latent Dirichlet Allocation (LDA)](https://medium.com/@lettier/how-does-lda-work-ill-explain-using-emoji-108abf40fa7d) + + - [Difference between LSI and LDA](https://www.quora.com/Whats-the-difference-between-Latent-Semantic-Indexing-LSI-and-Latent-Dirichlet-Allocation-LDA) + + - [Original LDA Paper](https://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf) + + - [alpha and beta in LDA](http://datascience.stackexchange.com/questions/199/what-does-the-alpha-and-beta-hyperparameters-contribute-to-in-latent-dirichlet-a) + + - [Intuitive explanation of the Dirichlet distribution](https://www.quora.com/What-is-an-intuitive-explanation-of-the-Dirichlet-distribution) + - [topicmodels: An R Package for Fitting Topic Models](https://cran.r-project.org/web/packages/topicmodels/vignettes/topicmodels.pdf) + + - [Topic modeling made just simple enough](https://tedunderwood.com/2012/04/07/topic-modeling-made-just-simple-enough/) + + - [Online LDA](http://alexminnaar.com/online-latent-dirichlet-allocation-the-best-option-for-topic-modeling-with-large-data-sets.html), [Online LDA with Spark](http://alexminnaar.com/distributed-online-latent-dirichlet-allocation-with-apache-spark.html) + + - [LDA in Scala](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-i-the-theory.html), [Part 2](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-ii-the-code.html) + + - [Segmentation of Twitter Timelines via Topic Modeling](https://alexisperrier.com/nlp/2015/09/16/segmentation_twitter_timelines_lda_vs_lsa.html) + + - [Topic Modeling of Twitter Followers](http://alexperrier.github.io/jekyll/update/2015/09/04/topic-modeling-of-twitter-followers.html) + + - [Multilingual Latent Dirichlet Allocation (LDA)](https://github.com/ArtificiAI/Multilingual-Latent-Dirichlet-Allocation-LDA). ([Tutorial here](https://github.com/ArtificiAI/Multilingual-Latent-Dirichlet-Allocation-LDA/blob/master/Multilingual-LDA-Pipeline-Tutorial.ipynb)) + + - [Deep Belief Nets for Topic Modeling](https://github.com/larsmaaloee/deep-belief-nets-for-topic-modeling) + - [Gaussian LDA for Topic Models with Word Embeddings](http://www.cs.cmu.edu/~rajarshd/papers/acl2015.pdf) + - Python + - [Series of lecture notes for probabilistic topic models written in ipython notebook](https://github.com/arongdari/topic-model-lecture-note) + - [Implementation of various topic models in Python](https://github.com/arongdari/python-topic-model) + + + +- word2vec + + - [Google word2vec](https://code.google.com/archive/p/word2vec) + + - [Bag of Words Model Wiki](https://en.wikipedia.org/wiki/Bag-of-words_model) + + - [word2vec Tutorial](https://rare-technologies.com/word2vec-tutorial/) + + - [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf) + + - [Skip Gram Model Tutorial](http://alexminnaar.com/word2vec-tutorial-part-i-the-skip-gram-model.html), [CBoW Model](http://alexminnaar.com/word2vec-tutorial-part-ii-the-continuous-bag-of-words-model.html) + + - [Word Vectors Kaggle Tutorial Python](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors) + + - [Making sense of word2vec](http://rare-technologies.com/making-sense-of-word2vec/) + + - [word2vec explained on deeplearning4j](http://deeplearning4j.org/word2vec.html) + + - [Quora word2vec](https://www.quora.com/How-does-word2vec-work) + + - [Other Quora Resources](https://www.quora.com/What-are-the-continuous-bag-of-words-and-skip-gram-architectures-in-laymans-terms), [2](https://www.quora.com/What-is-the-difference-between-the-Bag-of-Words-model-and-the-Continuous-Bag-of-Words-model), [3](https://www.quora.com/Is-skip-gram-negative-sampling-better-than-CBOW-NS-for-word2vec-If-so-why) + + - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html) + +- Text Clustering + + - [How string clustering works](http://stackoverflow.com/questions/8196371/how-clustering-works-especially-string-clustering) + + - [Levenshtein distance for measuring the difference between two sequences](https://en.wikipedia.org/wiki/Levenshtein_distance) + + - [Text clustering with Levenshtein distances](http://stackoverflow.com/questions/21511801/text-clustering-with-levenshtein-distances) + +- Text Classification + + - [Classification Text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/) + +- Named Entity Recognitation + + - [Stanford Named Entity Recognizer (NER)](https://nlp.stanford.edu/software/CRF-NER.shtml) + + - [Named Entity Recognition: Applications and Use Cases- Towards Data Science](https://towardsdatascience.com/named-entity-recognition-applications-and-use-cases-acdbf57d595e) + +- [Language learning with NLP and reinforcement learning](http://blog.dennybritz.com/2015/09/11/reimagining-language-learning-with-nlp-and-reinforcement-learning/) + +- [Kaggle Tutorial Bag of Words and Word vectors](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 3](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors) + +- [What would Shakespeare say (NLP Tutorial)](https://gigadom.wordpress.com/2015/10/02/natural-language-processing-what-would-shakespeare-say/) + +- [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf) + + + +## Computer Vision +- [Awesome computer vision (github)](https://github.com/jbhuang0604/awesome-computer-vision) + +- [Awesome deep vision (github)](https://github.com/kjw0612/awesome-deep-vision) + + + + +## Support Vector Machine + +- [Highest Voted Questions about SVMs on Cross Validated](http://stats.stackexchange.com/questions/tagged/svm) + +- [Help me Understand SVMs!](http://stats.stackexchange.com/questions/3947/help-me-understand-support-vector-machines) + +- [SVM in Layman's terms](https://www.quora.com/What-does-support-vector-machine-SVM-mean-in-laymans-terms) + +- [How does SVM Work | Comparisons](http://stats.stackexchange.com/questions/23391/how-does-a-support-vector-machine-svm-work) + +- [A tutorial on SVMs](http://alex.smola.org/papers/2003/SmoSch03b.pdf) + +- [Practical Guide to SVC](http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf), [Slides](http://www.csie.ntu.edu.tw/~cjlin/talks/freiburg.pdf) + +- [Introductory Overview of SVMs](http://www.statsoft.com/Textbook/Support-Vector-Machines) + +- Comparisons + + - [SVMs > ANNs](http://stackoverflow.com/questions/6699222/support-vector-machines-better-than-artificial-neural-networks-in-which-learn?rq=1), [ANNs > SVMs](http://stackoverflow.com/questions/11632516/what-are-advantages-of-artificial-neural-networks-over-support-vector-machines), [Another Comparison](http://www.svms.org/anns.html) + + - [Trees > SVMs](http://stats.stackexchange.com/questions/57438/why-is-svm-not-so-good-as-decision-tree-on-the-same-data) + + - [Kernel Logistic Regression vs SVM](http://stats.stackexchange.com/questions/43996/kernel-logistic-regression-vs-svm) + + - [Logistic Regression vs SVM](http://stats.stackexchange.com/questions/58684/regularized-logistic-regression-and-support-vector-machine), [2](http://stats.stackexchange.com/questions/95340/svm-v-s-logistic-regression), [3](https://www.quora.com/Support-Vector-Machines/What-is-the-difference-between-Linear-SVMs-and-Logistic-Regression) + +- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf) + +- [Variable Importance from SVM](http://stats.stackexchange.com/questions/2179/variable-importance-from-svm) + +- Software + + - [LIBSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvm/) + + - [Intro to SVM in R](http://cbio.ensmp.fr/~jvert/svn/tutorials/practical/svmbasic/svmbasic_notes.pdf) + +- Kernels + - [What are Kernels in ML and SVM?](https://www.quora.com/What-are-Kernels-in-Machine-Learning-and-SVM) + + - [Intuition Behind Gaussian Kernel in SVMs?](https://www.quora.com/Support-Vector-Machines/What-is-the-intuition-behind-Gaussian-kernel-in-SVM) + +- Probabilities post SVM + + - [Platt's Probabilistic Outputs for SVM](http://www.csie.ntu.edu.tw/~htlin/paper/doc/plattprob.pdf) + + - [Platt Calibration Wiki](https://en.wikipedia.org/wiki/Platt_scaling) + + - [Why use Platts Scaling](http://stats.stackexchange.com/questions/5196/why-use-platts-scaling) + + - [Classifier Classification with Platt's Scaling](http://fastml.com/classifier-calibration-with-platts-scaling-and-isotonic-regression/) + + + + +## Reinforcement Learning + +- [Awesome Reinforcement Learning (GitHub)](https://github.com/aikorea/awesome-rl) + +- [RL Tutorial Part 1](http://outlace.com/Reinforcement-Learning-Part-1/), [Part 2](http://outlace.com/Reinforcement-Learning-Part-2/) + + + +## Decision Trees + +- [Wikipedia Page - Lots of Good Info](https://en.wikipedia.org/wiki/Decision_tree_learning) + +- [FAQs about Decision Trees](http://stats.stackexchange.com/questions/tagged/cart) + +- [Brief Tour of Trees and Forests](https://statistical-research.com/index.php/2013/04/29/a-brief-tour-of-the-trees-and-forests/) + +- [Tree Based Models in R](http://www.statmethods.net/advstats/cart.html) + +- [How Decision Trees work?](http://www.aihorizon.com/essays/generalai/decision_trees.htm) + +- [Weak side of Decision Trees](http://stats.stackexchange.com/questions/1292/what-is-the-weak-side-of-decision-trees) + +- [Thorough Explanation and different algorithms](http://www.ise.bgu.ac.il/faculty/liorr/hbchap9.pdf) + +- [What is entropy and information gain in the context of building decision trees?](http://stackoverflow.com/questions/1859554/what-is-entropy-and-information-gain) + +- [Slides Related to Decision Trees](http://www.slideshare.net/pierluca.lanzi/machine-learning-and-data-mining-11-decision-trees) + +- [How do decision tree learning algorithms deal with missing values?](http://stats.stackexchange.com/questions/96025/how-do-decision-tree-learning-algorithms-deal-with-missing-values-under-the-hoo) + +- [Using Surrogates to Improve Datasets with Missing Values](https://www.salford-systems.com/videos/tutorials/tips-and-tricks/using-surrogates-to-improve-datasets-with-missing-values) + +- [Good Article](https://www.mindtools.com/dectree.html) + +- [Are decision trees almost always binary trees?](http://stats.stackexchange.com/questions/12187/are-decision-trees-almost-always-binary-trees) + +- [Pruning Decision Trees](https://en.wikipedia.org/wiki/Pruning_(decision_trees)), [Grafting of Decision Trees](https://en.wikipedia.org/wiki/Grafting_(decision_trees)) + +- [What is Deviance in context of Decision Trees?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart) + +- [Discover structure behind data with decision trees](http://vooban.com/en/tips-articles-geek-stuff/discover-structure-behind-data-with-decision-trees/) - Grow and plot a decision tree to automatically figure out hidden rules in your data + +- Comparison of Different Algorithms + + - [CART vs CTREE](http://stats.stackexchange.com/questions/12140/conditional-inference-trees-vs-traditional-decision-trees) + + - [Comparison of complexity or performance](https://stackoverflow.com/questions/9979461/different-decision-tree-algorithms-with-comparison-of-complexity-or-performance) + + - [CHAID vs CART](http://stats.stackexchange.com/questions/61230/chaid-vs-crt-or-cart) , [CART vs CHAID](http://www.bzst.com/2006/10/classification-trees-cart-vs-chaid.html) + + - [Good Article on comparison](http://www.ftpress.com/articles/article.aspx?p=2248639&seqNum=11) + +- CART + + - [Recursive Partitioning Wikipedia](https://en.wikipedia.org/wiki/Recursive_partitioning) + + - [CART Explained](http://documents.software.dell.com/Statistics/Textbook/Classification-and-Regression-Trees) + + - [How to measure/rank “variable importance” when using CART?](http://stats.stackexchange.com/questions/6478/how-to-measure-rank-variable-importance-when-using-cart-specifically-using) + + - [Pruning a Tree in R](http://stackoverflow.com/questions/15318409/how-to-prune-a-tree-in-r) + + - [Does rpart use multivariate splits by default?](http://stats.stackexchange.com/questions/4356/does-rpart-use-multivariate-splits-by-default) + + - [FAQs about Recursive Partitioning](http://stats.stackexchange.com/questions/tagged/rpart) + +- CTREE + + - [party package in R](https://cran.r-project.org/web/packages/party/party.pdf) + + - [Show volumne in each node using ctree in R](http://stackoverflow.com/questions/13772715/show-volume-in-each-node-using-ctree-plot-in-r) + + - [How to extract tree structure from ctree function?](http://stackoverflow.com/questions/8675664/how-to-extract-tree-structure-from-ctree-function) + +- CHAID + + - [Wikipedia Artice on CHAID](https://en.wikipedia.org/wiki/CHAID) + + - [Basic Introduction to CHAID](https://smartdrill.com/Introduction-to-CHAID.html) + + - [Good Tutorial on CHAID](http://www.statsoft.com/Textbook/CHAID-Analysis) + +- MARS + + - [Wikipedia Article on MARS](https://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines) + +- Probabilistic Decision Trees + + - [Bayesian Learning in Probabilistic Decision Trees](http://www.stats.org.uk/bayesian/Jordan.pdf) + + - [Probabilistic Trees Research Paper](http://people.stern.nyu.edu/adamodar/pdfiles/papers/probabilistic.pdf) + + + +## Random Forest / Bagging + +- [Awesome Random Forest (GitHub)**](https://github.com/kjw0612/awesome-random-forest) + +- [How to tune RF parameters in practice?](https://www.kaggle.com/forums/f/15/kaggle-forum/t/4092/how-to-tune-rf-parameters-in-practice) + +- [Measures of variable importance in random forests](http://stats.stackexchange.com/questions/12605/measures-of-variable-importance-in-random-forests) + +- [Compare R-squared from two different Random Forest models](http://stats.stackexchange.com/questions/13869/compare-r-squared-from-two-different-random-forest-models) + +- [OOB Estimate Explained | RF vs LDA](https://stat.ethz.ch/education/semesters/ss2012/ams/slides/v10.2.pdf) + +- [Evaluating Random Forests for Survival Analysis Using Prediction Error Curve](https://www.jstatsoft.org/index.php/jss/article/view/v050i11) + +- [Why doesn't Random Forest handle missing values in predictors?](http://stats.stackexchange.com/questions/98953/why-doesnt-random-forest-handle-missing-values-in-predictors) + +- [How to build random forests in R with missing (NA) values?](http://stackoverflow.com/questions/8370455/how-to-build-random-forests-in-r-with-missing-na-values) + +- [FAQs about Random Forest](http://stats.stackexchange.com/questions/tagged/random-forest), [More FAQs](http://stackoverflow.com/questions/tagged/random-forest) + +- [Obtaining knowledge from a random forest](http://stats.stackexchange.com/questions/21152/obtaining-knowledge-from-a-random-forest) + +- [Some Questions for R implementation](http://stackoverflow.com/questions/20537186/getting-predictions-after-rfimpute), [2](http://stats.stackexchange.com/questions/81609/whether-preprocessing-is-needed-before-prediction-using-finalmodel-of-randomfore), [3](http://stackoverflow.com/questions/17059432/random-forest-package-in-r-shows-error-during-prediction-if-there-are-new-fact) + + + +## Boosting + +- [Boosting for Better Predictions](http://www.datasciencecentral.com/profiles/blogs/boosting-algorithms-for-better-predictions) + +- [Boosting Wikipedia Page](https://en.wikipedia.org/wiki/Boosting_(machine_learning)) + +- [Introduction to Boosted Trees | Tianqi Chen](https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf) + +- Gradient Boosting Machine + + - [Gradiet Boosting Wiki](https://en.wikipedia.org/wiki/Gradient_boosting) + + - [Guidelines for GBM parameters in R](http://stats.stackexchange.com/questions/25748/what-are-some-useful-guidelines-for-gbm-parameters), [Strategy to set parameters](http://stats.stackexchange.com/questions/35984/strategy-to-set-the-gbm-parameters) + + - [Meaning of Interaction Depth](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm), [2](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm) + + - [Role of n.minobsinnode parameter of GBM in R](http://stats.stackexchange.com/questions/30645/role-of-n-minobsinnode-parameter-of-gbm-in-r) + + - [GBM in R](http://www.slideshare.net/mark_landry/gbm-package-in-r) + + - [FAQs about GBM](http://stats.stackexchange.com/tags/gbm/hot) + + - [GBM vs xgboost](https://www.kaggle.com/c/higgs-boson/forums/t/9497/r-s-gbm-vs-python-s-xgboost) + +- xgboost + + - [xgboost tuning kaggle](https://www.kaggle.com/khozzy/rossmann-store-sales/xgboost-parameter-tuning-template/log) + + - [xgboost vs gbm](https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/13012/question-to-experienced-kagglers-and-anyone-who-wants-to-take-a-shot/68296#post68296) + + - [xgboost survey](https://www.kaggle.com/c/higgs-boson/forums/t/10335/xgboost-post-competition-survey) + + - [Practical XGBoost in Python online course (free)](http://education.parrotprediction.teachable.com/courses/practical-xgboost-in-python) + +- AdaBoost + + - [AdaBoost Wiki](https://en.wikipedia.org/wiki/AdaBoost), [Python Code](https://gist.github.com/tristanwietsma/5486024) + + - [AdaBoost Sparse Input Support](http://hamzehal.blogspot.com/2014/06/adaboost-sparse-input-support.html) + + - [adaBag R package](https://cran.r-project.org/web/packages/adabag/adabag.pdf) + + - [Tutorial](http://math.mit.edu/~rothvoss/18.304.3PM/Presentations/1-Eric-Boosting304FinalRpdf.pdf) + +- CatBoost + + - [CatBoost Documentation](https://catboost.ai/docs/) + + - [Benchmarks](https://catboost.ai/#benchmark) + + - [Tutorial](https://github.com/catboost/tutorials) + + - [GitHub Project](https://github.com/catboost) + + - [CatBoost vs. Light GBM vs. XGBoost](https://towardsdatascience.com/catboost-vs-light-gbm-vs-xgboost-5f93620723db) + + + +## Ensembles + +- [Wikipedia Article on Ensemble Learning](https://en.wikipedia.org/wiki/Ensemble_learning) + +- [Kaggle Ensembling Guide](http://mlwave.com/kaggle-ensembling-guide/) + +- [The Power of Simple Ensembles](http://www.overkillanalytics.net/more-is-always-better-the-power-of-simple-ensembles/) + +- [Ensemble Learning Intro](http://machine-learning.martinsewell.com/ensembles/) + +- [Ensemble Learning Paper](http://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/springerEBR09.pdf) + +- [Ensembling models with R](http://amunategui.github.io/blending-models/), [Ensembling Regression Models in R](http://stats.stackexchange.com/questions/26790/ensembling-regression-models), [Intro to Ensembles in R](http://www.vikparuchuri.com/blog/intro-to-ensemble-learning-in-r/) + +- [Ensembling Models with caret](http://stats.stackexchange.com/questions/27361/stacking-ensembling-models-with-caret) + +- [Bagging vs Boosting vs Stacking](http://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning) + +- [Good Resources | Kaggle Africa Soil Property Prediction](https://www.kaggle.com/c/afsis-soil-properties/forums/t/10391/best-ensemble-references) + +- [Boosting vs Bagging](http://www.chioka.in/which-is-better-boosting-or-bagging/) + +- [Resources for learning how to implement ensemble methods](http://stats.stackexchange.com/questions/32703/resources-for-learning-how-to-implement-ensemble-methods) + +- [How are classifications merged in an ensemble classifier?](http://stats.stackexchange.com/questions/21502/how-are-classifications-merged-in-an-ensemble-classifier) + + + +## Stacking Models + +- [Stacking, Blending and Stacked Generalization](http://www.chioka.in/stacking-blending-and-stacked-generalization/) + +- [Stacked Generalization (Stacking)](http://machine-learning.martinsewell.com/ensembles/stacking/) + +- [Stacked Generalization: when does it work?](http://www.ijcai.org/Proceedings/97-2/011.pdf) + +- [Stacked Generalization Paper](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.56.1533&rep=rep1&type=pdf) + + + +## Vapnik–Chervonenkis Dimension + +- [Wikipedia article on VC Dimension](https://en.wikipedia.org/wiki/VC_dimension) + +- [Intuitive Explanantion of VC Dimension](https://www.quora.com/Explain-VC-dimension-and-shattering-in-lucid-Way) + +- [Video explaining VC Dimension](https://www.youtube.com/watch?v=puDzy2XmR5c) + +- [Introduction to VC Dimension](http://www.svms.org/vc-dimension/) + +- [FAQs about VC Dimension](http://stats.stackexchange.com/questions/tagged/vc-dimension) + +- [Do ensemble techniques increase VC-dimension?](http://stats.stackexchange.com/questions/78076/do-ensemble-techniques-increase-vc-dimension) + + + + +## Bayesian Machine Learning + +- [Bayesian Methods for Hackers (using pyMC)](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers) + +- [Should all Machine Learning be Bayesian?](http://videolectures.net/bark08_ghahramani_samlbb/) + +- [Tutorial on Bayesian Optimisation for Machine Learning](http://www.iro.umontreal.ca/~bengioy/cifar/NCAP2014-summerschool/slides/Ryan_adams_140814_bayesopt_ncap.pdf) + +- [Bayesian Reasoning and Deep Learning](http://blog.shakirm.com/2015/10/bayesian-reasoning-and-deep-learning/), [Slides](http://blog.shakirm.com/wp-content/uploads/2015/10/Bayes_Deep.pdf) + +- [Bayesian Statistics Made Simple](http://greenteapress.com/wp/think-bayes/) + +- [Kalman & Bayesian Filters in Python](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python) + +- [Markov Chain Wikipedia Page](https://en.wikipedia.org/wiki/Markov_chain) + + + + +## Semi Supervised Learning + +- [Wikipedia article on Semi Supervised Learning](https://en.wikipedia.org/wiki/Semi-supervised_learning) + +- [Tutorial on Semi Supervised Learning](http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf) + +- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf) + +- [Taxonomy](http://is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/taxo_[0].pdf) + +- [Video Tutorial Weka](https://www.youtube.com/watch?v=sWxcIjZFGNM) + +- [Unsupervised, Supervised and Semi Supervised learning](http://stats.stackexchange.com/questions/517/unsupervised-supervised-and-semi-supervised-learning) + +- [Research Papers 1](http://mlg.eng.cam.ac.uk/zoubin/papers/zglactive.pdf), [2](http://mlg.eng.cam.ac.uk/zoubin/papers/zgl.pdf), [3](http://icml.cc/2012/papers/616.pdf) + + + + +## Optimization + +- [Mean Variance Portfolio Optimization with R and Quadratic Programming](http://www.wdiam.com/2012/06/10/mean-variance-portfolio-optimization-with-r-and-quadratic-programming/?utm_content=buffer04c12&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer) + +- [Algorithms for Sparse Optimization and Machine Learning](http://www.ima.umn.edu/2011-2012/W3.26-30.12/activities/Wright-Steve/sjw-ima12) + +- [Optimization Algorithms in Machine Learning](http://pages.cs.wisc.edu/~swright/nips2010/sjw-nips10.pdf), [Video Lecture](http://videolectures.net/nips2010_wright_oaml/) + +- [Optimization Algorithms for Data Analysis](http://www.birs.ca/workshops/2011/11w2035/files/Wright.pdf) + +- [Video Lectures on Optimization](http://videolectures.net/stephen_j_wright/) + +- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf) + +- [The Interplay of Optimization and Machine Learning Research](http://jmlr.org/papers/volume7/MLOPT-intro06a/MLOPT-intro06a.pdf) + +- [Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters](http://vooban.com/en/tips-articles-geek-stuff/hyperopt-tutorial-for-optimizing-neural-networks-hyperparameters/) + + + + +## Other Tutorials + +- For a collection of Data Science Tutorials using R, please refer to [this list](https://github.com/ujjwalkarn/DataScienceR). + +- For a collection of Data Science Tutorials using Python, please refer to [this list](https://github.com/ujjwalkarn/DataSciencePython). diff --git a/contributing.md b/contributing.md index 134caf1..8711389 100644 --- a/contributing.md +++ b/contributing.md @@ -5,6 +5,7 @@ If you want to contribute to this list (please do), send me a pull request. Sinc Please ensure your pull request adheres to the following guidelines: - **Please make an individual pull request for each suggestion.** +- Please only submit resources that are completly free to access. - The pull request and commit should have a useful title. - Please search previous suggestions before making a new one, as yours may be a duplicate. - Make sure your link has a useful and relevant title.