Skip to content

Commit 7836e23

Browse files
sdgilleysdgilley
authored
Gh pages dev (#1)
* initial checkin * updates * update * update * updates * updates * add _site * remove * add images * updates * updates * updates * update
1 parent 561c809 commit 7836e23

29 files changed

+4423
-0
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
*.userprefs
1414

1515
# Build results
16+
_site/
1617
[Dd]ebug/
1718
[Dd]ebugPublic/
1819
[Rr]elease/

‎Powershell_Instructions.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
---
2+
layout: default
3+
title: PowerShell Instructions
4+
---
5+
6+
7+
## PowerShell Instructions
8+
---------------------------
9+
10+
<div class="row">
11+
<div class="col-md-6">
12+
<div class="toc">
13+
<li> <a href="#setup">Setup</a></li>
14+
<li> <a href="#execute-powershell-script">Execute PowerShell Script</a></li>
15+
<li> <a href="#score-production-data">Score Production Data</a></li>
16+
<li> <a href="#review-data">Review Data</a></li>
17+
<li> <a href="#visualizing-results">Visualizing Results</a> </li>
18+
<li> <a href="#other-steps">Other Steps</a></li>
19+
</div>
20+
</div>
21+
<div class="col-md-6">
22+
If you have deployed a VM through the
23+
<a href="{{ site.aka_url }}">Azure AI Gallery</a>, all the steps below have already been performed and your database on that machine has all the resulting tables and stored procedures. You can explore this solution in more detail by examining the folders and running Python or stored procedures to re-create the model, or skip to trying out the model in the included [Jupyter notebook](jupyter.html).
24+
</div>
25+
</div>
26+
27+
If you are configuring your own server, continue with the steps below to run the PowerShell script.
28+
29+
## Setup
30+
-----------
31+
32+
First, make sure you have set up your SQL Server by <a href="SetupSQL.html">following these instructions</a>. Then proceed with the steps below to run the solution template using the automated PowerShell file.
33+
34+
## Execute PowerShell Script
35+
----------------------------
36+
37+
Running this PowerShell script will create the data tables and stored procedures for the the operationalization of this solution in R in the `{{ site.db_name }}` database. It will also execute these procedures to create full database with results of the steps – dataset creation, modeling, and scoring as described [here](dba.html).
38+
39+
40+
1. Log onto the machine that contains the SQL Server you wish to use.
41+
42+
1. Install [Git](https://gitforwindows.org/) if it is not already present. During the install, check the box to add LFS support.
43+
44+
2. Download <a href="https://raw.githubusercontent.com/Microsoft/ml-server-text-classification/dev/Resources/ActionScripts/TextClassificationSetup.ps1" download>TextClassificationSetup.ps1</a> to your computer.
45+
46+
3. Open a command or PowerShell window as Administrator.
47+
48+
4. CD to the directory where you downloaded the above .ps1 file and execute the command:
49+
50+
.\TextClassificationSetup.ps1
51+
52+
5. Answer the prompts if any.
53+
54+
This will make the following modification to your SQL Server:
55+
56+
* Creates the SLQRUserGroup for running R and Python code.
57+
* Reconfigures SQL Server to allow running of external scripts.
58+
* Installs the latest SQL Server 2017 Cumulative Update if no updates have been installed (this solution requires at least CU1 to run successfully).
59+
* Clones the solution code and data into the c:\Solutions\{{ site.folder_name }} directory.
60+
* Creates the solution databases `{{ site.db_name }}_R` and `{{ site.db_name }}_Py`
61+
* Runs the solution workflow to populate all database tables.
62+
63+
<div class="alert alert info">
64+
If you wish to run the solution code on a different computer than SQL Server machine, see <a href="local.html">Setup for Local Code Execution</a>.
65+
</div>
66+
67+
## Review Data
68+
--------------
69+
70+
Once the PowerShell script has completed successfully, log into the SQL Server Management Studio to view all the datasets that have been created in the `{{ site.db_name }}_R` or `{{ site.db_name }}_Py` databases.
71+
Hit `Refresh` if necessary.
72+
73+
[Click here](tables.html) to view the details all tables created in this solution.
74+

‎SetupSQL.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
layout: default
3+
title: "On-Prem: Setup SQL Server 2017"
4+
---
5+
6+
## On-Prem: Setup SQL Server
7+
--------------------------
8+
9+
<div class="row">
10+
<div class="col-md-6">
11+
<div class="toc">
12+
<li><a href="#prepare-your-sql-server-installation">Prepare your SQL Server Installation</a></li>
13+
<li><a href="#ready-to-run-code">Ready to Run Code</a></li>
14+
</div>
15+
</div>
16+
<div class="col-md-6">
17+
The instructions on this page will help you to add this solution to your on premises SQL Server 2016 or higher.
18+
<p>
19+
If you instead would like to try this solution out on a virtual machine, visit the <a href="{{ site.aka_url }}">Azure AI Gallery</a> and use the Deploy button. All the configuration described below will be done for you, as well as the initial deployment of the solution. </p>
20+
</div>
21+
</div>
22+
23+
## Prepare your SQL Server Installation
24+
-------------------------------------------
25+
26+
The rest of this page assumes you are configuring your on premises SQL Server 2016 or 2017 for this solution.
27+
28+
If you need a trial version of SQL Server see [What's New in SQL Server 2017](https://docs.microsoft.com/en-us/sql/sql-server/what-s-new-in-sql-server-2017)for download or VM options.
29+
30+
Complete the steps in the Set up Microsoft Machine Learning Services (In-Database) Instructions. The set up instructions file can found at <a href="https://msdn.microsoft.com/en-us/library/mt696069.aspx" target="_blank"> https://msdn.microsoft.com/en-us/library/mt696069.aspx</a>
31+
32+
Make sure to install both SQL and Standalone version of R and Python.
33+
34+
35+
### Ready to Run Code
36+
---------------------
37+
38+
You are now ready to run the code for this solution.
39+
40+
* Install the solution by following these <a href="Powershell_Instructions.html">PowerShell Instructions</a> for deployment.
41+
42+
* Typically a data scientist will create and test a predictive model from their favorite R IDE, at which point the models will be stored in SQL Server and then scored in production using Transact-SQL (T-SQL) stored procedures.
43+
You can follow along with this by following the steps in [For the Data Scientist](data-scientist.html) or [For the Database Analyst](dba.html).
44+
45+
46+

‎_config.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
author:
2+
name: Microsoft
3+
4+
description: Text Classificaion using SQL Server 2017 + ML Services with R or Python
5+
# names
6+
solution_name: TextClassification
7+
folder_name: TextClassification
8+
db_name: TextClassification
9+
ps1_name: TextClassificationSetup.ps1
10+
pbix_name: TextClassification.pbix
11+
12+
13+
# urls
14+
code_url: "https://github.com/Microsoft/ml-server-text-classification"
15+
website_url: "https://microsoft.github.io/ml-server-text-classification/"
16+
aka_url: "http://aka.ms/text-classification"
17+

‎_includes/finalsteps.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
<a name="step4"></a>
2+
3+
### Step 4: Visualize the Results
4+
-------------------------
5+
The Power BI file **TextClassification.pbix** included with this solution can be used to visualize how the model performs.
6+
7+
On the `Training Summary` tab, you can see the predicted and actual labels for the Test data, along with the Micro Average Accuracy and Macro Average Accuracy values.
8+
9+
<img src="images/pbi1.png" />
10+
11+
On the `Scoring New Text` tab you can view the predicted labels for new text.
12+
13+
<img src="images/pbi2.png" />
14+
15+
## Template Contents
16+
---------------------
17+
18+
[View the contents of this solution template](contents.html).
19+
20+
21+
To try this out yourself:
22+
23+
* View the [Quick Start](quick.html).
24+
25+
[&lt; Home](index.html)

‎_includes/inputdata.html‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
This solution uses a preprocessed version of the [NewsGroups20](http://scikit-learn.org/stable/datasets/twenty_newsgroups.html), containing a Subject, a Text, and a Label. It has a similar structure to a support ticket data set which would also have two data fields: Title, and Problem description. Thus, you can easily change this solution to use your support ticket data simply by providing the same structure for the test and train datasets.

‎_includes/pysetup.html‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
<p>Debra would work on her own machine, using <a href="https://docs.microsoft.com/en-us/sql/advanced-analytics/python/sql-server-python-services">Machine Learning Services with Python</a> to execute these Python scripts. In case you want to run the code from the VM, ML Services Python has already been installed.</p>
2+
3+
<p>The Python code is present in the <strong>{{site.folder_name}}/Python</strong> directory. </p>
4+
5+
<p>OPTIONAL: You can execute the Python code on your local computer if you wish, but you must first <a href="local.html">prepare both the VM and your computer</a>. </p>
6+
7+
Follow these instructions to <a href="jupyter.html">view and execute the Python code with the Jupyter Notebook</a>.
8+
9+
You can also execute the Python code with an IDE. Both PyCharms and Visual Studio are installed on your VM. For each, you must first configure the Python interpreter to use <code>C:\Program Files\Microsoft\ML Server\PYTHON_SERVER\python.exe</code>.

‎_includes/requirements.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Running the R or Python scripts requires the following:
2+
3+
* SQL server 2017 with RevoscalePy (version 9.2) and MicrosoftML (version 1.5.0) installed and configured;
4+
* SQL Database which the user has write permission and execute stored procedures;
5+
* For more information about Machine Learning Services in SQL Server, please visit: [https://msdn.microsoft.com/en-us/library/mt604847.aspx](https://msdn.microsoft.com/en-us/library/mt604847.aspx)

‎_layouts/default.html‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<link rel="shortcut icon" type="image/png" href="images/favicon.png">
6+
<title>{{ page.title }}</title>
7+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
8+
<meta name="description" content="{{ site.description }}">
9+
<meta name="author" content="{{ site.author.name }}">
10+
11+
<!-- Le styles -->
12+
<link href="stylesheets/bootstrap.min.css" rel="stylesheet"/>
13+
14+
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
15+
<!--[if lt IE 9]>
16+
<script src="/js/html5shiv.js"></script>
17+
<![endif]-->
18+
<link rel="stylesheet" href="stylesheets/mystyles.css">
19+
20+
</head>
21+
<body>
22+
<div class="container">
23+
<div class="row">
24+
<div class="col-sm-9">
25+
<div class="jumboton teal">
26+
<h1>{{ site.solution_name }}</h1>
27+
<p>Implemented with Microsoft Machine Learning Services</p>
28+
</div>
29+
<div class="content">
30+
{{ content }}
31+
</div>
32+
</div><!--/col -->
33+
34+
<div class="col-sm-3 sidebar-offcanvas" id="sidebar">
35+
<img src="images/TextAnalysis.png">
36+
<div class="list-group">
37+
<a href="index.html" class="list-group-item">Home</a>
38+
<a href="data-scientist.html" class="list-group-item">For the Data Scientist</a>
39+
<a href="dba.html" class="list-group-item">For the Database Analyst</a>
40+
<a href="quick.html" class="list-group-item">Quick Start</a>
41+
</div>
42+
<hr />
43+
<div class="center">
44+
<a class="btn btn-large btn-info" href="{{ site.code_url }}">
45+
View On <strong>GitHub</strong></a>
46+
</div>
47+
<hr />
48+
<p class="details">Other Links</p>
49+
<div class="toc">
50+
<li><a href="contents.html">Packet Contents</a></li>
51+
<li><a href="sitemap.html">Site Map</a></li>
52+
<li><a href="https://aka.ms/ml-server-samples">Other ML Server Solutions</a></li>
53+
54+
</div>
55+
</div><!--/col -->
56+
</div><!-- /row -->
57+
58+
<div class="row">
59+
<hr />
60+
<footer>
61+
<p>
62+
This project has adopted the <a href="https://opensource.microsoft.com/codeofconduct/">Microsoft Open Source Code of Conduct</a>. For more information see the <a href="https://opensource.microsoft.com/codeofconduct/faq/">Code of Conduct FAQ</a> or contact <a href="mailto:[email protected]">[email protected]</a> with any additional questions or comments.</p>
63+
<p><small>Hosted on GitHub Pages</small> </p>
64+
</footer>
65+
</div>
66+
67+
</div><!-- /container -->
68+
69+
<script src="//ajax.aspnetcdn.com/ajax/jQuery/jquery-3.3.1.min.js"></script>
70+
71+
72+
<!-- <script>
73+
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
74+
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
75+
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
76+
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
77+
78+
ga('create', 'UA-88854735-5', 'auto');
79+
ga('send', 'pageview');
80+
81+
</script> -->
82+
83+
84+
</body>
85+
</html>

‎contents.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
---
2+
layout: default
3+
title: Template Contents
4+
---
5+
6+
## Template Contents
7+
--------------------
8+
9+
The following is the directory structure for this template:
10+
11+
- [**Data**](#data) This contains Data for scoring. Other data is downloaded during the solution workflow
12+
- [**R**](#model-development-in-R) This contains the R code to prepare training/testing/evaluation set, train the multi-class classifier and evaluate the model.
13+
- [**Python**](#model-development-in-python) This contains the Python code to prepare training/testing/evaluation set, train the multi-class classifier and evaluate the model.
14+
- [**SQLR**](#operationalize-in-sql-r ) Stored procedures in SQL implement the model training workflow with R code.
15+
- [**SQLPy**](#operationalize-in-sql-python ) Stored procedures in SQL implement the model training workflow with Python code.
16+
- [**Resources**](#resources-for-the-solution-packet) This directory contains other resources for the solution package.
17+
18+
19+
20+
21+
### Data
22+
----------------------------
23+
Data for training and testing will also be downloaded and added to this directory, so more files will be present once the solution has been run once.
24+
<table class="table table-striped table-condensed">
25+
<tr><th> File </th><th> Description</th></tr>
26+
<tr><td>News_To_Score </td><td> Text file containing new data for scoring. </td></tr>
27+
</table>
28+
29+
### Model Development in R
30+
-------------------------
31+
32+
<table class="table table-striped table-condensed">
33+
<tr><th> File </th><th> Description </th></tr>
34+
<tr><td>TextClassificationR.ipynb </td><td> Create features on the fly for the training and testing set, train model, make predictions, and evaluate the model in Jupyter notebook.</td></tr>
35+
<tr><td>run_modeling_main.R </td><td> Create features on the fly for the training and testing set, train model, make predictions, and evaluate the model.</td></tr>
36+
</table>
37+
38+
* See [For the Data Scientist](data_scientist.html) for more details about these files.
39+
40+
41+
### Operationalize in SQL R
42+
-------------------------------------------------------
43+
Stored procedures in SQL implement the model training workflow with R code.
44+
45+
<table class="table table-striped table-condensed">
46+
<tr><th> File </th><th> Description </th></tr>
47+
<tr><td>Load_Data.ps1</td><td>Loads all data for the solution if you'd like to create a second instance of the solution on the same server</td></tr>
48+
<tr><td>execute_yourself.sql</td><td>Runs through all the steps of the solution</td></tr>
49+
<tr><td>step0_create_tables.sql</td><td>Create data tables, invoked in Load_Data.ps1</td></tr>
50+
<tr><td>step1_create_features_train.sql</td><td>Create features on the fly and train model </td></tr>
51+
<tr><td>step2_score.sql</td><td>Scores data with model created in step1 </td></tr>
52+
<tr><td>step3_evaluate.sql</td><td>Evaluates model created in step1 </td></tr>
53+
</table>
54+
55+
* See [ For the Database Analyst](dba.html) for more information.
56+
* Follow the [PowerShell Instructions](Powershell_Instructions.html) to execute the PowerShell script which creates these stored procedures.
57+
58+
### Model Development in Python
59+
-------------------------
60+
61+
<table class="table table-striped table-condensed">
62+
<tr><th> File </th><th> Description </th></tr>
63+
<tr><td>TextClassificationR.ipynb </td><td> Create features on the fly for the training and testing set, train model, make predictions, and evaluate the model in Jupyter notebook.</td></tr>
64+
<tr><td>run_modeling_main.py </td><td> Create features on the fly for the training and testing set, train model, make predictions, and evaluate the model.</td></tr>
65+
</table>
66+
67+
68+
* See [For the Data Scientist](data_scientist.html) for more details about these files.
69+
70+
71+
### Operationalize in SQL Python
72+
-------------------------------------------------------
73+
Stored procedures in SQL implement the model training workflow with Python code.
74+
75+
<table class="table table-striped table-condensed">
76+
<tr><th> File </th><th> Description </th></tr>
77+
<tr><td>Load_Data.ps1</td><td>Loads all data for the solution if you'd like to create a second instance of the solution on the same server</td></tr>
78+
<tr><td>execute_yourself.sql</td><td>Runs through all the steps of the solution</td></tr>
79+
<tr><td>step0_create_tables.sql</td><td>Create data tables, invoked in Load_Data.ps1</td></tr>
80+
<tr><td>step1_create_features_train.sql</td><td>Create features on the fly and train model </td></tr>
81+
<tr><td>step2_score.sql</td><td>Scores data with model created in step1 </td></tr>
82+
<tr><td>step3_evaluate.sql</td><td>Evaluates model created in step1 </td></tr>
83+
</table>
84+
85+
* See [ For the Database Analyst](dba.html) for more information.
86+
* Follow the [PowerShell Instructions](Powershell_Instructions.html) to execute the PowerShell script which creates these stored procedures.
87+
88+
### Resources for the Solution Package
89+
------------------------------------
90+
91+
<table class="table table-striped table-condensed">
92+
<tr><th> File </th><th> Description </th></tr>
93+
94+
<tr><td> .\Resources\ActionScripts\ConfigureSQL.ps1</td><td>Configures SQL, called from SetupVM.ps1 </td></tr>
95+
<tr><td> .\Resources\ActionScripts\CreateDatabase.sql</td><td>Creates the database for this solution, called from ConfigureSQL.ps1 </td></tr>
96+
<tr><td> .\Resources\ActionScripts\CreateSQLObjectsPy.sql</td><td>Creates the tables and stored procedures for this solution, called from ConfigureSQL.ps1 </td></tr>
97+
<tr><td> .\Resources\ActionScripts\CreateSQLObjectsR.sql</td><td>Creates the tables and stored procedures for this solution, called from ConfigureSQL.ps1 </td></tr>
98+
<tr><td> .\Resources\ActionScripts\TextClassificationSetup.ps1</td><td>Configures SQL, creates and populates database</td></tr>
99+
<tr><td> .\Resources\ActionScripts\SolutionHelp.url</td><td>URL to the help page </td></tr>
100+
101+
</table>
102+
103+
104+
105+
106+
[&lt; Home](index.html)

0 commit comments

Comments
 (0)