update samples from Release-18 as a part of 1.1.0rc0 SDK experimental release (Azure#760)

Co-authored-by: vizhur <[email protected]>

Author: harneetvirk

README.md

@@ -2,7 +2,7 @@
 
 This repository contains example notebooks demonstrating the [Azure Machine Learning](https://azure.microsoft.com/en-us/services/machine-learning-service/) Python SDK which allows you to build, train, deploy and manage machine learning solutions using Azure.  The AML SDK allows you the choice of using local or cloud compute resources, while managing and maintaining the complete data science workflow from the cloud.
 
-![Azure ML Workflow](https://raw.githubusercontent.com/MicrosoftDocs/azure-docs/master/articles/machine-learning/service/media/concept-azure-machine-learning-architecture/workflow.png)
+![Azure ML Workflow](https://raw.githubusercontent.com/MicrosoftDocs/azure-docs/master/articles/machine-learning/media/concept-azure-machine-learning-architecture/workflow.png)
 
 
 ## Quick installation
@@ -17,7 +17,7 @@ This [index](.index.md) should assist in navigating the Azure Machine Learning n
 
 If you want to...
 
- * ...try out and explore Azure ML, start with image classification tutorials: [Part 1 (Training)](./tutorials/img-classification-part1-training.ipynb) and [Part 2 (Deployment)](./tutorials/img-classification-part2-deploy.ipynb).
+ * ...try out and explore Azure ML, start with image classification tutorials: [Part 1 (Training)](./tutorials/image-classification-mnist-data/img-classification-part1-training.ipynb) and [Part 2 (Deployment)](./tutorials/image-classification-mnist-data/img-classification-part2-deploy.ipynb).
  * ...learn about experimentation and tracking run history, first [train within Notebook](./how-to-use-azureml/training/train-within-notebook/train-within-notebook.ipynb), then try [training on remote VM](./how-to-use-azureml/training/train-on-remote-vm/train-on-remote-vm.ipynb) and [using logging APIs](./how-to-use-azureml/training/logging-api/logging-api.ipynb).
  * ...train deep learning models at scale, first learn about [Machine Learning Compute](./how-to-use-azureml/training/train-on-amlcompute/train-on-amlcompute.ipynb), and then try [distributed hyperparameter tuning](./how-to-use-azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-pytorch/train-hyperparameter-tune-deploy-with-pytorch.ipynb) and [distributed training](./how-to-use-azureml/training-with-deep-learning/distributed-pytorch-with-horovod/distributed-pytorch-with-horovod.ipynb).
  * ...deploy models as a realtime scoring service, first learn the basics by [training within Notebook and deploying to Azure Container Instance](./how-to-use-azureml/training/train-within-notebook/train-within-notebook.ipynb), then learn how to [production deploy models on Azure Kubernetes Cluster](./how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks.ipynb).

configuration.ipynb

@@ -103,7 +103,7 @@
       "source": [
         "import azureml.core\n",
         "\n",
-        "print(\"This notebook was created using version 1.0.85 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.1.0rc0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/classification-credit-card-fraud/auto-ml-classification-credit-card-fraud.ipynb

@@ -282,7 +282,11 @@
     {
       "cell_type": "code",
       "execution_count": null,
-      "metadata": {},
+      "metadata": {
+        "tags": [
+          "widget-rundetails-sample"
+        ]
+      },
       "outputs": [],
       "source": [
         "from azureml.widgets import RunDetails\n",

how-to-use-azureml/automated-machine-learning/forecasting-grouping/build.py

@@ -17,7 +17,7 @@
 
 def _get_groups(data: Dataset, group_column_names: List[str]) -> pd.DataFrame:
     return data._dataflow.distinct(columns=group_column_names)\
-        .keep_columns(columns=group_column_names).to_pandas_dataframe()
+        .keep_columns(columns=group_column_names).to_pandas_dataframe()[group_column_names]
 
 
 def _get_configs(automlconfig: AutoMLConfig,
@@ -28,6 +28,9 @@ def _get_configs(automlconfig: AutoMLConfig,
     # remove invalid characters regex
     valid_chars = re.compile('[^a-zA-Z0-9-]')
     groups = _get_groups(data, group_column_names)
+    if groups.shape[0] > 40:
+        raise RuntimeError("AutoML only supports 40 or less groups. Please modify your "
+                           "group_column_names to ensure no more than 40 groups are present.")
     configs = {}
     for i, group in groups.iterrows():
         single = data._dataflow
@@ -118,7 +121,7 @@ def build_pipeline_steps(automlconfig: AutoMLConfig,
         # add deployment step
         pp_group_column_names = PipelineParameter(
             "group_column_names",
-            default_value="#####".join(list(reversed(group_column_names))))
+            default_value="#####".join(list(group_column_names)))
 
         pp_model_names = PipelineParameter(
             "model_names",

how-to-use-azureml/automated-machine-learning/forecasting-grouping/deploy/myenv.yml

@@ -6,6 +6,7 @@ dependencies:
 
 - python=3.6.2
 - numpy>=1.16.0,<=1.16.2
+- pandas>=0.21.0,<=0.23.4
 - scikit-learn>=0.19.0,<=0.20.3
 - conda-forge::fbprophet==0.5
 

how-to-use-azureml/automated-machine-learning/forecasting-grouping/deploy/score.py

@@ -37,7 +37,7 @@ def run(raw_data):
             elif isinstance(grain, str):
                 cur_group = grain
             else:
-                cur_group = "#####".join(list(grain))
+                cur_group = "#####".join([str(v) for v in list(grain)])
             cur_group = valid_chars.sub('', cur_group)
             print("Query model for group {}".format(cur_group))
             if cur_group not in models:

how-to-use-azureml/automated-machine-learning/regression-hardware-performance-explanation-and-featurization/auto-ml-regression-hardware-performance-explanation-and-featurization.ipynb

@@ -717,10 +717,10 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "myenv = automl_run.get_environment().python.conda_dependencies\n",
+        "conda_dep = automl_run.get_environment().python.conda_dependencies\n",
         "\n",
         "with open(\"myenv.yml\",\"w\") as f:\n",
-        "    f.write(myenv.serialize_to_string())\n",
+        "    f.write(conda_dep.serialize_to_string())\n",
         "\n",
         "with open(\"myenv.yml\",\"r\") as f:\n",
         "    print(f.read())"
@@ -761,16 +761,16 @@
         "from azureml.core.model import InferenceConfig\n",
         "from azureml.core.webservice import AciWebservice\n",
         "from azureml.core.model import Model\n",
+        "from azureml.core.environment import Environment\n",
         "\n",
         "aciconfig = AciWebservice.deploy_configuration(cpu_cores=1, \n",
         "                                               memory_gb=1, \n",
         "                                               tags={\"data\": \"Machine Data\",  \n",
         "                                                     \"method\" : \"local_explanation\"}, \n",
         "                                               description='Get local explanations for Machine test data')\n",
         "\n",
-        "inference_config = InferenceConfig(runtime= \"python\", \n",
-        "                                   entry_script=\"score_explain.py\",\n",
-        "                                   conda_file=\"myenv.yml\")\n",
+        "myenv = Environment.from_conda_specification(name=\"myenv\", file_path=\"myenv.yml\")\n",
+        "inference_config = InferenceConfig(entry_script=\"score_explain.py\", environment=myenv)\n",
         "\n",
         "# Use configs and models generated above\n",
         "service = Model.deploy(ws, 'model-scoring', [scoring_explainer_model, original_model], inference_config, aciconfig)\n",

how-to-use-azureml/explain-model/azure-integration/scoring-time/train-explain-model-on-amlcompute-and-deploy.ipynb

@@ -409,16 +409,6 @@
         "    print(f.read())"
       ]
     },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "%%writefile dockerfile\n",
-        "RUN apt-get update && apt-get install -y g++ "
-      ]
-    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -439,17 +429,17 @@
         "from azureml.core.model import InferenceConfig\n",
         "from azureml.core.webservice import AciWebservice\n",
         "from azureml.core.model import Model\n",
+        "from azureml.core.environment import Environment\n",
+        "\n",
         "\n",
         "aciconfig = AciWebservice.deploy_configuration(cpu_cores=1, \n",
         "                                               memory_gb=1, \n",
         "                                               tags={\"data\": \"IBM_Attrition\",  \n",
         "                                                     \"method\" : \"local_explanation\"}, \n",
         "                                               description='Get local explanations for IBM Employee Attrition data')\n",
         "\n",
-        "inference_config = InferenceConfig(runtime= \"python\", \n",
-        "                                   entry_script=\"score_remote_explain.py\",\n",
-        "                                   conda_file=\"myenv.yml\",\n",
-        "                                   extra_docker_file_steps=\"dockerfile\")\n",
+        "myenv = Environment.from_conda_specification(name=\"myenv\", file_path=\"myenv.yml\")\n",
+        "inference_config = InferenceConfig(entry_script=\"score_remote_explain.py\", environment=myenv)\n",
         "\n",
         "# Use configs and models generated above\n",
         "service = Model.deploy(ws, 'model-scoring-service', [scoring_explainer_model, original_model], inference_config, aciconfig)\n",

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/README.md

@@ -18,5 +18,6 @@ These notebooks below are designed to go in sequence.
 13. [aml-pipelines-showcasing-datapath-and-pipelineparameter.ipynb](https://aka.ms/pl-datapath): This notebook showcases how to use DataPath and PipelineParameter in AML Pipeline.
 14. [aml-pipelines-how-to-use-pipeline-drafts.ipynb](http://aka.ms/pl-pl-draft): This notebook shows how to use Pipeline Drafts. Pipeline Drafts are mutable pipelines which can be used to submit runs and create Published Pipelines.
 15. [aml-pipelines-hot-to-use-modulestep.ipynb](https://aka.ms/pl-modulestep): This notebook shows how to define Module, ModuleVersion and how to use them in an AML Pipeline using ModuleStep.
+16. [aml-pipelines-with-notebook-runner-step.ipynb](https://aka.ms/pl-nbrstep): This notebook shows how you can run another notebook as a step in Azure Machine Learning Pipeline.
 
  ![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/README.png)

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-with-notebook-runner-step.ipynb

@@ -70,8 +70,6 @@
         "from azureml.pipeline.core import PipelineData\n",
         "from azureml.core.datastore import Datastore\n",
         "\n",
-        "from azureml.widgets import RunDetails\n",
-        "\n",
         "from azureml.core import Workspace, Experiment\n",
         "from azureml.contrib.notebook import NotebookRunConfig, AzureMLNotebookHandler\n",
         "\n",
@@ -149,7 +147,7 @@
       "metadata": {},
       "source": [
         "### Create or Attach an AmlCompute cluster\n",
-        "You will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#compute-target) for your AutoML run. In this tutorial, you get the default `AmlCompute` as your training compute resource."
+        "You will need to create a [compute target](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.computetarget?view=azure-ml-py) for your remote run. In this tutorial, you get the default `AmlCompute` as your training compute resource."
       ]
     },
     {
@@ -205,7 +203,7 @@
         "conda_run_config.environment.docker.enabled = True\n",
         "conda_run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
         "\n",
-        "cd = CondaDependencies.create(pip_packages=['azureml-sdk'], pin_sdk_version=False)\n",
+        "cd = CondaDependencies.create(pip_packages=['azureml-sdk'])\n",
         "conda_run_config.environment.python.conda_dependencies = cd\n",
         "\n",
         "print('run config is ready')"
@@ -298,7 +296,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "from azureml.pipeline.core import PipelineParameter, TrainingOutput\n",
+        "from azureml.pipeline.core import PipelineParameter\n",
         "\n",
         "output_from_notebook = PipelineData(name=\"notebook_processed_data\",\n",
         "                                    datastore=Datastore.get(ws, \"workspaceblobstore\"))\n",
@@ -354,6 +352,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
+        "from azureml.widgets import RunDetails\n",
         "RunDetails(pipeline_run1).show()"
       ]
     },