Merge pull request Azure#72 from rastala/master

Add logging API notebook

Author: rastala

01.getting-started/06.logging-api/06.logging-api.ipynb

@@ -0,0 +1,328 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
+        "\n",
+        "Licensed under the MIT License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# 06. Logging APIs\n",
+        "This notebook showcase various ways to use the Azure Machine Learning service run logging APIs, and view the results in the Azure portal."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Prerequisites\n",
+        "Make sure you go through the [00. Installation and Configuration](../../00.configuration.ipynb) Notebook first if you haven't. Also make sure you have tqdm and matplotlib installed in the current kernel.\n",
+        "\n",
+        "```\n",
+        "(myenv) $ conda install -y tqdm matplotlib\n",
+        "```"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Validate Azure ML SDK installation and get version number for debugging purposes"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": [
+          "install"
+        ]
+      },
+      "outputs": [],
+      "source": [
+        "from azureml.core import Experiment, Run, Workspace\n",
+        "import azureml.core\n",
+        "import numpy as np\n",
+        "\n",
+        "# Check core SDK version number\n",
+        "print(\"SDK version:\", azureml.core.VERSION)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Initialize Workspace\n",
+        "\n",
+        "Initialize a workspace object from persisted configuration."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": [
+          "create workspace"
+        ]
+      },
+      "outputs": [],
+      "source": [
+        "ws = Workspace.from_config()\n",
+        "print('Workspace name: ' + ws.name, \n",
+        "      'Azure region: ' + ws.location, \n",
+        "      'Subscription id: ' + ws.subscription_id, \n",
+        "      'Resource group: ' + ws.resource_group, sep='\\n')"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Set experiment\n",
+        "Create a new experiment (or get the one with such name)."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "exp = Experiment(workspace=ws, name='logging-api-test')"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Log metrics\n",
+        "We will start a run, and use the various logging APIs to record different types of metrics during the run."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from tqdm import tqdm\n",
+        "\n",
+        "# start logging for the run\n",
+        "run = exp.start_logging()\n",
+        "\n",
+        "# log a string value\n",
+        "run.log(name='Name', value='Logging API run')\n",
+        "\n",
+        "# log a numerical value\n",
+        "run.log(name='Magic Number', value=42)\n",
+        "\n",
+        "# Log a list of values. Note this will generate a single-variable line chart.\n",
+        "run.log_list(name='Fibonacci', value=[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89])\n",
+        "\n",
+        "# create a dictionary to hold a table of values\n",
+        "sines = {}\n",
+        "sines['angle'] = []\n",
+        "sines['sine'] = []\n",
+        "\n",
+        "for i in tqdm(range(-10, 10)):\n",
+        "    # log a metric value repeatedly, this will generate a single-variable line chart.\n",
+        "    run.log(name='Sigmoid', value=1 / (1 + np.exp(-i)))\n",
+        "    angle = i / 2.0\n",
+        "    \n",
+        "    # log a 2 (or more) values as a metric repeatedly. This will generate a 2-variable line chart if you have 2 numerical columns.\n",
+        "    run.log_row(name='Cosine Wave', angle=angle, cos=np.cos(angle))\n",
+        "        \n",
+        "    sines['angle'].append(angle)\n",
+        "    sines['sine'].append(np.sin(angle))\n",
+        "\n",
+        "# log a dictionary as a table, this will generate a 2-variable chart if you have 2 numerical columns\n",
+        "run.log_table(name='Sine Wave', value=sines)\n",
+        "\n",
+        "run.complete()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Even after the run is marked completed, you can still log things."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Log an image\n",
+        "This is how to log a _matplotlib_ pyplot object."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%matplotlib inline\n",
+        "import matplotlib.pyplot as plt\n",
+        "angle = np.linspace(-3, 3, 50)\n",
+        "plt.plot(angle, np.tanh(angle), label='tanh')\n",
+        "plt.legend(fontsize=12)\n",
+        "plt.title('Hyperbolic Tangent', fontsize=16)\n",
+        "plt.grid(True)\n",
+        "\n",
+        "run.log_image(name='Hyperbolic Tangent', plot=plt)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Upload a file"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can also upload an abitrary file. First, let's create a dummy file locally."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%%writefile myfile.txt\n",
+        "\n",
+        "This is a dummy file."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Now let's upload this file into the run record as a run artifact, and display the properties after the upload."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "props = run.upload_file(name='myfile_in_the_cloud.txt', path_or_stream='./myfile.txt')\n",
+        "props.serialize()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Examine the run"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Now let's take a look at the run detail page in Azure portal. Make sure you checkout the various charts and plots generated/uploaded."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "run"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can get all the metrics in that run back."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "run.get_metrics()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can also see the files uploaded for this run."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "run.get_file_names()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can also download all the files locally."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "import os\n",
+        "os.makedirs('files', exist_ok=True)\n",
+        "\n",
+        "for f in run.get_file_names():\n",
+        "    dest = os.path.join('files', f.split('/')[-1])\n",
+        "    print('Downloading file {} to {}...'.format(f, dest))\n",
+        "    run.download_file(f, dest)   "
+      ]
+    }
+  ],
+  "metadata": {
+    "authors": [
+      {
+        "name": "haining"
+      }
+    ],
+    "kernelspec": {
+      "display_name": "Python 3.6",
+      "language": "python",
+      "name": "python36"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.6"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}