|
30 | 30 | "\n", |
31 | 31 | "## Prerequisites\n", |
32 | 32 | "\n", |
33 | | - "See prerequisites in the [Azure Machine Learning documentation](https://docs.microsoft.com/azure/machine-learning/service/tutorial-train-models-with-aml#prerequisites)." |
| 33 | + "See prerequisites in the [Azure Machine Learning documentation](https://docs.microsoft.com/azure/machine-learning/service/tutorial-train-models-with-aml#prerequisites).\n", |
| 34 | + "\n", |
| 35 | + "On the computer running this notebook, conda install matplotlib, numpy, scikit-learn=0.22.1" |
34 | 36 | ] |
35 | 37 | }, |
36 | 38 | { |
|
309 | 311 | "import glob\n", |
310 | 312 | "\n", |
311 | 313 | "from sklearn.linear_model import LogisticRegression\n", |
312 | | - "from sklearn.externals import joblib\n", |
| 314 | + "import joblib\n", |
313 | 315 | "\n", |
314 | 316 | "from azureml.core import Run\n", |
315 | 317 | "from utils import load_data\n", |
|
397 | 399 | "source": [ |
398 | 400 | "### Create an estimator\n", |
399 | 401 | "\n", |
400 | | - "An estimator object is used to submit the run. Azure Machine Learning has pre-configured estimators for common machine learning frameworks, as well as generic Estimator. Create SKLearn estimator for scikit-learn model, by specifying\n", |
| 402 | + "An estimator object is used to submit the run. Azure Machine Learning has pre-configured estimators for common machine learning frameworks, as well as generic Estimator. Create an estimator by specifying\n", |
401 | 403 | "\n", |
402 | 404 | "* The name of the estimator object, `est`\n", |
403 | 405 | "* The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for execution. \n", |
404 | 406 | "* The compute target. In this case you will use the AmlCompute you created\n", |
405 | 407 | "* The training script name, train.py\n", |
406 | | - "* Parameters required from the training script \n", |
| 408 | + "* An environment that contains the libraries needed to run the script\n", |
| 409 | + "* Parameters required from the training script. \n", |
| 410 | + "\n", |
| 411 | + "In this tutorial, the target is AmlCompute. All files in the script folder are uploaded into the cluster nodes for execution. The data_folder is set to use the dataset.\n", |
| 412 | + "\n", |
| 413 | + "First, create the environment that contains: the scikit-learn library, azureml-dataprep required for accessing the dataset, and azureml-defaults which contains the dependencies for logging metrics. The azureml-defaults also contains the dependencies required for deploying the model as a web service later in the part 2 of the tutorial.\n", |
407 | 414 | "\n", |
408 | | - "In this tutorial, the target is AmlCompute. All files in the script folder are uploaded into the cluster nodes for execution. The data_folder is set to use the dataset." |
| 415 | + "Once the environment is defined, register it with the Workspace to re-use it in part 2 of the tutorial." |
409 | 416 | ] |
410 | 417 | }, |
411 | 418 | { |
|
418 | 425 | "from azureml.core.conda_dependencies import CondaDependencies\n", |
419 | 426 | "\n", |
420 | 427 | "# to install required packages\n", |
421 | | - "env = Environment('my_env')\n", |
422 | | - "cd = CondaDependencies.create(pip_packages=['azureml-sdk','scikit-learn==0.22.1','azureml-dataprep[pandas,fuse]>=1.1.14'])\n", |
| 428 | + "env = Environment('tutorial-env')\n", |
| 429 | + "cd = CondaDependencies.create(pip_packages=['azureml-dataprep[pandas,fuse]>=1.1.14', 'azureml-defaults'], conda_packages = ['scikit-learn==0.22.1'])\n", |
423 | 430 | "\n", |
424 | | - "env.python.conda_dependencies = cd" |
| 431 | + "env.python.conda_dependencies = cd\n", |
| 432 | + "\n", |
| 433 | + "# Register environment to re-use later\n", |
| 434 | + "env.register(workspace = ws)" |
| 435 | + ] |
| 436 | + }, |
| 437 | + { |
| 438 | + "cell_type": "markdown", |
| 439 | + "metadata": {}, |
| 440 | + "source": [ |
| 441 | + "Then, create the estimator by specifying the training script, compute target and environment." |
425 | 442 | ] |
426 | 443 | }, |
427 | 444 | { |
|
434 | 451 | }, |
435 | 452 | "outputs": [], |
436 | 453 | "source": [ |
437 | | - "from azureml.train.sklearn import SKLearn\n", |
| 454 | + "from azureml.train.estimator import Estimator\n", |
438 | 455 | "\n", |
439 | 456 | "script_params = {\n", |
440 | 457 | " # to mount files referenced by mnist dataset\n", |
441 | 458 | " '--data-folder': mnist_file_dataset.as_named_input('mnist_opendataset').as_mount(),\n", |
442 | 459 | " '--regularization': 0.5\n", |
443 | 460 | "}\n", |
444 | 461 | "\n", |
445 | | - "est = SKLearn(source_directory=script_folder,\n", |
| 462 | + "est = Estimator(source_directory=script_folder,\n", |
446 | 463 | " script_params=script_params,\n", |
447 | 464 | " compute_target=compute_target,\n", |
448 | 465 | " environment_definition=env,\n", |
|
667 | 684 | "name": "python", |
668 | 685 | "nbconvert_exporter": "python", |
669 | 686 | "pygments_lexer": "ipython3", |
670 | | - "version": "3.6.9" |
| 687 | + "version": "3.7.6" |
671 | 688 | }, |
672 | 689 | "msauthor": "roastala" |
673 | 690 | }, |
|
0 commit comments