[PYTHON] Try using Jupyter Notebook of Azure Machine Learning

Azure Machine Learning is a cloud-based machine learning service provided by Microsoft.

Azure Machine Learning allows you to perform various machine learning techniques using the GUI, but you can also use Jupyter.

Use Jupyter Notebook with Azure Machine Learning

You can try Azure Machine Learning for free if you have a Microsoft account. Jupyter can also be used with a free plan.

See the link below for pricing details.

Price --Machine Learning | Microsoft Azure

Create a Microsoft account

If you do not have a Microsoft account, create one from the link.

Home | Microsoft Account

Create workspace

Open the link below and click the "Get Started" button to sign in to your Microsoft account.

Microsoft Azure Machine Learning Studio

When the screen that says "Workspace Not Found" appears, open Microsoft Azure Machine Learning Studio again in your browser and press the "Get Started" button.

If successful, a free workspace has been created and a screen like the one below will open.

Creating a Jupyter Notebook

Click "+ NEW" at the bottom left of the screen to open the screen below. Select the Python version from NOTEBOOK.

This time I will select Python2. You will be asked for the name of the notebook, so enter it appropriately.

The created Jupyter Notebook is displayed in NOTEBOOKS, so click it to open it.

An empty Notebook opens. After this, you can use it as usual.

Available libraries

Jupyter for Azure Machine Learning uses Anaconda, and the packages listed below are available.

Anaconda Package List — Continuum documentation

The version of Anaconda is 2.1, which is a little old and may not be the latest depending on the package.

Integration of Azure Machine Learning and Jupyter Notebook

Azure Machine Learning's Jupyter Notebook has the Azure Machine Learning Python client installed. You can use this to operate Azure Machine Learning and exchange data.

Azure/Azure-MachineLearning-ClientLibrary-Python

Connect to workspace

First, connect to the workspace you are currently using. You can find the workspace ID and token required for the connection from SETTINGS in studio.

from azureml import Workspace
ws = Workspace(
    workspace_id="YOUR_WORKSPACE_ID", 
    authorization_token="YOUR_AUTHORIZATION_TOKEN",
    endpoint="https://studio.azureml.net"
)

Use a dataset

You can check the available datasets as follows.

ws.datasets

You can get the data by specifying the name of the dataset you want to use and convert it to a dataframe with `` `to_dataframe```.

df = ws.datasets['Bike Rental UCI dataset'].to_dataframe()

Get data from Experiment

To get the intermediate data of Experiment with Jupyter, enter the data in "Convert to CSV".

Click the output port of "Convert to CSV" and select "Generate Data Access Code" to display the code for retrieving the intermediate data as a data frame. (The code for connecting to the workspace is also displayed, but I got an error if the endpoint was not set. It worked when I added the endpoint as in [above](#Connect to workspace).)

Add Jupyter data to your Azure Machine Learning dataset

You can add data that you are working with in Jupyter to your Azure Machine Learning dataset.

Use add_from_dataframe when adding to the dataset.

import pandas as pd
import numpy as np
from sklearn.datasets import load_boston

boston = load_boston()
df = pd.DataFrame(
  np.column_stack([boston.data, boston.target]),
  columns=boston.feature_names
)

dataset = ws.datasets.add_from_dataframe(
    dataframe=df,
    data_type_id='GenericCSV',
    name='boston',
    description=boston.DESCR,
)

Create a web service

You can also create a web service from Jupyter.

For example, running the following code will create a web service called add.

from azureml import services

@services.publish(ws.workspace_id, ws.authorization_token)
@services.types(a = float, b = float)
@services.returns(float)
def add(a, b):
    return a + b

Use an existing web service

You can also use an existing web service from Jupyter.

When I used a web service created with Jupyter, authentication did not pass and it did not work, so with GUI Let's create a service to add and use it from Jupyter.

The contents of "Execute Python Script" are as follows.

def azureml_main(dataframe1 = None):
    dataframe1['c'] = dataframe1.a + dataframe1.b
    return dataframe1

If you deploy this as a web service, you will have a service that adds the two numbers as follows.

To use this web service from Notebook, define a function using a decorator as follows: The url and api_key can be obtained from the help page of REQUEST / RESPONSE, the dashboard of the created web service.

from azureml import services

url = 'WEB_SERVICE_URL'
api_key = 'WEB_SERVICE_API_KEY'
@services.service(url, api_key)
@services.types(a = float, b = float)
@services.returns(float)
def add(a, b):
    pass

You can use this function as a web service. (It will take some time to execute the first time.)

Finally

There are still many things that can not be done from the client Jupyter, but I am looking forward to the cooperation function being strengthened in the future. Azure Machine Learning has many examples, so it seems interesting to take a closer look at the intermediate results and draw a graph with Jupyter.

Reference material

• Introducing Jupyter Notebooks in Azure ML Studio - Machine Learning - Site Home - TechNet Blogs
• Accessing Python datasets using the Machine Learning Python client library | Microsoft Azure