[PYTHON] Microsoft's Deep Learning Library "CNTK" Tutorial

Do you know CNTK? CNTK is an open source Deep Learning library led by Microsoft. To be honest, I recently started using it, but this CNTK is not like Microsoft in a good way, and it is quite good. I think it's cool. This time, I will first briefly introduce the features of CNTK, [I will look at a code example using Jupyter Notebook in the second half](http://qiita.com/usuyama/items/bb7600ab7cfc1f5b8be3#python%E3%81%A7cntk] % E3% 82% 92% E4% BD% BF% E3% 81% A3% E3% 81% A6% E3% 81% BF% E3% 82% 8B).

** [Tutorial here](http://qiita.com/usuyama/items/bb7600ab7cfc1f5b8be3#python%E3%81%A7cntk%E3%82%92%E4%BD%BF%E3%81%A3%E3% 81% A6% E3% 81% BF% E3% 82% 8B) **

Open development policy that does not stick to "Microsoft"

Open Source

https://github.com/Microsoft/CNTK

Development is underway on GitHub. There is no in-house version.

Official wiki

Python and C ++ API

• Python API
• C++ API

There is no .Net API (yet). Brainscript is also supported.

Available on Linux and Windows

"Both Linux and Windows are first citizens of CNTK," the development team said. It seems that both platforms support the same functionality, although there may be a slight time lag. Docker is also available for Linux, so maybe Linux is easier to use. not.

High speed & scale

Scale on multiple GPUs

This is a performance comparison with Therano, TensorFlow, Torch 7, and Caffe. This is a comparison of how many frames can be processed per second by a fully connected 4-layer neural network (minibatch size is 8192). The blue and orange settings are clusters of Linux machines with one or four GPUs (Nvidia K40), and green is a cluster of two Linux machines (2 x 4 GPUs). ** With one GPU, each library has almost the same speed (CNTK is faster than TensorFlow?), But only CNTK can effectively utilize multiple GPUs. ** With 4 GPUs, CNTK is almost quadrupled, but Torch 7 and Caffe are about doubling. TensorFlow is almost unchanged.

Microsoft Research Blog

October 2016: CNTK reaches human level in speech recognition accuracy

In an experiment in October 2016, a speech recognition engine developed by Microsoft's team recorded a word error rate of 5.9%. This was equivalent to asking a professional transcription service for the same audio file. What's more, this transcription service uses a two-step process, with the first person transcribing from 1 and the second person making corrections.

The detailed part of the method is published in Paper, but it uses a neural network with a very large number of layers. Again, it is stated in the Paper that CNTK was demonstrating its power and that high-speed processing of CNTK was essential.

• Historic Achievement: Microsoft researchers reach human parity in conversational speech recognition
• Microsoft researchers achieve speech recognition milestone

Try using CNTK in Python

Introduction to CNTK (Python)

My colleague made a good tutorial, so I translated it. Since it is a Jupyter Notebook, you can edit and execute the code on the spot without preparing the environment. We plan to add more in sequence.

-CNTK 101: Logistic Regression and Introduction to Machine Learning ――First, let's create a linear model and solve a simple two-class classification problem. Introducing the general flow from model construction, learning to evaluation. Not only does it follow the code, but it also briefly introduces the mathematics behind it. -CNTK 102: Learning simulation data with Feed Forward Network --Create a 4-layer neural network and learn the same simulation data as 101. - -CNTK 103 Part A: MNIST (handwritten number) data preparation --In Part A, you will download the MNIST data, convert it to an easy-to-use format, and save it in a file. -

-CNTK 103 Part B: Learn MNIST data with Feed Forward Network --Learning the MNIST data downloaded in Part A with the 4-layer neural network created in 102.

Jupyter Notebook (Azure Notebook) If you log in with your Microsoft account, you will be able to clone your notes for editing and execution.

It is very convenient to be able to change the code on the spot and execute it, so please try changing the code on the tutorial and playing with it.

reference

--If you want to prepare the environment locally, Official or [here](http://qiita.com/ The article of hiiyan0402 / items / ec41908f2aa81b73047e) seems to be helpful. -(I haven't tried it yet) Prepare the environment on Azure seems to be easier. You can also use GPU.