[PYTHON] Introduction to Thano Function definition and automatic differentiation

This is the article on the 6th day of Hiroshima University IT Engineer Advent Calendar 2019.

I'm Yuto Araki, a 4th year student at the Faculty of Engineering, Hiroshima University! I will be an IT engineer in Fukuoka from next year! I'm doing Twitter with the handle name Imikoto, so please check that as well. （https://twitter.com/es__135 ）

Getting Started with Thano

python

import numpy as np
import theano.tensor as T
from theano import function

x = T.dscalar('x')
y = T.dscalar('y')
z = x + y
f = function([x,y],z)
f(2,3) #array(5.)Is output.

(Quoted from http://deeplearning.net/software/theano/tutorial/adding.html#exercise)

Define the variables used in the function below on the 5th and 6th lines, and create the function on the 7th and 8th lines. It may seem a little strange before you get used to it, but once you get used to it, it will become easier to write.

Let's write the three-square theorem by applying this.

x = T.dscalar('x')
y = T.dscalar('y')
Pythagoras = ( x**2 + y**2 )**(1/2)
length     = function([x,y],Pythagoras)
length(3,4) #5.0 is output.

Automatic differentiation

For more information, read Wikipedia, but you can calculate the derivative of the defined function.

x = T.dscalar('x')

#y = sin(x) + cos(x)
y = T.sin(x) + T.cos(x)

#The meaning of differentiating y with respect to x
gy = T.grad(cost=y, wrt=x)

f = function(inputs=[x], outputs=gy)

#Find the derivative by giving a concrete x
print (f(0))        #1.0
print (f(np.pi / 4))#1.1102230246251565e-16
print (f(np.pi))    #-1.0000000000000002

(This article was used as a reference. I recommend this article because it introduces many other examples.)

Also,

x = T.dscalar('x')
y = T.dscalar('y')
z =  T.sin(x) + T.cos(y)
#z = sin(x) + cos(y)

#The meaning of differentiating y with respect to x
dx    = T.grad(cost=z, wrt=x)
dy    = T.grad(cost=z, wrt=y)
dz_dx = function(inputs=[x,y], outputs=dx)
dz_dy = function(inputs=[x,y], outputs=dy)
#Find the derivative by giving a concrete x
pi = math.pi
dz_dx(pi/2,pi/2) #array(6.123234e-17)
dz_dy(0,pi/2)    #array(-1.)

In this way, you can also calculate the derivative of two variables. Could you somehow understand how to use it?

Summary

It's a little special way of writing, but once I got used to it, I thought it was surprisingly easy to write and use. If I have time, I would like to write an article about implementing neural networks using Thano. This is the library used in the early Deep Learning, and I can't imagine how it has evolved into a modern library. Will study

References

theano documentation http://deeplearning.net/software/theano/tutorial/adding.html#exercise

A record of artificial intelligence http://aidiary.hatenablog.com/entry/20150518/1431954329

github https://github.com/Theano/Theano