[PYTHON] Tensorflow memo [updated from time to time]

Tensorflow memo

It is a memo that I have left behind

Structure of Tensorflow

Concept of graphs, arithmetic nodes, and tensors

What is a graph?

A figure representing one layer consisting of multiple neurons φ(X * W + b)

Consists of nodes and edges connecting the nodes There are arithmetic nodes, variable nodes, press holder nodes, etc.

What is a tensor?

Amount flowing through the graph

A tensor is an n-dimensional array or list

Main notation

Variable tf.Variable Matrix multiplication tf.matmul Application of φ tf.nn.relu

Graph definition

graph.py


#Definition of variable to put W
weights = tf.Variable()
#Definition of variable to put b
bias = tf.Variable()
#The function of the layer φ(X*W+b)Defined in
#Here φ uses relu
#images is the input that this layer receives
#hidden1 is the output of this layer

#1st layer
hidden1 = tf.nn.relu(tf.matmul(images, weights) + bias)

#2nd layer
hidden2 = tf.nn.relu(tf.matmul(hidden1,weights) + bias)
#images, weights, bias,All hidden1 are tensors

Variable definition

Variable initialization

init.py


w = tf.Variable(tf.random_norml([784, 200], stddev = 0.35), name = "weights")
b  =tf.Variable(tf.zeros([200], name = "biases")

#Operation to initialize this variable
#Caution! It hasn't been executed yet, just a node has been added.
init_op = tf.initialize_all_variables()


#Call this initialization after launching the model
#The defined model works for the first time in Session.
#Use run to call
with tf.Session() as sess:
    # Run the init operation.
    sess.run(init_op)




** Saving and restoring variables

save.py


#Create a variable
v1 = tf.variable(..., name = "v1")
v2 = tf.variable(..., name = "v2")

#Initialize variables init_op node
init_op = tf.initalize_all_variables()
#Save all variables,Add save node to restore
saver = tf.train.Saver()

#After launching the model, initializing variables, doing some work
#Save variables to disk
with tf.Session() as sess:
  sess.run(init_op)
  #Do something with the model
  ##########
  
  #Save variables to disk
  save_path = saver.save(sess, "/tmp/model.ckpt")
  print("Modef saved in file: %s" % save_path)

  #Variable restore
  saver.retore(sess, "/tmp/model.ckpt")
  print ("Modell restored")

Parameter optimization (training)

Parameter (weight and bias) optimization

  1. Build a graph and calculate the output with the input data in the graph
  2. Compare the output with the correct answer. Use loss function for comparison
  3. Use Gradient Descent to modify the value of the loss function to a smaller value
  4. Calculate the output with the new parameters. Repeat until the loss function is small

Main functions

GradientDescentOptimizer() Optimization operation for parameter optimization. Optimize the loss function with this Optimizer

opt.py



###For numerical prediction###
 loss = tf.reduce_mean(tf.square(y - y_data))
#Learning rate 0.Gradient descent at 5
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)

###In case of classification###
y_ = tf.placeholder("float", [None , 10])
cross_enttopy = -tf.reduce_sum(y_ * tf.log(y))
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(cross_entropy)



Recommended Posts

Tensorflow memo [updated from time to time]
progate Python learning memo (updated from time to time)
vtkXMLUnstructuredGridReader Summary (updated from time to time)
vtkOpenFOAMReader Summary (Updated from time to time)
Engineer vocabulary (updated from time to time)
Private Python handbook (updated from time to time)
vtkClipPolyData / DataSet Summary (Updated from time to time)
[Updated from time to time] PostmarketOS related notes
Summary of vtkThreshold (updated from time to time)
Summary of gcc options (updated from time to time)
[Updated from time to time] LetCode algorithm and library
Notes on machine learning (updated from time to time)
OpenFOAM post-processing cheat sheet (updated from time to time)
Useful help sites, etc. (updated from time to time)
Anaconda updated from 4.2.0 to 4.3.0 (python3.5 updated to python3.6)
Machine learning python code summary (updated from time to time)
Apache settings, log confirmation, etc. (* Updated from time to time)
[Updated from time to time] Review of Let Code NumPy
I read the Chainer reference (updated from time to time)
[Updated from time to time] Summary of design patterns in Java
Python (from first time to execution)
(Updated from time to time) Storage location of various VS Code configuration files Memorandum memo
[Notes / Updated from time to time] This and that of Azure Functions
[Note] AI / machine learning / python related websites [updated from time to time]
Easy conversion from UTC to local time
Updated to Python 2.7.9
TensorFlow API memo
Learn how to inflate images from TensorFlow code
Understand design patterns by comparing implementations in JavaScript and Java [Updated from time to time]
Changes from Python 3.0 to Python 3.5
Flow memo to move LOCUST for the time being
Transition from WSL1 to WSL2
Django memo # 1 from scratch
[Updated from time to time] Python memos often used for data analysis [N division, etc.]
A memorandum of commands, packages, terms, etc. used in linux (updated from time to time)
From running MINST on TensorFlow 2.0 to visualization on TensorBoard (2019 edition)
TensorFlow API memo (Python)
From editing to execution
Updated Hospital_dashboard to ver.2.0
List of my articles that may be useful in competition pros (updated from time to time)
[Introduction to matplotlib] Read the end time from COVID-19 data ♬
DJango Memo: From the beginning (more edits to the management screen)
"Deep Learning from scratch" self-study memo (No. 15) TensorFlow beginner tutorial
Summary of folders where Ruby, Python, PostgreSQL, etc. are installed on macOS (updated from time to time)
Precautions when upgrading TensorFlow (to 1.3)
Post from Python to Slack
Python execution time measurement memo
An introduction to private TensorFlow
Cheating from PHP to Python
Time series analysis related memo
Porting from argparse to hydra
Migrating from Chainer v1 to Chainer v2
Migrated from Flask-RESTPlus to Flask-RESTX
Update python-social-auth from 0.1.x to 0.2.x
Migrate from requirements.txt to pipenv
Switch from python2.7 to python3.6 (centos7)
Connect to sqlite from python
[Python] Hit Keras from TensorFlow and TensorFlow from c ++ to speed up execution
A memo when connecting bluetooth from a smartphone / PC to Raspberry Pi 4
git / python> git log analysis (v0.1, v0.2)> Implementation to estimate work time from git log