# [PYTHON] Random seeds fixed in TensorFlow

## Roughly speaking

• In TensorFlow, you can set the random number seed with the `tensorflow.set_random_seed` function or the` seed` argument of the random number operator.
• There are two types of random number seeds called "graph level" and "operation level" in TensorFlow.
• After creating the graph, you need to set the random number seed before generating the random number operator.

## background

When implementing machine learning code in TensorFlow, we often use pseudo-random numbers (hereinafter simply "random numbers") for weight initialization and so on. As a result, the result changes each time the code is re-executed, and it is not possible to distinguish between the change in the result due to changes in data and parameters and the change in the result due to random numbers. Random numbers are generated based on a value called a "seed", which, unless otherwise specified, uses a somewhat variable default value (often a time or process ID). TensorFlow also provides a way to fix the random seed, but I was a little worried about its behavior, so I tried to organize it.

In addition to TensorFlow, we will also discuss fixing random number seeds in the random module and NumPy package.

## version

• Python: v3.5.1
• NumPy: v1.11.0
• TensorFlow: v0.9.0rc0

## random module

In the Python standard `random` module, you can set the random seed with the` random.seed` function. The experimental code is as follows.

#### `seed_random.py`

``````
import random
random.seed(0)
print("1:", random.random())
print("2:", random.random())
print("3:", random.random())
random.seed(0)
print("4:", random.random())
print("5:", random.random())
print("6:", random.random())
``````

The result when executed 3 times is as follows. The same value is output each time a random number seed is set or executed.

``````\$ python3 seed_random.py
1: 0.8444218515250481
2: 0.7579544029403025
3: 0.420571580830845
4: 0.8444218515250481
5: 0.7579544029403025
6: 0.420571580830845
\$ python3 seed_random.py
1: 0.8444218515250481
2: 0.7579544029403025
3: 0.420571580830845
4: 0.8444218515250481
5: 0.7579544029403025
6: 0.420571580830845
\$ python3 seed_random.py
1: 0.8444218515250481
2: 0.7579544029403025
3: 0.420571580830845
4: 0.8444218515250481
5: 0.7579544029403025
6: 0.420571580830845
``````

## NumPy package

In the NumPy package, you can set the random seed with the `numpy.random.seed` function. The experimental code is as follows.

#### `seed_numpy.py`

``````
import numpy as np
np.random.seed(0)
print("1:", np.random.rand(3))
print("2:", np.random.rand(3))
print("3:", np.random.rand(3))
np.random.seed(0)
print("4:", np.random.rand(3))
print("5:", np.random.rand(3))
print("6:", np.random.rand(3))
``````

The result when executed 3 times is as follows. The same value is output each time a random number seed is set or executed.

``````\$ python3 seed_numpy.py
1: [ 0.5488135   0.71518937  0.60276338]
2: [ 0.54488318  0.4236548   0.64589411]
3: [ 0.43758721  0.891773    0.96366276]
4: [ 0.5488135   0.71518937  0.60276338]
5: [ 0.54488318  0.4236548   0.64589411]
6: [ 0.43758721  0.891773    0.96366276]
\$ python3 seed_numpy.py
1: [ 0.5488135   0.71518937  0.60276338]
2: [ 0.54488318  0.4236548   0.64589411]
3: [ 0.43758721  0.891773    0.96366276]
4: [ 0.5488135   0.71518937  0.60276338]
5: [ 0.54488318  0.4236548   0.64589411]
6: [ 0.43758721  0.891773    0.96366276]
\$ python3 seed_numpy.py
1: [ 0.5488135   0.71518937  0.60276338]
2: [ 0.54488318  0.4236548   0.64589411]
3: [ 0.43758721  0.891773    0.96366276]
4: [ 0.5488135   0.71518937  0.60276338]
5: [ 0.54488318  0.4236548   0.64589411]
6: [ 0.43758721  0.891773    0.96366276]
``````

## TensorFlow package

In the TensorFlow package, you can set the random seed with the `tensorflow.set_random_seed` function or the` seed` argument of the random number operator. There are two types of random number seeds in TensorFlow called "graph level" and "operation level". You also need to be careful about the order of the settings, and you need to set the random seeds "after creating the graph" and "before generating the operator".

### An example that works as expected

The experimental code is as follows.

#### `seed_tensorflow1.py`

``````
import tensorflow as tf
with tf.Graph().as_default():
tf.set_random_seed(0)
rand_op1 = tf.random_normal()
rand_op2 = tf.random_normal(, seed=1)
print("session1")
with tf.Session() as sess1:
print("op1-1:", sess1.run(rand_op1))
print("op1-2:", sess1.run(rand_op1))
print("op2-1:", sess1.run(rand_op2))
print("op2-2:", sess1.run(rand_op2))
print("session2")
with tf.Session() as sess2:
print("op1-1:", sess2.run(rand_op1))
print("op1-2:", sess2.run(rand_op1))
print("op2-1:", sess2.run(rand_op2))
print("op2-2:", sess2.run(rand_op2))
``````

The result when executed 3 times is as follows.

``````\$ python3 seed_tensorflow1.py
session1
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]
session2
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]
\$ python3 seed_tensorflow1.py
session1
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]
session2
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]
\$ python3 seed_tensorflow1.py
session1
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]
session2
op1-1: [-1.40955448 -0.53668278 -0.56523788]
op1-2: [-1.07107699  0.4139019   2.29180121]
op2-1: [ 0.68034536  0.8777824  -0.64773595]
op2-2: [-1.21607268  0.95542693 -0.16866584]

``````

### Doesn't work as expected Example 1: Set a random seed before creating the graph

Here is an example of setting a random seed before creating a graph. The experimental code is as follows.

#### `seed_tensorflow2.py`

``````
import tensorflow as tf
tf.set_random_seed(0) #Set random seeds before creating the graph
with tf.Graph().as_default():
rand_op1 = tf.random_normal()
rand_op2 = tf.random_normal(, seed=1)
with tf.Session() as sess1:
print("op1-1:", sess1.run(rand_op1))
print("op1-2:", sess1.run(rand_op1))
print("op2-1:", sess1.run(rand_op2))
print("op2-2:", sess1.run(rand_op2))
``````

The result when executed 3 times is as follows. `Rand_op2`, which sets the random number seed at the operation level, outputs the same value three times, but` rand_op1` outputs a different value three times.

``````\$ python3 seed_tensorflow2.py
op1-1: [ 0.22292495  0.67142487  0.80927771]
op1-2: [-1.11188841  0.57819426  0.19088539]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
\$ python3 seed_tensorflow2.py
op1-1: [-0.96930581  1.48854125  0.52752507]
op1-2: [ 0.91646689  0.85830265 -0.18069462]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
\$ python3 seed_tensorflow2.py
op1-1: [ 0.79890805  1.35702407 -0.12329593]
op1-2: [ 1.69212222  0.22590902 -0.73435217]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
``````

### Doesn't work as expected Example 2: Set random seed after generating random operator

The following example shows how to set the random seed after generating the random operator. The experimental code is as follows.

#### `seed_tensorflow3.py`

``````
import tensorflow as tf
with tf.Graph().as_default():
rand_op1 = tf.random_normal()
rand_op2 = tf.random_normal(, seed=1)
tf.set_random_seed(0) #Set random number seed after generating random number operator
with tf.Session() as sess:
print("op1-1:", sess.run(rand_op1))
print("op1-2:", sess.run(rand_op1))
print("op2-1:", sess.run(rand_op2))
print("op2-2:", sess.run(rand_op2))
``````

The result when executed 3 times is as follows. Similar to the above example, `rand_op2`, which sets the random number seed at the operation level, outputs the same value three times, but` rand_op1` outputs a different value three times.

``````\$ python3 seed_tensorflow3.py
op1-1: [ 0.03802272  1.69988739 -0.59952497]
op1-2: [ 0.62614048  0.07537607 -1.19501412]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
\$ python3 seed_tensorflow3.py
op1-1: [ 1.02020776  1.70896292  0.45571503]
op1-2: [-0.46230376 -1.22950804  0.51038951]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
\$ python3 seed_tensorflow3.py
op1-1: [ 1.31134415 -1.12688231  0.1805287 ]
op1-2: [-0.57391566  0.94440365 -1.07441545]
op2-1: [-0.81131822  1.48459876  0.06532937]
op2-2: [ 0.55171245 -0.13107552 -0.04481386]
``````

### Other

It is unconfirmed information, but it seems that it will not be the same value even if you set a random number seed when running on GPU.

## Acknowledgments

We received very useful information from Mr. Hirai, a fellow study group on TensorFlow. Thank you here.