11# Variables: Creation, Initialization, Saving, and Loading <a class =" md-anchor " id =" AUTOGENERATED-variables--creation--initialization--saving--and-loading " ></a >
22
3- When you train a model, you use [ Variables ] ( ../../api_docs/python/state_ops.md )
3+ When you train a model, you use [ variables ] ( ../../api_docs/python/state_ops.md )
44to hold and update parameters. Variables are in-memory buffers containing
5- tensors. They need to be explicitly initialized and can be saved to disk during
5+ tensors. They must be explicitly initialized and can be saved to disk during
66and after training. You can later restore saved values to exercise or analyse
77the model.
88
99This document references the following TensorFlow classes. Follow the links to
1010their reference manual for a complete description of their API:
1111
12- * The ` Variable ` class [ tf.Variable] ( ../../api_docs/python/state_ops.md#Variable ) .
13- * The ` Saver ` class [ tf.train.Saver] ( ../../api_docs/python/state_ops.md#Saver ) .
12+ * The [ ` tf.Variable ` ] ( ../../api_docs/python/state_ops.md#Variable ) class .
13+ * The [ ` tf.train.Saver ` ] ( ../../api_docs/python/state_ops.md#Saver ) class .
1414
1515
1616## Creation <a class =" md-anchor " id =" AUTOGENERATED-creation " ></a >
1717
1818When you create a [ Variable] ( ../../api_docs/python/state_ops.md ) you pass a
1919` Tensor ` as its initial value to the ` Variable() ` constructor. TensorFlow
20- provides a collection of Ops that produce tensors often used for initialization
20+ provides a collection of ops that produce tensors often used for initialization
2121from [ constants or random values] ( ../../api_docs/python/constant_op.md ) .
2222
23- Note that all these Ops require you to specify the shape of the tensors. That
23+ Note that all these ops require you to specify the shape of the tensors. That
2424shape automatically becomes the shape of the variable. Variables generally
2525have a fixed shape, but TensorFlow provides advanced mechanisms to reshape
2626variables.
@@ -32,28 +32,28 @@ weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),
3232biases = tf.Variable(tf.zeros([200 ]), name = " biases" )
3333```
3434
35- Calling ` tf.Variable() ` adds a few Ops to the graph:
35+ Calling ` tf.Variable() ` adds several ops to the graph:
3636
37- * A ` variable ` Op that holds the variable value.
38- * An initializer Op that sets the variable to its initial value. This is
39- actually a ` tf.assign ` Op .
40- * The Ops for the initial value, such as the ` zeros ` Op for the ` biases `
37+ * A ` variable ` op that holds the variable value.
38+ * An initializer op that sets the variable to its initial value. This is
39+ actually a ` tf.assign ` op .
40+ * The ops for the initial value, such as the ` zeros ` op for the ` biases `
4141 variable in the example are also added to the graph.
4242
4343The value returned by ` tf.Variable() ` value is an instance of the Python class
4444` tf.Variable ` .
4545
4646## Initialization <a class =" md-anchor " id =" AUTOGENERATED-initialization " ></a >
4747
48- Variable initializers must be run explicitly before other Ops in your model can
49- be run. The easiest way to do that is to add an Op that runs all the variable
50- initializers, and run that Op before using the model.
48+ Variable initializers must be run explicitly before other ops in your model can
49+ be run. The easiest way to do that is to add an op that runs all the variable
50+ initializers, and run that op before using the model.
5151
5252You can alternatively restore variable values from a checkpoint file, see
5353below.
5454
55- Use ` tf.initialize_all_variables() ` to add an Op to run variable initializers.
56- Only run that Op after you have fully constructed your model and launched it in
55+ Use ` tf.initialize_all_variables() ` to add an op to run variable initializers.
56+ Only run that op after you have fully constructed your model and launched it in
5757a session.
5858
5959``` python
@@ -62,13 +62,13 @@ weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),
6262 name = " weights" )
6363biases = tf.Variable(tf.zeros([200 ]), name = " biases" )
6464...
65- # Add an Op to initialize the variables.
65+ # Add an op to initialize the variables.
6666init_op = tf.initialize_all_variables()
6767
6868# Later, when launching the model
6969with tf.Session() as sess:
7070 # Run the init operation.
71- sess.Run (init_op)
71+ sess.run (init_op)
7272 ...
7373 # Use the model
7474 ...
@@ -77,7 +77,7 @@ with tf.Session() as sess:
7777### Initialization from another Variable <a class =" md-anchor " id =" AUTOGENERATED-initialization-from-another-variable " ></a >
7878
7979You sometimes need to initialize a variable from the initial value of another
80- variable. As the Op added by ` tf.initialize_all_variables() ` initializes all
80+ variable. As the op added by ` tf.initialize_all_variables() ` initializes all
8181variables in parallel you have to be careful when this is needed.
8282
8383To initialize a new variable from the value of another variable use the other
@@ -98,27 +98,29 @@ w_twice = tf.Variable(weights.initialized_value() * 0.2, name="w_twice")
9898
9999### Custom Initialization <a class =" md-anchor " id =" AUTOGENERATED-custom-initialization " ></a >
100100
101- The convenience function ` tf.initialize_all_variables() ` adds an Op to
101+ The convenience function ` tf.initialize_all_variables() ` adds an op to
102102initialize * all variables* in the model. You can also pass it an explicit list
103103of variables to initialize. See the
104104[ Variables Documentation] ( ../../api_docs/python/state_ops.md ) for more options,
105105including checking if variables are initialized.
106106
107107## Saving and Restoring <a class =" md-anchor " id =" AUTOGENERATED-saving-and-restoring " ></a >
108108
109- The easiest way to save and restore a model is to use a ` tf.train.Saver `
110- object. The constructor adds ` save ` and ` restore ` Ops to the graph for all, or
111- a specified list, of variables. The saver object provides methods to run these
112- Ops, specifying paths for the checkpoint files to write to or read from.
109+ The easiest way to save and restore a model is to use a ` tf.train.Saver ` object.
110+ The constructor adds ` save ` and ` restore ` ops to the graph for all, or a
111+ specified list, of the variables in the graph. The saver object provides
112+ methods to run these ops, specifying paths for the checkpoint files to write to
113+ or read from.
113114
114115### Checkpoint Files <a class =" md-anchor " id =" AUTOGENERATED-checkpoint-files " ></a >
115116
116- Variables are saved in binary files that, roughly, contains a map from variable
117- names to tensors .
117+ Variables are saved in binary files that, roughly, contain a map from variable
118+ names to tensor values .
118119
119- When you create a ` Saver ` object, you can optionally chose names for the
120- variables in the checkpoint files. By default, it uses the names passed to the
121- ` tf.Variable() ` call.
120+ When you create a ` Saver ` object, you can optionally choose names for the
121+ variables in the checkpoint files. By default, it uses the value of the
122+ [ ` Variable.name ` ] ( ../../api_docs/python/state_ops.md#Variable.name ) property for
123+ each variable.
122124
123125### Saving Variables <a class =" md-anchor " id =" AUTOGENERATED-saving-variables " ></a >
124126
@@ -130,20 +132,20 @@ the model.
130132v1 = tf.Variable(... , name = " v1" )
131133v2 = tf.Variable(... , name = " v2" )
132134...
133- # Add an Op to initialize the variables.
135+ # Add an op to initialize the variables.
134136init_op = tf.initialize_all_variables()
135137
136- # Add Ops to save and restore all the variables.
138+ # Add ops to save and restore all the variables.
137139saver = tf.train.Saver()
138140
139141# Later, launch the model, initialize the variables, do some work, save the
140142# variables to disk.
141143with tf.Session() as sess:
142- sess.Run (init_op)
144+ sess.run (init_op)
143145 # Do some work with the model.
144146 ..
145147 # Save the variables to disk.
146- save_path = saver.Save (sess, " /tmp/model.ckpt" )
148+ save_path = saver.save (sess, " /tmp/model.ckpt" )
147149 print " Model saved in file: " , save_path
148150```
149151
@@ -157,23 +159,23 @@ restore variables from a file you do not have to initialize them beforehand.
157159v1 = tf.Variable(... , name = " v1" )
158160v2 = tf.Variable(... , name = " v2" )
159161...
160- # Add Ops to save and restore all the variables.
162+ # Add ops to save and restore all the variables.
161163saver = tf.train.Saver()
162164
163165# Later, launch the model, use the saver to restore variables from disk, and
164166# do some work with the model.
165167with tf.Session() as sess:
166168 # Restore variables from disk.
167- saver.Restore (sess, " /tmp/model.ckpt" )
169+ saver.restore (sess, " /tmp/model.ckpt" )
168170 print " Model restored."
169171 # Do some work with the model
170172 ...
171173```
172174
173- ### Chosing which Variables to Save and Restore <a class =" md-anchor " id =" AUTOGENERATED-chosing -which-variables-to-save-and-restore " ></a >
175+ ### Choosing which Variables to Save and Restore <a class =" md-anchor " id =" AUTOGENERATED-choosing -which-variables-to-save-and-restore " ></a >
174176
175- If you do not pass any argument to ` tf.train.Saver() ` the saver
176- handles all variables . Each one of them is saved under the name that was
177+ If you do not pass any argument to ` tf.train.Saver() ` the saver handles all
178+ variables in the graph . Each one of them is saved under the name that was
177179passed when the variable was created.
178180
179181It is sometimes useful to explicitly specify names for variables in the
@@ -196,10 +198,10 @@ Notes:
196198* You can create as many saver objects as you want if you need to save and
197199 restore different subsets of the model variables. The same variable can be
198200 listed in multiple saver objects, its value is only changed when the saver
199- ` Restore ()` method is run.
201+ ` restore ()` method is run.
200202
201203* If you only restore a subset of the model variables at the start
202- of a session, you have to run an initialize Op for the other variables. See
204+ of a session, you have to run an initialize op for the other variables. See
203205 [ ` tf.initialize_variables() ` ] ( ../../api_docs/python/state_ops.md#initialize_variables )
204206 for more information.
205207
@@ -208,7 +210,7 @@ Notes:
208210v1 = tf.Variable(... , name = " v1" )
209211v2 = tf.Variable(... , name = " v2" )
210212...
211- # Add Ops to save and restore only 'v2' using the name "my_v2"
213+ # Add ops to save and restore only 'v2' using the name "my_v2"
212214saver = tf.train.Saver({" my_v2" : v2})
213215# Use the saver object normally after that.
214216...
0 commit comments