Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Adding automatic mixed precision support #1637

Merged
afrozenator merged 6 commits into from
Jul 30, 2019
Merged

Adding automatic mixed precision support #1637

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
812f644
adding GPU auto mixed precision training
vinhngx Jul 22, 2019
e31d476
add warning: manual vs. amp
vinhngx Jul 23, 2019
d028d0c
fix os environment check
vinhngx Jul 23, 2019
8687315
add _use_locking attribute to optimizer
vinhngx Jul 23, 2019
752220e
add _name attribute to optimizer
vinhngx Jul 23, 2019
2a5d1d2
change OS flag to TF_ENABLE_AUTO_MIXED_PRECISION
vinhngx Jul 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix os environment check
  • Loading branch information
@vinhngx
vinhngx committed Jul 23, 2019
commit d028d0c0fa5210b46d5be838cf400bf46bf358f1
2 changes: 1 addition & 1 deletion tensor2tensor/utils/optimize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ def optimize(loss, learning_rate, hparams, use_tpu=False, variables=None, gpu_au
opt = ConditionalOptimizer(hparams.optimizer, learning_rate, hparams, use_tpu)
if use_tpu:
opt = tf.contrib.tpu.CrossShardOptimizer(opt)
if os.environ.get('GPU_AUTO_MIXED_PRECISION', default=False) or gpu_auto_mixed_precision:
if os.environ.get('GPU_AUTO_MIXED_PRECISION', default='0') == '1' or gpu_auto_mixed_precision:
if use_tpu:
raise(RuntimeError("GPU auto mixed precision cannot be used with TPU"))
elif _mixed_precision_is_enabled(hparams):
Expand Down