fix links

Author: Ryan Sepassi

README.md

@@ -1,15 +1,16 @@
 # T2T: Tensor2Tensor Transformers
 
-[T2T](https://github.com/tensorflow/t2t) is a modular and extensible library and
-binaries for supervised learning with TensorFlow and with a focus on sequence
-tasks. Actively used and maintained by researchers and engineers within Google
-Brain, T2T strives to maximize idea bandwidth and minimize execution latency.
+[T2T](https://github.com/tensorflow/tensor2tensor) is a modular and extensible
+library and binaries for supervised learning with TensorFlow and with a focus on
+sequence tasks. Actively used and maintained by researchers and engineers within
+Google Brain, T2T strives to maximize idea bandwidth and minimize execution
+latency.
 
 T2T is particularly well-suited to researchers working on sequence tasks. We're
 eager to collaborate with you on extending T2T's powers, so please feel free to
 open an issue on GitHub to kick off a discussion and send along pull requests,
-See [our contribution doc](CONTRIBUTING.md) for details and our
-[open issues](https://github.com/tensorflow/t2t/issues).
+See [our contribution doc](CONTRIBUTING.md) for details and our [open
+issues](https://github.com/tensorflow/tensor2tensor/issues).
 
 ## T2T overview
 
@@ -51,18 +52,21 @@ use in the above commands.
 ### Datasets
 
 **Datasets** are all standardized on TFRecord files with `tensorflow.Example`
-protocol buffers. All datasets are registered and generated with
-[`generator.py`](data_generators/generator.py) and many common
-sequence datasets are already available for generation and use.
+protocol buffers. All datasets are registered and generated with the
+[data
+generator](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/bin/t2t-datagen)
+and many common sequence datasets are already available for generation and use.
 
 ### Problems and Modalities
 
 **Problems** define training-time hyperparameters for the dataset and task,
 mainly by setting input and output **modalities** (e.g. symbol, image, audio,
 label) and vocabularies, if applicable. All problems are defined in
-[`problem_hparams.py`](data_generators/problem_hparams.py). **Modalities**,
-defined in [`modality.py`](utils/modality.py), abstract away the input and
-output data types so that **models** may deal with modality-independent tensors.
+[`problem_hparams.py`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/problem_hparams.py).
+**Modalities**, defined in
+[`modality.py`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/utils/modality.py),
+abstract away the input and output data types so that **models** may deal with
+modality-independent tensors.
 
 ### Models
 
@@ -71,19 +75,22 @@ input/output modality or task. Models take dense tensors in and produce dense
 tensors that may then be transformed in a final step by a **modality** depending
 on the task (e.g. fed through a final linear transform to produce logits for a
 softmax over classes). All models are imported in
-[`models.py`](models/models.py), inherit from `T2TModel` - defined in
-[`t2t_model.py`](utils/t2t_model.py) - and are registered with
-[`@registry.register_model`](utils/registry.py).
+[`models.py`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/models/models.py),
+inherit from `T2TModel` - defined in
+[`t2t_model.py`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/utils/t2t_model.py)
+- and are registered with
+[`@registry.register_model`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/utils/registry.py).
 
 ### Hyperparameter Sets
 
 **Hyperparameter sets** are defined and registered in code with
-[`@registry.register_hparams`](utils/registry.py) and are encoded in
+[`@registry.register_hparams`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/utils/registry.py)
+and are encoded in
 [`tf.contrib.training.HParams`](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/training/python/training/hparam.py)
 objects. The `HParams` are available to both the problem specification and the
 model. A basic set of hyperparameters are defined in
-[`common_hparams.py`](models/common_hparams.py) and hyperparameter set
-functions can compose other hyperparameter set functions.
+[`common_hparams.py`](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/models/common_hparams.py)
+and hyperparameter set functions can compose other hyperparameter set functions.
 
 ### Trainer
 
@@ -95,7 +102,8 @@ related flags control local and distributed training/evaluation.
 
 ## Adding a dataset
 
-See the data generators [README](data_generators/README.md).
+See the data generators
+[README](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/README.md).
 
 ---
 

setup.py

@@ -1,23 +1,29 @@
 """Install tensor2tensor."""
 
-from distutils.core import setup
+from setuptools import setup, find_packages
 
 setup(
     name='tensor2tensor',
-    version='1.0',
+    version='1.0.1.dev1',
     description='Tensor2Tensor',
     author='Google Inc.',
     author_email='[email protected]',
     url='http://github.com/tensorflow/tensor2tensor',
     license='Apache 2.0',
-    packages=[
-        'tensor2tensor', 'tensor2tensor.utils', 'tensor2tensor.data_generators',
-        'tensor2tensor.models'
-    ],
+    packages=find_packages(),
     scripts=['tensor2tensor/bin/t2t-trainer', 'tensor2tensor/bin/t2t-datagen'],
     install_requires=[
         'numpy',
         'sympy',
         'six',
         'tensorflow-gpu>=1.2.0rc1',
-    ],)
+    ],
+    classifiers = [
+        'Development Status :: 4 - Beta',
+        'Intended Audience :: Developers',
+        'Intended Audience :: Science/Research',
+        'License :: OSI Approved :: Apache Software License',
+        'Topic :: Scientific/Engineering :: Artificial Intelligence',
+    ],
+    keywords='tensorflow',
+)

tensor2tensor/data_generators/README.md

@@ -49,14 +49,15 @@ example, downloading data, generating vocabulary files, etc.
 
 Some examples:
 
-*   [Algorithmic generators](https://github.com/tensorflow/tensor2tensor/tree/master/data_generators/algorithmic.py)
-    and their [unit tests](https://github.com/tensorflow/tensor2tensor/tree/master/data_generators/algorithmic_test.py)
-*   [WMT generators](https://github.com/tensorflow/tensor2tensor/tree/master/data_generators/wmt.py)
-    and their [unit tests](https://github.com/tensorflow/tensor2tensor/tree/master/data_generators/wmt_test.py)
+*   [Algorithmic generators](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/algorithmic.py)
+    and their [unit tests](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/algorithmic_test.py)
+*   [WMT generators](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/wmt.py)
+    and their [unit tests](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/wmt_test.py)
 
 When your python generator is ready and tested, add it to the
-`_SUPPORTED_PROBLEM_GENERATORS` dictionary in
-[generator.py](https://github.com/tensorflow/tensor2tensor/tree/master/data_generators/generator.py).
+`_SUPPORTED_PROBLEM_GENERATORS` dictionary in the
+[data
+generator](https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/bin/t2t-datagen).
 The keys are problem names, and the values are pairs of (training-set-generator
 function, dev-set-generator function). For the generator above, one could add
 the following lines: