dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
transformers/setup.py

102 lines
3.8 KiB
Python
Raw Normal View History

include version number + comment in setup.py
2018-12-11 19:20:22 +08:00
"""
Simple check list from AllenNLP repo: https://github.com/allenai/allennlp/blob/master/setup.py
To create the package for pypi.
Release: 2.1.0
2019-10-10 00:14:03 +08:00
1. Change the version in __init__.py, setup.py as well as docs/source/conf.py.
include version number + comment in setup.py
2018-12-11 19:20:22 +08:00
2. Commit these changes with the message: "Release: VERSION"
3. Add a tag in git to mark the release: "git tag VERSION -m'Adds tag VERSION for pypi' "
Push the tag to git: git push --tags origin master
4. Build both the sources and the wheel. Do not change anything in setup.py between
creating the wheel and the source distribution (obviously).
update readme and setup.py
2019-09-26 19:47:58 +08:00
For the wheel, run: "python setup.py bdist_wheel" in the top level directory.
Update comments mentioning Python 2.
2019-12-23 01:22:29 +08:00
(this will build a wheel for the python version you use to build it).
include version number + comment in setup.py
2018-12-11 19:20:22 +08:00
For the sources, run: "python setup.py sdist"
update readme and setup.py
2019-09-26 19:47:58 +08:00
You should now have a /dist directory with both .whl and .tar.gz source versions.
include version number + comment in setup.py
2018-12-11 19:20:22 +08:00
5. Check that everything looks correct by uploading the package to the pypi test server:
twine upload dist/* -r pypitest
(pypi suggest using twine as other methods upload files via plaintext.)
Check that you can install it in a virtualenv by running:
[BIG] pytorch-transformers => transformers
2019-09-26 16:15:53 +08:00
pip install -i https://testpypi.python.org/pypi transformers
include version number + comment in setup.py
2018-12-11 19:20:22 +08:00
6. Upload the final version to actual pypi:
twine upload dist/* -r pypi
7. Copy the release notes from RELEASE.md to the tag in github once everything is looking hunky-dory.
"""
Sort imports with isort. This is the result of: $ isort --recursive examples templates transformers utils hubconf.py setup.py
2019-12-21 22:57:32 +08:00
preparing for first release
2018-11-16 03:56:10 +08:00
from setuptools import find_packages, setup
CLI for authenticated file sharing
2019-12-04 13:52:23 +08:00
Include all optional dependencies in extras. Take advantage of this to simplify the Circle CI configuration. Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.
2019-12-23 03:28:26 +08:00
extras = {}
extras["mecab"] = ["mecab-python3"]
extras["sklearn"] = ["scikit-learn"]
extras["tf"] = ["tensorflow"]
extras["torch"] = ["torch"]
extras["serving"] = ["pydantic", "uvicorn", "fastapi"]
extras["all"] = extras["serving"] + ["tensorflow", "torch"]
extras["testing"] = ["pytest", "pytest-xdist"]
extras["quality"] = ["black", "isort", "flake8"]
extras["docs"] = ["recommonmark", "sphinx", "sphinx-markdown-tables", "sphinx-rtd-theme"]
extras["dev"] = extras["testing"] + extras["quality"] + ["mecab-python3", "scikit-learn", "tensorflow", "torch"]
CLI for authenticated file sharing
2019-12-04 13:52:23 +08:00
preparing for first release
2018-11-16 03:56:10 +08:00
setup(
[BIG] pytorch-transformers => transformers
2019-09-26 16:15:53 +08:00
name="transformers",
Release: v2.3.0
2019-12-21 05:22:20 +08:00
version="2.3.0",
update setup.py metadata
2019-09-26 19:52:24 +08:00
author="Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Google AI Language Team Authors, Open AI team Authors, Facebook AI Authors, Carnegie Mellon University Authors",
preparing for first release
2018-11-16 03:56:10 +08:00
author_email="thomas@huggingface.co",
update setup.py metadata
2019-09-26 19:52:24 +08:00
description="State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch",
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
long_description=open("README.md", "r", encoding="utf-8").read(),
preparing for first release
2018-11-16 03:56:10 +08:00
long_description_content_type="text/markdown",
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
keywords="NLP deep learning transformer pytorch tensorflow BERT GPT GPT-2 google openai CMU",
license="Apache",
[BIG] pytorch-transformers => transformers
2019-09-26 16:15:53 +08:00
url="https://github.com/huggingface/transformers",
Review and update setup.py.
2019-12-23 02:14:07 +08:00
package_dir={"": "src"},
Move source code inside a src subdirectory. This prevents transformers from being importable simply because the CWD is the root of the git repository, while not being importable from other directories. That led to inconsistent behavior, especially in examples. Once you fetch this commit, in your dev environment, you must run: $ pip uninstall transformers $ pip install -e .
2019-12-22 20:54:22 +08:00
packages=find_packages("src"),
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
install_requires=[
"numpy",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# accessing files from S3 directly
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"boto3",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# filesystem locks e.g. to prevent parallel downloads
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"filelock",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# for downloading models over HTTPS
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"requests",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# progress bars in model download and training scripts
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"tqdm",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# for OpenAI GPT
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"regex != 2019.12.17",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# for XLNet
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"sentencepiece",
Remove requirements.txt. It's redundant with setup.py and, also, incomplete (e.g. numpy).
2019-12-23 03:33:08 +08:00
# for XLM
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"sacremoses",
[WIP] - CLI
2019-10-16 20:17:58 +08:00
],
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
extras_require=extras,
scripts=["transformers-cli"],
Review and update setup.py.
2019-12-23 02:14:07 +08:00
python_requires=">=3.5.0",
preparing for first release
2018-11-16 03:56:10 +08:00
classifiers=[
Review and update setup.py.
2019-12-23 02:14:07 +08:00
"Development Status :: 5 - Production/Stable",
"Intended Audience :: Developers",
"Intended Audience :: Education",
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
Review and update setup.py.
2019-12-23 02:14:07 +08:00
"Operating System :: OS Independent",
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"Programming Language :: Python :: 3",
Review and update setup.py.
2019-12-23 02:14:07 +08:00
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
Reformat source code with black. This is the result of: $ black --line-length 119 examples templates transformers utils hubconf.py setup.py There's a lot of fairly long lines in the project. As a consequence, I'm picking the longest widely accepted line length, 119 characters. This is also Thomas' preference, because it allows for explicit variable names, to make the code easier to understand.
2019-12-21 22:46:46 +08:00
"Topic :: Scientific/Engineering :: Artificial Intelligence",
preparing for first release
2018-11-16 03:56:10 +08:00
],
)