dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

doc mismatch fixed (#13345)

This commit is contained in:
Apoorv Garg 2021-08-31 15:58:37 +05:30 committed by GitHub
parent 41c559415a
commit 286ccefb48
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 6 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
docs/source/preprocessing.rst
View File

@ -243,15 +243,16 @@ three arguments you need to know for this are :obj:`padding`, :obj:`truncation`
- :obj:`truncation` controls the truncation. It can be a boolean or a string which should be: - :obj:`truncation` controls the truncation. It can be a boolean or a string which should be:
- :obj:`True` or :obj:`'only_first'` truncate to a maximum length specified by the :obj:`max_length` argument or - :obj:`True` or :obj:`'longest_first'` truncate to a maximum length specified by the :obj:`max_length` argument or
the maximum length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will the maximum length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will
only truncate the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided. truncate token by token, removing a token from the longest sequence in the pair until the proper length is
reached.
- :obj:`'only_second'` truncate to a maximum length specified by the :obj:`max_length` argument or the maximum - :obj:`'only_second'` truncate to a maximum length specified by the :obj:`max_length` argument or the maximum
length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will only truncate length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will only truncate
the second sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided. the second sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.
- :obj:`'longest_first'` truncate to a maximum length specified by the :obj:`max_length` argument or the maximum - :obj:`'only_first'` truncate to a maximum length specified by the :obj:`max_length` argument or the maximum
length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will truncate token length accepted by the model if no :obj:`max_length` is provided (``max_length=None``). This will only truncate
by token, removing a token from the longest sequence in the pair until the proper length is reached. the first sentence of a pair if a pair of sequence (or a batch of pairs of sequences) is provided.
- :obj:`False` or :obj:`'do_not_truncate'` to not truncate the sequences. As we have seen before, this is the - :obj:`False` or :obj:`'do_not_truncate'` to not truncate the sequences. As we have seen before, this is the
default behavior. default behavior.