Arthur 30b3c46ff5 [`split_special_tokens`] Add support for `split_special_tokens` argument to encode (#25081) ... * draft changes * update and add tests * styling for no * move test * path to usable model * update test * small update * update bertbased tokenizers * don'tuse kwargs for _tokenize * don'tuse kwargs for _tokenize * fix copies * update * update test for special tokenizers * fixup * skip two tests * remove pdb breakpiont() * wowo * rewrite custom tests * nits * revert chang in target keys * fix markup lm * update documentation of the argument		2023-08-18 13:26:27 +02:00
..
__init__.py	…
test_processor_layoutxlm.py	Update old existing feature extractor references (#24552)	2023-06-29 10:17:36 +01:00
test_tokenization_layoutxlm.py	[`split_special_tokens`] Add support for `split_special_tokens` argument to encode (#25081)	2023-08-18 13:26:27 +02:00