dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
transformers/utils/test_module/custom_tokenization_fast.py

9 lines
193 B
Python
Raw Permalink Blame History

from transformers import BertTokenizerFast
from .custom_tokenization import CustomTokenizer
class CustomTokenizerFast(BertTokenizerFast):
slow_tokenizer_class = CustomTokenizer
pass
Reference in New Issue View Git Blame Copy Permalink