5 Commits

Author	SHA1	Message	Date
thomwolf	c946bb51a6	fix xlnet tokenizer and python2	2019-06-22 22:28:49 +02:00
thomwolf	181075635d	updating model loading and adding special tokens ids	2019-06-21 23:23:37 +02:00
thomwolf	ebd2cb8d74	update from_pretrained to load XLNetModel as well	2019-06-21 21:08:44 +02:00
thomwolf	32da75486b	add tokenizer and tests	2019-06-21 11:09:51 +02:00
thomwolf	de713fa9b4	starting	2019-06-20 10:54:19 +02:00