thomwolf
|
9d0a11a68c
|
update dependencies and circle-ci
|
2019-09-08 15:02:06 +03:00 |
|
Shijie Wu
|
ca4baf8ca1
|
Match order of casing in OSS XLM; Improve document; Clean up dependency
|
2019-08-27 20:03:18 -04:00 |
|
Shijie Wu
|
e85123d398
|
Add custom tokenizer for zh and ja
|
2019-08-23 20:27:52 -04:00 |
|
Shijie Wu
|
436ce07218
|
Tokenization behave the same as original XLM proprocessing for most languages except zh, ja and th; Change API to allow specifying language in `tokenize`
|
2019-08-23 14:40:17 -04:00 |
|
thomwolf
|
58830807d1
|
inidicate we only support pytorch 1.0.0+ now
|
2019-08-05 14:38:59 +02:00 |
|
thomwolf
|
32da75486b
|
add tokenizer and tests
|
2019-06-21 11:09:51 +02:00 |
|
thomwolf
|
e0855e8929
|
forgot to add regex to requirements :(
|
2019-02-18 11:54:51 +01:00 |
|
thomwolf
|
ce52177638
|
added version in __init__.py
|
2018-12-13 12:50:44 +01:00 |
|
thomwolf
|
a99b971738
|
bump up version minor
|
2018-11-17 10:43:39 +01:00 |
|
thomwolf
|
1de35b624b
|
preparing for first release
|
2018-11-15 20:56:10 +01:00 |
|
thomwolf
|
88c1037991
|
update requirements
|
2018-11-04 21:26:18 +01:00 |
|
thomwolf
|
0d8d2285ba
|
fix optimization_test
|
2018-11-03 12:23:00 +01:00 |