Yu Liu 762cba3bda Albert pretrain datasets/ datacollator (#6168) ... * add dataset for albert pretrain * datacollator for albert pretrain * naming, comprehension, file reading change * data cleaning is no needed after this modification * delete prints * fix a bug * file structure change * add tests for albert datacollator * remove random seed * add back len and get item function * sample file for testing and test code added * format change for black * more format change * Style * var assignment issue resolve * add back wrongly deleted DataCollatorWithPadding in init file * Style Co-authored-by: Lysandre Debut <lysandre@huggingface.co> Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>		2020-09-10 07:56:29 -04:00
..
wiki_00	Albert pretrain datasets/ datacollator (#6168)	2020-09-10 07:56:29 -04:00