Do not warn about unexpected decoder weights when loading T5EncoderModel and LongT5EncoderModel (#26211)

Ignore decoder weights when using T5EncoderModel and LongT5EncoderModel

Both T5EncoderModel and LongT5EncoderModel do not have any decoder layers, so
loading a pretrained model checkpoint such as t5-small will give warnings about
keys found in the model checkpoint that are not in the model itself.

To prevent this log warning, r"decoder" has been added to _keys_to_ignore_on_load_unexpected for
both T5EncoderModel and LongT5EncoderModel
This commit is contained in:
fleance 2023-09-28 11:27:43 +02:00 committed by GitHub
parent 38e96324ef
commit 216dff7549
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 2 additions and 0 deletions

View File

@ -2152,6 +2152,7 @@ class LongT5ForConditionalGeneration(LongT5PreTrainedModel):
)
class LongT5EncoderModel(LongT5PreTrainedModel):
_tied_weights_keys = ["encoder.embed_tokens.weight"]
_keys_to_ignore_on_load_unexpected = [r"decoder"]
def __init__(self, config: LongT5Config):
super().__init__(config)

View File

@ -1866,6 +1866,7 @@ class T5ForConditionalGeneration(T5PreTrainedModel):
)
class T5EncoderModel(T5PreTrainedModel):
_tied_weights_keys = ["encoder.embed_tokens.weight"]
_keys_to_ignore_on_load_unexpected = [r"decoder"]
def __init__(self, config: T5Config):
super().__init__(config)