dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Remove duplicated mish activation function (#7856) 

* Remove duplicated mish activation function

* Update activations.py
This commit is contained in:
Raza Habib 2020-10-17 22:31:53 +01:00 committed by GitHub
parent f5c45a19e6
commit c65863ce53
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/transformers/activations.py
View File

@ -66,7 +66,3 @@ def get_activation(activation_string):
return ACT2FN[activation_string]
else:
raise KeyError("function {} not found in ACT2FN mapping {}".format(activation_string, list(ACT2FN.keys())))
def mish(x):
return x * torch.tanh(torch.nn.functional.softplus(x))