[Docs] Add language identifiers to fenced code blocks (#28955)

Add language identifiers to code blocks

docs/source/en/chat_templating.md

@@ -390,7 +390,7 @@ If your model expects those, they won't be added automatically by `apply_chat_te
 text will be tokenized with `add_special_tokens=False`. This is to avoid potential conflicts between the template and
 the `add_special_tokens` logic. If your model expects special tokens, make sure to add them to the template!
 
-```
+```python
 tokenizer.chat_template = "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
 ```
 

docs/source/en/custom_models.md

@@ -310,7 +310,7 @@ Use `register_for_auto_class()` if you want the code files to be copied. If you
 you don't need to call it. In cases where there's more than one auto class, you can modify the `config.json` directly using the 
 following structure:
 
-```
+```json
 "auto_map": {     
 	"AutoConfig": "<your-repo-name>--<config-name>",     
 	"AutoModel": "<your-repo-name>--<config-name>",

docs/source/en/custom_tools.md

@@ -405,7 +405,7 @@ Assistant:
 Therefore it is important that the examples of the custom `chat` prompt template also make use of this format.
 You can overwrite the `chat` template at instantiation as follows.
 
-```
+```python
 template = """ [...] """
 
 agent = HfAgent(url_endpoint=your_endpoint, chat_prompt_template=template)

docs/source/en/installation.md

@@ -72,7 +72,7 @@ pip install 'transformers[tf-cpu]'
 M1 / ARM Users
 
 You will need to install the following before installing TensorFLow 2.0
-```
+```bash
 brew install cmake
 brew install pkg-config
 ```

docs/source/en/model_doc/fastspeech2_conformer.md

@@ -41,7 +41,7 @@ You can run FastSpeech2Conformer locally with the 🤗 Transformers library.
 
 1. First install the 🤗 [Transformers library](https://github.com/huggingface/transformers), g2p-en:
 
-```
+```bash
 pip install --upgrade pip
 pip install --upgrade transformers g2p-en
 ```

docs/source/en/model_doc/layoutlmv2.md

@@ -50,7 +50,7 @@ this https URL.*
 
 LayoutLMv2 depends on `detectron2`, `torchvision` and `tesseract`. Run the
 following to install them:
-```
+```bash
 python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
 python -m pip install torchvision tesseract
 ```

docs/source/en/model_doc/lilt.md

@@ -39,7 +39,7 @@ The original code can be found [here](https://github.com/jpwang/lilt).
 - To combine the Language-Independent Layout Transformer with a new RoBERTa checkpoint from the [hub](https://huggingface.co/models?search=roberta), refer to [this guide](https://github.com/jpWang/LiLT#or-generate-your-own-checkpoint-optional).
 The script will result in `config.json` and `pytorch_model.bin` files being stored locally. After doing this, one can do the following (assuming you're logged in with your HuggingFace account):
 
-```
+```python
 from transformers import LiltModel
 
 model = LiltModel.from_pretrained("path_to_your_files")

docs/source/en/model_doc/musicgen.md

@@ -136,7 +136,7 @@ The same [`MusicgenProcessor`] can be used to pre-process an audio prompt that i
 following example, we load an audio file using the 🤗 Datasets library, which can be pip installed through the command
 below:
 
-```
+```bash
 pip install --upgrade pip
 pip install datasets[audio]
 ```

docs/source/en/model_doc/pop2piano.md

@@ -54,7 +54,7 @@ The original code can be found [here](https://github.com/sweetcocoa/pop2piano).
 ## Usage tips
 
 * To use Pop2Piano, you will need to install the 🤗 Transformers library, as well as the following third party modules:  
-```
+```bash
 pip install pretty-midi==0.2.9 essentia==2.1b6.dev1034 librosa scipy
 ```
 Please note that you may need to restart your runtime after installation.

docs/source/en/perf_hardware.md

@@ -64,7 +64,7 @@ Next let's have a look at one of the most important aspects when having multiple
 
 If you use multiple GPUs the way cards are inter-connected can have a huge impact on the total training time. If the GPUs are on the same physical node, you can run:
 
-```
+```bash
 nvidia-smi topo -m
 ```
 

docs/source/en/perf_train_cpu.md

@@ -38,7 +38,7 @@ IPEX release is following PyTorch, to install via pip:
 | 1.12              |  1.12.300+cpu  |
 
 Please run `pip list | grep torch` to get your `pytorch_version`, so you can get the `IPEX version_name`.
-```
+```bash
 pip install intel_extension_for_pytorch==<version_name> -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 You can check the latest versions in [ipex-whl-stable-cpu](https://developer.intel.com/ipex-whl-stable-cpu) if needed.

docs/source/en/perf_train_cpu_many.md

@@ -39,7 +39,7 @@ Wheel files are available for the following Python versions:
 | 1.12.0            |            | √          | √          | √          | √           |
 
 Please run `pip list | grep torch` to get your `pytorch_version`.
-```
+```bash
 pip install oneccl_bind_pt=={pytorch_version} -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 where `{pytorch_version}` should be your PyTorch version, for instance 2.1.0.
@@ -59,13 +59,13 @@ Use this standards-based MPI implementation to deliver flexible, efficient, scal
 oneccl_bindings_for_pytorch is installed along with the MPI tool set. Need to source the environment before using it.
 
 for Intel® oneCCL >= 1.12.0
-```
+```bash
 oneccl_bindings_for_pytorch_path=$(python -c "from oneccl_bindings_for_pytorch import cwd; print(cwd)")
 source $oneccl_bindings_for_pytorch_path/env/setvars.sh
 ```
 
 for Intel® oneCCL whose version < 1.12.0
-```
+```bash
 torch_ccl_path=$(python -c "import torch; import torch_ccl; import os;  print(os.path.abspath(os.path.dirname(torch_ccl.__file__)))")
 source $torch_ccl_path/env/setvars.sh
 ```
@@ -154,7 +154,7 @@ This example assumes that you have:
 
 The snippet below is an example of a Dockerfile that uses a base image that supports distributed CPU training and then
 extracts a Transformers release to the `/workspace` directory, so that the example scripts are included in the image:
-```
+```dockerfile
 FROM intel/ai-workflows:torch-2.0.1-huggingface-multinode-py3.9
 
 WORKDIR /workspace
@@ -286,7 +286,7 @@ set the same CPU and memory amounts for both the resource limits and requests.
 
 After the PyTorchJob spec has been updated with values appropriate for your cluster and training job, it can be deployed
 to the cluster using:
-```
+```bash
 kubectl create -f pytorchjob.yaml
 ```
 
@@ -304,7 +304,7 @@ transformers-pytorchjob-worker-3                         1/1     Running
 ```
 
 The logs for worker can be viewed using `kubectl logs -n kubeflow <pod name>`. Add `-f` to stream the logs, for example:
-```
+```bash
 kubectl logs -n kubeflow transformers-pytorchjob-worker-0 -f
 ```
 

docs/source/en/perf_train_gpu_many.md

@@ -140,7 +140,7 @@ Here is the benchmarking code and outputs:
 
 **DP**
 
-```
+```bash
 rm -r /tmp/test-clm; CUDA_VISIBLE_DEVICES=0,1 \
 python examples/pytorch/language-modeling/run_clm.py \
 --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 \
@@ -151,7 +151,7 @@ python examples/pytorch/language-modeling/run_clm.py \
 
 **DDP w/ NVlink**
 
-```
+```bash
 rm -r /tmp/test-clm; CUDA_VISIBLE_DEVICES=0,1 \
 torchrun --nproc_per_node 2 examples/pytorch/language-modeling/run_clm.py \
 --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 \
@@ -162,7 +162,7 @@ torchrun --nproc_per_node 2 examples/pytorch/language-modeling/run_clm.py \
 
 **DDP w/o NVlink**
 
-```
+```bash
 rm -r /tmp/test-clm; NCCL_P2P_DISABLE=1 CUDA_VISIBLE_DEVICES=0,1 \
 torchrun --nproc_per_node 2 examples/pytorch/language-modeling/run_clm.py \
 --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 \

docs/source/en/perf_train_gpu_one.md

@@ -201,7 +201,7 @@ of 23 bits precision it has only 10 bits (same as fp16) and uses only 19 bits in
 you can use the normal fp32 training and/or inference code and by enabling tf32 support you can get up to 3x throughput 
 improvement. All you need to do is to add the following to your code:
 
-```
+```python
 import torch
 torch.backends.cuda.matmul.allow_tf32 = True
 torch.backends.cudnn.allow_tf32 = True

docs/source/en/tasks/video_classification.md

@@ -483,7 +483,7 @@ You can also manually replicate the results of the `pipeline` if you'd like.
 
 Now, pass your input to the model and return the `logits`:
 
-```
+```py
 >>> logits = run_inference(trained_model, sample_test_video["video"])
 ```
 

docs/source/fr/installation.md

@@ -74,7 +74,7 @@ Pour les architectures mac M1 / ARM
 
 Vous devez installer les outils suivants avant d'installer TensorFLow 2.0
 
-```
+```bash
 brew install cmake
 brew install pkg-config
 ```

docs/source/it/perf_hardware.md

@@ -63,7 +63,7 @@ Diamo quindi un'occhiata a uno degli aspetti più importanti quando si hanno pi
 
 Se utilizzi più GPU, il modo in cui le schede sono interconnesse può avere un enorme impatto sul tempo totale di allenamento. Se le GPU si trovano sullo stesso nodo fisico, puoi eseguire:
 
-```
+```bash
 nvidia-smi topo -m
 ```
 

docs/source/ja/chat_templating.md

@@ -215,7 +215,7 @@ LLM（Language Model）はさまざまな入力形式を処理できるほどス
 
 If you like this one, here it is in one-liner form, ready to copy into your code:
 
-```
+```python
 tokenizer.chat_template = "{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}"
 ```
 

docs/source/ja/custom_tools.md

@@ -385,7 +385,7 @@ Assistant:
 
 したがって、カスタム`chat`プロンプトテンプレートの例もこのフォーマットを使用することが重要です。以下のように、インスタンス化時に`chat`テンプレートを上書きできます。
 
-```
+```python
 template = """ [...] """
 
 agent = HfAgent(url_endpoint=your_endpoint, chat_prompt_template=template)

docs/source/ja/main_classes/deepspeed.md

@@ -2202,7 +2202,7 @@ print(f"rank{rank}:\n   in={text_in}\n  out={text_out}")
 
 それを`t0.py`として保存して実行しましょう。
 
-```
+```bash
 $ deepspeed --num_gpus 2 t0.py
 rank0:
    in=Is this review positive or negative? Review: this is the best cast iron skillet you will ever buy
@@ -2226,13 +2226,13 @@ DeepSpeed 統合を含む PR を送信する場合は、CircleCI PR CI セット
 
 DeepSpeed テストを実行するには、少なくとも以下を実行してください。
 
-```
+```bash
 RUN_SLOW=1 pytest tests/deepspeed/test_deepspeed.py
 ```
 
 モデリングまたは pytorch サンプル コードのいずれかを変更した場合は、Model Zoo テストも実行します。以下はすべての DeepSpeed テストを実行します。
 
-```
+```bash
 RUN_SLOW=1 pytest tests/deepspeed
 ```
 

docs/source/ja/perf_hardware.md

@@ -64,7 +64,7 @@ GPUが重要な負荷の下でどのような温度を目指すべきかを正
 複数のGPUを使用する場合、カードの相互接続方法はトータルのトレーニング時間に大きな影響を与える可能性があります。GPUが同じ物理ノードにある場合、次のように実行できます：
 
 
-```
+```bash
 nvidia-smi topo -m
 ```
 

docs/source/ja/perf_torch_compile.md

@@ -42,7 +42,7 @@ model = AutoModelForImageClassification.from_pretrained(MODEL_ID).to("cuda")
 
 ### Image Classification with ViT
 
-```
+```python
 from PIL import Image
 import requests
 import numpy as np

docs/source/ja/perf_train_cpu.md

@@ -36,7 +36,7 @@ IPEXのリリースはPyTorchに従っており、pipを使用してインスト
 | 1.11              |  1.11.200+cpu  |
 | 1.10              |  1.10.100+cpu  |
 
-```
+```bash
 pip install intel_extension_for_pytorch==<version_name> -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 

docs/source/ja/perf_train_cpu_many.md

@@ -38,7 +38,7 @@ Wheelファイルは、以下のPythonバージョン用に利用可能です:
 | 1.11.0            |            | √          | √          | √          | √           |
 | 1.10.0            | √          | √          | √          | √          |             |
 
-```
+```bash
 pip install oneccl_bind_pt=={pytorch_version} -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 
@@ -70,13 +70,13 @@ oneccl_bindings_for_pytorchはMPIツールセットと一緒にインストー
 
 
 for Intel® oneCCL >= 1.12.0
-```
+```bash
 oneccl_bindings_for_pytorch_path=$(python -c "from oneccl_bindings_for_pytorch import cwd; print(cwd)")
 source $oneccl_bindings_for_pytorch_path/env/setvars.sh
 ```
 
 for Intel® oneCCL whose version < 1.12.0
-```
+```bash
 torch_ccl_path=$(python -c "import torch; import torch_ccl; import os;  print(os.path.abspath(os.path.dirname(torch_ccl.__file__)))")
 source $torch_ccl_path/env/setvars.sh
 ```

docs/source/ja/perf_train_gpu_many.md

@@ -131,7 +131,7 @@ DPとDDPの他にも違いがありますが、この議論には関係ありま
 `NCCL_P2P_DISABLE=1`を使用して、対応するベンチマークでNVLink機能を無効にしました。
 
 
-```
+```bash
 
 # DP
 rm -r /tmp/test-clm; CUDA_VISIBLE_DEVICES=0,1 \

docs/source/ja/perf_train_gpu_one.md

@@ -151,7 +151,7 @@ training_args = TrainingArguments(bf16=True, **default_args)
 
 アンペアハードウェアは、tf32という特別なデータ型を使用します。これは、fp32と同じ数値範囲（8ビット）を持っていますが、23ビットの精度ではなく、10ビットの精度（fp16と同じ）を持ち、合計で19ビットしか使用しません。これは通常のfp32トレーニングおよび推論コードを使用し、tf32サポートを有効にすることで、最大3倍のスループットの向上が得られる点で「魔法のよう」です。行う必要があるのは、次のコードを追加するだけです：
 
-```
+```python
 import torch
 torch.backends.cuda.matmul.allow_tf32 = True
 torch.backends.cudnn.allow_tf32 = True

docs/source/ja/tasks/video_classification.md

@@ -490,7 +490,7 @@ def compute_metrics(eval_pred):
 
 次に、入力をモデルに渡し、`logits `を返します。
 
-```
+```py
 >>> logits = run_inference(trained_model, sample_test_video["video"])
 ```
 

docs/source/ko/custom_tools.md

@@ -373,7 +373,7 @@ Assistant:
 따라서 사용자 정의 `chat` 프롬프트 템플릿의 예제에서도 이 형식을 사용하는 것이 중요합니다. 
 다음과 같이 인스턴스화 할 때 `chat` 템플릿을 덮어쓸 수 있습니다.
 
-```
+```python
 template = """ [...] """
 
 agent = HfAgent(url_endpoint=your_endpoint, chat_prompt_template=template)

docs/source/ko/perf_hardware.md

@@ -64,7 +64,7 @@ GPU가 과열될 때 정확한 적정 온도를 알기 어려우나, 아마도 +
 
 다중 GPU를 사용하는 경우 GPU 간의 연결 방식은 전체 훈련 시간에 큰 영향을 미칠 수 있습니다. 만약 GPU가 동일한 물리적 노드에 있을 경우, 다음과 같이 확인할 수 있습니다:
 
-```
+```bash
 nvidia-smi topo -m
 ```
 

docs/source/ko/perf_train_cpu.md

@@ -36,7 +36,7 @@ IPEX 릴리스는 PyTorch를 따라갑니다. pip를 통해 설치하려면:
 | 1.11              |  1.11.200+cpu  |
 | 1.10              |  1.10.100+cpu  |
 
-```
+```bash
 pip install intel_extension_for_pytorch==<version_name> -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 

docs/source/ko/perf_train_cpu_many.md

@@ -37,7 +37,7 @@ rendered properly in your Markdown viewer.
 | 1.11.0            |            | √          | √          | √          | √           |
 | 1.10.0            | √          | √          | √          | √          |             |
 
-```
+```bash
 pip install oneccl_bind_pt=={pytorch_version} -f https://developer.intel.com/ipex-whl-stable-cpu
 ```
 `{pytorch_version}`은 1.13.0과 같이 PyTorch 버전을 나타냅니다.
@@ -57,13 +57,13 @@ PyTorch 1.12.1은 oneccl_bindings_for_pytorch 1.12.10 버전과 함께 사용해
 oneccl_bindings_for_pytorch는 MPI 도구 세트와 함께 설치됩니다. 사용하기 전에 환경을 소스로 지정해야 합니다.
 
 Intel® oneCCL 버전 1.12.0 이상인 경우
-```
+```bash
 oneccl_bindings_for_pytorch_path=$(python -c "from oneccl_bindings_for_pytorch import cwd; print(cwd)")
 source $oneccl_bindings_for_pytorch_path/env/setvars.sh
 ```
 
 Intel® oneCCL 버전이 1.12.0 미만인 경우
-```
+```bash
 torch_ccl_path=$(python -c "import torch; import torch_ccl; import os;  print(os.path.abspath(os.path.dirname(torch_ccl.__file__)))")
 source $torch_ccl_path/env/setvars.sh
 ```

docs/source/ko/perf_train_gpu_many.md

@@ -133,7 +133,7 @@ DP와 DDP 사이에는 다른 차이점이 있지만, 이 토론과는 관련이
 
 해당 벤치마크에서 `NCCL_P2P_DISABLE=1`을 사용하여 NVLink 기능을 비활성화했습니다.
 
-```
+```bash
 
 # DP
 rm -r /tmp/test-clm; CUDA_VISIBLE_DEVICES=0,1 \

docs/source/ko/tasks/video_classification.md

@@ -485,7 +485,7 @@ def compute_metrics(eval_pred):
 
 모델에 입력값을 넣고 `logits`을 반환받으세요:
 
-```
+```py
 >>> logits = run_inference(trained_model, sample_test_video["video"])
 ```
 

docs/source/zh/installation.md

@@ -72,7 +72,7 @@ pip install 'transformers[tf-cpu]'
 M1 / ARM用户
 
 在安装 TensorFlow 2.0 前，你需要安装以下库：
-```
+```bash
 brew install cmake
 brew install pkg-config
 ```

docs/source/zh/main_classes/deepspeed.md

@@ -2048,7 +2048,7 @@ print(f"rank{rank}:\n   in={text_in}\n  out={text_out}")
 ```
 
 让我们保存它为 `t0.py`并运行：
-```
+```bash
 $ deepspeed --num_gpus 2 t0.py
 rank0:
    in=Is this review positive or negative? Review: this is the best cast iron skillet you will ever buy
@@ -2074,13 +2074,13 @@ rank1:
 
 要运行DeepSpeed测试，请至少运行以下命令：
 
-```
+```bash
 RUN_SLOW=1 pytest tests/deepspeed/test_deepspeed.py
 ```
 
 如果你更改了任何模型或PyTorch示例代码，请同时运行多模型测试。以下将运行所有DeepSpeed测试：
 
-```
+```bash
 RUN_SLOW=1 pytest tests/deepspeed
 ```
 

docs/source/zh/perf_hardware.md

@@ -64,7 +64,7 @@ rendered properly in your Markdown viewer.
 
 如果您使用多个GPU，则卡之间的互连方式可能会对总训练时间产生巨大影响。如果GPU位于同一物理节点上，您可以运行以下代码：
 
-```
+```bash
 nvidia-smi topo -m
 ```
 

examples/legacy/seq2seq/README.md

@@ -228,7 +228,7 @@ Contributions that implement this command for other distributed hardware setups
 When using `run_eval.py`, the following features can be useful:
 
 * if you running the script multiple times and want to make it easier to track what arguments produced that output, use `--dump-args`. Along with the results it will also dump any custom params that were passed to the script. For example if you used: `--num_beams 8 --early_stopping true`, the output will be:
-   ```
+   ```json
    {'bleu': 26.887, 'n_obs': 10, 'runtime': 1, 'seconds_per_sample': 0.1, 'num_beams': 8, 'early_stopping': True}
    ```
 
@@ -236,13 +236,13 @@ When using `run_eval.py`, the following features can be useful:
 
    If using `--dump-args --info`, the output will be:
 
-   ```
+   ```json
    {'bleu': 26.887, 'n_obs': 10, 'runtime': 1, 'seconds_per_sample': 0.1, 'num_beams': 8, 'early_stopping': True, 'info': '2020-09-13 18:44:43'}
    ```
 
    If using `--dump-args --info "pair:en-ru chkpt=best`, the output will be:
 
-   ```
+   ```json
    {'bleu': 26.887, 'n_obs': 10, 'runtime': 1, 'seconds_per_sample': 0.1, 'num_beams': 8, 'early_stopping': True, 'info': 'pair=en-ru chkpt=best'}
    ```
 

examples/pytorch/README.md

@@ -53,7 +53,7 @@ Coming soon!
 Most examples are equipped with a mechanism to truncate the number of dataset samples to the desired length. This is useful for debugging purposes, for example to quickly check that all stages of the programs can complete, before running the same setup on the full dataset which may take hours to complete.
 
 For example here is how to truncate all three splits to just 50 samples each:
-```
+```bash
 examples/pytorch/token-classification/run_ner.py \
 --max_train_samples 50 \
 --max_eval_samples 50 \
@@ -62,7 +62,7 @@ examples/pytorch/token-classification/run_ner.py \
 ```
 
 Most example scripts should have the first two command line arguments and some have the third one. You can quickly check if a given example supports any of these by passing a `-h` option, e.g.:
-```
+```bash
 examples/pytorch/token-classification/run_ner.py -h
 ```
 

examples/pytorch/speech-recognition/README.md

@@ -277,7 +277,7 @@ language or concept the adapter layers shall be trained. The adapter weights wil
 accordingly be called `adapter.{<target_language}.safetensors`.
 
 Let's run an example script. Make sure to be logged in so that your model can be directly uploaded to the Hub.
-```
+```bash
 huggingface-cli login
 ```
 

examples/research_projects/README.md

@@ -20,7 +20,7 @@ This folder contains various research projects using 🤗 Transformers. They are
 version of 🤗 Transformers that is indicated in the requirements file of each folder. Updating them to the most recent version of the library will require some work.
 
 To use any of them, just run the command
-```
+```bash
 pip install -r requirements.txt
 ```
 inside the folder of your choice.

examples/research_projects/bertabs/README.md

@@ -8,7 +8,7 @@ The model is loaded with the pre-trained weights for the abstractive summarizati
 
 ## Setup
 
-```
+```bash
 git clone https://github.com/huggingface/transformers && cd transformers
 pip install .
 pip install nltk py-rouge

examples/research_projects/deebert/README.md

@@ -34,7 +34,7 @@ This is for evaluating fine-tuned DeeBERT models, given a number of different ea
 ## Citation
 
 Please cite our paper if you find the resource useful:
-```
+```bibtex
 @inproceedings{xin-etal-2020-deebert,
     title = "{D}ee{BERT}: Dynamic Early Exiting for Accelerating {BERT} Inference",
     author = "Xin, Ji  and

examples/research_projects/distillation/README.md

@@ -183,7 +183,7 @@ Happy distillation!
 
 If you find the resource useful, you should cite the following paper:
 
-```
+```bibtex
 @inproceedings{sanh2019distilbert,
   title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
   author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},

examples/research_projects/information-gain-filtration/README.md

@@ -84,7 +84,7 @@ python run_clm_igf.py\
 
 If you find the resource useful, please cite the following paper
 
-```
+```bibtex
 @inproceedings{antonello-etal-2021-selecting,
     title = "Selecting Informative Contexts Improves Language Model Fine-tuning",
     author = "Antonello, Richard and Beckage, Nicole and Turek, Javier and Huth, Alexander",

examples/research_projects/jax-projects/README.md

@@ -311,7 +311,7 @@ library from source to profit from the most current additions during the communi
 
 Simply run the following steps:
 
-```
+```bash
 $ cd ~/
 $ git clone https://github.com/huggingface/datasets.git
 $ cd datasets
@@ -389,13 +389,13 @@ source ~/<your-venv-name>/bin/activate
 
 Next you should install JAX's TPU version on TPU by running the following command: 
 
-```
+```bash
 $ pip install requests
 ```
 
 and then:
 
-```
+```bash
 $ pip install "jax[tpu]>=0.2.16" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html
 ```
 
@@ -468,7 +468,7 @@ library from source to profit from the most current additions during the communi
 
 Simply run the following steps:
 
-```
+```bash
 $ cd ~/
 $ git clone https://github.com/huggingface/datasets.git
 $ cd datasets
@@ -568,7 +568,7 @@ class ModelPyTorch:
 
 Instantiating an object `model_pytorch` of the class `ModelPyTorch` would actually allocate memory for the model weights and attach them to the attributes `self.key_proj`, `self.value_proj`, `self.query_proj`, and `self.logits.proj`. We could access the weights via:
 
-```
+```python
 key_projection_matrix = model_pytorch.key_proj.weight.data
 ```
 
@@ -1224,25 +1224,25 @@ Sometimes you might be using different libraries or a very specific application
 
 A common use case is how to load files you have in your model repository in the Hub from the Streamlit demo. The `huggingface_hub` library is here to help you!
 
-```
+```bash
 pip install huggingface_hub
 ```
 
 Here is an example downloading (and caching!) a specific file directly from the Hub
-```
+```python
 from huggingface_hub import hf_hub_download
 filepath = hf_hub_download("flax-community/roberta-base-als", "flax_model.msgpack");
 ```
 
 In many cases you will want to download the full repository. Here is an example downloading all the files from a repo. You can even specify specific revisions!
 
-```
+```python
 from huggingface_hub import snapshot_download
 local_path = snapshot_download("flax-community/roberta-base-als");
 ```
 
 Note that if you're using 🤗 Transformers library, you can quickly load the model and tokenizer as follows
-```
+```python
 from transformers import AutoTokenizer, AutoModelForMaskedLM
   
 tokenizer = AutoTokenizer.from_pretrained("REPO_ID")

examples/research_projects/jax-projects/dataset-streaming/README.md

@@ -42,20 +42,20 @@ Here we call the model `"english-roberta-base-dummy"`, but you can change the mo
 You can do this either directly on [huggingface.co](https://huggingface.co/new) (assuming that
 you are logged in) or via the command line:
 
-```
+```bash
 huggingface-cli repo create english-roberta-base-dummy
 ```
 
 Next we clone the model repository to add the tokenizer and model files.
 
-```
+```bash
 git clone https://huggingface.co/<your-username>/english-roberta-base-dummy
 ```
 
 To ensure that all tensorboard traces will be uploaded correctly, we need to 
 track them. You can run the following command inside your model repo to do so.
 
-```
+```bash
 cd english-roberta-base-dummy
 git lfs track "*tfevents*"
 ```

examples/research_projects/jax-projects/hybrid_clip/README.md

@@ -43,17 +43,17 @@ Here we call the model `"clip-roberta-base"`, but you can change the model name
 You can do this either directly on [huggingface.co](https://huggingface.co/new) (assuming that
 you are logged in) or via the command line:
 
-```
+```bash
 huggingface-cli repo create clip-roberta-base
 ```
 Next we clone the model repository to add the tokenizer and model files.
-```
+```bash
 git clone https://huggingface.co/<your-username>/clip-roberta-base
 ```
 To ensure that all tensorboard traces will be uploaded correctly, we need to 
 track them. You can run the following command inside your model repo to do so.
 
-```
+```bash
 cd clip-roberta-base
 git lfs track "*tfevents*"
 ```

examples/research_projects/jax-projects/wav2vec2/README.md

@@ -18,20 +18,20 @@ Here we call the model `"wav2vec2-base-robust"`, but you can change the model na
 You can do this either directly on [huggingface.co](https://huggingface.co/new) (assuming that
 you are logged in) or via the command line:
 
-```
+```bash
 huggingface-cli repo create wav2vec2-base-robust
 ```
 
 Next we clone the model repository to add the tokenizer and model files.
 
-```
+```bash
 git clone https://huggingface.co/<your-username>/wav2vec2-base-robust
 ```
 
 To ensure that all tensorboard traces will be uploaded correctly, we need to 
 track them. You can run the following command inside your model repo to do so.
 
-```
+```bash
 cd wav2vec2-base-robust
 git lfs track "*tfevents*"
 ```

examples/research_projects/mm-imdb/README.md

@@ -6,7 +6,7 @@ Based on the script [`run_mmimdb.py`](https://github.com/huggingface/transformer
 
 ### Training on MM-IMDb
 
-```
+```bash
 python run_mmimdb.py \
     --data_dir /path/to/mmimdb/dataset/ \
     --model_type bert \

examples/research_projects/movement-pruning/README.md

@@ -173,7 +173,7 @@ In particular, hardware manufacturers are announcing devices that will speedup i
 
 If you find this resource useful, please consider citing the following paper:
 
-```
+```bibtex
 @article{sanh2020movement,
     title={Movement Pruning: Adaptive Sparsity by Fine-Tuning},
     author={Victor Sanh and Thomas Wolf and Alexander M. Rush},

examples/research_projects/quantization-qdqbert/README.md

@@ -30,17 +30,17 @@ Required:
 ## Setup the environment with Dockerfile
 
 Under the directory of `transformers/`, build the docker image:
-```
+```bash
 docker build . -f examples/research_projects/quantization-qdqbert/Dockerfile -t bert_quantization:latest
 ```
 
 Run the docker:
-```
+```bash
 docker run --gpus all --privileged --rm -it --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 bert_quantization:latest
 ```
 
 In the container:
-```
+```bash
 cd transformers/examples/research_projects/quantization-qdqbert/
 ```
 
@@ -48,7 +48,7 @@ cd transformers/examples/research_projects/quantization-qdqbert/
 
 Calibrate the pretrained model and finetune with quantization awared:
 
-```
+```bash
 python3 run_quant_qa.py \
   --model_name_or_path bert-base-uncased \
   --dataset_name squad \
@@ -60,7 +60,7 @@ python3 run_quant_qa.py \
   --percentile 99.99
 ```
 
-```
+```bash
 python3 run_quant_qa.py \
   --model_name_or_path calib/bert-base-uncased \
   --dataset_name squad \
@@ -80,7 +80,7 @@ python3 run_quant_qa.py \
 
 To export the QAT model finetuned above:
 
-```
+```bash
 python3 run_quant_qa.py \
   --model_name_or_path finetuned_int8/bert-base-uncased \
   --output_dir ./ \
@@ -97,19 +97,19 @@ Recalibrating will affect the accuracy of the model, but the change should be mi
 
 ### Benchmark the INT8 QAT ONNX model inference with TensorRT using dummy input
 
-```
+```bash
 trtexec --onnx=model.onnx --explicitBatch --workspace=16384 --int8 --shapes=input_ids:64x128,attention_mask:64x128,token_type_ids:64x128 --verbose
 ```
 
 ### Benchmark the INT8 QAT ONNX model inference with [ONNX Runtime-TRT](https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html) using dummy input
 
-```
+```bash
 python3 ort-infer-benchmark.py
 ```
 
 ### Evaluate the INT8 QAT ONNX model inference with TensorRT
 
-```
+```bash
 python3 evaluate-hf-trt-qa.py \
   --onnx_model_path=./model.onnx \
   --output_dir ./ \
@@ -126,7 +126,7 @@ python3 evaluate-hf-trt-qa.py \
 
 Finetune a fp32 precision model with [transformers/examples/pytorch/question-answering/](../../pytorch/question-answering/):
 
-```
+```bash
 python3 ../../pytorch/question-answering/run_qa.py \
   --model_name_or_path bert-base-uncased \
   --dataset_name squad \
@@ -145,7 +145,7 @@ python3 ../../pytorch/question-answering/run_qa.py \
 
 ### PTQ by calibrating and evaluating the finetuned FP32 model above:
 
-```
+```bash
 python3 run_quant_qa.py \
   --model_name_or_path ./finetuned_fp32/bert-base-uncased \
   --dataset_name squad \
@@ -161,7 +161,7 @@ python3 run_quant_qa.py \
 
 ### Export the INT8 PTQ model to ONNX
 
-```
+```bash
 python3 run_quant_qa.py \
   --model_name_or_path ./calib/bert-base-uncased \
   --output_dir ./ \
@@ -175,7 +175,7 @@ python3 run_quant_qa.py \
 
 ### Evaluate the INT8 PTQ ONNX model inference with TensorRT
 
-```
+```bash
 python3 evaluate-hf-trt-qa.py \
   --onnx_model_path=./model.onnx \
   --output_dir ./ \

examples/research_projects/rag/README.md

@@ -45,7 +45,7 @@ We publish two `base` models which can serve as a starting point for finetuning
 The `base` models initialize the question encoder with [`facebook/dpr-question_encoder-single-nq-base`](https://huggingface.co/facebook/dpr-question_encoder-single-nq-base) and the generator with [`facebook/bart-large`](https://huggingface.co/facebook/bart-large).
 
 If you would like to initialize finetuning with a base model using different question encoder and generator architectures, you can build it with a consolidation script, e.g.:
-```
+```bash
 python examples/research_projects/rag/consolidate_rag_checkpoint.py \
     --model_type rag_sequence \
     --generator_name_or_path facebook/bart-large-cnn \

examples/research_projects/robust-speech-event/README.md

@@ -216,7 +216,7 @@ library from source to profit from the most current additions during the communi
 
 Simply run the following steps:
 
-```
+```bash
 $ cd ~/
 $ git clone https://github.com/huggingface/datasets.git
 $ cd datasets

examples/research_projects/vqgan-clip/README.md

@@ -21,7 +21,7 @@ To install locally:
 
 In the root of the repo run:
 
-```
+```bash
 conda create -n vqganclip python=3.8
 conda activate vqganclip
 git-lfs install
@@ -30,7 +30,7 @@ pip install -r requirements.txt
 ```
 
 ### Generate new images
-```
+```python
 from VQGAN_CLIP import VQGAN_CLIP
 vqgan_clip = VQGAN_CLIP()
 vqgan_clip.generate("a picture of a smiling woman")
@@ -41,7 +41,7 @@ To get a test image, run
 `git clone https://huggingface.co/datasets/erwann/vqgan-clip-pic test_images`
 
 To edit:
-```
+```python
 from VQGAN_CLIP import VQGAN_CLIP
 vqgan_clip = VQGAN_CLIP()
 

examples/research_projects/wav2vec2/FINE_TUNE_XLSR_WAV2VEC2.md

@@ -138,20 +138,20 @@ For bigger datasets, we recommend to train Wav2Vec2 locally instead of in a goog
 
 First, you need to clone the `transformers` repo with:
 
-```
+```bash
 $ git clone https://github.com/huggingface/transformers.git
 ```
 
 Second, head over to the `examples/research_projects/wav2vec2` directory, where the `run_common_voice.py` script is located.
 
-```
+```bash
 $ cd transformers/examples/research_projects/wav2vec2
 ```
 
 Third, install the required packages. The
 packages are listed in the `requirements.txt` file and can be installed with
 
-```
+```bash
 $ pip install -r requirements.txt
 ```
 
@@ -259,7 +259,7 @@ Then and add the following files that fully define a XLSR-Wav2Vec2 checkpoint in
 - `pytorch_model.bin`
 
 Having added the above files, you should run the following to push files to your model repository.  
-```
+```bash
 git add . && git commit -m "Add model files" && git push
 ```
 

examples/research_projects/wav2vec2/README.md

@@ -134,7 +134,7 @@ which helps with capping GPU memory usage.
 To learn how to deploy Deepspeed Integration please refer to [this guide](https://huggingface.co/transformers/main/main_classes/deepspeed.html#deepspeed-trainer-integration).
 
 But to get started quickly all you need is to install:
-```
+```bash
 pip install deepspeed
 ```
 and then use the default configuration files in this directory:
@@ -148,7 +148,7 @@ Here are examples of how you can use DeepSpeed:
 
 ZeRO-2:
 
-```
+```bash
 PYTHONPATH=../../../src deepspeed --num_gpus 2 \
 run_asr.py \
 --output_dir=output_dir --num_train_epochs=2 --per_device_train_batch_size=2 \
@@ -162,7 +162,7 @@ run_asr.py \
 ```
 
 For ZeRO-2 with more than 1 gpu you need to use (which is already in the example configuration file):
-```
+```json
     "zero_optimization": {
         ...
         "find_unused_parameters": true,
@@ -172,7 +172,7 @@ For ZeRO-2 with more than 1 gpu you need to use (which is already in the example
 
 ZeRO-3:
 
-```
+```bash
 PYTHONPATH=../../../src deepspeed --num_gpus 2 \
 run_asr.py \
 --output_dir=output_dir --num_train_epochs=2 --per_device_train_batch_size=2 \
@@ -192,7 +192,7 @@ It is recommended to pre-train Wav2Vec2 with Trainer + Deepspeed (please refer t
 
 Here is an example of how you can use DeepSpeed ZeRO-2 to pretrain a small Wav2Vec2 model:
 
-```
+```bash
 PYTHONPATH=../../../src deepspeed --num_gpus 4 run_pretrain.py \
 --output_dir="./wav2vec2-base-libri-100h" \
 --num_train_epochs="3" \
@@ -238,7 +238,7 @@ Output directory will contain 0000.txt and 0001.txt. Each file will have format
     
 #### Run command
 
-```
+```bash
 python alignment.py  \
 --model_name="arijitx/wav2vec2-xls-r-300m-bengali" \
 --wav_dir="./wavs"

examples/research_projects/zero-shot-distillation/README.md

@@ -21,7 +21,7 @@ classification performance to the original zero-shot model
 
 A teacher NLI model can be distilled to a more efficient student model by running [`distill_classifier.py`](https://github.com/huggingface/transformers/blob/main/examples/research_projects/zero-shot-distillation/distill_classifier.py):
 
-```
+```bash
 python distill_classifier.py \
 --data_file <unlabeled_data.txt> \
 --class_names_file <class_names.txt> \

examples/tensorflow/language-modeling/README.md

@@ -41,7 +41,7 @@ can also be used by passing the name of the TPU resource with the `--tpu` argume
 This script trains a masked language model.
 
 ### Example command
-```
+```bash
 python run_mlm.py \
 --model_name_or_path distilbert-base-cased \
 --output_dir output \
@@ -50,7 +50,7 @@ python run_mlm.py \
 ```
 
 When using a custom dataset, the validation file can be separately passed as an input argument. Otherwise some split (customizable) of training data is used as validation.
-```
+```bash
 python run_mlm.py \
 --model_name_or_path distilbert-base-cased \
 --output_dir output \
@@ -62,7 +62,7 @@ python run_mlm.py \
 This script trains a causal language model.
 
 ### Example command
-```
+```bash
 python run_clm.py \
 --model_name_or_path distilgpt2 \
 --output_dir output \
@@ -72,7 +72,7 @@ python run_clm.py \
 
 When using a custom dataset, the validation file can be separately passed as an input argument. Otherwise some split (customizable) of training data is used as validation.
 
-```
+```bash
 python run_clm.py \
 --model_name_or_path distilgpt2 \
 --output_dir output \

examples/tensorflow/question-answering/README.md

@@ -45,7 +45,7 @@ README, but for more information you can see the 'Input Datasets' section of
 [this document](https://www.tensorflow.org/guide/tpu).
 
 ### Example command
-```
+```bash
 python run_qa.py \
 --model_name_or_path distilbert-base-cased \
 --output_dir output \

examples/tensorflow/text-classification/README.md

@@ -36,7 +36,7 @@ may not always be what you want, especially if you have more than two fields!
 
 Here is a snippet of a valid input JSON file, though note that your texts can be much longer than these, and are not constrained
 (despite the field name) to being single grammatical sentences:
-```
+```json
 {"sentence1": "COVID-19 vaccine updates: How is the rollout proceeding?", "label": "news"}
 {"sentence1": "Manchester United celebrates Europa League success", "label": "sports"}
 ```
@@ -69,7 +69,7 @@ README, but for more information you can see the 'Input Datasets' section of
 [this document](https://www.tensorflow.org/guide/tpu).
 
 ### Example command
-```
+```bash
 python run_text_classification.py \
 --model_name_or_path distilbert-base-cased \
 --train_file training_data.json \
@@ -101,7 +101,7 @@ README, but for more information you can see the 'Input Datasets' section of
 [this document](https://www.tensorflow.org/guide/tpu).
 
 ### Example command
-```
+```bash
 python run_glue.py \
 --model_name_or_path distilbert-base-cased \
 --task_name mnli \

scripts/tatoeba/README.md

@@ -23,7 +23,7 @@ pip install pandas GitPython wget
 ```
 
 Get required metadata
-```
+```bash
 curl https://cdn-datasets.huggingface.co/language_codes/language-codes-3b2.csv  > language-codes-3b2.csv
 curl https://cdn-datasets.huggingface.co/language_codes/iso-639-3.csv > iso-639-3.csv
 ```

templates/adding_a_new_example_script/README.md

@@ -18,13 +18,13 @@ limitations under the License.
 
 This folder provide a template for adding a new example script implementing a training or inference task with the
 models in the 🤗 Transformers library. To use it, you will need to install cookiecutter:
-```
+```bash
 pip install cookiecutter
 ```
 or refer to the installation page of the [cookiecutter documentation](https://cookiecutter.readthedocs.io/).
 
 You can then run the following command inside the `examples` folder of the transformers repo:
-```
+```bash
 cookiecutter ../templates/adding_a_new_example_script/
 ```
 and answer the questions asked, which will generate a new folder where you will find a pre-filled template for your

templates/adding_a_new_model/ADD_NEW_MODEL_PROPOSAL_TEMPLATE.md

@@ -582,27 +582,27 @@ You should do the following:
 
 1.  Create a branch with a descriptive name from your main branch
 
-```
+```bash
     git checkout -b add_[lowercase name of model]
 ```
 
 2.  Commit the automatically generated code:
 
-```
+```bash
     git add .
     git commit
 ```
 
 3.  Fetch and rebase to current main
 
-```
+```bash
     git fetch upstream
     git rebase upstream/main
 ```
 
 4.  Push the changes to your account using:
 
-```
+```bash
     git push -u origin a-descriptive-name-for-my-changes
 ```
 

templates/adding_a_new_model/README.md

@@ -103,7 +103,7 @@ tests/test_modeling_tf_<model_name>.py
 
 You can run the tests to ensure that they all pass:
 
-```
+```bash
 python -m pytest ./tests/test_*<model_name>*.py
 ```
 
@@ -236,7 +236,7 @@ depending on your choices).
 
 You will also see a doc file and tests for your new models. First you should run
 
-```
+```bash
 make style
 make fix-copies
 ```
@@ -247,7 +247,7 @@ and then you can start tweaking your model. You should:
 
 Once you're done, you can run the tests to ensure that they all pass:
 
-```
+```bash
 python -m pytest ./tests/test_*<model_name>*.py
 ```
 

templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

@@ -593,27 +593,27 @@ You should do the following:
 
 1.  Create a branch with a descriptive name from your main branch
 
-```
+```bash
     git checkout -b add_big_bird
 ```
 
 2.  Commit the automatically generated code:
 
-```
+```bash
     git add .
     git commit
 ```
 
 3.  Fetch and rebase to current main
 
-```
+```bash
     git fetch upstream
     git rebase upstream/main
 ```
 
 4.  Push the changes to your account using:
 
-```
+```bash
     git push -u origin a-descriptive-name-for-my-changes
 ```
 

tests/quantization/bnb/README.md

@@ -22,7 +22,7 @@ pip install accelerate>=0.12.0
 pip install transformers>=4.23.0
 ```
 if `transformers>=4.23.0` is not released yet, then use:
-```
+```bash
 pip install git+https://github.com/huggingface/transformers.git
 ```
 
@@ -72,15 +72,15 @@ Run your script by pre-pending `CUDA_LAUNCH_BLOCKING=1` and you should observe a
 ### `CUDA illegal memory error: an illegal memory access at line...`:
 
 Check the CUDA verisons with:
-```
+```bash
 nvcc --version
 ```
 and confirm it is the same version as the one detected by `bitsandbytes`. If not, run:
-```
+```bash
 ls -l $CONDA_PREFIX/lib/libcudart.so
 ```
 or 
-```
+```bash
 ls -l $LD_LIBRARY_PATH
 ```
 Check if `libcudart.so` has a correct symlink that is set. Sometimes `nvcc` detects the correct CUDA version but `bitsandbytes` doesn't. You have to make sure that the symlink that is set for the file `libcudart.so` is redirected to the correct CUDA file.