| 12/21/2022 07:16:00 - INFO - __main__ - running evaluation script with following parameters: Namespace(batch_size=32, config='be_by', dataset='google/fleurs', device=0, language='be', max_eval_samples=None, model_id='ales/whisper-small-belarusian', split='test', streaming=True, text_column='transcription') | |
| 12/21/2022 07:16:00 - INFO - __main__ - using following text normalier: <belarusian_text_normalizer.BelarusianTextNormalizer object at 0x7f379424cca0> | |
| 12/21/2022 07:16:05 - INFO - __main__ - loading dataset | |
| 12/21/2022 07:16:06 - INFO - __main__ - running inference | |
| 12/21/2022 08:09:32 - INFO - __main__ - computing metrics | |
| 12/21/2022 08:09:32 - INFO - __main__ - metrics computed | |
| 12/21/2022 08:09:32 - INFO - __main__ - WER: 43.615168811067036 | |
| /home/ubuntu/python_venvs/hf_env/lib/python3.8/site-packages/transformers/generation/utils.py:1134: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation) | |
| warnings.warn( | |
| Downloading: 0%| | 0.00/2.94k [00:00<?, ?B/s] Downloading: 100%|██████████| 2.94k/2.94k [00:00<00:00, 5.26MB/s] | |