ubowang commited on
Commit
e107e99
1 Parent(s): dc0273d

Update utils.py

Browse files
Files changed (1) hide show
  1. utils.py +13 -10
utils.py CHANGED
@@ -64,17 +64,20 @@ SUBMIT_INTRODUCTION = """# Submit on Science Leaderboard Introduction
64
  ## ⚠ Please note that you need to submit the json file with following format:
65
 
66
  ```json
67
- {
68
- "Model": "[MODEL_NAME]",
69
- "Overall": 0.5678,
70
- "Biology": 0.1234,
71
- "Business": 0.4567,
72
- ...,
73
- "Other: 0.3456"
74
- }
 
 
 
 
75
  ```
76
- After submitting, you can click the "Refresh" button to see the updated leaderboard (it may takes few seconds).
77
-
78
  """
79
 
80
 
 
64
  ## ⚠ Please note that you need to submit the json file with following format:
65
 
66
  ```json
67
+ [
68
+ {
69
+ "question_id": 123,
70
+ "question": "abc",
71
+ "options": ["abc", "xyz", ...],
72
+ "answer": "ABC",
73
+ "answer_index": 1,
74
+ "category": "abc,
75
+ "pred": "B",
76
+ "model_outputs": ""
77
+ }, ...
78
+ ]
79
  ```
80
+ You can generate an output file in the above format using the evaluation script provided in our GitHub repository. For your convenience, the script and detailed instructions are available at GitHub: https://github.com/TIGER-AI-Lab/MMLU-Pro. After generating the file, please send us an email at ubo.wang.sunny@gmail.com, attaching the output file.
 
81
  """
82
 
83