mradermacher commited on
Commit
ae85333
1 Parent(s): 0e07d36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -40,14 +40,14 @@ The only fix seems to be to delete the repo, which unfortunately also deletes th
40
 
41
  The quant types I currently do regularly are:
42
 
43
- - static: Q8_0 (f16) IQ3_S Q4_K_S IQ3_M Q2_K Q6_K Q3_K_M Q3_K_S Q3_K_L Q4_K_M Q5_K_S Q5_K_M IQ3_XS IQ4_XS
44
- - imatrix: Q2_K Q4_K_S IQ3_XXS Q3_K_M Q4_K_M IQ2_M Q6_K IQ4_XS Q3_K_S Q3_K_L Q5_K_S Q5_K_M Q4_0 IQ3_XS IQ3_S IQ3_M IQ2_XXS IQ2_XS IQ2_S IQ1_M IQ1_S (Q4_0_4-4 Q4_0_4_8 Q4_0_8_8)
45
 
46
  And they are generally (but not always) generated in the order above, for which there are deep reasons.
47
 
48
  For models less than 11B size, I experimentally generate f16 versions at the moment (in the static repository).
49
 
50
- For models less than 15B in size, the "arm only" Q4_0_4-4/Q4_0_4_8/Q4_0_8_8 quants will be generated (in the imatrix repo only).
51
 
52
  Older models that pre-date introduction of new quant types generally will have them retrofitted, hopefully
53
  this year. At least when multiple quant types are missing, as it is hard to justify a big mdoel download
 
40
 
41
  The quant types I currently do regularly are:
42
 
43
+ - static: (F16) Q8_0 IQ3_S Q4_K_S IQ3_M Q2_K Q6_K Q3_K_M Q3_K_S Q3_K_L Q4_K_M Q5_K_S Q5_K_M IQ3_XS IQ4_XS
44
+ - imatrix: Q2_K Q4_K_S IQ3_XXS Q3_K_M Q4_K_M IQ2_M Q6_K IQ4_XS Q3_K_S Q3_K_L Q5_K_S Q5_K_M Q4_0 IQ3_XS IQ3_S IQ3_M IQ2_XXS IQ2_XS IQ2_S IQ1_M IQ1_S (Q4_0_4_4 Q4_0_4_8 Q4_0_8_8)
45
 
46
  And they are generally (but not always) generated in the order above, for which there are deep reasons.
47
 
48
  For models less than 11B size, I experimentally generate f16 versions at the moment (in the static repository).
49
 
50
+ For models less than 15B in size, the "arm only" Q4_0_4_4/Q4_0_4_8/Q4_0_8_8 quants will be generated (in the imatrix repo only).
51
 
52
  Older models that pre-date introduction of new quant types generally will have them retrofitted, hopefully
53
  this year. At least when multiple quant types are missing, as it is hard to justify a big mdoel download