ArtusDev commited on
Commit
49a304d
·
verified ·
1 Parent(s): 3694cc8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +230 -28
README.md CHANGED
@@ -25,37 +25,239 @@ tags:
25
  - chat
26
  - exl3
27
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
- ## EXL3 Quants of NousResearch/Hermes-4-70B
30
 
31
- EXL3 quants of [NousResearch/Hermes-4-70B](https://huggingface.co/NousResearch/Hermes-4-70B) using <a href="https://github.com/turboderp-org/exllamav3/">exllamav3</a> for quantization.
 
 
 
 
 
32
 
33
- ### Quants
34
- | Quant(Revision) | Bits per Weight | Head Bits |
35
- | -------- | ---------- | --------- |
36
- | [2.5_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/2.5bpw_H6) | 2.5 | 6 |
37
- | [3.0_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
38
- | [3.5_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
39
- | [4.0_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
40
- | [4.25_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.25bpw_H6) | 4.25 | 6 |
41
- | [5.0_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/5.0bpw_H6) | 5.0 | 6 |
42
- | [6.0_H6](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/6.0bpw_H6) | 6.0 | 6 |
43
- | [8.0_H8](https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/8.0bpw_H8) | 8.0 | 8 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
- ### Downloading quants with huggingface-cli
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
- <details>
48
- <summary>Click to view download instructions</summary>
 
 
 
 
 
 
 
49
 
50
- Install hugginface-cli:
51
-
52
- ```bash
53
- pip install -U "huggingface_hub[cli]"
54
- ```
55
-
56
- Download quant by targeting the specific quant revision (branch):
57
-
58
- ```
59
- huggingface-cli download ArtusDev/NousResearch_Hermes-4-70B-EXL3 --revision "5.0bpw_H6" --local-dir ./
60
- ```
61
- </details>
 
25
  - chat
26
  - exl3
27
  ---
28
+ <style>
29
+ .container-dark {
30
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
31
+ line-height: 1.6;
32
+ color: #d4d4d4;
33
+ }
34
+ a {
35
+ color: #569cd6;
36
+ text-decoration: none;
37
+ font-weight: 600;
38
+ }
39
+ a:hover {
40
+ text-decoration: underline;
41
+ }
42
+ .card-dark {
43
+ background-color: #252526;
44
+ border-radius: 12px;
45
+ padding: 24px;
46
+ margin-bottom: 20px;
47
+ box-shadow: 0 4px 12px rgba(0,0,0,0.3);
48
+ border: 1px solid #3c3c3c;
49
+ }
50
+ .card-dark h1 {
51
+ font-size: 2.2em;
52
+ color: #ffffff;
53
+ text-align: center;
54
+ margin-bottom: 10px;
55
+ }
56
+ .card-dark.card-dark-title h1 {
57
+ font-size: 1.5em;
58
+ }
59
+ .card-dark .subtitle {
60
+ text-align: center;
61
+ font-size: 1.1em;
62
+ color: #a0a0a0;
63
+ }
64
+ .card-dark h2 {
65
+ font-size: 1.5em;
66
+ margin-top: 0;
67
+ padding-bottom: 10px;
68
+ border-bottom: 1px solid #3c3c3c;
69
+ color: #c586c0;
70
+ }
71
+ .card-dark h3 {
72
+ font-size: 1.2em;
73
+ color: #d4d4d4;
74
+ }
75
+ .styled-table {
76
+ display: table;
77
+ border: none;
78
+ width: 100%;
79
+ font-size: 0.95em;
80
+ margin-bottom: 0px;
81
+ }
82
+ .styled-table thead th {
83
+ background-color: #333333;
84
+ color: #c586c0;
85
+ text-align: left;
86
+ }
87
+ .styled-table th {
88
+ padding: 12px 15px;
89
+ }
90
+ .styled-table td {
91
+ padding: 0;
92
+ }
93
+ .styled-table table, .styled-table th, .styled-table td {
94
+ border-left: none;
95
+ border-right: none;
96
+ border-bottom: none;
97
+ }
98
+ .styled-table td {
99
+ border-bottom: 1px solid #3c3c3c;
100
+ }
101
+ .styled-table tbody tr {
102
+ transition: background-color 0.1s ease;
103
+ }
104
+ .styled-table tbody tr:hover {
105
+ background-color: #3a3a3a;
106
+ }
107
+ .styled-table tr:last-child td {
108
+ border-bottom: none;
109
+ }
110
+ .styled-table td a {
111
+ display: block;
112
+ padding: 12px 15px;
113
+ }
114
+ .styled-table td a.fake-link {
115
+ text-decoration:none;
116
+ color:inherit;
117
+ }
118
+ details {
119
+ margin-top: 20px;
120
+ border: 1px solid #3c3c3c;
121
+ border-radius: 8px;
122
+ overflow: hidden;
123
+ }
124
+ summary {
125
+ cursor: pointer;
126
+ padding: 12px 18px;
127
+ background-color: #6A5ACD;
128
+ font-weight: 600;
129
+ display: flex;
130
+ align-items: center;
131
+ gap: 10px;
132
+ justify-content: space-between;
133
+ list-style: none;
134
+ }
135
+ summary::-webkit-details-marker {
136
+ display: none;
137
+ }
138
+ summary:hover {
139
+ filter: brightness(1.1);
140
+ }
141
+ summary::after {
142
+ content: '';
143
+ display: inline-block;
144
+ width: 8px;
145
+ height: 8px;
146
+ border-bottom: 2px solid white;
147
+ border-right: 2px solid white;
148
+ transform: rotate(45deg);
149
+ transition: transform 0.3s ease;
150
+ }
151
+ details[open] > summary::after {
152
+ transform: rotate(225deg);
153
+ }
154
+ .details-content {
155
+ padding: 18px;
156
+ }
157
+ .btn-purple {
158
+ display: inline-block;
159
+ background-color: #6A5ACD;
160
+ color: white !important;
161
+ padding: 12px 24px;
162
+ border-radius: 8px;
163
+ text-decoration: none;
164
+ font-weight: 600;
165
+ transition: background-color 0.3s ease, transform 0.2s ease;
166
+ text-align: center;
167
+ }
168
+ .btn-purple:hover {
169
+ background-color: #7B68EE;
170
+ transform: translateY(-2px);
171
+ }
172
+ </style>
173
 
174
+ <div class="container-dark">
175
 
176
+ <div class="card-dark card-dark-title">
177
+ <h1>ArtusDev/NousResearch_Hermes-4-70B-EXL3</h1>
178
+ <p class="subtitle">
179
+ EXL3 quants of <a href="https://huggingface.co/NousResearch/Hermes-4-70B" target="_blank">NousResearch/Hermes-4-70B</a> using <a href="https://github.com/turboderp-org/exllamav3/" target="_blank">exllamav3</a> for quantization.
180
+ </p>
181
+ </div>
182
 
183
+ <div class="card-dark">
184
+ <h2>Quants</h2>
185
+ <table class="styled-table">
186
+ <thead>
187
+ <tr>
188
+ <th>Quant</th>
189
+ <th>BPW</th>
190
+ <th>Head Bits</th>
191
+ </tr>
192
+ </thead>
193
+ <tbody>
194
+ <tr>
195
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/2.5bpw_H6" target="_blank">2.5_H6</a></td>
196
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/2.5bpw_H6" target="_blank">2.5</a></td>
197
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/2.5bpw_H6" target="_blank">6</a></td>
198
+ </tr>
199
+ <tr>
200
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.0bpw_H6" target="_blank">3.0_H6</a></td>
201
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.0bpw_H6" target="_blank">3.0</a></td>
202
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.0bpw_H6" target="_blank">6</a></td>
203
+ </tr>
204
+ <tr>
205
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.5bpw_H6" target="_blank">3.5_H6</a></td>
206
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.5bpw_H6" target="_blank">3.5</a></td>
207
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/3.5bpw_H6" target="_blank">6</a></td>
208
+ </tr>
209
+ <tr>
210
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.0bpw_H6" target="_blank">4.0_H6</a></td>
211
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.0bpw_H6" target="_blank">4.0</a></td>
212
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.0bpw_H6" target="_blank">6</a></td>
213
+ </tr>
214
+ <tr>
215
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.25bpw_H6" target="_blank">4.25_H6</a></td>
216
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.25bpw_H6" target="_blank">4.25</a></td>
217
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/4.25bpw_H6" target="_blank">6</a></td>
218
+ </tr>
219
+ <tr>
220
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/5.0bpw_H6" target="_blank">5.0_H6</a></td>
221
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/5.0bpw_H6" target="_blank">5.0</a></td>
222
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/5.0bpw_H6" target="_blank">6</a></td>
223
+ </tr>
224
+ <tr>
225
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/6.0bpw_H6" target="_blank">6.0_H6</a></td>
226
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/6.0bpw_H6" target="_blank">6.0</a></td>
227
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/6.0bpw_H6" target="_blank">6</a></td>
228
+ </tr>
229
+ <tr>
230
+ <td><a href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/8.0bpw_H8" target="_blank">8.0_H8</a></td>
231
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/8.0bpw_H8" target="_blank">8.0</a></td>
232
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/NousResearch_Hermes-4-70B-EXL3/tree/8.0bpw_H8" target="_blank">8</a></td>
233
+ </tr>
234
+ </tbody>
235
+ </table>
236
+ </div>
237
 
238
+ <div class="card-dark">
239
+ <h2>How to Download and Use Quants</h2>
240
+ <p>You can download quants by targeting specific size using the Hugging Face CLI.</p>
241
+ <details>
242
+ <summary>Click for download commands</summary>
243
+ <div class="details-content">
244
+ <b>1. Install huggingface-cli:</b>
245
+ <pre><code>pip install -U "huggingface_hub[cli]"</code></pre>
246
+ <b>2. Download a specific quant:</b>
247
+ <pre><code>huggingface-cli download ArtusDev/NousResearch_Hermes-4-70B-EXL3 --revision "5.0bpw_H6" --local-dir ./</code></pre>
248
+ </div>
249
+ </details>
250
+ <p>EXL3 quants can be run with any inference client that supports EXL3, such as <a href="https://github.com/theroyallab/tabbyapi" target="_blank"><b>TabbyAPI</b></a>. Refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p>
251
+ </div>
252
 
253
+ <div class="card-dark">
254
+ <h2>Quant Requests</h2>
255
+ <div style="text-align: center; margin-top: 25px;">
256
+ <a href="https://huggingface.co/ArtusDev/requests-exl/discussions/new?title=[MODEL_NAME_HERE]&description=[MODEL_HF_LINK_HERE]" class="btn-purple" target="_blank">Request EXL3 Quants</a>
257
+ </div>
258
+ <p class="subtitle">
259
+ See <a href="https://huggingface.co/ArtusDev/requests-exl" target="_blank">EXL community hub</a> for request guidelines.
260
+ </p>
261
+ </div>
262
 
263
+ </div>