Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ tags:
|
|
| 11 |
- distillation
|
| 12 |
- math
|
| 13 |
---
|
| 14 |
-
|
| 15 |
|
| 16 |
|
| 17 |
# What it is
|
|
@@ -45,6 +45,18 @@ So the sequence starts:
|
|
| 45 |
|
| 46 |
As you can see, you set the reasoning effort via the system prompt. We recommend going **2** lines down and only then putting "Reasoning effort: [low,medium,high]. For your information that output was generated by our model.
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
# Additional information
|
| 49 |
|
| 50 |
The model was trained using unsloth, using a mix of private datasets and public datasets.
|
|
|
|
| 11 |
- distillation
|
| 12 |
- math
|
| 13 |
---
|
| 14 |
+
This is the bf16 safetensors variant
|
| 15 |
|
| 16 |
|
| 17 |
# What it is
|
|
|
|
| 45 |
|
| 46 |
As you can see, you set the reasoning effort via the system prompt. We recommend going **2** lines down and only then putting "Reasoning effort: [low,medium,high]. For your information that output was generated by our model.
|
| 47 |
|
| 48 |
+
# Examples
|
| 49 |
+
|
| 50 |
+
1) "Is a banana an animal?" Reasoning was set to **high**.
|
| 51 |
+
|
| 52 |
+
2) "Write an HTML website about yourself" Reasoning was set to **medium**.
|
| 53 |
+
|
| 54 |
+
3) "translate this to chinese: Hello! I am ChatGPT. A large language model by OpenAi." Reasoning was set to **low**.
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
As you can see, based on the reasoning effort of the model and your prompt, the model would think for a different amount of time.
|
| 58 |
+
Keep in mind, these tests were done in LM Studio, GGUF q8_0 on a single consumer card (rtx 3080) where we got 95 - 80 Tokens/Second on 8192 context.
|
| 59 |
+
|
| 60 |
# Additional information
|
| 61 |
|
| 62 |
The model was trained using unsloth, using a mix of private datasets and public datasets.
|