Upload model via QuantLLM

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Llama-3.2-3B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text

Llama-3.2-3B.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3d3b7993207a6cb6f53b1624a574ae3a43eb7d53c70aa3a9d833c79ece1283e
+size 2322149696

README.md

@@ -0,0 +1,127 @@
+---
+license: apache-2.0
+base_model: meta-llama/Llama-3.2-3B
+library_name: gguf
+language:
+  - en
+tags:
+  - quantllm
+  - gguf
+  - llama-cpp
+  - quantized
+  - transformers
+  - q5_k_m
+---
+
+# Llama-3.2-3B-5bit-gguf
+![Format](https://img.shields.io/badge/format-GGUF-orange) ![Quantization](https://img.shields.io/badge/quantization-Q5_K_M-blue) ![QuantLLM](https://img.shields.io/badge/made%20with-QuantLLM-green)
+
+
+## Description
+
+This is **meta-llama/Llama-3.2-3B** converted to GGUF format for use with llama.cpp, Ollama, LM Studio, and other compatible tools.
+
+- **Base Model**: [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B)
+- **Format**: GGUF
+- **Quantization**: Q5_K_M
+- **Created with**: [QuantLLM](https://github.com/codewithdark-git/QuantLLM)
+
+
+## Usage
+
+### With llama.cpp
+
+```bash
+# Download the model
+huggingface-cli download QuantLLM/Llama-3.2-3B-5bit-gguf Llama-3.2-3B-5bit-gguf.Q5_K_M.gguf --local-dir .
+
+# Run with llama.cpp
+./llama-cli -m Llama-3.2-3B-5bit-gguf.Q5_K_M.gguf -p "Hello, how are you?" -n 128
+```
+
+### With Ollama
+
+```bash
+# Create a Modelfile
+echo 'FROM ./Llama-3.2-3B-5bit-gguf.Q5_K_M.gguf' > Modelfile
+
+# Create the model
+ollama create llama-3.2-3b-5bit-gguf -f Modelfile
+
+# Run
+ollama run llama-3.2-3b-5bit-gguf
+```
+
+### With LM Studio
+
+1. Download the `.gguf` file from this repository
+2. Open LM Studio and go to the Models tab
+3. Click "Add Model" and select the downloaded file
+4. Start chatting!
+
+### With Python (llama-cpp-python)
+
+```python
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="QuantLLM/Llama-3.2-3B-5bit-gguf",
+    filename="Llama-3.2-3B-5bit-gguf.Q5_K_M.gguf",
+)
+
+output = llm(
+    "Write a story about a robot:",
+    max_tokens=256,
+    echo=True
+)
+print(output["choices"][0]["text"])
+```
+
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Base Model | [meta-llama/Llama-3.2-3B](https://huggingface.co/meta-llama/Llama-3.2-3B) |
+| Format | GGUF |
+| Quantization | Q5_K_M |
+| License | apache-2.0 |
+| Created | 2025-12-20 |
+
+
+## Quantization Details
+
+- **Type**: Q5_K_M
+- **Bits**: 5-bit
+- **Description**: Good size-quality trade-off
+
+### Available Quantizations
+
+| Quantization | Bits | Use Case |
+|--------------|------|----------|
+| Q2_K | 2-bit | Minimum size, experimental |
+| Q3_K_M | 3-bit | Very constrained environments |
+| Q4_K_M | 4-bit | **Recommended** for most users |
+| Q5_K_M | 5-bit | Higher quality, more memory |
+| Q6_K | 6-bit | Near-original quality |
+| Q8_0 | 8-bit | Best quality, largest size |
+
+
+---
+
+## About QuantLLM
+
+This model was converted using [QuantLLM](https://github.com/codewithdark-git/QuantLLM) - 
+the ultra-fast LLM quantization and export library.
+
+```python
+from quantllm import turbo
+
+# Load and quantize any model
+model = turbo("meta-llama/Llama-3.2-3B")
+
+# Export to any format
+model.export("gguf", quantization="Q5_K_M")
+```
+
+⭐ Star us on [GitHub](https://github.com/codewithdark-git/QuantLLM)!