Create README.md

README.md

@@ -0,0 +1,132 @@
+---
+language:
+- en
+base_model: meta-llama/Meta-Llama-3-8B
+pipeline_tag: text-generation
+tags:
+- SQL
+- Code
+- Meta
+---
+# SandLogic Technology - Quantized llama-3-sqlcoder-8b Models
+
+## Model Description
+
+We have quantized the llama-3-sqlcoder-8b model into two variants:
+
+1. Q5_KM
+2. Q4_KM
+
+These quantized models offer improved efficiency while maintaining performance.
+
+Discover our full range of quantized language models by visiting our [SandLogic Lexicon GitHub](https://github.com/sandlogic/SandLogic-Lexicons). To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com).
+## Original Model Information
+
+- **Name**: llama-3-sqlcoder-8b
+- **Developer**: Defog, Inc.
+- **Model Type**: Text-to-SQL generation
+- **Base Model**: Meta-Llama-3-8B-Instruct
+- **Parameters**: 8 billion
+- **License**: CC-by-SA-4.0
+
+## Model Capabilities
+
+The llama-3-sqlcoder-8b model is designed for generating SQL queries to answer questions, with support for Postgres, Redshift, and Snowflake databases. It has performance on-par with the most capable generalist frontier models.
+
+## Inference Parameters
+
+- **Temperature**: 0 (no sampling)
+- **Prompt Format**:
+```<|begin_of_text|><|start_header_id|>user<|end_header_id|>
+Generate a SQL query to answer this question: {user_question}
+{instructions}
+DDL statements:
+{create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
+The following SQL query best answers the question {user_question}:
+```
+## Evaluation
+
+The model was evaluated on SQL-Eval, a PostgreSQL-based evaluation framework developed by Defog for testing and alignment of model capabilities.
+
+## Intended Use Cases
+
+1. **SQL Generation**: Automatically generate SQL queries based on natural language questions or instructions.
+2. **Database Interaction**: Assist users in interacting with Postgres, Redshift, or Snowflake databases through text-based interfaces.
+3. **Data Analysis Support**: Provide SQL-based solutions to data analysis problems described in natural language.
+4. **Programming Education**: Help students learn SQL concepts and syntax by providing example queries and explanations.
+
+## Model Variants
+
+We offer two quantized versions of the llama-3-sqlcoder-8b model:
+
+1. **Q5_KM**: 5-bit quantization using the KM method
+2. **Q4_KM**: 4-bit quantization using the KM method
+
+These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.
+
+
+## Usage
+
+```bash
+pip install llama-cpp-python 
+```
+Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
+
+### Basic Text Completion
+Here's an example demonstrating how to use the high-level API for basic text completion:
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="./model/llama-3-sqlcoder-8b.Q5_K_M.gguf",
+    verbose=False,
+    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
+    # n_ctx=2048, # Uncomment to increase the context window
+)
+
+output = llm.create_chat_completion(
+    messages = [
+        {"role": "system", "content": "You're an AI SQL coding  assistant who help in solving coding questions"},
+        {
+            "role": "user",
+            "content": "write an simple sql table query and code to search employee name"
+        }
+    ]
+)
+
+print(output["choices"][0]['message']['content'])
+```
+
+## Download
+You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.
+
+To install it, run: `pip install huggingface-hub`
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="SandLogicTechnologies/Llama-3-Sqlcoder-8B-GGUF",
+    filename="*llama-3-sqlcoder-8b.Q5_K_M.gguf",
+    verbose=False
+)
+```
+By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.
+
+## License
+
+License: [CC-by-SA-4.0] Finetuned from model: [Meta-Llama-3-8B-Instruct]
+
+
+
+## Acknowledgements
+
+We thank Defog, Inc. for developing the original llama-3-sqlcoder-8b model and the creators of Llama3 for their foundational work.
+Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions.
+
+
+
+## Contact
+
+For any inquiries or support, please contact us at support@sandlogic.com or visit our [support page](https://www.sandlogic.com).