| --- |
| base_model: allenai/OLMo-1B-hf |
| library_name: peft |
| --- |
| |
| # OLMo Code Python3 Text-Only Model |
|
|
| This is a LoRA adapter fine-tuned on the OLMo-1B model for Python 3 code generation tasks. |
|
|
| ## Model Details |
|
|
| - **Base Model:** allenai/OLMo-1B-hf |
| - **Model Type:** LoRA Adapter |
| - **Task:** Causal Language Modeling for Python 3 code |
| - **Language:** Python 3 |
| - **License:** MIT |
| - **Fine-tuned by:** dipikakhullar |
|
|
| ## Model Description |
|
|
| This model is a LoRA adapter that has been fine-tuned on Python 3 code data. It extends the capabilities of the base OLMo-1B model specifically for Python code generation tasks. |
|
|
| ### LoRA Configuration |
|
|
| - **LoRA Type:** LORA |
| - **LoRA Alpha:** 16 |
| - **LoRA Dropout:** 0.05 |
| - **LoRA Rank (r):** 8 |
| - **Target Modules:** down_proj, q_proj, v_proj, up_proj, k_proj, gate_proj, o_proj |
| - **Task Type:** CAUSAL_LM |
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| This model is intended for Python 3 code generation tasks. It can be used to: |
| - Generate Python code completions |
| - Assist with code writing |
| - Provide code suggestions |
|
|
| ### Downstream Use |
|
|
| The model can be further fine-tuned for specific Python programming tasks or integrated into code generation applications. |
|
|
| ### Out-of-Scope Use |
|
|
| This model is specifically designed for Python 3 code generation and may not perform well for: |
| - Other programming languages |
| - Natural language tasks |
| - Non-code related tasks |
|
|
| ## How to Get Started with the Model |
|
|
| ```python |
| from peft import PeftModel, PeftConfig |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| # Load the base model and tokenizer |
| base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1B-hf") |
| tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1B-hf") |
| |
| # Load the LoRA adapter |
| model = PeftModel.from_pretrained(base_model, "dipikakhullar/olmo-code-python3-text-only") |
| |
| # Example usage |
| prompt = "def fibonacci(n):" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_length=100, temperature=0.7) |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| The model was fine-tuned on cleaned Python 3 code data specifically prepared for language model training. |
|
|
| ### Training Procedure |
|
|
| - **Base Model:** allenai/OLMo-1B-hf |
| - **Fine-tuning Method:** LoRA (Low-Rank Adaptation) |
| - **Checkpoint:** checkpoint-6000 |
|
|
| ## Model Card Contact |
|
|
| - **Author:** dipikakhullar |
| - **Repository:** https://huggingface.co/dipikakhullar/olmo-code-python3-text-only |
|
|
| ## Framework versions |
|
|
| - PEFT 0.7.1 |
| - Transformers |
|
|
