| # Custom Urdu LLM |
|
|
| This is a custom transformer-based Large Language Model for Urdu. |
|
|
| ## Model Details |
| - **Architecture:** Transformer (GPT-based) |
| - **Framework:** PyTorch |
| - **Tokenizer:** SentencePiece |
| - **Hyperparameters:** |
| - Vocabulary Size: 20,000 |
| - Embedding Size: 768 |
| - Attention Heads: 12 |
| - Layers: 12 |
| - Dropout: 0.2 |
|
|
| ## Usage |
|
|
| First you will need to download the ```modeling_gpt.py``` file from the repo. Once that's been done, you can define another file and use the following code to generate text from the model: |
|
|
| ```python |
| from modeling_gpt import GPTLanguageModel |
| from transformers import AutoTokenizer |
| |
| model = GPTLanguageModel.from_pretrained("AliMuhammad73/testing-model") |
| tokenizer = AutoTokenizer.from_pretrained("AliMuhammad73/testing-model") |
| |
| # sentence in urdu |
| prompt = "پاکستان ایک ایسا ملک ہے جو جنوبی ایشیا میں واقع ہے۔ اس کی سرحدیں ہندوستان، چین، افغانستان، اور " |
| encoded = tokenizer.encode(prompt) |
| encoded_tensor = torch.tensor(encoded).unsqueeze(0) |
| output = model.generate(encoded_tensor, max_new_tokens=64) |
| |
| response = tokenizer.decode(output[0].squeeze().tolist()) |
| ``` |
|
|
| --- |
| license: apache-2.0 |
| --- |
