Update README.md
Browse files
README.md
CHANGED
|
@@ -6,24 +6,20 @@ inference: false
|
|
| 6 |
<img src="seal_logo.png" width="200" />
|
| 7 |
</p>
|
| 8 |
|
| 9 |
-
#
|
| 10 |
|
| 11 |
|
| 12 |
-
<!-- - DEMO: [DAMO-NLP-SG/damo-seal-v0](https://huggingface.co/spaces/DAMO-NLP-SG/damo-seal-v0) -->
|
| 13 |
-
|
| 14 |
<p align="center">
|
| 15 |
-
๐ค <a href="https://huggingface.co/spaces/
|
| 16 |
</p>
|
| 17 |
|
| 18 |
-
We introduce SeaLLM - a family of language models optimized for
|
| 19 |
|
| 20 |
-
The [SeaLLM-chat](https://huggingface.co/spaces/
|
| 21 |
|
| 22 |
Our customized SFT process helps enhance our models' ability to understand, respond and serve communities whose languages are often neglected by previous [English-dominant LLMs](https://arxiv.org/abs/2307.09288), while outperforming existing polyglot LLMs, like [BLOOM](https://arxiv.org/abs/2211.05100) or [PolyLM](https://arxiv.org/pdf/2307.06018.pdf).
|
| 23 |
|
| 24 |
-
Our [first released SeaLLM](https://huggingface.co/spaces/
|
| 25 |
-
|
| 26 |
-
<!-- - Model links: [DAMO-NLP-SG/seal-13b-chat-a](https://huggingface.co/DAMO-NLP-SG/seal-13b-chat-a) -->
|
| 27 |
|
| 28 |
|
| 29 |
<blockquote style="color:red">
|
|
@@ -204,7 +200,7 @@ If you find our project useful, hope you can star our repo and cite our work as
|
|
| 204 |
```
|
| 205 |
@article{damonlpsg2023seallm,
|
| 206 |
author = {???},
|
| 207 |
-
title = {
|
| 208 |
year = 2023,
|
| 209 |
}
|
| 210 |
```
|
|
|
|
| 6 |
<img src="seal_logo.png" width="200" />
|
| 7 |
</p>
|
| 8 |
|
| 9 |
+
# SeaLLMs - Large Language Models for Southeast Asia
|
| 10 |
|
| 11 |
|
|
|
|
|
|
|
| 12 |
<p align="center">
|
| 13 |
+
๐ค <a href="https://huggingface.co/spaces/SeaLLMs/SeaLLM-chat-13b-demo">Hugging Face DEMO</a>
|
| 14 |
</p>
|
| 15 |
|
| 16 |
+
We introduce SeaLLM - a family of language models optimized for Southeast Asian (SEA) languages. The SeaLLM-base models (to be released) were pre-trained from [Llama-2](https://huggingface.co/meta-llama/Llama-2-13b-hf), on a tailored publicly-available dataset, which comprises mainly Vietnamese ๐ป๐ณ, Indonesian ๐ฎ๐ฉ and Thai ๐น๐ญ texts, along with those in English ๐ฌ๐ง and Chinese ๐จ๐ณ. The pre-training stage involves multiple stages with dynamic data control to preserve the original knowledge base of Llama-2 while gaining new abilities in SEA languages.
|
| 17 |
|
| 18 |
+
The [SeaLLM-chat](https://huggingface.co/spaces/SeaLLMs/SeaLLM-chat-13b-demo) model underwent supervised finetuning (SFT) on a mix of public instruction data (e.g. [OpenORCA](https://huggingface.co/datasets/Open-Orca/OpenOrca)) and a small internally-collected amount of natural queries from SEA native speakers, which **adapt to the local cultural norms, customs, styles and laws in these regions**, as well as other SFT enhancement techniques (to be revealed later).
|
| 19 |
|
| 20 |
Our customized SFT process helps enhance our models' ability to understand, respond and serve communities whose languages are often neglected by previous [English-dominant LLMs](https://arxiv.org/abs/2307.09288), while outperforming existing polyglot LLMs, like [BLOOM](https://arxiv.org/abs/2211.05100) or [PolyLM](https://arxiv.org/pdf/2307.06018.pdf).
|
| 21 |
|
| 22 |
+
Our [first released SeaLLM](https://huggingface.co/spaces/SeaLLMs/SeaLLM-chat-13b-demo) supports Vietnamese ๐ป๐ณ, Indonesian ๐ฎ๐ฉ and Thai ๐น๐ญ. Future verions endeavor to cover all languages spoken in Southeast Asia.
|
|
|
|
|
|
|
| 23 |
|
| 24 |
|
| 25 |
<blockquote style="color:red">
|
|
|
|
| 200 |
```
|
| 201 |
@article{damonlpsg2023seallm,
|
| 202 |
author = {???},
|
| 203 |
+
title = {SeaLLMs - Large Language Models for Southeast Asia},
|
| 204 |
year = 2023,
|
| 205 |
}
|
| 206 |
```
|