WYF3634076 commited on
Commit
a952c9a
·
verified ·
1 Parent(s): 91f9254

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -104,7 +104,7 @@ print("generate_text:", generate_text)
104
 
105
  ### Using vLLM
106
 
107
- vLLM is currently being adapted, priority can be given to using our forked repository [vllm](https://github.com/CSWYF3634076/vllm/tree/ernie). We are working with the community to fully support ERNIE4.5 models, stay tuned.
108
 
109
  ```bash
110
  # 80G * 16 GPU
@@ -112,7 +112,7 @@ vllm serve baidu/ERNIE-4.5-300B-A47B-PT --trust-remote-code
112
  ```
113
 
114
  ```bash
115
- # FP8 online quantification 80G * 8 GPU
116
  vllm serve baidu/ERNIE-4.5-300B-A47B-PT --trust-remote-code --quantization fp8
117
  ```
118
 
 
104
 
105
  ### Using vLLM
106
 
107
+ [vllm](https://github.com/vllm-project/vllm/tree/main) github library. Python-only [build](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html#set-up-using-python-only-build-without-compilation).
108
 
109
  ```bash
110
  # 80G * 16 GPU
 
112
  ```
113
 
114
  ```bash
115
+ # FP8 online quantification 80G * 16 GPU
116
  vllm serve baidu/ERNIE-4.5-300B-A47B-PT --trust-remote-code --quantization fp8
117
  ```
118