lightx2v commited on
Commit
537d519
Β·
verified Β·
1 Parent(s): 235c3a6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +203 -0
README.md CHANGED
@@ -9,3 +9,206 @@ base_model:
9
  - Wan-AI/Wan2.2-I2V-A14B
10
  library_name: diffusers
11
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  - Wan-AI/Wan2.2-I2V-A14B
10
  library_name: diffusers
11
  ---
12
+ # 🎬 Wan2.2 Distilled Models
13
+
14
+ ### ⚑ High-Performance Video Generation with 4-Step Inference
15
+
16
+ *Distillation-accelerated version of Wan2.2 - Dramatically faster speed with excellent quality*
17
+
18
+ ![img_lightx2v](https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/tTnp8-ARpj3wGxfo5P55c.png)
19
+
20
+ ---
21
+
22
+ [![πŸ€— HuggingFace](https://img.shields.io/badge/πŸ€—-HuggingFace-yellow)](https://huggingface.co/lightx2v/Wan2.2-Distill-Models)
23
+ [![GitHub](https://img.shields.io/badge/GitHub-LightX2V-blue?logo=github)](https://github.com/ModelTC/LightX2V)
24
+ [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](LICENSE)
25
+
26
+ ---
27
+
28
+ ## 🌟 What's Special?
29
+
30
+ <table>
31
+ <tr>
32
+ <td width="50%">
33
+
34
+ ### ⚑ Ultra-Fast Generation
35
+ - **4-step inference** (vs traditional 50+ steps)
36
+ - Approximately **2x faster** using LightX2V than ComfyUI
37
+ - Near real-time video generation capability
38
+
39
+ </td>
40
+ <td width="50%">
41
+
42
+ ### 🎯 Flexible Options
43
+ - **Dual noise control**: High/Low noise variants
44
+ - Multiple precision formats (BF16/FP8/INT8)
45
+ - Full 14B parameter models
46
+
47
+ </td>
48
+ </tr>
49
+ <tr>
50
+ <td width="50%">
51
+
52
+ ### πŸ’Ύ Memory Efficient
53
+ - FP8/INT8: **~50% size reduction**
54
+ - CPU offload support
55
+ - Optimized for consumer GPUs
56
+
57
+ </td>
58
+ <td width="50%">
59
+
60
+ ### πŸ”§ Easy Integration
61
+ - Compatible with LightX2V framework
62
+ - ComfyUI support
63
+ - Simple configuration files
64
+
65
+ </td>
66
+ </tr>
67
+ </table>
68
+
69
+ ---
70
+
71
+ ## πŸ“¦ Model Catalog
72
+
73
+ ### πŸŽ₯ Model Types
74
+
75
+ <table>
76
+ <tr>
77
+ <td align="center" width="50%">
78
+
79
+ #### πŸ–ΌοΈ **Image-to-Video (I2V) - 14B Parameters**
80
+ Transform static images into dynamic videos with advanced quality control
81
+
82
+ - 🎨 **High Noise**: More creative, diverse outputs
83
+ - 🎯 **Low Noise**: More faithful to input, stable outputs
84
+
85
+ </td>
86
+ <td align="center" width="50%">
87
+
88
+ #### πŸ“ **Text-to-Video (T2V) - 14B Parameters**
89
+ Generate videos from text descriptions
90
+
91
+ - 🎨 **High Noise**: More creative, diverse outputs
92
+ - 🎯 **Low Noise**: More stable and controllable outputs
93
+ - πŸš€ Full 14B parameter model
94
+
95
+ </td>
96
+ </tr>
97
+ </table>
98
+
99
+ ### 🎯 Precision Versions
100
+
101
+ | Precision | Model Identifier | Model Size | Framework | Quality vs Speed |
102
+ |:---------:|:-----------------|:----------:|:---------:|:-----------------|
103
+ | πŸ† **BF16** | `lightx2v_4step` | ~28.6 GB | LightX2V | ⭐⭐⭐⭐⭐ Highest Quality |
104
+ | ⚑ **FP8** | `scaled_fp8_e4m3_lightx2v_4step` | ~15 GB | LightX2V | ⭐⭐⭐⭐ Excellent Balance |
105
+ | 🎯 **INT8** | `int8_lightx2v_4step` | ~15 GB | LightX2V | ⭐⭐⭐⭐ Fast & Efficient |
106
+ | πŸ”· **FP8 ComfyUI** | `scaled_fp8_e4m3_lightx2v_4step_comfyui` | ~15 GB | ComfyUI | ⭐⭐⭐ ComfyUI Ready |
107
+
108
+ ### πŸ“ Naming Convention
109
+
110
+ ```bash
111
+ # Format: wan2.2_{task}_A14b_{noise_level}_{precision}_lightx2v_4step.safetensors
112
+
113
+ # I2V Examples:
114
+ wan2.2_i2v_A14b_high_noise_lightx2v_4step.safetensors # I2V High Noise - BF16
115
+ wan2.2_i2v_A14b_high_noise_scaled_fp8_e4m3_lightx2v_4step.safetensors # I2V High Noise - FP8
116
+ wan2.2_i2v_A14b_low_noise_int8_lightx2v_4step.safetensors # I2V Low Noise - INT8
117
+ wan2.2_i2v_A14b_low_noise_scaled_fp8_e4m3_lightx2v_4step_comfyui.safetensors # I2V Low Noise - FP8 ComfyUI
118
+
119
+ ```
120
+ > πŸ’‘ **Browse All Models**: [View Full Model Collection β†’](https://huggingface.co/lightx2v/Wan2.2-Distill-Models/tree/main)
121
+
122
+ ---
123
+
124
+ ## πŸš€ Usage
125
+
126
+ ### Method 1: LightX2V (Recommended ⭐)
127
+
128
+ **LightX2V is a high-performance inference framework optimized for these models, approximately 2x faster than ComfyUI with better quantization accuracy. Highly recommended!**
129
+
130
+ #### Quick Start
131
+
132
+ 1. Download model (using I2V FP8 as example)
133
+ ```bash
134
+ huggingface-cli download lightx2v/Wan2.2-Distill-Models \
135
+ --local-dir ./models/wan2.2_i2v \
136
+ --include "wan2.2_i2v_A14b_high_noise_scaled_fp8_e4m3_lightx2v_4step.safetensors"
137
+ ```
138
+
139
+ ```bash
140
+ huggingface-cli download lightx2v/Wan2.2-Distill-Models \
141
+ --local-dir ./models/wan2.2_i2v \
142
+ --include "wan2.2_i2v_A14b_low_noise_scaled_fp8_e4m3_lightx2v_4step.safetensors"
143
+ ```
144
+
145
+ > πŸ’‘ **Tip**: For T2V models, follow the same steps but replace `i2v` with `t2v` in the filenames
146
+
147
+ 2. Clone LightX2V repository
148
+
149
+ ```bash
150
+ git clone https://github.com/ModelTC/LightX2V.git
151
+ cd LightX2V
152
+ ```
153
+
154
+ 3. Install dependencies
155
+
156
+ ```bash
157
+ pip install -r requirements.txt
158
+ ```
159
+ Or refer to [Quick Start Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/quickstart.html) to use docker
160
+
161
+ 4. Select and modify configuration file
162
+
163
+ Choose appropriate configuration based on your GPU memory:
164
+
165
+ **80GB+ GPUs (A100/H100)**
166
+ - I2V: [wan_moe_i2v_distill.json](https://github.com/ModelTC/LightX2V/blob/main/configs/wan22/wan_moe_i2v_distill.json)
167
+
168
+ **24GB+ GPUs (RTX 4090)**
169
+ - I2V: [wan_moe_i2v_distill_4090.json](https://github.com/ModelTC/LightX2V/blob/main/configs/wan22/wan_moe_i2v_distill_4090.json)
170
+
171
+
172
+ 5. Run inference (using [I2V]((https://github.com/ModelTC/LightX2V/blob/main/scripts/wan22/run_wan22_moe_i2v_distill.sh)) as example)
173
+ ```bash
174
+ cd scripts
175
+ bash wan22/run_wan22_moe_i2v_distill.sh
176
+ ```
177
+
178
+ > πŸ“ **Note**: Update model paths in the script to point to your Wan2.2 model. Also refer to [LightX2V Model Structure Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/model_structure.html)
179
+
180
+
181
+ #### LightX2V Documentation
182
+ - **Quick Start Guide**: [LightX2V Quick Start](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/quickstart.html)
183
+ - **Complete Usage Guide**: [LightX2V Model Structure Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/model_structure.html)
184
+ - **Configuration File Instructions**: [Configuration Files](https://github.com/ModelTC/LightX2V/tree/main/configs/distill)
185
+ - **Quantized Model Usage**: [Quantization Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/quantization.html)
186
+ - **Parameter Offloading**: [Offload Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/offload.html)
187
+
188
+ ---
189
+
190
+ ### Method 2: ComfyUI
191
+
192
+ Please refer to [workflow](https://huggingface.co/lightx2v/Wan2.2-Distill-Models/blob/main/wan2.2_moe_i2v_scale_fp8_comfyui.json)
193
+
194
+
195
+
196
+ ## ⚠️ Important Notes
197
+
198
+ **Other Components**: These models only contain DIT weights. Additional components needed at runtime:
199
+ - T5 text encoder
200
+ - CLIP vision encoder
201
+ - VAE encoder/decoder
202
+ - Tokenizer
203
+
204
+ Please refer to [LightX2V Documentation](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/model_structure.html) for instructions on organizing the complete model directory.
205
+
206
+
207
+ ## 🀝 Community
208
+
209
+ - **GitHub Issues**: https://github.com/ModelTC/LightX2V/issues
210
+ - **HuggingFace**: https://huggingface.co/lightx2v/Wan2.2-Distill-Models
211
+
212
+ If you find this project helpful, please give us a ⭐ on [GitHub](https://github.com/ModelTC/LightX2V)
213
+
214
+ </div>