| --- |
| license: apache-2.0 |
| datasets: |
| - AlexFierro9/Kinetics400 |
| - imagenet-1k |
| - HuggingFaceM4/something_something_v2 |
| language: |
| - en |
| pipeline_tag: video-classification |
| extra_gated_fields: |
| Name: text |
| Company/Organization: text |
| Country: text |
| E-Mail: text |
| --- |
| |
|
|
|
|
| <br> |
|
|
| # VideoMamba |
|
|
| ## Model Details |
|
|
| VideoMamba is a purely SSM-based model for video understanding. |
|
|
| - **Developed by:** [OpenGVLab](https://github.com/OpenGVLab) |
| - **Model type:** An efficient backbone based on the bidirectional state space model. |
| - **License:** Non-commercial license |
|
|
|
|
| ### Model Sources |
|
|
| - **Repository:** https://github.com/OpenGVLab/VideoMamba |
| - **Paper:** https://arxiv.org/abs/2403.06977 |
|
|
| ## Uses |
|
|
| The primary use of VideoMamba is research on image and video tasks, e.g., image classification, action recognition, long-term video understanding, and video-text retrieval, with an SSM-based backbone. |
| The primary intended users of the model are researchers and hobbyists in computer vision, machine learning, and artificial intelligence. |
|
|
| ## How to Get Started with the Model |
|
|
| - You can replace the backbone for video tasks with the proposed VideoMamba: https://github.com/OpenGVLab/VideoMamba/blob/main/videomamba/video_sm/models/videomamba.py |
| - Then you can load this checkpoint and start training. |
| |
| |
| ### Citation Information |
| |
| ``` |
| @misc{li2024videomamba, |
| title={VideoMamba: State Space Model for Efficient Video Understanding}, |
| author={Kunchang Li and Xinhao Li and Yi Wang and Yinan He and Yali Wang and Limin Wang and Yu Qiao}, |
| year={2024}, |
| eprint={2403.06977}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV} |
| } |
| ``` |
