YangKai0616 commited on
Commit
a590715
·
verified ·
1 Parent(s): 9e9a4c5

Fix vision_config.model_type

Browse files

Fixed the `model_type` error in `vision_config`. This PR allows the visual model to be loaded correctly instead of loading the entire GLM-4V model.

A simple reproduction script is as follows:
```
import torch
from transformers import Glm46VForConditionalGeneration


def main():
model = Glm46VForConditionalGeneration.from_pretrained(
"THUDM/GLM-4.1V-9B-Thinking",
dtype="auto",
device_map="auto",
)

visual = model.model.visual
language = model.model.language_model

print(f"type(model.model.visual): {type(visual)}")
print(f"type(model.model.language_model): {type(language)}")

assert visual.__class__.__name__ == "Glm4vVisionModel", (
"vision_config mistakenly sets model_type='glm4v', so AutoModel.from_config "
"instantiates the full Glm4vModel (visual + language) instead of the pure "
"vision backbone Glm4vVisionModel."
)


if __name__ == "__main__":
main()
```

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -46,7 +46,7 @@
46
  }
47
  },
48
  "vision_config": {
49
- "model_type": "glm4v",
50
  "hidden_size": 1536,
51
  "depth": 24,
52
  "num_heads": 12,
 
46
  }
47
  },
48
  "vision_config": {
49
+ "model_type": "glm4v_vision",
50
  "hidden_size": 1536,
51
  "depth": 24,
52
  "num_heads": 12,