OPEA
/

Qwen2.5-14B-Instruct-int4-sym-inc

4-bit precision

Model card Files Files and versions

cicdatopea commited on Nov 29, 2024

Commit

68c5c2b

·

verified ·

1 Parent(s): 3a8f745

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -23,7 +23,7 @@ model = AutoModelForCausalLM.from_pretrained(
     quantized_model_dir,
     torch_dtype='auto',
     device_map="auto",
-    ##revision="f86a564" ##AutoGPTQ format
 )
 ##import habana_frameworks.torch.core as htcore ## uncommnet it for HPU

     quantized_model_dir,
     torch_dtype='auto',
     device_map="auto",
+    ##revision="98b3137" ##AutoGPTQ format
 )
 ##import habana_frameworks.torch.core as htcore ## uncommnet it for HPU