allenai

/

Olmo-3-7B-Think

like 65

Follow

Ai2 4.73k

Hi, first, thank you for making this model :)

When using LLM Compressor to quantize in FP8, I got the following error when first loading the model:

model = AutoModelForCausalLM.from_pretrained(MODEL_ID, dtype=torch.bfloat16, low_cpu_mem_usage=True, device_map="cuda", local_files_only=True)
rope_scaling's beta_fast field must be a float, got 32
rope_scaling's beta_slow field must be a float, got 1

It might be my local environment only (my transformers version is "4.57.3"), but just in case you observe the same.