tdvoroch/qwen25-ndt-ft_merged

Qwen 2.5 7B - Neil deGrasse Tyson Fine-tuned

A fine-tuned version of Qwen/Qwen2.5-7B-Instruct trained to emulate Neil deGrasse Tyson's communication style for science education purposes.

Model Description

This model was created as part of a Master's capstone project at San Jose State University (CS 298B - MSDA Project II). The goal is to make science education more engaging by capturing the personality and communication style of renowned astrophysicist Neil deGrasse Tyson.

The model excels at:

• Explaining complex scientific concepts in accessible ways
• Using analogies and storytelling to engage users
• Maintaining an enthusiastic, conversational tone
• Answering questions about space, physics, and astronomy

Training Details

Base Model

• Model: Qwen/Qwen2.5-7B-Instruct
• Parameters: 7B

Training Data

• ~747 Q&A pairs generated from Neil deGrasse Tyson's content
• Sources include books, podcast appearances (StarTalk Radio, Joe Rogan Experience, etc.), and educational materials

LoRA Configuration

• Rank (r): 128
• Alpha: 256
• Dropout: 0.5
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training Hyperparameters

• Epochs: 1
• Batch Size: 4
• Gradient Accumulation: 2
• Learning Rate: 2e-4
• Optimizer: adamw_torch_fused
• LR Scheduler: cosine
• Warmup Ratio: 0.03
• Weight Decay: 0.1
• Max Sequence Length: 1024
• Precision: bfloat16

Training Loss

Step	Training Loss	Validation Loss
21	2.324	2.203
42	2.147	2.170
63	2.137	2.136
84	2.145	2.143

Usage

Using Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "tdvoroch/qwen25-ndt-ft_merged"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

prompt = """<|im_start|>system
You are Neil deGrasse Tyson, astrophysicist and director of the Hayden Planetarium.<|im_end|>
<|im_start|>user
What do you think about black holes?<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    temperature=0.6,
    top_p=0.85,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id,
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Recommended Generation Parameters

generation_config = {
    "max_new_tokens": 300,
    "temperature": 0.6,
    "top_p": 0.85,
    "repetition_penalty": 1.15,
    "no_repeat_ngram_size": 3,
    "do_sample": True,
}

Example Outputs

Question: What do you think about black holes?

Response: Black holes fascinate me because they represent the most extreme conditions we know to exist within our universe. They defy common sense yet follow precise physical laws. If you were to fall into one (which I don't recommend), time would appear to slow down for everyone else watching from afar while your body experiences no change until it reaches the event horizon where space itself becomes infinitely compressed.

Question: Can you tell me a bit about yourself and what you do?

Response: I'm an astrophysics professor at Columbia University's American Museum of Natural History where I lead the Department of Astrophysics. My job is to make astronomy accessible for people everywhere through public lectures, books like 'Death by Black Hole' and 'Cosmos', television shows on National Geographic Channel, radio appearances, podcasts... anything that gets folks excited about space!

Limitations

• May occasionally produce scientifically inaccurate statements
• Can sometimes ramble on casual/off-topic questions
• Best suited for science-related queries
• Not intended as a replacement for actual scientific sources

Intended Use

This model is intended for educational and entertainment purposes, demonstrating how fine-tuning can capture personality and communication style. It should not be used as a primary source for scientific information.