EADST

Transformers Demo for DeepSeek-R1-Distill-Qwen-7B

Transformers Demo for DeepSeek-R1-Distill-Qwen-7B

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "/your_deepseek-ai_DeepSeek-R1-Distill-Qwen-7B_path"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=2048
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=False)[0] # show special tokens

print("Question: \n", text)
print("Answer: \n", response)
相关标签
About Me
XD
Goals determine what you are going to be.
Category
标签云
ResNet-50 Animate Rebuttal Distillation VPN Quantization Input Disk git-lfs Baidu Tensor DeepSeek Django 论文速读 Augmentation LLM GGML 个人博客 Mixtral InvalidArgumentError Plate Video PyCharm Knowledge Food Claude Dataset Conda RAR Paper printf Zip Crawler Ptyhon ONNX ModelScope IndexTTS2 RL 阿里云 Base64 NLTK SQL FlashAttention CLAP C++ BeautifulSoup EXCEL Pandas 净利润 Qwen Hilton GoogLeNet Cloudreve YOLO git Miniforge MD5 SPIE Github CC LLAMA uWSGI v0.dev LaTeX COCO Template torchinfo 签证 hf Windows SQLite SAM Harness ChatGPT Ubuntu TensorFlow CAM Anaconda 报税 Shortcut Bert CSV FastAPI Safetensors Translation diffusers Tracking Python 音频 Quantize NameSilo Data 版权 Gemma 论文 Vmess Breakpoint Image2Text 域名 GPT4 Nginx Jupyter Bipartite NLP API网关 TTS uwsgi UI FP32 FP16 Paddle Datetime LeetCode News SVR Pillow Heatmap Bitcoin Michelin Pytorch Attention 证件照 BF16 VGG-16 OpenCV CTC 强化学习 Logo CUDA Proxy Excel 递归学习法 VSCode CEIR Use QWEN Google Qwen2.5 Password RGB Statistics GPTQ BTC Agent FP64 HuggingFace PDB LoRA Permission DeepStream UNIX Jev Git Linux Color Card Numpy Clash 财报 v2ray PyTorch transformers 飞书 logger Review 继承 OCR WebCrawler 公式 Plotly 第一性原理 Interview 腾讯云 GIT Math Firewall PDF icon mmap JSON 云服务器 CV TensorRT Search 长期记忆 Diagram 算法题 Domain Bin llama.cpp Jetson scipy XML FP8 Pickle Land AI tqdm Sklearn PIP 顶会 图标 XGBoost 搞笑 多线程 Freesound API Tiktoken tar Transformers Magnet Qwen2 Docker Hungarian Markdown Vim OpenAI TSV Algorithm Streamlit 多进程 Llama Random 关于博主 图形思考法 WAN Web HaggingFace ms-swift Hotel 自我改进 Website
站点统计

本站现有博文340篇,共被浏览1019312次

本站已经建立2719天!

热门文章
文章归档
回到顶部