EADST

LLAMA Model Save with INT8 Format

LLAMA Model Save with INT8 Format

from transformers import BitsAndBytesConfig
from transformers import AutoModelForCausalLM

config = BitsAndBytesConfig(
    load_in_8bit=True,
)
path = "/home/llm/model/path/"
model = AutoModelForCausalLM.from_pretrained(path, device_map="cpu", quantization_config=config)
model.save_pretrained("model_save_folder-8bit")
相关标签
About Me
XD
Goals determine what you are going to be.
Category
标签云
Cloudreve Vmess mmap HaggingFace 报税 Zip Sklearn Food 第一性原理 Distillation PDB SQLite LoRA torchinfo Permission DeepStream Harness logger Review SQL TSV Clash 域名 Quantize Color Logo News OpenCV 顶会 GGML 算法题 Land SPIE YOLO Website JSON VPN Diagram CAM CLAP GoogLeNet LLAMA SAM 公式 Data Disk BTC Miniforge 图标 音频 Template SVR Dataset 自我改进 ModelScope CUDA Tiktoken transformers BF16 ResNet-50 QWEN ChatGPT icon 多进程 Django Interview LeetCode VGG-16 tar Firewall C++ Hilton Claude Llama CSV uWSGI CTC Gemma FP16 Bert Crawler Shortcut VSCode WebCrawler Numpy Paddle 版权 论文速读 OpenAI Tracking 多线程 Vim Safetensors Windows Rebuttal AI LaTeX DeepSeek TensorFlow Input API网关 递归学习法 CC Statistics Git v0.dev PIP Github FP32 hf Heatmap Pickle Docker printf HuggingFace diffusers PyTorch 长期记忆 PyCharm Datetime WAN 关于博主 Animate CEIR Mixtral Proxy 签证 Excel UI Video FastAPI NLP Algorithm Bin NLTK Pillow 图形思考法 Password Python Baidu Random Jev LLM PDF CV Base64 EXCEL GIT git-lfs Math UNIX git Jupyter TTS Plate NameSilo Attention BeautifulSoup ONNX Image2Text MD5 TensorRT 飞书 RGB scipy Knowledge Paper Pytorch 财报 Domain Pandas API OCR ms-swift Linux Quantization Magnet Breakpoint FlashAttention Freesound Streamlit Translation Plotly 阿里云 Ubuntu Google 腾讯云 净利润 llama.cpp 云服务器 Jetson Card Qwen Bipartite GPTQ Search Nginx Qwen2.5 搞笑 Conda RL Hotel Web Ptyhon Agent 证件照 IndexTTS2 FP8 v2ray 继承 强化学习 论文 Augmentation Qwen2 Hungarian Transformers XGBoost Bitcoin FP64 XML 个人博客 Markdown GPT4 Use InvalidArgumentError Anaconda uwsgi Michelin COCO tqdm RAR Tensor
站点统计

本站现有博文340篇,共被浏览1017826次

本站已经建立2718天!

热门文章
文章归档
回到顶部