EADST

Sharding and SafeTensors in Hugging Face Transformers

In the Hugging Face transformers library, managing large models efficiently is crucial, especially when working with limited disk space or specific file size requirements. Two key features that help with this are sharding and the use of SafeTensors.

Sharding

Sharding is the process of splitting a large model's weights into smaller files or "shards." This is particularly useful when dealing with large models that exceed file size limits or when you want to manage storage more effectively.

Usage

To shard a model during the saving process, you can use the max_shard_size parameter in the save_pretrained method. Here's an example:

# Save the model with sharding, setting the maximum shard size to 1GB
model.save_pretrained('./model_directory', max_shard_size="1GB")

In this example, the model's weights will be divided into multiple files, each not exceeding 1GB. This can make storage and transfer more manageable, especially when dealing with large-scale models.

SafeTensors

The safetensors library provides a new format for storing tensors in a safe and efficient way. Unlike traditional formats like PyTorch's .pt files, SafeTensors ensures that the tensor data cannot be accidentally executed as code, offering an additional layer of security. This is particularly important when sharing models across different systems or with the community.

Usage

To save a model using SafeTensors, simply specify the safe_serialization parameter when saving:

# Save the model using SafeTensors format
model.save_pretrained('./model_directory', safe_serialization=True)

This will create files with the .safetensors extension, ensuring the saved tensors are stored safely.

Combining Sharding and SafeTensors

You can combine both sharding and SafeTensors to save a large model securely and efficiently:

# Save the model with sharding and SafeTensors
model.save_pretrained('./model_directory', max_shard_size="1GB", safe_serialization=True)

This setup splits the model into shards, each in the SafeTensors format, offering both manageability and security.

Conclusion

By leveraging sharding and SafeTensors, Hugging Face transformers users can handle large models more effectively. Sharding helps manage file sizes, while SafeTensors ensures the safe storage of tensor data. These features are essential for anyone working with large-scale models, providing both practical and security benefits.

相关标签
About Me
XD
Goals determine what you are going to be.
Category
标签云
Pytorch Markdown NameSilo v0.dev JSON AI LLM 音频 DeepSeek HaggingFace Numpy 报税 SQL Jetson UNIX CV EXCEL uwsgi Conda 多线程 Base64 CAM Transformers Disk 签证 腾讯云 PDF llama.cpp FP16 Web SPIE Ubuntu Shortcut ModelScope Tensor Algorithm Clash Hotel MD5 CC CSV 公式 InvalidArgumentError Distillation Heatmap Google Hilton icon 多进程 mmap Interview Augmentation BeautifulSoup printf Breakpoint Bert Safetensors Michelin Firewall Review 强化学习 GoogLeNet 图标 Proxy git FastAPI 版权 CEIR XGBoost Statistics Vim Translation Anaconda Sklearn UI diffusers Password Pickle Mixtral ms-swift Bitcoin SQLite Pandas RAR Attention 飞书 Logo Claude 净利润 Excel 图形思考法 Template XML BTC Input Card API Website tqdm Quantization Use C++ git-lfs SAM 云服务器 FlashAttention Jev OpenAI News Paddle 自我改进 Hungarian Jupyter hf Docker 阿里云 RL Plate Tiktoken LLAMA CUDA Rebuttal NLTK logger ONNX TSV Django WAN Animate Git LoRA Dataset OCR 继承 GPTQ Quantize Baidu RGB Datetime GPT4 Linux 算法题 OpenCV Diagram Knowledge uWSGI GIT Agent CTC torchinfo YOLO Math Pillow Tracking VPN VSCode HuggingFace IndexTTS2 顶会 论文速读 Miniforge 域名 API网关 Nginx TTS 个人博客 BF16 FP8 LeetCode QWEN Github Streamlit Color Cloudreve Llama VGG-16 GGML tar 财报 FP64 TensorFlow Bin 证件照 Permission FP32 COCO 递归学习法 WebCrawler Qwen2.5 Random TensorRT Freesound CLAP v2ray SVR transformers LaTeX Crawler PyTorch Search Paper 论文 ChatGPT PDB Python Land Harness PyCharm Ptyhon Video Gemma NLP Bipartite Magnet 关于博主 scipy PIP Zip ResNet-50 第一性原理 Vmess Data Domain 搞笑 长期记忆 DeepStream Image2Text Food Qwen Qwen2 Plotly Windows
站点统计

本站现有博文340篇,共被浏览1021392次

本站已经建立2719天!

热门文章
文章归档
回到顶部