Hugging Face xet & datasets download cache: safe to delete?

What it is

The huggingface_hub and datasets libraries keep intermediate download data outside the main model cache. The xet folder is a chunk cache used by the Xet storage backend to speed up and deduplicate downloads, and datasets/downloads holds the raw files the datasets library fetched before converting them into its own format.

Why it gets so big

Raw dataset downloads are often large archives or CSV, JSON or Parquet files kept alongside the processed copy, so you effectively store each dataset twice. The xet chunk cache grows as you pull more models and files.

Is it safe to delete?

Yes. These are intermediate copies: models already in the hub cache and datasets already processed will still load. If something needs the raw files again, they're downloaded again. Nothing here is unique to you.

Tip

Setting HF_HOME moves all of Hugging Face's caches to another location, such as an external drive, if your internal disk is tight.