Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/hub/datasets-adding.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,6 @@ After uploading your dataset, make sure the Dataset Viewer correctly shows your

The Hugging Face Hub supports large scale datasets, usually uploaded in Parquet (e.g. via `push_to_hub()` using [🤗 Datasets](/docs/datasets/main/en/package_reference/main_classes#datasets.Dataset.push_to_hub)) or [WebDataset](https://github.com/webdataset/webdataset) format.

You can upload large scale datasets at high speed using the `huggingface_hub` library.
Use `upload_folder()` or `hf upload` from the `huggingface_hub` library to upload large folders: they split the upload into several commits and resume automatically if interrupted.

See [how to upload a folder by chunks](/docs/huggingface_hub/guides/upload#upload-a-large-folder), the [tips and tricks for large uploads](/docs/huggingface_hub/guides/upload#tips-and-tricks-for-large-uploads) and the [repository storage limits and recommendations](./storage-limits).
See [how to upload a large folder](/docs/huggingface_hub/guides/upload#upload-a-large-folder), the [tips and tricks for large uploads](/docs/huggingface_hub/guides/upload#tips-and-tricks-for-large-uploads) and the [repository storage limits and recommendations](./storage-limits).
2 changes: 1 addition & 1 deletion docs/hub/datasets-upload-guide-llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -433,7 +433,7 @@ hf upload username/dataset ./data --repo-type=dataset

## Dataset Viewer Configuration

**Note**: This section is primarily for datasets uploaded directly to the Hub (via UI or `upload_large_folder`). Datasets uploaded with `push_to_hub()` typically configure the viewer automatically.
**Note**: This section is primarily for datasets uploaded directly to the Hub (via the UI, `upload_folder` or `hf upload`). Datasets uploaded with `push_to_hub()` typically configure the viewer automatically.

### When automatic detection works

Expand Down