About

The files of a dataset, of any type a hub takes: their registry of file types, storage by key, upload staging, view, raw and download, and the file workbench of the dataset page

Provides

Models
HubfileHubfileAliasHubfileDownloadRecordHubfileViewRecordStagedFileUploadSession
Services
HubfileDownloadRecordServiceHubfileServiceHubfileViewRecordServiceStorageServiceUploadIngestServiceUploadSessionService
Routes
/datasets/unsynchronized/<int:dataset_id>/files/<int:file_id>/doi/<path:doi>/files/<string:filename>/doi/<path:doi>/files/raw/<path:filename>/doi/<path:doi>/files/raw/<path:filename>//hubfile/clear_temp/hubfile/delete/hubfile/list-github-files/hubfile/upload/hubfile/upload-github/hubfiles/<int:file_id>/workbench-content/hubfiles/download/<int:file_id>/hubfiles/raw/<int:file_id>/<path:filename>/records/<string:public_id>/files/<string:filename>/records/<string:public_id>/files/raw/<path:filename>
Hooks filled
dataset.form.scriptsdataset.form.uploaddataset.view.scriptsdataset.view.workbench
Signals
hubfile-createdhubfile-downloadedhubfile-viewed
Translations
es

Commands

  • $ splent feature:hubfile storage-migrate [OPTIONS]

    Move the files of the old layout (user_<owner>/dataset_<id>/...) to their keys.

  • $ splent feature:hubfile uploads-cleanup [OPTIONS]

    Remove the expired uploads under way and the old temporaries.

Run them in your workspace once the feature is installed in the selected product.

Configuration

Setting Default What it does
HUBFILE_TASKS_MODE empty No description yet.
HUBFILE_STORAGE_ROOT empty Where the files are kept, by key (storage.py). Empty is the "hubfile" folder of the protected uploads (PROTECTED_UPLOADS_DIR), where hubfile 0.4 kept them, so in production it lives on that volume and moving the old layout to the keys is a rename. A folder elsewhere has to be a volume too.
HUBFILE_STORAGE_BACKEND local Which store holds them. Only "local", a folder, for now.
HUBFILE_UPLOAD_EXPIRY_HOURS 24 How long an upload under way keeps its staged files after the last one came, and how old the other temporaries may get, before the sweep (splent feature:hubfile uploads-cleanup) removes them.
HUBFILE_FILE_EXTENSIONS empty The generic file type, for a hub without the UVL layer: the extensions it takes, whatever their case (["*"] takes any file), and how big a file of it may be. Empty takes nothing, so only the types other features register (uvl's UVL models) are taken.
HUBFILE_FILE_MAX_SIZE 20M No description yet.
HUBFILE_UNPACK_ARCHIVES true Whether a ZIP that comes in is unpacked into the files it holds, its folders kept. False keeps a ZIP as a file, which a type has to take (["*"] does), within that type's largest file.
HUBFILE_ZIP_MAX_SIZE 100M The largest ZIP an upload takes, before it unpacks. The upload's request cap, the zone's limit and its help text come from it and from the largest file of each type.
HUBFILE_MAX_UNZIPPED_BYTES 536870912 The most bytes the ZIPs of one upload may unpack to (512 MiB), counted as they unpack, whatever their headers claim.
HUBFILE_MAX_FILES 20000 The most files the ZIPs of one upload may hold.
HUBFILE_MAX_ZIP_RATIO 200 The most a file of more than 1 MiB may grow from its compressed size; a ZIP bomb grows by a thousand times or more.
HUBFILE_GITHUB_IMPORT true Whether the upload zone imports files from a GitHub repository. False leaves out its part of the zone and its routes.

These are the defaults. A product changes them in its own pyproject.toml, under [tool.splent.config], or in .env.deploy in production.

Dependencies

Requires
Used by
Listens to
dataset-deleteddataset-hidden-changeddataset-publisheddataset-updateddataset-withdrawnuser-deleting

In product lines

Extension points

Hook slots
hubfile.workbench.actionshubfile.workbench.contenthubfile.workbench.dialogshubfile.workbench.export_itemshubfile.workbench.file_urlshubfile.workbench.file_viewhubfile.workbench.panelhubfile.workbench.scriptshubfile.workbench.side_panelhubfile.workbench.uvl_actionshubfile.workbench.views
Templates
hooks/upload_zone.htmlhooks/workbench.htmlhooks/workbench_scripts.html
Services
HubfileDownloadRecordServiceHubfileServiceHubfileViewRecordServiceStorageServiceUploadIngestServiceUploadSessionService
Models
HubfileHubfileAliasHubfileDownloadRecordHubfileViewRecordStagedFileUploadSession
Routes
A refinement can add routes to its blueprint
Refined by
uvl

Other features fill its hook slots. A refinement can override its templates and services or extend its models without forking it.

Compatibility

Python
>=3.13
SPLENT framework
>=1.30.0
Python packages
requests