Interop
PyTorch and HuggingFace plumbing: applying a PyTorch state_dict to a Lux (ps, st) pair, resolving and caching weights through the HuggingFace Hub, and loading .safetensors blobs.
State-dict application
Luximm.Interop.apply_state_dict — Function
apply_state_dict(ps, state_dict, mapping) -> psRebuild a Lux parameter NamedTuple by replacing leaves according to mapping, an iterable of (pytorch_key, lux_path, transform) triples where:
pytorch_key: dotted name as written by Python'smodel.state_dict().keys().lux_path: tuple of Symbols naming the leaf inps, e.g.(:stage1, :layer_1, :norm1, :gn, :scale).transform: functionArray{Float32} -> Array{Float32}applied to the raw HDF5-read array. Common transforms areidentity(HDF5-natural matches Lux) andaxis_reverse(full axis reversal, used when the Julia tensor layout matches PyTorch's logical axis order rather than its reversed storage order).
The original ps is not mutated; the caller must bind the return value.
HuggingFace Hub
Luximm.Interop.hf_hub_download — Function
hf_hub_download(repo_id, filename; revision="main",
cache_dir=hf_hub_cache_dir(),
repo_type="model") -> StringResolve <repo_id>/<filename> against revision (a branch, tag, or commit) and return the local snapshot path, downloading only what is not already cached. The on-disk layout matches huggingface_hub:
<cache_dir>/models--<org>--<name>/blobs/<etag>
<cache_dir>/models--<org>--<name>/snapshots/<commit>/<filename>
-> ../../blobs/<etag>
<cache_dir>/models--<org>--<name>/refs/<revision> # text: <commit>This means a timm.create_model(..., pretrained=True) call and a hf_hub_download call against the same repo see each other's cached blob.
The function always performs a no-redirect HEAD against https://huggingface.co/<repo_id>/resolve/<revision>/<filename> to look up the current commit (X-Repo-Commit) and the blob's etag (X-Linked-ETag for LFS-backed files, ETag otherwise). If the HEAD fails (e.g. offline), the function falls back to the most recently recorded commit in refs/<revision> and returns the existing snapshot path if present; otherwise the original error is rethrown.
Set repo_type="dataset" for dataset repos; default "model" matches timm's usage.
Luximm.Interop.hf_download — Function
hf_download(url, dest) -> StringDownload url to dest unless dest already exists. Returns dest.
If HUGGING_FACE_HUB_TOKEN is set, it is sent as a Bearer token in the Authorization header. The download streams into a sibling temp file and is renamed into place only on success, so an interrupted call (network drop, ^C) never leaves a partial file at dest that a later call would mistake for a cache hit.
Luximm.Interop.hf_hub_cache_dir — Function
hf_hub_cache_dir() -> StringCache root that matches huggingface_hub (Python). Honors the same env-var precedence:
HF_HUB_CACHEif set,- otherwise
$HF_HOME/hubifHF_HOMEis set, - otherwise
~/.cache/huggingface/hub.
Files downloaded into this directory by Luximm are visible to timm / huggingface_hub, and vice versa.
SafeTensors
Luximm.Interop.load_safetensors_state_dict — Function
load_safetensors_state_dict(path; reverse_axes=true) -> Dict{String, Array{Float32}}Read a .safetensors file from disk into a dict of Float32 arrays.
When reverse_axes = true (default), every tensor's axes are reversed so the resulting layout matches read_parity's HDF5-natural Julia layout (PyTorch axes reversed). Set reverse_axes = false to keep PyTorch's logical axis order if a caller wants that layout explicitly.