Optimizing Resource Use with a Shared Peer-to-Peer Cache in Kubernetes
A practical lab setup leveraging Dragonfly and vCluster demonstrates how a platform-managed model distribution layer can effectively share AI models across multiple tenants without granting node-level access to tenant workloads.
In a multi-tenant Kubernetes architecture, each internal team typically operates within its own API boundary. While this separation fosters team autonomy, it inadvertently leads to redundant downloads of the same sizable AI model files. When tenants directly retrieve identical model weights, it results in excessive network traffic and redundant storage of data across nodes.
A cache implementation within each tenant preserves data ownership but also restricts content reuse. On the other hand, a node-level cache can facilitate broader sharing of data but often requires significant privileges like hostPath mounts or privileged containers. Therefore, I constructed a test environment to determine if a platform-owned peer-to-peer cache could efficiently distribute model payloads across virtual clusters without compromising tenant isolation.
The Architecture of the Shared Data Path
In this setup, Dragonfly operated exclusively on the host cluster in a manager-less configuration. Schedulers and seed peers orchestrated task management, while a client DaemonSet was deployed on each node. Each client provided an HTTP proxy on port 4001 using host networking, all owned by the platform team, thereby controlling the cache.
Tenant A and Tenant B each interacted with their own open-source vCluster API server. The jobs for downloading models were configured as standard non-root Pods, devoid of hostPath, hostNetwork, hostPID, hostIPC, or any privileged settings. The job utilized the Kubernetes downward API to identify its node IP and fetched the model payload via the local Dragonfly proxy.
The job initiated a HEAD request to Hugging Face to retrieve the signed CDN URL of the model file and subsequently downloaded it through Dragonfly. The cache was configured to remove specific signed query parameters from the cache identity, ensuring both tenants aligned their content paths to the same Dragonfly task ID. This measure was crucial because modifications to signed URLs could inhibit effective reuse.
Figure 1. Host-managed Dragonfly data plane shared by two vCluster tenants.
Observations from the Test Runs
Tenant A, allocated on the host control-plane node, successfully downloaded 267,954,768 bytes, generating a SHA-256 checksum of 5e3f1108e3cb34ee048634875d8482665b65ac713291a7e32396fb18f6ff0063. The elapsed wall-clock time for this operation was approximately 11 seconds.
Subsequently, I positioned Tenant B on a separate node, dragonfly-host-worker2, where the request couldn’t be fulfilled by the same node’s cache. Dragonfly assigned both tasks the same task ID. The control-plane client reported sending its existing data pieces to Worker 2, resulting in Tenant B also receiving the 267,954,768-byte file and registering the same checksum. The execution period for Tenant B was around three seconds.
While the quicker turnaround was noteworthy, the more significant finding lay in the shared task ID, the parent node list, and the log from remote peer uploads alongside differing host nodes—all establishing that the model payload was indeed delivered remotely. It’s essential to mention that I didn’t independently track total bytes from the origin, so the inference shouldn’t exaggerate the reduction of all requests to Hugging Face.
Key Insights for Platform Engineering
This experiment showcases that the cache functions effectively as a platform service. Tenant interactions are simplified through a narrow data interface, while the platform retains authority over deployment, capacity, observability, and future upgrades. Moreover, the integrity of cache identity must be ingrained within the system architecture—proper normalization of signed model-hub URLs is critical, as the adopted rules from this lab shouldn't be directly applied to private models without comprehensive authorization checks.
Thirdly, the significance of transfer-source logs outweighs mere timing observations. While a faster subsequent download suggests cache utilization, it does not definitively clarify if the data originated from a local cache, a remote peer, or the original source. Lastly, it’s pertinent to differentiate API separation from host-tenant security, as this lab had only analyzed standard tenant workloads with restricted configurations—they were not tested under adversarial conditions.
Exploring Alternatives and Establishing Production Standards
Aside from Dragonfly, a platform team might explore alternatives, such as per-node registry mirrors, pull-through caches, or shared persistent-volume caches. The optimal selection truly hinges on factors like artifact formats, trust boundaries, and storage architecture. The goal in this investigation was to facilitate cross-tenant P2P reuse under platform governance.
The guiding principle moving forward would be to centralize the distribution of repeated artifacts as a platform feature, exposing only minimal tenant-accessible interfaces while validating the source of actual data transfers. Starting with public artifacts and specific cache keys should set the stage for effective management. Treating aspects such as private-model authorization, cache eviction, failure recovery, and capacity oversight as distinct production necessities will be essential.
Author Bio
Pavan Madduri is a Senior Cloud Platform Engineer at W.W. Grainger and the elected Tech Lead of CNCF’s TAG Workloads Foundation (2026 to 2028). He actively contributes to Dragonfly, a CNCF graduated project, and serves as a vCluster Ambassador as well as a CNCF Golden Kubestronaut.
Disclosure: The author is a member of the vCluster Ambassador Program. This article makes use of only open-source vCluster features and underwent no external review or sponsorship by vCluster Labs.
