@@ -18,6 +18,8 @@ Here, we observed that caches are filled with same data, which leads to waste of
We suggest a coordination of those distributed caches, so that data scheduling to caches as well as workflow sheduling to resources or caches is optimized.
The aim is to reach data locality by bringing workflows to most suitable resource or cache.
For that purpose we developed NaviX, a intermediate between the workflow scheduler HTCondor and the data transfer protocol XRootD.
This protocol was developed by the High Energy Physics community for streaming of event-based data and already supports basic caching functionality.
Proxy servers cache files on-the-fly during access and redirect iterated accesses to the cached version instead of the remote one.
...
...
@@ -25,8 +27,6 @@ On top, XRootD managers coordinate data accesses and collect meta-data of the un
This hierarchical infrastructure enables utillization of meta-data for redirecting data accesses to the most suitable cache proxy.
Here, NaviX integrates all coordination decisions into the overlaying batch system HTCondor to influence the job to resource matching using cache meta-data.
## Requirements
* HTCondor batch system
...
...
@@ -80,6 +80,7 @@ Please adopt the configs listed in the configs folder to your local setup. Layou
*[XRootD project](http://xrootd.org/)
*[HTCondor - High Throughput Computing](https://research.cs.wisc.edu/htcondor/)
