Skip to content

Update law to `v0.1.12`, reduction of code duplication, and various small fixes.

Daniel Savoiu requested to merge law_v0.1.12 into master

This MR contains implements several changes to adapt the setup to the newer law version 0.1.12. It includes the changes made in MR !5 (closed) and therefore supersedes it.

The main changes are summarized below:

  • local file targets using relative paths are no longer supported and are adapted to use absolute paths instead
  • the same is done for temporary directories created by tasks in their respective run directories
  • the behavior of the load and dump methods for certain targets and file types has changed, such that supplying an explicit path is necessary. The tar formatter also needs to be specified explicitly, since the one inferred from the filename (ending in .tar.gz) seems to be GZipFormatter, which does not support gzipped tar archives.
  • the custom SpaceSeparatedListParameter is adapted to changes in the base class CSVParameter
  • some deprecated call signatures are changed to the new recommended way

Beyond that, this MR contains a few other fixes and changes meant to reduce code duplication, as summarized below.

  • relative package imports are replaced with absolute imports
  • the util function law.util.interruptable_popen is reimplemented to ignore bytes in stdout/stderr that cannot be decoded to unicode text using the UTF-8 encoding. This is needed because the output from NNLOJET sometimes contains non-unicode characters, which triggers an error using law's original interruptable_popen and causes the task to fail even if NNLOJET finished successfully.
  • two new mixin classes from which tasks can inherit, and which implement parameters/methods used by more than one task in a single place:
    • TarballExtractionMixin: implements a method extract_tarball that can be used to load the contents of a (remote) archive into a local directory and optionally check for the presence of extracted files
    • HTCondorRemoteStorageMixin: adds the htcondor_remote_storage boolean parameter and a method remote_or_local_target; depending on the parameter value, the method calls either self.remote_target or self.local_target.

Merge request reports