Slurm

[Gautzilla]

🍹Slurp the SLURM

This PR aims at adding the osekit.job.scheduler.slurm class, which can be used to submit jobs in a SLURM scheduler.

It'll be developed alongside #465, as some of the code in #465 is still PBS-locked. I'll do some edits in #465 when I feel like some features that I thought were PBS-specific can be moved up in the Scheduler class, and I'll rebase this branch on the Scheduler one from time to time.

In the end, the user should be able to run public transforms in a slurm workload manager easily:

from osekit.job.builder import JobBuilder
from osekit.job.config import JobConfig
from osekit.job.scheduler.slurm import Slurm
from osekit.public.project import Project

project = Project(...)  # See the Project documentation

job_config = JobConfig(...)  # See the JobConfig documentation

scheduler = Slurm(partition="cpu")  # Scheduler in which the job is submitted

project.job_builder = JobBuilder(
    config=job_config,
    scheduler=scheduler,
)

# Now the dataset has a non-None job_builder attribute,
# running a transform will write a SLURM file in the logs directory
# and submit it through the selected scheduler.

project.run(...)  # See the Transform documentation

resolves #279

[coveralls]

coverage: 99.122% (+0.03%) from 99.089% — Gautzilla:slurm into Project-OSmOSE:main

[Gautzilla]

I still have an issue with Slurm.update_info() since squeue doesn't work as PBS's qstat -X (where both running and completed jobs are listed).

For Slurm, I might have to complete squeue with sacct to account for completed jobs.

While I'm at it I might as well ease the parsing of the info directly in the INFO_CMD part, e.g. for slurm:

sacct --jobs 3647208 --allocations --noheader --parsable2 --format=jobID,Partition,JobName,User,State,Elapsed,NodeList,Reason

Which will replace the full report:

JobID           JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
3647208            job1        cpu hpc.ifrem+          2  COMPLETED      0:0
3647208.bat+      batch            hpc.ifrem+          2  COMPLETED      0:0
3647208.ext+     extern            hpc.ifrem+          2  COMPLETED      0:0

With a synthetic one:

3647208|cpu|job1|gberthom|COMPLETED|00:02:02|compute-116-7|None