Job failure: Error in file util/session_dir.c
Hey, got such error in one of three similar jobs
Error in rule hoso_tomi_h1_solve:
jobid: 50
output: /home/gitlabrun/builds/55-349-3-134250/openfoam/fwdc/Cases/baseline/2013_Hosokawa_and_Tomiyama/H1/log.solver
shell:
mpirun -oversubscribe -np 8 HZDRreactingMultiphaseEulerFoam -case /home/gitlabrun/builds/55-349-3-134250/openfoam/fwdc/Cases/baseline/2013_Hosokawa_and_Tomiyama/H1 -parallel > /home/gitlabrun/builds/55-349-3-134250/openfoam/fwdc/Cases/baseline/2013_Hosokawa_and_Tomiyama/H1/log.solver
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Removing output files of failed job hoso_tomi_h1_solve since they might be corrupted:
/home/gitlabrun/builds/55-349-3-134250/openfoam/fwdc/Cases/baseline/2013_Hosokawa_and_Tomiyama/H1/log.solver
Job failed, going on with independent jobs.
[csk088.cluster:137555] [[3914,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 106
[csk088.cluster:137555] [[3914,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 382
[csk088.cluster:137558] [[3919,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 106
[csk088.cluster:137558] [[3919,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 382
[csk088.cluster:137563] [[3906,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 106
[csk088.cluster:137563] [[3906,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 382
[csk088.cluster:137571] [[3962,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 106
[csk088.cluster:137575] [[3966,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 106
[csk088.cluster:137571] [[3962,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 382
[csk088.cluster:137575] [[3966,0],0] ORTE_ERROR_LOG: Error in file util/session_dir.c at line 382
[Thu Jun 25 10:43:29 2020]
May be it signals about problems on certain node?
Full log https://gitlab.hzdr.de/openfoam/fwdc/Cases/-/jobs/134250/raw
Two others (RUNNING now) https://gitlab.hzdr.de/openfoam/fwdc/Cases/-/jobs/134229 https://gitlab.hzdr.de/openfoam/fwdc/Cases/-/jobs/134231