register_dopar_cmq's n_jobs vs template cores

[liutiming]

Thanks for this helpful package again.

I am not very sure how will foreach jobs be submitted when both n_jobs and cores are given? Does one override the other?

[mschubert]

Jobs is the number of jobs submitted, cores is the number of cores requested per job.

So, for instance, you can use mclapply within each job that then makes use of the cores. But generally, I would recommend to parallelize in jobs with one core.

Hope that helps 😄

[liutiming]

Thanks that does help a lot!

So if I submit n_jobs = 32 and cores = 32, it will actually submit 32*32 core requests, right? So as you said, the wise thing to do is n_jobs = 32 and core = 1 since whatever is inside foreach loop may not parallelize well...

[mschubert]

Yes, exactly!

This is also true for using Q directly, not only for foreach