https://www.orchest.io/ logo
Title
a

Altieris Peixoto

10/08/2021, 6:04 PM
Hi everyone! I've increased the
MAX_JOB_RUNS_PARALLELISM =3
but i cannot run parallel pipelines using web tool yet. There's another configuration i should do?
y

Yannick

10/08/2021, 6:15 PM
Hi there :) If with "parallel pipelines" you meaning running multiple pipelines in parallel in the pipeline editor, then this is indeed not possible right now. We are looking to enable this any time soon (applicable GitHub issue). Do note that increasing
MAX_JOB_RUNS_PARALLELISM
requires a restart of Orchest (
./orchest restart
).
🙌 1
r

Rick Lamers

10/08/2021, 7:43 PM
Just to clarify, that setting controls how many pipelines run in parallel in jobs. If you have three jobs, they can each run their pipeline simultaneously.
If a single job had multiple pipeline runs (through parameterization) they will also run in parallel.
If you’d like an example I can record a video for you 🙌🏻
a

Altieris Peixoto

10/08/2021, 8:08 PM
thanks guys, my issue was about running two separate project pipelines. Actually if i try to run from one project, the another one stays with pending status
this is from the first project
image.png
this is from second one
r

Rick Lamers

10/09/2021, 7:22 AM
If you run both pipelines as a job (e.g. scheduled to run once) and you have the configuration of MAX_JOB_RUNS_PARALLELISM on 2 or higher (under global Orchest Settings) they will execute in parallel. We’re working on MAX_INTERACTIVE_RUN_PARALLELISM which would solve the situation you’re describing.
You can track progress on the interactive run parallelism in this GH issue https://github.com/orchest/orchest/issues/501
y

Yannick

11/04/2021, 1:36 PM
Hi @Altieris Peixoto I've picked this issue up today so it will be included in the next release 🙌 (hopefully tomorrow we will release
v2021.11.0
)
🎉 1
🔥 2
a

Altieris Peixoto

11/05/2021, 1:17 PM
That's great @Yannick!!
🙌 1
y

Yannick

11/05/2021, 8:33 PM
I ended up making some larger changes around how we handle global config changes for Orchest, the PR is pretty much finished. Just requires a final review 🎺