Hi guys Couple questions here Is it possible to externalize

Question

Hi guys Couple questions here Is it possible to externalize the execution of the code from the instance running Orchard What if you have multiple hundreds thousands of tasks that you want to execute in parallel Can we connect that with an external kurbernetes cluster or lambda funtions for example Basically how to have unlimited resources Also what s the scheduler running under orchest Celery is it able to handle huge amount of tasks Thanks

juanlu · Answer

hello Louis it s Juan Luis here from Orchest while I gather some answers from the team I wanted to ask some follow up questions when you talk about tasks to execute in parallel are you thinking of a large number of independent batch jobs or rather complex task graphs with interdependent steps

juanlu · Answer

our scheduled tasks run on Celery and the steps are isolated so it should work just fine for a considerable amount of tasks if you find a bottleneck here we would be happy to take a closer look about the resources we are undergoing a migration to Kubernetes and when we are done hopefully end of Q1 you will have as much computational power as you want to pay for

Louis · Answer

Thanks for your answer < juanlu> This is good news I ll give it a try slightly smiling face