Hello, how does Orchest compare to Dagaster/Prefect/MLFlow?

Hi Jean. Good question. I would consider Dagster and Prefect to be focused on data engineering workloads where ahead of time more information is known about the system you’re building. In Orchest it’s really easy to move quickly and try out many different ideas through the session based pipelines that connect directly to a running JupyterLab instance. In that sense, it’s more suitable for prototyping and experimentation. MLFlow as a framework I would put more closely to an experiment tracking tool, and is something that you could actually use within Orchest for measuring various parts of your data pipeline.

Oh I see. Thanks for the prompt response. I'm gonna try going for the three of them then. Still on the fence between Dagster and Prefect tho. I'm already booting up orchest

Happy to help you navigate! There are many tools, and I think your approach of trying them makes most sense. See what works for you. If you run into any issues with Orchest, know we’re here to help.