Rick Lamers
Rick Lamers
git
.
• Environments (simple container environments to make dependencies actually easy!)
• Directly import git
projects (from GitHub/GitLab)
• New pipeline editor renderer (try zooming in and out!)Foren Power
12/02/2020, 11:42 PMOleg L
12/03/2020, 9:43 AMRick Lamers
Rick Lamers
Saabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:26 AMSaabu
12/05/2020, 12:26 AMSaabu
12/05/2020, 12:26 AMYannick
orchest.get_inputs()
will store the data of the previous step in RAM (more specifically the RAM of the pipeline step in which the function is invoked). When making the data available from the previous step you have the ability to use orchest.transfer.output_to_disk()
instead of orchest.output()
.
If you are working with large datasets and still want to pass them through memory, then it is recommended to enable auto_eviction
(https://orchest.readthedocs.io/en/latest/user_guide/other.html#pipeline-level-configurations) otherwise it will be kept in memory until you reboot the interactive session.
Hope that answers your question :)Yannick
Rick Lamers
orchest.output(some_path)
to pass the path to the next step (to not hardcode where the files can be found).Rick Lamers
Rick Lamers
./orchest update
to check it out. Would love to get some feedback on our first attemptKarolína
12/10/2020, 1:15 PMRick Lamers
Rick Lamers
Heads up: this requires updating your old code. In short:• File viewer now contains a refresh button (see your Notebook fill up as it runs on the scheduler) • File viewer now allows you to easily navigate to the parent and child pipeline steps • Log output / build output to Xterm cleaned up to support loading style line rollback (\r) (edited screenhots)becomesorchest.output(data)
.orchest.output(data, name)
Rick Lamers
Vivan
12/16/2020, 7:47 AMRick Lamers
Rick Lamers
Vivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMRick Lamers