Rick Lamers
12/02/2020, 5:20 PMRick Lamers
12/02/2020, 5:20 PMhttps://www.youtube.com/watch?v=j0nySMu1-DQ▾
Rick Lamers
12/02/2020, 5:26 PMgit
.
• Environments (simple container environments to make dependencies actually easy!)
• Directly import git
projects (from GitHub/GitLab)
• New pipeline editor renderer (try zooming in and out!)Foren Power
12/02/2020, 11:42 PMOleg L
12/03/2020, 9:43 AMRick Lamers
12/03/2020, 5:14 PMRick Lamers
12/04/2020, 12:38 PMSaabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:25 AMSaabu
12/05/2020, 12:26 AMSaabu
12/05/2020, 12:26 AMSaabu
12/05/2020, 12:26 AMYannick
12/05/2020, 7:33 AMorchest.get_inputs()
will store the data of the previous step in RAM (more specifically the RAM of the pipeline step in which the function is invoked). When making the data available from the previous step you have the ability to use orchest.transfer.output_to_disk()
instead of orchest.output()
.
If you are working with large datasets and still want to pass them through memory, then it is recommended to enable auto_eviction
(https://orchest.readthedocs.io/en/latest/user_guide/other.html#pipeline-level-configurations) otherwise it will be kept in memory until you reboot the interactive session.
Hope that answers your question :)Yannick
12/05/2020, 7:36 AMRick Lamers
12/05/2020, 8:33 AMorchest.output(some_path)
to pass the path to the next step (to not hardcode where the files can be found).Rick Lamers
12/08/2020, 12:28 PMRick Lamers
12/08/2020, 5:12 PM./orchest update
to check it out. Would love to get some feedback on our first attemptKarolína
12/10/2020, 1:15 PMRick Lamers
12/10/2020, 1:15 PMRick Lamers
12/11/2020, 5:45 PMHeads up: this requires updating your old code. In short:• File viewer now contains a refresh button (see your Notebook fill up as it runs on the scheduler) • File viewer now allows you to easily navigate to the parent and child pipeline steps • Log output / build output to Xterm cleaned up to support loading style line rollback (\r) (edited screenhots)becomesorchest.output(data)
.orchest.output(data, name)
Rick Lamers
12/11/2020, 7:17 PMVivan
12/16/2020, 7:47 AMRick Lamers
12/16/2020, 9:01 AMRick Lamers
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMVivan
12/16/2020, 9:02 AMRick Lamers
12/16/2020, 9:04 AMRick Lamers
12/16/2020, 9:04 AM