Yannick
orchest.get_inputs()
will store the data of the previous step in RAM (more specifically the RAM of the pipeline step in which the function is invoked). When making the data available from the previous step you have the ability to use orchest.transfer.output_to_disk()
instead of orchest.output()
.
If you are working with large datasets and still want to pass them through memory, then it is recommended to enable auto_eviction
(https://orchest.readthedocs.io/en/latest/user_guide/other.html#pipeline-level-configurations) otherwise it will be kept in memory until you reboot the interactive session.
Hope that answers your question :)