Can there a be an option of keeping instance auto clean? (like keeping only last 2 jobs data) I mean we have to manually remove job files from the instance if I understand correctly? Or setup a job to clean previous jobs? Otherwise I need always keep monitoring instance memory, and if I have some cron jobs running it might be very hard

Hi @Serhii, thanks for reporting this. As you have said, currently there isn't an option to do that. I can see that having a long running cron job would eventually lead to a lot of disk space being used because of all the past runs. My suggestion would be, if possible, to open an issue on github so that the requirements of what you are looking for are clear and tracked. As you have mentioned, a possible workaround would be having a cronjob periodically clean up old runs snapshots. Note that you will be losing the ability to inspect those past runs and that this is more of an hack rather than a solution, and will lead to some broken state if you try to access the old runs. My guess is that the repository you are using has a considerable size or each job run is downloading a decent amount of data. Depending on the particular job at hand you could try any of the following: • at the end of each run have a final step that cleans up any data that doesn't actually need to be there anymore • make use of the

/data

directory to store any data that you need to download, and have jobs make use of this data and/or periodically clean it up Remember that interactive runs take place in the project directory, while each job run takes place in its own snapshot of the project directory.

Great feedback and something we can definitely work on to make job history more pragmatic. E.g. allowing you to specify a maximum count to act more as a ring buffer.

@Rick Lamers exactly. My repo is big because I failed to install Node js library. Have a big legacy script file, hard to rewrite, so I added node modules inside project. That's why this issue grows, but even without that log files are taking 100 mb per run. One of the problems I have right now is I want to give instance away to the client and say goodbye, which I cannot as I need to watch the disc size. I will try adding a job step cleaning old builds and logs as they won't be needed and hope it won't break anything

I created a feature request issue on GitHub for this 👍 https://github.com/orchest/orchest/issues/550