https://www.orchest.io/ logo
Docs
Join the conversationJoin Slack
Channels
announcements
be-shameless
hacktoberfest-22
introduce-yourself
random
tech-support
welcome
Powered by Linen
announcements
  • r

    Rick Lamers

    12/02/2020, 5:20 PM
    Hi everyone! We have some exciting news to share. The first stable release of Orchest is live 🎉 (0.3.0)! We can't wait to see what you'll build with it.
    🎉 6
  • r

    Rick Lamers

    12/02/2020, 5:20 PM
    Check out this quickstart walkthrough to see what it's all about

    https://www.youtube.com/watch?v=j0nySMu1-DQ▾

  • r

    Rick Lamers

    12/02/2020, 5:26 PM
    The most important new features are: • Projects: share code between pipelines, add pipelines to existing projects, and simply version Orchest projects using
    git
    . • Environments (simple container environments to make dependencies actually easy!) • Directly import
    git
    projects (from GitHub/GitLab) • New pipeline editor renderer (try zooming in and out!)
    😎 1
  • f

    Foren Power

    12/02/2020, 11:42 PM
    Thankful for being invited to check this out. Just what is needed because spreadsheets solvers are crashing with too many nonlinear equations and putting them into pandas with SciPy proves too difficult because I'm unable to visualize it.
    r
    • 2
    • 2
  • o

    Oleg L

    12/03/2020, 9:43 AM
    Hello everyone! I'm a software engineer and co-founder of a civic tech startup with a small DS team. I'm interested in open source and involved in some related projects in the open data community. I checked out Orchest after the HN post a couple of weeks ago and liked what I saw. Looking forward to getting more experience in the platform, and thanks for developing this inspiring product in the open!
    👍 2
    😀 1
    r
    • 2
    • 1
  • r

    Rick Lamers

    12/03/2020, 5:14 PM
    We just shared Orchest on /r/datascience. Feel free to join the discussion! https://www.reddit.com/r/datascience/comments/k61tby/orchest_an_open_source_tool_for_data_science/
    🙌 4
    a
    y
    • 3
    • 4
  • r

    Rick Lamers

    12/04/2020, 12:38 PM
    Why can't I resize you say? Now you can!
    resize-orchest.mp4
    👍 1
  • s

    Saabu

    12/05/2020, 12:25 AM
    Hi Rick
  • s

    Saabu

    12/05/2020, 12:25 AM
    Saw your post on reddit and wanted to try out your software
  • s

    Saabu

    12/05/2020, 12:25 AM
    When I do data = orchest.get_inputs()
  • s

    Saabu

    12/05/2020, 12:26 AM
    Are you storing all of the previous data in RAM
  • s

    Saabu

    12/05/2020, 12:26 AM
    If I am running it on a large dataset
  • s

    Saabu

    12/05/2020, 12:26 AM
    Is this the way to be writing the source or is there any other option?
  • y

    Yannick

    12/05/2020, 7:33 AM
    Hi there! Yes, calling
    orchest.get_inputs()
    will store the data of the previous step in RAM (more specifically the RAM of the pipeline step in which the function is invoked). When making the data available from the previous step you have the ability to use
    orchest.transfer.output_to_disk()
    instead of
    orchest.output()
    . If you are working with large datasets and still want to pass them through memory, then it is recommended to enable
    auto_eviction
    (https://orchest.readthedocs.io/en/latest/user_guide/other.html#pipeline-level-configurations) otherwise it will be kept in memory until you reboot the interactive session. Hope that answers your question :)
    ✅ 1
    💯 1
  • y

    Yannick

    12/05/2020, 7:36 AM
    On another note, we are currently improving on the serialization that is happening when passing data from one step to another. What is the size of the datasets we are talking about?
  • r

    Rick Lamers

    12/05/2020, 8:33 AM
    In addition, for data set sizes that are very large you could also manually manage read/write by just writing to /data. That mounted directory is shared between all pipeline steps. You could use the data passing of
    orchest.output(some_path)
    to pass the path to the next step (to not hardcode where the files can be found).
  • r

    Rick Lamers

    12/08/2020, 12:28 PM
    We're getting ready to welcome Julia to the list of supported languages for Orchest. Stay tuned! (https://github.com/orchest/orchest/pull/85/files)
  • r

    Rick Lamers

    12/08/2020, 5:12 PM
    You can now try Julia in the latest release (v0.3.6)! Run
    ./orchest update
    to check it out. Would love to get some feedback on our first attempt
  • k

    Karolína

    12/10/2020, 1:15 PM
    Hi all! I'm Karolina, Data Engineer and colleague of @Oleg L 🙂 Come here for more insights
    👋 3
  • r

    Rick Lamers

    12/10/2020, 1:15 PM
    Glad to have you Karolína! If you have any questions, don't hesitate to ask 🙂!
    👍 2
    👋 1
  • r

    Rick Lamers

    12/11/2020, 5:45 PM
    We just pushed an exciting new release: 0.3.9! What does it bring? • Passing data between pipeline steps now supports named data passing
    Heads up: this requires updating your old code. In short: 
    orchest.output(data)
     becomes 
    orchest.output(data, name)
    .
    • File viewer now contains a refresh button (see your Notebook fill up as it runs on the scheduler) • File viewer now allows you to easily navigate to the parent and child pipeline steps • Log output / build output to Xterm cleaned up to support loading style line rollback (\r) (edited screenhots)
  • r

    Rick Lamers

    12/11/2020, 7:17 PM
    image (3).png,image (2).png,Group 1.png
  • v

    Vivan

    12/16/2020, 7:47 AM
    hi everyone, i was searching to smoothen my workflow on Jupyter kernels and more efficient work with it an then i stumbled upon, orchest.io glad to contribute in any way possible.
  • r

    Rick Lamers

    12/16/2020, 9:01 AM
    Hi there! That’s great. Let us know if you have any questions. Make sure to check the docs https://orchest.readthedocs.io/en/stable/ and import some of the sample projects https://github.com/orchest/awesome-orchest-pipelines
  • r

    Rick Lamers

    12/16/2020, 9:02 AM
    Glad to have you!
  • v

    Vivan

    12/16/2020, 9:02 AM
    thanks rick
  • v

    Vivan

    12/16/2020, 9:02 AM
    was going through the repo
  • v

    Vivan

    12/16/2020, 9:02 AM
    and did some minor changes in documentation
  • v

    Vivan

    12/16/2020, 9:02 AM
    if you still reviewing PL's would be glad to have reviews.
  • r

    Rick Lamers

    12/16/2020, 9:04 AM
    That’s epic. Are you saying you want to create a PR for the changes?
Powered by Linen
Title
r

Rick Lamers

12/16/2020, 9:04 AM
That’s epic. Are you saying you want to create a PR for the changes?
View count: 1