https://www.orchest.io/ logo
Docs
Join the conversationJoin Slack
Channels
announcements
be-shameless
hacktoberfest-22
introduce-yourself
random
tech-support
welcome
Powered by Linen
announcements
  • a

    Alessio

    02/03/2021, 4:10 PM
    Any Docker gotchas I should be aware of? Trying to run
    ./orchest install
    but I’m getting stuck at pulling images. Just updated and restarted Docker too 🤔 Do I need to sign in to bypass the DockerHub rate limit?
    r
    • 2
    • 4
  • a

    Alessio

    02/03/2021, 8:39 PM
    Trying to
    apt install
    something in my environment, but it says I don’t have privileges. I even tried
    apt install vim
    like the docs suggest, but same issue. Am I missing something?
    WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
    
    E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
    E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
    r
    • 2
    • 4
  • r

    Rick Lamers

    02/08/2021, 4:32 PM
    For those of you who haven't read it yet, we wrote a bit about why we believe Notebooks are useful in the context of data science. https://hackernoon.com/why-jupyter-notebooks-are-the-future-of-data-science-e7133uv7 If you have any ideas or opinions about the use of Notebooks would love to open a bit of discussion 🙂!
    🙌 1
  • r

    Rick Lamers

    02/09/2021, 1:14 PM
    A new release! v0.8.0 We introduce the concept of Environment variables. Read more about in the release notes below:
    This release introduces the concept of Environment variables 📇 to Orchest! With its introduction we're deprecating and removing Data sources.
    We've been testing Orchest and listening to feedback 📣. As a result we discovered everybody handles data sources slightly differently in their workflow. We want Orchest to be helpful, but allow for flexibility in for example the choice of the data source connector (e.g. Python offers various Postgres database connectors).
    Environment variables allow for simple and effective secret passing (e.g. hostnames, passwords, usernames) to make it easy to connect and configure your favorite data sources.
    In addition, this release includes many UI/UX improvements. Such as a dedicated project settings pane, and a cleaned up ✨ job pane.
    https://github.com/orchest/orchest/releases/tag/v0.8.0
    👀 3
  • r

    Rick Lamers

    02/11/2021, 8:45 PM
    Hi community, our friends at Data Council have asked to share their OSS survey. Feel free to share your opinion (and potentially give Orchest some love!) https://twitter.com/DataCouncilAI/status/1359905254205247491
  • m

    Médéric Descazeaux

    02/12/2021, 11:05 AM
    Hello all, Thank you for the recent updates! After upgrading from 0.6.4 to 0.8.0 (yup, I got distracted), I am getting an error when switching to JupyterLab:
    Error attempting to connect to Gateway server url '<http://jupyter-EG-someid:8888>'.  Ensure gateway url is valid and the Gateway instance is running.
    I can run the scripts from the pipeline view, and I have successfully tried
    curl <http://jupyter-EG-someid:8888>
    from orchest-api container to check connectivity. Do you have pointers to help me out?
    j
    r
    • 3
    • 11
  • m

    Marcelo Felippe Guimarães

    02/19/2021, 2:28 AM
    Hello everyone! I'm trying to get started with this amazing project in a volunteer job, which I participate in. And I'd also like to congrats every contributor because the project is getting increadible! Well, I'm trying to install the Orchest in a Ubuntu 20.04 Amazon EC2. First, I cloned the repo and run
    ./orchest install
    ,
    ./orchest update
    and
    ./orchest start
    . But it always return this message for me:
    Before starting Orchest, make sure Orchest is installed. Run:
    	orchest install
    I repeated the cycle many times, installing it again but it always returns the same massage. However, when I run
    ./orchest version
    it returns to me the version v0.8.0. I'd appreciate a lot if someone can help me with this. Below is the execution cycle in my terminal.
    ubuntu@ip-:~/orchest$ ./orchest start
    Before starting Orchest, make sure Orchest is installed. Run:
    	orchest install
    
    ubuntu@ip-172-31-53-217:~/orchest$ ./orchest install
    Some images have been pulled before. Don't forget to run:
    	orchest update
    after the installation is finished to ensure that all images are running
    the same version of Orchest.
    Installing Orchest...
    Pulling images: 6/6|###################################################|
    
    ubuntu@ip-:~/orchest$ ./orchest update
    latest: Pulling from orchest/orchest-ctl
    Digest: sha256:9c6a3beb21f4fbb6f244881ec6643bfd07306e18c86729b9ee4fe385f5cf0af3
    Status: Image is up to date for orchest/orchest-ctl:latest
    <http://docker.io/orchest/orchest-ctl:latest|docker.io/orchest/orchest-ctl:latest>
    Updating...
    From <https://github.com/orchest/orchest>
     * branch              HEAD       -> FETCH_HEAD
    Already up to date.
    HEAD is up to date.
    From <https://github.com/orchest/orchest>
     * branch              HEAD       -> FETCH_HEAD
    Pulling images: 8/8|###################################################|
    Don't forget to restart Orchest for the changes to take effect:
    	orchest restart
    
    ubuntu@ip-:~/orchest$ ./orchest start
    Before starting Orchest, make sure Orchest is installed. Run:
    	orchest install
    
    ubuntu@ip-:~/orchest$ ./orchest version
    Orchest version: v0.8.0
    r
    • 2
    • 4
  • r

    Rick Lamers

    02/22/2021, 8:49 PM
    Hi everyone, a new release is waiting for you all! v0.9.0 Most importantly, we finally have proper support for URL based navigation in Orchest. Read all about in the release notes:
    We are very proud of this release and we call it: The Big One 🔥.
    This release introduces a massive new feature to Orchest: full URL deep linking support. Easy back/forward browser navigation to open the right views in your browser. Reloading the URL and directly opening the right view. Furthermore, environment build dependent actions (e.g. running a job, or interactive pipeline editing) are more automated: no more hard choices to make when builds complete.
    In addition, we now support running an exhaustive debug dump using the simple 
    orchest debug
     command. This will make troubleshooting much easier for both the users & developers. Many tests were added to the 
    orchest-api
     test suite for much improved reliability thanks mainly to @fruttasecca.
    Finally, many smaller bug fixes and performance improvements were added. Have fun building in Orchest!
    https://github.com/orchest/orchest/releases/tag/v0.9.0
  • r

    Rick Lamers

    03/11/2021, 10:46 PM
    Hi everyone! Another week, another release. This time it's a pretty big overhaul of the user interface of Orchest. The result is a more intuitive and consistent user experience with minimal visual distraction. More room for data, plots, and code! Check it out by running
    ./orchest update
    or update directly from the web UI settings pane. P.S. you might have noticed we switched to CalVer (calver.org). This sets better expectations about what the version means, and doesn't mean. P.P.S. you might not have seen it yet, but we now have full support for JupyterLab extensions, including server extensions. Test it out, and let us know which extensions are you favorite. We are big fans of https://pypi.org/project/jupyterlab-git/!
    💯 1
  • r

    Rick Lamers

    03/17/2021, 6:11 PM
    Hi all! We added a cool new feature for your GitHub repos that contain Orchest pipelines. ✅ You can now add a dynamically updating image of your pipeline to your markdown files! All you have to do is add this line to a README.md:
    ![My pipeline](<https://pviz.orchest.io/?pipeline=https://github.com/orchest/quickstart/blob/main/california_housing.orchest|https://pviz.orchest.io/?pipeline=https://github.com/orchest/quickstart/blob/main/california_housing.orchest>)
    Check it out in our quickstart repo https://github.com/orchest/quickstart
  • a

    Arman Khalatyan

    03/24/2021, 9:22 PM
    Hi
    🙌 3
    r
    • 2
    • 2
  • r

    Rick Lamers

    03/25/2021, 6:06 PM
    Another release! Happy data coding everybody 🥳
    This release brings a UI refactor for managing your pipeline sessions. We simplify the session management between the JupyterLab and Pipeline view by moving it to the top header bar.
    In addition, we have a number of quality of life, performance and bug fixes. Enjoy!
    https://github.com/orchest/orchest/releases/tag/v2021.03.6
    🎉 2
    • 1
    • 1
  • j

    Jean

    03/29/2021, 2:48 PM
    Hello, how does Orchest compare to Dagaster/Prefect/MLFlow?
    r
    • 2
    • 3
  • r

    Rick Lamers

    03/30/2021, 10:25 AM
    This repo contains an Orchest pipeline that features the workshop notebooks for the Ray Core workshop that will be held later today: https://github.com/ricklamers/ray_tutorial_orchest Fork of: https://github.com/DerwenAI/ray_tutorial Workshop: https://anyscale.zoom.us/j/98407189275 https://www.eventbrite.com/e/anyscale-academy-ray-core-hands-on-tutorial-tickets-146678245581
    • 1
    • 2
  • r

    Rick Lamers

    04/02/2021, 9:03 PM
    Hi everyone! We're super excited to announce that today we completed final testing of the first iteration of Orchest Cloud. A fully scalable architecture based on AWS for running independent Orchest instances at your command. We're opening up priority access to the Orchest Cloud private beta for members of this Slack community. If you're interested sign up here and shoot me a message so I can prioritize your onboarding.
    🎯 2
    🙌 1
    🚀 1
  • i

    Ian Yu

    04/18/2021, 2:45 AM
    Hi, joined as I’m interested in the development progress
    r
    • 2
    • 1
  • r

    Rick Lamers

    04/22/2021, 3:51 PM
    Hey y'all! I created a couple of videos to explain some of the core concepts in Orchest. Check them out here: https://www.orchest.io/knowledge-base
    🚀 2
  • r

    Rick Lamers

    05/13/2021, 3:57 PM
    Find out the latest about the data ecosystem & Orchest in Data Overload ⚡ Vol. 4! https://mailchi.mp/a471def8d2eb/orchest-presents-data-overload-vol-6527425
  • r

    Rick Lamers

    05/21/2021, 3:20 PM
    This new release brings an exciting new feature to Orchest: real-time collaboration 🤝 in JupyterLab. It's made possible by the awesome work from the Jupyter core team and yjs.
    In addition we have done some internal refactoring to improve the extensibility of the codebase, and as a result there are numerous improvements to stability and performance.
    We can't wait to see you build more awesome data pipelines with this release!
    https://github.com/orchest/orchest/releases/tag/v2021.05.0 Hi all! Today we have a new release. Check it out if you want to see "spooky action at a distance" (real-time colab in JupyterLab baked into Orchest that is).
    ./orchest update
    Should be all you need. Or use the web updater in Orchest itself (Global settings -> Update).
    🙌 2
  • r

    Rick Lamers

    05/25/2021, 7:53 AM
    We just crossed 1K stars on GitHub 🤩 Thanks everyone for your enthusiasm about what we’re building! Even though GitHub stars is just one metric, we’re glad the community is growing. https://github.com/orchest/orchest If you ever feel like contributing a small piece of code, just reach out to me and I’ll gladly assist you in the process 🤓
    🚀 1
    🤩 4
  • s

    Shrikant Karve

    05/28/2021, 2:25 PM
    I am seeing an issue while running "pnpm run dev".. I posted on git issue but thats not the right place. I figured the slack channel today. Your help is appreciated.
    j
    r
    y
    • 4
    • 64
  • s

    Shrikant Karve

    05/29/2021, 6:01 AM
    https://github.com/microsoft/WSL/issues/4885#
  • r

    Rick Lamers

    06/04/2021, 4:24 PM
    Quick announcement: a big new feature is coming up. It's called: Services. For the impatient: the alpha release is currently available on this development branch. More details will follow! Yes, you're seeing it right. That's VS Code, TensorBoard, and Streamlit running directly in Orchest. 🤯
    🤩 3
    🥳 1
    🙌 1
  • r

    Rick Lamers

    06/11/2021, 3:28 PM
    Super exciting news! Services are now available on the latest stable release. Read more about this new feature 👇
    📣 This release introduces a major new feature to Orchest: Services ✨
    Services allow you to easily integrate your favorite data science applications without any hassle. Simply use one of the existing service templates to add to your Orchest pipeline:
    - TensorBoard
    - Streamlit
    - PostgreSQL
    - Redis
    - and even VS Code!
    After adding a service to a pipeline it will show up in your pipeline editor (see attached image).
    If you want to start exploring services, simply import the 
    <https://github.com/ricklamers/orchest-language-classifier>
     project to Orchest. Make sure you upgrade Orchest to the latest version by running: 
    ./orchest update
    .
    We can't wait to see what ideas you come up with 🤩!
    You can find more details in the Services docs, including how to set up your own custom services.
    https://github.com/orchest/orchest/releases/tag/v2021.06.0
    🙌 4
  • m

    maotai zou

    06/23/2021, 9:17 AM
    I have upgraded to the latest version, but I still get the following error
    r
    • 2
    • 8
  • r

    Rick Lamers

    06/25/2021, 5:20 PM
    Hi everyone! Our most stable and performant release to date just went up: https://github.com/orchest/orchest/releases/tag/v2021.06.6 Update by running
    ./orchest update
    or update through the web UI. One of the core improvements is the conversion of the job scheduler to a managed daemon process to support auto restarting of the worker processes. ➡️ In short: more robust job execution 🔥
    🤩 1
    🙌 2
  • a

    Amer Haider

    07/05/2021, 6:34 PM
    Hello! I am new here. Thanks for all the work you guys are doing. Any plans to integrate this with Google Colab?
    r
    • 2
    • 3
  • c

    Charly Wargnier

    07/05/2021, 9:05 PM
    Hi! Glad to be here! 👋 I'm looking to sync Python scripts/notebooks with GCP or AWS. Can Orchest help?
    r
    • 2
    • 1
  • t

    Tarun Singh

    08/09/2021, 1:38 PM
    Hey guys, Im trying to run orchest on AWS i did docker installation and everything seems to be fine but can't start orchest for some reason
    j
    y
    • 3
    • 21
  • a

    Abid

    08/11/2021, 2:53 PM
    stuck at this during installation.
    👀 1
    y
    r
    • 3
    • 13
Powered by Linen
Title
a

Abid

08/11/2021, 2:53 PM
stuck at this during installation.
👀 1
y

Yannick

08/11/2021, 3:00 PM
Seems to be caused by the
orchest
script itself trying to create the global configuration dir. Are you sure you are working inside the Linux environment of WSL 2?
r

Rick Lamers

08/11/2021, 3:00 PM
Are you running this in Git Bash on Windows? For Windows you need to drop into a WSL2 container. https://orchest.readthedocs.io/en/latest/getting_started/installation.html#prerequisites
Hahaha you're fast @Yannick
🏍️ 2
y

Yannick

08/11/2021, 3:10 PM
Thanks for opening an issue @Abid. Might be better for us to continue the conversation there if this doesn't solve your problem (@Rick Lamers) The issue: https://github.com/orchest/orchest/issues/416
👍 1
a

Abid

08/11/2021, 3:18 PM
let me try
r

Rick Lamers

08/11/2021, 4:42 PM
Let me know if you need any help 👍
a

Abid

08/11/2021, 4:47 PM
Every day learning new things
Finally
issue closed
r

Rick Lamers

08/11/2021, 6:22 PM
Glad you made it Abid!
👍 1
a

Abid

08/12/2021, 5:44 AM
I don't think this was easy way for Data Scientist. We just want to pip install everything or run .exe files. Are you planning to make installation easier for non developer individuals?
r

Rick Lamers

08/12/2021, 8:50 AM
We're opening up https://cloud.orchest.io/ to more people soon, which will be as simple as Google Colab
a

Abid

08/12/2021, 12:41 PM
Just signed up for the cloud service.
👍 2
View count: 1