@Felipe da Silva has joined the channel

Good morning. Could you answer the following questions for me? 1. Do you provide a Helm package or Kubernetes manifests for tool installation via GitOps (to configuration as code)? 2. About their architecture (do you have architecture diagram)? I would need to know which component of the architecture is responsible for reading the pipeline settings, packaging the source code and running the jobs in Kubernetes

@William Cowan has joined the channel

Hi! Has anyone have tried running Selenium on Orchest pipeline before? Now, I have tried it with both Firefox and Chrome drivers, but both of them doesn't work. For example, with Chrome drivers, I am getting this error:

unknown error: DevToolsActivePort file doesn't exist

Simple Google search would lead to this thread. Top answer says that Selenium web drivers should be initialized as regular user instead of root user. So, I thought maybe I could retake ownership with this command line:

sudo chown -R $(whoami):$(whoami) ~/

Unfortunately, it doesn't help. Maybe you have any other ideas how I could initialize Selenium as a regular user? In comments section I am attaching both environment and python code I am using.

@Linen has joined the channel

Guys, am I missing something? When I try to put "R" Language/Environment to a step with

ipynb

, it works. But when I try to put this on a R Script, Orchest doesn't show me the R Env.

Hey, I added an image to python notebooks using markdown and it works fine in Jupyterlab, but in Preview it does not show correctly. Does anyone have any tips on how to make it works?

Hey, good afternoon. Is there an easy way to set an environment variable that is "global" to all projects within an orchest deployment?

Guys, I increased the EBS disk to 150GB used by userdir-pvc. I also updated on EKS, but the mount still has 50GB.

Hi, my Orchest instance appears to be offline. How can I resolve this?

Guys, today, out of the blue our deployment started to get the following error:

Unable to attach or mount volumes: unmounted volumes=[userdir-pvc], unattached volumes=[container-runtime-socket kube-api-access-jh5mn userdir-pvc]: timed out waiting for the condition

We sshed into the worker machine and the userdir volume is attached. Most pods are failing with the above message. Any thoughts on what may be the cause of this issue?

Hi! We have deployed Orchest on Linux server using Minikube.

minikube ip

didn't give us access to Orchest app, so we used port-forwarding

minikube kubectl -- port-forward --address public_ip -n orchest orchest-webserver-5d8585bd64-8c6bh 8448:80

in order to have Orchest app available through our public_ip:8448. This allowed us to reach Orchest web interface, create project, but failed at openning JupyterLab notebooks - it tries to go back to project list page and then tries to enter JupyterLab, then tries to go back to project list page and then JupyterLab and gets into this loop. So, I am thinking, could it be that JupyterLab needs to be port-forwarded as well? But if that's the case, newly created JupyterLab server pod would still fail. What other options we have?

Hi, when I tried to run a pipeline I got this error (X's replace our workspace url data)

xl@https://quizzical-newton-XXXXXXX.orchestapp.io/main.js?hash=5a1c400cf83bc017afd3e242c6230259:37:XXXXXXX
Tze</fromResponse/<@https://quizzical-newton-XXXXXXX.orchestapp.io/main.js?hash=5a1c400cf83bc017afd3e242c6230259:37:XXXXXXX
a@https://quizzical-newton-XXXXXXX.orchestapp.io/main.js?hash=5a1c400cf83bc017afd3e242c6230259:1:XXXXXXX

Hi all, I have expanded the limit of file size and upload a large file using streamlit service, the progress bar shows it full but it never finish uploading then ends with network error , is it streamlit problem or hosting streamlit on orchest is related to the issue ?

Hey community! I am facing issues with the installation of Orchest on my local machine (Windows). Is it possible to install minikube and set up Orchest using the commands

curl -fsSL <https://get.orchest.io> > convenience_install.sh \
    && bash convenience_install.sh

on WSL 2 and access the service by running

minikube ip

on WSL 2 command line and accessing it on my Windows browser?

Also are there any difference in the features of the open source and cloud based version of Orchest?

I am not able to configure it on a Linux VM too. Can someone please share proper steps of installation? The official documentation is not working for me.

Not a tech related question but can anyone explain how the pricing model works? The costs are based on usage, but how frequently are you billed and what is the cancellation term?

My use case is the following: I have a Streamlit application that depends on a job that is scheduled in Orchest. It runs every 6 hours, but I would like to make it easier for the users to update the data by setting a trigger for the job inside the application. I'm planning to use the orchest routes

/jobs

and

/jobs/{job_uuid}/runs/trigger

. Is there a better way to do what I'm planning to? If this is the best solution, how can I authenticate my requests to be able to make them inside orchest?

i have been working on Orchest for 2 months now (on cloud) , i always get storage issue after working on an instance after while, and it's not related to the number of created files or pipelines and even when i deleted all the files still i can not see significant change in the free storage size, any advice about how i can use storage better? (attached image for instance has 5 files only and 4 environment , 1 pipeline)

Hi! I am trying to connect to MSSQL database using pyodbc, so I need to install pyodbc on orchest in Environment Setup. In order to have pyodbc, I also need to install unixodbc-dev and that's fine, I've installed it successfully. However, when I try to connect to database using FreeTDS, I get this error:

[unixODBC][Driver Manager]Can't open lib 'FreeTDS' : file not found (0) (SQLDriverConnect)")

How would I solve it on my local machine? I'd configure /etc/odbcinst.ini in such a way:

echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so\n\
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so" | sudo tee -a /etc/odbcinst.ini

I tried the same on Orchest, but that doesn't fix the issue, because driver is not necessarily in the same location. In order to locate the needed location, I'd like to navigate through directories and see if it even exists. One way to do that on Orchest is to do

ls

commands to find if file is existent while Setting up and environment. But as you can understand that's very inconvenient. My question: is there any other way how I could get to look into files? Or even better maybe you know a solution to FreeTDS issue?

Hi everybody! Something is wrong with the password reset. Could somebody check?

Hey Guys, can you please help me a link/procedure to install Orchest tool in an on-prime Kubernetes (not with GKE/EKS).

my on-prime Kubernetes don't have access to Internet.

Guys, we are having problems with a possible memory leak on the Orchest Cluster, the memory is rising until it bursts. We are monitoring with node_exporter (prometheus) and we can see this behavior.

If you have recently updated to

v2023.01.0

or

v2023.01.1

and you are experiencing issues related to git imports or ssh/git config setup please update to

v2023.01.2

Does anybody have a best practice/prereq install guide for Ubuntu latest LTS?

Hi guys, today our node was decommissioned by kubernetes because kubernetes was unable to connect to it. Kubernetes has created another and our sessions are unable to start up. I have already deleted the reference to the old node in the orchest database. Any idea on how to stabilize the sessions?

I am facing issues with starting the postgres service as per the configuration mentioned in this blog post https://www.orchest.io/blog/google-analytics-data-with-orchest-and-meltano#:~:text=Add%20a%20new%20Service%20w[…]ctory%3A%20/var/lib/postgresql/data This is the error I am facing. Need urgent help

Hey community! So, recently I have been trying to integrate meltano in orchest and it is working fine except for that fact that hidden .ipynb_checkpoints folder gets createed into the models folder inside the transform directory of meltano. This causes issues when I am trying to run dbt transforms. Meltano goes into the checkpoints folder and tries to read in the sql files that were created out of the blue. This happens every time I run my pipeline. I am not sure why these folders are getting created because there are no jupyter notebook files inside those folders. Any help would be appreciated. Thank you