https://www.orchest.io/ logo
Join Slack
Channels
Powered by Linen
tech-support
  • h

    Hansal Shah

    12/14/2022, 1:55 AM
    Hey community! I am facing issues with the installation of Orchest on my local machine (Windows). Is it possible to install minikube and set up Orchest using the commands 
    curl -fsSL <https://get.orchest.io> > convenience_install.sh \
    && bash convenience_install.sh
    on WSL 2 and access the service by running 
    minikube ip
    on WSL 2 command line and accessing it on my Windows browser?
    r
    • 2
    • 11
  • h

    Hansal Shah

    12/14/2022, 2:06 AM
    Also are there any difference in the features of the open source and cloud based version of Orchest?
  • h

    Hansal Shah

    12/14/2022, 7:10 AM
    I am not able to configure it on a Linux VM too. Can someone please share proper steps of installation? The official documentation is not working for me.
  • r

    Ruben Bosch

    12/14/2022, 12:10 PM
    Not a tech related question but can anyone explain how the pricing model works? The costs are based on usage, but how frequently are you billed and what is the cancellation term?
    r
    • 2
    • 3
  • r

    Rafael Rodrigues Santana

    12/14/2022, 2:13 PM
    My use case is the following: I have a Streamlit application that depends on a job that is scheduled in Orchest. It runs every 6 hours, but I would like to make it easier for the users to update the data by setting a trigger for the job inside the application. I'm planning to use the orchest routes 
    /jobs
    and 
    /jobs/{job_uuid}/runs/trigger
    . Is there a better way to do what I'm planning to? If this is the best solution, how can I authenticate my requests to be able to make them inside orchest?
    r
    • 2
    • 2
  • e

    Esraa Madi

    12/17/2022, 3:05 PM
    i have been working on Orchest for 2 months now (on cloud) , i always get storage issue after working on an instance after while, and it's not related to the number of created files or pipelines and even when i deleted all the files still i can not see significant change in the free storage size, any advice about how i can use storage better? (attached image for instance has 5 files only and 4 environment , 1 pipeline)
    n
    r
    j
    • 4
    • 6
  • e

    ekeras

    12/22/2022, 12:34 PM
    Hi! I am trying to connect to MSSQL database using pyodbc, so I need to install pyodbc on orchest in Environment Setup. In order to have pyodbc, I also need to install unixodbc-dev and that's fine, I've installed it successfully. However, when I try to connect to database using FreeTDS, I get this error: 
    [unixODBC][Driver Manager]Can't open lib 'FreeTDS' : file not found (0) (SQLDriverConnect)")
    How would I solve it on my local machine? I'd configure /etc/odbcinst.ini in such a way: 
    echo "[FreeTDS]\n\
Description = FreeTDS unixODBC Driver\n\
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so\n\
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so" | sudo tee -a /etc/odbcinst.ini
    I tried the same on Orchest, but that doesn't fix the issue, because driver is not necessarily in the same location. In order to locate the needed location, I'd like to navigate through directories and see if it even exists. One way to do that on Orchest is to do 
    ls
    commands to find if file is existent while Setting up and environment. But as you can understand that's very inconvenient. My question: is there any other way how I could get to look into files? Or even better maybe you know a solution to FreeTDS issue?
    y
    • 2
    • 3
  • j

    Johan F

    12/23/2022, 8:43 PM
    Hi everybody! Something is wrong with the password reset. Could somebody check?
    r
    • 2
    • 1
  • v

    vihansh reddy

    12/30/2022, 8:39 AM
    Hey Guys, can you please help me a link/procedure to install Orchest tool in an on-prime Kubernetes (not with GKE/EKS).
    j
    y
    • 3
    • 4
  • v

    vihansh reddy

    12/30/2022, 8:40 AM
    my on-prime Kubernetes don't have access to Internet.
  • a

    Alexsander Pereira

    12/29/2022, 5:02 PM
    Guys, we are having problems with a possible memory leak on the Orchest Cluster, the memory is rising until it bursts. We are monitoring with node_exporter (prometheus) and we can see this behavior.
    j
    r
    +2
    • 5
    • 61
  • j

    Jacopo

    01/07/2023, 12:07 PM
    If you have recently updated to 
    v2023.01.0
    or 
    v2023.01.1
    and you are experiencing issues related to git imports or ssh/git config setup please update to 
    v2023.01.2
    đź‘Ť 1
  • r

    Ryan Cronin

    01/11/2023, 9:41 PM
    Does anybody have a best practice/prereq install guide for Ubuntu latest LTS?
    r
    • 2
    • 4
  • r

    Rafael Rodrigues Santana

    01/18/2023, 9:19 PM
    Hi guys, today our node was decommissioned by kubernetes because kubernetes was unable to connect to it. Kubernetes has created another and our sessions are unable to start up. I have already deleted the reference to the old node in the orchest database. Any idea on how to stabilize the sessions?
    j
    • 2
    • 46
  • h

    Hansal Shah

    01/21/2023, 7:59 AM
    I am facing issues with starting the postgres service as per the configuration mentioned in this blog post https://www.orchest.io/blog/google-analytics-data-with-orchest-and-meltano#:~:text=Add%20a%20new%20Service%20w[…]ctory%3A%20/var/lib/postgresql/data This is the error I am facing. Need urgent help
    y
    • 2
    • 2
  • h

    Hansal Shah

    01/22/2023, 12:07 PM
    Hey community! So, recently I have been trying to integrate meltano in orchest and it is working fine except for that fact that hidden .ipynb_checkpoints folder gets createed into the models folder inside the transform directory of meltano. This causes issues when I am trying to run dbt transforms. Meltano goes into the checkpoints folder and tries to read in the sql files that were created out of the blue. This happens every time I run my pipeline. I am not sure why these folders are getting created because there are no jupyter notebook files inside those folders. Any help would be appreciated. Thank you
  • h

    Hansal Shah

    01/22/2023, 12:11 PM
    It turns out that this is true for every folder that I create. By default a hidden .ipynb_checkpoints folder gets created in all of them with or without the presence of a jupyter notebook.
  • h

    Hansal Shah

    01/22/2023, 12:17 PM
    Would disabling the auto save documents option solve this issue?
    r
    • 2
    • 3
  • h

    Hansal Shah

    01/23/2023, 4:56 PM
    Is it possible to collaborate in orchest? Like can multiple users have access to the same project?
    y
    • 2
    • 3
  • h

    Hansal Shah

    01/27/2023, 8:03 AM
    My free orchest instance keeps on failing to start. Any ideas why?
    j
    • 2
    • 6
  • r

    Rafael Rodrigues Santana

    01/27/2023, 8:27 PM
    Hi guys, I've created a new orchest cluster, however, the celery worker is unable to access the celery-backend-result database while trying to build new images. Any thoughts on why this may happen?
    r
    j
    • 3
    • 7
  • k

    Kevin Schaich

    02/08/2023, 10:04 PM
    Hi all, if I'm hosting locally with 
    minikube
    , how can I get it to serve a set of files on my local Mac to be available for consumption within Orchest? I tried slightly modifying the instructions for local development: 
    minikube delete

minikube start \
  --cpus max \
  --memory max \
  --addons ingress metrics-server \
  --mount-string="/Users/kevinschaich/shared-data:/shared-data" --mount

orchest install

sudo minikube tunnel
    but there's no volume under /shared-data in the Orchest sidebar nor viewable if I do 
    ls
    from the Jupyter terminal.
    đź‘€ 1
    j
    • 2
    • 2
  • s

    Salomon DION

    02/14/2023, 7:19 AM
    Hello ! I'm trying to deploy orchest on GKE without sucess. Can anyone helps me or suggest me a better way to deploy orchest ?
    y
    • 2
    • 2
  • r

    Rafael Rodrigues Santana

    02/16/2023, 2:04 PM
    Hello guys, some of the jupyter-servers in our custom deployment are stucked in the following state: 
    Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
Awaiting boot lock...
    Because of this, we are unable to create sessions in those projects.

I have also noticed that by restarting the orchest cluster using the **restart** button, some of the projects that were not working are fixed, but other projects that were working starts to have the same issue.

Some important points:
- I have not found a lock in the $lockdir
- By running the /start.sh with the same parameters, I could reproduce the same result in jupyter-servers that are not working.
    j
    • 2
    • 19
  • n

    Nikita Chistikov

    03/06/2023, 11:56 AM
    Hi ! I have a strange problem: jupyter lab inside orchest can not connect to python3 kernel. But jupyter files in pipelines executing without problems. Any ideas how to solve it?
    y
    j
    • 3
    • 2
  • b

    Bruno Oliveira dos Santos

    03/07/2023, 2:48 PM
    Hello guys! I'm Bruno from the Dadosfera platform team. We are carrying out some studies to change the Orchest architecture, which is currently multi-cluster, to a single-cluster one. Do you have any material, studies or suggestions that could contribute to this subject?
    j
    r
    • 3
    • 18
  • a

    Allan Sene

    03/18/2023, 2:30 PM
    Hi, Guys! Do you know why some steps we can duplicate and others we don't, even with the session stopped?
    y
    • 2
    • 1
  • j

    Jericho Cruz

    03/20/2023, 3:37 PM
    Hi everyone, I'm unable to input text in the this field. Is this a setting that needs to be enabled? On a separate note, where can these settings be edited/configured so I modify AUTH_ENABLED? 
    {
  "AUTH_ENABLED": false,
  "MAX_BUILDS_PARALLELISM": 1,
  "MAX_INTERACTIVE_RUNS_PARALLELISM": 4,
  "MAX_JOB_RUNS_PARALLELISM": 4,
  "TELEMETRY_DISABLED": false,
  "TELEMETRY_UUID": "69b40767-e315-4953-8a2b-355833e344b8"
}
    j
    • 2
    • 17
  • b

    Bruno Oliveira dos Santos

    03/21/2023, 7:28 PM
    Hello guys! I need help with a problem. I will detail in the thread:
    j
    • 2
    • 6
  • r

    Reginaldo RĂ©

    03/22/2023, 11:49 AM
    Hi everyone! I need some help to configure my dev environment. I followed the development workflow steps (https://docs.orchest.io/en/stable/development/development_workflow.html). Everything installed without any problems, but after that, I didn't see orchest running on minikube when I took a look on k9s. Any tips?
    j
    y
    • 3
    • 26
Title
r

Reginaldo RĂ©

03/22/2023, 11:49 AM
Hi everyone! I need some help to configure my dev environment. I followed the development workflow steps (https://docs.orchest.io/en/stable/development/development_workflow.html). Everything installed without any problems, but after that, I didn't see orchest running on minikube when I took a look on k9s. Any tips?
j

Jacopo

03/22/2023, 11:51 AM
Hi Reginaldo, what are you getting when running 
orchest status
?
r

Reginaldo RĂ©

03/22/2023, 11:56 AM
Failed to fetch Orchest Cluster status. The Orchest Cluster named 'cluster-1' in namespace 'orchest' could not be found.
j

Jacopo

03/22/2023, 11:57 AM
Is the 
orchest
namespace there? Did you get any interesting message when running 
orchest install
?
r

Reginaldo RĂ©

03/22/2023, 11:58 AM
orchest install --dev results in:
Installing into existing namespace: orchest. Installing the Orchest Controller to manage the Orchest Cluster...
nothing else
Namespaces active are:
default Active 102m ingress-nginx Active 102m kube-node-lease Active 102m kube-public Active 102m kube-system Active 102m orchest Active 89m
trying to access orchest using chrome results in 404 from nginx
j

Jacopo

03/22/2023, 12:05 PM
What pods are running in the orchest namespace? Any chance that the minikube mount is failing? The 
--dev
flag expects the Orchest repo to be mounted to support hot reloading (see 
--mount-string="$(pwd):/orchest-dev-repo" --mount
). Maybe you weren't in the repository directory when starting minikube?
r

Reginaldo RĂ©

03/22/2023, 12:07 PM
Humm...
j

Jacopo

03/22/2023, 12:08 PM
About the mounting, see cluster-for-development in the same page
r

Reginaldo RĂ©

03/22/2023, 12:09 PM
I think it is the problem, Jacopo... I will try again.. thanks for a while...
đź‘Ť 1
Your help was great...
đź‘Ť 1
y

Yannick

03/22/2023, 12:13 PM
When installing through the dev flag you indeed need the mount and run the build containers script. The dev flag expects some files in the repo to be present.
r

Reginaldo RĂ©

03/22/2023, 12:14 PM
The repo is in my home dir, but I don't point correctly to it when run with dev flag....
But... it is possible to install a local image version without use dev flag ?
j

Jacopo

03/22/2023, 12:28 PM
Could you clarify? Do you mean applying code changes locally without using --dev?
r

Reginaldo RĂ©

03/22/2023, 1:43 PM
Sorry. Considering you need to use the build-container.sh script when changes are made, and, to avoid issues with hot reloading, couldn't I just build a set of imagens (for example, scripts/build_container.sh -i orchest-api -t $TAG -o $TAG) and run orchest install without dev flag?
I tested a new minikube mounting using local repo directory and the problem isn't on hot reloading. Also, the issues remain, there are no resources in orchest namespace and the pods running are: ingress-nginx-admission-create-t98nq, ingress-nginx-admission-patch-sb5v9, ingress-nginx-controller-77669ff58-7868j, coredns-787d4945fb-v57dx, kube-apiserver-minikube, kube-controller-manager-minikube, kube-proxy-cnqhh, kube-scheduler-minikube, storage-provisioner.
đź‘€ 1
j

Jacopo

03/22/2023, 2:33 PM
Sorry. Considering you need to use the build-container.sh script when changes are made, and, to avoid issues with hot reloading, couldn't I just build a set of imagens (for example, scripts/build_container.sh -i orchest-api -t $TAG -o $TAG) and run orchest install without dev flag?
Sorry forgot to reply here, yes, you can indeed do that, it' something we did often for development, see https://docs.orchest.io/en/stable/development/development_workflow.html#redeploying-orchest-after-code-changes
r

Reginaldo RĂ©

03/22/2023, 3:00 PM
Thanks for your attention. In that case, to make a first deploy (without any changes) the steps are: #Enter inside the root of the orchest repository cd orchest_local_repo #Run minikube without --mount-string option minikube delete minikube start \ --cpus max \ --memory max \ --addons ingress metrics-server # Use minikube's Docker daemon: eval $(minikube -p minikube docker-env) # Set the tag to the latest Orchest version available export TAG="$(orchest version --latest)" # Build the minimal set of images for Orchest to run scripts/build_container.sh -M -t $TAG -o $TAG # Install orchest orchest install So, after any change (like in orchest-api), the steps are: # Rebuild the images that need it scripts/build_container.sh -i orchest-api -t $TAG -o $TAG # Kill the pods of the orchest-api, so that the new image gets used # when new pod gets automatically deployed. kubectl delete pods -n orchest -l "app.kubernetes.io/name=orchest-api"
j

Jacopo

03/22/2023, 4:05 PM
Note, if Orchest is not installed, 
export TAG="$(orchest version --latest)"
will silently fail
aside from that, this looks correct, but you need to run install before setting the $TAG variable
r

Reginaldo RĂ©

03/22/2023, 4:13 PM
ok, thank you Jacopo.
đź‘Ť 1
Just to give a feedback. After to follow the steps discussed (without use dev flag), pods in the namespace orchestra are ok and orchestra seems to be correctly installed. I will conduct some tests. Hot reloading might have been the root of the problems I faced.Thank you, again!
View count: 2