Rafael Rodrigues Santana

10/31/2022, 5:21 PM
Hello everyone. We have a Orchest EKS deployment using version
. Sometimes, when trying to open Jupyter Lab, it get stuck trying to open it. Usually, this occurs when there is something wrong with the environment configuration. However, today we started to experience this in all projects that we have. By inspect one of the containers that was trying to launch the Jupyter Lab, it seems that the problem is occurring in the docker pull step as shown by the log below:
Error response from daemon: Get "<>": dial tcp connect: connection refused
Docker pull failed, pulling with buildah.
Trying to pull
initializing source <docker://>: pinging container registry Get "<>": dial tcp i/o timeout
Any thoughts on this issue?
šŸ‘€ 1


10/31/2022, 9:05 PM
hello @Rafael Rodrigues Santana, sorry you're having a bad experience! hopefully my colleagues @Yannick, @Navid H and @Jacopo can lend you a hand
āœ… 1


11/01/2022, 8:14 AM
@Jacopo Does this ring a bell for you? Seems like something is going wrong with the image pulling in the
( @Rafael Rodrigues Santana Did you change any configuration of the container runtime on your cluster (seems like you are using Docker)?
šŸ‘€ 1


11/01/2022, 8:39 AM
Hi @Rafael Rodrigues Santana, some questions: ā€¢ is this a single or multi node deployment? ā€¢ like @Yannick mentioned, did you perform any change to the container runtime configuration? For example, did you install Orchest on a docker based EKS cluster and then migrated the cluster to containerd based nodes? ā€¢ was any change done to the registry service that is internally deployed by Orchest? Is the registry reachable? Can you verify that the IP
is the IP in use by the registry service created by Orchest? (
kubectl get services -n orchest | grep docker-registry
) ā€¢ what AMI is being used for the EKS node? Anything custom?
šŸ‘€ 2

Navid H

11/01/2022, 9:13 AM
The registry IP seems to be created by orchest, can you also verify that the docker registry deployment is up and running.
kubectl get deployment docker-registry -n orchest

Rafael Rodrigues Santana

11/01/2022, 6:03 PM
I'll check and let you guys know! Thanks for the suggestions.
šŸ™ 1