Alexsander Pereira
10/05/2022, 5:47 PM[W 2022-10-05 17:45:33.160 ServerApp] HTTP 500: Internal Server Error (Error attempting to connect to Gateway server url '<http://jupyter-eg-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e:8888/jupyter-server-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e>'. Ensure gateway url is valid and the Gateway instance is running.)
Rick Lamers
10/05/2022, 5:49 PMorchest/jupyter-enterprise-gateway
k9s
?Alexsander Pereira
10/05/2022, 5:52 PMRick Lamers
10/05/2022, 5:53 PMk9s
can be very useful. You could quickly check for example the logs of the jupyter-enterprise-gateway to see whether there is an issue with it (some error).Alexsander Pereira
10/05/2022, 5:54 PMRick Lamers
10/05/2022, 5:55 PMAlexsander Pereira
10/05/2022, 6:01 PMRick Lamers
10/05/2022, 6:05 PMrecreate a jupiter notebookDo you mean restore a Jupyter notebook file (e.g.
my-notebook.ipynb
)?Alexsander Pereira
10/05/2022, 6:06 PMRick Lamers
10/05/2022, 6:07 PMorchest-controller
.main.orchest
) in the pipeline editorAlexsander Pereira
10/05/2022, 6:11 PMHTTP 500: Internal Server Error (Error attempting to connect to Gateway server url '<http://jupyter-eg-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e:8888/jupyter-server-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e>' . Ensure gateway url is valid and the Gateway instance is running.)
Rick Lamers
10/05/2022, 6:14 PMAlexsander Pereira
10/05/2022, 6:18 PMRick Lamers
10/05/2022, 6:18 PMorchest-api
could not be reached by the jupyter-enterprise-gateway
. Meaning that while the gateway tried to start the kernel by calling orchest-api
(a POST request), it failed to reach orchest-api
. Can you confirm the orchest-api
pod is running without errors (as reported by the k8s dashboard)?Alexsander Pereira
10/05/2022, 6:39 PMYannick
10/06/2022, 8:06 AMRick Lamers
10/06/2022, 8:33 AMAlexsander Pereira
10/06/2022, 1:34 PMRick Lamers
10/06/2022, 1:47 PMorchest-api
. (It's getting a 404).
Running wget -O - <http://orchest-api/api/sessions>
in the gateway container would establish that the Orchest API is reachable from the gateway. Worth checking that quickly to establish a baseline.Alexsander Pereira
10/06/2022, 1:58 PM(base) root@jupyter-eg-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e-b6fd6b89-qpb4s:/usr/local/bin# wget -O - <http://orchest-api/api/sessions>
--2022-10-06 13:56:18-- <http://orchest-api/api/sessions>
Resolving orchest-api (orchest-api)... 10.100.209.91
Connecting to orchest-api (orchest-api)|10.100.209.91|:80...connected.
HTTP request sent, awaiting response... 308 PERMANENT REDIRECT
Location: <http://orchest-api/api/sessions/> [following]
--2022-10-06 13:56:18-- <http://orchest-api/api/sessions/>
Reusing existing connection to orchest-api:80.
HTTP request sent, awaiting response... 200 OK
Length: 2135 (2.1K) [application/json]
Saving to: 'STDOUT'
Rick Lamers
10/06/2022, 2:02 PMenv
or checking the k8s manifest).Alexsander Pereira
10/06/2022, 2:07 PM{
"project_uuid": "e4fd61e1-cfee-426a-83aa-1b0230681b16",
"pipeline_uuid": "0bee97bf-8b41-4d2e-a502-45fd3f139f5d",
"status": "RUNNING",
"base_url": "/jupyter-server-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e",
"user_services": {
"data-app": {
"name": "data-app",
"image": "environment@d839e5bc-ce97-479b-841f-2403e6e86a33",
"order": 1,
"ports": [
8000
],
"scope": [
"interactive",
"noninteractive"
],
"exposed": false,
"requires_authentication": true
},
"fast-api": {
"name": "fast-api",
"image": "environment@d839e5bc-ce97-479b-841f-2403e6e86a33",
"order": 2,
"ports": [
8000
],
"scope": [
"interactive",
"noninteractive"
],
"exposed": false,
"requires_authentication": true
},
"tensorboard-demo": {
"name": "tensorboard-demo",
"image": "environment@d839e5bc-ce97-479b-841f-2403e6e86a33",
"order": 3,
"ports": [
8000
],
"scope": [
"interactive",
"noninteractive"
],
"exposed": false,
"requires_authentication": true
}
}
}
Yannick
10/06/2022, 3:15 PMRick Lamers
10/07/2022, 10:26 AMAlexsander Pereira
10/07/2022, 12:51 PMYannick
10/07/2022, 2:02 PMAnd the build didn't work either.Okay that is strange... Could you share the
OrchestCluster
object so that we can see what parts of the system you've configured. ThanksAlexsander Pereira
10/07/2022, 2:15 PMapiVersion: <http://orchest.io/v1alpha1|orchest.io/v1alpha1>
kind: OrchestCluster
metadata:
name: cluster-1
namespace: orchest
annotations:
<http://controller.orchest.io/deploy-ingress|controller.orchest.io/deploy-ingress>: "false"
spec:
singleNode: true
orchest:
version: v2022.10.0
authServer:
image: public.ecr.aws/u5k1d2l0/orchest-auth-server:v2022.10.0-1.0.2
orchestWebServer:
image: public.ecr.aws/u5k1d2l0/orchest-webserver:v2022.10.0-1.0.2
Yannick
10/07/2022, 2:51 PMOrchestCluster
you can't build JupyterLab and you can't connect to the Gateway
? (Asking again to make sure I have everything correct.)Alexsander Pereira
10/07/2022, 2:52 PMYannick
10/07/2022, 2:55 PMAlexsander Pereira
10/07/2022, 2:55 PMYannick
10/07/2022, 3:11 PMNavid H
10/07/2022, 3:14 PMAlexsander Pereira
10/07/2022, 5:05 PMYannick
10/10/2022, 8:06 AMI'm creating a new cluster to see if the problem can only be in this one.Any luck? 🤞