Fran
08/24/2021, 2:30 PMRick Lamers
Fran
08/24/2021, 2:38 PMRick Lamers
Fran
08/24/2021, 2:40 PMRick Lamers
can make it work by running a process inside the step (remember steps run in their own container in Orchest) that needs to connect to the production database.For example in a pipeline step with a Notebook you can have a first cell that starts a process that runs AutoSSH:
!autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -L 5000:localhost:3306 user@bastion.tld &
Fran
08/24/2021, 3:17 PMautossh -M 0 \
-o ServerAliveInterval=30 \
-o ServerAliveCountMax=3 \
-v -N -L 3306:$PROD_DATABASE:3306 \
$SSH_USER@$SSH_BASTION_HOST \
-i keyfile.pem \
-o StrictHostKeyChecking=no
Then I connect to localhost on port 3306 and I'm connected. Maybe I'm wrong by saying the prod database is in the network the bastion is on.Rick Lamers
But if AutoSSH is not the best way maybe you can point me in the right direction? I'm open to suggestions. Also, what would be the best way to deploy Orchest? A VM?A VM would be easiest.
autossh -M 0 \
-o ServerAliveInterval=30 \
-o ServerAliveCountMax=3 \
-v -N -L 3306:$PROD_DATABASE:3306 \
$SSH_USER@$SSH_BASTION_HOST \
-i keyfile.pem \
-o StrictHostKeyChecking=noThis could work. One common alternative would be to deploy Orchest on a VM in the network that can directly reach the database.
Fran
08/24/2021, 5:24 PMRick Lamers