Fran08/24/2021, 2:30 PM
Fran08/24/2021, 2:38 PM
Fran08/24/2021, 2:40 PM
can make it work by running a process inside the step (remember steps run in their own container in Orchest) that needs to connect to the production database.For example in a pipeline step with a Notebook you can have a first cell that starts a process that runs AutoSSH:
!autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -L 5000:localhost:3306 email@example.com &
Fran08/24/2021, 3:17 PM
Then I connect to localhost on port 3306 and I'm connected. Maybe I'm wrong by saying the prod database is in the network the bastion is on.
autossh -M 0 \ -o ServerAliveInterval=30 \ -o ServerAliveCountMax=3 \ -v -N -L 3306:$PROD_DATABASE:3306 \ $SSH_USER@$SSH_BASTION_HOST \ -i keyfile.pem \ -o StrictHostKeyChecking=no
But if AutoSSH is not the best way maybe you can point me in the right direction? I'm open to suggestions. Also, what would be the best way to deploy Orchest? A VM?A VM would be easiest.
autossh -M 0 \
-o ServerAliveInterval=30 \
-o ServerAliveCountMax=3 \
-v -N -L 3306:$PROD_DATABASE:3306 \
-i keyfile.pem \
-o StrictHostKeyChecking=noThis could work. One common alternative would be to deploy Orchest on a VM in the network that can directly reach the database.
Fran08/24/2021, 5:24 PM