https://www.orchest.io/ logo
Title
s

Serhii

11/23/2021, 7:49 AM
Morning guys! I often see jupyter hang eternally, what can I do with it? Might be related to occasionally it is not saving files. "Saving started" and kernel not initialized Additional question - how to run python file separately with environment I built? (not through step -> run) Can VS Code service reuse environment? (Just trying to figure out correct expected workflow working with orchest)
r

Rick Lamers

11/23/2021, 8:58 AM
What version of JupyterLab do you see (top right Help -> About)?
If you want to run a Python file outside of the pipeline directly the easiest way is to run
!python myscript.py
in a Notebook kernel. We currently donโ€™t provide a way to run in one of the environments from VS Code. Which is why JupyterLab is sort of considered the โ€œprimaryโ€ editor.
s

Serhii

11/23/2021, 10:00 AM
Yep so I went as far as trying to setup this tool to use by small team and It's a bit complicated to come up with some nice choice. I worked in Jupyterlab (Version 3.1.18), but it is too glitchy at the moment. VS Code service is super responsive and works fine but there is no way to actually develop a pipeline there... Really a complicated choice. I ended up moving notebooks to python files for portability and as they tend to be less glitchy.
Using jypiter only for temporary development then transfer final things -> to python files....
As far as I understand real cross-team usage would require setting up orchest locally for everyone, developing and pushing commits to the repo...
@Rick Lamers I dont know about are there per-project services but having to setup VS Code for each pipeline is unrealistic ๐Ÿ˜ž I dont think VS Code should be a service anyway
r

Rick Lamers

11/23/2021, 10:21 AM
As far as I understand real cross-team usage would require setting up orchest locally for everyone, developing and pushing commits to the repo...
We're working on making this more seamless. Right now that would be preferred in order to not have everyone hog each other's compute resources. I'll get an release of Orchest out today that upgrades JupyterLab to the latest release. They move to 3.2.x and I hope they improved the stability. Will do some internal testing.
VS Code as a service is kind of a clutch at the moment. Ideally we integrate it more deeply like we did with JupyterLab.
Right now it's not really there as the main editor
s

Serhii

11/23/2021, 10:23 AM
thanks for clarifications
r

Rick Lamers

11/23/2021, 10:25 AM
We'll be launching the k8s version in a couple of months, this will make it a lot more feasible to run Orchest on a small cluster and have multiple people use one and the same Orchest instance where their compute will be fanned out over multiple 'worker' nodes.
๐Ÿ‘ 1
s

Serhii

11/23/2021, 10:49 AM
@Rick Lamers also can we make environmental variables propagate more? I stumble third+ time upon this. Or they work only in ipynb and during pipeline run
r

Rick Lamers

11/23/2021, 10:54 AM
Found an issue in JupyterLab 3.2.4 so holding off on releasing it for now: https://github.com/jupyterlab/jupyterlab/issues/11527 Will keep you posted.
Or they work only in ipynb and during pipeline run
Where else would you like them to work? Perhaps we're overseeing an important case
s

Serhii

11/23/2021, 11:00 AM
Well, like I said before, probably I`m not a target audience. But mostly it was needed for some edge cases: 1. I tried to use them to make nodejs processing script to work in my environment 2. Now I tried to use them to setup git repository ssh rights in terminal for interactive editing (SSH_KEY env variable) I guess there are more cases where you would expect your project /pipeline level env variable be accessible
And yep i cannot make ssh git auth work across pipelines ๐Ÿ˜ž all the files get reset
have to change to https and specifying password i guess ๐Ÿ˜ž
r

Rick Lamers

11/23/2021, 11:51 AM
Are you using git from VS Code?
s

Serhii

11/23/2021, 12:06 PM
@Rick Lamers tried to setup there with env variables, then from jypiter, havent found an option to save my ssh key anywhere
This should help you out with the SSH/git config
๐Ÿ‘ 1
s

Serhii

11/23/2021, 3:33 PM
@Rick Lamers ok, but this won't work with jupyterlab buttons as I checked, right?
r

Rick Lamers

11/23/2021, 5:01 PM
It should work, the only issue is that it needs to trust the host at least once (which needs to happen through terminal, as the JupyterLab git extension doesn't support asking you)
I'll look into that for you ๐Ÿ‘
The workaround is to
git pull
once through terminal, after accepting the host (marking it secure) you can use the Git extension GUI buttons in JupyterLab
s

Serhii

11/24/2021, 6:55 AM
okay, thanks