Title
#tech-support
Mayara Marinho

Mayara Marinho

11/03/2022, 5:03 PM
Hi everyone, I'm trying to install Snowpark on Orchest, but the build is failing:
ERROR: Ignored the following versions that require a different python version: 0.10.0 Requires-Python ==3.8.*; 0.11.0 Requires-Python ==3.8.*; 0.12.0 Requires-Python ==3.8.*; 0.6.0 Requires-Python ==3.8.*; 0.7.0 Requires-Python ==3.8.*; 0.8.0 Requires-Python ==3.8.*; 0.9.0 Requires-Python ==3.8.*; 1.0.0 Requires-Python ==3.8.*
ERROR: Could not find a version that satisfies the requirement snowflake-snowpark-python (from versions: none)
ERROR: No matching distribution found for snowflake-snowpark-python
Any tips?
juanlu

juanlu

11/03/2022, 9:39 PM
hi @Mayara Marinho! apparently
snowflake-snowpark-python
requires Python 3.8 https://github.com/snowflakedb/snowpark-python/issues/377 so you'd need to create a custom environment with that Python version. you can follow these instructions: https://docs.orchest.io/en/stable/fundamentals/environments.html#using-a-different-python-version (replace
python=3.10
with
3.8
and
py310
with
py38
)
Mayara Marinho

Mayara Marinho

11/04/2022, 2:55 PM
@juanlu this solution it's not working in Orchest version
v2021.12.0
, apparently because of
mamba
2:57 PM
Do you know how to make it work on this version? We couldn't migrate to the new version yet
juanlu

juanlu

11/04/2022, 4:03 PM
could you try replacing
mamba
with
conda
instead? indeed, in
v2021.12.0
mamba was not preinstalled
Mayara Marinho

Mayara Marinho

11/04/2022, 5:03 PM
@juanlu like this?
#!/bin/bash
# Install Python 3.10 and get minimum set of dependencies
conda create -y -n py38 python=3.8 future
conda install -y -n py38 ipykernel jupyter_client ipython_genutils pycryptodomex future "pyarrow<8.0.0"
conda run -n py38 pip install orchest
# Jupyter environment variable that specifies
# a path to search for kernels data files
# See <https://jupyter-core.readthedocs.io/en/latest/paths.html>
echo "export JUPYTER_PATH=/opt/conda/envs/py38/share/jupyter" >> /home/jovyan/.orchestrc
# Orchest related environment variable that can be set to specify
# the conda environment to use to start Jupyter kernels
echo "export CONDA_ENV=py38" >> /home/jovyan/.orchestrc
the error persists
juanlu

juanlu

11/04/2022, 5:26 PM
sorry, my bad - there are more changes needed indeed. could you try the instructions that we had online earlier this year? https://web.archive.org/web/20220126205503/https://docs.orchest.io/en/latest/fundamentals/environments.html
conda create -y -n py38 python=3.8 future
conda install -y -n py38 ipykernel jupyter_client ipython_genutils pycryptodomex future
conda run -n py38 pip install orchest

# Set environment variables so that the new Python version is
# used when executing the pipeline and inside kernels. The variables
# are set here so that they are isolated within the environment.
# NOTE: We are first overwriting the `.bashrc` file to make sure the
# environment variables are unaffected by existing code in the file.
echo "export JUPYTER_PATH=/opt/conda/envs/py38/share/jupyter" > /home/jovyan/.bashrc
echo "export CONDA_ENV=py38" >> /home/jovyan/.bashrc
Mayara Marinho

Mayara Marinho

11/04/2022, 6:21 PM
still not working 😕
6:23 PM
ERROR: Ignored the following versions that require a different python version: 0.10.0 Requires-Python ==3.8.*; 0.11.0 Requires-Python ==3.8.*; 0.12.0 Requires-Python ==3.8.*; 0.6.0 Requires-Python ==3.8.*; 0.7.0 Requires-Python ==3.8.*; 0.8.0 Requires-Python ==3.8.*; 0.9.0 Requires-Python ==3.8.*; 1.0.0 Requires-Python ==3.8.*
ERROR: Could not find a version that satisfies the requirement snowflake-snowpark-python (from versions: none)
ERROR: No matching distribution found for snowflake-snowpark-python
juanlu

juanlu

11/05/2022, 8:51 PM
hmmm weird - apparently the Python version still isn't 3.8? otherwise the pip error doesn't make sense. are you using exactly the contents I pasted for the environment? I don't know if @Jacopo or someone else has more ideas, Orchest 2021.12.0 is really old
Jacopo

Jacopo

11/07/2022, 8:54 AM
I'd say the first thing to do is understanding why python 3.8 is not being used, what happens if you print the python version after
conda run
? Given that this is an old version of Orchest it might be that switching between python versions like this is not fully supported, meaning that things could break when running steps or kernels using the resulting environment image; so after understanding why/if python 3.8 is not being picked up the due diligence would be understanding if
services/base-images/runnable-shared/bootscript.sh
and
services/base-images/base-kernel-py/Dockerfile
need some changes in order to support that . The versions of Orchest that you are using is based on this branch https://github.com/orchest/orchest/tree/stale/pre-k8s-orchest, so comparison should happen between that branch and master During this week I might have time to deploy this old version of Orchest and try this out, but I'm a bit time constrained, so I can't really make any promises
Mayara Marinho

Mayara Marinho

12/06/2022, 2:32 PM
We updated the version and now it works fine, thank you!