Hi everyone, I'm trying to install Snowpark on Orchest, but the build is failing:

ERROR: Ignored the following versions that require a different python version: 0.10.0 Requires-Python ==3.8.*; 0.11.0 Requires-Python ==3.8.*; 0.12.0 Requires-Python ==3.8.*; 0.6.0 Requires-Python ==3.8.*; 0.7.0 Requires-Python ==3.8.*; 0.8.0 Requires-Python ==3.8.*; 0.9.0 Requires-Python ==3.8.*; 1.0.0 Requires-Python ==3.8.*

ERROR: Could not find a version that satisfies the requirement snowflake-snowpark-python (from versions: none)

ERROR: No matching distribution found for snowflake-snowpark-python

Any tips?

hi @Mayara Marinho! apparently

snowflake-snowpark-python

requires Python 3.8 https://github.com/snowflakedb/snowpark-python/issues/377 so you'd need to create a custom environment with that Python version. you can follow these instructions: https://docs.orchest.io/en/stable/fundamentals/environments.html#using-a-different-python-version (replace

python=3.10

with

3.8

and

py310

with

py38

)

@juanlu this solution it's not working in Orchest version

v2021.12.0

, apparently because of

mamba

could you try replacing

mamba

with

conda

instead? indeed, in

v2021.12.0

mamba was not preinstalled

@juanlu like this?

#!/bin/bash

# Install Python 3.10 and get minimum set of dependencies

conda create -y -n py38 python=3.8 future

conda install -y -n py38 ipykernel jupyter_client ipython_genutils pycryptodomex future "pyarrow<8.0.0"

conda run -n py38 pip install orchest

# Jupyter environment variable that specifies

# a path to search for kernels data files

# See <https://jupyter-core.readthedocs.io/en/latest/paths.html>

echo "export JUPYTER_PATH=/opt/conda/envs/py38/share/jupyter" >> /home/jovyan/.orchestrc

# Orchest related environment variable that can be set to specify

# the conda environment to use to start Jupyter kernels

echo "export CONDA_ENV=py38" >> /home/jovyan/.orchestrc

the error persists

sorry, my bad - there are more changes needed indeed. could you try the instructions that we had online earlier this year? https://web.archive.org/web/20220126205503/https://docs.orchest.io/en/latest/fundamentals/environments.html

conda create -y -n py38 python=3.8 future
conda install -y -n py38 ipykernel jupyter_client ipython_genutils pycryptodomex future
conda run -n py38 pip install orchest

# Set environment variables so that the new Python version is
# used when executing the pipeline and inside kernels. The variables
# are set here so that they are isolated within the environment.
# NOTE: We are first overwriting the `.bashrc` file to make sure the
# environment variables are unaffected by existing code in the file.
echo "export JUPYTER_PATH=/opt/conda/envs/py38/share/jupyter" > /home/jovyan/.bashrc
echo "export CONDA_ENV=py38" >> /home/jovyan/.bashrc

still not working 😕

hmmm weird - apparently the Python version still isn't 3.8? otherwise the pip error doesn't make sense. are you using exactly the contents I pasted for the environment? I don't know if @Jacopo or someone else has more ideas, Orchest 2021.12.0 is really old

I'd say the first thing to do is understanding why python 3.8 is not being used, what happens if you print the python version after

conda run

? Given that this is an old version of Orchest it might be that switching between python versions like this is not fully supported, meaning that things could break when running steps or kernels using the resulting environment image; so after understanding why/if python 3.8 is not being picked up the due diligence would be understanding if

services/base-images/runnable-shared/bootscript.sh

and

services/base-images/base-kernel-py/Dockerfile

need some changes in order to support that . The versions of Orchest that you are using is based on this branch https://github.com/orchest/orchest/tree/stale/pre-k8s-orchest, so comparison should happen between that branch and master During this week I might have time to deploy this old version of Orchest and try this out, but I'm a bit time constrained, so I can't really make any promises

We updated the version and now it works fine, thank you!