https://www.orchest.io/ logo
#announcements
Title
# announcements
f

Felix Becker

10/22/2021, 9:46 PM
Hey guys - I am having issues with transferring a dataframe from one step to another. orchest.output((predictions_df_all), name="predictions") when reading the dataframe: data = orchest.get_inputs() predictions_df_all = data['predictions'] I receive this error:
Copy code
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/bin/kernel-launchers/python/scripts/launch_ipykernel.py in <module>
      1 import orchest
----> 2 data = orchest.get_inputs()
      3 predictions_df_all = data['predictions']

/orchest/orchest-sdk/python/orchest/transfer.py in get_inputs(ignore_failure, verbose)
   1016         # continue with other steps.
   1017         try:
-> 1018             incoming_step_data = get_output_method(*args, **kwargs)
   1019         except error.OutputNotFoundError as e:
   1020             if not ignore_failure:

/orchest/orchest-sdk/python/orchest/transfer.py in _get_output_memory(step_uuid, consumer)
    727     obj_id = _convert_uuid_to_object_id(step_uuid)
    728     try:
--> 729         obj = _deserialize_output_memory(obj_id, client)
    730 
    731     except error.ObjectNotFoundError:

/orchest/orchest-sdk/python/orchest/transfer.py in _deserialize_output_memory(obj_id, client)
    696         # Can load the buffer directly because its a bytes-like-object:
    697         # <https://docs.python.org/3/library/pickle.html#pickle.loads>
--> 698         return pickle.loads(buffer)
    699     else:
    700         raise ValueError("Object was serialized with an unsupported serialization")

AttributeError: Can't get attribute 'new_block' on <module 'pandas.core.internals.blocks' from '/opt/conda/lib/python3.7/site-packages/pandas/core/internals/blocks.py'>
I already reloaded the session + sent "abcdef" from one step to another. There must be somethign in the dataframe that messes up the transfer. The problem is between my pipeline step: Inf_MakePrediction -> Inf_sendToFB Thank you guys in advance and keep up the great work!
r

Rick Lamers

10/22/2021, 10:12 PM
Do you have multiple environments in this pipeline? It looks like there might be differences between the dependencies (Python libraries) between the two steps.
Thanks for the compliment!
f

Felix Becker

10/23/2021, 8:48 AM
Amazing - this was the issue 🙂
r

Rick Lamers

10/23/2021, 2:44 PM
Glad it was resolved! I’ll make note to put this in the docs somewhere as a hint.
❤️ 1