@Alice Miller has joined the channel

@Eric Hansander has joined the channel

@Thierry has joined the channel

Release - v2022.09.9 New release published by yannickperrenet Full Changelog: v2022.09.8...v2022.09.9 orchest/orchest

@Marco Bianco has joined the channel

@Jerome Montino has joined the channel

Hi, everyone. Hope you're doing well. I just found Orchest via @Rick Lamers' post on LinkedIn. Currently working as a one-man DE team in my current company so I had some questions. Currently, our Django app uses a Postgres RDS instance hosted in AWS. Every day, I move the data to Redshift and S3 via Glue, very straightforward. I then use dbt Cloud to transform the data to fact tables and so on for the tables in Redshift, no issues here. However, the final part of my workflow is running some notebooks, the output of which I have to dump back into Redshift/S3 then run some dbt Cloud jobs again. This is where I hope Orchest can do two (2) things. 1. Integrate with dbt Cloud to remove the manual part of running these notebooks in between dbt Cloud runs. 2. Ideally be able to have Orchest speak to our Redshift instance that runs behind a VPC. Point (1), I'm exploring thanks to some docs Rick shared in LinkedIn. My problem is Point (2). I believe I need to add an inbound rule for Orchest to be allowed to communicate with Redshift. Thing is, I don't think I've seen Orchest's IP address in the docs. Anyone with any thoughts on Point (2)?

Release - v2022.09.10 New release published by yannickperrenet Full Changelog: v2022.09.9...v2022.09.10 orchest/orchest

@Nathan Sweet has joined the channel

@uki has joined the channel

@AJ Elliott has joined the channel

@zross has joined the channel

Release - v2022.09.11 New release published by yannickperrenet What's Changed Improvements 👷 • ORC-908: Background color for chips in run table by @mausworks in #1321 Full Changelog: v2022.09.10...v2022.09.11 orchest/orchest

@Sriram Sivakumar has joined the channel

@Ruben Bosch has joined the channel

Has anyone had any luck installing the package

RPostgreSQL

in their R environment? Keep getting the same error message in the screenshot, even if I install Postgres first.

sudo apt-get install libpq-dev
sudo sh -c 'echo "deb <http://apt.postgresql.org/pub/repos/apt> $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - <https://www.postgresql.org/media/keys/ACCC4CF8.asc> | sudo apt-key add -
sudo apt-get update
sudo apt-get -y install postgresql
Rscript -e 'install.packages("RPostgreSQL", dependencies=TRUE, repos="<http://cran.rstudio.com/>")'

@viresh dhawan has joined the channel

Is anyone using orchest for a realtime messaging kind of a system? How is the parallelization achieved here?

Seems auto-migration didnt work and no clue what to do next in orchest cloud

@Luca Scimone has joined the channel

@Linh-quan HA has joined the channel

Release - v2022.10.0 New release published by yannickperrenet What's Changed Improvements 👷 • Sticky header shadows and unify layout components by @mausworks in #1320 Full Changelog: v2022.09.11...v2022.10.0 orchest/orchest

@Lex Avstreikh has joined the channel

hi everyone! it is 🎃 Hacktoberfest ✨ season, which means that if you contribute 4 valid pull requests before the end of the month you can have a tree planted in your name 🌳 or the Hacktoberfest 2022 t-shirt! 👕 Orchest is participating, so this is how you can contribute to the project: 1️⃣ Send a bug fix or new code feature. We have labeled the easier issues with "hacktoberfest" so you know where to start https://github.com/orchest/orchest/issues?q=is%3Aopen+is%3Aissue+label%3Ahacktoberfest but all meaningful contributions are welcome! Just drop as a message before starting the work to make sure we're aligned. In particular, why not adding a dark mode to Orchest? :ninja:🏼 https://github.com/orchest/orchest/issues/1331 2️⃣ Improve our documentation. Yes! this year Hacktoberfest allows low-code and no-code contributions. Just make sure that they're in the form of a pull request (and that they're not just minor typo edits!) 3️⃣ Give a talk or write a blog post about Orchest. Again, to make sure you meet the eligibility requirements, ping us in advance and we'll be happy to have a chat over it. Happy hacking! https://github.com/orchest/orchest/

@Marc H. Guirand has joined the channel

I updated to the latest version using EKS and orchest is showing this error in the controller:

Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "dbc00419165f73b0ba32733416e822c3c5c42a30787782f78475670ac50171fe" network for pod "orchest-controller-85df7b958b-9kn4c": networkPlugin cni failed to set up pod "orchest-controller-85df7b958b-9kn4c_orchest" network: add cmd: Error received from AddNetwork gRPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:50051: connect: connection refused", failed to clean up sandbox container "dbc00419165f73b0ba32733416e822c3c5c42a30787782f78475670ac50171fe" network for pod "orchest-controller-85df7b958b-9kn4c": networkPlugin cni failed to teardown pod "orchest-controller-85df7b958b-9kn4c_orchest" network: del cmd: error received from DelNetwork gRPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:50051: connect: connection refused"]

Can someone help me? It's a production cluster.

What would this error be?

[W 2022-10-05 17:45:33.160 ServerApp] HTTP 500: Internal Server Error (Error attempting to connect to Gateway server url '<http://jupyter-eg-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e:8888/jupyter-server-e4fd61e1-cfee-426a0bee97bf-8b41-4d2e>'.  Ensure gateway url is valid and the Gateway instance is running.)

Release - v2022.10.1 New release published by yannickperrenet What's Changed Improvements 👷 • Fix: add missing "back to settings" buttons by @iannbing in #1332 • Fix: incorrect warning after saving a recurring Job. by @iannbing in #1327 • Improve step selector performance by @mausworks in #1322 • Fix race condition between job scheduler and job cancellation by @fruttasecca in #1333 Full Changelog: v2022.10.0...v2022.10.1 orchest/orchest

Hi, Guys! I was wandering: does make sense to use Spark inside Orchest? I mean, not just connect a Spark Session and submit jobs to other cluster, but to have an infrastructure of containers inside the same k8s and highly coupled with Orchest administration