https://www.orchest.io/ logo
#announcements
Title
# announcements
e

Eshwaran Venkat

05/12/2022, 7:29 AM
Hey! Is there a feature for the pipelines yet where a single node can just be a sql query as opposed to a full fledged notebook?
orchest 1
j

juanlu

05/12/2022, 7:32 AM
hi @Eshwaran Venkat! at the moment the only supported technologies are Python/Julia/R/Bash scripts and Jupyter notebooks. having SQL queries would need some way to configure the connection details I assume. you could do it with Python using something like this:
Copy code
import psycopg2
con = psycopg2.connect(host="your-host", user="postgres", dbname="postgres", password="postgres")

with con.cursor() as cursor:
    cursor.execute("SELECT * FROM pg_stat_activity WHERE pid <> pg_backend_pid( ) AND datname = current_database()")
    while row := cursor.fetchone():
        print(row)
1
e

Eshwaran Venkat

05/12/2022, 7:37 AM
Understood. I am thinking of probably even using a "template" notebook (that can generically read a query from file and run it) where I can pass the SQL query filename as the environment variable. And then use the output in another notebook. It would be something along the lines of SQL runner.
j

Jacopo

05/12/2022, 7:43 AM
@Eshwaran Venkat that's a very neat idea, perhaps using step parameters might even be nicer since environment variables are shared across all steps of the pipeline, while step parameters aren't
1
j

juanlu

05/12/2022, 7:45 AM
notice that, to run a
.sql
file in Python, it's enough to read all the contents and pass the string to the
execute
method of the psycopg2 cursor, see https://stackoverflow.com/a/50006361/554319
r

Rick Lamers

05/12/2022, 11:37 AM
We've had the idea of "template" steps for a while! This is another good reason to do more with that idea. @Eshwaran Venkat thanks for nudging us again.
e

Eshwaran Venkat

05/14/2022, 2:51 PM
Thanks for the feedback guys! I'll give some of the other methods a shot.
✌️ 1
r

Rick Lamers

05/14/2022, 2:53 PM
If you have any other questions, just let us know 🙂!
e

Eshwaran Venkat

05/14/2022, 2:54 PM
Sure thing, thanks! And likewise if there's anything I can do to help with your project then by all means :)
🙏 2