Allan Sene
08/11/2022, 6:46 PMorchest-webserver
(or other ones) image to use a custom one?
I know that there is a property orchestWebServer
in the orchest-controller, but I'm not sure how is the best way to change this value.juanlu
Navid H
08/12/2022, 6:53 AMYannick
Registry
option in the OrchestSpec
to specify the registry where you store your custom images. For example, when not specifying it it will default to DockerHub (<http://docker.io|docker.io>
) and use our images.
This way you can host the images that you want to use in Orchest. To make sure the in-app updating still works your version tags need to follow CalVer (given that we enforce it in the orchest-cli and call it when updating through the orchest-api).
When looking to extend Orchest it is good to be aware of the fact that the Orchest product depends on some external endpoints for some functionality. For example, to incorporate examples into the product it will pull the applicable information from https://github.com/orchest/orchest-examples, and to show a update dialog (indicating that a new version is available) it will also poll from an external endpoint (so you would have to make sure this endpoint returns a version of your choosing). Of course you can change, in the code, the endpoints that are called (and how).
---
For the case of development, you just need to make sure that your images are on the nodes under the orchest/orchest-webserver
etc. names so that the imagePullPolicy
will resort to using your images instead of pulling them from DockerHub (or your custom registry). For this, we use the scripts/build_container.sh
script (as you can read in the development docs).
I hope that answers your question. If it doesn't or if it doesn't cover your use case, then please feel free to reach out again πAllan Sene
08/12/2022, 9:54 PMin-app update
feature as well, right? This way, we can keep track about the updates, making sure that our changes does not conflict with the main repoYannick
Alexsander Pereira
08/16/2022, 1:57 PMYannick
I saw that you suggested modifying the services/orchest-api/app/app/core/environment_image_builds.py file in the orchest-api to pull my custom images.Please be aware that these changes only apply to Environment base images, not to changes for e.g. the
orchest-webserver
, auth-server
or other Orchest services.
But how do I register the custom orchest-api image in EKS?This is done by pushing all the images to another registry (other than DockerHub because that is where we host our images) and specifying the
Registry
option (under OrchestSpec
) in the CR Object to point to that registry. Then the orchest-controller
should pull the images from the registry you configured, containing your custom images.
Does that answer your question?Alexsander Pereira
08/16/2022, 2:36 PMapiVersion: <http://orchest.io/v1alpha1|orchest.io/v1alpha1>
kind: OrchestCluster
metadata:
name: cluster-1
namespace: orchest
spec:
singleNode: true
Yannick
apiVersion: <http://orchest.io/v1alpha1|orchest.io/v1alpha1>
kind: OrchestCluster
metadata:
name: cluster-1
namespace: orchest
spec:
singleNode: true
orchest:
registry: ...
Isn't there a way to change just the image of the orchest-webserver?At this point in time there is not. It would require some changes in the
orchest-controller
to make this possible. For your information, we don't allow changing only the image for the orchest-webserver
(and other services) to ensure that Orchest is in a consistent state and we know we can update to a new version.
Of course we can make some changes to make it possible to specify a custom image. Could you share a little more about your use case and what you are looking to achieve? E.g. just specifying dadosfera-orcehst-webserver:v2022.06.6
as the image to be used for the orchest-webserver
?Alexsander Pereira
08/16/2022, 2:47 PMNavid H
08/16/2022, 2:49 PMAlexsander Pereira
08/16/2022, 3:12 PMapiVersion: <http://orchest.io/v1alpha1|orchest.io/v1alpha1>
kind: OrchestCluster
metadata:
name: cluster-1
namespace: orchest
spec:
singleNode: true
orchest:
orchestWebServer:
image: alexsanderdadosfera/orchest-webserver:latest
Yannick
The Spec file lets you define an image, shouldn't that work?This doesn't work currently to prevent users from "breaking things". Given that we can no longer ensure that Orchest works correctly when specifying a custom image. Nevertheless, @Navid H and I have just had a discussion and we can make it possible to specify a custom image (just like you did above) π . Be aware that this would result in updating to no longer work and you would have to manually update the CR Object defining your
OrchestCluster
in case you want to deploy another Orchest version with your custom changes. Reason being is that we can only update an OrchestCluster
that uses only "Orchest owned" images, thus if you specify a custom image then updating will no longer work.
In summary, we will do a new Orchest release (hopefully today or tomorrow) that allows you to specify a custom image in the CR Spec
at the cost of losing the ability to update from within the application (or using the orchest-cli
) and having to manually update the OrchestCluster
Spec
Does that work for you @Alexsander Pereira?Alexsander Pereira
08/16/2022, 5:47 PMYannick
Alexsander Pereira
08/18/2022, 2:13 PMNavid H
08/18/2022, 2:17 PMAlexsander Pereira
08/18/2022, 2:23 PMYannick
Allan Sene
08/18/2022, 8:20 PMAlexsander Pereira
08/19/2022, 7:03 PMYannick