https://www.orchest.io/ logo
Powered by
e

Esraa Madi

11/11/2022, 10:17 AM
Hey, i tried to configure elsticsearch as a service, there are some environment variables i need to set up but the service does not accept them since has a dot in the names, btw I run the service without these variables it gives me 503 error when accessing via url, could you please help me on this >
y

Yannick

11/11/2022, 10:36 AM
I quickly set up a the provided service and I noticed the following error in the logs:
Copy code
Aborting auto configuration because of config dir ownership mismatch. Config dir is owned by elasticsearch but auto-configuration directory would be owned by root
{"@timestamp":"2022-11-11T10:30:30.091Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch-f8e9a7a0-daf0-4bce846a697d-11a5-414c-7666979l6kqg","elasticsearch.cluster.name":"docker-cluster","error.type":"java.lang.RuntimeException","error.message":"can not run elasticsearch as root"}
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log
ERROR: Elasticsearch exited unexpectedly
In addition, when following their installation docs they talk about an authentication token that you need to provide when making requests. And thus I don't expect the request to succeed anyways, but in your case the service completely fails to start due to the above error. Regarding not being allowed to enter dots in the names, you can convert the names according to the their docs:
If you cannot do this, for example because your orchestration platform forbids periods in environment variable names, then you can use an alternative style by converting the setting name as follows.
1. Change the setting name to uppercase
2. Prefix it with 
ES_SETTING_
3. Escape any underscores (
_
) by duplicating them
4. Convert all periods (
.
) to underscores (
_
)
For example, 
-e bootstrap.memory_lock=true
becomes 
-e ES_SETTING_BOOTSTRAP_MEMORY__LOCK=true
.
I am unfamiliar with Elasticsearch so I am shooting a bit in the dark here. The error messages says 
auto-configuration directory would be owned by root
so we need to make sure this directory is not owned by root. Since you are using an image that is provided by elasticsearch it would be strange if they provide an image that doesn't start and thus I expect that we (in Orchest) automatically mount to a location in the Docker image that elasticsearch also uses for the 
auto-configuration
directory. Maybe it helps to change the directory (possibly this page in the elasticsearch docs). We mount the following paths by default: 
/project-dir
, 
/data
Hope that helps!
r

Rick Lamers

11/11/2022, 12:29 PM
@Yannick what is the regex validation we use to limit environment variable names at the moment based on? Is there a spec we are following that elastic is breaking or are we arbitrarily constraining the environment variable names too much?
y

Yannick

11/11/2022, 12:58 PM
Seems like we use: https://github.com/orchest/orchest/blob/0e5893d0ed0332d28a515f93f98d9b0464b65ab9/lib/python/orchest-internals/_orchest/internals/utils.py#L180-L182 I think it can be extended to include dots 
.
as well, but would be good to check.
r

Rick Lamers

11/11/2022, 1:39 PM
Looks like Bash does not allow periods in variable names https://tldp.org/LDP/abs/html/gotchas.html
👍 1
y

Yannick

11/11/2022, 1:41 PM
Might be good to add it as a comment in the code 💯
r

Rick Lamers

11/11/2022, 1:41 PM
Sounds good, go for it 👍
e

Esraa Madi

11/12/2022, 1:58 PM
@Yannick I used the docs as you mentioned to setup env variables but still getting an error that i have to setup these variables
To workaround this issue, I build a custom docker image that does not need a setting for any variables with periods, and the elasticsearch service launched without any errors when checking logs, but still, I can't access it correctly using the given URL. (below screenshot for hitting the service locally and another for 502 error from orchest server (note: there are no errors in logs and print the desired output) could someone help me with this please
i changed the ports and worked thank you
y

Yannick

11/14/2022, 9:35 AM
Great to hear that you got it working! 💯
but still getting an error that i have to setup these variables
That sucks... sadly I can't really help with this given that this is at the side of elastic. When setting up a custom Docker image for your service, you can also use Environments in Orchest. That way it should be easier to iterate on the custom image.
r

Rick Lamers

11/14/2022, 9:41 AM
@Esraa Madi services are a bit complex at the moment, great to hear you found your way to a working solution. True programmer mentality 🤓
🕺 1
122 Views
Powered by