Title
e
Esraa Madi
11/11/2022, 10:17 AMHey, i tried to configure elsticsearch as a service, there are some environment variables i need to set up but the service does not accept them since has a dot in the names, btw I run the service without these variables it gives me 503 error when accessing via url, could you please help me on this >
y
Yannick
11/11/2022, 10:36 AMI quickly set up a the provided service and I noticed the following error in the logs:
Aborting auto configuration because of config dir ownership mismatch. Config dir is owned by elasticsearch but auto-configuration directory would be owned by root
{"@timestamp":"2022-11-11T10:30:30.091Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch-f8e9a7a0-daf0-4bce846a697d-11a5-414c-7666979l6kqg","elasticsearch.cluster.name":"docker-cluster","error.type":"java.lang.RuntimeException","error.message":"can not run elasticsearch as root"}
ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log
ERROR: Elasticsearch exited unexpectedlyIf you cannot do this, for example because your orchestration platform forbids periods in environment variable names, then you can use an alternative style by converting the setting name as follows.
1. Change the setting name to uppercase
2. Prefix it withES_SETTING_
3. Escape any underscores () by duplicating them_
4. Convert all periods () to underscores (.)_
For example,becomes-e bootstrap.memory_lock=true.-e ES_SETTING_BOOTSTRAP_MEMORY__LOCK=true
I am unfamiliar with Elasticsearch so I am shooting a bit in the dark here. The error messages says
auto-configuration directory would be owned by rootauto-configuration/project-dir/datar
Rick Lamers
11/11/2022, 12:29 PM@Yannick what is the regex validation we use to limit environment variable names at the moment based on? Is there a spec we are following that elastic is breaking or are we arbitrarily constraining the environment variable names too much?
y
Yannick
11/11/2022, 12:58 PMSeems like we use: https://github.com/orchest/orchest/blob/0e5893d0ed0332d28a515f93f98d9b0464b65ab9/lib/python/orchest-internals/_orchest/internals/utils.py#L180-L182
I think it can be extended to include dots
.r
Rick Lamers
11/11/2022, 1:39 PMLooks like Bash does not allow periods in variable names https://tldp.org/LDP/abs/html/gotchas.html
👍 1
y
Yannick
11/11/2022, 1:41 PMMight be good to add it as a comment in the code 💯
r
Rick Lamers
11/11/2022, 1:41 PMSounds good, go for it 👍
e
Esraa Madi
11/12/2022, 1:58 PM@Yannick I used the docs as you mentioned to setup env variables but still getting an error that i have to setup these variables
To workaround this issue, I build a custom docker image that does not need a setting for any variables with periods, and the elasticsearch service launched without any errors when checking logs, but still, I can't access it correctly using the given URL. (below screenshot for hitting the service locally and another for 502 error from orchest server (note: there are no errors in logs and print the desired output) could someone help me with this please
i changed the ports and worked
thank you
y
Yannick
11/14/2022, 9:35 AMGreat to hear that you got it working! 💯
but still getting an error that i have to setup these variablesThat sucks... sadly I can't really help with this given that this is at the side of elastic. When setting up a custom Docker image for your service, you can also use Environments in Orchest. That way it should be easier to iterate on the custom image.
r
Rick Lamers
11/14/2022, 9:41 AM@Esraa Madi services are a bit complex at the moment, great to hear you found your way to a working solution. True programmer mentality 🤓
🕺 1