Yaml types and Nameko environment variable substitution

googlegroup
1

When we use flow
Jijnja2 -> Nameko
yaml files receive substitution values BEFORE PyYaml type resolver
#before Jinja2 processing
b: {{var_b}}

#after Jinja2 processing
b: 4242

#in nameko config
{ 'a': True, 'b': 4242}

When we start using
http://nameko.readthedocs.io/en/stable/cli.html#environment-variable-substitution
- we receive always string

#Yaml
b: ${var_b}

#in nameko config
{ 'a': 'true', 'b': '4242'}

Maybe trow this strin back to PyYaml resolver?
My fixes in main.py

def _env_var_constructor(loader, node):
    raw_value = loader.construct_scalar(node)
    value = ENV_VAR_MATCHER.sub(_replace_env_var, raw_value)
    new_tag = loader.resolve(yaml.ScalarNode, value, (True, False)) # <- re-resolve yaml tag
    return loader.yaml_constructors[new_tag](loader, yaml.ScalarNode(new_tag, value)) # <- rebuild yaml node

···

a: {{var_a}}
a: true
a: ${var_a}

2

This seems like a good idea to me. Again, you want to raise a pull request?

···

On Friday, March 17, 2017 at 4:27:12 PM UTC, trex....@gmail.com wrote:

When we use flow
Jijnja2 -> Nameko
yaml files receive substitution values BEFORE PyYaml type resolver
#before Jinja2 processing
a: {{var_a}}
b: {{var_b}}

#after Jinja2 processing
a: true
b: 4242

#in nameko config
{ 'a': True, 'b': 4242}

When we start using
Command Line Interface — nameko 2.12.0 documentation
- we receive always string

#Yaml
a: ${var_a}
b: ${var_b}

#in nameko config
{ 'a': 'true', 'b': '4242'}

Maybe trow this strin back to PyYaml resolver?
My fixes in main.py

def _env_var_constructor(loader, node):
    raw_value = loader.construct_scalar(node)
    value = ENV_VAR_MATCHER.sub(_replace_env_var, raw_value)
    new_tag = loader.resolve(yaml.ScalarNode, value, (True, False)) # <- re-resolve yaml tag
    return loader.yaml_constructors[new_tag](loader, yaml.ScalarNode(new_tag, value)) # <- rebuild yaml node

3

Yes, if this change not broke any plans and no unwanted side effect.

···

On Saturday, March 18, 2017 at 1:28:39 PM UTC+2, Matt Yule-Bennett wrote:

This seems like a good idea to me. Again, you want to raise a pull request?

4

are there any security implications to consider?

if we go ahead, we should make sure we document the behaviour

thanks for the suggestion!

best,
david

···

On Saturday, 18 March 2017 16:26:57 UTC, trex....@gmail.com wrote:

On Saturday, March 18, 2017 at 1:28:39 PM UTC+2, Matt Yule-Bennett wrote:

This seems like a good idea to me. Again, you want to raise a pull
request?

Yes, if this change not broke any plans and no unwanted side effect.

5

Thinking about the security -- we're loading the contents of an environment
variable as yaml. That doesn't seem to be any more dangerous than loading
the contents of a file.

···

On Saturday, March 18, 2017 at 5:10:54 PM UTC, David Szotten wrote:

are there any security implications to consider?

if we go ahead, we should make sure we document the behaviour

thanks for the suggestion!

best,
david

On Saturday, 18 March 2017 16:26:57 UTC, trex....@gmail.com wrote:

On Saturday, March 18, 2017 at 1:28:39 PM UTC+2, Matt Yule-Bennett wrote:

This seems like a good idea to me. Again, you want to raise a pull
request?

Yes, if this change not broke any plans and no unwanted side effect.

6
···

On Sunday, March 19, 2017 at 6:49:31 PM UTC+2, Matt Yule-Bennett wrote:

Thinking about the security -- we're loading the contents of an
environment variable as yaml. That doesn't seem to be any more dangerous
than loading the contents of a file.

On Saturday, March 18, 2017 at 5:10:54 PM UTC, David Szotten wrote:

are there any security implications to consider?

if we go ahead, we should make sure we document the behaviour

thanks for the suggestion!

best,
david

On Saturday, 18 March 2017 16:26:57 UTC, trex....@gmail.com wrote:

On Saturday, March 18, 2017 at 1:28:39 PM UTC+2, Matt Yule-Bennett wrote:

This seems like a good idea to me. Again, you want to raise a pull
request?

Yes, if this change not broke any plans and no unwanted side effect.