Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporary way of passing *some* environment variables to remote nodes #1564

Merged
merged 1 commit into from
Oct 4, 2023

Conversation

romain-intel
Copy link
Contributor

This will be revisited with a more generalized solution where each individual decorator will be able to set the variables it wants to pass along

This will be revisited with a more generalized solution where each
individual decorator will be able to set the variables it wants to
pass along
Copy link
Collaborator

@saikonen saikonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a temp solution this seems great. No worry about leaking user-specific envs with the whitelisting+config_values approach.

If I understood it correctly, then one gotcha that I thought of regarding extensions was that any env var that should be passable, needs to be accessed by from_conf before it is made available through config_values

What was the specific use-case for this improvement?

metaflow/plugins/aws/batch/batch.py Show resolved Hide resolved
metaflow/plugins/kubernetes/kubernetes.py Show resolved Hide resolved
metaflow/plugins/argo/argo_workflows.py Show resolved Hide resolved
@romain-intel
Copy link
Contributor Author

@saikonen : this is to make sure the netflix conda extension works cleanly here as well. And yes, things need to go through from_conf (any conf needs to go through that). Future improvement will probably use something like @Environment to pass env vars (and some combo of the functionality from config_values). But for now this should be good.

@savingoyal savingoyal merged commit ad51394 into master Oct 4, 2023
20 checks passed
@savingoyal savingoyal deleted the fix/pass-some-envvars branch October 4, 2023 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants