Deprecate storing aws profile within the encrypted file / with the key

[ajvb]

There seems to have been a number of issues surrounding our use of AWS profiles:

• support --kms, --aws-profile options when decryption #471
• Decryption failed when different aws-profile names used #614
• AWS_PROFILE not respected with aws credentials as environment variables #619
• EC2 Instance Metadata #474
• AWS Profile flag doesn't work #439

Storing the AWS Profile in the encrypted file breaks expectations because it cannot be decrypted on systems where that profile doesn't exist. This is very different from how folks are used to interacting with AWS.

I'd propose that we deprecate storing the aws profile within the key and instead require users to pass --aws-profile every time they want to use an AWS profile.

[ajvb]

@autrilla What do you think?

[autrilla]

Given all the confusion around it I think we should probably deprecate it, and also deprecate --aws-profile. The AWS lets you use environment variables to control what profile is used, users should just use that.

[pnguyen-okta]

It's really useful to pass the profile using --aws-profile. I can understand reducing confusion but is it possible to just deprecate storing the aws profile in the encrypted file and keep --aws-profile?

thanks

[ajvb]

I really like how we implemented this in frost: https://github.com/mozilla/frost/blob/master/aws/client.py#L17-L32

You can pass in a profile using --aws-profiles (though for sops it would be --aws-profile) and it gets passed to the session creation, otherwise we just do the default AWS auth flow.

I'd also consider expanding this to removing support for setting AWS roles in the config. Most tools that interact with AWS via the command line require you to authenticate to AWS outside of the context of the tool and it makes sense to me that sops functions the same way. It seems like most of the issues are around people being confused it doesn't work that way (i.e. #555).

[autrilla]

Why do you prefer using a flag vs an environment variable? It’s kind of the same thing with different syntax, no? Using a flag would let us simplify the code a bit...

…

On Thu, 20 Feb 2020 at 23:21, Paul Nguyen ***@***.***> wrote: It's really useful to pass the profile using --aws-profile. I can understand reducing confusion but is it possible to just deprecate storing the aws profile in the encrypted file and keep --aws-profile? thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#634>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARH4V7JG5KURGVFRD6LHWDRD366JANCNFSM4KYXW3EQ> .

[jvehent]

I'd also consider expanding this to removing support for setting AWS roles in the config.

Original support for that came from a requirement to invoke KMS from various accounts transparently, by having Sops assume roles in those accounts. I believe this feature is still in use in our own teams.

[dsalaza4]

I agree with @jvehent, we use sops in the company I work for and being able to use multiple accounts transparently on the same shell is very important to us. Changing environment variables before running a sops command is pretty dirty, especially in automation scripts.

[autrilla]

I agree with @jvehent, we use sops in the company I work for and being able to use multiple accounts transparently on the same shell is very important to us. Changing environment variables before running a sops command is pretty dirty, especially in automation scripts.

I assume you mean the removal of the feature from the config file.

I personally find it very hard to care about the difference in spelling between:

AWS_PROFILE=foo sops foo.yaml

and

sops --aws-profile=foo foo.yaml

But if there's some specific reason you care about this, I'd love to hear about it.

I'd also consider expanding this to removing support for setting AWS roles in the config.

Original support for that came from a requirement to invoke KMS from various accounts transparently, by having Sops assume roles in those accounts. I believe this feature is still in use in our own teams.

Hm, I wonder if it is? I don't think the .sops.yaml config file is used at all when decrypting or editing. My understanding is that it'll only be used when creating new files. IIRC, I had to change profiles manually back then, but maybe things have changed. I think if you use MFA, SOPS's profile switching is not going to work anyway, because we don't support token input.

[nakatsuchi]

I agree with ajvb.
When I decrypt a file encrypted by other member of our team, I have to manually delete profile entry in the file every time.

[casey-robertson]

One vote to deprecate - it's standard practice in our pipelines and locally to set up the AWS env for each CI/CD step. We've used SOPS for a while now and never touched (maybe never even knew about) embedding the aws_profile in the encrypted file.

[scjudd]

I'd add that, although the AWS CLI could just pick one of AWS_PROFILE or --profile, it supports both and I've used both. It also seems like SOPS follows this pattern for most other flags/env vars, so I'm not sure why this one should be different.