Skip to content

[SPARK-10532][EC2]Added --profile option to specify the name of profile #8696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
teramonagi wants to merge 4 commits into from
Closed

[SPARK-10532][EC2]Added --profile option to specify the name of profile #8696

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
acea71f
Added --profile option to specify names profiles
teramonagi Sep 10, 2015
a726bc1
Update spark_ec2.py
teramonagi Sep 10, 2015
5b5c56f
Added aws/boto comment
teramonagi Oct 17, 2015
5152fa4
Updated the default value for --profile option
teramonagi Oct 18, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Added --profile option to specify names profiles
  • Loading branch information
@teramonagi
teramonagi committed Sep 10, 2015
commit acea71f7254a29ba34cdac8d1e62e6aa2e51ce57
8 changes: 7 additions & 1 deletion ec2/spark_ec2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,9 @@ def parse_args():
parser.add_option(
"-i", "--identity-file",
help="SSH private key file to use for logging into instances")
parser.add_option(
"-p", "--profile", default="default",
help=" If you have multiple profiles, you can configure additional, named profiles by using this option (default: %default)")
Copy link
Contributor

@JoshRosen JoshRosen Oct 16, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you expand this to explain what type of profile this is talking about? Like, maybe clarify whether it's an IAM or boto profile or whatever? As it stands now, I worry that this description isn't clear to people unfamiliar with this feature.

Copy link
Contributor Author

@teramonagi teramonagi Oct 17, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually speaking, we can use any types of profile(AWS/boto). The remaining problem is just the loading order of these "profiles" as Jan Vlcinsky explained. I think that it is little bit long to explain that in commandline help.

I adde little modification(aws or boto config) in that help.

parser.add_option(
"-t", "--instance-type", default="m1.large",
help="Type of instance to launch (default: %default). " +
Expand Down Expand Up @@ -1311,7 +1314,10 @@ def real_main():
sys.exit(1)

try:
conn = ec2.connect_to_region(opts.region)
if opts.profile != "default":
Copy link
Contributor

@JoshRosen JoshRosen Oct 17, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I wanted to have a profile that's named "default"? I think it would better to have None as the default and to replace this with an is not None check.

Copy link
Contributor Author

@teramonagi teramonagi Oct 18, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I will fix this quickly

conn = ec2.connect_to_region(opts.region, profile_name=opts.profile)
else:
conn = ec2.connect_to_region(opts.region)
except Exception as e:
print((e), file=stderr)
sys.exit(1)
Expand Down