[ARM] az group export: Add new parameters `--skip-resource-name-params` and `--skip-all-params` to support skip parameterization #13558

jorgecotillo · 2020-05-19T00:13:12Z

Description

Add --skip-resource-name-params argument which removes the use of parameters from name properties
Add --skip-all-params argument which removes the use of all parameters from the response
Add

--resource-ids argument to filter the results based on a string containing whitespace-separated

Testing Guide

To export all resource templates from a resource group without resource name parameterization: az group export --name contoso_rg --skip-resource-name-params --resource-ids "*"
To export all resource templates from a resource group without parameters: az group export --name contoso_rg --skip-all-params --resource-ids "*"

This checklist is used to make sure that common guidelines for a pull request are followed.

The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.

…ons (#12771)

yonzhan · 2020-05-19T01:49:22Z

add to S170

anthony-c-martin · 2020-05-20T22:13:23Z

LGTM!

zhoxing-ms · 2020-05-21T07:01:35Z

@jorgecotillo Hi, I suggest that it might be better to change the PR title, like:
[ARM] az group export: Add new parameters `--skip-resource-name-parameterization` and `--skip-all-parameterization` to support skip parameterization

Please refer to the document when writing PR title: doc link

zhoxing-ms · 2020-05-21T07:18:24Z

Testing Guide

To export all resource templates from a resource group without resource name parameterization: az resource group export --name contoso_rg --skip-resource-name-parameterization --resource-ids "*"
To export all resource templates from a resource group without parameters: az resource group export --name contoso_rg --skip-all-parameterization --resource-ids "*"

This command should actually be az group export, right?

zhoxing-ms · 2020-05-21T07:21:38Z

.vscode/launch.json

            "type": "python",
            "request": "launch",
-            "pythonPath": "${config:python.pythonPath}",
+            "pythonPath": "${config:python.interpreterPath}",


Why is it modified here?

zhoxing-ms · 2020-05-21T07:24:17Z

src/azure-cli/azure/cli/command_modules/resource/custom.py


 def export_group_as_template(
-        cmd, resource_group_name, include_comments=False, include_parameter_default_value=False):
+        cmd, resource_group_name, resource_ids=None, include_comments=False, include_parameter_default_value=False, skip_resource_name_parameterization=False, skip_all_parameterization=False):


resource_ids=None It is recommended to put the new parameters after the existing parameter of the method

--skip-all-parameterization and --skip-resource-name-parameterization are too long for the customers to input. Could you make them shorter?

How about --skip-all-params and --skip-resource-name-params? @jorgecotillo

zhoxing-ms · 2020-05-21T07:35:22Z

src/azure-cli/azure/cli/command_modules/resource/custom.py

+                raise CLIError('az resource: error: argument --resource_ids: invalid ResourceId value: \'%s\'' % i)



--resource_ids --> --resource-ids

zhoxing-ms · 2020-05-21T07:46:18Z

src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py

+
+        self.cmd('network vnet create -g {rg} -n {vnet}')
+        self.kwargs['vnet_id'] = self.cmd('network vnet show -g {rg} -n {vnet}').get_output_in_json()['id']
+        result = self.cmd('group export --name {rg} --resource-ids "{vnet_id}" --skip-all-parameterization --query "contentVersion"')


Could you add checks to this test to confirm whether the --skip-all-parameterization is effective?

zhoxing-ms · 2020-05-21T07:46:44Z

src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py

+
+        self.cmd('network vnet create -g {rg} -n {vnet}')
+        self.kwargs['vnet_id'] = self.cmd('network vnet show -g {rg} -n {vnet}').get_output_in_json()['id']
+        result = self.cmd('group export --name {rg} --resource-ids "{vnet_id}" --skip-resource-name-parameterization --query "contentVersion"')


Could you add checks to this test to confirm whether the --skip-resource-name-parameterization is effective?

jorgecotillo · 2020-05-25T20:44:13Z

Can someone give me a hint on how to solve the following issue:

AssertionError: No match for the request (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/cli_test_rg_scenario000001/providers/Microsoft.Network/virtualNetworks/vnet1?api-version=2020-04-01>) was found. Can't overwrite existing cassette ('/home/vsts/work/1/s/src/azure-cli/azure/cli/command_modules/resource/tests/latest/recordings/test_resource_group_export_skip_all_params.yaml') in your current record mode ('once').

When I run the test locally, I get vnet api-version 2020-03-01, I just merged the latest from master and still seeing the same api-version and an additional error: test_resource_scenario is also complaining.

Can I get some assistance please?

zhoxing-ms · 2020-05-26T01:58:58Z

Can someone give me a hint on how to solve the following issue:
AssertionError: No match for the request (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/cli_test_rg_scenario000001/providers/Microsoft.Network/virtualNetworks/vnet1?api-version=2020-04-01>) was found. Can't overwrite existing cassette ('/home/vsts/work/1/s/src/azure-cli/azure/cli/command_modules/resource/tests/latest/recordings/test_resource_group_export_skip_all_params.yaml') in your current record mode ('once').
When I run the test locally, I get vnet api-version 2020-03-01, I just merged the latest from master and still seeing the same api-version and an additional error: test_resource_scenario is also complaining.
Can I get some assistance please?

@jorgecotillo Hi, this problem is because the version of ResourceType.MGMT_NETWORK was upgraded in the dev branch at 05-21. You need to merge with dev branch and retest locally for record.

jorgecotillo · 2020-05-26T07:45:56Z

@zhoxing-ms merged from dev and now seeing different errors ...

env/lib/python3.8/site-packages/_pytest/junitxml.py:417
/home/vsts/work/1/s/env/lib/python3.8/site-packages/_pytest/junitxml.py:417: PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0.
Add 'junit_family=xunit1' to your pytest.ini file to keep the current format in future versions of pytest and silence this warning.
_issue_warning_captured(deprecated.JUNIT_XML_DEFAULT_FAMILY, config.hook, 2)

-- Docs: https://docs.pytest.org/en/latest/warnings.html

generated xml file: /home/vsts/.azdev/env_config/home/vsts/work/1/s/env/test_results.xml -
=========================== short test summary info ============================
FAILED src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_cli_version.py::VersionTest::test_version
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
==================== 1 failed, 6 passed, 1 warning in 1.47s ====================

zhoxing-ms · 2020-05-26T14:20:43Z

@jorgecotillo This is a known issue: the problem is caused by an incompatibility with Python 3.8 in the azure-python-devtools project on which azure-cli-testsdk is dependent

self = <contextlib.ExitStack object at 0x7f4b609ab070>, exc_details = ()
    def __exit__(self, *exc_details):
>       received_exc = exc_details[0] is not None
E       IndexError: tuple index out of range

We already have a PR follow-up to solve this problem: #13661

Juliehzl · 2020-05-27T08:40:54Z

src/azure-cli/azure/cli/command_modules/resource/custom.py


 def export_group_as_template(
-        cmd, resource_group_name, include_comments=False, include_parameter_default_value=False):
+        cmd, resource_group_name, include_comments=False, include_parameter_default_value=False, resource_ids=None, skip_resource_name_params=False, skip_all_params=False):


Can we register all new arguments in _params.py? Especially for --resource-ids, which should be an array, right?

Done, had to add nargs to the param.

Juliehzl · 2020-05-27T08:54:44Z

src/azure-cli/azure/cli/command_modules/resource/custom.py

+    :param bool skip_resource_name_params: export template and skip resource name parameterization.
+    :param bool skip_all_params: export template parameter and skip all parameterization.


For these parameters, I will recommend you to add them as storage_true and give short name for them.

yonzhan · 2020-05-31T12:54:52Z

add to S171

jorgecotillo · 2020-06-01T17:06:39Z

@Juliehzl can you please confirm the recent updates? Let me know if they look ok to you or if additional updates are needed.

zhoxing-ms

LGTM

jorgecotillo · 2020-06-03T03:35:03Z

@qianwens please let me know if the PR looks good or if updates are required.

azcloudfarmer · 2020-06-05T21:19:52Z

@qianwens please let us know if the PR is good to go or if there are any other updates we need to make. Thanks!

qianwens · 2020-06-08T02:35:20Z

src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py

        self.assertEqual('"1.0.0.0"\n', result.output)

+    @ResourceGroupPreparer(name_prefix='cli_test_rg_scenario')
+    def test_resource_group_export_skip_all_params(self, resource_group):


test_resource_group_export_skip_all_params [](start = 8, length = 42)

please also add test case for parameter --resource-ids

qianwens · 2020-06-08T02:41:21Z

src/azure-cli/azure/cli/command_modules/resource/custom.py

+        export_options.append('SkipAllParameterization')
+
+    resources = []
+    if resource_ids is None or resource_ids == "*":


resource_ids [](start = 31, length = 12)

just to confirm here if it should be resource_ids[0] == "*"?

updated the code.

jorgecotillo · 2020-06-11T17:29:32Z

@Juliehzl @qianwens @zhoxing-ms if the PR looks good, is it possible to merge it please? Otherwise, let me know if there are additional updates required.

dkmiller and others added 8 commits April 27, 2020 16:13

{Docs} Remove stale reference in README to closed issue about extensi…

00b9235

…ons (#12771)

Merge remote-tracking branch 'upstream/release'

ab16df0

Merge remote-tracking branch 'upstream/release'

03436bc

added new export options

dc510d6

fixed parameter reference

b9b8fb6

renamed argument

20b80e4

unit tests

2518451

fixed style issues

15003b5

jorgecotillo requested review from Juliehzl, qianwens and zhoxing-ms as code owners May 19, 2020 00:13

yonzhan assigned zhoxing-ms May 19, 2020

yonzhan added this to the S170 milestone May 19, 2020

Merge remote-tracking branch 'upstream/release'

7d98b1f

zhoxing-ms reviewed May 21, 2020

View reviewed changes

jorgecotillo added 2 commits May 21, 2020 23:03

reverted launch.json and updated based on feedback

422e831

added validation to skip params

c06cdb4

jorgecotillo changed the title ~~[ARM] Add new options to az resource group export~~ [ARM] az group export: Add new parameters --skip-resource-name-parameterization and --skip-all-parameterization to support skip parameterization May 22, 2020

jorgecotillo added 3 commits May 22, 2020 08:57

updated recordings

6d683f0

renamed test cases'

4061af8

Merge branch 'master' of https://github.com/Azure/azure-cli into dev

c112ce2

jorgecotillo added 2 commits May 25, 2020 23:48

Merge branch 'dev' of https://github.com/Azure/azure-cli into dev

d78eb48

fixed api version

90cc75f

Juliehzl reviewed May 27, 2020

View reviewed changes

jorgecotillo added 3 commits May 28, 2020 14:18

Merge branch 'dev' of https://github.com/Azure/azure-cli into dev

29e3df8

updated based on feedback

ad96692

removed str reference from param

497020f

yonzhan modified the milestones: S170, S171 May 31, 2020

zhoxing-ms self-requested a review June 2, 2020 02:43

zhoxing-ms approved these changes Jun 2, 2020

View reviewed changes

qianwens reviewed Jun 8, 2020

View reviewed changes

check index zero array

2f8a565

Juliehzl approved these changes Jun 9, 2020

View reviewed changes

qianwens approved these changes Jun 9, 2020

View reviewed changes

zhoxing-ms merged commit d780bb4 into Azure:dev Jun 12, 2020

		raise CLIError('az resource: error: argument --resource_ids: invalid ResourceId value: \'%s\'' % i)

		:param bool skip_resource_name_params: export template and skip resource name parameterization.
		:param bool skip_all_params: export template parameter and skip all parameterization.

[ARM] az group export: Add new parameters --skip-resource-name-params and --skip-all-params to support skip parameterization #13558

[ARM] az group export: Add new parameters --skip-resource-name-params and --skip-all-params to support skip parameterization #13558

Uh oh!

Conversation

jorgecotillo commented May 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonzhan commented May 19, 2020

Uh oh!

anthony-c-martin commented May 20, 2020

Uh oh!

zhoxing-ms commented May 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhoxing-ms commented May 21, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

azcloudfarmer May 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgecotillo commented May 25, 2020

Uh oh!

zhoxing-ms commented May 26, 2020

Uh oh!

jorgecotillo commented May 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhoxing-ms commented May 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgecotillo May 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yonzhan commented May 31, 2020

Uh oh!

jorgecotillo commented Jun 1, 2020

Uh oh!

zhoxing-ms left a comment

Choose a reason for hiding this comment

Uh oh!

jorgecotillo commented Jun 3, 2020

Uh oh!

azcloudfarmer commented Jun 5, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorgecotillo commented Jun 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

[ARM] az group export: Add new parameters `--skip-resource-name-params` and `--skip-all-params` to support skip parameterization #13558

[ARM] az group export: Add new parameters `--skip-resource-name-params` and `--skip-all-params` to support skip parameterization #13558

jorgecotillo commented May 19, 2020 •

edited

Loading

zhoxing-ms commented May 21, 2020 •

edited

Loading

azcloudfarmer May 22, 2020 •

edited

Loading

jorgecotillo commented May 26, 2020 •

edited

Loading

zhoxing-ms commented May 26, 2020 •

edited

Loading

jorgecotillo May 27, 2020 •

edited

Loading