Skip to content

Conversation

@evelyn-ys
Copy link
Member

@evelyn-ys evelyn-ys commented Jul 8, 2025

Related command

All cli commands

Description

Previously we added dict transformation for typespec generated SDK models in #30339, we called typespec generated SDK model's as_dict func to get each attribute name and attribute value. But this still causes issues like #31529 reported when

  • the attribute is renamed during SDK generation
  • the attribute has some formatter defined in the SDK model

We can take the expired attribute from Keyvault SDK as an example:

    expires: Optional[datetime.datetime] = rest_field(
        name="exp", visibility=["read", "create", "update", "delete", "query"], format="unix-timestamp"
    )

The expires is renamed from exp in http response and has unix-timestamp time format. By calling as_dict

  • we are expecting to get {'expires': '2027-05-28T16:43:07+00:00'} (from swagger generated SDK model)
  • but we actually get {'exp': 1811522587} (from typespec generated SDK model).

The renaming and formatting was not applied to as_dict and this behavior applies for all typespec generated SDK models.

To resolve this breaking change, SDK team added related serialization support in azure-core with Azure/azure-sdk-for-python#41445, Azure/azure-sdk-for-python#41571. They now have two functions to handle both swagger generated SDK and typespec generated SDK:

  • is_generated_model to identify auto generated SDK models
  • attribute_list to list all attribute names which are after renaming
  • we can call getattr(obj, attr) for the attribute we get from attribute_list to get the attribute value which is after formatting

This PR changes the dict formatting for typespec generated SDK models with new attribute_list func. We didn't adopt is_generated_model because we don't want to change swagger generated SDK models' dict formatting to avoid any regressions.

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature


This checklist is used to make sure that common guidelines for a pull request are followed.

Copilot AI review requested due to automatic review settings July 8, 2025 05:50
@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Jul 8, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️latest
️✔️3.12
️✔️3.9

@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Jul 8, 2025

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Jul 8, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link

github-actions bot commented Jul 8, 2025

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the CLI’s todict utility to use Azure Core’s serialization helpers (is_generated_model and attribute_list) instead of the older as_dict logic, ensuring correct handling of renamed/ formatted SDK model attributes.

  • Import is_generated_model and attribute_list from azure.core.serialization
  • Replace the old as_dict branch with a single path using is_generated_model
  • Remove legacy typespec as_dict handling
Comments suppressed due to low confidence (2)

src/azure-cli-core/azure/cli/core/util.py:654

  • Add unit tests for this new is_generated_model branch to cover both swagger and typespec generated models, verifying that attribute_list correctly enumerates renamed/formatted attributes.
    if is_generated_model(obj):

src/azure-cli-core/azure/cli/core/util.py:629

  • Update the function docstring to mention the support for Azure Core generated models via is_generated_model and attribute_list.
def todict(obj, post_processor=None):

@microsoft-github-policy-service microsoft-github-policy-service bot added Auto-Assign Auto assign by bot Core CLI core infrastructure labels Jul 8, 2025
@evelyn-ys evelyn-ys changed the title {Core} Adopt serialization func from azure-core for CLI's todict formatting {Core} Adopt serialization func from azure-core for CLI's todict formatting Jul 8, 2025
if is_generated_model(obj):
result = {}
for attr in attribute_list(obj):
if hasattr(obj, attr):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check whether the attr exist because we may import SDK model and then do some customization like keyvault certificate L122-127

def _default_certificate_profile(cmd):
def get_model(x):
return cmd.loader.get_sdk(x, resource_type=ResourceType.DATA_KEYVAULT_CERTIFICATES, mod='_generated_models')
Action = get_model('Action')
ActionType = get_model('ActionType')
KeyUsageType = get_model('KeyUsageType')
CertificateAttributes = get_model('CertificateAttributes')
CertificatePolicy = get_model('CertificatePolicy')
IssuerParameters = get_model('IssuerParameters')
KeyProperties = get_model('KeyProperties')
LifetimeAction = get_model('LifetimeAction')
SecretProperties = get_model('SecretProperties')
X509CertificateProperties = get_model('X509CertificateProperties')
Trigger = get_model('Trigger')
template = CertificatePolicy(
key_properties=KeyProperties(
exportable=True,
key_type='RSA',
key_size=2048,
reuse_key=True
),
secret_properties=SecretProperties(
content_type='application/x-pkcs12'
),
x509_certificate_properties=X509CertificateProperties(
key_usage=[
KeyUsageType.c_rl_sign,
KeyUsageType.data_encipherment,
KeyUsageType.digital_signature,
KeyUsageType.key_encipherment,
KeyUsageType.key_agreement,
KeyUsageType.key_cert_sign
],
subject='CN=CLIGetDefaultPolicy',
validity_in_months=12
),
lifetime_actions=[LifetimeAction(
trigger=Trigger(
days_before_expiry=90
),
action=Action(
action_type=ActionType.auto_renew
)
)],
issuer_parameters=IssuerParameters(
name='Self',
),
attributes=CertificateAttributes(
enabled=True
)
)
del template.id
del template.attributes
del template.issuer_parameters.certificate_type
del template.lifetime_actions[0].trigger.lifetime_percentage
del template.x509_certificate_properties.subject_alternative_names
del template.x509_certificate_properties.ekus
return template

Some attributes are deleted

# The way to detect if it's a typespec generated model is to check the private `_is_model` attribute
# azure-core provided new function `attribute_list` to list all attribute names
# so that we don't need to use raw __dict__ directly
if getattr(obj, "_is_model", False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did it handle the nested case? When the internal property is also an object?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will call todict recursively

@evelyn-ys evelyn-ys merged commit 4254e54 into Azure:dev Jul 10, 2025
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auto-Assign Auto assign by bot Core CLI core infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants