[DataFactory]Add AzureMySqlSink and two datasets #6820

wenbof-zz · 2019-07-31T13:43:41Z

Latest improvements:

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Contribution checklist:

I have reviewed the documentation for the workflow.
Validation tools were run on swagger spec(s) and have all been fixed in this PR.
The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

openapi-sdkautomation · 2019-07-31T13:43:43Z

In Testing, Please Ignore

[Logs] (Generated from 640aecc, Iteration 6)

Python: test-repo-billy/azure-sdk-for-python [Logs] [Diff]

azure-mgmt-datafactory [Logs] [Instructions] [Generation PR]
- azure-mgmt-datafactory-0.7.0.zip
- azure_mgmt_datafactory-0.7.0-py2.py3-none-any.whl

Java: test-repo-billy/azure-sdk-for-java [Logs] [Diff]

datafactory/resource-manager/v2017_09_01_preview [Logs]
datafactory/resource-manager/v2018_06_01 [Logs]

Go: test-repo-billy/azure-sdk-for-go [Logs] [Diff]

datafactory/mgmt/2018-06-01 [Logs] [Instructions] [Generation PR]
- datafactory.mgmt.2018-06-01.zip
preview/datafactory/mgmt/2017-09-01-preview [Logs]

JavaScript: test-repo-billy/azure-sdk-for-js [Logs] [Diff]

@azure/arm-datafactory [Logs]

wenbof-zz · 2019-07-31T13:45:04Z

the related SDK built and local tests which run well when testing locally: wenbof-zx/azure-sdk-for-net@a2f1622

AutorestCI · 2019-07-31T13:46:24Z

Automation for azure-sdk-for-python

A PR has been created for you based on this PR content.

Once this PR will be merged, content will be added to your service PR:
Azure/azure-sdk-for-python#5747

AutorestCI · 2019-07-31T13:51:03Z

Automation for azure-sdk-for-go

The initial PR has been merged into your service PR:
Azure/azure-sdk-for-go#5451

azuresdkci · 2019-07-31T14:05:04Z

Can one of the admins verify this patch?

...atafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json

majastrz · 2019-08-02T18:32:24Z

...atafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json

      }
    },
+    "AmazonRedshiftTableDataset": {
+      "x-ms-discriminator-value": "AmazonRedshiftTable",


AmazonRedshiftTable [](start = 35, length = 19)

So you're adding a new discriminator value to an existing stable API. The SDK generated from the swagger before the change will be able to deal with the discriminator value of "AmazonRedshiftTable" because it's modeled as a string, but it will not be able to deserialize the payload specific to AmazonRedshiftTable. This unfortunately makes it an SDK breaking change. Can you add this in a new API version? #Closed

Sure. it will be generated in a new API version when making changes to SDK. #Closed

as similar as this SDK change where you can see that we change the API version Azure/azure-sdk-for-net#7044 #Closed

To generate a new API version when making changes to SDK, you need the swagger to exist for the new API version, no?

In reply to: 310406114 [](ancestors = 310406114)

right. the order I understand is also the same as you : merge swagger change first, then generate SDK based on the latest swagger. #Closed

the new SDK version is always built based on the latest swagger. which means the old SDK were built based on swagger at specific time and then kept intact there. #Closed

Let's not assume. Can you verify the behavior? What I want to know if it will throw. If the SDK just accepts the payload even if the user can't really interact with it, I'd be ok with that.

@michaeljqzq, can you chime in here as well? What is the behavior with the generated SDK if a new discriminator value is being returned in the backend, but the SDK doesn't know about it yet? Will it throw an exception? #Closed

I double checked our changes on SDK and can confirm that we manually changed the version and appending the new changes on existing one every time to the new version. @hvermis could you please also help confirm as well? thanks. #Closed

We do not update the API version. This is a known behavior and we have been adding new types like this. Our customers are aware of this. #Closed

@wenbof and @hvermis, you are not actually answering my question, which is delaying this review. I'm not asking whether you have done this before (you clearly have). Azure is a distributed system. There's no mechanism to ensure that all customers simultaneously upgrade to latest and greatest SDK that you have generated, which is why we have breaking changes guidance.

So I will ask the question again in a different way. Let's say you have SDK v2 that supports connectors a and b and SDK v3 that supports connectors a, b, and c. User A has been happily using SDK v2 to manage resources using connectors a and b. Suddenly user B creates resources using connector c in the same subscription or RG via SDK v3.

What will User A now see when he/she queries for all resources?

Will the old SDK v2 throw an exception?

Will the responses deserialize successfully but incompletely?

Please pick either answer 1 or 2. #Closed

...tafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Pipeline.json

majastrz

I added some questions. Please take a look.

wenbof-zz · 2019-08-04T22:16:30Z

I added some questions. Please take a look.
replied with my comments. could you please double review? thanks

wenbof-zz · 2019-08-05T18:48:50Z

@majastrz could you please check whether you have any further concerns or comments? thanks.

hvermis · 2019-08-06T19:32:13Z

@majastrz Answer to yourquestion is #2. This done by ClientRuntime library which is one of common libraries owned by the Azure SDK team. #Closed

majastrz · 2019-08-06T19:33:33Z

@majastrz Answer to yourquestion is #2. This done by ClientRuntime library which is one of common libraries owned by the Azure SDK team.

Great! Thanks for confirming #Closed

majastrz

Signed off from ARM side.

[DataFactory]Add AzureMySqlSink and two datasets

1b18c20

azuresdkci assigned michaeljqzq Jul 31, 2019

azuresdkci requested a review from michaeljqzq July 31, 2019 13:45

AutorestCI mentioned this pull request Jul 31, 2019

[AutoPR datafactory/resource-manager] [DataFactory]Add AzureMySqlSink and two datasets Azure/azure-sdk-for-python#6576

Closed

AutorestCI mentioned this pull request Jul 31, 2019

[AutoPR datafactory/resource-manager] [DataFactory]Add AzureMySqlSink and two datasets Azure/azure-sdk-for-go#5395

Merged

openapi-sdkautomation bot mentioned this pull request Jul 31, 2019

[AutoPR datafactory/resource-manager/v2018_06_01] [DataFactory]Add AzureMySqlSink and two datasets Azure/azure-sdk-for-java#4735

Closed

openapi-sdkautomation bot mentioned this pull request Jul 31, 2019

[AutoPR @azure/arm-datafactory] [DataFactory]Add AzureMySqlSink and two datasets Azure/azure-sdk-for-js#4556

Closed

michaeljqzq reviewed Aug 1, 2019

View reviewed changes

...atafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Dataset.json Show resolved Hide resolved

michaeljqzq added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Aug 1, 2019

hbzhxying added 2 commits August 1, 2019 15:03

split name

ad04152

don't split name for odbc

640aecc

majastrz reviewed Aug 2, 2019

View reviewed changes

...tafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Pipeline.json Show resolved Hide resolved

majastrz added the ARMReviewInProgress label Aug 2, 2019

majastrz reviewed Aug 2, 2019

View reviewed changes

majastrz added ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review and removed ARMReviewInProgress WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Aug 6, 2019

majastrz approved these changes Aug 6, 2019

View reviewed changes

michaeljqzq approved these changes Aug 7, 2019

View reviewed changes

michaeljqzq merged commit 0e1793c into Azure:master Aug 7, 2019

xiaoyingQQ mentioned this pull request Aug 8, 2019

Add support for azure mysql sink Azure/azure-sdk-for-net#7207

Merged

[DataFactory]Add AzureMySqlSink and two datasets #6820

[DataFactory]Add AzureMySqlSink and two datasets #6820

Uh oh!

Conversation

wenbof-zz commented Jul 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Latest improvements:

Contribution checklist:

ARM API Review Checklist

Uh oh!

openapi-sdkautomation bot commented Jul 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

In Testing, Please Ignore

[Logs] (Generated from 640aecc, Iteration 6)

Uh oh!

wenbof-zz commented Jul 31, 2019

Uh oh!

AutorestCI commented Jul 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Automation for azure-sdk-for-python

Uh oh!

AutorestCI commented Jul 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Automation for azure-sdk-for-go

Uh oh!

azuresdkci commented Jul 31, 2019

Uh oh!

Uh oh!

majastrz Aug 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenbof-zz Aug 4, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenbof-zz Aug 4, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majastrz Aug 5, 2019

Choose a reason for hiding this comment

Uh oh!

wenbof-zz Aug 5, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenbof-zz Aug 5, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majastrz Aug 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wenbof-zz Aug 5, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hvermis Aug 6, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majastrz Aug 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

majastrz left a comment

Choose a reason for hiding this comment

Uh oh!

wenbof-zz commented Aug 4, 2019

Uh oh!

wenbof-zz commented Aug 5, 2019

Uh oh!

hvermis commented Aug 6, 2019 • edited by majastrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

majastrz commented Aug 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

wenbof-zz commented Jul 31, 2019 •

edited

Loading

openapi-sdkautomation bot commented Jul 31, 2019 •

edited

Loading

AutorestCI commented Jul 31, 2019 •

edited

Loading

AutorestCI commented Jul 31, 2019 •

edited

Loading

majastrz Aug 2, 2019 •

edited

Loading

wenbof-zz Aug 4, 2019 •

edited by majastrz

Loading

wenbof-zz Aug 4, 2019 •

edited by majastrz

Loading

wenbof-zz Aug 5, 2019 •

edited by majastrz

Loading

wenbof-zz Aug 5, 2019 •

edited by majastrz

Loading

majastrz Aug 5, 2019 •

edited

Loading

wenbof-zz Aug 5, 2019 •

edited by majastrz

Loading

hvermis Aug 6, 2019 •

edited by majastrz

Loading

majastrz Aug 6, 2019 •

edited

Loading

hvermis commented Aug 6, 2019 •

edited by majastrz

Loading

majastrz commented Aug 6, 2019 •

edited

Loading