fix(integ-runner): integ-runner produces snapshot that doesn't validate #666
Conversation
The integ-runner uses `synthFast` to produce the assembly that will be used to compare the snapshot to, to determine if there is "a change". However, when a snapshot has changed and a test is run, there is a fork: - In dry-run mode: `synthFast` will be used to produce the updated snapshot. - In real-mode, the result of `this.deploy()` will be the updated snapshot (but also: the next time it will be compared to the result of `synthFast`). We are running into a situation where the results of `synthFast` and `this.deploy` are different, and snapshot validation fails. In this change, we always generate the snapshot using `synthFast`, and ignore the template produced by `this.deploy()`. This makes sure that the templates will at least compile equal. I'm not sure why this ever worked this way, because the template that is deployed can be specialized to an account and region, but the snapshot can never be, so we need to investigate a little more.
|
Okay the actual issue we had related to the flag Don't know what that was about, but I still feel the snapshot-that-gets-commited and the snapshot-that-gets-compared should use the same code paths. |
|
To close this out: The source of this bug was that during We would pass the boolean This behavior will be fixed when we make the new |
…okups: false` In #666 we used `synthFast()` to always generate a snapshot in the same was as was used for validating (which was also using `synthFast()`). The difference being, that the context we load depends on the `enableLookups: false|true` flag that's passed to `new IntegTest()` in the test case itself. So when writing the snapshot we have to take that same field into account.
…okups: false` (#685) In #666 we used `synthFast()` to always generate a snapshot in the same was as was used for validating (which was also using `synthFast()`). The difference being, that the context we load depends on the `enableLookups: false|true` flag that's passed to `new IntegTest()` in the test case itself. So when writing the snapshot we have to take that same field into account. --- By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license
3 participants
The integ-runner uses
synthFastto produce the assembly that will be used to compare the snapshot to, to determine if there is "a change".However, when a snapshot has changed and a test is run, there is a fork:
synthFastwill be used to produce the updated snapshot.this.deploy()will be the updated snapshot (but also: the next time it will be compared to the result ofsynthFast).We are running into a situation where the results of
synthFastandthis.deployare different, and snapshot validation fails.In this change, we always generate the snapshot using
synthFast, and ignore the template produced bythis.deploy(). This makes sure that the templates will at least compile equal.I'm not sure why this ever worked this way, because the template that is deployed can be specialized to an account and region, but the snapshot can never be, so we need to investigate a little more.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license