Skip to content

Commit 9853b58

Browse files
committed
Edit the Virtual Data Environments post
1 parent 5f63e2b commit 9853b58

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

posts/virtual_environments.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ Variations of this approach can be seen in dbt’s [state](https://docs.getdbt.c
5555
This approach, though clearly an improvement, still has several shortcomings:
5656
- Users are not informed about the **indirect** impact their changes have on downstream datasets, so they are forced to rely on their judgment and choose to either recompute the entire dependency graph (correctness) or recompute modified datasets only (efficiency). Anything in between requires careful manual intervention on the user’s part, which is error-prone and doesn’t scale to large dependency graphs.
5757
- Existing datasets can be reused when creating new development environments, but not when deploying changes to production. As mentioned earlier, this leads to changes being recomputed all over again without any guarantee of yielding the same results. Meanwhile, existing production data and deployed code remain out of sync for the duration of the recomputation.
58+
- State is discarded once the changes are deployed to production, making it impossible to reuse previously built datasets when reverting applied changes.
5859

5960
I will demonstrate that there doesn’t have to be a tradeoff between correctness and efficiency. Both can be achieved without compromising either.
6061

0 commit comments

Comments
 (0)