Skip to content

Introduce forward-only plans#109

Merged
izeigerman merged 3 commits intomainfrom
forward-only-plan
Dec 29, 2022
Merged

Introduce forward-only plans#109
izeigerman merged 3 commits intomainfrom
forward-only-plan

Conversation

@izeigerman
Copy link
Copy Markdown
Collaborator

This PR pretty much covers the entire forward-only functionality. Here's what remains and will be covered in subsequent PRs:

  • Prompt user for the "preview" time range when in forward-only mode.
  • Add schema migration to the Plan evaluation.
  • Support for clones instead of just empty temporary tables.

Comment thread sqlmesh/core/plan/definition.py Outdated
and upstream_snapshot.is_paused
):
raise PlanError(
f"Modified model '{model_name}' depends on a paused forward-only snapshot {upstream_snapshot.snapshot_id}. "
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we show the forward-only model name instead of snapshot id? I believe this is currently the only place where we tell users about the concept of snapshots.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point. The challenge here is that we don't talk about just any version of that parent model, but a very specific instance of it. How do you think we can covey this without exposing a snapshot?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can use the name here but then still use a snapshot id when I say "2) promote the snapshot ...".

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or I guess I might just drop option 2) from suggestions

Copy link
Copy Markdown
Collaborator

@eakmanrq eakmanrq Dec 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think option 2) is useful. How hard would it be to say something like "model that is currently active in these environments: needs to be promoted to production environment". Basically check where all that snapshot is active in. In most cases it should be a single environment.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you helped me realize that we can refer to it as just "the current version" since we know that this snapshot is currently in the user's local state. How about this then:

raise PlanError(
    f"Modified model '{model_name}' depends on the paused version of model '{upstream}'. "
    "Possible remedies: "
    "1) make sure your codebase is up-to-date; "
    f"2) promote the current version of model '{upstream}' in the production environment; "
    "3) recreate this plan in a forward-only mode."
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that works. I think for this person, especially if they are less technical, it might be difficult to understand how to promote just that upstream model to product but it is likely in that case they will be pinging an engineer anyways to resolve the issue. Also telling them the environment helps some but not completely since in this scenario it is likely someone else's environment and they are pinging them anyways. So good for now.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, pushed

@izeigerman izeigerman merged commit d1ffd0b into main Dec 29, 2022
@izeigerman izeigerman deleted the forward-only-plan branch December 29, 2022 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants