[CT-2054] Add deprecation warnings for LOG_PATH and TARGET_PATH in dbt_project.yml

[ChenyuLInx]

Issue

Currently, user would be able to set log-path and target-path in dbt_project.yml.

But since we initialize all of the flags before loading the project object, we need to
manually load the project object from a profile dir(link), read out the log-path and set it there.

One other issue with this setup is that dbt-server would load the project object before
invoking the rest of the core functionality, during that process we are actually calling
get_flags function with will create a global flag. This causes circular dependency between
project and flags.

Solution

Move log_path and target-path out of dbt_project and move them into user_config that's
currently in profiles.yml. This would be a breaking change for certain user but clean up the
circular dependency of project and Flags.

Acceptance Criteria

LOG_PATH and TARGET_PATH are no longer being specified in dbt_project.yml, Flags no longer depend on project object.

[jtcohen6]

Thanks for opening @ChenyuLInx, and for facilitating the conversation yesterday. We take these sorts of changes seriously; they ought to be exceedingly rare.

While at first glance it seems like target-path and log-path are "paths" just like model-paths, seed-paths, etc, we've realized that these are really runtime configs, more so than project configs, and so they rightly belong as CLI flags / env vars, rather than things defined in dbt_project.yml. Insofar as it provides us a better foundation upon which to build better, cleaner APIs—ones that the community really stands to benefit from!—the benefit here outweighs the cost.

Let's make this change in an early beta, and invite feedback from the community. It is technically a breaking change, I don't anticipate this being a significant inconvenience for anyone; a thing is still functionally possible, through (IMO) an even-more-sensible mechanism.

[jtcohen6]

The default values should remain the same, relative to --project-dir (dbt_project.yml):

• target/ for compiled files, artifacts, etc
• logs/ for logs

I don't even think we need to move these into UserConfig. (Given where it lives today, in profiles.yml, feels odd to define a relative path in a file that potentially lives elsewhere on the machine.) I think the param (CLI flag + env var) are sufficient to enable end user configuration.

Ideal outcome: Keep existing functionality for v1.5, raise a deprecation warning if target-path and log-path are set in dbt_project.yml. We remove the functionality in a future v1.x release.

Acceptable outcome: Raise a deprecation warning if target-path and log-path are specified in dbt_project.yml. Ignore their values.

[stu-k]

@jtcohen6 Could you choose one of the outcomes for this ticket to address? This will help us point in the next BLG.

[jtcohen6]

Let's estimate this on the basis of the Ideal Outcome. If it's not possible, we can fall back to the Acceptable Outcome, which is still better than doing nothing.

[stu-k]

Created a spike to better understand how to address this missing functionality in #7140.