plots: flexible dvc.yaml spec

[dberenbaum]

YAML doesn't permit dup keys also even if we change that to arrays we will have the issue of duplication and discrepancy, i.e. one template under one key and another one under the second key. So plot names should be unique string, whichever you write in target I suppose and datafile should go as key. We may still use a key if datafile is absent, this will simplify a very common one file case. So:

plots:
- train_vs_val:
    x: epoch
    y: loss
    data: [train_loss.csv, val_loss.csv]


- val_f1.csv:
    x: epoch
    y: [f1_class_0, f1_class_1]
    y_label: f1
    # data is absent, using key value: val_f1.csv

# Separately plot since we have TWO plots, even though with the same data file
- scores_acc:
    x: epoch
    y: acc
    data: scores.csv
- scores_auc:
    x: epoch
    y: auc
    data: scores.csv

Originally posted by @Suor in #5980 (reply in thread)

[dberenbaum]

@pared I think it makes sense to implement in reverse order of how it's listed above.

1. Multiple plot specs from the same file:

# Separately plot since we have TWO plots, even though with the same data file
- scores_acc:
    x: epoch
    y: acc
    data: scores.csv
- scores_auc:
    x: epoch
    y: auc
    data: scores.csv

Requirements:

• Have a way to define plots spec separate from a stage output.
• Add an optional data field to specify file, in which case plot key is used as a title.
• Update plots commands.
• Nice to have: dvc plots add command to generate plots.

2. Multiple series per axis:

- val_f1.csv:
    x: epoch
    y: [f1_class_0, f1_class_1]
    y_label: f1
    # data is absent, using key value: val_f1.csv

Requirements:

• Enable x, y, x-axis, and y-axis fields to be arrays.
• Show multiple lines/series in plots.
• Support diffs of multi-series plots -- most likely as a facet grid.

3. Multiple files within one plot:

- train_vs_val:
    x: epoch
    y: loss
    data: [train_loss.csv, val_loss.csv]

Requirements:

• Enable data field to be an array.
• Show multiple lines/series in plots and support diffs (similar to above).
• Out of scope: x or y field arrays combined with data array. This can throw an error.

[pared]

Question: can we provide parallel coordinates plots config alongside/inside plots spec?

[mattlbeck]

Related: #6316

[dberenbaum]

Question: can we provide parallel coordinates plots config alongside/inside plots spec?

IMO let's not worry about it as part of this issue.