Skip to content

chore(backend): correct node id for metrics reporting#12213

Merged
google-oss-prow[bot] merged 1 commit intokubeflow:masterfrom
HumairAK:fix_metrics
Sep 3, 2025
Merged

chore(backend): correct node id for metrics reporting#12213
google-oss-prow[bot] merged 1 commit intokubeflow:masterfrom
HumairAK:fix_metrics

Conversation

@HumairAK
Copy link
Collaborator

@HumairAK HumairAK commented Sep 3, 2025

Description of your changes:

This fixes metrics reporting for v1 pipeline runs that log metrics. Previously the wrong node id was sent to the api server.

You can verify this with this sample pipeline:

sample.py

from typing import NamedTuple

import kfp
from kfp import dsl
from kfp.components import create_component_from_func


# Step 1: Define a Python function as a component
def train_op() -> NamedTuple('Outputs', [('mlpipeline_metrics', 'Metrics')]):
    import json
    import random
    from pathlib import Path

    # Generate a dummy metric (accuracy)
    accuracy = random.uniform(0.7, 0.99)

    # Write metrics to the designated file
    metrics = {
        "metrics": [
            {
                "name": "accuracy",
                "numberValue": accuracy,
                "format": "PERCENTAGE"
            }
        ]
    }

    # KFP convention: write metrics.json to /mlpipeline-metrics.json
    # Path("/mlpipeline-metrics.json").write_text(json.dumps(metrics))

    return [json.dumps(metrics)]


# Step 2: Convert function to KFP component
train_component = create_component_from_func(
    func=train_op,
    base_image="python:3.9"
)


# Step 3: Define the pipeline
@dsl.pipeline(
    name="Simple Metric Logging Pipeline",
    description="A minimal example that logs a single metric."
)
def simple_pipeline():
    train_task = train_component()


# Step 4: Compile the pipeline
if __name__ == "__main__":
    kfp.compiler.Compiler().compile(
        pipeline_func=simple_pipeline,
        package_path="simple_pipeline.yaml"
    )

Here's the resulting json if you query this run:

sample.json

{
  "run": {
    "id": "aee677a3-c3a1-47a2-bbbb-81821f47c447",
    "name": "Run of simple_pipeline (261da)",
    "pipeline_spec": {}, # omitted
    "resource_references": [
      {
        "key": {
          "type": "EXPERIMENT",
          "id": "fa3653ac-312d-46bc-a67a-a9eaab61c779"
        },
        "relationship": "OWNER"
      },
      {
        "key": {
          "type": "PIPELINE_VERSION",
          "id": "428f5c85-d0c6-4062-b795-76b846272c59"
        },
        "relationship": "CREATOR"
      }
    ],
    "service_account": "pipeline-runner",
    "created_at": "2025-09-03T19:50:04Z",
    "scheduled_at": "2025-09-03T19:50:04Z",
    "finished_at": "2025-09-03T19:50:14Z",
    "status": "Succeeded",
    "metrics": [
      {
        "name": "accuracy",
        "node_id": "simple-metric-logging-pipeline-gxltq-train-op-3397519976",
        "number_value": 0.7160825905928586,
        "format": "PERCENTAGE"
      }
    ]
  },
  "pipeline_runtime": { } # omitted
}

Checklist:

Copy link
Collaborator

@mprahl mprahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
/lgtm

Signed-off-by: Humair Khan <HumairAK@users.noreply.github.com>
Copy link
Collaborator

@mprahl mprahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Sep 3, 2025
@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mprahl

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 98632a1 into kubeflow:master Sep 3, 2025
86 checks passed
krishanbhasin-px pushed a commit to krishanbhasin-px/kubeflow-pipelines that referenced this pull request Sep 18, 2025
Signed-off-by: Humair Khan <HumairAK@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants