IssueRaiser

Joining different labels

If we are monitoring complex systems, we might get metrics on different connected entities from different metric providers. The label naming might not be 100% aligned, i.e. in Kubernetes we get container metrics with a `pod_name` label from cAdvisor and metrics with a `pod` label from kube-state-metrics. The values are the same pod identifying names. However, we have to use `label_replace()` to be able to join them properly, which is always somewhat awkward and hard to decipher. I think this will be a common case across many systems and first-class support to join such labels could be beneficial. I imagine a syntax along the lines of: ``` container_last_seen and on(pod_name=pod) kube_pod_info ``` Where the names left and right of `=` refer to the matched on labels of the left and right metrics respectively. I'd hope it to be easy enough to implement in our parser and think it's generally an rather obvious syntax for the semantics. I know that's not a trivial extension to PromQL, so this is just to start a discussion. @brian-brazil

Posted by fabxc on Nov 18, 2016, 11:23 AM

Tags

Related issues

Raise real-world problems, find people facing the same issue, and discuss solutions.4,500 issues0 comments

ux (Sysadmin)

Joining different labels

fabxcNov 18, 2016, 11:23 AM

If we are monitoring complex systems, we might get metrics on different connected entities from different metric providers. The label naming might not be 100% aligned, i.e. in Kubernetes we get container metrics with a `pod_name` label from cAdvisor and metrics with a `pod` label from kube-state-metrics. The values are the same pod identifying names. However, we have to use `label_replace()` to be able to join them properly, which is always somewhat awkward and hard to decipher. I think this will be a common case across many systems and first-class support to join such labels could be beneficial. I imagine a syntax along the lines of: ``` container_last_seen and on(pod_name=pod) kube_pod_info ``` Where the names left and right of `=` refer to the matched on labels of the left and right metrics respectively. I'd hope it to be easy enough to implement in our parser and think it's generally an rather obvious syntax for the semantics. I know that's not a trivial extension to PromQL, so this is just to start a discussion. @brian-brazil

Comments (0)

No comments yet. Start the discussion.

Comment on this issue