Skip to content

Commit

Permalink
feat: enable field label in TenantStageSpec of PodLogs pipeline (graf…
Browse files Browse the repository at this point in the history 
…ana#2427)

* feat: enable field label in TenantStageSpec of PodLogs pipeline

Signed-off-by: Mon Si <siiimoooooon@gmail.com>
Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com>
  • Loading branch information
@siiimooon @tpaschalis
siiimooon and tpaschalis committed Oct 27, 2022
1 parent 7c1fb2c commit f42768d
Show file tree
Hide file tree
Showing 7 changed files with 87 additions and 28 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ Main (unreleased)
Grafana Agent, but are not meant to act as a replacement for fully featured
components like `prometheus.integration.node_exporter`. (@rfratto)

- Enable field label in TenantStageSpec of PodLogs pipeline. (@siiimooon)

- Enable reporting of enabled integrations. (@marctc)

- Grafana Agent Flow: `prometheus.remote_write` and `prometheus.relabel` will
Expand Down
18 changes: 15 additions & 3 deletions docs/sources/operator/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4510,14 +4510,26 @@ picking it from a field in the extracted data map.</p>
<tbody>
<tr>
<td>
<code>label</code><br/>
<em>
string
</em>
</td>
<td>
<p>Name from labels to whose value should be set as tenant ID. Mutually exclusive with
source and value.</p>
</td>
</tr>
<tr>
<td>
<code>source</code><br/>
<em>
string
</em>
</td>
<td>
<p>Name from extracted data to use as the tenant ID. Mutually exclusive with
value.</p>
label and value.</p>
</td>
</tr>
<tr>
Expand All @@ -4529,7 +4541,7 @@ string
</td>
<td>
<p>Value to use for the template ID. Useful when this stage is used within a
conditional pipeline such as match. Mutually exclusive with source.</p>
conditional pipeline such as match. Mutually exclusive with label and source.</p>
</td>
</tr>
</tbody>
Expand Down Expand Up @@ -4614,5 +4626,5 @@ Can be skip or fudge. Defaults to fudge.</p>
<hr/>
<p><em>
Generated with <code>gen-crd-api-reference-docs</code>
on git commit <code>53ddc06d</code>.
on git commit <code>87429255</code>.
</em></p>
8 changes: 6 additions & 2 deletions pkg/operator/apis/monitoring/v1alpha1/types_logs.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -526,12 +526,16 @@ type TemplateStageSpec struct {
// TenantStageSpec is an action stage that sets the tenant ID for the log entry
// picking it from a field in the extracted data map.
type TenantStageSpec struct {
// Name from labels whose value should be set as tenant ID. Mutually exclusive with
// source and value.
Label string `json:"label,omitempty"`

// Name from extracted data to use as the tenant ID. Mutually exclusive with
// value.
// label and value.
Source string `json:"source,omitempty"`

// Value to use for the template ID. Useful when this stage is used within a
// conditional pipeline such as match. Mutually exclusive with source.
// conditional pipeline such as match. Mutually exclusive with label and source.
Value string `json:"value,omitempty"`
}

Expand Down
2 changes: 2 additions & 0 deletions pkg/operator/config/logs_templates_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -415,12 +415,14 @@ func TestLogsStages(t *testing.T) {
name: "tenant",
input: map[string]interface{}{"spec": &gragent.PipelineStageSpec{
Tenant: &gragent.TenantStageSpec{
Label: "__meta_kubernetes_pod_label_fake",
Source: "customer_id",
Value: "fake",
},
}},
expect: util.Untab(`
tenant:
label: __meta_kubernetes_pod_label_fake
source: customer_id
value: fake
`),
Expand Down
1 change: 1 addition & 0 deletions pkg/operator/config/templates/component/logs/stages.libsonnet
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ local new_stage = function(spec) {

// spec.Tenant :: *TenantStageSpec
tenant: if spec.Tenant != null then {
label: optionals.string(spec.Tenant.Label),
source: optionals.string(spec.Tenant.Source),
value: optionals.string(spec.Tenant.Value),
},
Expand Down
76 changes: 55 additions & 21 deletions production/operator/crds/monitoring.grafana.com_grafanaagents.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1642,13 +1642,13 @@ spec:
Cannot be updated.
type: string
ports:
description: List of ports to expose from the container. Exposing
a port here gives the system additional information about
the network connections a container uses, but is primarily
informational. Not specifying a port here DOES NOT prevent
that port from being exposed. Any port which is listening
on the default "0.0.0.0" address inside a container will be
accessible from the network. Cannot be updated.
description: List of ports to expose from the container. Not
specifying a port here DOES NOT prevent that port from being
exposed. Any port which is listening on the default "0.0.0.0"
address inside a container will be accessible from the network.
Modifying this array with strategic merge patch may corrupt
the data. For more information See https://github.com/kubernetes/kubernetes/issues/108255.
Cannot be updated.
items:
description: ContainerPort represents a network port in a
single container.
Expand Down Expand Up @@ -2897,13 +2897,13 @@ spec:
Cannot be updated.
type: string
ports:
description: List of ports to expose from the container. Exposing
a port here gives the system additional information about
the network connections a container uses, but is primarily
informational. Not specifying a port here DOES NOT prevent
that port from being exposed. Any port which is listening
on the default "0.0.0.0" address inside a container will be
accessible from the network. Cannot be updated.
description: List of ports to expose from the container. Not
specifying a port here DOES NOT prevent that port from being
exposed. Any port which is listening on the default "0.0.0.0"
address inside a container will be accessible from the network.
Modifying this array with strategic merge patch may corrupt
the data. For more information See https://github.com/kubernetes/kubernetes/issues/108255.
Cannot be updated.
items:
description: ContainerPort represents a network port in a
single container.
Expand Down Expand Up @@ -5505,6 +5505,19 @@ spec:
only "value". The requirements are ANDed.
type: object
type: object
matchLabelKeys:
description: MatchLabelKeys is a set of pod label keys to select
the pods over which spreading will be calculated. The keys
are used to lookup values from the incoming pod labels, those
key-value labels are ANDed with labelSelector to select the
group of existing pods over which spreading will be calculated
for the incoming pod. Keys that don't exist in the incoming
pod labels will be ignored. A null or empty list means only
match against labelSelector.
items:
type: string
type: array
x-kubernetes-list-type: atomic
maxSkew:
description: 'MaxSkew describes the degree to which pods may
be unevenly distributed. When `whenUnsatisfiable=DoNotSchedule`,
Expand Down Expand Up @@ -5545,22 +5558,43 @@ spec:
minimum\" is treated as 0. In this situation, new pod with
the same labelSelector cannot be scheduled, because computed
skew will be 3(3 - 0) if new Pod is scheduled to any of the
three zones, it will violate MaxSkew. \n This is an alpha
field and requires enabling MinDomainsInPodTopologySpread
feature gate."
three zones, it will violate MaxSkew. \n This is a beta field
and requires the MinDomainsInPodTopologySpread feature gate
to be enabled (enabled by default)."
format: int32
type: integer
nodeAffinityPolicy:
description: "NodeAffinityPolicy indicates how we will treat
Pod's nodeAffinity/nodeSelector when calculating pod topology
spread skew. Options are: - Honor: only nodes matching nodeAffinity/nodeSelector
are included in the calculations. - Ignore: nodeAffinity/nodeSelector
are ignored. All nodes are included in the calculations. \n
If this value is nil, the behavior is equivalent to the Honor
policy. This is a alpha-level feature enabled by the NodeInclusionPolicyInPodTopologySpread
feature flag."
type: string
nodeTaintsPolicy:
description: "NodeTaintsPolicy indicates how we will treat node
taints when calculating pod topology spread skew. Options
are: - Honor: nodes without taints, along with tainted nodes
for which the incoming pod has a toleration, are included.
- Ignore: node taints are ignored. All nodes are included.
\n If this value is nil, the behavior is equivalent to the
Ignore policy. This is a alpha-level feature enabled by the
NodeInclusionPolicyInPodTopologySpread feature flag."
type: string
topologyKey:
description: TopologyKey is the key of node labels. Nodes that
have a label with this key and identical values are considered
to be in the same topology. We consider each <key, value>
as a "bucket", and try to put balanced number of pods into
each bucket. We define a domain as a particular instance of
a topology. Also, we define an eligible domain as a domain
whose nodes match the node selector. e.g. If TopologyKey is
"kubernetes.io/hostname", each Node is a domain of that topology.
And, if TopologyKey is "topology.kubernetes.io/zone", each
zone is a domain of that topology. It's a required field.
whose nodes meet the requirements of nodeAffinityPolicy and
nodeTaintsPolicy. e.g. If TopologyKey is "kubernetes.io/hostname",
each Node is a domain of that topology. And, if TopologyKey
is "topology.kubernetes.io/zone", each zone is a domain of
that topology. It's a required field.
type: string
whenUnsatisfiable:
description: 'WhenUnsatisfiable indicates how to deal with a
Expand Down
8 changes: 6 additions & 2 deletions production/operator/crds/monitoring.grafana.com_podlogs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -386,14 +386,18 @@ spec:
data map. If the field is missing, the default LogsClientSpec.tenantId
will be used.
properties:
label:
description: Name from labels to whose value should be set
as tenant ID. Mutually exclusive with source and value.
type: string
source:
description: Name from extracted data to use as the tenant
ID. Mutually exclusive with value.
ID. Mutually exclusive with label and value.
type: string
value:
description: Value to use for the template ID. Useful when
this stage is used within a conditional pipeline such
as match. Mutually exclusive with source.
as match. Mutually exclusive with label and source.
type: string
type: object
timestamp:
Expand Down

0 comments on commit f42768d

Please sign in to comment.