Adds: Active time interval (#2779)

* add active time interval Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix active time interval Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix unittests for active time interval Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update notify/notify.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update dispatch/route.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * split the stage for active and mute intervals Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update notify/notify.go Adds doc for a helper function Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update notify/notify.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update notify/notify.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update notify/notify.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix code after commit suggestions Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Making mute_time_interval and time_intervals can coexist in the config Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * docs: configuration's doc has been updated about time intervals Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update config/config.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * updates configuration readme to improve active time description Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * merge deprecated mute_time_intervals and time_intervals Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update cmd/alertmanager/main.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update cmd/alertmanager/main.go Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fmt main.go Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix lint error Signed-off-by: clyang82 <chuyang@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Document that matchers are ANDed together Signed-off-by: Mac Chaffee <me@macchaffee.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Remove extra parentheticals Signed-off-by: Mac Chaffee <me@macchaffee.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * config: root route should have empty matchers Unmarshal should validate that the root route does not contain any matchers. Prior to this change, only the deprecated match structures were checked. Signed-off-by: Philip Gough <philip.p.gough@gmail.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * chore: Let git ignore temporary files for ui/app Signed-off-by: nekketsuuu <nekketsuuu@users.noreply.github.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * adding max_alerts parameter to slack webhook config correcting the logic to trucate fields instead of dropping alerts in the slack integration Signed-off-by: Prashant Balachandran <pnair@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * *: bump to Go 1.17 (#2792) * *: bump to Go 1.17 Signed-off-by: Simon Pasquier <spasquie@redhat.com> * *: fix yamllint errors Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Automate CSS-inlining for default HTML email template (#2798) * Automate CSS-inlining for default HTML email template The original HTML email template was added in `template/email.html`. It looks like the CSS was manually inlined. Most likely using the premailer.dialect.ca web form, which is mentioned in the README for the Mailgun transactional-email-templates project. The resulting HTML with inlined CSS was then copied into `template/default.tmpl`. This has resulted in `email.html` and `default.tmpl` diverging at times. This commit adds build automation to inline the CSS automatically using [juice][1]. The Go template containing the resulting HTML has been moved into its own file to avoid the script that performs the CSS inlining having to parse the `default.tmpl` file to insert it there. Fixes #1939. [1]: https://www.npmjs.com/package/juice Signed-off-by: Brad Ison <bison@xvdf.io> * Update asset/assets_vfsdata.go Signed-off-by: Brad Ison <bison@xvdf.io> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * go.{mod,sum}: update Go dependencies Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * amtool to support http_config to access alertmanager (#2764) * Support http_config for amtool Co-authored-by: Julien Pivotto <roidelapluie@gmail.com> Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: clyang82 <chuyang@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * notify/sns: detect FIFO topic based on the rendered value Since the TopicARN field is a template string, it's safer to check for the ".fifo" suffix in the rendered string. Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * config: delegate Sigv4 validation to the inner type This change also adds unit tests for SNS configuration. Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix unittests Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix comment about active time interval Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * fix another comment about active time interval Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Fix typo in documentation Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> * Update docs/configuration.md Co-authored-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Sinuhe Tellez <dubyte@gmail.com> Co-authored-by: Simon Pasquier <spasquie@redhat.com> Co-authored-by: clyang82 <chuyang@redhat.com> Co-authored-by: Mac Chaffee <me@macchaffee.com> Co-authored-by: Philip Gough <philip.p.gough@gmail.com> Co-authored-by: nekketsuuu <nekketsuuu@users.noreply.github.com> Co-authored-by: Prashant Balachandran <pnair@redhat.com> Co-authored-by: Simon Pasquier <pasquier.simon@gmail.com> Co-authored-by: Brad Ison <brad.ison@redhat.com> Co-authored-by: Julien Pivotto <roidelapluie@gmail.com>
prometheus · Mar 4, 2022 · d155153 · d155153
1 parent 3f42c5e
commit d155153
Show file tree

Hide file tree

Showing 8 changed files with 330 additions and 47 deletions.
diff --git a/cmd/alertmanager/main.go b/cmd/alertmanager/main.go
@@ -440,10 +440,14 @@ func run() int {
 			integrationsNum += len(integrations)
 		}
 
-		// Build the map of time interval names to mute time definitions.
-		muteTimes := make(map[string][]timeinterval.TimeInterval, len(conf.MuteTimeIntervals))
+		// Build the map of time interval names to time interval definitions.
+		timeIntervals := make(map[string][]timeinterval.TimeInterval, len(conf.MuteTimeIntervals)+len(conf.TimeIntervals))
 		for _, ti := range conf.MuteTimeIntervals {
-			muteTimes[ti.Name] = ti.TimeIntervals
+			timeIntervals[ti.Name] = ti.TimeIntervals
+		}
+
+		for _, ti := range conf.TimeIntervals {
+			timeIntervals[ti.Name] = ti.TimeIntervals
 		}
 
 		inhibitor.Stop()
@@ -465,7 +469,7 @@ func run() int {
 			waitFunc,
 			inhibitor,
 			silencer,
-			muteTimes,
+			timeIntervals,
 			notificationLog,
 			pipelinePeer,
 		)

diff --git a/config/config.go b/config/config.go
@@ -269,14 +269,34 @@ func (mt *MuteTimeInterval) UnmarshalYAML(unmarshal func(interface{}) error) err
 	return nil
 }
 
+// TimeInterval represents a named set of time intervals for which a route should be muted.
+type TimeInterval struct {
+	Name          string                      `yaml:"name" json:"name"`
+	TimeIntervals []timeinterval.TimeInterval `yaml:"time_intervals" json:"time_intervals"`
+}
+
+// UnmarshalYAML implements the yaml.Unmarshaler interface for MuteTimeInterval.
+func (ti *TimeInterval) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	type plain TimeInterval
+	if err := unmarshal((*plain)(ti)); err != nil {
+		return err
+	}
+	if ti.Name == "" {
+		return fmt.Errorf("missing name in time interval")
+	}
+	return nil
+}
+
 // Config is the top-level configuration for Alertmanager's config files.
 type Config struct {
-	Global            *GlobalConfig      `yaml:"global,omitempty" json:"global,omitempty"`
-	Route             *Route             `yaml:"route,omitempty" json:"route,omitempty"`
-	InhibitRules      []*InhibitRule     `yaml:"inhibit_rules,omitempty" json:"inhibit_rules,omitempty"`
-	Receivers         []*Receiver        `yaml:"receivers,omitempty" json:"receivers,omitempty"`
-	Templates         []string           `yaml:"templates" json:"templates"`
+	Global       *GlobalConfig  `yaml:"global,omitempty" json:"global,omitempty"`
+	Route        *Route         `yaml:"route,omitempty" json:"route,omitempty"`
+	InhibitRules []*InhibitRule `yaml:"inhibit_rules,omitempty" json:"inhibit_rules,omitempty"`
+	Receivers    []*Receiver    `yaml:"receivers,omitempty" json:"receivers,omitempty"`
+	Templates    []string       `yaml:"templates" json:"templates"`
+	// Deprecated. Remove before v1.0 release.
 	MuteTimeIntervals []MuteTimeInterval `yaml:"mute_time_intervals,omitempty" json:"mute_time_intervals,omitempty"`
+	TimeIntervals     []TimeInterval     `yaml:"time_intervals,omitempty" json:"time_intervals,omitempty"`
 
 	// original is the input from which the config was parsed.
 	original string
@@ -491,18 +511,32 @@ func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error {
 		return fmt.Errorf("root route must not have any mute time intervals")
 	}
 
+	if len(c.Route.ActiveTimeIntervals) > 0 {
+		return fmt.Errorf("root route must not have any active time intervals")
+	}
+
 	// Validate that all receivers used in the routing tree are defined.
 	if err := checkReceiver(c.Route, names); err != nil {
 		return err
 	}
 
 	tiNames := make(map[string]struct{})
+
+	// read mute time intervals until deprecated
 	for _, mt := range c.MuteTimeIntervals {
 		if _, ok := tiNames[mt.Name]; ok {
 			return fmt.Errorf("mute time interval %q is not unique", mt.Name)
 		}
 		tiNames[mt.Name] = struct{}{}
 	}
+
+	for _, mt := range c.TimeIntervals {
+		if _, ok := tiNames[mt.Name]; ok {
+			return fmt.Errorf("time interval %q is not unique", mt.Name)
+		}
+		tiNames[mt.Name] = struct{}{}
+	}
+
 	return checkTimeInterval(c.Route, tiNames)
 }
 
@@ -529,12 +563,16 @@ func checkTimeInterval(r *Route, timeIntervals map[string]struct{}) error {
 			return err
 		}
 	}
-	if len(r.MuteTimeIntervals) == 0 {
-		return nil
+
+	for _, ti := range r.ActiveTimeIntervals {
+		if _, ok := timeIntervals[ti]; !ok {
+			return fmt.Errorf("undefined time interval %q used in route", ti)
+		}
 	}
-	for _, mt := range r.MuteTimeIntervals {
-		if _, ok := timeIntervals[mt]; !ok {
-			return fmt.Errorf("undefined time interval %q used in route", mt)
+
+	for _, tm := range r.MuteTimeIntervals {
+		if _, ok := timeIntervals[tm]; !ok {
+			return fmt.Errorf("undefined time interval %q used in route", tm)
 		}
 	}
 	return nil
@@ -694,11 +732,12 @@ type Route struct {
 	// Deprecated. Remove before v1.0 release.
 	Match map[string]string `yaml:"match,omitempty" json:"match,omitempty"`
 	// Deprecated. Remove before v1.0 release.
-	MatchRE           MatchRegexps `yaml:"match_re,omitempty" json:"match_re,omitempty"`
-	Matchers          Matchers     `yaml:"matchers,omitempty" json:"matchers,omitempty"`
-	MuteTimeIntervals []string     `yaml:"mute_time_intervals,omitempty" json:"mute_time_intervals,omitempty"`
-	Continue          bool         `yaml:"continue" json:"continue,omitempty"`
-	Routes            []*Route     `yaml:"routes,omitempty" json:"routes,omitempty"`
+	MatchRE             MatchRegexps `yaml:"match_re,omitempty" json:"match_re,omitempty"`
+	Matchers            Matchers     `yaml:"matchers,omitempty" json:"matchers,omitempty"`
+	MuteTimeIntervals   []string     `yaml:"mute_time_intervals,omitempty" json:"mute_time_intervals,omitempty"`
+	ActiveTimeIntervals []string     `yaml:"active_time_intervals,omitempty" json:"active_time_intervals,omitempty"`
+	Continue            bool         `yaml:"continue" json:"continue,omitempty"`
+	Routes              []*Route     `yaml:"routes,omitempty" json:"routes,omitempty"`
 
 	GroupWait      *model.Duration `yaml:"group_wait,omitempty" json:"group_wait,omitempty"`
 	GroupInterval  *model.Duration `yaml:"group_interval,omitempty" json:"group_interval,omitempty"`

diff --git a/config/config_test.go b/config/config_test.go
@@ -177,9 +177,35 @@ receivers:
 
 }
 
-func TestMuteTimeHasName(t *testing.T) {
+func TestActiveTimeExists(t *testing.T) {
 	in := `
-mute_time_intervals:
+route:
+    receiver: team-Y
+    routes:
+    -  match:
+        severity: critical
+       active_time_intervals:
+       - business_hours
+
+receivers:
+- name: 'team-Y'
+`
+	_, err := Load(in)
+
+	expected := "undefined time interval \"business_hours\" used in route"
+
+	if err == nil {
+		t.Fatalf("no error returned, expected:\n%q", expected)
+	}
+	if err.Error() != expected {
+		t.Errorf("\nexpected:\n%q\ngot:\n%q", expected, err.Error())
+	}
+
+}
+
+func TestTimeIntervalHasName(t *testing.T) {
+	in := `
+time_intervals:
 - name: 
   time_intervals:
   - times:
@@ -199,7 +225,7 @@ route:
 `
 	_, err := Load(in)
 
-	expected := "missing name in mute time interval"
+	expected := "missing name in time interval"
 
 	if err == nil {
 		t.Fatalf("no error returned, expected:\n%q", expected)
@@ -358,6 +384,35 @@ route:
 
 }
 
+func TestRootRouteNoActiveTimes(t *testing.T) {
+	in := `
+time_intervals:
+- name: my_active_time
+  time_intervals:
+  - times:
+     - start_time: '09:00'
+       end_time: '17:00'
+
+receivers:
+- name: 'team-X-mails'
+
+route:
+  receiver: 'team-X-mails'
+  active_time_intervals:
+  - my_active_time
+`
+	_, err := Load(in)
+
+	expected := "root route must not have any active time intervals"
+
+	if err == nil {
+		t.Fatalf("no error returned, expected:\n%q", expected)
+	}
+	if err.Error() != expected {
+		t.Errorf("\nexpected:\n%q\ngot:\n%q", expected, err.Error())
+	}
+}
+
 func TestRootRouteHasNoMatcher(t *testing.T) {
 	testCases := []struct {
 		name string

diff --git a/dispatch/dispatch.go b/dispatch/dispatch.go
@@ -447,6 +447,7 @@ func (ag *aggrGroup) run(nf notifyFunc) {
 			ctx = notify.WithReceiverName(ctx, ag.opts.Receiver)
 			ctx = notify.WithRepeatInterval(ctx, ag.opts.RepeatInterval)
 			ctx = notify.WithMuteTimeIntervals(ctx, ag.opts.MuteTimeIntervals)
+			ctx = notify.WithActiveTimeIntervals(ctx, ag.opts.ActiveTimeIntervals)
 
 			// Wait the configured interval before calling flush again.
 			ag.mtx.Lock()

diff --git a/dispatch/route.go b/dispatch/route.go
@@ -118,6 +118,7 @@ func NewRoute(cr *config.Route, parent *Route) *Route {
 	sort.Sort(matchers)
 
 	opts.MuteTimeIntervals = cr.MuteTimeIntervals
+	opts.ActiveTimeIntervals = cr.ActiveTimeIntervals
 
 	route := &Route{
 		parent:    parent,
@@ -210,6 +211,9 @@ type RouteOpts struct {
 
 	// A list of time intervals for which the route is muted.
 	MuteTimeIntervals []string
+
+	// A list of time intervals for which the route is active.
+	ActiveTimeIntervals []string
 }
 
 func (ro *RouteOpts) String() string {

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -116,9 +116,14 @@ receivers:
 inhibit_rules:
   [ - <inhibit_rule> ... ]
 
+# DEPRECATED: use time_intervals below.
 # A list of mute time intervals for muting routes.
 mute_time_intervals:
   [ - <mute_time_interval> ... ]
+
+# A list of time intervals for muting/activating routes.
+time_intervals:
+  [ - <time_interval> ... ]
 ```
 
 ## `<route>`
@@ -189,6 +194,16 @@ matchers:
 mute_time_intervals:
   [ - <string> ...]
 
+# Times when the route should be active. These must match the name of a
+# time interval defined in the time_intervals section. An empty value
+# means that the route is always active.
+# Additionally, the root node cannot have any active times.
+# The route will send notifications only when active, but otherwise
+# acts normally (including ending the route-matching process
+# if the `continue` option is not set).
+active_time_intervals:
+  [ - <string> ...]
+
 # Zero or more child routes.
 routes:
   [ - <route> ... ]
@@ -221,12 +236,32 @@ route:
     group_by: [product, environment]
     matchers:
     - team="frontend"
+
+  # All alerts with the service=inhouse-service label match this sub-route.
+  # the route will be muted during offhours and holidays time intervals.
+  # even if it matches, it will continue to the next sub-route
+  - receiver: 'dev-pager'
+    matchers:
+      - service="inhouse-service"
+    mute_time_intervals:
+      - offhours
+      - holidays
+    continue: true
+
+    # All alerts with the service=inhouse-service label match this sub-route
+    # the route will be active only during offhours and holidays time intervals.
+  - receiver: 'on-call-pager'
+    matchers:
+      - service="inhouse-service"
+    active_time_intervals:
+      - offhours
+      - holidays
 ```
 
-## `<mute_time_interval>`
+## `<time_interval>`
 
-A `mute_time_interval` specifies a named interval of time that may be referenced
-in the routing tree to mute particular routes for particular times of the day.
+A `time_interval` specifies a named interval of time that may be referenced
+in the routing tree to mute/activate particular routes for particular times of the day.
 
 ```yaml
 name: <string>