> ## Documentation Index
> Fetch the complete documentation index at: https://test-8ad8522e-feat-ai-sre.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Noise Reduction
> Reduce alert noise through grouping, silencing, inhibition, and other strategies to improve processing efficiency
**Plan requirement**: Alert aggregation, storm warning, and inhibit rules require an On-call Standard or higher subscription. Silence rules are available on all plans. [Learn more](https://flashcat.cloud/flashduty/price/)
Noise reduction is one of the core capabilities of Flashduty On-call. When an alert storm hits, you might receive hundreds of similar notifications. The noise reduction feature groups these alerts into a single incident, so you only need to handle it once instead of being overwhelmed by repeated notifications.
## Why Noise Reduction
| Scenario | Without Noise Reduction | With Noise Reduction |
| :------------------------------------------------------ | :---------------------------------------------- | :--------------------------------------- |
| Server outage triggers 100 alerts | Receive 100 notifications, must handle each one | Receive 1 incident, handle uniformly |
| Network flapping causes repeated alert trigger/recovery | Notification bombardment, exhausting to respond | Marked as flapping, reduced interference |
| Batch alerts at midnight | Woken up multiple times by calls/SMS | Notified only once, sleep unaffected |
**Core value of noise reduction**:
* Reduce notification frequency, avoid alert fatigue
* Focus on issues that truly need attention
* Improve incident response and handling efficiency
## Core Concepts
Before understanding noise reduction, you need to understand the relationship between three core objects:
```
Monitoring System → Event → Alert → Incident
```
| Object | Definition | Source |
| :----------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------- |
| **Event** | Raw notification from monitoring system, each trigger or recovery is an event | Zabbix, Prometheus, etc. |
| **Alert** | Automatically triggered by events. Events sharing the same `alert_key` merge into one alert within the channel's **Event Aggregation** window | Automatically created by Flashduty |
| **Incident** | Primary object processed by Flashduty, triggered by alerts or created manually | Auto-triggered or manually created |
**Key understanding**:
* One alert can contain multiple events (events with the same `alert_key` that arrive within the Event Aggregation window, plus the corresponding recovery events)
* One incident can contain multiple alerts (similar alerts grouped together)
* Noise reduction happens at two stages: **Event Aggregation** controls "Event → Alert"; **Alert Grouping** controls "Alert → Incident"
## Noise Reduction Process
When a monitoring system pushes alerts to Flashduty On-call, the system automatically executes the following workflow:
Determine whether to merge into an existing alert, otherwise create a new alert.
Determine whether to merge into an existing incident, otherwise create a new incident.
New incidents notify relevant personnel according to escalation rules.
Subsequent alerts merge into existing incidents without repeated notifications.
## Event Aggregation
Go to Channel Details → **Noise Reduction** → **Event Aggregation** to configure.
Event Aggregation controls the "Event → Alert" merge behavior: when the upstream monitoring system keeps pushing events that share the same `alert_key`, whether those events are merged into the same existing alert or each one creates its own independent alert.
New channels have Event Aggregation enabled by default, with a window of **24 hours** (1440 minutes).
### Configuration
| Configuration | Description | Default | Range |
| :--------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------- | :------------------------------------------------------------------------------------------------------------- |
| **Enable Event Aggregation** | When enabled, events with the same `alert_key` merge into the same alert within the aggregation window; when disabled, every event creates its own alert | Enabled | Enabled / Disabled |
| **Aggregation Window** | Starts counting from the alert's creation time; events arriving after this duration create a new alert. Only configurable when Event Aggregation is enabled | 1440 minutes (24 hours) | 1–1440 minutes; capped at 24 hours by default, extendable up to 7 days on request (contact the Flashduty team) |
`alert_key` is the identifier used for alert correlation and deduplication. It is either reported by the upstream integration or generated automatically by integration rules.
### Versus Alert Grouping
| Stage | Setting | Behavior it controls |
| :------------------- | :---------------- | :-------------------------------------------------------------------- |
| **Event → Alert** | Event Aggregation | Whether events sharing the same `alert_key` merge into a single alert |
| **Alert → Incident** | Alert Grouping | Whether similar alerts merge into a single incident |
## Alert Grouping
Go to Channel Details → **Noise Reduction** to configure.
Alert grouping merges multiple similar alerts into a single incident for unified assignment and notification. When an alert storm hits, you only need to handle one incident instead of hundreds of repeated notifications.
New channels have alert grouping disabled by default. When disabled, each alert creates an independent incident.
### Grouping Modes
Flashduty On-call provides two grouping modes:
| Mode | Use Case | Characteristics |
| :----------------------- | :---------------------------------------- | :------------------------------------------------------------------------------------------ |
| **Intelligent Grouping** | Quick start, lower precision requirements | Based on machine learning semantic similarity analysis, no manual rule configuration needed |
| **Rule-based Grouping** | Need precise control over grouping logic | Exact matching by specified dimensions (attributes, labels) |
### Common Configuration
| Configuration | Description |
| :---------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Aggregation Window** | Optional toggle. When disabled, new alerts continue merging into the incident until the incident is closed. When enabled, alerts within the window are merged; alerts arriving after the window expires are grouped into a new incident |
| **Window Timing Start** | Only configurable when the aggregation window is enabled. **Incident trigger** (default): Fixed timer starts from incident creation, stops grouping when the window duration is reached. **Alert merges into incident**: Timer resets each time a new alert merges in, the window recalculates from the last merge |
| **Window Duration** | Only configurable when the aggregation window is enabled. Set the duration of the aggregation window, minimum 1 minute. Rule-based grouping and intelligent grouping share the same cap: 24 hours by default, extendable to 30 days on request (contact the Flashduty team to enable) |
| **Alert Storm Warning** | When merged alert count reaches a configured threshold, the system records an alert storm event in the incident timeline and triggers a warning notification, prompting urgent handling. You can configure up to 5 thresholds, each ranging from 2 to 10,000 |
| **Strict Grouping** | When enabled, empty label values are treated as different; when disabled, empty values are treated as the same (not supported for intelligent grouping) |
When new alerts are highly similar to active incidents, automatically merge into the incident. The system uses machine learning to calculate semantic similarity between alerts, requiring no manual rule definition.
Select **Intelligent Grouping** mode
Select the fields used for similarity calculation. Default fields include: title, description, labels.service, labels.resource. You can add or remove fields based on your needs, up to a maximum of **10** fields. Click **Reset** to restore default settings.
The system calculates alert similarity based on the selected fields' content. When similarity reaches the threshold, new alerts automatically merge into existing incidents.
Exact matching by specified dimensions, only groups when all dimension values are identical.
**Unified Control**: All alerts in the channel use the same dimensions for grouping.
**Fine-grained Control**: Match different grouping dimensions by conditions, applying different grouping strategies for different types of alerts. You can add multiple condition branches, each containing match conditions and corresponding grouping dimensions:
* The system matches conditions from top to bottom, using the grouping dimensions of the first matched branch
* Supports reordering condition branch priority by moving them up or down
* Set **Default Dimensions** at the bottom as a fallback; when alerts don't match any condition, they are grouped by the default dimensions
### Configuration Limits
To protect grouping performance and stability, the following fields have hard backend caps:
| Grouping mode | Limit | Cap | Description |
| :----------------------- | :----------------------------------------------- | :------------ | :----------------------------------------------------------------------------- |
| **Rule-based grouping** | Grouping dimensions (equals) | ≤ 5 per group | Number of dimensions per rule in both unified control and fine-grained control |
| **Rule-based grouping** | Fine-grained branches (cases) | ≤ 100 | Total condition branches you can configure under fine-grained control |
| **Intelligent grouping** | Fields used for similarity calculation (i\_keys) | ≤ 10 | Default 4: `title`, `description`, `labels.service`, `labels.resource` |
### Grouping Effect
After setting grouping by **Alert Check Item**, 5 alert notifications are grouped into 1 incident:
```
Incident: cpu idle < 20% / es.nj.03, Critical
- Alert cpu idle < 20% / es.nj.03:
- Event1: es.nj.03, cpu.idle = 10%, Critical
- Event2: es.nj.03, cpu.idle = 18%, Warning
- Event4: es.nj.03, cpu.idle = 10%, Ok
- Alert cpu idle < 20% / es.nj.01:
- Event3: es.nj.01, cpu.idle = 15%, Warning
- Alert cpu idle < 20% / es.nj.02:
- Event5: es.nj.02, cpu.idle = 19%, Warning
```
View grouping relationships on the incident details page:
* Click alert title to view alert timeline and associated events
* Click event point to view specific event content
## Flapping Detection
When the same incident triggers and recovers frequently, the system marks it as "flapping" status to avoid notification bombardment.
Go to Channel Details → Noise Reduction → **Flapping Detection**:
| Option | Behavior |
| :--------------------- | :------------------------------------------------------------ |
| **Off** | Don't detect flapping status |
| **Alert Only** | Mark flapping status, continue notifications per policy |
| **Alert Then Silence** | Mark flapping status, no more notifications after first alert |
Flapping detection is enabled by default for new channels in **Alert Then Silence** mode, with default parameters: 60-minute observation window, 4 state changes, and a 120-minute mute duration.
### Configurable Parameters
| Parameter | Description | Default | Range |
| :-------------------------------- | :---------------------------------------------------------------------------------------------------- | :---------- | :-------------- |
| **State changes** (max\_changes) | Number of alert state changes within the observation window to trigger flapping detection | 4 | 2–100 |
| **Observation window** (in\_mins) | Time window for counting state changes | 60 minutes | 1–1440 minutes |
| **Mute duration** (mute\_mins) | Duration to mute notifications after flapping is detected (only applies in "Alert Then Silence" mode) | 120 minutes | 30–1440 minutes |
"Same incident" refers to incidents with the same Alert Key, typically using the alert ID pushed from the upstream system as a unique identifier.
## Silence Rules
During maintenance windows or known issue periods, silence rules can suppress alert notifications for specific conditions.
Go to Channel Details → Noise Reduction → **Silence Rules**.
### Silence Time
| Type | Description |
| :------------------------------------ | :---------------------------------------------------------------------------------------------- |
| **One-time Silence** | Active during specified time period, rule retained but inactive after expiration |
| **Recurring Silence - Weekly Mode** | Active at fixed weekly time periods, e.g., every Saturday 00:00-06:00 |
| **Recurring Silence - Calendar Mode** | Active on workdays/rest days per [Service Calendar](/en/on-call/configuration/service-calendar) |
**Time input for One-time Silence**: a duration input on the left (default `1d`) plus an absolute time-range picker on the right. The two controls stay in sync:
* Duration accepts shorthand such as `30m`, `1h`, `12h`, `1d`, `1w`, `2w`. Changing the duration recomputes the end time from the current moment.
* You can also pick the start and end directly on the right; the duration input updates accordingly.
* Start time must be earlier than end time, and neither field may be empty.
**Auto-delete on expiration** (`is_auto_delete`): one-time silence rules can opt into this switch. When enabled, the rule is automatically removed by the system 24 hours after its end time, matching the cleanup behavior of [Quick Silence](#quick-silence). When disabled (the default), the rule stays in the list after expiration but is no longer active — you can keep it for reuse or delete it manually.
### Silence Conditions
Define which alerts should be silenced, supports multiple condition combinations.
| Match Item | Description | Example |
| :-------------- | :--------------------------------- | :---------------------------------------------- |
| **Severity** | Match by alert level | Only silence `Info` level |
| **Title** | Match by alert title keywords | Title contains "Planned Maintenance" |
| **Description** | Match by alert description content | Description contains "restart" |
| **Integration** | Match by alert integration source | Only silence alerts from a specific integration |
| **Labels** | Match by label key-value pairs | `host=db-master-01` |
**Combination Logic**:
* **AND**: All conditions must be met to silence
* **OR**: Any condition met triggers silence
See [Configure Filter Conditions](/en/on-call/configuration/filter-conditions) for details.
### Silence Behavior
| Behavior | Description |
| :------------------ | :------------------------------------------------------------------------------------ |
| **Drop Directly** | Alert doesn't appear in any list, no record |
| **Retain and Mark** | Alert appears in **Raw Alerts List** marked as "Silenced", can be filtered and viewed |
The next time you create a silence rule, your most recently chosen behavior is used as the default.
### Quick Silence
Quickly create temporary silence rules based on existing incidents.
**Operation Path**: Incident Details → More Actions → **Quick Silence**
* Rule name defaults to "Quick Silence - #short-ID", with the incident title included in the description
* Effective scope is the incident's channel (cannot be changed)
* **One-time silence** is selected by default and lasts 1 day. You can type a custom duration in the left input box (formats such as `30m`, `1h`, `12h`, `1d`, `1w`, `2w` are supported) or pick an absolute time range on the right. A one-time rule is automatically deleted after expiration
* You can instead select **recurring silence** and configure the silenced time windows with the standard recurring schedule
* Conditions default to severity and filtered label matching. Labels are automatically excluded in two ways: **system format rules** (label values that are JSON, numeric, datetime-formatted, or overly long) and **account exclusion rules** (globally excluded labels and per-integration excluded labels — see [Quick-silence label exclusion rules](/en/platform/organization-info#quick-silence-label-exclusion-rules))
When creating a quick silence, the drawer shows "N labels identified, M labels automatically excluded" and lists each excluded label with its reason: system format rules are marked as **JSON / Numeric / Too long / Datetime**, account rules as **Globally excluded** or the owning integration name. From the drawer you can click **Manage exclusion rules** or **View exclusion rules** to open the account configuration (only the main account or Account.Admin can modify it; other members can view it read-only). The matching conditions are generated automatically from the source incident's labels — conditions that are too precise may cause similar alerts to escape silencing, so review the list carefully before creating.
When repeatedly using quick silence on the same incident, it edits the original rule rather than creating a new one.
### Copy a Silence Rule
Every rule card in the silence rule list (except rules created by **Quick Silence**) shows a **Copy** icon on hover. Clicking it opens the copy dialog, where you can set the new rule's name and description and choose the target channel:
| Configuration | Description |
| :-------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Rule name** | Required; pre-filled with "Copy " plus the original name (e.g. `Copy Database Maintenance`), **39 characters max** — overlong input is truncated in the field |
| **Description** | Pre-filled with the original description, **500 characters max** — overlong input is truncated in the field |
| **Channel** | Choose **This channel** (the current one) or **Another channel**. Copying to another channel requires picking the target from a dropdown that excludes the source rule's channel; non-admin users only see channels they manage |
The result is **a new, independent rule**: it reuses the source rule's silence conditions, silence behavior (Drop directly / Retain and mark), auto-delete-on-expiration setting, and schedule (one-time or recurring), but it does not carry over identity fields such as `rule_id` or `from_incident_id`, nor the quick-silence link to the source incident.
* **Copying to the current channel**: on success, the new rule appears in the list immediately and a "Copied" toast is shown.
* **Copying to another channel**: on success, a **Copy succeeded** dialog appears showing the target channel, rule name, description, and creator. Click **Go to view** to open the target channel and jump straight to the new rule's detail page.
**Quick silence** rules cannot be copied: a quick-silence rule is bound to its source incident (repeating the action on the same incident edits the original rule), so its card has no **Copy** action.
## Inhibit Rules
When a root cause alert exists, automatically inhibit related secondary alerts. For example: When a Critical level incident exists, inhibit Warning/Info level incidents for the same check item.
### Configuration Path
| Location | Path | Characteristics |
| :-------------------- | :------------------------------------------------------------------ | :------------------------------------------- |
| **Channel** | Channel Details → Noise Reduction → **Inhibit Rules** | Only effective for alerts in current channel |
| **Alert Integration** | Alert Integration Details → Alert Processing → **Alert Inhibition** | Effective for alerts from this integration |
### Inhibit Conditions
When a new alert meets the conditions and there is a matching **active alert** (any alert in an active, unrecovered state can serve as the inhibit source, whether or not it has been acknowledged) within the last 10 minutes, and both share equal items, the new alert is inhibited.
| Configuration | Description |
| :-------------------------- | :----------------------------------------------------------------------------------- |
| **New Alert Conditions** | Conditions the inhibited alert must meet, e.g., severity is Warning/Info |
| **Active Alert Conditions** | Conditions the inhibiting source alert must meet, e.g., severity is Critical |
| **Equal Items** | Attributes or labels that must be identical between both, e.g., check item, hostname |
### Inhibit Behavior
| Behavior | Description |
| :------------------ | :-------------------------------------------------------------------------------- |
| **Drop Directly** | Alert doesn't appear in any list, no record |
| **Retain and Mark** | Alert appears in **Alert List** marked as "Inhibited", can be filtered and viewed |
### Configuration Example
**Scenario**: When Critical level alerts exist, inhibit Warning/Info level alerts for the same check item.
## FAQ
No. The incident title matches the first alert that triggered it and can be manually modified at any time; it won't change with new alerts.
* **Manually created incidents**: No, labels list always remains empty
* **Auto-triggered incidents**: Possibly, incident labels stay consistent with the first alert; if that alert's labels change, incident labels update accordingly
Yes. Alert labels always stay consistent with the latest merged event. However, if the new event is a recovery event, the alert keeps existing labels and only adds labels that didn't exist before.
Up to 5000, mainly to ensure console rendering performance. Due to backend concurrent processing, actual count may slightly exceed this limit.
Whether an event can merge into an existing alert is controlled by the channel's **Event Aggregation** setting (which governs the "Event → Alert" stage — enabled by default with a 24-hour window, configurable from 1 to 1440 minutes, extendable up to 7 days on request, or can be turned off entirely):
* **Event Aggregation enabled**: events with the same `alert_key` merge into the same alert within the window; events arriving after the window expires create a new alert
* **Event Aggregation disabled**: every event creates an independent alert, with no merging
Note that **Alert Grouping** controls the merge window at the "Alert → Incident" stage (24 hours by default, extendable to 30 days). It is a separate concern from whether events merge into an alert — don't conflate the two.
## Related Topics
Define alert notification and escalation rules
Learn about condition matching syntax