Merge pull request #591 from flashcatcloud/sync/alert-batch5-e2e-docs-test

docs: sync OpenStatus/Dead Man's Snitch/Axiom/Komodor/HyperDX/OpenNMS/Pandora FMS alert-integration fixes

Author: ysyneu

en/on-call/integration/alert-integration/alert-sources/axiom.mdx

@@ -35,9 +35,13 @@ You can get the integration push URL in either of the following ways.
 <Steps>
 <Step title="Create a custom webhook notifier">
 
-1. In Axiom, open the **Monitors** tab and click **Manage notifiers** on the right
+<Note>
+The **Custom Webhook** notifier is not available on Axiom's free Personal plan: it is greyed out in the notifier list. Use a paid Axiom plan.
+</Note>
+
+1. In Axiom, open the **Monitors** tab, click **New monitor** (or open an existing monitor), and click **Manage notifiers** in the top right
 2. Click **New notifier** in the top right and name it `Flashduty`
-3. Select **Custom webhook** and paste the full Flashduty push URL into **Webhook URL**
+3. Select **Custom Webhook** and paste the full Flashduty push URL into **Webhook URL**
 4. Keep Axiom's default body template unchanged. If you edited it before, replace it with the template below
 5. No headers are needed. Click **Create**
 

en/on-call/integration/alert-integration/alert-sources/dead-mans-snitch.mdx

@@ -38,16 +38,17 @@ You can get the integration push URL in either of the following ways.
 <Step title="Add a webhook integration">
 
 1. Sign in to Dead Man's Snitch and click **Integrations** in the top navigation bar
-2. In the list of available integrations, click **Add Webhook**
+2. In the **Available Integrations** list, click **ADD** next to **Webhooks**
 3. Paste the full Flashduty push URL (including `integration_key`) into the **Hook URL** field and click **Save**
+4. Click **SEND TEST**. Dead Man's Snitch sends a sample `snitch.reporting` notification; Flashduty returns success and creates no alert, and Dead Man's Snitch shows **Webhook is working correctly.**
 
 </Step>
 
 <Step title="Filter by tag (optional)">
 
 By default, an integration receives notifications for every snitch in the current case. To send only some snitches to Flashduty:
 
-1. On the **Integrations** page, click the filter icon next to the webhook
+1. On the **Integrations** page, click the **Manage Tags** icon next to the webhook
 2. Add one or more tags and click **Save**
 
 After that, only snitches that carry at least one of these tags are sent to Flashduty. Remove all tags to send every snitch again.
@@ -112,7 +113,7 @@ Dead Man's Snitch webhooks carry no severity. Flashduty maps them as follows:
 <AccordionGroup>
 <Accordion title="Does a snitch that stays missing send repeated notifications?">
 
-Yes. Until the snitch is paused or checks in again, Dead Man's Snitch sends one missing notification for every failed interval. These notifications have the same Alert Key as the original alert and are merged into that active alert instead of creating new ones.
+The Dead Man's Snitch FAQ says email alerts repeat once per failed period until the snitch is paused or checks in again. With a webhook integration, a snitch on a 1-minute interval sent only one missing notification over several failed periods. Any repeated missing notification has the same Alert Key as the original alert and is merged into that active alert instead of creating a new one.
 
 </Accordion>
 

en/on-call/integration/alert-integration/alert-sources/hyperdx.mdx

@@ -37,7 +37,7 @@ Both open-source HyperDX and managed ClickStack in ClickHouse Cloud support Gene
 <Steps>
 <Step title="Create a Generic webhook">
 
-1. Open **Team Settings** and click **Add Webhook** under **Webhooks**. You can also select **Add New Webhook** while creating an alert
+1. Open **Team Settings** → **Integrations** and click **Add Webhook** under **Webhooks**. You can also click **Add New Incoming Webhook** while creating an alert
 2. Set **Service Type** to **Generic**
 3. Enter `Flashduty` as the **Webhook Name** and paste the full Flashduty push URL into **Webhook URL**
 4. Leave **Webhook Headers** empty. HyperDX sends `Content-Type: application/json` by default
@@ -73,8 +73,8 @@ Older HyperDX releases do not have the `{{alertId}}` and `{{groupKey}}` variable
 **Search alerts**:
 
 1. Run a query on the **Search** page and click **Alerts** in the top-right corner
-2. Save the search if it is not saved yet, set the threshold and time window, and fill in **grouped by** if needed
-3. Select the `Flashduty` webhook as the notification target and save the alert
+2. Set the threshold and time window, and fill in **grouped by** under **Advanced Settings** if needed
+3. Under **Send to**, select the `Flashduty` webhook, then click **Save Search with Alert**
 
 **Dashboard chart alerts**:
 
@@ -130,7 +130,7 @@ Flashduty uses `state` to decide between trigger and recovery, and `severity` to
 | :--- | :--- |
 | `event_id` | `{{eventId}}`, the Alert Key |
 | `alert_id` | `{{alertId}}`, the HyperDX alert ID |
-| `group` | `{{groupKey}}`, the group value of a grouped alert |
+| `group` | `{{groupKey}}`, the group of a grouped alert as `<column>:<value>`, for example `ServiceName:checkout` |
 | `severity` | Raw `severity` value from the template |
 | `link` | `{{link}}`, the link to the search or chart in HyperDX |
 

en/on-call/integration/alert-integration/alert-sources/komodor.mdx

@@ -4,7 +4,7 @@ description: "Send Kubernetes workload, node, PVC, and Job issues and their reco
 keywords: ["alert integration", "Komodor", "Kubernetes", "Realtime Health Monitors", "webhook"]
 ---
 
-Use the webhook notification channel of Komodor Realtime Health Monitors to send the issues Komodor detects in your Kubernetes clusters to Flashduty On-call. One kind of monitor issue on one resource in one namespace of one cluster maps to one Flashduty alert: an `open` message from Komodor triggers the alert, and a `close` message recovers it.
+Use the webhook notification channel of Komodor Realtime Health Monitors to send the issues Komodor detects in your Kubernetes clusters to Flashduty On-call. One kind of monitor issue on one resource in one namespace of one cluster maps to one Flashduty alert: an `open` message from Komodor triggers the alert, and a `closed` message recovers it.
 
 <div className="hide">
 
@@ -49,33 +49,37 @@ Choose this method when you need to route alerts to different channels based on
 ---
 
 - The Komodor Agent is installed in your Kubernetes cluster, and the cluster appears in the Komodor console.
-- Your account can edit Komodor **Realtime Health Monitors**.
+- Your account can edit Komodor **Health Policies**.
 
 ## In Komodor
 ---
 
 <Steps>
-<Step title="Open a monitor rule">
+<Step title="Open a monitor">
 
-Sign in to the Komodor console, go to the **Realtime Health Monitors** page, select the cluster, and open a monitor rule (for example, the Availability monitor), or click **Add rule** to create one.
+Sign in to the Komodor console and open **Organization Settings** → **Health Policies** → **Realtime Monitors**. Expand the cluster and a monitor type (for example **Availability monitor**), then click a monitor (for example **Default Runtime Availability**) to open **Edit Monitor**, or click **+ Add Monitor** to create one.
 
 </Step>
 
 <Step title="Add a webhook notification channel">
 
-1. In the rule's **Edit Rule** section, select **Webhook** as the notification method
+1. Under **Where do you want to receive notifications?**, select **Webhook**
 2. Click **Add New Webhook**:
-   - **Webhook Name**: a recognizable name, for example `Flashduty`
    - **Webhook URL**: the full Flashduty push URL, including the `integration_key` parameter
-3. Save the rule
+   - **Webhook Name**: a recognizable name, for example `Flashduty`
+   - Leave **Headers** empty
+3. Click **Test Webhook** (see the next step), then **Add Webhook**
+4. Select `Flashduty` under **Select webhooks** and click **Save Monitor**
 
-Once created, the webhook channel can be selected in any other monitor rule. Select it in every rule that should send alerts to Flashduty.
+Once created, the webhook can be selected in any other monitor. Select it in every monitor that should send alerts to Flashduty.
 
 </Step>
 
 <Step title="Confirm delivery">
 
-The Komodor documentation does not provide a test send for webhooks. The next time the rule detects an issue, check that the alert appears in the channel's alert list in Flashduty, and that it recovers once the issue is resolved.
+**Test Webhook** posts `{"type":"Test!"}` from your browser. Flashduty returns success and creates no alert, so the test only confirms that the push URL is reachable.
+
+The next time the monitor detects an issue, check that the alert appears in the channel's alert list in Flashduty, and that it recovers once the issue is resolved.
 
 </Step>
 </Steps>
@@ -87,7 +91,7 @@ Deploy monitor notifications describe a rollout (successful or failed), not a he
 ## Alert Key
 ---
 
-Flashduty builds the Alert Key from the `monitorType`, `cluster`, `namespace`, and `resourceName` fields. For one kind of issue on one resource, the `open` and `close` messages carry the same four values, so they merge into one alert and the `close` message recovers it.
+Flashduty builds the Alert Key from the `monitorType`, `cluster`, `namespace`, and `resourceName` fields. For one kind of issue on one resource, the `open` and `closed` messages carry the same four values, so they merge into one alert and the `closed` message recovers it.
 
 - Different resources, namespaces, or clusters, or different monitor types on the same resource (for example Availability and Job), produce different alerts.
 - Changes to the issue details (`issueDetails`), the issue link (`issueURL`), and the time fields do not change the Alert Key.
@@ -101,7 +105,7 @@ Komodor webhook notifications carry no severity field, so Flashduty uses Critica
 | Komodor `status` | Flashduty status | Flashduty severity |
 | :--- | :--- | :--- |
 | `open` | Triggered | Critical |
-| `close` | Recovered | Critical |
+| `closed` | Recovered | Critical |
 
 If `status` has any other value, or `monitorType`, `cluster`, or `resourceName` is missing, Flashduty returns an error and Komodor records a failed delivery.
 
@@ -115,8 +119,8 @@ If `status` has any other value, or `monitorType`, `cluster`, or `resourceName`
 | `service` | Komodor service identifier `serviceName` |
 | `resource` | Resource name `resourceName` |
 | `check` | Monitor type, for example `availability`, `node`, `pvc`, or `job` |
-| `status` | `open` or `close` |
-| `issue_details` | Issue details, for example `[Terminating ContainersNotReady]` |
+| `status` | `open` or `closed` |
+| `issue_details` | Issue details, for example `NonZeroExitCode - Exit code: 1`; several details are joined with `; ` |
 | `issue_url` | Link to the issue in Komodor |
 
 The alert title is `<monitor type> issue on <resource name> (<cluster>/<namespace>)`, and the description contains the issue details and the Komodor link.
@@ -133,15 +137,15 @@ No. Notification sinks deliver agent run and investigation events (such as `run.
 
 <Accordion title="What if an alert does not recover?">
 
-- Check that the monitor rule still selects the Flashduty webhook channel. After a rule is changed or deleted, Komodor no longer sends its `close` message.
+- Check that the monitor rule still selects the Flashduty webhook channel. After a rule is changed or deleted, Komodor no longer sends its `closed` message.
 - On the alert details page, compare the `check`, `cluster`, `namespace`, and `resource` labels. The recovery message must carry the same four values to recover the alert.
-- Changing the scope of an Availability monitor can remove existing issues, and Komodor may then send no `close` message. Close the alert in Flashduty by hand.
+- Changing the scope of an Availability monitor can remove existing issues, and Komodor may then send no `closed` message. Close the alert in Flashduty by hand.
 
 </Accordion>
 
 <Accordion title="Why does one workload have several alerts?">
 
-Different monitor types produce different alerts. For example, when a CronJob triggers both the Availability and the CronJob monitors, Flashduty creates two alerts, and each recovers with its own `close` message.
+Different monitor types produce different alerts. For example, when a CronJob triggers both the Availability and the CronJob monitors, Flashduty creates two alerts, and each recovers with its own `closed` message.
 
 </Accordion>
 </AccordionGroup>

en/on-call/integration/alert-integration/alert-sources/opennms.mdx

@@ -68,7 +68,13 @@ Notes:
 
 - Every `${name}` in the body needs an `<argument>` with the matching name; a missing one is replaced with an empty string. `notice_id` is the field Flashduty uses to identify the alert, so do not remove it
 - If the push URL contains `&` (for example, because you appended other query parameters), write it as `&amp;` in the XML
-- Changes to this file take effect immediately, without restarting OpenNMS
+
+After saving, reload Notifd; until then the `flashduty` command does not appear in the destination path wizard in the next step. On the OpenNMS server, run `$OPENNMS_HOME/bin/send-event.pl -p 'daemonName Notifd' uei.opennms.org/internal/reloadDaemonConfig`. The official container image has no Perl, so send the same event through the REST API instead (replace `<OpenNMS>` with the OpenNMS address and enter the admin password when prompted):
+
+```bash
+curl -u admin -X POST -H 'Content-Type: application/json' http://<OpenNMS>:8980/opennms/rest/events \
+  -d '{"uei": "uei.opennms.org/internal/reloadDaemonConfig", "source": "flashduty", "parms": [{"parmName": "daemonName", "value": "Notifd"}]}'
+```
 
 </Step>
 
@@ -81,7 +87,7 @@ OpenNMS runs the notification command once for each target user in a destination
 3. Name it `Flashduty`, click **Edit**, select only `flashduty` under **Send to Selected Users**, and click **Next Step**
 4. Select the `flashduty` command, set **Auto Notify** to **On**, click **Next Step**, and then click **Finish**
 
-**Auto Notify** must be **On**. With the default value, OpenNMS does not send the resolution notice for a notice that someone has already acknowledged by hand in OpenNMS, and the Flashduty alert does not close.
+**Auto Notify** must be **On**, which the wizard preselects. With **Off** or **Auto**, OpenNMS does not send the resolution notice for a notice that someone has already acknowledged by hand in OpenNMS, and the Flashduty alert does not close.
 
 </Step>
 
@@ -128,6 +134,15 @@ $OPENNMS_HOME/bin/send-event.pl -n <node ID> -i <IP address> uei.opennms.org/nod
 $OPENNMS_HOME/bin/send-event.pl -n <node ID> -i <IP address> uei.opennms.org/nodes/nodeUp
 ```
 
+The official container image has no Perl to run `send-event.pl`; send the same events through the REST API instead:
+
+```bash
+curl -u admin -X POST -H 'Content-Type: application/xml' http://<OpenNMS>:8980/opennms/rest/events \
+  -d '<event><uei>uei.opennms.org/nodes/nodeDown</uei><source>flashduty</source><nodeid><node ID></nodeid><interface><IP address></interface></event>'
+curl -u admin -X POST -H 'Content-Type: application/xml' http://<OpenNMS>:8980/opennms/rest/events \
+  -d '<event><uei>uei.opennms.org/nodes/nodeUp</uei><source>flashduty</source><nodeid><node ID></nodeid><interface><IP address></interface></event>'
+```
+
 OpenNMS has no button to test a notification command on its own.
 
 </Step>
@@ -165,7 +180,7 @@ To push other notices that have no auto-acknowledgement (for example, threshold
 <auto-acknowledge-alarm resolution-prefix="RESOLVED: " notify="true"/>
 ```
 
-When the event's alarm is cleared by its clearing event, OpenNMS acknowledges the notice and sends the resolution notice. After the change, run `$OPENNMS_HOME/bin/send-event.pl -p 'daemonName Notifd' uei.opennms.org/internal/reloadDaemonConfig` to reload. For notices that still get no resolution notice, turn on the [auto-resolve timeout](/en/on-call/channel/create-edit) in the channel that receives this integration's alerts, set **Window timing start** to **Incident trigger**, and set the timeout to 24 hours.
+When the event's alarm is cleared by its clearing event, OpenNMS acknowledges the notice and sends the resolution notice. After the change, reload Notifd as in the first step. For notices that still get no resolution notice, turn on the [auto-resolve timeout](/en/on-call/channel/create-edit) in the channel that receives this integration's alerts, set **Window timing start** to **Incident trigger**, and set the timeout to 24 hours.
 
 ## Severity
 ---
@@ -183,14 +198,14 @@ Severity comes from `severity` in the request, which is the OpenNMS event severi
 | `Cleared` | Info |
 | Empty or any other value | Critical |
 
-A resolution notice carries the severity of the original outage event, so the recovery event keeps the severity the alert was triggered with.
+In a resolution notice, `severity` is the numeric severity id OpenNMS stores (for example `5` for `Minor` and `6` for `Major`). Flashduty converts ids 1–7 back to severity names, so the recovery event keeps the severity the alert was triggered with.
 
 ## Alert content
 ---
 
 - **Title**: the notice subject `subject` without the `RESOLVED: ` prefix; `OpenNMS notice #<notice ID>` when the subject is empty
 - **Description**: the notice text `message` without the `RESOLVED: ` prefix
-- **Labels**: `notice_id`, `event_id`, `event_uei`, `node_id`, `interface`, `service`, `severity`. Labels with empty values are not added
+- **Labels**: `notice_id`, `event_id`, `event_uei`, `node_id`, `interface`, `service`, `severity` (the severity name; the numeric id in a resolution notice is converted to its name). Labels with empty values are not added
 
 ## Troubleshooting
 ---

en/on-call/integration/alert-integration/alert-sources/openstatus.mdx

@@ -35,24 +35,24 @@ You can get the integration push URL in either of the following ways.
 <Steps>
 <Step title="Create a Webhook notification channel">
 
-1. Sign in to the OpenStatus dashboard, open **Notifications** in the sidebar, and click **Create Notification Channel**
-2. Select **Webhook** as the channel type
-3. Name the channel `Flashduty`
+1. Sign in to the OpenStatus dashboard and open **Notifications** in the sidebar
+2. Under **Create a new notifier**, click **Webhook**
+3. Enter `Flashduty` as the **Name**
 4. Paste the full Flashduty push URL into **Webhook URL**
 5. Leave **Request Headers** empty. OpenStatus always sends `application/json`
-6. Under **Monitors**, select the monitors that should alert, then save
+6. Under **Monitors**, select the monitors that should alert, then click **Submit**. You can also attach the notifier later from a monitor's **Settings → Notifications**
 
 </Step>
 
 <Step title="Send a test">
 
-In the same form, click **Send Test**. The test message uses a fixed sample monitor (`monitor.id` `1`, name `test`, URL `http://openstat.us`). Flashduty returns success and creates no alert. When OpenStatus shows **Test sent**, the push URL works.
+In the same form, click **Send Test**. The test message uses a fixed sample monitor (`monitor.id` `1`, name `test`, URL `http://openstat.us`). Flashduty returns success and creates no alert.
 
 </Step>
 
 <Step title="Verify the alert lifecycle">
 
-1. Create a monitor with the URL `https://openstat.us/500` (always returns 500) and attach the notification channel from the first step
+1. Create a monitor with the URL `https://openstat.us/500` (always returns 500) and attach the notification channel from the first step. OpenStatus checks the URL before saving and asks **Still save?** because it returns 500; click **Save**
 2. After at least half of the monitor's regions report an error, confirm that Flashduty receives a Critical alert
 3. Point the monitor back at a healthy URL, wait for the regions to recover, and confirm that the alert recovers
 4. Delete the test monitor when you are done

en/on-call/integration/alert-integration/alert-sources/pandora-fms.mdx

@@ -85,9 +85,9 @@ Go to **Management → Alerts → Templates**, open the template you use, and in
 
 <Step title="Add the action to module alerts">
 
-1. Go to **Management → Alerts → Module Alerts** and create a module alert, or open an existing one
-2. Select the **Agent**, **Module**, and **Template**, and add `Flashduty` under **Actions**
-3. Save
+1. Go to **Management → Alerts → List of Alerts** and click **Create** to create a module alert, or find an existing one in the list
+2. Select the **Agent**, **Module**, and **Template**, and select `Flashduty` under **Actions**
+3. Click **Add alert**
 
 You can also add an alert for a module from the agent's **Alerts** tab and select the `Flashduty` action. For an existing module alert, just add the `Flashduty` action; existing actions such as email are not affected.