Docs
uk8s
Cluster Monitoring
Prometheus monitoring solution
Add Recipients

Add Recipients

The monitoring center supports the addition of recipient emails, corporate WeChat and DingTalk in three ways of alarm. Please configure in UK8S Console’s “Monitoring Center” → “Send and Receive Settings”.

1. Configuring the Outgoing Server

Before configuring the recipient, you need to first set up the sender’s email information. The sending of information is currently carried out via email.

Different email service providers have relatively detailed descriptions of the configuration of outgoing servers; here are two points to emphasize:

  1. TLS is not currently supported, so do not fill in the TLS port;
  2. The password is recommended to be the client password, filling in the mail account password may not send emails.

2. Configuring Email Recipients

Multiple email recipients can be added

  • Name: Recipient’s name
  • Email Address: The recipient’s email address

3. Configuring Corporate WeChat Recipients

Note: For the corporate WeChat robot type, please refer to the ‘Configuration of webhook recipients (DingTalk/Corporate WeChat robot method)‘.

Before using WeChat recipients, we must create an application in the corporate WeChat management background and obtain information such as application ID, corporate ID, application secret, department ID, and corporate WeChat User ID, you need to consult your WeChat administrator or manager to obtain relevant information. Please refer to the official documentation on alertmanager configuration here.

  • Name: The name defined for the recipient used for alertmanager configuration
  • Corporate ID: Unique Corporate WeChat ID; Administrators log in to the corporate WeChat web page, query in My Company->Company Information; (Cannot be queried when only one person in the corporate WeChat)
  • Application ID: Administrators log in to the corporate WeChat web page to view the details of self-built applications in Application Management, the AgentId of the application is the ID of this application; If there is no self-built application, you can create an application first;
  • Application secret: The same as the query of the Application ID, there is a secret option in the application details page, this is the secret of the application;
  • Recipient:
    • Recipient’s tag: Create a tag for the recipient
    • Recipient’s department: Recipient’s company department ID, The administrator enters the address book, find the recipient’s department in communication, click the three points next to the department, and then check the department ID
    • Receiver: The recipient’s user email.

Since the monitoring center has configured a watchdog alarm rule, as long as the information of the WeChat Enterprise is filled in correctly, you can usually get the alarm information from the WeChat Enterprise within 10 minutes.

If corporate WeChat can’t receive the alert messages, you need to set a trusted IP, please refer to the official documentation of corporate WeChat. Administrators log in to the WeChat corporative web page and click on the self-built application in “Application Management” to enter the details page; find “Enterprise Trusted IP” for configuration. The IP filled in is the IP of visiting corporate WeChat (usually the external elastic IP of the NAT gateway related to the cluster).

4. Configuring Webhook Recipients (DingTalk/Corporate WeChat Robot Method)

4.1 Create DingTalk/Corporate WeChat Robot, Get Webhook Address

Before using webhook recipients (DingTalk/Corporate WeChat robot method), we must create a custom bot in DingTalk/Corporate WeChat management background, and get its Webhook address, please refer to the relevant documentation of DingTalk/Corporate WeChat.

4.2 Deploy Configuration File

AlertManager does not support direct access to DingTalk/Corporate WeChat alerts, so adaptation conversion is required. Here is a sample deployment yaml file referenced by the community.

Please replace the webhook address and image in the yaml according to the prompts in the yaml and the actual situation.

apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
data:
  config.yaml: |-
     targets:
      webhook1:  
        # Please replace with your DingTalk/Office WeChat robot Webhook address
        url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxx
---
apiVersion: v1
kind: Service
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
spec:
  selector:
    k8s-app: webhook
  ports:
    - name: http
      port: 80
      targetPort: 8060
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: webhook
  template:
    metadata:
      labels:
        k8s-app: webhook
    spec:
      volumes:
        - name: config
          configMap:
            name: alertmanager-webhook
      containers:
        - name: alertmanager-webhook
          # Replace image value to:
          # Corporate WeChat bot: uhub.ucloud-global.com/uk8s/prometheus-webhook-wechat:v2.0.0
          # DingTalk Bot: uhub.ucloud-global.com/uk8s/prometheus-webhook-dingtalk:v2.0.0
          image: xxx:xxx
          args:
            - --web.listen-address=:8060
            - --config.file=/config/config.yaml
          volumeMounts:
            - name: config
              mountPath: /config
          resources:
            limits:
              cpu: 100m
              memory: 100Mi
          ports:
            - name: http
              containerPort: 8060

4.3 Add Recipients

On the “Send and Receive Settings” page of the control panel, click “Add” in the “Recipient” panel, and fill in the according to the type in the Webhook address bar

DingTalk Robot: http://alertmanager-webhook.uk8s-monitor.svc/dingtalk/webhook1/send

Corporate WeChat Robot: http://alertmanager-webhook.uk8s-monitor.svc/wechat/webhook/send

5. Configuring Webhook Recipients (WeChat Official Account)

5.1 Create an official account and template, and get the appid, secret and templateid of the official account

This alert method is based on the template message of the WeChat official account. Please follow the WeChat official account template message operation specification when using.

You can refer to the following for template message content (the template name is arbitrary)

Alarm status: {{ status.DATA }}
Alarm type: {{ alertname.DATA }}
Alarm level: {{ severity.DATA }}
Alarm instance: {{ instance.DATA }}
Alarm content: {{ message.DATA }}
Alarm time: {{ startsat.DATA }}

5.2 Deploy Configuration File

AlertManager also does not support direct access to WeChat’s official account alerts. Adaptation conversion is required.

Please replace the configuration in the yaml according to the prompts in the yaml and the actual situation.

apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
data:
  config.yaml: |
    # Please replace with your WeChat public account's appid
    appid: "xxx"
    # Please replace with your WeChat public account's secret
    secret: "xxx"
    # Please replace with your WeChat public account's templateid
    templateid: "xxx"
    # Alert groups: name is the group name, chatids below are the openid of users in this alert group
    chatgroups:
      - name: uk8s
        chatids:
          - "openid1"
      - name: all
        chatids:
          - "openid1"
          - "openid2"
---
apiVersion: v1
kind: Service
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
spec:
  selector:
    k8s-app: webhook
  ports:
    - name: http
      port: 80
      targetPort: 8060
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: alertmanager-webhook
  namespace: uk8s-monitor
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: webhook
  template:
    metadata:
      labels:
        k8s-app: webhook
    spec:
      volumes:
        - name: config
          configMap:
            name: alertmanager-webhook
      containers:
        - name: alertmanager-webhook
          image: uhub.ucloud-global.com/uk8s/prometheus-webhook-wechat-public:v2.0.0
          args:
            - --web.listen-address=:8060
            - --config.file=/config/config.yaml
          volumeMounts:
            - name: config
              mountPath: /config
          resources:
            limits:
              cpu: 100m
              memory: 100Mi
          ports:
            - name: http
              containerPort: 8060

5.3 Add Recipients

On the “Send and Receive Settings” page of the console, click “Add” in the “Recipients” pane, and fill in the Webhook address bar

http://alertmanager-webhook.uk8s-monitor.svc/wechat/{groupname}/send

Where {groupname} is the chatgroups.name in config.yaml

Start Monitoring CenterCommon Questions about Monitoring