EXPAND ALL
  • Home
Open Source Docs

Slack Alerts using the Pixie API

This tutorial will show you how to create a Slackbot to monitor your Kubernetes cluster using data from Pixie's observability platform. This Slackbot reports the number of HTTP errors per service in your cluster. However, the example code can be modified to alert based on any data available from Pixie.

Slackbot alerting for per service HTTP errors.

Before you begin

This tutorial assumes:

  1. You have a Slack workspace to install your Slack App to. If not, please create a workspace.

  2. You have a Kubernetes cluster with Pixie installed. If you don't have a cluster, you can create a minikube test cluster and install Pixie, using our install guides.

Setup Slack

First, you'll need to create a Slack App, which will allow our application to post messages in a Slack channel.

Create a Slack app

  1. Go to https://api.slack.com/apps and select Create New App on the top right.
Configuring your new Slack app.
  1. Name your app Pixie Alerts and select the workspace to test your app in.

  2. After creating the Pixie Alerts App, you'll be taken to the Basic Information page for the app. Under Add features and functionality, select Bots.

Add Bot functionality to your Pixie Alerts Slack App.
  1. On the next page, select the Review Scopes to Add button.

  2. On the next page, scroll down to the Scopes section, select the Add an OAuth Scope button and then the chat:write option.

The Pixie Alerts App will simply write to a channel, but if you wanted to add support for direct messaging or other functionality, you'll need to increase the bot's scope on this page.

Add OAuth scope for `chat:write`.
  1. Scroll up to the top of this page (OAuth & Permissions) and select Install to Workspace to install the Pixie Alerts App to your workspace.
Install Pixie Alerts App to workspace.
  1. Copy the Bot User OAuth Access Token.
Save Bot OAuth Access Token after installation.

Create an alerts channel

  1. Create a new channel called #pixie-alerts for the Pixie Alerts App to post in. If you use a different channel name, you'll need to update the CHANNEL variable in the slackbot code.

  2. Invite the Pixie Alerts App to the channel, using /invite @Pixie Alerts.

Adding the Pixie Alerts App to a channel.

Setup your cluster

  1. Use Pixie's CLI to deploy the px-sock-shop demo. Our Pixie Alerts App will report HTTP errors for services in the px-sock-shop namespace.
px demo deploy px-sock-shop
  1. Find your cluster's ID following the directions here. Save this string, we'll use it in Part 3.

  2. Create an API Key following the directions here. Save this string, we'll use it in Part 3.

Run the Slackbot app

This tutorial uses example code from Pixie's GitHub repository.

  1. Get the example directory.
# Clone the Pixie repository.
git clone https://github.com/pixie-io/pixie-demos.git
# Change to the app directory.
cd slack-alert-app/<language>
  1. (Python only) Install the dependencies.
# Requires Python 3.8.7+
pip install -r requirements.txt
  1. Create the environment variables required by the slackbot code:
# Slackbot token from Part 1.
export SLACK_BOT_TOKEN="your_slackbot_token"
# Pixie Cluster ID from Part 2.
export PIXIE_CLUSTER_ID="your_cluster_id"
# Pixie API Key from Part 2.
export PIXIE_API_KEY="your_api_key"
  1. Run the slackbot app:
go run slackbot.go

Congrats, your Pixie Alerts App will now post automated alerts to the #pixie-alerts channel in your Slack workspace!

(Optional) Modify the PxL Script

The slackbot can be modified to alert based on any information available from Pixie's observability platform. Some notes:

  • We recommend testing your PxL code in the Live UI. Once it works, you can replace the PXL_SCRIPT string in the slackbot app code.

  • The example PXL_SCRIPT filters for services in the px-sock-shop namespace only. Make sure to modify or remove this line to fit your cluster's needs.

  • Make sure that the start_time variable in your PXL_SCRIPT matches the slack messaging interval to get data for that specific interval only.

  • The table name supplied in the call to px.display(df, "<table_name>") needs to match the table name in the slackbot app call to the Pixie API, else you will receive a ValueError: Table 'table_name' not received error.

References

© 2018-21 New Relic, Inc. All Rights Reserved.
This site uses cookies to provide you with a better user experience. By using Pixie, you consent to our use of cookies.