Slack Notifications
Configure error notifications and alerts for the Watch Tower.
The Watch Tower can send notifications to Slack when errors occur during operation.
Setting up Slack webhook
To enable Slack notifications, you need to provide a Slack webhook URL.
Creating a webhook
- Go to your Slack workspace settings
- Create an Incoming Webhook integration
- Copy the webhook URL (starts with
https://hooks.slack.com/services/...)
Configuring the watch-tower
Provide the webhook URL using the --slack-webhook option:
Or via environment variable:
What gets notified
The watch-tower sends Slack notifications for:
Execution errors
When the watch-tower encounters an error during execution:
- Exception messages
- Critical failures that prevent processing
- Link to Tenderly dashboard for debugging
Example notification:
Silent mode
You can disable all notifications using the --silent flag:
This is useful for:
- Local development and testing
- CI/CD environments
- When using alternative monitoring solutions
The --silent and --slack-webhook options are mutually exclusive. You cannot use both at the same time.
Error rate limiting
To prevent notification spam, the watch-tower implements automatic rate limiting:
Rate limit period
2 hours - Only one error notification is sent every 2 hours.
How it works
- When an error occurs, the watch-tower checks
LAST_NOTIFIED_ERROR in the database
- If less than 2 hours have passed since the last notification, the new error is logged but not sent to Slack
- If 2 hours or more have passed, the notification is sent and
LAST_NOTIFIED_ERROR is updated
This prevents flooding your Slack channel during incidents while still ensuring you’re alerted to ongoing issues.
Rate limit behavior
When a notification is rate limited, you’ll see a log message:
All errors are still logged locally at the configured log level, so you can review them even if not sent to Slack.
Slack notifications are sent as simple text messages using the webhook’s text field:
You can customize the webhook integration in Slack to:
- Set a custom bot name
- Add an icon
- Configure which channel receives notifications
Database persistence
The LAST_NOTIFIED_ERROR timestamp is persisted to the database atomically with other state updates. This ensures:
- Rate limiting survives watch-tower restarts
- No duplicate notifications after crashes
- Consistent behavior across deployments
Requirements
Slack notifications require one of:
- A valid
--slack-webhook URL is provided, OR
- The
--silent flag is enabled
If neither condition is met, the watch-tower will throw an error:
This prevents accidentally running the watch-tower without any error notification mechanism.
