About
Errors to GlitchTip, Sentry or email, and alerts by email, Matrix or Telegram on unhandled errors, failed readiness and degraded services
Provides
Services
ObservabilityService
Commands
-
$
splent feature:observability statusWhere the errors go and who the alerts reach.
-
$
splent feature:observability testSend a test alert to every channel and a test error to the backend.
Run them in your workspace once the feature is installed in the selected product.
Configuration
| Setting | Default | What it does |
|---|---|---|
OBSERVABILITY_BACKEND |
none
|
Where the errors go: glitchtip or sentry (both through the Sentry SDK and OBSERVABILITY_DSN), email (one mail per kind of error to OBSERVABILITY_EMAIL_TO, through splent_feature_mail), or none (the log). |
OBSERVABILITY_DSN |
empty | The DSN of the GlitchTip or Sentry project, https://<key>@<host>/<id>. |
OBSERVABILITY_ENVIRONMENT |
empty | The environment the reports are filed under. Empty means production, development or testing, after the app's profile. |
OBSERVABILITY_RELEASE |
empty | The release the reports are filed under. Empty means <product>@<version>, the version the image was built with. |
OBSERVABILITY_SAMPLE_RATE |
1.0
|
The share of errors sent to GlitchTip or Sentry, from 0 to 1. |
OBSERVABILITY_TRACES_SAMPLE_RATE |
0.0
|
The share of requests traced for performance, from 0 to 1. Off by default: a trace carries every query of the request. |
OBSERVABILITY_ALERT_CHANNELS |
empty | Who is told when something breaks: email, matrix and telegram, any of them. Empty alerts nobody; the backend still gets the errors. |
OBSERVABILITY_ALERT_ON |
errors,readiness,degraded
|
What is alerted on: errors (an unhandled error in a request), readiness (a check of /readyz failing, and recovering) and degraded (what a feature reports with report_degraded). |
OBSERVABILITY_ALERT_DEDUP_MINUTES |
60
|
Minutes during which the same alert is not sent again. The next one says how many were held back. |
OBSERVABILITY_READINESS_GRACE_SECONDS |
60
|
Seconds a readiness check fails before it is alerted on, so a deploy that waits a few seconds for its database alerts nobody. |
OBSERVABILITY_EMAIL_TO |
empty | The addresses of the email backend and of the email channel. |
OBSERVABILITY_MATRIX_HOMESERVER |
empty | The Matrix homeserver of the account that posts the alerts. |
OBSERVABILITY_MATRIX_ROOM_ID |
empty | The room the alerts go to, !id:server, and the access token of that account. Both in the deployment's .env. |
OBSERVABILITY_MATRIX_TOKEN |
empty | No description yet. |
OBSERVABILITY_TELEGRAM_BOT_TOKEN |
empty | The bot that posts the alerts, and the chat they go to. Both in the deployment's .env. |
OBSERVABILITY_TELEGRAM_CHAT_ID |
empty | No description yet. |
OBSERVABILITY_HTTP_TIMEOUT |
5
|
Seconds a Matrix or Telegram request may take. |
OBSERVABILITY_STATE_DIR |
empty | Where the processes of a container agree on what was alerted. Empty means a directory in the system's temporary directory. |
These are the defaults. A product changes them in its own pyproject.toml, under [tool.splent.config], or in .env.deploy in production.
Dependencies
No hard dependencies.
In product lines
- openhubs_spl Optional in this line
Extension points
Services
ObservabilityService
Routes
A refinement can add routes to its blueprint
Other features fill its hook slots. A refinement can override its templates and services or extend its models without forking it.
Compatibility
Python
>=3.13
SPLENT framework
>=1.30.0
Python packages
sentry-sdk[flask]>=2.20,<3requests>=2.33,<3