Multi-tenant observability platform operated since December 2024, consolidating heterogeneous infrastructure telemetry across 12+ clients.
Selected work / Data Ops
SystemFrame
Multi-tenant observability platform for 12+ clients and 390+ endpoints, with GitOps deploy and an alert pipeline integrated with GLPI and ServiceNow.
-60%
MTTR50MIN → 20MIN
-
01 / The context
The platform monitors heterogeneous infrastructure across 12+ clients: Aruba and Meraki equipment, plus Zabbix and GLPI already installed in each client's fleet. There was no consolidated telemetry -- each client was an island, and diagnosing an incident could take a whole day. Alerts reached the team but didn't turn into a ticket on their own.
-
02 / My contribution
I took ownership of the platform layer with K3s, Helm, ArgoCD, and Ansible, sustaining continuous GitOps deployment. I wrote 5 custom exporters for Aruba, Meraki, Zabbix, and GLPI, consolidating 390+ endpoints into a single format. On top of that I built the alert pipeline connecting Alertmanager to GLPI and ServiceNow, running 296 rules, plus analysis tools built on the Prometheus and VictoriaMetrics APIs distributed across 2 availability zones.
-
03 / The outcome
MTTR dropped from 50 to 20 minutes (-60%), and incident diagnosis time went from 1 day to 2 hours. The GitOps deploy pipeline accumulated more than 3,500 CI/CD runs, averaging around 8 deploys per business day.
-
04 / What I learned
A custom exporter is maintenance debt: each one becomes proprietary code that has to track vendor firmware and API changes. It was worth it because unified-format telemetry was what made the 296 rules possible -- without it, each client would need its own set of alerts. And an alert that doesn't automatically become a ticket doesn't reduce MTTR: the gain came from the GLPI and ServiceNow integration, not from detecting issues faster.
Architecture
Explore the components, the decisions behind them, and the trade-offs involved.
Drag to explore the diagram.
entryservicedataexternalasync
Click a component to see the decision made there and what it cost.
Read all architecture decisions
- 5 custom exporters
- Write proprietary exporters for each source, instead of adopting a single off-the-shelf agent or accepting the native formats.
- Trade-off: Each exporter is code that has to track vendor firmware and API changes. It was what enabled unified-format telemetry across heterogeneous fleets -- and without it, the 296 rules wouldn't exist.
- Prometheus / VictoriaMetrics
- Keep VictoriaMetrics alongside Prometheus and distribute across 2 availability zones.
- Trade-off: Two metrics systems to operate and understand instead of one. It buys longer retention and survival if one zone goes down.
- Alertmanager
- Concentrate the 296 rules in Alertmanager and handle deduplication there, before any external integration.
- Trade-off: The rules become a large, centralized artifact that requires discipline to keep from turning into a mess. It prevents each integration from reimplementing its own silencing logic.
- Analysis tools
- Build query tools on top of the Prometheus and VictoriaMetrics APIs instead of relying solely on visual dashboards.
- Trade-off: More internal code to maintain, outside the product's core path. It cut incident diagnosis time from a day to two hours, because investigating no longer depended on building a dashboard on the fly.
- ArgoCD
- GitOps deployment with ArgoCD and Helm, with desired state versioned instead of applied by hand.
- Trade-off: Every change now requires a commit and sync, which is slower for a one-off hotfix. In exchange, onboarding a new client became Helm values, and cluster state stopped depending on who applied what.
- K3s + Helm
- K3s instead of full Kubernetes to sustain the multi-tenant fleet.
- Trade-off: Fewer components out of the box, so part of what a managed cluster delivers ready-made had to be built. It paid off through lower per-client resource consumption and simpler operations.