Chat
I've been tinkering for a while. The main recent change has been monitoring. I used a Prometheus package and packaged everything (including Alertmanager and Grafana). I also added monitoring for the mail server (mox).
This was when I kept marveling at how good AI is — the monitoring was entirely written by AI. Although I later found a bunch of issues, that's hindsight; the point is it worked on the web.
GitOps
Dashboard RRR
For a GitOps setup with ArgoCD I think it's pretty great. At least it provides a dashboard where I can see things. Debugging is convenient, especially when you don't really understand what's happening. praising AI again
But one habit needs to change: staring at the dashboard. This happens because you can't be sure when problems will occur. Watching the dashboard is convenient, and sometimes you're just impatient. This should be alleviated as Alertmanager gets better — it's loud (X).
Dind...
For runner deployment, I originally used a Gitea act-runner Warp that included a Dind running on a Pi (deployed to the cluster via ArgoCD). Unfortunately its performance was really poor. We probably need a separate machine instead of using Dind. It has many problems: downloads are slow and often fail. It seems running Docker directly would be a better solution.
Monitoring
It's you again, AI
Yes — because I'm basically unfamiliar with that stuff, it's AI time again. But AI also made mistakes here; for example, the Discord key field was filled incorrectly (a small issue but it took a while to find lol). Ironically, the root cause was discovered by AI — basically AI built it.
Lots of dashboards
Maybe because things were bundled, there are so many dashboards it made me dizzy. Most of the time I don't know what the charts mean. The formulas are kind of magical, but they provide an intuitive way to observe things like capacity — and my capacity is pretty tight lol.
Summary
Lately, instead of working on apps or websites, infrastructure has become my new favorite. I might set up a cluster-shared Redis or PostgreSQL later. BTW, PG Vector is hard to install ;-;