Guides, Nov 13, 2026 # 3x-ui monitoring: what the panel can't tell you The 3x-ui panel tells you whether Xray is running, how busy the server is and how much each client has used. It cannot tell you whether a client on the internet can complete a handshake with each inbound, because it only looks from inside the server. To monitor 3x-ui properly, keep the panel's bot alerts for the host, and add four checks from outside: a real connection through every inbound, the subscription server, the certificates the inbounds actually serve, and Xray state through the API. Tahir Mehmood, 12 min read Everything below is checked against 3x-ui v3.11.0 (released 25 October 2026) and its documentation at [docs.sanaei.dev](https://docs.sanaei.dev/docs/). The project moves fast: v3.9.0, v3.10.0 and v3.11.0 all came out in October 2026. If you run an older panel, the menu numbers and some settings may differ. ## What does the 3x-ui panel monitor on its own? The panel watches the host and the Xray process, not the client's view. Out of the box you get: - **A status snapshot**: CPU, memory, swap, disk, network I/O, load, open connections, and Xray state (`running`, `stop` or `error`) with its version. The panel refreshes it every 2 seconds. - **Per-client data**: traffic, online status, expiry, and quota. - **Bot alerts** through Telegram or Discord, with the same events by email if you set up SMTP. - **An optional tunnel health monitor** (`XUI_TUNNEL_HEALTH_MONITOR=true`). It requests a URL every 30 seconds by default and restarts Xray after 3 failures in a row. The bot alerts are the most useful part, but only two events are on by default: `login.attempt` and `cpu.high`. In the [bot settings](https://docs.sanaei.dev/docs/operations/telegram-bot) (`tgEnabledEvents`) I turn on the rest: | Event | Fires when | | --- | --- | | `xray.crash` | Xray-core crashes | | `memory.high` | memory goes over `tgMemory` percent (default 80) | | `outbound.down` / `outbound.up` | an outbound goes down or comes back | | `node.down` / `node.up` | a managed node goes offline or comes back | Be careful with the tunnel health monitor. The [3x-ui README](https://github.com/MHSanaei/3x-ui) says it plainly: a restart drops all clients. Its probe also runs on the server itself. Unless you set `XUI_TUNNEL_HEALTH_PROXY` to a local Xray inbound, it only checks that the host can reach the internet, which says nothing about whether your users can reach the host. ## Why can an inbound look fine in the panel and still fail? An enabled inbound only means Xray has it in its config. Whether a client can use it depends on things the panel never sees: the network path, the firewall in front of the server, the certificate the client receives, and the country the client is in. These are the failures I see most, and how each one shows up: | Problem | What the panel shows | What a client sees | | --- | --- | --- | | Port closed in the cloud firewall or `ufw` | inbound enabled, Xray `running` | TCP connect timeout | | REALITY target (`dest`) wrong or unreachable | nothing unusual | handshake fails, see [Xray REALITY not working](https://tunnelhq.com/blog/xray-reality-not-working/) | | TLS certificate expired, or renewed on disk but not reloaded | nothing unusual | TLS error about the certificate | | Xray failed to start after an edit | Xray `error` | every inbound fails | | Another service holds the port, often nginx on 443 | Xray `error`, log says `address already in use` | connection reaches the wrong service | | Server DNS or outbound broken | nothing unusual | tunnel connects, but no website loads | | Client hit its traffic, expiry or IP limit | client marked depleted, or an IP in the ban log | that client is refused, or one of its IPs is banned | | Server IP blocked in the user's country | nothing unusual | works from your test host, fails for those users | Only three of these eight show up in the panel at all. That is the gap the rest of this post fills. When the panel itself is down ("3x-ui panel not working"), start on the server: `x-ui status` for the service, `x-ui log` for the debug log, and check that nothing else took the panel port (2053 by default) and that `/etc/x-ui/x-ui.db` is writable. ## How do I check Xray state through the 3x-ui API? Use the status endpoint with a read-only token. In the panel, create an API token with the `monitor` scope. It can read status and metrics but cannot change anything. Then, from another machine: ```bash PANEL='https://vpn.yourdomain.com:2053/your-web-base-path' curl -fsS -m 10 -H "Authorization: Bearer $XUI_TOKEN" "$PANEL/panel/api/server/status" | jq '.obj.xray' ``` ```json { "state": "running", "version": "v25.10.31" } ``` I run this from cron and report the result to a heartbeat, so that a dead panel, a wrong token and a stopped Xray all end in an alert: ```bash #!/usr/bin/env bash # Report Xray state on a 3x-ui panel to a heartbeat URL. set -uo pipefail state=$(curl -fsS -m 10 -H "Authorization: Bearer $XUI_TOKEN" \ "$PANEL/panel/api/server/status" | jq -r '.obj.xray.state') || exit 1 if [ "$state" = "running" ]; then curl -fsS -m 10 --retry 3 -o /dev/null "$HEARTBEAT_URL" else echo "xray state: $state" curl -fsS -m 10 --retry 3 -o /dev/null "$HEARTBEAT_URL/fail" fi ``` If the panel does not answer, the script exits without calling the heartbeat, and the heartbeat raises the alert when its period runs out. The `/fail` suffix reports an immediate failure. Healthchecks.io and TunnelHQ heartbeats both accept it. This check is still an inside view. `running` means the process is up, not that anyone can connect to it. ## How do I test every inbound from outside? Connect through each inbound with a real client, from a network that is not the server's own. A port check is not enough: Xray accepts the TCP connection even when the client's key or REALITY settings are wrong, and only drops it after the handshake fails. The port looks open while every client fails. The easiest way I have found uses 3x-ui's own JSON subscription: 1. In every inbound you want to watch, add a client called `monitor` with no expiry and no traffic limit. Give all of them the same Sub ID, so one subscription returns them all. 2. Leave its IP limit at 0, or add your checker's address to the trusted list, so Fail2ban never bans your own test host. 3. Turn on the JSON subscription in **Settings, Subscription** (`subJsonEnable`). The JSON endpoint returns a complete Xray client config for each inbound, with a local SOCKS inbound on `127.0.0.1:10808`. This script runs each config with Xray and fetches a URL through it: ```bash #!/usr/bin/env bash # Test every config in a 3x-ui JSON subscription with a real Xray client. set -u SUB_URL="${1:?usage: $0 }" TEST_URL="https://www.gstatic.com/generate_204" work=$(mktemp -d); trap 'rm -rf "$work"' EXIT curl -fsS -m 20 -o "$work/sub.json" "$SUB_URL" || { echo "FAIL subscription fetch"; exit 2; } count=$(jq 'if type == "array" then length else 1 end' "$work/sub.json") failed=0 for i in $(seq 0 $((count - 1))); do jq "if type == \"array\" then .[$i] else . end" "$work/sub.json" > "$work/one.json" name=$(jq -r '.remarks // "config \($i)"' --argjson i "$i" "$work/one.json") port=$(jq '[.inbounds[] | select(.protocol == "socks")][0].port' "$work/one.json") xray run -c "$work/one.json" > "$work/xray.log" 2>&1 & pid=$! sleep 2 if curl -fsS -m 15 -o /dev/null --socks5-hostname "127.0.0.1:$port" "$TEST_URL"; then echo "OK $name" else echo "FAIL $name"; failed=$((failed + 1)) fi kill "$pid" 2>/dev/null; wait "$pid" 2>/dev/null done echo "$failed of $count failed" [ "$failed" -eq 0 ] ``` The output looks like this, and the exit code is 0 only when every config passed: ``` OK monitor-vless-reality FAIL monitor-trojan-ws 1 of 2 failed ``` Three details make this check honest: - `--socks5-hostname` makes the server resolve the test URL's name. A broken DNS setting on the server then fails the check, as it would for users. - 3x-ui returns one object for a single config and an array for several. The `jq` lines handle both. - TUIC and AmneziaWG inbounds are left out of the JSON endpoints. Test those from the raw share links with a client that speaks them. Run it from a host with a different provider than your servers, and if you can, from a network like the one your users are on. For checking one link by hand, see [How to check a VLESS key](https://tunnelhq.com/blog/how-to-check-vless-key/). ## How do I check the 3x-ui subscription server? Fetch the subscription the way a client does and count the links. The [subscription server](https://docs.sanaei.dev/docs/config/subscription) is a separate listener from the panel, on port 2096 by default, and it answers `HEAD` requests with the same status and headers as `GET`: ```bash SUB='https://sub.yourdomain.com:2096/your-sub-path/your-sub-id' curl -sSI "$SUB" | grep -iE '^(HTTP|subscription-userinfo|profile-update-interval)' curl -sS "$SUB" | base64 -d | grep -c '://' ``` If you turned `subEncrypt` off, the body is already plain text, so drop the `base64 -d`. Alert when the status is not 200, or when the link count drops below what you expect. With the `monitor` client in every inbound, that count is the number of inbounds you serve. For reading the body and the headers in detail, see [How to decode a V2Ray subscription link](https://tunnelhq.com/blog/decode-v2ray-subscription-link/). When users report "3x-ui subscription not working", the cause is almost always one of these: 1. **Port 2096 is closed.** Opening the panel port is not enough, the subscription server has its own. 2. **`subDomain` is set.** The server then only answers for that host name, so links that use the IP or another domain fail. 3. **HTTP and HTTPS are mixed up.** The server speaks HTTPS only when `subCertFile` and `subKeyFile` are both set. An `https://` link to a plain HTTP listener fails, and so does the reverse. 4. **The path changed.** Fresh panels generate a random `subPath`. A reinstall or a settings reset gives a new path and breaks every old link. 5. **The client is disabled or depleted.** Only enabled clients on enabled inbounds appear, so the list shrinks without an error. 6. **A Clash client gets the base64 list.** Clash and Mihomo clients need the Clash output turned on (`subClashEnable`), and then either the Clash path or user-agent detection on the raw path. The `/clash-legacy/` path for Clash for Windows returns 422 when no inbound is compatible. A browser that opens the link shows an HTML info page. That is normal: the server sends the page to any request with `Accept: text/html`. ## How do I catch certificate problems before clients do? Check the certificate each TLS listener actually serves, not the file on disk. The [3x-ui SSL docs](https://docs.sanaei.dev/docs/config/ssl-certificates) warn that the panel keeps reading the files at their fixed paths and picks up a renewed certificate on its next restart. So the file can be fresh while the listener still serves the old one. ```bash # what the listener serves (panel 2053, subscription 2096, TLS inbounds) echo | openssl s_client -connect vpn.yourdomain.com:443 -servername vpn.yourdomain.com 2>/dev/null \ | openssl x509 -noout -enddate # what is on disk (path used by the x-ui SSL menu) openssl x509 -in /root/cert/vpn.yourdomain.com/fullchain.pem -noout -enddate # exit code 1 if it expires within 14 days echo | openssl s_client -connect vpn.yourdomain.com:443 -servername vpn.yourdomain.com 2>/dev/null \ | openssl x509 -noout -checkend 1209600 ``` If the date on disk is later than the date served, the process has not reloaded the file. Restart the panel with `x-ui restart` for the panel and subscription certificates, and Xray with `x-ui restart-xray` for inbound certificates. Pick the warning time from the certificate's lifetime. Let's Encrypt domain certificates last 90 days, so 14 days of warning is plenty. The short-lived IP certificates from the x-ui menu last about 6 days, so warn at 3. Skip this check on REALITY inbounds. They have no certificate of their own, and `s_client` shows the target site's certificate instead. ## What about traffic limits, expiry and renewals? Expect failures that come from your own limits, and keep them out of your alerts. When a client reaches its quota or expiry date, its connections fail authentication, and once it is disabled it also leaves the subscription. For that user this is correct behaviour, not an outage. Two habits keep it from confusing your monitoring: - The `monitor` clients never expire and have no quota. If one of them fails, it is the server, not a limit. - The bot's periodic report (`tgRunTime`, daily by default) lists exhausted and expiring clients. Set `expireDiff` and `trafficDiff` to get warnings before a paying user is cut off. Both default to 0, which means off. ## How do I update 3x-ui without losing track of it? Update with `x-ui update`, or menu item **Update**. Docker installs pull the new image and recreate the container instead. Updates keep the database and settings, but take a backup before a major-version jump. After an update, check three things: the status endpoint still says `running`, the Xray version is the one you expected, and the outside check above still passes for every inbound. A new Xray-core release can change behaviour for a transport, and the panel will not tell you. `x-ui update-all-geofiles` updates the geoip and geosite files and restarts if they changed. A restart drops live connections, so run it at a quiet hour. ## How do I monitor 3x-ui nodes? On v3, a master panel can manage other 3x-ui panels as nodes. The master sends each node a heartbeat every few seconds, marks it `online` or `offline`, and fires `node.down` and `node.up` events. Turn those events on in the bot. A node heartbeat proves the node's API answers the master. It does not prove the node's inbounds work for users, so the outside check above still applies to every node. Managed hosts make the subscription longer: each enabled host on an inbound adds another link with its own address, so count those in the number of links you expect. ## What should I back up, and how do I know the backup ran? Back up the database and the certificates. The database is `/etc/x-ui/x-ui.db` with SQLite (the default), or your PostgreSQL database. Certificates from the x-ui menu live under `/root/cert/`. The simplest off-server copy is the bot: with `tgBotBackup` on, the periodic report carries the database and Xray's `config.json`. If you back up with your own cron job instead, make the job prove it ran: ```bash 0 4 * * * cp /etc/x-ui/x-ui.db /backup/x-ui-$(date +\%F).db && curl -fsS -m 10 --retry 5 -o /dev/null "$HEARTBEAT_URL" ``` A backup that has silently failed for a month is a problem you only find on the day you need it. ## A monitoring plan for a 3x-ui server | What | How | How often | Alert when | | --- | --- | --- | --- | | Every inbound | real client connection from outside | 1 to 5 minutes | the same inbound fails twice in a row | | Subscription server | `HEAD` plus a link count | 5 to 15 minutes | not 200, or fewer links than expected | | Certificates | `openssl ... -checkend` on each TLS listener | daily | less than 14 days left (3 for IP certs) | | Xray state | status API with a `monitor` token | 1 minute | not `running`, or no answer | | Host | bot events: `xray.crash`, `cpu.high`, `memory.high`, `node.down` | as they happen | any event | | Backups | heartbeat at the end of the job | daily | heartbeat missed | Full disclosure: TunnelHQ does the first row of this table. Paste each inbound's share link, or the `monitor` client's subscription URL, and it completes the real protocol handshake through every inbound from outside, every 1 to 5 minutes depending on plan, opening and closing incidents on its own. Its Uptime checks (in beta) cover the heartbeat rows. Every plan has a 14-day free trial, and you can try a single link in the [free tester](https://tunnelhq.com/test/) first. Source: https://tunnelhq.com/blog/monitoring-3x-ui-servers/