Troubleshooting
Common symptoms/errors and their fixes. Quick triage → detailed resolution flow.
status.sh— one-line summary of containers + ports + API + key app.properties entrieshealth.sh— comprehensive health check based on exit codes (suitable for cron)doctor.sh— bulk diagnostic tarball (api / docker / systemd / 7 component logs / config redacted / host metrics) → support escalation/api/v1/system/health— status map for each of the 8 components + 503 branching (immediately identifies which component is DOWN)/opt/kopens/install/RUNBOOK.mdin the box — 5-scenario matrix (A–E) + command cheat sheet
Index by symptom
| Symptom | Candidate causes |
|---|---|
| App hangs during boot | Cassandra connection failure, DDL application failure |
/api/v1/edge responds with started=false | Collector initialization failure (driver / cache issue) |
Only a specific OPC shows connection_status=DISCONNECTED | PLC network / authentication / address format |
| All OPCs occasionally drop at once | Full OPCUAServerManager restart caused by OPC add/modify |
| MQTT publish not working | mqtt.enable=false or broker authentication failure |
| Sparkplug topics not published | sparkplug.enable=false or missing jar |
| Screen loads slowly | opcList() N+1 queries + accumulated sleep |
| OPC start takes 5 seconds each | Impact of Thread.sleep (target of Phase A1) |
Boot / initialization
Cassandra connection failure
Caused by: com.datastax.oss.driver.core.exceptions.NoHostAvailableException
| Check | Verification |
|---|---|
| Cassandra running | nodetool status |
| Port reachable | telnet <app.db.host> 9042 |
| Keyspace exists | cqlsh -u cassandra -p cassandra then DESCRIBE KEYSPACE pe; |
app.db.* matches oltp.cassandra.* | Review entries on both sides of app.properties |
app.sql.path is a typo for resouces
The resouces in app.sql.path = classpath:resouces/sql of app.properties is a typo, but the directory is kept under the same
name so it works. If you get an error about SQL resources not being found, check whether the WEB-INF/resouces/sql/
path actually exists.
OPC / driver
A specific OPC will not connect
| Protocol | First checks |
|---|---|
| OPCUA | opc_agent_port reachable, discovery=true attempted, user/password |
| Modbus TCP | Port 502 reachable, holding-register:... format |
| MELSEC | MC protocol enabled, controller-type=Q_L matches |
| S7 | rack/slot match, "PUT/GET" enabled |
| LS | If NetUtils.isReachable fails, suspect ICMP blocking; port 2004 |
| EIP | rack/slot, port 44818 |
Refer first to the "Common errors + fixes" table on each driver page.
LS-PLC Ping failed
The LS driver checks reachability with an ICMP ping before connecting. If ICMP is blocked on the corporate network, a healthy PLC will also be reported as failed.
Fix:
- Allow ICMP on the firewall
- Or a follow-up change to make the ping check in
LSDriver.connect()optional (currently requires a code change)
32-bit / 64-bit values are inconsistent
In most cases the cause is format not specified.
| Protocol | Wrong pattern → correct pattern |
|---|---|
| LS | D00600 + Integer (read as 16-bit) → D00600 + Integer + format=DW |
| Modbus | holding-register:1 (16-bit) → holding-register:1:DINT |
| S7 | %DB1.DBW0 + Float → %DB1.DBD0 + format=REAL |
32-bit float values are NaN / huge numbers
A byte order (endianness) problem. Check the byte/word swap policy of the slave / master:
- Modbus → use PLC4j options such as
:UDINT_LSWORD_FIRST - For other protocols, post-process with a fomula
Transmission (MQTT / Sparkplug)
MQTT publish not working
# broker 도달 확인
MQTT_USER="${MQTT_USER:-edge}"
MQTT_PASSWORD="$(tr -d '\r\n' < /run/secrets/mqtt-password)"
mosquitto_pub -h <mqtt.server.host> -p 1883 -u "$MQTT_USER" -P "$MQTT_PASSWORD" -t /edge/point -m '{"test":1}'
Checklist:
mqtt.enable=truemqtt.server.host/port/user/passwordcorrect- Whether the broker's ACL allows publishing from the client
MQTT connectedor reconnection logs in catalina.out
Sparkplug topics are not visible
| Check | Method |
|---|---|
Whether it is sparkplug.enable=true | Check properties + restart required |
| Whether the jar is in lib | ls WebContent/WEB-INF/lib/tahu-core*.jar |
| Whether it is the same broker | mqtt.server.* and SPB are identical |
| Broker ACL for SPB topics | spBv1.0/# publish permission |
NDEATH appears to be published twice
This is normal behavior. Explicit publication on normal shutdown + (since a will is also registered) if the broker does not recognize the disconnection as a will-bypass, the will may be published as well. In that case the host side must be implemented so that an NDEATH with the same bdSeq is processed only once.
Performance / responsiveness
OPC start/stop responses take 5 seconds
This is due to the intentional Thread.sleep in ConnectService (to guarantee collector warm-up).
- Temporary workaround: reload the screen to check the polling response
- Permanent fix: follow-up PR applying
REFACTORING_PLAN.mdPhase A1 (removeThread.sleep/CountDownLatch)
First screen load is slow
opcList() N+1 queries + distributed LastValueMap lookups can add roughly 1–2 seconds of extra delay.
- Expected to improve once Phase B1 (bulk IN select) and B2 (single facade) are applied
Changing one OPC temporarily suspends other OPCs
This is because OPCUAServerManager.restart() is a full restart (Phase A2 — target for partial reload).
Emergency workaround: perform OPC changes outside operating hours.
Data / Cassandra
Tombstone accumulation warning
When many OPCs are deleted in operation, the tombstone metric of nodetool cfstats pe.app_tag may spike temporarily.
- Not serious, since the number of OPCs/Tags on this gateway is usually ≤ 1000
- Time series (
TM_TAG_POINT) is the responsibility ofplantpulse-timeseries-engine, not this gateway
tag not found: <id>
Occurs frequently on PUT /api/v1/tag/{tagId} or on read.
Causes:
tag_idtypo- Since it is a
ALLOW FILTERINGquery ofAPP_TAG.xmlR03, the partition key is unspecified. tag_id must be exact.
Verification:
cqlsh -u cassandra -p cassandra -k pe -e "SELECT tag_id FROM app_tag;"
tag write not supported for opc_type=... when writing a value
Protocols other than HTTP currently do not support write.
- Only
HTTPDriver.bind()is implemented - Write for other protocols requires SPI extension together with Sparkplug Phase 3 (NCMD/DCMD)
Security / operations
/api/* is exposed without authentication
By design this assumes a corporate internal network (/api/* is not included in SecurityFilter check-pattern). If exposed
externally, place mTLS / API Key / Basic auth in front of a reverse proxy.
CSRF
CSRF is applied only to the *_.do pattern. v1 mutation endpoints are out of scope. Review together when exposing externally.
Log collection guide
Attaching the following when reporting an issue speeds up analysis.
# 환경
curl -s http://localhost:8080/api/v1/edge
# OPC 상태
curl -s http://localhost:8080/api/v1/opc
# 시스템 메트릭
curl -s http://localhost:8080/api/v1/monitoring
# 최근 로그
tail -n 500 $CATALINA_HOME/logs/catalina.out
Mask the password in app.properties before attaching.