Troubleshooting
This document provides guidance on common issues and resolutions for the PlantPulse platform operating via one-liner / Docker installation. Most issues can be resolved by following the procedures below in order.
The Platform operates as a Docker Compose stack — one certificate (plantpulse-certs), one data lake, six apps, and one proxy. All nine must be running constantly, and (healthy) is normal. There is no container named plantpulse-platform.
Narrow down «what is broken» first. ./status.sh provides answers by service → Installation Layout
First Step: Regardless of the issue, please check the following three items first.
If the issue persists, please generate a diagnostic tarball via
./doctor.shand send it to the technical support team.
Container Startup Issues
Symptom: Container does not recover from unhealthy state
| Cause | Resolution |
|---|---|
| Aster TSDB schema migration failure | Confirm error via ./logs.sh cassandra -n 300 then attempt ./restart.sh. If failure repeats, use ./doctor.sh |
| Data directory permission issue | Check /data1/pp-data permissions on the host. sudo chown -R root:root /data1/pp-data && sudo chmod -R 755 /data1/pp-data |
| Out of Memory (OOMKilled) | Confirm via docker inspect <컨테이너> --format '{{.State.OOMKilled}}'. Limits vary by container — Data Lake is DOCKER_DATALAKE_MEMORY, Apps are DOCKER_SERVER_MEMORY, etc. (Environment Variables) |
| JVM warm-up incomplete | Booting takes 3–5 minutes. Diagnose further if unhealthy state persists beyond 5 minutes |
# 어떤 서비스가 비정상인가 (0 = 정상 / 2 = 비정상)
./status.sh
# 스택 전체 준비 판정
./stack-verify-boot.sh
# 컨테이너별 메모리 / CPU
docker stats --no-stream
# 헬스체크 엔드포인트 응답 확인
curl -kfsS https://<server-ip>:4950/api/health | jq
plantpulse-certs bakes the certificate once and stays active to monitor expiration. (healthy) is normal. The health check verifies whether the volume materials are usable rather than just whether the process is alive — it checks the time remaining until expiration and whether the keystore opens.
Symptom: no such service · Container is reported as missing
The initial installation was not completed, or containers were removed via ./remove.sh.
If you were looking for plantpulse-platform following an old runbook, there is no container by that name — check for names as ./status.sh or ./logs.sh --list.
cd /opt/kopens/plantpulse-platform-docker/bin
./install.sh # 최초 설치
# 또는 OS 설정이 이미 끝났다면
./up.sh # 컨테이너 생성 + 기동
Symptom: address already in use (Port Conflict)
This occurs when an older version of PlantPulse is running natively on the host or another service is occupying the port.
# 호스트 native plantpulse 정지
pkill -ef plantpulse
# 특정 포트 점유 프로세스 확인 (예: 80)
ss -tlnp | grep :80
# 충돌 프로세스를 종료한 후 재시도
./restart.sh
Refer to the Ports and Service Management page for a detailed port list.
Image / Registry Issues
Symptom: docker pull → unauthorized: authentication required
Registry authentication has expired or credentials are missing.
docker login docker.kopens.io
# Username/Password 입력 후, 하던 설치 · 업그레이드 원라이너를 다시 실행
Symptom: Image Download Failure (Network)
# 1. 레지스트리 접근 가능 여부 확인
curl -fsSL https://docker.kopens.io/v2/
# 2. DNS 확인
nslookup docker.kopens.io
# 3. 회사 방화벽 / 프록시 차단 가능성 — 네트워크 관리자에게 다음 도메인 허용 요청
# docker.kopens.io, download.kopens.io
For air-gapped network environments, please follow the air-gapped installation procedure on the Docker Installation page.
Database Issues
Database components operate inside containers. Inspections are performed after entering the container via ./shell.sh.
PostgreSQL (Meta DB)
Role: Stores platform metadata such as user information, site settings, and asset settings.
./shell.sh # 컨테이너 진입
# 컨테이너 내부에서
/opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node psql # PostgreSQL 셸 접속
psql=# SELECT 1;
psql=# SHOW max_connections;
psql=# SELECT count(*) FROM pg_stat_activity;
| Symptom | Cause | Resolution |
|---|---|---|
| Connection refused | PostgreSQL component down | Run pd restart storage inside the container. From host: ./restart.sh |
| Too many connections | Connection pool exceeded | Review connection pool settings in properties. Restart storage if temporary |
| Authentication failed | Password mismatch | Verify consistency between PP_PG_PASSWORD in /etc/kopens/plantpulse-platform.env and application properties |
Path Verification — The authoritative copy of the secret sidecar is only
/etc/kopens/plantpulse-platform.env(Permission0600). Duplicate names may exist under/opt/kopens/from old installations, but they are not read; the installation script restores the authoritative copy → Environment Variable Reference
Aster TSDB (Time Series DB)
Role: Stores large-scale data such as sensor time series data and alarm history.
./shell.sh
# 컨테이너 내부에서
/opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node status # 클러스터 상태
/opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node compactionstats # 컴팩션 진행 상태
/opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node cql # CQL 셸 접속
| Symptom | Cause | Resolution |
|---|---|---|
| Connection timeout | Node down | Check node status via pd node status then pd restart storage |
| WriteTimeout | Write latency (Disk I/O saturation) | Check pd node compactionstats, perform manual compaction via pd node compact |
| ReadTimeout | Read latency (Large partition) | Check partition size via pd node table-histograms |
| Insufficient disk space | SSTable accumulation | Run pd node cleanup then verify with df -h /data1 |
Valkey (Redis Cache)
./shell.sh
# 컨테이너 내부에서
redis-cli -a "$PP_REDIS_PASSWORD" ping
redis-cli -a "$PP_REDIS_PASSWORD" INFO memory
Login / Console Access Issues
Symptom: Unable to access console from browser
# 1. 컨테이너 health 확인
./status.sh
# 2. 외부 노출 IP 설정 확인
grep DOCKER_PP_EXTERNAL_IP env.sh
# 3. 호스트 방화벽 확인
sudo firewall-cmd --list-ports # RHEL/Rocky/Oracle
sudo ufw status # Ubuntu
# 4. 포트 점유 확인 — 프록시가 80/443 을 엽니다
ss -tlnp | grep -E ':(80|443)\s'
| Symptom | Cause | Resolution |
|---|---|---|
| Page does not open | plantpulse-proxy is abnormal | Check proxy status via ./status.sh. The proxy is the sole entry point for 80/443 |
| Page does not open | Firewall blocking | Request allowance for 80, 443, 4950 in corporate/cloud firewall |
| Login failure | Password mismatch | Check default admin / admin123!. If changed, request reset from administrator |
| 403 Forbidden | Insufficient user permissions | Request role verification from administrator |
| Session expired | Auto logout after 30 minutes of inactivity | Log in again |
Data Collection Issues
Data collection path: OPC Server → Message Broker (Kafka/MQTT) → Engine Pipeline → Aster TSDB. If any point in this path is blocked, data will not be collected.
Symptom: Data not being collected
| Inspection Item | Verification Method |
|---|---|
| OPC connection status | Check CONNECTED in Web Console > Connection Management > Status |
| Engine status | Check RUNNING in Console > Monitoring > System Status |
| Pipeline | Reception stopped if MPS (Messages Per Second) is 0 |
| Message Broker | Enter ./shell.sh then verify Kafka topics via /opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node topic |
| Network | Ping / telnet from OPC Server → Platform Host IP |
Symptom: Data Latency
| Cause | Resolution |
|---|---|
| Pipeline queue congestion | Increase engine.pipeline.threads in properties |
| Aster TSDB write latency | Check pd node compactionstats, monitor disk I/O |
| Kafka lag | Verify consumer lag via pd node topic |
| Network latency | Measure RTT between OPC Server ↔ Platform Host |
Performance Issues
Symptom: Slow Console / API Response
# 호스트에서 컨테이너 자원 사용량
docker stats --no-stream
# 앱 컨테이너의 JVM 메모리 (그 앱이 도는 컨테이너 안에서)
./shell.sh plantpulse-server-web
jmap -heap $(pgrep -f plantpulse-server)
Since the six apps run in separate containers, app processes will not be found inside the data lake. Identify the target container via ./status.sh then enter via ./shell.sh <컨테이너>.
| Cause | Resolution |
|---|---|
| Insufficient container memory | Increase limit for the container then ./restart.sh — Data Lake is DOCKER_DATALAKE_MEMORY, Apps are DOCKER_SERVER_MEMORY, etc. (Environment Variables) |
| Insufficient JVM memory | Adjust heap size in setenv inside the container (Details in Performance Tuning) |
| Frequent GC | Analyze GC logs, tune G1GC options |
| DB slow query | Check PostgreSQL slow query logs |
| Host CPU saturation | Check top / htop then consider increasing DOCKER_PP_CPUS |
Symptom: OutOfMemoryError
# 컨테이너 내부에서 heap dump 활성화
./shell.sh
# setenv 또는 JAVA_TOOL_OPTIONS 에 -XX:+HeapDumpOnOutOfMemoryError 추가
# 생성된 hprof 를 호스트로 복사
docker cp plantpulse-datalake:/path/to/heap.hprof /tmp/
Analyze using Eclipse MAT / VisualVM.
Alarm Issues
| Symptom | Cause | Resolution |
|---|---|---|
| Alarm not triggering | Alarm setting is disabled | Check in Alarm > Advanced Alarm → Selected Alarm Setting Batch |
| Value exceeded but alarm not in list | Momentarily hidden · Planned maintenance or repair suppression · Service exclusion | Check Hide · Suppress · Service Exclusion. Hidden occurrences can be viewed via GET /api/v5/alarm/suppressed |
| Repeated alarm pushes | Suppression time is 0 | Alarm settings: Duplicate Alarm Suppression → Suppression Time (min) |
| Recovery rejected | Recovery inspection failed (value not in normal range or no value within 10 min) | Verify value then recover again, or write reason and select Recover Anyway |
| Statement storage rejected | Missing mandatory output fields per type | Mandatory Fields by Type |
| Alarm setting creation 422 (API) | Manually created ID | Use the ID received via GET /api/v5/alarm/config/new-id |
| Email notification not sent | SMTP setting error | Check mail.properties (Internal container /opt/kopens/plantpulse-platform/plantpulse-server/config/) |
| No notification sound | Browser autoplay policy | Allow site autoplay in browser settings |
UI Issues
| Symptom | Resolution |
|---|---|
| Screen glitching | Clear browser cache (Ctrl+Shift+Delete) |
| Chart not displaying | Check JS errors in Browser Developer Tools (F12) > Console |
| Real-time update disconnect | Check SSE (HTTP streaming) buffering/timeout settings in proxy/firewall |
| Dashboard load failure | Verify existence of connected tags / data sources |
| Korean character corruption | Check PP_LANG=ko, PP_TZ=Asia/Seoul in env.sh then ./restart.sh |
Object Store (RustFS) Issues
«AccessDenied» is printed endlessly in logs
[RUSTFS] level=ERROR op=GetObject err=S3Error(code: AccessDenied, message: "Access Denied")
This is not a credential issue. The metric collector of the Management API calls the old MinIO metric path (/minio/v2/metrics/cluster) every 2 seconds, but since that path does not exist in RustFS, the request falls to the S3 router. This is read as a query with bucket name minio, and because the collector does not attach a signature, it becomes an anonymous request and is rejected.
| Question | Answer |
|---|---|
| Is the data at risk? | No. A single read attempt is being rejected; it is unrelated to stored data |
| Should I fix the password? | No — this is not for the operator to handle. The requester is our collector that does not attach signatures; even if it were made to sign, the metric would still be empty (because the endpoint does not exist) |
| How often is it printed? | 2-second interval — over 40,000 lines per day |
| What to do now? | Treat this signature as «known noise» and ignore it. To prevent real errors from being buried, filter out this specific line when viewing logs |
| When will it disappear? | It will be gone in the September 2026 release — the collector has been fixed to not call non-existent endpoints — approximately 43,000 lines per day will disappear (Measured at 41: 0 lines for 36 consecutive minutes after application) |
What disappears in the next release is the log line, not the value. Offline drives · offline nodes · capacity alarms will still not receive values — this will only change when a metric path is created in RustFS. Do not interpret the silence as «fixed».
Back then, the metric path was open without authentication (MINIO_PROMETHEUS_AUTH_TYPE=public), allowing collection. With the change to RustFS, there is no corresponding setting, so those seven metrics (minio_cluster_*) no longer arrive. This is also why those values appear empty in the console.
Disk / Volume Issues
Symptom: Insufficient disk space
# 호스트 디스크 사용량
df -h
df -h /data1 # 데이터 디스크
# Docker 사용량 (이미지 / 볼륨 / 빌드 캐시)
docker system df
# 컨테이너 내부 사용량
./shell.sh
df -h
du -sh /opt/kopens/plantpulse-platform/plantpulse-storage/db/cassandra/data
| Cause | Resolution |
|---|---|
| Accumulation of old backups | Clean up old tar.gz in /data1/pp-backup/docker-volume |
| Unused Docker images | Clean up via docker system prune (images/networks/cache) |
| Aster TSDB SSTable accumulation | Run pd node cleanup, pd node compact inside the container |
| Bloated log directory | Clean up /opt/kopens/plantpulse-platform-docker/logs |
Caution:
docker volume prunedeletes all volumes not in use. Do not leave containers in a stopped state to prevent accidental deletion ofpp-*volumes.
Symptom: Volume data recovery required
Refer to the recovery procedure on the Operational Management - Backup and Recovery page.
Update / Rollback Issues
Symptom: Containers not operating normally after upgrade
The official procedure for versioning up is Rerunning the Installation One-liner. If an upgrade fails, the one-liner automatically reverts to the version immediately prior to the upgrade using the backup (UPGRADE FAILED block contains the backup location).
Identify the cause from the listed inspection items and run the same one-liner again. For manual rollbacks, use bin/upgrade-rollback.sh <백업 위치>. Do not revert by changing only the image tag via ./update.sh — operational scripts and compose will remain as the new version.
Symptom: NOT FOUND CONTAINER during update
The initial installation (./install.sh) has not been performed. Please perform the installation first.
Network / Cluster Issues
Symptom: Worker not joining Master
# 1. 워커 컨테이너 진입 (워커는 기본 스택의 서비스가 아니라 shell.sh 로는 잡히지 않습니다)
docker exec -ti plantpulse-worker-1 /bin/bash
# 워커 내부에서 마스터 IP 로 연결 테스트
ping ${PP_MASTER_IP}
# 2. 마스터에서 링 확인 — 이것이 조인의 «유일한» 증거입니다
cd /opt/kopens/plantpulse-platform-docker/bin
./shell.sh
/opt/kopens/plantpulse-platform/plantpulse-datalake-cli/bin/pd node status
# 3. JGroups / Cassandra 포트 (7000, 7001, 7800, 7801, 9042) 방화벽 허용 확인
Workers that fail to join report as normal exactly like successful workers — because the container's self-check only verifies «can I reach the Master». In a measurement on 2026-08-31, a worker where Aster TSDB was OOMKilled maintained health: starting while the ring remained at 1 node.
Always determine status via the ring list in pd node status. For adding workers, bin/worker-add.sh handles this verification → Cluster Installation
Symptom: MQTT / Kafka external client connection failure
Verify that 1883/1884 (MQTT) and 9092/9093/9094 (Kafka) are allowed in the host firewall and corporate/cloud firewalls. Refer to Port Configuration Information for a detailed port list.
- MQTT is received by
plantpulse-proxy. Equipment only needs to know this host; equipment settings do not need to be touched even if the broker moves or the name changes. 1884 is TLS passthrough, so the broker terminates the TLS. - Kafka does not go through the proxy. This is because clients reconnect to the
advertised.listenersaddress after bootstrapping — if only the bootstrap succeeds and subsequent attempts fail silently, suspect this address first.
Emergency Response
When containers are unresponsive
cd /opt/kopens/plantpulse-platform-docker/bin
# 1. 상태 확인
docker ps -a
./status.sh
# 2. 로그에서 마지막 에러 확인
./logs.sh -n 200
# 3. 안전 재시작
./restart.sh
# 4. 위 단계로 회복 안 될 경우 진단 tarball 생성
./doctor.sh
# 생성된 tarball 을 webmaster@kopens.com 으로 전달
When data corruption is suspected
# 1. 즉시 정지 (추가 손상 방지)
./down.sh
# 2. 최신 백업 확인
ls -lh /data1/pp-backup/docker-volume/
# 3. 진단 tarball 생성 (절대 데이터를 임의로 수정하지 마세요)
./doctor.sh
# 4. 기술 지원팀 연락
Prohibitions during data corruption: Do not perform direct deletion of
pd node repair,pd node cleanup, or SSTables. Incorrect recovery operations can expand the damage. Always proceed after consultation with the technical support team.
Information Required for Support Requests
If the above methods do not resolve the issue, providing the following information will enable faster analysis.
| Item | Collection Method |
|---|---|
| Diagnostic tarball | File generated after running ./doctor.sh |
| Container logs | ./logs.sh -n 500 > /tmp/container.log 2>&1 (all eight containers) |
| Logs by module | Result of ./tools/copy-log-to-local.sh (/tmp/plantpulse-log/) |
| Image version | Output of ./stack-version.sh |
| Environment variables | env.sh (passwords masked) |
| System information | OS, CPU, Memory, Disk (uname -a, free -h, df -h) |
| Error message | Exact error message / Browser console (F12) capture |
| Reproduction steps | Sequence of operations performed immediately before the issue occurred |
Technical Support: webmaster@kopens.com
Related Documents
- FAQ — Frequently Asked Questions
- Operational Management — Daily Operational Commands
- Startup Guide — Start/Stop/Restart Procedures
- Backup and Recovery — Detailed Backup Procedures
- Performance Tuning — JVM / DB Tuning
- Ports and Service Management — Port Conflict Diagnosis