DataLake Overview
DataLake is the infrastructure layer of the PlantPulse Platform. Everything where data is stored and flows — databases, message brokers, analytics engines, time series engines, workflow engines — lives here, and above it sit applications like the web console, batch jobs, and the warehouse.
- Where are the configuration files — what survives a restart when you edit it
- How to change settings — values, node addresses, templates, memory
- Rotate passwords and API keys —
passwd.shin one command - Admin console — view status, logs, and backups in your browser
- FAQ
One container, eight services
DataLake runs as plantpulse-datalake one container on the host. Inside it, eight services start in a defined order. The authoritative order is in one file plantpulse-datalake-cli/services/order.txt inside the container.
storage → analytics → messaging → timeseries → cep → workflow → data-gateway → admin-api
| Order | Service | Contains | Ports | Purpose |
|---|---|---|---|---|
| 1 | storage | Valkey · PostgreSQL · Cassandra · MinIO | 6379 · 5432 · 9042 · 9000 | Caching · metadata · time series · object storage |
| 2 | analytics | Spark · Hive · Gravitino · Kyuubi | 7077/4440 · 9083 · 19001 · 10000 | Distributed analytics and SQL gateway |
| 3 | messaging | Kafka · HiveMQ(MQTT) | 9092 · 1883 | Field data collection and inter-module event delivery |
| 4 | timeseries | Time series engine · Grafana | 7800 · 3000 | Time series storage, query, dashboards |
| 5 | cep | CEP server | 7400 | Complex event processing (EQL) |
| 6 | workflow | Temporal · Kestra | 7233/8233 · 8380 | Long-running jobs and data pipelines |
| 7 | data-gateway | Data gateway | 5500 | External data query portal (includes SQL console) |
| 8 | admin-api | Admin console backend | 4949 · 4950 | Operations console and health API for this node |
storage and analytics are gatekeepers. If they do not start, no service behind them will even try. The rest will warn and continue if they do not start.
plantpulse-sql is deprecated (2026-09-07)The old documentation's "SQL Query Service (port 4000)" is no longer in the image. The place where you run SQL from the console is now the Data Gateway console (https://<server-ip>:5501/) → plantpulse-sql (deprecated)
Where to do what
There are three places where you work with DataLake. Once you know which place does what, the rest of this section is reference material.
| Place | Commands · screens | Task |
|---|---|---|
Host (/opt/kopens/plantpulse-platform-docker/bin/) | up.sh down.sh restart.sh restart-datalake.sh status.sh logs.sh shell.sh passwd.sh | Start and stop the container, rotate passwords, edit config files. Operator's first entry point |
Inside container (docker exec plantpulse-datalake pd …) | pd status pd doctor pd logs pd config diff pd restart <서비스> pd backup | Restart a single service, check if config rendered correctly, take backups |
Browser (https://<server-ip>:4950/) | Admin console | View status, metrics, logs, diagnostics, backups, and config diffs. Run pd verbs allowed from the command screen |
Config files inside the container (/opt/kopens/plantpulse-platform/<module>/conf/…) are all generated. Changes you make there vanish at next startup. The source of truth is a set of files under /etc/kopens/ on the host; details are in Where are the configuration files.
First 5 minutes — verify it works
All the commands below are read-only. They change nothing, so run them anytime.
# 1. 호스트에서 — 컨테이너와 헬스 요약. 종료 코드 0 = 정상, 2 = 비정상
cd /opt/kopens/plantpulse-platform-docker/bin
./status.sh
# 2. 컨테이너 안 — 서비스별 포트 표. 마지막 줄 "0 STOPPED" 면 정상
docker exec plantpulse-datalake pd status
# 3. 컨테이너 안 — 환경이 조용히 어긋난 곳은 없나. 마지막 줄 "FAIL 0" 이면 정상
docker exec plantpulse-datalake pd doctor
# 4. 브라우저 — 관리 콘솔
# https://<server-ip>:4950/?lang=ko
It is normal to see one STOPPED in pd status for a few tens of seconds after startup. Kyuubi inside analytics starts last (observed: 54 seconds). Check again after 1 minute.
This section's structure
| Page | When to read |
|---|---|
| Path map | When you ask "where is that file" — host, container, volumes, logs |
| Environment configuration | Config file locations, how to change them, variable reference |
| Credentials | Service password and API key rotation, web console login account |
| Admin console | Console access, login, screen layout |
pd CLI | Complete operations commands inside the container |
| Operations | Start, stop, restart; logs and health; backup and restore |
| Components | Storage, messaging, analytics, time series, CEP, workflow, gateway in detail |
| FAQ | Common symptoms and answers |
Related documentation
- System startup and shutdown — starting and stopping the entire stack
- Ports and services — complete list of public ports and firewall rules
- Backup and recovery — backup tools and disaster recovery scenarios
- Change initial password — commissioning checklist