plantpulse-storage (Storage Layer)
Role
This layer permanently stores all PlantPulse data. It bundles four specialized engines into a single module, each handling a different data type. pd It is the first module to start in the boot sequence. If it fails, no subsequent services are attempted — it acts as the gatekeeper.
| Item | Value |
|---|---|
| Module Name | plantpulse-storage |
| Install Path | /opt/kopens/plantpulse-platform/plantpulse-storage/ |
| Data Location | /data1/pp-data/ (PP_DATA_DIR, Volume pp-data) |
| Backup Location | /data1/pp-backup/ (PP_BACKUP_DIR, Volume pp-backup) |
pd Service | storage — MASTER · WORKER (RustFS is MASTER only) |
| Boot Sequence Order | valkey → postgres → cassandra → rustfs |
Configuration
Directory Structure
plantpulse-storage/
├── cache/valkey/ # conf/valkey.conf (생성물) · 로그는 /data1/pp-data/valkey/logs
├── db/
│ ├── cassandra/ # bin/ (cqlsh · nodetool) · conf/cassandra.yaml · jvm-server.options (생성물) · logs/
│ └── postgres/ # bin/ (psql · pg_ctl) · conf/postgresql.conf · pg_hba.conf (생성물) · logs/
├── object/
│ ├── rustfs/ # bin/ · lib/(rustfs · rc) · logs/
├── graph/janusgraph/ # 그래프 DB — 이미지에 있지만 pd 가 기동하지 않습니다
└── tools/reaper/ # Cassandra Reaper — 이미지에 있지만 pd 가 기동하지 않습니다
conf/ The files below are all generated artifacts. The place to edit is the host's /etc/kopens/conf/*.template → Changing Settings
Each Engine
Aster TSDB — Time Series · Events
| Item | Value |
|---|---|
| Port | 9042 (CQL) · 7000/7001 (Inter-node) · 7099 (JMX) |
| Keyspace | pp (PP_KEYSPACE) |
| Account | cassandra (PP_CASSANDRA_USER) / PP_CASSANDRA_PASSWORD — PasswordAuthenticator |
| Configuration | cassandra.yaml.template · jvm-server.options.template (WORKER uses the cluster/ version) |
| Advertised Address | broadcast_rpc_address — if 127.0.0.1, the render will reject it |
Key tables: pp.ts_data_points(Time Series Engine data points, TTL per row), pp.tm_tag_point · tm_tag_point_archive · tm_tag_point_snapshot(tags), pp.tm_asset_data(assets), pp.tw_*(records for items moved to the cold tier). Per-table TTL, compression strategy, and size are shown in pd retention.
docker exec plantpulse-datalake pd node status # 링 상태 — UN 이 정상
docker exec plantpulse-datalake pd node cql # cqlsh
docker exec plantpulse-datalake pd node errors # 최근 WARN/ERROR
docker exec plantpulse-datalake pd node compactionstats
docker exec plantpulse-datalake pd node repair # 무겁습니다
docker exec plantpulse-datalake pd node cleanup # 파괴적 — 스냅샷 삭제
| Tuning Point | Template | Recommended |
|---|---|---|
| Heap | jvm-server.options.template | 1/4 of host memory, max 32GB |
compaction_throughput | cassandra.yaml.template | 128 MB/s (NVMe) / 16 MB/s (HDD) |
concurrent_compactors | cassandra.yaml.template | min(코어 수, 8) |
Aster TSDB Reaper is included in the tools/reaper/ image, but it is a manual tool that pd does not start. Regular repairs are done via pd node repair.
PostgreSQL — Metadata
| Item | Value |
|---|---|
| Port | 5432 |
| Databases | pp (PP_DB_NAME) — plus temporal · temporal_visibility · kestra · hive-metastore-230 |
| Account | plantpulse / PP_PG_PASSWORD. Roles temporal · hive · gravitino · kestra · replica are also here |
| Authentication | scram-sha-256 (pg_hba.conf.template) |
| Configuration | postgresql.conf.template · pg_hba.conf.template |
At initial boot, the schema and roles are created without passwords. Then pd aligns the role passwords with its sidecar value (idempotent on every boot — ALTER ROLE). So for PostgreSQL accounts that rotation tools don't handle, like PP_GRAVITINO_PASSWORD · PP_KESTRA_DB_PASSWORD, you simply modify the sidecar and restart.
docker exec -it plantpulse-datalake pd node psql # postgres OS 사용자로 psql
docker exec plantpulse-datalake pd storage # WAL · 복제 슬롯 · max_slot_wal_keep_size
docker exec plantpulse-datalake pd backup # 논리 덤프
| Tuning Point | Recommended |
|---|---|
shared_buffers | 25% of host memory |
effective_cache_size | 50 ~ 75% |
max_connections | 200 (500 for large deployments) |
max_slot_wal_keep_size | Set a cap — prevents slots from removed workers from holding onto WAL indefinitely (default in the image is 8GB) |
Valkey — In-memory Cache
| Item | Value |
|---|---|
| Port | 6379 (Plaintext) · 6380 (TLS) |
| Account | redis (PP_REDIS_USER) / PP_REDIS_PASSWORD — requirepass |
| Configuration | valkey.conf.template (WORKER uses the cluster/ version — replicaof master) |
| Log | /data1/pp-data/valkey/logs/system.log — located under the data volume |
docker exec -it plantpulse-datalake bash -c '"$PP_HOME/tools/valkey/bin/valkey-cli" -h 127.0.0.1 -a "$PP_REDIS_PASSWORD"'
> INFO memory
> CONFIG GET maxmemory
RustFS — Object Storage
| Item | Value |
|---|---|
| Port | 9000 (S3 API — monitored by pd) · 9001 (Console — the human-facing screen) |
| Account | rustfs (PP_S3_USER) / PP_S3_PASSWORD — injected via startup env RUSTFS_ACCESS_KEY·RUSTFS_SECRET_KEY |
| Default Buckets | plantpulse (PP_S3_BUCKET). Iceberg data, backups, attachments |
| Console | http://<server-ip>:9001/ — login uses the same credentials as the account above |
RustFS runs on the MASTER node only. The client rc(rustfs/cli) is included in the image. The startup script registers the pp-s3 alias and creates buckets if they don't exist.
Integrated Operation
docker exec plantpulse-datalake pd status storage # 넷의 UP/DOWN
docker exec plantpulse-datalake pd restart storage # 넷 전부 — 의존하는 모든 서비스가 연결을 잃습니다
docker exec plantpulse-datalake pd storage # 볼륨 · WAL · 슬롯 · 스냅샷 · 크기
storage disconnects everything aboveWhenever possible, restart the entire data lake using the host's restart-datalake.sh. This also determines dependency application states → Start · Stop · Restart
Backup / Recovery
| Engine | Tool | Method |
|---|---|---|
| PostgreSQL | pgBackRest (physical) · pg_dump (logical) | full / diff · point-in-time recovery |
| Aster TSDB | Medusa | full / diff snapshots |
| RustFS | rc mirror | External S3 replication |
Procedures are detailed in Backup · Restore.
Common Issues
| Symptom | Cause | Action |
|---|---|---|
Aster TSDB WriteTimeout | Disk I/O saturation | pd node compactionstats, adjust throughput |
| Aster TSDB node down | Network · Disk · OOM | pd logs storage GC pause, host dmesg |
PostgreSQL Too many connections | Connection pool leak | increase max_connections + check the application pool |
| WAL keeps growing | Replication slots from removed workers | pd storage — slot.orphan → host ops-check.sh --reclaim |
| Valkey out of memory | maxmemory-policy not set | allkeys-lru or volatile-lru |
| RustFS disk full | Accumulated Iceberg · backups | Set lifecycle policy in console 9001 |
| Slow response after restart | Aster TSDB warmup | Normal (3 ~ 5 minutes) |
Recommended Disk Layout
/data1/
├── pp-data/ # 데이터 볼륨 (NVMe SSD)
│ ├── cassandra/ # SSTable · commitlog · saved_caches
│ ├── postgres/ # data · dump
│ ├── rustfs/
│ ├── valkey/
│ └── kafka/
├── pp-temp/
└── pp-backup/ # pgbackrest · medusa
Separating the Aster TSDB commitlog and data onto separate disks (or separate LVs) significantly improves write throughput.