Database Model
This document explains where and in what shape the AI stack stores data. There are four stores, each with a different role. For operating procedures, see Database Management.
PostgreSQL and Aster TSDB are shared with the PlantPulse Platform installation. The AI stack does not spin up its own DB. This means "AI backup" and "Platform backup" overlap — if you don't sort out which side covers what, you end up with gaps that neither side covers.
| Store | Owner | Holds |
|---|---|---|
PostgreSQL pi · schema co | AI (separate DB inside the platform PG) | Chat · incidents · settings · MCP (Copilot) |
PostgreSQL pi · schema dt | AI (〃) | External system connector registry · bindings · action log (Digital Thread) |
PostgreSQL lightrag | AI (〃) | LightRAG's document processing status (PGDocStatusStorage) |
PostgreSQL rag_ingest | AI (〃) | Status of the document ingestion pipeline (rag-api) — see "PostgreSQL rag_ingest" below |
PostgreSQL litellm | AI (〃) | Keys and usage of the LiteLLM gateway |
Aster TSDB pi | AI | Feature store · anomaly detection outputs |
Aster TSDB pp | Platform (read-only) | Raw sensors · alarms · OEE |
Neo4j plantpulse-neo4j | AI | LightRAG knowledge graph |
Neo4j plantpulse-digital-thread-neo4j | AI | Mirror of the Digital Thread's factory graph — reference only |
| Qdrant | AI | Document embedding vectors |
lightrag-storage volume | AI | LightRAG's KV cache — local JSON files |
lightrag-inputs volume | AI | Uploaded source documents |
AI does not run its own PostgreSQL; it uses the Platform's PostgreSQL. Inside it, four
DBs — pi · lightrag · rag_ingest · litellm — are created (idempotently, by provision-platform-db.sh) and
used. pi is a DB shared by the AI apps, with one schema per app
(co Copilot · dt Digital Thread). DB ch from 2026.09 and earlier is moved to
pi.co at startup → Database Management. Aster TSDB · Valkey ·
RustFS · the MQTT broker (equipment alarm subscription) are likewise attached and used
from the platform's instance — do not size these four separately when sizing an AI box.
PostgreSQL pi.co — Conversations and operational state
This is schema co of the separate database pi inside the platform PostgreSQL
(jdbc:postgresql://<platform-host>:5432/pi?currentSchema=co). Copilot connects seeing only this schema, so any table name without a schema
prefix below refers to co.
ch from 2026.09 and earlierThese are tables that used to live in public of DB ch. Names and shapes are unchanged
— only the location moved to pi.co.
| Group | Tables |
|---|---|
| Conversations | chats · chat_messages · prompts · prompt_history — chats.context holds context pinned to a conversation (2026.10 re-release) |
| Chat behavior · answer quality (2026.10 re-release) | chat_action (one row per draft suggestion delivered in a chat answer — who approved it, when, and why) · chat_turn_metric (per answered turn: latency · tokens · model · citation count · evidence type · 👍👎, plus the outcome flag outcome — empty for a normal answer, no_answer for an empty answer even after retry, loop_cut to break a loop) · chat_bench_run (benchmark run scores) |
| Users | users · user_roles · employees · web_push_subscription — employees.work_role is the work role an admin assigns (approval authority); employees.home_role is the start screen a user used to pick in the old UI (2026.10 re-release 3 — not used by the new UI) → Work Roles |
Incidents (table names remain incident) | incident · incident_event · assignees |
| Watches (condition monitoring — a removed feature) | watch · watch_run. Not created or used from the UI replacement release onward. Tables from older installs remain as-is |
| Anomaly detection ops | anomaly_mute · hourly_anomaly · asset_tag_profile |
| MCP · tools | agent_tool_call_log — if a human approved the action, the approver is recorded in approved_by. The old tables mcp_server · mcp_tool_pref · mcp_tool_call_log are dropped at startup in 2026.10 re-release (2026-10-20) — the source of truth for servers · on/off · call logs is the MCP Gateway DB (the gw schema in pi) |
| Public API | api_key |
| Settings | config · config_audit_log · metadata |
The "2026.10 re-release" tables and columns above are created by Copilot at startup via
CREATE TABLE IF NOT EXISTS · ADD COLUMN IF NOT EXISTS. There is no migration to run by hand. chat_turn_metric does not hold the raw
question/answer text (that's in chat_messages), and chat_action does not hold the voice or answer body
— only tool arguments and result indices.
agent_tool_call_logEvery single tool call AI makes is recorded here — without exception — timestamp ·
user · session · tool name · source MCP server · arguments (secrets masked with ***) ·
result (ok · error · denied) · elapsed time. denied is a change-type tool call blocked
by an approval gate → MCP. mcp_tool_call_log only has success/duration
stats per registered MCP server, so "who did what" is found in this table.
api_keyKeys are never stored in plaintext — only the hash (key_hash) and a recognizable prefix
(key_prefix) are kept. Revoking does not delete the row; it stamps revoked_at. An empty last_used_at
means the key is unused → Copilot Admin Console
Before the 2026-07 restructuring, Aster TSDB also held conversation history. Now
chat_messages is the single source. Don't rely on old docs and look for it in Aster TSDB.
PostgreSQL pi.dt — Digital Thread registry
A schema owned by the Digital Thread API. Tables are created when the service starts.
Every statement is scoped to dt., so it never touches other schemas.
| Group | Tables |
|---|---|
| Connectors | dt_connector · dt_connector_tool |
| Bindings | dt_binding · dt_binding_log (operator-defined column bindings and their change history — who, when, before/after) |
| Action log | dt_action_log (drafts/requests sent to external systems after approval) |
| Episodes | dt_episode · dt_episode_label |
| Identity declarations | dt_identity · dt_identity_log (an operator's declaration that "key A and key B are the same thing at this site," and its change history — never attached automatically) |
Credentials in dt_connector are stored sealed with PI_DT_SECRET_KEY. If you lose this key, restoring the DB
doesn't help — the stored credentials become unusable and must be re-entered per
connector. Make sure to include this when backing up the host env file
→ Digital Thread
PostgreSQL rag_ingest — Document ingestion status
A DB owned by rag-api. Tables are created when rag-api starts (idempotently).
| Group | Tables | When created |
|---|---|---|
| Sync | rag_asset_doc_sync · rag_sync_run · rag_asset_card | Always |
| Drawings | rag_drawing_doc · rag_drawing_chunk · rag_drawing_ident · rag_drawing_dim · rag_drawing_kg · rag_drawing_outbox | Only when DRAWING_INGEST_ENABLED=true |
| Video · audio | rag_media_job · rag_media_segment · rag_media_sidecar | Only on installs with video/audio enabled (PI_MEDIA_ENABLED=true — called MEDIA_INGEST_ENABLED=true inside rag-api) |
rag_drawing_doc.sourcetracks where a drawing came from — either uploaded via the admin console (MANUAL) or inserted by the sync that pulls in platform asset documents (PLATFORM, 2026.10 re-release 3). The latter is the source of truth from the platform, so the console does not overwrite or delete it.rag_drawing_dimholds, for each dimension line, the upper/lower bounds (upper_limit·lower_limit) and their basis (limits_basis— whether it's a value written on the drawing, or a "formula estimate" computed from a fit formula).rag_media_jobis one row per video/audio file (sourcesource· processing status · retry count · thumbnail count, etc.). The RAG › Index › Video screen in the admin console shows this → Video · Audio Search
The structure/previews of the original drawing and the video transcript are not stored
in tables but in the object storage bucket plantpulse-rag →
Database Management
Aster TSDB pi — Feature store
12 tables owned and created by the Time Series Insight service. The prefix is tsi_.
| Group | Tables |
|---|---|
| Features | tsi_tag_feature_hourly · tsi_tag_feature_daily · tsi_asset_feature_hourly · tsi_asset_feature_daily |
| Context/profile | tsi_asset_context_daily · tsi_asset_profile |
| Anomalies/failures | tsi_anomaly_embedding · tsi_failure_signature · tsi_precursor_alert |
| Pipeline | tsi_pipeline_status · tsi_pipeline_history · tsi_backfill_status |
Tables are created idempotently by the service at startup. However, the keyspace
pi itself is not created by the app — that's handled by the installation script
(provision-platform-db.sh).
The anomaly detection and prediction screens in the admin console read exclusively from these tables. If results are empty, check the pipeline first, not the model → Pipeline Status / Run History in the admin console (AI Admin Console).
Copilot also keeps its own tables in pi (e.g., morning cards). In the 2026.10
re-release, PI_NAVIGATOR_ACTION_CHECK was added to hold action-check status for structured reports, created
by Copilot at startup.
pp.ti_* no longer existsOn 2026-07-30, it was moved from keyspace pp to pi, and the old table was
dropped. Querying under the old name returns nothing.
Aster TSDB pp — Platform source (read-only)
AI only reads data collected by the platform. It never writes here.
| Table | Use |
|---|---|
tm_asset_data_based_second / _minute / _hour | Equipment sensor time series |
tm_tag_point | Tag points |
tm_asset_alarm | Alarms |
tm_kpi_oee_history | OEE |
tm_kpi_ram_history · tm_kpi_ems_history | RAM · EMS |
tm_monitor_platform_status | System resources (CPU · memory · disk) |
Partition keys differ by resolution
second : PRIMARY KEY ((asset_id, date, hour), timestamp)
minute : PRIMARY KEY ((asset_id, date), timestamp)
hour : PRIMARY KEY (asset_id, timestamp)
kpi : PRIMARY KEY ((asset_id, day), timestamp) -- tm_kpi_*_history
The day of KPI history (tm_kpi_*_history) is a UTC date (yyyymmdd). A local calendar day can
span two UTC buckets, so you need to expand the period into a list of UTC dates and read
each bucket. A row is a cumulative order count up to that time, so rows must not be
summed. See the detailed rules in
KPI Tables.
At second-level resolution, the partition includes down to equipment · date · time.
Scanning a long period at second resolution touches thousands of partitions. Use _hour
for wide periods, and _second only for narrow windows.
The full schema on the platform side is documented in Platform Database Model.
Neo4j · Qdrant — Knowledge graph and vectors
| Store | Holds |
|---|---|
Neo4j (plantpulse-neo4j) | LightRAG's knowledge graph — entities/relations extracted from documents |
| Qdrant | Embedding vectors for document chunks |
The Digital Thread's factory graph mirror lives in a separate Neo4j instance
(plantpulse-digital-thread-neo4j). It is not LightRAG's store, so it is not among the "four" below.
The RAG engine's (LightRAG's) storage is split across four places. If you only know about two of them, recovery will go wrong.
| What | Where | Config |
|---|---|---|
| Vectors | Qdrant | LIGHTRAG_VECTOR_STORAGE=QdrantVectorDBStorage |
| Graph | Neo4j | LIGHTRAG_GRAPH_STORAGE=Neo4JStorage |
| KV cache | Local JSON in the lightrag-storage volume | LIGHTRAG_KV_STORAGE not configured — LightRAG default (JsonKVStorage) |
| Document processing status | lightrag DB in the platform PG | LIGHTRAG_DOC_STATUS_STORAGE=PGDocStatusStorage |
The KV cache being local JSON is intentional — it's enough for a single instance and
avoids depending on the platform's Valkey. When scaling to multiple instances, switch to
LIGHTRAG_KV_STORAGE=RedisKVStorage + REDIS_URI.
Only the document processing status lives in PostgreSQL. If you restore only PG and leave Qdrant/Neo4j empty, LightRAG thinks the documents are "already ingested" and will not re-ingest them — searches will keep coming back empty. Conversely, if you restore only the graph/vectors, the state mismatch causes duplicate ingestion.
When restoring, either align all four to the same point in time, or empty all four and
re-ingest from the source. The source lives in the lightrag-inputs volume — this volume is
effectively the backup, and if you lose it too, there's no way to regenerate the index.
See Understanding Ontology for ontology concepts.
Backup scope summary
| Store | Who must back it up | If missing |
|---|---|---|
PostgreSQL pi (co · dt) | AI side must cover this | Conversations, incidents, settings, and the connector registry are lost |
PostgreSQL lightrag · rag_ingest | AI side must cover this | Document processing status is lost — see the danger box above |
PostgreSQL litellm | AI | Gateway key/usage history is lost |
Aster TSDB pi | AI side must cover this | Features are lost and analysis screens go blank (recomputable, but slow) |
Aster TSDB pp | Platform backup | — |
| Neo4j · Qdrant | AI | Document index is lost (regenerable from source) |
| Digital Thread Neo4j | AI (bin/backup.sh, whole volume) | The mirror is lost (rebuildable from the source) |
lightrag-storage volume | AI | KV cache is lost (regenerable) |
lightrag-inputs volume | AI — the most important | Source documents are lost and there's no way to regenerate the index |
See Database Management for procedures.
Related documents
- Database Management — access, inspection, operation
- AI Admin Console — checking pipeline status
- Service Port Map
- Platform Database Model