Skip to content

Monitoring ​

Purpose: every executed unit of work in DDL is a process — a SQL query, an import or sync, a transform, an export, a transfer, an upload, an action, a backup, a publish. Monitoring is where you find out whether it ran, how long it took, and what it said when it failed. It is the entry point for "why did this sync fail?".

DDL offers the same log in two scopes:

ViewWhereExtra column
Global/monitoring (profile menu → Monitoring)PROJECT NAME — spans every project you can see
ProjectProject sidebar → Monitoring (/projects/<id>/monitoring)— (the project is implicit; the browser tab title becomes DDL - <project>)

The list ​

Global MonitoringGlobal view: PROJECT NAME · STATUS · PROCESS TYPE · NODE NAME · PROCESS START · PROCESS DURATION — here with both successful and failed SQL Query rows.

Project MonitoringThe same data in project scope, without the project column.

Project Monitoring with runsSQL Query runs recorded against the project.

ColumnNotes
STATUSColour-coded: Pending / Running / Success / Failed / Upstream Failed
PROCESS TYPEAction / Backup / Export / Import / Publish / SQL Query / Transfer / Transform / Upload File
NODE NAMEThe node the process belongs to. Empty for SQL runs launched from the editor — those are not tied to a node
PROCESS STARTRendered as relative time ("15 minutes ago")
PROCESS DURATIONSub-second runs display 0 seconds
Last cellView Log

An account with no runs yet shows "There is no process to show".


Filter to what you need ​

The filter bar is identical in both views.

The filter barStatus filter, process-type filter, a start and end datetime range (both labelled GMT+00:00), and Reset Filter .

  1. Open the status filter.

Status filter openThe status dropdown.

  1. Its exact options:

    Status optionsPending · Running · Success · Failed · Upstream Failed.

  2. Open the process type filter.

    Type filter openThe type dropdown.

    Its exact options:

    Type optionsAction · Backup · Export · Import · Publish · SQL Query · Transfer · Transform · Upload File.

  3. Narrow the window with the two datetime-local range inputs, and clear everything with Reset Filter.

"Upstream Failed" is a hint, not a mystery

It means the process never got to run because something it depends on failed. Start at the upstream node's own failed row — the Lineage view tells you which node that is.


Read a process log ​

  1. Click View Log on a row (or click the row itself).

Process log drawerThe drawer: process metadata at the top, Download Log , and the technical log below.

  1. Read the metadata block:

    FieldExample
    Processthe process UUID
    Process TypeSql Query
    Process StatusSuccess
    Node Name- when the run is not tied to a node
    Process Start / Process Endrelative times
    Process Duration0 seconds for sub-second runs
  2. Read the technical log — "Complete technical details for further investigation" — a terminal block with timestamped lines:

    text
    2026-11-18 05:28:47,245329 | root/INFO | Starting sql query e1945b89-71ab-4068-ac1f-bd46759dc92e (PID: 4437f375-bfcd-4190-8fef-26823d4938ad)
    2026-11-18 05:28:47,508883 | root/INFO | Sql query e1945b89-71ab-4068-ac1f-bd46759dc92e is completed successfully
  3. Use Download Log to attach the full log to an incident ticket, and copy the URL to share the exact view.

The log drawer is deep-linkable

Opening a log rewrites the URL to /monitoring?process_id=<uuid>&page=1. Reload it, or send it to a colleague with access, and the same drawer opens on the same process. This is the best way to ask a question about a failed run.


Diagnose a failed run (worked example) ​

A failed SQL run is the clearest case, because the error tells you exactly what went wrong:

text
psycopg2.errors.UndefinedTable: relation "demo_orders" does not exist
utils.exceptions.QueryError: Query fails: …
  1. Find the Failed row in Monitoring (status filter → Failed).
  2. Open View Log and read the last line of the technical log.
  3. Match the error to its cause:
Error in the logCauseFix
relation "…" does not existTable name was not written as a back-quoted node nameUse `node_name` — see SQL Query
relation "schema.node" does not existSchema-qualified nameDrop the prefix
Import/transfer failure with a connection nameCredentials, permissions or network reachabilityTest the connection — see Connections
Upstream FailedA node upstream did not succeedFix the upstream node first
  1. Re-run: there is no retry button inside the log drawer. Re-run the query from the SQL editor, or use Sync now on the node — Pipelines (ETL).

Behaviour worth knowing ​

Monitoring is eventually consistent

Right after a Sync now, the list may still read "There is no process to show". Wait and refresh rather than launching the run again.

Only your visible work appears here

Monitoring lists runs made by/for you within the projects you can access. Instance-wide totals are elsewhere: Settings → Storage Usage, Settings → Process Usage and — for AI — Settings → AI → AI Ops. A number that looks too big usually means you are looking at an aggregate page, not at Monitoring.

Timestamps are relative here, absolute elsewhere

Monitoring renders "15 minutes ago" in both the list and the drawer; the absolute timestamp only appears inside the log body. Storage Usage and Versions use absolute times. If you are correlating a run with an external log, take the time from the technical log lines.


Next: AI & MCP integration · Related: Pipelines (ETL), SQL Query, FAQ

Delman Data Lab — user guide for the DDL web application at ddl.delman.io.