tkg API

Temporal Knowledge Graph (TKG)¶

TKG is a time-aligned knowledge graph built from multi-processor outputs (face, yolo, ocr, pose, asrx, gaze, lip, appearance). It produces 9 node types and 14 edge types stored in dev.tkg_nodes and dev.tkg_edges.

Node naming convention: All trace types use _track suffix. Text uses _region (non-temporal).

See also: docs_v1.0/DESIGN/TKG_FORMATION_V1.0.md for formation phases, flow diagrams, and query examples.

Node Types¶

Node Type	External ID Format	Description	Key Properties
`face_track`	`trace_{trace_id}`	A tracked face identity over time	`trace_id`, `frame_count`, `status`, `avg_bbox`, `avg_yaw`, `avg_pitch`, `avg_roll`, `start_frame`, `end_frame`, `pose_count`
`gaze_track`	`gaze_track_{id}`	Gaze direction over time	`direction` (frontal/left/right/up/down + diagonals)
`lip_track`	`lip_track_{id}`	Lip movement synced with speech	`speaker_id`, `lip_area_range`
`text_region`	`text_region_{id}`	Spoken text aligned to time	`speaker_id`, `text`, `start_time`, `end_time`
`appearance_trace`	`appearance_{trace_id}`	Human appearance (clothing) over time	`clothing_color`, `upper_cloth`, `lower_cloth`
`accessory`	`accessory_{id}`	Detected accessories	`type` (glasses/hat/etc.), `confidence`
`object`	`object_{class}_{id}`	YOLO-detected object	`class`, `confidence`, `frame_count`
`speaker`	`speaker_{speaker_id}`	ASRX speaker segment	`speaker_id`, `segment_count`, `total_duration`

Identity Agent Integration (face_track nodes)¶

Identity Agent marks face_track nodes with identity binding status.

face_track Status Values¶

Status	Description	Properties
`pending`	No identity suggestion	Default state
`suggested`	Identity Agent suggested	`pending_identity_name`, `pending_identity_uuid`, `suggested_by`, `confidence`
`confirmed`	User confirmed binding	`identity_uuid`, `identity_id`, `identity_ref`, `identity_name`
`stranger`	Stranger cluster member	`stranger_id`, `stranger_ref`

Suggested By Values¶

Value	Description
`tmdb`	TMDb seed matched
`propagation`	Confirmed trace propagation
`manual`	User manual selection

Example face_track Node¶

{
  "node_type": "face_track",
  "external_id": "face_track_1",
  "label": "Face Track 1",
  "properties": {
    "trace_id": 1,
    "frame_count": 45,
    "start_frame": 100,
    "end_frame": 300,
    "avg_bbox": {"x": 100, "y": 200, "width": 80, "height": 100},
    "status": "suggested",
    "pending_identity_name": "Tom Hanks",
    "pending_identity_uuid": "xxx-xxx",
    "suggested_by": "tmdb",
    "confidence": 0.91
  }
}

Edge Types¶

Edge Type	Storage Name	Source → Target	Description
`co_occurs`	`CO_OCCURS_WITH`	object ↔ object	Two objects appear together in same frame
`speaker_face`	`SPEAKS_AS`	speaker → face_track	Speaker matched to face track via lip sync
`face_face`	`INTERACTS_WITH`	face_track ↔ face_track	Two face tracks interact (mutual gaze)
`mutual_gaze`	`MUTUAL_GAZE`	gaze_track ↔ gaze_track	Two people looking at each other
`lip_sync`	`LIP_SYNC`	lip_track → text_region	Lip movement aligned with spoken text
`has_appearance`	`HAS_APPEARANCE`	face_track → appearance_trace	Face has specific appearance
`wears`	`WEARS`	face_track → accessory	Face wears an accessory
`hand_object`	`HOLDS`	hand → object	Hand holding object

`POST /api/v1/file/:file_uuid/tkg/rebuild`¶

Auth: Required Scope: file-level

Rebuild the Temporal Knowledge Graph for a file. Reads processor JSON outputs (face, yolo, ocr, pose, asrx, gaze, lip, appearance) and generates TKG nodes and edges. Clears existing nodes/edges for the file first, then rebuilds from scratch.

Example¶

curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/rebuild" \
  -H "X-API-Key: $KEY"

Response (200)¶

{
  "success": true,
  "file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
  "result": {
    "face_track_nodes": 16,
    "gaze_track_nodes": 16,
    "lip_track_nodes": 12,
    "text_region_nodes": 24,
    "appearance_trace_nodes": 8,
    "skin_tone_trace_nodes": 5,
    "accessory_nodes": 3,
    "object_nodes": 26,
    "speaker_nodes": 4,
    "co_occurrence_edges": 94,
    "speaker_face_edges": 12,
    "face_face_edges": 8,
    "mutual_gaze_edges": 2,
    "lip_sync_edges": 10,
    "has_appearance_edges": 16,
    "wears_edges": 3
  },
  "error": null
}

Field	Type	Description
`success`	boolean	True if rebuild completed
`file_uuid`	string	32-char hex UUID
`result`	object	Node and edge counts by type
`error`	string/null	Error message if failed

`POST /api/v1/file/:file_uuid/tkg/nodes`¶

Auth: Required Scope: file-level

Query TKG nodes with pagination and optional type filter.

Request Parameters¶

Field	Type	Required	Default	Description
`node_type`	string	No	all	Filter by node type: `face_track`, `gaze_track`, `lip_track`, `text_region`, `appearance_trace`, `skin_tone_trace`, `accessory`, `object`, `speaker`
`page`	integer	No	1	Page number
`page_size`	integer	No	100	Items per page (max 500)

Example¶

# Get all face_track nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
  -H "X-API-Key: $KEY" \
  -H "Content-Type: application/json" \
  -d '{"node_type": "face_track", "page": 1, "page_size": 50}'

# Get all nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
  -H "X-API-Key: $KEY" \
  -H "Content-Type: application/json" \
  -d '{}'

Response (200)¶

{
  "success": true,
  "file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
  "total": 16,
  "page": 1,
  "page_size": 50,
  "nodes": [
    {
      "id": 1,
      "node_type": "face_track",
      "external_id": "face_track_0",
      "label": "Face Track 0",
      "properties": {
        "trace_id": 0,
        "frame_count": 142,
        "avg_confidence": 0.87,
        "best_confidence": 0.92,
        "avg_capture_quality": 0.61,
        "best_capture_quality": 0.78,
        "key_frame": 120,
        "key_face": "tkg:face/0:point_xyz",
        "avg_bbox": {"x": 200, "y": 150, "width": 60, "height": 60},
        "best_bbox": {"x": 210, "y": 155, "width": 62, "height": 60}
      }
    }
  ]
}

Field	Type	Description
`success`	boolean	Always true on 200
`file_uuid`	string	32-char hex UUID
`total`	integer	Total matching node count
`page`	integer	Current page
`page_size`	integer	Items per page
`nodes`	array	Array of node objects
`nodes[].id`	integer	Database primary key
`nodes[].node_type`	string	Node type (see table above)
`nodes[].external_id`	string	External identifier (e.g., `trace_0`, `gaze_1`)
`nodes[].label`	string	Human-readable label
`nodes[].properties`	object	Type-specific properties as JSON

`POST /api/v1/file/:file_uuid/tkg/edges`¶

Auth: Required Scope: file-level

Query TKG edges with pagination and optional filters.

Request Parameters¶

Field	Type	Required	Default	Description
`edge_type`	string	No	all	Filter by edge type: `co_occurs`, `speaker_face`, `face_face`, `mutual_gaze`, `lip_sync`, `has_appearance`, `wears`
`source_type`	string	No	—	Filter by source node type
`target_type`	string	No	—	Filter by target node type
`page`	integer	No	1	Page number
`page_size`	integer	No	100	Items per page (max 500)

Example¶

# Get all co_occurs edges
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
  -H "X-API-Key: $KEY" \
  -H "Content-Type: application/json" \
  -d '{"edge_type": "co_occurs"}'

# Get edges between face_track and speaker nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
  -H "X-API-Key: $KEY" \
  -H "Content-Type: application/json" \
  -d '{"source_type": "speaker", "target_type": "face_track"}'

Response (200)¶

{
  "success": true,
  "file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
  "total": 94,
  "page": 1,
  "page_size": 100,
  "edges": [
    {
      "id": 1,
      "edge_type": "co_occurs",
      "source_node_id": 10,
      "target_node_id": 15,
      "properties": {
        "frame_count": 45,
        "confidence": 0.92
      }
    }
  ]
}

Field	Type	Description
`success`	boolean	Always true on 200
`file_uuid`	string	32-char hex UUID
`total`	integer	Total matching edge count
`page`	integer	Current page
`page_size`	integer	Items per page
`edges`	array	Array of edge objects
`edges[].id`	integer	Database primary key
`edges[].edge_type`	string	Edge type
`edges[].source_node_id`	integer	Source node ID (FK to tkg_nodes)
`edges[].target_node_id`	integer	Target node ID (FK to tkg_nodes)
`edges[].properties`	object	Edge-specific properties as JSON

`GET /api/v1/file/:file_uuid/tkg/node/:node_id`¶

Auth: Required Scope: file-level

Get detail for a specific TKG node including its connected edges.

Example¶

curl -s "$API/api/v1/file/$FILE_UUID/tkg/node/1" \
  -H "X-API-Key: $KEY"

Response (200)¶

{
  "success": true,
  "node": {
    "id": 1,
    "node_type": "face_track",
    "external_id": "face_track_0",
    "label": "Face Track 0",
    "properties": {
      "trace_id": 0,
      "frame_count": 142,
      "avg_confidence": 0.87
    }
  },
  "connected_edges": [
    {
      "id": 5,
      "edge_type": "co_occurs",
      "source_node_id": 1,
      "target_node_id": 10,
      "properties": {"frame_count": 45}
    }
  ],
  "edge_count": 3
}

Field	Type	Description
`success`	boolean	Always true on 200
`node`	object	Node detail (same format as nodes query)
`connected_edges`	array	Edges connected to this node
`edge_count`	integer	Total connected edge count

Error Codes¶

HTTP	When
`404`	Node not found

`GET /api/v1/file/:file_uuid/processor-counts`¶

Auth: Required Scope: file-level

Get counts of processor JSON output files for a file. Scans the output directory for {file_uuid}.{processor}.json files and extracts frame counts, segment counts, and chunk counts from each file.

Supports short UUID prefix matching (e.g., d3f9ae8e → resolves to full d3f9ae8e471a1fc4d47022c66091b920).

Example¶

curl -s "$API/api/v1/file/$FILE_UUID/processor-counts" \
  -H "X-API-Key: $KEY"

Response (200)¶

{
  "file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
  "output_dir": "/Users/accusys/momentry/output_dev",
  "processors": [
    {
      "processor": "cut",
      "has_json": true,
      "frame_count": 5391,
      "segment_count": null,
      "chunk_count": null,
      "last_modified": "2026-06-16T18:48:01.987241061+00:00"
    },
    {
      "processor": "face",
      "has_json": true,
      "frame_count": 1112,
      "segment_count": null,
      "chunk_count": null,
      "last_modified": "2026-06-18T17:21:37.408383765+00:00"
    },
    {
      "processor": "asrx",
      "has_json": true,
      "frame_count": null,
      "segment_count": 6,
      "chunk_count": null,
      "last_modified": "2026-06-18T17:21:40.872063642+00:00"
    },
    {
      "processor": "story",
      "has_json": true,
      "frame_count": null,
      "segment_count": null,
      "chunk_count": 12,
      "last_modified": "2026-06-18T17:22:00.000000000+00:00"
    },
    {
      "processor": "mediapipe",
      "has_json": false,
      "frame_count": null,
      "segment_count": null,
      "chunk_count": null,
      "last_modified": null
    }
  ]
}

Field	Type	Description
`file_uuid`	string	Full 32-char hex UUID (resolved from prefix)
`output_dir`	string	Output directory scanned
`processors`	array	Per-processor output info
`processors[].processor`	string	Processor name
`processors[].has_json`	boolean	Whether JSON file exists
`processors[].frame_count`	integer/null	Total frames processed (frame-based processors)
`processors[].segment_count`	integer/null	Segment count (ASRX segments, etc.)
`processors[].chunk_count`	integer/null	Chunk count (Story chunks, etc.)
`processors[].last_modified`	string/null	ISO 8601 timestamp of last modification

Error Codes¶

HTTP	When
`404`	File UUID not found in database

Trace Management¶

Endpoints for managing face traces: list, delete, restore, and merge.

`DELETE /api/v1/file/:file_uuid/trace/:trace_id`¶

Auth: Required

Soft-delete a face trace (default) or hard-delete with {"hard_delete": true}.

Soft delete marks Qdrant points with status: "deleted" and TKG nodes with status: "deleted" in properties. Deleted traces are excluded from the traces list.

Hard delete permanently removes Qdrant points and TKG nodes.

Request Body (optional):

Field	Type	Default	Description
`hard_delete`	boolean	`false`	Permanently delete instead of marking

Example:

# Soft delete
curl -X DELETE "$API/api/v1/file/$FILE_UUID/trace/8" \
  -H "X-API-Key: $KEY" -H "Content-Type: application/json" -d '{}'

# Hard delete
curl -X DELETE "$API/api/v1/file/$FILE_UUID/trace/8" \
  -H "X-API-Key: $KEY" -H "Content-Type: application/json" \
  -d '{"hard_delete": true}'

Response:

{
  "success": true,
  "file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
  "trace_id": 8,
  "hard_delete": false,
  "qdrant_marked": true,
  "tkg_nodes_marked": 1
}

`POST /api/v1/file/:file_uuid/trace/:trace_id/restore`¶

Auth: Required

Undo a soft-deleted trace. Clears status: "deleted" from Qdrant points and TKG node properties.

Example:

curl -X POST "$API/api/v1/file/$FILE_UUID/trace/8/restore" \
  -H "X-API-Key: $KEY"

Response:

{
  "success": true,
  "file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
  "trace_id": 8,
  "qdrant_restored": true,
  "tkg_nodes_restored": 1
}

`POST /api/v1/file/:file_uuid/trace/:source_trace_id/merge/:target_trace_id`¶

Auth: Required

Merge all face points from source trace into target trace. Updates Qdrant trace_id and deletes source TKG node.

Example:

curl -X POST "$API/api/v1/file/$FILE_UUID/trace/16/merge/3" \
  -H "X-API-Key: $KEY"

Response:

{
  "success": true,
  "file_uuid": "9f6a9cd55a5809f977f5a6589b9045c5",
  "source_trace_id": 16,
  "target_trace_id": 3,
  "points_moved": 58,
  "tkg_nodes_deleted": 1
}

Updated: 2026-07-21 01:00:00 Updated: 2026-08-12 — face_track node properties now include capture_quality fields (avg/best) and key_frame/key_face are selected by best capture quality

Version History¶

Version	Date	Changes
1.0	2026-10-24	Initial documentation

tkg API

Temporal Knowledge Graph (TKG)¶

Node Types¶

Identity Agent Integration (face_track nodes)¶

face_track Status Values¶

Suggested By Values¶

Example face_track Node¶

Edge Types¶

POST /api/v1/file/:file_uuid/tkg/rebuild¶

Example¶

Response (200)¶

POST /api/v1/file/:file_uuid/tkg/nodes¶

Request Parameters¶

Example¶

Response (200)¶

POST /api/v1/file/:file_uuid/tkg/edges¶

Request Parameters¶

Example¶

Response (200)¶

GET /api/v1/file/:file_uuid/tkg/node/:node_id¶

Example¶

Response (200)¶

Error Codes¶

GET /api/v1/file/:file_uuid/processor-counts¶

Example¶

Response (200)¶

Error Codes¶

Trace Management¶

DELETE /api/v1/file/:file_uuid/trace/:trace_id¶

POST /api/v1/file/:file_uuid/trace/:trace_id/restore¶

POST /api/v1/file/:file_uuid/trace/:source_trace_id/merge/:target_trace_id¶

Version History¶

`POST /api/v1/file/:file_uuid/tkg/rebuild`¶

`POST /api/v1/file/:file_uuid/tkg/nodes`¶

`POST /api/v1/file/:file_uuid/tkg/edges`¶

`GET /api/v1/file/:file_uuid/tkg/node/:node_id`¶

`GET /api/v1/file/:file_uuid/processor-counts`¶

`DELETE /api/v1/file/:file_uuid/trace/:trace_id`¶

`POST /api/v1/file/:file_uuid/trace/:trace_id/restore`¶

`POST /api/v1/file/:file_uuid/trace/:source_trace_id/merge/:target_trace_id`¶