Digg Live

Tables

Contracts

Blob outputs

Docs

Edges

Default view.

Source

X List Posts API

GET /2/lists/{id}/tweets

6 fields0 indexes

Control

x_lists

poll configuration and cursor state

11 fields1 indexes

X Posts Raw Incoming Data

Incoming data fields directly from X API.

shovel/x/tweet-data.yaml

X API fields

Digg post metadata

Digg app owned fields

31 fields0 indexes

Content

posts

first-class List posts

data/x/posts/utc/{created_at_utc_date}/{id}.md

Postgres fields

31 fields4 indexes

Include

referenced_posts

quoted, replied-to, retweeted, edited

Postgres fields

20 fields1 indexes

Include

users

authors and expanded users

18 fields1 indexes

Include

media

expanded media attachments

13 fields1 indexes

Include

polls

expanded poll attachments

8 fields1 indexes

Derived

post_urls

canonicalized links

10 fields3 indexes

Audit

poll_runs

workflow run audit

11 fields2 indexes

Audit

poll_events

timeline of poll activity

5 fields1 indexes

Blob

post markdown

Vercel Blob export

data/x/posts/utc/{created_at_utc_date}/{id}.md

5 fields0 indexes

Doc

data-model.md

shovel/x

shovel/x/data-model.md

2 fields0 indexes

X Posts Raw Incoming Data

Paths

shovel/x/tweet-data.yaml

Fields

Mirrored X post fields

id (string)
Unique identifier of the post, returned by X. Stored as a string so its 64-bit value never loses precision when parsed as a number.
text (string)
The post's UTF-8 body text as returned by X, capped at 280 characters. Content longer than that is delivered separately in the note_tweet field.
created_at (string)
ISO 8601 / RFC 3339 UTC timestamp of when the post was published on X. Every derived date field and output path is computed from this value.
author_id (string)
X user ID of the account that authored the post. Resolves to a full user object through the author_id expansion.
conversation_id (string)
ID of the post that started the thread this one belongs to. It equals the post's own id when the post begins a new conversation.
note_tweet (any)
Long-form content payload for posts that exceed 280 characters (X Premium long posts), carrying the extended text and its own parsed entities.
in_reply_to_user_id (string)
X user ID of the account this post is replying to. Present only when the post is itself a reply.
referenced_tweets (any)
Other posts this one points at, each given as a type (replied_to, quoted, or retweeted) plus the referenced post ID.
attachments (any)
References to attached media and polls by media_keys and poll_ids; the full media and poll objects arrive through expansions in separate tables.
context_annotations (any)
Topic and interest labels X's classifiers infer from the post, each pairing a domain (broad category) with an entity (the specific subject).
entities (any)
Elements parsed out of the text — URLs, hashtags, cashtags, mentions, and annotations — each with its character offsets in the post.
public_metrics (any)
Engagement counts visible to everyone: replies, reposts, likes, quotes, bookmarks, and impressions.
possibly_sensitive (boolean)
Boolean flag indicating the post may contain sensitive media or links that some audiences should be warned about before viewing.
lang (string)
BCP-47 language tag X detected for the post text (e.g. "en"), or "und" when no language could be determined.
geo (any)
Location attached to the post, containing a place_id and, when the author shared it, precise coordinates.
reply_settings (string)
Who is allowed to reply to the post — everyone, mentionedUsers, following, subscribers, or verified.
withheld (any)
Country-withholding metadata present when the content is blocked in certain jurisdictions, including copyright, the affected country_codes, and scope.

Digg app owned fields

status (enum(active | deleted | withheld))app
App-managed lifecycle state of the stored post — active, withheld, or deleted — derived from X availability and takedown handling.
x_list_id (string)app
ID of the X List whose poll first caused this post to be ingested.
list_ids (string[])app
Every X List in which this post has since been observed, accumulated across polls.
created_at_utc_date (string)app
UTC calendar date (YYYY-MM-DD) derived from created_at, used to partition generated Blob output stably.
created_at_pt (string)app
Pacific-time timestamp derived from created_at, for reasoning about the local publication day.
created_at_pt_date (string)app
Pacific calendar date (YYYY-MM-DD) derived from created_at, supporting 24-hour PT-day queries.
first_seen_at (string)app
Timestamp of the first time the app ingested and stored this row.
last_seen_at (string)app
Timestamp of the most recent poll in which the app observed this post.
blob_pathname (string)app
Deterministic path of the post's generated markdown file in Vercel Blob.
blob_url (string)app
Public Vercel Blob URL of the post's generated markdown file.
blob_etag (string)app
ETag returned by Vercel Blob for the uploaded markdown, used to detect changes between runs.
blob_uploaded_at (string)app
Timestamp of the last successful markdown upload to Vercel Blob.
content_hash (string)app
Hash of the rendered markdown, compared on each run to skip rewriting unchanged Blob output.
raw (any)app
Complete raw X post object as returned, preserved for forward compatibility and future backfills.

Indexes

No indexes.

Relationships

Defines the mirrored X and Digg app-owned field contracts implemented by posts.
Defines the tweet-shaped subset implemented by referenced_posts.
Excludes polling, pagination, cursor, retry, and endpoint transport state.

Notes

Raw-only candidate X fields are tracked in the YAML coverage section, not counted as stored columns.