Vector Temporal Shift

EigenLake compares two time windows and finds semantic shifts in the records inside each window. Use it for questions like “what changed this week?”, “which support issues are growing?”, or “what new failure modes appeared?”.

Temporal shift runs topic discovery separately for the baseline and current windows, aligns similar topics, and ranks new, growing, shrinking, drifted, and restructured patterns.

Run temporal shift

result = idx.search.temporal_shift(
    baseline={
        "start": "2026-06-01T00:00:00Z",
        "end": "2026-06-08T00:00:00Z",
    },
    current={
        "start": "2026-06-08T00:00:00Z",
        "end": "2026-06-15T00:00:00Z",
    },
    timestamp_field="event_timestamp",
    filter={"status": {"$eq": "open"}},
    group_by=["product"],
    limit_per_window=10_000,
    min_clusters=2,
    max_clusters=20,
    text_fields=["subject", "description"],
    metadata_fields=["priority", "region"],
    summary_mode="deterministic",
)

for shift in result["shifts"]:
    print(shift["shift_id"], shift["kind"], shift["direction"], shift["score"])
    print(shift["label"])
    print(shift["explanation"])

The API uses timestamp_field to select records in each window. filter applies to both windows. group_by runs independent comparisons per metadata group, which is useful for products, regions, tenants, or services.

Shift kinds

Each shift has:

• kind: high-level category such as emerging, growing, shrinking, drift, or restructured.
• direction: new, up, down, shifted, or restructured.
• score: ranking score for the shift.
• label: human-readable phrase for the pattern.
• explanation: deterministic or LLM-generated summary.
• baseline_topics and current_topics: the topics involved in the shift.
• signals: raw count, prevalence, similarity, and drift signals.

Growing and shrinking shifts use both relative and absolute count thresholds:

result = idx.search.temporal_shift(
    baseline={"start": "...", "end": "..."},
    current={"start": "...", "end": "..."},
    min_relative_shift=0.25,
    min_count_shift=5,
)

Optional LLM summaries

result = idx.search.temporal_shift(
    baseline={"start": "...", "end": "..."},
    current={"start": "...", "end": "..."},
    text_fields=["subject", "description"],
    summary_mode="llm",
)

LLM summaries are a presentation layer. The underlying windows, topics, alignments, and shift scores are calculated first. If summary generation is not available, the API falls back to deterministic labels and explanations.

Workload limits

• Default limit_per_window: 10,000 records
• Default top_n: 50 returned shifts
• Minimum useful data: several valid embeddings in each window
• Distance: cosine similarity over existing vectors
• Default client timeout: 240 seconds

For long-running production jobs, deploy the temporal shift Lambda and configure the API backend to invoke it.