AI product manager who builds · safety, security & developer tools

I research how AI systems behave.I build the tools to test them.

Product work on AI systems that need to be trusted. I build and publish research: open questions turned into experiments, working tools and product decisions, with the evidence and its limits left in view.

Same ten items, a different mix.

In the published fixed-ten-item comparison, mixed-relevance context improved reported relevance-score accuracy within the studied corpus.

Concept illustration, not a photograph or product screenshot.

01 · Context research

Context-Window Research

Public preprint and research harness · hover or focus a step to show it on the monitor

QUESTION

Can less-relevant context help?

Same ten items in every prompt; only the mix changes. Can less-relevant items help a model tell what matters?

STUDY

An inspectable research harness

The published study reports 2,420 confirmatory trials across 11 model configurations using 661 anonymized workplace items scored for relevance. The main comparison holds the prompt at ten items and changes its composition: high-relevance items only versus a mixture of high- and same-domain low-relevance items.

FINDING

Judge context against the task

In the paper's all-11 paired comparison, the 50:50 condition improved relevance-score accuracy by 0.077 relative to the all-high-relevance condition (n = 220 paired comparisons). Those cells are not independent. The paper also reports a nine-model-family aggregation and explicitly treats it as a within-corpus descriptive comparison, not population inference.

Within-corpus findings, not a universal rule for context or model quality. One corpus and task family. Descriptive, a public preprint: not a universal rule or independent replication.

Scope and source notes
  • Links and reported comparison refer only to public preprint v1; within-corpus descriptive findings, not population inference or later assurance completion. Private corpus and unreleased packages remain excluded.
  • Version 1 only; paired cells are not independent. Within-corpus descriptive comparison, not population inference, universal context advice, or completion of later human calibration/assurance.
01 · CONTEXTFINDING
All high relevance
50:50 mix

Within-corpus findings.

Composition comparison.

Prompt layout; reported finding in the text.

02 · Decisions

Quorum

Released VS Code extension

QUESTION

A recommendation isn't the whole answer.

How can a developer inspect several model perspectives without losing the disagreement behind the recommendation?

COUNCIL

A council inside VS Code

3–5 models deliberate in roles; high disagreement triggers extra critique rounds. The memo: recommendation, dissent, confidence, evidence. Follow-ups stay in the thread.

MEMO

Keep the reasoning inspectable

Public example: a five-role council recommends extracting the payment module first and deferring a full rewrite until evidence justifies it, as advisory guidance, not approval.

Installable software; decision-quality gains and sustained external use are not established. A released extension with source to inspect. It does not establish better decisions, adoption or independent validation. A related paper is in preparation.

Scope and source notes
  • Availability does not establish decision-quality improvement, active external use, retention, or commercial success.
  • Quorum public repository: Published VS Code extension with a Marketplace installation path. No adoption or decision-quality claim inferred.
02 · QUORUMCOUNCIL
Question→ ChairArchitectTesterRisk→ Memo ↻ high disagreement → extra critique round

3–5 models per panel.

How a council run works.

03 · Agent safety

Intent-monitor

Research prototype

QUESTION

Explanations aren't evidence.

What can recorded actions tell us that an agent's stated plan alone cannot?

COMPARISON

The plan versus the tool ledger

Compares reasoning-trace claims with recorded tool calls, against text-only monitoring and a tool-name blocklist. Benign, licensed cases test false positives.

FINDING

Action evidence, not motives

A reassuring explanation isn't independent evidence of what an agent did. The ledger shows what ran, not why.

Constructed examples; real-world effectiveness is not established. Hand-built fixtures and a default regex extractor: a mechanism demonstration, not validated detection, a production safety product or a guarantee.

Scope and source notes
  • Constructed examples and a default regex extractor; not a naturalistic catch rate, true-intent inference, or production safety guarantee.
03 · INTENTCOMPARISON
Stated planTool ledger read✓readtest✓test —⚑write

Constructed example.

Stated plan versus recorded actions.

04 · Agent controls

Convention-Gate

Public process-control mechanism

QUESTION

Move the check out of the prompt.

Which part of a process requirement can be checked at the action boundary, rather than left inside a prompt?

MECHANISM

Registered state, checked by a hook

An MCP server records prerequisite state; under valid, enabled configuration a Git pre-commit hook checks the required PASS registrations before a commit.

BOUNDARY

A precise promise

An instruction to follow a process is not an external check of registered state.

Registered prerequisites are not proof that a review happened or covered the current changes. A PASS is self-attestation: it doesn't prove a review happened, was good, or covered this change. Bypassed, disabled or unmediated paths aren't governed.

Scope and source notes
  • Registered state, not authenticated review or review quality; valid enabled mediated path only, with bypass/configuration limits.
  • Convention-Gate public repository: Supports the bounded registered-state mechanism. Broad historical README/study interpretations are not repeated as universal or isolated causal claims.
04 · GATEMECHANISM
reviewPASStestsmissing

pre-commit hook → blocked

Illustrative prerequisites.

Registered prerequisites checked at commit.

What I conclude

Four conclusions, each with its limit.

  1. 01
    Judge context against the task, not by window size.Context-Window Research
  2. 02
    Keep the disagreement inspectable, not just the polished answer.Quorum
  3. 03
    Treat recorded actions, not explanations, as evidence.Intent-monitor
  4. 04
    Put checks at the action boundary, and say what they don't prove.Convention-Gate

All four are public: a preprint and harness, an installable extension, and two inspectable repositories.

A way of working

Questions first.
Tools with a purpose.

I'm interested in the gap between what an AI system can demonstrate and what the evidence actually supports.

As a product manager, that means making the question precise, building something inspectable, and keeping the limits close to the result.

More about Dvir ↗

Continue the conversation

Something worth exploring?

Research questions, product problems, and thoughtful collaborations are welcome.

Let's talk on LinkedIn