Challenging AI Outputs#

AI makes it sound right; you make sure it is right. The strongest users treat AI outputs as preliminary analysis requiring adversarial review, not finished work product. Below are prompts to help you systematically challenge AI-generated content.

For example, you might start with a prompt like:

"List all of the evidence that tends to support the allegation of improper expense reporting in violation of the expense policy."

After the AI generates its response, it's your job as an investigator to challenge, verify, and strengthen. Below are quality control prompts to help you systematically challenge AI-generated content.

Where to use these prompts: Copy and paste them into the Report drafting page or into the Search bars on the Documents and Interviews pages.

PROMPTS#

Unused Evidence Audit#

Review all uploaded materials. List every document or statement NOT cited in your analysis. For each, explain specifically why it's not relevant to the findings.

Single-Source Vulnerability Check#

Identify every material fact supported by only one source. For each: (a) why wasn't corroboration available? (b) what factors affect that source's reliability? (c) how would conclusions change if that source were deemed unreliable?

Timeline Gap Analysis#

Create a timeline of all events mentioned in this analysis. Identify gaps of [specify threshold]. What potentially significant events could have occurred during those gaps? What evidence addresses those periods?

Opposing Counsel Perspective#

You are defense counsel for [respondent/subject]. Based solely on this evidence, construct the three strongest arguments that would challenge these findings. For each argument, identify the specific evidence you would emphasize and which gaps or ambiguities you would exploit.

Exculpatory Evidence Review#

Identify all evidence that could be viewed as exculpatory or mitigating for [respondent/subject]. How is that evidence addressed in the current analysis? What's the strongest interpretation of events that incorporates this evidence?

Assumption Interrogation#

List every assumption embedded in this analysis regarding: (a) witness credibility, (b) timeline sequencing, (c) motive or intent, (d) interpretation of ambiguous communications. For each assumption, identify what evidence would disprove it and whether such evidence exists in the file.

Inference Chain Mapping#

For [specific conclusion], work backward to identify each inferential step required. Which steps are supported by direct evidence versus circumstantial evidence versus assumption? Where does the reasoning depend on "more likely than not" judgments?

Conflicting Evidence Reconciliation#

Identify all instances where evidence conflicts or where different sources provide inconsistent accounts. How was each conflict resolved? What's the alternative interpretation if the conflict were resolved differently?

Unasked Questions#

Based on the findings in this report, generate a list of follow-up questions that a thorough investigation would pursue. For each question, identify: (a) what additional evidence would be needed, (b) whether that evidence is reasonably available, (c) why the investigation stopped short of answering it.

Witness Coverage Gap Analysis#

Identify all individuals mentioned in the evidence who were not interviewed or did not provide statements. For each person, explain: (a) what relevant knowledge they likely possess, (b) why they weren't interviewed, (c) how their absence affects confidence in the findings.

Confidence Calibration#

Rate confidence in each major finding as HIGH (direct evidence, multiple sources, no material conflicts), MEDIUM (circumstantial evidence, single source, or minor conflicts), or QUALIFIED (inferential reasoning, credibility judgments, or significant gaps). For medium and qualified findings, explain the limitations.

Alternate Conclusion Test#

Could a reasonable investigator reviewing this same evidence reach a materially different conclusion? If yes, describe that alternate conclusion and identify which evidence would need to be weighted differently or interpreted differently to support it.

Adversarial Review#

You are an external reviewer hired to challenge this report. Identify the three most significant vulnerabilities: evidentiary gaps, unsupported inferences, unaddressed contradictions, or procedural deficiencies. How would you address each vulnerability if you were defending this work product?

---

Pick 3-5 prompts and build them into your workflow.