Overview
Reproducibility means that others can understand what was done, why it was done, and how the final analytic decisions were made. AI-assisted analysis should be documented with the same care as data cleaning, model selection, code development, and interpretation.
StatWiseAI automatically stores users’ prompt history and AI-generated outputs. These records support reproducibility, troubleshooting, quality review, and research evaluation. Access to stored prompt and output history is limited to the StatWiseAI research team.
Because prompt and output history is stored, users should avoid entering proprietary, sensitive, private, identifiable, HIPAA-regulated, FERPA-regulated, or otherwise restricted information.
What should be documented?
For important analytic decisions, users should document:
- What question was asked.
- What information was provided to StatWiseAI.
- What StatWiseAI recommended.
- What decision the researcher made.
- Whether the recommendation was accepted, modified, or rejected.
- What verification steps were completed.
- Who reviewed the final decision.
- Where the code and output are stored.
AI Use Log
Users may use this template to document important AI-assisted research decisions.
AI Use Log Template
Project name:
Dataset or documentation source:
Research question:
Date:
StatWiseAI task:
Prompt or prompt summary:
AI-generated recommendation:
Human decision:
Rationale for decision:
Verification steps:
Software/code used:
Reviewer or collaborator consulted:
Remaining concerns:
Location of final code/output:
Example AI Use Log entry
Project name: NHANES food insecurity and depressive symptoms analysis
Dataset or documentation source: Public NHANES documentation
Research question: Is food insecurity associated with depressive symptoms among adults?
Date: June 8, 2026
StatWiseAI task: Analysis planning
Prompt or prompt summary: Asked for guidance on relevant documentation, survey design features, candidate variables, and common mistakes.
AI-generated recommendation: Review questionnaire documentation, demographic files, depression measure documentation, food security variables, survey weights, strata, PSUs, and cycle-combination rules.
Human decision: Proceed with a survey-weighted analysis plan after confirming variable eligibility and appropriate weights.
Rationale for decision: NHANES uses complex survey design features that must be incorporated for population-level inference.
Verification steps: Review official NHANES analytic guidelines and variable documentation; consult statistician if combining cycles.
Software/code used: R or Stata, depending on analyst preference.
Reviewer or collaborator consulted: PI or Co-I.
Remaining concerns: Missingness, eligibility restrictions, and whether depression items are available for selected cycles.
Location of final code/output: Project repository or approved storage location.
Reproducibility checklist
Before relying on AI-assisted work, users should confirm:
- The research question is clearly stated.
- Dataset documentation has been reviewed.
- Variable definitions and coding decisions are documented.
- Inclusion and exclusion criteria are recorded.
- Missing data decisions are justified.
- Survey weights, clustering, repeated measures, or longitudinal features are addressed when relevant.
- AI-generated code has been reviewed and tested.
- Model assumptions are checked.
- Sensitivity analyses are considered.
- Final interpretations are based on actual output.
- AI assistance is documented.
- A human researcher has reviewed the final analytic decision.
Prompt history is helpful, but not sufficient
Stored prompt history can help researchers reconstruct how an analysis plan developed. However, prompt history alone is not enough. Users should still maintain project documentation that includes code, output, analytic decisions, assumptions, and verification steps.
Recommended practice
When using StatWiseAI for a major analytic decision, users should save or summarize the final decision in their project notes. A good documentation note should answer:
- What did StatWiseAI help with?
- What did the researcher decide?
- Why was that decision made?
- How was the decision checked?
Start Here: Responsible Use Rules
AI Basics for Researchers
Prompting for Data Analysis
Working with Dataset Documentation
Reviewing AI Outputs
Requesting Code
Reproducibility and Prompt History
Practice Use Cases: HRS and NHANES
Templates and Checklists
Return to StatWiseAI AI Literacy Tutorial Home
