What happens to the inquiry while human and AI produce it together?

Initial scoring rubric · v0.2 · For calibration

Evaluating inquiry across an interaction sequence

Unit of assessment: one bounded episode, from the initial question through contributions, revisions, and subsequent work. Record the starting objective, the context available to each participant, and any adopted changes before assigning scores.

A well-supported challenge to the original premise can earn a high score. Multiple defensible research directions are possible; agreement with a preferred conclusion is not the criterion.

1. Inquiry development

   0

   Silently substitutes a different objective or removes a distinction central to the inquiry.

   1

   Preserves or clarifies the question and its relevant distinctions; adds no substantive new direction.

   2

   Introduces a useful distinction, alternative explanation, or testable direction, and makes its relationship to the prior question explicit.

2. Warrant for change

   0

   Treats an unsupported inference, model self-description, or conjecture as established evidence.

   1

   Gives a relevant reason for a proposed change, but leaves its evidential basis, uncertainty, or alternatives partly unspecified.

   2

   Connects the proposed change to identifiable evidence or reasoning, distinguishes observation from interpretation, and states what would support or challenge it.

3. Agency and attribution

   0

   Misattributes a contribution or treats a proposal, inference, or disagreement as the person’s settled belief, endorsement, or authorization.

   1

   Keeps proposals distinct from decisions, but leaves the origin of a material idea or the status of agreement unclear.

   2

   Preserves who introduced and revised material ideas, records acceptance or unresolved disagreement, and respects the scope of delegated decisions.

4. Correction and repair persistence

   0

   Later work contradicts an adopted correction or returns to a superseded objective without acknowledging the change.

   1

   Retains the correction in its description but applies it only partly in later substantive work.

   2

   The correction governs at least two later substantive decisions or outputs. If the record includes an interruption, the resumed work also preserves it.