Questions
What each provider was asked
Jev and Claude answer the same narrow questions, rendered from build_questions() in relay/decisions/questions.py. Code, not the model, turns the answers into the five decisions. Every text on these pages is exported from that code, and each hash is checked against the hash the committed traces record.
Question sets and prompts
12 questions
Jev · 2 runs
sha256:b395531673a5…
first set12 questions
Jev · 5 runs · code default
sha256:89717c795a2d…
19 questions
Jev · 6 runs · latest adopted
sha256:4589d78b40d2…
12 questions
Claude · 4 runs · the q-v0.2 questions in one system prompt (claude-prompt-v1)
sha256:d24c74fa140c…
rules-v0.1
no prompt
rules-v0.1 sends no prompt. It is a deterministic provider that matches keyword patterns and fixed phrases (for example "never tried methotrexate" or "inadequate response") and checks the member-ID field; the same policy engine and thresholds then decide the action. Its patterns are described in RESULTS.md.
The questions read only the policy's indication and required therapy. immunara-v0.1 and immunara-v0.2 both name rheumatoid arthritis and methotrexate, so every set's wording and hash are identical under both. immunara-v0.2's recency rule changes how step therapy is composed, not what is asked.
How answers become decisions
- Diagnosis supporteddiagnosis_support
- p_yes of diagnosis_support, used as it is.
- Step therapy metstep_therapy
- Composed in code: P(a methotrexate course of at least 12 weeks) × p_yes of mtx_inadequate_response. The model reads date parts; code does all the date arithmetic. A month-only date becomes a range and duration is measured conservatively, latest possible start to earliest possible end; an unknown part, or an end that is not stated, adds no probability toward satisfied. Date parts are treated as independent.
- q-v0.1, q-v0.2: One course: the first start to the final end, from mtx_start_*, mtx_end_status and mtx_end_*. An ongoing course ends at the request's as_of_date.
- q-v0.3: Pause and restart segments: p = (1 − p_int) · P(first start → final end) + p_int · P(start → pause or restart → end), where p_int is p_yes of mtx_interrupted. The two segments are combined by inclusion-exclusion; an unknown pause or restart adds nothing.
Under immunara-v0.2 (max_days_since_therapy 365) the qualifying course must also be ongoing or have ended, at its earliest possible end, no more than 365 days before as_of. The rule applies on both paths; immunara-v0.1 has none.
- Documentation completedocumentation_complete
- p_yes of documentation_complete, used as it is.
- Material contradictionmaterial_contradiction
- p_yes of material_contradiction, used as it is.
- Missing evidencemissing_evidence
- The missing_evidence answer and its probability for each of the six labels.
relay/decisions/composition.py, relay/decisions/step_therapy.py
The default set: q-v0.2
DEFAULT_QUESTION_SET_VERSION in the code, and the set every Claude run used. q-v0.3 is the latest adopted set. Year options are the years found in the case.
- question_set_hash
- sha256:89717c795a2dfea7efe30e038fb483c6d4ed72d343bfbfcefeabfc6483d7a4aaunder immunara-v0.1 and immunara-v0.2 (identical)
- Recorded
- the same hash is recorded in 100 committed gold and smoke traces
01diagnosis_support
yes/no · Noul
Do the clinical documents establish a clinician-documented diagnosis of rheumatoid arthritis for this patient, as the policy requires?
- yes (true)
- A clinician states that the patient has rheumatoid arthritis as an established diagnosis.
- no (false)
- The diagnosis is absent, only suspected, pending workup, or describes someone other than the patient.
02documentation_complete
yes/no · Noul
Does this request include every item the policy lists under submission documentation?
- yes (true)
- The insurance member ID, a clinician note supporting the diagnosis, and the patient's treatment history (including whether and when methotrexate was taken) are all present (a statement that the patient never took methotrexate counts as treatment history).
- no (false)
- At least one required item is missing, unavailable, or marked as not provided.
03material_contradiction
yes/no · Noul
Do the case documents contain a material contradiction about the patient's diagnosis or treatment history that affects this authorization?
- yes (true)
- Two sources make incompatible claims about the same fact, for example one record shows a medication was taken while another says it was never tried.
- no (false)
- The sources agree, or differ only in wording, detail, or precision without conflicting.
04missing_evidence
choice · Choice
What is the most important information missing from this request for the policy decision, if any?
Options: DIAGNOSIS, TREATMENT_HISTORY, LAB_RESULT, DOSAGE, INSURANCE_INFORMATION, NONE
- DIAGNOSIS
- No established diagnosis of rheumatoid arthritis is documented.
- TREATMENT_HISTORY
- The records do not say whether or when the patient took methotrexate. A record stating that the patient never took methotrexate counts as documented treatment history.
- LAB_RESULT
- A lab result that the decision depends on is missing.
- DOSAGE
- The requested dose or regimen is missing.
- INSURANCE_INFORMATION
- The insurance member ID or plan information is missing.
- NONE
- Nothing that the policy requires is missing.
05mtx_start_month
choice · Choice
In which month did the patient (not a relative or other person) start taking methotrexate? Answer 'none' if the documents do not state the month.
Options: January … December, none
- none
- The documents do not state this, or the patient never took the medication.
06mtx_start_day
choice · Choice
On which day of the month (1-31) did the patient (not a relative or other person) start taking methotrexate? Answer 'none' if the day is not stated, for example when only a month is given.
Options: 1 … 31, none
- none
- The documents do not state this, or the patient never took the medication.
07mtx_start_year
choice · Choice
In which year did the patient (not a relative or other person) start taking methotrexate? Answer 'none' if no year is stated for this event.
Options: years found in the case, none
- none
- The documents do not state this, or the patient never took the medication.
08mtx_end_status
choice · Choice
What is the status of the patient's own methotrexate treatment (not a relative's)?
Options: ended, ongoing, not_stated
- ended
- The documents say the patient stopped taking methotrexate.
- ongoing
- The documents say the patient is still taking methotrexate.
- not_stated
- The documents do not say, or the patient never took methotrexate.
09mtx_end_month
choice · Choice
In which month did the patient (not a relative or other person) stop taking methotrexate? Answer 'none' if the documents do not state the month.
Options: January … December, none
- none
- The documents do not state this, or the patient never took the medication.
10mtx_end_day
choice · Choice
On which day of the month (1-31) did the patient (not a relative or other person) stop taking methotrexate? Answer 'none' if the day is not stated, for example when only a month is given.
Options: 1 … 31, none
- none
- The documents do not state this, or the patient never took the medication.
11mtx_end_year
choice · Choice
In which year did the patient (not a relative or other person) stop taking methotrexate? Answer 'none' if no year is stated for this event.
Options: years found in the case, none
- none
- The documents do not state this, or the patient never took the medication.
12mtx_inadequate_response
yes/no · Noul
Is it documented that the patient's own methotrexate treatment was ineffective (inadequate response) or was stopped because of intolerance or a contraindication?
- yes (true)
- A clinician documents that methotrexate did not adequately control this patient's disease, or that it caused intolerance or is contraindicated for this patient.
- no (false)
- There is no such documentation for this patient; statements about relatives or other people do not count, and neither does a different medication.