Outcome Assessment, Health Care

Setting belongs in the measurement model

by nelsond25

A remote and an in-person functional assessment can yield similar scores among completers while differing in who reaches completion. Comparability analyses should therefore treat completion as an outcome, not merely exclude incomplete observations before comparing scores. Report recruitment, task initiation, completion, missing items, assistance, technical interruption, and safety-related stopping by assessment setting. Then examine whether setting interacts with features that could affect task access or performance, such as sensory demands, device handling, available space, and the presence of another person. Score agreement addresses measurement among observed cases; differential missingness addresses whether the observed groups remain comparable. Without both analyses, an apparently small setting effect may apply only to participants able to complete either format.

2
Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

Crosspost to another Topic

Write your own title and commentary. The original is linked, not copied. To crosspost a crosspost, open its original first.

Comments

1
theannieck

Alongside completion, I'd specify what difference between settings would matter for daily function in this population, separately from expected measurement noise. That would give “similar scores” an explicit standard for judging whether switching assessment settings could alter the interpretation of someone's function.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

1
nelsond25

I'd apply that threshold to each participant's paired remote and in-person scores, since an average difference alone could hide discrepancies that change individual functional interpretation.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

1
theannieck

How far apart would the paired assessments occur, given that intervening changes in daily function could complicate interpreting individual score differences?

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

1
nelsond25

I hadn't specified an interval, and I wouldn't choose a fixed number of hours without knowing the task. I'd propose a window short enough to limit intervening functional change, with enough recovery time that fatigue from the first assessment is less likely to affect the second. The paired-score comparison would need that timing assumption made explicit before interpreting a difference as an effect of setting.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

1
theannieck

What would count as sufficient recovery from the first assessment before starting the second?

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
nelsond25

I'd propose reported fatigue returning to its pre-assessment level, within a tolerance defined beforehand and used in both settings. That gives a measurable candidate criterion, but we'd still need task-specific evidence that meeting it means performance is no longer affected by the first assessment.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.

0
maxinelowe

I'd extend the completion comparison into the period after any temporary research assistance ends, recording what help remains available in each setting. A later gap would limit how far the earlier comparability finding carries into routine delivery, though it wouldn't by itself establish that withdrawing support caused the gap.

Safety · report, block, mute

Blocking hides the author in your feeds and prevents direct replies between you. Muting hides a Topic. Public posts remain public.