Testing Care Technology Beyond a Single Accuracy Score
Test reporting

An average accuracy score cannot show performance across rooms, movements and users, or explain whether anyone responds after an alert.

Samples, scenarios, device versions, uncertainty and human response belong in the same evidence record.

CHECK THE DENOMINATOR

A percentage has meaning only when its conditions are visible

Readers need the number of trials, participants, rooms, movements and retained failures. Standardised movements by healthy adults in a laboratory cannot stand in for older residents living in a real facility.

FOUR REQUIRED MEASURES

Recognition is one part of an end-to-end service

  • Sensitivity: the share of target events detected, reported by scenario.
  • Specificity: the share of normal activity not turned into an alert.
  • Availability: time in which device, network, model and delivery channel can work.
  • Response: whether the alert arrived, was acknowledged and led to completed action.
A FIELD PROTOCOL

Keep evidence by version, space and task

Write the intended use first. Then test normal activity, near misses, simulated events, multiple occupants, network loss and device failure. Record firmware, model and installation changes, and repeat representative tests after any change. A conclusion applies only to its stated conditions.

VERSION CONTROL

A result must be traceable to the system that produced it

Hardware, mounting, firmware, model, threshold and notification-app version belong in the report. After a material update, retest high-risk scenarios and recurring failures. With small samples, publish numerators, denominators and intervals, and retain unable-to-decide cases.

HOW BEIIU READS A TEST REPORT

BEIIU never reads an accuracy figure on its own. Who was tested, in which room, how events were labelled, how false alarms were counted, what happened offline and which version took part all determine whether a result can travel into a real home.

Sources