Surgical video annotation: phases, instruments, and action labels

What can be annotated in surgical footage?

Annotation can describe an entire case, a time interval, a single frame, or a spatial region. These levels answer different questions and have different review costs. Choose the smallest amount of detail that supports the intended experiment, then test whether it is sufficient.

The CAMMA: CholecT50 dataset documentation uses instrument, verb, and target labels as a concrete example of action annotation. Its vocabulary is specific to its task and collection. Reusing a published label set still requires checking whether it fits your procedures and intended use.

What belongs in the annotation guide?

A label name is not a definition. Explain what observable evidence supports each label, provide positive and negative examples, and show how to handle uncertainty. Include examples from difficult footage rather than only ideal frames.

• Definitions and boundaries for every label.
• Rules for partial visibility, simultaneous actions, and off-screen activity.
• An explicit unknown or not-assessable state where appropriate.
• Timestamp units, frame indexing, interval inclusivity, and video version.
• Reviewer qualifications, escalation rules, and adjudication responsibilities.

Resolve temporal conventions early. An interval expressed in seconds may not map cleanly to a variable-frame-rate export without a defined time base. A label file can be syntactically valid and still align to the wrong moment if the video has been re-edited.

How should an annotation pilot work?

Choose a representative sample and have more than one qualified reviewer assess selected segments. Compare their decisions and discuss disagreements before scaling the task. Some disagreements reveal unclear instructions; others reflect information that the video simply does not contain.

Record the decision and revise the guide when needed. If a revision changes the meaning of a label, determine which earlier work requires review. Keep guide versions connected to annotation releases, so a researcher can understand why two deliveries differ.

Measure time spent on instruction, labeling, review, and correction separately. A pilot that only counts the time spent clicking labels can underestimate the effort required to produce a usable dataset.

What does annotation quality mean?

Quality includes correct meaning, alignment, coverage, and consistency. A perfect file schema does not prove that the labels are clinically sensible. Agreement between reviewers is useful evidence, but high agreement can also reflect shared instructions that omit an important distinction.

Select measures suited to the task. For temporal labels, review boundaries and transitions. For spatial labels, examine difficult object edges and occlusions. For descriptions, assess whether claims are supported by the visible sequence. Keep clinically important error types visible instead of collapsing every issue into one average score.

What should an annotation delivery contain?

Request a machine-readable export, a label dictionary, the guide version, a mapping to media identifiers, quality review results, and known limitations. Include examples that show how to load a label and locate its corresponding frame or interval.

Define correction and versioning rules. Researchers should be able to tell whether a new release adds cases, changes labels, or fixes timestamps. Operavue can scope clinical annotation and expert evaluation alongside a data inquiry, with the actual deliverables agreed before production work.