AWS is demonstrating how edge-to-edge cloud operations with its AWS IoT Greengrass open-source framework, in tandem with NVIDIA’s Holoscan platform, are enabling sub-20ms clinical decision support. This refers to processing at the edge where operating room data is generated, with cloud infrastructure providing scalable model training on anonymized data.

This architecture addresses the interoperability standards that 98% of healthcare respondents said would deliver meaningful improvements, according to AWS, because it  unlocks the surgical video data that has remained trapped in the “black box” of the OR.

The goal is to show how interoperable AI infrastructure may be deployed to advance surgical intelligence at scale.

Over 300 million surgeries are performed globally every year, the company said. While remarkable human skill is required in an OR, it also means that different surgeons doing the same procedure in the same hospital, may potentially have very different approaches and outcomes.

A recent Johnson and Johnson report found that 95% of surgeon respondents said better surgical software would help them improve care, yet, the infrastructure to deliver it remains fragmented. Because video is increasingly capturing procedures, there is a significant  opportunity to unlock clinical insights from this data, AWS said.

Just one minute of HD surgical video contains 25 times the data of a CT scan — creating massive storage and transmission demands –- and most healthcare facilities operate on legacy infrastructure with insufficient bandwidth and limited throughput, where network congestion delays critical data transmission, the company said.

With a growing ecosystem of participating healthcare and technology leaders, the initiative represents one of the first efforts to create a scalable foundation for AI in surgery — from real-time intelligence in the OR to research, model development, and future clinical applications, according to AWS.

The process starts with transforming raw video into structured clinical knowledge during the surgical recognition phase. AI classifies which step of a procedure is underway in real time — systems have achieved 93–95% accuracy in procedures such as sleeve gastrectomy, with the ability to detect missing or unexpected steps, AWS said. Knowing the phase transforms how the surgical team is supported — it enables real-time workflow analytics, training feedback, and creates the temporal context that makes outcome correlation meaningful. A recording is only video, but a recording tagged by phase is a structured dataset, AWS noted.

Instrument detection is designed to close the loop between what tools were used, how, and with what outcome. Real-time identification and tracking supports intraoperative awareness, instrument counts, and safety checks — combined video and sensor approaches have achieved 90.3% detection rates, with recent multi-tool tracking systems demonstrating real-time inference at state-of-the-art performance, AWS said. Beyond the procedure itself, instrument tracking feeds surgical training, quality review, and outcome correlation that identifies which tool patterns lead to better results.

The architecture is organized into three layers that form a continuous loop: the edge layer in the OR, where real-time inference happens; the cloud layer where models are trained and improved; and the bridge layer that is designed to connect them securely and manages the fleet at scale. The company said each layer has a distinct role, and the value of the system comes from how they work together.