Development and Assessment of Real-Time Visual Recognition System for Scrub Nurse Robot

Development and Assessment of Real-Time Visual Recognition System for Scrub Nurse Robot

&nbsp;&nbsp;We have been developing the Scrub Nurse Robot (SNR) for endoscopic and laparoscopic surgery. The aim of this study was to develop a real-time visual recognition system (RTVRS) in order for the second version of SNR to be capable of recognizing surgeon&quot;s actions during exchange of surgical instruments.<BR>&nbsp;&nbsp;The RTVRS consisted of 4 digital video cameras, a commercially-available 3D position tracking system (QuickMag Ⅳ), and two PCs. One PC had an action-recognizing algorithm and a timed-automata-based model of surgeon&quot;s actions, both of which we had developed. The other PC had a motion controller of SNR. The algorithm can recognize the surgeon&quot;s actions in real time from the physical movements of the surgeon with three differently-colored makers each on his right wrist, right elbow and the center of his chest. The time series positional data of those markers were obtained in real time with QuickMag Ⅳ. When the action-recognizing algorithm in RTVRS detects the moment that the surgeon starts drawing a surgical instrument after use from the operative field, a signal is produced and sent to the motion controller of SNR. Then it drives SNR to hold out its hand with an instrument needed next toward the surgeon.<BR>&nbsp;&nbsp;We tested if the RTVRS worked properly in a laboratory, and also analyzed how a real surgeon and human scrub nurses behaved in clinical cases. As a result, we confirmed that the RTVRS-driven SNR was capable of giving a surgeon&quot;s stand-in a surgical instrument with satisfactory timing (without making him wait for the instrument) by recognizing his two key actions during exchange of instruments: drawing a surgical instrument from and inserting another into the operative field. However, if the current SNR is used in the clinical cases where a surgical tray is used instead of the tool changer, a counterpart in the SNR system, the SNR will make real surgeons wait slightly longer than the human scrub nurses. Therefore, in order to achieve the clinical use of the SNR, design and mechanism of the device to return and store surgical instruments should be investigated and also the data processing rather than the machine movement in the RTVRS-driven SNR needs to be sped up.