Advancing AMIE towards expert-level audio-visual clinical consultations
Health & Bioscience
Get smart on it
Google Research has advanced AMIE, a medical AI system, to conduct real-time video consultations by perceiving visual and auditory cues that are lost in text-only interfaces, such as a patient's gait, breathing, and visible signs of distress. This capability matters because physicians rely on these non-verbal signals for diagnosis, patient trust, and clinical communication, and a text-based system cannot independently observe them or guide patients through physical examination maneuvers. AMIE (Video) uses three specialized agents working in parallel to balance natural conversational speed with clinical reasoning and continuous processing of audio and visual information. In a randomized study with patient actors and board-certified physicians, AMIE (Video) demonstrated expert-level performance in real-time clinical video consultations.
Related news
Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3
In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepar
Meta’s ‘open’ AI, and a $250M deal gone very wrong
Meta released Glimmer this week, an open-weight AI model anyone can download and run on their own hardware — a contrast to Muse Spark, the company’s more powerful model that stays locked behind its own APIs. The release landed alongside a letter from Mark Zuckerberg arguing AI should be “for everyone” rather than controlled by a handful of labs, but as Equity’s […]
