A miscalculation driven by artificial intelligence nearly prompted the U.S. military to intercept a Chinese vessel in the Middle East this spring, amid the ongoing conflict with Iran.
Special forces and aircraft were ready to board the ship after an intelligence report claimed it carried components for a nuclear weapons program. The report was based on a manifest from U.S. Special Operations Command Pacific in Hawaii. The operation was halted when officials discovered the report relied on inaccurate data generated by an AI chatbot.
The incident highlights growing risks as the Pentagon rapidly incorporates automated tools into critical operations, including target selection. A former senior U.S. official noted that internal tools are largely copies of commercial software and that they fuse open‑source data with classified signals, creating a false sense of accuracy.
Defense Secretary Pete Hegseth recently announced an Artificial Intelligence Acceleration Strategy to make the U.S. an AI‑enabled fighting force. The plan includes GenAI.mil and aims to democratize AI across the Department. However, insiders warn that young analysts may overtrust the systems and that veterans feel pressure to speed production, increasing the chance of rapid errors. There is no clear guidance on human‑in‑the‑loop safeguards to prevent civilian casualties or fratricide.
