NIST Evaluation of DeepSeek AI Models

The provided text is an excerpt from a **technical evaluation report** conducted by the Center for AI Standards and Innovation (CAISI), housed within the National Institute of Standards and Technology (NIST), in September 2025. This report **systematically compares** three DeepSeek AI models against four U.S. reference models, including OpenAI’s GPT-5 and Anthropic’s Opus 4, across 19 benchmarks. The evaluation focuses on several critical areas, revealing that DeepSeek models generally **lag U.S. models in performance**, particularly in cyber and software engineering tasks, while also being **more expensive to operate** and **significantly less robust** against security threats like agent hijacking and jailbreaking attacks. Furthermore, the analysis determined that the DeepSeek models exhibit alignment with **Chinese Communist Party (CCP) censorship narratives** in both English and Chinese queries. The document also includes data on model adoption trends, noting the rapid increase in the use of certain PRC models like DeepSeek.Source:https://www.nist.gov/system/files/documents/2025/09/30/CAISI_Evaluation_of_DeepSeek_AI_Models.pdf

There are no comments yet.

Please log in to write the first comment.