NVIDIA 发布 Nemotron 3.5 Content Safety 多模态内容安全模型
NVIDIA 发布 Nemotron 3.5 Content Safety，基于 Gemma 3 4B IT 微调，将多模态输入、12 语言覆盖、自定义企业策略执行与可审计推理统一到单次推理调用中。
AI安全动态
Hugging Face Blog推荐指数8181
Anthropic精选推荐指数8080 Anthropic 发布 LLM ATT&CK Navigator，将 832 个账号的 AI 网络攻击映射到 MITRE ATT&CK
Anthropic 前沿红队发布 LLM ATT&CK Navigator，把 2025 年 3 月至 2026 年 3 月间因违反使用政策被封禁的 832 个账号的 13,873 次恶意行为映射到 MITRE ATT&CK V18，覆盖全部 14 个战术和 482 个子技术。
推荐理由：Anthropic 把 832 个被封账号的 AI 滥用行为映射到 MITRE ATT&CK，并给出 ARiES 风险评分方法，可看攻击者如何从造工具转向自主编排。
Anthropic精选推荐指数8282 Anthropic 用 ExploitBench、ExploitGym 与 SCONE-bench 测量大模型开发漏洞利用的能力
Anthropic 前沿红队发布研究，用 ExploitBench、ExploitGym 和更新版 SCONE-bench 三个基准测量大模型开发漏洞利用的能力，Claude Mythos Preview 在三者上均优于所有其他受测模型。
推荐理由：Anthropic 红队用三个漏洞利用基准量化 Claude Mythos Preview 的能力，读者可据此看到模型攻破 V8 沙箱与内核的实际差距。
Fortinet推荐指数2222 Fortinet：OT 安全为何需要威胁驱动的方法
Fortinet 指出，OT 安全不能简单照搬 IT 控制，需采用威胁驱动方法：激进漏洞扫描可能干扰 PLC/RTU，端点代理会占用 HMI 资源，补丁引入非计划停机，过度分段可能阻断工业协议。CISO 需先建立资产可见性（控制器、工程站、HMI、远程接入点及 OT/IT 连接），再以分段限制爆炸半径，识别最可能被攻击的系统与路径，用补偿性控制加固而不中断运营。
嘶吼推荐指数3434 CACTER 反钓鱼演练系统（PhishSim）迭代上线：四步配置，员工中招率从 23.88% 降至 4.16%
CACTER 反钓鱼演练系统（PhishSim）全新迭代上线，以高仿真模拟钓鱼场景替代传统安全培训，支持模拟虚假链接、恶意附件、伪装二维码三类钓鱼形式，并可仿冒发件人和官方域名，覆盖 APT 攻击与鱼叉式钓鱼场景。
Hugging Face Blog Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality
TL;DR: Two new Apache 2.0 multilingual embedding models built on ModernBERT — a 97M-parameter compact model that beats every open sub-100M multilingual embedder on MTEB Multilingual Retrieval (60.3), and a 311M full-size model that scores 65.2 on MTEB Multilingual Retrieval (#2 among open models under 500M parameters) with Matryoshka support. Both cover 200+ languages, are tuned on 52 languages, handle 32K-token context (64x R1), and add code retrieval across 9 programming languages. In this post: Enterprise-Ready by Design · A Strong Sub-100M Multilingual Model · What Changed from R1 · Training the Full-Size 311M Model · Building the compact 97M Multilingual Model · Benchmark Results · Matryoshka Embeddings · Deployment Options · For Framework Integrators · Which Model Should You Use? · Try The Models Multilingual embedding models face a persistent tension: broad language coverage usually comes at the cost of model size, and small models usually sacrifice languages. If you work across languages — retrieval-augmented generation over multilingual corpora, cross-lingual search, code retrieval in international teams — you've likely had to choose between a model that's fast enough and one that's good enough. The Granite Embedding Multilingual R2 release narrows that gap considerably. We're releasing two new multilingual embedding models: granite-embedding-311m-multilingual-r2 — A 311M-parameter full-size model with 768-dimensional embeddings, Matryoshka dimension support, and top-tier multilingual retrieval quality. granite-embedding-97m-multilingual-r2 — A 97M-parameter compact model with 384-dimensional embeddings that delivers strong retrieval quality for its size. Both models support 200+ languages with enhanced retrieval quality for 52 languages and programming code , handle context lengths up to 32,768 tokens (a 64x increase over their R1 predecessors), and are released under the Apache 2.0 license. They work out of the box with sentence-transformers and transformers , require no task-specific instructions, and are compatible as drop-in replacements in LangChain , LlamaIndex , Haystack , and Milvus with a one-line model name change. For frameworks currently using an English-only default, that one line gives every user in your community support for 200+ languages — no API changes, no new dependencies, no code changes required on their end. Both models ship with ONNX and OpenVINO weights for CPU-optimized inference. 52 enhanced-support languages (click to expand) The underlying encoder was pretrained on text from 200+ languages, producing general-purpose embeddings for any of them. The following 52 languages receive explicit retrieval-pair and cross-lingual training for higher-quality retrieval: Albanian (sq), Arabic (ar), Azerbaijani (az), Bengali (bn), Bulgarian (bg), Catalan (ca), Chinese (zh), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), Georgian (ka), German (de), Greek (el), Hebrew (he), Hindi (hi), Hungarian (hu), Icelandic (is), Indonesian (id), Italian (it), Japanese (ja), Kazakh (kk), Khmer (km), Korean (ko), Latvian (lv), Lithuanian (lt), Malay (ms), Marathi (mr), Norwegian (no), Persian (fa), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Serbian (sr), Slovak (sk), Slovenian (sl), Spanish (es), Swahili (sw), Swedish (sv), Tagalog (tl), Telugu (te), Thai (th), Turkish (tr), Ukrainian (uk), Urdu (ur), Uzbek (uz), Vietnamese (vi). Additionally, the models are trained on programming code (Python, Go, Java, JavaScript, PHP, Ruby, SQL, C, C++) and support cross-lingual code retrieval. Enterprise-Ready by Design Both embedding models are trained on a mixture of IBM‑curated datasets, publicly available data, and internally generated or synthetic data. Public web‑derived data used in training is selected and filtered using IBM‑developed quality, deduplication, and governance processes intended to reduce risk in downstream commercial use. We intentionally avoid the use of the MS‑MARCO training dataset and datasets with explicit non‑commercial licensing restrictions. The models are pretrained using GneissWeb , an IBM‑curated dataset derived from publicly available web content and processed using IBM’s data preparation and governance tooling—along with additional IBM‑curated and other publicly available sources. Datasets undergo IBM governance review to assess licensing considerations, ownership signals, and personal data risks. These processes are designed to contribute to responsible use and enterprise deployment. A Strong Sub-100M Multilingual Model The standout of this release is granite-embedding-97m-multilingual-r2 . At 97 million parameters, it scores 60.3 on Multilingual MTEB Retrieval across 18 languages — the highest retrieval score we've found for any open multilingual embedding model under 100M parameters. The next-best model in that size class, multilingual-e5-small, scores 50.9 on the same benchmark — a +9.4 point gap on a mature benchmark. At roughly one-third the size of the 311M full-size model, it retains the majority of its retrieval quality across multilingual, code, and long-document benchmarks — a +12.2 point gain on MTEB Multilingual Retrieval over its direct predecessor, driven by a new architecture, better training data, and a novel pruning methodology (more on that below). The full-size granite-embedding-311m-multilingual-r2 scores 65.2 on the same benchmark, a +13.0 point gain over its R1 predecessor. What Changed from R1 The Granite Embedding Multilingual R1 models were built on XLM-RoBERTa encoders with a 512-token context window. The R2 generation is a ground-up rebuild: ModernBERT is a recent encoder architecture that revisits the original BERT design with techniques from the last five years of transformer research. The shift brings several practical benefits: alternating attention lengths reduce computation on long sequences (improves throughput on long sequences significantly), rotary position embeddings allow the 32K context window without the positional interpolation hacks that plague older architectures, and Flash Attention 2.0 support speeds up encoding on modern GPUs. The new multilingual tokenizers are worth highlighting. Rather than reusing XLM-RoBERTa's 250K-token vocabulary, we adopted existing tokenizers with strong multilingual and code coverage. The 311M model uses the Gemma 3 tokenizer (262K tokens); the 97M model starts from the GPT-OSS tokenizer and prunes it down to a compact 180K-token vocabulary that preserves broad multilingual coverage while reducing the embedding table's parameter footprint. Tokenizer efficiency matters more than people realize — a 32K-token window sounds impressive until your tokenizer burns half of it encoding a single paragraph of Thai. Training the Full-Size 311M Model The 311M model is a 22-layer ModernBERT encoder with a 262K-token multilingual vocabulary, trained through a multi-stage pipeline: Knowledge distillation : The model learns from multiple teacher models simultaneously. The teachers are Granite 3.3 Instruct and Mistral v0.2 Instruct decoder models, further finetuned for text embeddings, which transfer retrieval-specific knowledge into the 311M encoder architecture. Contrastive fine-tuning : Standard contrastive training on multilingual retrieval pairs — queries matched with relevant and hard-negative passages across 52 languages and code — sharpens the model's ability to distinguish relevant from irrelevant results. Model merging : After training, we merge checkpoints from different training stages and configurations. This combines the strengths of models optimized for different objectives (e.g., multilingual breadth vs. English depth) into a single set of weights without additional training compute. Matryoshka Representation Learning : The model is trained with Matryoshka objectives so that its 768-dimensional embeddings can be truncated to 512, 384, 256, or 128 dimensions with minimal quality loss (see Matryoshka Embeddings below). The result is a model that scores 65.2 on MTEB Multilingual Retrieval and 56.3 on the overall average — a +14.5 point average gain over its R1 predecessor. Building the compact 97M Multilingual model The 97M model is trained through a combination of vocabulary selection and knowledge distillation : Vocabulary selection : The 262K-token vocabulary is reduced to a purpose-trained 180K-token vocabulary that preserves broad multilingual coverage while cutting the embedding table size substantially. Knowledge distillation : The pruned model is then finetuned using knowledge distillation from multiple teacher models (including a Granite 4.1 8B and Mistral Instruct decoder-based teacher) and contrastive training to improve retrieval quality. This approach transfers retrieval-specific knowledge from multiple strong teachers, while reducing the model parameters without sacrificing language coverage. The result is a highly efficient compact model — scoring 60.3 on MTEB Multilingual Retrieval vs. 65.2 for the full-size model, while being approximately 3x smaller. Benchmark Results Multilingual Retrieval Performance across the main benchmark suite sorted by model size. Scores are averages across tasks within each benchmark (higher is better): Model Params Active Params Embed Dim MTEB Multilingual Retrieval (18) Code (12) English Retrieval (10) LongEmbed (6) RaR-b (17) F2LLM-v2-80M 80M 32M 320 50.1 68.0 47.5 31.7 17.9 multilingual-e5-small 118M 22M 384 50.9 53.5 46.5 38.8 20.3 granite-embedding-107m-multilingual (R1) 107M 11M 384 48.1 40.7 47.9 34.3 17.1 paraphrase-multilingual-MiniLM-L12-v2 118M 22M 384 36.6 23.5 35.9 20.9 10.9 jina-embeddings-v5-text-nano 212M 113M 768 63.3 71.2 58.8 63.6 25.2 harrier-oss-v1-270m 268M 100M 640 66.4 62.4 52.1 64.9 32.9 multilingual-e5-base 278M 86M 768 52.7 52.6 49.0 40.5 23.4 granite-embedding-278m-multilingual (R1) 278M 86M 768 52.2 48.5 51.5 37.7 18.9 embeddinggemma-300m 308M 106M 768 62.5 68.7 54.6 55.4 26.1 gte-multilingual-base 305M 113M 768 57.2 57.5 50.8 62.1 19.0 snowflake-arctic-embed-m-v2.0 305M 113M 768 54.8 55.2 58.4 55.4 23.3 multilingual-e5-large 560M 304M 1024 53.7 55.8 51.5 40.4 25.4 text-embedding-3-small (OpenAI, API only) — — 1536 50.7 — 53.8 53.6 23.2 granite-embedding-97m-multilingual-r2 97M 28M 384 60.3 60.4 50.1 65.6 24.9 granite-embedding-311m-multilingual-r2 311M 110M 768 65.2 (#2) 63.8 (#3) 52.6 (#5) 71.7 (#1) 28.0 (#2) A few things stand out: The 97M R2 model beats multilingual-e5-base and gte-multilingual-base (~300M parameter models) on average and on most individual benchmarks, despite being roughly 3x smaller. paraphrase-multilingual-MiniLM-L12-v2 — a widely-used framework default — scores 36.6 , a full +23.7 points behind the 97M R2 model, which is also slightly smaller (97M vs 110M parameters) with the same 384-dimensional output. LongEmbed is the biggest R1-to-R2 gain : +31.3 points for the 97M model, +34.0 for the 311M. This is the direct payoff of the 32K context window — R1's 512-token limit meant your legal contract was being judged by its first page. Many practical multilingual workloads involve long documents (legal contracts, technical manuals, research papers, multi-page reports) that R1 simply could not see in full. Code retrieval improves dramatically : +19.7 (97M) and +15.3 (311M) over R1, reflecting the new code training set, larger context window, and better training methodology. In the broader competitive field , harrier-oss-v1-270m leads on MTEB Multilingual Retrieval (66.4) and RaR-b (32.9), while jina-embeddings-v5-text-nano leads on Code (71.2) and English Retrieval (58.8). The 311M Granite model is competitive on average (56.3) and leads on LongEmbed (71.7), while offering substantially higher encoding throughput than jina-embeddings-v5-text-nano (see speed table below). Speed and Throughput Encoding speed matters for production workloads, especially when you're indexing millions of documents or need low-latency query encoding. We measured latency and throughput on a single NVIDIA H100 GPU using 512-token chunks: The 97M model encodes over 2,500 documents per second — comparable throughput to multilingual-e5-small — while delivering substantially higher retrieval quality. The 311M model, at ~1,800 docs/sec, performs better than jina-embeddings-v5-text-nano on retrieval quality (65.2 vs. 63.3) at over 5.5x the encoding speed (note: speed numbers are computed with the latest transformer code, which had a speed regression vs the last 4.57 version - for both the Jina and granite models - see our technical report for details). harrier-oss-v1-270m offers the best combination of speed and retrieval score among the competitors listed here. Matryoshka Embeddings (311M) The 311M model supports Matryoshka Representation Learning , which lets you truncate embeddings from the full 768 dimensions down to 512, 384, 256, or 128 with graceful quality degradation. This is useful when storage, memory, or similarity-computation cost is a concern — a 256-dimensional embedding takes one-third the storage of a 768-dimensional one, and cosine similarity is proportionally cheaper to compute. Here's how retrieval quality holds up across embedding dimensions: The quality loss from dimension reduction is remarkably small. Cutting from 768 to 256 dimensions — a 3x reduction in storage and similarity-computation cost — drops MTEB Multilingual Retrieval by just 0.5 points (65.2 → 64.7) and Code Retrieval by 0.5 points (63.9 → 63.4). Even at 128 dimensions (a 6x reduction ), the model still scores 63.7 on MTEB Multilingual Retrieval and 62.3 on Code — retaining over 97% of its full-dimension performance. In practice, this means you can substantially reduce your index size and search latency with minimal impact on result quality. (Note,results in the above picture were evaluated with a context length of 1024 for English and Multilingual Retrieval, and 8192 for Code). For comparison, the 311M model truncated to 384 dimensions (the same dimensionality as the 97M model's native output) still outperforms the 97M model across all three benchmarks. If you need 384-dimensional embeddings and can afford the 311M model's encoding cost, Matryoshka truncation is the stronger option. from sentence_transformers import SentenceTransformer model = SentenceTransformer( "ibm-granite/granite-embedding-311m-multilingual-r2" ) # Full 768-dimensional embeddings full = model.encode([ "example text" ]) print (full.shape) # (1, 768) # Truncated to 384 dimensions small = model.encode([ "example text" ], truncate_dim= 384 ) print (small.shape) # (1, 384) The 97M model does not support Matryoshka — 384 dimensions is already compact. Cross-lingual Retrieval Average performance on cross-lingual tasks within MTEB Retrieval. Belebele measures cross-lingual passage matching across 122 languages; MLQA measures extractive cross-lingual question answering retrieval across 7 languages. Model Belebele Retrieval MLQA Retrieval granite-embedding-107m-multilingual (R1) 55.1 60.5 granite-embedding-278m-multilingual (R1) 62.2 63.0 granite-embedding-97m-multilingual-r2 52.9 60.5 granite-embedding-311m-multilingual-r2 66.5 67.1 The 311M R2 model gains +4.3 on Belebele and +4.1 on MLQA over its R1 predecessor, showing improved cross-lingual transfer at the larger scale across both benchmarks. The 97M R2 model scores lower on Belebele (52.9 vs 55.1, −2.2) while matching its R1 predecessor on MLQA (60.5). The Belebele gap is a tradeoff inherent in the pruning and vocabulary reduction process — the R2 model's training prioritized the broader 18-language MTEB Multilingual Retrieval set (where it gains +12.2 over R1) and long-document retrieval (+31.3), while the smaller vocabulary (180K vs. 250K tokens) and reduced layer count (12 vs. 22) affect narrow cross-lingual transfer tasks. If cross-lingual transfer across many language pairs is your primary use case, the full-size 311M model is the better choice. Deployment Options Both models ship with multiple deployment paths for production use. Install the core library with: pip install sentence-transformers Sentence Transformers (recommended for most users): from sentence_transformers import SentenceTransformer, util model = SentenceTransformer( "ibm-granite/granite-embedding-97m-multilingual-r2" ) queries = [ "What is the tallest mountain in Japan?" , # English "Wer hat das Lied Achy Breaky Heart geschrieben?" , # German "ドイツの首都はどこですか？" , # Japanese ] passages = [ "富士山は、静岡県と山梨県にまたがる活火山で、標高3776.12 mで日本最高峰の独立峰である。" , # Japanese "Achy Breaky Heart is a country song written by Don Von Tress." , # English "Berlin ist die Hauptstadt und ein Land der Bundesrepublik Deutschland." , # German ] q_emb = model.encode(queries) p_emb = model.encode(passages) print (util.cos_sim(q_emb, p_emb)) # Each query scores highest against its matching passage — across languages LangChain ( pip install langchain-huggingface ): from langchain_huggingface import HuggingFaceEmbeddings embeddings = HuggingFaceEmbeddings( model_name= "ibm-granite/granite-embedding-97m-multilingual-r2" ) docs = embeddings.embed_documents([ "富士山は日本最高峰の独立峰です。" , "Mount Fuji is Japan's highest peak." , ]) query = embeddings.embed_query( "What is Japan's tallest mountain?" ) # Drop-in replacement anywhere LangChain accepts an Embeddings object LlamaIndex ( pip install llama-index-embeddings-huggingface ): from llama_index.embeddings.huggingface import HuggingFaceEmbedding from llama_index.core import Settings embed_model = HuggingFaceEmbedding( model_name= "ibm-granite/granite-embedding-97m-multilingual-r2" ) Settings.embed_model = embed_model # applies globally to any index or pipeline Haystack ( pip install sentence-transformers haystack-ai ) from haystack.components.embedders import ( SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, ) from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever from haystack.dataclasses import Document from haystack.document_stores.in_memory import InMemoryDocumentStore doc_embedder = SentenceTransformersDocumentEmbedder( model= "ibm-granite/granite-embedding-97m-multilingual-r2" ) query_embedder = SentenceTransformersTextEmbedder( model= "ibm-granite/granite-embedding-97m-multilingual-r2" ) doc_embedder.warm_up() query_embedder.warm_up() # Embed and index documents document_store = InMemoryDocumentStore() result_docs = doc_embedder.run(documents=[ Document(content= "富士山は日本最高峰の独立峰です。" ), Document(content= "Mount Fuji is Japan's highest peak." ), Document(content= "Achy Breaky Heart is a country song written by Don Von Tress." ), Document(content= "Berlin ist die Hauptstadt und ein Land der Bundesrepublik Deutschland." ), ]) document_store.write_documents(result_docs[ "documents" ]) # Embed query and retrieve result_query = query_embedder.run(text= "What is Japan's tallest mountain?" ) retriever = InMemoryEmbeddingRetriever(document_store=document_store) results = retriever.run(query_embedding=result_query[ "embedding" ], top_k= 2 ) for doc in results[ "documents" ]: print ( f" {doc.score: .3 f} {doc.content} " ) # 0.961 Mount Fuji is Japan's highest peak. # 0.913 富士山は日本最高峰の独立峰です。 Milvus ( pip install pymilvus sentence-transformers ) from pymilvus import MilvusClient from sentence_transformers import SentenceTransformer model = SentenceTransformer( "ibm-granite/granite-embedding-97m-multilingual-r2" ) # Use "./milvus.db" for local persistence or a server URI for production client = MilvusClient( ":memory:" ) client.create_collection(collection_name= "multilingual_docs" , dimension= 384 ) docs = [ "富士山は日本最高峰の独立峰です。" , "Mount Fuji is Japan's highest peak." , "Achy Breaky Heart is a country song written by Don Von Tress." , "Berlin ist die Hauptstadt und ein Land der Bundesrepublik Deutschland." , ] embeddings = model.encode(docs).tolist() client.insert( collection_name= "multilingual_docs" , data=[{ "id" : i, "vector" : emb, "text" : doc} for i, (emb, doc) in enumerate ( zip (embeddings, docs))], ) query_emb = model.encode([ "What is Japan's tallest mountain?" ]).tolist() results = client.search( collection_name= "multilingual_docs" , data=query_emb, limit= 2 , output_fields=[ "text" ], ) for hit in results[ 0 ]: print ( f" {hit[ 'distance' ]: .3 f} {hit[ 'entity' ][ 'text' ]} " ) # 0.961 Mount Fuji is Japan's highest peak. # 0.913 富士山は日本最高峰の独立峰です。 Both models also ship with pre-converted ONNX and OpenVINO weights for optimized CPU/accelerator inference, work as embedding endpoints via vLLM ( vllm serve ... --task embed ), and can be converted to GGUF for Ollama using llama.cpp . See the model cards for full deployment examples. For Framework Integrators If you maintain an embedding framework, vector store, or RAG pipeline library and are evaluating these models as a default, here's what you need to know: License : Apache 2.0, trained without MS-MARCO Drop-in behavior : No task-specific instruction prefix required — behaves like all-MiniLM-L6-v2 at the API level. Existing code that calls .encode() works unchanged. Dimensionality : 384-dimensional output (97M) and 768-dimensional output (311M), matching the most common existing defaults. No index migration required. Model size : The 97M model's weights are 195 MB (safetensors) — less than half the size of paraphrase-multilingual-MiniLM-L12-v2 (471 MB), the most common multilingual default. The quantized ONNX weights are just 98 MB, comparable to all-MiniLM-L6-v2 (91 MB) while covering 200+ languages. CPU-friendly : Ships with ONNX and OpenVINO weights for optimized CPU inference. No GPU dependency for a getting-started tutorial. Multilingual by default : If your current default is English-only, this is a one-line swap that gives every user in your community support for 200+ languages — without touching their code. Stable identifier : ibm-granite/granite-embedding-97m-multilingual-r2 on Hugging Face, maintained by IBM under the Granite model family. To discuss adopting these models as a default in your project, open an issue at ibm-granite/granite-embedding-models . Which Model Should You Use? These two multilingual models are part of the broader Granite Embedding R2 family, which also includes two high-performing English-focused models: granite-embedding-english-r2 (149M parameters) and granite-embedding-small-english-r2 (47M parameters). If your data is predominantly English, the English models offer higher retrieval quality on English benchmarks at a smaller footprint, since they don't need to allocate capacity across 200+ languages. If you need... Use Best multilingual retrieval quality granite-embedding-311m-multilingual-r2 Flexible embedding dimensions (storage/speed tradeoff) granite-embedding-311m-multilingual-r2 (Matryoshka) Maximum throughput / edge deployment / low latency granite-embedding-97m-multilingual-r2 Best cross-lingual transfer across many language pairs granite-embedding-311m-multilingual-r2 Predominantly English data granite-embedding-english-r2 or granite-embedding-small-english-r2 Try The Models Both models are available now on Hugging Face under the IBM Granite Embedding collection : granite-embedding-311m-multilingual-r2 granite-embedding-97m-multilingual-r2 You can try the small models interactively (on CPU) via a Granite Embedding demo here on Hugging Face Spaces, or run the full examples notebook in Google Colab: You can access our detailed technical report covering the full training methodology, per-language evaluations, and pruning ablations here Granite Multilingual Embedding R2 report . For questions, feedback, or issues, visit ibm-granite/granite-embedding-models on GitHub. Framework maintainers: If you'd like to adopt these models as a default in your project, open an issue at ibm-granite/granite-embedding-models — we're happy to help with integration, testing, and any questions about licensing or deployment. Give them a try, and if the embeddings spark joy, smash that ❤️ button on Hugging Face. Our models have feelings too, and every +1 keeps them warm at night.
嘶吼推荐指数2424 嘶吼2026网络安全产业图谱发布：AI重构安全产品架构成行业共识
嘶吼安全产业研究院正式发布《嘶吼2026网络安全产业图谱》，基于累计回收的400余份有效问卷，呈现中国网络安全产业发展格局。调研识别出三个确定性趋势：AI重构安全产品架构已成行业共识、产业格局从"大而全"走向"专而精"、中国网络安全产业黄金发展期仍在持续。已完成AI能力整合的厂商在客户留存、增长韧性、溢价空间等关键指标上均展现出明显优势。
OWASP精选推荐指数7878 OWASP ASI06 负责人：AI Agent 的记忆也是攻击面
Idan Habler 作为 OWASP ASI06: Memory & Context Poisoning 条目共同负责人撰文，结合 Cisco 对 Claude Code 的研究（漏洞代号 MemoryTrap）指出。
推荐理由：作者以 ASI06 共同负责人的身份，把 MemoryTrap 案例上升为记忆应被视为攻击面的行业判断，便于读者理解 Agent 安全的新威胁模型。
Hugging Face Blog Adding Benchmaxxer Repellant to the Open ASR Leaderboard
"When a measure becomes a target, it ceases to be a good measure." (Goodhart’s Law) TLDR : Appen Inc. and DataoceanAI have provided high-quality English ASR datasets covering scripted and conversational speech over multiple accents. To prevent potential risks of benchmaxxing or test-set contamination, we will keep these datasets private for a high-quality measure of performance on multiple tasks. We’re not updating the average WER at this time : by default, the leaderboard’s Average WER remains computed on public datasets only. You can optionally include the private datasets using the toggle to see their impact 👀 Since its launch in September 2023, the Open ASR Leaderboard has been visited over 710K times. We’re blown away by the community’s interest and motivation to keep pushing speech recognition 🗣️ Two words sum up the objectives (but also challenges) in maintaining a benchmark like the Open ASR Leaderboard: Standardization : models can have different conventions for their usage and outputs, e.g. with/without punctuation and casing. Datasets have the same challenges and can be structured differently. To this end, all test sets have been gathered into a single dataset on the Hub for easy access and previewing. Moreover, to standardize model outputs and dataset transcripts, we use a normalizer that (among other things) removes punctuation and casing, and maps to American spelling. It is based on the normalizer of Whisper . Openness : the UI code and evaluation scripts are open-sourced. This has helped not only to incorporate new models, but also to improve the quality of the evaluation procedure through community feedback and contributions. Standardization and openness are essential for meaningful benchmarking, but they also make benchmarks more susceptible to benchmark-specific optimization ("benchmaxxing"), where models improve leaderboard performance without corresponding gains in real-world robustness. As models and use cases evolve, the Open ASR Leaderboard will continue incorporating high-quality datasets and new evaluation settings to better reflect real-world performance and improve robustness against benchmark-specific optimization. As discussed in our report , there is no single "catch-all" ASR model: some perform better on American English, others on diverse accents and multilingual settings, while others are optimized for speed or conversational audio. Different applications also prioritize different capabilities, so a model that performs less well on one dimension is not necessarily a worse model overall. The goal of the Open ASR Leaderboard is to capture these nuances and provide a more holistic view of ASR performance. New high-quality, private datasets To this end, we have worked with Appen Inc. and DataoceanAI to curate high-quality datasets for ASR benchmarking. Below is some information on the various splits. Dataset Accent Duration [h] Male (%) / Female (%) Style Transcription Appen Scripted AU Australian 1.42 49 / 51 Read Punctuated, cased. Appen Scripted CA Canadian 1.53 52 / 48 Read Punctuated, cased. Appen Scripted IN Indian 1.02 49 / 51 Read Punctuated, cased. Appen Scripted US American 1.45 49 / 51 Read Punctuated, cased. Appen Conversational IN Indian 1.37 51 / 49 Conversational, spontaneous Punctuated, disfluencies. Appen Conversational US003 American 1.64 49 / 51 Conversational, spontaneous Punctuated, cased, disfluencies. Appen Conversational US004 American 1.65 49 / 51 Conversational, spontaneous Punctuated, disfluencies. DataoceanAI Scripted US American 2.43 54 / 46 Read Punctuated, cased (proper nouns), disfluencies. DataoceanAI Scripted GB British 2.43 47 / 53 Read Punctuated, disfluencies. DataoceanAI Conversational US American 8.82 NA Conversational, spontaneous Punctuated, disfluencies. DataoceanAI Conversational GB British 5.96 NA Conversational, spontaneous Punctuated, disfluencies. Below are sample audio showing the variety of content (scripted, conversational, acronyms, disfluencies, proper nouns). While private datasets may sound contrary to the spirit of openness, we believe that incorporating such datasets will increase the trustworthiness of the Open ASR Leaderboard, as they are less likely to be exploited for benchmaxxing, whether by model developers who explicitly use the public test sets or who try to find training data that closely resembles a particular dataset to boost their score in the macroaverage. With these datasets, we can also provide targeted metrics to highlight gaps and biases between controlled and often saturated settings (scripted, American accent) and more nuanced conditions (conversational and non-American accents). Below is a screenshot of the new "Private data" tab. Below is how each column is computed. "Average WER" computes a macroaverage of the data provider averages, so that they are weighted equally. "Avg Scripted" performs a macroaverage of all scripted datasets. "Avg Conversational" performs a macroaverage of all conversational datasets. "Avg US" performs a macroaverage of all datasets with American accents. "Avg non-US" performs a macroaverage of all datasets with non-American accents. We intentionally do not provide a score on each split, to avoid model developers from boosting their score with a specific data provider or accent. How can I evaluate my model on this data? Get your model on the Open ASR Leaderboard, and we'll run the evaluation! As before, the process for adding a model to the leaderboard takes place on the Open ASR Leaderboard GitHub : Open a pull request, and a model checklist will appear. As before, you should report your results on the public datasets. We will verify the results on the public sets and compute the metrics on the private ones. Confirm the results we’ve obtained. While you wait for your model to be added to the Open ASR Leaderboard, you can self-report your metrics on the public sets by adding a YAML file like this to your model card. Your model will then appear on an (unverified) leaderboard that appears on the dataset page (see screenshot below). More on this approach to decentralized evaluation can be read here . Do models trained on the data providers have an advantage? They could. We’ve asked Appen and DataoceanAI to not provide this data to their clients. But even if they do not provide this exact data, data from a similar distribution could still help the model on the corresponding evaluation set (similar to benchmaxxing by optimizing for a challenging task from the public sets). To this end, having multiple data providers balances out the advantage a model may get from having used data from one of the providers. And we are open to more data providers and eval sets for the "Private data" tab! Moreover, to ensure that the private sets do not affect the model ranking, we’ve defaulted the Average WER to not include the Private sets in its macroaverage. In the screenshot below, you can see that "Private data" is toggled off. This means that the macroaverage across datasets does not include it. Simply toggle on "Private data" splits to include them in the macroaverage. The "Rank Δ" column shows how the ordering changes relative to the default macroaverage configuration. Including or excluding public datasets also changes the macroaverage, allowing users to tailor the evaluation to the use cases and data distributions most relevant to their application. What's next? We’re excited to hear the community’s feedback on how the new track and dataset toggling features help users identify the model(s) that best fit their application(s). We’re also looking into evaluations that better reflect real-world noisy conditions, and you can expect some news on that 😉 While preparing the private evaluation sets, we took extra care to ensure consistent audio and transcript quality across datasets, including developing tooling to identify challenging cases such as low signal-to-noise conditions or transcript mismatches, since these factors can meaningfully affect WER. More on that in a future post!
Hugging Face Blog精选推荐指数6060 Hugging Face 实测 OpenAI Privacy Filter：用 gradio.Server 构建三个 PII 脱敏 Web 应用
Hugging Face 博客作者用 OpenAI 本周在 Hub 上开源的 Privacy Filter（1.5B 参数、50M 激活参数、Apache 2.0 许可。
推荐理由：原文用三个可运行示例演示了 Privacy Filter 的 PII 检测能力与 gradio.Server 的前后端分工，读者可直接照搬这套工程做法。
嘶吼推荐指数3838 烽台科技纵横网络靶场社区正式发布，以虚实融合技术构建工业信息安全实战生态
烽台科技正式发布纵横网络靶场社区，以“虚实融合”技术为核心，整合AI智能体与数字孪生能力，为高校、企业、科研院所提供工业安全实训与技术交流空间。
Hugging Face Blog QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard
QIMMA validates benchmarks before evaluating models, ensuring reported scores reflect genuine Arabic language capability in LLMs. If you've been tracking Arabic LLM evaluation, you've probably noticed a growing tension: the number of benchmarks and leaderboards is expanding rapidly, but are we actually measuring what we think we're measuring? We built QIMMA قمّة (Arabic for "summit"), to answer that question systematically. Instead of aggregating existing Arabic benchmarks as-is and running models on them, we applied a rigorous quality validation pipeline before any evaluation took place. What we found was sobering: even widely-used, well-regarded Arabic benchmarks contain systematic quality issues that can quietly corrupt evaluation results. This post walks through what QIMMA is, how we built it, what problems we found, and what the model rankings look like once you clean things up. 🔍 The Problem: Arabic NLP Evaluation Is Fragmented and Unvalidated Arabic is spoken by over 400 million people across diverse dialects and cultural contexts, yet the Arabic NLP evaluation landscape remains fragmented. A few key pain points have motivated this work: Translation issues. Many Arabic benchmarks are translations from English. This introduces distributional shifts. Questions that feel natural in English become awkward or culturally misaligned in Arabic, making benchmark data less representative of how Arabic is naturally used. Absent quality validation. Even native Arabic benchmarks are often released without rigorous quality checks. Annotation inconsistencies, incorrect gold answers, encoding errors, and cultural bias in ground-truth labels have all been documented in established resources. Reproducibility gaps. Evaluation scripts and per-sample outputs are rarely released publicly, making it hard to audit results or build on prior work. Coverage fragmentation. Existing leaderboards cover isolated tasks and narrow domains, making holistic model assessment difficult. To illustrate where QIMMA sits relative to existing platforms: Leaderboard Open Source Native Arabic Quality Validation Coding Eval Public Outputs OALL v1 ✅ Mixed ❌ ❌ ✅ OALL v2 ✅ Mostly ❌ ❌ ✅ BALSAM Partial 50% ❌ ❌ ❌ AraGen ✅ 100% ✅ ❌ ❌ SILMA ABL ✅ 100% ✅ ❌ ✅ ILMAAM Partial 100% ✅ ❌ ❌ HELM Arabic ✅ Mixed ❌ ❌ ✅ ⛰ QIMMA ✅ 99% ✅ ✅ ✅ QIMMA is the only platform combining all five properties: open source, predominantly native Arabic content, systematic quality validation, code evaluation, and public per-sample inference outputs. ⛰ What's in QIMMA? QIMMA consolidates 109 subsets from 14 source benchmarks into a unified evaluation suite of over 52,000 samples , spanning 7 domains: Domain Benchmarks Task Types Cultural AraDiCE-Culture, ArabCulture, PalmX MCQ STEM ArabicMMLU, GAT, 3LM STEM MCQ Legal ArabLegalQA, MizanQA MCQ, QA Medical MedArabiQ, MedAraBench MCQ, QA Safety AraTrust MCQ Poetry & Literature FannOrFlop QA Coding 3LM HumanEval+, 3LM MBPP+ Code A few things stand out about this design: 99% native Arabic content. The only exception is code evaluation, which is inherently language-agnostic. First Arabic leaderboard with code evaluation. QIMMA integrates Arabic-adapted versions of HumanEval+ and MBPP+, making it possible to assess coding capability with Arabic-language problem statements. Diversity in Domains and Tasks. QIMMA evaluates real-world competency areas including education, governance, healthcare, creative expression, and software development. 🔬 The Quality Validation Pipeline This is the methodological heart of QIMMA. Before running a single model, we applied a multi-stage validation pipeline to every sample in every benchmark. Stage 1: Multi-Model Automated Assessment Each sample was independently evaluated by two state-of-the-art LLMs: Qwen3-235B-A22B-Instruct DeepSeek-V3-671B We chose two models with strong Arabic capability but different training data compositions, so that their combined judgment is more robust than either alone. Each model scores a sample against a 10-point rubric , with binary scores (0 or 1) per criterion: A sample is eliminated if either model scores it below 7/10. Samples where both models agree on elimination are dropped immediately. However, where only one model flags a sample, it proceeds to human review in Stage 2. Stage 2: Human Annotation and Review Flagged samples are reviewed by native Arabic speakers with cultural and dialectal familiarity. Human annotators make final calls on: Cultural context and regional variation Dialectal nuance Subjective interpretation Subtle quality issues automated assessment may miss For culturally sensitive content, multiple perspectives are considered, since "correctness" can genuinely vary across Arab regions. ⚠️ What We Found: Systematic Quality Problems The pipeline revealed recurring quality issues across benchmarks; not isolated errors, but systematic patterns reflecting gaps in how benchmarks were originally constructed. By the Numbers Benchmark Total Samples Discarded Discard Rate ArabicMMLU 14,163 436 3.1% MizanQA 1,769 41 2.3% PalmX 3,001 25 0.8% MedAraBench 4,960 33 0.7% FannOrFlop 6,984 43 0.6% ArabCulture 3,482 7 0.2% MedArabiQ 499 1 0.2% GAT 13,986 1 ~0.0% 3LM STEM 2,609 1 ~0.0% AraDiCE-Culture 180 0 0.0% ArabLegalQA 79 0 0.0% AraTrust 522 0 0.0% Taxonomy of Issues Found ⚖️ Answer Quality False or mismatched gold indices, factually wrong answers, missing or raw text answers. 📄 Text & Formatting Quality Corrupt or illegible text, spelling and grammar errors, and duplicate samples. 💬 Cultural Sensitivity Stereotype reinforcement and monolithic generalizations about diverse communities. 🤝 Gold Answer Compliance Misalignment of gold answers with evaluation protocols. 💻 Code Benchmark: A Different Kind of Quality Work Code benchmarks required a different intervention. Rather than discarding samples, we refined the Arabic problem statements in 3LM's Arabic adaptations of HumanEval+ and MBPP+, leaving task identifiers, reference solutions, and test suites completely unchanged. The modification rates were striking: Benchmark Total Prompts Modified Unchanged Modification Rate 3LM HumanEval+ 164 145 19 88% 3LM MBPP+ 378 308 70 81% Modifications fell into five categories: Linguistic refinement : normalizing toward natural Modern Standard Arabic and consistent imperative style Clarity improvements : fixing ambiguous instructions and unclear constraints Consistency normalization : standardizing mathematical terminology, punctuation, and example formatting Structural corrections : fixing broken triple-quoted strings, indentation errors, corrupted text fragments Semantic refinements : clarifying whether ranges are inclusive/exclusive, preserving task intent ⚙️ Evaluation Setup Evaluation Framework QIMMA uses LightEval , EvalPlus and FannOrFlop as its evaluation framework, chosen for consistency, multilingual community adoption, and reproducibility. Metrics by Task Type Task Type Metric Benchmarks MCQ Normalized Log-Likelihood Accuracy AraDiCE-Culture, ArabicMMLU, ArabCulture, PalmX, 3LM STEM, MedArabiQ, GAT, MedAraBench, AraTrust Multi-select MCQ Probability Mass on Gold Choices MizanQA Generative QA F1 BERTScore (AraBERT v02) MedArabiQ, ArabLegalQA, FannOrFlop Code Pass@1 3LM HumanEval+, 3LM MBPP+ Prompt Templates QIMMA standardizes prompting by question format, with six template types: MCQ : generic multiple choice · MCQ-C : multiple choice with context passage · MCQ-I : multiple choice with specific instructions (GAT analogy/completion) · QA : generic open-ended QA · QA-C : QA with context · QA-F : fill-in-the-blank QA All prompts are in Arabic. For MizanQA and ArabCulture, benchmark-specific system prompts from the original papers are preserved. 🏆 Leaderboard Results Results as of April 2026; covering top 10 evaluated models. Visit the live leaderboard for current rankings. Rank Model AVERAGE AraDiCE-Culture ArabicMMLU ArabCulture PALMX 3LM STEM AraTrust MizanQA MedArabiQ ArabLegalQA GAT MedAraBench HumanEval+ MBPP+ FannOrFlop 🥇 1 Qwen/Qwen3.5-397B-A17B-FP8 68.06 82.78 77.54 61.75 83.91 88.67 90.04 73.36 47.30 54.94 55.89 47.97 67.68 76.72 44.33 🥈 2 Applied-Innovation-Center/Karnak 66.20 73.33 80.94 53.49 81.40 93.10 89.08 55.92 55.78 71.58 61.06 54.19 33.54 64.55 58.91 🥉 3 inceptionai/Jais-2-70B-Chat 65.81 78.89 81.29 83.24 83.73 87.96 90.23 71.78 52.79 69.60 51.67 50.89 19.51 43.65 56.13 #4 Qwen/Qwen2.5-72B-Instruct 65.75 77.22 73.78 63.83 77.77 87.55 88.51 63.49 50.06 70.74 55.90 44.19 37.20 72.75 57.51 #5 Applied-Innovation-Center/AIC-1 65.37 73.33 72.02 77.52 76.11 88.13 90.61 56.36 53.75 68.96 62.11 50.78 28.05 69.58 47.83 #6 Qwen/Qwen3.5-122B-A10B 64.84 74.44 73.17 37.78 81.46 86.18 86.97 64.01 47.04 55.11 50.90 52.49 65.24 72.43 60.54 #7 Sakalti/Ultiima-72B 64.49 78.33 72.28 68.79 76.75 83.70 89.08 60.44 44.58 69.12 46.91 42.25 39.02 74.07 57.56 #8 meta-llama/Llama-3.3-70B-Instruct 63.96 77.22 71.57 78.05 77.95 88.28 85.63 67.44 56.25 64.00 51.13 54.86 27.44 71.16 24.43 #9 Qwen/Qwen2.5-32B-Instruct 63.26 70.56 68.76 75.80 72.07 81.03 85.82 53.78 48.08 69.27 56.94 36.51 34.15 72.75 93.10 #10 FreedomIntelligence/AceGPT-v2-32B-Chat 61.14 76.67 70.62 79.79 74.46 84.88 86.97 63.89 49.96 71.46 56.04 47.32 23.78 54.50 15.56 Scale does not guarantee best performance. The top 10 spans models from 32B to 397B parameters, with several mid-size models outperforming larger ones on specific domains. Arabic-specialized models lead on cultural and linguistic tasks. Jais-2-70B-Chat ranks highest on ArabicMMLU and ArabCulture, while Karnak leads on 3LM STEM and ArabLegalQA. Coding remains the hardest domain for Arabic-specialized models. The top HumanEval+ and MBPP+ scores belong to multilingual models, with Qwen3.5-397B leading both. The Size-Performance Relationship Across the full leaderboard (46 models), a clear but imperfect size-performance correlation emerges. However, there are interesting exceptions: Arabic-specialized models often outperform size-matched multilingual models Instruction-tuned models consistently outperform their base counterparts except for Qwen3 Some smaller Arabic-specialized models (Fanar-1-9B, ALLaM-7B) outperform much larger multilingual models on specific domains 🌟 What Makes QIMMA Different To summarize the distinctive properties of QIMMA: Property Details Quality-first philosophy Validation runs before evaluation, not as an afterthought Multi-model validation Two LLMs with different training + human review for flagged cases 99% native Arabic Avoids translation artifacts almost entirely Multi-domain, multi-task 7 domains, 3 task types (MCQ, QA, code), 109 subsets Code evaluation First Arabic leaderboard to include code generation Full transparency Per-sample inference outputs publicly released, not just aggregate scores LightEval-based Unified, reproducible evaluation codebase Dialectal awareness Explicit handling of MSA vs. dialectal variation in prompts and rubrics 🔗 Resources 🏆 Leaderboard : QIMMA Leaderboard 💻 Code : GitHub 📄 Paper : Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluation 🔖 Citation @misc{alqadi2026arabicbenchmarksreliableqimmas, title={Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluation}, author={Leen AlQadi and Ahmed Alzubaidi and Mohammed Alyafeai and Hamza Alobeidli and Maitha Alhammadi and Shaikha Alsuwaidi and Omar Alkaabi and Basma El Amel Boussaha and Hakim Hacid}, year={2026}, eprint={2604.03395}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2604.03395}, }
Hugging Face Blog推荐指数5858 Hugging Face 撰文论述 AI 网络安全中开放性为何关键
Hugging Face 发布博文，围绕 Mythos 与 Project Glasswing 的发布，分析 AI 网络安全的现状并论证开放性的作用。文章指出 Mythos 的能力来自系统配方（大算力、软件数据训练的模型、漏洞探测与修补脚手架、一定自主性），而非单一模型，且较小模型嵌入具备深厚安全专长的系统也可能以更低成本实现类似效果。
嘶吼推荐指数4343 梆梆安全发布《2026年Q1移动应用安全风险报告》：超八成APP存隐私违规，数据境外外发风险需高度警惕
梆梆安全发布《2026年Q1移动应用安全风险报告》，基于2026年一季度威胁监测数据，抽样6,701款Android APP中80.69%存在中高危漏洞，4,089款抽样中82.22%涉及隐私违规。抽样4,089款APP中302款存在向境外IP传输数据行为，发往美国占比70.2%、澳大利亚14.57%；抽样47,266款APP中仅30.85%完成应用加固。
Fortinet推荐指数5656 Fortinet：前沿AI模型重塑攻防节奏，防御方需以自动化缓解应对
Fortinet 发文指出 AI 正在压缩网络风险时间线，漏洞从披露到被利用的间隔从数周数月缩短到数小时，防御方优势取决于能否在开发、检测、响应全生命周期更快协同。
OWASP精选推荐指数7171 OWASP FinBot CTF 上线，作为 Agentic Security Initiative 的动手实践平台
OWASP 宣布 FinBot CTF 平台正式上线，它是 OWASP GenAI Security Project 旗下 Agentic Security Initiative 的一部分，被定位为 Agentic AI 领域的 Juice Shop，围绕模拟的金融服务应用提供动手环境。
推荐理由：原文说明了 FinBot 的挑战范围与风险映射，读者可据此了解它如何把 Agentic Top 10 的抽象条目落到可操作的实测场景。
OWASP精选推荐指数7575 OWASP 发布 2026 年 Q1 GenAI 漏洞事件汇总报告
OWASP GenAI Security Project 发布 GenAI Exploit Round-up Report Q1 2026，覆盖 2026 年 1 月 1 日至 4 月 11 日。
推荐理由：报告逐条把事件映射到 OWASP LLM 与 Agentic Top 10，并指出多数 AI 安全事件尚未进入 CVE 体系，读者可据此理解传统漏洞管理与 AI 风险之间的缺口。
嘶吼推荐指数3030 公安部通报37款违规移动应用，电商类占比超七成，小程序不再是“法外之地”
公安部计算机信息系统安全产品质量监督检验中心检测通报37款违法违规收集使用个人信息的移动应用，其中小程序占比近90%，电商商城类超四分之三，涉及多家知名品牌。违规问题涵盖未公开收集使用规则、未经同意提前或超范围收集个人信息、缺乏删除与注销途径等9类。上期通报的54款违规应用经复测仍有8款存在问题，相关分发平台已予以下架。
嘶吼推荐指数6464 Progress ShareFile 两处漏洞可组合实现未认证远程代码执行
watchTowr 在 Progress ShareFile 5.x 的 Storage Zones Controller 组件中发现认证绕过漏洞 CVE-2026-2699 和远程代码执行漏洞 CVE-2026-2701，二者可组合利用，在无需身份认证的情况下实现远程代码执行。
嘶吼推荐指数2525 嘶吼安全动态｜八部门发布《科技数据安全管理暂行规定》4月10日起实施，黑客利用像素级SVG隐藏信用卡窃密代码
八部门联合发布《科技数据安全管理暂行规定》，4月10日起实施，明确科技数据分类分级、算法备案、跨境管控等要求，强化科研与算力设施安全。黑客利用像素级SVG技巧隐藏信用卡窃密代码，影响近100个使用Magento电子商务平台的在线商店，将信用卡盗取代码隐藏在一个像素大小的可缩放矢量图形（SVG）图像中。
网信办 · 网络安全推荐指数4040 中央网信办、国家铁路局联合约谈7家涉火车票销售第三方平台
中央网信办、国家铁路局近日依据《网络安全法》《关键信息基础设施安全保护条例》联合约谈携程、同程、去哪儿、飞猪、美团、智行火车票、高铁管家等7家涉火车票销售第三方平台，要求不得利用自动化程序实施大规模、高频次抢票操作干扰铁路12306平台的安全核验措施。相关部门将加强技术监测，对利用技术手段干扰、危害12306平台安全的行为依法严肃查处。
嘶吼推荐指数4444 CrystalRAT 恶意软件新增远程控制、数据窃取等功能
卡巴斯基研究人员发现名为 CrystalRAT 的新型远程控制木马正以恶意软件即服务（MaaS）模式在 Telegram 上推广，提供远程控制、数据窃取、键盘记录与剪贴板劫持等核心功能，并内置大量恶作剧功能以干扰受害者。
Anthropic推荐指数4242 Anthropic 详解如何构建可信 AI Agent：从 Claude Code 的 Plan Mode 到四层组件安全设计
Anthropic 发布《Trustworthy agents in practice》，阐述其去年 8 月提出的可信 Agent 五原则（人类控制、价值对齐、交互安全、透明、隐私）如何落地到 Claude Code 和 Claude Cowork 等产品。
Anthropic推荐指数8888 Anthropic 红队评估 Claude Mythos Preview 的网络安全能力
Anthropic 红队发布对 Claude Mythos Preview 的网络安全能力评估，称该模型可在用户指令下发现并利用各大操作系统与浏览器中的零日漏洞，并已启动 Project Glasswing 向关键行业伙伴与开源开发者限量开放。
AICSO.AI汇总精选推荐指数7777 Meta 在重大 AI 数据泄露后暂停与 Mercor 合作
Meta 暂停与 AI 数据初创公司 Mercor 的全部合同，起因是一次针对开源库 LiteLLM 的供应链攻击：黑客组织 TeamPCP 于 3 月 27 日利用 LiteLLM 维护者泄露的凭证入侵其 CI/CD 流水线。
推荐理由：原文梳理了从开源依赖到多家 AI 厂商的连锁泄露路径，读者可据此理解共享第三方数据供应商的集中风险。
Hugging Face Blog A New Framework for Evaluating Voice Agents (EVA)
Introduction Conversational voice agents present a distinct evaluation challenge: they must simultaneously satisfy two objectives — accuracy (completing the user's task correctly and faithfully) and conversational experience (doing so naturally, concisely, and in a way appropriate for spoken interaction). These objectives are deeply intertwined: mishearing a confirmation code renders perfect LLM reasoning meaningless, a wall of options overwhelms a caller who can't skim spoken output, and delayed responses can pass every accuracy check while remaining unusable in practice. Existing frameworks treat these as separate concerns — evaluating task success or conversational dynamics, but not both. We introduce EVA, an end-to-end evaluation framework for conversational voice agents that evaluates complete, multi-turn spoken conversations using a realistic bot-to-bot architecture. EVA produces two high-level scores, EVA-A (Accuracy) and EVA-X (Experience), and is designed to surface failures along each dimension. EVA is the first to jointly score task success and conversational experience. We release EVA with an initial airline dataset of 50 scenarios covering flight rebooking, cancellation handling, vouchers, and more — the first in a planned series of domains. We also provide benchmark results for 20 cascade and audio-native systems, such as speech-to-speech (S2S) models and Large Audio Language Models (LALMs). Our biggest finding is that there is a consistent Accuracy-Experience tradeoff; agents that perform well on task completion tend to deliver worse user experiences, and vice versa. 🌐 Website — Explore the full framework, early results, and a demo. 💻 GitHub — Dive into the code, framework, and judge prompts. 📂 HF Dataset - Explore the dataset Background and Motivation The field currently lacks a framework that evaluates the full quality of voice agent interactions, as most existing efforts assess individual components in isolation. For example, AudioBench , SD-Eval , VoxEval , Kimi-Audio-Evalkit , VoiceBench and VoxDialogue evaluate core speech understanding capabilities for Speech-to-Text (STT) — transcription, paralinguistics, acoustic cues — but remain confined to single-turn, non-interactive settings. On the other hand, EmergentTTS-Eval and SHEET assess perceived speech quality using subjective listening tests (e.g., Mean Opinion Score). Beyond speech perception, FD-Bench , Talking Turns , Full-Duplex-Bench provide deeper analyses of conversational dynamics — interruptions, backchanneling, turn-taking — yet evaluate these in isolation from task-oriented tool use, leaving the relationship between dialogue quality and agentic capability unexamined. More recent efforts, notably VoiceAgentBench and CAVA , take steps towards evaluating the agentic capabilities of commercial voice agent systems, including tool-calling and complex instruction-following. However, these voice-agentic capabilities are not evaluated within complete conversational workflows that voice agents must navigate in practice: from initial user request through multi-step tool orchestration to final task resolution. The lack of frameworks that jointly capture accuracy and experience underscores the need for a framework that treats voice agent quality as an integrated whole. This means evaluating not only whether the task succeeded, but whether the agent communicated accurately, concisely, and naturally throughout, and surfacing how these dimensions trade off against one another in realistic deployment conditions. EVA The Framework End-to-end evaluation reveals interaction dynamics that are not apparent at the component level: whether the agent interrupts users during natural pauses in speech, whether it recovers smoothly when a user corrects a transcription error, or whether high latency disrupts the conversational flow enough to prompt users to repeat themselves or abandon the task entirely. EVA simulates multi-turn spoken conversations over live audio in which the agent must invoke appropriate tools, adhere to task-specific policies, and reach a deterministically verifiable end state. EVA evaluates voice agents using a bot-to-bot audio architecture composed of five core components: User Simulator — A conversational AI configured with a specific goal and persona that plays the role of a caller. It operates in audio using high-quality Text-to-Speech (TTS) models, ensuring the evaluation captures representative speech-understanding challenges in natural-sounding conversational speech and realistic turn-taking dynamics. Voice Agent — The voice agent being evaluated, built with Pipecat, an open-source Python framework for real-time voice applications. EVA supports both cascade architectures (STT → LLM → TTS) and audio-native models (S2S or LALM → TTS). Tool Executor — The engine that provides deterministic, reproducible tool responses via custom Python functions. It dynamically queries and modifies a predefined per-scenario database. Validators — A set of validation metrics that check that conversations are complete and that the user faithfully reproduced the intended behavior and speech, with no human annotation required. Any conversation that fails in this validation step is regenerated, ensuring that only valid, correctly executed conversations enter evaluation. This stands in contrast to approaches that rely on post-hoc human labeling to identify simulator errors. Metrics Suite — A suite of metrics evaluates the voice agent using the conversation recording, transcript, and tool call logs. Data Each test case (scenario) in our framework is an evaluation record, structured to make tests reproducible: User Goal — What the caller is trying to accomplish. Includes a highly specific user objective with an exact decision tree that guides the user simulator through the conversation, leaving no ambiguity about the intended outcome. User Persona — How the caller should behave — their speaking style, patience level, and personality traits. Scenario Database — The backend data the agent's tools will query. Ground Truth — The expected final state of the scenario database after a successful conversation. We release EVA with a synthetic airline dataset of 50 scenarios and 15 tools, spanning IRROPS rebooking, voluntary itinerary changes, cancellations, same-day standby, and compensation vouchers. Scenarios are designed to test temporal reasoning, policy-following, constraint satisfaction, and named-entity handling. See the full demo here . Evaluation Methodology EVA evaluates voice agents across two fundamental dimensions, EVA-A for accuracy, and EVA-X for experience. EVA also includes a set of diagnostic metrics. Unlike the primary metrics, these are not used directly to compare or rank models — rather, they offer granular insight into why a model scores the way it does, helping identify and understand specific failure modes (e.g., ASR, speech synthesis, etc.). We report pass@k (the probability that at least one of k runs succeeds) and pass^k (the probability that all k runs succeed) across three trials per scenario (k = 3), capturing both peak performance and behavioral consistency. EVA uses two evaluation methods: deterministic code-based metrics, which compute scores directly from structured data and are fast; and LLM-as-Judge metrics, which use Large Language Models (LLMs) to assess qualitative aspects of the conversation, or Large Audio Language Models (LALM) to evaluate speech directly. Each judge-based metric uses the model that performs best on a curated evaluation dataset for that specific metric. EVA-A: Accuracy Task completion alone is a necessary but insufficient measure of accuracy. An agent can reach the correct end state while fabricating a policy detail, misreading a confirmation code aloud, or hallucinating a flight number mid-conversation. These failures are invisible to a binary pass/fail check but directly harm users. EVA-A therefore measures three dimensions of accuracy: Task Completion [Deterministic] — Measures whether the agent correctly completed the task by comparing the expected end state of the scenario database against the actual end state after the conversation. Faithfulness [LLM-as-Judge] — Measures whether the agent's responses were grounded in its instructions, policies, user inputs, and tool call results — flagging fabrications, misrepresentations, policy violations, and hallucinations. Speech Fidelity [LALM-as-Judge] — Measures whether the speech system faithfully reproduced the intended text in spoken audio, with particular focus on entities critical to get right in a voice context, such as confirmation codes, flight numbers, and dollar amounts. This is the only metric in any end-to-end voice agent benchmark that evaluates the quality of the agent's own spoken output at the audio level. EVA-X: Experience Turn-taking timing matters, but it tells only part of the story. An agent can have perfect timing while overwhelming a caller with a wall of spoken options they cannot skim, or repeatedly asking for information already given. These failures degrade the experience without ever involving a mistimed response. EVA-X therefore measures three dimensions of experience: Conciseness [LLM-as-Judge] — Measures whether the agent's responses were appropriately brief and focused for spoken delivery, since phone users cannot skim, re-read, or scroll back through long responses. Conversation Progression [LLM-as-Judge] — Measures whether the agent moved the conversation forward effectively — avoiding repetition, retaining context across turns, and driving toward task completion without stalling. Turn-Taking [LLM-as-Judge] — Measures whether the agent spoke at the right time — neither interrupting the user nor introducing excessive silence after they finish speaking. Findings We evaluated 20 systems — proprietary and open-source, cascade and audio-native — and find a consistent accuracy-experience tradeoff: agents that perform well on task completion tend to deliver worse user experiences, and vice versa — a tradeoff invisible to benchmarks that score only task completion. No single configuration dominates both axes, confirming that accuracy and experience must be measured jointly. Additionally, we identified named entity transcription as a dominant failure mode. A single misheard character can cascade into an authentication failure and a full conversation breakdown. Also, multi-step workflows break agents in predictable ways. Rebooking a flight while preserving ancillary services — seats, baggage — is the dominant complexity breaker across all configurations. Finally, we observed that additional calibration is needed for real-world use cases. The gap between pass@3 and pass^3 is substantial across all configurations. Even agents that can complete a task often cannot do so consistently, which is critical for real-world success. View the early results here . Limitations EVA is designed to provide rigorous, end-to-end evaluation of conversational voice agents, but several limitations are important to acknowledge, across the framework, data, and metrics dimensions: Metrics — LLM-as-judge models carry inherent biases and may favor certain response styles independent of quality, with additional risk of systematic bias when the evaluated and judge models share a provider. While we validate our judges against labeled datasets and report accuracy measurements on our website, these alignment scores do not eliminate systematic bias entirely. Additionally, task completion is measured as binary, which does not capture partial credits and may understate the relative quality of systems that fail gracefully versus catastrophically. Simulation — The current release covers 50 English-language scenarios in a single domain (airline); results may not generalize to other domains, languages, or accents. Also, the user simulator may not perfectly replicate real caller behavior (e.g., disfluencies, hesitations, emotions) or guarantee full policy adherence. Framework — The user simulator relies on a single commercial provider whose voice characteristics may systematically favor certain ASR systems, and the bot-to-bot pipeline — including audio format conversions and real-time audio interfaces — may not fully represent production deployments. Also, full reproduction requires commercial API access, and latency measurements will vary across providers and infrastructure. What's Next On the evaluation side, we plan to add prosodic quality assessment (pronunciation, rhythm, expressiveness) — currently an open problem after finding very low alignment between LALM-as-Judge and human judgments. We also plan robustness testing under noisy conditions, diverse accents, multilingual users, and varied speaker behaviors, alongside affect-aware evaluation of how agents respond to user distress. In terms of data, we are developing additional domain datasets — each with distinct policy structures, named entity profiles, and conversational dynamics — and more complex scenarios involving compound requests, multi-step follow-ups, and longer conversational memory. On the tooling front, we will release a results and error analysis application that automatically identifies errors per metric and model, surfaces representative examples for exploration, and generates structured summaries of each model’s strengths and weaknesses. Finally, we intend to expand the leaderboard continuously to provide an up-to-date assessment of voice agent capabilities across the field. View more details about limitations and our upcoming roadmap here . Acknowledgements Core contributors include Tara Bogavelli, Gabrielle Gauthier Melançon, Katrina Stankiewicz, Oluwanifemi Bamgbose, Hoang Nguyen, Raghav Mehndiratta, and Hari Subramani. We also thank Lindsay Brin, Akshay Kalkunte, Joseph Marinier, Jishnu Nair, and Aman Tiwari for their careful data review and thoughtful contributions to the framework, and Fanny Riols, Anil Madamala, Sridhar Nemala, and Srinivas Sunkara for their management, leadership, and support throughout. We also extend our thanks to the PAVA and CLAE ServiceNow teams, whose prior work on evaluations and voice agents provided valuable inspiration for this project. Citation @misc{eva-2026, title={A New End-to-end Framework for Evaluating Voice Agents (EVA)}, author={Bogavelli, Tara and Gauthier Melançon, Gabrielle and Stankiewicz, Katrina and Bamgbose, Oluwanifemi and Nguyen, Hoang and Mehndiratta, Raghav and Subramani, Hari}, year={2026}, url={https://github.com/ServiceNow/eva} }
OWASP精选推荐指数6060 OWASP GenAI Security Project 在 RSA 2026 前发布 LLM 与 Agentic 安全新版指南
OWASP GenAI Security Project 宣布发布 Q2 2026 Updated Landscape Guide for LLM and Agentic Security，新增更新的厂商与工具生态文档以及新的 agentic 红队分类法，覆盖识别、度量、缓解和治理 AI 风险的全生命周期框架。
推荐理由：原文列出了新增的 Agentic 红队分类法和 MCP 安全指南等具体产出，读者可据此了解 OWASP 在 Agent 安全方向的框架布局。
Fortinet推荐指数4646 世界经济论坛《Global Cybersecurity Outlook 2026》发布：CISO 必读要点解读
世界经济论坛发布《Global Cybersecurity Outlook 2026》，Fortinet 参与报告撰写并将在 Fortinet Accelerate 大会圆桌讨论中解读关键发现。
Anthropic精选推荐指数7878 Anthropic 红队逆向分析 Claude 为 CVE-2026-2796 编写的漏洞利用
Anthropic 前沿红队公开了 Claude Opus 4.6 为 Firefox 的 CVE-2026-2796（已修复）编写漏洞利用的逆向分析过程。
推荐理由：Anthropic 红队复盘 Claude Opus 4.6 自主写出 Firefox 漏洞利用的完整过程，可看到模型如何一步步搭出利用原语。
腾讯玄武实验室精选推荐指数8080 腾讯玄武实验室提出 Agentic Coding 幽灵依赖供应链威胁并发布 Atuin 防护插件
腾讯玄武实验室提出 Agentic Coding 范式下的新型供应链安全威胁幽灵依赖，包含版本幽灵（LLM 倾向引入过时组件版本）和名称幽灵（LLM 捏造不存在的组件名）两类风险，并定义了利用过时组件 N-day 漏洞和针对幻觉组件名定向投毒两种攻击方式。
推荐理由：研究给出了 Agentic Coding 场景下幽灵依赖的威胁模型与实验数据，读者可据此理解 AI 自主供应链决策带来的新型攻击面及防御思路。
Fortinet推荐指数3434 Fortinet：认证为何成为 CISO 的战略控制手段
Fortinet 目前维持超过 130 项有效认证，最新一项是其安全产品开发生命周期（SPDL）获得 IEC 62443-4-1 成熟度等级 2（ML2）认证。
Hugging Face Blog OpenEnv in Practice: Evaluating Tool-Using Agents in Real-World Environments
AI agents often perform impressively in controlled research settings, yet struggle when deployed in real-world systems where they must reason across multiple steps, interact with real tools and APIs, operate under partial information, and recover from errors in stateful, permissioned environments—highlighting a persistent gap between research success and production reliability. OpenEnv is an open-source framework from Meta and Hugging Face designed to address this challenge by standardizing how agents interact with real environments. As part of this collaboration, Turing contributed a production-grade calendar management environment to study tool-using agents under realistic constraints such as access control, temporal reasoning, and multi-agent coordination. In this post, we explore how OpenEnv works in practice, why calendars serve as a powerful benchmark for real-world agent evaluation, and what our findings reveal about the current limitations of tool-using agents. What Is OpenEnv? OpenEnv is a framework for evaluating AI agents against real systems rather than simulations . It provides a standardized way to connect agents to real tools and workflows while preserving the structure needed for consistent and reliable evaluation. OpenEnv uses a gym-oriented API ( reset , step , action , observations ) like OpenAI's Gymnasium . Also, OpenEnv uses a standard MCP tool call interface to connect to envs which provides a consistent interface across domains and simulation to production environments. The environments maintain state across multiple actions—enabling long-horizon reasoning—and can connect directly to real APIs and tools such as browsers, code repositories, or calendars. This shifts evaluation from "Can this work in a controlled demo?" to "Can this operate reliably in the real world?" The Calendar Gym: A Production-Grade Benchmark Calendar systems are deceptively complex. While scheduling a meeting seems simple, real-world calendar management requires agents to reason over time, permissions, multiple users, and incomplete information—often across several dependent steps. These properties make calendars a powerful testbed for evaluating tool-using agents outside controlled simulations. To ground OpenEnv in this kind of realistic, demanding use case, Turing built a production-grade calendar management environment referred to as the Calendar Gym . Rather than simulating scheduling in the abstract, it exposes agents to the same constraints they would face in real calendar systems: Access Control Lists across users and calendars, limited visibility into other users' state, and multi-step workflows where actions must be chained in the correct order. Agents interact with a rich set of calendar operations—from listing calendars to modifying events and permissions—and must handle failed actions, incorrect assumptions, and missing permissions. Each session runs in an isolated environment, enabling reliable comparisons across runs. Below is a code example of how to use the Calendar Gym. We explore the environment, discover available tools, list calendars, create an event, and print the result. from openenv_wrapper.client import MCPEnvClient from openenv_wrapper.data_models import MCPAction with MCPEnvClient.from_hub(base_url= "TuringEnterprises/calendar-gym" ) as client: # Connect and reset the environment result = client.reset() print ( "Reset successful:" , result.observation.success) # Discover available tools result = client.step(MCPAction(action_type= "ListToolsAction" )) print ( "Available tools:" , len (result.observation.tools_list)) # List calendars result = client.step(MCPAction( action_type= "ToolCallAction" , tool_name= "calendars_list" , arguments={} )) calendars = result.observation.tool_result[ "items" ] print ( "Calendars:" , calendars) # Create an event result = client.step(MCPAction( action_type= "ToolCallAction" , tool_name= "events_insert" , arguments={ "calendarId" : "primary" , "summary" : "Team Sync" , "start" : { "dateTime" : "2026-01-15T14:00:00Z" }, "end" : { "dateTime" : "2026-01-15T15:00:00Z" } } )) print ( "Event created:" , result.observation.success) Below is an excerpt of what the Calendar Gym returns when you call ListToolsAction . Each entry includes the tool name plus an input schema (what arguments the tool accepts). Click to expand output { "tools_list" : [ { "name" : "calendars_list" , "description" : "List calendars visible to the current user." , "input_schema" : { "type" : "object" , "properties" : { } , "additionalProperties" : false } } , { "name" : "events_insert" , "description" : "Create an event in a calendar." , "input_schema" : { "type" : "object" , "properties" : { "calendarId" : { "type" : "string" } , "summary" : { "type" : "string" } , "start" : { "type" : "object" , "properties" : { "dateTime" : { "type" : "string" } } , "required" : [ "dateTime" ] } , "end" : { "type" : "object" , "properties" : { "dateTime" : { "type" : "string" } } , "required" : [ "dateTime" ] } } , "required" : [ "calendarId" , "summary" , "start" , "end" ] } } ] } What We Learned Evaluating agents in the Calendar Gym revealed consistent patterns which were common across multiple domains. While agents often perform well on individual game like actions, reliability breaks down as tasks become longer, more ambiguous, and more constrained. Multi-step reasoning is the primary bottleneck. Agents struggle to correctly chain actions across longer workflows, suggesting that benchmarks need to test sustained reasoning over multiple dependent steps—not just single tool calls. Ambiguity significantly degrades performance. Agents achieved close to 90% success on tasks with explicit calendar identifiers, but success dropped to roughly 40% when the same tasks were phrased using natural language descriptions. Building stronger lookup and validation into agent loops—rather than relying on the LLM to resolve references unaided—appears essential. Correct tool choice isn't enough. Across failed interactions, more than half of errors stemmed from malformed tool arguments or incorrect ordering, even when the right tool was selected. Reliable agent behavior depends as much on execution quality and structured feedback as on tool selection—environment design matters. These challenges are not unique to scheduling and calendars. They reflect broader limitations that emerge whenever agents operate in changing systems over long periods of time, and they point toward evaluation frameworks that test permissions, partial observability, and multi-step workflows together. Looking Ahead OpenEnv provides a foundation for testing agents under realistic conditions, and the Calendar Gym demonstrates how seemingly simple domains can surface deep challenges in reasoning, ambiguity resolution, and tool use. By evaluating agents where failure is measurable and constraints are real, we gain clearer insight into what it takes to build agents that operate reliably in production. For a deeper dive into the Calendar Gym's design, benchmarking methodology, and quantitative results, explore the full technical article on Turing's site . To explore a clone of the Calendar Gym, visit the Calendar Gym space . Appendix: Common error cases in tool use In practice, tool integrations rarely fail in dramatic ways; they fail in small, predictable ones. When wiring up MCP tools to real APIs (like calendar operations), we encountered a handful of recurring issues. Specific error cases found in the wild Below are three common failure modes we’ve seen in production, along with representative error payloads and mitigation strategies. These examples illustrate not just what can go wrong, but how structured errors can help agents recover gracefully. 1. Schema validation errors (missing or malformed arguments) The agent calls a valid tool (e.g. events_insert ), but the arguments do not match the declared JSON schema. Missing required fields like calendarId Incorrect nesting of start / end Passing a string where an object is expected. Click to expand error payload { "ok" : false , "error_type" : "validation_error" , "tool_name" : "events_insert" , "message" : "Invalid arguments for tool 'events_insert'." , "details" : { "missing_required_fields" : [ "calendarId" , "end" ] , "invalid_fields" : [ { "field" : "start" , "expected_type" : "object" , "received_type" : "string" } ] } } We can mitigate this by providing one canonical example of a correct 'events_insert' call in your prompt. Return structured validation errors so the model can repair and retry instead of failing silently. 2. Permission / authorization errors (401/403) The tool call is syntactically correct, but the API rejects it due to insufficient permissions. Missing OAuth scopes Expired access token User lacks write access to the target calendar Click to expand error payload { "ok" : false , "error_type" : "permission_error" , "tool_name" : "events_insert" , "http_status" : 403 , "message" : "The authenticated user does not have write access to calendar 'primary'." , "remediation" : [ "Ensure the OAuth token includes calendar write scope." , "Verify the user has edit access to the target calendar." , "Reconnect the integration if the token has expired." ] } We can mitigate this by clearly documenting the required OAuth scopes. Return structured, actionable remediation steps so the agent can guide the user instead of retrying the same failing call. Clearly document required OAuth scopes. Return structured, actionable remediation steps so the agent can guide the user instead of retrying the same failing call. 3. Datetime / format errors (RFC3339 & timezone issues) The event is rejected by the API, or it is created at an unexpected time. Missing timezone offset Non-RFC3339 datetime format Incorrect nesting of start.dateTime or end.dateTime Mixing local time and UTC without specifying an offset Click to expand error payload { "ok" : false , "error_type" : "format_error" , "tool_name" : "events_insert" , "message" : "Invalid datetime format for field 'start.dateTime'." , "details" : { "received" : "02/11/2026 9:30 AM" , "expected_format" : "RFC3339 (e.g. 2026-02-11T09:30:00-05:00)" } } We can mitigate this by standardizing on RFC3339 with explicit timezone offsets (e.g. 2026-02-11T09:30:00-05:00). Include at least one correct datetime example in your documentation to anchor model behavior and reduce repair retries.
Anthropic推荐指数8282 Anthropic 评估并缓解 LLM 发现 0-day 漏洞的风险
Anthropic 前沿红队发布研究，称 Claude Opus 4.6 无需专用脚手架或定制提示即可在长期运行模糊测试的代码库中发现高危漏洞，并已找到并验证超过 500 个高危漏洞，开始向开源维护者报告且部分补丁已合入。
Hugging Face Blog Community Evals: Because we're done trusting black-box leaderboards over the community
TL;DR: Benchmark datasets on Hugging Face can now host leaderboards. Models store their own eval scores. Everything links together. The community can submit results via PR. Verified badges prove that the results can be reproduced. Evaluation is broken Let's be real about where we are with evals in 2026. MMLU is saturated above 91%. GSM8K hit 94%+. HumanEval is conquered. Yet some models that ace benchmarks still can't reliably browse the web, write production code, or handle multi-step tasks without hallucinating, based on usage reports. There is a clear gap between benchmark scores and real-world performance. Furthermore, there is another gap within reported benchmark scores. Multiple sources report different results. From Model Cards, to papers, to evaluation platforms, there is no alignment in reported scores. The result is that the community lacks a single source of truth. What We're Shipping Decentralized and transparent evaluation reporting. We are going to take evaluations on the Hugging Face Hub in a new direction by decentralizing reporting and allowing the entire community to openly report scores for benchmarks. At first, we will start with a shortlist of 4 benchmarks and over time we’ll expand to the most relevant benchmarks. For Benchmarks: Dataset repos can now register as benchmarks ( MMLU-Pro , GPQA , HLE are already live). They automatically aggregate reported results from across the Hub and display leaderboards in the dataset card. The benchmark defines the eval spec via eval.yaml , based on the Inspect AI format, so anyone can reproduce it. The reported results need to align with the task definition. For Models: Eval scores live in .eval_results/*.yaml in the model repo. They appear on the model card and are fed into benchmark datasets. Both the model author’s results and open pull requests for results will be aggregated. Model authors will be able to close score PR and hide results. For the Community: Any user can submit evaluation results for any model via a PR. Results get shown as "community", without waiting for model authors to merge or close. The community can link to sources like a paper, Model Card, third-party evaluation platform, or inspect eval logs. The community can discuss scores like any PR. Since the Hub is Git based, there is a history of when evals were added, when changes were made, etc. The sources look like below. To learn more about evaluation results, check out the docs . Model scores in the Hub Why This Matters Decentralizing evaluation will expose scores that already exist across the community in sources like model cards and papers. By exposing these scores, the community can build on top of them to aggregate, track, and understand scores across the field. Also, all scores will be exposed via Hub APIs, making it easy to aggregate and build curated leaderboards, dashboards, etc. Community evals do not replace benchmarks so leaderboards and closed evals with published results are still crucial. However, we believe it's important to contribute to the field with open eval results based on reproducible eval specs. This won't solve benchmark saturation or close the benchmark-reality gap. Nor will it stop training on test sets. But it makes the game visible by exposing what is evaluated, how, when, and by whom. Mostly, we hope to make the Hub an active place to build and share reproducible benchmarks. Particularly focusing on new tasks and domains that challenge SOTA models more. Get Started Read the docs: To learn more about evaluation results, check out the docs . Add eval results: Publish the evals you conducted as YAML files in .eval_results/ on any model repo. Check out the scores on the benchmark dataset . Register a new benchmark: Add eval.yaml to your dataset repo and contact us to be included in the shortlist. The feature is in beta. We're building in the open. Feedback welcome.
腾讯玄武实验室精选推荐指数8686 腾讯玄武实验室发布《AI网络爬虫安全白皮书》并开源SEChrome防护方案
腾讯玄武实验室发布《AI网络爬虫安全白皮书》，系统分析AI时代服务端浏览器/爬虫的攻击面，指出多款产品存在沙箱外远程代码执行（RCE）风险，相关产品服务端用户规模累计超10亿，涵盖多家头部科技企业。
推荐理由：白皮书梳理了服务端浏览器的完整攻击链与四个真实RCE案例，并给出可复用的纵深防御框架和开源方案，工程落地参考价值高。
Hugging Face Blog Alyah ⭐️: Toward Robust Evaluation of Emirati Dialect Capabilities in Arabic LLMs
Arabic is one of the most widely spoken languages in the world, with hundreds of millions of speakers across more than twenty countries. Despite this global reach, Arabic is not a monolithic language. Modern Standard Arabic coexists with a rich landscape of regional dialects that differ significantly in vocabulary, syntax, phonology, and cultural grounding. These dialects are the primary medium of daily communication, oral storytelling, poetry, and social interaction. However, most existing benchmarks for Arabic large language models focus almost exclusively on Modern Standard Arabic, leaving dialectal Arabic largely under-evaluated and under-represented. This gap is particularly problematic as large language models increasingly interact with users in informal, culturally grounded, and conversational settings. A model that performs well on formal newswire text may still fail to understand a greeting, an idiomatic expression, or a short anecdote expressed in a local dialect. To address this limitation, our team introduces Alyah الياه (which means North Star ⭐️ in Emirati), an Emirati-centric benchmark designed to assess how well Arabic LLMs capture the linguistic, cultural, and pragmatic aspects of the Emirati dialect. Benchmark Motivation and Scope The Emirati dialect is deeply intertwined with local culture, heritage, and history. It appears in everyday greetings, oral poetry, proverbs, folk narratives, and expressions whose meanings cannot be inferred through literal translation alone. Our benchmark is intentionally designed to probe this depth. Rather than testing surface-level lexical knowledge, it challenges models on their ability to interpret culturally embedded meaning, pragmatic usage, and dialect-specific nuances. The benchmark covers a wide range of content, including common and uncommon local expressions, culturally grounded greetings, short anecdotes, heritage-related questions, and references to Emirati poetry. The goal is not only to measure correctness, but also to understand where models systematically succeed or fail when confronted with authentic Emirati language use. Dataset Structure Following further development and consolidation, the benchmark has been unified into a single dataset called Alyah . The final benchmark contains 1,173 samples , all collected manually from native Emirati speakers to ensure linguistic authenticity and cultural grounding. This manual curation step was essential to capture expressions, meanings, and usages that are rarely documented in written resources and are difficult to infer from Modern Standard Arabic alone. Each sample is formulated as a multiple-choice question with four candidate answers , exactly one of which is correct. Large language models were used to synthetically generate the distractor choices, after which they were reviewed to ensure plausibility and semantic closeness to the correct answer. To avoid positional bias during evaluation, the index of the correct answer follows a randomized distribution across the dataset. Below is the distribution of word count per query and candidate answers. Alyah spans a broad spectrum of linguistic and cultural phenomena in the Emirati dialect, ranging from everyday expressions to culturally sensitive and figurative language. The distribution across categories is summarized below. Category Number of Samples Difficulty Greetings & Daily Expressions 61 Easy Religious & Social Sensitivity 78 Medium Imagery & Figurative Meaning 121 Medium Etiquette & Values 173 Medium Poetry & Creative Expression 32 Difficult Historical & Heritage Knowledge 89 Difficult Language & Dialect 619 Difficult Below are examples of each category: This composition allows Alyah to jointly evaluate surface-level conversational fluency and deeper cultural, semantic, and pragmatic understanding, with a particular emphasis on dialect-specific language phenomena that remain challenging for current models. Model Evaluation Setup We evaluated a total of 54 language models , comprising 23 base models and 31 instruction-tuned models , spanning several architectural and training paradigms. These include Arabic-native LLMs such as Jais and Allam, multilingual models with strong Arabic support such as Qwen and LLaMA, and adapted or regionally specialized models such as Fanar and AceGPT. For each family, both base and instruction-tuned variants were evaluated in order to understand the impact of alignment and instruction tuning on dialectal performance. All models were evaluated under a consistent prompting and scoring protocol. Responses were assessed for semantic correctness and appropriateness with respect to Emirati usage, rather than literal overlap with a reference answer. This is particularly important for dialectal evaluation, where multiple valid phrasings may exist. For each question category, we estimated difficulty empirically based on model performance. Categories where most models struggled were labeled as harder, while those consistently answered correctly across model families were considered easier. This approach allows difficulty to emerge from observed behavior rather than from subjective annotation alone. Evaluation Results on Alyah (Emirati Dialect) We evaluate a broad set of contemporary Arabic and multilingual large language models on Alyah , using accuracy on multiple-choice questions as the primary metric. The evaluation covers 53 models in total, including 22 base models and 31 instruction-tuned models , spanning Arabic-native, multilingual, and regionally adapted systems. Below is a radar plot showing the performance of top models of different sizes per question category. These results are intended as reference measurements within the scope of Alyah, rather than absolute rankings across all Arabic benchmarks. Base Models Model Accuracy google/gemma-3-27b-pt 74.68 tiiuae/Falcon-H1-34B-Base 73.66 FreedomIntelligence/AceGPT-v2-32B 67.35 google/gemma-3-4b-pt 63.17 QCRI/Fanar-1-9B 62.75 tiiuae/Falcon-H1-7B-Base 60.78 meta-llama/Llama-3.1-8B 58.23 Qwen/Qwen3-14B-Base 57.29 inceptionai/jais-adapted-13b 56.01 Qwen/Qwen2.5-32B 53.03 FreedomIntelligence/AceGPT-13B 50.81 Qwen/Qwen2.5-72B 47.91 Qwen/Qwen2.5-14B 46.8 google/gemma-2-2b 41.86 tiiuae/Falcon3-7B-Base 41.43 Qwen/Qwen3-8B-Base 40.75 tiiuae/Falcon-H1-3B-Base 40.41 Qwen/Qwen2.5-7B 36.57 Qwen/Qwen2.5-3B 35.29 meta-llama/Llama-3.2-3B 35.12 inceptionai/jais-adapted-7b 33.5 Qwen/Qwen3-4B-Base 27.45 Instruction-Tuned Models Model Accuracy falcon-h1-arabic-7b-instruct 82.18 humain-ai/ALLaM-7B-Instruct-preview 77.24 google/gemma-3-27b-it 74.68 falcon-h1-arabic-3b-instruct 74.51 Qwen/Qwen2.5-72B-Instruct 74.6 CohereForAI/aya-expanse-32b 73.66 Navid-AI/Yehia-7B-preview 73.32 FreedomIntelligence/AceGPT-v2-32B-Chat 72.8 Qwen/Qwen2.5-32B-Instruct 71.61 tiiuae/Falcon-H1-34B-Instruct 71.1 meta-llama/Llama-3.3-70B-Instruct 69.74 QCRI/Fanar-1-9B-Instruct 69.22 tiiuae/Falcon-H1-7B-Instruct 65.13 CohereForAI/c4ai-command-r7b-arabic-02-2025 64.54 silma-ai/SILMA-9B-Instruct-v1.0 63.94 FreedomIntelligence/AceGPT-v2-8B-Chat 63.43 CohereLabs/aya-expanse-8b 61.21 yasserrmd/kallamni-2.6b-v1 61.13 yasserrmd/kallamni-4b-v1 60.7 microsoft/Phi-4-mini-instruct 58.57 tiiuae/Falcon-H1-3B-Instruct 57.12 silma-ai/SILMA-Kashif-2B-Instruct-v1.0 48.51 Qwen/Qwen2.5-7B-Instruct 45.44 google/gemma-3-4b-it 46.12 meta-llama/Llama-3.1-8B-Instruct 46.29 meta-llama/Llama-3.2-3B-Instruct 39.64 yasserrmd/kallamni-1.2b-v1 37.77 Qwen/Qwen3-4B 26.26 google/gemma-2-2b-it 26.00 Qwen/Qwen3-14B 26.00 Qwen/Qwen3-8B 25.66 Analysis and Observed Trends Figure 1: Models' accuracy across categories based on size. Figure 2: Models' accuracy across categories based on language. Several trends emerge from the evaluation. Instruction-tuned models generally outperform their base counterparts as shown in Figures 1 and 2. This is particularly the case on questions involving conversational norms and culturally appropriate responses (i.e. the Etiquette & Values Category). Furthermore, it is the case with questions that test imagery and figurative meaning. This can be attributed, to the model’s original strong capabilities with understanding MSA-based imagery and figurative language regardless of the dialect at hand. The models are able to draw patterns of non-literal description regardless of dialect. Generally, the most difficult categories for the models were consistently “Language and Dialect” and “Greeting and Daily expressions” across model sizes as shown in figure 1. These results reflect the current state of Emirati dialect presence in written media, as the dialect is mostly spoken rarely written, which explains its novelty relative to the evaluated models. Nonetheless, there is a clear benefit to instruct models with understanding the dialect (and the other evaluation categories) in comparison to their counterparts, especially in small and medium models. This is particularly noticeable with the Poetry and Creative Expression category, which is where the large instruct models performed marginally better than the smaller models. Figure 3: Evaluated models average accuracy. As shown in Figure 3, even strong multilingual models show notable degradation on the most challenging Alyah questions, suggesting that dialect-specific semantic knowledge is not easily acquired through generic multilingual training alone. It must be noted that while Arabic-native models tend to perform more robustly on culturally grounded content, their performances are not uniform across all categories (figure 2). In particular, questions involving implicit meanings and rare expressions remain difficult across nearly all evaluated models. This highlights a persistent gap between surface-level dialect familiarity and deeper cultural understanding. The high variance in performance across categories , where a model that excels at imagery and figurative meaning may still struggle with poetry or heritage-related creative questions, indicates that dialectal competence is multi-dimensional and cannot be captured by a single score. Figure 3 shows that the highest scoring large model in Jais-2-70B, followed by the two small models jais-2-8B and ALLaM-7B-instruct, which are all Arabic instruct-tuned models. Conclusion and Community Impact This benchmark represents a step toward more realistic and culturally grounded evaluation of Arabic language models. By focusing on the Emirati dialect, we aim to support the development of models that better serve local communities, institutions, and users in the UAE. Beyond model ranking, the benchmark is intended as a diagnostic tool to guide future data collection, training, and adaptation efforts. We invite researchers, practitioners, and the broader community to use the benchmark, explore the results, and share feedback. Community input will be essential to refining the dataset, expanding coverage, and ensuring that dialectal Arabic receives the attention it deserves in the evaluation of Large Language Models. Citation @misc{emirati_dialect_benchmark_2026, title = {Alyah: An Emirati Dialect Benchmark for Evaluating Arabic Large Language Models}, author={Omar Alkaabi and Ahmed Alzubaidi and Hamza Alobeidli and Shaikha Alsuwaidi and Mohammed Alyafeai and Leen AlQadi and Basma El Amel Boussaha and Hakim Hacid}, year = {2026}, month = {january}, }
Hugging Face Blog AssetOpsBench: Bridging the Gap Between AI Agent Benchmarks and Industrial Reality
AssetOpsBench is a comprehensive benchmark and evaluation system with six qualitative dimensions that bridges the gap for agentic AI in domain-specific settings, starting with industrial Asset Lifecycle Management. Introduction While existing AI benchmarks excel at isolated tasks such as coding or web navigation, they often fail to capture the complexity of real-world industrial operations. To bridge this gap, we introduce AssetOpsBench , a framework specifically designed to evaluate agent performance across six critical dimensions of industrial applications. Unlike traditional benchmarks, AssetOpsBench emphasizes the need for multi-agent coordination—moving beyond `lone wolf' models to systems that can handle complex failure modes, integrate multiple data streams, and manage intricate work orders. By focusing on these high-stakes, multi-agent dynamics, the benchmark ensures that AI agents are assessed on their ability to navigate the nuances and safety-critical demands of a true industrial environment. AssetOpsBench is built for asset operations such as chillers and air handling units. It comprises: 2.3M sensor telemetry points 140+ curated scenarios across 4 agents 4.2K work orders for diverse scenarios 53 structured failure modes Experts helped curate 150+ scenarios. Each scenario includes metadata: task type, output format, category, and sub-agents. The tasks designed span across: Anomaly detection in sensor streams Failure mode reasoning and diagnostics KPI forecasting and analysis Work order summarization and prioritization Evaluation Framework and Overall Feedback AssetOpsBench evaluates agentic systems across six qualitative dimensions designed to reflect real operational constraints in industrial asset management. Rather than optimizing for a single success metric, the benchmark emphasizes decision trace quality, evidence grounding, failure awareness, and actionability under incomplete and noisy data. Each agent run is scored across six criteria: Task Completion Retrieval Accuracy Result Verification Sequence Correctness Clarity and Justification Hallucination rate Across early evaluations, we observe that many general-purpose agents perform well on surface-level reasoning but struggle with sustained multi-step coordination involving work orders, failure semantics, and temporal dependencies. Agents that explicitly model operational context and uncertainty tend to produce more stable and interpretable trajectories, even when final task completion is partial. This feedback-oriented evaluation is intentional: in industrial settings, understanding why an agent fails is often more valuable than a binary success signal. Failure Modes in Industrial Agentic Workflows A central contribution of AssetOpsBench is the explicit treatment of failure modes as first-class evaluation signals in agentic industrial workflows. Rather than treating failure as a binary outcome, AssetOpsBench analyzes full multi-agent execution trajectories to identify where , how , and why agent behavior breaks down under realistic operational constraints. Failure analysis in AssetOpsBench is implemented through a dedicated trajectory-level pipeline ( TrajFM ), which combines LLM-based reasoning with statistical clustering to surface interpretable failure patterns from agent execution traces. This pipeline operates in three stages: (1) trajectory-level failure extraction using an LLM-guided diagnostic prompt, (2) embedding-based clustering to group recurring failure patterns, and (3) analysis and visualization to support developer feedback and iteration. Across industrial scenarios, recurrent failure modes include: Misalignment between sensor telemetry, alerts, and historical work orders Overconfident conclusions drawn under missing, delayed, or insufficient evidence Inconsistent aggregation of heterogeneous data modalities across agents Premature action selection without adequate verification or validation steps Breakdowns in multi-agent coordination, such as ignored inputs or action–reasoning mismatches Importantly, AssetOpsBench does not rely solely on a fixed, hand-crafted failure taxonomy. While a structured set of predefined failure categories (e.g., verification errors, step repetition, role violations) is used for consistency, the system is explicitly designed to discover new failure patterns that emerge in practice. Additional failure modes identified by the LLM are embedded and clustered automatically, allowing the taxonomy to evolve as new agent designs and behaviors are evaluated. To preserve industrial confidentiality, raw execution traces are never exposed. Instead, agents receive aggregated scores across six evaluation dimensions together with clustered failure-mode summaries that explain why an agent failed, without revealing sensitive data or intermediate reasoning steps. This feedback-driven design enables developers to diagnose weaknesses, refine agent workflows, and iteratively resubmit improved agents. This failure-aware evaluation reflects the realities of industrial asset management, where cautious, degradation-aware reasoning—and the ability to recognize uncertainty, defer action, or escalate appropriately—is often preferable to aggressive but brittle automation. Submit an Agent for Evaluation AssetOpsBench-Live is designed as an open, competition-ready benchmark , and we welcome submissions of agent implementations from the community. Agents are evaluated in a controlled, privacy-preserving environment that reflects real industrial asset management constraints. To submit an agent, developers first validate their implementation locally using a provided simulated environment, which includes representative sensor data, work orders, alerts, and failure-mode catalogs. Agents are then containerized and submitted for remote execution on hidden evaluation scenarios. Submitted agents are evaluated across six qualitative dimensions—task completion, accuracy, result verification, action sequencing, clarity, and hallucination—using a consistent, reproducible evaluation protocol. Execution traces are not exposed; instead, participants receive aggregated scores and structured failure-mode feedback that highlights where and why an agent’s reasoning or coordination broke down. This feedback-driven evaluation loop enables iterative improvement: developers can diagnose failure patterns, refine agent design or workflow structure, and resubmit updated agents for further evaluation. Both planning-focused and execution-focused agents are supported, allowing researchers and practitioners to explore diverse agentic designs within the same benchmark framework. Experiment and Observations We performed a community evaluation where we tested two tracks: Planning-oriented multi-agent orchestration Execution-oriented dynamic multi-agent workflow. Across 225 users and 300+ agents and leading open source models, here are the observations: Model Family Best Planning Score Best Execution Score Key Limitation GPT-4.1 68.2 72.4 Hallucinated completion on complex workflows Mistral-Large 64.7 69.1 Struggled with multi-hop tool sequences LLaMA-4 Maverick 66.0 70.8 Missed clarifying questions (fixable) LLaMA-3-70B 52.3 58.9 Collapsed under multi-agent coordination Note: None of the models could pass our evaluation criteria benchmark and get 85 points, which is the threshold for deployment readiness. Distribution of Failures Across 881 agent execution traces, failure distribution was as follows: Ineffective Error Recovery: 31.2% Overstated Completion: 23.8% Formatting Issues: 21.4% Unhandled Tool Errors: 10.3% Ignored Feedback: 8.0% Other: 5.3% Beyond this, 185 traces had one new failure pattern and 164 had multiple novel failures. Key Error Findings "Sounds Right, Is Wrong": Agents claim to have completed tasks (23.8%) and output success even after unsuccessful failure recovery (31.2%). AssetOps benchmarking is important to uncover this so that operators do not act upon incorrect information. Tool Usage: This is the biggest differentiator between high and low performing agents, with top agents having 94% tool accuracy compared to 61% of low performers. Multi-agent Multiplies Failures: Task accuracy between single agent (68%) vs multi-agent (47%) shows the complexity multi-agent brings with context loss, asynchronous issues, and cascaded failures. Domain Knowledge: Agents with access to failure mode databases and maintenance manuals performed better. However, RAG knowledge wasn’t always used correctly, suggesting a need for structured reasoning. Ambiguity: Missing sensors, conflicting logs, and vague operator descriptions caused the success rate to drop 34%. Agents must have clarification strategies embedded. Where to get started? Read our technical report AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance How to run AssetOpsBench locally - Video AssetOpsBench Local Execution Try out AssetOpsBench in the HuggingFace Space Playground Find More Detail AssetOpsBench GitHub , fork the repo and get started.
Anthropic精选推荐指数7272 Anthropic 红队：Claude Sonnet 4.5 在真实网络靶场中自主利用公开 CVE
Anthropic 前沿红队与 Incalmo 合作，在 25-50 台主机的模拟网络靶场上评测 Claude，发现 Claude Sonnet 4.5 在少数网络上无需此前几代所需的定制网络工具包即可成功。
推荐理由：Anthropic 红队披露 Claude Sonnet 4.5 在真实网络靶场中无需专用工具即可自主利用公开 CVE，并给出与上一代的对照数据。
Anthropic推荐指数4848 Anthropic 用 Claude 与基于属性的测试在 Python 生态中发现数百个 bug
Anthropic 开发了一个基于 Claude Code 自定义命令的 AI agent，通过推断代码应满足的通用属性并用 Hypothesis 编写基于属性的测试，在 NumPy、SciPy、Pandas 等热门 Python 包中发现数百个潜在 bug。
Fortinet推荐指数4040 Fortinet：2026 年 CISO 的韧性之年——五大战略应对 AI 驱动的风险
Fortinet 在《CISO Predictions for 2026》中指出，AI 加速了创新也放大了风险，CISO 的角色正从安全负责人转向首席韧性官（Chief Resilience Officer）。