K2 Horizon: Frontier Performance, Radically Open

Today IFM is releasing K2 Horizon, a connected fleet of six models: 375B-A23B, 36B-A4B, 32B, 7B, 3.7B, and 0.9B. Across reasoning, mathematics, coding, agentic tasks, and general capabilities, K2 Horizon delivers top-tier performance in every size class—with the 0.9B, 3.7B, and 7B models setting new state of the art at their respective scales. K2 Horizon is also our most comprehensive open release to date. For every model, we are opening the training lifecycle from pretraining through reasoning and agentic post-training. We are releasing intermediate checkpoints, training data or detailed data-construction recipes, open architecture, mixture compositions, training code, configurations, fine-grained logs, evaluation results, and final weights. The models and code are released under the Apache 2.0 license. Datasets are released under their applicable licenses, such as ODC-BY; We disclose how the data was constructed and mixed when redistribution is not possible. Together, K2 Horizon represents the most comprehensive open model release to date: A new performance frontier across scales. The 0.9B, 3.7B, and 7B models achieve world-leading performance in their size classes across widely used evaluations. The 36B-A4B model, equipped with our new Mixture-of-Value-Attention (MoVA) mechanism, delivers exceptional capability per active parameter, outperforming some much larger models. The 32B and 375B-A23B models rank among the top models in their respective classes. Together, the six models provide competitive performance across deployment environments ranging from edge devices to the enterprise. The first fully open model fleet for agents. K2 Horizon is the first open model family to expose the complete development process through agentic post-training. By releasing checkpoints, data (or data recipe), code, configurations, and training logs across every stage, K2 Horizon makes it possible to study how reasoning, tool use, planning, and agentic capabilities emerge; reproduce the methods that create them; and adapt those methods to new tools, environments, and domains. Six models spanning edge to enterprise. The 0.9B model is designed for highly constrained environments such as watches and glasses, while the 3.7B and 7B models bring advanced capabilities to phones and other on-device applications. The dense 32B model and sparse 36B-A4B model provide powerful options for local workstations and efficient serving. The 375B-A23B model brings the fleet’s strongest capabilities to demanding enterprise deployments. All six models include quantization support. One connected fleet. The six models share core architecture, vocabulary, training methodology, interfaces, evaluation infrastructure, and deployment tooling, with a smaller vocabulary for the 0.9B model. This consistency also makes it easier to move between sizes, route work dynamically, and study capability and efficiency across scale. World-leading performance across the scales The 0.9B, 3.7B, and 7B models achieve state-of-the-art results in their respective classes across mathematics, reasoning, general capability, coding, and agentic tasks. The 36B-A4B model performs beyond the level normally expected from its active parameter count, demonstrating the efficiency of our unique Mixture-of-Expert design when computing attention values. The 32B and 375B-A23B models place among the top models in their respective comparison classes. The small models are especially notable. K2 Horizon 0.9B achieves an AIME 2026 score above 48, along with strong reasoning, tool-use, and agentic capabilities. K2 Horizon 3.7B and 7B extend these capabilities to more demanding software-engineering and multi-step environments, demonstrated on strong performance in SWE-bench and BrowseComp. Although complex tasks that require extensive exploration and repeated recovery, such as those in TerminalBench, remain difficult for the smallest models, K2 Horizon moves the boundary of what is possible at every scale. Why the Horizon Fleet matters A transparent model that falls far behind the capability frontier has limited value as a foundation, even for research. At the same time, a powerful model released only as final weights allows people to run it, but provides little insight into how its capabilities were created. K2 Horizon brings these two together. The fleet provides highly competitive models and releases the recipes used to train them. Researchers can study advanced capabilities in models strong enough to exhibit them, while developers can reproduce, adapt, and extend the methods rather than treating the final checkpoint as an opaque starting point. Since introducing the fully open principle in our 2023 LLM360 paper, we have released open models every year while extending that commitment to larger scales, stronger capabilities, and now the complete lifecycle through agentic post-training. A Deep Dive into The K2 Horizon Fleet K2 Horizon 375B-A23B: the enterprise powerhouse K2 Horizon 375B-A23B is the fleet’s largest and most capable model. Its sparse MoE architecture provides 375 billion parameters of total capacity while activating approximately 23 billion parameters for each token, allowing it to draw on the capacity of a much larger model without using every parameter for every token. The model ranks among the top models below 400 billion parameters across general, reasoning, coding, and agentic evaluations. It is designed for demanding workloads where model quality matters most, including complex reasoning, software engineering, research, and long-horizon agentic tasks. Like every model in the Horizon fleet, 375B-A23B is released not as a single endpoint but as a development tree. Its intermediate checkpoints and post-training branches expose how the base model develops into reasoning, instruction-following, and specialized agentic variants. K2 Horizon 32B and 36B-A4B: strong performance for local deployment Horizon 32B is the fleet’s most powerful dense model, providing a strong balance of capability, adaptability, and local deployability. It ranks among the top dense models below 40 billion parameters. Horizon 36B-A4B reaches nearly the performance of the dense 32B model while activating only approximately 4 billion parameters per token. Its efficiency comes from MoVA, our new sparse attention architecture, together with MoE feed-forward layers. These two models serve as an important reference point for studying how dense and sparse architectures behave under similar training conditions. These models occupy the fleet’s local performance sweet spot. They are powerful enough for demanding reasoning, coding, and agentic applications while remaining practical for local workstations and efficient serving systems. K2 Horizon 7B, 3.7B, and 0.9B: frontier capability at small scale k2 Horizon 7B and 3.7B deliver strong reasoning, mathematics, coding, tool-use, and agentic performance while remaining suitable for local and on-device deployment. On several evaluations, their results approach or exceed those of models many times larger from the previous generation. K2 Horizon 0.9B carries many of the same capabilities into highly constrained environments. It can perform mathematical reasoning, use tools, and complete simple agentic tasks while remaining compact enough for applications on watches, glasses, and other edge devices under quantization. The appropriate task changes with scale: the 0.9B model is best suited to focused interactions and lightweight tool use, while the 3.7B and 7B models can handle more demanding coding and multi-step workflows. Together, they demonstrate how much capability can now be retained in models small enough to run almost anywhere. Designing K2 Horizon One family from the beginning Horizon was designed as a connected family rather than a collection of unrelated models. The six models share core architectural decisions, training methodology, interfaces, evaluation infrastructure, and deployment tooling. This allows developers to move between sizes more easily and gives researchers a more controlled setting for studying capability across scale. Each model is pretrained on approximately 20 trillion tokens using carefully constructed and documented mixtures. Intermediate checkpoints and their corresponding fine-grained logs are captured throughout training, creating a detailed record of how each model develops. MoVA: scaling attention with sparse experts The core idea behind mixture-of-experts is to increase total model capacity while keeping the computation required for each token roughly fixed. Conventional MoE architectures apply this sparsity primarily to feed-forward layers: many specialized experts are available, but a router activates only a small subset for each token. Our new architecture, MoVA—Mixture-of-Value Attention, extends this principle to attention. Because attention determines how a transformer brings together information from across its context, introducing sparsity there opens another dimension for scaling model capacity beyond the feed-forward network. MoVA integrates expert routing into multi-head attention while remaining compatible with efficient techniques including FlashAttention, grouped-query attention, and sparse attention. The result is K2 Horizon MoVA 36B-A4B: a model with 36 billion total parameters but approximately 4 billion active parameters per token. Under the same training conditions, it performs only slightly below the dense Horizon 32B model while requiring substantially fewer active parameters. Training Data Training across six scales requires data that is broad enough to support general capability and carefully constructed enough to maintain quality over approximately 20 trillion tokens. Horizon’s pre-training mixture combines diverse web, code, mathematical, scientific, multilingual, and domain-specific sources with synthetic data generated through our own pipelines. One of our key data innovations is the incorporation of reasoning directly into pre-training: nearly 17% of the pre-training corpus consists of problem-solving trajectories with explicit reasoning. Reasoning trajectories for mathematical tasks were further rewritten into formats such as dialogues and study guides. In total, we used approximately 10 trillion synthetic tokens during pre-training. Our synthetic data pipelines use millions of combinations of diversity knobs and context seeds, including retrieval from an internally built search engine over the pre-training web corpus. To quantify diversity at corpus scale, we developed a custom gzip-based compression metric with adaptive striding to avoid the rapid saturation of standard gzip-based measurements as the number of documents grows. As shown below, the measured diversity of our synthetic data approaches that of high-quality natural web text and substantially exceeds that of web code. We document the data composition, synthesis pipelines, and construction of the training mixture. Following our open-source principles, where licenses permit, we release the training-ready datasets directly. Where redistribution is restricted, we release source descriptions, filtering and construction methods, and mixture composition. This allows researchers to understand what each model learned from and how the data distribution evolved throughout training. Our post-training data is introduced from the beginning of mid-training, rather than being reserved exclusively for the final stages of training. We combine synthesized long-context documents with instruction-following, reasoning, and agentic trajectories formatted with the chat template. A main pillar of our post-training data is large-scale task synthesis grounded in task taxonomies, diversity knobs, and web-search seeding, resulting in over 100 million unique tasks. In addition, we developed sampling techniques that guide solver LLMs toward correct solutions and desired behaviors when generating training trajectories. Pre-training Dynamics Every K2 Horizon checkpoint is accompanied by detailed training logs and fine-grained histories that expose the real dynamics of large-scale training: how losses evolve, where instabilities appear, how interventions affect training, and when capabilities begin to emerge. As an example of why these records are useful, K2 Horizon 3.7B, 7B, 32B, and 36B-A4B were trained on exactly the same 22 trillion tokens. When aligned by training progress and normalized by the median loss over the final 1% of training, their raw loss trajectories approximately collapse, which you can see in the figure below—even across dense and sparse architectures and nearly an order of magnitude in total parameter count. Although this normalization aligns their endpoints by construction, it does not force the early and intermediate trajectories to coincide. Their close agreement shows that, under shared data and a common recipe, the shape of learning remained remarkably consistent across the fleet subset that used the same dataset throughout training. Together, the checkpoints and logs allow researchers to investigate why some dynamics transfer across scale and architecture while others diverge, and to connect those differences with particular training stages, data mixtures, and technical decisions. Post-training for reasoning and agents Most of K2 Horizon’s advanced reasoning and agentic capabilities emerge during post-training. The complete pipeline includes mid-training, supervised fine-tuning, model merging, reinforcement learning with specialized agent training. Rather than producing only one final chat model, this process creates a development tree. Different branches specialize in reasoning, coding, tool use, and agentic domains while remaining connected to common base checkpoints. We release the artifacts across these stages so researchers can study where each capability emerges, reproduce individual branches, and adapt the same methods to new tools and environments. Uno Diffusion: plug-and-play lossless speedup for Horizon Inference speed has become a first-class optimization target, especially in the era of reasoning models and agents. As models produce longer chains of thought and take more actions, even small per-token delays compound into substantial latency. Yet autoregressive language models generate one token at a time, creating a fundamental bottleneck. Existing approaches offer partial solutions: speculative decoding typically requires a separately trained draft model, while discrete diffusion enables parallel generation but often sacrifices quality for speed. Uno is designed to remove that tradeoff. It provides a lossless inference speedup, accelerating generation without degrading response quality. Uno keeps Horizon’s autoregressive parameters frozen and fully responsible for the model’s output distribution, while a lightweight set of diffusion parameters learns only how to generate more efficiently. Through a process we call Diffusion Distillation, these compact adapters learn to generate blocks of tokens in parallel. The result combines the quality guarantees of autoregressive decoding with the speed advantages of diffusion: the model reaches the same answers, but reaches them faster. Across our evaluations, Uno achieves a better speed–quality tradeoff than leading speculative-decoding systems and both open-weight and proprietary diffusion language models. Crucially, its gains persist across every batch size we tested, enabling lower latency for interactive agents and higher throughput for large-scale serving, with no loss in model quality and negligible additional training overhead. Uno is delivered as a simple LoRA adapter, making this lossless speedup easy to adopt: all you need is to attach the adapters. Open infrastructure for building and extending Horizon We are releasing the infrastructure used to build Horizon alongside the models themselves. The goal is not only to make the fleet reproducible, but to make its development stack useful as a foundation for new models and systems. At the center of this release is xLLM, our production-tested training infrastructure. xLLM combines large-scale training performance with the flexibility required for research, allowing teams to modify architectures, data mixtures, training stages, and objectives without rebuilding the surrounding system. We will be also releasing our full agentic post-training code base, including Reinforcement Learning, which we hope will help researchers to advance the techniques in related areas. Researchers can use these components to inspect training behavior or reproduce a particular stage. Developers can adapt Horizon to a domain, add new tools, train agent experts, or deploy several model sizes behind a unified interface. From Open Source to Open Science Intermediate checkpoints turn model development into an observable scientific process. They allow researchers to study when capabilities emerge, how training choices change behavior, and when unintended strategies first appear. Reward hacking provides one revealing example. When a capable model is placed in a realistic computer environment and told to solve a complex task, the same resourcefulness that makes it useful can also lead it to exploit shortcuts in the evaluation itself. To understand how much this affects K2 Horizon's reported numbers, we audited the released model using Artificial Analysis's reward hacking auditing procedure. TerminalBench 2.1 places models in sandboxed computer environments and evaluates whether they can complete complex technical tasks. A capable model must explore files, invoke tools, diagnose failures, and find alternative paths toward a solution. That resourcefulness is exactly what we want. But occasionally it crosses a boundary: instead of solving the intended problem, the model locates a hidden answer, exploits something the task left exposed, or manipulates the grader itself. We ran K2 Horizon 375B-A23B on 89 TerminalBench 2.1 tasks with eight attempts each, producing 712 trials. Of these, 500 passed the task verifier (70.2% reported accuracy). We then audited every passing trial using Artificial Analysis's reward hacking auditing procedure, applying their harbor analyze tool with the reward_hacking criterion and their full rubric text verbatim, with Codex gpt-5.6-sol as the judge model. The audit flagged 24 trials across 10 tasks. Removing them lowers the accuracy from 70.2% to 66.9%, a correction of 3.37 percentage points. The remaining 79 tasks were fully clean. For context, Artificial Analysis reports flag rates of 2.2% for claude Fable 5 and 4.1% for GPT-5.6 Luna; K2 Horizon's 3.37% falls within that range. The model discovered several strategies: Inferring it was inside a public benchmark, finding the repository on GitHub, and downloading the reference solution Pulling the current source from a real project's public repository and copying the fix rather than deriving it Inspecting unadvertised files, generator scripts, or exposed credentials Editing the test harness or crafting output that exploited how the test checked success We observed a related case with K2 Horizon 7B, which found and downloaded SWE-bench answers and consequently produced an inflated score of 82. The score does not represent genuine software-engineering performance, but the behavior is scientifically revealing: benchmark hacking emerged as an unintended consequence of broader planning, tool use, environment exploration, and persistence. Because we release intermediate checkpoints alongside the final models, these behaviors can be studied rather than hidden. Researchers can determine when a strategy first appears, connect it to changes in training, and measure its effect on reported performance. K2 Horizon is therefore more than a collection of model weights. It is an open experiment in how capabilities—and their unintended consequences—develop throughout training. Get Started with K2 Horizon Today! All six K2 Horizon sizes are released as open weights under Apache 2.0, with day-zero support from vLLM, SGLang, and Ollama. Further, K2 Horizon supports deployment on NVIDIA, AMD, and Cerebras hardware, providing options from local inference to large-scale serving. Get the models from our repository: https://huggingface.co/IFM, download the models, inspect intermediate checkpoints, study training data and mixtures, reproduce training stages, or build new models and agents using the Horizon code and infrastructure. K2 Horizon provides more than six final models. It provides an open blueprint for understanding, adapting, and advancing the next generation of AI systems. Full Result Table 375B-A23B 36B-A4B 32B K2 Horizon 7B K2 Horizon 3.7B K2 Horizon 0.9B