Hy4 Preview: Tencent Open-Sources 770B-Parameter MoE Model with 1M+ Context Window

Chinese technology conglomerate Tencent has officially released and open-sourced Hy4 preview, its latest flagship large language model designed specifically for demanding real-world productivity tasks. Built on an advanced Mixture-of-Experts (MoE) architecture, Hy4 preview comprises 770 billion total parameters-with approximately 49 billion active parameters per token request-and features an expansive context window exceeding 1 million tokens.

The model is available immediately as open-source software across public repositories including GitHub, Hugging Face, ModelScope, and AtomGit. Tencent has simultaneously deployed the model across its enterprise and consumer ecosystems, integrating it into developer-centric tools like CodeBuddy and WorkBuddy, as well as apps like Yuanbao and ima. Tencent is offering a two-week free access window on WorkBuddy and CodeBuddy, while extending complimentary access to its previous Hy3 model through September 30.

Scaling Up: 770B Parameters and a 1M-Token Context Window

Tencent scaled Hy4 preview across three core dimensions: model size, context length, and dataset volume. According to the company, substantial advances across both pre-training and post-training runs resulted in the largest generation-over-generation intelligence leap measured within its Hunyuan AI division, positioning Hy4 preview at the frontier of open-source artificial intelligence.

The 1-million-token context window allows Hy4 preview to parse and analyze extensive multi-file software repositories, long-form legal and financial documentation, and full books without suffering from contextual drift or degradation in retrieval accuracy.

Inside the Architecture: How Hy4 Preview Works

Hy4 preview relies on a specialized Mixture-of-Experts (MoE) topology designed to maximize computing efficiency while keeping inference overhead manageable. Key architectural highlights include:

• Layer Configuration: The model consists of 78 total layers. Layer 0 operates as a dense Feed-Forward Network (FFN), followed by 77 specialized MoE layers.
• Expert Routing Mechanism: The network contains 256 routed experts and 1 permanently active shared expert. For each token, the router dynamically activates 8 routed experts alongside the shared expert, engaging roughly 49 billion parameters.
• Attention & Memory Optimization: The model integrates Gated DeepSeek Sparse Attention, paired with proprietary memory caching layers (IndexCache and iHC) to streamline latency during extended context processing.
• Multi-Token Prediction (MTP): An added MTP speculative decoding layer accelerates token generation throughput during complex inference tasks.

Internal Blind Benchmarks: Outperforming Domestic Competitors

To evaluate performance in production environments, Tencent conducted an internal blind evaluation involving 163 technical experts evaluating model outputs across 203 complex engineering tasks. Hy4 preview achieved an average composite score of 2.99 out of 4.00, slightly outperforming leading domestic models:

• Overall Score Comparison: Hy4 preview (2.99/4.00) vs. Moonshot AI’s Kimi K3 (2.94/4.00) vs. Zhipu AI’s GLM-5.3 (2.92/4.00).
• Head-to-Head vs. GLM-5.3: Hy4 preview recorded a 46.8% win rate, a 12.8% tie rate, and a 40.4% loss rate.
• Head-to-Head vs. Kimi K3: Hy4 preview achieved a 51.2% win rate, a 7.9% tie rate, and a 40.9% loss rate.

Tencent noted that these gains stem from a “model-product co-design” strategy, where foundational model training directly incorporates real-world feedback from internal engineering and product teams.

Software Engineering and Front-End UI Design

Tencent co-developed training datasets with internal specialists in software engineering, cybersecurity, finance, and game design. Hy4 preview shows notable improvements in planning, debugging, and verifying long-horizon codebases.

In one front-end demonstration highlighted by Tencent, the model generated a 3D desktop viewer for seven historic Chinese buildings using Three.js. Moving away from standard dark-mode interfaces, the model crafted a research-file aesthetic featuring a three-column layout, rice-paper textured backgrounds, serif typography, and indigo accents, complete with interactive 3D drag, zoom, and rotation controls.

Instant Game Prototyping from Natural Language

In game development, Hy4 preview can translate a single natural language prompt into a functional, playable prototype compatible with leading game engines like Unity and China’s Tuanjie engine.

In a demonstration provided by the company, the model built a third-person 3D cartoon game featuring a penguin exploring a snowy environment, gathering resources, opening chests, and battling enemies. The model wrote the core game logic and generated menu systems, pause and game-over screens, animation blend trees, an inventory HUD, sound effects, and exposed physics and gameplay variables directly within the engine editor for live tweaking.

Enterprise File Processing and Financial Auditing

Hy4 preview handles multi-document reasoning, turning disparate and unstructured corporate files into formatted reports, financial spreadsheets, and interactive presentations.

In a financial audit scenario demonstrated by Tencent, the model reviewed 24 employee reimbursement claims. It autonomously cross-referenced conflicting policy versions to determine which applied, verified invoices against budget ledgers, approved or adjusted claim amounts, and output a structured audit table citing mathematical breakdowns and flagged suspicious transactions for potential fraud.

Scientific Research and Multi-Session Codex Management

Across pure and applied sciences, Hy4 preview is optimized for complex multi-step reasoning in fields such as AI research, molecular dynamics simulation, condensed-matter physics, and pure mathematics.

According to Tencent’s benchmark data, Hy4 preview successfully managed multiple parallel Codex programming sessions during a model post-training experiment. Acting as a supervisory researcher, the model adjusted experimental parameters based on incoming logs and outperformed an autonomous Codex baseline across all eight standardized evaluation benchmarks tested.

Autonomous Infrastructure: A 31.8% Boost in Inference Throughput

A notable aspect of the Hy4 development cycle was the model’s active participation in optimizing its own training and deployment pipelines. The model contributed to:

• Training & Operator Refinement: Automating data filtering strategies, low-level operator adjustments, and evaluation frameworks.
• Inference Bottleneck Analysis: Independently analyzing hardware communication bottlenecks and optimizing operator fusion.
• Throughput Gains: Achieving a 31.8% increase in end-to-end inference throughput over Tencent’s baseline across varying context lengths and concurrent workloads.

API Pricing and Commercial Availability

Tencent is offering API access via Tencent Cloud TokenHub and OpenRouter with a transparent pricing model:

• Input Tokens: $0.834 per 1 million tokens
• Output Tokens: $2.501 per 1 million tokens
• Cache Hits: $0.042 per 1 million tokens

Model weights are available for download on GitHub and Hugging Face, allowing researchers and enterprises to self-host or fine-tune the architecture for private deployments.

Known Limitations and the “Preview-First” Roadmap

Tencent positioned Hy4 as an early preview build, acknowledging several areas currently undergoing rapid refinement:

• Inference Latency: The model can occasionally spend longer than necessary reasoning through multi-step logic tasks.
• Over-Verification: An inclination to repeatedly verify intermediate steps, which can extend response times.
• Training Headroom: Ongoing headroom remaining in both pre-training and post-training runs.

Adopting a “preview-first” release cadence, Tencent plans to integrate user and developer feedback from this launch into subsequent production builds of the Hy4 family in the coming months.