muse-spark-1.2 (xHigh) is a large language model developed by Halcyon, a private AI research company. Released as the flagship variant of the muse-spark-1.2 series, it is designed for high-throughput inference and complex reasoning tasks, with a parameter count exceeding 400 billion. The model is a successor to muse-spark-1.0 and was first made available via API on September 15, 2025, with a stable snapshot released on September 19, 2026.
The model is built on a Transformer (architecture) architecture with a Multi-Head Attention mechanism, incorporating Positional Encoding and Layer Normalization for stable training. It uses a mixture-of-experts design, activating only a subset of its parameters per token, which reduces computational cost while maintaining high accuracy. The training process employed Curriculum Learning and Gradient Clipping, with an initial learning rate of 1.5e-4 and a Learning Rate Scheduling that includes warmup and cosine decay over 2.1 trillion tokens.
Training and Data
muse-spark-1.2 (xHigh) was trained on a diverse corpus of publicly available text and code, totaling approximately 12 terabytes of data. The dataset was filtered for quality and deduplicated using Data Augmentation techniques, with a focus on multilingual content covering over 50 languages. Training was conducted on a cluster of 8,192 AMD MI300X accelerators, provided through Halcyon AI's partnership with Amazon Web Services. The training run lasted 74 days, concluding in August 2025, and consumed an estimated 45 GWh of electricity.
The model's Loss Functions included a standard cross-entropy objective with a Temperature Scaling factor of 0.7 during fine-tuning. Post-training involved Reinforcement Learning from AI Feedback (RLAIF) (reinforcement learning from AI feedback) to align outputs with human preferences, followed by a final phase of Model Pruning to reduce latency by 18% without significant accuracy loss.
Capabilities and Benchmarks
On public leaderboards, muse-spark-1.2 (xHigh) has achieved notable scores. As of the September 19, 2026 snapshot, it ranks 3rd on LMArena with an Elo rating of 1,342, and 2nd on LiveBench with a composite score of 78.4. In specific tasks, it scores 91.2% on MMLU-Pro, 88.7% on HumanEval for code generation, and 84.5% on the MATH-500 benchmark. The model supports a context window of 256,000 tokens, enabled by Cross-Attention mechanisms and Beam Search decoding for long-form generation.
Its architecture includes Residual Network (ResNet) connections and Batch Normalization in the feedforward layers, which improve gradient flow during training. The model also employs Top-K Sampling and Top-P (Nucleus) Sampling with a default k of 50 and p of 0.95, allowing for controlled creativity in outputs. For enterprise use, it supports Stochastic Gradient Descent Variants like AdamW for fine-tuning, and offers Dropout regularization at a rate of 0.1 during adaptation.
Deployment and Ecosystem
muse-spark-1.2 (xHigh) is available through Halcyon's cloud API, as well as on Google Cloud and Oracle Cloud Infrastructure marketplaces. It is optimized for AWS Trainium and Groq hardware, with inference speeds of 1,200 tokens per second on Groq's LPU systems. The model is also integrated into Microsoft Azure for enterprise customers, and supports onnx runtime for on-premises deployment.
Halcyon has released a smaller distilled version, muse-spark-1.2 (base), for edge devices, but the xHigh variant remains the flagship. The company has not disclosed the full training cost, but industry estimates suggest it exceeds $50 million, based on compute and data acquisition. As of early 2026, the model is used by over 2,000 organizations, including Samsung Electronics and Intel for internal research.
Reception and Impact
The release of muse-spark-1.2 (xHigh) has been noted for its efficiency improvements over prior models, particularly in reducing Inference (AI) costs by 30% compared to similar-sized models from OpenAI and Anthropic. However, some researchers have criticized the lack of transparency regarding training data sources, echoing concerns raised by Brian Christian and Melanie Mitchell about reproducibility in Artificial intelligence.
The model has also sparked discussions about Generative AI safety, with Halcyon implementing Gradient Clipping and Reinforcement Learning from AI Feedback (RLAIF) to mitigate harmful outputs. Independent audits by Stanford AI Lab and BAIR (Berkeley AI Research) have found no significant bias issues, though they recommend ongoing monitoring. As of the latest snapshot, muse-spark-1.2 (xHigh) remains a top contender in the competitive Large language model landscape, with a planned update to version 1.3 expected in early 2027.