# muse-spark-1.2 (xHigh)

muse-spark-1.2 (xHigh) is a large language model developed by Halcyon, first released in September 2025. It is a high-capacity variant of the muse-spark series, optimized for complex reasoning and long-context tasks, and has ranked on public leaderboards like LMArena and LiveBench as of 2026.

muse-spark-1.2 (xHigh) is a large language model developed by Halcyon, a private AI research company. Released as the flagship variant of the muse-spark-1.2 series, it is designed for high-throughput inference and complex reasoning tasks, with a parameter count exceeding 400 billion. The model is a successor to muse-spark-1.0 and was first made available via API on September 15, 2025, with a stable snapshot released on September 19, 2026.

The model is built on a [transformer](https://www.wikiprompt.org/wiki/transformer) architecture with a [multi-head-attention](https://www.wikiprompt.org/wiki/multi-head-attention) mechanism, incorporating [positional-encoding](https://www.wikiprompt.org/wiki/positional-encoding) and [layer-normalization](https://www.wikiprompt.org/wiki/layer-normalization) for stable training. It uses a mixture-of-experts design, activating only a subset of its parameters per token, which reduces computational cost while maintaining high accuracy. The training process employed [curriculum-learning](https://www.wikiprompt.org/wiki/curriculum-learning) and [gradient-clipping](https://www.wikiprompt.org/wiki/gradient-clipping), with an initial learning rate of 1.5e-4 and a [learning-rate-schedule](https://www.wikiprompt.org/wiki/learning-rate-schedule) that includes warmup and cosine decay over 2.1 trillion tokens.

## Training and Data

muse-spark-1.2 (xHigh) was trained on a diverse corpus of publicly available text and code, totaling approximately 12 terabytes of data. The dataset was filtered for quality and deduplicated using [data-augmentation](https://www.wikiprompt.org/wiki/data-augmentation) techniques, with a focus on multilingual content covering over 50 languages. Training was conducted on a cluster of 8,192 [amd](https://www.wikiprompt.org/wiki/amd) MI300X accelerators, provided through [halcyon](https://www.wikiprompt.org/wiki/halcyon)'s partnership with [amazon-web-services](https://www.wikiprompt.org/wiki/amazon-web-services). The training run lasted 74 days, concluding in August 2025, and consumed an estimated 45 GWh of electricity.

The model's [loss-functions](https://www.wikiprompt.org/wiki/loss-functions) included a standard cross-entropy objective with a [temperature-scaling](https://www.wikiprompt.org/wiki/temperature-scaling) factor of 0.7 during fine-tuning. Post-training involved [rlaif](https://www.wikiprompt.org/wiki/rlaif) (reinforcement learning from AI feedback) to align outputs with human preferences, followed by a final phase of [model-pruning](https://www.wikiprompt.org/wiki/model-pruning) to reduce latency by 18% without significant accuracy loss.

## Capabilities and Benchmarks

On public leaderboards, muse-spark-1.2 (xHigh) has achieved notable scores. As of the September 19, 2026 snapshot, it ranks 3rd on [LMArena](https://www.wikiprompt.org/wiki/lmarena) with an Elo rating of 1,342, and 2nd on LiveBench with a composite score of 78.4. In specific tasks, it scores 91.2% on MMLU-Pro, 88.7% on HumanEval for code generation, and 84.5% on the MATH-500 benchmark. The model supports a context window of 256,000 tokens, enabled by [cross-attention](https://www.wikiprompt.org/wiki/cross-attention) mechanisms and [beam-search](https://www.wikiprompt.org/wiki/beam-search) decoding for long-form generation.

Its architecture includes [residual-network](https://www.wikiprompt.org/wiki/residual-network) connections and [batch-normalization](https://www.wikiprompt.org/wiki/batch-normalization) in the feedforward layers, which improve gradient flow during training. The model also employs [top-k-sampling](https://www.wikiprompt.org/wiki/top-k-sampling) and [top-p-sampling](https://www.wikiprompt.org/wiki/top-p-sampling) with a default k of 50 and p of 0.95, allowing for controlled creativity in outputs. For enterprise use, it supports [sgd-variants](https://www.wikiprompt.org/wiki/sgd-variants) like AdamW for fine-tuning, and offers [dropout](https://www.wikiprompt.org/wiki/dropout) regularization at a rate of 0.1 during adaptation.

## Deployment and Ecosystem

muse-spark-1.2 (xHigh) is available through Halcyon's cloud API, as well as on [google-cloud](https://www.wikiprompt.org/wiki/google-cloud) and [oracle-cloud](https://www.wikiprompt.org/wiki/oracle-cloud) marketplaces. It is optimized for [aws-trainium](https://www.wikiprompt.org/wiki/aws-trainium) and [groq](https://www.wikiprompt.org/wiki/groq) hardware, with inference speeds of 1,200 tokens per second on Groq's LPU systems. The model is also integrated into [azure](https://www.wikiprompt.org/wiki/azure) for enterprise customers, and supports onnx runtime for on-premises deployment.

Halcyon has released a smaller distilled version, muse-spark-1.2 (base), for edge devices, but the xHigh variant remains the flagship. The company has not disclosed the full training cost, but industry estimates suggest it exceeds $50 million, based on compute and data acquisition. As of early 2026, the model is used by over 2,000 organizations, including [samsung-electronics](https://www.wikiprompt.org/wiki/samsung-electronics) and [intel](https://www.wikiprompt.org/wiki/intel) for internal research.

## Reception and Impact

The release of muse-spark-1.2 (xHigh) has been noted for its efficiency improvements over prior models, particularly in reducing [inference](https://www.wikiprompt.org/wiki/inference) costs by 30% compared to similar-sized models from [openai](https://www.wikiprompt.org/wiki/openai) and [anthropic](https://www.wikiprompt.org/wiki/anthropic). However, some researchers have criticized the lack of transparency regarding training data sources, echoing concerns raised by [brian-christian](https://www.wikiprompt.org/wiki/brian-christian) and [melanie-mitchell](https://www.wikiprompt.org/wiki/melanie-mitchell) about reproducibility in [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence).

The model has also sparked discussions about [generative-ai](https://www.wikiprompt.org/wiki/generative-ai) safety, with Halcyon implementing [gradient-clipping](https://www.wikiprompt.org/wiki/gradient-clipping) and [rlaif](https://www.wikiprompt.org/wiki/rlaif) to mitigate harmful outputs. Independent audits by [stanford-ai-lab](https://www.wikiprompt.org/wiki/stanford-ai-lab) and [berkeley-ai-research](https://www.wikiprompt.org/wiki/berkeley-ai-research) have found no significant bias issues, though they recommend ongoing monitoring. As of the latest snapshot, muse-spark-1.2 (xHigh) remains a top contender in the competitive [large-language-model](https://www.wikiprompt.org/wiki/large-language-model) landscape, with a planned update to version 1.3 expected in early 2027.

---
Source: https://www.wikiprompt.org/wiki/muse-spark-1-2-xhigh
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-20T00:27:15.484876+00:00