WizardLM is a family of large language models developed by Microsoft Research Asia. The project introduced the Evolve-Instruct method, a technique for automatically generating complex instruction data used in machine learning fine-tuning. The first WizardLM model was released in 2023 and was based on the LLaMA architecture, a Transformer (architecture) model that relies on multi-head attention and layer normalization.
The name "WizardLM" reflects the goal of enabling models to handle intricate user instructions, similar to a wizard's expertise. The models are part of the broader field of generative AI and build on advances in artificial intelligence research. WizardLM was designed to address the challenge of instruction following, where models must interpret and execute user requests with high accuracy.
Background and motivation
Instruction tuning is a standard practice for adapting neural networks to follow user prompts. In this process, a base model is fine-tuned on pairs of instructions and desired responses. However, manually crafting diverse and complex instructions is labor-intensive and often results in datasets that lack depth. WizardLM addresses this by using an AI system to evolve simple seed instructions into more complex forms. This approach is related to curriculum learning, where models are trained on progressively harder examples. The goal is to expose the model to a wider range of reasoning patterns and constraints, thereby improving its ability to handle real-world user requests.
The Evolve-Instruct method was introduced in a paper titled "WizardLM: Empowering Large Language Models to Follow Complex Instructions," posted on arXiv in April 2023. The authors were affiliated with Microsoft Research Asia. The paper proposed a two-stage process: deep evolution and in-depth evolution. In deep evolution, the model generates new instructions by adding constraints, deepening reasoning, or complicating input. In in-depth evolution, the model rewrites existing instructions to increase their complexity. The resulting dataset is then used to fine-tune a base model.
Evolve-Instruct method
The Evolve-Instruct method uses a teacher model, such as ChatGPT, to rewrite seed instructions through operations like adding constraints, deepening reasoning, and complicating inputs. The evolved instructions serve as a form of data augmentation for fine-tuning. The method includes multiple evolution types, including add constraints, deepen, concretize, and increase reasoning steps. For example, a simple instruction like "Write a poem" might be evolved into "Write a poem in the style of Shakespeare about a robot learning to love." This increases the complexity and specificity of the task.
The process is iterative, with the model repeatedly exposed to more challenging instructions. The authors also introduced a quality control mechanism to filter out instructions that are too ambiguous or impossible to answer. This ensures that the training data remains useful and does not degrade the model's performance. The evolved instructions are then combined with the original seed instructions to create a final training set.
Model versions and benchmarks
The first WizardLM model, WizardLM-7B, was fine-tuned from LLaMA-7B. Later versions included WizardLM-13B and WizardLM-30B. The models were evaluated on the Evol-Instruct test set and other benchmarks. The authors reported that WizardLM-7B outperformed ChatGPT on complex instruction following in their evaluations. Specifically, they used GPT-4 as a judge to compare responses and found that WizardLM-7B achieved a higher win rate on the Evol-Instruct test set. Subsequent versions, such as WizardLM v1.1 and v1.2, incorporated further refinements, including larger training datasets and improved evolution strategies.
The model weights were released publicly, allowing researchers and developers to use and fine-tune them. WizardLM models were also integrated into various open-source projects and served as baselines for later instruction-following models. As of 2024, the WizardLM family includes models of different sizes, and the Evolve-Instruct method has been applied to other domains.
Impact and related work
WizardLM's Evolve-Instruct method influenced later models, including WizardCoder for code generation and WizardMath for mathematical reasoning. These models applied similar evolution techniques to specialized domains, demonstrating the versatility of the approach. The approach has been adopted by other researchers in the field, and the paper has been widely cited. WizardLM is part of a broader trend in deep learning to improve instruction following through automated data generation. The method also contributed to the development of more sophisticated evaluation benchmarks for instruction-following models.