# Yinfei Yang

Yinfei Yang is a computer scientist at Google Research, known for co-authoring BERT, a transformer-based model for natural language representation learning, and for contributions to natural language generation and sentence embeddings.

Yinfei Yang is a computer scientist and research scientist at Google, recognized for co-authoring BERT, a [transformer](https://www.wikiprompt.org/wiki/transformer)-based model that advanced [representation learning](https://www.wikiprompt.org/wiki/deep-learning) in natural language processing. His research interests include [natural language generation](https://www.wikiprompt.org/wiki/generative-ai), [sequence-to-sequence](https://www.wikiprompt.org/wiki/sequence-to-sequence) modeling, and [machine-learning](https://www.wikiprompt.org/wiki/machine-learning). He has contributed to both the theoretical foundations and practical applications of [artificial-intelligence](https://www.wikiprompt.org/wiki/artificial-intelligence) in language understanding.

## Career and research

Yang works at Google Research, where he has focused on natural language understanding and generation. He co-authored the 2018 paper introducing BERT, which became a foundational model for [large language models](https://www.wikiprompt.org/wiki/large-language-model). His other notable work includes the Universal Sentence Encoder, a model for learning sentence embeddings that has been widely adopted in production systems. He has also contributed to research on text generation, including work on [encoder-decoder](https://www.wikiprompt.org/wiki/encoder-decoder) architectures and [multi-head-attention](https://www.wikiprompt.org/wiki/multi-head-attention) mechanisms. His research often involves [neural networks](https://www.wikiprompt.org/wiki/neural-network) trained at scale.

## BERT and bidirectional representation learning

BERT, short for Bidirectional Encoder Representations from Transformers, was introduced in the paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" published in 2018. The model uses a transformer encoder with [positional encodings](https://www.wikiprompt.org/wiki/positional-encoding), [layer normalization](https://www.wikiprompt.org/wiki/layer-normalization), [residual connections](https://www.wikiprompt.org/wiki/residual-network), and [dropout](https://www.wikiprompt.org/wiki/dropout) during training. It was pre-trained on a large corpus using masked language modeling and next-sentence prediction objectives. BERT-base contains 110 million parameters, while BERT-large contains 340 million parameters. The model achieved state-of-the-art results on several natural language processing benchmarks, including the GLUE score and the SQuAD question answering task. Its bidirectional training approach allowed it to capture context from both directions, improving performance on tasks that require understanding of word relationships. Training such models required careful [learning rate scheduling](https://www.wikiprompt.org/wiki/learning-rate-schedule) and the use of the [Adam optimizer](https://www.wikiprompt.org/wiki/adam-optimizer).

## Universal Sentence Encoder

Yang co-authored the Universal Sentence Encoder, a model that maps sentences to fixed-length vectors. The model is based on a transformer architecture and was trained on a variety of data sources, including web news, question-answer pages, and discussion forums. It supports transfer learning for tasks such as semantic similarity, text classification, and clustering. The Universal Sentence Encoder has been released as a tool for developers and researchers, enabling efficient sentence-level representations without the need for task-specific training. This work has been influential in the area of sentence embedding and semantic search.

## Natural language generation and impact

Yang's contributions to natural language generation include work on sequence-to-sequence models and techniques for generating coherent and contextually relevant text. In this area, decoding strategies such as [beam search](https://www.wikiprompt.org/wiki/beam-search) are commonly used to improve output quality. His work has been applied to tasks such as summarization, dialogue systems, and question answering. The representation learning methods he helped develop, particularly BERT, have had a lasting impact on the field. They enabled more effective fine-tuning for downstream tasks and served as building blocks for subsequent large language models and generative AI systems. His work on sentence embeddings also influenced approaches to semantic text similarity and retrieval.

---
Source: https://www.wikiprompt.org/wiki/yinfei-yang
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-12T22:24:32.383181+00:00