Google Brain was a deep learning artificial intelligence research team that served as the sole AI branch of Google before being incorporated under the newer umbrella of Google AI, a research division at Google dedicated to artificial intelligence. Formed in 2011, it combined open-ended machine learning research with information systems and large-scale computing resources. It created tools such as TensorFlow, which allow neural networks to be used by the public, and multiple internal AI research projects, and aimed to create research opportunities in machine learning and natural language processing. It was merged into former Google sister company DeepMind to form Google DeepMind in April 2023.
The project originated at Google X as a collaboration between Google researchers Jeff Dean and Greg Corrado and Stanford professor Andrew Ng. It became a key driver of deep learning advances within the company, producing influential tools and models that shaped the broader field of Artificial intelligence.
History
The Google Brain project began in 2011 at Google X as a research collaboration involving Google researchers Jeff Dean and Greg Corrado and Stanford professor Andrew Ng. In 2012, the team trained a large-scale Neural network on 10 million images from YouTube videos; the system learned to recognize high-level concepts including cats without being explicitly trained to identify them. This demonstration highlighted the potential of Deep learning to learn features from raw data.
Google expanded its deep-learning research in 2013 by hiring Geoffrey Hinton and acquiring DNNResearch, which he had established with Alex Krizhevsky and Ilya Sutskever. Later that year, Tomáš Mikolov and his colleagues at Google Brain developed word2vec, a method for learning word embeddings from large collections of text. This technique became foundational for natural language processing.
In 2015, Google open-sourced TensorFlow, a framework developed by the Google Brain team for developing and training machine-learning models. TensorFlow became one of the most widely used Machine learning libraries, enabling both research and production applications.
In 2017, Google researchers introduced the Transformer architecture in the paper "Attention Is All You Need." Initially developed for machine translation, it later became the dominant architecture for Large language models, powering systems like GPT and BERT. The Transformer's self-attention mechanism allowed for more efficient parallel processing and better handling of long-range dependencies in text.
In April 2023, Google Brain merged with DeepMind to form Google DeepMind, as part of the company's continued efforts to accelerate work on AI. The combined entity aimed to consolidate research talent and resources under one umbrella, focusing on advancing AI safely and responsibly.
Team and location
Google Brain was initially established by Google Fellow Jeff Dean and visiting Stanford professor Andrew Ng. In 2014, the team included Jeff Dean, Quoc V. Le, Ilya Sutskever, Alex Krizhevsky, Samy Bengio, and Vincent Vanhoucke. In 2017, team members included Anelia Angelova, Samy Bengio, Greg Corrado, George Dahl, Michael Isard, Anjuli Kannan, Hugo Larochelle, Chris Olah, Benoit Steiner, Vincent Vanhoucke, Vijay Vasudevan, and Fernanda Viegas. Chris Lattner, who created Apple's programming language Swift and then ran Tesla's autonomy team for six months, joined Google Brain's team in August 2017. Lattner left the team in January 2020 and joined SiFive.
As of 2021, Google Brain was led by Jeff Dean, Geoffrey Hinton, and Zoubin Ghahramani. Other members included Katherine Heller, Pi-Chuan Chang, Ian Simon, Jean-Philippe Vert, Nevena Lazic, Anelia Angelova, Lukasz Kaiser, Carrie Jun Cai, Eric Breck, Ruoming Pang, Carlos Riquelme, Hugo Larochelle, and David Ha. Samy Bengio left the team in April 2021, and Zoubin Ghahramani took on his responsibilities.
Google Research includes Google Brain and is based in Mountain View. It also has satellite groups in Accra, Amsterdam, Atlanta, Beijing, Berlin, Cambridge, Israel, Los Angeles, London, Montreal, Munich, New York City, Paris, Pittsburgh, Princeton, San Francisco, Seattle, Tokyo, Toronto, and Zurich.
Projects
Artificial-intelligence-devised encryption system
In October 2016, Google Brain designed an experiment to determine that neural networks are capable of learning secure symmetric encryption. In this experiment, three neural networks were created: Alice, Bob and Eve. Adhering to the idea of a generative adversarial network (GAN), the goal of the experiment was for Alice to send an encrypted message to Bob that Bob could decrypt, but the adversary, Eve, could not. Alice and Bob maintained an advantage over Eve, in that they shared a key used for encryption and decryption. In doing so, Google Brain demonstrated the capability of neural networks to learn secure encryption.
Image enhancement
In February 2017, Google Brain determined a probabilistic method for converting pictures with 8x8 resolution to a resolution of 32x32. The method built upon an already existing probabilistic model called pixelCNN to generate pixel translations. The proposed software utilizes two neural networks to make approximations for the pixel makeup of translated images. The first network, known as the "conditioning network," downsizes high-resolution images to 8x8 and attempts to create mappings from the original 8x8 image to these higher-resolution ones. The other network, known as the "prior network," uses the mappings from the previous network to add more detail to the original image. The resulting translated image is not the same image in higher resolution, but rather a 32x32 resolution estimation based on other existing high-resolution images. Google Brain's results indicate the possibility for neural networks to enhance images.
Google Translate
The Google Brain contributed to the Google Translate project by employing a new deep learning system that combines artificial neural networks with vast databases of multilingual texts. In September 2016, Google Neural Machine Translation (GNMT) was launched, an end-to-end learning framework, able to learn from a large number of examples. Previously, Google Translate's Phrase-Based Machine Translation (PBMT) approach would statistically analyze word by word and try to match corresponding words in other languages without considering the surrounding phrases in the sentence. But rather than choosing a replacement for each individual word in the desired language, GNMT evaluates word segments in the context of the rest of the sentence to choose more accurate replacements. Compared to older PBMT models, the GNMT model scored a 24% improvement in similarity to human translation, with a 60% reduction in errors. The GNMT has also shown significant improvement for notoriously difficult translations, like Chinese to English.
While the introduction of the GNMT has increased the quality of Google Translate's translations for the pilot languages, it was very difficult to create such improvements for all of its 103 languages. Addressing this problem, the Google Brain Team was able to develop a Multilingual GNMT system, which extended the previous one by enabling translations between multiple languages. Furthermore, it allows for Zero-Shot Translations, which are translations between two languages that the system has never explicitly seen before. Google announced that Google Translate can now also translate without transcribing, using neural networks. This means that it is possible to translate speech in one language directly into text in another language, without first transcribing it to text. According to the researchers at Google Brain, this intermediate step can be avoided using neural networks. In order for the system to learn this, they exposed it to audio and text pairs, allowing it to map sounds directly to written translations.
The work on GNMT and the Transformer architecture positioned Google Brain as a leader in Sequence-to-Sequence (Seq2Seq) learning and Machine learning research, influencing subsequent developments in Generative AI and Transformer (architecture)-based models.