Places365 is a large-scale dataset for scene recognition, containing over 1.8 million images from 365 distinct scene categories. It was introduced in 2016 by researchers at the Massachusetts Institute of Technology (MIT), including Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. The dataset is designed to advance the field of computer vision, particularly in the area of scene understanding, which involves identifying the context or environment in which an image was captured. Places365 has become a standard benchmark for evaluating the performance of deep learning models on scene classification tasks, complementing object-centric datasets like ImageNet.
The dataset is organized into two versions: Places365-Standard and Places365-Challenge. Places365-Standard contains approximately 1.8 million training images, with each category having between 3,000 and 5,000 images. Places365-Challenge is a larger version with over 8 million training images, used for the Places Challenge competition held at the Conference on Computer Vision and Pattern Recognition (CVPR) in 2016. The images are sourced from the web and are annotated with one of 365 scene categories, ranging from natural scenes like "mountain" and "beach" to indoor environments like "kitchen" and "office". The categories are derived from the Places2 dataset, which itself is an extension of the earlier Places205 dataset.
Development and Motivation
Places365 was created to address the need for a comprehensive scene recognition dataset. While object recognition had seen significant progress with datasets like ImageNet, scene understanding remained relatively underdeveloped. The MIT team aimed to provide a large, diverse, and well-annotated dataset that could support training of neural network models for scene classification. The dataset was built by collecting images from the web and then using a combination of automated and human-based annotation processes. The final set of 365 categories was chosen to cover a wide range of everyday environments, ensuring that models trained on Places365 could generalize to real-world applications such as autonomous driving, robotics, and image retrieval.
The creation of Places365 was part of a broader effort to build the Places database, which also includes the Places205 dataset and the Places2 dataset. Places205, introduced in 2014, contained 205 scene categories and was used as a preliminary benchmark. Places365 expanded this to 365 categories, providing a more comprehensive coverage of scene types. The dataset has been widely adopted by the research community, and its release has spurred numerous advances in scene recognition algorithms.
Architecture and Content
Places365 consists of images with varying resolutions, typically around 224×224 pixels after preprocessing. Each image is labeled with a single scene category, making it a single-label classification task. The dataset includes a training set, a validation set, and a test set. The validation and test sets have 50 images per category, while the training set has thousands of images per category. The images are stored in JPEG format and are organized into directories by category.
One notable aspect of Places365 is its focus on scene-level context rather than individual objects. This makes it distinct from object recognition datasets, as models must learn to recognize the overall environment. The dataset also includes a semantic hierarchy, grouping the 365 categories into 16 super-categories such as "indoor", "outdoor natural", and "outdoor man-made". This hierarchy allows for more structured evaluation and analysis of model performance.
Applications and Impact
Places365 has been used extensively in machine learning research, particularly for training convolutional neural networks (CNNs) for scene classification. Many state-of-the-art models, such as ResNet and DenseNet, have been evaluated on Places365, and the dataset has served as a benchmark for comparing different architectures. Beyond classification, Places365 has been used for tasks such as scene parsing, semantic segmentation, and transfer learning. For example, models pretrained on Places365 have been fine-tuned for other vision tasks, including object detection and image captioning.
The dataset has also influenced the development of artificial intelligence systems in industry. Companies like Google DeepMind and OpenAI have incorporated scene understanding capabilities into their models, and Places365 has been used to evaluate the performance of large-scale vision models. In addition, the dataset has been used in studies on human scene perception, providing insights into how humans categorize environments.
Limitations and Future Directions
Despite its success, Places365 has several limitations. The dataset is biased towards Western, English-speaking contexts, as images were collected from the web, which may not represent all global environments equally. Additionally, the single-label format does not capture the complexity of real-world scenes, which often contain multiple overlapping contexts. Researchers have proposed extensions and alternatives, such as multi-label scene datasets and datasets with more diverse geographic coverage. As generative AI and large language models continue to advance, there is growing interest in using scene understanding to enhance multimodal models, and Places365 remains a relevant resource for training and evaluating such systems.