# COL774 : A3 New — Data

来源：https://www.kaggle.com/competitions/col-774-a-3-new/data

本地 Meta Kaggle 快照；不是本次实时页面。

# Dataset Description

This section provides an overview of the `train.pkl` and `test.pkl` files included in this competition, outlining their formats and key definitions. This information will help participants understand what to expect from the datasets, what they need to predict, and any relevant terminology.

### Files

* **train.pkl**: This file contains the training dataset, which includes image data and corresponding labels. Participants will use this data to train their models and learn patterns within the dataset.

* **test.pkl**: This file contains the test dataset, which includes image data along with an `id` for each image, but does not contain labels. Participants will use this dataset to evaluate their models and generate predictions for submission.

### Data Format

Both `train.pkl` and `test.pkl` are structured as lists of tuples, where each tuple contains:

1. **Image Data**: A tensor representation of an image, typically of shape `(C, H, W)`, where:
   - `C` represents the number of channels (e.g., 3 for RGB images).
   - `H` and `W` represent the height and width of the image.

2. **Label** (only in `train.pkl`): An integer representing the class label associated with the image. This label indicates the category to which the image belongs.

3. **ID** (only in `test.pkl`): A unique identifier for each image, allowing participants to match their predictions to the corresponding images.

### Example Structure

The following illustrates the structure of each dataset:

* **`train.pkl`**:
   - Entry: `(image_tensor, label)`
   - Example: `(<tensor of shape (3, 32, 32)>, 5)` where `5` is the label for the corresponding image.

* **`test.pkl`**:
   - Entry: `(id, image_tensor)`
   - Example: `(12345, <tensor of shape (3, 32, 32)>)` where `12345` is the unique identifier for the image.

### Dataset Analysis

#### Train Dataset
- **Number of samples**: 40,000
- **Sample batch shape**: `torch.Size([64, 3, 32, 32])`
- **Sample labels shape**: `torch.Size([64])`
- **Min pixel value**: 0.0
- **Max pixel value**: 1.0

#### Test Dataset
- **Number of samples**: 20,000
- **Sample batch shape**: `torch.Size([64, 3, 32, 32])`
- **Sample labels shape**: `torch.Size([64])`
- **Min pixel value**: 0.0
- **Max pixel value**: 1.0

### Predictions

Participants are expected to predict the class labels for the images in the `test.pkl` dataset. The goal of the competition is to develop a model that accurately assigns labels to these unlabeled images based on the training data. The predictions should be submitted alongside the corresponding IDs to ensure correct association.

### Sample Code

Here is a code snippet to help participants load and analyze the datasets:

```python
import pickle
import torch
from torch.utils.data import Dataset, DataLoader
import matplotlib.pyplot as plt
import numpy as np

class CIFAR100Dataset(Dataset):
    def __init__(self, file_path):
        with open(file_path, 'rb') as f:
            self.data = pickle.load(f)

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        image, label = self.data[idx]
        return image, label

def analyze_dataset(dataset, dataset_name):
    """Function to analyze the dataset"""
    print(f"\n--- Analysis of {dataset_name} Dataset ---")
    print(f"Number of samples: {len(dataset)}")
    
    # Get a sample batch
    sample_batch, sample_labels = next(iter(DataLoader(dataset, batch_size=64)))
    
    print(f"Sample batch shape: {sample_batch.shape}")
    print(f"Sample labels shape: {sample_labels.shape}")
    
    # Check the range of pixel values
    print(f"Min pixel value: {sample_batch.min()}")
    print(f"Max pixel value: {sample_batch.max()}")
    
    # Display a few sample images with their labels
    visualize_samples(sample_batch, sample_labels, dataset_name)

def visualize_samples(images, labels, dataset_name):
    """Function to visualize sample images"""
    num_images = min(8, len(images))  # Display a maximum of 8 images
    plt.figure(figsize=(10, 5))
    for i in range(num_images):
        plt.subplot(2, 4, i + 1)
        plt.imshow(images[i].permute(1, 2, 0))  # Change to HWC format for plotting
        plt.title(f'ID: {labels[i].item()}' if dataset_name == "Test" else f'Label: {labels[i].item()}')
        plt.axis('off')
    plt.suptitle(f'Sample Images from {dataset_name} Dataset')
    plt.tight_layout()
    plt.show()

# Load datasets
train_dataset = CIFAR100Dataset('train.pkl')
test_dataset = CIFAR100Dataset('test.pkl')

# Analyze datasets
analyze_dataset(train_dataset, "Train")
analyze_dataset(test_dataset, "Test")
```