{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":87323,"databundleVersionId":9934427,"sourceType":"competition"}],"dockerImageVersionId":30823,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Vision Transformer, EfficientNet, and RegNet for Fake vs Real Scene Classification\n\nIn this notebook, we explore three different models to tackle the CIDAUT AI Fake Scene Classification 2024 competition:\n\n- [Vision Transformer (ViT)](https://arxiv.org/pdf/2010.11929) (AUC ROC 0.9263)\n- [EfficientNet V2](https://arxiv.org/pdf/2104.00298) (AUC ROC 0.9476)\n- [RegNet](https://arxiv.org/pdf/2101.00590) (AUC ROC 0.9323)\n\nEach model is fine-tuned from pre-trained weights to classify images as either “fake” (editada) or “real.” We also walk through the data preparation process, model training loops, and final inference steps to generate Kaggle submission files. Results are compared based on validation metrics and leaderboard scores.\n\n---\n\n## A Challenge with the Leaderboard\n\nOne key issue with this competition was the absence of a **split between the public and private leaderboard**, which meant that the most effective strategy to reach the top was to **brute force the leaderboard** by learning from multiple submissions on Kaggle. This approach rewarded fine-tuning predictions to match the leaderboard rather than genuinely optimizing the model for the perfect result.\n\nThis is apparent in many Top 10 solutions. For instance, **bestwater** (ranked 1st) openly admits to this approach in their post ([link here](https://www.kaggle.com/competitions/cidaut-ai-fake-scene-classification-2024/discussion/557589#3101028)). Additionally, it's almost certain that **solosid69** (top 2, 4, 5, 6, 7) used the same technique, leveraging **five different accounts** 😶‍🌫️ to maximize submissions and brute force his way up the leaderboard.\n\nIt’s worth noting that bestwater's honesty in admitting this approach is commendable, as it highlights the real issue—the competition design itself. The lack of a private leaderboard effectively incentivized brute-forcing, which overshadowed more rigorous model optimization efforts.\n\nEven with thorough hyperparameter tuning, the results presented here are likely close to the best achievable without resorting to brute-forcing the competition.\n\nHower, I enjoyed participating in this competition. It was a fun and rewarding experience that allowed me to explore new models and gain a deeper understanding of their performance on real-world datas :).\n\n[CIDAUT AI Fake Scene Classification 2024](https://www.kaggle.com/competitions/cidaut-ai-fake-scene-classification-2024)","metadata":{}},{"cell_type":"markdown","source":"## Import","metadata":{}},{"cell_type":"code","source":"from __future__ import print_function\n\nimport glob\nimport os\nimport random\nimport pandas as pd\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom PIL import Image\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import roc_auc_score\nfrom torch.optim.lr_scheduler import StepLR\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision import transforms, models\nfrom tqdm.notebook import tqdm\nimport matplotlib.pyplot as plt","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Vision Transformer\n\nThis was the first model I tried because I am most familiar with it. After some minor hyperparameter adjustments, the model achieved decent results. My best leaderboard score using Vision Transformer was [0.9263](https://www.kaggle.com/code/quentinducoulombier/vit-efficientnet-regnet-cidaut-ai?scriptVersionId=217148512)","metadata":{}},{"cell_type":"markdown","source":"### Configuration and Reproducibility¶","metadata":{}},{"cell_type":"code","source":"batch_size = 8\nepochs = 10\nlr = 5e-5\ngamma = 0.7\nseed = 42\n\n\ndef seed_everything(seed):\n    \"\"\"\n    Sets the random seed for reproducibility in python, numpy, and torch.\n    \"\"\"\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n\nseed_everything(seed)\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(\"Device used :\", device)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Data Preparation and Transformation¶","metadata":{}},{"cell_type":"code","source":"# -- Paths to the data --\nbase_dir = '/kaggle/input/cidaut-ai-fake-scene-classification-2024'\ntrain_csv_path = os.path.join(base_dir, 'train.csv')\n\n# Directory containing training images\ntrain_images_dir = os.path.join(base_dir, 'Train')\n\n# Directory containing test images (without labels)\ntest_images_dir = os.path.join(base_dir, 'Test')\n\n# -- Reading the train CSV file --\ndf = pd.read_csv(train_csv_path)\n# df contains two columns: \"images\" (file name), \"label\" (\"editada\" or \"real\")\n\n# -- Converting labels: editada -> 0, real -> 1 --\ncls_to_idx = {'editada': 0, 'real': 1}\ndf['label'] = df['label'].map(cls_to_idx)\n\n# -- Retrieving image paths and labels --\nall_image_paths = [os.path.join(train_images_dir, img_name) for img_name in df['image']]\nall_labels = df['label'].values\n\n# -- Train/validation split (85% / 15%) --\ntrain_paths, val_paths, train_labels, val_labels = train_test_split(\n    all_image_paths,\n    all_labels,\n    test_size=0.15,\n    stratify=all_labels,\n    random_state=seed\n)\n\nprint(f\"Train Data: {len(train_paths)}\")\nprint(f\"Validation Data: {len(val_paths)}\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# -- Transforms --\ntrain_transforms = transforms.Compose([\n    transforms.Resize((224, 224)),\n    transforms.RandomResizedCrop(224),\n    transforms.RandomHorizontalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])\n\nval_transforms = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])\n\ntest_transforms = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Custom Dataset Implementation","metadata":{}},{"cell_type":"code","source":"class FakeRealDataset(Dataset):\n    \"\"\"\n    Dataset for training (train/val) where labels are available.\n    \"\"\"\n    def __init__(self, file_list, labels, transform=None):\n        self.file_list = file_list\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.file_list)\n\n    def __getitem__(self, idx):\n        img_path = self.file_list[idx]\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        label = self.labels[idx]\n        return img, label\n\n\nclass FakeRealTestDataset(Dataset):\n    \"\"\"\n    Dataset for testing (no labels).\n    Returns the transformed image and the image name for submission.\n    \"\"\"\n    def __init__(self, directory, transform=None):\n        self.transform = transform\n        # Retrieve all file names in the Test directory\n        self.image_names = sorted(os.listdir(directory))\n        self.image_paths = [os.path.join(directory, img) for img in self.image_names]\n\n    def __len__(self):\n        return len(self.image_paths)\n\n    def __getitem__(self, idx):\n        img_path = self.image_paths[idx]\n        img_name = self.image_names[idx]  # for constructing the submission file\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        return img, img_name\n\n# -- Datasets and DataLoaders --\ntrain_data = FakeRealDataset(train_paths, train_labels, transform=train_transforms)\nval_data   = FakeRealDataset(val_paths,   val_labels,   transform=val_transforms)\n\ntrain_loader = DataLoader(dataset=train_data, batch_size=batch_size, shuffle=True,  num_workers=4)\nval_loader   = DataLoader(dataset=val_data,   batch_size=batch_size, shuffle=False, num_workers=4)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Model Initialization","metadata":{}},{"cell_type":"code","source":"# -- Loading the pre-trained ViT model --\nmodel = models.vit_l_16(weights=models.ViT_L_16_Weights.DEFAULT)\nmodel.to(device)\n\n# -- Modifying the last layer to output 2 classes (0 or 1) --\nnum_ftrs = model.heads.head.in_features\nmodel.heads.head = nn.Linear(num_ftrs, 2)\n\n# Use multiple GPUs if available (ONLY if u use GPU T4*2)\n# model = nn.DataParallel(model)\nmodel.to(device)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Training the Model","metadata":{}},{"cell_type":"code","source":"criterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=lr)\nscheduler = StepLR(optimizer, step_size=1, gamma=gamma)\n\ntrain_losses = []\ntrain_accuracies = []\nval_losses = []\nval_accuracies = []","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for epoch in range(epochs):\n    model.train()\n    epoch_loss = 0\n    epoch_accuracy = 0\n\n    for data, label in tqdm(train_loader, desc=f\"Training Epoch {epoch+1}\"):\n        data = data.to(device)\n        label = label.to(device)\n\n        optimizer.zero_grad()\n        output = model(data)\n        loss = criterion(output, label)\n        loss.backward()\n        optimizer.step()\n\n        epoch_loss += loss.item()\n        preds = output.argmax(dim=1)\n        acc = (preds == label).float().mean()\n        epoch_accuracy += acc.item()\n\n    # Average per batch\n    epoch_loss /= len(train_loader)\n    epoch_accuracy /= len(train_loader)\n\n    train_losses.append(epoch_loss)\n    train_accuracies.append(epoch_accuracy)\n\n    # -- Validation --\n    model.eval()\n    val_loss = 0\n    val_acc = 0\n\n    with torch.no_grad():\n        for data, label in tqdm(val_loader, desc=f\"Validation Epoch {epoch+1}\"):\n            data = data.to(device)\n            label = label.to(device)\n\n            output = model(data)\n            loss = criterion(output, label)\n            val_loss += loss.item()\n\n            preds = output.argmax(dim=1)\n            acc = (preds == label).float().mean()\n            val_acc += acc.item()\n\n    val_loss /= len(val_loader)\n    val_acc /= len(val_loader)\n\n    val_losses.append(val_loss)\n    val_accuracies.append(val_acc)\n\n    print(\n        f\"Epoch [{epoch+1}/{epochs}] \"\n        f\"Train Loss: {epoch_loss:.4f} | Train Acc: {epoch_accuracy:.4f} | \"\n        f\"Val Loss: {val_loss:.4f} | Val Acc: {val_acc:.4f}\"\n    )\n\n    scheduler.step()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Visualization of Loss and Accuracy","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Loss\")\nplt.plot(train_losses, label=\"Training\")\nplt.plot(val_losses, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.show()\n\nplt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Accuracy\")\nplt.plot(train_accuracies, label=\"Training\")\nplt.plot(val_accuracies, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Accuracy\")\nplt.legend()\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Inference on Test Data and Submission Generation","metadata":{}},{"cell_type":"code","source":"# Create a test dataset/loader without labels\ntest_dataset = FakeRealTestDataset(test_images_dir, transform=test_transforms)\ntest_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=4)\n\nmodel.eval()\npredictions = []\nimage_names = []\n\nwith torch.no_grad():\n    for data, names in tqdm(test_loader, desc=\"Predicting\"):\n        data = data.to(device)\n        output = model(data)\n        # output has size (batch_size, 2)\n        probs = nn.Softmax(dim=1)(output)  # (batch_size, 2)\n        # Directly retrieve the probability of class 1 (real)\n        real_probs = probs[:, 1].cpu().numpy()\n        predictions.extend(real_probs)\n        image_names.extend(names)\n\n# -- Construct the submission DataFrame --\nsubmission_df = pd.DataFrame({\n    'image': image_names,\n    'label': predictions  # probability of class 1 (real)\n})","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# -- Saving to a CSV file --\nsubmission_df.to_csv(\"submit_ViT.csv\", index=False)\nprint(\"Submission file generated: submit_ViT.csv\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## EfficientNet V2\n\nThis was the second model I tried. After tuning some hyperparameters, it provided **the best results among all the models**, reaching a score of [0.9476](https://www.kaggle.com/code/quentinducoulombier/vit-efficientnet-regnet-cidaut-ai?scriptVersionId=217742924), which placed me in the top 10 on the leaderboard. Below is the code for training and inference.","metadata":{}},{"cell_type":"markdown","source":"### Configuration and Reproducibility¶\n","metadata":{}},{"cell_type":"code","source":"batch_size = 8\nepochs = 25\nlr = 5e-5\ngamma = 0.7\nseed = 42\n\n\ndef seed_everything(seed):\n    \"\"\"\n    Sets the random seed for reproducibility in python, numpy, and torch.\n    \"\"\"\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n\nseed_everything(seed)\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(\"Device used :\", device)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Data Preparation and Transformation¶","metadata":{}},{"cell_type":"code","source":"# Paths to the dataset\nbase_dir = '/kaggle/input/cidaut-ai-fake-scene-classification-2024'\ntrain_csv_path = os.path.join(base_dir, 'train.csv')\ntrain_images_dir = os.path.join(base_dir, 'Train')\ntest_images_dir = os.path.join(base_dir, 'Test')\n\n# Load training labels\ntrain_df = pd.read_csv(train_csv_path)\n# Map labels 'editada' -> 0 and 'real' -> 1\ncls_to_idx = {'editada': 0, 'real': 1}\ntrain_df['label'] = train_df['label'].map(cls_to_idx)\n\n# Extract image paths and labels\nall_image_paths = [os.path.join(train_images_dir, img_name) for img_name in train_df['image']]\nall_labels = train_df['label'].values\n\n# Split train/validation (95% / 5%) --\ntrain_paths, val_paths, train_labels, val_labels = train_test_split(\n    all_image_paths,\n    all_labels,\n    test_size=0.05,        \n    stratify=all_labels,\n    random_state=seed\n)\n\nprint(f\"Train Data: {len(train_paths)}\")\nprint(f\"Validation Data: {len(val_paths)}\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_transforms = transforms.Compose([\n    transforms.Resize((224, 224)),\n    transforms.RandomResizedCrop(224),\n    transforms.RandomHorizontalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])\n\nval_transforms = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])\n\ntest_transforms = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406],\n                         std=[0.229, 0.224, 0.225]),\n])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Custom Dataset Implementation","metadata":{}},{"cell_type":"code","source":"class FakeRealDataset(Dataset):\n    \"\"\"\n    Dataset for training and validation with labels.\n    \"\"\"\n    def __init__(self, file_list, labels, transform=None):\n        self.file_list = file_list\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.file_list)\n\n    def __getitem__(self, idx):\n        img_path = self.file_list[idx]\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        label = self.labels[idx]\n        return img, label\n\nclass FakeRealTestDataset(Dataset):\n    \"\"\"\n    Dataset for testing without labels.\n    \"\"\"\n    def __init__(self, directory, transform=None):\n        self.transform = transform\n        self.image_names = sorted(os.listdir(directory))\n        self.image_paths = [os.path.join(directory, img) for img in self.image_names]\n\n    def __len__(self):\n        return len(self.image_paths)\n\n    def __getitem__(self, idx):\n        img_path = self.image_paths[idx]\n        img_name = self.image_names[idx]\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        return img, img_name\n\n# Create DataLoaders\ntrain_data = FakeRealDataset(train_paths, train_labels, transform=train_transforms)\nval_data = FakeRealDataset(val_paths, val_labels, transform=val_transforms)\n\ntrain_loader = DataLoader(dataset=train_data, batch_size=batch_size, shuffle=True, num_workers=4)\nval_loader = DataLoader(dataset=val_data, batch_size=batch_size, shuffle=False, num_workers=4)\n\nprint(f\"Train Dataset size: {len(train_data)}\")\nprint(f\"Validation Dataset size: {len(val_data)}\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Model Initialization","metadata":{}},{"cell_type":"code","source":"# Load pre-trained EfficientNet\nmodel = models.efficientnet_v2_l(weights=models.EfficientNet_V2_L_Weights.IMAGENET1K_V1)\n\n# Modify the classifier to output 2 classes (real and fake)\nnum_ftrs = model.classifier[1].in_features\nmodel.classifier[1] = nn.Linear(num_ftrs, 2)\n\n# Use multiple GPUs if available (ONLY if u use GPU T4*2)\n# model = nn.DataParallel(model)\nmodel.to(device)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Training the Model","metadata":{}},{"cell_type":"code","source":"# Define loss, optimizer, and scheduler\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=lr)\nscheduler = StepLR(optimizer, step_size=1, gamma=gamma)\n\ntrain_losses = []\ntrain_accuracies = []\nval_losses = []\nval_accuracies = []","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Training loop\nfor epoch in range(epochs):\n    model.train()\n    epoch_loss = 0\n    epoch_accuracy = 0\n\n    for data, label in tqdm(train_loader, desc=f\"Training Epoch {epoch+1}\"):\n        data = data.to(device)\n        label = label.to(device)\n\n        optimizer.zero_grad()\n        output = model(data)\n        loss = criterion(output, label)\n        loss.backward()\n        optimizer.step()\n\n        epoch_loss += loss.item()\n        preds = output.argmax(dim=1)\n        acc = (preds == label).float().mean()\n        epoch_accuracy += acc.item()\n\n    epoch_loss /= len(train_loader)\n    epoch_accuracy /= len(train_loader)\n\n    train_losses.append(epoch_loss)\n    train_accuracies.append(epoch_accuracy)\n\n    # -- Validation --\n    model.eval()\n    val_loss = 0\n    val_acc = 0\n    val_preds_list = []\n    val_labels_list = []\n\n\n    with torch.no_grad():\n        for data, label in tqdm(val_loader, desc=f\"Validation Epoch {epoch+1}\"):\n            data = data.to(device)\n            label = label.to(device)\n\n            output = model(data)\n            loss = criterion(output, label)\n            val_loss += loss.item()\n\n            preds = output.argmax(dim=1)\n            acc = (preds == label).float().mean()\n            val_acc += acc.item()\n            \n            val_preds_list.extend(output[:, 1].cpu().numpy()) \n            val_labels_list.extend(label.cpu().numpy())\n\n    val_loss /= len(val_loader)\n    val_acc /= len(val_loader)\n\n    val_auc = roc_auc_score(val_labels_list, val_preds_list)\n\n\n    val_losses.append(val_loss)\n    val_accuracies.append(val_acc)\n\n    print(\n        f\"Epoch [{epoch+1}/{epochs}] \"\n        f\"Train Loss: {epoch_loss:.4f} | Train Acc: {epoch_accuracy:.4f} | \"\n        f\"Val Loss: {val_loss:.4f} | Val Acc: {val_acc:.4f} | Val AUC: {val_auc:.4f}\"\n    )\n\n    scheduler.step()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Visualization of Loss and Accuracy","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Loss\")\nplt.plot(train_losses, label=\"Training\")\nplt.plot(val_losses, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.show()\n\nplt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Accuracy\")\nplt.plot(train_accuracies, label=\"Training\")\nplt.plot(val_accuracies, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Accuracy\")\nplt.legend()\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Inference on Test Data and Submission Generation","metadata":{}},{"cell_type":"code","source":"test_dataset = FakeRealTestDataset(test_images_dir, transform=test_transforms)\ntest_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=4)\n\nmodel.eval()\npredictions = []\nimage_names = []\n\nwith torch.no_grad():\n    for data, names in tqdm(test_loader, desc=\"Predicting\"):\n        data = data.to(device)\n        output = model(data)\n        probs = nn.Softmax(dim=1)(output)\n        real_probs = probs[:, 1].cpu().numpy()\n        predictions.extend(real_probs)\n        image_names.extend(names)\n\nsubmission_df = pd.DataFrame({\n    'image': image_names,\n    'label': predictions\n})","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_df.to_csv(\"submit_efficientNet.csv\", index=False)\nprint(\"Submission file generated: submit_efficientNet.csv\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Regnet","metadata":{}},{"cell_type":"markdown","source":"Finally, I experimented with RegNet, expecting strong performance based on its theoretical scores. During training, I even saw 100% accuracy on the training set. However, the public leaderboard score turned out to be lower, probably due to overfitting. Since I did not have enough time to fine-tune the number of epochs or experiment with more advanced methods, my best result with RegNet was [0.9323](https://www.kaggle.com/code/quentinducoulombier/regnet-cidaut?scriptVersionId=218231833).  \nDespite everything, RegNet is still a powerful model that could likely achieve higher performance with additional regularization and tuning.","metadata":{}},{"cell_type":"markdown","source":"### Configuration and Reproducibility","metadata":{}},{"cell_type":"code","source":"batch_size = 8\nepochs = 10 # 5 for the best result (0.9323 AUC ROC) \nlr = 5e-5\ngamma = 0.7\nseed = 42\n\n\ndef seed_everything(seed):\n    \"\"\"\n    Sets the random seed for reproducibility in python, numpy, and torch.\n    \"\"\"\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n\nseed_everything(seed)\n\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(\"Device used :\", device)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Data Preparation and Transformation","metadata":{}},{"cell_type":"code","source":"# Paths to the dataset\nbase_dir = '/kaggle/input/cidaut-ai-fake-scene-classification-2024'\ntrain_csv_path = os.path.join(base_dir, 'train.csv')\ntrain_images_dir = os.path.join(base_dir, 'Train')\ntest_images_dir = os.path.join(base_dir, 'Test')\n\n# Load training labels\ntrain_df = pd.read_csv(train_csv_path)\n# Map labels 'editada' -> 0 and 'real' -> 1\ncls_to_idx = {'editada': 0, 'real': 1}\ntrain_df['label'] = train_df['label'].map(cls_to_idx)\n\n# Extract image paths and labels\nall_image_paths = [os.path.join(train_images_dir, img_name) for img_name in train_df['image']]\nall_labels = train_df['label'].values\n\n# Split train/validation (95% / 5%) --\ntrain_paths, val_paths, train_labels, val_labels = train_test_split(\n    all_image_paths,\n    all_labels,\n    test_size=0.05,        \n    stratify=all_labels,\n    random_state=seed\n)\n\nprint(f\"Train Data: {len(train_paths)}\")\nprint(f\"Validation Data: {len(val_paths)}\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import torchvision.transforms as T\nfrom torchvision.transforms import InterpolationMode\n\ntrain_transforms = T.Compose([\n    T.Resize(224, interpolation=InterpolationMode.BICUBIC),\n    T.RandomCrop(224),\n    T.RandomHorizontalFlip(),\n    T.ToTensor(),\n    T.Normalize(mean=[0.485, 0.456, 0.406],\n                std=[0.229, 0.224, 0.225]),\n])\n\nval_transforms = T.Compose([\n    T.Resize(224, interpolation=InterpolationMode.BICUBIC),\n    T.CenterCrop(224),\n    T.ToTensor(),\n    T.Normalize(mean=[0.485, 0.456, 0.406],\n                std=[0.229, 0.224, 0.225]),\n])\n\ntest_transforms = T.Compose([\n    T.Resize(224, interpolation=InterpolationMode.BICUBIC),\n    T.CenterCrop(224),\n    T.ToTensor(),\n    T.Normalize(mean=[0.485, 0.456, 0.406],\n                std=[0.229, 0.224, 0.225]),\n])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Custom Dataset Implementation","metadata":{}},{"cell_type":"code","source":"class FakeRealDataset(Dataset):\n    \"\"\"\n    Dataset for training and validation with labels.\n    \"\"\"\n    def __init__(self, file_list, labels, transform=None):\n        self.file_list = file_list\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.file_list)\n\n    def __getitem__(self, idx):\n        img_path = self.file_list[idx]\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        label = self.labels[idx]\n        return img, label\n\nclass FakeRealTestDataset(Dataset):\n    \"\"\"\n    Dataset for testing without labels.\n    \"\"\"\n    def __init__(self, directory, transform=None):\n        self.transform = transform\n        self.image_names = sorted(os.listdir(directory))\n        self.image_paths = [os.path.join(directory, img) for img in self.image_names]\n\n    def __len__(self):\n        return len(self.image_paths)\n\n    def __getitem__(self, idx):\n        img_path = self.image_paths[idx]\n        img_name = self.image_names[idx]\n        img = Image.open(img_path).convert(\"RGB\")\n        if self.transform:\n            img = self.transform(img)\n        return img, img_name\n\n# Create DataLoaders\ntrain_data = FakeRealDataset(train_paths, train_labels, transform=train_transforms)\nval_data = FakeRealDataset(val_paths, val_labels, transform=val_transforms)\n\ntrain_loader = DataLoader(dataset=train_data, batch_size=batch_size, shuffle=True, num_workers=4)\nval_loader = DataLoader(dataset=val_data, batch_size=batch_size, shuffle=False, num_workers=4)\n\nprint(f\"Train Dataset size: {len(train_data)}\")\nprint(f\"Validation Dataset size: {len(val_data)}\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Model Initialization","metadata":{}},{"cell_type":"code","source":"# Load pre-trained Regnet\nmodel = models.regnet_y_32gf(weights=models.RegNet_Y_32GF_Weights.IMAGENET1K_SWAG_E2E_V1)\n\nnum_ftrs = model.fc.in_features\n\nmodel.fc = nn.Linear(num_ftrs, 2)\n\n\n#model = nn.DataParallel(model)\nmodel.to(device)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Training the Model","metadata":{}},{"cell_type":"code","source":"# Define loss, optimizer, and scheduler\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=lr)\nscheduler = StepLR(optimizer, step_size=1, gamma=gamma)\n\ntrain_losses = []\ntrain_accuracies = []\nval_losses = []\nval_accuracies = []","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Training loop\nfor epoch in range(epochs):\n    model.train()\n    epoch_loss = 0\n    epoch_accuracy = 0\n\n    for data, label in tqdm(train_loader, desc=f\"Training Epoch {epoch+1}\"):\n        data = data.to(device)\n        label = label.to(device)\n\n        optimizer.zero_grad()\n        output = model(data)\n        loss = criterion(output, label)\n        loss.backward()\n        optimizer.step()\n\n        epoch_loss += loss.item()\n        preds = output.argmax(dim=1)\n        acc = (preds == label).float().mean()\n        epoch_accuracy += acc.item()\n\n    epoch_loss /= len(train_loader)\n    epoch_accuracy /= len(train_loader)\n\n    train_losses.append(epoch_loss)\n    train_accuracies.append(epoch_accuracy)\n\n    # -- Validation --\n    model.eval()\n    val_loss = 0\n    val_acc = 0\n    val_preds_list = []\n    val_labels_list = []\n\n\n    with torch.no_grad():\n        for data, label in tqdm(val_loader, desc=f\"Validation Epoch {epoch+1}\"):\n            data = data.to(device)\n            label = label.to(device)\n\n            output = model(data)\n            loss = criterion(output, label)\n            val_loss += loss.item()\n\n            preds = output.argmax(dim=1)\n            acc = (preds == label).float().mean()\n            val_acc += acc.item()\n            \n            val_preds_list.extend(output[:, 1].cpu().numpy()) \n            val_labels_list.extend(label.cpu().numpy())\n\n    val_loss /= len(val_loader)\n    val_acc /= len(val_loader)\n\n    val_auc = roc_auc_score(val_labels_list, val_preds_list)\n\n\n    val_losses.append(val_loss)\n    val_accuracies.append(val_acc)\n\n    print(\n        f\"Epoch [{epoch+1}/{epochs}] \"\n        f\"Train Loss: {epoch_loss:.4f} | Train Acc: {epoch_accuracy:.4f} | \"\n        f\"Val Loss: {val_loss:.4f} | Val Acc: {val_acc:.4f} | Val AUC: {val_auc:.4f}\"\n    )\n\n    scheduler.step()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Visualization of Loss and Accuracy","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Loss\")\nplt.plot(train_losses, label=\"Training\")\nplt.plot(val_losses, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.show()\n\nplt.figure(figsize=(10,5))\nplt.title(\"Training and Validation Accuracy\")\nplt.plot(train_accuracies, label=\"Training\")\nplt.plot(val_accuracies, label=\"Validation\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Accuracy\")\nplt.legend()\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Inference on Test Data and Submission Generation","metadata":{}},{"cell_type":"code","source":"test_dataset = FakeRealTestDataset(test_images_dir, transform=test_transforms)\ntest_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=4)\n\nmodel.eval()\npredictions = []\nimage_names = []\n\nwith torch.no_grad():\n    for data, names in tqdm(test_loader, desc=\"Predicting\"):\n        data = data.to(device)\n        output = model(data)\n        probs = nn.Softmax(dim=1)(output)\n        real_probs = probs[:, 1].cpu().numpy()\n        predictions.extend(real_probs)\n        image_names.extend(names)\n\nsubmission_df = pd.DataFrame({\n    'image': image_names,\n    'label': predictions\n})","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_df.to_csv(\"submit_regnet.csv\", index=False)\nprint(\"Submission file generated: submit_regnet.csv\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Summary of Results:\n\n- Vision Transformer (ViT): Best public score around 0.9263.\n- EfficientNet: Best public score around 0.9476, which positioned me in the competition’s top 10.\n- RegNet: Achieved a best public score of 0.9323, though it showed strong training accuracy (indicating possible overfitting).","metadata":{}}]}