{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.13"},"colab":{"provenance":[]},"kaggle":{"accelerator":"gpu","dataSources":[],"dockerImageVersionId":31154,"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"papermill":{"default_parameters":{},"duration":40046.684692,"end_time":"2025-10-11T22:45:57.547744","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2025-10-11T11:38:30.863052","version":"2.6.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"8c02b22f","cell_type":"markdown","source":"# Tâche 2 : Estimation de l’âge à partir d’une photo\n* * *\nCe notebook présente mes travaux pour la tâche 2 , axée sur l'estimation de l’âge à partir d’images de visages. Le pipeline couvre le chargement et la préparation des données spécifiques à cette tâche, la construction et l'entraînement d'un modèle de deep learning basé sur **EfficientNet-B7**, l'évaluation initiale des performances, et le test du modèle obtenu sur les données de test.\n\nDans ce cas, le modèle **EfficientNet-B7**, pré-entraîné sur ImageNet, est utilisé comme extracteur de caractéristiques. La couche de classification finale est remplacée par un nouveau module pour prédire à la fois l'âge (en tant que tâche de **régression**) et le genre (en tant que tâche de classification). Une fonction de perte combinée est utilisée pour optimiser simultanément les deux tâches.\n\n* * *","metadata":{"id":"8c02b22f","papermill":{"duration":0.009505,"end_time":"2025-10-11T11:38:34.703881","exception":false,"start_time":"2025-10-11T11:38:34.694376","status":"completed"},"tags":[]}},{"id":"cbb8f439","cell_type":"markdown","source":"## Traitement des données\n\nCette section gère le chargement, la préparation et la visualisation des données pour l'entraînement et la validation.\n\n\n1. **Téléchargement des fichiers nécessaires:** Les fichiers `kaggle.json` (pour l'authentification Kaggle) et le dataset sont téléchargés à partir des URLs spécifiées.","metadata":{"id":"cbb8f439","papermill":{"duration":0.007822,"end_time":"2025-10-11T11:38:34.720004","exception":false,"start_time":"2025-10-11T11:38:34.712182","status":"completed"},"tags":[]}},{"id":"0cebbfba","cell_type":"code","source":"!wget -q \"https://drive.google.com/uc?export=download&id=1mEjJxNUN0LmeeXnrGESDwe_aKx6Jb-zN\" -O kaggle.json","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:38:34.737135Z","iopub.status.busy":"2025-10-11T11:38:34.736895Z","iopub.status.idle":"2025-10-11T11:38:36.233041Z","shell.execute_reply":"2025-10-11T11:38:36.232046Z"},"id":"0cebbfba","papermill":{"duration":1.506362,"end_time":"2025-10-11T11:38:36.234596","exception":false,"start_time":"2025-10-11T11:38:34.728234","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488809738,"user_tz":-60,"elapsed":1226,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"fe1813df","cell_type":"markdown","source":"2. **Configuration de Kaggle:** Le fichier `kaggle.json` est copié dans le répertoire de configuration de Kaggle (`/root/.config/kaggle/`) pour permettre l'accès aux datasets Kaggle.","metadata":{"papermill":{"duration":0.008015,"end_time":"2025-10-11T11:38:36.251253","exception":false,"start_time":"2025-10-11T11:38:36.243238","status":"completed"},"tags":[],"id":"fe1813df"}},{"id":"bea9c19c","cell_type":"code","source":"import shutil\nimport os\n\ndestination_dir = '/root/.config/kaggle/'\ndestination_file = os.path.join(destination_dir, 'kaggle.json')\n\nos.makedirs(destination_dir, exist_ok=True)\n\nshutil.copy('kaggle.json', destination_file)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:38:36.269652Z","iopub.status.busy":"2025-10-11T11:38:36.268962Z","iopub.status.idle":"2025-10-11T11:38:36.276037Z","shell.execute_reply":"2025-10-11T11:38:36.27548Z"},"papermill":{"duration":0.017948,"end_time":"2025-10-11T11:38:36.277146","exception":false,"start_time":"2025-10-11T11:38:36.259198","status":"completed"},"tags":[],"id":"bea9c19c","executionInfo":{"status":"ok","timestamp":1767488809822,"user_tz":-60,"elapsed":35,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}},"outputId":"bd3e9dc1-f99c-4697-8ea8-4d067f7629e8"},"outputs":[],"execution_count":null},{"id":"99820aa2","cell_type":"code","source":"!kaggle competitions download -c anip-reconnaissance-faciale-estimation-ages-ocr","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2025-10-11T11:38:36.294519Z","iopub.status.busy":"2025-10-11T11:38:36.294023Z","iopub.status.idle":"2025-10-11T11:39:54.045538Z","shell.execute_reply":"2025-10-11T11:39:54.04468Z"},"executionInfo":{"elapsed":60352,"status":"ok","timestamp":1767488870167,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"99820aa2","outputId":"33fe975f-60fe-4c21-da2a-f6b66508ae8d","papermill":{"duration":77.761526,"end_time":"2025-10-11T11:39:54.047022","exception":false,"start_time":"2025-10-11T11:38:36.285496","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"0f1d55ce","cell_type":"markdown","source":"3. **Extraction du dataset:** Le fichier zip contenant le dataset est extrait dans un répertoire `data`. Le répertoire existant est supprimé avant l'extraction pour garantir un état propre.","metadata":{"id":"0f1d55ce","papermill":{"duration":0.009051,"end_time":"2025-10-11T11:39:54.065791","exception":false,"start_time":"2025-10-11T11:39:54.05674","status":"completed"},"tags":[]}},{"id":"40b83440","cell_type":"code","source":"!rm -rf data\n!mkdir -p data\n!unzip -oq anip-reconnaissance-faciale-estimation-ages-ocr.zip -d data","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:39:54.086067Z","iopub.status.busy":"2025-10-11T11:39:54.085612Z","iopub.status.idle":"2025-10-11T11:40:18.415559Z","shell.execute_reply":"2025-10-11T11:40:18.414655Z"},"id":"40b83440","papermill":{"duration":24.342011,"end_time":"2025-10-11T11:40:18.417049","exception":false,"start_time":"2025-10-11T11:39:54.075038","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488906153,"user_tz":-60,"elapsed":35983,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"9050a7ef","cell_type":"markdown","source":"4. **Vérification du nombre d'images:** Le nombre d'images dans le répertoire d'entraînement est compté et affiché pour vérifier que les données ont été correctement extraites.","metadata":{"id":"9050a7ef","papermill":{"duration":0.008846,"end_time":"2025-10-11T11:40:18.435395","exception":false,"start_time":"2025-10-11T11:40:18.426549","status":"completed"},"tags":[]}},{"id":"d121073a","cell_type":"code","source":"n_images=len(os.listdir('data/dataset_tache_2/dataset_tache_2/train'))\nprint(f\"{n_images} images trouvées \")","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:18.454374Z","iopub.status.busy":"2025-10-11T11:40:18.454118Z","iopub.status.idle":"2025-10-11T11:40:18.480027Z","shell.execute_reply":"2025-10-11T11:40:18.479347Z"},"executionInfo":{"elapsed":64,"status":"ok","timestamp":1767488906173,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"d121073a","outputId":"bf74c0c9-dc7c-4897-9270-27e0cd3183b4","papermill":{"duration":0.03669,"end_time":"2025-10-11T11:40:18.480991","exception":false,"start_time":"2025-10-11T11:40:18.444301","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"3de68f11","cell_type":"markdown","source":"5. **Configuration de l'environnement et import des bibliothèques:** Les variables d'environnement pour CUDA sont configurées pour améliorer la gestion de la mémoire GPU. Les bibliothèques Python nécessaires pour le traitement des données, la construction du modèle, l'entraînement et l'évaluation sont importées.","metadata":{"id":"3de68f11","papermill":{"duration":0.008821,"end_time":"2025-10-11T11:40:18.499166","exception":false,"start_time":"2025-10-11T11:40:18.490345","status":"completed"},"tags":[]}},{"id":"27e130eb","cell_type":"code","source":"os.environ[\"CUDA_LAUNCH_BLOCKING\"] = \"1\"\nos.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'\n\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader, WeightedRandomSampler\nfrom torch.amp import GradScaler, autocast\nfrom torchvision import models, transforms\nfrom PIL import Image\nimport pandas as pd\nimport numpy as np\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn.metrics import mean_absolute_error\nimport matplotlib.pyplot as plt\nfrom collections import Counter\nfrom sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score\nimport seaborn as sns\nfrom collections import Counter","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:18.518421Z","iopub.status.busy":"2025-10-11T11:40:18.517765Z","iopub.status.idle":"2025-10-11T11:40:32.652813Z","shell.execute_reply":"2025-10-11T11:40:32.652156Z"},"id":"27e130eb","papermill":{"duration":14.146154,"end_time":"2025-10-11T11:40:32.654127","exception":false,"start_time":"2025-10-11T11:40:18.507973","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917132,"user_tz":-60,"elapsed":10957,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"7a944f06","cell_type":"markdown","source":"6. **Définition de la classe `AgeDataset`:** Cette classe personnalisée hérite de `torch.utils.data.Dataset`. Elle est conçue pour charger les images, extraire les étiquettes (âge et genre) à partir des noms de fichiers et appliquer les transformations d'images spécifiées.","metadata":{"id":"7a944f06","papermill":{"duration":0.009633,"end_time":"2025-10-11T11:40:32.673327","exception":false,"start_time":"2025-10-11T11:40:32.663694","status":"completed"},"tags":[]}},{"id":"3c7a8a13","cell_type":"code","source":"class AgeDataset(Dataset):\n    def __init__(self, image_paths, ages=None, genders=None, transform=None):\n        self.image_paths = image_paths\n        self.ages = ages\n        self.genders = genders\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.image_paths)\n\n    def __getitem__(self, idx):\n        img_path = self.image_paths[idx]\n        image = Image.open(img_path).convert('RGB')\n        if self.transform:\n            image = self.transform(image)\n        else:\n            image = transforms.ToTensor()(image)\n        if self.ages is not None:\n            age = self.ages[idx]\n            gender = self.genders[idx] if self.genders is not None else None\n            return image, age, gender\n        return image, os.path.basename(img_path).split('.')[0]","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:32.692899Z","iopub.status.busy":"2025-10-11T11:40:32.692581Z","iopub.status.idle":"2025-10-11T11:40:32.698341Z","shell.execute_reply":"2025-10-11T11:40:32.697631Z"},"id":"3c7a8a13","papermill":{"duration":0.016448,"end_time":"2025-10-11T11:40:32.699455","exception":false,"start_time":"2025-10-11T11:40:32.683007","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917142,"user_tz":-60,"elapsed":5,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"cc4f7edb","cell_type":"markdown","source":"7. **Fonction `parse_train_data`:** Cette fonction parcourt le répertoire d'entraînement, extrait l'âge et le genre du nom de fichier de chaque image en analysant la structure du nom de fichier (supposant le format `quelquechose_ageZ.JPG` ou `quelquechose_ageZ.PNG` où Z est 'M' ou 'F'). Elle collecte les chemins d'accès aux images, les âges et les genres correspondants.","metadata":{"id":"cc4f7edb","papermill":{"duration":0.008723,"end_time":"2025-10-11T11:40:32.717222","exception":false,"start_time":"2025-10-11T11:40:32.708499","status":"completed"},"tags":[]}},{"id":"0f045073","cell_type":"code","source":"def parse_train_data(train_dir):\n    image_paths = []\n    ages = []\n    genders = []\n    for filename in os.listdir(train_dir):\n        if filename.endswith(('.JPG', '.PNG')):\n            parts = filename.split('_')\n            if len(parts) == 2:\n                suffix = parts[1].split('.')[0]\n                if len(suffix) >= 3:\n                    gender = suffix[-3]  # Z: M or F\n                    age_str = suffix[-2:]\n                    try:\n                        age = float(age_str)\n                        image_paths.append(os.path.join(train_dir, filename))\n                        ages.append(age)\n                        genders.append(1 if gender == 'M' else 0)  # Binary: 1=M, 0=F\n                    except ValueError:\n                        print(f\"Invalid age in {filename}\")\n            else :\n                print(filename)\n\n    return image_paths, ages, genders","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:32.735943Z","iopub.status.busy":"2025-10-11T11:40:32.735753Z","iopub.status.idle":"2025-10-11T11:40:32.741097Z","shell.execute_reply":"2025-10-11T11:40:32.740423Z"},"id":"0f045073","papermill":{"duration":0.01598,"end_time":"2025-10-11T11:40:32.742213","exception":false,"start_time":"2025-10-11T11:40:32.726233","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917191,"user_tz":-60,"elapsed":45,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"a2188f82","cell_type":"markdown","source":"8. **Définition des transformations d'images:** Des pipelines de transformation d'images sont définis à l'aide de `torchvision.transforms`. `train_transform` inclut des augmentations de données (redimensionnement, recadrage aléatoire, retournement horizontal aléatoire) pour améliorer la robustesse du modèle pendant l'entraînement. `val_transform` inclut des transformations nécessaires (redimensionnement, recadrage central) pour la validation. Les deux pipelines incluent une normalisation basée sur les statistiques des images courantes.","metadata":{"id":"a2188f82","papermill":{"duration":0.008828,"end_time":"2025-10-11T11:40:32.759917","exception":false,"start_time":"2025-10-11T11:40:32.751089","status":"completed"},"tags":[]}},{"id":"6b579a20","cell_type":"code","source":"train_transform = transforms.Compose([\n    transforms.Resize(256),\n    transforms.RandomResizedCrop(224, scale=(0.8, 1.0)),\n    transforms.RandomHorizontalFlip(p=0.5),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n])","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:32.778582Z","iopub.status.busy":"2025-10-11T11:40:32.778346Z","iopub.status.idle":"2025-10-11T11:40:32.78257Z","shell.execute_reply":"2025-10-11T11:40:32.781853Z"},"id":"6b579a20","papermill":{"duration":0.014833,"end_time":"2025-10-11T11:40:32.783676","exception":false,"start_time":"2025-10-11T11:40:32.768843","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917233,"user_tz":-60,"elapsed":38,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"79688685","cell_type":"code","source":"val_transform = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n])","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:32.802511Z","iopub.status.busy":"2025-10-11T11:40:32.802069Z","iopub.status.idle":"2025-10-11T11:40:32.805482Z","shell.execute_reply":"2025-10-11T11:40:32.804971Z"},"id":"79688685","papermill":{"duration":0.013775,"end_time":"2025-10-11T11:40:32.806474","exception":false,"start_time":"2025-10-11T11:40:32.792699","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917242,"user_tz":-60,"elapsed":4,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"677117d7","cell_type":"markdown","source":"9. **Chargement et division des données d'entraînement:** La fonction `parse_train_data` est utilisée pour charger les chemins d'accès aux images, les âges et les genres. Les données sont ensuite divisées en ensembles d'entraînement et de validation en utilisant `train_test_split` de scikit-learn. La stratification par genre (`stratify=train_genders`) est utilisée pour garantir une répartition similaire des genres dans les ensembles d'entraînement et de validation.","metadata":{"id":"677117d7","papermill":{"duration":0.008828,"end_time":"2025-10-11T11:40:32.82431","exception":false,"start_time":"2025-10-11T11:40:32.815482","status":"completed"},"tags":[]}},{"id":"69aaf507","cell_type":"code","source":"train_dir = 'data/dataset_tache_2/dataset_tache_2/train'\ntrain_image_paths, train_ages, train_genders = parse_train_data(train_dir)\ntrain_paths, val_paths, train_ages, val_ages, train_genders, val_genders = train_test_split(\n    train_image_paths, train_ages, train_genders, test_size=0.2, random_state=42, stratify=train_genders\n)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:32.843029Z","iopub.status.busy":"2025-10-11T11:40:32.84261Z","iopub.status.idle":"2025-10-11T11:40:32.966006Z","shell.execute_reply":"2025-10-11T11:40:32.965398Z"},"id":"69aaf507","papermill":{"duration":0.134023,"end_time":"2025-10-11T11:40:32.967165","exception":false,"start_time":"2025-10-11T11:40:32.833142","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488917303,"user_tz":-60,"elapsed":57,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"1098fb7d","cell_type":"markdown","source":"10. **Affichage des statistiques des données d'entraînement:** Le nombre total d'images dans l'ensemble d'entraînement, l'âge minimum, l'âge maximum et l'âge moyen sont affichés pour fournir un aperçu des données.","metadata":{"id":"1098fb7d","papermill":{"duration":0.009034,"end_time":"2025-10-11T11:40:32.986999","exception":false,"start_time":"2025-10-11T11:40:32.977965","status":"completed"},"tags":[]}},{"id":"d9c32ad4","cell_type":"code","source":"print(f\"Nombre d'images: {len(train_ages)}\")\nprint(f\"Âge min: {min(train_ages):.1f}, Âge max: {max(train_ages):.1f}, Moyenne: {np.mean(train_ages):.1f}\")","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.005803Z","iopub.status.busy":"2025-10-11T11:40:33.005599Z","iopub.status.idle":"2025-10-11T11:40:33.012984Z","shell.execute_reply":"2025-10-11T11:40:33.012117Z"},"executionInfo":{"elapsed":51,"status":"ok","timestamp":1767488917422,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"d9c32ad4","outputId":"26eba894-0b42-4659-ea80-9edcbedbe250","papermill":{"duration":0.018271,"end_time":"2025-10-11T11:40:33.014169","exception":false,"start_time":"2025-10-11T11:40:32.995898","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"2b3df8ac","cell_type":"markdown","source":"11. **Visualisation de la distribution des âges:** Un histogramme de la distribution des âges dans les données d'entraînement est généré à l'aide de Seaborn et Matplotlib pour visualiser la répartition des âges. Cela aide à identifier d'éventuels déséquilibres de classes.","metadata":{"id":"2b3df8ac","papermill":{"duration":0.008917,"end_time":"2025-10-11T11:40:33.032236","exception":false,"start_time":"2025-10-11T11:40:33.023319","status":"completed"},"tags":[]}},{"id":"52c74e80","cell_type":"code","source":"sns.set(style=\"whitegrid\", palette=\"muted\")\nplt.figure(figsize=(10, 6))\n\nsns.histplot(train_ages, bins=50, kde=True, color='skyblue', stat='density', linewidth=0)\n\nplt.title(\"Distribution des âges dans les données d'entraînement\", fontsize=14, pad=10)\nplt.xlabel(\"Âge (années)\", fontsize=12)\nplt.ylabel(\"Densité\", fontsize=12)\nplt.xlim(0, 100)\nplt.xticks(np.arange(0, 101, 10))  # Étiquettes d'âge tous les 10 ans\nplt.grid(True, alpha=0.3)\n\nplt.show()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.05173Z","iopub.status.busy":"2025-10-11T11:40:33.051117Z","iopub.status.idle":"2025-10-11T11:40:33.614854Z","shell.execute_reply":"2025-10-11T11:40:33.614089Z"},"executionInfo":{"elapsed":885,"status":"ok","timestamp":1767488918302,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"52c74e80","outputId":"89ec95e1-c51e-4b70-cf51-680660cdec5b","papermill":{"duration":0.57466,"end_time":"2025-10-11T11:40:33.61604","exception":false,"start_time":"2025-10-11T11:40:33.04138","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"1167b2a3","cell_type":"markdown","source":"12. **Calcul des poids d'échantillonnage pour la stratification:** Les âges sont regroupés en bins pour la stratification. Les poids d'échantillonnage sont calculés pour chaque bin d'âge en fonction de la fréquence des âges dans ce bin. Un `WeightedRandomSampler` est créé à partir de ces poids. Ce sampler sera utilisé pendant l'entraînement pour pondérer l'échantillonnage des données et aider à gérer le déséquilibre potentiel des classes d'âge.","metadata":{"id":"1167b2a3","papermill":{"duration":0.009461,"end_time":"2025-10-11T11:40:33.635984","exception":false,"start_time":"2025-10-11T11:40:33.626523","status":"completed"},"tags":[]}},{"id":"c1890d6c","cell_type":"code","source":"age_bins = np.digitize(train_ages, bins=np.arange(0, 101, 10))  # Bin ages for stratification\nclass_counts = Counter(age_bins)\nsample_weights = [1.0 / class_counts[age_bins[i]] for i in range(len(train_ages))]\nsampler = WeightedRandomSampler(sample_weights, len(sample_weights))","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.656595Z","iopub.status.busy":"2025-10-11T11:40:33.655883Z","iopub.status.idle":"2025-10-11T11:40:33.677292Z","shell.execute_reply":"2025-10-11T11:40:33.67678Z"},"id":"c1890d6c","papermill":{"duration":0.032748,"end_time":"2025-10-11T11:40:33.678337","exception":false,"start_time":"2025-10-11T11:40:33.645589","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488918349,"user_tz":-60,"elapsed":20,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"069d3d4f","cell_type":"markdown","source":"13. **Création des datasets PyTorch:** Les datasets d'entraînement et de validation sont créés en utilisant la classe `AgeDataset` définie précédemment, en passant les chemins d'accès aux images, les âges, les genres et les transformations d'images appropriées (`train_transform` pour l'entraînement et `val_transform` pour la validation).","metadata":{"id":"069d3d4f","papermill":{"duration":0.009633,"end_time":"2025-10-11T11:40:33.697759","exception":false,"start_time":"2025-10-11T11:40:33.688126","status":"completed"},"tags":[]}},{"id":"079cf73f","cell_type":"code","source":"train_dataset = AgeDataset(train_paths, train_ages, train_genders,train_transform)\nval_dataset = AgeDataset(val_paths, val_ages,val_genders,val_transform)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.718706Z","iopub.status.busy":"2025-10-11T11:40:33.718465Z","iopub.status.idle":"2025-10-11T11:40:33.721968Z","shell.execute_reply":"2025-10-11T11:40:33.721251Z"},"id":"079cf73f","papermill":{"duration":0.014748,"end_time":"2025-10-11T11:40:33.722952","exception":false,"start_time":"2025-10-11T11:40:33.708204","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488918358,"user_tz":-60,"elapsed":6,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"23723e76","cell_type":"markdown","source":"14. **Création des DataLoaders PyTorch:** Les DataLoaders d'entraînement et de validation sont créés pour charger les données par batch pendant l'entraînement et la validation. Le `train_loader` utilise le `WeightedRandomSampler` pour l'échantillonnage pondéré, tandis que le `val_loader` ne mélange pas les données (`shuffle=False`) car l'ordre n'est pas important pour la validation. `num_workers` et `pin_memory` sont configurés pour optimiser le chargement des données.","metadata":{"id":"23723e76","papermill":{"duration":0.009866,"end_time":"2025-10-11T11:40:33.742561","exception":false,"start_time":"2025-10-11T11:40:33.732695","status":"completed"},"tags":[]}},{"id":"2b2e6f11","cell_type":"code","source":"train_loader = DataLoader(train_dataset, batch_size=16, sampler=sampler, num_workers=0, pin_memory=True)\nval_loader = DataLoader(val_dataset, batch_size=16, shuffle=False, num_workers=0, pin_memory=True)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.763021Z","iopub.status.busy":"2025-10-11T11:40:33.76241Z","iopub.status.idle":"2025-10-11T11:40:33.76649Z","shell.execute_reply":"2025-10-11T11:40:33.765928Z"},"id":"2b2e6f11","papermill":{"duration":0.015339,"end_time":"2025-10-11T11:40:33.767527","exception":false,"start_time":"2025-10-11T11:40:33.752188","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488918400,"user_tz":-60,"elapsed":8,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"a6c7955a","cell_type":"markdown","source":"## Configuration du Modèle et de l'Entraînement\n\nCette section configure le modèle de réseau neuronal, la fonction de perte, l'optimiseur et le scheduler.\n\n15. **Définition du modèle:** Un modèle pré-entraîné `efficientnet_b7` de `torchvision.models` est chargé. La couche de classification finale est remplacée par un nouveau séquentiel de couches linéaires et de dropout pour s'adapter à la tâche combinée d'estimation de l'âge (régression) et de classification du genre.","metadata":{"id":"a6c7955a","papermill":{"duration":0.00933,"end_time":"2025-10-11T11:40:33.786777","exception":false,"start_time":"2025-10-11T11:40:33.777447","status":"completed"},"tags":[]}},{"id":"872551eb","cell_type":"code","source":"model = models.efficientnet_b7()\nnum_features = model.classifier[1].in_features\nmodel.classifier = nn.Sequential(\n    nn.Dropout(0.5),\n    nn.Linear(num_features, 512),\n    nn.ReLU(),\n    nn.Dropout(0.3),\n    nn.Linear(512, 3)  # Sortie : age (regression) + genre (classification)\n)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:33.80673Z","iopub.status.busy":"2025-10-11T11:40:33.806516Z","iopub.status.idle":"2025-10-11T11:40:35.042801Z","shell.execute_reply":"2025-10-11T11:40:35.041937Z"},"id":"872551eb","papermill":{"duration":1.247867,"end_time":"2025-10-11T11:40:35.044211","exception":false,"start_time":"2025-10-11T11:40:33.796344","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488918987,"user_tz":-60,"elapsed":592,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"46f796b9","cell_type":"markdown","source":"16. **Définition de la fonction de perte combinée:** Une classe personnalisée `CombinedLoss` est définie, héritant de `nn.Module`. Cette fonction de perte combine la perte L1 pour l'estimation de l'âge (régression) et la perte d'entropie croisée pour la classification du genre. Elle inclut également une pénalité pour les âges prédits en dehors d'une plage spécifiée (`min_age`, `max_age`). Un paramètre `alpha` permet de pondérer l'importance relative des pertes d'âge et de genre.","metadata":{"id":"46f796b9","papermill":{"duration":0.009609,"end_time":"2025-10-11T11:40:35.064309","exception":false,"start_time":"2025-10-11T11:40:35.0547","status":"completed"},"tags":[]}},{"id":"6e65cc11","cell_type":"code","source":"class CombinedLoss(nn.Module):\n    def __init__(self, alpha=0.8,min_age=10, max_age=80):\n        super().__init__()\n        self.age_loss = nn.L1Loss()\n        self.gender_loss = nn.CrossEntropyLoss()\n        self.alpha = alpha\n        self.min_age = min_age\n        self.max_age = max_age\n\n    def forward(self, age_pred, gender_pred, age_target, gender_target):\n        age_l = self.age_loss(age_pred, age_target)\n\n        penalty = torch.mean(torch.relu(self.min_age - age_pred)) + torch.mean(torch.relu(age_pred - self.max_age)) # Pénalité pour âges hors plage\n        gender_l = self.gender_loss(gender_pred, gender_target)\n        return self.alpha * (age_l + 0.1 * penalty) + (1 - self.alpha) * gender_l","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.088404Z","iopub.status.busy":"2025-10-11T11:40:35.087873Z","iopub.status.idle":"2025-10-11T11:40:35.095745Z","shell.execute_reply":"2025-10-11T11:40:35.094886Z"},"id":"6e65cc11","papermill":{"duration":0.0229,"end_time":"2025-10-11T11:40:35.096919","exception":false,"start_time":"2025-10-11T11:40:35.074019","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919041,"user_tz":-60,"elapsed":20,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"f6084cb4","cell_type":"markdown","source":"17. **Instanciation de la fonction de perte:** La fonction de perte combinée est instanciée avec un poids `alpha` de 0.9, donnant plus d'importance à la perte d'âge.","metadata":{"id":"f6084cb4","papermill":{"duration":0.016008,"end_time":"2025-10-11T11:40:35.123074","exception":false,"start_time":"2025-10-11T11:40:35.107066","status":"completed"},"tags":[]}},{"id":"8b4b2e61","cell_type":"code","source":"criterion = CombinedLoss(alpha=0.9)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.148945Z","iopub.status.busy":"2025-10-11T11:40:35.148642Z","iopub.status.idle":"2025-10-11T11:40:35.153048Z","shell.execute_reply":"2025-10-11T11:40:35.152115Z"},"id":"8b4b2e61","papermill":{"duration":0.017877,"end_time":"2025-10-11T11:40:35.154511","exception":false,"start_time":"2025-10-11T11:40:35.136634","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919048,"user_tz":-60,"elapsed":3,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"e19fcd19","cell_type":"markdown","source":"18. **Configuration du périphérique et déplacement du modèle:** Le périphérique d'entraînement (GPU si disponible, sinon CPU) est détecté. Le modèle est ensuite déplacé vers ce périphérique. Le modèle est également converti en `float` pour une compatibilité potentielle avec l'entraînement en précision mixte.","metadata":{"id":"e19fcd19","papermill":{"duration":0.009699,"end_time":"2025-10-11T11:40:35.178623","exception":false,"start_time":"2025-10-11T11:40:35.168924","status":"completed"},"tags":[]}},{"id":"2a7a71c4","cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nprint(f\"GPU available ?{torch.cuda.is_available()}\")\nmodel = model.to(device)\nmodel = model.float()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.198682Z","iopub.status.busy":"2025-10-11T11:40:35.198485Z","iopub.status.idle":"2025-10-11T11:40:35.533377Z","shell.execute_reply":"2025-10-11T11:40:35.532789Z"},"executionInfo":{"elapsed":51,"status":"ok","timestamp":1767488919102,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"2a7a71c4","outputId":"e6540477-d7f5-4afe-95f6-b116f0a564ab","papermill":{"duration":0.346423,"end_time":"2025-10-11T11:40:35.534667","exception":false,"start_time":"2025-10-11T11:40:35.188244","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"a21e2b4f","cell_type":"markdown","source":"19. **Configuration de l'optimiseur et du scheduler:** Un optimiseur AdamW est configuré pour mettre à jour les poids du modèle pendant l'entraînement. Un scheduler `CosineAnnealingWarmRestarts` est utilisé pour ajuster dynamiquement le taux d'apprentissage au cours de l'entraînement, en le faisant redémarrer périodiquement.","metadata":{"id":"a21e2b4f","papermill":{"duration":0.009715,"end_time":"2025-10-11T11:40:35.554586","exception":false,"start_time":"2025-10-11T11:40:35.544871","status":"completed"},"tags":[]}},{"id":"e219a253","cell_type":"code","source":"optimizer = optim.AdamW(model.parameters(), lr=0.0003, weight_decay=0.01)\nscheduler = optim.lr_scheduler.CosineAnnealingWarmRestarts(optimizer, T_0=10, T_mult=2, eta_min=1e-7)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.574924Z","iopub.status.busy":"2025-10-11T11:40:35.574709Z","iopub.status.idle":"2025-10-11T11:40:35.581155Z","shell.execute_reply":"2025-10-11T11:40:35.580468Z"},"id":"e219a253","papermill":{"duration":0.01795,"end_time":"2025-10-11T11:40:35.582199","exception":false,"start_time":"2025-10-11T11:40:35.564249","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919132,"user_tz":-60,"elapsed":27,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"2a0daa3f","cell_type":"markdown","source":"20. **Configuration du GradScaler pour l'entraînement en précision mixte:** Un `GradScaler` est instancié. Il est utilisé pour faciliter l'entraînement en précision mixte (`autocast`), ce qui peut accélérer l'entraînement et réduire l'utilisation de la mémoire GPU en utilisant des types de données à virgule flottante de précision inférieure (comme float16) pour certaines opérations.","metadata":{"id":"2a0daa3f","papermill":{"duration":0.009529,"end_time":"2025-10-11T11:40:35.601325","exception":false,"start_time":"2025-10-11T11:40:35.591796","status":"completed"},"tags":[]}},{"id":"8081ab88","cell_type":"code","source":"scaler = GradScaler('cuda')","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.621353Z","iopub.status.busy":"2025-10-11T11:40:35.621142Z","iopub.status.idle":"2025-10-11T11:40:35.624493Z","shell.execute_reply":"2025-10-11T11:40:35.623771Z"},"id":"8081ab88","papermill":{"duration":0.01474,"end_time":"2025-10-11T11:40:35.625662","exception":false,"start_time":"2025-10-11T11:40:35.610922","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919177,"user_tz":-60,"elapsed":41,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}},"outputId":"5b667bec-a27c-4b2a-d5c8-6f3cc2334012"},"outputs":[],"execution_count":null},{"id":"5f6bd3a1","cell_type":"markdown","source":"21. **Initialisation des paramètres d'entraînement:** Le nombre total d'époques pour l'entraînement est défini. Les variables `best_val_mae`, `patience`, et `counter` sont initialisées pour le mécanisme d'arrêt anticipé.","metadata":{"id":"5f6bd3a1","papermill":{"duration":0.009502,"end_time":"2025-10-11T11:40:35.644857","exception":false,"start_time":"2025-10-11T11:40:35.635355","status":"completed"},"tags":[]}},{"id":"05c0c2ad","cell_type":"code","source":"num_epochs = 32\nbest_val_mae = float('inf')\npatience = 10\ncounter = 0","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.665194Z","iopub.status.busy":"2025-10-11T11:40:35.664957Z","iopub.status.idle":"2025-10-11T11:40:35.66832Z","shell.execute_reply":"2025-10-11T11:40:35.667666Z"},"id":"05c0c2ad","papermill":{"duration":0.014982,"end_time":"2025-10-11T11:40:35.669582","exception":false,"start_time":"2025-10-11T11:40:35.6546","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919183,"user_tz":-60,"elapsed":3,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"7c369149","cell_type":"markdown","source":"22. **Initialisation des listes pour suivre les métriques:** Des listes vides sont créées pour stocker les pertes d'entraînement et les MAE de validation à chaque époque, ce qui permettra de visualiser les progrès de l'entraînement.","metadata":{"id":"7c369149","papermill":{"duration":0.009448,"end_time":"2025-10-11T11:40:35.688857","exception":false,"start_time":"2025-10-11T11:40:35.679409","status":"completed"},"tags":[]}},{"id":"63d2072a","cell_type":"code","source":"train_losses = []\nval_maes = []","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.709195Z","iopub.status.busy":"2025-10-11T11:40:35.708598Z","iopub.status.idle":"2025-10-11T11:40:35.711721Z","shell.execute_reply":"2025-10-11T11:40:35.711158Z"},"id":"63d2072a","papermill":{"duration":0.014366,"end_time":"2025-10-11T11:40:35.712762","exception":false,"start_time":"2025-10-11T11:40:35.698396","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919195,"user_tz":-60,"elapsed":4,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"097659fa","cell_type":"markdown","source":"23. **Vidage du cache CUDA:** Le cache de la mémoire GPU est vidé pour libérer de la mémoire avant de commencer l'entraînement.","metadata":{"id":"097659fa","papermill":{"duration":0.009384,"end_time":"2025-10-11T11:40:35.731919","exception":false,"start_time":"2025-10-11T11:40:35.722535","status":"completed"},"tags":[]}},{"id":"a77de3ac","cell_type":"code","source":"torch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.751787Z","iopub.status.busy":"2025-10-11T11:40:35.751578Z","iopub.status.idle":"2025-10-11T11:40:35.754891Z","shell.execute_reply":"2025-10-11T11:40:35.754184Z"},"id":"a77de3ac","papermill":{"duration":0.014552,"end_time":"2025-10-11T11:40:35.756","exception":false,"start_time":"2025-10-11T11:40:35.741448","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919245,"user_tz":-60,"elapsed":47,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"1423e148","cell_type":"markdown","source":"24. **Initialisation des listes pour les prédictions de genre de validation:** Des listes vides sont créées pour stocker les prédictions et les cibles de genre pendant la validation.","metadata":{"id":"1423e148","papermill":{"duration":0.00936,"end_time":"2025-10-11T11:40:35.774959","exception":false,"start_time":"2025-10-11T11:40:35.765599","status":"completed"},"tags":[]}},{"id":"fe0b62c9","cell_type":"code","source":"val_preds_gender = []\nval_targets_gender = []","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.794928Z","iopub.status.busy":"2025-10-11T11:40:35.794722Z","iopub.status.idle":"2025-10-11T11:40:35.797955Z","shell.execute_reply":"2025-10-11T11:40:35.797261Z"},"id":"fe0b62c9","papermill":{"duration":0.014419,"end_time":"2025-10-11T11:40:35.799081","exception":false,"start_time":"2025-10-11T11:40:35.784662","status":"completed"},"tags":[],"executionInfo":{"status":"ok","timestamp":1767488919247,"user_tz":-60,"elapsed":11,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"9e1hP14ujR3z","cell_type":"code","source":"next(iter(train_loader))[0].shape","metadata":{"id":"9e1hP14ujR3z","executionInfo":{"status":"ok","timestamp":1767489135698,"user_tz":-60,"elapsed":66,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}},"outputId":"c8e57f29-806d-4223-94cb-37cae2ac923e"},"outputs":[],"execution_count":null},{"id":"017b0a59","cell_type":"markdown","source":"## Entraînement du Modèle\n\nCette section contient la boucle principale d'entraînement et de validation.\n\n25. **Boucle d'entraînement par époque:** Le code itère sur le nombre spécifié d'époques. Pour chaque époque, le modèle est entraîné sur l'ensemble d'entraînement en calculant la perte combinée (âge et genre) et en mettant à jour les poids. Ensuite, le modèle est évalué sur l'ensemble de validation pour calculer la MAE de l'âge et les métriques du genre. Le meilleur modèle est sauvegardé en fonction de la MAE de validation, et un arrêt anticipé est mis en œuvre.","metadata":{"id":"017b0a59","papermill":{"duration":0.009441,"end_time":"2025-10-11T11:40:35.818147","exception":false,"start_time":"2025-10-11T11:40:35.808706","status":"completed"},"tags":[]}},{"id":"b0d56b75","cell_type":"code","source":"for epoch in range(num_epochs):\n    model.train()\n    running_loss = 0.0\n    for images, ages, genders in train_loader:\n        images = images.to(device, non_blocking=True)\n        ages = ages.float().unsqueeze(1).to(device, non_blocking=True)\n        genders = genders.long().to(device, non_blocking=True)\n\n        # Vérification formes\n        if ages.shape[0] != images.shape[0]:\n            print(f\"Shape mismatch in batch {epoch}: ages {ages.shape}, images {images.shape}\")\n            continue\n\n        optimizer.zero_grad()\n        with autocast(device_type='cuda' if device.type == 'cuda' else 'cpu'):\n            outputs = model(images)\n            age_pred = outputs[:, 0:1]  # (batch,1)\n            gender_pred = outputs[:, 1:3]\n            loss = criterion(age_pred, gender_pred, ages, genders)\n\n        scaler.scale(loss).backward()\n        scaler.step(optimizer)\n        scaler.update()\n        running_loss += loss.item() * images.size(0)\n\n    epoch_loss = running_loss / len(train_loader.dataset)\n    train_losses.append(epoch_loss)\n    scheduler.step()\n\n    # Validation\n    model.eval()\n    val_preds = []\n    val_targets = []\n    val_gender_preds = []\n    val_gender_targets = []\n    val_loss = 0.0\n    with torch.no_grad():\n        for images, ages, genders in val_loader:\n            images = images.to(device)\n            ages = ages.float().unsqueeze(1).to(device, non_blocking=True)\n            with autocast(device_type='cuda' if device.type == 'cuda' else 'cpu'):\n                features = model.features(images)\n                x = model.avgpool(features)\n                x = torch.flatten(x, 1)\n                classifier_out = model.classifier(x)\n                age_pred = classifier_out[:, 0:1]\n                val_loss += nn.L1Loss(reduction='sum')(age_pred, ages).item()\n                val_preds.extend(age_pred.cpu().numpy().flatten())\n                val_targets.extend(ages.cpu().numpy().flatten())\n\n                val_gender_preds.extend(torch.argmax(gender_pred, dim=1).cpu().numpy())\n                val_gender_targets.extend(genders.cpu().numpy())\n\n    val_mae = mean_absolute_error(val_targets, val_preds)\n    val_maes.append(val_mae)\n    print(f'Epoch [{epoch+1}/{num_epochs}], Train Loss: {epoch_loss:.4f}, Val MAE: {val_mae:.4f}')\n\n    # Early stopping\n    if val_mae < best_val_mae:\n        best_val_mae = val_mae\n        torch.save(model.state_dict(), 'best_age_model.pth')\n        best_val_gender_preds = val_gender_preds\n        best_val_gender_targets = val_gender_targets\n        counter = 0\n    else:\n        counter += 1\n        if counter >= patience:\n            print(\"Early stopping\")\n            break","metadata":{"execution":{"iopub.execute_input":"2025-10-11T11:40:35.83861Z","iopub.status.busy":"2025-10-11T11:40:35.838043Z","iopub.status.idle":"2025-10-11T22:45:49.457256Z","shell.execute_reply":"2025-10-11T22:45:49.456473Z"},"id":"b0d56b75","outputId":"1cefbd78-e45c-467c-a4ca-a68d1f9c2d60","papermill":{"duration":39913.64216,"end_time":"2025-10-11T22:45:49.469975","exception":false,"start_time":"2025-10-11T11:40:35.827815","status":"completed"},"tags":[],"executionInfo":{"status":"error","timestamp":1767488939359,"user_tz":-60,"elapsed":20118,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"bc7a95a1","cell_type":"markdown","source":"## Visualisation des Résultats de l'Entraînement\n\nCette section génère des graphiques pour visualiser l'évolution de la perte d'entraînement et de la MAE de validation au cours des époques.\n\n26. **Tracé des courbes de perte et de MAE:** Un graphique est créé avec deux sous-graphiques : l'un montrant la perte d'entraînement par époque et l'autre montrant la MAE de validation par époque. Les graphiques sont enregistrés dans un fichier (`training_progress.png`).","metadata":{"id":"bc7a95a1","papermill":{"duration":0.010716,"end_time":"2025-10-11T22:45:49.491704","exception":false,"start_time":"2025-10-11T22:45:49.480988","status":"completed"},"tags":[]}},{"id":"27272121","cell_type":"code","source":"plt.figure(figsize=(10, 5))\nplt.subplot(1, 2, 1)\nplt.plot(train_losses, label='Train Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.legend()\nplt.subplot(1, 2, 2)\nplt.plot(val_maes, label='Val MAE')\nplt.xlabel('Epochs')\nplt.ylabel('MAE')\nplt.legend()\nplt.show()\nplt.savefig('training_progress.png')\nplt.close()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:49.51448Z","iopub.status.busy":"2025-10-11T22:45:49.514216Z","iopub.status.idle":"2025-10-11T22:45:49.890654Z","shell.execute_reply":"2025-10-11T22:45:49.889799Z"},"id":"27272121","papermill":{"duration":0.389521,"end_time":"2025-10-11T22:45:49.892096","exception":false,"start_time":"2025-10-11T22:45:49.502575","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939426,"user_tz":-60,"elapsed":381,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"c4922567","cell_type":"code","source":"print(f\"\\nMétriques finales :\")\nprint(f\"MAE (Age): {best_val_mae:.4f}\")","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:49.917145Z","iopub.status.busy":"2025-10-11T22:45:49.916914Z","iopub.status.idle":"2025-10-11T22:45:49.92051Z","shell.execute_reply":"2025-10-11T22:45:49.919699Z"},"id":"c4922567","papermill":{"duration":0.016957,"end_time":"2025-10-11T22:45:49.921569","exception":false,"start_time":"2025-10-11T22:45:49.904612","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939430,"user_tz":-60,"elapsed":381,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"euNMiBP6s1vX","cell_type":"markdown","source":"Nous avons une **MAE** de **2.886** ce qui montre que le modèle obtenu est très performant.","metadata":{"id":"euNMiBP6s1vX"}},{"id":"967ed543","cell_type":"markdown","source":"## Test et Génération du Fichier de Soumission\n\nCette section charge le meilleur modèle entraîné, effectue des prédictions sur l'ensemble de test et génère le fichier de soumission.","metadata":{"id":"967ed543","papermill":{"duration":0.011237,"end_time":"2025-10-11T22:45:49.94434","exception":false,"start_time":"2025-10-11T22:45:49.933103","status":"completed"},"tags":[]}},{"id":"28d59156","cell_type":"markdown","source":"27. **Chargement du meilleur modèle entraîné:** Un nouveau modèle `efficientnet_b7` est instancié avec la même structure que le modèle entraîné. L'état du meilleur modèle sauvegardé (`best_age_model.pth`) est chargé dans ce nouveau modèle. Le modèle est mis en mode évaluation (`model.eval()`).","metadata":{"id":"28d59156","papermill":{"duration":0.011279,"end_time":"2025-10-11T22:45:49.967059","exception":false,"start_time":"2025-10-11T22:45:49.95578","status":"completed"},"tags":[]}},{"id":"1960f6fe","cell_type":"code","source":"model = models.efficientnet_b7()\nnum_features = model.classifier[1].in_features\nmodel.classifier = nn.Sequential(\n    nn.Dropout(0.5),\n    nn.Linear(num_features, 512),\n    nn.ReLU(),\n    nn.Dropout(0.3),\n    nn.Linear(512, 3)  # 1 (âge) + 2 (genre)\n)\nmodel = model.to(device)\nmodel = model.float()\n\nmodel.load_state_dict(torch.load('best_age_model.pth', map_location=device))\nmodel.eval()\nprint(\"Modèle chargé avec succès\")","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:49.990927Z","iopub.status.busy":"2025-10-11T22:45:49.990703Z","iopub.status.idle":"2025-10-11T22:45:51.839011Z","shell.execute_reply":"2025-10-11T22:45:51.838204Z"},"executionInfo":{"elapsed":382,"status":"aborted","timestamp":1767488939433,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"1960f6fe","papermill":{"duration":1.861696,"end_time":"2025-10-11T22:45:51.840171","exception":false,"start_time":"2025-10-11T22:45:49.978475","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"42e8fef2","cell_type":"markdown","source":"28. **Préparation des données de test:** Les chemins d'accès aux images de l'ensemble de test sont générés. Un `AgeDataset` est créé pour l'ensemble de test (sans âges ni genres car ce sont les valeurs à prédire). Un `DataLoader` est créé pour charger les données de test par batch.","metadata":{"id":"42e8fef2","papermill":{"duration":0.011671,"end_time":"2025-10-11T22:45:51.863931","exception":false,"start_time":"2025-10-11T22:45:51.85226","status":"completed"},"tags":[]}},{"id":"942012bb","cell_type":"code","source":"test_dir = 'data/dataset_tache_2/dataset_tache_2/test'\ntest_image_paths = [os.path.join(test_dir, f'{i}.JPG') for i in range(200)]\ntest_dataset = AgeDataset(test_image_paths)\ntest_loader = DataLoader(test_dataset, batch_size=64, shuffle=False, num_workers=0, pin_memory=True)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:51.888775Z","iopub.status.busy":"2025-10-11T22:45:51.888512Z","iopub.status.idle":"2025-10-11T22:45:51.893088Z","shell.execute_reply":"2025-10-11T22:45:51.892387Z"},"id":"942012bb","papermill":{"duration":0.018664,"end_time":"2025-10-11T22:45:51.894238","exception":false,"start_time":"2025-10-11T22:45:51.875574","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939438,"user_tz":-60,"elapsed":367,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"24d22453","cell_type":"markdown","source":"29. **Inférence sur l'ensemble de test:** Les prédictions sont générées pour les images de l'ensemble de test.\n    - **Désactivation du calcul du gradient:** Le calcul du gradient est désactivé.\n    - **Parcours du DataLoader de test:** Pour chaque batch d'images et d'IDs :\n        - Les images sont déplacées vers le périphérique.\n        - **Inférence avec précision mixte (`autocast`):** L'inférence est effectuée avec `autocast`.\n        - **Obtention des sorties du modèle:** Les sorties du modèle (contenant les prédictions d'âge et de genre) sont obtenues.\n        - **Extraction des prédictions d'âge:** Les prédictions d'âge sont extraites des sorties.\n        - **Stockage des prédictions:** Les prédictions d'âge sont arrondies à l'entier le plus proche et stockées avec l'ID de l'image.","metadata":{"id":"24d22453","papermill":{"duration":0.011497,"end_time":"2025-10-11T22:45:51.918043","exception":false,"start_time":"2025-10-11T22:45:51.906546","status":"completed"},"tags":[]}},{"id":"ef1580b0","cell_type":"code","source":"predictions = []\nwith torch.no_grad():\n    for images, ids in test_loader:\n        images = images.to(device)\n        with autocast(device_type='cuda' if device.type == 'cuda' else 'cpu'):\n            outputs = model(images)\n\n        age_preds = outputs[:, 0].cpu().numpy()\n        age_preds = age_preds.flatten()\n        for id_, pred in zip(ids, age_preds):\n            rounded_pred = round(pred)\n            predictions.append({'image_id': int(id_), 'predicted_age': rounded_pred})","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:51.942007Z","iopub.status.busy":"2025-10-11T22:45:51.941773Z","iopub.status.idle":"2025-10-11T22:45:53.127995Z","shell.execute_reply":"2025-10-11T22:45:53.127402Z"},"id":"ef1580b0","papermill":{"duration":1.199559,"end_time":"2025-10-11T22:45:53.129251","exception":false,"start_time":"2025-10-11T22:45:51.929692","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939441,"user_tz":-60,"elapsed":368,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"151fea00","cell_type":"markdown","source":"30. **Création et sauvegarde du fichier de soumission:** Les prédictions collectées sont organisées dans un DataFrame pandas, triées par `image_id`, et sauvegardées dans un fichier CSV nommé `submission_task_2.csv` au format requis pour la soumission.","metadata":{"id":"151fea00","papermill":{"duration":0.011719,"end_time":"2025-10-11T22:45:53.153569","exception":false,"start_time":"2025-10-11T22:45:53.14185","status":"completed"},"tags":[]}},{"id":"c4e57942","cell_type":"code","source":"pd.DataFrame(predictions).sort_values('image_id').to_csv('submission_task_2.csv', index=False)\nprint('Predictions saved to submission_task_2.csv')","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.177626Z","iopub.status.busy":"2025-10-11T22:45:53.177375Z","iopub.status.idle":"2025-10-11T22:45:53.190922Z","shell.execute_reply":"2025-10-11T22:45:53.190143Z"},"executionInfo":{"elapsed":131141,"status":"aborted","timestamp":1767488939445,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"c4e57942","papermill":{"duration":0.026852,"end_time":"2025-10-11T22:45:53.192004","exception":false,"start_time":"2025-10-11T22:45:53.165152","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"fe54d9e8","cell_type":"code","source":"df=pd.read_csv('submission_task_2.csv')\ndf.head()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.216192Z","iopub.status.busy":"2025-10-11T22:45:53.216013Z","iopub.status.idle":"2025-10-11T22:45:53.235847Z","shell.execute_reply":"2025-10-11T22:45:53.235235Z"},"executionInfo":{"elapsed":131138,"status":"aborted","timestamp":1767488939448,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"fe54d9e8","papermill":{"duration":0.033192,"end_time":"2025-10-11T22:45:53.236888","exception":false,"start_time":"2025-10-11T22:45:53.203696","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"7e3b218d","cell_type":"markdown","source":"31. **Affichage des 50 premières lignes du DataFrame de soumission:** Les 50 premières lignes du DataFrame de soumission sont affichées.","metadata":{"id":"7e3b218d","papermill":{"duration":0.011864,"end_time":"2025-10-11T22:45:53.262063","exception":false,"start_time":"2025-10-11T22:45:53.250199","status":"completed"},"tags":[]}},{"id":"dbdca18a","cell_type":"code","source":"df.head(50)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.286742Z","iopub.status.busy":"2025-10-11T22:45:53.286248Z","iopub.status.idle":"2025-10-11T22:45:53.294017Z","shell.execute_reply":"2025-10-11T22:45:53.293273Z"},"executionInfo":{"elapsed":131136,"status":"aborted","timestamp":1767488939452,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"dbdca18a","papermill":{"duration":0.021476,"end_time":"2025-10-11T22:45:53.295152","exception":false,"start_time":"2025-10-11T22:45:53.273676","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"fff72adf","cell_type":"code","source":"df[df['predicted_age']<15]","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.321527Z","iopub.status.busy":"2025-10-11T22:45:53.320854Z","iopub.status.idle":"2025-10-11T22:45:53.331156Z","shell.execute_reply":"2025-10-11T22:45:53.33041Z"},"executionInfo":{"elapsed":131133,"status":"aborted","timestamp":1767488939455,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"},"user_tz":-60},"id":"fff72adf","papermill":{"duration":0.024618,"end_time":"2025-10-11T22:45:53.332306","exception":false,"start_time":"2025-10-11T22:45:53.307688","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"a0d98aed","cell_type":"code","source":"from PIL import Image","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.357729Z","iopub.status.busy":"2025-10-11T22:45:53.357528Z","iopub.status.idle":"2025-10-11T22:45:53.360579Z","shell.execute_reply":"2025-10-11T22:45:53.36004Z"},"id":"a0d98aed","papermill":{"duration":0.017059,"end_time":"2025-10-11T22:45:53.361692","exception":false,"start_time":"2025-10-11T22:45:53.344633","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939458,"user_tz":-60,"elapsed":131135,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"32bae842","cell_type":"code","source":"def display_images_with_predictions(predictions_df, test_dir, num_images=10):\n    selected_ids = predictions_df['image_id'].iloc[:num_images].tolist()\n    plt.figure(figsize=(15, 10))\n    for i, image_id in enumerate(selected_ids):\n        img_path = os.path.join(test_dir, f'{image_id}.JPG')\n        image = np.array(Image.open(img_path).convert('RGB'))\n        age_pred = predictions_df[predictions_df['image_id'] == image_id]['predicted_age'].iloc[0]\n        plt.subplot(2, 5, i + 1)\n        plt.imshow(image)\n        plt.title(f\"Image ID: {image_id}\\nÂge prédit: {age_pred}\")\n        plt.axis('off')\n    plt.tight_layout()\n    plt.show()","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.387162Z","iopub.status.busy":"2025-10-11T22:45:53.386919Z","iopub.status.idle":"2025-10-11T22:45:53.391858Z","shell.execute_reply":"2025-10-11T22:45:53.391314Z"},"id":"32bae842","papermill":{"duration":0.019063,"end_time":"2025-10-11T22:45:53.392901","exception":false,"start_time":"2025-10-11T22:45:53.373838","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939461,"user_tz":-60,"elapsed":131137,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"c7142328","cell_type":"code","source":"display_images_with_predictions(df, test_dir, num_images=10)","metadata":{"execution":{"iopub.execute_input":"2025-10-11T22:45:53.418236Z","iopub.status.busy":"2025-10-11T22:45:53.418031Z","iopub.status.idle":"2025-10-11T22:45:55.058402Z","shell.execute_reply":"2025-10-11T22:45:55.05765Z"},"id":"c7142328","papermill":{"duration":1.674728,"end_time":"2025-10-11T22:45:55.08006","exception":false,"start_time":"2025-10-11T22:45:53.405332","status":"completed"},"tags":[],"executionInfo":{"status":"aborted","timestamp":1767488939464,"user_tz":-60,"elapsed":131134,"user":{"displayName":"Parfait BOTCHI","userId":"13971785570063439557"}}},"outputs":[],"execution_count":null},{"id":"1133c217","cell_type":"markdown","source":"## Conclusion\n\nCette tentative pour la tâche 2 du challenge a permis de mettre en place un pipeline fonctionnel pour l'estimation de l'âge. Nous avons réussi à charger et prétraiter les données, entraîner un modèle initial, et tester la robustesse du modèle. Les résultats obtenus, incluant une **MAE** de validation de **2.8860** pour l'âge et les métriques de genre, servent de base d'évaluation pour cette première approche. Il sera important d'analyser ces résultats en détail pour identifier les axes d'amélioration pour les prochaines tentatives dans le cadre du challenge, notamment en explorant des architectures de modèles plus performantes, des techniques d'augmentation de données adaptées, ou une stratégie d'entraînement affinée.","metadata":{"id":"1133c217","papermill":{"duration":0.030044,"end_time":"2025-10-11T22:45:55.225127","exception":false,"start_time":"2025-10-11T22:45:55.195083","status":"completed"},"tags":[]}}]}