Author: Will Hinton
\n\n\nThis notebook builds, trains, evaluates, visualizes, and prepares a Kaggle competition submission for a Generative Adversarial Network (GAN) project using the I’m Something of a Painter Myself dataset. The project objective is to generate Monet-style synthetic artwork from photographic source images while demonstrating the adversarial interaction between a generator and a discriminator. This version uses the official Kaggle competition mount path /kaggle/input/gan-getting-started, prioritizes the TFRecord inputs, and packages the final submission in the style of a successful public Kaggle notebook for this competition.\n
\nThe dataset contains four directories: monet_tfrec, photo_tfrec, monet_jpg, and photo_jpg. The Monet folders contain the target-domain paintings used to learn the Monet style, and the photo folders contain source-domain photographs that are translated into Monet-like outputs. The public competition dataset contains 300 Monet paintings and 7,028 source photographs, all sized 256 × 256. The notebook dynamically sets the submission image count from the available source-photo count when it falls within Kaggle’s 7,000–10,000 image requirement; otherwise, it falls back to a safe 7,000-image target.\n
\nSubmissions are evaluated using MiFID, the Memorization-informed Fréchet Inception Distance used by Kaggle for this competition. The final scoreable artifact is /kaggle/working/images.zip, a top-level archive of RGB JPEG images sized 256 × 256. At a high level, the notebook performs environment setup, official dataset loading, preprocessing and normalization, CycleGAN-style model construction and training, visual evaluation, compact diagnostic evaluation, submission packaging, results documentation, conclusion, and references.\n
/kaggle/working/images.zip. The preferred public submission size is 7,028 images, matching the public photo_tfrec count, while the notebook retains a safe 7,000-image fallback for private reruns or alternate mounts.\n","metadata":{"_cell_guid":"4466b83d-ca3e-4a28-bd1f-553b53f9f0c7","_uuid":"9a0dde02-787a-4565-8bae-7ef13a187f55","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.004005,"end_time":"2026-05-13T23:26:13.990309+00:00","exception":false,"start_time":"2026-05-13T23:26:13.986304+00:00","status":"completed"},"tags":[]}},{"id":"e1b34503","cell_type":"code","source":"# ============================================================\n# 0. Library Imports and Reproducibility\n# Author: Will Hinton\n# Kaggle Python 3.12 / GPU-compatible setup\n# ============================================================\n\n# ---- TensorFlow/CUDA log controls must be set before TensorFlow import ----\nimport os\n\nos.environ[\"TF_CPP_MIN_LOG_LEVEL\"] = \"3\"\nos.environ[\"TF_FORCE_GPU_ALLOW_GROWTH\"] = \"true\"\nos.environ[\"TF_CUDNN_USE_FRONTEND\"] = \"0\"\nos.environ[\"PYTHONHASHSEED\"] = \"42\"\nos.environ[\"AUTOGRAPH_VERBOSITY\"] = \"0\"\nos.environ[\"PYDEVD_DISABLE_FILE_VALIDATION\"] = \"1\"\n\n# ---- Standard library imports ----\nimport io\nimport gc\nimport math\nimport time\nimport random\nimport shutil\nimport zipfile\nimport tarfile\nimport warnings\nfrom pathlib import Path\nfrom typing import Dict, Iterable, List, Optional, Tuple\n\nwarnings.filterwarnings(\"ignore\")\n\n# ---- Data and visualization imports ----\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom PIL import Image\n\n# ---- Suppress TensorFlow's C-level stderr messages during import ----\nclass _SuppressStderr:\n def __enter__(self):\n self._stderr_fd = os.dup(2)\n self._devnull_fd = os.open(os.devnull, os.O_WRONLY)\n os.dup2(self._devnull_fd, 2)\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n os.dup2(self._stderr_fd, 2)\n os.close(self._stderr_fd)\n os.close(self._devnull_fd)\n\n\nwith _SuppressStderr():\n import tensorflow as tf\n from tensorflow.keras import layers\n\n# ---- TensorFlow Python-level logging controls ----\ntf.get_logger().setLevel(\"ERROR\")\ntry:\n from absl import logging as absl_logging\n absl_logging.set_verbosity(absl_logging.ERROR)\nexcept Exception:\n pass\n\nAUTHOR = \"Will Hinton\"\nPROJECT_TITLE = \"Generating Monet-Style Images with a Generative Adversarial Network (GAN)\"\n\nIMG_SIZE = 256\nIMG_SHAPE = (IMG_SIZE, IMG_SIZE, 3)\nBATCH_SIZE = 1\nINFERENCE_BATCH_SIZE = 16\nEPOCHS = 15\nMAX_STEPS_PER_EPOCH = 300\nMIN_SUBMISSION_IMAGE_COUNT = 7000\nMAX_SUBMISSION_IMAGE_COUNT = 10000\nPREFERRED_PUBLIC_SUBMISSION_COUNT = 7028\nSUBMISSION_IMAGE_COUNT = PREFERRED_PUBLIC_SUBMISSION_COUNT\nLEARNING_RATE = 2e-4\nBETA_1 = 0.5\nLAMBDA_CYCLE = 10.0\nLAMBDA_IDENTITY = 5.0\nJPEG_QUALITY = 90\n\nDEBUG_FAST_RUN = False\nif DEBUG_FAST_RUN:\n EPOCHS = 1\n MAX_STEPS_PER_EPOCH = 5\n SUBMISSION_IMAGE_COUNT = 128\n\n# ---- Reproducibility ----\nSEED = 42\nrandom.seed(SEED)\nnp.random.seed(SEED)\ntf.keras.utils.set_random_seed(SEED)\n\n# Deterministic ops are intentionally not enabled here.\n# On Kaggle GPU/P100, deterministic convolution selection can trigger extra cuDNN warnings.\n\n# ---- GPU configuration ----\ngpus = tf.config.list_physical_devices(\"GPU\")\nfor gpu in gpus:\n try:\n tf.config.experimental.set_memory_growth(gpu, True)\n except RuntimeError:\n pass\n\nKAGGLE_INPUT_DIR = Path(\"/kaggle/input\")\nKAGGLE_WORKING_DIR = Path(\"/kaggle/working\")\nKAGGLE_TEMP_DIR = Path(\"/kaggle/temp\")\nWORKING_DIR = KAGGLE_WORKING_DIR if KAGGLE_WORKING_DIR.exists() else Path.cwd()\nOUTPUT_ZIP = WORKING_DIR / \"images.zip\"\nTEMP_DATA_DIR = Path(\"/tmp/gan_monet_data\")\nTEMP_IMAGE_DIR = (KAGGLE_TEMP_DIR if KAGGLE_TEMP_DIR.exists() else Path(\"/tmp\")) / \"hinton_monet_submission_images\"\nKAGGLE_COMPETITION_DIR = KAGGLE_INPUT_DIR / \"gan-getting-started\"\n\nprint(\"Author: Will Hinton\")\nprint(f\"TensorFlow: {tf.__version__}\")\nprint(f\"GPU count: {len(gpus)}\")\nprint(f\"Competition path target: {KAGGLE_COMPETITION_DIR}\")\nprint(f\"Final submission path: {OUTPUT_ZIP}\")\n","metadata":{"_cell_guid":"016f67e9-865a-49d1-8e98-43e9a0e20f58","_uuid":"239dc076-2452-48a5-86ff-627a7a88a3f8","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:14.000263Z","iopub.status.busy":"2026-05-13T23:26:13.999885Z","iopub.status.idle":"2026-05-13T23:26:46.118190Z","shell.execute_reply":"2026-05-13T23:26:46.117264Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":32.130412,"end_time":"2026-05-13T23:26:46.124527+00:00","exception":false,"start_time":"2026-05-13T23:26:13.994115+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"65b976f5","cell_type":"markdown","source":"## Step 1 — Load and Inventory the Dataset\n\nThis section verifies and inventories the Kaggle competition data. The notebook first looks for the official competition mount at /kaggle/input/gan-getting-started. It also includes limited fallback discovery for alternate Kaggle mounts, local testing, or compressed dataset copies, but the intended production path is the official competition input.\n\nThe inventory logic prioritizes monet_tfrec and photo_tfrec, then falls back to monet_jpg and photo_jpg if needed. TFRecord example counts are computed directly, and JPEG counts are computed from the directory contents. The notebook then selects the dataset mode as tfrec, jpg, or synthetic-fallback. If the available photo count is between 7,000 and 10,000, that count becomes the submission target; otherwise, the target is reset to 7,000 so the private Kaggle rerun remains inside the competition’s required submission range.\n","metadata":{"_cell_guid":"917ed181-53fa-4a4a-acdd-42476e25e507","_uuid":"2867ba72-9dbf-4e85-ba49-4419f7f32155","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.004805,"end_time":"2026-05-13T23:26:46.133347+00:00","exception":false,"start_time":"2026-05-13T23:26:46.128542+00:00","status":"completed"},"tags":[]}},{"id":"09ea92c7","cell_type":"code","source":"# Step 1: Locate the official Kaggle data root and inventory TFRecord/JPEG inputs.\n\nREQUIRED_DIRS = {\"monet_jpg\", \"photo_jpg\", \"monet_tfrec\", \"photo_tfrec\"}\nTFREC_DIRS = {\"monet_tfrec\", \"photo_tfrec\"}\nJPG_DIRS = {\"monet_jpg\", \"photo_jpg\"}\nARCHIVE_SUFFIXES = (\".zip\", \".tar\", \".tar.gz\", \".tgz\")\n\n\ndef existing_child_dirs(path: Path) -> set:\n return {p.name for p in path.iterdir() if p.is_dir()} if path.exists() and path.is_dir() else set()\n\n\ndef has_expected_data(path: Path) -> bool:\n children = existing_child_dirs(path)\n return bool(TFREC_DIRS.issubset(children) or JPG_DIRS.issubset(children))\n\n\ndef extract_archive(archive_path: Path, destination: Path) -> None:\n destination.mkdir(parents=True, exist_ok=True)\n name = archive_path.name.lower()\n if name.endswith(\".zip\"):\n with zipfile.ZipFile(archive_path, \"r\") as zf:\n zf.extractall(destination)\n elif name.endswith((\".tar\", \".tar.gz\", \".tgz\")):\n with tarfile.open(archive_path, \"r:*\") as tfp:\n tfp.extractall(destination)\n\n\ndef find_data_root() -> Path:\n direct_candidates = [\n KAGGLE_COMPETITION_DIR,\n KAGGLE_INPUT_DIR / \"competitions\" / \"gan-getting-started\",\n KAGGLE_INPUT_DIR,\n Path.cwd(),\n TEMP_DATA_DIR,\n ]\n for candidate in direct_candidates:\n if has_expected_data(candidate):\n return candidate\n\n for base in [KAGGLE_INPUT_DIR, Path.cwd(), TEMP_DATA_DIR]:\n if base.exists():\n for candidate in base.rglob(\"*\"):\n if candidate.is_dir() and has_expected_data(candidate):\n return candidate\n\n archives = []\n for base in [KAGGLE_INPUT_DIR, Path.cwd()]:\n if base.exists():\n archives.extend([p for p in base.rglob(\"*\") if p.is_file() and str(p).lower().endswith(ARCHIVE_SUFFIXES)])\n\n for archive in archives:\n extract_archive(archive, TEMP_DATA_DIR / archive.stem)\n\n for candidate in TEMP_DATA_DIR.rglob(\"*\"):\n if candidate.is_dir() and has_expected_data(candidate):\n return candidate\n\n # Do not raise here; later cells can create a valid fallback submission if a private rerun changes mounts.\n return KAGGLE_COMPETITION_DIR if KAGGLE_COMPETITION_DIR.exists() else KAGGLE_INPUT_DIR\n\n\ndef list_images(directory: Path) -> List[Path]:\n files = []\n for pattern in [\"*.jpg\", \"*.jpeg\", \"*.JPG\", \"*.JPEG\"]:\n files.extend(directory.glob(pattern))\n return sorted(files)\n\n\ndef tfrec_glob(directory: Path) -> List[str]:\n patterns = [\"*.tfrec\", \"*.tfrec*\", \"*.tfrecord\", \"*.tfrecord*\"]\n files: List[str] = []\n for pattern in patterns:\n files.extend(tf.io.gfile.glob(str(directory / pattern)))\n return sorted(set(files))\n\n\ndef count_tfrecord_examples(filenames: List[str]) -> int:\n if not filenames:\n return 0\n count = 0\n for _ in tf.data.TFRecordDataset(filenames):\n count += 1\n return int(count)\n\n\nDATA_ROOT = find_data_root()\nMONET_TFREC_FILENAMES = tfrec_glob(DATA_ROOT / \"monet_tfrec\")\nPHOTO_TFREC_FILENAMES = tfrec_glob(DATA_ROOT / \"photo_tfrec\")\nmonet_files = list_images(DATA_ROOT / \"monet_jpg\")\nphoto_files = list_images(DATA_ROOT / \"photo_jpg\")\n\nMONET_TFREC_COUNT = count_tfrecord_examples(MONET_TFREC_FILENAMES)\nPHOTO_TFREC_COUNT = count_tfrecord_examples(PHOTO_TFREC_FILENAMES)\n\nif MONET_TFREC_COUNT > 0 and PHOTO_TFREC_COUNT > 0:\n DATASET_MODE = \"tfrec\"\n available_photo_count = PHOTO_TFREC_COUNT\n available_monet_count = MONET_TFREC_COUNT\nelif len(monet_files) > 0 and len(photo_files) > 0:\n DATASET_MODE = \"jpg\"\n available_photo_count = len(photo_files)\n available_monet_count = len(monet_files)\nelse:\n DATASET_MODE = \"synthetic-fallback\"\n available_photo_count = 0\n available_monet_count = 0\n\nif not DEBUG_FAST_RUN:\n if MIN_SUBMISSION_IMAGE_COUNT <= available_photo_count <= MAX_SUBMISSION_IMAGE_COUNT:\n # Public competition behavior: 7,028 photos -> 7,028 submitted images.\n SUBMISSION_IMAGE_COUNT = available_photo_count\n else:\n # Hidden reruns or alternate mounts: stay safely within Kaggle's 7,000–10,000 requirement.\n SUBMISSION_IMAGE_COUNT = MIN_SUBMISSION_IMAGE_COUNT\n\ninventory = pd.DataFrame([\n {\"directory\": \"monet_tfrec\", \"count\": MONET_TFREC_COUNT, \"path\": str(DATA_ROOT / \"monet_tfrec\")},\n {\"directory\": \"photo_tfrec\", \"count\": PHOTO_TFREC_COUNT, \"path\": str(DATA_ROOT / \"photo_tfrec\")},\n {\"directory\": \"monet_jpg\", \"count\": len(monet_files), \"path\": str(DATA_ROOT / \"monet_jpg\")},\n {\"directory\": \"photo_jpg\", \"count\": len(photo_files), \"path\": str(DATA_ROOT / \"photo_jpg\")},\n])\n\ndisplay(inventory)\nprint(f\"Dataset root: {DATA_ROOT}\")\nprint(f\"Dataset mode: {DATASET_MODE}\")\nprint(f\"Submission image target: {SUBMISSION_IMAGE_COUNT:,}\")\n","metadata":{"_cell_guid":"5c67357f-40a7-42b5-a24b-c153c2df653d","_uuid":"b4ae17a8-779f-4435-8b06-8c2ea9902722","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:46.142779Z","iopub.status.busy":"2026-05-13T23:26:46.142250Z","iopub.status.idle":"2026-05-13T23:26:46.376057Z","shell.execute_reply":"2026-05-13T23:26:46.374667Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.240732,"end_time":"2026-05-13T23:26:46.377810+00:00","exception":false,"start_time":"2026-05-13T23:26:46.137078+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"dafac26d","cell_type":"markdown","source":"## Step 2 — Data Cleaning, Preprocessing, and Normalization\n\nThis section constructs TensorFlow datasets for training, previewing, and submission generation. TFRecords are decoded first using the official competition path pattern /kaggle/input/gan-getting-started/monet_tfrec/*.tfrec and /kaggle/input/gan-getting-started/photo_tfrec/*.tfrec. JPEG decoding remains available as a fallback when TFRecords are not available.\n\nEach image is decoded as RGB, resized or reshaped to 256 × 256, and normalized to [-1, 1] so it matches the generator’s tanh output range. Training examples use random jitter through resizing, random cropping, and horizontal flipping. The training datasets repeat indefinitely, while the submission dataset is limited to the chosen SUBMISSION_IMAGE_COUNT. When a rerun exposes fewer source images than expected, the pipeline repeats available records or falls back to synthetic tensors so the notebook can still produce a scoreable images.zip rather than throwing an exception.\n","metadata":{"_cell_guid":"950ab64c-bbb0-4783-84f3-d4edace93217","_uuid":"179424e1-aecc-4e21-962d-75091bc32bd9","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.004184,"end_time":"2026-05-13T23:26:46.386471+00:00","exception":false,"start_time":"2026-05-13T23:26:46.382287+00:00","status":"completed"},"tags":[]}},{"id":"86c23393","cell_type":"code","source":"# Step 2a: Build TensorFlow datasets from TFRecords first, then JPEGs, then a no-fail fallback.\n\nAUTOTUNE = tf.data.AUTOTUNE\n\n\ndef normalize_image(image: tf.Tensor) -> tf.Tensor:\n return (tf.cast(image, tf.float32) / 127.5) - 1.0\n\n\ndef denormalize_image(image: tf.Tensor) -> tf.Tensor:\n return tf.clip_by_value((image + 1.0) / 2.0, 0.0, 1.0)\n\n\ndef random_jitter(image: tf.Tensor) -> tf.Tensor:\n image = tf.image.resize(image, [286, 286], method=tf.image.ResizeMethod.BICUBIC)\n image = tf.image.random_crop(image, [IMG_SIZE, IMG_SIZE, 3])\n image = tf.image.random_flip_left_right(image)\n image.set_shape(IMG_SHAPE)\n return image\n\n\ndef decode_tfrecord_example(example: tf.Tensor, training: bool = False) -> tf.Tensor:\n tfrecord_format = {\n \"image_name\": tf.io.FixedLenFeature([], tf.string),\n \"image\": tf.io.FixedLenFeature([], tf.string),\n }\n parsed = tf.io.parse_single_example(example, tfrecord_format)\n image = tf.image.decode_jpeg(parsed[\"image\"], channels=3)\n image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE], method=tf.image.ResizeMethod.BICUBIC)\n image.set_shape(IMG_SHAPE)\n if training:\n image = random_jitter(image)\n return normalize_image(image)\n\n\ndef decode_jpeg_path(path: tf.Tensor, training: bool = False) -> tf.Tensor:\n image = tf.image.decode_jpeg(tf.io.read_file(path), channels=3)\n image = tf.image.resize(image, [IMG_SIZE, IMG_SIZE], method=tf.image.ResizeMethod.BICUBIC)\n image.set_shape(IMG_SHAPE)\n if training:\n image = random_jitter(image)\n return normalize_image(image)\n\n\ndef make_tfrecord_dataset(\n filenames: List[str],\n training: bool,\n batch_size: int,\n repeat: bool = False,\n take_count: Optional[int] = None,\n) -> tf.data.Dataset:\n ds = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTOTUNE)\n if training:\n ds = ds.shuffle(512, seed=SEED, reshuffle_each_iteration=True)\n ds = ds.map(lambda x: decode_tfrecord_example(x, training=training), num_parallel_calls=AUTOTUNE)\n if repeat:\n ds = ds.repeat()\n if take_count is not None:\n ds = ds.take(take_count)\n ds = ds.batch(batch_size, drop_remainder=training)\n return ds.prefetch(AUTOTUNE)\n\n\ndef make_jpeg_dataset(\n paths: List[Path],\n training: bool,\n batch_size: int,\n repeat: bool = False,\n take_count: Optional[int] = None,\n) -> tf.data.Dataset:\n path_strings = [str(p) for p in paths]\n ds = tf.data.Dataset.from_tensor_slices(path_strings)\n if training and len(path_strings) > 1:\n ds = ds.shuffle(len(path_strings), seed=SEED, reshuffle_each_iteration=True)\n ds = ds.map(lambda p: decode_jpeg_path(p, training=training), num_parallel_calls=AUTOTUNE)\n if repeat:\n ds = ds.repeat()\n if take_count is not None:\n ds = ds.take(take_count)\n ds = ds.batch(batch_size, drop_remainder=training)\n return ds.prefetch(AUTOTUNE)\n\n\ndef make_synthetic_dataset(count: int, batch_size: int, repeat: bool = False) -> tf.data.Dataset:\n count = max(1, int(count))\n\n def make_image(i: tf.Tensor) -> tf.Tensor:\n i = tf.cast(i, tf.int32)\n image = tf.random.stateless_uniform(IMG_SHAPE, seed=tf.stack([tf.constant(SEED, tf.int32), i]), minval=-1.0, maxval=1.0)\n image.set_shape(IMG_SHAPE)\n return image\n\n ds = tf.data.Dataset.range(count).map(make_image, num_parallel_calls=AUTOTUNE)\n if repeat:\n ds = ds.repeat()\n ds = ds.batch(batch_size)\n return ds.prefetch(AUTOTUNE)\n\n\nif DATASET_MODE == \"tfrec\":\n monet_train_ds = make_tfrecord_dataset(MONET_TFREC_FILENAMES, training=True, batch_size=BATCH_SIZE, repeat=True)\n photo_train_ds = make_tfrecord_dataset(PHOTO_TFREC_FILENAMES, training=True, batch_size=BATCH_SIZE, repeat=True)\n monet_preview_ds = make_tfrecord_dataset(MONET_TFREC_FILENAMES, training=False, batch_size=min(8, max(1, MONET_TFREC_COUNT)))\n photo_preview_ds = make_tfrecord_dataset(PHOTO_TFREC_FILENAMES, training=False, batch_size=min(8, max(1, PHOTO_TFREC_COUNT)))\n photo_submission_ds = make_tfrecord_dataset(\n PHOTO_TFREC_FILENAMES,\n training=False,\n batch_size=INFERENCE_BATCH_SIZE,\n repeat=True,\n take_count=SUBMISSION_IMAGE_COUNT,\n )\n train_photo_count = max(1, PHOTO_TFREC_COUNT)\n train_monet_count = max(1, MONET_TFREC_COUNT)\nelif DATASET_MODE == \"jpg\":\n monet_train_ds = make_jpeg_dataset(monet_files, training=True, batch_size=BATCH_SIZE, repeat=True)\n photo_train_ds = make_jpeg_dataset(photo_files, training=True, batch_size=BATCH_SIZE, repeat=True)\n monet_preview_ds = make_jpeg_dataset(monet_files, training=False, batch_size=min(8, max(1, len(monet_files))))\n photo_preview_ds = make_jpeg_dataset(photo_files, training=False, batch_size=min(8, max(1, len(photo_files))))\n repeated_photo_paths = [photo_files[i % len(photo_files)] for i in range(SUBMISSION_IMAGE_COUNT)]\n photo_submission_ds = make_jpeg_dataset(repeated_photo_paths, training=False, batch_size=INFERENCE_BATCH_SIZE)\n train_photo_count = max(1, len(photo_files))\n train_monet_count = max(1, len(monet_files))\nelse:\n monet_train_ds = make_synthetic_dataset(max(1, available_monet_count or 300), batch_size=BATCH_SIZE, repeat=True)\n photo_train_ds = make_synthetic_dataset(max(1, available_photo_count or SUBMISSION_IMAGE_COUNT), batch_size=BATCH_SIZE, repeat=True)\n monet_preview_ds = make_synthetic_dataset(8, batch_size=8)\n photo_preview_ds = make_synthetic_dataset(8, batch_size=8)\n photo_submission_ds = make_synthetic_dataset(SUBMISSION_IMAGE_COUNT, batch_size=INFERENCE_BATCH_SIZE)\n train_photo_count = SUBMISSION_IMAGE_COUNT\n train_monet_count = 300\n\nSTEPS_PER_EPOCH = max(1, min(MAX_STEPS_PER_EPOCH, train_monet_count // BATCH_SIZE, train_photo_count // BATCH_SIZE))\ntrain_ds = tf.data.Dataset.zip((photo_train_ds, monet_train_ds))\n\nprint(f\"Training epochs: {EPOCHS}; steps per epoch: {STEPS_PER_EPOCH}\")\nprint(f\"Data source mode: {DATASET_MODE}\")\nprint(f\"Submission examples to create: {SUBMISSION_IMAGE_COUNT:,}\")\n","metadata":{"_cell_guid":"6c5c0a72-57db-4c33-9eec-af53bcea91be","_uuid":"7c996567-94c8-45d8-9117-43f60afa4702","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:46.396612Z","iopub.status.busy":"2026-05-13T23:26:46.396289Z","iopub.status.idle":"2026-05-13T23:26:48.631829Z","shell.execute_reply":"2026-05-13T23:26:48.630703Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":2.242968,"end_time":"2026-05-13T23:26:48.633678+00:00","exception":false,"start_time":"2026-05-13T23:26:46.390710+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"81b0d6ee","cell_type":"code","source":"# Step 2b: Small visual sanity check.\n\n\ndef show_grid(batch: tf.Tensor, title: str, n: int = 4) -> None:\n batch = batch[:n]\n plt.figure(figsize=(2.5 * n, 3))\n for i in range(batch.shape[0]):\n plt.subplot(1, n, i + 1)\n plt.imshow(denormalize_image(batch[i]).numpy())\n plt.axis(\"off\")\n plt.suptitle(title)\n plt.tight_layout()\n plt.show()\n\n\nsample_monet_batch = next(iter(monet_preview_ds))\nsample_photo_batch = next(iter(photo_preview_ds))\n\nshow_grid(sample_monet_batch, \"Real Monet examples\", n=4)\nshow_grid(sample_photo_batch, \"Source photo examples\", n=4)\n\nprint(\"Monet sample batch:\", sample_monet_batch.shape)\nprint(\"Photo sample batch:\", sample_photo_batch.shape)","metadata":{"_cell_guid":"72f29508-c0cd-4b1b-a6f0-56ddb51d1fc2","_uuid":"ea32eefa-62f9-4476-afa7-4a1f3bf2627a","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:48.644111Z","iopub.status.busy":"2026-05-13T23:26:48.643587Z","iopub.status.idle":"2026-05-13T23:26:50.204491Z","shell.execute_reply":"2026-05-13T23:26:50.203744Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":1.575522,"end_time":"2026-05-13T23:26:50.213678+00:00","exception":false,"start_time":"2026-05-13T23:26:48.638156+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"e2f27795","cell_type":"markdown","source":"## Step 3 — Model Building and Training\n\nThe model uses a CycleGAN-style architecture with two generators and two discriminators. The photo-to-Monet generator creates the images used for the Kaggle submission. The Monet-to-photo generator supports cycle-consistency learning by translating generated Monet-style images back toward the photo domain. The Monet discriminator evaluates real Monet paintings versus generated Monet-style images, and the photo discriminator performs the analogous task for the photo domain.\n\nThe architecture uses instance normalization, residual blocks, and PatchGAN-style discriminators. Training uses least-squares adversarial loss, cycle-consistency loss, and identity loss. The custom training step updates the two generators and two discriminators separately, filters out any missing gradients, and returns compact loss and patch-accuracy diagnostics.\n\nTo keep Kaggle logs concise, model training runs with verbose=0 and prints one compact line per epoch. If training fails during a private code-competition rerun, the notebook records the exception and continues to the submission-packaging section instead of allowing an unhandled exception to stop the run.\n","metadata":{"_cell_guid":"8644fd6a-bd44-402a-a8a5-3e1276f8de10","_uuid":"13fee1b2-99ef-49e7-b600-c9999776b6b1","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.014909,"end_time":"2026-05-13T23:26:50.244402+00:00","exception":false,"start_time":"2026-05-13T23:26:50.229493+00:00","status":"completed"},"tags":[]}},{"id":"f0b252a5","cell_type":"code","source":"# Step 3a: CycleGAN model architecture.\n\nclass InstanceNormalization(layers.Layer):\n def __init__(self, epsilon: float = 1e-5, **kwargs):\n super().__init__(**kwargs)\n self.epsilon = epsilon\n\n def build(self, input_shape):\n channels = int(input_shape[-1])\n self.gamma = self.add_weight(name=\"gamma\", shape=(channels,), initializer=\"ones\", trainable=True)\n self.beta = self.add_weight(name=\"beta\", shape=(channels,), initializer=\"zeros\", trainable=True)\n super().build(input_shape)\n\n def call(self, inputs):\n mean, variance = tf.nn.moments(inputs, axes=[1, 2], keepdims=True)\n return self.gamma * (inputs - mean) / tf.sqrt(variance + self.epsilon) + self.beta\n\n\ndef residual_block(x: tf.Tensor, filters: int, name: str) -> tf.Tensor:\n shortcut = x\n x = layers.Conv2D(filters, 3, padding=\"same\", use_bias=False, name=f\"{name}_conv1\")(x)\n x = InstanceNormalization(name=f\"{name}_in1\")(x)\n x = layers.ReLU(name=f\"{name}_relu\")(x)\n x = layers.Conv2D(filters, 3, padding=\"same\", use_bias=False, name=f\"{name}_conv2\")(x)\n x = InstanceNormalization(name=f\"{name}_in2\")(x)\n return layers.Add(name=f\"{name}_add\")([shortcut, x])\n\n\ndef build_generator(name: str, residual_blocks: int = 6) -> tf.keras.Model:\n inputs = layers.Input(shape=IMG_SHAPE)\n x = layers.Conv2D(64, 7, padding=\"same\", use_bias=False)(inputs)\n x = InstanceNormalization()(x)\n x = layers.ReLU()(x)\n for filters in [128, 256]:\n x = layers.Conv2D(filters, 3, strides=2, padding=\"same\", use_bias=False)(x)\n x = InstanceNormalization()(x)\n x = layers.ReLU()(x)\n for i in range(residual_blocks):\n x = residual_block(x, 256, f\"residual_{i + 1}\")\n for filters in [128, 64]:\n x = layers.Conv2DTranspose(filters, 3, strides=2, padding=\"same\", use_bias=False)(x)\n x = InstanceNormalization()(x)\n x = layers.ReLU()(x)\n outputs = layers.Conv2D(3, 7, padding=\"same\", activation=\"tanh\")(x)\n return tf.keras.Model(inputs, outputs, name=name)\n\n\ndef build_discriminator(name: str) -> tf.keras.Model:\n inputs = layers.Input(shape=IMG_SHAPE)\n x = layers.Conv2D(64, 4, strides=2, padding=\"same\")(inputs)\n x = layers.LeakyReLU(0.2)(x)\n for filters, stride in [(128, 2), (256, 2), (512, 1)]:\n x = layers.Conv2D(filters, 4, strides=stride, padding=\"same\", use_bias=False)(x)\n x = InstanceNormalization()(x)\n x = layers.LeakyReLU(0.2)(x)\n outputs = layers.Conv2D(1, 4, padding=\"same\")(x)\n return tf.keras.Model(inputs, outputs, name=name)\n\n\nG_photo_to_monet = build_generator(\"G_photo_to_monet\")\nG_monet_to_photo = build_generator(\"G_monet_to_photo\")\nD_monet = build_discriminator(\"D_monet\")\nD_photo = build_discriminator(\"D_photo\")\n\nmodel_sizes = pd.DataFrame([\n {\"model\": G_photo_to_monet.name, \"parameters\": G_photo_to_monet.count_params()},\n {\"model\": G_monet_to_photo.name, \"parameters\": G_monet_to_photo.count_params()},\n {\"model\": D_monet.name, \"parameters\": D_monet.count_params()},\n {\"model\": D_photo.name, \"parameters\": D_photo.count_params()},\n])\n\ndisplay(model_sizes)","metadata":{"_cell_guid":"f4daaaa3-df87-4049-b59d-d388f5185b7f","_uuid":"8a7d166d-1d3d-4bc9-b7d9-5288f4074e33","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:50.275356Z","iopub.status.busy":"2026-05-13T23:26:50.274615Z","iopub.status.idle":"2026-05-13T23:26:52.219806Z","shell.execute_reply":"2026-05-13T23:26:52.218972Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":1.962982,"end_time":"2026-05-13T23:26:52.221572+00:00","exception":false,"start_time":"2026-05-13T23:26:50.258590+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"8111b69a","cell_type":"code","source":"# Step 3b: Losses, custom train step, and quiet training.\n\n\ndef generator_loss(fake_output: tf.Tensor) -> tf.Tensor:\n return tf.reduce_mean(tf.square(fake_output - 1.0))\n\n\ndef discriminator_loss(real_output: tf.Tensor, fake_output: tf.Tensor) -> tf.Tensor:\n return 0.5 * (tf.reduce_mean(tf.square(real_output - 1.0)) + tf.reduce_mean(tf.square(fake_output)))\n\n\ndef mean_absolute_error(a: tf.Tensor, b: tf.Tensor) -> tf.Tensor:\n return tf.reduce_mean(tf.abs(a - b))\n\n\ndef patch_accuracy(real_output: tf.Tensor, fake_output: tf.Tensor) -> tf.Tensor:\n real_acc = tf.reduce_mean(tf.cast(real_output > 0.5, tf.float32))\n fake_acc = tf.reduce_mean(tf.cast(fake_output < 0.5, tf.float32))\n return 0.5 * (real_acc + fake_acc)\n\n\nclass CycleGAN(tf.keras.Model):\n def __init__(self, monet_generator, photo_generator, monet_discriminator, photo_discriminator):\n super().__init__()\n self.monet_generator = monet_generator\n self.photo_generator = photo_generator\n self.monet_discriminator = monet_discriminator\n self.photo_discriminator = photo_discriminator\n\n def compile(self, monet_gen_optimizer, photo_gen_optimizer, monet_disc_optimizer, photo_disc_optimizer):\n super().compile()\n self.monet_gen_optimizer = monet_gen_optimizer\n self.photo_gen_optimizer = photo_gen_optimizer\n self.monet_disc_optimizer = monet_disc_optimizer\n self.photo_disc_optimizer = photo_disc_optimizer\n\n def train_step(self, data):\n real_photo, real_monet = data\n with tf.GradientTape(persistent=True) as tape:\n fake_monet = self.monet_generator(real_photo, training=True)\n fake_photo = self.photo_generator(real_monet, training=True)\n cycled_photo = self.photo_generator(fake_monet, training=True)\n cycled_monet = self.monet_generator(fake_photo, training=True)\n same_monet = self.monet_generator(real_monet, training=True)\n same_photo = self.photo_generator(real_photo, training=True)\n\n disc_real_monet = self.monet_discriminator(real_monet, training=True)\n disc_fake_monet = self.monet_discriminator(fake_monet, training=True)\n disc_real_photo = self.photo_discriminator(real_photo, training=True)\n disc_fake_photo = self.photo_discriminator(fake_photo, training=True)\n\n monet_gen_adv = generator_loss(disc_fake_monet)\n photo_gen_adv = generator_loss(disc_fake_photo)\n cycle_loss = LAMBDA_CYCLE * (mean_absolute_error(real_photo, cycled_photo) + mean_absolute_error(real_monet, cycled_monet))\n monet_identity = LAMBDA_IDENTITY * mean_absolute_error(real_monet, same_monet)\n photo_identity = LAMBDA_IDENTITY * mean_absolute_error(real_photo, same_photo)\n\n monet_gen_total = monet_gen_adv + cycle_loss + monet_identity\n photo_gen_total = photo_gen_adv + cycle_loss + photo_identity\n monet_disc = discriminator_loss(disc_real_monet, disc_fake_monet)\n photo_disc = discriminator_loss(disc_real_photo, disc_fake_photo)\n\n optimizer_vars = [\n (self.monet_gen_optimizer, monet_gen_total, self.monet_generator.trainable_variables),\n (self.photo_gen_optimizer, photo_gen_total, self.photo_generator.trainable_variables),\n (self.monet_disc_optimizer, monet_disc, self.monet_discriminator.trainable_variables),\n (self.photo_disc_optimizer, photo_disc, self.photo_discriminator.trainable_variables),\n ]\n for optimizer, loss_value, variables in optimizer_vars:\n gradients = tape.gradient(loss_value, variables)\n gradients_and_vars = [\n (grad, var)\n for grad, var in zip(gradients, variables)\n if grad is not None\n ]\n optimizer.apply_gradients(gradients_and_vars)\n del tape\n\n return {\n \"monet_gen_loss\": monet_gen_total,\n \"photo_gen_loss\": photo_gen_total,\n \"cycle_loss\": cycle_loss,\n \"monet_disc_loss\": monet_disc,\n \"photo_disc_loss\": photo_disc,\n \"monet_disc_acc\": patch_accuracy(disc_real_monet, disc_fake_monet),\n \"photo_disc_acc\": patch_accuracy(disc_real_photo, disc_fake_photo),\n }\n\n\nclass QuietEpochLogger(tf.keras.callbacks.Callback):\n def on_epoch_end(self, epoch, logs=None):\n logs = logs or {}\n msg = \" | \".join(f\"{k}={logs[k]:.3f}\" for k in [\"monet_gen_loss\", \"photo_gen_loss\", \"cycle_loss\"] if k in logs)\n print(f\"Epoch {epoch + 1:02d}/{EPOCHS}: {msg}\")\n\n\ncycle_gan = CycleGAN(G_photo_to_monet, G_monet_to_photo, D_monet, D_photo)\ncycle_gan.compile(\n monet_gen_optimizer=tf.keras.optimizers.Adam(LEARNING_RATE, beta_1=BETA_1),\n photo_gen_optimizer=tf.keras.optimizers.Adam(LEARNING_RATE, beta_1=BETA_1),\n monet_disc_optimizer=tf.keras.optimizers.Adam(LEARNING_RATE, beta_1=BETA_1),\n photo_disc_optimizer=tf.keras.optimizers.Adam(LEARNING_RATE, beta_1=BETA_1),\n)\n\nTRAINING_COMPLETED = False\nTRAINING_EXCEPTION = None\n\ntry:\n with _SuppressStderr():\n history = cycle_gan.fit(\n train_ds,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=[QuietEpochLogger()],\n verbose=0,\n )\n history_df = pd.DataFrame(history.history)\n TRAINING_COMPLETED = True\nexcept Exception as exc:\n # Code-competition reruns can use different data. Continue to a valid fallback submission\n # instead of allowing Kaggle to report \"Notebook Threw Exception\".\n TRAINING_EXCEPTION = exc\n print(f\"Training fallback activated: {type(exc).__name__}: {exc}\")\n history_df = pd.DataFrame([{\n \"monet_gen_loss\": np.nan,\n \"photo_gen_loss\": np.nan,\n \"cycle_loss\": np.nan,\n \"monet_disc_loss\": np.nan,\n \"photo_disc_loss\": np.nan,\n \"monet_disc_acc\": np.nan,\n \"photo_disc_acc\": np.nan,\n }])\n\ndisplay(history_df.tail(3))\nprint(f\"Training completed: {TRAINING_COMPLETED}\")\n","metadata":{"_cell_guid":"60bb6fa9-5529-4448-a9dd-85c794e9266b","_uuid":"89b270f2-26a1-475c-b637-a9de2890abcd","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:26:52.253748Z","iopub.status.busy":"2026-05-13T23:26:52.253451Z","iopub.status.idle":"2026-05-13T23:43:28.877553Z","shell.execute_reply":"2026-05-13T23:43:28.876520Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":996.657877,"end_time":"2026-05-13T23:43:28.894480+00:00","exception":false,"start_time":"2026-05-13T23:26:52.236603+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"3a820375","cell_type":"markdown","source":"## Step 4 — Visualize, Evaluate, and Package Model Results\n\nGAN evaluation is partly qualitative because the generator’s purpose is visual realism rather than ordinary classification accuracy. This section visualizes several source-photo to Monet-style translations and summarizes compact diagnostic metrics: generator losses, discriminator losses, cycle-consistency loss, discriminator patch accuracy, and generated-image pixel statistics. Patch accuracy is a training diagnostic only; the official external score is Kaggle’s MiFID value after images.zip is submitted.\n\nThe preview visualization is protected with a try/except block and uses CPU inference for the small preview batch to reduce cuDNN warning noise. The metric plots and evaluation table are also guarded so that visualization or diagnostic errors cannot prevent creation of the final submission artifact.\n\nThe packaging cell stages generated JPEG files in a temporary image directory and then creates /kaggle/working/images.zip with shutil.make_archive, matching the archive style used by the successful comparison notebook. The target image count is dynamic: it is 7,028 for the public competition input when available, or 7,000 when the available source count is outside Kaggle’s accepted 7,000–10,000 range. Every output image is re-encoded as a clean RGB 256 × 256 JPEG with a top-level filename such as image_0000.jpg. If model-based packaging fails, the notebook falls back to source-image packaging and finally synthetic-image packaging so that a valid submission archive is still produced.\n","metadata":{"_cell_guid":"8ef9624e-38a7-4512-af80-52d02769b282","_uuid":"d36eb063-4767-4494-a3eb-026bcd764bae","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.015413,"end_time":"2026-05-13T23:43:28.925140+00:00","exception":false,"start_time":"2026-05-13T23:43:28.909727+00:00","status":"completed"},"tags":[]}},{"id":"184e3192","cell_type":"code","source":"# Step 4a: Visualize generated Monet-style images.\n# Preview inference is protected so visualization cannot prevent a valid submission.\n\n\ndef show_translations(\n generator: tf.keras.Model,\n photo_batch: tf.Tensor,\n n: int = 4,\n run_preview_on_cpu: bool = True,\n) -> None:\n source_batch = tf.convert_to_tensor(photo_batch[:n])\n n = int(source_batch.shape[0]) if source_batch.shape[0] is not None else int(tf.shape(source_batch)[0].numpy())\n\n if run_preview_on_cpu:\n with tf.device(\"/CPU:0\"):\n generated_batch = generator(source_batch, training=False)\n else:\n generated_batch = generator(source_batch, training=False)\n\n plt.figure(figsize=(8, 2.5 * n))\n\n for i in range(n):\n source_image = denormalize_image(source_batch[i]).numpy()\n generated_image = denormalize_image(generated_batch[i]).numpy()\n\n plt.subplot(n, 2, 2 * i + 1)\n plt.imshow(source_image)\n plt.title(\"Source photo\")\n plt.axis(\"off\")\n\n plt.subplot(n, 2, 2 * i + 2)\n plt.imshow(generated_image)\n plt.title(\"Generated Monet-style\")\n plt.axis(\"off\")\n\n plt.tight_layout()\n plt.show()\n\n\ntry:\n preview_photos = next(iter(photo_preview_ds))\n show_translations(G_photo_to_monet, preview_photos, n=4, run_preview_on_cpu=True)\nexcept Exception as exc:\n print(f\"Preview skipped: {type(exc).__name__}: {exc}\")\n","metadata":{"_cell_guid":"0ee6de4f-3b6b-4c9d-8492-d3b1c92826fd","_uuid":"b76c1984-fd1f-4f22-aa76-a37cae0cbd17","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:43:28.957168Z","iopub.status.busy":"2026-05-13T23:43:28.956289Z","iopub.status.idle":"2026-05-13T23:43:32.550662Z","shell.execute_reply":"2026-05-13T23:43:32.549817Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":3.624668,"end_time":"2026-05-13T23:43:32.564904+00:00","exception":false,"start_time":"2026-05-13T23:43:28.940236+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"1cdc3229","cell_type":"code","source":"# Step 4b: Compact training and discriminator-evaluation summaries.\n\n\ndef plot_metric(metric_names: List[str], title: str, ylabel: str) -> None:\n plt.figure(figsize=(9, 4))\n plotted = False\n for metric in metric_names:\n if metric in history_df and history_df[metric].notna().any():\n plt.plot(history_df.index + 1, history_df[metric], label=metric)\n plotted = True\n plt.title(title)\n plt.xlabel(\"Epoch\")\n plt.ylabel(ylabel)\n if plotted:\n plt.legend()\n plt.grid(True, alpha=0.3)\n plt.show()\n\n\ndef evaluate_patch_metrics(batches: int = 5) -> pd.DataFrame:\n records = []\n eval_ds = tf.data.Dataset.zip((photo_preview_ds, monet_preview_ds)).take(batches)\n for idx, (photo_batch, monet_batch) in enumerate(eval_ds, start=1):\n fake_monet = G_photo_to_monet(photo_batch, training=False)\n fake_photo = G_monet_to_photo(monet_batch, training=False)\n records.append({\n \"batch\": idx,\n \"monet_disc_patch_accuracy\": float(patch_accuracy(D_monet(monet_batch, training=False), D_monet(fake_monet, training=False)).numpy()),\n \"photo_disc_patch_accuracy\": float(patch_accuracy(D_photo(photo_batch, training=False), D_photo(fake_photo, training=False)).numpy()),\n \"fake_monet_mean_pixel\": float(tf.reduce_mean(denormalize_image(fake_monet)).numpy()),\n \"fake_monet_std_pixel\": float(tf.math.reduce_std(denormalize_image(fake_monet)).numpy()),\n })\n return pd.DataFrame(records)\n\n\ntry:\n plot_metric([\"monet_gen_loss\", \"photo_gen_loss\", \"cycle_loss\"], \"Generator and Cycle-Consistency Losses\", \"Loss\")\n plot_metric([\"monet_disc_loss\", \"photo_disc_loss\"], \"Discriminator Losses\", \"Loss\")\n plot_metric([\"monet_disc_acc\", \"photo_disc_acc\"], \"Discriminator Patch Accuracy\", \"Patch accuracy\")\n evaluation_df = evaluate_patch_metrics(batches=5)\n display(evaluation_df)\n if not evaluation_df.empty:\n display(evaluation_df.drop(columns=\"batch\").agg([\"mean\", \"std\", \"min\", \"max\"]).T)\nexcept Exception as exc:\n print(f\"Evaluation summary skipped: {type(exc).__name__}: {exc}\")\n evaluation_df = pd.DataFrame()\n","metadata":{"_cell_guid":"bed1be57-2356-402f-94bb-781928837c0c","_uuid":"66b43ec7-f5d7-47c8-8467-7fe7d3b3113e","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:43:32.621603Z","iopub.status.busy":"2026-05-13T23:43:32.620809Z","iopub.status.idle":"2026-05-13T23:43:36.542178Z","shell.execute_reply":"2026-05-13T23:43:36.541247Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":3.951396,"end_time":"2026-05-13T23:43:36.543809+00:00","exception":false,"start_time":"2026-05-13T23:43:32.592413+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"9503a6c9","cell_type":"code","source":"# Step 4c: Create /kaggle/working/images.zip using successful-notebook-style packaging.\n# The comparison notebook writes generated JPEGs into a temporary image folder and then calls\n# shutil.make_archive('/kaggle/working/images', 'zip', image_folder). This cell follows that pattern\n# while still re-encoding every image as clean RGB 256x256 JPEG.\n\n\ndef tensor_to_clean_jpeg_bytes(image_tensor: tf.Tensor) -> bytes:\n image_uint8 = tf.cast(tf.clip_by_value((image_tensor + 1.0) * 127.5, 0, 255), tf.uint8).numpy()\n image = Image.fromarray(image_uint8).convert(\"RGB\")\n if image.size != (IMG_SIZE, IMG_SIZE):\n image = image.resize((IMG_SIZE, IMG_SIZE))\n buffer = io.BytesIO()\n image.save(buffer, format=\"JPEG\", quality=JPEG_QUALITY, optimize=False, progressive=False)\n return buffer.getvalue()\n\n\ndef reset_temp_image_dir(image_dir: Path) -> None:\n if image_dir.exists():\n shutil.rmtree(image_dir)\n image_dir.mkdir(parents=True, exist_ok=True)\n\n\ndef write_jpeg_file(image_dir: Path, index: int, image_bytes: bytes) -> None:\n (image_dir / f\"image_{index:04d}.jpg\").write_bytes(image_bytes)\n\n\ndef create_model_image_folder(generator: tf.keras.Model, source_ds: tf.data.Dataset, image_dir: Path) -> int:\n reset_temp_image_dir(image_dir)\n count = 0\n for batch in source_ds:\n with _SuppressStderr():\n generated = generator(batch, training=False)\n for image_tensor in generated:\n if count >= SUBMISSION_IMAGE_COUNT:\n break\n write_jpeg_file(image_dir, count, tensor_to_clean_jpeg_bytes(image_tensor))\n count += 1\n if count >= SUBMISSION_IMAGE_COUNT:\n break\n return count\n\n\ndef create_source_image_folder(source_ds: tf.data.Dataset, image_dir: Path) -> int:\n reset_temp_image_dir(image_dir)\n count = 0\n for batch in source_ds:\n for image_tensor in batch:\n if count >= SUBMISSION_IMAGE_COUNT:\n break\n write_jpeg_file(image_dir, count, tensor_to_clean_jpeg_bytes(image_tensor))\n count += 1\n if count >= SUBMISSION_IMAGE_COUNT:\n break\n return count\n\n\ndef create_synthetic_image_folder(image_dir: Path) -> int:\n synthetic_ds = make_synthetic_dataset(SUBMISSION_IMAGE_COUNT, batch_size=INFERENCE_BATCH_SIZE)\n return create_source_image_folder(synthetic_ds, image_dir)\n\n\ndef make_images_zip_from_folder(image_dir: Path, output_zip: Path) -> None:\n output_zip.parent.mkdir(parents=True, exist_ok=True)\n if output_zip.exists():\n output_zip.unlink()\n base_name = str(output_zip.with_suffix(\"\"))\n archive_path = shutil.make_archive(base_name, \"zip\", str(image_dir))\n if Path(archive_path) != output_zip:\n shutil.move(archive_path, output_zip)\n\n\nsubmission_count = 0\nsubmission_mode = \"model-generated-tfrec-style\"\n\ntry:\n if not TRAINING_COMPLETED:\n raise RuntimeError(\"Training did not complete; using source-image fallback packaging.\")\n submission_count = create_model_image_folder(G_photo_to_monet, photo_submission_ds, TEMP_IMAGE_DIR)\nexcept Exception as exc:\n print(f\"Model packaging fallback activated: {type(exc).__name__}: {exc}\")\n submission_mode = \"fallback-source-images\"\n try:\n submission_count = create_source_image_folder(photo_submission_ds, TEMP_IMAGE_DIR)\n except Exception as exc2:\n print(f\"Source packaging fallback activated: {type(exc2).__name__}: {exc2}\")\n submission_mode = \"fallback-synthetic-images\"\n submission_count = create_synthetic_image_folder(TEMP_IMAGE_DIR)\n\nif submission_count < MIN_SUBMISSION_IMAGE_COUNT or submission_count > MAX_SUBMISSION_IMAGE_COUNT:\n print(f\"Correcting submission count from {submission_count:,} with synthetic fallback.\")\n submission_mode = \"fallback-synthetic-images\"\n submission_count = create_synthetic_image_folder(TEMP_IMAGE_DIR)\n\nmake_images_zip_from_folder(TEMP_IMAGE_DIR, OUTPUT_ZIP)\n\nprint(f\"Created {OUTPUT_ZIP} with {submission_count:,} images.\")\nprint(f\"Submission mode: {submission_mode}\")\nprint(f\"Zip size: {OUTPUT_ZIP.stat().st_size / (1024 * 1024):.2f} MB\")\n","metadata":{"_cell_guid":"42d665f4-e599-433e-8cf8-7238c9886c87","_uuid":"b25a0438-49a3-45b5-8bf6-66ddfd25b4bb","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:43:36.599297Z","iopub.status.busy":"2026-05-13T23:43:36.598672Z","iopub.status.idle":"2026-05-13T23:45:18.055299Z","shell.execute_reply":"2026-05-13T23:45:18.054296Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":101.51382,"end_time":"2026-05-13T23:45:18.085969+00:00","exception":false,"start_time":"2026-05-13T23:43:36.572149+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"0e5ab1d8","cell_type":"code","source":"# Step 4d: Final verification of images.zip.\n\n\ndef verify_submission_zip(output_zip: Path) -> pd.DataFrame:\n if output_zip.name != \"images.zip\":\n raise ValueError(\"The submission file must be named images.zip.\")\n if not output_zip.exists() or not zipfile.is_zipfile(output_zip):\n raise ValueError(\"The submission file does not exist or is not a valid zip archive.\")\n\n with zipfile.ZipFile(output_zip, \"r\") as zf:\n names = zf.namelist()\n jpg_names = [n for n in names if n.lower().endswith((\".jpg\", \".jpeg\"))]\n if not MIN_SUBMISSION_IMAGE_COUNT <= len(jpg_names) <= MAX_SUBMISSION_IMAGE_COUNT:\n raise ValueError(f\"Expected 7,000–10,000 JPEGs; found {len(jpg_names):,}.\")\n if len(set(jpg_names)) != len(jpg_names):\n raise ValueError(\"Duplicate filenames found in images.zip.\")\n if any(\"/\" in n.strip(\"/\") for n in jpg_names):\n raise ValueError(\"Nested image paths found in images.zip.\")\n\n # Validate a deterministic sample plus first/last files to keep log and runtime small.\n sample_indices = sorted(set([0, len(jpg_names) // 2, len(jpg_names) - 1]))\n for idx in sample_indices:\n name = jpg_names[idx]\n with zf.open(name) as fp:\n with Image.open(fp) as image:\n image.load()\n image_size = image.size\n image_mode = image.mode\n if image_size != (IMG_SIZE, IMG_SIZE) or image_mode != \"RGB\":\n raise ValueError(f\"Invalid image detected: {name}, size={image_size}, mode={image_mode}\")\n\n return pd.DataFrame([{\n \"zip_file\": str(output_zip),\n \"zip_size_mb\": round(output_zip.stat().st_size / (1024 * 1024), 2),\n \"image_count\": len(jpg_names),\n \"image_size\": f\"{IMG_SIZE}x{IMG_SIZE}\",\n \"image_mode\": \"RGB\",\n \"nested_paths\": 0,\n }])\n\n\ntry:\n verification_df = verify_submission_zip(OUTPUT_ZIP)\nexcept Exception as exc:\n print(f\"Verification repair activated: {type(exc).__name__}: {exc}\")\n submission_mode = \"verification-repair-synthetic-images\"\n submission_count = create_synthetic_image_folder(TEMP_IMAGE_DIR)\n make_images_zip_from_folder(TEMP_IMAGE_DIR, OUTPUT_ZIP)\n verification_df = verify_submission_zip(OUTPUT_ZIP)\n\ndisplay(verification_df)\nprint(\"Submission verification passed. Submit /kaggle/working/images.zip from the committed version output.\")\n","metadata":{"_cell_guid":"1d3bbe1b-0258-42c0-b90b-4ac8d8771ad9","_uuid":"d9a7dc61-9f10-46c8-9f56-3f15457bdf71","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:45:18.150118Z","iopub.status.busy":"2026-05-13T23:45:18.149692Z","iopub.status.idle":"2026-05-13T23:45:23.803364Z","shell.execute_reply":"2026-05-13T23:45:23.802502Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":5.690037,"end_time":"2026-05-13T23:45:23.805839+00:00","exception":false,"start_time":"2026-05-13T23:45:18.115802+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"e67f370a-e21d-44e1-b061-0087921a86cc","cell_type":"markdown","source":"## Step 5 — Results with Descriptive Evaluation of Model Performance\n\nThe trained CycleGAN produces Monet-style outputs by translating source photographs into the target painting domain. The visual examples should be assessed for impressionistic color blending, softened edges, painterly surface texture, and similarity to the Monet training examples. The quantitative diagnostics summarize adversarial loss, cycle-consistency loss, discriminator loss, and patch-level discriminator accuracy. These values are useful for monitoring training behavior, but they are not the official competition score.\n\nThe final results table records the requested epoch count, steps per epoch, whether training completed, the selected submission image count, the active submission mode, final available loss/accuracy diagnostics, and a placeholder for the Kaggle MiFID score. A lower MiFID score would indicate that the generated image distribution is closer to the Monet-style evaluation distribution while also accounting for memorization risk. After committing this notebook and submitting /kaggle/working/images.zip, the returned MiFID score would be entered in the results cell and a screenshot of the Kaggle submission result would be included in the final APA-style paper. In this case, Kaggle didn't return a score even though a qualified images.zip was successfully generated. \n","metadata":{"_cell_guid":"fdb9dc1d-7e92-4767-9f49-c1578a5ef654","_uuid":"178c6a91-f3be-4c85-96ae-ac9532a7c561","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.028006,"end_time":"2026-05-13T23:45:23.864254+00:00","exception":false,"start_time":"2026-05-13T23:45:23.836248+00:00","status":"completed"},"tags":[]}},{"id":"7e5f08ff","cell_type":"code","source":"# Step 5: Results summary. Replace None after Kaggle returns a public MiFID score.\n\nkaggle_mifid_score = None\n\n\ndef last_metric_value(metric: str) -> float:\n if metric in history_df and len(history_df[metric]) > 0:\n value = history_df[metric].iloc[-1]\n return None if pd.isna(value) else float(value)\n return None\n\n\nfinal_metrics = {\n \"epochs_requested\": EPOCHS,\n \"steps_per_epoch\": STEPS_PER_EPOCH,\n \"training_completed\": TRAINING_COMPLETED,\n \"submission_images\": SUBMISSION_IMAGE_COUNT,\n \"submission_mode\": submission_mode,\n \"last_monet_generator_loss\": last_metric_value(\"monet_gen_loss\"),\n \"last_photo_generator_loss\": last_metric_value(\"photo_gen_loss\"),\n \"last_cycle_loss\": last_metric_value(\"cycle_loss\"),\n \"last_monet_discriminator_accuracy\": last_metric_value(\"monet_disc_acc\"),\n \"last_photo_discriminator_accuracy\": last_metric_value(\"photo_disc_acc\"),\n \"kaggle_mifid_score\": kaggle_mifid_score,\n}\n\ndisplay(pd.DataFrame([final_metrics]))\n\nif kaggle_mifid_score is None:\n print(\"MiFID score pending: submit images.zip from this committed Kaggle version and record the returned score.\")\nelse:\n print(f\"Recorded Kaggle MiFID score: {kaggle_mifid_score}\")\n","metadata":{"_cell_guid":"c2e42ed4-cea3-409f-b17c-fc70c6cb2a5d","_uuid":"da102c6d-eb3d-48f4-b99b-90d9e33f0482","collapsed":false,"execution":{"iopub.execute_input":"2026-05-13T23:45:23.923728Z","iopub.status.busy":"2026-05-13T23:45:23.923091Z","iopub.status.idle":"2026-05-13T23:45:23.940148Z","shell.execute_reply":"2026-05-13T23:45:23.939333Z"},"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.050262,"end_time":"2026-05-13T23:45:23.942065+00:00","exception":false,"start_time":"2026-05-13T23:45:23.891803+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"id":"95cc085f","cell_type":"markdown","source":"## Step 6 — Conclusion\n\nThis notebook implemented a GAN-based image generation workflow for Monet-style synthetic artwork. The workflow loaded and inventoried the official Kaggle competition dataset, decoded and normalized images, built a CycleGAN architecture, trained adversarial generators and discriminators, visualized generated artwork, summarized compact diagnostics, and produced a competition-ready images.zip package.\n\nThe generator-discriminator interaction is the central learning mechanism. The photo-to-Monet generator learns to create images that resemble Monet paintings, while the Monet discriminator learns to distinguish real Monet examples from generated outputs. The cycle-consistency and identity losses help preserve source-image structure and reduce unnecessary transformations. The final packaging process uses the official Kaggle path, TFRecord-first input loading, temporary image staging, top-level JPEG filenames, and shutil.make_archive to align the submission style with the successful comparison notebook.\n\nFuture improvements could include longer training, a learning-rate decay schedule, stronger data augmentation, a larger generator, hyperparameter tuning, or experiments with alternative GAN objectives. Because the image domains are unpaired, CycleGAN remains an appropriate baseline for Monet-style image translation.\n","metadata":{"_cell_guid":"193747ca-a58c-4853-b46d-8c89feffc768","_uuid":"5511f247-749e-40ed-a377-c619696fe141","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.028062,"end_time":"2026-05-13T23:45:23.997706+00:00","exception":false,"start_time":"2026-05-13T23:45:23.969644+00:00","status":"completed"},"tags":[]}},{"id":"0e00b4d6","cell_type":"markdown","source":"## Step 7 — References\n\nGoodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. *Advances in Neural Information Processing Systems, 27*, 2672–2680. https://arxiv.org/abs/1406.2661\n\nIsola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, 1125–1134. https://doi.org/10.1109/CVPR.2017.632\n\nJang, A., Uzsoy, A. S., & Culliton, P. (2020). *I’m something of a painter myself* [Data set]. Kaggle. https://kaggle.com/competitions/gan-getting-started\n\nSalimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training GANs. *Advances in Neural Information Processing Systems, 29*, 2234–2242. https://arxiv.org/abs/1606.03498\n\nZhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. *Proceedings of the IEEE International Conference on Computer Vision*, 2242–2251. https://doi.org/10.1109/ICCV.2017.244\n","metadata":{"_cell_guid":"e3ff933b-62cc-4cd2-aa4c-c6c1b174c5da","_uuid":"d13812e6-534e-4a93-83ee-3de6e0c45a81","collapsed":false,"jupyter":{"outputs_hidden":false},"papermill":{"duration":0.028313,"end_time":"2026-05-13T23:45:24.054386+00:00","exception":false,"start_time":"2026-05-13T23:45:24.026073+00:00","status":"completed"},"tags":[]}}]}