{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":19989,"databundleVersionId":1160143,"isSourceIdPinned":false,"sourceType":"competition"},{"sourceId":278819654,"sourceType":"kernelVersion"},{"sourceId":628693,"sourceType":"modelInstanceVersion","isSourceIdPinned":false,"modelInstanceId":468144,"modelId":483974}],"dockerImageVersionId":31192,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🌾 **Global Wheat Detection: KerasCV YOLOv8 Inference**\n\n## I. Overview\nThis notebook outlines the **Inference Pipeline** for the Global Wheat Detection competition. It utilizes a fine-tuned **YOLOv8** model built using the **KerasCV** library. The goal is to accurately localize wheat heads in agricultural images, even under varying lighting conditions and growth stages.\n\nTo maximize predictive performance and robustness, this pipeline incorporates **Test Time Augmentation (TTA)** and **Weighted Boxes Fusion (WBF)**. TTA generates predictions on multiple augmented versions of the same image (e.g., flips), and WBF consolidates these predictions into a single, more accurate set of bounding boxes.\n\n## II. Training Methodology (Recap)\nThe model used in this inference pipeline was trained using a **Stratified Optimization Strategy** in a separate notebook. Key highlights of the training process include:\n* **Architecture:** KerasCV YOLOv8 Backbones.\n* **Phased Training:** 1.  **Warmup:** Stabilizing weights with a gradual learning rate ramp-up.\n    2.  **Mid-Tune:** Primary training phase with cosine decay.\n    3.  **Fine-Tune:** Low learning rate refinement for maximum precision.\n\n## III. Project Resources\n* **Training Notebook:** [🌾 KerasCV YOLOv8 Fine-Tuning](https://www.kaggle.com/code/amirmohamadaskari/wheat-detection-yolov8-fine-tuning) - *See how this model was trained.*\n* **Trained Model:** [Wheat Detection YOLOv8 Model](https://www.kaggle.com/models/amirmohamadaskari/wheat-detection) - *Download or use the fine-tuned weights.*\n* **Dataset:** [Global Wheat Detection Data](https://www.kaggle.com/competitions/global-wheat-detection/data) - *Competition dataset.*\n\n---\n\n##  IV. Pipeline Summary\n1.  **Environment Setup:** Loading offline utility libraries (e.g., `ensemble_boxes`).\n2.  **Hardware Initialization:** Detecting TPUs/GPUs and setting mixed-precision policies.\n3.  **Model Loading:** Importing the saved KerasCV model with custom objects.\n4.  **Inference Utilities:** Defining functions for TTA and Box Fusion.\n5.  **Prediction Loop:** Iterating through test images and generating submissions.","metadata":{}},{"cell_type":"markdown","source":"## **Section 1. Environment Setup & Library Imports**\n\nIn this section, we import the necessary core libraries for image processing (`cv2`), numerical operations (`numpy`), and deep learning (`tensorflow`, `keras_cv`).\n\n**Note on Offline Inference:**\nSince internet access is often disabled during Kaggle code competitions, external libraries like `ensemble_boxes` (used for WBF) must be loaded from local datasets.\n\n---","metadata":{}},{"cell_type":"markdown","source":"### **1.1 Importing Required Libraries**\nWe begin by importing all necessary packages used throughout the inference notebook:\n\n- **TensorFlow / KerasCV** — model loading & inference  \n- **OpenCV** — image pre-processing  \n- **NumPy / Pandas** — data manipulation  \n- **Matplotlib** — visualization  \n- **ensemble_boxes** — Weighted Boxes Fusion for merging multiple TTA predictions  \n\nTensorFlow logging is suppressed so the notebook output stays clean.","metadata":{}},{"cell_type":"code","source":"import ast\nimport cv2\nimport keras_cv\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport os\nimport pandas as pd\nimport sys\nimport tensorflow as tf\n# Suppress TensorFlow warning messages for cleaner output\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:28:51.515614Z","iopub.execute_input":"2025-11-16T18:28:51.516275Z","iopub.status.idle":"2025-11-16T18:28:51.519990Z","shell.execute_reply.started":"2025-11-16T18:28:51.516250Z","shell.execute_reply":"2025-11-16T18:28:51.519268Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **1.2 Listing Available Compute Devices**\nWe print all detected logical devices (CPU / GPU / TPU).  \nThis helps verify what kind of accelerator is provided by the Kaggle environment.","metadata":{}},{"cell_type":"code","source":"print(\"Available devices: \\n\")\nfor device in tf.config.list_logical_devices():\n    print(device.name, device.device_type)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:32:22.297296Z","iopub.execute_input":"2025-11-16T18:32:22.297876Z","iopub.status.idle":"2025-11-16T18:32:22.536809Z","shell.execute_reply.started":"2025-11-16T18:32:22.297853Z","shell.execute_reply":"2025-11-16T18:32:22.535974Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **1.3 Selecting the Optimal Distribution Strategy**\nA helper function automatically selects the best TensorFlow strategy:\n\n1. **TPU** (fastest for object detection on Kaggle)  \n2. **GPU** (MirroredStrategy for multi-GPU)  \n3. **CPU** as fallback  \n\nThe chosen strategy ensures performance and compatibility when running the model.","metadata":{}},{"cell_type":"code","source":"def get_strategy():\n    \"\"\"\n    Detects and returns the best TensorFlow distribution strategy.\n    - TPUStrategy for TPU(s)\n    - MirroredStrategy for GPU(s)\n    - Default strategy for CPU\n    \"\"\"\n    try:\n        # Try TPU first\n        tpu = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='local')\n        tf.config.experimental_connect_to_cluster(tpu)\n        tf.tpu.experimental.initialize_tpu_system(tpu)\n        strategy = tf.distribute.TPUStrategy(tpu)\n        print(\"Using TPU strategy:\", type(strategy).__name__)\n    except Exception:\n        # If TPU not available, try GPU\n        gpus = tf.config.list_physical_devices('GPU')\n        if gpus:\n            strategy = tf.distribute.MirroredStrategy()\n            print(\"Using GPU strategy:\", type(strategy).__name__)\n        else:\n            # Fallback CPU\n            strategy = tf.distribute.get_strategy()\n            print(\"No TPU/GPU found. Using CPU strategy:\", type(strategy).__name__)\n\n    print(\"REPLICAS:\", strategy.num_replicas_in_sync)\n    return strategy\n\n# Call it\nstrategy = get_strategy()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:32:27.241156Z","iopub.execute_input":"2025-11-16T18:32:27.241893Z","iopub.status.idle":"2025-11-16T18:32:27.616557Z","shell.execute_reply.started":"2025-11-16T18:32:27.241867Z","shell.execute_reply":"2025-11-16T18:32:27.615862Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n### **1.4 Loading Offline `ensemble_boxes` Package**\nKaggle competition environments do not allow internet installation.  \nTherefore, the project includes an **isolated version** of `ensemble_boxes`, which is manually appended to `sys.path`.\n\nThis package provides **Weighted Boxes Fusion (WBF)** used later for combining predictions across TTA augmentations.\n","metadata":{}},{"cell_type":"code","source":"# --- Configuration for Offline Libraries ---\n# Path to the dataset containing the 'ensemble_boxes' library\nPACKAGE_DIR = '/kaggle/input/wheat-detection-ensemble-boxes/isolated_ensemble_boxes/'\nif PACKAGE_DIR not in sys.path:\n    sys.path.append(PACKAGE_DIR)\n\nprint(f\"Added isolated package path to sys.path: {PACKAGE_DIR}\")\n\nimport ensemble_boxes\nprint(\"✅ ensemble_boxes imported successfully.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:53:29.311773Z","iopub.execute_input":"2025-11-16T18:53:29.312093Z","iopub.status.idle":"2025-11-16T18:53:29.317470Z","shell.execute_reply.started":"2025-11-16T18:53:29.312067Z","shell.execute_reply":"2025-11-16T18:53:29.316619Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n### **1.5 Verifying WBF Import**\nWe attempt to import `weighted_boxes_fusion`.  \nIf not available, a clear warning is shown so the user understands the missing dependency.\n","metadata":{}},{"cell_type":"code","source":"try:\n    from ensemble_boxes import weighted_boxes_fusion\nexcept ImportError:\n    print(\"WARNING: The 'ensemble_boxes' library is required for WBF.\")\n    print(\"Please run: pip install ensemble-boxes\")\n    # Placeholder to prevent crash, you must install the library to proceed\n    weighted_boxes_fusion = None ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:53:44.594256Z","iopub.execute_input":"2025-11-16T18:53:44.594970Z","iopub.status.idle":"2025-11-16T18:53:44.598582Z","shell.execute_reply.started":"2025-11-16T18:53:44.594942Z","shell.execute_reply":"2025-11-16T18:53:44.597883Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Section 2. Global Configuration & Data Paths**\n\nHere we define the hyperparameters for inference, such as image size, confidence thresholds, and Intersection over Union (IoU) thresholds.\n\n* **`CONF_THRESHOLD`**: Only predictions with confidence above this value are kept.\n* **`IOU_THRESHOLD`**: Used in Non-Max Suppression (NMS) or WBF to decide when boxes overlap enough to be merged.\n* **`NUM_TTA`**: The number of augmentations (e.g., flips/rotations) to apply per image.","metadata":{}},{"cell_type":"code","source":"# --- Inference Hyperparameters ---\nAUTO = tf.data.AUTOTUNE  # Auto from TensorFlow\nIMG_SIZE = (1024, 1024)  # Input size for the YOLOv8 model\nNUM_CLASSES = 1          # Wheat heads\nNUM_TTA = 8              # Number of Test Time Augmentations\nCONF_THRESHOLD = 0.40    # Minimum confidence score to keep a box\nIOU_THRESHOLD = 0.50     # IoU threshold for Weighted Boxes Fusion\nBATCH_SIZE_PER_REPLICA = 1\nBATCH_SIZE = BATCH_SIZE_PER_REPLICA * strategy.num_replicas_in_sync\n\n# --- Data & Model Paths ---\n# Root directory for competition data\nDATA_DIR = '/kaggle/input/global-wheat-detection' \nTRAIN_DIR = os.path.join(DATA_DIR, 'train')\nTEST_DIR = os.path.join(DATA_DIR, 'test')\nCSV_PATH = os.path.join(DATA_DIR, 'train.csv')\n\n# Path to the fine-tuned KerasCV model (ensure this matches your uploaded model dataset)\nMODEL_DIR = '/kaggle/input/wheat-detection/keras/warmup/3/finetune_best_model.keras'\n\nprint(f\"Global Batch size: {BATCH_SIZE}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:12:34.508787Z","iopub.execute_input":"2025-11-16T19:12:34.509049Z","iopub.status.idle":"2025-11-16T19:12:34.514433Z","shell.execute_reply.started":"2025-11-16T19:12:34.509030Z","shell.execute_reply":"2025-11-16T19:12:34.513562Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Section 3. Model Loading**\n\nWe load the fine-tuned YOLOv8 model within the distribution strategy scope.\n\n**Crucial Step:**\nBecause `KerasCV` models use custom layers and backbones (like `YOLOV8Detector` and `YOLOV8Backbone`), we must pass these in the `custom_objects` dictionary when loading the model. This allows TensorFlow to reconstruct the architecture correctly.","metadata":{}},{"cell_type":"code","source":"with strategy.scope():\n    print(\"Loading model ...\")\n    # Load the model providing necessary custom objects from KerasCV\n    yolo_model = tf.keras.models.load_model(MODEL_DIR,\n            custom_objects = {\n                'YOLOV8Detector': keras_cv.models.YOLOV8Detector,\n                'YOLOV8Backbone': keras_cv.models.YOLOV8Backbone\n            }\n    )\n    print(\"Model loaded successfully. Ready for Visualization and Test Time !\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:32:35.476784Z","iopub.execute_input":"2025-11-16T18:32:35.477565Z","iopub.status.idle":"2025-11-16T18:32:41.507458Z","shell.execute_reply.started":"2025-11-16T18:32:35.477541Z","shell.execute_reply":"2025-11-16T18:32:41.506660Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## **Section 4. Data Preparation & Basic Visualization Utilities**\n\nThis section handles the preprocessing of bounding-box annotations and provides helper\nfunctions for visualizing sample images from the **training** and **test** datasets.  \nThese steps are not part of inference preprocessing — they are only used for validation, inspection,\nand manual quality checks.\n\n---","metadata":{}},{"cell_type":"markdown","source":"### **4.1 — Load CSV & Expand Bounding Boxes**\n\nThe Kaggle Wheat Detection dataset stores bounding boxes as Python-like string lists  \n(e.g., `\"[x, y, w, h]\"`).  \nWe convert them into actual lists using `ast.literal_eval`, then expand them to  \nthe more convenient `x_min, y_min, x_max, y_max` format for visualization.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(CSV_PATH)\n# Convert the string representation of the list in 'bbox' column to an actual list\ndf['bbox'] = df['bbox'].apply(ast.literal_eval)\n# Extract coordinates from the list: [x_min, y_min, x_max, y_max]\ndf['x_min'] = df['bbox'].apply(lambda b: b[0])\ndf['y_min'] = df['bbox'].apply(lambda b: b[1])\ndf['x_max'] = df['bbox'].apply(lambda b: b[0] + b[2])\ndf['y_max'] = df['bbox'].apply(lambda b: b[1] + b[3])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:49:35.077997Z","iopub.execute_input":"2025-11-16T18:49:35.078746Z","iopub.status.idle":"2025-11-16T18:49:37.254883Z","shell.execute_reply.started":"2025-11-16T18:49:35.078722Z","shell.execute_reply":"2025-11-16T18:49:37.254275Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **4.2 — Visualize Random Training Images with Ground-Truth Boxes**  \nThis function randomly selects images from the training directory, loads them using OpenCV, and overlays all ground-truth bounding boxes belonging to that image.  \nUseful for verifying annotation quality before training.\n","metadata":{}},{"cell_type":"code","source":"def show_images_with_bboxes(df, image_dir, nrows, ncols):\n    # Pick random images from the train dir\n    files = os.listdir(image_dir)\n    fig, axs = plt.subplots(nrows, ncols, figsize=(4*ncols, 4*nrows))\n\n    for ax, fname in zip(axs.flatten(), files):\n        image_id = fname.replace('.jpg', '')\n\n        # Load image\n        img_path = os.path.join(image_dir, fname)\n        img = cv2.imread(img_path)\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n\n        # Get bboxes if exists\n        if image_id in df['image_id'].values:\n            bboxes = df[df['image_id'] == image_id][['x_min', 'y_min', 'x_max', 'y_max']].values\n            for (x_min, y_min, x_max, y_max) in bboxes:\n                start_point = (int(x_min), int(y_min))\n                end_point = (int(x_max), int(y_max))\n                color = (255, 0, 0)\n                thickness = 2\n                cv2.rectangle(img, start_point, end_point, color, thickness)\n\n        # Show image\n        ax.imshow(img)\n        ax.axis('off')\n        ax.set_title(fname, fontsize=8)\n        \n    plt.tight_layout()\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:33:41.894485Z","iopub.execute_input":"2025-11-16T18:33:41.895252Z","iopub.status.idle":"2025-11-16T18:33:41.902015Z","shell.execute_reply.started":"2025-11-16T18:33:41.895212Z","shell.execute_reply":"2025-11-16T18:33:41.900929Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"show_images_with_bboxes(df, TRAIN_DIR, 2, 2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:33:43.954853Z","iopub.execute_input":"2025-11-16T18:33:43.955173Z","iopub.status.idle":"2025-11-16T18:33:45.106118Z","shell.execute_reply.started":"2025-11-16T18:33:43.955123Z","shell.execute_reply":"2025-11-16T18:33:45.104990Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **4.3 — Utility: Load a Single Image for Visualization**  \nWe define a helper function that loads and resizes a single image to the model’s expected input size.  \nThis is **not** a preprocessing pipeline for inference — it's only for visualization helper tools.\n","metadata":{}},{"cell_type":"code","source":"def load_single_image(image_path):\n    \"\"\"Loads and resizes a single image for prediction.\"\"\"\n    image = tf.io.read_file(image_path)\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = tf.image.resize(image, IMG_SIZE)\n    return image","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:49:27.060326Z","iopub.execute_input":"2025-11-16T18:49:27.060593Z","iopub.status.idle":"2025-11-16T18:49:27.064869Z","shell.execute_reply.started":"2025-11-16T18:49:27.060574Z","shell.execute_reply":"2025-11-16T18:49:27.064153Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **4.4 — Visualize Model Predictions on Test Images**  \nThis function loads a small batch of test images, runs YOLO model inference, filters out low-confidence predictions, and visualizes the results using `keras_cv.visualization.plot_bounding_box_gallery()`.  \nThis helps verify whether the model produces meaningful predictions before moving to full inference.\n","metadata":{}},{"cell_type":"code","source":"def detections(model, image_path, bounding_box_format=\"xyxy\", \n                         confidence_threshold=CONF_THRESHOLD):\n    files = os.listdir(image_path)[:4]\n    paths = [os.path.join(image_path, f) for f in files]\n    num_images = len(paths)\n\n    image_list = [load_single_image(path) for path in paths]\n    images = tf.stack(image_list)\n    print(f\"Loaded {num_images} images into a batch of shape {images.shape}\")\n    \n    # Run model inference on the batch\n    y_pred = model.predict(images)\n\n    # y_pred is a dictionary: {'boxes': ..., 'confidence': ..., 'classes': ...}\n    # Filter low-confidence boxes manually (since keras_cv.bounding_box has no filter)\n    conf_mask = y_pred[\"confidence\"] > confidence_threshold\n\n    # Create filtered prediction dict\n    y_pred_filtered = {\n        \"boxes\": tf.ragged.boolean_mask(y_pred[\"boxes\"], conf_mask),\n        \"classes\": tf.ragged.boolean_mask(y_pred[\"classes\"], conf_mask),\n        \"confidence\": tf.ragged.boolean_mask(y_pred[\"confidence\"], conf_mask),\n    }\n\n    # Visualize with KerasCV utility\n    keras_cv.visualization.plot_bounding_box_gallery(\n        images,\n        value_range=(0, 255),\n        bounding_box_format=bounding_box_format,\n        y_pred=y_pred_filtered,\n        scale=4,\n        rows=2,\n        cols=2,\n        show=True,\n        font_scale=0.7,\n    )\n\n    return y_pred\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:09:32.689598Z","iopub.execute_input":"2025-11-16T19:09:32.690259Z","iopub.status.idle":"2025-11-16T19:09:32.696539Z","shell.execute_reply.started":"2025-11-16T19:09:32.690235Z","shell.execute_reply":"2025-11-16T19:09:32.695798Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"y_pred = detections(yolo_model, TRAIN_DIR, bounding_box_format=\"xyxy\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:09:34.681542Z","iopub.execute_input":"2025-11-16T19:09:34.681831Z","iopub.status.idle":"2025-11-16T19:09:37.115572Z","shell.execute_reply.started":"2025-11-16T19:09:34.681809Z","shell.execute_reply":"2025-11-16T19:09:37.114691Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Also we can see what's the `Structure` of `Prediction` is like","metadata":{}},{"cell_type":"code","source":"print(y_pred)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:09:55.668806Z","iopub.execute_input":"2025-11-16T19:09:55.669099Z","iopub.status.idle":"2025-11-16T19:09:55.677546Z","shell.execute_reply.started":"2025-11-16T19:09:55.669077Z","shell.execute_reply":"2025-11-16T19:09:55.676765Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"y_pred = detections(yolo_model, TEST_DIR)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:10:02.710778Z","iopub.execute_input":"2025-11-16T19:10:02.711275Z","iopub.status.idle":"2025-11-16T19:10:05.256978Z","shell.execute_reply.started":"2025-11-16T19:10:02.711251Z","shell.execute_reply":"2025-11-16T19:10:05.256164Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"##  Section 5. Test-Time Augmentation (TTA) and Box Coordinate Reversal\n\nIn object detection, **Test-Time Augmentation (TTA)** is a powerful technique where we run inference on multiple augmented versions of an image (e.g., flips, color jitter) and then convert all predictions back into the *original* coordinate system.\n\nThis is especially important for flips:\n- A **horizontal flip** mirrors the image on the x-axis → bounding boxes must be reversed accordingly.\n- A **vertical flip** mirrors the image on the y-axis → bounding boxes must also be reversed.\n\nThis section focuses on:\n1. Applying deterministic augmentations (HFlip, VFlip).\n2. Predicting with the model on each augmented image.\n3. Reversing the predicted bounding boxes back to original image coordinates.\n4. Verifying visually that the reversal logic is correct.\n","metadata":{}},{"cell_type":"markdown","source":"### **5.1 — Reverse Horizontal Flip (HFlip)**\n\nA horizontal flip mirrors the image across the **vertical midline**, which inverts the x-coordinates:\n\n- `xmin_new = image_width  - xmax_old`\n- `xmax_new = image_width  - xmin_old`\n\nY-coordinates remain unchanged.\n\nThis subsection implements the exact mathematical reversal needed to convert horizontally-flipped predictions back into the original coordinate system.","metadata":{}},{"cell_type":"code","source":"def reverse_h_flip(boxes, image_width= IMG_SIZE[1]):\n    \"\"\"Reverses a horizontal flip for pixel-space [xmin, ymin, xmax, ymax] boxes.\"\"\"\n    xmin = image_width - boxes[:, 2]\n    ymin = boxes[:, 1]\n    xmax = image_width - boxes[:, 0]\n    ymax = boxes[:, 3]\n    return tf.stack([xmin, ymin, xmax, ymax], axis=-1)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:59:34.690150Z","iopub.execute_input":"2025-11-16T18:59:34.690419Z","iopub.status.idle":"2025-11-16T18:59:34.695754Z","shell.execute_reply.started":"2025-11-16T18:59:34.690400Z","shell.execute_reply":"2025-11-16T18:59:34.694859Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.2 — Reverse Vertical Flip (VFlip)**\n\nA vertical flip mirrors the image across the **horizontal midline**, which inverts the y-coordinates:\n\n- `ymin_new = image_height - ymax_old`\n- `ymax_new = image_height - ymin_old`\n\nX-coordinates remain unchanged.\n\nThis subsection provides the correct formula for restoring bounding boxes after vertical flipping.\n","metadata":{}},{"cell_type":"code","source":"def reverse_v_flip(boxes, image_height= IMG_SIZE[0]):\n    \"\"\"Reverses a vertical flip for normalized [xmin, ymin, xmax, ymax] boxes.\"\"\"\n    # ymin_new = 1 - ymax_old; ymax_new = 1 - ymin_old\n    xmin = boxes[:, 0]     # Use xmin index (0)\n    ymin = image_height - boxes[:, 3] # Use ymax index (3)\n    xmax = boxes[:, 2]     # Use xmax index (2)\n    ymax = image_height - boxes[:, 1] # Use ymin index (1)\n    \n    # Must stack in the original [xmin, ymin, xmax, ymax] order\n    return tf.stack([xmin, ymin, xmax, ymax], axis=-1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:59:36.124967Z","iopub.execute_input":"2025-11-16T18:59:36.125746Z","iopub.status.idle":"2025-11-16T18:59:36.130327Z","shell.execute_reply.started":"2025-11-16T18:59:36.125716Z","shell.execute_reply":"2025-11-16T18:59:36.129507Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.3 — Mild Color Jitter (Non-Geometric)**\n\nColor jitter modifies pixel intensities but **does not affect spatial geometry**, so bounding boxes stay the same.\n\nWe apply light augmentations:\n- Random Hue (±0.04)\n- Random Contrast (0.85–1.15)\n- Random Saturation (0.85–1.15)\n- Random Brightness (±0.04)\n\nThese improve robustness to lighting variations without distorting object locations.\n","metadata":{}},{"cell_type":"code","source":"# Function to apply multiple, slightly random color/light transforms\ndef random_color_jitter(img):\n    # Apply small random hue shift\n    img = tf.image.random_hue(img, max_delta=0.04) \n    # Apply small random contrast\n    img = tf.image.random_contrast(img, lower=0.85, upper=1.15) \n    # Apply small random saturation\n    img = tf.image.random_saturation(img, lower=0.85, upper=1.15)\n    # Apply small random brightness\n    img = tf.image.random_brightness(img, max_delta=0.04)\n    return img","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T18:59:38.498946Z","iopub.execute_input":"2025-11-16T18:59:38.499671Z","iopub.status.idle":"2025-11-16T18:59:38.504192Z","shell.execute_reply.started":"2025-11-16T18:59:38.499647Z","shell.execute_reply":"2025-11-16T18:59:38.503426Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.4 — Predict & Filter Detections**\n\nA reusable helper function that:\n\n1. Expands the input image to a batch  \n2. Runs `model.predict()`  \n3. Filters boxes by confidence threshold  \n4. Returns a RaggedTensor prediction dictionary:\n   - `\"boxes\"`\n   - `\"classes\"`\n   - `\"confidence\"`\n\nUsed for:\n- Original image prediction  \n- Flipped image prediction  \n- TTA inference pipelines  \n","metadata":{}},{"cell_type":"code","source":"test_img_path = os.path.join(TEST_DIR, os.listdir(TEST_DIR)[3])\ntest_img = load_single_image(test_img_path)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:03:16.404738Z","iopub.execute_input":"2025-11-16T19:03:16.405035Z","iopub.status.idle":"2025-11-16T19:03:16.429486Z","shell.execute_reply.started":"2025-11-16T19:03:16.405014Z","shell.execute_reply":"2025-11-16T19:03:16.428905Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def predict_and_filter(model, image_tensor, conf_threshold):\n    \"\"\"\n    Runs model prediction on a single image, filters results by confidence.\n    \n    Args:\n        model: The trained Keras model.\n        image_tensor: The input image (H, W, 3).\n        conf_threshold: Minimum confidence score to keep a detection.\n        \n    Returns:\n        dict: A dictionary of filtered predictions {\"boxes\", \"classes\", \"confidence\"}\n              in KerasCV RaggedTensor format, with batch dimension [1, None, ...].\n    \"\"\"\n    # Add batch dimension: [1, H, W, 3]\n    input_tensor = tf.expand_dims(image_tensor, 0)\n    \n    # Predict\n    y_pred = model.predict(input_tensor, verbose=0)\n\n    # Filter by confidence\n    conf_mask = y_pred[\"confidence\"] > conf_threshold\n    \n    # Apply mask to all prediction components\n    y_pred_filtered = {\n        \"boxes\": tf.ragged.boolean_mask(y_pred[\"boxes\"], conf_mask),\n        \"classes\": tf.ragged.boolean_mask(y_pred[\"classes\"], conf_mask),\n        \"confidence\": tf.ragged.boolean_mask(y_pred[\"confidence\"], conf_mask),\n    }\n    \n    return y_pred_filtered","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:00:02.908335Z","iopub.execute_input":"2025-11-16T19:00:02.908904Z","iopub.status.idle":"2025-11-16T19:00:02.913648Z","shell.execute_reply.started":"2025-11-16T19:00:02.908884Z","shell.execute_reply":"2025-11-16T19:00:02.912926Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.5 — Visualize Predictions**\n\nA visualization utility that wraps  \n`keras_cv.visualization.plot_bounding_box_gallery()`.\n\nWe use it to:\n- Inspect original predictions  \n- Inspect flip-space predictions  \n- Inspect reversed predictions on the original image  \n\nThis is essential to verify that flip reversals align with ground truth spatial structure.\n","metadata":{}},{"cell_type":"code","source":"def visualize_prediction(image_tensor, predictions, title, **kwargs):\n    \"\"\"\n    Visualizes the prediction on the given image tensor.\n    \n    Args:\n        image_tensor: The image to plot on (H, W, 3).\n        predictions: The prediction dictionary from predict_and_filter.\n        title: Title to print before visualization.\n        **kwargs: Optional keyword arguments for plot_bounding_box_gallery.\n    \"\"\"\n    print(f\"\\n--- {title} ---\")\n    \n    keras_cv.visualization.plot_bounding_box_gallery(\n        tf.expand_dims(image_tensor, 0),\n        value_range=(0, 255),\n        bounding_box_format=\"xyxy\",\n        y_pred=predictions,\n        scale=4,\n        rows=1,\n        cols=1,\n        show=True,\n        font_scale=0.7,\n        **kwargs\n    )\n    print(f\"✅ {title} visualized.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:00:04.269832Z","iopub.execute_input":"2025-11-16T19:00:04.270421Z","iopub.status.idle":"2025-11-16T19:00:04.275337Z","shell.execute_reply.started":"2025-11-16T19:00:04.270396Z","shell.execute_reply":"2025-11-16T19:00:04.274544Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.6 — Step 1 — Original Image Prediction**\n\nWe begin with the baseline:  \nRun inference on the unaltered image to produce reference-coordinate bounding boxes.\n\nAll TTA-corrected boxes are later compared to this reference.","metadata":{}},{"cell_type":"code","source":"# --- STEP 1: Original Image Prediction ---\ny_pred_original = predict_and_filter(\n    yolo_model, \n    test_img, \n    CONF_THRESHOLD\n)\n\nvisualize_prediction(\n    test_img, \n    y_pred_original, \n    \"Original Prediction\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:00:05.772897Z","iopub.execute_input":"2025-11-16T19:00:05.773558Z","iopub.status.idle":"2025-11-16T19:00:06.595336Z","shell.execute_reply.started":"2025-11-16T19:00:05.773536Z","shell.execute_reply":"2025-11-16T19:00:06.594589Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.6 — Step 2 — Prediction on Vertically Flipped Image**\n\nWe flip the input vertically and run the model again.\n\nImportant:\n- Predictions now exist **in flipped coordinate space**\n- Boxes must not be used directly\n- They must first be corrected (next step)\n","metadata":{}},{"cell_type":"code","source":"# --- STEP 2: Vertical Flip, Predict, and Visualize Flipped ---\nvflipped_img = tf.image.flip_up_down(test_img)\n\ny_pred_flip = predict_and_filter(\n    yolo_model, \n    vflipped_img, \n    CONF_THRESHOLD\n)\n\nvisualize_prediction(\n    vflipped_img, \n    y_pred_flip, \n    \"Flipped Image Prediction\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:01:53.255410Z","iopub.execute_input":"2025-11-16T19:01:53.255942Z","iopub.status.idle":"2025-11-16T19:01:54.091375Z","shell.execute_reply.started":"2025-11-16T19:01:53.255922Z","shell.execute_reply":"2025-11-16T19:01:54.090493Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **5.6 — Step 3 — Reverse Vertical Flip & Visualize on Original Image**\n\nWe take vertical-flip predictions and mathematically map them back to the original coordinate system using the reversal formula.\n\nThen we visualize these corrected boxes **on the original image**.\n\nThis final step verifies:\n- Box reversal logic is correct  \n- Predictions align normally after TTA  \n- No shift/offset/misalignment occurs  \n","metadata":{}},{"cell_type":"code","source":"# --- STEP 3: Reverse Boxes and Visualize on Original Image ---\nprint(\"\\nReversing flipped boxes to original coordinates...\")\n\n# Extract boxes from the filtered RaggedTensor (get the dense tensor at index 0)\nboxes_augmented = y_pred_flip[\"boxes\"][0] \n\n# Note: Your original reverse_v_flip function requires the image dimension\n# We assume it takes IMG_SIZE[0] (1024) as the image_height argument.\n# We must ensure reverse_v_flip is passed the image size as defined in your setup.\nboxes_reversed = reverse_v_flip(boxes_augmented)\n\n# Reformat the reversed boxes into the prediction dictionary for plotting\ny_pred_reversed = {\n    \"boxes\": tf.RaggedTensor.from_tensor(tf.expand_dims(boxes_reversed, axis=0)),\n    \"classes\": y_pred_flip[\"classes\"],\n    \"confidence\": y_pred_flip[\"confidence\"],\n}\n\nvisualize_prediction(\n    test_img, \n    y_pred_reversed, \n    \"Reversed Flipped Boxes on Original Image\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:02:09.189422Z","iopub.execute_input":"2025-11-16T19:02:09.189975Z","iopub.status.idle":"2025-11-16T19:02:09.545068Z","shell.execute_reply.started":"2025-11-16T19:02:09.189953Z","shell.execute_reply":"2025-11-16T19:02:09.544364Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## Section **6 — TTA Sequences & TTA Aggregation Pipeline**\n\nThis section defines the complete TTA (Test-Time Augmentation) pipeline.  \nTTA improves detection robustness by running multiple augmented versions of the same input image through the model and then aggregating the predictions back into the original coordinate system.\n\nWe define:\n- A set of **TTA sequences** (image augmentation + reverse-mapping function)\n- A **TTA engine** that:\n  1. Applies each augmentation  \n  2. Runs inference  \n  3. Reverses bounding boxes  \n  4. Aggregates all predictions  \n","metadata":{}},{"cell_type":"markdown","source":"### 6.1 TTA Sequence\n\nTest-Time Augmentation (TTA) requires defining a set of image transformations and how their predicted bounding boxes should be reversed back to the original coordinate space.\n\nHere we define **TTA_SEQUENCES**:\n\n1. **Baseline (Identity)** — no augmentation, boxes unchanged.\n2. **Horizontal Flip** — uses `tf.image.flip_left_right` and `reverse_h_flip`.\n3. **Vertical Flip** — uses `tf.image.flip_up_down` and `reverse_v_flip`.\n4-7. **Random Color Variants** — mild color jittering; these **do not alter spatial coordinates**, so reversal is identity.\n\nThis structure ensures:\n- Geometric transforms → boxes reversed mathematically  \n- Photometric transforms → boxes unchanged\n","metadata":{}},{"cell_type":"code","source":"# --- TTA Sequence Definition (for [xmin, ymin, xmax, ymax] format) ---\nTTA_SEQUENCES = [\n    # 0. Baseline (Identity)\n    (lambda img: img, lambda boxes: boxes),  \n    \n    # 1. Horizontal Flip\n    (tf.image.flip_left_right, reverse_h_flip),\n    \n    # 2. Vertical Flip\n    (tf.image.flip_up_down, reverse_v_flip),\n    \n    # 3-6. Four different, random photometric variants (No box reversal needed)\n    (lambda img: random_color_jitter(img), lambda boxes: boxes), \n    (lambda img: random_color_jitter(img), lambda boxes: boxes), \n    (lambda img: random_color_jitter(img), lambda boxes: boxes), \n    (lambda img: random_color_jitter(img), lambda boxes: boxes), \n]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:02:13.287395Z","iopub.execute_input":"2025-11-16T19:02:13.288000Z","iopub.status.idle":"2025-11-16T19:02:13.292795Z","shell.execute_reply.started":"2025-11-16T19:02:13.287978Z","shell.execute_reply":"2025-11-16T19:02:13.292013Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### 6.2 TTA Function\n\nThe `tta` function performs inference across all TTA sequences and aggregates the results:\n\n1. **Augment Image** — apply the augmentation function for this TTA sequence.\n2. **Predict** — run model inference on the augmented image.\n3. **Filter by Confidence** — remove low-confidence boxes.\n4. **Reverse Boxes** — map boxes back to the original coordinate system if needed.\n5. **Aggregate** — collect boxes, confidences, and classes from all sequences into single arrays.\n\nThe output is ready for **Weighted Boxes Fusion (WBF)** or other aggregation techniques.  \nThis function logs sequence-wise results for debugging and ensures all TTA predictions are aligned.\n","metadata":{}},{"cell_type":"code","source":"def tta(model, image_tensor, tta_sequences, conf_threshold):\n    \"\"\"\n    Applies TTA sequences, collects predictions, reverses boxes, and aggregates results \n    for a single image. **Note: Relies on external global variable IMG_SIZE[0] for dimensions\n    if not passed to reverse_boxes_fn.**\n\n    Args:\n        model: The trained Keras model.\n        image_tensor: The input image tensor (e.g., test_img, shape [H, W, 3]).\n        tta_sequences: The list of (augment_fn, reverse_fn) tuples.\n        conf_threshold: The minimum confidence for filtering individual TTA predictions.\n        \n    Returns:\n        tuple: (all_boxes_np, all_confidences_np, all_classes_np) as concatenated NumPy arrays.\n    \"\"\"\n    all_tta_predictions = []\n\n    print(f\"--- Running TTA for 1 image ({len(tta_sequences)} sequences) ---\")\n\n    for i, (augment_image_fn, reverse_boxes_fn) in enumerate(TTA_SEQUENCES):\n        # 1. Augment and Prepare Input\n        augmented_img = augment_image_fn(image_tensor)\n        input_tensor = tf.expand_dims(augmented_img, 0)\n        \n        # 2. Predict (verbose=0 suppresses Keras logging)\n        y_pred = model.predict(input_tensor, verbose=0)\n\n        # 3. Filter by confidence\n        conf_mask = y_pred[\"confidence\"] > conf_threshold\n        \n        # Extract and un-batch filtered tensors\n        boxes_augmented = tf.ragged.boolean_mask(y_pred[\"boxes\"], conf_mask)[0]\n        classes = tf.ragged.boolean_mask(y_pred[\"classes\"], conf_mask)[0]\n        confidence = tf.ragged.boolean_mask(y_pred[\"confidence\"], conf_mask)[0]\n\n        n_detections = tf.shape(boxes_augmented)[0].numpy()\n        \n        if n_detections > 0:\n            # Note: We must now pass the image_dim back, as reverse functions typically need it\n            # unless they were hard-coded to 1024 (which is poor practice).\n            # Assuming reverse_boxes_fn still expects the dimension for safety:\n            boxes_reversed = reverse_boxes_fn(boxes_augmented) \n            all_tta_predictions.append((boxes_reversed, confidence, classes))\n        \n        # New: Log every single TTA sequence\n        print(f\"  > Sequence {i+1}/{len(tta_sequences)} ({n_detections} boxes) processed.\")\n\n\n    if not all_tta_predictions:\n        print(\"--- Aggregation: No detections found above confidence threshold. ---\")\n        return None, None, None\n\n    # 4. Aggregate all predictions into single NumPy arrays\n    list_of_boxes = [p[0].numpy() for p in all_tta_predictions]\n    list_of_confidences = [p[1].numpy() for p in all_tta_predictions]\n    list_of_classes = [p[2].numpy() for p in all_tta_predictions]\n\n    all_boxes_np = np.concatenate(list_of_boxes, axis=0)\n    all_confidences_np = np.concatenate(list_of_confidences, axis=0)\n    # Corrected potential bug: Use list_of_classes instead of all_classes_np\n    all_classes_np = np.concatenate(list_of_classes, axis=0)\n\n    print(f\"--- Aggregation Complete. Total boxes for WBF: {len(all_boxes_np)} ---\")\n    \n    return all_boxes_np, all_confidences_np, all_classes_np","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:12:44.382768Z","iopub.execute_input":"2025-11-16T19:12:44.383421Z","iopub.status.idle":"2025-11-16T19:12:44.391493Z","shell.execute_reply.started":"2025-11-16T19:12:44.383399Z","shell.execute_reply":"2025-11-16T19:12:44.390592Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"all_boxes_np, all_confidences_np, all_classes_np = tta(yolo_model, test_img, \n                                                       TTA_SEQUENCES, CONF_THRESHOLD)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:12:46.821244Z","iopub.execute_input":"2025-11-16T19:12:46.821961Z","iopub.status.idle":"2025-11-16T19:12:50.337750Z","shell.execute_reply.started":"2025-11-16T19:12:46.821928Z","shell.execute_reply":"2025-11-16T19:12:50.337032Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## **Section 7. Weighted Boxes Fusion (WBF) for TTA Predictions**\n\nAfter performing Test-Time Augmentation (TTA), a single image can produce multiple overlapping predictions for the same object.  \nWeighted Boxes Fusion (WBF) is a robust method to merge these overlapping boxes into a single, more accurate prediction.  \n\n**Key Steps of WBF Implementation:**\n\n1. **Normalization** — Convert pixel-space coordinates [xmin, ymin, xmax, ymax] to the [0, 1] range, which is required by the WBF algorithm.  \n2. **Fusion** — Apply the `weighted_boxes_fusion()` function using confidence scores and IoU threshold to merge overlapping boxes.  \n3. **Filtering & Denormalization** — Remove boxes below the confidence threshold and scale them back to original image pixel dimensions.  \n4. **Submission Formatting** — Convert the fused boxes from xyxy format to Kaggle-friendly xywh format and create a space-delimited prediction string.  \n5. **Optional Visualization** — Prepare a KerasCV-compatible dictionary for plotting fused boxes and confidence scores.\n\n**Benefits:**\n- Reduces duplicate detections from multiple TTA variants\n- Improves box localization accuracy\n- Produces clean, submission-ready predictions for Kaggle competitions\n","metadata":{}},{"cell_type":"code","source":"def wbf(all_boxes_np, all_confidences_np, \n        all_classes_np, img_size, conf_threshold, iou_threshold):\n    \"\"\"\n    Performs WBF on aggregated predictions and converts results to Kaggle submission string.\n\n    Returns:\n        str: Space-delimited prediction string in \"conf x y w h\" format.\n        dict: Fused prediction dictionary for visualization (KerasCV format).\n    \"\"\"\n    image_dim = img_size[0]\n\n    # 1. Normalize pixel coordinates to [0.0, 1.0] for WBF\n    normalized_boxes = all_boxes_np / image_dim\n    \n    # WBF expects list of lists\n    boxes = [normalized_boxes.tolist()]\n    scores = [all_confidences_np.tolist()]\n    labels = [all_classes_np.astype(float).tolist()] \n\n    # 2. Apply Weighted Boxes Fusion (WBF)\n    fused_boxes_norm, fused_scores, fused_labels = weighted_boxes_fusion(\n        boxes,\n        scores,\n        labels,\n        weights=None,\n        iou_thr=iou_threshold,\n        conf_type='max',\n        skip_box_thr=0.0001\n    )\n    print(f\"--- WBF Applied. Boxes reduced to {len(fused_boxes_norm)} before final filter. ---\")\n    \n    # 3. Final Filtering and Denormalization\n    final_mask = fused_scores >= conf_threshold\n    final_boxes_norm = fused_boxes_norm[final_mask]\n    final_scores = fused_scores[final_mask]\n    final_labels_filtered = fused_labels[final_mask] # Capture filtered labels\n    \n    # Denormalize boxes back to [0, 1024] pixel space (xyxy format)\n    final_boxes_denorm_xyxy = final_boxes_norm * image_dim\n    \n    print(f\"--- Final {len(final_boxes_denorm_xyxy)} boxes remain after final confidence filter. ---\")\n\n    # 4. Format Conversion to Kaggle \"conf x y w h\"\n    prediction_strings = []\n    \n    # Iterate over the fused, filtered results\n    for box, score in zip(final_boxes_denorm_xyxy, final_scores):\n        # Convert to integer pixel values (Kaggle standard)\n        xmin, ymin, xmax, ymax = box.astype(np.int32)\n        \n        # Convert xyxy to xywh\n        w = xmax - xmin\n        h = ymax - ymin\n        \n        # Format: confidence xmin ymin w h (space-delimited)\n        box_string = f\"{score:.4f} {xmin} {ymin} {w} {h}\"\n        prediction_strings.append(box_string)\n        \n    submission_string = \" \".join(prediction_strings)\n    \n    # Optional: Show example submission string\n    if submission_string:\n        print(f\"--- Submission String Example: {submission_string[:100]}... ---\")\n    else:\n        print(\"--- Submission String is empty (No confident detections). ---\")\n    \n    # 5. Prepare KerasCV format for optional visualization\n    final_boxes_denorm_batched = tf.expand_dims(\n        tf.convert_to_tensor(final_boxes_denorm_xyxy, dtype=tf.float32), axis=0\n    )\n    final_scores_batched = tf.expand_dims(\n        tf.convert_to_tensor(final_scores, dtype=tf.float32), axis=0\n    )\n    \n    y_pred_tta_fused = {\n        \"boxes\": tf.RaggedTensor.from_tensor(final_boxes_denorm_batched),\n        # Use the filtered labels, converted to int32, and batched\n        \"classes\": tf.RaggedTensor.from_tensor(\n            tf.expand_dims(tf.convert_to_tensor(final_labels_filtered, dtype=tf.int32), axis=0)\n        ),\n        \"confidence\": tf.RaggedTensor.from_tensor(final_scores_batched),\n    }\n\n    return submission_string, y_pred_tta_fused","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:12:52.465432Z","iopub.execute_input":"2025-11-16T19:12:52.465707Z","iopub.status.idle":"2025-11-16T19:12:52.475104Z","shell.execute_reply.started":"2025-11-16T19:12:52.465684Z","shell.execute_reply":"2025-11-16T19:12:52.474323Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_string, y_pred_fused = wbf(all_boxes_np, \n                                      all_confidences_np, \n                                      all_classes_np,\n                                      IMG_SIZE,\n                                      CONF_THRESHOLD,\n                                      IOU_THRESHOLD)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:12:55.488273Z","iopub.execute_input":"2025-11-16T19:12:55.488774Z","iopub.status.idle":"2025-11-16T19:12:55.530853Z","shell.execute_reply.started":"2025-11-16T19:12:55.488754Z","shell.execute_reply":"2025-11-16T19:12:55.530189Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Assuming you have the fused result from wbf_and_format:\n# submission_string, y_pred_fused = wbf_and_format(...)\n\nvisualize_prediction(\n    image_tensor=test_img, \n    predictions=y_pred_fused, \n    title=\"Final WBF-Fused TTA Result\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:13:00.466538Z","iopub.execute_input":"2025-11-16T19:13:00.467248Z","iopub.status.idle":"2025-11-16T19:13:00.775940Z","shell.execute_reply.started":"2025-11-16T19:13:00.467218Z","shell.execute_reply":"2025-11-16T19:13:00.775169Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## **Section 8. Submission file and Test dataset**\n\nThis section handles **end-to-end inference** on the test dataset.  \nIt applies **Test-Time Augmentation (TTA)** and **Weighted Boxes Fusion (WBF)** for each image, formats predictions for Kaggle submission, and optionally visualizes results.  \n\nThe pipeline includes:\n1. Preparing the test dataset.\n2. Running TTA/WBF for each image.\n3. Collecting results into a submission DataFrame.\n\n---","metadata":{}},{"cell_type":"markdown","source":"### **8.1 Preprocessing Individual Test Images**\n\n- **Function:** `preprocess_for_inference(image_path)`  \n- **Purpose:** Loads a single image, resizes it to the model’s input shape, and extracts the image ID (filename without extension).  \n- **Output:** `(image_tensor, image_id)` ready for inference.  \n\nThis ensures all test images are properly formatted for the model.\n\n---","metadata":{}},{"cell_type":"code","source":"def preprocess_for_inference(image_path):\n    \"\"\"Loads and resizes a single image for model prediction.\"\"\"\n    image = load_single_image(image_path)\n    image_id = tf.strings.split(image_path, os.sep)[-1]       # filename\n    image_id = tf.strings.regex_replace(image_id, \".jpg$\", \"\") # remove extension\n    return image, image_id","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:39:35.286898Z","iopub.execute_input":"2025-11-16T19:39:35.287557Z","iopub.status.idle":"2025-11-16T19:39:35.291880Z","shell.execute_reply.started":"2025-11-16T19:39:35.287534Z","shell.execute_reply":"2025-11-16T19:39:35.291089Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **8.2 Creating the Test Dataset**\n\n- **Function:** `test_dataset(TEST_DIR)`  \n- **Purpose:** Generates a `tf.data.Dataset` from all images in the test directory.  \n- **Steps:** \n  - Collect image file paths.\n  - Map each path through `preprocess_for_inference`.\n  - Batch and prefetch for efficient GPU/CPU utilization.  \n- **Output:** A TensorFlow dataset suitable for batch-wise TTA/WBF inference.","metadata":{}},{"cell_type":"code","source":"def test_dataset(TEST_DIR):\n    '''Create dataset with loading all images in Test directory'''\n    test_images_path = [\n        os.path.join(TEST_DIR, fname) for fname in os.listdir(TEST_DIR)\n    ]\n    test_ds = tf.data.Dataset.from_tensor_slices(test_images_path)\n    test_ds = (test_ds.map(preprocess_for_inference, num_parallel_calls= AUTO)\n                      .batch(BATCH_SIZE)\n                      .prefetch(AUTO))\n    \n    print('Test Dataset created successfully !')\n    return test_ds","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:39:48.371605Z","iopub.execute_input":"2025-11-16T19:39:48.371916Z","iopub.status.idle":"2025-11-16T19:39:48.376849Z","shell.execute_reply.started":"2025-11-16T19:39:48.371893Z","shell.execute_reply":"2025-11-16T19:39:48.376187Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### **8.3 Running TTA/WBF & Creating Submission File**\n\n- **Function:** `create_submission_file(model, test_ds, tta_sequences, img_size, conf_threshold, iou_threshold)`  \n- **Purpose:** Processes all test images, applies TTA, fuses predictions via WBF, and creates a Kaggle-compatible submission.  \n- **Steps:**\n  1. Loop over each image in the test dataset.\n  2. Run TTA and aggregate predictions.\n  3. Apply WBF to merge overlapping boxes and filter by confidence.\n  4. Format predictions into Kaggle \"conf xmin ymin w h\" strings.\n  5. Collect all image IDs and prediction strings into a DataFrame.\n  6. Optionally visualize the first image’s fused predictions.  \n\n- **Output:** `pd.DataFrame` containing `image_id` and `PredictionString`, ready for submission.","metadata":{}},{"cell_type":"code","source":"def create_submission_file(model, test_ds, tta_sequences, img_size, conf_threshold, iou_threshold):\n    \"\"\"\n    Processes the entire test dataset using TTA/WBF and creates the submission DataFrame.\n    \n    Args:\n        model: The Keras model.\n        test_ds: The tf.data.Dataset for the test images.\n        ... all other necessary parameters\n        \n    Returns:\n        pd.DataFrame: The final submission dataframe.\n    \"\"\"\n    submission_data = {\"image_id\": [], \"PredictionString\": []}\n    \n    print(\"\\n=======================================================\")\n    print(\">>> Starting Inference on Test Dataset with TTA/WBF <<<\")\n    print(\"=======================================================\")\n    \n    # Iterate through the test dataset\n    for step, (images, image_ids) in enumerate(test_ds):\n        \n        # Loop through images in the current batch (usually batch_size=1 for TTA)\n        for image_tensor, image_id in zip(images, image_ids):\n            \n            image_id_str = image_id.numpy().decode('utf-8')\n            print(f\"\\n[IMAGE {step+1}] Processing image: {image_id_str}\")\n\n            # 1. Run TTA and Aggregate\n            all_boxes, all_confidences, all_classes = tta(\n                model, image_tensor, tta_sequences, conf_threshold\n            )\n            \n            if all_boxes is None:\n                # No detections, prediction string is empty\n                submission_string = \"\"\n                y_pred_fused = None\n            else:\n                # 2. Run WBF and Format\n                submission_string, y_pred_fused = wbf(\n                    all_boxes, all_confidences, all_classes, \n                    img_size, conf_threshold, iou_threshold\n                )\n            \n            # 3. Collect Result\n            submission_data[\"image_id\"].append(image_id_str)\n            submission_data[\"PredictionString\"].append(submission_string)\n\n            # 4. Optional: Visualize the first image result\n            if step == 1 and y_pred_fused is not None:\n                visualize_prediction(image_tensor, \n                                     predictions=y_pred_fused, \n                                     title=\"Final WBF-Fused TTA Result\")\n\n    print(\"\\n=======================================================\")\n    print(\">>> TTA/WBF Inference Complete. Creating Submission File <<<\")\n    \n    submission_df = pd.DataFrame(submission_data)\n    return submission_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:14:25.142618Z","iopub.execute_input":"2025-11-16T19:14:25.142836Z","iopub.status.idle":"2025-11-16T19:14:25.149745Z","shell.execute_reply.started":"2025-11-16T19:14:25.142820Z","shell.execute_reply":"2025-11-16T19:14:25.148937Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_ds = test_dataset(TEST_DIR)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:39:49.561557Z","iopub.execute_input":"2025-11-16T19:39:49.561833Z","iopub.status.idle":"2025-11-16T19:39:49.692855Z","shell.execute_reply.started":"2025-11-16T19:39:49.561812Z","shell.execute_reply":"2025-11-16T19:39:49.692010Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_df = create_submission_file(yolo_model,\n                       test_ds,\n                       TTA_SEQUENCES,\n                       IMG_SIZE,\n                       CONF_THRESHOLD,\n                       IOU_THRESHOLD)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:39:57.946344Z","iopub.execute_input":"2025-11-16T19:39:57.946631Z","iopub.status.idle":"2025-11-16T19:40:34.081320Z","shell.execute_reply.started":"2025-11-16T19:39:57.946610Z","shell.execute_reply":"2025-11-16T19:40:34.080526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"submission_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:40:36.636947Z","iopub.execute_input":"2025-11-16T19:40:36.637694Z","iopub.status.idle":"2025-11-16T19:40:36.645238Z","shell.execute_reply.started":"2025-11-16T19:40:36.637674Z","shell.execute_reply":"2025-11-16T19:40:36.644547Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Save the final submission file\nsubmission_df.to_csv('submission.csv', index=False)\n\nprint('Submission file created successfully!')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-16T19:40:38.433442Z","iopub.execute_input":"2025-11-16T19:40:38.434002Z","iopub.status.idle":"2025-11-16T19:40:38.440322Z","shell.execute_reply.started":"2025-11-16T19:40:38.433973Z","shell.execute_reply":"2025-11-16T19:40:38.439540Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# ✅ Conclusion\n\nIn this notebook, we implemented a **professional inference pipeline** for the Wheat Detection Kaggle competition. Key takeaways include:\n\n- **Efficient preprocessing:** Test images are resized and prepared for model input, ensuring consistent tensor shapes.  \n- **Test-Time Augmentation (TTA):** Horizontal, vertical flips and color jittering help improve prediction robustness.  \n- **Weighted Boxes Fusion (WBF):** Aggregates predictions from multiple TTA variants, reducing duplicate boxes and improving localization accuracy.  \n- **Submission-ready pipeline:** Predictions are converted into Kaggle-compatible format (`conf xmin ymin w h`) and stored in a DataFrame.  \n- **Optional visualization:** Enables visual verification of predictions, helping debug or showcase model performance.\n\nThis modular design allows **easy extension** to new augmentations, different backbones, or updated models without modifying the core workflow.\n\n---\n\n# 📚 References\n\n- **KerasCV YOLOv8 Documentation:** [KerasCV Object Detection Guide](https://keras.io/api/keras_cv/models/yolo/)  \n  Provides guidance on **fine-tuning YOLOv8**, preprocessing images, handling bounding boxes, and visualization utilities.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}