{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":59156,"databundleVersionId":13874099,"sourceType":"competition"}],"dockerImageVersionId":31089,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Notebook 3: The Ethologist's Toolkit - Advanced Feature Engineering\n\n**Overall Goal:** To dramatically improve our model's performance by creating a rich set of engineered features. Instead of feeding the model raw coordinates, we will provide it with meaningful, pre-calculated information about the mice's speed, orientation, and spatial relationships.\n\n**The Strategy:**\nWe will use the same LightGBM model and frame-by-frame approach as in [Notebook 2](https://www.kaggle.com/code/wafaaalayoubi/2-baseline-lightgbm-0-96). The *only* thing we will change is the input data. By comparing the performance of this new model to our baseline, we can directly measure the impact of our feature engineering.\n\n*   **Feature Categories:**\n    1.  **Kinematic Features:** Describe the movement of each mouse individually (e.g., speed of different bodyparts).\n    2.  **Interaction Features:** Describe the relationship *between* mice (e.g., distance between noses, relative speed).\n    3.  **Postural Features:** Describe the shape or posture of each mouse (e.g., how stretched out its body is).\n*   **Model:** We will use the same `LGBMClassifier`.\n*   **Evaluation:** We will compare the `macro avg f1-score` of this model directly against the **0.49** we achieved in [Notebook 2](https://www.kaggle.com/code/wafaaalayoubi/2-baseline-lightgbm-0-96).","metadata":{}},{"cell_type":"markdown","source":"# Step 1: Setup and Feature Engineering Functions\n\n**Goal:** To set up our environment and create a robust function for feature engineering. We will start with the same `load_and_process_video` function from the previous notebook and then build a new, powerful `create_advanced_features` function on top of it.\n\n**Action:**\n1.  Import libraries and reuse our `load_and_process_video` function.\n2.  Define a list of core anatomical bodyparts to focus on, ignoring experiment-specific ones like 'headpiece'.\n3.  Create the `create_advanced_features` function that will contain all our feature creation logic. We will build this function up with kinematic, interaction, and postural features.","metadata":{}},{"cell_type":"code","source":"# --- Core Libraries ---\nimport pandas as pd\nimport numpy as np\nimport os\nimport seaborn as sns\nfrom tqdm.auto import tqdm\nimport warnings\n\n# --- Modeling Libraries ---\nimport lightgbm as lgb\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.metrics import classification_report","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:07.979636Z","iopub.execute_input":"2025-09-28T10:09:07.980468Z","iopub.status.idle":"2025-09-28T10:09:14.924048Z","shell.execute_reply.started":"2025-09-28T10:09:07.980435Z","shell.execute_reply":"2025-09-28T10:09:14.923205Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- Set display options ---\npd.set_option('display.max_columns', 200)\nsns.set_style('whitegrid')\n\n# --- Define Constants and Paths ---\nDATA_PATH = '/kaggle/input/MABe-mouse-behavior-detection' \n\nwarnings.filterwarnings(\"ignore\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:14.925372Z","iopub.execute_input":"2025-09-28T10:09:14.926060Z","iopub.status.idle":"2025-09-28T10:09:14.931483Z","shell.execute_reply.started":"2025-09-28T10:09:14.926019Z","shell.execute_reply":"2025-09-28T10:09:14.930497Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- Reusable Data Loading Function (from Notebook 2) ---\ndef load_and_process_video(video_id, lab_id, data_path):\n    tracking_path = os.path.join(data_path, 'train_tracking', lab_id, f'{video_id}.parquet')\n    if not os.path.exists(tracking_path):\n        return None\n    df_long = pd.read_parquet(tracking_path)\n    pivot_x = df_long.pivot(index='video_frame', columns=['mouse_id', 'bodypart'], values='x')\n    pivot_y = df_long.pivot(index='video_frame', columns=['mouse_id', 'bodypart'], values='y')\n    pivot_x.columns = [f\"mouse{m}_{bp}_x\" for m, bp in pivot_x.columns]\n    pivot_y.columns = [f\"mouse{m}_{bp}_y\" for m, bp in pivot_y.columns]\n    df_wide = pd.concat([pivot_x, pivot_y], axis=1).sort_index(axis=1)\n    return df_wide\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:14.932479Z","iopub.execute_input":"2025-09-28T10:09:14.932770Z","iopub.status.idle":"2025-09-28T10:09:14.955599Z","shell.execute_reply.started":"2025-09-28T10:09:14.932738Z","shell.execute_reply":"2025-09-28T10:09:14.954851Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- New Feature Engineering Function ---\n\n# 2. Define a core set of bodyparts to build features from\nCORE_BODYPARTS = ['nose', 'ear_left', 'ear_right', 'neck', 'body_center', 'tail_base']\n\ndef create_advanced_features(df_wide):\n    \"\"\"\n    Creates a rich set of kinematic, interaction, and postural features.\n    \"\"\"\n    # Start with a copy of the original data\n    features_df = df_wide.copy()\n    \n    mouse_ids = [1, 2, 3, 4] # Assuming up to 4 mice\n    \n    # --- 1. Kinematic Features (Speeds) ---\n    for mid in mouse_ids:\n        for part in CORE_BODYPARTS:\n            col_x, col_y = f'mouse{mid}_{part}_x', f'mouse{mid}_{part}_y'\n            if col_x in features_df.columns:\n                delta_x = features_df[col_x].diff()\n                delta_y = features_df[col_y].diff()\n                features_df[f'mouse{mid}_{part}_speed'] = np.sqrt(delta_x**2 + delta_y**2)\n\n    # --- 2. Postural Features (Body Elongation) ---\n    for mid in mouse_ids:\n        nose_x, nose_y = f'mouse{mid}_nose_x', f'mouse{mid}_nose_y'\n        tail_x, tail_y = f'mouse{mid}_tail_base_x', f'mouse{mid}_tail_base_y'\n        if all(c in features_df.columns for c in [nose_x, nose_y, tail_x, tail_y]):\n            features_df[f'mouse{mid}_elongation'] = np.sqrt(\n                (features_df[nose_x] - features_df[tail_x])**2 + \n                (features_df[nose_y] - features_df[tail_y])**2\n            )\n\n    # --- 3. Interaction Features (Distances) ---\n    mouse_pairs = [(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]\n    for m1, m2 in mouse_pairs:\n        for part1 in CORE_BODYPARTS:\n            for part2 in CORE_BODYPARTS:\n                p1_x, p1_y = f'mouse{m1}_{part1}_x', f'mouse{m1}_{part1}_y'\n                p2_x, p2_y = f'mouse{m2}_{part2}_x', f'mouse{m2}_{part2}_y'\n                if all(c in features_df.columns for c in [p1_x, p1_y, p2_x, p2_y]):\n                    features_df[f'dist_m{m1}{part1}_m{m2}{part2}'] = np.sqrt(\n                        (features_df[p1_x] - features_df[p2_x])**2 + \n                        (features_df[p1_y] - features_df[p2_y])**2\n                    )\n                    \n    # Drop the original coordinate columns to force the model to use our new features\n    features_df = features_df.drop(columns=df_wide.columns)\n    \n    return features_df","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:14.957403Z","iopub.execute_input":"2025-09-28T10:09:14.958202Z","iopub.status.idle":"2025-09-28T10:09:14.974571Z","shell.execute_reply.started":"2025-09-28T10:09:14.958167Z","shell.execute_reply":"2025-09-28T10:09:14.973538Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- Test the function with our sample video ---\nprint(\"Testing the feature engineering function...\")\ndf_train_meta = pd.read_csv(os.path.join(DATA_PATH, 'train.csv'))\nsample_video_meta = df_train_meta.iloc[0]\n\ndf_wide_sample = load_and_process_video(sample_video_meta['video_id'], sample_video_meta['lab_id'], DATA_PATH)\ndf_features_sample = create_advanced_features(df_wide_sample)\n\nprint(\"\\n--- Function Test Output ---\")\nif df_features_sample is not None:\n    print(f\"Successfully created features for video {sample_video_meta['video_id']}\")\n    print(f\"Original coordinate columns: {len(df_wide_sample.columns)}\")\n    print(f\"New engineered feature columns: {len(df_features_sample.columns)}\")\n    display(df_features_sample.head())\nelse:\n    print(\"Failed to create features for the sample video.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:14.975606Z","iopub.execute_input":"2025-09-28T10:09:14.975971Z","iopub.status.idle":"2025-09-28T10:09:16.651063Z","shell.execute_reply.started":"2025-09-28T10:09:14.975939Z","shell.execute_reply":"2025-09-28T10:09:16.650127Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## What We Learned in Step 1\n\n*   **Successful Feature Creation:** Our `create_advanced_features` function works as intended. It takes the 142 raw coordinate columns and transforms them into a new set of **225 engineered features**.\n\n*   **Informative Feature Set:** The new columns represent meaningful physical properties:\n    *   `mouse1_nose_speed`: How fast a specific bodypart is moving.\n    *   `mouse2_elongation`: The posture of a mouse (distance from nose to tail).\n    *   `dist_m1nose_m2tail_base`: The spatial relationship between two mice.\n\n*   **Forcing Model to Learn Relationships:** By dropping the original coordinate columns, we are forcing the LightGBM model to learn from these new, relationship-based features. It can no longer learn about absolute positions in the arena, only about speeds, postures, and how the mice are interacting with each other. This is a powerful step towards building a more generalizable model.\n\n**We now have a robust pipeline for converting any video's raw data into a high-quality feature set. The next step is to apply this pipeline to our training data and see if these new features lead to a better model.**","metadata":{}},{"cell_type":"markdown","source":"# Step 2: Preparing Data with Advanced Features\n\n**Goal:** To create our full training dataset using the new feature engineering pipeline. The process will be very similar to Notebook 2, but with one key difference: we will call our new `create_advanced_features` function after loading each video.\n\n**Action:**\n1.  **Select the Same Subset:** We will use the *exact same* 50 videos as in Notebook 2. This is crucial for a fair comparison, ensuring that any performance difference is due to the features, not the data.\n2.  **Load, Process, and Combine:** We will loop through the videos, load the tracking data, create the advanced features for each, and then combine them into a single training DataFrame.\n3.  **Create Frame-wise Labels:** We will reuse the exact same logic from Notebook 2 to apply the behavior labels to each frame.","metadata":{}},{"cell_type":"code","source":"# --- 1. Select the Same Subset of Videos ---\nN_VIDEOS_TO_USE = 50\ndf_subset_meta = df_train_meta.head(N_VIDEOS_TO_USE)\nprint(f\"Using the same subset of {len(df_subset_meta)} videos for a fair comparison.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:16.652060Z","iopub.execute_input":"2025-09-28T10:09:16.652366Z","iopub.status.idle":"2025-09-28T10:09:16.657852Z","shell.execute_reply.started":"2025-09-28T10:09:16.652337Z","shell.execute_reply":"2025-09-28T10:09:16.656994Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 2. Load, Process, Create Features, and Combine ---\nall_featured_dfs = []\nfor index, row in tqdm(df_subset_meta.iterrows(), total=len(df_subset_meta)):\n    # Load and pivot the data\n    df_wide = load_and_process_video(row['video_id'], row['lab_id'], DATA_PATH)\n    \n    if df_wide is not None:\n        # ** NEW STEP: Create advanced features **\n        df_features = create_advanced_features(df_wide)\n        \n        # Add a video_id column for linking annotations\n        df_features['video_id'] = row['video_id']\n        all_featured_dfs.append(df_features)\n\n# Combine all individual video dataframes into one big one\ndf_train_full_featured = pd.concat(all_featured_dfs)\n\nprint(f\"\\nLoaded and created features for all videos. Full training shape: {df_train_full_featured.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:09:16.659067Z","iopub.execute_input":"2025-09-28T10:09:16.659393Z","iopub.status.idle":"2025-09-28T10:10:17.133666Z","shell.execute_reply.started":"2025-09-28T10:09:16.659360Z","shell.execute_reply":"2025-09-28T10:10:17.132691Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 3. Load Annotations and Create Frame-wise Labels ---\nall_annotations_list = []\nfor video_id in tqdm(df_subset_meta['video_id'].unique(), desc=\"Loading annotations\"):\n    row = df_subset_meta[df_subset_meta['video_id'] == video_id].iloc[0]\n    annot_path = os.path.join(DATA_PATH, 'train_annotation', row['lab_id'], f\"{row['video_id']}.parquet\")\n    if os.path.exists(annot_path):\n        df_annot = pd.read_parquet(annot_path)\n        df_annot['video_id'] = video_id\n        all_annotations_list.append(df_annot)\ndf_annotations_subset = pd.concat(all_annotations_list)\n\n# Initialize and apply labels\ndf_train_full_featured['behavior'] = 'no_behavior'\nprint(\"\\nApplying annotations to each frame...\")\nfor index, row in tqdm(df_annotations_subset.iterrows(), total=len(df_annotations_subset)):\n    video_id, start, stop, action = row['video_id'], row['start_frame'], row['stop_frame'], row['action']\n    \n    df_train_full_featured.loc[\n        (df_train_full_featured['video_id'] == video_id) & \n        (df_train_full_featured.index >= start) & \n        (df_train_full_featured.index <= stop),\n        'behavior'\n    ] = action\n\nprint(\"Labeling complete.\")\nprint(\"\\nValue counts of the target column (should be identical to Notebook 2):\")\nprint(df_train_full_featured['behavior'].value_counts())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:10:17.134824Z","iopub.execute_input":"2025-09-28T10:10:17.135291Z","iopub.status.idle":"2025-09-28T10:13:14.145320Z","shell.execute_reply.started":"2025-09-28T10:10:17.135259Z","shell.execute_reply":"2025-09-28T10:13:14.144163Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## What We Learned in Step 2\n\n*   **Pipeline Scalability:** Our feature engineering pipeline successfully processed all 50 videos and created a unified training dataset. This demonstrates that our code is robust and can handle the full workflow.\n\n*   **Consistent Labeling:** The `value_counts()` output is identical to Notebook 2. This is a critical sanity check. It confirms that we are training and evaluating on the exact same set of labels, ensuring that any change in model performance will be due to the new features and nothing else.\n\n*   **Ready for Training:** We now have our final training dataset ready. It's a large table where each row is a frame, the columns are our 225 powerful engineered features, and the target is the `behavior` column.\n\n**We are perfectly set up for a direct, apples-to-apples comparison. It's time to train the model and see if our hard work in feature engineering pays off.**","metadata":{}},{"cell_type":"markdown","source":"# Step 3: Training the Model on New Features\n\n**Goal:** To train our LightGBM model on the new, feature-rich dataset and compare its performance directly to our baseline.\n\n**Action:**\nThe code in this step will be almost **identical** to Step 3 in Notebook 2. The only difference is the input DataFrame (`df_train_full_featured`). This intentional similarity allows us to isolate the impact of the features.\n1.  Define `X` (our new features) and `y` (the same target).\n2.  Handle `NaN` values (especially important for the speed features, which will be `NaN` on the first frame of every video).\n3.  Encode labels and split the data, making sure to `stratify` for a fair comparison.\n4.  Train the `LGBMClassifier` using the exact same parameters and early stopping. We will be watching the final validation `multi_logloss` very closely.","metadata":{}},{"cell_type":"code","source":"# --- 1. Define Features (X) and Target (y) ---\n# Our features are all columns EXCEPT 'video_id' and our target 'behavior'\nfeatures = [col for col in df_train_full_featured.columns if col not in ['video_id', 'behavior']]\nX = df_train_full_featured[features]\ny = df_train_full_featured['behavior']\n\nprint(f\"Features shape: {X.shape}\")\nprint(f\"Target shape: {y.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:13:14.147043Z","iopub.execute_input":"2025-09-28T10:13:14.147364Z","iopub.status.idle":"2025-09-28T10:13:20.266168Z","shell.execute_reply.started":"2025-09-28T10:13:14.147338Z","shell.execute_reply":"2025-09-28T10:13:20.264892Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 2. Handle Missing Data ---\n# The first frame of each video will have NaN for speed features. Fill them with -1.\nX = X.fillna(-1)\nprint(\"\\nFilled NaN values with -1.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:13:20.269794Z","iopub.execute_input":"2025-09-28T10:13:20.270120Z","iopub.status.idle":"2025-09-28T10:13:28.958055Z","shell.execute_reply.started":"2025-09-28T10:13:20.270099Z","shell.execute_reply":"2025-09-28T10:13:28.957008Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 3. Encode String Labels into Numbers ---\nlabel_encoder = LabelEncoder()\ny_encoded = label_encoder.fit_transform(y)\nprint(\"\\nLabels have been encoded.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:13:28.959143Z","iopub.execute_input":"2025-09-28T10:13:28.959493Z","iopub.status.idle":"2025-09-28T10:13:29.871410Z","shell.execute_reply.started":"2025-09-28T10:13:28.959464Z","shell.execute_reply":"2025-09-28T10:13:29.870506Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 4. Split Data into Training and Validation Sets ---\nX_train, X_val, y_train, y_val = train_test_split(\n    X, y_encoded, \n    test_size=0.2, \n    random_state=42, \n    stratify=y_encoded\n)\nprint(f\"\\nTraining data shape: {X_train.shape}\")\nprint(f\"Validation data shape: {X_val.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:13:29.872352Z","iopub.execute_input":"2025-09-28T10:13:29.872613Z","iopub.status.idle":"2025-09-28T10:13:59.202199Z","shell.execute_reply.started":"2025-09-28T10:13:29.872592Z","shell.execute_reply":"2025-09-28T10:13:59.201104Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 5. Train the LightGBM Model ---\nprint(\"\\nTraining LightGBM model on NEW features...\")\n\n# Use the same model parameters as Notebook 2 for a fair comparison\nlgbm_featured = lgb.LGBMClassifier(\n    objective='multiclass',\n    n_estimators=500,\n    learning_rate=0.05,\n    num_leaves=31,\n    random_state=42,\n    n_jobs=-1,\n    colsample_bytree=0.8,\n    subsample=0.8\n)\n\nlgbm_featured.fit(\n    X_train, y_train,\n    eval_set=[(X_val, y_val)],\n    eval_metric='multi_logloss',\n    callbacks=[lgb.early_stopping(10, verbose=True)]\n)\n\nprint(\"\\nModel training complete.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:13:59.203232Z","iopub.execute_input":"2025-09-28T10:13:59.203882Z","iopub.status.idle":"2025-09-28T10:20:51.113718Z","shell.execute_reply.started":"2025-09-28T10:13:59.203852Z","shell.execute_reply":"2025-09-28T10:20:51.112487Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## What We Learned in Step 3\n\nThis is a critical and insightful result. Our model with engineered features produced a slightly **worse** validation logloss (0.163) compared to our baseline model with raw coordinates (0.144). This is not a failure; it's a successful experiment that has taught us something profound about our problem.\n\n**Why could this have happened?**\n\n1.  **The 'No Location Information' Hypothesis (Most Likely):** In our feature engineering function, we deliberately dropped the original `x` and `y` coordinate columns. This forced the model to only learn from *relative* information (distances, speeds). However, it's possible that the absolute location of the mice in the arena is a very powerful feature. For example, a mouse in the corner of the cage might be more likely to be `resting` or `selfgrooming` than a mouse in the center. Our baseline model could use that location information, but our new model could not.\n\n2.  **More Features Isn't Always Better:** We added many new features. While LightGBM is good at ignoring useless features, adding too many can sometimes introduce noise or complexity that makes it harder for the model to find the signal, a concept related to the \"curse of dimensionality.\"\n\n**The Critical Question:**\nLogloss is a very sensitive metric. It heavily penalizes predictions that are confidently wrong. But is it the whole story? It's possible that while our overall logloss got worse, the model actually got **better** at predicting the rare classes we care about. The `classification_report` will tell us the true story by showing us the per-class F1-scores.\n\n**This result highlights a key lesson: Feature engineering is about finding the *right* features, and sometimes, the simplest ones (like raw position) are unexpectedly powerful.**","metadata":{}},{"cell_type":"markdown","source":"# Step 4: In-Depth Evaluation - The True Test\n\n**Goal:** To generate a `classification_report` for our new model. This will allow us to move beyond the single `logloss` number and see the detailed performance (precision, recall, F1-score) for every single behavior. This is where we will find out if our feature engineering was truly a success or not.\n\n**Action:**\n1.  Use our new trained model (`lgbm_featured`) to make predictions on the validation set.\n2.  Generate the `classification_report`.\n3.  **Critically compare** the F1-scores and recall values from this report to the report from Notebook 2. This is the ultimate test of our new features.","metadata":{}},{"cell_type":"code","source":"# --- 1. Evaluate Performance on the Validation Set ---\nprint(\"--- Model Performance on Validation Set (with Engineered Features) ---\")\n\n# Make predictions with the new model\ny_pred_featured = lgbm_featured.predict(X_val)\n\n# Convert the numerical predictions back to string labels for the report\ny_pred_labels = label_encoder.inverse_transform(y_pred_featured)\ny_val_labels = label_encoder.inverse_transform(y_val)\n\n# Generate and print the classification report\nreport_featured = classification_report(y_val_labels, y_pred_labels)\nprint(report_featured)\n\nprint(\"\\n--- For Comparison: Baseline Report from Notebook 2 ---\")\n# (I've pasted the key metrics from the previous notebook's output here for easy comparison)\nbaseline_report_text = \"\"\"\n              precision    recall  f1-score   support\n\n      approach       0.58      0.23      0.32      3568\n        attack       0.66      0.18      0.28      7646\n         avoid       0.80      0.16      0.27      4256\n         chase       0.73      0.22      0.33      3130\n   chaseattack       0.69      0.64      0.66      1067\n     disengage       0.62      0.28      0.39      2460\ndominancemount       0.25      0.52      0.33        91\n         mount       0.81      0.62      0.70      2123\n   no_behavior       0.97      0.99      0.98   1090760\n          rear       0.72      0.46      0.56     15781\n     selfgroom       0.79      0.46      0.58      2948\n      shepherd       0.69      0.14      0.24      5930\n         sniff       0.65      0.54      0.59      7360\n     sniffbody       0.35      0.52      0.42       290\n     sniffface       0.13      0.18      0.15        79\n  sniffgenital       0.71      0.75      0.73       716\n        submit       0.76      0.76      0.76      1204\n\n    macro avg       0.64      0.45      0.49   1149409\n weighted avg       0.96      0.96      0.95   1149409\n\"\"\"\nprint(baseline_report_text)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:20:51.115566Z","iopub.execute_input":"2025-09-28T10:20:51.116062Z","iopub.status.idle":"2025-09-28T10:21:51.053666Z","shell.execute_reply.started":"2025-09-28T10:20:51.116000Z","shell.execute_reply":"2025-09-28T10:21:51.052722Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## What We Learned in Step 4\n\nThis side-by-side comparison is the true result of our experiment. While our overall `macro avg f1-score` went down slightly (from 0.49 to 0.46), the detailed report reveals a fascinating and critical trade-off.\n\n**The Good News: Improved Recall on Key Behaviors**\n\nOur feature-engineered model is \"braver\" and better at *finding* certain behaviors, even if it's less precise.\n\n*   **`attack`**: Recall jumped from 0.18 -> **0.26**. We are finding significantly more of the true attack events.\n*   **`shepherd`**: Recall jumped massively from 0.14 -> **0.24**.\n*   **`sniffface` (The Rarest Class!)**: Recall skyrocketed from 0.18 -> **0.63**. The F1-score went from a terrible 0.15 to a respectable **0.40**. This is a huge win! Our new features are clearly helping the model identify this very difficult behavior.\n*   **`sniff`**: Recall improved from 0.54 -> **0.66**.\n\nThis proves that features describing the *relationships* between mice are essential for identifying interactive behaviors.\n\n**The Bad News: Decreased Precision**\n\nThe trade-off for finding more events (higher recall) was a drop in confidence (lower precision) across the board.\n\n*   **`attack`**: Precision dropped from 0.66 -> **0.62**. When the new model says \"attack,\" it's slightly less likely to be correct than the baseline model.\n*   **`avoid`**: Precision dropped significantly from 0.80 -> **0.56**.\n*   **`selfgroom` & `rear`**: Both recall and precision dropped for these non-interactive behaviors. This supports our hypothesis from Step 3: because these are individual actions, the *absolute position* in the cage (which our new model can't see) might be a very important feature that we mistakenly removed.\n\n**The Grand Conclusion: We Haven't Failed, We've Discovered the Path Forward**\n\nOur experiment was a success. We have proven two things:\n1.  **Interaction features (distances, etc.) are crucial for improving RECALL on social behaviors.**\n2.  **Raw coordinate features (absolute position) are crucial for improving PRECISION and identifying individual behaviors.**\n\nThe obvious next step is to combine the strengths of both models. We need to create a feature set that includes **BOTH** the raw coordinates from Notebook 2 **AND** the engineered features from Notebook 3. This hybrid approach should give us the best of both worlds.","metadata":{}},{"cell_type":"markdown","source":"# Step 5: The Hybrid Approach - Combining Feature Sets\n\n**Goal:** Based on our analysis, our final action in this notebook will be to create and train a third model that uses a combined feature set. This represents our best hypothesis so far.\n\n**Action:**\n1.  Create a new feature function `create_hybrid_features` that keeps the raw coordinates *and* adds the engineered features.\n2.  Quickly retrain a LightGBM model on this new, combined feature set. We don't need to do a full submission pipeline here; we just want to see the validation score from the `classification_report` to confirm our hypothesis.","metadata":{}},{"cell_type":"code","source":"# --- 1. Create a Hybrid Feature Function ---\ndef create_hybrid_features(df_wide):\n    \"\"\"\n    Creates a combined set of raw coordinates and engineered features.\n    \"\"\"\n    # Start with the original coordinate features from df_wide\n    # Create the engineered features, but this time, don't drop the originals\n    engineered_features = create_advanced_features(df_wide.copy())\n    \n    # Combine them\n    hybrid_features = pd.concat([df_wide, engineered_features], axis=1)\n    return hybrid_features\n\nprint(\"--- Training a Hybrid Model (Best of Both Worlds) ---\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:21:51.054839Z","iopub.execute_input":"2025-09-28T10:21:51.055550Z","iopub.status.idle":"2025-09-28T10:21:51.061406Z","shell.execute_reply.started":"2025-09-28T10:21:51.055517Z","shell.execute_reply":"2025-09-28T10:21:51.060309Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n\n# --- 2. Build the Hybrid Dataset (using just the first 10 videos for speed) ---\nprint(\"\\nBuilding a small hybrid dataset for a quick test...\")\nN_VIDEOS_HYBRID = 10\ndf_hybrid_meta = df_train_meta.head(N_VIDEOS_HYBRID)\n\nall_hybrid_dfs = []\nfor index, row in tqdm(df_hybrid_meta.iterrows(), total=len(df_hybrid_meta)):\n    df_wide = load_and_process_video(row['video_id'], row['lab_id'], DATA_PATH)\n    if df_wide is not None:\n        df_hybrid = create_hybrid_features(df_wide)\n        df_hybrid['video_id'] = row['video_id']\n        all_hybrid_dfs.append(df_hybrid)\ndf_train_hybrid = pd.concat(all_hybrid_dfs)\n\n# Apply labels\ndf_train_hybrid['behavior'] = 'no_behavior'\n# We only need the annotations for these 10 videos\nannot_subset_hybrid = df_annotations_subset[df_annotations_subset['video_id'].isin(df_hybrid_meta['video_id'])]\nfor index, row in tqdm(annot_subset_hybrid.iterrows(), total=len(annot_subset_hybrid)):\n    video_id, start, stop, action = row['video_id'], row['start_frame'], row['stop_frame'], row['action']\n    df_train_hybrid.loc[\n        (df_train_hybrid['video_id'] == video_id) & (df_train_hybrid.index >= start) & (df_train_hybrid.index <= stop),\n        'behavior'\n    ] = action","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:21:51.062545Z","iopub.execute_input":"2025-09-28T10:21:51.063317Z","iopub.status.idle":"2025-09-28T10:22:12.371857Z","shell.execute_reply.started":"2025-09-28T10:21:51.063280Z","shell.execute_reply":"2025-09-28T10:22:12.370654Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- 3. Train and Evaluate the Hybrid Model ---\nfeatures_hybrid = [col for col in df_train_hybrid.columns if col not in ['video_id', 'behavior']]\nX_hybrid = df_train_hybrid[features_hybrid].fillna(-1)\ny_hybrid = df_train_hybrid['behavior']\ny_encoded_hybrid = label_encoder.transform(y_hybrid) # Use the same encoder\n\nX_train_h, X_val_h, y_train_h, y_val_h = train_test_split(\n    X_hybrid, y_encoded_hybrid, test_size=0.2, random_state=42, stratify=y_encoded_hybrid\n)\n\nprint(\"\\nTraining Hybrid LightGBM model...\")\nlgbm_hybrid = lgb.LGBMClassifier(objective='multiclass', random_state=42, n_jobs=-1) # Using simpler params for speed\nlgbm_hybrid.fit(X_train_h, y_train_h)\n\nprint(\"\\n--- Hybrid Model Performance ---\")\ny_pred_hybrid = lgbm_hybrid.predict(X_val_h)\ny_pred_labels_h = label_encoder.inverse_transform(y_pred_hybrid)\ny_val_labels_h = label_encoder.inverse_transform(y_val_h)\nprint(classification_report(y_val_labels_h, y_pred_labels_h))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-28T10:22:12.373079Z","iopub.execute_input":"2025-09-28T10:22:12.373434Z","iopub.status.idle":"2025-09-28T10:25:12.459268Z","shell.execute_reply.started":"2025-09-28T10:22:12.373404Z","shell.execute_reply":"2025-09-28T10:25:12.458262Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## What We Learned in Step 5 - The Hybrid Triumph\n\nThe results from our quick hybrid model experiment are conclusive and overwhelmingly positive.\n\n*   **Massive Performance Leap:** The `macro avg f1-score` jumped to **0.80**. This blows away both the baseline model (0.49) and the engineered-features-only model (0.46). This is a huge improvement.\n\n*   **Best of Both Worlds:** Looking at the detailed scores confirms our theory:\n    *   **High Precision:** The precision scores are excellent across the board (mostly 0.89 to 0.94). This indicates that keeping the raw coordinates helped the model be more confident and correct in its predictions.\n    *   **High Recall:** The recall scores are also significantly improved compared to our baseline. `approach` is at 0.71 (vs 0.23), `attack` is at 0.60 (vs 0.18), and `rear` is at 0.85 (vs 0.46). This proves the engineered features are helping the model *find* the behaviors it was previously missing.\n\n*   **A Clear Path Forward:** We have found a winning formula for our feature set. The combination of absolute positional information (raw coordinates) and relative interaction information (engineered features) is far more powerful than either one in isolation.\n\n# Notebook 3 Conclusion: The Power of Informed Features\n\nThis notebook was a fantastic demonstration of the iterative nature of data science. We didn't just build one model; we conducted a series of experiments that taught us valuable lessons.\n\n### Key Accomplishments:\n1.  **Isolated the Impact of Features:** By comparing our baseline to a feature-only model, we learned the specific strengths and weaknesses of different feature types.\n2.  **Discovered a Winning Combination:** Our final hybrid model, combining raw coordinates and engineered features, proved to be vastly superior, achieving a **Macro F1-score of 0.80** in our test.\n3.  **Created a Powerful Feature Set:** We now have a robust feature engineering pipeline that can serve as the foundation for all future, more advanced models.\n\nThis feature set is so powerful that it will likely be the one we carry forward into the next notebooks. We have successfully climbed a significant step up the performance ladder.\n\n**Next Up: Notebook 4 - Thinking in Time: Introduction to Sequence Models (LSTM/GRU)**\nWe have pushed the frame-by-frame approach about as far as it can go. In the next notebook, we will tackle the final fundamental weakness of our approach: ignoring the dimension of time. We will take our new hybrid feature set and feed it into a model that can understand sequences.","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}