{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":102335,"databundleVersionId":12518947,"sourceType":"competition"}],"dockerImageVersionId":31089,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"a519fecf","cell_type":"markdown","source":"# 🧠 CMI Sensor Data EDA – Let’s Spy on the Sensors!","metadata":{}},{"id":"78f4dcf2-9163-49ff-b7fe-299816d326f7","cell_type":"markdown","source":"## Section 1: ⬇️ Introduction\n\nWelcome to the **Exploratory Data Analysis** of the CMI competition: *Detect Behavior with Sensor Data*.\n\nIn this notebook, we'll:\n\n- Load the data (without waking it up too hard)\n- Understand its structure\n- Check for missing values\n- Explore gestures and demographics\n- Peek into sensor patterns\n- Prepare for future modeling (we’re getting serious 😎)\n","metadata":{}},{"id":"4488b51c-57d7-43e7-923f-676829c868c5","cell_type":"markdown","source":"## Section 2: 📦 Importing the Nerdy Essentials\nLet’s start by bringing in our usual crew of Python data whisperers.\n","metadata":{}},{"id":"72ef69f9-13c4-4d41-ba36-e0a2471cc083","cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom pathlib import Path\nfrom tqdm.notebook import tqdm\n\npd.set_option('display.max_columns', 400)","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:38:58.337102Z","iopub.execute_input":"2025-07-24T11:38:58.338143Z","iopub.status.idle":"2025-07-24T11:38:59.858893Z","shell.execute_reply.started":"2025-07-24T11:38:58.338096Z","shell.execute_reply":"2025-07-24T11:38:59.857895Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"4c812080-69ad-40b1-a3a9-fe8465f81e5e","cell_type":"markdown","source":"## Section 3: 📁 Loading the Data\nNow, let’s open the files and see what we’re working with. No judgment yet.\n","metadata":{}},{"id":"97da79f4-4a15-4ddc-9eec-98768bc87bdf","cell_type":"code","source":"DATA_DIR = Path(\"/kaggle/input/cmi-detect-behavior-with-sensor-data\")\nassert os.path.exists(DATA_DIR)\n\ntrain_df = pd.read_csv(DATA_DIR / \"train.csv\")\ntrain_demo_df = pd.read_csv(DATA_DIR / \"train_demographics.csv\")\n\nprint(\"👉 train.csv shape:\", train_df.shape)\nprint(\"👉 train_demographics.csv shape:\", train_demo_df.shape)\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:38:59.860722Z","iopub.execute_input":"2025-07-24T11:38:59.861311Z","iopub.status.idle":"2025-07-24T11:39:38.156542Z","shell.execute_reply.started":"2025-07-24T11:38:59.861274Z","shell.execute_reply":"2025-07-24T11:39:38.155347Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"ffb2e1f2-b861-4e37-8163-3b507716009c","cell_type":"markdown","source":"## Section 4: 👀 Sneak Peek at the Sensor Data\nA random look at a few rows, just to feel the data’s vibe.\n","metadata":{}},{"id":"39c028b7-0056-496e-ac8f-fc0aefc02b54","cell_type":"code","source":"train_df.sample(3)","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:39:38.157533Z","iopub.execute_input":"2025-07-24T11:39:38.157806Z","iopub.status.idle":"2025-07-24T11:39:38.414304Z","shell.execute_reply.started":"2025-07-24T11:39:38.157782Z","shell.execute_reply":"2025-07-24T11:39:38.412818Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"33d9a959-58e0-42b0-b12d-be93730080ef","cell_type":"markdown","source":"## Section 5: 🔍 Basic Info + Missing Data Check\nLet’s investigate how messy (or clean) our data really is.\n","metadata":{}},{"id":"3bb4506d-8337-49a3-aa0d-5aa3f34296ce","cell_type":"code","source":"train_df.info()\ntrain_df.isnull().sum().sort_values(ascending=False).head(20)","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:39:38.416818Z","iopub.execute_input":"2025-07-24T11:39:38.417190Z","iopub.status.idle":"2025-07-24T11:39:39.345820Z","shell.execute_reply.started":"2025-07-24T11:39:38.417157Z","shell.execute_reply":"2025-07-24T11:39:39.344882Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"1152d67f-496f-43ed-a8e1-13a4a98956a5","cell_type":"code","source":"train_df.describe(include='all')","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:39:39.346885Z","iopub.execute_input":"2025-07-24T11:39:39.347266Z","iopub.status.idle":"2025-07-24T11:39:50.829211Z","shell.execute_reply.started":"2025-07-24T11:39:39.347234Z","shell.execute_reply":"2025-07-24T11:39:50.828059Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"279e45b9-2e7d-4859-a2d5-04fa03e170f3","cell_type":"code","source":"train_df.describe(include='all').T","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:39:50.830395Z","iopub.execute_input":"2025-07-24T11:39:50.830716Z","iopub.status.idle":"2025-07-24T11:40:01.859723Z","shell.execute_reply.started":"2025-07-24T11:39:50.830689Z","shell.execute_reply":"2025-07-24T11:40:01.858854Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"6ff34676-266b-4a00-ae38-b155c1aa504d","cell_type":"markdown","source":"## Section 6: 🧼 Visualizing Missingness\nMissing values are like awkward silences — better if visualized and understood.\n","metadata":{}},{"id":"c9476b96-7d0d-4235-864d-be2ec402241f","cell_type":"code","source":"missing_cols = train_df.columns[train_df.isnull().any()]\nmissing_summary = train_df[missing_cols].isnull().mean().sort_values(ascending=False)\n\nplt.figure(figsize=(12,5))\nmissing_summary.plot(kind='bar', color='coral')\nplt.title(\"Columns with Missing Data (%)\")\nplt.ylabel(\"Fraction Missing\")\nplt.xticks(rotation=90)\nplt.grid(axis='y', linestyle='--', alpha=0.5)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:01.860854Z","iopub.execute_input":"2025-07-24T11:40:01.861222Z","iopub.status.idle":"2025-07-24T11:40:06.438624Z","shell.execute_reply.started":"2025-07-24T11:40:01.861187Z","shell.execute_reply":"2025-07-24T11:40:06.437699Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0d7f0562-7847-453d-b337-f3265e1f3a4c","cell_type":"markdown","source":"## Section 7: 🙋 Gesture Distribution\nLet’s see how often each gesture shows up. Are some more popular than others?\n","metadata":{}},{"id":"c39a006f-ac3c-4a31-b153-94f28686ec2d","cell_type":"code","source":"gesture_counts = train_df.groupby(['gesture'])['sequence_id'].nunique().sort_values(ascending=False)\n\nplt.figure(figsize=(12,6))\nsns.barplot(y=gesture_counts.index, x=gesture_counts.values, palette=\"viridis\")\nplt.title(\"Gesture Distribution (by sequence count)\")\nplt.xlabel(\"Number of sequences\")\nplt.ylabel(\"Gesture label\")\nplt.grid(axis='x', linestyle='--', alpha=0.6)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:06.440023Z","iopub.execute_input":"2025-07-24T11:40:06.440414Z","iopub.status.idle":"2025-07-24T11:40:06.854491Z","shell.execute_reply.started":"2025-07-24T11:40:06.440376Z","shell.execute_reply":"2025-07-24T11:40:06.853579Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"beee179c-4e6c-4fd8-8e2e-0a460693bb2c","cell_type":"markdown","source":"## Section 8: 🧬 Who Are the People Behind the Gestures? (Demographics)\nA brief look into the participants and their stats — height, age, handedness, etc.\n","metadata":{}},{"id":"51ce79f8-fb63-4d38-a11d-bd1b0b744525","cell_type":"code","source":"train_demo_df.describe(include='all').T","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:06.855949Z","iopub.execute_input":"2025-07-24T11:40:06.856766Z","iopub.status.idle":"2025-07-24T11:40:06.888314Z","shell.execute_reply.started":"2025-07-24T11:40:06.856725Z","shell.execute_reply":"2025-07-24T11:40:06.887438Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"e6d1c8e0-8453-4040-a645-b181f591cf6d","cell_type":"markdown","source":"## Section 9: 🧩 Merging Demographics with Sensor Data\nLet’s glue the demographics info to our main dataset so we can play with it later.\n","metadata":{}},{"id":"b663ac0c-9f2d-4fb2-8fa5-32c08b07002c","cell_type":"code","source":"train_merged = train_df.merge(train_demo_df, on=\"subject\", how=\"left\")\nprint(\"Merged dataset shape:\", train_merged.shape)\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:06.891410Z","iopub.execute_input":"2025-07-24T11:40:06.891735Z","iopub.status.idle":"2025-07-24T11:40:08.009282Z","shell.execute_reply.started":"2025-07-24T11:40:06.891707Z","shell.execute_reply":"2025-07-24T11:40:08.008152Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"946ae713-239b-4a45-b749-9bed66ba64a3","cell_type":"code","source":"train_merged.info()\ntrain_merged.describe(include='all').T","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:08.010541Z","iopub.execute_input":"2025-07-24T11:40:08.010911Z","iopub.status.idle":"2025-07-24T11:40:18.924121Z","shell.execute_reply.started":"2025-07-24T11:40:08.010882Z","shell.execute_reply":"2025-07-24T11:40:18.923038Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"2041b0d5-8884-4af2-8d4e-db98f142d955","cell_type":"markdown","source":"## Section 10: 📈 IMU Feature Summary\nSummarize sensor readings (acceleration + rotation) for each gesture sequence.\n","metadata":{}},{"id":"bd65be4e-de22-462e-9de3-3efad3c4e769","cell_type":"code","source":"imu_features = ['acc_x', 'acc_y', 'acc_z', 'rot_w', 'rot_x', 'rot_y', 'rot_z']\n\nimu_stats = train_merged.groupby(\"sequence_id\")[imu_features].agg(\n    ['mean', 'std', 'min', 'max', 'skew']\n)\n\nimu_stats.columns = ['_'.join(col).strip() for col in imu_stats.columns.values]\nimu_stats.reset_index(inplace=True)\n\nimu_stats.sample(3)\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:18.925291Z","iopub.execute_input":"2025-07-24T11:40:18.925691Z","iopub.status.idle":"2025-07-24T11:40:19.213490Z","shell.execute_reply.started":"2025-07-24T11:40:18.925658Z","shell.execute_reply":"2025-07-24T11:40:19.212188Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"312b7abb-7634-4d5d-a6fa-3aa243065bc0","cell_type":"markdown","source":"## Section 11: 🔦 ToF & Thermopile Sensors – Are They Alive?\nSome sensors may be ghosting us. Let's check how many are missing or inactive.\n","metadata":{}},{"id":"d2b49558-c017-4a03-83d5-3caab7416c32","cell_type":"code","source":"tof_cols = [col for col in train_df.columns if col.startswith(\"tof_\")]\nthm_cols = [col for col in train_df.columns if col.startswith(\"thm_\")]\n\nprint(\"🔹 ToF Columns:\", len(tof_cols), \"🔹 Thermopile Columns:\", len(thm_cols))\n\ntof_nulls = train_df[tof_cols].isnull().mean().mean()\nthm_nulls = train_df[thm_cols].isnull().mean().mean()\n\nprint(f\"💀 Avg Missing (ToF): {tof_nulls:.2%}\")\nprint(f\"💀 Avg Missing (Thermopile): {thm_nulls:.2%}\")\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:19.214753Z","iopub.execute_input":"2025-07-24T11:40:19.215122Z","iopub.status.idle":"2025-07-24T11:40:20.874623Z","shell.execute_reply.started":"2025-07-24T11:40:19.215089Z","shell.execute_reply":"2025-07-24T11:40:20.873619Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"f1cd3174-a74a-409b-9e84-99aff983dee4","cell_type":"markdown","source":"## Section 12: ⏱ Sequence Lengths – How Long is a Gesture?\nGestures vary in length. Let’s see how many timestamps each sequence has.\n","metadata":{}},{"id":"9d139613-c580-4e81-b5e5-1f55e1305aff","cell_type":"code","source":"seq_lengths = train_df.groupby(\"sequence_id\")[\"sequence_counter\"].max() + 1\n\nplt.hist(seq_lengths, bins=30, color=\"mediumseagreen\", edgecolor=\"black\")\nplt.title(\"⏳ Sequence Lengths\")\nplt.xlabel(\"Number of timestamps\")\nplt.ylabel(\"Count\")\nplt.grid(True, linestyle='--', alpha=0.6)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:20.875829Z","iopub.execute_input":"2025-07-24T11:40:20.876141Z","iopub.status.idle":"2025-07-24T11:40:21.164321Z","shell.execute_reply.started":"2025-07-24T11:40:20.876116Z","shell.execute_reply":"2025-07-24T11:40:21.163169Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"f1e271ae-5f25-4fd4-9392-78af899fcc62","cell_type":"markdown","source":"## Section 12+1 : 🎬 Visualizing a Gesture in Detail\nLet’s pick one sequence, zoom into its sensor signals, convert rotation to pitch-roll-yaw, and even draw a ToF heatmap!\n","metadata":{}},{"id":"6e23ce06-d271-4661-bcdc-6c90d7ca9765","cell_type":"code","source":"import random\nfrom scipy.spatial.transform import Rotation as R\nimport matplotlib.patches as mpatches\nimport cv2  # for image resizing\n\n# Pick a random sequence\nselected_sequence = random.choice(train_df['sequence_id'].unique())\nprint(f\"🎯 Selected sequence: {selected_sequence}\")\n\nseq_df = train_df[train_df['sequence_id'] == selected_sequence].copy()\n\n# Reset index for plotting\nseq_df.reset_index(drop=True, inplace=True)\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:21.165343Z","iopub.execute_input":"2025-07-24T11:40:21.165626Z","iopub.status.idle":"2025-07-24T11:40:21.476257Z","shell.execute_reply.started":"2025-07-24T11:40:21.165603Z","shell.execute_reply":"2025-07-24T11:40:21.474956Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"29e72dcc-c773-43fe-b29f-30b83ec72272","cell_type":"markdown","source":"## Section 14: 🌡️ Thermopile Trends – Global Heat Over Time\n\nUnlike ToF sensors, Thermopile values are single global readings per sensor.  \nLet’s plot their changes over time to catch any hot events 🔥.\n","metadata":{}},{"id":"c470d2eb-5558-462d-a64e-f4e608d40134","cell_type":"code","source":"# Thermopile columns\nthm_cols = [col for col in seq_df.columns if col.startswith(\"thm_\")]\n\n# Plot them over time\nplt.figure(figsize=(14, 5))\nfor col in thm_cols:\n    plt.plot(seq_df[\"sequence_counter\"], seq_df[col], label=col)\n\nplt.title(f\"Thermopile Sensor Readings – Sequence {selected_sequence}\")\nplt.xlabel(\"Timestep\")\nplt.ylabel(\"Temperature\")\nplt.legend()\nplt.grid(True, linestyle='--', alpha=0.6)\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:21.477462Z","iopub.execute_input":"2025-07-24T11:40:21.477909Z","iopub.status.idle":"2025-07-24T11:40:21.794861Z","shell.execute_reply.started":"2025-07-24T11:40:21.477879Z","shell.execute_reply":"2025-07-24T11:40:21.793719Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"7307c082-8ad2-4c8d-9807-f1d95bc324ba","cell_type":"markdown","source":"## Section 15: 📊 Sensor Signals – Acceleration & Rotation\nLet’s draw acc and rot values. Colored background shows `behavior` & `phase` info.\n","metadata":{}},{"id":"13d5bf92-2c68-41c4-83df-31d6bad4c410","cell_type":"code","source":"fig, axes = plt.subplots(2, 1, figsize=(16, 8), sharex=True)\n\n# Plot acceleration\naxes[0].plot(seq_df['acc_x'], label='acc_x')\naxes[0].plot(seq_df['acc_y'], label='acc_y')\naxes[0].plot(seq_df['acc_z'], label='acc_z')\naxes[0].set_ylabel(\"Acceleration\")\naxes[0].legend(loc='upper right')\naxes[0].set_title(\"Acceleration over Time\")\n\n# Plot rotation\naxes[1].plot(seq_df['rot_x'], label='rot_x')\naxes[1].plot(seq_df['rot_y'], label='rot_y')\naxes[1].plot(seq_df['rot_z'], label='rot_z')\naxes[1].plot(seq_df['rot_w'], label='rot_w')\naxes[1].set_ylabel(\"Rotation\")\naxes[1].legend(loc='upper right')\naxes[1].set_title(\"Rotation over Time\")\n\n# Add shaded backgrounds for behavior\nfor ax in axes:\n    for i in range(len(seq_df)):\n        color = \"orange\" if seq_df.loc[i, \"behavior\"] == 1 else \"lightgray\"\n        ax.axvspan(i, i+1, color=color, alpha=0.05)\n\nplt.xlabel(\"Timestep\")\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:21.796239Z","iopub.execute_input":"2025-07-24T11:40:21.796580Z","iopub.status.idle":"2025-07-24T11:40:22.533510Z","shell.execute_reply.started":"2025-07-24T11:40:21.796551Z","shell.execute_reply":"2025-07-24T11:40:22.532121Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"bcac6fac-e729-459e-8e4e-16bc8ecef5f6","cell_type":"markdown","source":"## Section 16: 🎯 Euler Angles (Pitch, Roll, Yaw)\n\nRotations are cool, but Euler angles are cooler! Let’s convert quaternions into pitch-roll-yaw for better interpretability.\n","metadata":{}},{"id":"e735bf51-4f23-4f76-8012-1babde57eaf1","cell_type":"code","source":"# Ensure quaternion order is (x, y, z, w)\nquaternions = seq_df[['rot_x', 'rot_y', 'rot_z', 'rot_w']].to_numpy()\neuler_angles = R.from_quat(quaternions).as_euler('xyz', degrees=True)\n\nseq_df['pitch'], seq_df['roll'], seq_df['yaw'] = euler_angles.T\n\n# Plot\nplt.figure(figsize=(16,6))\nplt.plot(seq_df['pitch'], label='Pitch')\nplt.plot(seq_df['roll'], label='Roll')\nplt.plot(seq_df['yaw'], label='Yaw')\nplt.title(\"Euler Angles (Degrees)\")\nplt.xlabel(\"Timestep\")\nplt.ylabel(\"Angle (°)\")\nplt.legend(loc='upper right')\n\n# Highlight behavior\nfor i in range(len(seq_df)):\n    color = \"orange\" if seq_df.loc[i, \"behavior\"] == 1 else \"lightgray\"\n    plt.axvspan(i, i+1, color=color, alpha=0.05)\n\nplt.grid(True, linestyle='--', alpha=0.3)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:22.534834Z","iopub.execute_input":"2025-07-24T11:40:22.535201Z","iopub.status.idle":"2025-07-24T11:40:22.915380Z","shell.execute_reply.started":"2025-07-24T11:40:22.535169Z","shell.execute_reply":"2025-07-24T11:40:22.914267Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"539b0ea1-026c-441c-81a6-573228764429","cell_type":"markdown","source":"## Section 17: 🧱 ToF Sensor – Visual as 8x8 Infrared Heatmap\n\nLet’s pick a ToF frame, reshape it to 8x8, and visualize it like a thermal camera frame.\n","metadata":{}},{"id":"675bffa9-b315-4856-a659-14eae5fde616","cell_type":"code","source":"# Group ToF columns by sensor index (v1, v2, ..., v5)\ntof_groups = {\n    f\"tof_v{i}\": [col for col in seq_df.columns if f\"tof_{i}_\" in col] \n    for i in range(1,6)\n}\n\n# Pick a middle timestep\nmid_index = len(seq_df) // 2\n\nfig, axes = plt.subplots(1, 5, figsize=(20, 4))\n\nfor i, (sensor_name, cols) in enumerate(tof_groups.items()):\n    # Get 64 ToF values for this sensor\n    tof_values = seq_df[cols].iloc[mid_index].values.reshape(8, 8)\n    \n    # Resize to 128x128 for better visuals\n    tof_resized = cv2.resize(tof_values, (128, 128), interpolation=cv2.INTER_NEAREST)\n    \n    ax = axes[i]\n    im = ax.imshow(tof_resized, cmap='hot')\n    ax.set_title(f\"Sensor {sensor_name.split('_')[-1]}\")\n    ax.axis('off')\n\nfig.suptitle(f\"ToF Heatmaps – Sequence {selected_sequence}, Timestep {mid_index}\", fontsize=16)\nplt.colorbar(im, ax=axes.ravel().tolist(), shrink=0.6, label=\"Distance\")\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2025-07-24T11:40:22.917053Z","iopub.execute_input":"2025-07-24T11:40:22.917449Z","iopub.status.idle":"2025-07-24T11:40:23.975734Z","shell.execute_reply.started":"2025-07-24T11:40:22.917415Z","shell.execute_reply":"2025-07-24T11:40:23.974586Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"da80bded-678f-428b-a856-6c6d1ab0e8bd","cell_type":"markdown","source":"## Section 18: 🎬 Full Sensor Animation – ToF, Acceleration, Euler Angles & Thermopile in Action!\n  \nLet's visualize the full sensor story for a selected sequence:\n\n- Five ToF sensors as 8x8 heatmaps  \n- Acceleration x, y, z over time  \n- Euler angles (Pitch, Yaw, Roll)  \n- Thermopile temperature readings with live numeric values  \n\nAll animated and synchronized frame-by-frame!\n","metadata":{}},{"id":"06a8aabe-16d6-452a-96c1-ee71b9042e55","cell_type":"code","source":"## In Google Colab:\n# !sudo apt-get update\n# !sudo apt-get install -y ffmpeg\n\n\n## In Kaggle Notebooks:\n# !apt-get update\n# !apt-get install -y ffmpeg\n\n## In Windows \n# import matplotlib as mpl\n# mpl.rcParams['animation.ffmpeg_path'] = r'path\\to\\ffmpeg\\bin\\ffmpeg.exe'\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-24T11:40:23.976957Z","iopub.execute_input":"2025-07-24T11:40:23.977676Z","iopub.status.idle":"2025-07-24T11:40:23.983135Z","shell.execute_reply.started":"2025-07-24T11:40:23.977635Z","shell.execute_reply":"2025-07-24T11:40:23.981651Z"}},"outputs":[],"execution_count":null},{"id":"3403da2f-d70a-4d78-8baa-77b72f2986ad","cell_type":"code","source":"!apt-get update\n!apt-get install -y ffmpeg","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-24T11:40:23.984339Z","iopub.execute_input":"2025-07-24T11:40:23.984707Z","iopub.status.idle":"2025-07-24T11:40:34.741757Z","shell.execute_reply.started":"2025-07-24T11:40:23.984679Z","shell.execute_reply":"2025-07-24T11:40:34.740404Z"}},"outputs":[],"execution_count":null},{"id":"202ac563","cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib.animation as animation\nfrom matplotlib import gridspec\nfrom scipy.spatial.transform import Rotation as R\nfrom IPython.display import HTML\nfrom matplotlib.animation import FFMpegWriter\n\n# Pick a random sequence\nselected_sequence = random.choice(train_df['sequence_id'].unique())\nprint(f\"🎯 Selected sequence: {selected_sequence}\")\n\nsequence_data = train_df[train_df['sequence_id'] == selected_sequence].reset_index(drop=True)\nn_frames = len(sequence_data)\n\n# --- Detect phase and behavior changes ---\ncombined_changes = (\n    sequence_data[\"phase\"].ne(sequence_data[\"phase\"].shift()) |\n    sequence_data[\"behavior\"].ne(sequence_data[\"behavior\"].shift())\n)\nchange_frames = np.where(combined_changes)[0]\n\nchange_labels = [\n    f\"{sequence_data.loc[i, 'phase']}\\n{sequence_data.loc[i, 'behavior']}\"\n    for i in change_frames\n]\n\n# --- Prepare figure layout ---\nfig = plt.figure(figsize=(20, 13))\ngs = gridspec.GridSpec(5, 5, height_ratios=[0.2, 2, 1, 1, 1])  # Row 0: Title text\n\n# --- Dynamic Text for Target Info ---\ntitle_ax = fig.add_subplot(gs[0, :])\ntitle_ax.axis(\"off\")\ntitle_text = title_ax.text(\n    0.01, 0.5, \"\", fontsize=18, ha=\"left\", va=\"center\", transform=title_ax.transAxes\n)\n\n# --- ToF plots (row 1) ---\ntof_groups = {\n    f\"tof_v{i}\": [col for col in sequence_data.columns if f\"tof_{i}_\" in col]\n    for i in range(1, 6)\n}\ntof_axes = [fig.add_subplot(gs[1, i]) for i in range(5)]\ntof_imshows = []\nfor ax in tof_axes:\n    im = ax.imshow(np.zeros((8, 8)), cmap=\"hot\", vmin=-1, vmax=255)\n    ax.axis(\"off\")\n    tof_imshows.append(im)\n\n# --- Acceleration plot (row 2) ---\nacc_ax = fig.add_subplot(gs[2, :])\n(acc_x_line,) = acc_ax.plot([], [], label=\"acc_x\")\n(acc_y_line,) = acc_ax.plot([], [], label=\"acc_y\")\n(acc_z_line,) = acc_ax.plot([], [], label=\"acc_z\")\nacc_ax.legend(loc=\"upper right\")\nacc_ax.set_ylabel(\"Acceleration\", fontsize=14)\nacc_ax.set_xlim(0, n_frames)\nacc_ax.set_ylim(\n    sequence_data[[\"acc_x\", \"acc_y\", \"acc_z\"]].min().min() - 1,\n    sequence_data[[\"acc_x\", \"acc_y\", \"acc_z\"]].max().max() + 1,\n)\n\n# --- Euler angles (row 3) ---\nquats = sequence_data[[\"rot_x\", \"rot_y\", \"rot_z\", \"rot_w\"]].to_numpy()\neulers = R.from_quat(quats).as_euler(\"xyz\", degrees=True)\nsequence_data[\"pitch\"], sequence_data[\"roll\"], sequence_data[\"yaw\"] = eulers.T\n\neuler_ax = fig.add_subplot(gs[3, :])\n(pitch_line,) = euler_ax.plot([], [], label=\"pitch\")\n(roll_line,) = euler_ax.plot([], [], label=\"roll\")\n(yaw_line,) = euler_ax.plot([], [], label=\"yaw\")\neuler_ax.legend(loc=\"upper right\")\neuler_ax.set_ylabel(\"Euler Angles (°)\", fontsize=14)\neuler_ax.set_xlim(0, n_frames)\neuler_ax.set_ylim(\n    sequence_data[[\"pitch\", \"roll\", \"yaw\"]].min().min() - 5,\n    sequence_data[[\"pitch\", \"roll\", \"yaw\"]].max().max() + 5,\n)\n\n# --- Thermal sensors (row 4) ---\nthm_cols = [col for col in sequence_data.columns if col.startswith(\"thm_\")]\nthm_ax = fig.add_subplot(gs[4, :])\nthm_lines = []\nfor col in thm_cols:\n    (line,) = thm_ax.plot([], [], label=col, linewidth=1)\n    thm_lines.append(line)\nthm_ax.set_xlim(0, n_frames)\nthm_ax.set_ylim(\n    sequence_data[thm_cols].min().min() - 1, sequence_data[thm_cols].max().max() + 1\n)\nthm_ax.set_ylabel(\"Temperature\", fontsize=14)\nthm_ax.legend(ncol=len(thm_cols) // 2, loc=\"upper right\", fontsize=8)\n\n\ndef draw_change_lines(ax, let_text=False):\n    y_min, y_max = ax.get_ylim()\n    label_spacing = (y_max - y_min) * 0.05  # spacing between labels\n    used_y_positions = {}  # track how many labels placed at each x\n\n    for i, frame in enumerate(change_frames):\n        ax.axvline(frame, color=\"purple\", linestyle=\"--\", alpha=0.5)\n\n        if let_text:\n            # Count how many labels already placed at this x to stack\n            count = used_y_positions.get(frame, 0)\n            y_pos = y_min + label_spacing * count\n            used_y_positions[frame] = count + 1\n\n            ax.text(\n                frame + 0.5,\n                y_pos,\n                change_labels[i],\n                color=\"purple\",\n                rotation=90,\n                fontsize=16,\n                ha=\"left\",\n                va=\"bottom\",\n            )\n\n\n\ndraw_change_lines(acc_ax)\ndraw_change_lines(euler_ax)\ndraw_change_lines(thm_ax,let_text=True)\n\n\n# --- Update function ---\ndef update(frame):\n    # 📌 Update title info\n    s = sequence_data.loc[\n        frame, [\"subject\", \"orientation\", \"behavior\", \"phase\", \"gesture\"]\n    ]\n    title_text.set_text(\n        # f\"Sequence: {selected_sequence} | Subject: {s.subject} | Orientation: {s.orientation} | \"\n        # f\"Behavior: {s.behavior} | Phase: {s.phase} | Gesture: {s.gesture}\"\n        \n        f\"{selected_sequence} | {s.subject} | Frame {frame:04}\\n\"+\n        f\"Orientation: {s.orientation} | Gesture: {s.gesture}\\n\"+\n        f\"Phase: {s.phase} | Behavior: {s.behavior}\"\n    )\n\n    # 🎛️ ToF\n    for i, (sensor_name, cols) in enumerate(tof_groups.items()):\n        vals = sequence_data.loc[frame, cols].astype(float).values.reshape(8, 8)\n        tof_imshows[i].set_array(vals)\n\n    # 📈 Acc\n    acc_x_line.set_data(range(frame), sequence_data[\"acc_x\"][:frame])\n    acc_y_line.set_data(range(frame), sequence_data[\"acc_y\"][:frame])\n    acc_z_line.set_data(range(frame), sequence_data[\"acc_z\"][:frame])\n\n    # 🧭 Euler\n    pitch_line.set_data(range(frame), sequence_data[\"pitch\"][:frame])\n    roll_line.set_data(range(frame), sequence_data[\"roll\"][:frame])\n    yaw_line.set_data(range(frame), sequence_data[\"yaw\"][:frame])\n\n    # 🌡️ Thermal\n    for i, col in enumerate(thm_cols):\n        thm_lines[i].set_data(range(frame), sequence_data[col][:frame])\n\n    return (\n        [title_text]\n        + tof_imshows\n        + [acc_x_line, acc_y_line, acc_z_line, pitch_line, roll_line, yaw_line]\n        + thm_lines\n    )\n\n\n# 🎬 Create Animation\nani = animation.FuncAnimation(fig, update, frames=n_frames, interval=100, blit=True)\nplt.close()\n\n# 💾 Save to MP4\nwriter = FFMpegWriter(fps=10, metadata=dict(artist=\"You\"), bitrate=1800)\nani.save(f\"sequence_animation_{selected_sequence}.mp4\", writer=writer, dpi=150)\n\n# 💻 Display inline (for Jupyter)\nHTML(ani.to_jshtml())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-24T11:40:34.743412Z","iopub.execute_input":"2025-07-24T11:40:34.743734Z"}},"outputs":[],"execution_count":null},{"id":"74723a7c-c29b-4c0d-9e8a-69550c7a8c28","cell_type":"markdown","source":"## Section x_01: 🎭 Metadata Overlay (Orientation, Behavior, Phase, Gesture)\n\nEach frame comes with metadata. Let's check a few rows to remind ourselves what context surrounds the movement.\n","metadata":{}},{"id":"ebdac8e1-a125-4595-94d3-5e3388ea770e","cell_type":"code","source":"display(seq_df[['sequence_counter', 'orientation', 'behavior', 'phase', 'gesture']].head(10))\n","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}