{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":102335,"databundleVersionId":12518947,"sourceType":"competition"}],"dockerImageVersionId":31089,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## 🛑 Please Read this first.\n* This section presents an initial Exploratory Data Analysis (EDA) of the subject demographics, aiming to build a foundational understanding of our participant cohort. By examining feature distributions, we can spot potential biases, guide feature engineering, and provide essential context for future modeling.\n\n* These insights, while not directly used in model training, are critical for building a reliable and interpretable machine learning model to distinguish BFRB from non-BFRB activities.\n","metadata":{}},{"cell_type":"markdown","source":"# ⚪ **Import Libraries**","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nsns.set()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:52.705092Z","iopub.execute_input":"2025-07-25T12:47:52.705426Z","iopub.status.idle":"2025-07-25T12:47:53.787363Z","shell.execute_reply.started":"2025-07-25T12:47:52.705401Z","shell.execute_reply":"2025-07-25T12:47:53.786429Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# ⚪ **Args**","metadata":{}},{"cell_type":"code","source":"Dir = \"/kaggle/input/cmi-detect-behavior-with-sensor-data\"","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.789250Z","iopub.execute_input":"2025-07-25T12:47:53.790363Z","iopub.status.idle":"2025-07-25T12:47:53.795332Z","shell.execute_reply.started":"2025-07-25T12:47:53.790336Z","shell.execute_reply":"2025-07-25T12:47:53.794059Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# ⚪ **EDA**","metadata":{}},{"cell_type":"markdown","source":"### ⚪ **Demographics**","metadata":{}},{"cell_type":"code","source":"df_Demographic  = pd.read_csv(f\"{Dir}/train_demographics.csv\") # \"dmgc\" means demographics\ndf_Demographic.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.796821Z","iopub.execute_input":"2025-07-25T12:47:53.797153Z","iopub.status.idle":"2025-07-25T12:47:53.834727Z","shell.execute_reply.started":"2025-07-25T12:47:53.797129Z","shell.execute_reply":"2025-07-25T12:47:53.833722Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### 🟠  **Demographic Features Description**\n\nThe `train_demographics.csv` file provides detailed demographic and anthropometric information for each participant. Below is a description of each column:\n\n| Column                 | Description                                                                                                         | Type         |\n|------------------------|---------------------------------------------------------------------------------------------------------------------|--------------|\n| `subject`              | Unique participant ID.                                                                                              | Nominal Categorical  |\n| `adult_child`          | Whether the participant is a child (`0`) or an adult (`1`). Adults are 18 years or older.                           | Binary Categorical   |\n| `age`                  | Age of participant in years at the time of data collection.                                                         | Continuous Numerical |\n| `sex`                  | Sex assigned at birth (`0`: female, `1`: male).                                                                     | Binary Categorical   |\n| `handedness`           | Dominant hand (`0`: left-handed, `1`: right-handed).                                                                | Binary Categorical   |\n| `height_cm`            | Participant's height in centimeters.                                                                                | Continuous Numerical |\n| `shoulder_to_wrist_cm` | Distance from shoulder to wrist in centimeters.                                                                     | Continuous Numerical |\n| `elbow_to_wrist_cm`    | Distance from elbow to wrist in centimeters.                                                                        | Continuous Numerical |\n\n\n---\n\n\n**Note:**  \n1. Demographic features can be used to analyze differences in behavior or sensor data by age, sex, handedness, or anthropometric characteristics.\n2. Correctly identifying whether a feature is numerical or categorical is crucial—misclassifying data types can lead to unreliable analyses, incorrect modeling choices, and ultimately, invalid conclusions.\n","metadata":{}},{"cell_type":"code","source":"df_Demographic.shape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.837193Z","iopub.execute_input":"2025-07-25T12:47:53.838164Z","iopub.status.idle":"2025-07-25T12:47:53.844572Z","shell.execute_reply.started":"2025-07-25T12:47:53.838133Z","shell.execute_reply":"2025-07-25T12:47:53.843134Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_Demographic.info()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.845594Z","iopub.execute_input":"2025-07-25T12:47:53.846020Z","iopub.status.idle":"2025-07-25T12:47:53.873827Z","shell.execute_reply.started":"2025-07-25T12:47:53.845995Z","shell.execute_reply":"2025-07-25T12:47:53.872587Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# ⚪ **Statistical summary**","metadata":{}},{"cell_type":"code","source":"df_Demographic.describe()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.875327Z","iopub.execute_input":"2025-07-25T12:47:53.875764Z","iopub.status.idle":"2025-07-25T12:47:53.911274Z","shell.execute_reply.started":"2025-07-25T12:47:53.875728Z","shell.execute_reply":"2025-07-25T12:47:53.909945Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### 🟠 **Concise Summary of Demographic Analysis**\n\nAnalysis of the 81 participants' demographic data reveals two critical issues requiring action:\n\n1.  **Clear Outlier/Data Error:** A significant anomaly was found in the anatomical data. One subject has a forearm length (`elbow_to_wrist_cm` = 44 cm) that is anatomically improbable compared to their total arm length (71 cm). This data point must be investigated and likely corrected or removed.\n\n2.  **Severe Feature Imbalance:** The `handedness` feature is extremely imbalanced (~88% right-handed), giving it very low variance and minimal predictive value. It is a strong candidate for removal to reduce model noise.\n\nOther less critical observations include a mild gender imbalance and a well-balanced distribution between adults and children. The dataset has no missing values.","metadata":{}},{"cell_type":"code","source":"df_Demographic['adult_child'] = df_Demographic['adult_child'].astype('object')\ndf_Demographic['sex'] = df_Demographic['sex'].astype('object')\ndf_Demographic['handedness'] = df_Demographic['handedness'].astype('object')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.912401Z","iopub.execute_input":"2025-07-25T12:47:53.912785Z","iopub.status.idle":"2025-07-25T12:47:53.926634Z","shell.execute_reply.started":"2025-07-25T12:47:53.912760Z","shell.execute_reply":"2025-07-25T12:47:53.925583Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_Demographic.info()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.927975Z","iopub.execute_input":"2025-07-25T12:47:53.928706Z","iopub.status.idle":"2025-07-25T12:47:53.949071Z","shell.execute_reply.started":"2025-07-25T12:47:53.928647Z","shell.execute_reply":"2025-07-25T12:47:53.948155Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_Demographic.describe(include='all')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.950375Z","iopub.execute_input":"2025-07-25T12:47:53.950734Z","iopub.status.idle":"2025-07-25T12:47:53.993504Z","shell.execute_reply.started":"2025-07-25T12:47:53.950703Z","shell.execute_reply":"2025-07-25T12:47:53.992192Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### 🟠 **Summary of New Insights**\n\nThis summary highlights the critical, actionable insights derived from combining descriptive statistics with the data dictionary.\n\n*   **Data Integrity Confirmed:** The dataset contains records for **81 unique participants**.\n\n*   **Demographic Distribution:**\n    *   **`adult_child` (Balanced):** 42 Adults vs. 39 Children.\n    *   **`sex` (Mildly Imbalanced):** 50 Males vs. 31 Females.\n    *   **`handedness` (Severely Imbalanced):** 71 Right-handed vs. 10 Left-handed.\n\n*   **Critical Modeling Implications:**\n1.  **Multicollinearity Warning:** The `adult_child` feature is engineered directly from `age`. To prevent model instability, **only one of these two columns should be used** for feature selection.\n2.  **Feature Removal Candidate:** Due to its severe imbalance, the `handedness` column has extremely low variance. It is a prime candidate for **removal**, as it is unlikely to have significant predictive power.\n","metadata":{}},{"cell_type":"code","source":"# Reverting demographic columns to integer type for further analysis.\n\ndf_Demographic['adult_child'] = df_Demographic['adult_child'].astype('int64')\ndf_Demographic['sex'] = df_Demographic['sex'].astype('int64')\ndf_Demographic['handedness'] = df_Demographic['handedness'].astype('int64')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:53.996261Z","iopub.execute_input":"2025-07-25T12:47:53.996560Z","iopub.status.idle":"2025-07-25T12:47:54.003243Z","shell.execute_reply.started":"2025-07-25T12:47:53.996538Z","shell.execute_reply":"2025-07-25T12:47:54.002229Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Checking\ndf_Demographic[['adult_child', 'sex', 'handedness']].dtypes","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:54.004225Z","iopub.execute_input":"2025-07-25T12:47:54.004474Z","iopub.status.idle":"2025-07-25T12:47:54.024233Z","shell.execute_reply.started":"2025-07-25T12:47:54.004455Z","shell.execute_reply":"2025-07-25T12:47:54.023167Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Its ok now. We will continue","metadata":{}},{"cell_type":"markdown","source":"# ⚪ **Distribution Overview**","metadata":{}},{"cell_type":"code","source":"# Choose Only numeric columns\nnumeric_columns = df_Demographic.select_dtypes(include='number').columns\n\n# Set number of rows and columns for subplots\nnum_cols = 3\nnum_rows = (len(numeric_columns) + num_cols) - 1 // num_cols\n\n# subplots\nfig, axes = plt.subplots(num_rows, num_cols, figsize = (15, 4 * num_rows))\naxes = axes.flatten()\n\n# Plot for each numeric columns\nfor i, col in enumerate(numeric_columns):\n    axes[i].hist(df_Demographic[col], bins=20, color='lightblue', edgecolor='black')\n    axes[i].set_title(f'Histogram of {col}')\n    axes[i].set_xlabel(col)\n    axes[i].set_ylabel('Absolute Frequency')\n\n# Remove unused subplots if any\nfor j in range(i + 1, len(axes)):\n    fig.delaxes(axes[j])\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-25T12:47:54.025200Z","iopub.execute_input":"2025-07-25T12:47:54.025465Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Correlation Matrix\ncor_mat = df_Demographic.corr(numeric_only=True)\n\nplt.figure(figsize=(10, 8))\n\nsns.heatmap(cor_mat, annot=True, cmap='coolwarm', fmt='.2f', square=True)\n\nplt.title('Correlation Matrix')\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.scatter(df_Demographic.age, df_Demographic.height_cm)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.scatter(df_Demographic.shoulder_to_wrist_cm, df_Demographic.height_cm)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(8, 6))\nscatter = plt.scatter(\n    df_Demographic['age'],\n    df_Demographic['height_cm'],\n    c = df_Demographic['sex'],\n    cmap = 'coolwarm',\n    edgecolor = 'k',\n    s = 100,\n    alpha = 0.8\n)\nplt.xlabel('Age (year)', fontsize = 14)\nplt.ylabel('Height (centimetre)', fontsize = 14)\n\nplt.title('Age vs Height Colored by Sex', fontsize = 16)\n\ncbar = plt.colorbar(scatter)\ncbar.set_label('Sex (Blue=Female, Red=Male)', fontsize =13)\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### Result:\nThe plot visually indicates a distinct trend: male participants tend to be taller than female participants. This observation holds true across all age groups shown, as the red dots (males) consistently appear above the blue dots (females) on the vertical axis.","metadata":{}},{"cell_type":"markdown","source":"### Finish 👋","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}