{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":102335,"databundleVersionId":12518947,"sourceType":"competition"},{"sourceId":242954653,"sourceType":"kernelVersion"}],"dockerImageVersionId":31041,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## CMI - Detect Behavior with Sensor Data","metadata":{}},{"cell_type":"markdown","source":"The goal of this competition is to develop a predictive model that distinguishes BFRB-like and non-BFRB-like activity using data from a variety of sensors collected via a wrist-worn device. Successfully disentangling these behaviors will improve the design and accuracy of wearable BFRB-detection devices, which are relevant to a wide range of mental illnesses, ultimately strengthening the tools available to support their treatment.","metadata":{}},{"cell_type":"markdown","source":"### Importing the necessary Libraries","metadata":{}},{"cell_type":"code","source":"# Two‑Branch Human‑Activity‑Recognition Pipeline (IMU + Thermopile/TOF  + SE‑CNN + BiLSTM + Attention)\nimport os, json, joblib, numpy as np, pandas as pd\nfrom pathlib import Path\nimport warnings \nwarnings.filterwarnings(\"ignore\")\n\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom sklearn.utils.class_weight import compute_class_weight\n\nfrom tensorflow.keras.utils import Sequence, to_categorical, pad_sequences\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.layers import (\n    Input, Conv1D, BatchNormalization, Activation, add, MaxPooling1D, Dropout,\n    Bidirectional, LSTM, GlobalAveragePooling1D, Dense, Multiply, Reshape,\n    Lambda, Concatenate\n)\nfrom tensorflow.keras.regularizers import l2\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import EarlyStopping\nfrom tensorflow.keras import backend as K\nimport tensorflow as tf\nimport polars as pl\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.156526Z","iopub.execute_input":"2025-07-27T04:02:58.157427Z","iopub.status.idle":"2025-07-27T04:02:58.163556Z","shell.execute_reply.started":"2025-07-27T04:02:58.157400Z","shell.execute_reply":"2025-07-27T04:02:58.162848Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Configuration","metadata":{}},{"cell_type":"markdown","source":"This sets the hyperparameters and paths. Can play around the these values to get higher LB score. \n\nTraining hyperparameters:\n\nBATCH_SIZE: Minibatch size.\n\nPAD_PERCENTILE: Used for sequence padding.\n\nLR_INIT: Learning rate.\n\nWD: Weight decay (L2).\n\nMIXUP_ALPHA: If using mixup augmentation.\n\nEPOCHS: Max training epochs.\n\nPATIENCE: Early stopping patience.","metadata":{}},{"cell_type":"code","source":"# (Competition metric will only be imported when TRAINing)\nTRAIN = True                     # ← set to True when you want to train\nRAW_DIR = Path(\"/kaggle/input/cmi-detect-behavior-with-sensor-data\")\nPRETRAINED_DIR = Path(\"/kaggle/input/pretrained-model\")  # used when TRAIN=False\nEXPORT_DIR = Path(\"./\")                                    # artefacts will be saved here\nBATCH_SIZE = 64\nPAD_PERCENTILE = 95\nLR_INIT = 1e-3\nWD = 3e-4\nMIXUP_ALPHA = 0.4\nEPOCHS = 160\nPATIENCE = 40\n\n\nprint(\"▶ imports ready · tensorflow\", tf.__version__)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.164956Z","iopub.execute_input":"2025-07-27T04:02:58.165326Z","iopub.status.idle":"2025-07-27T04:02:58.180054Z","shell.execute_reply.started":"2025-07-27T04:02:58.165310Z","shell.execute_reply":"2025-07-27T04:02:58.179521Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Utility Functions","metadata":{}},{"cell_type":"markdown","source":"Utility Fucntions are having modular helper fucntions which simplify the architecture design in TensorFlow/Keras models.","metadata":{}},{"cell_type":"code","source":"#Tensor Manipulations\ndef time_sum(x):\n    return K.sum(x, axis=1)\n\ndef squeeze_last_axis(x):\n    return tf.squeeze(x, axis=-1)\n\ndef expand_last_axis(x):\n    return tf.expand_dims(x, axis=-1)\n\ndef se_block(x, reduction=8):\n    ch = x.shape[-1]\n    se = GlobalAveragePooling1D()(x)\n    se = Dense(ch // reduction, activation='relu')(se)\n    se = Dense(ch, activation='sigmoid')(se)\n    se = Reshape((1, ch))(se)\n    return Multiply()([x, se])\n\n\n# Residual CNN Block with SE\ndef residual_se_cnn_block(x, filters, kernel_size, pool_size=2, drop=0.3, wd=1e-4):\n    shortcut = x\n    for _ in range(2):\n        x = Conv1D(filters, kernel_size, padding='same', use_bias=False,\n                   kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x)\n        x = Activation('relu')(x)\n    x = se_block(x)\n    if shortcut.shape[-1] != filters:\n        shortcut = Conv1D(filters, 1, padding='same', use_bias=False,\n                          kernel_regularizer=l2(wd))(shortcut)\n        shortcut = BatchNormalization()(shortcut)\n    x = add([x, shortcut])\n    x = Activation('relu')(x)\n    x = MaxPooling1D(pool_size)(x)\n    x = Dropout(drop)(x)\n    return x\n\ndef attention_layer(inputs):\n    score = Dense(1, activation='tanh')(inputs)\n    score = Lambda(squeeze_last_axis)(score)\n    weights = Activation('softmax')(score)\n    weights = Lambda(expand_last_axis)(weights)\n    context = Multiply()([inputs, weights])\n    context = Lambda(time_sum)(context)\n    return context\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.180867Z","iopub.execute_input":"2025-07-27T04:02:58.181347Z","iopub.status.idle":"2025-07-27T04:02:58.197615Z","shell.execute_reply.started":"2025-07-27T04:02:58.181323Z","shell.execute_reply":"2025-07-27T04:02:58.197021Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Data Helpers","metadata":{}},{"cell_type":"code","source":"# 時系列データの前処理（欠損値の補完と標準化）\ndef preprocess_sequence(df_seq: pd.DataFrame, feature_cols: list[str], scaler: StandardScaler):\n    mat = df_seq[feature_cols].ffill().bfill().fillna(0).values  # 欠損値を補完（前方・後方）、さらにゼロ埋め\n    return scaler.transform(mat).astype('float32')  # 標準化（平均0・分散1に変換）、データ型をfloat32に変換\n\n# MixUpデータ拡張を行い、ニューラルネットワークの正則化を強化するクラス\nclass MixupGenerator(Sequence):\n    def __init__(self, X, y, batch_size, alpha=0.2):\n        self.X, self.y = X, y  # 入力データとラベル\n        self.batch = batch_size  # バッチサイズ\n        self.alpha = alpha  # ベータ分布のパラメータ（MixUpの比率調整）\n        self.indices = np.arange(len(X))  # データのインデックスを作成\n        \n    def __len__(self):\n        return int(np.ceil(len(self.X) / self.batch))  # ミニバッチの総数を計算\n    \n    def __getitem__(self, i):\n        idx = self.indices[i*self.batch:(i+1)*self.batch]  # バッチに対応するインデックスを取得\n        Xb, yb = self.X[idx], self.y[idx]  # 対応するデータとラベルを取得\n        \n        lam = np.random.beta(self.alpha, self.alpha)  # ベータ分布からMixUp係数を取得\n        perm = np.random.permutation(len(Xb))  # データのシャッフル用のインデックス\n        \n        X_mix = lam * Xb + (1 - lam) * Xb[perm]  # MixUpによりデータを線形補間\n        y_mix = lam * yb + (1 - lam) * yb[perm]  # ラベルも同様にMixUp\n        \n        return X_mix, y_mix  # MixUp後のデータとラベルを返す\n    \n    def on_epoch_end(self):\n        np.random.shuffle(self.indices)  # エポック終了時にデータのインデックスをシャッフル\n\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.198485Z","iopub.execute_input":"2025-07-27T04:02:58.198742Z","iopub.status.idle":"2025-07-27T04:02:58.216593Z","shell.execute_reply.started":"2025-07-27T04:02:58.198719Z","shell.execute_reply":"2025-07-27T04:02:58.215919Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Model Definition - Two Branch Architecture","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.layers import Lambda\n\n# ✅ FFTを用いた低・中・高周波の特徴抽出関数（3分割）\ndef fft_low_mid_high_features(tensor):\n    fft_values = tf.signal.fft(tf.cast(tensor, dtype=tf.complex64))  # FFT変換\n    fft_magnitude = tf.abs(fft_values)  # 振幅スペクトル（絶対値）\n\n    freq_len = tf.shape(fft_magnitude)[-1]  # 周波数数（センサーデータの長さ）\n\n    # 比率を TensorFlow 定数に変換（型安全に）\n    ratio_30 = tf.constant(0.3, dtype=tf.float32)\n    ratio_70 = tf.constant(0.7, dtype=tf.float32)\n\n    # 三分割ポイント（low: 0〜30%、mid: 30〜70%、high: 70〜100%）\n    low_end = tf.cast(tf.math.floor(tf.cast(freq_len, tf.float32) * ratio_30), tf.int32)\n    high_start = tf.cast(tf.math.floor(tf.cast(freq_len, tf.float32) * ratio_70), tf.int32)\n\n    # 各周波数帯の統計特徴を抽出\n    low_freq = tf.reduce_mean(fft_magnitude[:, :, :low_end], axis=-1, keepdims=True)\n    mid_freq = tf.reduce_mean(fft_magnitude[:, :, low_end:high_start], axis=-1, keepdims=True)\n    high_freq = tf.reduce_mean(fft_magnitude[:, :, high_start:], axis=-1, keepdims=True)\n\n    # 低・中・高周波特徴を結合\n    return tf.concat([low_freq, mid_freq, high_freq], axis=-1)\n\n# ✅ 2ブランチモデルの構築（FFT三分割版）\ndef build_two_branch_model(pad_len, imu_dim, tof_dim, n_classes, wd=1e-4):\n    inp = Input(shape=(pad_len, imu_dim + tof_dim))\n\n    # 入力をIMU / TOFに分離\n    imu = Lambda(lambda t: t[:, :, :imu_dim])(inp)\n    tof = Lambda(lambda t: t[:, :, imu_dim:])(inp)\n\n    # ✅ FFT特徴をIMUに追加（低・中・高周波の三分割）\n    imu_freq_features = Lambda(fft_low_mid_high_features)(imu)\n    imu = Concatenate()([imu, imu_freq_features])\n\n    # IMU処理ブランチ（Residual SE-CNN）\n    x1 = residual_se_cnn_block(imu, 128, 3, drop=0.4, wd=wd)\n    x1 = residual_se_cnn_block(x1, 256, 5, drop=0.4, wd=wd)\n\n    # TOF処理ブランチ（軽量Conv1D）\n    x2 = Conv1D(64, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(tof)\n    x2 = BatchNormalization()(x2); x2 = Activation('relu')(x2)\n    x2 = MaxPooling1D(2)(x2); x2 = Dropout(0.3)(x2)\n    x2 = Conv1D(128, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(x2)\n    x2 = BatchNormalization()(x2); x2 = Activation('relu')(x2)\n    x2 = MaxPooling1D(2)(x2); x2 = Dropout(0.3)(x2)\n\n    # 結合 → LSTM → Attention\n    merged = Concatenate()([x1, x2])\n    x = Bidirectional(LSTM(256, return_sequences=True, kernel_regularizer=l2(wd)))(merged)\n    x = Dropout(0.5)(x)\n    x = attention_layer(x)\n\n    # Dense層による分類\n    for units, drop in [(256, 0.5), (128, 0.3)]:\n        x = Dense(units, use_bias=False, kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x); x = Activation('relu')(x)\n        x = Dropout(drop)(x)\n\n    out = Dense(n_classes, activation='softmax', kernel_regularizer=l2(wd))(x)\n    return Model(inp, out)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.218191Z","iopub.execute_input":"2025-07-27T04:02:58.218442Z","iopub.status.idle":"2025-07-27T04:02:58.235946Z","shell.execute_reply.started":"2025-07-27T04:02:58.218427Z","shell.execute_reply":"2025-07-27T04:02:58.235221Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Training / Inference Pipeline","metadata":{}},{"cell_type":"markdown","source":"📥 データ処理\ntrain.csv → sequence_id ごとに時系列前処理（スケーリング＋欠損補完）\n\npad_len はシーケンス長の 95パーセンタイル\n\nfeature_cols は IMU+Thermopile+ToF の全センサ特徴量\n\nimu_cols / tof_cols に自動分離\n\n\n\n🏷 ラベル処理\nLabelEncoder で gesture_int を作成（np.save で永続化）\n\nto_categorical により One-Hot ベクトル化\n\n\n\n🧪 モデル学習\nTwo-Branchモデルを構築（IMU + ToFブランチ → BiLSTM → Attention）\n\nMixupGenerator によるバッチ単位のデータ拡張\n\nCosineDecayRestarts による学習率スケジューリング\n\nCategoricalCrossentropy(label_smoothing=0.1) でラベル曖昧化（汎化向上）\n\n\n\n📊 評価\nhold-outによる検証セット (validation_data=(X_val, y_val))\n\nカスタムスコア CompetitionMetric().calculate_hierarchical_f1() にて評価（ツリー構造ラベルを考慮）\n\n","metadata":{}},{"cell_type":"markdown","source":"✅ モデルとパイプラインの強み\n項目\t内容\n\n🧠 Two-Branch構造\tモダリティ（IMU / ToF+Thermal）ごとに分けた前処理 → 精度向上に貢献しやすい\n\n🧪 MixUp適用\tデータが比較的少ない中で汎化性能UPに効果大\n\n⏳ FFT特徴 + BiLSTM + Attention\t時系列性・周波数性・時刻ごとの重要度を統合的に学習できる設計\n\n⚖️ クラス重み\tcompute_class_weight の活用で 不均衡ラベル対応 も適切","metadata":{}},{"cell_type":"code","source":"if TRAIN:\n    print(\"▶ TRAIN MODE – loading dataset …\")\n    df = pd.read_csv(RAW_DIR / \"train.csv\")  # 学習データを読み込む\n\n    # ラベルエンコーディング\n    le = LabelEncoder(); df['gesture_int'] = le.fit_transform(df['gesture'])  # gestureを整数に変換\n    np.save(EXPORT_DIR / \"gesture_classes.npy\", le.classes_)  # 変換後のクラス情報を保存\n\n    # 特徴量リストの作成\n    meta_cols = {'gesture', 'gesture_int', 'sequence_type', 'behavior', 'orientation',\n                 'row_id', 'subject', 'phase', 'sequence_id', 'sequence_counter'}  # メタデータの列\n    feature_cols = [c for c in df.columns if c not in meta_cols]  # メタデータを除いた特徴量の列\n\n    imu_cols  = [c for c in feature_cols if not (c.startswith('thm_') or c.startswith('tof_'))]  # IMUセンサーデータ\n    tof_cols  = [c for c in feature_cols if c.startswith('thm_') or c.startswith('tof_')]  # TOF/THMセンサーデータ\n    print(f\"  IMU {len(imu_cols)} | TOF/THM {len(tof_cols)} | total {len(feature_cols)} features\")  # 特徴量数を出力\n\n    # [修正] センサーモダリティごとに別々のStandardScalerを学習させる\n    #  IMUは動きのスケール、TOF/THMは距離・温度という異なる物理量を持つため、別スケーリングが有効\n    scaler_imu = StandardScaler().fit(df[imu_cols].fillna(0))  # IMUスケーラー\n    scaler_tof = StandardScaler().fit(df[tof_cols].fillna(0))  # TOF/THMスケーラー\n\n    # [修正] スケーラーを保存（inference用に）\n    joblib.dump(scaler_imu, EXPORT_DIR / \"scaler_imu.pkl\")\n    joblib.dump(scaler_tof, EXPORT_DIR / \"scaler_tof.pkl\")\n\n    # シーケンスの構築\n    seq_gp = df.groupby('sequence_id')  # sequence_idごとにグループ化\n    X_list, y_list, lens = [], [], []\n\n    for seq_id, seq in seq_gp:\n        # [修正] モダリティごとに標準化処理を分離して適用\n        imu = scaler_imu.transform(seq[imu_cols].fillna(0))\n        tof = scaler_tof.transform(seq[tof_cols].fillna(0))\n        mat = np.hstack([imu, tof]).astype('float32')  # 結合して1つの入力配列にする\n        X_list.append(mat)  # 前処理済み特徴量を保存\n        y_list.append(seq['gesture_int'].iloc[0])  # ジェスチャークラスの保存\n        lens.append(len(mat))  # シーケンスの長さを記録\n\n    pad_len = int(np.percentile(lens, PAD_PERCENTILE))  # シーケンス長のパディング閾値を設定\n    np.save(EXPORT_DIR / \"sequence_maxlen.npy\", pad_len)  # 最大シーケンス長を保存\n    np.save(EXPORT_DIR / \"feature_cols.npy\", np.array(feature_cols))  # 使用する特徴量リストを保存\n\n    X = pad_sequences(X_list, maxlen=pad_len, padding='post', truncating='post', dtype='float32')  # シーケンスをパディング\n    y = to_categorical(y_list, num_classes=len(le.classes_))  # One-hotエンコーディングでカテゴリ化\n\n    # 訓練データと検証データに分割\n    X_tr, X_val, y_tr, y_val = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y_list)  # stratified split\n\n    # クラスの重み付けを計算\n    cw_vals = compute_class_weight('balanced', classes=np.arange(len(le.classes_)), y=y_list)  # クラスバランスを考慮した重み\n    class_weight = dict(enumerate(cw_vals))  # 重みを辞書に変換\n\n    # モデル構築\n    model = build_two_branch_model(pad_len, len(imu_cols), len(tof_cols), len(le.classes_), wd=WD)  # モデルを定義\n    steps = len(X_tr)//BATCH_SIZE\n    lr_sched = tf.keras.optimizers.schedules.CosineDecayRestarts(LR_INIT, first_decay_steps=5*steps)  # 学習率のスケジュール設定\n    model.compile(optimizer=Adam(lr_sched),\n                  loss=tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.1),\n                  metrics=['accuracy'])  # モデルのコンパイル\n\n    train_gen = MixupGenerator(X_tr, y_tr, batch_size=BATCH_SIZE, alpha=MIXUP_ALPHA)  # Mixupによるデータ拡張\n    cb = EarlyStopping(patience=PATIENCE, restore_best_weights=True, verbose=1)  # 早期停止の設定\n    model.fit(train_gen, epochs=EPOCHS, validation_data=(X_val, y_val),\n              class_weight=class_weight, callbacks=[cb], verbose=1)  # モデルの学習\n\n    model.save(EXPORT_DIR / \"gesture_two_branch_mixup.h5\")  # 訓練済みモデルを保存\n    print(\"✔ Training done – artefacts saved in\", EXPORT_DIR)  # 訓練完了メッセージ\n\n    # Hold-outデータの評価\n    from cmi_2025_metric_copy_for_import import CompetitionMetric\n    preds = model.predict(X_val).argmax(1)  # 推論結果を取得\n    true  = y_val.argmax(1)  # 正解ラベル\n    h_f1 = CompetitionMetric().calculate_hierarchical_f1(\n        pd.DataFrame({'gesture': le.classes_[true]}),\n        pd.DataFrame({'gesture': le.classes_[preds]}))  # ツリー構造に基づくF1スコアの計算\n    print(\"Hold‑out H‑F1 =\", round(h_f1, 4))  # スコアを出力","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-27T04:02:58.274672Z","iopub.execute_input":"2025-07-27T04:02:58.275255Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Predicting","metadata":{}},{"cell_type":"markdown","source":"学習時のスケーラー・特徴量・pad長を 完全再利用（再現性確保）\n\nPolars→pandas変換でI/O効率と学習互換性を両立\n\n関数単体で 本番環境（Kaggle）で動作可能な設計\n\n","metadata":{}},{"cell_type":"code","source":"# [修正] 正しい保存先（workingディレクトリ）を指定\nPRETRAINED_DIR = Path(\"/kaggle/working\")\n\n# make sure gesture_classes exists in both modes\nif TRAIN:\n    gesture_classes = le.classes_\n\n# [修正] 推論時に使用する2つのスケーラーをロード（IMU用・TOF/THM用）\nscaler_imu = joblib.load(PRETRAINED_DIR / \"scaler_imu.pkl\")\nscaler_tof = joblib.load(PRETRAINED_DIR / \"scaler_tof.pkl\")\n\ndef predict(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n    global gesture_classes\n    if gesture_classes is None:\n        gesture_classes = np.load(PRETRAINED_DIR / \"gesture_classes.npy\", allow_pickle=True)\n\n    df_seq = sequence.to_pandas()\n\n    # IMUとTOF/THMを個別スケーリングし、結合する\n    imu = scaler_imu.transform(df_seq[imu_cols].fillna(0))\n    tof = scaler_tof.transform(df_seq[tof_cols].fillna(0))\n    mat = np.hstack([imu, tof]).astype('float32')\n\n    # シーケンス長をパディングしてモデル入力に整形\n    pad = pad_sequences([mat], maxlen=pad_len, padding='post', truncating='post', dtype='float32')\n\n    # 予測クラスを返す\n    idx = int(model.predict(pad, verbose=0).argmax(1)[0])\n    return str(gesture_classes[idx])\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Integration with Kaggle Evaluation","metadata":{}},{"cell_type":"code","source":"import kaggle_evaluation.cmi_inference_server\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n    inference_server.serve()\nelse:\n    inference_server.run_local_gateway(\n        data_paths=(\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n        )\n    )","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Acknowledgement:\n  [https://www.kaggle.com/code/vonmainstein/imu-tof](https://www.kaggle.com/code/vonmainstein/imu-tof)\n\n  All creadits goes to this notebook. Have tried some played with some parametes to get better score. Hope this is useful. ","metadata":{}}]}