{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":53666,"databundleVersionId":6589269,"sourceType":"competition"},{"sourceId":151724150,"sourceType":"kernelVersion"}],"dockerImageVersionId":30588,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Child Mind Institute - Detect Sleep States\n\nCredits:\n\n - https://www.kaggle.com/code/itsuki9180/cmi-dss-debertav3-baseline-train?scriptVersionId=150120807","metadata":{}},{"cell_type":"markdown","source":"# Import modules and Config","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport gc\nimport time\nimport json\nfrom datetime import datetime\nimport matplotlib.pyplot as plt\nimport os, glob\nimport joblib\nimport random\nimport math\nfrom tqdm import tqdm \nfrom collections import OrderedDict\n\nfrom scipy.interpolate import interp1d\nfrom scipy import signal\nfrom scipy.signal import argrelmax\n\nfrom sklearn.metrics import mean_squared_error\n\nfrom math import pi, sqrt, exp\nimport sklearn,sklearn.model_selection\nimport torch\nfrom torch import nn,Tensor\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader, Dataset, SubsetRandomSampler\nfrom torch.optim import AdamW\nfrom sklearn.metrics import average_precision_score\nfrom timm.scheduler import CosineLRScheduler\n\nfrom transformers import get_cosine_schedule_with_warmup\nfrom transformers import AutoTokenizer, AutoConfig\nfrom transformers import AutoModel\n\nplt.style.use(\"ggplot\")\n\nfrom pyarrow.parquet import ParquetFile\nimport pyarrow as pa \nimport ctypes","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:42.121532Z","iopub.execute_input":"2023-12-02T06:02:42.122038Z","iopub.status.idle":"2023-12-02T06:02:46.848959Z","shell.execute_reply.started":"2023-12-02T06:02:42.121990Z","shell.execute_reply":"2023-12-02T06:02:46.847985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.850788Z","iopub.execute_input":"2023-12-02T06:02:46.851780Z","iopub.status.idle":"2023-12-02T06:02:46.857079Z","shell.execute_reply.started":"2023-12-02T06:02:46.851738Z","shell.execute_reply":"2023-12-02T06:02:46.856192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fundamental config\nNOTDEBUG = True # False -> DEBUG, True -> normally train\nWORKERS = os.cpu_count() // 2\nN_FOLDS = 5\nTRAIN_FOLD = 3\n\nSIGMA = 720 #average length of day is 24*60*12 = 17280 for comparison\nSAMPLE_FREQ = 12 # 1 obs per minute\n\nMAX_LEN = (24*60*12) # 1day per chunk\nMAX_CHUNK_LEN = MAX_LEN // SAMPLE_FREQ\nUSE_AMP = False\nSEED = 8620","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.858629Z","iopub.execute_input":"2023-12-02T06:02:46.859338Z","iopub.status.idle":"2023-12-02T06:02:46.871229Z","shell.execute_reply.started":"2023-12-02T06:02:46.859157Z","shell.execute_reply":"2023-12-02T06:02:46.869723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Model config\nOUTPUT_DIR='./Model'\nBS = 4","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.874400Z","iopub.execute_input":"2023-12-02T06:02:46.875424Z","iopub.status.idle":"2023-12-02T06:02:46.883798Z","shell.execute_reply.started":"2023-12-02T06:02:46.875376Z","shell.execute_reply":"2023-12-02T06:02:46.882938Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def torch_fix_seed(seed=42):\n    # Python random\n    random.seed(seed)\n    # Numpy\n    np.random.seed(seed)\n    # Pytorch\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    # torch.backends.cudnn.deterministic = True\n    # torch.use_deterministic_algorithms = True\n    # torch.backends.cudnn.benchmark = True\n\ntorch_fix_seed(SEED)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.885277Z","iopub.execute_input":"2023-12-02T06:02:46.885964Z","iopub.status.idle":"2023-12-02T06:02:46.899020Z","shell.execute_reply.started":"2023-12-02T06:02:46.885920Z","shell.execute_reply":"2023-12-02T06:02:46.898149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = 'cuda' if torch.cuda.is_available() else 'cpu'\ndevice","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.900152Z","iopub.execute_input":"2023-12-02T06:02:46.901572Z","iopub.status.idle":"2023-12-02T06:02:46.913769Z","shell.execute_reply.started":"2023-12-02T06:02:46.901531Z","shell.execute_reply":"2023-12-02T06:02:46.912459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define Model","metadata":{}},{"cell_type":"markdown","source":"## Define Dataset and Fold split","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import StratifiedKFold, StratifiedGroupKFold\nskf = StratifiedKFold(n_splits=N_FOLDS, random_state=SEED, shuffle=True)\nmetadata = pd.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv')\nunique_ids = metadata['series_id'].unique()\nmeta_cts = pd.DataFrame(unique_ids, columns=['series_id'])\nfor i, (train_index, valid_index) in enumerate(skf.split(X=meta_cts['series_id'], y=[1]*len(meta_cts))):\n    if i != TRAIN_FOLD:\n        continue\n    print(f\"Fold = {i}\")\n    train_ids = meta_cts.loc[train_index, 'series_id']\n    valid_ids = meta_cts.loc[valid_index, 'series_id']\n    print(f\"Length of Train = {len(train_ids)}, Length of Valid = {len(valid_ids)}\")\n    \n    if i == TRAIN_FOLD:\n        break\n        \ntrain_fpaths = [f\"/kaggle/input/cmi-data-preprocessing/train_csvs/{_id}.csv\" for _id in train_ids]\nvalid_fpaths = [f\"/kaggle/input/cmi-data-preprocessing/train_csvs/{_id}.csv\" for _id in valid_ids]\ntrain_ids[:5], train_fpaths[:5] ,len(train_fpaths)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.915004Z","iopub.execute_input":"2023-12-02T06:02:46.915785Z","iopub.status.idle":"2023-12-02T06:02:46.959265Z","shell.execute_reply.started":"2023-12-02T06:02:46.915750Z","shell.execute_reply":"2023-12-02T06:02:46.958066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class SleepDataset(Dataset):\n    def __init__(\n        self,\n        folder,\n        max_len=17280,\n        is_train=False,\n        sample_per_epoch=10000\n    ):\n        self.enmo_mean = np.load('/kaggle/input/cmi-data-preprocessing/enmo_mean.npy')\n        self.enmo_std = np.load('/kaggle/input/cmi-data-preprocessing/enmo_std.npy')\n        \n        self.max_len = max_len\n        assert max_len % SAMPLE_FREQ == 0\n        \n        self.is_train = is_train\n        \n        self.max_df_size = 0\n        self.min_df_size = 1e9\n        \n        self.sample_per_epoch = sample_per_epoch\n        \n        self.feat_list = ['anglez','enmo']\n        \n        self.Xys = self.read_csvs(folder)        \n        \n        self.label_list = ['onset', 'wakeup']\n        \n        self.hour_feat= ['hour']\n        \n        self.compress_methods = ['mean', 'median', 'fixed']\n        \n    def read_csvs(self, folder):\n        res = []\n        if type(folder) is str:\n            files = glob.glob(f'{folder}/*.csv')\n        else:\n            files = folder\n        for i, f in tqdm(enumerate(files), total=len(files), leave=False):\n            df = pd.read_csv(f)\n            df = self.norm_feat_eng(df, init=True if i==0 else False)\n                \n            res.append(df)\n            self.max_df_size = max(self.max_df_size, len(df))\n            self.min_df_size = min(self.min_df_size, len(df))\n        return res\n    \n    def compress(self, xt, method='mean', sample_freq=SAMPLE_FREQ):\n        x, t = xt\n        seq_len = x.shape[0]\n        if method == 'mean':\n            x = x.reshape(seq_len//sample_freq, sample_freq, -1).mean(1)\n            t = t.reshape(seq_len//sample_freq, sample_freq, 1).mean(1)\n            \n        if method == 'median':\n            x = np.median(x.reshape(seq_len//sample_freq, sample_freq, -1), axis=1)\n            t = np.median(t.reshape(seq_len//sample_freq, sample_freq, 1), axis=1)\n            \n        if method == 'fixed':\n            start = torch.randint(0, sample_freq, size=(1,))[0].numpy()\n            x = x[start::sample_freq]\n            t = t[start::sample_freq]\n        \n        return x.astype(np.float32), t.astype(np.int32)\n\n    def norm_feat_eng(self, X, init=False):\n        # 'anglez'와 'enmo' 열의 정규화:\n        # 'anglez' 열의 값은 90.0으로 나누어 정규화됩니다. 이렇게 하면 값의 범위가 0에서 1로 스케일링됩니다.\n        # 'enmo' 열의 값은 평균(self.enmo_mean)을 뺀 후 표준편차(self.enmo_std)로 나누어 정규화됩니다. 이 때 self.enmo_mean과 self.enmo_std는 클래스 초기화 과정에서 미리 계산된 값입니다. 작은 값을 더하여 분모가 0이 되지 않도록 합니다.\n        X['anglez'] = X['anglez'] / 90.0\n        X['enmo'] = (X['enmo'] - self.enmo_mean) / (self.enmo_std + 1e-12) # Standardization\n        \n        # 시프트된 특징 생성:\n        # 시계열 데이터에서 현재 값을 기준으로 일정 시간 간격(w)에 대한 이전 및 이후 값을 새로운 열로 추가합니다. 이를 통해 시계열 데이터의 시간적 패턴을 포착할 수 있습니다.\n        # 예를 들어 'anglez' 열을 기반으로 'anglez_shift_pos_w'와 'anglez_shift_neg_w' 열을 생성하며, 'enmo' 열을 기반으로 'enmo_shift_pos_w'와 'enmo_shift_neg_w' 열을 생성합니다.\n        # init 매개변수가 True인 경우(첫 번째 파일 처리 시)에는 이러한 새로운 열의 이름을 self.feat_list 리스트에 추가합니다.                \n        for w in [1, 2, 4, 8, 16]:    \n            X['anglez_shift_pos_' + str(w)] = X['anglez'].shift(w).fillna(0)\n            X['anglez_shift_neg_' + str(w)] = X['anglez'].shift(-w).fillna(0)\n            \n            X['enmo_shift_pos_' + str(w)] = X['enmo'].shift(w).fillna(0)\n            X['enmo_shift_neg_' + str(w)] = X['enmo'].shift(-w).fillna(0)\n\n            if init:\n                self.feat_list.append('anglez_shift_pos_' + str(w))\n                self.feat_list.append('anglez_shift_neg_' + str(w))\n                \n                self.feat_list.append('enmo_shift_pos_' + str(w))\n                self.feat_list.append('enmo_shift_neg_' + str(w))\n\n        # 롤링 통계 특징 생성:\n        # 롤링 평균과 표준편차를 계산하여 새로운 열로 추가합니다. 롤링 평균과 표준편차는 이동 평균과 이동 표준편차를 의미하며, 시계열 데이터의 전반적인 추세를 파악하는 데 도움이 됩니다.\n        # r은 롤링 윈도우의 크기를 나타냅니다. 여기서는 [17, 33, 65] 크기의 롤링 윈도우에 대한 평균과 표준편차를 계산합니다.\n        # init 매개변수가 True인 경우(첫 번째 파일 처리 시)에는 이러한 새로운 열의 이름을 self.feat_list 리스트에 추가합니다.\n        for r in [17, 33, 65]:\n            tmp_anglez = X['anglez'].rolling(r, center=True)\n            X[f'anglez_mean_{r}'] = tmp_anglez.mean()\n            X[f'anglez_std_{r}'] = tmp_anglez.std()            \n            \n            tmp_enmo = X['enmo'].rolling(r, center=True)\n            X[f'enmo_mean_{r}'] = tmp_enmo.mean()\n            X[f'enmo_std_{r}'] = tmp_enmo.std()\n            \n            if init:\n                self.feat_list.append(f'anglez_mean_{r}')\n                self.feat_list.append(f'anglez_std_{r}')\n\n                self.feat_list.append(f'enmo_mean_{r}')\n                self.feat_list.append(f'enmo_std_{r}')\n        \n        # 결측값 처리:\n        # 모든 결측값(NaN)을 0으로 채웁니다. 결측값은 누락된 데이터를 나타내며, 데이터의 무결성을 유지하기 위해 0으로 대체합니다.                \n        X = X.fillna(0)\n        \n        return X.astype(np.float32)\n\n    def gauss(self, n=SIGMA, sigma=SIGMA*0.15):\n        # guassian distribution function\n        r = range(-int(n/2),int(n/2)+1)\n        return [1 / (sigma * sqrt(2*pi)) * exp(-float(x)**2/(2*sigma**2)) for x in r]\n    \n    def __len__(self):\n        return self.sample_per_epoch if self.is_train else len(self.Xys)\n    \n    def __getitem__(self, index):\n# __getitem__ 메서드는 PyTorch 데이터셋 클래스의 핵심 메서드 중 하나로, 데이터셋에서 특정 인덱스의 데이터를 검색하는 역할을 합니다. 이 메서드는 주로 데이터셋을 반복(iterate)할 때 사용되며, 각 반복(iteration)마다 데이터셋에서 다음 데이터 샘플을 가져와 모델에 공급합니다.\n\n# 여기서 __getitem__ 메서드는 다음과 같이 두 가지 경우로 나뉩니다:\n\n# 훈련 데이터셋에서 데이터를 가져오는 경우:\n\n# self.is_train이 True인 경우, 훈련 데이터셋에서 데이터를 무작위로 선택하고 데이터 전처리를 수행합니다.\n# ind 변수를 사용하여 훈련 데이터셋(self.Xys)에서 무작위로 데이터 인덱스를 선택합니다.\n# 선택된 데이터의 특징(X), 레이블(y), 시간(t)을 추출하고, 이를 self.max_len에 맞게 패딩합니다.\n# 또한 데이터 압축(compression)을 무작위로 선택한 방법(self.compress_methods)으로 수행합니다.\n# 마지막으로, 특징(X), 시간(t), 레이블(y)을 반환합니다.\n# 검증 데이터셋에서 데이터를 가져오는 경우:\n# __getitem__ 메서드는 데이터셋을 반복(iterate)하면서 모델에 입력으로 제공됩니다. 모델은 이러한 데이터를 사용하여 훈련 또는 평가를 수행하게 됩니다. 이 메서드를 사용하면 데이터셋의 내용을 모델 학습 루프나 평가 루프에서 쉽게 처리할 수 있습니다.\n        if self.is_train:\n            ind = torch.randint(0, len(self.Xys), size=(1,))[0].numpy()\n            Xy = self.Xys[ind]\n            \n            X = Xy[self.feat_list].values.astype(np.float32)\n            y = Xy[self.label_list].values.astype(np.float32)\n            t = Xy[self.hour_feat].values.astype(np.int32)\n\n            if len(Xy)+1<self.max_len:\n                res = self.max_len - len(Xy) + 1\n                X = np.pad(X, ((0, res), (0, 0)))\n                y = np.pad(y, ((0, res), (0, 0)))\n                t = np.pad(t, ((0, res), (0, 0)))\n\n            start = torch.randint(0, len(X)-self.max_len, size=(1,))[0].numpy()\n\n            X = X[start:start+self.max_len]\n            y = y[start:start+self.max_len]\n            t = t[start:start+self.max_len]\n            \n            method_idx = torch.randint(0, len(self.compress_methods), size=(1,))[0].numpy()\n            X, t = self.compress((X, t), method=self.compress_methods[method_idx])\n\n            \n# self.is_train이 False인 경우, 검증 데이터셋에서 인덱스 index에 해당하는 데이터를 가져오고 데이터 전처리를 수행합니다.\n# 검증 데이터셋에서 데이터를 순차적으로 가져옵니다.\n# 가져온 데이터의 특징(X), 레이블(y), 시간(t)을 추출하고, 데이터의 길이를 SAMPLE_FREQ의 배수로 맞게 패딩합니다.\n# 데이터 압축(compression)은 항상 'mean' 방법을 사용합니다.\n# 마지막으로, 특징(X), 시간(t), 레이블(y)을 반환합니다.\n\n        else:\n            Xy = self.Xys[index]\n            X = Xy[self.feat_list].values.astype(np.float32)\n            y = Xy[self.label_list].values.astype(np.float32)        \n            t = Xy[self.hour_feat].values.astype(np.int32)\n            \n            if len(Xy)%SAMPLE_FREQ!=0:\n                res = SAMPLE_FREQ - (len(Xy)%SAMPLE_FREQ)\n                X = np.pad(X, ((0, res), (0, 0)))\n                y = np.pad(y, ((0, res), (0, 0)))\n                t = np.pad(t, ((0, res), (0, 0)))\n            \n            X, t = self.compress((X, t), method='mean')\n            \n        return X, t, y\n","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.961096Z","iopub.execute_input":"2023-12-02T06:02:46.961498Z","iopub.status.idle":"2023-12-02T06:02:46.996565Z","shell.execute_reply.started":"2023-12-02T06:02:46.961464Z","shell.execute_reply":"2023-12-02T06:02:46.995493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# NOTDEBUG = True\n# NOTDEBUG = False\n    \ntrain_fpaths = train_fpaths if NOTDEBUG else train_fpaths[:25]\nvalid_fpaths = valid_fpaths if NOTDEBUG else valid_fpaths[:5]\nsample_per_epoch = 20_000 if NOTDEBUG else 100\n\ntrain_ds = SleepDataset(train_fpaths, max_len=MAX_LEN, is_train=True, sample_per_epoch=sample_per_epoch)\nval_ds = SleepDataset(valid_fpaths, is_train=False)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:02:46.997869Z","iopub.execute_input":"2023-12-02T06:02:46.998276Z","iopub.status.idle":"2023-12-02T06:03:05.914852Z","shell.execute_reply.started":"2023-12-02T06:02:46.998242Z","shell.execute_reply":"2023-12-02T06:03:05.913365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import matplotlib.pyplot as plt\n# from sklearn.preprocessing import MinMaxScaler\n\n# # train_ds.Xys 데이터프레임에서 필요한 열을 추출\n# step_data = train_ds.Xys[0]['step']\n# anglez_data = train_ds.Xys[0]['anglez']\n# enmo_data = train_ds.Xys[0]['enmo']\n# onset_data = train_ds.Xys[0]['onset']\n# wakeup_data = train_ds.Xys[0]['wakeup']\n\n# # Min-Max 스케일링을 사용하여 Anglez와 Enmo 열 스케일링\n# scaler = MinMaxScaler()\n# anglez_scaled = scaler.fit_transform(anglez_data.values.reshape(-1, 1))\n# enmo_scaled = scaler.fit_transform(enmo_data.values.reshape(-1, 1))\n\n# # 그래프 그리기\n# plt.figure(figsize=(20, 6))  # 그래프 크기 설정 (선택 사항)\n\n# # x축은 step 데이터로 설정하고, 나머지 열을 y축으로 그립니다.\n# plt.plot(step_data, anglez_scaled, label='anglez', linewidth=0.5, linestyle='--', alpha=0.7, color=\"gray\")\n# plt.plot(step_data, enmo_scaled, label='enmo', linewidth=0.5, linestyle='-', alpha=0.6, color=\"black\")\n# plt.plot(step_data, onset_data, label='onset', linewidth=2, color=\"#FF6347\")\n# plt.plot(step_data, wakeup_data, label='wakeup', linewidth=2, color=\"#4682B4\")\n\n# # 그래프에 제목과 범례 추가 (선택 사항)\n# # plt.title(f'Step vs. Anglez, Enmo, Onset, Wakeup')\n# plt.title(f'[{train_ids[0]}]     Step vs. Anglez, Enmo, Onset, Wakeup', fontsize=16)\n# plt.xlabel('Step', fontsize=14)\n# plt.yticks([])\n# plt.ylabel('Anglez, Enmo, Onset, Wakeup', fontsize=14)\n# plt.legend()\n\n# plt.tight_layout()\n# graph_filepath = 'graph.png'\n# plt.savefig(graph_filepath)\n\n# # 그래프 표시\n# plt.show()\n\n","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:05.919932Z","iopub.execute_input":"2023-12-02T06:03:05.920429Z","iopub.status.idle":"2023-12-02T06:03:10.878433Z","shell.execute_reply.started":"2023-12-02T06:03:05.920386Z","shell.execute_reply":"2023-12-02T06:03:10.877250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.save('feature_list.npy', train_ds.feat_list)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:10.879817Z","iopub.execute_input":"2023-12-02T06:03:10.880801Z","iopub.status.idle":"2023-12-02T06:03:10.886587Z","shell.execute_reply.started":"2023-12-02T06:03:10.880761Z","shell.execute_reply":"2023-12-02T06:03:10.884893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dl = DataLoader(\n    train_ds,\n    batch_size=BS,\n    pin_memory=True,\n    num_workers=WORKERS,\n    shuffle=True,\n    drop_last=True\n)\nval_dl = DataLoader(\n    val_ds,\n    batch_size=1,\n    pin_memory=True,\n    num_workers=WORKERS,\n    shuffle=False,\n    drop_last=False\n)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:10.888197Z","iopub.execute_input":"2023-12-02T06:03:10.888681Z","iopub.status.idle":"2023-12-02T06:03:10.900415Z","shell.execute_reply.started":"2023-12-02T06:03:10.888645Z","shell.execute_reply":"2023-12-02T06:03:10.898789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"input_size = len(train_ds.feat_list)\noutput_size = len(train_ds.label_list)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:10.902490Z","iopub.execute_input":"2023-12-02T06:03:10.903706Z","iopub.status.idle":"2023-12-02T06:03:10.913448Z","shell.execute_reply.started":"2023-12-02T06:03:10.903664Z","shell.execute_reply":"2023-12-02T06:03:10.912150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del train_ds, val_ds\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:10.914987Z","iopub.execute_input":"2023-12-02T06:03:10.916139Z","iopub.status.idle":"2023-12-02T06:03:11.150656Z","shell.execute_reply.started":"2023-12-02T06:03:10.916087Z","shell.execute_reply":"2023-12-02T06:03:11.149439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_path = OUTPUT_DIR\nos.makedirs(model_path, exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.152371Z","iopub.execute_input":"2023-12-02T06:03:11.152929Z","iopub.status.idle":"2023-12-02T06:03:11.161221Z","shell.execute_reply.started":"2023-12-02T06:03:11.152886Z","shell.execute_reply":"2023-12-02T06:03:11.160101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define Model","metadata":{}},{"cell_type":"code","source":"# https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch\n\nclass FocalLoss(nn.Module):\n    def __init__(self, weight=None, size_average=True, alpha=1., gamma=2.):\n        super(FocalLoss, self).__init__()\n        self.alpha = alpha\n        self.gamma = gamma\n\n    def forward(self, inputs, targets):\n        \n        #comment out if your model contains a sigmoid or equivalent activation layer\n        #inputs = F.sigmoid(inputs)       \n        \n        #flatten label and prediction tensors\n#         inputs = inputs.view(-1)\n#         targets = targets.view(-1)\n        inputs = inputs.reshape(-1)\n        targets = targets.reshape(-1)\n        \n        #first compute binary cross-entropy \n        BCE = F.binary_cross_entropy_with_logits(inputs, targets, reduction='none')\n        BCE_EXP = torch.exp(-BCE)\n        focal_loss = self.alpha * (1-BCE_EXP)**self.gamma * BCE\n        \n        return focal_loss.mean()\n    \n# alpha 파라미터: alpha는 클래스 불균형을 다룰 때 사용됩니다. 두 클래스가 비슷한 비율로 존재한다면, alpha를 0.5 근처의 값으로 설정하는 것이 좋습니다. 하지만 한 클래스가 다른 클래스보다 훨씬 더 적게 존재한다면, alpha를 더 큰 클래스에 낮은 값을, 더 적은 클래스에 높은 값을 주어야 합니다. 예를 들어, alpha를 0.75 정도로 설정할 수 있습니다.\n\n# gamma 파라미터: gamma는 모델이 얼마나 쉽게 분류할 수 있는 예시에 덜 집중하게 할 것인지를 결정합니다. gamma가 높을수록 쉬운 예시들에 대한 손실의 영향이 줄어듭니다. 일반적으로 gamma는 0에서 5 사이의 값을 가지며, 2 정도가 자주 사용되는 값입니다. 만약 모델이 너무 쉬운 예시들에만 집중하는 것으로 보인다면, gamma 값을 증가시켜 볼 수 있습니다.","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.162833Z","iopub.execute_input":"2023-12-02T06:03:11.163280Z","iopub.status.idle":"2023-12-02T06:03:11.177962Z","shell.execute_reply.started":"2023-12-02T06:03:11.163236Z","shell.execute_reply":"2023-12-02T06:03:11.176246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Trainer(object):\n    def __init__(self, train_dataloader, val_dataloader, model, criterion, optimizer, save_model_name):\n        self.USE_AMP = USE_AMP\n        self.model_path = OUTPUT_DIR\n        self.train_dataloader = train_dataloader\n        self.val_dataloader = val_dataloader\n        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n        self.model = model\n        self.model = self.model.to(self.device)\n        self.save_model_name = save_model_name\n        \n        self.eval_metric = torch.nn.L1Loss()\n       \n        self.best_pred = 0\n        self.eval_best_loss = 10000000\n        self.history = {'loss':[], 'val_loss':[]}\n        \n        self.criterion = criterion\n        self.optimizer = optimizer\n\n        \n    def training(self):\n        self.model.train()\n        running_loss = 0.0\n              \n        for i, (inputs, times, targets) in enumerate(self.train_dataloader):\n            inputs, times, targets = inputs.to(self.device), times.to(self.device), targets.to(self.device)\n\n            self.optimizer.zero_grad()\n\n            with torch.cuda.amp.autocast(enabled=self.USE_AMP, dtype=torch.half):\n                outputs = self.model(inputs)\n                loss = self.criterion(outputs, targets)\n            \n            loss.backward()\n            self.optimizer.step()\n            \n            running_loss += loss.item()\n                        \n        epoch_loss = running_loss / len(self.train_dataloader)\n        return epoch_loss\n\n    \n    def validation(self):\n        self.model.eval()\n        total_loss = 0.0\n\n        with torch.no_grad():\n            for i, (inputs, times, targets) in enumerate(self.val_dataloader):\n                inputs, times, targets = inputs.to(self.device), times.to(self.device), targets.to(self.device)\n\n                with torch.cuda.amp.autocast(enabled=self.USE_AMP, dtype=torch.half):\n                    outputs = self.model(inputs)\n                    loss = self.criterion(outputs, targets)\n\n                total_loss += loss.item()\n\n        avg_loss = total_loss / len(self.val_dataloader)\n        return avg_loss\n\n\n    def fit(self, epochs):\n        self.history = {'loss': [], 'val_loss': []}\n        pbar = tqdm(total=epochs, leave=False, desc='Epoch', dynamic_ncols=True)\n\n        for epoch in range(epochs):\n            pbar.set_description(f'Epoch {epoch + 1}/{epochs}', refresh=True)\n            train_loss = self.training()\n            self.history['loss'].append(train_loss)\n\n            val_loss = self.validation()\n            self.history['val_loss'].append(val_loss)\n\n            # Update best validation loss\n            if val_loss < self.eval_best_loss:\n                self.eval_best_loss = val_loss\n                torch.save(\n                    model.state_dict(),\n                    os.path.join(self.model_path, save_model_name),\n                )\n            pbar.set_postfix(train_loss=train_loss, val_loss=val_loss, refresh=True)\n            pbar.update()\n            print(\n                f\"{epoch+1}/{epochs} -- \",\n                f\"train_loss = {train_loss:.6f} -- \",\n                f\"valid_loss = {val_loss:.6f}\"\n            )\n\n        pbar.close()\n        print('Best prediction (L1 loss) on Validation set:', self.eval_best_loss)\n        \n        \n    def plot_history(self):\n        fig = plt.figure(figsize=(10, 5))\n\n        plt.figure()\n        plt.plot(self.history['loss'], label='Train')\n        plt.plot(self.history['val_loss'], label='Val')\n        plt.xlabel('Epochs')\n        plt.ylabel('Loss')\n        plt.title('Loss')\n        plt.legend()\n\n        plt.tight_layout()\n        plt.show()\n\n        print('Max validation mae:', max(self.history['val_loss']))","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.179870Z","iopub.execute_input":"2023-12-02T06:03:11.180261Z","iopub.status.idle":"2023-12-02T06:03:11.201892Z","shell.execute_reply.started":"2023-12-02T06:03:11.180224Z","shell.execute_reply":"2023-12-02T06:03:11.200623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class LSTM(nn.Module):\n    def __init__(self, input_size, hidden_size, num_layers, output_size):\n        super(ImprovedLSTM, self).__init__()\n        self.hidden_size = hidden_size\n        self.num_layers = num_layers\n        self.output_size = output_size\n\n        # LSTM Layer\n        self.lstm = nn.LSTM(input_size, hidden_size, num_layers, batch_first=True)\n\n        # Transpose Convolution Layer\n        self.transpose_conv = nn.ConvTranspose1d(in_channels=hidden_size, \n                                                 out_channels=hidden_size, \n                                                 kernel_size=SAMPLE_FREQ, \n                                                 stride=SAMPLE_FREQ)\n\n        # Fully Connected Layer\n        self.fc = nn.Sequential(\n            nn.Linear(hidden_size, output_size),\n#             nn.Sigmoid()  # Ensure the output is between 0 and 1\n        )\n\n        # Layer Normalization\n        self.layer_norm = nn.LayerNorm(hidden_size)\n\n    def forward(self, x):\n        # Initialize hidden state and cell state\n        h0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size, device=x.device)\n        c0 = torch.zeros(self.num_layers, x.size(0), self.hidden_size, device=x.device)\n\n        # Forward propagate LSTM\n        out, _ = self.lstm(x, (h0, c0))\n\n        # Apply Layer Normalization\n        out = self.layer_norm(out)\n\n        # Transpose Convolution\n        out = out.permute(0, 2, 1)  # [batch_size, hidden_size, sequence_length]\n        out = self.transpose_conv(out)  # Upsample\n        out = out.permute(0, 2, 1)  # [batch_size, target_seq_length, hidden_size]\n\n        # Fully Connected Layer\n        out = self.fc(out)\n\n        return out","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-12-02T06:03:11.203870Z","iopub.execute_input":"2023-12-02T06:03:11.204232Z","iopub.status.idle":"2023-12-02T06:03:11.217607Z","shell.execute_reply.started":"2023-12-02T06:03:11.204200Z","shell.execute_reply":"2023-12-02T06:03:11.216070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ImprovedLSTM(nn.Module):\n    def __init__(self, input_size, hidden_size, num_layers, output_size, dropout_prob=0.5):\n        super(ImprovedLSTM, self).__init__()\n        self.hidden_size = hidden_size\n        self.num_layers = num_layers\n\n        # Bidirectional LSTM Layer\n        self.lstm = nn.LSTM(input_size, hidden_size, num_layers, \n                            batch_first=True, dropout=dropout_prob, bidirectional=True)\n\n        # Transpose Convolution Layer for Upsampling\n        self.transpose_conv = nn.ConvTranspose1d(in_channels=hidden_size * 2,  # Adjust for bidirectional\n                                                 out_channels=hidden_size * 2,  # Adjust for bidirectional\n                                                 kernel_size=SAMPLE_FREQ, \n                                                 stride=SAMPLE_FREQ)\n\n        # Fully Connected Layer\n        self.fc = nn.Linear(hidden_size * 2, output_size)  # Adjust for bidirectional\n\n        # Layer Normalization\n        self.layer_norm = nn.LayerNorm(hidden_size * 2)  # Adjust for bidirectional\n\n        # Dropout Layer\n        self.dropout = nn.Dropout(dropout_prob)\n\n    def forward(self, x):\n        # Forward propagate LSTM\n        out, _ = self.lstm(x)\n\n        # Apply Layer Normalization\n        out = self.layer_norm(out)\n\n        # Transpose Convolution for Upsampling\n        out = out.permute(0, 2, 1)  # Change to [batch_size, hidden_size * 2, sequence_length]\n        out = self.transpose_conv(out)  # Upsample\n        out = out.permute(0, 2, 1)  # Change back to [batch_size, target_seq_length, hidden_size * 2]\n\n        # Apply Layer Normalization\n        out = self.layer_norm(out)\n        # Apply Dropout\n        out = self.dropout(out)\n        \n        # Fully Connected Layer\n        out = self.fc(out)\n\n        return out\n","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.218908Z","iopub.execute_input":"2023-12-02T06:03:11.219283Z","iopub.status.idle":"2023-12-02T06:03:11.234589Z","shell.execute_reply.started":"2023-12-02T06:03:11.219250Z","shell.execute_reply":"2023-12-02T06:03:11.233493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class LSTM_CNN(nn.Module):\n    def __init__(self, input_size, hidden_size, num_layers, output_size, dropout_prob=0.5, num_channels=34*2, kernel_size=3):\n        super(LSTM_CNN, self).__init__()\n        self.hidden_size = hidden_size\n        self.num_layers = num_layers\n\n        # Bidirectional LSTM Layer\n        self.lstm = nn.LSTM(input_size, hidden_size, num_layers, \n                            batch_first=True, dropout=dropout_prob, bidirectional=True)\n\n        # CNN Layer\n        self.conv1 = nn.Conv1d(in_channels=hidden_size * 2, out_channels=num_channels, kernel_size=kernel_size, padding=kernel_size // 2)\n        self.relu = nn.ReLU()\n\n        # Transpose Convolution Layer for Upsampling\n        self.transpose_conv = nn.ConvTranspose1d(in_channels=num_channels,\n                                                 out_channels=num_channels,\n                                                 kernel_size=SAMPLE_FREQ,\n                                                 stride=SAMPLE_FREQ)\n        \n        # Fully Connected Layer (Adjust output size if needed)\n        self.fc = nn.Linear(num_channels, output_size)\n\n        # Layer Normalization\n        self.layer_norm = nn.LayerNorm(hidden_size * 2)  # Adjust for bidirectional\n\n        # Dropout Layer\n        self.dropout = nn.Dropout(dropout_prob)\n\n    def forward(self, x):\n        # Initialize hidden state and cell state for bidirectional LSTM\n        h0 = torch.zeros(self.num_layers * 2, x.size(0), self.hidden_size, device=x.device)  # * 2 for bidirectional\n        c0 = torch.zeros(self.num_layers * 2, x.size(0), self.hidden_size, device=x.device)  # * 2 for bidirectional\n\n        # Forward propagate LSTM\n        out, _ = self.lstm(x, (h0, c0))\n\n        # Apply Layer Normalization\n        out = self.layer_norm(out)\n\n        # Apply Dropout\n        out = self.dropout(out)\n\n        # CNN\n        out = out.permute(0, 2, 1) # Reshape for CNN [batch_size, features, seq_length]\n        out = self.conv1(out)\n        out = self.relu(out)\n\n        # Transpose Convolution for Upsampling (if needed)\n        out = self.transpose_conv(out)\n\n        # Fully Connected Layer (Adjust output size if needed)\n        out = out.permute(0, 2, 1) # Reshape for FC layer\n        out = self.fc(out)\n\n        return out\n","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.236364Z","iopub.execute_input":"2023-12-02T06:03:11.236749Z","iopub.status.idle":"2023-12-02T06:03:11.251508Z","shell.execute_reply.started":"2023-12-02T06:03:11.236716Z","shell.execute_reply":"2023-12-02T06:03:11.250447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_weight_decay(model, weight_decay=1e-5, skip_list=()):\n    decay = []\n    no_decay = []\n    for name, param in model.named_parameters():\n        if not param.requires_grad:\n            continue\n        if len(param.shape) == 1 or np.any([v in name.lower()  for v in skip_list]):\n            # print(name, 'no decay')\n            no_decay.append(param)\n        else:\n            # print(name, 'decay')\n            decay.append(param)\n    return [\n        {'params': no_decay, 'weight_decay': 0.},\n        {'params': decay, 'weight_decay': weight_decay}]","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.252487Z","iopub.execute_input":"2023-12-02T06:03:11.252814Z","iopub.status.idle":"2023-12-02T06:03:11.266619Z","shell.execute_reply.started":"2023-12-02T06:03:11.252782Z","shell.execute_reply":"2023-12-02T06:03:11.265553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# input_size = len(train_ds.feat_list)\nhidden_size = 128\nnum_layers = 2\n# output_size = len(train_ds.label_list)\n\ncriterion = FocalLoss(alpha=1.0, gamma=3.5) # 0~1 / 0~5\n\nmodel = ImprovedLSTM(input_size, hidden_size, num_layers, output_size, dropout_prob=0.3)\n# model = LSTM_CNN(input_size, hidden_size, num_layers, output_size, dropout_prob=0.2)\nmodel","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.267635Z","iopub.execute_input":"2023-12-02T06:03:11.267991Z","iopub.status.idle":"2023-12-02T06:03:11.300828Z","shell.execute_reply.started":"2023-12-02T06:03:11.267960Z","shell.execute_reply":"2023-12-02T06:03:11.299683Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"WD = 1e-2\nLR = 1e-3\n\noptimizer_parameters = add_weight_decay(model, weight_decay=WD, skip_list=['bias'])\noptimizer = AdamW(optimizer_parameters, lr=LR, eps=1e-6, betas=(0.9, 0.999))","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.302052Z","iopub.execute_input":"2023-12-02T06:03:11.302705Z","iopub.status.idle":"2023-12-02T06:03:11.309407Z","shell.execute_reply.started":"2023-12-02T06:03:11.302666Z","shell.execute_reply":"2023-12-02T06:03:11.308213Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"save_model_name = f'model_LSTM1_CV_{TRAIN_FOLD}.pth'\ntrainer = Trainer(train_dl, val_dl, model, criterion, optimizer, save_model_name)\nnum_epochs = 5\ntrainer.fit(num_epochs)","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:03:11.311623Z","iopub.execute_input":"2023-12-02T06:03:11.312282Z","iopub.status.idle":"2023-12-02T06:08:51.621653Z","shell.execute_reply.started":"2023-12-02T06:03:11.312237Z","shell.execute_reply":"2023-12-02T06:08:51.619897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"trainer.plot_history()","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:51.623997Z","iopub.execute_input":"2023-12-02T06:08:51.624939Z","iopub.status.idle":"2023-12-02T06:08:52.035203Z","shell.execute_reply.started":"2023-12-02T06:08:51.624884Z","shell.execute_reply":"2023-12-02T06:08:52.033596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plot","metadata":{}},{"cell_type":"code","source":"def expand(value):\n    expanded = value.cpu().numpy()\n    expanded = np.repeat(expanded, SAMPLE_FREQ, axis=1)\n    return expanded","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.036733Z","iopub.execute_input":"2023-12-02T06:08:52.037234Z","iopub.status.idle":"2023-12-02T06:08:52.044059Z","shell.execute_reply.started":"2023-12-02T06:08:52.037188Z","shell.execute_reply":"2023-12-02T06:08:52.042414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def lpf(wave, fs=12*60*24, fe=30, n=3):\n    nyq = fs / 2.0\n    b, a = signal.butter(1, fe/nyq, btype='low')\n    for i in range(0, n):\n        wave = signal.filtfilt(b, a, wave)\n    return wave","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.045696Z","iopub.execute_input":"2023-12-02T06:08:52.046143Z","iopub.status.idle":"2023-12-02T06:08:52.058258Z","shell.execute_reply.started":"2023-12-02T06:08:52.046098Z","shell.execute_reply":"2023-12-02T06:08:52.056634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prediction(model, data, device, lpf_flag = True):\n    with torch.no_grad():\n        model.eval()\n        \n        inputs, steps, targets = data\n        inputs, steps = inputs.to(device), steps.to(device)\n\n        # Expand the steps for X axis\n        steps = expand(steps)\n        x_values = np.arange(0, steps[:, :, 0].size, 1)\n\n        # Prediction\n        outputs = model(inputs)\n        probabilities = torch.sigmoid(outputs)\n\n        onset_predictions = probabilities[:, :, 0].cpu().numpy()\n        wakeup_predictions = probabilities[:, :, 1].cpu().numpy()\n        if lpf_flag:\n            onset_predictions = lpf(onset_predictions)\n            wakeup_predictions = lpf(wakeup_predictions)\n        onset_pred = onset_predictions[0, :]\n        wakeup_pred = wakeup_predictions[0, :]\n        onset_pred_index = argrelmax(onset_pred, order=12 * 60 * 6)[0]  # 12*60*6 = 6 hours\n        wakeup_pred_index = argrelmax(wakeup_pred, order=12 * 60 * 6)[0]  # 12*60*6 = 6 hours\n\n        # Ground Truth\n        onset_ground_truth = targets[:, :, 0].cpu().numpy()\n        wakeup_ground_truth = targets[:, :, 1].cpu().numpy()\n        onset_gt = onset_ground_truth[0, :]\n        wakeup_gt = wakeup_ground_truth[0, :]\n\n        onset = [onset_gt, onset_pred, onset_pred_index]\n        wakeup = [wakeup_gt, wakeup_pred, wakeup_pred_index]\n\n        return onset, wakeup, x_values","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.064896Z","iopub.execute_input":"2023-12-02T06:08:52.065284Z","iopub.status.idle":"2023-12-02T06:08:52.077017Z","shell.execute_reply.started":"2023-12-02T06:08:52.065253Z","shell.execute_reply":"2023-12-02T06:08:52.075390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_pred(x_values, onset, wakeup):\n    onset_gt, onset_pred, onset_pred_index = onset\n    wakeup_gt, wakeup_pred, wakeup_pred_index = wakeup\n    \n    plt.figure(figsize=(20, 6))\n\n    # Onset actual and predictions\n    plt.plot(x_values, onset_gt, label='Onset Actual', linewidth=1, linestyle='-', color=\"#FF6347\")\n    plt.plot(x_values, onset_pred, label='Onset Prediction', linewidth=0.5, linestyle='--', color=\"#FF6347\", alpha=0.7)\n\n    # Wakeup actual and predictions\n    plt.plot(x_values, wakeup_gt, label='Wakeup Actual', linewidth=1, linestyle='-', color=\"#4682B4\")\n    plt.plot(x_values, wakeup_pred, label='Wakeup Prediction', linewidth=0.5, linestyle='--', color=\"#4682B4\", alpha=0.7)\n\n    # Prediction markers\n    plt.scatter(onset_pred_index, onset_pred[onset_pred_index], label='Onset Predicted Points', marker='o', s=50, color=\"#FF6347\", alpha=0.8)\n    plt.scatter(wakeup_pred_index, wakeup_pred[wakeup_pred_index], label='Wakeup Predicted Points', marker='s', s=50, color=\"#4682B4\", alpha=0.8)\n\n    # Enhancements\n    plt.title('Prediction: Onset and Wakeup', fontsize=16)\n    plt.xlabel('Step', fontsize=14)\n    plt.ylabel('Probabilities', fontsize=14)\n    plt.legend()\n    plt.grid(True)\n    plt.show()\n#     plt.close()","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.078586Z","iopub.execute_input":"2023-12-02T06:08:52.079041Z","iopub.status.idle":"2023-12-02T06:08:52.093507Z","shell.execute_reply.started":"2023-12-02T06:08:52.078997Z","shell.execute_reply":"2023-12-02T06:08:52.092092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load best model\n# MODEL_PATH = './Test/model_LSTM_0.pth'\n# MODEL_PATH = '/kaggle/input/model-test/model_best.pth'\nmodel.load_state_dict(torch.load(os.path.join(model_path, save_model_name), map_location=device))","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.095080Z","iopub.execute_input":"2023-12-02T06:08:52.096009Z","iopub.status.idle":"2023-12-02T06:08:52.120861Z","shell.execute_reply.started":"2023-12-02T06:08:52.095921Z","shell.execute_reply":"2023-12-02T06:08:52.119360Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for k, (inputs, steps, targets) in enumerate(val_dl):\n#     inputs, steps, targets = next(iter(val_dl))\n#     onset, wakeup = prediction(model, [inputs, steps, targets], device)\n    onset, wakeup, x_values = prediction(model, [inputs, steps, targets], device, lpf_flag =False)\n    plot_pred(x_values, onset, wakeup)\n    if k==4: break","metadata":{"execution":{"iopub.status.busy":"2023-12-02T06:08:52.123247Z","iopub.execute_input":"2023-12-02T06:08:52.123758Z","iopub.status.idle":"2023-12-02T06:11:32.099749Z","shell.execute_reply.started":"2023-12-02T06:08:52.123712Z","shell.execute_reply":"2023-12-02T06:11:32.098005Z"},"trusted":true},"execution_count":null,"outputs":[]}]}