{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Explained Critical point RNN \n\n# This Notebook is taken from [zulqarnain ali's](https://www.kaggle.com/zulqarnainali) shared notebook.\nThis notebook is available at the publically shared code in the copetition hosted by The Child Mind Institute.\n\n## Introduction 🌟\nWelcome to this Jupyter notebook developed for the child mind institute Predict AI Model Runtime! This notebook is designed to help you participate in the competition and to Detect sleep onset and wake from wrist-worn accelerometer data.\n\n### Inspiration and Credits 🙌\nThis notebook is inspired by the work of werus23, available at [this Kaggle project](https://www.kaggle.com/code/werus23/sleep-critical-point-infer/notebook). I extend my gratitude to werus23 for sharing their insights and code.\n\n🙏 Thank you for taking the time to review my work, and please give it a thumbs-up if you found it valuable! 👍\n\n## Purpose 🎯\nThe primary purpose of this notebook is to:\n- Load and preprocess the competition data 📁\n- Engineer relevant features for model training 🏋️‍♂️\n- Train predictive models to make target variable predictions 🧠\n- Submit predictions to the competition environment 📤\n\n## Notebook Structure 📚\nThis notebook is structured as follows:\n1. **Data Preparation**: In this section, we load and preprocess the competition data.\n2. **Feature Engineering**: We generate and select relevant features for model training.\n3. **Model Training**: We train machine learning models on the prepared data.\n4. **Prediction and Submission**: We make predictions on the test data and submit them for evaluation.\n\n\n## How to Use 🛠️\nTo use this notebook effectively, please follow these steps:\n1. Ensure you have the competition data and environment set up.\n2. Execute each cell sequentially to perform data preparation, feature engineering, model training, and prediction submission.\n3. Customize and adapt the code as needed to improve model performance or experiment with different approaches.\n\n**Note**: Make sure to replace any placeholder paths or configurations with your specific information.\n\n## Acknowledgments 🙏\nWe acknowledge The Child Mind Institute organizers for providing the dataset and the competition platform.\n\nLet's get started! Feel free to reach out if you have any questions or need assistance along the way.\n","metadata":{}},{"cell_type":"markdown","source":"## Import necessary libraries and modules 📚","metadata":{}},{"cell_type":"code","source":"import pandas as pd           # Import and alias pandas (data manipulation) 🐼\nimport numpy as np            # Import and alias numpy (numerical operations) 🧮\nimport gc                    # Import garbage collection for memory management 🗑️\nimport time                  # Import time module for time-related operations ⏰\nimport json                  # Import JSON handling for data 📝\nfrom datetime import datetime  # Import datetime for date and time manipulation 📅\nimport matplotlib.pyplot as plt  # Import and alias matplotlib for data visualization 📊\nimport os                    # Import os for operating system functions 📂\nimport joblib                # Import joblib for job (de)serialization 🧰\nimport random                # Import random for generating random numbers 🎲\nimport math                  # Import math module for mathematical functions ➗\nfrom tqdm.auto import tqdm   # Import tqdm for progress bars 📊\n\n# Data science and modeling libraries 📊🧪\nfrom scipy.interpolate import interp1d  # Interpolation functions for data 📈\nfrom math import pi, sqrt, exp        # Math constants and functions 🔵📏√\nimport sklearn, sklearn.model_selection  # Import scikit-learn for ML tasks 🧪📊\nimport torch                 # Import PyTorch for deep learning 🔥\nfrom torch import nn, Tensor  # Import neural network components 🧯\nimport torch.nn.functional as F  # Import PyTorch's functional API 🧯📊\nfrom torch.utils.data import DataLoader, Dataset, SubsetRandomSampler  # Data loading utilities 🧮📊\nfrom sklearn.metrics import average_precision_score  # Import a metric for evaluation 🧪📈\nfrom timm.scheduler import CosineLRScheduler  # Scheduler for learning rate adjustment 📅⏰\n\n# Set the style for matplotlib plots 📊📈\nplt.style.use(\"ggplot\")\n\n# Import libraries for working with Parquet files 📄\nfrom pyarrow.parquet import ParquetFile  # PyArrow Parquet file handling 📄\nimport pyarrow as pa          # PyArrow for data serialization 📄\nimport ctypes                 # Python library for calling C functions 📄\n\n# Set the number of threads for interoperation and processing 🔢\ntorch.set_num_interop_threads(4)  # Set the number of interop threads 🔢\ntorch.set_num_threads(4)     # Set the number of CPU threads 🔢\n\n# Determine if CUDA (GPU) is available, and set the device accordingly 💻\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'  # Set device to 'cuda' if GPU is available\n","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":7.964505,"end_time":"2023-10-18T23:14:08.173454","exception":false,"start_time":"2023-10-18T23:14:00.208949","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:19.860047Z","iopub.execute_input":"2023-10-22T05:44:19.860437Z","iopub.status.idle":"2023-10-22T05:44:23.848168Z","shell.execute_reply.started":"2023-10-22T05:44:19.860408Z","shell.execute_reply":"2023-10-22T05:44:23.847186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define paths to data files 📂","metadata":{}},{"cell_type":"markdown","source":"**Cell 2**\n\n**Explaination**:\n\nThis code defines a set of classes and functions for handling data loading, cleaning, and memory optimization. Let's break down each part:\n\n1. `PATHS` class:\n   - It defines paths to various data files, including submission data, training events data, training series data (in CSV format), and test series data (in Parquet format).\n\n2. `CFG` class:\n   - It defines configuration settings, such as `DEMO_MODE`, which can be set to `True` or `False` to enable or disable demo mode.\n\n3. `data_reader` class:\n   - It is responsible for handling data loading and processing.\n   - `__init__` method:\n     - Initializes the class with mappings for data loading and sets the demo mode.\n   - `verify` method:\n     - Verifies whether a given data name is valid. If it's not a valid data name, it prints an error message.\n   - `cleaning` method:\n     - Cleans the data by removing rows with missing timestamps.\n   - `reduce_memory_usage` method:\n     - Reduces memory usage by modifying data types (e.g., changing data types to use less memory).\n   - `load_data` method:\n     - Loads data from the provided data name, either from a CSV or Parquet file.\n     - Depending on the demo mode setting, it loads either a limited number of rows or the entire dataset.\n     - If the data has timestamps, it calls the `cleaning` method to remove rows with missing timestamps.\n     - After loading the data, it calls the `reduce_memory_usage` method to optimize memory usage.\n\n\n","metadata":{}},{"cell_type":"code","source":"class PATHS:\n    MAIN_DIR = \"/kaggle/input/child-mind-institute-detect-sleep-states/\"\n    # CSV FILES : \n    SUBMISSION = MAIN_DIR + \"sample_submission.csv\"\n    TRAIN_EVENTS = MAIN_DIR + \"train_events.csv\"\n    # PARQUET FILES:\n    TRAIN_SERIES = MAIN_DIR + \"train_series.parquet\"\n    TEST_SERIES = MAIN_DIR + \"test_series.parquet\"\nclass CFG:\n    DEMO_MODE = True\nclass data_reader:\n    def __init__(self, demo_mode):\n        super().__init__()\n        # MAPPING FOR DATA LOADING :\n        self.names_mapping = {\n            \"submission\" : {\"path\" : PATHS.SUBMISSION, \"is_parquet\" : False, \"has_timestamp\" : False}, \n            \"train_events\" : {\"path\" : PATHS.TRAIN_EVENTS, \"is_parquet\" : False, \"has_timestamp\" : True},\n            \"train_series\" : {\"path\" : PATHS.TRAIN_SERIES, \"is_parquet\" : True, \"has_timestamp\" : True},\n            \"test_series\" : {\"path\" : PATHS.TEST_SERIES, \"is_parquet\" : True, \"has_timestamp\" : True}\n        }\n        self.valid_names = [\"submission\", \"train_events\", \"train_series\", \"test_series\"]\n        self.demo_mode = demo_mode\n    \n    def verify(self, data_name):\n        \"function for data name verification\"\n        if data_name not in self.valid_names:\n            print(\"PLEASE ENTER A VALID DATASET NAME, VALID NAMES ARE : \", valid_names)\n        return\n    \n    def cleaning(self, data):\n        \"cleaning function : drop na values\"\n        before_cleaning = len(data)\n        print(\"Number of missing timestamps : \", len(data[data[\"timestamp\"].isna()]))\n        data = data.dropna(subset=[\"timestamp\"])\n        after_cleaning = len(data)\n        print(\"Percentage of removed rows : {:.1f}%\".format(100 * (before_cleaning - after_cleaning) / before_cleaning) )\n#         print(data.isna().any())\n#         data = data.bfill()\n        return data\n    \n    @staticmethod\n    def reduce_memory_usage(data):\n        \"iterate through all the columns of a dataframe and modify the data type to reduce memory usage.\"\n        start_mem = data.memory_usage().sum() / 1024**2\n        print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n        for col in data.columns:\n            col_type = data[col].dtype    \n            if col_type != object:\n                c_min = data[col].min()\n                c_max = data[col].max()\n                if str(col_type)[:3] == 'int':\n                    if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                        data[col] = data[col].astype(np.int8)\n                    elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                        data[col] = data[col].astype(np.int16)\n                    elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                        data[col] = data[col].astype(np.int32)\n                    elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                        data[col] = data[col].astype(np.int64)  \n                else:\n                    if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                        data[col] = data[col].astype(np.float16)\n                    elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                        data[col] = data[col].astype(np.float32)\n                    else:\n                        data[col] = data[col].astype(np.float64)\n            else:\n                data[col] = data[col].astype('category')\n\n        end_mem = data.memory_usage().sum() / 1024**2\n        print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n        print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n        return data\n    \n    def load_data(self, data_name):\n        \"function for data loading\"\n        self.verify(data_name)\n        data_props = self.names_mapping[data_name]\n        if data_props[\"is_parquet\"]:\n            if self.demo_mode:\n                pf = ParquetFile(data_props[\"path\"]) \n                demo_rows = next(pf.iter_batches(batch_size=20_000)) \n                data = pa.Table.from_batches([demo_rows]).to_pandas()\n            else:\n                data = pd.read_parquet(data_props[\"path\"])\n        else:\n            if self.demo_mode:\n                data = pd.read_csv(data_props[\"path\"], nrows=20_000)\n            else:\n                data = pd.read_csv(data_props[\"path\"])\n                \n        gc.collect()\n        if data_props[\"has_timestamp\"]:\n            print('cleaning')\n            data = self.cleaning(data)\n            gc.collect()\n        data = self.reduce_memory_usage(data)\n        return data","metadata":{"papermill":{"duration":0.022683,"end_time":"2023-10-18T23:14:08.19881","exception":false,"start_time":"2023-10-18T23:14:08.176127","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:23.850479Z","iopub.execute_input":"2023-10-22T05:44:23.851397Z","iopub.status.idle":"2023-10-22T05:44:23.881156Z","shell.execute_reply.started":"2023-10-22T05:44:23.851355Z","shell.execute_reply":"2023-10-22T05:44:23.880162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Initialize data reader and load test_series data 🆔","metadata":{}},{"cell_type":"markdown","source":"**Cell 3**\n\n**Explaination**:\n\n\n\n1. `reader = data_reader(demo_mode=False)`: This line creates an instance of the `data_reader` class with `demo_mode` set to `False`. The `data_reader` class is assumed to be a custom class responsible for data loading and management.\n\n2. `test_series = reader.load_data(data_name=\"test_series\")`: It uses the `reader` object to load data with the name \"test_series.\" This data is stored in the variable `test_series`.\n\n3. `ids = test_series.series_id.unique()`: It extracts unique series IDs from the `test_series` data using the `unique()` method. The resulting unique series IDs are stored in the `ids` variable.\n\n4. `gc.collect()`: This line triggers the Python garbage collector to free up memory by collecting and cleaning up any unreferenced objects, helping to manage memory usage efficiently.\n\nThis code is for data processing pipeline where data is loaded, specific information is extracted (unique series IDs), and memory management is handled by garbage collection.","metadata":{}},{"cell_type":"code","source":"# Create a data_reader object with demo_mode set to False 🧑‍💻\nreader = data_reader(demo_mode=False)\n\n# Load the 'test_series' data using the data_reader 📥\ntest_series = reader.load_data(data_name=\"test_series\")\n\n# Get unique series IDs from the 'test_series' data 🆔\nids = test_series.series_id.unique()\n\n# Perform garbage collection to free up memory 🗑️\ngc.collect()\n","metadata":{"papermill":{"duration":0.609886,"end_time":"2023-10-18T23:14:08.81095","exception":false,"start_time":"2023-10-18T23:14:08.201064","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:23.883233Z","iopub.execute_input":"2023-10-22T05:44:23.88429Z","iopub.status.idle":"2023-10-22T05:44:24.44809Z","shell.execute_reply.started":"2023-10-22T05:44:23.884247Z","shell.execute_reply":"2023-10-22T05:44:24.446864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Cell 4**\n\n**Explaination**:\n\nThe provided code defines two classes, `ResidualBiGRU` and `MultiResidualBiGRU`, which are components for a neural network model:\n\n1. `ResidualBiGRU` class:\n\n   This class defines a single bidirectional GRU layer with residual connections.\n\n   - `__init__` method:\n     - It initializes the class with parameters for the hidden size of the GRU, the number of layers, and whether it should be bidirectional.\n     - It creates a bidirectional GRU layer with the specified configuration.\n     - It defines fully connected layers (`fc1` and `fc2`) and LayerNorms (`ln1` and `ln2`) for residual connections.\n     \n   - `forward` method:\n     - It takes an input `x` and an optional hidden state `h` as arguments.\n     - It passes the input through the bidirectional GRU layer.\n     - It applies fully connected layers and LayerNorm for residual connections, using ReLU activation.\n     - It implements a skip connection (residual connection) by adding the original input `x` to the processed output.\n     - It returns the output and the new hidden state.\n\n2. `MultiResidualBiGRU` class:\n\n   This class creates a neural network model with multiple layers of `ResidualBiGRU` modules.\n\n   - `__init__` method:\n     - It initializes the class with parameters for the input size, hidden size, output size, the number of layers, and whether the GRU layers should be bidirectional.\n     - It defines an initial fully connected layer (`fc_in`) and LayerNorm for the input data.\n     - It creates a list of `ResidualBiGRU` layers, where each layer has a hidden size and is either unidirectional or bidirectional.\n     - It defines an output fully connected layer (`fc_out`).\n\n   - `forward` method:\n     - It takes an input `x` and an optional list of hidden states `h`.\n     - If no hidden states are provided, it initializes them as `None`.\n     - It applies the initial fully connected layer and LayerNorm to the input data.\n     - It iterates through the list of `ResidualBiGRU` layers, passing the input and the corresponding hidden state through each layer and collecting the new hidden states.\n     - It applies the output fully connected layer to the final output.\n     - It returns the final output and a list of hidden states.\n\nThese classes are designed for building a deep neural network with residual connections using GRU layers. The `MultiResidualBiGRU` class stacks multiple `ResidualBiGRU` layers to create a deep architecture for sequence processing tasks.","metadata":{}},{"cell_type":"code","source":"class ResidualBiGRU(nn.Module):\n    def __init__(self, hidden_size, n_layers=1, bidir=True):\n        super(ResidualBiGRU, self).__init__()\n\n        self.hidden_size = hidden_size\n        self.n_layers = n_layers\n\n        self.gru = nn.GRU(\n            hidden_size,\n            hidden_size,\n            n_layers,\n            batch_first=True,\n            bidirectional=bidir,\n        )\n        dir_factor = 2 if bidir else 1\n        self.fc1 = nn.Linear(\n            hidden_size * dir_factor, hidden_size * dir_factor * 2\n        )\n        self.ln1 = nn.LayerNorm(hidden_size * dir_factor * 2)\n        self.fc2 = nn.Linear(hidden_size * dir_factor * 2, hidden_size)\n        self.ln2 = nn.LayerNorm(hidden_size)\n\n    def forward(self, x, h=None):\n        res, new_h = self.gru(x, h)\n        # res.shape = (batch_size, sequence_size, 2*hidden_size)\n\n        res = self.fc1(res)\n        res = self.ln1(res)\n        res = nn.functional.relu(res)\n\n        res = self.fc2(res)\n        res = self.ln2(res)\n        res = nn.functional.relu(res)\n\n        # skip connection\n        res = res + x\n\n        return res, new_h\n\nclass MultiResidualBiGRU(nn.Module):\n    def __init__(self, input_size, hidden_size, out_size, n_layers, bidir=True):\n        super(MultiResidualBiGRU, self).__init__()\n\n        self.input_size = input_size\n        self.hidden_size = hidden_size\n        self.out_size = out_size\n        self.n_layers = n_layers\n\n        self.fc_in = nn.Linear(input_size, hidden_size)\n        self.ln = nn.LayerNorm(hidden_size)\n        self.res_bigrus = nn.ModuleList(\n            [\n                ResidualBiGRU(hidden_size, n_layers=1, bidir=bidir)\n                for _ in range(n_layers)\n            ]\n        )\n        self.fc_out = nn.Linear(hidden_size, out_size)\n\n    def forward(self, x, h=None):\n        # if we are at the beginning of a sequence (no hidden state)\n        if h is None:\n            # (re)initialize the hidden state\n            h = [None for _ in range(self.n_layers)]\n\n        x = self.fc_in(x)\n        x = self.ln(x)\n        x = nn.functional.relu(x)\n\n        new_h = []\n        for i, res_bigru in enumerate(self.res_bigrus):\n            x, new_hi = res_bigru(x, h[i])\n            new_h.append(new_hi)\n\n        x = self.fc_out(x)\n#         x = F.normalize(x,dim=0)\n        return x, new_h  # log probabilities + hidden state","metadata":{"papermill":{"duration":0.015415,"end_time":"2023-10-18T23:14:08.828854","exception":false,"start_time":"2023-10-18T23:14:08.813439","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:24.450796Z","iopub.execute_input":"2023-10-22T05:44:24.451149Z","iopub.status.idle":"2023-10-22T05:44:24.468166Z","shell.execute_reply.started":"2023-10-22T05:44:24.451111Z","shell.execute_reply":"2023-10-22T05:44:24.466718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Cell 5**\n\n**Explaination**:\n\n\n```python\n# SleepDataset class to create a custom PyTorch dataset 🛏️\nclass SleepDataset(Dataset):\n    def __init(\n        self,\n        series_ids,\n        series,\n    ):\n        series_ids = series_ids\n        series = series.reset_index()\n        self.data = []\n\n        # Loop through series_ids and prepare the data for each visualization\n        for viz_id in tqdm(series_ids):\n            self.data.append(series.loc[(series.series_id==viz_id)].copy().reset_index())\n```\n- This code defines a custom PyTorch dataset called `SleepDataset` by inheriting from the `Dataset` class.\n- The `__init__` method initializes the dataset. It takes two arguments: `series_ids` and `series`, which represent the series IDs and the series data, respectively.\n- The `series_ids` argument is assigned to a local variable `series_ids`, but it's not used in the constructor.\n- The `series` data is reset to have a new index using `reset_index()`.\n- An empty list `self.data` is created to store the data for each visualization.\n\n```python\n    # Function to downsample sequences and generate features 📉\n    def downsample_seq_generate_features(self, feat, downsample_factor):\n        if len(feat) % 12 != 0:\n            feat = np.concatenate([feat, np.zeros(12 - ((len(feat)) % 12)) + feat[-1]])\n        feat = np reshape(feat, (-1, 12))\n        feat_mean = np.mean(feat, 1)\n        feat_std = np.std(feat, 1)\n        feat_median = np.median(feat, 1)\n        feat_max = np.max(feat, 1)\n        feat_min = np.min(feat, 1)\n\n        return np.dstack([feat_mean, feat_std, feat_median, feat_max, feat_min])[0]\n```\n- The `downsample_seq_generate_features` method takes a sequence of features `feat` and a `downsample_factor` as arguments. It downsamples the sequence and generates statistical features.\n- If the length of the input sequence is not a multiple of 12, it pads the sequence with zeros to make its length a multiple of 12.\n- The padded sequence is then reshaped into a 2D array where each row has 12 values.\n- Statistical features (mean, standard deviation, median, maximum, and minimum) are calculated along each row.\n- The features are stacked into a 3D array and returned.\n\n```python\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, index):\n        X = self.data[index][['anglez', 'enmo']].values.astype(np.float32)\n\n        # Generate features by downsampling sequences\n        X = np.concatenate([self.downsample_seq_generate_features(X[:, i], 12) for i in range(X shape[1])], -1)\n        X = torch.from_numpy(X)\n\n        return X\n```\n- The `__len__` method returns the length of the dataset, which is the number of visualizations stored in `self.data`.\n- The `__getitem__` method retrieves an item from the dataset at the specified `index`.\n- It extracts features 'anglez' and 'enmo' from the data at the given index, converts them to a NumPy array of type `float32`, and stores it in the variable `X`.\n- The `downsample_seq_generate_features` function is called to generate features by downsampling the sequences in `X`.\n- The features are concatenated horizontally, and a PyTorch tensor is created from the result. This tensor is returned as the item from the dataset.\n\n```python\n# Create a test dataset using the SleepDataset class 🛏️\ntest_ds = SleepDataset(test_series.series_id.unique(), test_series)\n\n# Cleanup to save memory\ndel test_series\ngc.collect()\n```\n- The `test_ds` variable is assigned a new instance of the `SleepDataset` class, which is initialized with unique series IDs from the `test_series` data.\n- After creating the dataset, there's some cleanup to release memory. The `test_series` variable is deleted, and `gc.collect()` is used to trigger garbage collection, freeing up any unreferenced memory.\n\nThis code defines a custom PyTorch dataset for sleep data and initializes a test dataset with it. The dataset processes the data to generate features for use.\n","metadata":{}},{"cell_type":"code","source":"class SleepDataset(Dataset):\n    def __init__(\n        self,\n        series_ids,\n        series,\n    ):\n        series_ids = series_ids\n        series = series.reset_index()\n        self.data = []\n        \n        for viz_id in tqdm(series_ids):\n            self.data.append(series.loc[(series.series_id==viz_id)].copy().reset_index())\n            \n    def downsample_seq_generate_features(self,feat, downsample_factor):\n        \n        if len(feat)%12!=0:\n            feat = np.concatenate([feat,np.zeros(12-((len(feat))%12))+feat[-1]])\n        feat = np.reshape(feat, (-1,12))\n        feat_mean = np.mean(feat,1)\n        feat_std = np.std(feat,1)\n        feat_median = np.median(feat,1)\n        feat_max = np.max(feat,1)\n        feat_min = np.min(feat,1)\n\n        return np.dstack([feat_mean,feat_std,feat_median,feat_max,feat_min])[0]\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, index):\n        X = self.data[index][['anglez','enmo']].values.astype(np.float32)\n        X = np.concatenate([self.downsample_seq_generate_features(X[:,i],12) for i in range(X.shape[1])],-1)\n        X = torch.from_numpy(X)\n        return X\ntest_ds = SleepDataset(test_series.series_id.unique(),test_series)\ndel test_series\ngc.collect()","metadata":{"papermill":{"duration":0.185209,"end_time":"2023-10-18T23:14:09.016533","exception":false,"start_time":"2023-10-18T23:14:08.831324","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:24.469646Z","iopub.execute_input":"2023-10-22T05:44:24.469982Z","iopub.status.idle":"2023-10-22T05:44:24.682604Z","shell.execute_reply.started":"2023-10-22T05:44:24.469953Z","shell.execute_reply":"2023-10-22T05:44:24.68183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Cell 6**\n\n**Explaination**:\n\nCertainly, let's explain these two variable assignments:\n\n1. `max_chunk_size = 24 * 60 * 100`: This line sets the variable `max_chunk_size` to a value. It's calculated as follows:\n   - `24` represents the number of hours in a day.\n   - `60` represents the number of minutes in an hour.\n   - `100` appears to be a scaling factor.\n   \n   So, `max_chunk_size` is set to a value that represents the maximum chunk size in minutes. It's equivalent to 144,000 minutes, which is 100 days (24 hours * 60 minutes * 100). This value is used in the code to process data in chunks, with each chunk covering a maximum of 100 days of data.\n\n2. `min_interval = 30`: This line sets the variable `min_interval` to a value of `30`. This variable represents the minimum interval in minutes. It's used in the code to define a rolling window within which the maximum scores are calculated. The rolling window will have a width of 30 minutes. If you're working with time-series data, this parameter controls how closely the algorithm looks for maximum scores within a certain time interval.\n","metadata":{}},{"cell_type":"code","source":"max_chunk_size = 24 * 60 * 100  # Maximum chunk size in minutes 🕒\nmin_interval = 30  # Minimum interval in minutes 🕒\n","metadata":{"papermill":{"duration":0.010196,"end_time":"2023-10-18T23:14:09.029435","exception":false,"start_time":"2023-10-18T23:14:09.019239","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:24.683537Z","iopub.execute_input":"2023-10-22T05:44:24.683861Z","iopub.status.idle":"2023-10-22T05:44:24.688788Z","shell.execute_reply.started":"2023-10-22T05:44:24.683833Z","shell.execute_reply":"2023-10-22T05:44:24.687811Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Cell 7**\n\n**Explaination**:\n\n\n```python\n# Load the pre-trained MultiResidualBiGRU model for evaluation 🛏️\nmodel = MultiResidualBiGRU(input_size=10, hidden_size=64, out_size=2, n_layers=5).to(device).eval()\nmodel.load_state_dict(torch.load(f'/kaggle/input/sleep-critical-point-train/model_best.pth', map_location=device))\n```\n1. This code loads a pre-trained model called `MultiResidualBiGRU` for sleep data analysis. It sets the model to evaluation mode and loads the model's state dictionary from a saved file on the device.\n\n```python\n# Create an empty DataFrame for the submission data 📊\nsubmission = pd.DataFrame()\n```\n2. An empty DataFrame named `submission` is created to store the results of the analysis.\n\n```python\n# Loop through the test dataset\nfor i in range(len(test_ds)):\n```\n3. This code starts a loop that iterates over the test dataset (`test_ds`).\n\n```python\n    X = test_ds[i].half()\n    seq_len = X.shape[0]\n    h = None\n    pred = torch.zeros((len(X), 2)).half()\n```\n4. Inside the loop, it extracts a sample from the test dataset (`test_ds`) and converts it to half-precision (float16). It also initializes some variables (`seq_len`, `h`, and `pred`).\n\n```python\n    # Process data in chunks\n    for j in range(0, seq_len, max_chunk_size):\n```\n5. Another nested loop is started to process the data in chunks. It iterates over the sequence data in `max_chunk_size` increments.\n\n```python\n        y_pred, h = model(X[j: j + max_chunk_size].float(), h)\n        h = [hi.detach() for hi in h]\n        pred[j: j + max_chunk_size] = y_pred.detach()\n        del y_pred\n        gc.collect()\n```\n6. Within the nested loop, it feeds a chunk of data to the model and collects predictions (`y_pred`). It also updates the hidden state (`h`). Predictions are stored in the `pred` variable.\n\n```python\n    del h, X\n    gc.collect()\n    pred = pred.numpy()\n```\n7. After processing the chunk, it cleans up some variables and converts the `pred` tensor to a NumPy array.\n\n```python\n    series_id = ids[i]\n```\n8. It extracts the `series_id` associated with the current test sample.\n\n```python\n    # Calculate the number of days\n    days = len(pred) / (17280 / 12)\n```\n9. It calculates the number of days based on the length of the prediction data.\n\n```python\n    # Initialize arrays to store scores\n    scores0, scores1 = np.zeros(len(pred), dtype=np.float16), np.zeros(len(pred), dtype=np.float16)\n```\n10. Two arrays, `scores0` and `scores1`, are initialized to store scores. They are NumPy arrays with a data type of float16.\n\n```python\n    # Find the maximum scores in a rolling window\n    for index in range(len(pred)):\n```\n11. A loop is started to iterate over the predictions and calculate maximum scores within a rolling window.\n\n```python\n        if pred[index, 0] == max(pred[max(0, index - min_interval):index + min_interval, 0]):\n            scores0[index] = max(pred[max(0, index - min_interval):index + min_interval, 0])\n        if pred[index, 1] == max(pred[max(0, index - min_interval):index + min_interval, 1]):\n            scores1[index] = max(pred[max(0, index - min_interval):index + min_interval, 1])\n```\n12. Within the loop, it compares each prediction value to the maximum within a specified interval and updates the scores arrays accordingly.\n\n```python\n    # Identify candidates for \"onset\" and \"wakeup\" events\n    candidates_onset = np.argsort(scores0)[-max(1, round(days)):]\n    candidates_wakeup = np.argsort(scores1)[-max(1, round(days)):]\n```\n13. Candidates for \"onset\" and \"wakeup\" events are identified based on the highest scores.\n\n```python\n    # Extract the corresponding steps for \"onset\" and \"wakeup\" events\n    onset = test_ds.data[i][['step']].iloc[np.clip(candidates_onset * 12, 0, len(test_ds.data[i]) - 1)].astype(np.int32)\n    onset['event'] = 'onset'\n    onset['series_id'] = series_id\n    onset['score'] = scores0[candidates_onset]\n    \n    wakeup = test_ds.data[i][['step']].iloc[np.clip(candidates_wakeup * 12, 0, len(test_ds.data[i]) - 1)].astype(np.int32)\n    wakeup['event'] = 'wakeup'\n    wakeup['series_id'] = series_id\n    wakeup['score'] = scores1[candidates_wakeup]\n```\n14. Steps corresponding to the \"onset\" and \"wakeup\" events are extracted and stored in the `onset` and `wakeup` DataFrames.\n\n```python\n    # Concatenate event data to the submission DataFrame\n    submission = pd.concat([submission, onset, wakeup], axis=0)\n    del onset, wakeup, candidates_onset, candidates_wakeup, scores0, scores1, pred, series_id\n    gc.collect()\n```\n15. Event data is concatenated to the `submission` DataFrame, and some variables are cleaned up.\n\n```python\n# Sort and reset the index of the submission DataFrame\nsubmission = submission.sort_values(['series_id', 'step']).reset_index(drop=True)\nsubmission['row_id'] = submission.index.astype(int)\nsubmission['score'] = submission['score'].fillna(submission['score'].mean())\nsubmission = submission[['row_id', 'series_id', 'step', 'event', 'score']]\n```\n16. The `submission` DataFrame is sorted, and its index is reset. Additional columns ('row_id' and 'score') are added, and missing scores are filled with the mean score.\n\n```python\n# Save the submission data to a CSV file\nsubmission.to_csv('submission.csv', index=False)\n```\n17. The final submission data is saved to a CSV file named 'submission.csv' without including the index column.\n\n\n","metadata":{}},{"cell_type":"code","source":"model = MultiResidualBiGRU(input_size=10,hidden_size=64,out_size=2,n_layers=5).to(device).eval()\nmodel.load_state_dict(torch.load(f'/kaggle/input/sleep-critical-point-train/model_best.pth',map_location=device))\nsubmission = pd.DataFrame()\nfor i in range(len(test_ds)):\n    X = test_ds[i].half()\n    \n    seq_len = X.shape[0]\n    h = None\n    pred = torch.zeros((len(X),2)).half()\n    for j in range(0, seq_len, max_chunk_size):\n        y_pred, h = model(X[j: j + max_chunk_size].float(), h)\n        h = [hi.detach() for hi in h]\n        pred[j : j + max_chunk_size] = y_pred.detach()\n        del y_pred;gc.collect()\n    del h,X;gc.collect()\n    pred = pred.numpy()\n    \n    series_id = ids[i]\n    \n    days = len(pred)/(17280/12)\n    scores0,scores1 = np.zeros(len(pred),dtype=np.float16),np.zeros(len(pred),dtype=np.float16)\n    for index in range(len(pred)):\n        if pred[index,0]==max(pred[max(0,index-min_interval):index+min_interval,0]):\n            scores0[index] = max(pred[max(0,index-min_interval):index+min_interval,0])\n        if pred[index,1]==max(pred[max(0,index-min_interval):index+min_interval,1]):\n            scores1[index] = max(pred[max(0,index-min_interval):index+min_interval,1])\n    candidates_onset = np.argsort(scores0)[-max(1,round(days)):]\n    candidates_wakeup = np.argsort(scores1)[-max(1,round(days)):]\n    \n    onset = test_ds.data[i][['step']].iloc[np.clip(candidates_onset*12,0,len(test_ds.data[i])-1)].astype(np.int32)\n    onset['event'] = 'onset'\n    onset['series_id'] = series_id\n    onset['score']= scores0[candidates_onset]\n    wakeup = test_ds.data[i][['step']].iloc[np.clip(candidates_wakeup*12,0,len(test_ds.data[i])-1)].astype(np.int32)\n    wakeup['event'] = 'wakeup'\n    wakeup['series_id'] = series_id\n    wakeup['score']= scores1[candidates_wakeup]\n    submission = pd.concat([submission,onset,wakeup],axis=0)\n    del onset,wakeup,candidates_onset,candidates_wakeup,scores0,scores1,pred,series_id,\n    gc.collect()\nsubmission = submission.sort_values(['series_id','step']).reset_index(drop=True)\nsubmission['row_id'] = submission.index.astype(int)\nsubmission['score'] = submission['score'].fillna(submission['score'].mean())\nsubmission = submission[['row_id','series_id','step','event','score']]\nsubmission.to_csv('submission.csv',index=False)","metadata":{"papermill":{"duration":1.62678,"end_time":"2023-10-18T23:14:10.658889","exception":false,"start_time":"2023-10-18T23:14:09.032109","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:24.690695Z","iopub.execute_input":"2023-10-22T05:44:24.690982Z","iopub.status.idle":"2023-10-22T05:44:27.134192Z","shell.execute_reply.started":"2023-10-22T05:44:24.690958Z","shell.execute_reply":"2023-10-22T05:44:27.133235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission","metadata":{"papermill":{"duration":0.017502,"end_time":"2023-10-18T23:14:10.679758","exception":false,"start_time":"2023-10-18T23:14:10.662256","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-10-22T05:44:27.135687Z","iopub.execute_input":"2023-10-22T05:44:27.136677Z","iopub.status.idle":"2023-10-22T05:44:27.152675Z","shell.execute_reply.started":"2023-10-22T05:44:27.136642Z","shell.execute_reply":"2023-10-22T05:44:27.151474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Explore More! 👀\nThank you for exploring this notebook! If you found this notebook insightful or if it helped you in any way, I invite you to explore more of my work on my profile.\n\n👉 [Visit my Profile](https://www.kaggle.com/zulqarnainali) 👈\n\n## Feedback and Gratitude 🙏\nWe value your feedback! Your insights and suggestions are essential for our continuous improvement. If you have any comments, questions, or ideas to share, please don't hesitate to reach out.\n\n📬 Contact me via email: [zulqar445ali@gmail.com](mailto:zulqar445ali@gmail.com)\n\nI would like to express our heartfelt gratitude for your time and engagement. Your support motivates us to create more valuable content.\n\nHappy coding and best of luck in your data science endeavors! 🚀\n\n","metadata":{}}]}