{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Explained Baseline Solution 💨\n\n## Introduction 🌟\nWelcome to this Jupyter notebook developed for the Google - Fast or Slow? Predict AI Model Runtime! This notebook is designed to help you participate in the competition and to Detect sleep onset and wake from wrist-worn accelerometer data.\n\n### Inspiration and Credits 🙌\nThis notebook is inspired by the work of Bhukya Satheesh\n, available at [this Kaggle project](https://www.kaggle.com/code/satheeshbhukya1/google-fast-or-slow/notebook). I extend my gratitude to Bhukya Satheesh\n for sharing their insights and code.\n\n🌟 Explore my profile and other public projects, and don't forget to share your feedback! \n👉 [Visit my Profile](https://www.kaggle.com/zulqarnainali) 👈\n\n🙏 Thank you for taking the time to review my work, and please give it a thumbs-up if you found it valuable! 👍\n\n## Purpose 🎯\nThe primary purpose of this notebook is to:\n- Load and preprocess the competition data 📁\n- Engineer relevant features for model training 🏋️‍♂️\n- Train predictive models to make target variable predictions 🧠\n- Submit predictions to the competition environment 📤\n\n## Notebook Structure 📚\nThis notebook is structured as follows:\n1. **Data Preparation**: In this section, we load and preprocess the competition data.\n2. **Feature Engineering**: We generate and select relevant features for model training.\n3. **Model Training**: We train machine learning models on the prepared data.\n4. **Prediction and Submission**: We make predictions on the test data and submit them for evaluation.\n\n\n## How to Use 🛠️\nTo use this notebook effectively, please follow these steps:\n1. Ensure you have the competition data and environment set up.\n2. Execute each cell sequentially to perform data preparation, feature engineering, model training, and prediction submission.\n3. Customize and adapt the code as needed to improve model performance or experiment with different approaches.\n\n**Note**: Make sure to replace any placeholder paths or configurations with your specific information.\n\n## Acknowledgments 🙏\nWe acknowledge theChild Mind Institute organizers for providing the dataset and the competition platform.\n\nLet's get started! Feel free to reach out if you have any questions or need assistance along the way.\n👉 [Visit my Profile](https://www.kaggle.com/zulqarnainali) 👈","metadata":{}},{"cell_type":"markdown","source":"## 📚 Importing necessary libraries 📊\n","metadata":{}},{"cell_type":"code","source":"! pip install --upgrade bayesian-optimization\n\n# 📚 Importing necessary libraries 📊\nimport numpy as np              # NumPy for numerical operations\nimport pandas as pd             # Pandas for data manipulation\nimport plotly.express as px     # Plotly Express for interactive plotting\nimport matplotlib.pyplot as plt # Matplotlib for basic plotting\nimport seaborn as sns           # Seaborn for statistical data visualization\nimport random                   # Random for generating random numbers\nimport os                       # OS for interacting with the operating system\nimport gc                       # Garbage collector for memory management\nfrom copy import deepcopy      # Deepcopy for creating deep copies of objects\nfrom functools import partial  # Partial function application for function manipulation\nfrom itertools import combinations  # Combinations for creating combinations of elements\nfrom itertools import groupby  # Groupby for grouping elements in an iterable\nfrom tqdm import tqdm          # tqdm for progress bars\nimport polars as pl            # Polars for data manipulation\nimport datetime                # Datetime for date and time operations\n\n# 🧱 Importing specific functions and classes 🧱\nfrom sklearn.model_selection import train_test_split  # Splitting data into training and testing sets\nfrom sklearn.model_selection import StratifiedKFold, KFold  # Cross-validation techniques\nfrom sklearn.metrics import roc_auc_score, accuracy_score, log_loss  # Evaluation metrics\nfrom sklearn.model_selection import cross_validate  # Cross-validation scoring\nfrom sklearn.metrics import RocCurveDisplay, confusion_matrix, ConfusionMatrixDisplay, precision_score, average_precision_score  # Metrics and displays\nimport optuna  # Library for hyperparameter tuning\nimport xgboost as xgb  # XGBoost for gradient boosting\nimport lightgbm as lgb  # LightGBM for gradient boosting\nfrom bayes_opt import BayesianOptimization\nfrom sklearn.linear_model import LogisticRegression  # Logistic Regression\nfrom sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier  # Random Forest and Gradient Boosting\nfrom sklearn.pipeline import Pipeline  # Pipeline for building a sequence of data transformations\nfrom catboost import Pool  # CatBoost for gradient boosting\nfrom sklearn.ensemble import BaggingClassifier, StackingClassifier\n\n# ⚙️ Importing a custom metric function ⚙️\nfrom metric import score  # Importing a custom event detection AP score function\n\n# 📋 Define column names and tolerances for the score function 📋\ncolumn_names = {\n    'series_id_column_name': 'series_id',\n    'time_column_name': 'step',\n    'event_column_name': 'event',\n    'score_column_name': 'score',\n}\n\ntolerances = {\n    'onset': [12, 36, 60, 90, 120, 150, 180, 240, 300, 360], \n    'wakeup': [12, 36, 60, 90, 120, 150, 180, 240, 300, 360]\n}\n\n# 📊 Setting display options for Pandas DataFrames 📊\npd.set_option('display.max_columns', None)  # Show all columns\n\n# 🚫 Suppressing warnings 🚫\nimport warnings\nwarnings.filterwarnings(\"ignore\", category=UserWarning)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-09-27T15:25:52.650635Z","iopub.execute_input":"2023-09-27T15:25:52.651104Z","iopub.status.idle":"2023-09-27T15:28:36.513082Z","shell.execute_reply.started":"2023-09-27T15:25:52.651069Z","shell.execute_reply":"2023-09-27T15:28:36.511808Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 📂 Importing and transforming data 🔄\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination**\n\nThis code will do data transformations and loading data from input directory.\n\n1. `dt_transforms = [...]`: This defines a list called `dt_transforms` that will contain a series of transformations to be applied to a timestamp column in the data.\n\n2. `pl.col('timestamp').str.to_datetime()`: This line converts a column named 'timestamp' to datetime format using the `str.to_datetime()` method.\n\n3. `(pl.col('timestamp').str.to_datetime().dt.year() - 2000).cast(pl.UInt8).alias('year')`: This line extracts the year from the 'timestamp' column, subtracts 2000 from it, casts the result as an 8-bit unsigned integer, and assigns it an alias 'year'.\n\n4. `pl.col('timestamp').str.to_datetime().dt.month().cast(pl.UInt8).alias('month')`: Similar to line 3, this line extracts the month from the 'timestamp' column, casts it as an 8-bit unsigned integer, and assigns it an alias 'month'.\n\n5. `pl.col('timestamp').str.to_datetime().dt.day().cast(pl.UInt8).alias('day')`: This line extracts the day from the 'timestamp' column, casts it as an 8-bit unsigned integer, and assigns it an alias 'day'.\n\n6. `pl.col('timestamp').str.to_datetime().dt.hour().cast(pl.UInt8).alias('hour')`: Similar to lines 3 and 4, this line extracts the hour from the 'timestamp' column, casts it as an 8-bit unsigned integer, and assigns it an alias 'hour'.\n\n7. `data_transforms = [...]`: This defines another list called `data_transforms` that will contain transformations for columns other than the timestamp.\n\n8. `pl.col('anglez').cast(pl.Int16)`: This line casts the column 'anglez' to a 16-bit signed integer.\n\n9. `(pl.col('enmo') * 1000).cast(pl.UInt16)`: This line multiplies the 'enmo' column by 1000 and casts the result as a 16-bit unsigned integer.\n\n10. `train_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/train_series.parquet').with_columns(dt_transforms + data_transforms)`: This line reads a parquet file ('train_series.parquet'), applies the transformations defined in `dt_transforms` and `data_transforms`, and assigns the result to the `train_series` variable.\n\n11. `train_events = pl.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv').with_columns(dt_transforms)`: This line reads a CSV file ('train_events.csv'), applies the transformations defined in `dt_transforms`, and assigns the result to the `train_events` variable.\n\n12. `test_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet').with_columns(dt_transforms + data_transforms)`: Similar to line 10, this line reads a parquet file ('test_series.parquet'), applies the transformations defined in `dt_transforms` and `data_transforms`, and assigns the result to the `test_series` variable.\n\n13. `series_ids = train_events['series_id'].unique(maintain_order=True).to_list()`: This line extracts unique 'series_id' values from the `train_events` DataFrame while maintaining the original order and converts them to a Python list.\n\n14. `onset_counts = ...`: These lines calculate the counts of 'onset' events and 'wakeup' events for each 'series_id' and store them in `onset_counts` and `wakeup_counts` DataFrames.\n\n15. `counts = pl.DataFrame(...)`: This line creates a DataFrame called `counts` containing 'series_id', 'onset_counts', and 'wakeup_counts' columns by combining the results of the previous step.\n\n16. `count_mismatches = counts.filter(...)`: This line filters `counts` to find series where the 'onset_counts' do not match 'wakeup_counts'.\n\n17. `train_series = train_series.filter(...)` and `train_events = train_events.filter(...)`: These lines filter the `train_series` and `train_events` DataFrames to exclude series with count mismatches.\n\n18. `series_ids = train_events.drop_nulls()['series_id'].unique(maintain_order=True).to_list()`: This line updates the `series_ids` list by removing rows with null values in 'series_id' and extracting unique values while maintaining the original order.\n\nThis code loads and transforms data from different sources, applies various data transformations, and filters out series with count mismatches in 'onset' and 'wakeup' events.","metadata":{}},{"cell_type":"code","source":"# Column transformations for timestamp\ndt_transforms = [\n    pl.col('timestamp').str.to_datetime(),  # Convert timestamp to datetime\n    (pl.col('timestamp').str.to_datetime().dt.year() - 2000).cast(pl.UInt8).alias('year'),  # Extract and cast year\n    pl.col('timestamp').str.to_datetime().dt.month().cast(pl.UInt8).alias('month'),  # Extract and cast month\n    pl.col('timestamp').str.to_datetime().dt.day().cast(pl.UInt8).alias('day'),  # Extract and cast day\n    pl.col('timestamp').str.to_datetime().dt.hour().cast(pl.UInt8).alias('hour')  # Extract and cast hour\n]\n\n# Column transformations for data\ndata_transforms = [\n    pl.col('anglez').cast(pl.Int16),  # Casting 'anglez' to 16-bit integer\n    (pl.col('enmo') * 1000).cast(pl.UInt16)  # Convert 'enmo' to 16-bit unsigned integer\n]\n\n# Loading and transforming training series data\ntrain_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/train_series.parquet').with_columns(\n    dt_transforms + data_transforms\n)\n\n# Loading and transforming training events data\ntrain_events = pl.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv').with_columns(\n    dt_transforms\n)\n\n# Loading and transforming test series data\ntest_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet').with_columns(\n    dt_transforms + data_transforms\n)\n\n# Getting unique series IDs for convenience\nseries_ids = train_events['series_id'].unique(maintain_order=True).to_list()\n\n# Removing series with mismatched event counts (onset vs. wakeup)\nonset_counts = train_events.filter(pl.col('event') == 'onset').group_by('series_id').count().sort('series_id')['count']\nwakeup_counts = train_events.filter(pl.col('event') == 'wakeup').group_by('series_id').count().sort('series_id')['count']\n\ncounts = pl.DataFrame({'series_id': sorted(series_ids), 'onset_counts': onset_counts, 'wakeup_counts': wakeup_counts})\ncount_mismatches = counts.filter(counts['onset_counts'] != counts['wakeup_counts'])\n\n# Filtering out series with count mismatches\ntrain_series = train_series.filter(~pl.col('series_id').is_in(count_mismatches['series_id']))\ntrain_events = train_events.filter(~pl.col('series_id').is_in(count_mismatches['series_id']))\n\n# Updating the list of series IDs, excluding series with no non-null values\nseries_ids = train_events.drop_nulls()['series_id'].unique(maintain_order=True).to_list()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:17:53.665601Z","iopub.execute_input":"2023-09-27T15:17:53.666058Z","iopub.status.idle":"2023-09-27T15:17:53.970930Z","shell.execute_reply.started":"2023-09-27T15:17:53.666018Z","shell.execute_reply":"2023-09-27T15:17:53.969604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 🧮 Creating Features 📈\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination**\nThis code is focused on generating and adding various features to two DataFrames (`train_series` and `test_series`) based on different time windows and data transformations.\n\n1. `features, feature_cols = [pl.col('hour')], ['hour']`: This line initializes two lists, `features` and `feature_cols`. It starts with one feature, which is the 'hour' column, and one feature column name 'hour'.\n\n2. `for mins in [5, 30, 60*2, 60*8]:`: This is the beginning of a loop that iterates over different time window durations specified in minutes (5, 30, 120, and 480).\n\n3. `features += [...]`: This line appends new features to the `features` list. For each time window duration (`mins`), it calculates the following features for the 'enmo' column:\n   - Rolling mean with a window of 12 times the specified minutes (`12 * mins`) using `pl.col('enmo').rolling_mean(...)`.\n   - Rolling maximum with the same window using `pl.col('enmo').rolling_max(...)`.\n   It then casts these features to 16-bit unsigned integers and assigns aliases with the format 'enmo_Xm_mean' and 'enmo_Xm_max', where X is the time window duration in minutes.\n\n4. `feature_cols += [...]`: This line appends the corresponding feature column names to the `feature_cols` list, which are 'enmo_Xm_mean' and 'enmo_Xm_max' for each time window duration.\n\n5. The same steps are repeated for the 'anglez' column, resulting in features named 'anglez_Xm_mean' and 'anglez_Xm_max' for each time window duration.\n\n6. `id_cols = ['series_id', 'step', 'timestamp']`: This line defines a list of columns to keep in the final DataFrame. These columns are 'series_id', 'step', and 'timestamp'.\n\n7. `train_series = train_series.with_columns(features).select(id_cols + feature_cols)`: This line adds the calculated features to the `train_series` DataFrame using the `with_columns` method and then selects only the columns specified in `id_cols` and `feature_cols`. This effectively updates `train_series` with the newly generated features.\n\n8. `test_series = test_series.with_columns(features).select(id_cols + feature_cols)`: Similar to the previous line, this line adds the same calculated features to the `test_series` DataFrame and selects the specified columns.\n\nThis code generates features based on rolling statistics (mean and max) for both the 'enmo' and 'anglez' columns, with different time window durations. It then updates the `train_series` and `test_series` DataFrames with these calculated features while keeping the specified identifier and feature columns. This process is common in feature engineering for machine learning tasks, where additional features are created from existing data to improve model performance.","metadata":{}},{"cell_type":"code","source":"# Initializing features and feature column names\nfeatures, feature_cols = [pl.col('hour')], ['hour']\n\n# Generating features for different time windows\nfor mins in [5, 30, 60*2, 60*8]:\n    # Enmo rolling mean and max\n    features += [\n        pl.col('enmo').rolling_mean(12 * mins, center=True, min_periods=1).abs().cast(pl.UInt16).alias(f'enmo_{mins}m_mean'),\n        pl.col('enmo').rolling_max(12 * mins, center=True, min_periods=1).abs().cast(pl.UInt16).alias(f'enmo_{mins}m_max')\n    ]\n\n    feature_cols += [\n        f'enmo_{mins}m_mean', f'enmo_{mins}m_max'\n    ]\n\n    # Anglez and Enmo first variations\n    for var in ['enmo', 'anglez']:\n        features += [\n            (pl.col(var).diff().abs().rolling_mean(12 * mins, center=True, min_periods=1) * 10).abs().cast(pl.UInt32).alias(f'{var}_1v_{mins}m_mean'),\n            (pl.col(var).diff().abs().rolling_max(12 * mins, center=True, min_periods=1) * 10).abs().cast(pl.UInt32).alias(f'{var}_1v_{mins}m_max')\n        ]\n\n        feature_cols += [\n            f'{var}_1v_{mins}m_mean', f'{var}_1v_{mins}m_max'\n        ]\n\n# Defining columns to keep\nid_cols = ['series_id', 'step', 'timestamp']\n\n# Adding calculated features to the training and testing series\ntrain_series = train_series.with_columns(\n    features\n).select(id_cols + feature_cols)\n\ntest_series = test_series.with_columns(\n    features\n).select(id_cols + feature_cols)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:17:53.972973Z","iopub.execute_input":"2023-09-27T15:17:53.973445Z","iopub.status.idle":"2023-09-27T15:17:53.989778Z","shell.execute_reply.started":"2023-09-27T15:17:53.973403Z","shell.execute_reply":"2023-09-27T15:17:53.988265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 📊 Creating Training Dataset Function 📉\n\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination** :\n\nThis Python function, `make_train_dataset`, is designed to create a training dataset from raw data, which includes both feature data (`train_data`) and event information (`train_events`). It returns feature data `X` and target labels `y` for training our machine learning model.:\n\n1. `series_ids = train_data['series_id'].unique(maintain_order=True).to_list()`: This line extracts unique 'series_id' values from the `train_data` DataFrame while maintaining the original order and converts them to a Python list. These unique series identifiers will be used to process data for each individual series.\n\n2. `X, y = pl.DataFrame(), pl.DataFrame()`: This line initializes two empty DataFrames, `X` for features and `y` for target labels.\n\n3. `for idx in tqdm(series_ids):`: This loop iterates through each unique 'series_id' in the `series_ids` list, using `tqdm` to display a progress bar.\n\n4. `sample = train_data.filter(...)`: Within the loop, this line filters the `train_data` DataFrame to obtain the data for the current 'series_id'. It then normalizes the selected columns (except 'hour') by dividing each column by its standard deviation and casting the result as a 32-bit floating-point number (Float32).\n\n5. `events = train_events.filter(...)`: This line filters the `train_events` DataFrame to obtain the events data for the current 'series_id'.\n\n6. `if drop_nulls: ...`: If the `drop_nulls` parameter is set to `True`, this section removes data points in the `sample` DataFrame that correspond to dates where no events were recorded. It does this by filtering the `sample` DataFrame based on matching dates between `sample` and `events` using the 'timestamp' column.\n\n7. `X = X.vstack(...)`: This line vertically stacks the current 'sample' DataFrame onto the existing 'X' DataFrame, combining the features for multiple series.\n\n8. `onsets = events.filter(...)`: This line extracts the 'step' values corresponding to 'onset' events from the 'events' DataFrame and stores them in the `onsets` list.\n\n9. `wakeups = events.filter(...)`: Similar to the previous line, this extracts the 'step' values corresponding to 'wakeup' events and stores them in the `wakeups` list.\n\n10. `y = y.vstack(...)`: This line constructs target labels ('asleep') based on the 'onsets' and 'wakeups' values. It checks for each data point in 'sample' if it falls within any sleep interval defined by 'onset' and 'wakeup' events, and casts the result as a Boolean value. The resulting labels are stacked vertically onto the 'y' DataFrame.\n\n11. `y = y.to_numpy().ravel()`: After processing all series, this line converts the 'y' DataFrame to a NumPy array and flattens it to a 1D array. This is typically done to match the format expected by many machine learning models.\n\n12. Finally, the function returns `X` (the feature matrix) and `y` (the target labels) as the output.\n\nThis function takes raw feature data and event information, processes them for each series, and creates a dataset suitable for training machine learning models. It provides a way to drop null data points if desired and generates target labels based on sleep event information.","metadata":{}},{"cell_type":"code","source":"def make_train_dataset(train_data, train_events, drop_nulls=False):\n    \"\"\"\n    Create a training dataset from raw data.\n\n    Args:\n        train_data (pl.DataFrame): Raw training data with features.\n        train_events (pl.DataFrame): Training events data with event information.\n        drop_nulls (bool): Whether to drop null data points where no events were recorded.\n\n    Returns:\n        X (pl.DataFrame): Features for training.\n        y (numpy.ndarray): Target labels for training.\n    \"\"\"\n    \n    series_ids = train_data['series_id'].unique(maintain_order=True).to_list()\n    X, y = pl.DataFrame(), pl.DataFrame()\n    \n    for idx in tqdm(series_ids):\n        # Normalizing sample features\n        sample = train_data.filter(pl.col('series_id') == idx).with_columns(\n            [(pl.col(col) / pl.col(col).std()).cast(pl.Float32) for col in feature_cols if col != 'hour']\n        )\n        \n        events = train_events.filter(pl.col('series_id') == idx)\n        \n        if drop_nulls:\n            # Removing datapoints on dates where no data was recorded\n            sample = sample.filter(\n                pl.col('timestamp').dt.date().is_in(events['timestamp'].dt.date())\n            )\n        \n        X = X.vstack(sample[id_cols + feature_cols])\n\n        onsets = events.filter((pl.col('event') == 'onset') & (pl.col('step') != None))['step'].to_list()\n        wakeups = events.filter((pl.col('event') == 'wakeup') & (pl.col('step') != None))['step'].to_list()\n\n        # NOTE: This will break if there are event series without any recorded onsets or wakeups\n        y = y.vstack(sample.with_columns(\n            sum([(onset <= pl.col('step')) & (pl.col('step') <= wakeup) for onset, wakeup in zip(onsets, wakeups)]).cast(pl.Boolean).alias('asleep')\n            ).select('asleep')\n        )\n    \n    y = y.to_numpy().ravel()\n    \n    return X, y\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:17:53.993470Z","iopub.execute_input":"2023-09-27T15:17:53.993851Z","iopub.status.idle":"2023-09-27T15:17:54.010254Z","shell.execute_reply.started":"2023-09-27T15:17:53.993819Z","shell.execute_reply":"2023-09-27T15:17:54.009251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 🕒 Get Events Function 🌙","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\n\nThis Python function, `get_events`, is designed to generate sleep event predictions and format them as a submission DataFrame. It takes a time series dataset (`series`) with features and a trained classifier (`classifier`) for predicting sleep events. \n\n1. `series_ids = series['series_id'].unique(maintain_order=True).to_list()`: This line extracts unique 'series_id' values from the `series` DataFrame while maintaining the original order and converts them to a Python list. These unique series identifiers will be used to process data for each individual series.\n\n2. `events = pl.DataFrame(...)`: This line initializes an empty DataFrame called `events` with a specific schema containing columns: 'series_id' (string), 'step' (integer), 'event' (string), and 'score' (float). This DataFrame will store sleep event predictions.\n\n3. `for idx in tqdm(series_ids):`: This loop iterates through each unique 'series_id' in the `series_ids` list, using `tqdm` to display a progress bar.\n\n4. `scale_cols = [...]`: This line identifies the columns in the `feature_cols` list (excluding 'hour') that have a standard deviation not equal to zero. These columns will be used for feature scaling later.\n\n5. `X = series.filter(...)`: Within the loop, this line filters the `series` DataFrame to obtain data for the current 'series_id'. It selects the columns specified in `id_cols` and `feature_cols`. The selected columns are then normalized by dividing them by their standard deviation and casting the result as a 32-bit floating-point number (Float32).\n\n6. `preds, probs = ...`: This line applies the trained classifier (`classifier`) to predict sleep events and obtain corresponding probabilities for each data point in the 'X' DataFrame.\n\n7. `X = X.with_columns(...)`: This code adds two new columns to the 'X' DataFrame:\n   - 'prediction': It contains the integer predictions (0 or 1) for sleep events.\n   - 'probability': It contains the predicted probabilities of being in a sleep state.\n\n8. `pred_onsets = X.filter(...)`, `pred_wakeups = X.filter(...)`: These lines extract predicted 'onset' and 'wakeup' time steps based on changes in the 'prediction' column. These steps are converted to Python lists.\n\n9. `if len(pred_onsets) > 0: ...`: This conditional block checks if there are any predicted sleep events ('onset' and 'wakeup' pairs). If there are, it proceeds to process and filter these events.\n\n10. Inside the conditional block, it ensures that predicted sleep periods have a minimum duration of 30 minutes (12 * 30 time steps).\n\n11. It calculates a 'score' for each sleep period by taking the mean probability over that period.\n\n12. It then adds sleep events to the `events` DataFrame, including 'onset' and 'wakeup' events, along with their corresponding 'score'.\n\n13. The loop continues to process each series, accumulating sleep events in the `events` DataFrame.\n\n14. After processing all series, it converts the `events` DataFrame to a Pandas DataFrame, resets the index, and renames the index column to 'row_id'.\n\n15. Finally, the function returns the formatted sleep events as a Pandas DataFrame.\n\nThis function takes a time series dataset and a trained classifier, predicts sleep events, filters and processes them, and returns the results as a Pandas DataFrame suitable for submission or further analysis.","metadata":{}},{"cell_type":"code","source":"def get_events(series, classifier):\n    \"\"\"\n    Takes a time series and a classifier and returns a formatted submission dataframe.\n\n    Args:\n        series (pl.DataFrame): Time series data with features.\n        classifier: A trained classifier for predicting sleep events.\n\n    Returns:\n        events (pd.DataFrame): Formatted submission dataframe with sleep events.\n    \"\"\"\n    \n    series_ids = series['series_id'].unique(maintain_order=True).to_list()\n    events = pl.DataFrame(schema={'series_id': str, 'step': int, 'event': str, 'score': float})\n\n    for idx in tqdm(series_ids):\n        # Collecting sample and normalizing features\n        scale_cols = [col for col in feature_cols if (col != 'hour') & (series[col].std() != 0)]\n        X = series.filter(pl.col('series_id') == idx).select(id_cols + feature_cols).with_columns(\n            [(pl.col(col) / series[col].std()).cast(pl.Float32) for col in scale_cols]\n        )\n\n        # Applying classifier to get predictions and scores\n        preds, probs = classifier.predict(X[feature_cols]), classifier.predict_proba(X[feature_cols])[:, 1]\n\n        # NOTE: Considered using rolling max to get sleep periods excluding <30 min interruptions,\n        # but ended up decreasing performance\n        X = X.with_columns(\n            pl.lit(preds).cast(pl.Int8).alias('prediction'),\n            pl.lit(probs).alias('probability')\n        )\n        \n        # Getting predicted onset and wakeup time steps\n        pred_onsets = X.filter(X['prediction'].diff() > 0)['step'].to_list()\n        pred_wakeups = X.filter(X['prediction'].diff() < 0)['step'].to_list()\n        \n        if len(pred_onsets) > 0:\n            # Ensuring all predicted sleep periods begin and end\n            if min(pred_wakeups) < min(pred_onsets):\n                pred_wakeups = pred_wakeups[1:]\n\n            if max(pred_onsets) > max(pred_wakeups):\n                pred_onsets = pred_onsets[:-1]\n\n            # Keeping sleep periods longer than 30 minutes\n            sleep_periods = [(onset, wakeup) for onset, wakeup in zip(pred_onsets, pred_wakeups) if wakeup - onset >= 12 * 30]\n\n            for onset, wakeup in sleep_periods:\n                # Scoring using mean probability over period\n                score = X.filter((pl.col('step') >= onset) & (pl.col('step') <= wakeup))['probability'].mean()\n\n                # Adding sleep event to dataframe\n                events = events.vstack(pl.DataFrame().with_columns(\n                    pl.Series([idx, idx]).alias('series_id'),\n                    pl.Series([onset, wakeup]).alias('step'),\n                    pl.Series(['onset', 'wakeup']).alias('event'),\n                    pl.Series([score, score]).alias('score')\n                ))\n\n    # Adding row id column and converting to a pandas DataFrame\n    events = events.to_pandas().reset_index().rename(columns={'index': 'row_id'})\n\n    return events\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:17:54.011853Z","iopub.execute_input":"2023-09-27T15:17:54.012564Z","iopub.status.idle":"2023-09-27T15:17:54.032207Z","shell.execute_reply.started":"2023-09-27T15:17:54.012524Z","shell.execute_reply":"2023-09-27T15:17:54.031140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 📊 Collecting and Sampling Data 📊","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\n\nThis line of code is selecting and collecting a subset of data from the `train_series` DataFrame for further processing.\n\n1. `train_data = ...`: This part of the code initializes a variable named `train_data` to store the selected data points.\n\n2. `train_series.filter(...)`: Here, the `train_series` DataFrame is filtered using the `filter` method. The condition being applied is `pl.col('series_id').is_in(series_ids)`. This condition filters the rows in `train_series` to include only those where the 'series_id' column matches any of the values in the `series_ids` list.\n\n3. `.collect()`: After filtering, the `collect` method is used to retrieve all the rows that satisfy the condition and create a new DataFrame containing these rows. This step essentially materializes the filtered DataFrame.\n\n4. `.sample(int(1e6))`: Finally, the `sample` method is applied to the collected DataFrame. It randomly selects a specified number of samples from the DataFrame. In this case, it's selecting 1 million samples (`int(1e6)` represents 1 million). The purpose of taking a random sample is often to reduce the size of the data for quicker experimentation or analysis when working with a large dataset.\n\nSo, the overall purpose of this line of code is to filter the `train_series` DataFrame to include only rows where the 'series_id' matches any of the values in the `series_ids` list, and then randomly sample 1 million rows from this filtered data. This can be useful when you want to work with a manageable subset of data for model training or analysis.","metadata":{}},{"cell_type":"code","source":"# We will collect datapoints and take 1 million samples\ntrain_data = train_series.filter(pl.col('series_id').is_in(series_ids)).collect().sample(int(1e6))\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:38:01.653561Z","iopub.execute_input":"2023-09-27T15:38:01.654016Z","iopub.status.idle":"2023-09-27T15:41:22.695367Z","shell.execute_reply.started":"2023-09-27T15:38:01.653980Z","shell.execute_reply":"2023-09-27T15:41:22.693674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 🚂 Creating Training Dataset 🚂\n\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\nThis code is using a custom function called `make_train_dataset` to generate a training dataset from two data sources: `train_data` and `train_events`. It then assigns the resulting features and labels to `X_train` and `y_train`, respectively.\n\n1. `X_train, y_train = ...`: This line initializes two variables, `X_train` and `y_train`, which will store the features and target labels for the training dataset, respectively.\n\n2. `make_train_dataset(train_data, train_events)`: This is a function call to `make_train_dataset` with two arguments: `train_data` and `train_events`. It's invoking the function and passing these two data sources to it.\n\n3. `make_train_dataset` is presumably a custom function defined elsewhere in the code (not shown here). This function is designed to create a training dataset from raw data. Based on the function signature and comments provided earlier, it takes the following arguments:\n   - `train_data`: A DataFrame containing raw training data with features.\n   - `train_events`: A DataFrame containing training events data with event information.\n   - `drop_nulls`: An optional boolean parameter that determines whether to drop null data points where no events were recorded (this parameter is not explicitly passed in the code snippet you provided).\n\n4. The function returns two values: `X` and `y`, where:\n   - `X` is a DataFrame containing features for training.\n   - `y` is a NumPy array containing the target labels for training.\n\n5. The line `X_train, y_train = make_train_dataset(train_data, train_events)` assigns the returned values from the `make_train_dataset` function to `X_train` and `y_train`, respectively. This essentially populates `X_train` with the features and `y_train` with the corresponding target labels for training.\n\nIn summary, this line of code is invoking a custom function, `make_train_dataset`, to generate a training dataset from `train_data` and `train_events`, and then assigns the resulting features and labels to `X_train` and `y_train`. This is a common step in preparing data for machine learning, where the data is split into features (X) and labels (y) for model training.","metadata":{}},{"cell_type":"code","source":"# Generating training dataset using the 'make_train_dataset' function\nX_train, y_train = make_train_dataset(train_data, train_events)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:41:22.697977Z","iopub.execute_input":"2023-09-27T15:41:22.698859Z","iopub.status.idle":"2023-09-27T15:41:33.842697Z","shell.execute_reply.started":"2023-09-27T15:41:22.698796Z","shell.execute_reply":"2023-09-27T15:41:33.839606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 🧹 Recovering Memory 🧹","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\n\n\nThis code is responsible for removing the `train_data` variable from memory and then explicitly triggering garbage collection to free up any associated memory resources.:\n\n1. `del train_data`: The `del` statement is used in Python to delete a reference to an object. In this case, it is deleting the reference to the `train_data` variable. This means that the variable `train_data` will no longer exist in the current scope, and any memory associated with it will become eligible for garbage collection.\n\n2. `gc.collect()`: The `gc` module in Python provides an interface to the garbage collection mechanism. Calling `gc.collect()` explicitly triggers garbage collection. Garbage collection is the process by which Python automatically reclaims memory that is no longer being used by the program, such as objects that have no remaining references. By invoking `gc.collect()`, the program is requesting that the Python garbage collector run immediately to reclaim any memory that can be freed.\n\nIn summary, this code snippet is cleaning up memory by removing the reference to the `train_data` variable and then explicitly requesting garbage collection to release any memory resources associated with it. This can be useful when working with large datasets to ensure that memory is managed efficiently and that resources are released when they are no longer needed.","metadata":{}},{"cell_type":"code","source":"# Removing the 'train_data' variable and triggering garbage collection to free up memory\ndel train_data\ngc.collect()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T15:21:44.615626Z","iopub.execute_input":"2023-09-27T15:21:44.616109Z","iopub.status.idle":"2023-09-27T15:21:44.845095Z","shell.execute_reply.started":"2023-09-27T15:21:44.616068Z","shell.execute_reply":"2023-09-27T15:21:44.843669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🧮 LightGBM Classifier Training 🚀\n\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\n\nThis code segment is configuring, creating, and training a LightGBM (Light Gradient Boosting Machine) classifier for a machine learning task. \n\n1. `lgb_opt`: This is a dictionary containing hyperparameters for the LightGBM classifier. The hyperparameters specified are:\n   - `'num_leaves'`: This parameter controls the number of leaves (terminal nodes) in each tree in the gradient boosting model. A higher value can make the model more expressive, but it can also lead to overfitting if set too high.\n   - `'learning_rate'`: This parameter determines the step size at each iteration during the gradient boosting process. A smaller learning rate often requires more iterations but can result in a better-performing model.\n   - `'random_state'`: This parameter sets the random seed for reproducibility. It ensures that the same results are obtained when the model is trained with the same data and hyperparameters.\n\n2. `my_classifier = lgb.LGBMClassifier(**lgb_opt)`: This line creates an instance of the LightGBM classifier with the specified hyperparameters (`lgb_opt`). The double asterisks (`**`) are used to unpack the dictionary `lgb_opt` and pass its contents as keyword arguments to the `LGBMClassifier` constructor. This initializes the classifier with the specified hyperparameters.\n\n3. `my_classifier.fit(X_train[feature_cols], y_train)`: This line trains the LightGBM classifier (`my_classifier`) on the training dataset. Here's what each part does:\n   - `X_train[feature_cols]`: This is the feature matrix used for training. It selects only the columns specified in `feature_cols`, which typically represent the features used for prediction.\n   - `y_train`: This is the target labels or ground truth values corresponding to the training dataset.\n\nThe `fit` method is called to train the classifier using the specified features and target labels. During training, the LightGBM algorithm will iteratively build an ensemble of decision trees, optimizing their structure to minimize a specified loss function. The trained model (`my_classifier`) can then be used for making predictions on new data.\n\nIn summary, this code segment sets hyperparameters for a LightGBM classifier, creates an instance of the classifier with those hyperparameters, and trains it on the provided training dataset (`X_train` and `y_train`). The resulting model can be used for making predictions on new data.","metadata":{}},{"cell_type":"code","source":"X_train, X_val, y_train, y_val = train_test_split(X_train[feature_cols], y_train, test_size=0.2, random_state=42)\n# Define the function to optimize\ndtrain = xgb.DMatrix(X_train, label=y_train)\n\n# Định nghĩa hàm tối ưu\ndef bo_tune_xgb(max_depth, gamma,learning_rate):\n    params = {'max_depth': int(max_depth),\n              'gamma': gamma,\n              'learning_rate':learning_rate,\n              'subsample': 0.8,\n              'eta': 0.1,\n              'eval_metric': 'auc'}\n    cv_result = xgb.cv(params, dtrain, nfold=5)\n    return cv_result['test-auc-mean'].iloc[-1]\n\n# Sử dụng Bayesian Optimization\nxgb_bo = BayesianOptimization(bo_tune_xgb, {'max_depth': (3, 10),\n                                             'gamma': (0, 1),\n                                             'learning_rate':(0,1)\n                                            },\n                              random_state = 0)\nxgb_bo.maximize(n_iter=5, init_points=8)\n\n# Lấy ra tham số tốt nhất\nparams = xgb_bo.max['params']\nparams['max_depth'] = int(params['max_depth'])\n\n# Huấn luyện mô hình với tham số tốt nhất\nmodel = xgb.XGBClassifier(**params).fit(X_train,y_train)\n\n# Dự đoán kết quả trên tập kiểm tra\ny_pred = model.predict(X_val)\n\n# Tính toán độ chính xác\naccuracy = accuracy_score(y_val, y_pred)\nprint(\"Accuracy: %.2f%%\" % (accuracy * 100.0))","metadata":{"execution":{"iopub.status.busy":"2023-09-27T16:03:17.812619Z","iopub.execute_input":"2023-09-27T16:03:17.813830Z","iopub.status.idle":"2023-09-27T16:20:18.337430Z","shell.execute_reply.started":"2023-09-27T16:03:17.813763Z","shell.execute_reply":"2023-09-27T16:20:18.335657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 📝 Getting Event Predictions and Saving Submission 📄\n\n","metadata":{}},{"cell_type":"markdown","source":"**Explaination**:\n\nThis code snippet is responsible for generating sleep event predictions for the test set using a trained LightGBM classifier (`my_classifier`) and saving the predictions to a CSV file.\n\n1. `submission = get_events(test_series.collect(), my_classifier)`: This line calls a function called `get_events` to obtain sleep event predictions for the test set. Here's what each part of the code does:\n   - `test_series.collect()`: The `collect` method is used to gather all the data points from the `test_series` DataFrame. This step effectively retrieves all the test data for processing.\n   - `my_classifier`: This is the trained LightGBM classifier that will be used to predict sleep events based on the test data.\n\n   The `get_events` function presumably takes this collected test data and the classifier as input and returns a formatted DataFrame containing sleep event predictions.\n\n2. `submission.to_csv('submission.csv', index=False)`: This line saves the DataFrame named `submission` to a CSV file named 'submission.csv'. Here's what each part of the code does:\n   - `submission`: This is the DataFrame containing the sleep event predictions generated using the test data.\n   - `.to_csv(...)`: This is a method call on the DataFrame that exports the data to a CSV file. The parameters are as follows:\n      - `'submission.csv'`: This specifies the name of the CSV file to which the DataFrame will be saved.\n      - `index=False`: This parameter indicates that the DataFrame's index (row numbers) should not be included in the CSV file. When `index` is set to `False`, the CSV file will not have an extra column for row numbers.\n\nIn summary, this code segment applies a trained LightGBM classifier to the test data to predict sleep events, collects these predictions into a DataFrame named `submission`, and then saves this DataFrame to a CSV file named 'submission.csv'. The resulting CSV file can be used for submission in a machine learning competition or for further analysis.","metadata":{}},{"cell_type":"code","source":"# Getting event predictions for the test set using the trained classifier\nsubmission = get_events(test_series.collect(), model)\n\n# Saving the submission dataframe to a CSV file\nsubmission.to_csv('submission.csv', index=False)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T16:20:18.338666Z","iopub.status.idle":"2023-09-27T16:20:18.339125Z","shell.execute_reply.started":"2023-09-27T16:20:18.338906Z","shell.execute_reply":"2023-09-27T16:20:18.338925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Explore More! 👀\nThank you for exploring this notebook! If you found this notebook insightful or if it helped you in any way, I invite you to explore more of my work on my profile.\n\n👉 [Visit my Profile](https://www.kaggle.com/zulqarnainali) 👈\n\n## Feedback and Gratitude 🙏\nWe value your feedback! Your insights and suggestions are essential for our continuous improvement. If you have any comments, questions, or ideas to share, please don't hesitate to reach out.\n\n📬 Contact me via email: [zulqar445ali@gmail.com](mailto:zulqar445ali@gmail.com)\n\nI would like to express our heartfelt gratitude for your time and engagement. Your support motivates us to create more valuable content.\n\nHappy coding and best of luck in your data science endeavors! 🚀\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}