\n Predicting blood glucose fluctuations is crucial for managing type 1 diabetes. Effective algorithms can alleviate some of the challenges faced by individuals with this condition. The primary goal of this project is to forecast blood glucose levels one hour ahead using the previous six hours of participant data.\n
\n\n Type 1 diabetes is a chronic condition in which the body no longer produces insulin, making it difficult to regulate glucose levels in the bloodstream. Individuals with this condition must inject insulin to manage their blood glucose levels, which can be impacted by various factors such as diet, physical activity, stress, and illness. Accurately predicting blood glucose changes is essential for effective diabetes management, as it helps individuals make informed decisions about insulin dosage and lifestyle choices.\n
\n\n This project utilizes a newly collected dataset from young adults in the UK with type 1 diabetes, who used continuous glucose monitors and insulin pumps, along with a smartwatch to gather activity data. The challenge lies in developing a model that can effectively predict future blood glucose levels based on this historical data.\n
\n\n Submissions will be evaluated based on Root Mean Square Error (RMSE) between the predicted blood glucose levels and the actual values collected, highlighting the importance of accurate forecasting in diabetes management.\n
\n\n","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport xgboost as xgb\nfrom sklearn.model_selection import train_test_split\nimport optuna\nfrom sklearn.metrics import mean_squared_error\nimport re\nfrom datetime import datetime\nimport warnings\nimport seaborn as sns\nwarnings.filterwarnings('ignore')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-11-19T14:18:41.758373Z","iopub.execute_input":"2024-11-19T14:18:41.758656Z","iopub.status.idle":"2024-11-19T14:18:46.934713Z","shell.execute_reply.started":"2024-11-19T14:18:41.758627Z","shell.execute_reply":"2024-11-19T14:18:46.933973Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df=pd.read_csv('/kaggle/input/brist1d/train.csv')\ndt=pd.read_csv('/kaggle/input/brist1d/test.csv')\ndf.shape,dt.shape","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:18:46.936145Z","iopub.execute_input":"2024-11-19T14:18:46.936575Z","iopub.status.idle":"2024-11-19T14:18:58.114309Z","shell.execute_reply.started":"2024-11-19T14:18:46.936540Z","shell.execute_reply":"2024-11-19T14:18:58.113340Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# pd.set_option('display.max_rows', None)\n# print([i for i in df.columns])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:18:58.115555Z","iopub.execute_input":"2024-11-19T14:18:58.115828Z","iopub.status.idle":"2024-11-19T14:18:58.119818Z","shell.execute_reply.started":"2024-11-19T14:18:58.115803Z","shell.execute_reply":"2024-11-19T14:18:58.118919Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df.head(10)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:18:58.121938Z","iopub.execute_input":"2024-11-19T14:18:58.122246Z","iopub.status.idle":"2024-11-19T14:18:58.177592Z","shell.execute_reply.started":"2024-11-19T14:18:58.122212Z","shell.execute_reply":"2024-11-19T14:18:58.176740Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df.info()","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:18:58.178476Z","iopub.execute_input":"2024-11-19T14:18:58.178726Z","iopub.status.idle":"2024-11-19T14:18:58.228315Z","shell.execute_reply.started":"2024-11-19T14:18:58.178701Z","shell.execute_reply":"2024-11-19T14:18:58.227334Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df.describe()","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:18:58.229412Z","iopub.execute_input":"2024-11-19T14:18:58.229777Z","iopub.status.idle":"2024-11-19T14:19:01.429019Z","shell.execute_reply.started":"2024-11-19T14:18:58.229731Z","shell.execute_reply":"2024-11-19T14:19:01.428232Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ###\n This step helps us identify the number of missing values in each column, which is essential for data preprocessing. By knowing where data is missing, \n we can decide on suitable strategies for handling these gaps, such as filling or dropping them, to ensure clean and reliable data for model training.\n
\n \n","metadata":{}},{"cell_type":"code","source":"nan_percentage = df.isnull().mean() * 100\nprint(nan_percentage)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:01.430217Z","iopub.execute_input":"2024-11-19T14:19:01.430629Z","iopub.status.idle":"2024-11-19T14:19:01.838397Z","shell.execute_reply.started":"2024-11-19T14:19:01.430591Z","shell.execute_reply":"2024-11-19T14:19:01.837454Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# for i in df.columns:\n# plt.figure(figsize=(8,8))\n# plt.plot(df[i],label=i)\n# plt.title(f\"{i} - with NaN values marked\")\n# plt.xlabel(\"Index\")\n# plt.ylabel(i)\n# plt.legend()\n# plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:01.839370Z","iopub.execute_input":"2024-11-19T14:19:01.839633Z","iopub.status.idle":"2024-11-19T14:19:01.843324Z","shell.execute_reply.started":"2024-11-19T14:19:01.839606Z","shell.execute_reply":"2024-11-19T14:19:01.842544Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# plt.figure(figsize=(15, 10))\n# sns.heatmap(df.isnull(), cbar=False, cmap=\"viridis\")\n# plt.title(\"Missing Values Heatmap\")\n# plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:01.844661Z","iopub.execute_input":"2024-11-19T14:19:01.845023Z","iopub.status.idle":"2024-11-19T14:19:01.856426Z","shell.execute_reply.started":"2024-11-19T14:19:01.844984Z","shell.execute_reply":"2024-11-19T14:19:01.855736Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n In this section, we manage missing values for different column categories. These columns\n are identified by specific prefixes: bg- for blood glucose, insulin- for insulin levels, \n and hr- for heart rate. We apply forward fill (ffill) and \n backward fill (bfill) techniques to fill gaps in these columns, ensuring data continuity by\n using previous or next valid entries for missing values within each category.\n
\n\n","metadata":{}},{"cell_type":"code","source":"bg_col=[col for col in df.columns if col.startswith('bg-')]\ndf[bg_col]=df[bg_col].fillna(method='ffill',axis=1)\ndf[bg_col]=df[bg_col].fillna(method='bfill',axis=1)\ndt[bg_col]=dt[bg_col].fillna(method='ffill',axis=1)\ndt[bg_col]=dt[bg_col].fillna(method='bfill',axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:01.859007Z","iopub.execute_input":"2024-11-19T14:19:01.859727Z","iopub.status.idle":"2024-11-19T14:19:02.228667Z","shell.execute_reply.started":"2024-11-19T14:19:01.859698Z","shell.execute_reply":"2024-11-19T14:19:02.227631Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"hr_col=[col for col in df.columns if col.startswith('hr-')]\ndf[hr_col]=df[hr_col].fillna(method='ffill',axis=1)\ndf[hr_col]=df[hr_col].fillna(method='bfill',axis=1)\ndt[hr_col]=dt[hr_col].fillna(method='ffill',axis=1)\ndt[hr_col]=dt[hr_col].fillna(method='bfill',axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:02.229954Z","iopub.execute_input":"2024-11-19T14:19:02.230344Z","iopub.status.idle":"2024-11-19T14:19:02.607403Z","shell.execute_reply.started":"2024-11-19T14:19:02.230306Z","shell.execute_reply":"2024-11-19T14:19:02.606550Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"cals_col=[col for col in df.columns if col.startswith('cals-')]\ndf[cals_col]=df[cals_col].fillna(method='ffill',axis=1)\ndf[cals_col]=df[cals_col].fillna(method='bfill',axis=1)\ndt[cals_col]=dt[cals_col].fillna(method='ffill',axis=1)\ndt[cals_col]=dt[cals_col].fillna(method='bfill',axis=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:02.608540Z","iopub.execute_input":"2024-11-19T14:19:02.608831Z","iopub.status.idle":"2024-11-19T14:19:03.045566Z","shell.execute_reply.started":"2024-11-19T14:19:02.608804Z","shell.execute_reply":"2024-11-19T14:19:03.044662Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"inc_col=[col for col in df.columns if col.startswith('insulin-')]\ndf[inc_col]=df[inc_col].fillna(method='ffill',axis=1)\ndf[inc_col]=df[inc_col].fillna(method='bfill',axis=1)\ndt[inc_col]=dt[inc_col].fillna(method='ffill',axis=1)\ndt[inc_col]=dt[inc_col].fillna(method='bfill',axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:03.046722Z","iopub.execute_input":"2024-11-19T14:19:03.047031Z","iopub.status.idle":"2024-11-19T14:19:03.407720Z","shell.execute_reply.started":"2024-11-19T14:19:03.047003Z","shell.execute_reply":"2024-11-19T14:19:03.406710Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"steps_col=[col for col in df.columns if col.startswith('steps-')]\ndf[steps_col]=df[steps_col].fillna(method='ffill',axis=1)\ndf[steps_col]=df[steps_col].fillna(method='bfill',axis=1)\ndt[steps_col]=dt[steps_col].fillna(method='ffill',axis=1)\ndt[steps_col]=dt[steps_col].fillna(method='bfill',axis=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:03.408819Z","iopub.execute_input":"2024-11-19T14:19:03.409079Z","iopub.status.idle":"2024-11-19T14:19:03.800007Z","shell.execute_reply.started":"2024-11-19T14:19:03.409054Z","shell.execute_reply":"2024-11-19T14:19:03.799245Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"carbs_col=[col for col in df.columns if col.startswith('carbs-')]\ndf[carbs_col]=df[carbs_col].fillna(method='ffill',axis=1)\ndf[carbs_col]=df[carbs_col].fillna(method='bfill',axis=1)\ndt[carbs_col]=dt[carbs_col].fillna(method='ffill',axis=1)\ndt[carbs_col]=dt[carbs_col].fillna(method='bfill',axis=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:03.800831Z","iopub.execute_input":"2024-11-19T14:19:03.801054Z","iopub.status.idle":"2024-11-19T14:19:04.286420Z","shell.execute_reply.started":"2024-11-19T14:19:03.801031Z","shell.execute_reply":"2024-11-19T14:19:04.285599Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"activity_col=[col for col in df.columns if col.startswith('activity-')]\ndf[activity_col]=df[activity_col].fillna(method='ffill',axis=1)\ndf[activity_col]=df[activity_col].fillna(method='bfill',axis=1)\ndt[activity_col]=dt[activity_col].fillna(method='ffill',axis=1)\ndt[activity_col]=dt[activity_col].fillna(method='bfill',axis=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:04.287472Z","iopub.execute_input":"2024-11-19T14:19:04.287733Z","iopub.status.idle":"2024-11-19T14:19:14.772170Z","shell.execute_reply.started":"2024-11-19T14:19:04.287707Z","shell.execute_reply":"2024-11-19T14:19:14.771510Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# pd.set_option('display.max_rows', None)\nnan_percentage = df.isnull().mean() * 100\nprint(nan_percentage)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-11-19T14:19:14.773503Z","iopub.execute_input":"2024-11-19T14:19:14.774307Z","iopub.status.idle":"2024-11-19T14:19:16.154691Z","shell.execute_reply.started":"2024-11-19T14:19:14.774244Z","shell.execute_reply":"2024-11-19T14:19:16.153801Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\nIn this step, we assign unique numeric codes to each activity type, which provides \na streamlined way to represent activities in the dataset. We create a dictionary mapping each activity name to a unique integer. \nThis encoding method is valuable for data analysis and preparing categorical \ndata for machine learning models.\n
\n","metadata":{}},{"cell_type":"code","source":"data = {'activity': ['Indoor climbing','Run','Strength training','Swim','Bike','Dancing','Stairclimber','Spinning','Walking','HIIT','Outdoor Bike','Walk','Aerobic Workout','Tennis','Workout','Hike','Zumba','Sport','Yoga','Swimming','Weights','Running']}\ndata = pd.DataFrame(data)\nunique_act={act:i+1 for i,act in enumerate(data['activity'].dropna().unique())}\nunique_act","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:16.155899Z","iopub.execute_input":"2024-11-19T14:19:16.156533Z","iopub.status.idle":"2024-11-19T14:19:16.164713Z","shell.execute_reply.started":"2024-11-19T14:19:16.156484Z","shell.execute_reply":"2024-11-19T14:19:16.163760Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n Here, we assign a unique numeric code to each participant, making it easier to reference participants \n consistently across the dataset. This process involves generating a dictionary where each participant ID is mapped to a unique integer. \n Encoding participants in this way simplifies data manipulation and analysis.\n
\n\n ","metadata":{}},{"cell_type":"code","source":"data = {'participant': ['p01','p02','p03','p04','p05','p06','p07','p08','p09','p10','p11','p12','p13','p14','p15']}\ndata = pd.DataFrame(data)\nunique_pat={pat:i+1 for i,pat in enumerate(data['participant'].dropna().unique())}\nunique_pat","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:16.166388Z","iopub.execute_input":"2024-11-19T14:19:16.167121Z","iopub.status.idle":"2024-11-19T14:19:16.180216Z","shell.execute_reply.started":"2024-11-19T14:19:16.167080Z","shell.execute_reply":"2024-11-19T14:19:16.179430Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n In this step, we encode activity names and participant numbers with unique numeric codes. \n We first identify the columns related to activities by checking for prefixes. Then, we use \n the applymap function to transform each activity name into its corresponding \n unique code from the unique_act dictionary, replacing any empty values with -1.\n Similarly, we encode participant numbers using the unique_pat dictionary, ensuring that any empty entries \n are also set to -1. This encoding facilitates easier analysis and model training by standardizing \n the representation of categorical variables.\n
\n\n","metadata":{}},{"cell_type":"code","source":"act_col = [col for col in df.columns if col.startswith('activity')]\ndf[act_col] = df[act_col].applymap(lambda x: unique_act.get(x, -1) if x != '' else -1)\ndt[act_col] = dt[act_col].applymap(lambda x: unique_act.get(x, -1) if x != '' else -1)","metadata":{"execution":{"iopub.status.busy":"2024-11-19T14:19:16.181391Z","iopub.execute_input":"2024-11-19T14:19:16.181655Z"},"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df['p_num'] = df['p_num'].apply(lambda x: unique_pat.get(x, -1) if x != '' else -1)\ndt['p_num'] = dt['p_num'].apply(lambda x: unique_pat.get(x, -1) if x != '' else -1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_bg_features(df):\n bg_cols = [col for col in df.columns if 'bg-' in col]\n df['bg_current'] = df[bg_cols].iloc[:, 0] # Most recent bg reading\n df['bg_change'] = df[bg_cols].iloc[:, 0] - df[bg_cols].iloc[:, -1] # Change from earliest to latest reading\n df['bg_mean'] = df[bg_cols].mean(axis=1)\n df['bg_std'] = df[bg_cols].std(axis=1)\n df['bg_min'] = df[bg_cols].min(axis=1)\n df['bg_max'] = df[bg_cols].max(axis=1)\n return df\n\n# 2. Insulin Features\ndef extract_insulin_features(df):\n insulin_cols = [col for col in df.columns if 'insulin-' in col]\n df['insulin_recent'] = df[insulin_cols].iloc[:, 0] # Most recent insulin dose\n df['insulin_cumulative'] = df[insulin_cols].sum(axis=1) # Total insulin over the period\n df['insulin_frequency'] = (df[insulin_cols] > 0).sum(axis=1) # Count of non-zero doses\n return df\n\n# 3. Carbohydrate (carbs) Features\ndef extract_carbs_features(df):\n carbs_cols = [col for col in df.columns if 'carbs-' in col]\n df['carbs_recent'] = df[carbs_cols].iloc[:, 0] # Most recent carb intake\n df['carbs_total'] = df[carbs_cols].sum(axis=1) # Total carbs intake over the period\n df['carbs_frequency'] = (df[carbs_cols] > 0).sum(axis=1) # Count of non-zero carb intake\n return df\n\n# 4. Heart Rate (hr) Features\ndef extract_hr_features(df):\n hr_cols = [col for col in df.columns if 'hr-' in col]\n df['hr_current'] = df[hr_cols].iloc[:, 0] # Latest heart rate\n df['hr_mean'] = df[hr_cols].mean(axis=1)\n df['hr_std'] = df[hr_cols].std(axis=1)\n df['hr_min'] = df[hr_cols].min(axis=1)\n df['hr_max'] = df[hr_cols].max(axis=1)\n return df\n\n# 5. Steps and Calories Features\ndef extract_steps_cals_features(df):\n steps_cols = [col for col in df.columns if 'steps-' in col]\n cals_cols = [col for col in df.columns if 'cals-' in col]\n df['steps_total'] = df[steps_cols].sum(axis=1)\n df['cals_total'] = df[cals_cols].sum(axis=1)\n df['pace'] = df['steps_total'] / (df[cals_cols].count(axis=1) * 5) # Assuming each interval is 5 min\n df['calories_per_step'] = df['cals_total'] / (df['steps_total'] + 1e-5) # Avoid division by zero\n return df\n\n# 6. Activity Features\ndef extract_activity_features(df):\n activity_cols = [col for col in df.columns if 'activity-' in col]\n df['activity_recent'] = df[activity_cols].iloc[:, 0] # Most recent activity\n df['activity_changes'] = df[activity_cols].nunique(axis=1) - 1 # Number of changes in activity\n return df\n\n\n\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Apply all feature extraction functions\ndf = extract_bg_features(df)\ndf = extract_insulin_features(df)\ndf = extract_carbs_features(df)\ndf = extract_hr_features(df)\ndf = extract_steps_cals_features(df)\ndf = extract_activity_features(df)\ndt = extract_bg_features(dt)\ndt = extract_insulin_features(dt)\ndt = extract_carbs_features(dt)\ndt = extract_hr_features(dt)\ndt = extract_steps_cals_features(dt)\ndt = extract_activity_features(dt)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_bg_features(df):\n bg_cols = [col for col in df.columns if 'bg-' in col]\n df['bg_median'] = df[bg_cols].median(axis=1)\n df['bg_range'] = df[bg_cols].max(axis=1) - df[bg_cols].min(axis=1)\n df['bg_variability'] = df[bg_cols].diff(axis=1).abs().mean(axis=1) # Mean absolute difference\n df['bg_above_threshold'] = (df[bg_cols] > 180).sum(axis=1) # Count of bg readings above threshold\n df['bg_below_threshold'] = (df[bg_cols] < 70).sum(axis=1) # Count of bg readings below threshold\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_insulin_features(df):\n insulin_cols = [col for col in df.columns if 'insulin-' in col]\n df['insulin_mean'] = df[insulin_cols].mean(axis=1)\n df['insulin_std'] = df[insulin_cols].std(axis=1)\n df['insulin_median'] = df[insulin_cols].median(axis=1)\n df['insulin_max'] = df[insulin_cols].max(axis=1)\n df['insulin_min'] = df[insulin_cols].min(axis=1)\n df['insulin_last_nonzero'] = df[insulin_cols].apply(lambda row: row[row != 0].iloc[-1] if any(row != 0) else 0, axis=1)\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_carbs_features(df):\n carbs_cols = [col for col in df.columns if 'carbs-' in col]\n df['carbs_median'] = df[carbs_cols].median(axis=1)\n df['carbs_variance'] = df[carbs_cols].var(axis=1)\n df['carbs_std'] = df[carbs_cols].std(axis=1)\n df['carbs_above_threshold'] = (df[carbs_cols] > 50).sum(axis=1) # Count of high-carb intakes\n df['carbs_max'] = df[carbs_cols].max(axis=1)\n df['carbs_last_nonzero'] = df[carbs_cols].apply(lambda row: row[row != 0].iloc[-1] if any(row != 0) else 0, axis=1)\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_hr_features(df):\n hr_cols = [col for col in df.columns if 'hr-' in col]\n df['hr_range'] = df[hr_cols].max(axis=1) - df[hr_cols].min(axis=1)\n df['hr_above_threshold'] = (df[hr_cols] > 100).sum(axis=1) # Count of high HR readings\n df['hr_below_threshold'] = (df[hr_cols] < 60).sum(axis=1) # Count of low HR readings\n df['hr_median'] = df[hr_cols].median(axis=1)\n df['hr_rolling_mean'] = df[hr_cols].rolling(window=3, axis=1).mean().iloc[:, -1] # Rolling average for last 3 intervals\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_steps_cals_features(df):\n steps_cols = [col for col in df.columns if 'steps-' in col]\n cals_cols = [col for col in df.columns if 'cals-' in col]\n df['steps_variance'] = df[steps_cols].var(axis=1)\n df['steps_mean'] = df[steps_cols].mean(axis=1)\n df['cals_mean'] = df[cals_cols].mean(axis=1)\n df['steps_to_cals_ratio'] = df['steps_total'] / (df['cals_total'] + 1e-5) # Steps per calorie burned\n df['cals_per_min'] = df[cals_cols].sum(axis=1) / (len(cals_cols) * 5) # Assuming each interval is 5 min\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def extract_advanced_activity_features(df):\n activity_cols = [col for col in df.columns if 'activity-' in col]\n df['activity_unique_count'] = df[activity_cols].nunique(axis=1) # Count of unique activities\n df['activity_mode'] = df[activity_cols].mode(axis=1)[0] # Most frequent activity\n df['activity_duration'] = df[activity_cols].apply(lambda row: max((row != row.shift()).cumsum()), axis=1) # Longest continuous activity\n df['activity_high_intensity'] = (df[activity_cols] == 'high').sum(axis=1) # Count of high-intensity activities\n return df\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df = extract_advanced_bg_features(df)\ndf = extract_advanced_insulin_features(df)\ndf = extract_advanced_carbs_features(df)\ndf = extract_advanced_hr_features(df)\ndf = extract_advanced_steps_cals_features(df)\ndf = extract_advanced_activity_features(df)\n\ndt = extract_advanced_bg_features(dt)\ndt = extract_advanced_insulin_features(dt)\ndt = extract_advanced_carbs_features(dt)\ndt = extract_advanced_hr_features(dt)\ndt = extract_advanced_steps_cals_features(dt)\ndt = extract_advanced_activity_features(dt)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"missing_percentage = df.isnull().mean()\nmissing_percentage*100","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df = df.replace([np.inf, -np.inf], np.nan)\ndt = dt.replace([np.inf, -np.inf], np.nan)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n This function processes time data to extract useful features for analysis. It begins by converting the time \n column to a datetime format, enabling easy extraction of hours and minutes. Next, we create cyclical features:\n
\n These transformations help maintain the cyclical nature of time, which is essential for modeling. Finally, the original time \n column is dropped to keep the DataFrame clean. The function is then applied to both the df and dt DataFrames.\n
\n\n","metadata":{}},{"cell_type":"code","source":"def formate_time(df):\n time_col = pd.to_datetime(df['time'], format='%H:%M:%S')\n hours = time_col.dt.hour\n minutes = time_col.dt.minute\n df['hour_sin'] = np.sin(2 * np.pi * hours / 24)\n df['hour_cos'] = np.cos(2 * np.pi * hours / 24)\n df['minute_sin'] = np.sin(2 * np.pi * minutes / 60)\n df['minute_cos'] = np.cos(2 * np.pi * minutes / 60)\n df.drop(columns=['time'], inplace=True)\n return df\n\n# Apply the function\ndf = formate_time(df)\ndt = formate_time(dt)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"missing_percentage = df.isnull().mean()\nmissing_percentage*100","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"id = dt['id']","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df=df.drop('id',axis=1)\ndt=dt.drop('id',axis=1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# print(dt.dtypes)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X=df.drop('bg+1:00',axis=1)\ny=df['bg+1:00']\nX.shape,y.shape","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# X_train_split, X_valid_split, y_train_split, y_valid_split = train_test_split(X, y, test_size=0.11, random_state=28)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n In this section, we are using XGBoost, a powerful machine learning algorithm, along with Optuna, a framework for hyperparameter optimization. \n The goal is to improve the model's performance by systematically tuning its hyperparameters. \n Optuna helps automate the search for optimal parameter values, allowing us to efficiently test different configurations to find the best settings for our XGBoost model.\n By evaluating the model's performance through metrics such as RMSE, we aim to enhance predictive accuracy and achieve better results in our regression task.\n
\n\n","metadata":{}},{"cell_type":"code","source":"import xgboost as xgb\nfrom sklearn.metrics import mean_squared_error\n\ndef objective(trial):\n # Suggest hyperparameters for tuning\n params = {\n 'objective': 'reg:squarederror',\n 'eval_metric': 'rmse',\n 'booster': 'gbtree',\n 'n_estimators': trial.suggest_int('n_estimators', 500, 1600), \n 'eta': trial.suggest_loguniform('eta', 0.0005, 0.3), \n 'max_depth': trial.suggest_int('max_depth', 3, 15),\n 'tree_method': 'hist', \n 'device': 'cuda'\n }\n\n dtrain = xgb.DMatrix(X_train_split, label=y_train_split)\n dvalid = xgb.DMatrix(X_valid_split, label=y_valid_split)\n \n # Train the model\n model = xgb.train(params, dtrain, evals=[(dvalid, 'validation')], num_boost_round=1500, early_stopping_rounds=30, verbose_eval=False)\n \n # Predict on the validation set\n y_pred_valid = model.predict(dvalid)\n \n # Calculate RMSE on the validation set\n rmse = mean_squared_error(y_valid_split, y_pred_valid, squared=False)\n print(\"RMSE = \", rmse)\n return rmse\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# study = optuna.create_study(direction='minimize')\n# study.optimize(objective, n_trials=11)\n# best_params = study.best_params\n# print(f\"Best hyperparameters: {best_params}\")\n# best_params['objective'] = 'reg:squarederror'\n# best_params['eval_metric'] = 'rmse'\n# best_params['device'] = 'cuda'","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##\n In this section, we implement K-Fold cross-validation to train an XGBoost regression model and evaluate its performance. \n K-Fold cross-validation allows us to divide the dataset into several subsets (or folds) and iteratively train the model on different combinations of these folds to ensure robust performance. \n
\n\n","metadata":{}},{"cell_type":"code","source":"from xgboost import XGBRegressor\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import mean_squared_error\nimport numpy as np\nimport pandas as pd\n\n# Define the custom scoring function (Root Mean Squared Error)\ndef rmse(y_true, y_pred):\n return np.sqrt(mean_squared_error(y_true, y_pred))\n\n# Use KFold for cross-validation\nkf = KFold(n_splits=3, shuffle=True, random_state=42)\n\n# Best parameters from the Optuna trials\nbest_params = [\n {'n_estimators': 100, 'learning_rate': 0.025753933847930798, 'max_depth': 8, 'max_bin': 2048},\n {'n_estimators': 122, 'learning_rate': 0.010958676342527139, 'max_depth': 12, 'max_bin': 32},\n {'n_estimators': 118, 'learning_rate': 0.0023094901054996004, 'max_depth': 8, 'max_bin': 4096},\n {'n_estimators': 164, 'learning_rate': 0.023195470560234722, 'max_depth': 7, 'max_bin': 8192}\n]\n\n# List to store all predictions (from all parameter sets)\nall_predictions = []\n\n# Loop over all parameter sets\nfor params in best_params:\n test_predictions = []\n \n # Loop over each fold in KFold cross-validation\n for train_index, val_index in kf.split(X):\n X_train, X_val = X.iloc[train_index], X.iloc[val_index]\n y_train, y_val = y.iloc[train_index], y.iloc[val_index]\n \n # Train the model with the current parameters\n model = XGBRegressor(\n n_estimators=params['n_estimators'],\n learning_rate=params['learning_rate'],\n max_depth=params['max_depth'],\n max_bin=params['max_bin'],\n device='cuda',\n tree_method='hist',\n random_state=42\n )\n model.fit(X_train, y_train)\n \n # Make predictions on the test set (dt)\n y_test_pred = model.predict(dt)\n test_predictions.append(y_test_pred)\n \n # Take the mean of predictions from all folds for the current parameter set\n fold_mean_predictions = np.mean(np.array(test_predictions), axis=0)\n all_predictions.append(fold_mean_predictions)\n\n# Calculate the final mean of predictions across all parameter sets\nfinal_predictions = np.mean(np.array(all_predictions), axis=0)\n\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ##