{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":63460,"databundleVersionId":6938096,"sourceType":"competition"}],"dockerImageVersionId":30558,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Baseline \nBaseline создается больше как шаблон, где можно посмотреть, как происходит обращение с входящими данными и что нужно получить на выходе. При этом ML начинка может быть достаточно простой. Это помогает быстрее приступить к самому ML, а не тратить ценное время на инженерные задачи. Также baseline является хорошей опорной точкой по метрике. Если наше решение хуже baseline - мы явно делаем что-то не так и стоит попробовать другой путь)\n\n(it is created more like a template where you can see how incoming data is processed and what you need to get as an output. At the same time, the ML filling can be quite simple. This helps you get started on the ML itself faster, rather than wasting valuable time on engineering tasks. Also baseline is a good reference point for the metric. If our solution is worse than the baseline, we are clearly doing something wrong and it’s worth trying a different path)\n\n### In baseline we will do the following:\nBuild a \"naive\"/baseline model (to which we will compare other models)\n\nSteps:\n- Install libs\n- Import\n- Load the data\n- EDA - Exploratory Data Analysis\n- Data preprocessing\n- Generate new features\n- Make the first model based on gradient bousting with CatBoost (CatBoostClassifier)\n- Generate submission file and send it to Kaggle","metadata":{}},{"cell_type":"markdown","source":"# Install libs","metadata":{}},{"cell_type":"code","source":"!pip install -q pandas_ta","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:47:56.770935Z","iopub.execute_input":"2024-04-16T02:47:56.771265Z","iopub.status.idle":"2024-04-16T02:48:16.123021Z","shell.execute_reply.started":"2024-04-16T02:47:56.771235Z","shell.execute_reply":"2024-04-16T02:48:16.120895Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# всегда фиксируйте RANDOM_SEED, чтобы ваши эксперименты были воспроизводимы!\n# (Always fix RANDOM_SEED so your experiments are reproducible!)\nRANDOM_SEED = 42","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:48:21.086427Z","iopub.execute_input":"2024-04-16T02:48:21.086970Z","iopub.status.idle":"2024-04-16T02:48:21.093282Z","shell.execute_reply.started":"2024-04-16T02:48:21.086916Z","shell.execute_reply":"2024-04-16T02:48:21.092278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# зафиксируем версию пакетов, чтобы эксперименты были воспроизводимы:\n# (fix the package version so that the experiments are reproducible:)\n!pip freeze > requirements.txt","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:48:27.596757Z","iopub.execute_input":"2024-04-16T02:48:27.597331Z","iopub.status.idle":"2024-04-16T02:48:31.400655Z","shell.execute_reply.started":"2024-04-16T02:48:27.597293Z","shell.execute_reply":"2024-04-16T02:48:31.398726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\n# CatBoost is provided by Yandex: https://catboost.ai/en/docs/\nfrom catboost import CatBoostClassifier\nimport pandas_ta as ta\nfrom tqdm import tqdm\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:48:36.911750Z","iopub.execute_input":"2024-04-16T02:48:36.912400Z","iopub.status.idle":"2024-04-16T02:48:39.336898Z","shell.execute_reply.started":"2024-04-16T02:48:36.912347Z","shell.execute_reply":"2024-04-16T02:48:39.335678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data loading","metadata":{}},{"cell_type":"code","source":"# Just lists the files in the /kaggle/input folder\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:48:50.282328Z","iopub.execute_input":"2024-04-16T02:48:50.283021Z","iopub.status.idle":"2024-04-16T02:48:50.295381Z","shell.execute_reply.started":"2024-04-16T02:48:50.282982Z","shell.execute_reply":"2024-04-16T02:48:50.293751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load all of the data from the CSV files into training and testing datasets. Also loads a sample submission.\nfolder = '/kaggle/input/crypto-bull-prediction-challenge/'\n\ntrain = pd.read_csv(folder+'train.csv')\ntest = pd.read_csv(folder+'test.csv')\nsample_submission = pd.read_csv(folder+'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:48:58.191947Z","iopub.execute_input":"2024-04-16T02:48:58.192399Z","iopub.status.idle":"2024-04-16T02:49:11.474984Z","shell.execute_reply.started":"2024-04-16T02:48:58.192366Z","shell.execute_reply":"2024-04-16T02:49:11.473919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA\nExploratory Data Analysis","metadata":{}},{"cell_type":"code","source":"# A look at the training data, which is labelled data as shown by the \"target\" column.\n'''\nThe \"target\" field is calculated as follows:\n\n    data[price].pct_change(horizon).shift(-horizon).fillna(0) > threshold\n    \nWhere \"price,\" \"horizon,\" and \"threshold\" are fixes values for this competition.\n\nWhen target is 1, it signifies that there was an increase of the price over the horizon above the threshold\n'''\ntrain","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:34:14.849081Z","iopub.execute_input":"2024-04-16T03:34:14.849629Z","iopub.status.idle":"2024-04-16T03:34:14.881016Z","shell.execute_reply.started":"2024-04-16T03:34:14.849563Z","shell.execute_reply":"2024-04-16T03:34:14.879686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Unlabelled data, instead of the \"target\" field, there is now an \"id\" field which is an incremental ID\n# starting from 1.\ntest","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:20.078126Z","iopub.execute_input":"2024-04-16T02:49:20.078615Z","iopub.status.idle":"2024-04-16T02:49:20.107149Z","shell.execute_reply.started":"2024-04-16T02:49:20.078561Z","shell.execute_reply":"2024-04-16T02:49:20.106119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### date","metadata":{}},{"cell_type":"code","source":"# Group by the different markets, get the minimums for training dataset\ntrain.groupby('symbol')['date'].min()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:24.362976Z","iopub.execute_input":"2024-04-16T02:49:24.363584Z","iopub.status.idle":"2024-04-16T02:49:25.482950Z","shell.execute_reply.started":"2024-04-16T02:49:24.363543Z","shell.execute_reply":"2024-04-16T02:49:25.481651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Group by the different markets, get the maximums for training dataset\ntrain.groupby('symbol')['date'].max()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:30.371958Z","iopub.execute_input":"2024-04-16T02:49:30.372344Z","iopub.status.idle":"2024-04-16T02:49:31.467111Z","shell.execute_reply.started":"2024-04-16T02:49:30.372315Z","shell.execute_reply":"2024-04-16T02:49:31.465962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Group by the different markets, get the total rows for training dataset\ntrain.groupby('symbol')['close'].count().sort_values()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:34.951764Z","iopub.execute_input":"2024-04-16T02:49:34.952337Z","iopub.status.idle":"2024-04-16T02:49:35.352659Z","shell.execute_reply.started":"2024-04-16T02:49:34.952289Z","shell.execute_reply":"2024-04-16T02:49:35.351498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Group by the different markets, get the minimums for testing dataset\ntest.groupby('symbol')['date'].min()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:38.762699Z","iopub.execute_input":"2024-04-16T02:49:38.763179Z","iopub.status.idle":"2024-04-16T02:49:38.791624Z","shell.execute_reply.started":"2024-04-16T02:49:38.763146Z","shell.execute_reply":"2024-04-16T02:49:38.790277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Group by the different markets, get the maximums for testing dataset\ntest.groupby('symbol')['date'].max()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T02:49:46.499181Z","iopub.execute_input":"2024-04-16T02:49:46.499673Z","iopub.status.idle":"2024-04-16T02:49:46.529179Z","shell.execute_reply.started":"2024-04-16T02:49:46.499627Z","shell.execute_reply":"2024-04-16T02:49:46.527799Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Group by the different markets, get the total rows for testing dataset\ntest.groupby('symbol')['close'].count().sort_values()","metadata":{"execution":{"iopub.status.busy":"2024-04-10T16:46:44.404372Z","iopub.execute_input":"2024-04-10T16:46:44.404849Z","iopub.status.idle":"2024-04-10T16:46:44.421321Z","shell.execute_reply.started":"2024-04-10T16:46:44.404814Z","shell.execute_reply":"2024-04-10T16:46:44.419794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## symbol","metadata":{}},{"cell_type":"code","source":"# How many markets are being predicted from the training dataset\nlen(train['symbol'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:08:41.952297Z","iopub.execute_input":"2024-04-16T03:08:41.952953Z","iopub.status.idle":"2024-04-16T03:08:42.268736Z","shell.execute_reply.started":"2024-04-16T03:08:41.952905Z","shell.execute_reply":"2024-04-16T03:08:42.267199Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# How many markets are being predicted from the testing dataset\nlen(test['symbol'].unique())","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:08:45.749911Z","iopub.execute_input":"2024-04-16T03:08:45.751144Z","iopub.status.idle":"2024-04-16T03:08:45.762833Z","shell.execute_reply.started":"2024-04-16T03:08:45.751068Z","shell.execute_reply":"2024-04-16T03:08:45.761300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Target","metadata":{}},{"cell_type":"code","source":"'''\nvalue_counts() is a function in the pandas library that returns the frequency of each unique\nvalue in a categorical data column. This function is useful when you want to get a quick\nunderstanding of the distribution of a categorical variable, such as the most common categories\nand their frequency.\n'''\n# 0: price didn't meet the price increase threshold\n# 1: price increased over the threshold\ntrain.target.value_counts()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:25:26.563259Z","iopub.execute_input":"2024-04-16T03:25:26.563776Z","iopub.status.idle":"2024-04-16T03:25:26.603687Z","shell.execute_reply.started":"2024-04-16T03:25:26.563738Z","shell.execute_reply":"2024-04-16T03:25:26.602185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Outputting the sample submission provided by the baseline Kaggle project\nsample_submission","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:25:30.060526Z","iopub.execute_input":"2024-04-16T03:25:30.061016Z","iopub.status.idle":"2024-04-16T03:25:30.075570Z","shell.execute_reply.started":"2024-04-16T03:25:30.060981Z","shell.execute_reply":"2024-04-16T03:25:30.074288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show what datatypes for each field in the training dataset\ntrain.info()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:38:07.351631Z","iopub.execute_input":"2024-04-16T03:38:07.352954Z","iopub.status.idle":"2024-04-16T03:38:07.374563Z","shell.execute_reply.started":"2024-04-16T03:38:07.352909Z","shell.execute_reply":"2024-04-16T03:38:07.372920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show what datatypes for each field in the testing dataset\ntest.info()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:38:26.672811Z","iopub.execute_input":"2024-04-16T03:38:26.673198Z","iopub.status.idle":"2024-04-16T03:38:26.698026Z","shell.execute_reply.started":"2024-04-16T03:38:26.673170Z","shell.execute_reply":"2024-04-16T03:38:26.696496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data preprocessing","metadata":{}},{"cell_type":"code","source":"# IMPORTANT! for the correct processing of signs, we combine the train and the test into one dataset\ntrain['sample'] = 1 # Note: from training dataset\ntest['sample'] = 0 # Note: from the testing dataset\n\ntrain['id'] = 0\ntest['target'] = 0 # if \"target\" wasn't set, display anyway, because it's just polar zeros\n\n# Merge the two datasets, reindex\ndata = pd.concat([train, test]).reset_index(drop=True) # concat","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:41:11.846702Z","iopub.execute_input":"2024-04-16T03:41:11.847160Z","iopub.status.idle":"2024-04-16T03:41:12.797631Z","shell.execute_reply.started":"2024-04-16T03:41:11.847129Z","shell.execute_reply":"2024-04-16T03:41:12.796485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data","metadata":{"execution":{"iopub.status.busy":"2024-04-16T03:43:59.272951Z","iopub.execute_input":"2024-04-16T03:43:59.273445Z","iopub.status.idle":"2024-04-16T03:43:59.306031Z","shell.execute_reply.started":"2024-04-16T03:43:59.273413Z","shell.execute_reply":"2024-04-16T03:43:59.304468Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Generate new features\n> **Important Note on Time Series Data:**\n\n> Time series data has a natural temporal ordering, which makes it different from other types of data. When generating features or using libraries, it's paramount to ensure you don't inadvertently introduce future information into the past, a phenomenon known as \"data leakage\". Data leakage can result in overly optimistic performance metrics during model evaluation and can be a critical mistake in real-world scenarios.\n>\n> Always be meticulous about the indicators you incorporate and the libraries you leverage. Data leakage is a grave oversight in time series analysis and, if detected, will lead to disqualification from the competition.\n>\n> For a deeper understanding of data leakage in time series, refer to these articles:\n> - [How (not) to use Machine Learning for time series forecasting: Avoiding the pitfalls](https://towardsdatascience.com/how-not-to-use-machine-learning-for-time-series-forecasting-avoiding-the-pitfalls-19f9d7adf424)\n> - [Understanding Data Leakage in Machine Learning](https://machinelearningmastery.com/data-leakage-machine-learning/)\n","metadata":{}},{"cell_type":"code","source":"# Periods are in hours.\n# Creates a set of indicators for the model to analyze.\ndef gen_indicators(data, indicator_periods = [2, 3, 4, 6, 9, 12, 24, 48, 72, 96, 120, 168]):\n        stock_data_tech = data.copy() # Clone the data into a new data object\n        stock_data_tech = stock_data_tech.sort_values(by='date') # Ensure all values are chronological\n\n        fe_df = pd.DataFrame() # Create a new Pandas dataframe\n\n        # tqdm() is a progress bar function\n        for symbol in tqdm(stock_data_tech.symbol.unique()): # For each crypto market\n            data = stock_data_tech[stock_data_tech.symbol == symbol].copy() # Copy all rows with same market symbol\n\n            data.reset_index(inplace=True, drop=True) # Remove the index\n            data.set_index('date', drop=False, inplace=True) # Reindex using the date field\n\n            # Calculate the percentage of change for each period\n            ################## Base pct ################################################################\n            for period in ([1,] + indicator_periods):\n                data[f'pct_close_{period}'] = data['close'].pct_change(period)\n            \n            ################### indicators ############################################################\n            \n            # ta = Pandas TA (Technical Analysis), a library with over 150 indicators.\n            # period-independent indicators\n            data.ta.pvol(append=True) # Price-Volume\n            data.ta.bop(append=True) # Balance of Power\n\n            for period in indicator_periods:\n\n                if period > 2:\n                    data.ta.willr(length=period, append=True) # William's Percent R \n                    data.ta.stoch(length=period, append=True) # Stochastic Oscillator\n                    data.ta.rsi(length=period, append=True) # Relative Strength Index\n                    data.ta.roc(length=period, append=True) # Rate of Change\n                    data.ta.mfi(length=period, append=True) # Multiple Factor Analysis\n                    # data.ta.massi(length=period, append=True)  # EMA data leak !!!\n\n            data.fillna(method=\"ffill\", inplace=True) # Forward fill the last valid data\n            data.dropna(inplace=True) # Remove any remaining rows with NULL values\n            #data.fillna(method=\"bfill\", inplace=True)\n            #data.fillna(0, inplace=True)\n            data.reset_index(inplace=True, drop=True) # Reindex\n        \n            fe_df = pd.concat([fe_df, data], axis=0) # Add the resulting data to the output object\n        fe_df.reset_index(inplace=True, drop=True) # One more reindex, just in case\n        return fe_df","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:08:55.009337Z","iopub.execute_input":"2024-04-16T04:08:55.010098Z","iopub.status.idle":"2024-04-16T04:08:55.029943Z","shell.execute_reply.started":"2024-04-16T04:08:55.010058Z","shell.execute_reply":"2024-04-16T04:08:55.028718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Generate the technical indicators for all market pairs.\nfe_data = gen_indicators(data)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:09:03.842169Z","iopub.execute_input":"2024-04-16T04:09:03.842689Z","iopub.status.idle":"2024-04-16T04:13:34.915968Z","shell.execute_reply.started":"2024-04-16T04:09:03.842652Z","shell.execute_reply":"2024-04-16T04:13:34.914459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Split","metadata":{}},{"cell_type":"code","source":"# Restore the training and testing datasets to their own objects, drop the temporary \"sample\" field\nfe_train = fe_data.query('sample == 1').drop(['sample',], axis=1)\nfe_test = fe_data.query('sample == 0').drop(['sample',], axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:20:17.067141Z","iopub.execute_input":"2024-04-16T04:20:17.067640Z","iopub.status.idle":"2024-04-16T04:20:20.741830Z","shell.execute_reply.started":"2024-04-16T04:20:17.067607Z","shell.execute_reply":"2024-04-16T04:20:20.740752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Resort the testing dataset by the id field\nfe_test = fe_test.sort_values(by=['id',])\n\nprint(fe_test[['date', 'symbol', 'trades', 'id', 'target',]])","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:24:01.716158Z","iopub.execute_input":"2024-04-16T04:24:01.716657Z","iopub.status.idle":"2024-04-16T04:24:01.757557Z","shell.execute_reply.started":"2024-04-16T04:24:01.716624Z","shell.execute_reply":"2024-04-16T04:24:01.756354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set the X matrix for the training dataset to be only the technical analysis indicators\nX_train = fe_train.drop(['open', 'high', 'low', 'close', 'volume', 'trades', 'symbol', 'date', 'id', 'target', 'close_unixtime'], axis=1)\n# Set the y vector for the training dataset to be the target column\ny_train = fe_train.target.values\n\n# Set the X matrix for the testing dataset to be only the technical analysis indicators\nX_test = fe_test.drop(['open', 'high', 'low', 'close', 'volume', 'trades', 'symbol', 'date', 'id', 'target', 'close_unixtime'], axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:32:40.034262Z","iopub.execute_input":"2024-04-16T04:32:40.035209Z","iopub.status.idle":"2024-04-16T04:32:40.921947Z","shell.execute_reply.started":"2024-04-16T04:32:40.035171Z","shell.execute_reply":"2024-04-16T04:32:40.920720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View the remaining data for the testing dataset\nX_test","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:33:12.501414Z","iopub.execute_input":"2024-04-16T04:33:12.501869Z","iopub.status.idle":"2024-04-16T04:33:12.550090Z","shell.execute_reply.started":"2024-04-16T04:33:12.501837Z","shell.execute_reply":"2024-04-16T04:33:12.549065Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{}},{"cell_type":"code","source":"# Generate the model using Yandex's CatBoost library, using their Classifier\n# Output the status while working every 100 passes\n# Original learning rate was .01\nmodel = CatBoostClassifier(verbose=100, random_seed=RANDOM_SEED, learning_rate=.005)\nmodel.fit(X_train, y_train)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T04:35:58.030015Z","iopub.execute_input":"2024-04-16T04:35:58.030519Z","iopub.status.idle":"2024-04-16T04:49:22.544584Z","shell.execute_reply.started":"2024-04-16T04:35:58.030488Z","shell.execute_reply":"2024-04-16T04:49:22.543543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save_model('catboost_baseline.model')","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:00:13.271418Z","iopub.execute_input":"2024-04-16T05:00:13.271910Z","iopub.status.idle":"2024-04-16T05:00:13.298291Z","shell.execute_reply.started":"2024-04-16T05:00:13.271878Z","shell.execute_reply":"2024-04-16T05:00:13.296694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Feature Importance","metadata":{}},{"cell_type":"code","source":"# Get the most important features\nfeature_importances = model.get_feature_importance()\n\n# Создаем DataFrame для удобства / We create a DataFrame for convenience\nfeatures_df = pd.DataFrame({\n    'Feature': X_train.columns,\n    'Importance': feature_importances\n})\n\n# Сортируем по убыванию важности / Sort by descending importance\nsorted_features_df = features_df.sort_values(by='Importance', ascending=False).head(25)\n\n# Рисуем график / Drawing a graph\nplt.figure(figsize=(12, 10))\nsns.barplot(data=sorted_features_df, x='Importance', y='Feature', palette='viridis')\nplt.title('Top 25 Feature Importances')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:04:29.577539Z","iopub.execute_input":"2024-04-16T05:04:29.579044Z","iopub.status.idle":"2024-04-16T05:04:30.213916Z","shell.execute_reply.started":"2024-04-16T05:04:29.578985Z","shell.execute_reply":"2024-04-16T05:04:30.213021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## submission","metadata":{}},{"cell_type":"code","source":"# Predict the probabilities for each class\npredict_proba = model.predict_proba(X_test)[:, 1]","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:04:45.241635Z","iopub.execute_input":"2024-04-16T05:04:45.242097Z","iopub.status.idle":"2024-04-16T05:04:45.316143Z","shell.execute_reply.started":"2024-04-16T05:04:45.242065Z","shell.execute_reply":"2024-04-16T05:04:45.314252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(predict_proba)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:04:54.137051Z","iopub.execute_input":"2024-04-16T05:04:54.137525Z","iopub.status.idle":"2024-04-16T05:04:54.146158Z","shell.execute_reply.started":"2024-04-16T05:04:54.137491Z","shell.execute_reply":"2024-04-16T05:04:54.144584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(sample_submission)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:04:57.584262Z","iopub.execute_input":"2024-04-16T05:04:57.584658Z","iopub.status.idle":"2024-04-16T05:04:57.592111Z","shell.execute_reply.started":"2024-04-16T05:04:57.584628Z","shell.execute_reply":"2024-04-16T05:04:57.590675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Move the predictions to sample submission object\nsample_submission['target'] = predict_proba","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:07:41.084336Z","iopub.execute_input":"2024-04-16T05:07:41.084874Z","iopub.status.idle":"2024-04-16T05:07:41.093533Z","shell.execute_reply.started":"2024-04-16T05:07:41.084838Z","shell.execute_reply":"2024-04-16T05:07:41.091674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Convert the submission to CSV and save\nsample_submission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:07:46.728041Z","iopub.execute_input":"2024-04-16T05:07:46.728831Z","iopub.status.idle":"2024-04-16T05:07:46.920749Z","shell.execute_reply.started":"2024-04-16T05:07:46.728795Z","shell.execute_reply":"2024-04-16T05:07:46.919454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show a histogram\nsample_submission['target'].hist(bins=250, edgecolor='black')","metadata":{"execution":{"iopub.status.busy":"2024-04-16T05:08:48.241013Z","iopub.execute_input":"2024-04-16T05:08:48.241477Z","iopub.status.idle":"2024-04-16T05:08:49.194375Z","shell.execute_reply.started":"2024-04-16T05:08:48.241445Z","shell.execute_reply":"2024-04-16T05:08:49.192702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> # What's next?\nЧто можно сделать, чтоб улучшить результат:\n\n- Добавить крассвалидацию (cross-validation)\n- Очистить данные\n- Сгенерировать новые признаки\n- Сделать отбор признаков\n- Перебрать различные модели\n- Посмотреть где модель ошибается и как ей можно помочь\n- Подобрать гиперпараметры\n\nВ общем, процесс творческий и весьма увлекательный! Удачи в соревновании!","metadata":{}},{"cell_type":"markdown","source":"What can be done to improve the result:\n* Add cross-validation\n* Clear data\n* Generate new features\n* Make a feature selection\n* Try different models\n* See where the model is wrong and how you can help it\n* Select hyperparameters\n\nIn general, the process is creative and very exciting! Good luck in the competition!\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}