{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":63460,"databundleVersionId":6938096,"sourceType":"competition"}],"dockerImageVersionId":30746,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\nWelcome to this notebook! If you've been puzzled by the vague description of the target variable in the Crypto Bull Prediction Challenge, you're not alone. \n\nThis notebook aims to explain on how the target is calculated and why understanding it is crucial for fair and effective participation in the competition.","metadata":{}},{"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\n\nfrom sklearn.metrics import roc_auc_score\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:35:41.246289Z","iopub.execute_input":"2024-08-05T23:35:41.247033Z","iopub.status.idle":"2024-08-05T23:35:41.253183Z","shell.execute_reply.started":"2024-08-05T23:35:41.246987Z","shell.execute_reply":"2024-08-05T23:35:41.251955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:23:14.261619Z","iopub.execute_input":"2024-08-05T23:23:14.262602Z","iopub.status.idle":"2024-08-05T23:23:14.270861Z","shell.execute_reply.started":"2024-08-05T23:23:14.262568Z","shell.execute_reply":"2024-08-05T23:23:14.269674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"folder = '/kaggle/input/crypto-bull-prediction-challenge/'\n\ntrain = pd.read_csv(folder+'train.csv')\ntest = pd.read_csv(folder+'test.csv')\nsample_submission = pd.read_csv(folder+'sample_submission.csv')","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:23:23.555889Z","iopub.execute_input":"2024-08-05T23:23:23.556262Z","iopub.status.idle":"2024-08-05T23:23:33.603791Z","shell.execute_reply.started":"2024-08-05T23:23:23.556232Z","shell.execute_reply":"2024-08-05T23:23:33.602751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# The Problem\nThe competition description provides a formula for the target calculation:\n\ndata[price].pct_change(horizon).shift(-horizon).fillna(0) > threshold\n\nHowever, it fails to specify two critical parameters:\n\n- The horizon value\n- The threshold value\n\nThis lack of clarity creates an uneven playing field, potentially giving an unfair advantage to participants who can decipher these parameters.","metadata":{}},{"cell_type":"markdown","source":"# Why This Matters\nUnderstanding the target variable is fundamental in any machine learning task. Here's why it's particularly important in this competition:\n\n- Accurate Problem Framing: Knowing what we're predicting allows us to frame the problem correctly and choose appropriate models and features.\n- Feature Engineering: With a clear understanding of the target, we can create more relevant and powerful features.\n- Model Selection: Different prediction horizons and thresholds might favor different types of models or architectures.\n- Evaluation Understanding: Knowing how the target is calculated helps us better interpret our model's performance and the competition's evaluation metric.\n- Fair Competition: Equal access to information about the target ensures a level playing field for all participants.","metadata":{}},{"cell_type":"markdown","source":"# Our Solution\nTo address this issue, I've developed a code that reverse-engineer the parameters used to create the target. Here's what it does:\n\n1. Iterates through possible combinations of horizon and threshold values.\n2. Calculates the target for each combination.\n3. Compares the calculated target with the provided target.\n4. Identifies the parameters that produce the closest match.","metadata":{}},{"cell_type":"code","source":"def calculate_target(data, price_column, horizon, threshold):\n    return (data[price_column].pct_change(horizon).shift(-horizon).fillna(0) > threshold).astype(int)\n\ndef find_params(data, price_column, target_column, max_horizon=30, threshold_steps=1000):\n    best_horizon = 0\n    best_threshold = 0\n    best_auc = 0\n\n    total_iterations = max_horizon * threshold_steps\n    progress_bar = tqdm(total=total_iterations, desc=\"Searching parameters\")\n\n    for horizon in range(1, max_horizon + 1):\n        for threshold in np.linspace(0, 0.1, threshold_steps):  # Adjust range as needed\n            calculated_target = calculate_target(data, price_column, horizon, threshold)\n            auc = roc_auc_score(data[target_column], calculated_target)\n\n            if auc > best_auc:\n                best_auc = auc\n                best_horizon = horizon\n                best_threshold = threshold\n                print(f\"\\nNew best parameters found: Horizon = {horizon}, Threshold = {threshold:.6f}, AUC = {auc:.6f}\")\n\n            progress_bar.update(1)\n\n    progress_bar.close()\n    return best_horizon, best_threshold, best_auc\n\n# Load your data\nprint(\"Loading data...\")\ndata = train[train['symbol'] == '1000LUNC-USDT-PERP'].copy()\nprint(f\"Data loaded. Shape: {data.shape}\")\n\n# Specify the column names\nprice_column = 'close'  # Replace with your actual price column name\ntarget_column = 'target'  # Replace with your actual target column name\n\nprint(f\"\\nStarting parameter search...\")\nprint(f\"Price column: {price_column}\")\nprint(f\"Target column: {target_column}\")\n\n# Find the best parameters\nbest_horizon, best_threshold, best_auc = find_params(data, price_column, target_column)\n\nprint(\"\\nSearch completed.\")\nprint(f\"Best horizon: {best_horizon}\")\nprint(f\"Best threshold: {best_threshold:.6f}\")\nprint(f\"Best AUC: {best_auc:.6f}\")\n\n# Verify the results\nprint(\"\\nVerifying results...\")\ncalculated_target = calculate_target(data, price_column, best_horizon, best_threshold)\nfinal_auc = roc_auc_score(data[target_column], calculated_target)\nprint(f\"Verified AUC: {final_auc:.6f}\")\n\nprint(\"\\nDone!\")","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:24:03.403338Z","iopub.execute_input":"2024-08-05T23:24:03.404164Z","iopub.status.idle":"2024-08-05T23:25:51.467303Z","shell.execute_reply.started":"2024-08-05T23:24:03.404126Z","shell.execute_reply":"2024-08-05T23:25:51.466313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train['target_roc'] = train['close'].pct_change(3).shift(-3).fillna(0) > 0.009 # I rounded the threshold value here and it yielded a better result\ntrain['target_roc'] = train['target_roc'].astype(int)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:25:55.382466Z","iopub.execute_input":"2024-08-05T23:25:55.382816Z","iopub.status.idle":"2024-08-05T23:25:55.502924Z","shell.execute_reply.started":"2024-08-05T23:25:55.382787Z","shell.execute_reply":"2024-08-05T23:25:55.501722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"auc = roc_auc_score(train['target'], train['target_roc'])\nmismatches = train[train['target_roc'] != train['target']]\nprint(f\"Number of mismatches: {len(mismatches)}\")\nprint(f\"Percentage of mismatches: {len(mismatches) / len(train) * 100:.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2024-08-05T23:25:55.511978Z","iopub.execute_input":"2024-08-05T23:25:55.512269Z","iopub.status.idle":"2024-08-05T23:25:56.339110Z","shell.execute_reply.started":"2024-08-05T23:25:55.512245Z","shell.execute_reply":"2024-08-05T23:25:56.338151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Parameters Discovered\nAfter running our code, we found that the target variable is likely calculated using the following parameters:\n\n**Horizon: 3**\n\n**Threshold: 0.009 (0.9%)**\n\n---\n\nLet's break down what these parameters mean and their implications for the competition.\n\n\n1. **Horizon = 3**\n\nThe horizon of 3 indicates that the prediction is based on a 3-day future window. This means:\n\n*Short-term Focus*: We're dealing with very short-term price movements in the crypto market. \nThis aligns with the volatile nature of cryptocurrencies, where significant price changes can occur in just a few days.\n\n*Data Lag*: When building our models, we need to ensure that we're not using any data from the future 3-day window to make our predictions. This is crucial for preventing look-ahead bias.\n\n*Feature Engineering*: We might want to focus on features that capture recent market trends and sentiment, as these are likely to have the most impact on a 3-day horizon.\n\n*Model Selection*: This short horizon might favor models that are good at capturing recent patterns and quick market reactions, such as certain types of time series models or models that can effectively utilize high-frequency data.\n\n2. **Threshold = 0.009 (0.9%)**\n\nThe threshold of 0.9% provides more information about what the competition considers a significant price movement:\n\n*Definition of 'Bull'*: In this competition, a 'bull' movement is defined as a price increase of more than 0.9% over three days. This is a relatively small movement, reflecting the competition's focus on capturing even minor uptrends.\n\n*Frequency of Positive Cases*: With such a low threshold, we might expect to see a relatively balanced distribution of positive and negative cases. \nThis could be good for model training, as it might help avoid class imbalance issues.\n\n*Risk and Reward*: For a trader, a 0.9% price increase over three days might not seem significant, especially given typical crypto volatility. \nThis suggests that the competition might be more focused on frequency of correct predictions rather than magnitude of price movements.\n\n*Noise Sensitivity*: Such a low threshold means our models need to be sensitive enough to capture small price movements, but robust enough **not** to be thrown off by market noise.\n\n---\n\n# Conclusion\n\nI hope now all of us have a better understanding of what are we actually trying to predict and why target caclualtion is more complex than just a shift of a next candle with a 0 threshold. \n\nBecause of the market nature, it is beneficial to slightly increase the threshold for candles that might move up, which could provide a better risk-management, and mitigate the market noise when we will deploy our models live","metadata":{}}]}