![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F27467869%2Fd76a3f9f7fc641b83e285022f4b6418f%2FScreenshot%202025-12-24%20141741.png?generation=1766566077451197&alt=media\")
First, thanks to the competition organizers and congratulations to all participants.
\nThis solution focuses on understanding the distributional structure of the target and uncovering non-linear signals that are not captured well by standard correlation analysis or tree-based models.
Rather than relying on Gradient Boosting Trees, this approach uses a Support Vector Regressor (SVR) with a Polynomial kernel, which proved especially effective for this dataset.
\nThe key ideas behind this solution were:
\nThe first step was analyzing the distribution of the target variable, HOMELESS_RATE.
\nComparing the mean and median revealed a noticeable skew, indicating that extreme values were influencing the distribution.
This suggested that standard linear regression models would struggle, particularly in the tails.
\n
Decision:
\nA logarithmic transformation was applied to the target to stabilize variance and reduce skew, improving model behavior during training.
Instead of relying solely on Pearson correlation, which captures only linear relationships, the analysis included Mutual Information (MI) scores.
\nThis revealed several features with:
\nExamples included demographic percentages related to veterans, race, and age groups.
\nInsight:
\nImportant predictive signals existed in non-linear interactions that linear models would miss.\nThis directly motivated the choice of a non-linear kernel-based method.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F27467869%2Fab87ac0af88481ab7c466411bcfd1edc%2FScreenshot%202025-12-24%20140515.png?generation=1766565332411912&alt=media\")
Because Support Vector Machines are distance-based models, preprocessing played a crucial role.
\nThe pipeline was designed to be:
\nMissing values were handled conservatively to avoid introducing bias, and all features were scaled into a consistent range to ensure balanced distance computations across dimensions.
\nA Support Vector Regressor with a Polynomial kernel was selected for several reasons:
\nUnlike tree-based models, which partition feature space hierarchically, the polynomial kernel projects features into a higher-dimensional space where complex demographic interactions become linearly separable.
\nHomelessness rates are driven not by individual features in isolation, but by intersections of demographic groups.
\nFor example:
\nThe polynomial kernel captures these interactions implicitly, allowing the model to learn them directly from data.
\nSince the model was trained on a transformed target, predictions were converted back to the original scale before submission.
\nSeveral safeguards were applied to ensure a valid submission:
\nThese steps ensured that all predictions were physically meaningful and submission-safe.
\nThe primary driver of performance was the model’s ability to capture smooth, high-order non-linear relationships in demographic data.
\nRather than relying on complex ensembles, this solution succeeded by:
\nThis resulted in a stable and competitive solution that generalized well.
\nThis solution demonstrates that strong performance does not always require heavy ensembling or deep architectures.
\nBy:
\na relatively simple pipeline achieved top leaderboard performance.
\nThank you for reading, and I’m happy to discuss any part of the approach further.
", "comments": [ { "id": 3381582, "author_name": "Deep Pulusani", "author_url": "", "post_date": "2025-12-25T00:18:42.653000", "content": "Thanks for sharing! Simply plotting the distribution of the target variable in this competition makes a big difference in coming up with a solution!
", "votes": 1, "replies": [ { "id": 3381610, "author_name": "Abhishek Godara", "author_url": "", "post_date": "2025-12-25T03:11:06.960000", "content": "Absolutely agree — and congratulations on the 🏆 top result!
\nPlotting the target early really helped guide the solution.
\nWould love to read a write-up of your approach if you decide to share it.
yes i will plan on it after the holiday! thanks!
", "votes": 1, "replies": [] } ] } ] }, { "id": 3381538, "author_name": "David E.", "author_url": "", "post_date": "2025-12-24T21:25:22.020000", "content": "Nicely done. Was you decision to use a log transform based on its' performance versus other transforms ?
", "votes": 2, "replies": [ { "id": 3381609, "author_name": "Abhishek Godara", "author_url": "", "post_date": "2025-12-25T03:05:59.470000", "content": "Thanks for the question. I compared several target transforms after scaling HOMELESS_RATE × 100.
The raw target was heavily right-skewed. A log transform substantially reduced the tail while still preserving separation among high-rate regions.
\nMore aggressive power transforms (e.g. Yeo–Johnson) produced a more symmetric distribution, but they also over-compressed extreme values. Since high-homelessness regions carry real signal, I preferred the log transform as it offered a better trade-off between variance stabilization, interpretability, and robustness for SVR.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F27467869%2Fa8a3328d0c7f5468b4fb034d5167369c%2FScreenshot%202025-12-25%20082914.png?generation=1766631569164907&alt=media\")
First, thanks to the competition organizers and congratulations to all participants.
\nThis solution focuses on understanding the distributional structure of the target and uncovering non-linear signals that are not captured well by standard correlation analysis or tree-based models.
Rather than relying on Gradient Boosting Trees, this approach uses a Support Vector Regressor (SVR) with a Polynomial kernel, which proved especially effective for this dataset.
\nThe key ideas behind this solution were:
\nThe first step was analyzing the distribution of the target variable, HOMELESS_RATE.
\nComparing the mean and median revealed a noticeable skew, indicating that extreme values were influencing the distribution.
This suggested that standard linear regression models would struggle, particularly in the tails.
\n
Decision:
\nA logarithmic transformation was applied to the target to stabilize variance and reduce skew, improving model behavior during training.
Instead of relying solely on Pearson correlation, which captures only linear relationships, the analysis included Mutual Information (MI) scores.
\nThis revealed several features with:
\nExamples included demographic percentages related to veterans, race, and age groups.
\nInsight:
\nImportant predictive signals existed in non-linear interactions that linear models would miss.\nThis directly motivated the choice of a non-linear kernel-based method.
Because Support Vector Machines are distance-based models, preprocessing played a crucial role.
\nThe pipeline was designed to be:
\nMissing values were handled conservatively to avoid introducing bias, and all features were scaled into a consistent range to ensure balanced distance computations across dimensions.
\nA Support Vector Regressor with a Polynomial kernel was selected for several reasons:
\nUnlike tree-based models, which partition feature space hierarchically, the polynomial kernel projects features into a higher-dimensional space where complex demographic interactions become linearly separable.
\nHomelessness rates are driven not by individual features in isolation, but by intersections of demographic groups.
\nFor example:
\nThe polynomial kernel captures these interactions implicitly, allowing the model to learn them directly from data.
\nSince the model was trained on a transformed target, predictions were converted back to the original scale before submission.
\nSeveral safeguards were applied to ensure a valid submission:
\nThese steps ensured that all predictions were physically meaningful and submission-safe.
\nThe primary driver of performance was the model’s ability to capture smooth, high-order non-linear relationships in demographic data.
\nRather than relying on complex ensembles, this solution succeeded by:
\nThis resulted in a stable and competitive solution that generalized well.
\nThis solution demonstrates that strong performance does not always require heavy ensembling or deep architectures.
\nBy:
\na relatively simple pipeline achieved top leaderboard performance.
\nThank you for reading, and I’m happy to discuss any part of the approach further.
", "rawMarkdown": "# Polynomial SVR & Non-Linear Signal Analysis\n\nFirst, thanks to the competition organizers and congratulations to all participants. \nThis solution focuses on understanding the **distributional structure of the target** and uncovering **non-linear signals** that are not captured well by standard correlation analysis or tree-based models.\n\nRather than relying on Gradient Boosting Trees, this approach uses a **Support Vector Regressor (SVR) with a Polynomial kernel**, which proved especially effective for this dataset.\n\n---\n\n## TL;DR\n\nThe key ideas behind this solution were:\n\n- Careful analysis of the skewed target distribution \n- Explicit detection of non-linear feature signals \n- Choosing a model that **implicitly captures feature interactions** \n- Keeping the pipeline simple and stable \n\n---\n\n## 1. Exploratory Data Analysis (EDA) & Key Insights\n\n### Target Distribution\n\nThe first step was analyzing the distribution of the target variable, `HOMELESS_RATE`. \nComparing the **mean and median** revealed a noticeable skew, indicating that extreme values were influencing the distribution.\n\nThis suggested that standard linear regression models would struggle, particularly in the tails.\n\n\n\n\n**Decision:** \nA logarithmic transformation was applied to the target to stabilize variance and reduce skew, improving model behavior during training.\n\n---\n\n### Non-Linear Signal Detection\n\nInstead of relying solely on Pearson correlation, which captures only linear relationships, the analysis included **Mutual Information (MI)** scores.\n\nThis revealed several features with:\n- Low linear correlation\n- Strong non-linear dependency with the target\n\nExamples included demographic percentages related to veterans, race, and age groups.\n\n**Insight:** \nImportant predictive signals existed in **non-linear interactions** that linear models would miss.\nThis directly motivated the choice of a non-linear kernel-based method.\n\n\n\n---\n\n## 2. Preprocessing Strategy\n\nBecause Support Vector Machines are distance-based models, preprocessing played a crucial role.\n\nThe pipeline was designed to be:\n\n- **Robust** to missing values \n- **Numerically stable** \n- **Compatible** with kernel methods \n\nMissing values were handled conservatively to avoid introducing bias, and all features were scaled into a consistent range to ensure balanced distance computations across dimensions.\n\n---\n\n## 3. Modeling Strategy\n\n### Why Support Vector Regression?\n\nA Support Vector Regressor with a **Polynomial kernel** was selected for several reasons:\n\n- The dataset size was well-suited for kernel methods \n- Polynomial kernels naturally model feature interactions \n- No manual interaction engineering was required \n\nUnlike tree-based models, which partition feature space hierarchically, the polynomial kernel projects features into a higher-dimensional space where complex demographic interactions become linearly separable.\n\n---\n\n### Key Assumption\n\nHomelessness rates are driven not by individual features in isolation, but by **intersections of demographic groups**.\n\nFor example:\n- Age × Race \n- Veteran status × Income proxy \n- Compound demographic percentages \n\nThe polynomial kernel captures these interactions implicitly, allowing the model to learn them directly from data.\n\n---\n\n## 4. Post-Processing & Submission Integrity\n\nSince the model was trained on a transformed target, predictions were converted back to the original scale before submission.\n\nSeveral safeguards were applied to ensure a valid submission:\n- Non-negativity enforcement \n- Handling of numerical edge cases \n- Matching the competition’s expected target scale \n\nThese steps ensured that all predictions were physically meaningful and submission-safe.\n\n---\n\n## 5. Why This Approach Worked\n\nThe primary driver of performance was the model’s ability to **capture smooth, high-order non-linear relationships** in demographic data.\n\nRather than relying on complex ensembles, this solution succeeded by:\n- Choosing a model aligned with the data’s signal structure \n- Letting the kernel handle interactions automatically \n- Avoiding unnecessary model complexity \n\nThis resulted in a stable and competitive solution that generalized well.\n\n---\n\n## Conclusion\n\nThis solution demonstrates that strong performance does not always require heavy ensembling or deep architectures.\n\nBy:\n- Understanding the target distribution \n- Detecting non-linear signals early \n- Selecting a model that matches the data \n\na relatively simple pipeline achieved top leaderboard performance.\n\nThank you for reading, and I’m happy to discuss any part of the approach further.\n", "votes": 6 }, { "id": 3381582, "postDate": "2025-12-25T00:18:42.653Z", "content": "Thanks for sharing! Simply plotting the distribution of the target variable in this competition makes a big difference in coming up with a solution!
", "rawMarkdown": "Thanks for sharing! Simply plotting the distribution of the target variable in this competition makes a big difference in coming up with a solution!", "votes": 1, "replies": [ { "id": 3381610, "postDate": "2025-12-25T03:11:06.960Z", "content": "Absolutely agree — and congratulations on the 🏆 top result!
\nPlotting the target early really helped guide the solution.
\nWould love to read a write-up of your approach if you decide to share it.
yes i will plan on it after the holiday! thanks!
", "rawMarkdown": "yes i will plan on it after the holiday! thanks!", "votes": 1 } ] } ] }, { "id": 3381538, "postDate": "2025-12-24T21:25:22.020Z", "content": "Nicely done. Was you decision to use a log transform based on its' performance versus other transforms ?
", "rawMarkdown": "Nicely done. Was you decision to use a log transform based on its' performance versus other transforms ?", "votes": 2, "replies": [ { "id": 3381609, "postDate": "2025-12-25T03:05:59.470Z", "content": "Thanks for the question. I compared several target transforms after scaling HOMELESS_RATE × 100.
The raw target was heavily right-skewed. A log transform substantially reduced the tail while still preserving separation among high-rate regions.
\nMore aggressive power transforms (e.g. Yeo–Johnson) produced a more symmetric distribution, but they also over-compressed extreme values. Since high-homelessness regions carry real signal, I preferred the log transform as it offered a better trade-off between variance stabilization, interpretability, and robustness for SVR.
\n