![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F9ec7542ebf72b84b5aa239b919cc880a%2Finbox_113660_abc7b467c58faf9b3970782c00e2df04_Selection_999(8305).png?generation=1751465663931833&alt=media\")
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F94c58f1acf2bf2ff81f44916957a176f%2Finbox_113660_df6fa47aaa722296aae89d54f982821c_Selection_999(8306).png?generation=1751465698577730&alt=media\")
My current public LB score is 0.82, and it was achieved by my AI-agent application.
\nFirst, I implemented the solution myself and achieved a public LB score of 0.82. Afterwards, I built an agent application and, with only simple instructions, had the agent handle the implementation.I use Mastra as the agent-application framework.
The agent receives only the following information:
\nHere are my impressions of kaggle agent:
\nWhen using only a single model, the evaluation score is still low. With just the o3 model, the IMU-only CV score was 0.70.
Combining multiple models and having the agent evaluate and plan the implementation is extremely important; doing so raised the LB to 0.82.
Given the complexity of this competition’s task, it seems I don’t need to be very involved in the implementation. For more complex tasks, I’d like to try in the future and see what kind of ranking I can achieve.
", "messages": [ { "id": 3220848, "postDate": "2025-06-10T03:54:52.603Z", "content": "My current public LB score is 0.82, and it was achieved by my AI-agent application.
\nFirst, I implemented the solution myself and achieved a public LB score of 0.82. Afterwards, I built an agent application and, with only simple instructions, had the agent handle the implementation.I use Mastra as the agent-application framework.
The agent receives only the following information:
\nHere are my impressions of kaggle agent:
\nWhen using only a single model, the evaluation score is still low. With just the o3 model, the IMU-only CV score was 0.70.
Combining multiple models and having the agent evaluate and plan the implementation is extremely important; doing so raised the LB to 0.82.
Given the complexity of this competition’s task, it seems I don’t need to be very involved in the implementation. For more complex tasks, I’d like to try in the future and see what kind of ranking I can achieve.
", "rawMarkdown": "My current public LB score is 0.82, and it was achieved by my AI-agent application. \nFirst, I implemented the solution myself and achieved a public LB score of 0.82. Afterwards, I built an agent application and, with only simple instructions, had the agent handle the implementation.I use Mastra as the agent-application framework. \n\nThe agent receives only the following information:\n- Competition data\n- Competition page link\n- System prompt\n- Rule prompt: details on how to conduct experiments, when to run static analysis and tests, etc.\n- My past Kaggle competition code and its explanations: these aren’t critical; they’re mainly used as references for file names and code structure, which helps improve readability.\n\n\nHere are my impressions of kaggle agent:\n- When using only a single model, the evaluation score is still low. With just the o3 model, the IMU-only CV score was 0.70.\n\n- Combining multiple models and having the agent evaluate and plan the implementation is extremely important; doing so raised the LB to 0.82.\n\nGiven the complexity of this competition’s task, it seems I don’t need to be very involved in the implementation. For more complex tasks, I’d like to try in the future and see what kind of ranking I can achieve.", "votes": 150 }, { "id": 3220946, "postDate": "2025-06-10T07:29:56.910Z", "content": "This could be more impressive if you didnt reach that score by yourself first. It feels like you designed your agent to reach your solution. Anyway I haven't looked at this competition, so I may be wrong.
", "rawMarkdown": "This could be more impressive if you didnt reach that score by yourself first. It feels like you designed your agent to reach your solution. Anyway I haven't looked at this competition, so I may be wrong.", "votes": 15, "replies": [ { "id": 3221427, "postDate": "2025-06-11T02:08:09.593Z", "content": "I haven’t built anything specifically for this competition, but because the code is based on what I developed for last year’s leap competition, there might be some built-in biases that help it analyze time-series data effectively.
", "rawMarkdown": "I haven’t built anything specifically for this competition, but because the code is based on what I developed for last year’s leap competition, there might be some built-in biases that help it analyze time-series data effectively.", "votes": 1, "replies": [ { "id": 3221644, "postDate": "2025-06-11T09:59:43.977Z", "content": "@phalanx IMO gold solutions are really really impressive, and i would like to think that any code reference to that helped more than anyone would expected.
\n\n\nThe agent analyzes the CV scores and the types of mistakes in its predictions
\n
I personally find that the concept of having a proper CV is not getting as much attention as it should, outside of kaggle (or perhaps, LB-based competitions). Giving your previous gold workflow, probably helped significantly with the \"analyzes the CV scores\".
\nOr in other words, i am guessing agent is using the same CV scheme, for your 0.82 solution?
", "rawMarkdown": "@phalanx IMO gold solutions are really **really** impressive, and i would like to think that any code reference to that helped more than anyone would expected.\n\n\n> The agent analyzes the CV scores and the types of mistakes in its predictions\n\nI personally find that the concept of having a proper CV is *not* getting as much attention as it should, outside of kaggle (or perhaps, LB-based competitions). Giving your previous gold workflow, probably helped significantly with the \"analyzes the CV scores\". \n\nOr in other words, i am guessing agent is using the same CV scheme, for your 0.82 solution?\n", "votes": 2 }, { "id": 3397931, "postDate": "2026-01-28T09:53:54.337Z", "content": "@phalanx Would you like to share the source code of your agent?
", "rawMarkdown": "@phalanx Would you like to share the source code of your agent?" } ] } ] }, { "id": 3220915, "postDate": "2025-06-10T06:32:30.377Z", "content": "I don't use an agentic flow but I'm also vibing, it's pretty useful when you have so many ideas in your mind and you are lazy to try all of them. 😅
", "rawMarkdown": "I don't use an agentic flow but I'm also vibing, it's pretty useful when you have so many ideas in your mind and you are lazy to try all of them. 😅", "votes": 12 }, { "id": 3231302, "postDate": "2025-06-24T07:04:07.663Z", "content": "AI agent-driven model ensembling significantly improves Kaggle score. Minimal intervention, strong results—excited to try on harder tasks!
", "rawMarkdown": "AI agent-driven model ensembling significantly improves Kaggle score. Minimal intervention, strong results—excited to try on harder tasks!", "votes": 9 }, { "id": 3239130, "postDate": "2025-07-02T14:15:09.487Z", "content": "check my post for AI kaggler
\nhttps://www.kaggle.com/discussions/general/587477
how about a competition where only AI kaggler can take part?
\n
\n
A naive question - how much did it cost to you, in terms of API calls, to run this through your AI-Agent application?
\nI would like to start developing agents, but I'm worried about the costs. Do you have recommendations? Thank you.
Congratulations for the great results! Would it be possible for you share which models you combined to build the agent?
", "rawMarkdown": "Congratulations for the great results! Would it be possible for you share which models you combined to build the agent?", "votes": 3, "replies": [ { "id": 3221416, "postDate": "2025-06-11T01:52:25.040Z", "content": "I use below 4 models.
\nAmazing, insightful work and thank you for your post, just curious to know if you tried using the agent to provide a solution but this time without giving it your past kaggle code and explanation are you able to derive a decent solution/starting template etc ?
", "rawMarkdown": "Amazing, insightful work and thank you for your post, just curious to know if you tried using the agent to provide a solution but this time without giving it your past kaggle code and explanation are you able to derive a decent solution/starting template etc ? " } ] } ] }, { "id": 3220903, "postDate": "2025-06-10T06:02:04.450Z", "content": "This is quite interesting!
\nIf I understand correctly, you have a 0.82 solution, and your agent has built another 0.82 solution without using your existing one. Can we combine the two? Would that work better?
@phalanx do u think that in near future, kaggle competitions will be who writes better llm agent, not competition itself? That would actually be very interesting, who comes up with better ai coder.
", "rawMarkdown": "@phalanx do u think that in near future, kaggle competitions will be who writes better llm agent, not competition itself? That would actually be very interesting, who comes up with better ai coder.", "votes": 4, "replies": [ { "id": 3224049, "postDate": "2025-06-14T10:00:45.203Z", "content": "interesting
", "rawMarkdown": "interesting" }, { "id": 3226159, "postDate": "2025-06-17T10:22:36.430Z", "content": "That would be a bit sad - but unfortunately, likely to be true.
", "rawMarkdown": "That would be a bit sad - but unfortunately, likely to be true." } ] }, { "id": 3222249, "postDate": "2025-06-12T04:26:31.800Z", "content": "Congratulations mate.
", "rawMarkdown": "Congratulations mate.", "votes": 3 }, { "id": 3221307, "postDate": "2025-06-10T18:31:10.070Z", "content": "Really interesting experiment using Mastra to automate your Kaggle workflow! It's impressive that your agent-based implementation was able to match your manually achieved LB score of 0.82 with just high-level guidance.
", "rawMarkdown": "Really interesting experiment using Mastra to automate your Kaggle workflow! It's impressive that your agent-based implementation was able to match your manually achieved LB score of 0.82 with just high-level guidance.", "votes": 1 }, { "id": 3220856, "postDate": "2025-06-10T04:15:22.253Z", "content": "For your 0.82 LB score, are you seeing a similar level of improvement in local CV? With a few competitors reaching 0.80 now, I can see there being a conceptual 'step' to take to improve the model overall, I just ask because you don't state it. It's surprising that 'vibe coding' as you say can be so effective for the competition, though I'd imagine this is only possible due to your previous experience.
", "rawMarkdown": "For your 0.82 LB score, are you seeing a similar level of improvement in local CV? With a few competitors reaching 0.80 now, I can see there being a conceptual 'step' to take to improve the model overall, I just ask because you don't state it. It's surprising that 'vibe coding' as you say can be so effective for the competition, though I'd imagine this is only possible due to your previous experience.", "votes": 1, "replies": [ { "id": 3220873, "postDate": "2025-06-10T05:10:04.833Z", "content": "The agent analyzes the CV scores and the types of mistakes in its predictions, then devises and implements improvement ideas based on those insights.
", "rawMarkdown": "The agent analyzes the CV scores and the types of mistakes in its predictions, then devises and implements improvement ideas based on those insights.", "votes": 3 } ] }, { "id": 3220852, "postDate": "2025-06-10T04:02:33.720Z", "content": "awesome results! does the agent come up with experiment plan itself or it's given by rule prompts?
", "rawMarkdown": "awesome results! does the agent come up with experiment plan itself or it's given by rule prompts?", "votes": 1, "replies": [ { "id": 3220871, "postDate": "2025-06-10T05:07:00.333Z", "content": "agent drafts its own plans and carries out the implementation. The rules lay out in detail the guidelines to follow—such as how to perform EDA and how to build models—and the agent relies on them when planning and implementing.
", "rawMarkdown": "agent drafts its own plans and carries out the implementation. The rules lay out in detail the guidelines to follow—such as how to perform EDA and how to build models—and the agent relies on them when planning and implementing.", "votes": 2 } ] }, { "id": 3223321, "postDate": "2025-06-13T06:52:18.457Z", "content": "Fantastic sharing! That's amazing, just give me some hints!
", "rawMarkdown": "Fantastic sharing! That's amazing, just give me some hints!", "votes": 1 }, { "id": 3239068, "postDate": "2025-07-02T13:12:40.183Z", "content": "Very amazing!
", "rawMarkdown": "Very amazing!", "votes": -1 }, { "id": 3229072, "postDate": "2025-06-21T00:45:34.210Z", "content": "Congratulations! for a good result.
", "rawMarkdown": "Congratulations! for a good result.\n", "votes": -1 }, { "id": 3227117, "postDate": "2025-06-18T14:46:01.050Z", "content": "Thank you for giving me an heart attack as a first timer. I can't even go past 0.60 score. 🥲 lol
", "rawMarkdown": "Thank you for giving me an heart attack as a first timer. I can't even go past 0.60 score. 🥲 lol", "votes": -1 }, { "id": 3226274, "postDate": "2025-06-17T13:03:50.483Z", "content": "What kind of licenses do you have to do such a thing?
\nI know that openAI has licenses to use their API programatically, did you buy those?
Love to know more about this. I'll keep a tab on it.
", "rawMarkdown": "Love to know more about this. I'll keep a tab on it." }, { "id": 3224702, "postDate": "2025-06-15T10:51:50.627Z", "content": "Hello Everyone,
\nI am going to make project on Big data analytics using Machine Learning, Big Data Technologies and using Database give me some idea and resources to get best fit title for my project
oh gee, just read this. It blows my mind, agent has advanced to this level?
", "rawMarkdown": "oh gee, just read this. It blows my mind, agent has advanced to this level?" }, { "id": 3223657, "postDate": "2025-06-13T16:43:05.290Z", "content": "This is quite interesting….!
", "rawMarkdown": "This is quite interesting....!" }, { "id": 3223590, "postDate": "2025-06-13T14:27:30.273Z", "content": "Impressive work, 0.82 LB with an AI agent led pipeline demonstrates strong modeling, orchestration and prompt engineering, its exciting to see agents reaching this level of autonomy.👍
", "rawMarkdown": "Impressive work, 0.82 LB with an AI agent led pipeline demonstrates strong modeling, orchestration and prompt engineering, its exciting to see agents reaching this level of autonomy.👍" }, { "id": 3223167, "postDate": "2025-06-13T00:31:01.120Z", "content": "By \"using multiple models\" do you mean \"using different models to create a few different version of plans and codes, then let 1 model to evaluate all the options and choose the best one?
", "rawMarkdown": "By \"using multiple models\" do you mean \"using different models to create a few different version of plans and codes, then let 1 model to evaluate all the options and choose the best one?" }, { "id": 3222574, "postDate": "2025-06-12T09:18:50.440Z", "content": "nice very
", "rawMarkdown": "nice very " }, { "id": 3222234, "postDate": "2025-06-12T03:46:22.030Z", "content": "Which model did you use? I only used LightGBM, and my LB score is not exceeding 0.7.
", "rawMarkdown": "Which model did you use? I only used LightGBM, and my LB score is not exceeding 0.7." }, { "id": 3222219, "postDate": "2025-06-12T02:57:26.100Z", "content": "MO gold solutions are really great.
", "rawMarkdown": "MO gold solutions are really great." }, { "id": 3222138, "postDate": "2025-06-11T22:09:20.700Z", "content": "Congratulations! Doesn't matter if you automated it or not, you did it and someone in real life will benefit from your work.
", "rawMarkdown": "Congratulations! Doesn't matter if you automated it or not, you did it and someone in real life will benefit from your work. " }, { "id": 3221444, "postDate": "2025-06-11T03:01:53.893Z", "content": "@phalanx nice to see, you motivated everyone and make this competition as LLM competition and impressive results with autollm --> i read similar 2 papers auto agents in the similar but didn't remember exact names.
\nis possible to see your Rule prompt
", "rawMarkdown": "@phalanx nice to see, you motivated everyone and make this competition as LLM competition and impressive results with autollm --> i read similar 2 papers auto agents in the similar but didn't remember exact names.\n\nis possible to see your **Rule prompt**", "replies": [ { "id": 3221455, "postDate": "2025-06-11T03:18:04.167Z", "content": "I’ve built an agent for this competition, but I’m torn between releasing it (or parts of it) publicly and continuing to use it privately. If using agents becomes the norm, the cost of competing on Kaggle will rise dramatically. We should be lowering the barriers to entry for Kaggle; otherwise, people who are just beginning to learn machine learning may hesitate to participate in the future.
\nI’d also like to hear what the organizers and other participants think about this.
", "rawMarkdown": "I’ve built an agent for this competition, but I’m torn between releasing it (or parts of it) publicly and continuing to use it privately. If using agents becomes the norm, the cost of competing on Kaggle will rise dramatically. We should be lowering the barriers to entry for Kaggle; otherwise, people who are just beginning to learn machine learning may hesitate to participate in the future.\n\nI’d also like to hear what the organizers and other participants think about this.", "votes": 13, "replies": [ { "id": 3221627, "postDate": "2025-06-11T09:13:20.530Z", "content": "@phalanx I agree, this is already an issue on kaggle, and not only for beginners. Basically most contests are out of reach for those that don’t have dedicated hardware/compute.
\nA simple idea would be for contests to include a “kaggle compute tier” that also awards prizes/medals. These would be lower/less than the overall contests but would at least be a space where compute/resources isn’t the overriding factor. Only submissions that use kaggle resources would be valid, ie training locally or on some compute service and then loading the model for inference wouldn’t be a valid submission in this tier. This would still be fine for the overall contest.
Obviously this doesn't resolve the specific issue of paying for agents you raise, but it's in the same spirit of trying to keep kaggle a reasonably open and low barrier place.
", "rawMarkdown": "@phalanx I agree, this is already an issue on kaggle, and not only for beginners. Basically most contests are out of reach for those that don’t have dedicated hardware/compute. \nA simple idea would be for contests to include a “kaggle compute tier” that also awards prizes/medals. These would be lower/less than the overall contests but would at least be a space where compute/resources isn’t the overriding factor. Only submissions that use kaggle resources would be valid, ie training locally or on some compute service and then loading the model for inference wouldn’t be a valid submission in this tier. This would still be fine for the overall contest. \n\nObviously this doesn't resolve the specific issue of paying for agents you raise, but it's in the same spirit of trying to keep kaggle a reasonably open and low barrier place. ", "votes": 4 }, { "id": 3253861, "postDate": "2025-07-25T12:29:04.047Z", "content": "I think that depends on the sponsor of the competition. In some cases, I’ve seen that only Kaggle resources can be used because they’re looking for models with better computational efficiency (they might not have the necessary infrastructure). But in other competitions, I’ve seen it’s open, as the sponsors can afford to deploy more complex models.
", "rawMarkdown": "I think that depends on the sponsor of the competition. In some cases, I’ve seen that only Kaggle resources can be used because they’re looking for models with better computational efficiency (they might not have the necessary infrastructure). But in other competitions, I’ve seen it’s open, as the sponsors can afford to deploy more complex models." } ] } ] }, { "id": 3221330, "postDate": "2025-06-10T19:19:57.467Z", "content": "@phalanx I am trying to understand this. I believe that after implementing a baseline code, we then need to use an external AI agent framework—Mastra. Mastra receives the completion data, and it generates the solution to submit.
\nPlease correct me if I am wrong.
", "rawMarkdown": "@phalanx I am trying to understand this. I believe that after implementing a baseline code, we then need to use an external AI agent framework—Mastra. Mastra receives the completion data, and it generates the solution to submit. \n\nPlease correct me if I am wrong. ", "replies": [ { "id": 3221430, "postDate": "2025-06-11T02:17:33.793Z", "content": "I don’t need the baseline code for this competition. agent implement new code for this competition using the code architecture from past competitions as a reference.
", "rawMarkdown": "I don’t need the baseline code for this competition. agent implement new code for this competition using the code architecture from past competitions as a reference.", "votes": 1 } ] }, { "id": 3220972, "postDate": "2025-06-10T08:00:00.613Z", "content": "Thank you for the very interesting post!
\nBy the way, how many cycles did you go through? (i.e., how many times did you provide feedback to the Kaggle agent via CV or LB?)
I use a human-in-the-loop setup, so I had to answer a few follow-up questions whenever something wasn’t clear. Generally, though, once I give instructions the first time, no additional intervention is needed after that.
", "rawMarkdown": "I use a human-in-the-loop setup, so I had to answer a few follow-up questions whenever something wasn’t clear. Generally, though, once I give instructions the first time, no additional intervention is needed after that.", "votes": 3 } ] }, { "id": 3220925, "postDate": "2025-06-10T06:50:07.067Z", "content": "Interesting experiment, @phalanx. Even if the score did not improve, the time savings would still make it worthwhile. How faithfully did the agent adhere to your instructions? Did it make any surprising moves or discoveries?
", "rawMarkdown": "Interesting experiment, @phalanx. Even if the score did not improve, the time savings would still make it worthwhile. How faithfully did the agent adhere to your instructions? Did it make any surprising moves or discoveries?", "replies": [ { "id": 3221426, "postDate": "2025-06-11T02:02:26.703Z", "content": "I didn’t observe any quirks unique to Kaggle. Most issues are common ones—such as skipping static analysis or tests at the right moments, and editing code directly instead of creating a new branch for each experiment.
", "rawMarkdown": "I didn’t observe any quirks unique to Kaggle. Most issues are common ones—such as skipping static analysis or tests at the right moments, and editing code directly instead of creating a new branch for each experiment.", "votes": 1 } ] }, { "id": 3258634, "postDate": "2025-07-31T01:34:40.500Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 3237753, "postDate": "2025-07-01T09:28:19.047Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3227450, "postDate": "2025-06-19T02:54:20.043Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 3226476, "postDate": "2025-06-17T17:56:16.703Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3226461, "postDate": "2025-06-17T17:25:41.813Z", "rawMarkdown": "", "votes": -2, "isDeleted": true, "replies": [ { "id": 3228865, "postDate": "2025-06-20T15:54:06.890Z", "content": "TBH bro this comment looks like it is generated by some LLM lol
", "rawMarkdown": "TBH bro this comment looks like it is generated by some LLM lol", "votes": -1 } ] }, { "id": 3223025, "postDate": "2025-06-12T18:57:58.920Z", "content": "Great work
", "rawMarkdown": "Great work", "votes": -1 }, { "id": 3222570, "postDate": "2025-06-12T09:15:30.767Z", "content": "Congratulations
", "rawMarkdown": "Congratulations", "votes": -1 }, { "id": 3222554, "postDate": "2025-06-12T08:57:31.707Z", "content": "Congratulations
", "rawMarkdown": "Congratulations", "votes": -1 }, { "id": 3222407, "postDate": "2025-06-12T07:23:31.157Z", "content": "well done
", "rawMarkdown": "well done\n", "votes": -1 }, { "id": 3222393, "postDate": "2025-06-12T07:17:53.577Z", "content": "congratulations
", "rawMarkdown": "congratulations", "votes": -1 } ], "comments": [ { "id": 3220946, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2025-06-10T07:29:56.910000", "content": "This could be more impressive if you didnt reach that score by yourself first. It feels like you designed your agent to reach your solution. Anyway I haven't looked at this competition, so I may be wrong.
", "votes": 15, "replies": [ { "id": 3221427, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-11T02:08:09.593000", "content": "I haven’t built anything specifically for this competition, but because the code is based on what I developed for last year’s leap competition, there might be some built-in biases that help it analyze time-series data effectively.
", "votes": 1, "replies": [ { "id": 3221644, "author_name": "Gerwyn", "author_url": "", "post_date": "2025-06-11T09:59:43.977000", "content": "@phalanx IMO gold solutions are really really impressive, and i would like to think that any code reference to that helped more than anyone would expected.
\n\n\nThe agent analyzes the CV scores and the types of mistakes in its predictions
\n
I personally find that the concept of having a proper CV is not getting as much attention as it should, outside of kaggle (or perhaps, LB-based competitions). Giving your previous gold workflow, probably helped significantly with the \"analyzes the CV scores\".
\nOr in other words, i am guessing agent is using the same CV scheme, for your 0.82 solution?
", "votes": 2, "replies": [] }, { "id": 3397931, "author_name": "ironrro", "author_url": "", "post_date": "2026-01-28T09:53:54.337000", "content": "@phalanx Would you like to share the source code of your agent?
", "votes": 0, "replies": [] } ] } ] }, { "id": 3220915, "author_name": "Anil Ozturk", "author_url": "", "post_date": "2025-06-10T06:32:30.377000", "content": "I don't use an agentic flow but I'm also vibing, it's pretty useful when you have so many ideas in your mind and you are lazy to try all of them. 😅
", "votes": 12, "replies": [] }, { "id": 3231302, "author_name": "Pratyush Puri", "author_url": "", "post_date": "2025-06-24T07:04:07.663000", "content": "AI agent-driven model ensembling significantly improves Kaggle score. Minimal intervention, strong results—excited to try on harder tasks!
", "votes": 9, "replies": [] }, { "id": 3239130, "author_name": "hengck23", "author_url": "", "post_date": "2025-07-02T14:15:09.487000", "content": "check my post for AI kaggler
\nhttps://www.kaggle.com/discussions/general/587477
how about a competition where only AI kaggler can take part?
\n
\n
A naive question - how much did it cost to you, in terms of API calls, to run this through your AI-Agent application?
\nI would like to start developing agents, but I'm worried about the costs. Do you have recommendations? Thank you.
Congratulations for the great results! Would it be possible for you share which models you combined to build the agent?
", "votes": 3, "replies": [ { "id": 3221416, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-11T01:52:25.040000", "content": "I use below 4 models.
\nAmazing, insightful work and thank you for your post, just curious to know if you tried using the agent to provide a solution but this time without giving it your past kaggle code and explanation are you able to derive a decent solution/starting template etc ?
", "votes": 0, "replies": [] } ] } ] }, { "id": 3220903, "author_name": "Spiritmilk", "author_url": "", "post_date": "2025-06-10T06:02:04.450000", "content": "This is quite interesting!
\nIf I understand correctly, you have a 0.82 solution, and your agent has built another 0.82 solution without using your existing one. Can we combine the two? Would that work better?
@phalanx do u think that in near future, kaggle competitions will be who writes better llm agent, not competition itself? That would actually be very interesting, who comes up with better ai coder.
", "votes": 4, "replies": [ { "id": 3224049, "author_name": "Shalma Muji", "author_url": "", "post_date": "2025-06-14T10:00:45.203000", "content": "interesting
", "votes": 0, "replies": [] }, { "id": 3226159, "author_name": "Giovanni Marco Dall'Olio", "author_url": "", "post_date": "2025-06-17T10:22:36.430000", "content": "That would be a bit sad - but unfortunately, likely to be true.
", "votes": 0, "replies": [] } ] }, { "id": 3222249, "author_name": "", "author_url": "", "post_date": "2025-06-12T04:26:31.800000", "content": "Congratulations mate.
", "votes": 3, "replies": [] }, { "id": 3221307, "author_name": "Harsh Gupta", "author_url": "", "post_date": "2025-06-10T18:31:10.070000", "content": "Really interesting experiment using Mastra to automate your Kaggle workflow! It's impressive that your agent-based implementation was able to match your manually achieved LB score of 0.82 with just high-level guidance.
", "votes": 1, "replies": [] }, { "id": 3220856, "author_name": "Peter Moorhouse", "author_url": "", "post_date": "2025-06-10T04:15:22.253000", "content": "For your 0.82 LB score, are you seeing a similar level of improvement in local CV? With a few competitors reaching 0.80 now, I can see there being a conceptual 'step' to take to improve the model overall, I just ask because you don't state it. It's surprising that 'vibe coding' as you say can be so effective for the competition, though I'd imagine this is only possible due to your previous experience.
", "votes": 1, "replies": [ { "id": 3220873, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-10T05:10:04.833000", "content": "The agent analyzes the CV scores and the types of mistakes in its predictions, then devises and implements improvement ideas based on those insights.
", "votes": 3, "replies": [] } ] }, { "id": 3220852, "author_name": "Zacchaeus", "author_url": "", "post_date": "2025-06-10T04:02:33.720000", "content": "awesome results! does the agent come up with experiment plan itself or it's given by rule prompts?
", "votes": 1, "replies": [ { "id": 3220871, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-10T05:07:00.333000", "content": "agent drafts its own plans and carries out the implementation. The rules lay out in detail the guidelines to follow—such as how to perform EDA and how to build models—and the agent relies on them when planning and implementing.
", "votes": 2, "replies": [] } ] }, { "id": 3223321, "author_name": "", "author_url": "", "post_date": "2025-06-13T06:52:18.457000", "content": "Fantastic sharing! That's amazing, just give me some hints!
", "votes": 1, "replies": [] }, { "id": 3239068, "author_name": "ADORBA", "author_url": "", "post_date": "2025-07-02T13:12:40.183000", "content": "Very amazing!
", "votes": -1, "replies": [] }, { "id": 3229072, "author_name": "M- Zohaib-KH", "author_url": "", "post_date": "2025-06-21T00:45:34.210000", "content": "Congratulations! for a good result.
", "votes": -1, "replies": [] }, { "id": 3227117, "author_name": "Syed Muhammad Mehran", "author_url": "", "post_date": "2025-06-18T14:46:01.050000", "content": "Thank you for giving me an heart attack as a first timer. I can't even go past 0.60 score. 🥲 lol
", "votes": -1, "replies": [] }, { "id": 3226274, "author_name": "Gio", "author_url": "", "post_date": "2025-06-17T13:03:50.483000", "content": "What kind of licenses do you have to do such a thing?
\nI know that openAI has licenses to use their API programatically, did you buy those?
Love to know more about this. I'll keep a tab on it.
", "votes": 0, "replies": [] }, { "id": 3224702, "author_name": "Akshay Sutar", "author_url": "", "post_date": "2025-06-15T10:51:50.627000", "content": "Hello Everyone,
\nI am going to make project on Big data analytics using Machine Learning, Big Data Technologies and using Database give me some idea and resources to get best fit title for my project
oh gee, just read this. It blows my mind, agent has advanced to this level?
", "votes": 0, "replies": [] }, { "id": 3223657, "author_name": "Sarah Arshad", "author_url": "", "post_date": "2025-06-13T16:43:05.290000", "content": "This is quite interesting….!
", "votes": 0, "replies": [] }, { "id": 3223590, "author_name": "KartikSaini18", "author_url": "", "post_date": "2025-06-13T14:27:30.273000", "content": "Impressive work, 0.82 LB with an AI agent led pipeline demonstrates strong modeling, orchestration and prompt engineering, its exciting to see agents reaching this level of autonomy.👍
", "votes": 0, "replies": [] }, { "id": 3223167, "author_name": "Steven Boon", "author_url": "", "post_date": "2025-06-13T00:31:01.120000", "content": "By \"using multiple models\" do you mean \"using different models to create a few different version of plans and codes, then let 1 model to evaluate all the options and choose the best one?
", "votes": 0, "replies": [] }, { "id": 3222574, "author_name": "102203195_Rajveer_Singh", "author_url": "", "post_date": "2025-06-12T09:18:50.440000", "content": "nice very
", "votes": 0, "replies": [] }, { "id": 3222234, "author_name": "Rustam Bazarbayev", "author_url": "", "post_date": "2025-06-12T03:46:22.030000", "content": "Which model did you use? I only used LightGBM, and my LB score is not exceeding 0.7.
", "votes": 0, "replies": [] }, { "id": 3222219, "author_name": "Dhruv_Rajoria", "author_url": "", "post_date": "2025-06-12T02:57:26.100000", "content": "MO gold solutions are really great.
", "votes": 0, "replies": [] }, { "id": 3222138, "author_name": "Thiago Mania", "author_url": "", "post_date": "2025-06-11T22:09:20.700000", "content": "Congratulations! Doesn't matter if you automated it or not, you did it and someone in real life will benefit from your work.
", "votes": 0, "replies": [] }, { "id": 3221444, "author_name": "SeshuRaju 🧘♂️", "author_url": "", "post_date": "2025-06-11T03:01:53.893000", "content": "@phalanx nice to see, you motivated everyone and make this competition as LLM competition and impressive results with autollm --> i read similar 2 papers auto agents in the similar but didn't remember exact names.
\nis possible to see your Rule prompt
", "votes": 0, "replies": [ { "id": 3221455, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-11T03:18:04.167000", "content": "I’ve built an agent for this competition, but I’m torn between releasing it (or parts of it) publicly and continuing to use it privately. If using agents becomes the norm, the cost of competing on Kaggle will rise dramatically. We should be lowering the barriers to entry for Kaggle; otherwise, people who are just beginning to learn machine learning may hesitate to participate in the future.
\nI’d also like to hear what the organizers and other participants think about this.
", "votes": 13, "replies": [ { "id": 3221627, "author_name": "DanteTheAbstract", "author_url": "", "post_date": "2025-06-11T09:13:20.530000", "content": "@phalanx I agree, this is already an issue on kaggle, and not only for beginners. Basically most contests are out of reach for those that don’t have dedicated hardware/compute.
\nA simple idea would be for contests to include a “kaggle compute tier” that also awards prizes/medals. These would be lower/less than the overall contests but would at least be a space where compute/resources isn’t the overriding factor. Only submissions that use kaggle resources would be valid, ie training locally or on some compute service and then loading the model for inference wouldn’t be a valid submission in this tier. This would still be fine for the overall contest.
Obviously this doesn't resolve the specific issue of paying for agents you raise, but it's in the same spirit of trying to keep kaggle a reasonably open and low barrier place.
", "votes": 4, "replies": [] }, { "id": 3253861, "author_name": "Pablo Lucero", "author_url": "", "post_date": "2025-07-25T12:29:04.047000", "content": "I think that depends on the sponsor of the competition. In some cases, I’ve seen that only Kaggle resources can be used because they’re looking for models with better computational efficiency (they might not have the necessary infrastructure). But in other competitions, I’ve seen it’s open, as the sponsors can afford to deploy more complex models.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3221330, "author_name": "Samith Chimminiyan", "author_url": "", "post_date": "2025-06-10T19:19:57.467000", "content": "@phalanx I am trying to understand this. I believe that after implementing a baseline code, we then need to use an external AI agent framework—Mastra. Mastra receives the completion data, and it generates the solution to submit.
\nPlease correct me if I am wrong.
", "votes": 0, "replies": [ { "id": 3221430, "author_name": "phalanx", "author_url": "", "post_date": "2025-06-11T02:17:33.793000", "content": "I don’t need the baseline code for this competition. agent implement new code for this competition using the code architecture from past competitions as a reference.
", "votes": 1, "replies": [] } ] }, { "id": 3220972, "author_name": "", "author_url": "", "post_date": "2025-06-10T08:00:00.613000", "content": "", "votes": 0, "replies": [ { "id": 3221428, "author_name": "", "author_url": "", "post_date": "2025-06-11T02:13:25.290000", "content": "", "votes": 3, "replies": [] } ] }, { "id": 3220925, "author_name": "", "author_url": "", "post_date": "2025-06-10T06:50:07.067000", "content": "", "votes": 0, "replies": [ { "id": 3221426, "author_name": "", "author_url": "", "post_date": "2025-06-11T02:02:26.703000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 3258634, "author_name": "", "author_url": "", "post_date": "2025-07-31T01:34:40.500000", "content": "", "votes": -1, "replies": [] }, { "id": 3237753, "author_name": "", "author_url": "", "post_date": "2025-07-01T09:28:19.047000", "content": "", "votes": 0, "replies": [] }, { "id": 3227450, "author_name": "", "author_url": "", "post_date": "2025-06-19T02:54:20.043000", "content": "", "votes": 1, "replies": [] }, { "id": 3226476, "author_name": "", "author_url": "", "post_date": "2025-06-17T17:56:16.703000", "content": "", "votes": 0, "replies": [] }, { "id": 3226461, "author_name": "", "author_url": "", "post_date": "2025-06-17T17:25:41.813000", "content": "", "votes": -2, "replies": [ { "id": 3228865, "author_name": "", "author_url": "", "post_date": "2025-06-20T15:54:06.890000", "content": "", "votes": -1, "replies": [] } ] }, { "id": 3223025, "author_name": "", "author_url": "", "post_date": "2025-06-12T18:57:58.920000", "content": "", "votes": -1, "replies": [] }, { "id": 3222570, "author_name": "", "author_url": "", "post_date": "2025-06-12T09:15:30.767000", "content": "", "votes": -1, "replies": [] }, { "id": 3222554, "author_name": "", "author_url": "", "post_date": "2025-06-12T08:57:31.707000", "content": "", "votes": -1, "replies": [] }, { "id": 3222407, "author_name": "", "author_url": "", "post_date": "2025-06-12T07:23:31.157000", "content": "", "votes": -1, "replies": [] }, { "id": 3222393, "author_name": "", "author_url": "", "post_date": "2025-06-12T07:17:53.577000", "content": "", "votes": -1, "replies": [] } ], "raw_markdown_by_id": { "3220848": "My current public LB score is 0.82, and it was achieved by my AI-agent application. \nFirst, I implemented the solution myself and achieved a public LB score of 0.82. Afterwards, I built an agent application and, with only simple instructions, had the agent handle the implementation.I use Mastra as the agent-application framework. \n\nThe agent receives only the following information:\n- Competition data\n- Competition page link\n- System prompt\n- Rule prompt: details on how to conduct experiments, when to run static analysis and tests, etc.\n- My past Kaggle competition code and its explanations: these aren’t critical; they’re mainly used as references for file names and code structure, which helps improve readability.\n\n\nHere are my impressions of kaggle agent:\n- When using only a single model, the evaluation score is still low. With just the o3 model, the IMU-only CV score was 0.70.\n\n- Combining multiple models and having the agent evaluate and plan the implementation is extremely important; doing so raised the LB to 0.82.\n\nGiven the complexity of this competition’s task, it seems I don’t need to be very involved in the implementation. For more complex tasks, I’d like to try in the future and see what kind of ranking I can achieve.", "3220946": "This could be more impressive if you didnt reach that score by yourself first. It feels like you designed your agent to reach your solution. Anyway I haven't looked at this competition, so I may be wrong.", "3220915": "I don't use an agentic flow but I'm also vibing, it's pretty useful when you have so many ideas in your mind and you are lazy to try all of them. 😅", "3231302": "AI agent-driven model ensembling significantly improves Kaggle score. Minimal intervention, strong results—excited to try on harder tasks!", "3239130": "check my post for AI kaggler\nhttps://www.kaggle.com/discussions/general/587477\n\nhow about a competition where only AI kaggler can take part?\n\n.png?generation=1751465663931833&alt=media)\n.png?generation=1751465698577730&alt=media)", "3226386": "A naive question - how much did it cost to you, in terms of API calls, to run this through your AI-Agent application? \nI would like to start developing agents, but I'm worried about the costs. Do you have recommendations? Thank you.", "3220924": "Congratulations for the great results! Would it be possible for you share which models you combined to build the agent?", "3220903": "This is quite interesting! \nIf I understand correctly, you have a 0.82 solution, and your agent has built another 0.82 solution without using your existing one. Can we combine the two? Would that work better?", "3223785": "@phalanx do u think that in near future, kaggle competitions will be who writes better llm agent, not competition itself? That would actually be very interesting, who comes up with better ai coder.", "3222249": "Congratulations mate.", "3221307": "Really interesting experiment using Mastra to automate your Kaggle workflow! It's impressive that your agent-based implementation was able to match your manually achieved LB score of 0.82 with just high-level guidance.", "3220856": "For your 0.82 LB score, are you seeing a similar level of improvement in local CV? With a few competitors reaching 0.80 now, I can see there being a conceptual 'step' to take to improve the model overall, I just ask because you don't state it. It's surprising that 'vibe coding' as you say can be so effective for the competition, though I'd imagine this is only possible due to your previous experience.", "3220852": "awesome results! does the agent come up with experiment plan itself or it's given by rule prompts?", "3223321": "Fantastic sharing! That's amazing, just give me some hints!", "3239068": "Very amazing!", "3229072": "Congratulations! for a good result.\n", "3227117": "Thank you for giving me an heart attack as a first timer. I can't even go past 0.60 score. 🥲 lol", "3226274": "What kind of licenses do you have to do such a thing?\nI know that openAI has licenses to use their API programatically, did you buy those?", "3225646": "Love to know more about this. I'll keep a tab on it.", "3224702": "Hello Everyone,\nI am going to make project on Big data analytics using Machine Learning, Big Data Technologies and using Database give me some idea and resources to get best fit title for my project ", "3223980": "oh gee, just read this. It blows my mind, agent has advanced to this level?", "3223657": "This is quite interesting....!", "3223590": "Impressive work, 0.82 LB with an AI agent led pipeline demonstrates strong modeling, orchestration and prompt engineering, its exciting to see agents reaching this level of autonomy.👍", "3223167": "By \"using multiple models\" do you mean \"using different models to create a few different version of plans and codes, then let 1 model to evaluate all the options and choose the best one?", "3222574": "nice very ", "3222234": "Which model did you use? I only used LightGBM, and my LB score is not exceeding 0.7.", "3222219": "MO gold solutions are really great.", "3222138": "Congratulations! Doesn't matter if you automated it or not, you did it and someone in real life will benefit from your work. ", "3221444": "@phalanx nice to see, you motivated everyone and make this competition as LLM competition and impressive results with autollm --> i read similar 2 papers auto agents in the similar but didn't remember exact names.\n\nis possible to see your **Rule prompt**", "3221330": "@phalanx I am trying to understand this. I believe that after implementing a baseline code, we then need to use an external AI agent framework—Mastra. Mastra receives the completion data, and it generates the solution to submit. \n\nPlease correct me if I am wrong. ", "3220972": "Thank you for the very interesting post!\nBy the way, how many cycles did you go through? (i.e., how many times did you provide feedback to the Kaggle agent via CV or LB?)", "3220925": "Interesting experiment, @phalanx. Even if the score did not improve, the time savings would still make it worthwhile. How faithfully did the agent adhere to your instructions? Did it make any surprising moves or discoveries?", "3258634": "", "3237753": "", "3227450": "", "3226476": "", "3226461": "", "3223025": "Great work", "3222570": "Congratulations", "3222554": "Congratulations", "3222407": "well done\n", "3222393": "congratulations" } }