{ "id": 360097, "title": "2nd Place Solution", "content": "

Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! Since this competition can be approached in a variety of ways, I'm looking forward to see everyone's solution.

\n

Overview of my solution

\n

My pipeline consists of:

\n\n

Once I found that the attention or cropping near the ball is effective to improve the accuracy of action classifier, I focused on refining the performance of the detector and classifier.

\n

\"pipeline\"

\n
\n

1. Optical Flow

\n

It's difficult to detect the ball when:

\n\n

So my idea is to use multiple frames to detect the ball on play which is usually moving by the optical flow.
\nAt first, I implemented the optical flow with high resolution image since balls are very small, but it's slow and not suitable for this competition. So I adopted the low resolution OpticalFlow to predict only the global (camera) motion. Flow itself cannot tell which one is the ball on play, but the difference between warped image and original image is very informative.
\n\"opticalflow_motivation\"
\nI used RAFT to predict optical flow. It is trained with the provided video by self-supervised manner.
\n\"opticalflow_implementation\"

\n
\n

2. Ball Detection

\n

To make the ball detector, I repeated training and annotation several times.
\n(Annotation -> Train Detector -> Predict -> Annotation -> Train Detector -> Predict -> …)
\nI selected CenterNet because I do not like annotation. Annotating point is much easier than rectangle.

\n

Since the target(ball) is very small. Inputs to the model is high resolution images.
\nTo achieve both inference speed and accuracy, I used the initial part of the encoder.
\n\"balldetector\"

\n

Inputs to the detector consist of 9 channels:

\n\n

The latter two drastically improved the accuracy of detector especially in the crowded scene.
\n\"detection_example\"

\n
\n

3. Cost Minimization of Ball Trajectory

\n

Ball location can be estimated more accurately by using multi-frame information. I applied cost minimization on selecting the ball path.
\nIf we know the trajectory of the ball, we can estimate where it will go next. In the following case, people can select the next point correctly even if the ball is hard to seen.
\n\"ball_path\"

\n

As I want to consider both detector's score and distance based probability, I defined
\n Cost = Function (Confidence of Detector, Distance between Nodes)
\nand minimized the total cost of the path by Dijkstra Algorithm .

\n

It can create a more natural and accurate trajectory than just connecting points with the highest scores.
\n\"ball_path_image\"
\n(If this competition was for the ball detection, I would get first place!)

\n

Videos can be seen here. Left one is the max confidence path and right one is the min cost path.
\nhttps://twitter.com/i/status/1582747059165495297

\n
\n

4. Event Classifier (Action Recognition)

\n

Now we know where the ball is. Next step is to predict event. I used ball trajectory and images cropped near ball over 1 sec to predict action(4 class) and regression of residuals.
\nSince the labeled intervals are very short, it's difficult to make the long sequential training data. I extracted only 1 sec frames randomly from labeled intervals as an input to the model.

\n

The model consists of :
\n1) 2D CNN to extract features from cropped images
\n2) 1D CNN to extract features from ball path
\n3) 1D CNN to predict event from upper two sets of features
\n\"event_detect_1st\"

\n
\n

5. Event Classifier [2nd Stage]

\n

The first classifier cannot predict the event with long-term information.
\nAt second stage, long sequential inputs(4 sec) can be adopted because the inputs of the model is only feature vectors which is much more memory-efficient than the images.
\n\"second_stage\"

\n
\n

6. PostProcessing

\n

Simply, combination of gaussian weight and peak detection.

\n\n
\n

Final Thought

\n

Since the number of samples of the events is not so large and the most of the area in a image is useless, I thought the model would hard to learn and would overfit easily. So I soon decided to use the ball trajectory.
\nThat's why I really amazed that many kagglers suceeded the simple end-to-end approach. I've learned a lot and still need to learn a lot from this competition. Thanks again to kaggle community!

", "comments": [ { "id": 1993471, "author_name": "Guillermo García Cobo", "author_url": "", "post_date": "2022-10-18T12:08:14.303000", "content": "

Congrats on the great solution @kmat2019! The logic behind all the steps is great. Moreover, congratulations on the detailed post and the amazing visualizations. What tool did you use to generate them?

\n

First, I wanted to ask you about the reduction to 12.5 FPS. We also processed videos at 12.5 FPS, considering that it was enough both for the model and for the final score. Nonetheless, two days before the competition ended, we checked that performance of the best public model was degraded more than expected (-0.03) when reducing the inference FPS to 12.5. Did you check the performance of your architecture with 25 FPS?

\n

On the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?

\n

I liked the approach of using a 2nd stage classifier with the extracted features and more temporal information (although it seems from other solutions that not much temporal context was needed). Did you ensemble the predictions of both classifiers or just take the output of the 2nd? If you did combine them, how did you do it?

\n

Finally, what was approximately your inference time? Given all the steps that your solution requires, I am amazed that you manage to fit everything on time.

\n

Congrats again! Really liked the Dijkstra approach to smooth the ball trajectory!

", "votes": 2, "replies": [ { "id": 1993660, "author_name": "K_mat", "author_url": "", "post_date": "2022-10-18T14:50:36.927000", "content": "

Thanks @atmguille .

\n
\n

What tool did you use to generate them?

\n
\n

Sorry. PowerPoint is all you need.
\nI write my solution on a presentation template for winner call since it can be reused. I'm always believing my winning (:
\nhttps://www.kaggle.com/WinningModelDocumentationGuidelines

\n
\n

Did you check the performance of your architecture with 25 FPS?

\n
\n

No, I didn't. I designed my model to predict events at 12.5 fps, that couldn't be changed easily because my model is too complicated. To cope with the low frame rate (80ms/frame), I added the regression branch on the classifier to predict the time offset from the actual event occurrence time. But I think your point is correct. Using full 25 fps would lead more precise ball trajectory and time of the event. I should have look into that more.

\n
\n

On the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?

\n
\n

1D conv is applied on the normalized ball path and speed on image coordinates.
\nInput to 1D conv has the shape of [batch, frames, channels(= x,y location and delta_x, delta_y as speed)].

\n
\n

Did you ensemble the predictions of both classifiers or just take the output of the 2nd?

\n
\n

I used only the output of the second. The score of the second is approx. 0.02 better than first. Ensembling the 2nd with the 1st slightly deteriorated the score.

\n
\n

Really liked the Dijkstra approach to smooth the ball trajectory!

\n
\n

i’m glad to hear that! The Dijkstra approach is one of my confident ideas in this competition.

", "votes": 4, "replies": [] } ] }, { "id": 1988259, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2022-10-15T07:29:08.977000", "content": "

Wow, wonderful result @kmat2019! Hearty congratulations and thanks for the detailed explanation!

", "votes": 1, "replies": [] }, { "id": 2016059, "author_name": "CR_C1-10P", "author_url": "", "post_date": "2022-11-03T18:22:59.140000", "content": "

May I know How did you train CenterNet? Did you create annoannotation yourself? Could you share some reference code or blog for centernet that will be useful?

", "votes": 1, "replies": [] }, { "id": 1993336, "author_name": "Basel Ayman", "author_url": "", "post_date": "2022-10-18T09:49:32.100000", "content": "

Great Work!!

", "votes": 1, "replies": [] }, { "id": 1990016, "author_name": "Puru Behl 1", "author_url": "", "post_date": "2022-10-16T09:40:07.723000", "content": "

osm new thing

", "votes": 1, "replies": [] }, { "id": 1989511, "author_name": "assign", "author_url": "", "post_date": "2022-10-16T03:29:30.450000", "content": "

Thanks for sharing an interesting approach.

\n

1 Question about cost function

\n

information about the ball at t1 [x1, y1, c1]
\nFor the information about the ball at t2 [x2, y2, c2],
\nCan you provide a function to calculate the cost of moving to t1->t2?

\n

I'm asking because I think conf0 … conf9 could affect the costing.

\n

If the confidence does not affect the cost, the detector only constructs the smallest path out of 10 candidates to minimize the distance, and may not go through the frame with the greatest confidence in any frame.
\nStill, I think it's a reasonable enough approximation.

\n

2 How many manually labeled frames?

\n

Can you tell me how many frames you have labeled for ball detection?

", "votes": 1, "replies": [ { "id": 1990209, "author_name": "K_mat", "author_url": "", "post_date": "2022-10-16T11:59:32.237000", "content": "

1.Question about cost function

\n

Cost is computed as the combination of the confidence of the detector and gaussian probability of distance shown below.

\n
def cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):\n    x0_pred = x0 + inertia_x\n    y0_pred = y0 + inertia_y\n    dist = sqrt((x1 - x0_pred)**2 + (y1 - y0_pred)**2)\n    cost = - log(conf1 * gaussian_prob(dist, sigma=40))\n    return cost\n
\n

If the cost is too high, a new node can be added by paying some cost.

\n
if cost > > threshold:\n    add_new_node()\n    total_cost += new_node_cost\n
\n

2.How many manually labeled frames?

\n

1000 sets of 3 continuous frames for first training. From there, I repeated the training and predictions, finally it reached over 10000 sets. Since most of the predictions were accurate, the actual number of hand annotation is much fewer than 10000.
\n I scored over 0.80 with the first detector trained by 1000 sets. Increasing the hard samples improve the score by about 0.02 in my case.

", "votes": 2, "replies": [] } ] }, { "id": 1990909, "author_name": "Bice Weg", "author_url": "", "post_date": "2022-10-16T20:13:58.333000", "content": "

def cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):
\n x0_pred = x0 + inertia_x
\n y0_pred = y0 + inertia_y
\n dist = sqrt((x1 - x0_pred)2 + (y1 - y0_pred)2)
\n cost = - log(conf1 * gaussian_prob(dist, sigma=40))
\n return cost

", "votes": -1, "replies": [] }, { "id": 2023545, "author_name": "Alexander Ivkin", "author_url": "", "post_date": "2022-11-09T20:33:30.800000", "content": "

Great solution! Thank you for this detailed description.
\nCan I ask why you chose optical flow calculation instead of frame registration using the feature-matching approach?
\nCan you share more details about the detector and how you trained it? I didn't get why you annotated again after the detection - isn't it sufficient to just have the initial ground truth for multiple epochs training?
\nI'm working on a similar task for golf ball tracking. I'm currently experimenting with the T-Rex solution; however, yours look like the next step, or maybe it makes sense to combine the best parts. For example, they use a 3-channel grayscale image instead of 9-channel, and they use SSD as a detector with a custom backbone, which looks similar to yours.

\n

I'd be happy to chat about it, please send me a message if you're interested in a potential collaboration.

", "votes": 0, "replies": [] }, { "id": 1993068, "author_name": "Zhaoting Qiu", "author_url": "", "post_date": "2022-10-18T05:28:23.247000", "content": "

Thanks for sharing, this is an interesting approach

", "votes": 0, "replies": [] }, { "id": 1991160, "author_name": "Max Wei", "author_url": "", "post_date": "2022-10-17T02:49:15.593000", "content": "

Good job!

", "votes": 0, "replies": [] }, { "id": 1990659, "author_name": "Odudu Ochuko Richie", "author_url": "", "post_date": "2022-10-16T17:16:39.547000", "content": "

Wow, this is really nice. Congrats, keep it up

", "votes": 0, "replies": [] }, { "id": 1993194, "author_name": "", "author_url": "", "post_date": "2022-10-18T07:17:43.493000", "content": "", "votes": 0, "replies": [] }, { "id": 2080398, "author_name": "Om Ashish Mishra", "author_url": "", "post_date": "2022-12-30T06:31:36.210000", "content": "

Thank you for this wonderful details!

", "votes": 0, "replies": [] }, { "id": 2072992, "author_name": "Gaju Ahmed", "author_url": "", "post_date": "2022-12-22T15:17:42.657000", "content": "

Great, Thanks for the solution

", "votes": 0, "replies": [] } ], "comment_count": 16, "messages": [ { "id": 1987998, "postDate": "2022-10-15T01:48:47.450Z", "content": "

Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! Since this competition can be approached in a variety of ways, I'm looking forward to see everyone's solution.

\n

Overview of my solution

\n

My pipeline consists of:

\n\n

Once I found that the attention or cropping near the ball is effective to improve the accuracy of action classifier, I focused on refining the performance of the detector and classifier.

\n

\"pipeline\"

\n
\n

1. Optical Flow

\n

It's difficult to detect the ball when:

\n\n

So my idea is to use multiple frames to detect the ball on play which is usually moving by the optical flow.
\nAt first, I implemented the optical flow with high resolution image since balls are very small, but it's slow and not suitable for this competition. So I adopted the low resolution OpticalFlow to predict only the global (camera) motion. Flow itself cannot tell which one is the ball on play, but the difference between warped image and original image is very informative.
\n\"opticalflow_motivation\"
\nI used RAFT to predict optical flow. It is trained with the provided video by self-supervised manner.
\n\"opticalflow_implementation\"

\n
\n

2. Ball Detection

\n

To make the ball detector, I repeated training and annotation several times.
\n(Annotation -> Train Detector -> Predict -> Annotation -> Train Detector -> Predict -> …)
\nI selected CenterNet because I do not like annotation. Annotating point is much easier than rectangle.

\n

Since the target(ball) is very small. Inputs to the model is high resolution images.
\nTo achieve both inference speed and accuracy, I used the initial part of the encoder.
\n\"balldetector\"

\n

Inputs to the detector consist of 9 channels:

\n\n

The latter two drastically improved the accuracy of detector especially in the crowded scene.
\n\"detection_example\"

\n
\n

3. Cost Minimization of Ball Trajectory

\n

Ball location can be estimated more accurately by using multi-frame information. I applied cost minimization on selecting the ball path.
\nIf we know the trajectory of the ball, we can estimate where it will go next. In the following case, people can select the next point correctly even if the ball is hard to seen.
\n\"ball_path\"

\n

As I want to consider both detector's score and distance based probability, I defined
\n Cost = Function (Confidence of Detector, Distance between Nodes)
\nand minimized the total cost of the path by Dijkstra Algorithm .

\n

It can create a more natural and accurate trajectory than just connecting points with the highest scores.
\n\"ball_path_image\"
\n(If this competition was for the ball detection, I would get first place!)

\n

Videos can be seen here. Left one is the max confidence path and right one is the min cost path.
\nhttps://twitter.com/i/status/1582747059165495297

\n
\n

4. Event Classifier (Action Recognition)

\n

Now we know where the ball is. Next step is to predict event. I used ball trajectory and images cropped near ball over 1 sec to predict action(4 class) and regression of residuals.
\nSince the labeled intervals are very short, it's difficult to make the long sequential training data. I extracted only 1 sec frames randomly from labeled intervals as an input to the model.

\n

The model consists of :
\n1) 2D CNN to extract features from cropped images
\n2) 1D CNN to extract features from ball path
\n3) 1D CNN to predict event from upper two sets of features
\n\"event_detect_1st\"

\n
\n

5. Event Classifier [2nd Stage]

\n

The first classifier cannot predict the event with long-term information.
\nAt second stage, long sequential inputs(4 sec) can be adopted because the inputs of the model is only feature vectors which is much more memory-efficient than the images.
\n\"second_stage\"

\n
\n

6. PostProcessing

\n

Simply, combination of gaussian weight and peak detection.

\n\n
\n

Final Thought

\n

Since the number of samples of the events is not so large and the most of the area in a image is useless, I thought the model would hard to learn and would overfit easily. So I soon decided to use the ball trajectory.
\nThat's why I really amazed that many kagglers suceeded the simple end-to-end approach. I've learned a lot and still need to learn a lot from this competition. Thanks again to kaggle community!

", "rawMarkdown": "Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! Since this competition can be approached in a variety of ways, I'm looking forward to see everyone's solution.\n\n# Overview of my solution\nMy pipeline consists of:\n- **Optical Flow to get global(camera) motion.**\n- **Ball Detector to find a ball on the field.**\n- **Cost Minimization of Ball Trajectory to refine the ball locations over multiple frames.**\n- **2-stage Action Recognition to classify the events.**\n- **Ensemble and post-processing.**\n\nOnce I found that the attention or cropping near the ball is effective to improve the accuracy of action classifier, I focused on refining the performance of the detector and classifier. \n\n\"pipeline\"\n\n---\n\n# 1. Optical Flow\nIt's difficult to detect the ball when:\n- it is occluded behind players.\n- its background is the stand.\n- balls not on play are placed near sideline.\n\nSo my idea is to use multiple frames to detect the ball on play which is usually moving by the optical flow.\nAt first, I implemented the optical flow with high resolution image since balls are very small, but it's slow and not suitable for this competition. So I adopted the low resolution OpticalFlow to predict only the global (camera) motion. Flow itself cannot tell which one is the ball on play, but the difference between warped image and original image is very informative.\n\"opticalflow_motivation\"\nI used RAFT to predict optical flow. It is trained with the provided video by self-supervised manner.\n\"opticalflow_implementation\"\n\n---\n\n# 2. Ball Detection\nTo make the ball detector, I repeated training and annotation several times. \n(Annotation -> Train Detector -> Predict -> Annotation -> Train Detector -> Predict -> …)\nI selected CenterNet because I do not like annotation. Annotating point is much easier than rectangle. \n\nSince the target(ball) is very small. Inputs to the model is high resolution images.\nTo achieve both inference speed and accuracy, I used the initial part of the encoder. \n\"balldetector\"\n\nInputs to the detector consist of 9 channels:\n- normal RGB image of the current frame\n- difference between the current frame and previous frame reconstructed by the optical flow.\n- difference between the current frame and next frame reconstructed by the optical flow.\n\nThe latter two drastically improved the accuracy of detector especially in the crowded scene.\n\"detection_example\"\n\n---\n\n# 3. Cost Minimization of Ball Trajectory\nBall location can be estimated more accurately by using multi-frame information. I applied cost minimization on selecting the ball path. \nIf we know the trajectory of the ball, we can estimate where it will go next. In the following case, people can select the next point correctly even if the ball is hard to seen.\n\"ball_path\"\n\nAs I want to consider both detector's score and distance based probability, I defined \n Cost = Function (Confidence of Detector, Distance between Nodes)\nand minimized the total cost of the path by Dijkstra Algorithm .\n\nIt can create a more natural and accurate trajectory than just connecting points with the highest scores.\n\"ball_path_image\"\n(If this competition was for the ball detection, I would get first place!)\n\nVideos can be seen here. Left one is the max confidence path and right one is the min cost path.\nhttps://twitter.com/i/status/1582747059165495297\n\n---\n\n# 4. Event Classifier (Action Recognition)\nNow we know where the ball is. Next step is to predict event. I used ball trajectory and images cropped near ball over 1 sec to predict action(4 class) and regression of residuals.\nSince the labeled intervals are very short, it's difficult to make the long sequential training data. I extracted only 1 sec frames randomly from labeled intervals as an input to the model.\n\nThe model consists of :\n1) 2D CNN to extract features from cropped images\n2) 1D CNN to extract features from ball path\n3) 1D CNN to predict event from upper two sets of features \n\"event_detect_1st\"\n\n---\n\n# 5. Event Classifier [2nd Stage]\nThe first classifier cannot predict the event with long-term information.\nAt second stage, long sequential inputs(4 sec) can be adopted because the inputs of the model is only feature vectors which is much more memory-efficient than the images.\n\"second_stage\"\n\n---\n\n# 6. PostProcessing\nSimply, combination of gaussian weight and peak detection.\n- Peak detection by max pooling\n- Gaussian filter on the “challenge” score because the peak of “challenge” is ambiguous compared to the other two classes.\n\"post_process\"\n\n---\n\n# Final Thought\nSince the number of samples of the events is not so large and the most of the area in a image is useless, I thought the model would hard to learn and would overfit easily. So I soon decided to use the ball trajectory.\nThat's why I really amazed that many kagglers suceeded the simple end-to-end approach. I've learned a lot and still need to learn a lot from this competition. Thanks again to kaggle community!", "votes": 145 }, { "id": 1993471, "postDate": "2022-10-18T12:08:14.303Z", "content": "

Congrats on the great solution @kmat2019! The logic behind all the steps is great. Moreover, congratulations on the detailed post and the amazing visualizations. What tool did you use to generate them?

\n

First, I wanted to ask you about the reduction to 12.5 FPS. We also processed videos at 12.5 FPS, considering that it was enough both for the model and for the final score. Nonetheless, two days before the competition ended, we checked that performance of the best public model was degraded more than expected (-0.03) when reducing the inference FPS to 12.5. Did you check the performance of your architecture with 25 FPS?

\n

On the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?

\n

I liked the approach of using a 2nd stage classifier with the extracted features and more temporal information (although it seems from other solutions that not much temporal context was needed). Did you ensemble the predictions of both classifiers or just take the output of the 2nd? If you did combine them, how did you do it?

\n

Finally, what was approximately your inference time? Given all the steps that your solution requires, I am amazed that you manage to fit everything on time.

\n

Congrats again! Really liked the Dijkstra approach to smooth the ball trajectory!

", "rawMarkdown": "Congrats on the great solution @kmat2019! The logic behind all the steps is great. Moreover, congratulations on the detailed post and the amazing visualizations. What tool did you use to generate them?\n\nFirst, I wanted to ask you about the reduction to 12.5 FPS. We also processed videos at 12.5 FPS, considering that it was enough both for the model and for the final score. Nonetheless, two days before the competition ended, we checked that performance of the best public model was degraded more than expected (-0.03) when reducing the inference FPS to 12.5. Did you check the performance of your architecture with 25 FPS?\n\nOn the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?\n\nI liked the approach of using a 2nd stage classifier with the extracted features and more temporal information (although it seems from other solutions that not much temporal context was needed). Did you ensemble the predictions of both classifiers or just take the output of the 2nd? If you did combine them, how did you do it?\n\nFinally, what was approximately your inference time? Given all the steps that your solution requires, I am amazed that you manage to fit everything on time.\n\nCongrats again! Really liked the Dijkstra approach to smooth the ball trajectory!", "votes": 2, "replies": [ { "id": 1993660, "postDate": "2022-10-18T14:50:36.927Z", "content": "

Thanks @atmguille .

\n
\n

What tool did you use to generate them?

\n
\n

Sorry. PowerPoint is all you need.
\nI write my solution on a presentation template for winner call since it can be reused. I'm always believing my winning (:
\nhttps://www.kaggle.com/WinningModelDocumentationGuidelines

\n
\n

Did you check the performance of your architecture with 25 FPS?

\n
\n

No, I didn't. I designed my model to predict events at 12.5 fps, that couldn't be changed easily because my model is too complicated. To cope with the low frame rate (80ms/frame), I added the regression branch on the classifier to predict the time offset from the actual event occurrence time. But I think your point is correct. Using full 25 fps would lead more precise ball trajectory and time of the event. I should have look into that more.

\n
\n

On the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?

\n
\n

1D conv is applied on the normalized ball path and speed on image coordinates.
\nInput to 1D conv has the shape of [batch, frames, channels(= x,y location and delta_x, delta_y as speed)].

\n
\n

Did you ensemble the predictions of both classifiers or just take the output of the 2nd?

\n
\n

I used only the output of the second. The score of the second is approx. 0.02 better than first. Ensembling the 2nd with the 1st slightly deteriorated the score.

\n
\n

Really liked the Dijkstra approach to smooth the ball trajectory!

\n
\n

i’m glad to hear that! The Dijkstra approach is one of my confident ideas in this competition.

", "rawMarkdown": "Thanks @atmguille .\n\n> What tool did you use to generate them?\n\nSorry. PowerPoint is all you need.\nI write my solution on a presentation template for winner call since it can be reused. I'm always believing my winning (:\nhttps://www.kaggle.com/WinningModelDocumentationGuidelines\n\n> Did you check the performance of your architecture with 25 FPS?\n\nNo, I didn't. I designed my model to predict events at 12.5 fps, that couldn't be changed easily because my model is too complicated. To cope with the low frame rate (80ms/frame), I added the regression branch on the classifier to predict the time offset from the actual event occurrence time. But I think your point is correct. Using full 25 fps would lead more precise ball trajectory and time of the event. I should have look into that more.\n\n> On the other hand, I am not sure if I understand how you applied 1D convolutions to the ball path. Could you expand a little bit on this?\n\n1D conv is applied on the normalized ball path and speed on image coordinates. \nInput to 1D conv has the shape of [batch, frames, channels(= x,y location and delta_x, delta_y as speed)].\n\n> Did you ensemble the predictions of both classifiers or just take the output of the 2nd?\n\nI used only the output of the second. The score of the second is approx. 0.02 better than first. Ensembling the 2nd with the 1st slightly deteriorated the score.\n\n> Really liked the Dijkstra approach to smooth the ball trajectory!\n\ni’m glad to hear that! The Dijkstra approach is one of my confident ideas in this competition.", "votes": 4 } ] }, { "id": 1988259, "postDate": "2022-10-15T07:29:08.977Z", "content": "

Wow, wonderful result @kmat2019! Hearty congratulations and thanks for the detailed explanation!

", "rawMarkdown": "Wow, wonderful result @kmat2019! Hearty congratulations and thanks for the detailed explanation!", "votes": 1 }, { "id": 2016059, "postDate": "2022-11-03T18:22:59.140Z", "content": "

May I know How did you train CenterNet? Did you create annoannotation yourself? Could you share some reference code or blog for centernet that will be useful?

", "rawMarkdown": "May I know How did you train CenterNet? Did you create annoannotation yourself? Could you share some reference code or blog for centernet that will be useful?", "votes": 1 }, { "id": 1993336, "postDate": "2022-10-18T09:49:32.100Z", "content": "

Great Work!!

", "rawMarkdown": "Great Work!!", "votes": 1 }, { "id": 1990016, "postDate": "2022-10-16T09:40:07.723Z", "content": "

osm new thing

", "rawMarkdown": "osm new thing\n", "votes": 1 }, { "id": 1989511, "postDate": "2022-10-16T03:29:30.450Z", "content": "

Thanks for sharing an interesting approach.

\n

1 Question about cost function

\n

information about the ball at t1 [x1, y1, c1]
\nFor the information about the ball at t2 [x2, y2, c2],
\nCan you provide a function to calculate the cost of moving to t1->t2?

\n

I'm asking because I think conf0 … conf9 could affect the costing.

\n

If the confidence does not affect the cost, the detector only constructs the smallest path out of 10 candidates to minimize the distance, and may not go through the frame with the greatest confidence in any frame.
\nStill, I think it's a reasonable enough approximation.

\n

2 How many manually labeled frames?

\n

Can you tell me how many frames you have labeled for ball detection?

", "rawMarkdown": "Thanks for sharing an interesting approach.\n\n1 Question about cost function\n\ninformation about the ball at t1 [x1, y1, c1]\nFor the information about the ball at t2 [x2, y2, c2],\nCan you provide a function to calculate the cost of moving to t1->t2?\n\nI'm asking because I think conf0 ... conf9 could affect the costing.\n\nIf the confidence does not affect the cost, the detector only constructs the smallest path out of 10 candidates to minimize the distance, and may not go through the frame with the greatest confidence in any frame.\nStill, I think it's a reasonable enough approximation.\n\n2 How many manually labeled frames?\n\nCan you tell me how many frames you have labeled for ball detection?", "votes": 1, "replies": [ { "id": 1990209, "postDate": "2022-10-16T11:59:32.237Z", "content": "

1.Question about cost function

\n

Cost is computed as the combination of the confidence of the detector and gaussian probability of distance shown below.

\n
def cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):\n    x0_pred = x0 + inertia_x\n    y0_pred = y0 + inertia_y\n    dist = sqrt((x1 - x0_pred)**2 + (y1 - y0_pred)**2)\n    cost = - log(conf1 * gaussian_prob(dist, sigma=40))\n    return cost\n
\n

If the cost is too high, a new node can be added by paying some cost.

\n
if cost > > threshold:\n    add_new_node()\n    total_cost += new_node_cost\n
\n

2.How many manually labeled frames?

\n

1000 sets of 3 continuous frames for first training. From there, I repeated the training and predictions, finally it reached over 10000 sets. Since most of the predictions were accurate, the actual number of hand annotation is much fewer than 10000.
\n I scored over 0.80 with the first detector trained by 1000 sets. Increasing the hard samples improve the score by about 0.02 in my case.

", "rawMarkdown": "1.Question about cost function\n\nCost is computed as the combination of the confidence of the detector and gaussian probability of distance shown below.\n```\ndef cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):\n x0_pred = x0 + inertia_x\n y0_pred = y0 + inertia_y\n dist = sqrt((x1 - x0_pred)**2 + (y1 - y0_pred)**2)\n cost = - log(conf1 * gaussian_prob(dist, sigma=40))\n return cost\n```\nIf the cost is too high, a new node can be added by paying some cost.\n```\nif cost > > threshold:\n add_new_node()\n total_cost += new_node_cost\n```\n\n2.How many manually labeled frames?\n\n1000 sets of 3 continuous frames for first training. From there, I repeated the training and predictions, finally it reached over 10000 sets. Since most of the predictions were accurate, the actual number of hand annotation is much fewer than 10000.\n I scored over 0.80 with the first detector trained by 1000 sets. Increasing the hard samples improve the score by about 0.02 in my case.", "votes": 2 } ] }, { "id": 1990909, "postDate": "2022-10-16T20:13:58.333Z", "content": "

def cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):
\n x0_pred = x0 + inertia_x
\n y0_pred = y0 + inertia_y
\n dist = sqrt((x1 - x0_pred)2 + (y1 - y0_pred)2)
\n cost = - log(conf1 * gaussian_prob(dist, sigma=40))
\n return cost

", "rawMarkdown": "def cost_btw_nodes(x0, y0, x1, y1, conf1, inertia_x, inertia_y):\n x0_pred = x0 + inertia_x\n y0_pred = y0 + inertia_y\n dist = sqrt((x1 - x0_pred)**2 + (y1 - y0_pred)**2)\n cost = - log(conf1 * gaussian_prob(dist, sigma=40))\n return cost", "votes": -1 }, { "id": 2023545, "postDate": "2022-11-09T20:33:30.800Z", "content": "

Great solution! Thank you for this detailed description.
\nCan I ask why you chose optical flow calculation instead of frame registration using the feature-matching approach?
\nCan you share more details about the detector and how you trained it? I didn't get why you annotated again after the detection - isn't it sufficient to just have the initial ground truth for multiple epochs training?
\nI'm working on a similar task for golf ball tracking. I'm currently experimenting with the T-Rex solution; however, yours look like the next step, or maybe it makes sense to combine the best parts. For example, they use a 3-channel grayscale image instead of 9-channel, and they use SSD as a detector with a custom backbone, which looks similar to yours.

\n

I'd be happy to chat about it, please send me a message if you're interested in a potential collaboration.

", "rawMarkdown": "Great solution! Thank you for this detailed description.\nCan I ask why you chose optical flow calculation instead of frame registration using the feature-matching approach?\nCan you share more details about the detector and how you trained it? I didn't get why you annotated again after the detection - isn't it sufficient to just have the initial ground truth for multiple epochs training?\nI'm working on a similar task for golf ball tracking. I'm currently experimenting with the T-Rex solution; however, yours look like the next step, or maybe it makes sense to combine the best parts. For example, they use a 3-channel grayscale image instead of 9-channel, and they use SSD as a detector with a custom backbone, which looks similar to yours.\n\nI'd be happy to chat about it, please send me a message if you're interested in a potential collaboration." }, { "id": 1993068, "postDate": "2022-10-18T05:28:23.247Z", "content": "

Thanks for sharing, this is an interesting approach

", "rawMarkdown": "Thanks for sharing, this is an interesting approach" }, { "id": 1991160, "postDate": "2022-10-17T02:49:15.593Z", "content": "

Good job!

", "rawMarkdown": "Good job!\n" }, { "id": 1990659, "postDate": "2022-10-16T17:16:39.547Z", "content": "

Wow, this is really nice. Congrats, keep it up

", "rawMarkdown": "Wow, this is really nice. Congrats, keep it up" }, { "id": 1993194, "postDate": "2022-10-18T07:17:43.493Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2080398, "postDate": "2022-12-30T06:31:36.210Z", "content": "

Thank you for this wonderful details!

", "rawMarkdown": "Thank you for this wonderful details!" }, { "id": 2072992, "postDate": "2022-12-22T15:17:42.657Z", "content": "

Great, Thanks for the solution

", "rawMarkdown": "Great, Thanks for the solution" } ], "complete": true, "source": "live", "fetched_at": "2026-10-22T08:54:16Z", "competition_id": 37244, "competition_slug": "dfl-bundesliga-data-shootout", "team_id": 8957358, "solution_url": "/competitions/dfl-bundesliga-data-shootout/writeups/k-mat-2nd-place-solution" }