Good morning!
\n\nI am experimenting with Facebook's DETR and I am getting pretty good visual results. However, every submission I try I get very, very, very bad scores (always under 0.0051). I understand that the distribution of the whole test set may be different from the trainining set and even from the 10 test images visible. But I would not expect this low score, when the results (as shown in the kernel for the 10 visible test images) seem pretty good.
\n\nMaybe I did some mistake while generating the submission file, but maybe my perspective bias is not allowing me to spot it. Any thoughts? Help would be very appreciated.
\n\nhttps://www.kaggle.com/fmobrj1975/detr50-round19-800-50pct
", "messages": [ { "id": "941767", "postDate": "07/23/2020 12:03:57", "content": "Good morning!
\n\nI am experimenting with Facebook's DETR and I am getting pretty good visual results. However, every submission I try I get very, very, very bad scores (always under 0.0051). I understand that the distribution of the whole test set may be different from the trainining set and even from the 10 test images visible. But I would not expect this low score, when the results (as shown in the kernel for the 10 visible test images) seem pretty good.
\n\nMaybe I did some mistake while generating the submission file, but maybe my perspective bias is not allowing me to spot it. Any thoughts? Help would be very appreciated.
\n\nhttps://www.kaggle.com/fmobrj1975/detr50-round19-800-50pct
", "rawMarkdown": "Good morning!\n\nI am experimenting with Facebook's DETR and I am getting pretty good visual results. However, every submission I try I get very, very, very bad scores (always under 0.0051). I understand that the distribution of the whole test set may be different from the trainining set and even from the 10 test images visible. But I would not expect this low score, when the results (as shown in the kernel for the 10 visible test images) seem pretty good.\n\nMaybe I did some mistake while generating the submission file, but maybe my perspective bias is not allowing me to spot it. Any thoughts? Help would be very appreciated.\n \nhttps://www.kaggle.com/fmobrj1975/detr50-round19-800-50pct", "votes": null }, { "id": "941904", "postDate": "07/23/2020 13:39:25", "content": "I suggest to verify box format, the submission should be in x-y-w-h format.\nPorbably you have x1-y1-x2-y2 or something else.\nAlso check box scaling if you are not using full resolution.
", "rawMarkdown": "I suggest to verify box format, the submission should be in x-y-w-h format.\nPorbably you have x1-y1-x2-y2 or something else.\nAlso check box scaling if you are not using full resolution.", "votes": null }, { "id": "941968", "postDate": "07/23/2020 14:22:10", "content": "Thanks, Sergey. DETR already uses x, y, w, h as input. I just scale the results to the original image w, h and make the submission record. After that, for the 10 first images I just convert to x0,y0,x1,y1 for plotting the results and check if the scaling to the original image dimensions worked fine. There is some other catch I am missing.
", "rawMarkdown": "Thanks, Sergey. DETR already uses x, y, w, h as input. I just scale the results to the original image w, h and make the submission record. After that, for the 10 first images I just convert to x0,y0,x1,y1 for plotting the results and check if the scaling to the original image dimensions worked fine. There is some other catch I am missing.", "votes": null }, { "id": "942011", "postDate": "07/23/2020 14:43:23", "content": "Anyway, try to verify by drawing boxes from your submission.csv, not from the code.
", "rawMarkdown": "Anyway, try to verify by drawing boxes from your submission.csv, not from the code.", "votes": null }, { "id": "942021", "postDate": "07/23/2020 14:47:20", "content": "It seems you have boxes in x_center, y_center, w, h format after prediction (looking at your code for pictures).
Great idea! Thanks.
", "rawMarkdown": "Great idea! Thanks.", "votes": null }, { "id": "942033", "postDate": "07/23/2020 14:52:02", "content": "That's it. I assumed the competition data was dealing with data in coco format, once it used width, height intead of xmax, ymax. But it is slightly different, with xmin, ymin, width, height. Thank you very much! I will try it and post the results here. I am impressed with the simplicity of DETR. I dont think it will match Yolo results, but it is competitive.
\n\nThank you, very much!
", "rawMarkdown": "That's it. I assumed the competition data was dealing with data in coco format, once it used width, height intead of xmax, ymax. But it is slightly different, with xmin, ymin, width, height. Thank you very much! I will try it and post the results here. I am impressed with the simplicity of DETR. I dont think it will match Yolo results, but it is competitive.\n\nThank you, very much!", "votes": null }, { "id": "942069", "postDate": "07/23/2020 15:07:15", "content": "x_center, y_center, w, h is yolo format\nx_min, y_min, w, h is coco format (should be in submission)\nx_min, y_min, x_max, y_max is pascal format
Yeah, many formats :)
", "rawMarkdown": "` x_center, y_center, w, h` is yolo format\n` x_min, y_min, w, h` is coco format (should be in submission)\n` x_min, y_min, x_max, y_max` is pascal format\n\nYeah, many formats :)", "votes": null }, { "id": "942493", "postDate": "07/23/2020 19:15:45", "content": "@fmobrj1975 Hi there, could you share you training script? I've tried to build DETR pipeline weeks ago but failed to get a success. Maybe I was tortured by the box format as well but I didn't realize it... Thanks!
", "rawMarkdown": "fmobrj1975 Hi there, could you share you training script? I've tried to build DETR pipeline weeks ago but failed to get a success. Maybe I was tortured by the box format as well but I didn't realize it... Thanks!", "votes": null }, { "id": "942508", "postDate": "07/23/2020 19:32:50", "content": "No problem. I ran outside of kaggle, in our jupyter server. I am busy with a work deliverable here now. When I finish the work I will post in github and put the link here.
", "rawMarkdown": "No problem. I ran outside of kaggle, in our jupyter server. I am busy with a work deliverable here now. When I finish the work I will post in github and put the link here.", "votes": null }, { "id": "943139", "postDate": "07/24/2020 06:59:08", "content": "looking forward to receiving this link from you)
", "rawMarkdown": "looking forward to receiving this link from you)", "votes": null }, { "id": "943646", "postDate": "07/24/2020 13:42:35", "content": "Hi @fmobrj1975 Thank you for your kindness! Looking forward to it.
", "rawMarkdown": "Hi @fmobrj1975 Thank you for your kindness! Looking forward to it.", "votes": null }, { "id": "954139", "postDate": "08/01/2020 12:22:33", "content": "Hi @fmobrj1975 Any updates about the training script? I finally submit a model and got a pretty low score...even with TTA. I guess I made something wrong while submitting..
", "rawMarkdown": "Hi @fmobrj1975 Any updates about the training script? I finally submit a model and got a pretty low score...even with TTA. I guess I made something wrong while submitting..", "votes": null } ], "comments": [ { "id": 941904, "author_name": "sergeyzlobin", "author_url": "", "post_date": "07/23/2020 13:39:25", "content": "I suggest to verify box format, the submission should be in x-y-w-h format.\nPorbably you have x1-y1-x2-y2 or something else.\nAlso check box scaling if you are not using full resolution.
", "votes": null, "replies": [ { "id": 941968, "author_name": "fmobrj1975", "author_url": "", "post_date": "07/23/2020 14:22:10", "content": "Thanks, Sergey. DETR already uses x, y, w, h as input. I just scale the results to the original image w, h and make the submission record. After that, for the 10 first images I just convert to x0,y0,x1,y1 for plotting the results and check if the scaling to the original image dimensions worked fine. There is some other catch I am missing.
", "votes": null, "replies": [] }, { "id": 942011, "author_name": "sergeyzlobin", "author_url": "", "post_date": "07/23/2020 14:43:23", "content": "Anyway, try to verify by drawing boxes from your submission.csv, not from the code.
", "votes": null, "replies": [] }, { "id": 942021, "author_name": "sergeyzlobin", "author_url": "", "post_date": "07/23/2020 14:47:20", "content": "It seems you have boxes in x_center, y_center, w, h format after prediction (looking at your code for pictures).
Great idea! Thanks.
", "votes": null, "replies": [] }, { "id": 942033, "author_name": "fmobrj1975", "author_url": "", "post_date": "07/23/2020 14:52:02", "content": "That's it. I assumed the competition data was dealing with data in coco format, once it used width, height intead of xmax, ymax. But it is slightly different, with xmin, ymin, width, height. Thank you very much! I will try it and post the results here. I am impressed with the simplicity of DETR. I dont think it will match Yolo results, but it is competitive.
\n\nThank you, very much!
", "votes": null, "replies": [] }, { "id": 942069, "author_name": "sergeyzlobin", "author_url": "", "post_date": "07/23/2020 15:07:15", "content": "x_center, y_center, w, h is yolo format\nx_min, y_min, w, h is coco format (should be in submission)\nx_min, y_min, x_max, y_max is pascal format
Yeah, many formats :)
", "votes": null, "replies": [] } ] }, { "id": 942493, "author_name": "", "author_url": "", "post_date": "07/23/2020 19:15:45", "content": "@fmobrj1975 Hi there, could you share you training script? I've tried to build DETR pipeline weeks ago but failed to get a success. Maybe I was tortured by the box format as well but I didn't realize it... Thanks!
", "votes": null, "replies": [ { "id": 942508, "author_name": "fmobrj1975", "author_url": "", "post_date": "07/23/2020 19:32:50", "content": "No problem. I ran outside of kaggle, in our jupyter server. I am busy with a work deliverable here now. When I finish the work I will post in github and put the link here.
", "votes": null, "replies": [ { "id": 943646, "author_name": "", "author_url": "", "post_date": "07/24/2020 13:42:35", "content": "Hi @fmobrj1975 Thank you for your kindness! Looking forward to it.
", "votes": null, "replies": [] }, { "id": 954139, "author_name": "", "author_url": "", "post_date": "08/01/2020 12:22:33", "content": "Hi @fmobrj1975 Any updates about the training script? I finally submit a model and got a pretty low score...even with TTA. I guess I made something wrong while submitting..
", "votes": null, "replies": [] } ] }, { "id": 943139, "author_name": "kupchanski", "author_url": "", "post_date": "07/24/2020 06:59:08", "content": "looking forward to receiving this link from you)
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "941767": "Good morning!\n\nI am experimenting with Facebook's DETR and I am getting pretty good visual results. However, every submission I try I get very, very, very bad scores (always under 0.0051). I understand that the distribution of the whole test set may be different from the trainining set and even from the 10 test images visible. But I would not expect this low score, when the results (as shown in the kernel for the 10 visible test images) seem pretty good.\n\nMaybe I did some mistake while generating the submission file, but maybe my perspective bias is not allowing me to spot it. Any thoughts? Help would be very appreciated.\n \nhttps://www.kaggle.com/fmobrj1975/detr50-round19-800-50pct", "941904": "I suggest to verify box format, the submission should be in x-y-w-h format.\nPorbably you have x1-y1-x2-y2 or something else.\nAlso check box scaling if you are not using full resolution.", "941968": "Thanks, Sergey. DETR already uses x, y, w, h as input. I just scale the results to the original image w, h and make the submission record. After that, for the 10 first images I just convert to x0,y0,x1,y1 for plotting the results and check if the scaling to the original image dimensions worked fine. There is some other catch I am missing.", "942011": "Anyway, try to verify by drawing boxes from your submission.csv, not from the code.", "942021": "It seems you have boxes in `x_center, y_center, w, h` format after prediction (looking at your code for pictures).", "942023": "Great idea! Thanks.", "942033": "That's it. I assumed the competition data was dealing with data in coco format, once it used width, height intead of xmax, ymax. But it is slightly different, with xmin, ymin, width, height. Thank you very much! I will try it and post the results here. I am impressed with the simplicity of DETR. I dont think it will match Yolo results, but it is competitive.\n\nThank you, very much!", "942069": "` x_center, y_center, w, h` is yolo format\n` x_min, y_min, w, h` is coco format (should be in submission)\n` x_min, y_min, x_max, y_max` is pascal format\n\nYeah, many formats :)", "942493": "fmobrj1975 Hi there, could you share you training script? I've tried to build DETR pipeline weeks ago but failed to get a success. Maybe I was tortured by the box format as well but I didn't realize it... Thanks!", "942508": "No problem. I ran outside of kaggle, in our jupyter server. I am busy with a work deliverable here now. When I finish the work I will post in github and put the link here.", "943139": "looking forward to receiving this link from you)", "943646": "Hi @fmobrj1975 Thank you for your kindness! Looking forward to it.", "954139": "Hi @fmobrj1975 Any updates about the training script? I finally submit a model and got a pretty low score...even with TTA. I guess I made something wrong while submitting.." }, "source": "meta" }