{
  "fetched_at": "2026-10-19T06:36:34Z",
  "source_url": "https://www.kaggle.com/competitions/cvpr-metafood-3d-food-reconstruction-challenge/data",
  "pages": [
    {
      "name": "data-description",
      "content": "Challenge update (05/25/2024):\nThank you to everyone who participated in the Physically Informed 3D Food Reconstruction competition. We are now entering Phase 2 of the competition, where we will select the top 5 teams to submit their 3D object models. These models will be evaluated based on chamfer distance. The final score will be calculated by averaging the rankings from Phase 1 and Phase 2. In the event of a tie, the Phase 1 score will determine the final ranking.\n\nFor the Phase 2 evaluation (for the top 5 teams), we were unable to find a fair and automatic way to align the pose between the submissions and the ground truth objects. Therefore, we are updating the evaluation procedure, and the winner announcement will be delayed to June 7th. The revised steps are as follows:\n\n1. Upload Models: Upload all 20 of your 3D models (.obj files named with the object ID) to a location where we can download them.\n2. Send Link: Send us the link to your models along with your team ID (or other ways to identify yourself) by May 27th, 11:59 PM AOE. Email: metafood.cvpr@gmail.com. If you encounter any issues, please post on the Kaggle discussion forum.\n3. Model Verification: We will check your models against your final Phase 1 submission to ensure they are the same models used in Phase 1. We will also visually inspect the models to ensure no team is violating the rules by submitting primitive objects such as spheres.\n4. Ground Truth Models: We will upload our ground truth 3D models, along with the script used to compute the final chamfer distance, on Kaggle by May 28th, 11:59 PM AOE.\n5. Align Models: Align the models yourself and prepare a transform matrix for each object you submitted. Your final chamfer distance score will be computed using your submitted models and transform matrices.\n6. Submit Transform Matrices: Send us your transform matrices by June 4th, 11:59 PM AOE.\nWe will announce the winner on June 7th.\n\nThank you for your understanding and cooperation.\n\nDATASET UPDATE (04/23/2024): We added a script to extract depth information from the images. You can find the script in convert_depth folder. \nDATASET UPDATE (04/18/2024): Please use the data in the \"MTF_Challenge-20240418T205428Z-001\" directory. We fixed an issue where the masks for object 12 are incorrect. Note that you can still access the old data in the directory \"MTF_Challenge\".\n\nThe challenge features 20 selected food items, each meticulously scanned with a 3D scanner while corresponding videos were captured. These items are accompanied by a checkerboard and pattern mat during video capture, serving as physical references for scaling the output 3D models to ensure size accuracy—one of the key evaluation metrics. The difficulty level of the challenge varies with the number of 2D images available for reconstruction: simple levels are represented by approximately 200 images sampled from video, medium levels by about 30 images, and hard levels by a single monocular top-view image. Detailed information on these food items is provided below:\n\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n    <meta charset=\"UTF-8\">\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n    <title>Data stats</title>\n    <style>\n        table {\n            border: 2px solid black;\n            border-collapse: collapse;\n            width: 100%;\n        }\n        th, td {\n            border: 1px solid black;\n            padding: 8px;\n            text-align: center;\n        }\n    </style>\n</head>\n<body>\n\n<table>\n    <thead>\n        <tr>\n            <th>Object Index</th>\n            <th>Food Item</th>\n            <th>Difficulty Level</th>\n            <th>Number of Frames</th>\n        </tr>\n    </thead>\n    <tbody>\n        <tr>\n            <td>1</td>\n            <td>Strawberry</td>\n            <td>simple</td>\n            <td>199</td>\n        </tr>\n        <tr>\n            <td>2</td>\n            <td>Cinnamon bun</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>3</td>\n            <td>Pork rib</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>4</td>\n            <td>Corn</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>5</td>\n            <td>French toast</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>6</td>\n            <td>Sandwich</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>7</td>\n            <td>Burger</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>8</td>\n            <td>Cake</td>\n            <td>simple</td>\n            <td>200</td>\n        </tr>\n        <tr>\n            <td>9</td>\n            <td>Blueberry muffin</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>10</td>\n            <td>Banana</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>11</td>\n            <td>Salmon</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>12</td>\n            <td>Steak</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>13</td>\n            <td>Burrito</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>14</td>\n            <td>Hotdog</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>15</td>\n            <td>Chicken nugget</td>\n            <td>medium</td>\n            <td>30</td>\n        </tr>\n        <tr>\n            <td>16</td>\n            <td>Everything bagel</td>\n            <td>hard</td>\n            <td>1</td>\n        </tr>\n        <tr>\n            <td>17</td>\n            <td>Croissant</td>\n            <td>hard</td>\n            <td>1</td>\n        </tr>\n        <tr>\n            <td>18</td>\n            <td>Shrimp</td>\n            <td>hard</td>\n            <td>1</td>\n        </tr>\n        <tr>\n            <td>19</td>\n            <td>Waffle</td>\n            <td>hard</td>\n            <td>1</td>\n        </tr>\n        <tr>\n            <td>20</td>\n            <td>Pizza</td>\n            <td>hard</td>\n            <td>1</td>\n        </tr>\n    </tbody>\n</table>\n\n</body>\n</html>\n\nSample challenge data for “everything bagel”.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F19611415%2Fb9fa4bb29d5cf4b50898ae5e968b2c8c%2FPicture1.png?generation=1711251426407329&alt=media)\n\n\n<html lang=\"en\">\n</head>\n<body>\n    <h1>MTF_Challenge Dataset Structure</h1>\n    <ul>\n        <li><strong>object index (1 - 20)</strong>\n            <ul>\n                <li><strong>Depth:</strong> depth image for each frame</li>\n                <li><strong>Mask:</strong> segmentation mask for each frame</li>\n                <li><strong>Original:</strong> sampled original frames</li>\n            </ul>\n        </li>\n        <li><strong>source-files</strong>\n            <ul>\n                <li><strong>checkerboard.pdf</strong></li>\n                <li><strong>mat.pdf</strong></li>\n                <li><strong>data_meta.xls</strong></li>\n                <li><strong>eval</strong>\n                <ul>\n                    <li><strong>challenge_volume_calculation.py</strong> (evaluation script)</li>\n                    <li><strong>environment.yml</strong> (dependent libraries)</li>\n                    <li><strong>volume_calculation_instructions.docx</strong> (evaluation instruction)</li>\n                    <li><strong>sample_solution.csv</strong> (example submission file)</li>\n                </ul>\n                </li>\n</body>\n</html>\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "Today, computer vision algorithms show near-perfect performance, better than human when there are clear, well curated and enough amount of data. However, there remains a substantial gap when it comes to applying state-of-the-art computer vision algorithms to food data, particularly when dealing with food in its natural, uncontrolled environment, often referred to as “data in the wild.” This gap stems from the inherent challenges in noisy, watermarked, and low-quality food data readily available on the internet. The MetaFood Workshop (MTF) invites the CVPR community to engage with the food domain-related challenges. These challenges provide not only a demanding, real testing environment for the development of robust computer vision algorithms, but also an exciting opportunity to develop new algorithms in the fields of food data analysis and food digitization.\n\nThe aim of this challenge is to highlight the importance of accurate 3D food reconstruction through a monocular camera, especially from a single image. This precise 3D modeling serves crucial purposes in food computing, such as keeping dietary records, necessitating an accurate representation of food portion sizes.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation Criteria",
      "content": "Challenge update (05/25/2024):\nThank you to everyone who participated in the Physically Informed 3D Food Reconstruction competition. We are now entering Phase 2 of the competition, where we will select the top 5 teams to submit their 3D object models. These models will be evaluated based on chamfer distance. The final score will be calculated by averaging the rankings from Phase 1 and Phase 2. In the event of a tie, the Phase 1 score will determine the final ranking.\n\nFor the Phase 2 evaluation (for the top 5 teams), we were unable to find a fair and automatic way to align the pose between the submissions and the ground truth objects. Therefore, we are updating the evaluation procedure, and the winner announcement will be delayed to June 7th. The revised steps are as follows:\n\n1. Upload Models: Upload all 20 of your 3D models (.obj files named with the object ID) to a location where we can download them.\n2. Send Link: Send us the link to your models along with your team ID (or other ways to identify yourself) by May 27th, 11:59 PM AOE. Email: metafood.cvpr@gmail.com. If you encounter any issues, please post on the Kaggle discussion forum.\n3. Model Verification: We will check your models against your final Phase 1 submission to ensure they are the same models used in Phase 1. We will also visually inspect the models to ensure no team is violating the rules by submitting primitive objects such as spheres.\n4. Ground Truth Models: We will upload our ground truth 3D models, along with the script used to compute the final chamfer distance, on Kaggle by May 28th, 11:59 PM AOE.\n5. Align Models: Align the models yourself and prepare a transform matrix for each object you submitted. Your final chamfer distance score will be computed using your submitted models and transform matrices.\n6. Submit Transform Matrices: Send us your transform matrices by June 4th, 11:59 PM AOE.\nWe will announce the winner on June 7th.\n\nThank you for your understanding and cooperation.\n\n==============================================\nEvaluation will consist of a two-phase process centered on the precision of the reconstructed 3D models in terms of their shape (3D structure) and portion size (volume). Results will be compared to the models of the same food items that are captured by a 3D scanner. \nIn Phase-I, the Mean Absolute Percentage Error (MAPE) metric will be employed to assess the accuracy of the model's portion size. Only the volume accuracy of the models will be evaluated during this stage.\nIn Phase-II, the top-ranking teams from the phase-I will be asked to submit the complete 3D mesh files for each food item to validate their submission results of Phase-I. Then, the Chamfer distance metric will be used to evaluate the shape accuracy of these models.\nThe final winner will be determined by combining the scores from both phases for all 20 objects.  \n\n\n## Submission File\nFor each id in the test set, you must reconstruct a 3D model from the provided images and report the volume of the model. The file should contain a header and have the following format:\n\n    id,predicted\n    2,0\n    5,0\n    6,0\n    etc.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "Participants are required to submit 20 .obj mesh files corresponding to the 20 selected food items. These submissions will undergo evaluation based on the reconstruction's shape and size accuracy. It is essential for participants to review the following rules:\n\n•\tSubmissions must adhere to the provided instructions. Submissions will be disqualified if they contain inappropriate content or fail to meet the submission rules.\n\n•\tEach registered participant/group can only submit the result once per day.\n\n•\tThe contest is open to everyone. However, those directly involved in organizing and supporting the contest cannot enter.\n\n•\tParticipants are allowed to use publicly available data and models, but they must properly attribute the source by including references in their submitted code.\n\n•\tDuring the final phase, the organizers will closely examine the code and 3D models submitted by the top-performing teams. Disqualification will occur for teams that do not present substantial original work, use similar code to others, or are caught cheating. Additionally, teams will be disqualified if their submitted 3D models do not accurately represent the object class’s shape. For instance, a team will face disqualification for submitting a sphere model for an object classified as a strawberry, whereas a submission featuring a model resembling a hemisphere with cone for the strawberry class will remain eligible.\n\n•\tContestants are responsible for ensuring their submissions, including all files, data, and code, are legal and comply with regulations. The organizers are not liable for any legal issues that arise from the participants' submissions.\n\n•\tIf contestants publish papers related to their competition results, they must credit the competition.\n\n•\tContestants may use data and models available to the public, but must give credit by citing the sources in their code submissions.\n\n•\tThe challenge data is not public yet, is not allowed to use for other purposes. \n\nWinner and Prize: The top-3 teams will be selected as winner. The winner will be invited for oral presentation at workshop and also write the technical report about the challenge. Refer to the workshop website for prize details: https://sites.google.com/view/cvpr-metafood-2024 \n\nExternal sources: \n•\tParticipants are allowed to utilize external 3D object datasets for training purposes, such as OmniObject3D[1], NutritionVerse-3D[2], etc. The use of pre-trained models or any form of external data must be clearly disclosed in the submission.\n•\tBaseline 3D reconstruction methods include: \n        o\tMedium/Simple: Use COLMAP for automatic reconstruction with the checkerboard available and apply Meshlab for cleaning and simplification to create a sparse 3D model, save it as an .obj file for the challenge submission.\n        o\tHard: Apply existing methods such as One-2-3-4-5[3] to create multiple views from a single image, use them for 3D reconstruction, calibrate scale using a checkerboard pattern, and adjust the 3D model to real-world dimensions for accurate scene representation.\n\n\n[1] Wu, Tong, et al. \"Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation.\" CVPR 2023.\n[2] Tai, Chi-en Amy, et al. \"Nutritionverse-3d: A 3d food model dataset for nutritional intake estimation.\" arXiv preprint arXiv:2304.05619, 2023\n[3] Liu, Minghua, et al. \"One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization.\" Neurips, 2023.\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "abstract",
      "content": "3D Food Reconstruction is an innovative venture into the intersection of computer vision and culinary arts, with the goal of reconstructing three-dimensional models of food items from two-dimensional images. This challenge is designed to push the boundaries of 3D reconstruction technology by applying it to a dynamic and uncontrolled setting, such as capturing food images during eating occasions. Participants will tackle the challenge of creating precise and robust 3D models from single or multiple 2D views, using a visible physical reference (checkerboard) with the known size. The checkerboard serves as a reference object to ensure that the reconstructed 3D models have the same size as the real food items, which is crucial for accurate portion estimation. The objective is to reconstruct 3D models for 20 selected food items, provided at varying levels of difficulty: simple, medium, and hard. By reconstructing 3D models with the correct size, this technology can play a vital role in tracking nutritional intake and helping individuals maintain a healthy diet. This endeavor not only aims to enhance sharing food experiences in three dimensions but also has significant potential applications in the fields of nutrition and health monitoring. ",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}