{
  "fetched_at": "2026-10-19T07:19:54Z",
  "source_url": "https://www.kaggle.com/competitions/budgeted-model-inference-iccv-2023-rcv-workshop/data",
  "pages": [
    {
      "name": "data-description",
      "content": "**UltraMNIST Dataset**\nUltraMNIST dataset comprises very large-scale images, 3-5 digits per image. Each of these digits has been extracted from the original MNIST dataset. The label is the sum of the digits per image, and this number can be anything from 0 to 27.\n\n## Files\n\n*   **train.csv** - a csv file containing training image names and their corresponding labels.\n*   **sample_submission.csv** - a sample submission file in the correct format\n\n## Folders\n* **train** - contains training images\n* **test** - contains test images",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "## Motivation\n\nDespite the success of CNNs and, more recently, transformers, the scale of deep learning models has grown exponentially due to the rapid development of computational resources. Going forward, it is critical to focus on the practical training and inference efficiency of these models, as well as ensuring that even the largest models have viable real-world applications. Moreover, compute efficiency of computer vision models is becoming increasingly crucial for downstream tasks such as segmentation, object tracking, and action recognition, among others.\n\nThere exist several different directions towards making deep learning efficient for computer vision, thereby leading to a reduction in the required computational memory or the associated training and inference time. This competition will aim at bringing research efforts into practice for one such direction: *budget-aware model inference*. Most research that exists focuses on making deep learning methods efficient, however, the resource associated with real-world AI devices can vary drastically, and a method termed efficient for a certain choice of resource budgets might be completely inefficient for a different one. This competition will also focus on methods that aim at performing budget-aware model inference in best possible ways under limited GPU memory memory as well as limited inference time.\n\n**This challenge is part of the ICCV 2023 Resource-Efficient Deep Learning (RCV) Workshop to be held between Oct 2-5, 2023 in Paris, France.**\n\n## Problem description\nAs part of this budgeted inference challenge, we pose the problem of the classification of UltraMNIST digits. The problem at test time needs to be solved using limited GPU compute memory as well as time. The UltraMNIST dataset used here is an adapted variant of [UltraMNIST dataset](https://www.kaggle.com/competitions/ultra-mnist/data) comprises images of size  with 3-5 digits per image. Each of these digits has been extracted from the original MNIST dataset. Your task is to predict the sum of the digits per image, and this number can be anything from 0 to 27. However there are certain rules to be respected to qualify as a winning solution.\n\n1. For final evaluation, each submitted inference script along with trained models weights will be used to evaluate on a separate private test set than those revealed here. The inference speed and accuracy score on this corresponding test set will be used to decide the final score of the model.\n2. All the models will be inferenced on a V100 GPU with a memory constraint of 16 GB.\n3. Together with your code (only python scripts + one model weights file), you will be required to submit the run environment as a .yml that will be used to run the code. (we provide a basic .yml file environment in the code section; use this a starting point and only add packages if you need to.)\n4. The main script should be named main.py and should take only one command line argument --data_dir to produce submission.csv in the current directory.\n\nThis competition will be running in two phases: **Phase I** and **Phase II**.\n\n**Phase I:** This phase is a standard classification problem where the goal is to maximize the accuracy on the test set and improve the ranking on the leaderboard. This phase of the competition will be hosted here on Kaggle and the leaderboard of Kaggle will be used to rank the solutions. Note that this phase will only use accuracy as the evaluation measure.  Although this phase does not take the GPU memory and inference speed into consideration, participants are recommended to adhere to an approximate estimate and use the leaderboard to assess the performance of their efficient models that could pass on to the next Phase. Using very large networks that has slow inference speed, is highly discouraged since this will improve your ranking in Phase I, but will not be able to qualify you for Phase II.\n\n**Phase II:** We will post a baseline solution and all solutions that obtain an accuracy higher than baseline accuracy will be eligible to submit their solutions for Phase II. For details on the evaluation of the solutions for this phase, see the final evaluation details describe for the final evaluation above. This phase will be in an offline mode and results can be viewed at [Phase II Leaderboard](https://sites.google.com/view/rcv2023/leaderboard-workshop-challenges). We will keep updating the leaderboard on a rolling basis, however, every solution can be expected to get reflected on the leaderboard in 5 days from the day of the submission. Every team will be able to submit their two best solutions in one instance. For the start date of this phase, check the timeline page.\n\n## Challenge Organizer team\nFor details related to the organizing team, see https://sites.google.com/view/rcv2023/organizing-committee\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "### One account per participant\nYou cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.\n\n### No private sharing outside teams\nPrivately sharing code or data outside of teams is not permitted. It's okay to share code if made available to all participants on the forums.\n\n### Team Mergers\nThere are no team mergers permitted in this competition.\n\n### Team Limits\nEach team can be formed with a maximum of 3 members.\n\n### Submission Limits\nYou may submit a maximum of 5 entries per day.\n\nYou may select up to 2 final submissions for judging.\n\nPrize Eligibility\nIn order to be eligible for the prizes, you must:\n\nParticipate in Phase II of the competition that will be held in an offline model.\n\nSubmit code on time for the team to evaluate. Ensure that your code scripts respect the memory contraints and that it works with the python environment provided by the organizers.\n\nPlease note that people affiliated with Transmute AI Lab are not eligible for the prizes.\n\nCompetition Timeline\nCompetition Timeline dates (including Entry Deadline, Final Submission Deadline,  and Start Date, as applicable) are reflected on the competition’s Overview > Timeline page.\n\nCompetition Data\n\"Competition Data\" means the data or datasets available from the Competition Website for the purpose of use in the Competition, including any prototype or executable code provided on the Competition Website. The Competition Data will contain private and public test sets. Which data belongs to which set will not be made available to participants.\n\nA. Data Access and Use. Competition Use only: You may access and use the Competition Data only for participating in the Competition and on Kaggle.com forums. The Competition Sponsor reserves the right to disqualify any participant who uses the Competition Data other than as permitted by the Competition Website and these Rules.\n\nB. Data Security. You agree to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the Competition Data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Competition. You agree to notify Hosts immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with them to rectify any unauthorized transmission or access.\n\nC. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your Submissions. However, you will ensure the External Data is publicly available and equally accessible to use by all participants of the Competition for purposes of the competition at no cost to the other participants.\n\nSubmissions may not use or incorporate information from hand labeling or human prediction of the training or test data records.\n\nCompetition Code\nCode related to all submissions on the leaderboard should be submitted during the course of the competition. The submitted code, in its original form, should be able to reproduce the scores. Your code should not be using any external data to produce the reported scores.\nBy participating in the competition, you provide rights to the competition organizers to use your code, associated solution and any related ideas for any venue or publication outside the competition.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Timeline",
      "content": "## Timeline\n- Phase I starts: April 12, 2023\n- Phase II starts: June 12, 2023\n- Phase II ends: July 30, 2023\n- Final submission of code: Aug 01, 2023\n- Notification of results: Aug 05, 2023\n- Submission of report: Aug 09, 2023",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "All submissions will be evaluated with different metrics at the two phases of the competition.\n\n- **Phase I**: This phase is hosted on Kaggle and will be using accuracy as the evaluation measure. \n\n- **Phase II**: This competition requires that the resultant model is light-weight. Solutions will be ranked in terms of accuracy metric and latency:\n~~~~\nTWE = (accuracy^2) / inference_time (in minutes)\n~~~~\nThe following machine configuration will be used for inference -\nGPU : Quadro RTX 8000 \nGPU Mem. Limit : 16GB\nCPU Cores : 4\nRAM : 32GB\n## Submission File for Phase I\n\nFor each `image_id`, you should predict the corresponding image label in the `digit_sum` column\n\nThe file should be named submission.csv and containing the columns `image_id` and `digit_sum` in the following format:\n~~~~\nimage_id,digit_sum\nasdfghjklo,0\nzxcvbnmlkj,1\n~~~~\n\n## Submission File for Phase II\nEach qualifying team is required to submit scripts that will be run and evaluated on a private dataset. The details of qualification to Phase II and format of code scripts is specified in the **Description** tab.\n\n## Submission File for Phase I\n\nFor each `image_id`, you should predict the corresponding image label in the `digit_sum` column\n\nThe file should be named submission.csv and containing the columns `image_id` and `digit_sum` in the following format:\n~~~~\nimage_id,digit_sum\nasdfghjklo,0\nzxcvbnmlkj,1\n~~~~\n\n## Submission File for Phase II\nEach qualifying team is required to submit scripts that will be run and evaluated on a private dataset. The details of qualification to Phase II and format of code scripts is specified in the **Description** tab.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Prizes",
      "content": "## Prizes\n\nNote that Kaggle platform is only used to run Phase I of the challenge. The final prizes will be based on Phase II results and the details can be found here: https://sites.google.com/view/rcv2023/workshop-challenges",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}