{
  "fetched_at": "2026-10-19T07:47:40Z",
  "source_url": "https://www.kaggle.com/competitions/cs5785-fall19-final/data",
  "pages": [
    {
      "name": "rules",
      "content": "\n<h3>One account per participant</h3>\n<p>You cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.</p>\n<h3>No private sharing outside teams</h3>\n<p>\n    Privately sharing code or data outside of teams is not permitted.\n    It's okay to share code if made available to all participants on the forums.\n</p>\n<h3>Team Mergers</h3>\n<p>Team mergers are not allowed in this competition.</p>\n\n<h3>Team Limits</h3>\n<p>There is no maximum team size.</p>\n<h3>Submission Limits</h3>\n<p>You may submit a maximum of 2 entries per day.</p>\n<p>You may select up to 2 final submissions for judging.</p>\n<h3>Competition Timeline</h3>\n<p>Start Date: <strong>TBA</strong></p>\n<p>Merger Deadline: <strong>None</strong></p>\n<p>Entry Deadline: <strong>None</strong></p>\n<p>End Date: <strong>2/1/2014 12:00 AM UTC</strong></p>\n",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "data-description",
      "content": "<h1>File descriptions</h1>\n\n<ul>\n<li><strong>images_train</strong> - 10,000 training images of size 224x224.</li>\n<li><strong>images_test</strong> - 2,000 test images of size 224x224.</li>\n<li><strong>tags_train</strong> - image tags correspond to training images. Each image have several tags indicating the human-labeled object categories appear in the image, in the form of \"supercategory:category\".</li>\n<li><strong>tags_test</strong> - image tags correspond to test images. Each image have several tags indicating the human-labeled object categories appear in the image, in the form of \"supercategory:category\".</li>\n<li><strong>features_train</strong> - features extracted from a pre-trained Residual Network (ResNet) on training set, including 1,000 dimensional feature from classification layer (fc1000) and 2,048 dimensional feature from final convolution layer (pool5). Each dimension of the fc1000 feature corresponds to a WordNet synset here.</li>\n<li><strong>features_test</strong> - features extracted from the same Residual Network (ResNet) on test set, including 1,000 dimensional feature from classification layer (fc1000) and 2,048 dimensional feature from final convolution layer (pool5).</li>\n<li><strong>descriptions_train</strong> - image descriptions correspond to training images. Each image have 5 sentences for describing the image content.</li>\n<li><strong>descriptions_test</strong> - image descriptions for test images. Each image have 5 sentences for describing the image content. <em>Notice that one test description corresponds to one test image. The task you need to do is to return top 20 images in test set for each test description.</em></li>\n<li><strong>sample_submission.csv</strong> - a sample submission file in the correct format.</li>\n</ul>",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "<h1>About The Exam</h1>\n<p>The format of the exam is a mock peer-reviewed conference. You will develop an algorithm, prepare an anonymized professional paper, submit an anonymized version, and peer-review the work from other groups. There are four deliverables:</p>\n\n<hr>\n<h2><strong>- Phase 1: Develop your algorithm and write your writeup</strong></h2>\n<p>Develop your algorithm and submit your results to the Kaggle leaderboard. <br><strong>Deadline: Thursday, Dec 12, 11:59 PM EST.</strong>\n<\\p>\n\n<p>\nWrite your paper submission and submit an anonymized version to both HotCRP [<a href=\"https://cornell-cs5785-fa2019.hotcrp.com/\">link</a>] and CMSX. \n<br><strong>Deadline: Thursday, Dec 12, 11:59 PM EST</strong>.\n</p>\n\n<p>\nSubmit a brief paragraph that describes you and your teammates' contribution to the team to CMSX. \n<br><strong>Deadline: Thursday, Dec 12, 11:59 PM EST</strong>.\n</p>\n\n<p><strong>Describing your contributions</strong></p>\n<p>Each student should also submit a brief description of each team member's contribution to the project. \nThis should also include your own contribution to the project.\nFor example, which part of the algorithm you worked on, which the section of the report you wrote. \n</p>\n\n<p><strong>What must be included in the report?</strong></p>\n<ul>\n<li>A clear description of your approach.</li>\n<li>A screenshot of your team submission in your report. This will also be an important part of your grade.</li>\n</ul>\n\n<p><strong>IMPORTANT NOTES:</strong></p>\n<ul>\n<li>Your submission to HotCRP <strong>has to be</strong> anonymous.</li>\n<li>Your report should be no longer than <strong>four</strong> pages. You can include an additional page of reference.</li>\n<li>There should be only one competition submission from each team. </li>\n<li>You need to create an account on HotCRP with your <strong>Cornell email</strong>.</li>\n<li>You also need to submit your report to CMS, so that the graders could have access to it.</li>\n<li>The description of individual contribution is <strong>not</strong> a per team submission.  Every one has to submit his/her own.</li>\n<li>Remember to invite all your group members in CMSX and HotCRP.</li>\n<li>It would be great if you can make your team-name searchable. The graders will need to llok up your team and check your scores in the private leaderboard.</li>\n</ul>\n\n<hr>\n<h2><strong>- Phase 2: Peer-review other groups' work!</strong></h2>\n<p>Write reviews and submit them to HotCRP. \n<br><strong>Deadline: Friday, Dec 13, 11:59 PM EST</strong>. </p>\n<p>\nThe submission site will open after Phase 1 ends.\n<strong>Everybody on each team must complete peer reviews individually. (They are not \"per-team\")</strong></p>\n\n<hr>\n<h1>The Challenge: Build a large-scale image search engine!</h1>\n<p>You and your team of <strong>four Cornell Tech students</strong> are surely on the path to fame and fortune! You have been recruited by Google to disrupt Google Image Search by building a better search engine using novel statistical learning techniques.</p>\n<p>The specifications are simple: We need a way to <strong>search for relevant images</strong> given a natural language query. For instance, if a user types \"dog jumping to catch frisbee,\" your system will <strong>rank-order the most relevant images</strong> from a large database.</p>\n<p>Here are some details:</p>\n<ul>\n<li><p><strong>During training</strong>, you have a dataset of 10,000 samples. Each sample has the following data available for learning:</p>\n<ul>\n<li>A 224x224 JPG image. </li>\n<li>A list of tags indicating objects appeared in the image. </li>\n<li>Feature vectors extracted using <a href=\"https://arxiv.org/abs/1512.03385\" target=\"_blank\">ResNet</a>, a state-of-the-art Deep-learned CNN (You don't have to train or run ResNet -- we are providing the features for you). See <a href=\"http://ethereon.github.io/netscope/#/gist/b21e2aae116dc1ac7b50\" target=\"_blank\">here</a> for the illustration of the    ResNet-101 architecture. The features are extracted from pool5 and fc1000 layer.</li>\n<li>A five-sentence description, used to train your search    engine.</li></ul></li>\n<li><p><strong>During testing</strong>, your system matches a <em>single five-sentence description</em> against a pool of 2,000 <em>candidate samples</em> from the test set. Each sample has:</p>\n<ul>\n<li>A 224x224 JPEG image.</li>\n<li>A list of tags for that image.</li>\n<li>ResNet feature vectors for that image.</li></ul></li>\n</ul>\n<p>-<strong>Output:</strong> For each description, your system must rank-score each testing image with the likelihood of that image matches the given sentence. Your system then returns the name of the top 20 relevant images, delimited by space. See \"sample_submission.csv\" on the data page for more details on the output format.</p>\n<p>-<strong>Evaluation metric:</strong> There are 2,000 descriptions, and for each description, you must compare against the entire 2,000-image test set. That is, rank-order test images for each test description. We will use <strong>MAP@20</strong> as the evaluation metric. If the corresponding image of a description is among your algorithm's 20 highest scoring images, this metric gives you a certain score based on the ranking of the corresponding image. Please refer to the evaluation page for more details. <br>\nUse all of your skills, tools, and experience. It is OK to use libraries like numpy, scikit-learn, pandas, etc., as long as you cite them. Use cross-validation on training set to debug your algorithm. Submit your results to the Kaggle leaderboard and send your complete writeup to CMS. The data you use --- and the way you use the data --- is completely up to you.</p>\n<p>The best teams of <strong>four Cornell Tech students</strong> might use visualization techniques for debugging (e.g., show top images retrieved by your algorithm and see whether they make sense or not), preprocessing, a nice way to compare tags and descriptions, leveraging visual features and combining them with tags and descriptions, supervised and/or unsupervised learning to best understand how to best take advantage of each data source available to them.</p>\n<p>The best reports will be professionally written. We suggest including an <strong>Abstract</strong>, followed by an <strong>Introduction</strong> section including motivations of your method and a detailed description of each step in your pipeline / framework you used for this task. Then, include a detailed <strong>Experiment</strong> section and a <strong>Results</strong> section. If time permits, a <strong>Background / Related Work</strong> section can help to point out the relevant work in the image retrieval literature.</p>\n<p>The best reports use a professional style adopted by a major academic conference. NIPS is a good choice. Teams can download a template from \"Formatting instructions\" of the <a href=\"https://nips.cc/Conferences/2017/PaperInformation/EvaluationCriteria\" target=\"_blank\">NIPS Author Guidelines</a>. We recommend you to use the LaTeX template, but this is not a requirement. If you're more familiar with Word, use Word.</p>\n<p>The best peer reviews will be detailed, constructive, and <a href=\"https://compassionatecoding.com/blog/2016/10/11/be-a-mentor-not-a-bully\" target=\"_blank\">compassionate rather than nitpicky or showy</a>, pointing out potential areas of improvement as well as complimenting their peers' strengths.</p>\n<p>The best teams understand how to divide the precious time and energy among all members of the group. They might split into roles, preventing programming students from messing up the writing and ensuring writing gurus don't screw up the code. Close contact is always maintained within top-performing teams.</p>\n\n<hr>\n<h2>Rules</h2>\n<p>Your group can have a maximum of <strong>four</strong> students.</p>\n<p>Groups <strong>may not</strong> collaborate with other groups. You cannot cite other teams' work. This includes sharing data, intermediate and final results, writing, or discussing your detailed method with other groups. You <strong>may not</strong> view reports from other teams until <strong>after</strong> the peer-review phase begins!</p>\n<p>You <strong>can not</strong> use any external training data. This include the use of online pretrained model on a different dataset. If you want to use a pretrained model, please discuss with the TAs. Following is a list of pretrained model you are allowed to use (the list will be updated): </p>\n<ul>\n<li>ImageNet trained ResNet and DenseNets</li>\n<li>Word2vec and BERT.</li>\n</ul>\n\n<hr>\n<h2>How to anonymize your paper</h2>\n<p>The \"Peer Review\" phase will be double-blind. This means authors will not know who the reviewers are, and the reviewers should not know who the authors are. This protects both sides.</p>\n<p>To maintain anonymity, don't include team member names in the actual paper that you submit to CMS!</p>\n<p><img src=\"http://i.imgur.com/F255rFT.png\" alt=\"enter image description here\"></p>\n\n<hr>\n<h2>Peer Review Tips</h2>\n<p>See the following resources for tips on how to write a professional review:</p>\n<p>Matthew Might’s notes on how to peer review and <a href=\"http://matt.might.net/articles/peer-fortress/\" target=\"_blank\">Peer Fortress: The Scientific Battlefield</a> (the last one provides examples NOT to follow)<br>\nCVPR 2018 Reviewer’s Guidelines.<br>\n<a href=\"http://mobilehci.acm.org/2015/download/ExcellenceInReviewsforHCICommunity.pdf\" target=\"_blank\">So You're a Program Committee Member Now: On Excellence in Reviews and Meta-Reviews and Championing Submitted Work That Has Merit</a></p>\n\n<hr>\n<h2>Score Breakdown</h2>\n<ul>\n<li><strong>30% on Kaggle performance</strong>. This will be based on the score on the private test set.</li>\n<li><strong>30% on the report</strong>. Is the proposed method clearly described? is it professionally written? does it include enough detail for a professional (ie. skilled graduate student) to re-implement the results? A good report should at least contain a brief introduction of what you did, a detailed description of your method, an experimental evaluation part that shows the experimental results and analysis of your results. <\\li>\n<li><strong>20% on method</strong> Does it make sense? is it suited to the task? does it make use of the data in appropriate ways?<\\li>\n<li><strong> 5% on peer reviews from other classmates</strong>.<\\li>\n<li><strong>5% on peer reviews from your teammates</strong>.<\\li>\n<li><strong>10% on the quality of peer review</strong> Is it well-written and thoughtful? does it provide insight to the authors?<\\li>\nNote that we may also adjust the grades based on tindividual contributions.\n</p>",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation",
      "content": "<p>The evaluation metric is <strong>Mean Average Precision at 20 (MAP@20)</strong> on the test set. A good explanation of the metric can be found <a href=\"http://fastml.com/what-you-wanted-to-know-about-mean-average-precision/\" target=\"_blank\">here</a>.</p>\n\n<p>In our case, there is only one true corresponding image. So for a description, suppose your algorithm find the corresponding image in top 20 of your returned list and its ranked i-th, MAP@20 gives a score of</p>\n\n$$scores=\\frac{20+1-i}{20}$$\n\n<p>For example, if its ranked 1st, you get 1; if its 11th, you get 0.5; if its 20th, you get 0.05. If the corresponding image is not in your top 20 list, you get 0 score. The overall score is calculated as the average score on the test set.</p>\n<p>We divide the test set into two splits: public split (50%) and private split (50%). Results on the Leaderboard are evaluated on public split and the private split is used for grading.</p>",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}