The evaluation metric for this contest is [a version of macro-averaged ROC-AUC that skips classes which have no true positive labels](https://www.kaggle.com/code/metric/birdclef-roc-auc).

# Submission Format
For each `row_id`, you should predict the probability that a given species was present. There is one column per species. **Each row covers a five-second window of audio.**


## Working Note Award Criteria (optional)

Criteria for the BirdCLEF Best Working Note Award:

**Originality.** The value of a paper is a function of the degree to which it presents new or novel technical material. Does the paper present results previously unknown? Does it push forward the frontiers of knowledge? Does it present new methods for solving old problems or new viewpoints on old problems? Or, on the other hand, is it a rehash of information already known?

**Quality.** A paper's value is a function of the innate character or degree of excellence of the work described. Was the work performed or the study made with a high degree of thoroughness? Was high engineering skill demonstrated? Is an experiment described which has a high degree of elegance? Or, on the other hand, is the work described pretty much of a run-of-the-mill nature?

**Contribution.** The value of a paper is a function of the degree to which it represents an overall contribution to the advancement of the art. This is different from originality. A paper may be highly original but may be concerned with a very minor, or even insignificant, matter or problem. On the other hand, a paper may make a great contribution by collecting and analyzing known data and facts and pointing out their significance. Or, a fine exposition of a known but obscure or complex phenomenon or theory or system or operating technique may be a very real contribution to the art. Obviously, a paper may well score highly on both originality and contribution. Perhaps the important question is, will the engineer who reads the paper be able to practice his profession more effectively because of having read it?

**Presentation.** The value of the paper is a function of the ease with which the reader can determine what the author is trying to present. Regardless of the other criteria, a paper is not good unless the material is presented clearly and effectively. Is the paper well written? Is the meaning of the author clear? Are the tables, charts, and figures clear? Is their meaning readily apparent? Is the information presented in the paper complete? At the same time, is the paper concise?

*Evaluation of the submitted BirdCLEF working notes:*

Each working note will be reviewed by two reviewers and scores averaged. Maximum score: 15.

a) Evaluation of work and contribution

-   5 points: Excellent work and a major contribution
    
-   4 points: Good, solid work of some importance
    
-   3 points: Solid work but a marginal contribution
    
-   2 points: Marginal work and minor contribution
    
-   1 point: Work doesn't meet scientific standards
    

b) Originality and novelty

-   5 points Trailblazing
    
-   4 points: A pioneering piece of work
    
-   3 points: One step ahead of the pack
    
-   2 points: Yet another paper about…
    
-   1 point: It's been said many times before
    

c) Readability and organization

-   5 points: Excellent
    
-   4 points: Well written
    
-   3 points: Readable
    
-   2 points: Needs considerable work
    
-   1 point: Work doesn't meet scientific standards