Submissions are evaluated on the categorization accuracy. Each correct identification is counted (from a correct solution.csv file not visible to you) and the score is expressed as a percent correct; 0.91 = 91% correct.
Submission Format
For every connection in the test dataset
, submission files should contain two columns: Id and Category. Id represents the unique TCP connection (the specific sample from the test.csv file identified in column "id") while Category represents the type of attack predicted by your model(s), i.e. the predicted y (target, label) value.
The file should contain a header row, 489,844 prediction rows, and have the following format:
Id,Category
0,imap
1,normal
...
311028,neptune
A word about Kaggle Leaderboards. Rows in the solution.csv file are divided, for scoring purposes, into 30% for the Public Leaderboard and 70% for the Private Leaderboard. The Public Leaderboard gives you some idea of how you are doing against 30% of the correct solution. 70% is hidden from you, but show up on the Private Leaderboard only I can see. A combination of the Public and Private Leaderboard scores is used to determine the final score (and grade), see below.
I have set the sample_solution.csv file, which has all predictions as normal, as the benchmark score you should be able to beat. The sample_solutions.csv file has a score of 0.199, or 19.9% correct. This score shows up on the Public Leaderboard. You can make 20 submissions per day while the competition is running.
Final Scoring and Letter Grade
Your letter grade on this assignment is calculated as follows:
combined_score = (0.3 * Public_Score) + (0.7 * Private_Score)
The combined_score is then scaled to a point on a line defined by 2 points:
(x1, y1) = (0.199, 0.7) and,
(x2, y2) = (1.0, 1.0)
where y1=0.7 maps to a C- letter grade, and y2=1.0 maps to an A letter grade.
No submission receives a combined_score of 0, or a submission that has formatting errors and cannot be scored receives a combined_score of 0, both of which maps to an F letter grade.