In the training set, the Category column will have each connection labeled as either normal or an attack with exactly one type of attack protocol specified. It was curated by MIT Lincoln Labs as part of the 1998 DARPA Intrusion Detection Evaluation Program. Lincoln Labs set up an environment to acquire nine weeks of raw TCP data dumps for a local area network (LAN) simulating a typical U.S Air Force LAN. They operated the LAN in a manner comparable to a true Air Force environment except they peppered the LAN with numerous attacks. Raw training data amounts to four gigabytes of compressed TCP dump data across seven weeks of network traffic. Approx five million connection records. Each connection is ~ 100 bytes with 41 features.
\n\nIn this competition, we are predicting if a given network connection is malicious or not. In cases where a connection is malicious, we are to identify the specific attack mechanism. Note that the test data includes a slightly more diverse set of attack patterns than the train set.
\nOutside of a ‘normal’ connection, attacks can fall into one of 4 general categories. These are general categories of attack types, and are not the labels you are to predict, but to assist you in understanding the major attack categories. There are 4 general categories:
\n\n| Specific attack type (label) | \nGeneral attack category | \n
|---|---|
| back | dos |
| buffer_overflow | u2r |
| ftp_write | r2l |
| guess_passwd | r2l |
| imap | r2l |
| ipsweep | probe |
| land | dos |
| loadmodule | u2r |
| multihop | r2l |
| neptune | dos |
| nmap | probe |
| perl | u2r |
| phf | r2l |
| pod | dos |
| portsweep | probe |
| rootkit | u2r |
| satan | probe |
| smurf | dos |
| spy | r2l |
| teardrop | dos |
| warezclient | r2l |
| warezmaster | r2l |
| Feature | \nDescription | \nType | \n
|---|---|---|
| duration | \nlength (number of seconds) of the connection | \ncontinuous | \n
| protocol_type | \ntype of the protocol, e.g. tcp, udp, etc. | \ndiscrete | \n
| service | \nnetwork service on the destination, e.g., http, telnet, etc. | \ndiscrete | \n
| src_bytes | \nnumber of data bytes from source to destination | \ncontinuous | \n
| dst_bytes | \nnumber of data bytes from destination to source | \ncontinuous | \n
| flag | \nnormal or error status of the connection | \ndiscrete | \n
| land | \n1 if connection is from/to the same host/port; 0 otherwise | \ndiscrete | \n
| wrong_fragment | \nnumber of ``wrong'' fragments | \ncontinuous | \n
| urgent | \nnumber of urgent packets | \ncontinuous | \n
| hot | \nnumber of ``hot'' indicators | \ncontinuous | \n
| num_failed_logins | \nnumber of failed login attempts | \ncontinuous | \n
| logged_in | \n1 if successfully logged in; 0 otherwise | \ndiscrete | \n
| num_compromised | \nnumber of ``compromised'' conditions | \ncontinuous | \n
| root_shell | \n1 if root shell is obtained; 0 otherwise | \ndiscrete | \n
| su_attempted | \n1 if ``su root'' command attempted; 0 otherwise | \ndiscrete | \n
| num_root | \nnumber of ``root'' accesses | \ncontinuous | \n
| num_file_creations | \nnumber of file creation operations | \ncontinuous | \n
| num_shells | \nnumber of shell prompts | \ncontinuous | \n
| num_access_files | \nnumber of operations on access control files | \ncontinuous | \n
| num_outbound_cmds | \nnumber of outbound commands in an ftp session | \ncontinuous | \n
| is_hot_login | \n1 if the login belongs to the ``hot'' list; 0 otherwise | \ndiscrete | \n
| is_guest_login | \n1 if the login is a ``guest''login; 0 otherwise | \ndiscrete | \n
\n
Can machine learning identify when a network connection or intrusion is malicious? Network security is a universal concern. And, as attack mechanisms became more sophisticated, network security and identification mechanisms had to adopt newer strategies as well. This classroom challenge will introduce you to tackling network security issues with machine learning.
\n\nMachine learning is used in a variety of security fronts. It’s used to detect malicious activity and stop attacks, analyze mobile endpoints, enhance human analysis, automate repetitive security tasks, and close zero-day vulnerabilities.
\n\nMIT’s Lincoln Labs, as part of the 1998 Darpa Intrusion Detection Evaluation Program, set up an environment to acquire nine weeks of raw TCP data dumps for a local area network (LAN) simulating a typical U.S Air Force LAN. They operated the LAN in a manner comparable to a true Air Force environment except they peppered the LAN with numerous attacks. This labeled data is what you’ll be working with as you try to tackle network security.
\n\nIn this competition, you’ll build algorithms to identify if a connection is normal or part of a malicious attack. You’ll be building a multi-class classification model leveraging the network data provided by Lincoln Labs. Winning models will be sure to carefully consider feature engineering along with model selection to create flexible models able to handle varied network traffic.
\n\nTo be successful in this competition, you will need to build a machine learning classifier capable of identifying the type of connection and what type of traffic it is: normal or malicious. Along the way, you’ll gain familiarity with exploratory data analysis, data cleaning and preparation, feature engineering, model selection, and hyper parameter tuning.
\n\nWe thank the MIT Lincoln Lab & KDD.
\n\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "Submissions are evaluated on the categorization accuracy. Each correct identification is counted (from a correct solution.csv file not visible to you) and the score is expressed as a percent correct; 0.91 = 91% correct.
\n\nFor every connection in the test dataset, submission files should contain two columns: Id and Category. Id represents the unique TCP connection (the specific sample from the test.csv file identified in column \"id\") while Category represents the type of attack predicted by your model(s), i.e. the predicted y (target, label) value.
\nThe file should contain a header row, 489,844 prediction rows, and have the following format:
\nId,Category\n\n
0,imap
1,normal
...
311028,neptune
\nA word about Kaggle Leaderboards. Rows in the solution.csv file are divided, for scoring purposes, into 30% for the Public Leaderboard and 70% for the Private Leaderboard. The Public Leaderboard gives you some idea of how you are doing against 30% of the correct solution. 70% is hidden from you, but show up on the Private Leaderboard only I can see. A combination of the Public and Private Leaderboard scores is used to determine the final score (and grade), see below.\n
\n\n\nI have set the sample_solution.csv file, which has all predictions as normal, as the benchmark score you should be able to beat. The sample_solutions.csv file has a score of 0.199, or 19.9% correct. This score shows up on the Public Leaderboard. You can make 20 submissions per day while the competition is running.\n
\n\n\nYour letter grade on this assignment is calculated as follows:
\ncombined_score = (0.3 * Public_Score) + (0.7 * Private_Score)
\n
\nThe combined_score is then scaled to a point on a line defined by 2 points:
\n(x1, y1) = (0.199, 0.7) and,
\n(x2, y2) = (1.0, 1.0)
\nwhere y1=0.7 maps to a C- letter grade, and y2=1.0 maps to an A letter grade.
\n
\nNo submission receives a combined_score of 0, or a submission that has formatting errors and cannot be scored receives a combined_score of 0, both of which maps to an F letter grade.\n