![\"\"](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3163/media/Facebook%20Careers2.png\")
There is no monetary prize for this competition and no points will be awarded. Highly ranked contestants who indicate their interest will be considered by Facebook for further interviews based on their work in the competition and additional background. Positions\r\n are available in Menlo Park, Seattle, \r\nNew York City, and London; candidates must have, or be eligible to obtain, authorization to work in the US or UK.
\r\n
You cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.
\r\nAny sharing code or data outside of teams is not permitted. This includes making code or data made available to all players, such as on the forums.
\r\nThe maximum size of a team is 1 participants.
\r\nYou are allowed to make your team anonymous, which hides participant details.
\r\nYou can submit a maximum of 2 entries per day.
\r\nYou can select up to 5 final submissions for judging.
\r\nCandidates grant Facebook a license to use the Intellectual Property behind any submission\r\nonly for the purposes of assessing the submission and evaluate the suitability of the Participant for employment. The Participant must deliver the algorithm’s code and documentation to Sponsor if they wish to\r\n be considered for employment. The source code must contain a description of resources required to build and run the algorithm.
\r\nThe Official Rules document of the competition can be downloaded \r\nhere.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "data-description", "content": "
\r\nThere are 15 graphs in the training set and 5 in the test set. These are directed, dynamic, real-world Internet topology graphs with nodes that correspond to Autonomous\r\n Systems (AS). Each graph is sampled at a fixed time interval, \\\\(\\Delta t\\\\), apart. The 5 test graphs represent the status of the network at the 5 time points after the final training graph, but are not supplied to participants.
| Graph | \r\nTime | \r\nSupplied? | \r\n
| train1 | \r\n\\\\( t_0 \\\\) | \r\nY | \r\n
| train2 | \r\n\\\\( t_0 +\\Delta t \\\\) | \r\nY | \r\n
| train3 | \r\n\\\\( t_0 + 2\\Delta t \\\\) | \r\nY | \r\n
| train4 | \r\n\\\\( t_0 + 3\\Delta t \\\\) | \r\nY | \r\n
| train5 | \r\n\\\\( t_0 + 4\\Delta t \\\\) | \r\nY | \r\n
| train6 | \r\n\\\\( t_0 + 5\\Delta t \\\\) | \r\nY | \r\n
| train7 | \r\n\\\\( t_0 + 6\\Delta t \\\\) | \r\nY | \r\n
| train8 | \r\n\\\\( t_0 + 7\\Delta t \\\\) | \r\nY | \r\n
| train9 | \r\n\\\\( t_0 + 8\\Delta t \\\\) | \r\nY | \r\n
| train10 | \r\n\\\\( t_{0} + 9\\Delta t \\\\) | \r\nY | \r\n
| train11 | \r\n\r\n \\\\( t_{0} + 10\\Delta t \\\\) \r\n | \r\nY | \r\n
| train12 | \r\n\\\\( t_{0} + 11\\Delta t \\\\) | \r\nY | \r\n
| train13 | \r\n\\\\( t_{0} + 12\\Delta t \\\\) | \r\nY | \r\n
| train14 | \r\n\\\\( t_{0} + 13\\Delta t \\\\) | \r\nY | \r\n
| train15 | \r\n\\\\( t_{0} + 14\\Delta t \\\\) | \r\nY | \r\n
| test1 | \r\n\\\\( t_{0} + 15\\Delta t \\\\) | \r\nN | \r\n
| test2 | \r\n\\\\( t_{0} + 16\\Delta t \\\\) | \r\nN | \r\n
| test3 | \r\n\\\\( t_{0} + 17\\Delta t \\\\) | \r\nN | \r\n
| test4 | \r\n\\\\( t_{0} + 18\\Delta t \\\\) | \r\nN | \r\n
| test5 | \r\n\\\\( t_{0} + 19\\Delta t \\\\) | \r\nN | \r\n
Edges may appear and disappear as routing patterns change over time. Some nodes are peers (A->B means A can exchange traffic through B for free) and some are connected by regular edges (A->B means A pays B to route their traffic). Peers have weights of\r\n zero in the training data and regular edges have weights of one.
\r\nWe have shuffled the AS numbers and replaced them with distorted versions of the AS names. In other words, the names are arbitrary with respect to reality, but consistent within the data. For example, if the original graph has the AS number/name
\r\n26 CORNELL - Cornell University,
\r\nit would be remapped to a random new number and its corresponding name, say (hypothetically),
\r\n11 HARVARD - Harvard University,
\r\nwith the text altered in some fashion. The AS remapping is done to discourage reverse engineering the graph. The text alteration is done to test participants' ability to turn messy, real-world data in to a computable format. Some AS names are not available.\r\n For these, we have left the AS number intact.
\r\nFor the prediction task, you are given a path which, at one point in the training time period, was an optimal path from node A to B. Here, “optimal” accounts for the peer relationships with zero-weight edges (i.e. it is the weighted shortest path).\r\n Many paths can have the same best optimal weight and are all considered optimal, regardless of the path length.
\r\nThe training set graphs give links in the form:
\r\nMessy AS Name 1 | Messy AS Name 2 | Edge weight
\r\nThe test set paths have the form:
\r\nMessy AS Name Start | Messy AS Name | ... | Messy AS Name | Messy AS Name End
\r\nExample real data from the training set:
\r\n64649 | B--NSAORWKCPAEST Southern Cross Broadcasting. | 1\r\n
4715 | FISHNET-NZ Otago Regional Office | 1
NIC éc,ioxM S.C. | BCS CBS Technologies SA | 1
KOCAS-KW Kuwait Oil Company (K.S.C.) | 64998 | 0
You are supplied with 10000 paths (paths.txt). The task is then to predict, for all 5 test graphs, the probability that the given path is STILL an optimal path.
\r\nCompute your probabilities as a matrix of size 10000 x 5 (10000 path probabilities, predicted at each of the 5 test times), sorted according to the original order of the test set. Concatenate the five 10000 x 1 columns to get a 50000 x\r\n 1 column vector, and prepend a header row, "Probability", to create a 50001-row column vector. To be explicit, if your probabilities for path \\\\(i\\\\) at time \\\\(j\\\\) are \\\\( p_{i,j} \\\\), the format should look like:
\r\n$$ \\left( \\begin{array}{c}
\r\n\\text{Probability}\\\\
\r\np_{1,1}\\\\
\r\np_{2,1}\\\\
\r\n\\vdots \\\\
\r\np_{10000,1}\\\\
\r\np_{1,2}\\\\
\r\np_{2,2}\\\\
\r\n\\vdots \\\\
\r\np_{10000,2}\\\\
\r\n\\vdots \\\\
\r\np_{10000,5}\\\\
\r\n\\end{array} \\right) $$
For this challenge, potential Facebook recruits will be exploring the map of the entire internet. Unlike the map of a city, where best routes are relatively fixed except for the occasional construction or parade detour, the paths that information travels\r\n over the web are constantly changing. There is no centralized system of stop-lights or traffic cops. Instead, there are tens of thousands of autonomous systems using a common protocol to advertise the next available hops, updated depending on service-agreements,\r\n capacity, and load. This will be a test of both the candidates engineering know-how and their ability to statistically learn on complex, dynamic graph structures.
\r\nThe Task: you will be given a path which, at one point in the training time period, was an optimal path from node A to B. The question is then to make a probalistic prediction, for each of the 5 test graphs, whether the given path is STILL an optimal\r\n path. This is a much more difficult task than link prediction alone. The global structure of the graph may affect many optimal routes, paths can have varying lengths (and thus varying a priori probabilities of being optimal), and there may be multiple\r\n optimal routes for a given source & destination.
\r\n![\"Graph](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3163/media/time_series_graphic.png\")
The Prize: Facebook is seeking data-savvy software engineers (Data Engineers) to build the next generation of systems that will transform the online experience of over a billion users. Appropriate candidates should have experience with multiple components\r\n across the big data stack (check out the \r\nVisualization track for another way to highlight your skills). There are many teams that they could be a fit for depending on their backgrounds .\r\nPositions are available in Menlo Park, Seattle, New York City, and London; candidates must have, or be eligible to obtain, authorization to work in the US or UK.
\r\n\r\n
![\"Round-trip](\"https://storage.googleapis.com/kaggle-media/competitions/kaggle/3163/media/graph_inverted.gif\")
An example visualization of Internet topology (round-trip times) produced by Walrus
\r\n(courtesy of Young Hyun and inverted for display purposes)
Your predictions will be evaluated by the \r\nArea Under the ROC Curve (AUC) metric. While the ground-truth is binary, real-valued submissions are allowed. See the\r\nData Description page for instructions on formatting your submission.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Background Internet Routing Architecture", "content": ""The second [Autonomous System] level in the [internet] routing hierarchy is the interdomain routing domain. The interdomain routing environment describes how domains interconnect, but avoids the task of maintaining transit\r\n paths within each domain. In the interdomain space, a routing path to an address is described as a sequence of domains that must be transited to reach the domain that originates that particular address prefix. Today this interdomain space is maintained using\r\n Version 4 of the Border Gateway Protocol (BGPv4).
\r\nEach routing domain is a single administrative domain, operated within a uniform set of routing policies, and is operated independently from any other domain. The domain is in effect an autonomous unit in the overall routing architecture,\r\n and is termed an Autonomous System (AS). Each of these ASs is uniquely identified using an Autonomous System Number (ASN)."
\r\nfrom \r\nExploring Autonomous System Numbers
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Faqs", "content": "Q. Where are the forums?
\r\nThe forums have been disabled in this competition to help ensure that each entry reflects only the participants' own work. They will be turned on after the competition closes to compare techniques. If you have a question about the data during the competition,\r\n please email will.cukierski@kaggle.com.
\r\nQ: Where did the data come from?
\r\nCompetition rules prohibit de-anonymization. At a high level, this data comes from relationships inferred from router tables.
Q: How long is the time step between graphs?!
\r\nTo reduce the temptation of de-anonymization, we have obsured this info. It is longer than milliseconds but shorter than years.
Q: Is the anonymitization same across all graphs?
\r\nAS numbers are mapped to the same anonymized number across the entire data set, but the text is altered in different ways within and accross each graph. For example, the same node will\r\nnot be HARVARD in one graph and CORNELL in another, but it could be HARVARD in one graph and ARDVARH in another. We don't do this to be cruel, but rather because Facebook is interested in people who can work\r\n with messy problems.
Q: Some AS names can not be mapped to a unique node?
\r\nThis is part of the challenge of messy data. We understand that it may not be possible to get a perfect score on the data, as presented.
Q: How many possible nodes are in the graph?
\r\n65535
Q: Are the peer edges (with weight 0) directed edges?
\r\nYes, A|B|0 is still a directed edge in this case. A can send traffic through/to B for free, but it does not imply an edge exists from B->A.
Q: Some of the paths have only one node?
\r\nYou can treat these paths the same as the others (although not quite as interesting). The weight of an edge from a node to itself is, by definition, 0.