{ "id": 15567, "title": "Agreement Test Details", "url": "/competitions/flavours-of-physics/discussion/15567", "author_name": "", "post_date": "2015-07-27T06:50:20.357Z", "votes": 4, "comment_count": 2, "views": 640, "content": "

Looking at the details of the Agreement Test, we would like the classifier outputs to be from the same distribution for both the cases:

\n\n\n\n

Why is the distribution of the outputs supposed to be the same for both the cases? I would imagine that if we are not using any simulation specific features, the distribution should be the same only if we confine ourselves to signal events for both the data sources (or if we also simulate background events in the same proportion).

", "messages": [ { "id": "87135", "postDate": "07/27/2015 06:50:20", "content": "

Looking at the details of the Agreement Test, we would like the classifier outputs to be from the same distribution for both the cases:

\n\n\n\n

Why is the distribution of the outputs supposed to be the same for both the cases? I would imagine that if we are not using any simulation specific features, the distribution should be the same only if we confine ourselves to signal events for both the data sources (or if we also simulate background events in the same proportion).

", "rawMarkdown": "Looking at the details of the Agreement Test, we would like the classifier outputs to be from the same distribution for both the cases:\r\n\r\n- Real data : consisting of **both signal and background**\r\n- Simulated data: consisting **only of signal**\r\n\r\nWhy is the distribution of the outputs supposed to be the same for both the cases? I would imagine that if we are not using any simulation specific features, the distribution should be the same only if we confine ourselves to signal events for both the data sources (or if we also simulate background events in the same proportion).", "votes": null }, { "id": "87322", "postDate": "07/28/2015 17:23:11", "content": "

Hi, Neeraj!

\n\n

Thank you for this good question!

\n\n

The main thing is sPlot, which is applied to real data. If you look through weights for real data, you will find out that weights there are both positive and negative. They were computed in such a way, that contribution of the background is vanished, so effectively you're comparing real signal and Monte Carlo signal (without knowing labels in real data actually!)

\n\n

The idea of sPlot is the following:

\n\n
    \n
  1. You know signal and background distributions for some variable, in our case this is the mass.
    2. \n
  2. You have distribution of some feature X in your data (signal+background) (for example classifier’s output). It is necessary that this feature is not correlated with the mass.
    3. \n
  3. sPlot constructs you weights for each event in such way that distribution of the feature X with these weights will be signal distribution of the feature X (distribution with 1-weights will be background).
    4. \n
  4. That is why classifiers predictions with this sPlot weights will give signal distribution for real data.
    5. \n
", "rawMarkdown": "Hi, Neeraj!\r\n\r\nThank you for this good question!\r\n\r\nThe main thing is [sPlot](http://arxiv.org/abs/physics/0402083), which is applied to real data. If you look through weights for real data, you will find out that weights there are both positive and negative. They were computed in such a way, that contribution of the background is vanished, so effectively you're comparing real signal and Monte Carlo signal (without knowing labels in real data actually!)\r\n\r\n\r\nThe idea of sPlot is the following:\r\n\r\n1. You know signal and background distributions for some variable, in our case this is the mass.\r\n2. You have distribution of some feature X in your data (signal+background) (for example classifier’s output). It is necessary that this feature is not correlated with the mass.\r\n3. sPlot constructs you weights for each event in such way that distribution of the feature X with these weights will be signal distribution of the feature X (distribution with 1-weights will be background).\r\n4. That is why classifiers predictions with this sPlot weights will give signal distribution for real data.", "votes": null }, { "id": "87460", "postDate": "07/29/2015 17:30:29", "content": "

Thanks, that was really useful. It becomes clearer when I see the code now.

", "rawMarkdown": "Thanks, that was really useful. It becomes clearer when I see the code now.", "votes": null } ], "comments": [ { "id": 87322, "author_name": "tataantares", "author_url": "", "post_date": "07/28/2015 17:23:11", "content": "

Hi, Neeraj!

\n\n

Thank you for this good question!

\n\n

The main thing is sPlot, which is applied to real data. If you look through weights for real data, you will find out that weights there are both positive and negative. They were computed in such a way, that contribution of the background is vanished, so effectively you're comparing real signal and Monte Carlo signal (without knowing labels in real data actually!)

\n\n

The idea of sPlot is the following:

\n\n
    \n
  1. You know signal and background distributions for some variable, in our case this is the mass.
    2. \n
  2. You have distribution of some feature X in your data (signal+background) (for example classifier’s output). It is necessary that this feature is not correlated with the mass.
    3. \n
  3. sPlot constructs you weights for each event in such way that distribution of the feature X with these weights will be signal distribution of the feature X (distribution with 1-weights will be background).
    4. \n
  4. That is why classifiers predictions with this sPlot weights will give signal distribution for real data.
    5. \n
", "votes": null, "replies": [] }, { "id": 87460, "author_name": "neerajprad", "author_url": "", "post_date": "07/29/2015 17:30:29", "content": "

Thanks, that was really useful. It becomes clearer when I see the code now.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "87135": "Looking at the details of the Agreement Test, we would like the classifier outputs to be from the same distribution for both the cases:\r\n\r\n- Real data : consisting of **both signal and background**\r\n- Simulated data: consisting **only of signal**\r\n\r\nWhy is the distribution of the outputs supposed to be the same for both the cases? I would imagine that if we are not using any simulation specific features, the distribution should be the same only if we confine ourselves to signal events for both the data sources (or if we also simulate background events in the same proportion).", "87322": "Hi, Neeraj!\r\n\r\nThank you for this good question!\r\n\r\nThe main thing is [sPlot](http://arxiv.org/abs/physics/0402083), which is applied to real data. If you look through weights for real data, you will find out that weights there are both positive and negative. They were computed in such a way, that contribution of the background is vanished, so effectively you're comparing real signal and Monte Carlo signal (without knowing labels in real data actually!)\r\n\r\n\r\nThe idea of sPlot is the following:\r\n\r\n1. You know signal and background distributions for some variable, in our case this is the mass.\r\n2. You have distribution of some feature X in your data (signal+background) (for example classifier’s output). It is necessary that this feature is not correlated with the mass.\r\n3. sPlot constructs you weights for each event in such way that distribution of the feature X with these weights will be signal distribution of the feature X (distribution with 1-weights will be background).\r\n4. That is why classifiers predictions with this sPlot weights will give signal distribution for real data.", "87460": "Thanks, that was really useful. It becomes clearer when I see the code now." }, "source": "meta" }