{ "id": 2249, "title": "Doubt in the data of training-disambiguated-product-mentions.csv", "url": "/competitions/cprod1/discussion/2249", "author_name": "", "post_date": "2012-07-23T19:30:23.227Z", "votes": null, "comment_count": 1, "views": 1724, "content": "

@ADMIN- please reply to this..what does these in the file training-disambiguated-product-mentions.csv mean:-

\r\n

7b465aa355535d76323c0f89dfbec8b5:9-11,0

\r\n

7b465aa355535d76323c0f89dfbec8b5:14-16,0

\r\n

7b465aa355535d76323c0f89dfbec8b5:129-131,0

\r\n

These three document id are same, so whay are they listed so many times with different line numbers.

", "messages": [ { "id": "12676", "postDate": "07/23/2012 19:30:23", "content": "

@ADMIN- please reply to this..what does these in the file training-disambiguated-product-mentions.csv mean:-

\r\n

7b465aa355535d76323c0f89dfbec8b5:9-11,0

\r\n

7b465aa355535d76323c0f89dfbec8b5:14-16,0

\r\n

7b465aa355535d76323c0f89dfbec8b5:129-131,0

\r\n

These three document id are same, so whay are they listed so many times with different line numbers.

", "rawMarkdown": "", "votes": null }, { "id": "12688", "postDate": "07/24/2012 03:12:33", "content": "

The three records that you list are three distinct "disambiguated product mentions" - they are three distinct token spans in the training data each with an associated product list. For these three records:

\r\n

1) the product list is simply "0" which means that the product was not found in the product catalog,
\r\n2) the spans are three tokens long: 9-11; 14-16; and 129-131.
\r\n3) the spans are to three different product terms: "DirecTiVo HD HR10-250s", "DirecTV HD HR20-700s", and "Panny VHS/DVD Recorder" respectively (see the FYI below for a script to extract the "terms").

\r\n

The CPROD1 Team

\r\n

FYI, below is a one-liner perl script that can extract the span of text for a disambiguated product mention piped into STanDard INput. It assumes that you have installed the JSON package from CPAN, and that the json file being queried is training-annotated-text.json.\r\n The script is applied to the three records that you provide.

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:9-11,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "../training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

DirecTiVo HD HR10-250s

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:14-16,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "../training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

DirecTV HD HR20-700s

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:129-131,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

Panny VHS/DVD Recorder

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 12688, "author_name": "cprod1", "author_url": "", "post_date": "07/24/2012 03:12:33", "content": "

The three records that you list are three distinct "disambiguated product mentions" - they are three distinct token spans in the training data each with an associated product list. For these three records:

\r\n

1) the product list is simply "0" which means that the product was not found in the product catalog,
\r\n2) the spans are three tokens long: 9-11; 14-16; and 129-131.
\r\n3) the spans are to three different product terms: "DirecTiVo HD HR10-250s", "DirecTV HD HR20-700s", and "Panny VHS/DVD Recorder" respectively (see the FYI below for a script to extract the "terms").

\r\n

The CPROD1 Team

\r\n

FYI, below is a one-liner perl script that can extract the span of text for a disambiguated product mention piped into STanDard INput. It assumes that you have installed the JSON package from CPAN, and that the json file being queried is training-annotated-text.json.\r\n The script is applied to the three records that you provide.

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:9-11,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "../training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

DirecTiVo HD HR10-250s

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:14-16,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "../training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

DirecTV HD HR20-700s

\r\n

bash$ echo "7b465aa355535d76323c0f89dfbec8b5:129-131,0" | perl -e 'use JSON qw(decodejson) ; open $fh, "training-annotated-text.json" or die; $jtext; {local $/; $jtext = <$fh>} $json=decodejson($jtext); while () { ($tiid, $stok, $etok, $prodList)\r\n = split/[\\:-\\,]/; foreach my $tokenId ($stok .. $etok) {$token=@{$json->{TextItem}->{$tiid}}[$tokenId] ; print "$token "}}'

\r\n

Panny VHS/DVD Recorder

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "12676": "", "12688": "" }, "source": "meta" }