Since this is a new metric to Kaggle, I thought I'd share the code we use to calculate it (in C#):
\r\npublic static double Gini(this IList<double> a, IList<double> p) {
if(a.Count != p.Count) {
throw new ArgumentException();
}
var all =
p.Zip(a, (pred, actual) => new { actualValue = actual, predictedValue = pred }) // (actual, prediction)
.Zip(Enumerable.Range(1, a.Count), (ap, i) => new { ap.actualValue, ap.predictedValue, originalIndex = i })
.OrderByDescending(ap => ap.predictedValue) // important to sort descending by prediction
.ThenBy(ap => ap.originalIndex); // secondary sorts to ensure unambigious orders
var totalActualLosses = a.Sum();
double populationDelta = 1.0/a.Count;
double accumulatedPopulationPercentageSum = 0;
double accumulatedLossPercentageSum = 0;
double giniSum = 0.0;
foreach(var currentPair in all) {
accumulatedLossPercentageSum += (currentPair.actualValue/totalActualLosses);
accumulatedPopulationPercentageSum += populationDelta;
giniSum += accumulatedLossPercentageSum - accumulatedPopulationPercentageSum;
}
var gini = giniSum/a.Count;
return gini;
}
public static double GiniNormalized(this IList<double> a, IList<double> p) {
return a.Gini(p)/(a.Gini(a));
}\r\nLet me know if you have any questions about the code. Feel free to post your own ports of it to the languages of your choice.
", "messages": [ { "id": "4568", "postDate": "07/13/2011 13:49:34", "content": "Since this is a new metric to Kaggle, I thought I'd share the code we use to calculate it (in C#):
\r\npublic static double Gini(this IList<double> a, IList<double> p) {
if(a.Count != p.Count) {
throw new ArgumentException();
}
var all =
p.Zip(a, (pred, actual) => new { actualValue = actual, predictedValue = pred }) // (actual, prediction)
.Zip(Enumerable.Range(1, a.Count), (ap, i) => new { ap.actualValue, ap.predictedValue, originalIndex = i })
.OrderByDescending(ap => ap.predictedValue) // important to sort descending by prediction
.ThenBy(ap => ap.originalIndex); // secondary sorts to ensure unambigious orders
var totalActualLosses = a.Sum();
double populationDelta = 1.0/a.Count;
double accumulatedPopulationPercentageSum = 0;
double accumulatedLossPercentageSum = 0;
double giniSum = 0.0;
foreach(var currentPair in all) {
accumulatedLossPercentageSum += (currentPair.actualValue/totalActualLosses);
accumulatedPopulationPercentageSum += populationDelta;
giniSum += accumulatedLossPercentageSum - accumulatedPopulationPercentageSum;
}
var gini = giniSum/a.Count;
return gini;
}
public static double GiniNormalized(this IList<double> a, IList<double> p) {
return a.Gini(p)/(a.Gini(a));
}\r\nLet me know if you have any questions about the code. Feel free to post your own ports of it to the languages of your choice.
", "rawMarkdown": "", "votes": null }, { "id": "4573", "postDate": "07/13/2011 18:04:23", "content": "Hi Jeff!
\r\nI don't know C code, but I tried to read it and made some educated guesses in order to try and make an R function (good practice for me - as I generally hate reading code).
\r\nAnyway - can you tell me if I am on the right track:
\r\nnormalizedGini <- function(aa, pp){
\r\nGini <- function(a, p){
\r\n if(length(a) != length(p)) stop("Actual and Predicted need to be equal lengths!")
\r\n temp.df <- data.frame(actual = a, pred = p, range=c(1:length(a)))
\r\n temp.df <- temp.df[order(temp.df$pred, temp.df$range, decreasing=TRUE),]
\r\n population.delta <- 1 / length(a)
\r\n total.losses <- sum(a)
\r\n null.losses <- rep(population.delta, length(a)) # Hopefully is similar to accumulatedPopulationPercentageSum
\r\n accum.losses <- temp.df$actual / total.losses # Hopefully is similar to accumulatedLossPercentageSum
\r\n gini.sum <- cumsum(accum.losses - null.losses) # Not sure if this is having the same effect or not
\r\n sum(gini.sum)
\r\n}
\r\nGini(aa,pp) / Gini(aa,aa)
\r\n}
I get:
\r\n> normalizedGini(1:1000, 1:1000)", "rawMarkdown": "", "votes": null }, { "id": "4574", "postDate": "07/13/2011 18:58:11", "content": "
[1] 1\r\n> normalizedGini(1:1000, 99001:100000)
[1] 1\r\n> normalizedGini(1:1000, 1000:1)
[1] -1\r\n> normalizedGini(1:10000, runif(10000)) # Should be close to zero
[1] 0.01671304\r\n> normalizedGini(cars$Claim_Amount[1:1000000], cars$Var1[1:1000000]) #First million rows of training set
[1] -0.03384947
Thanks!
Hi Chris,
\r\nTwo observations:
\r\nYou may find this doc helpful.
\r\nhttp://www.rhinorisk.com/Publications/Gini%20Coefficients.pdf
Jeff, where do you get the perfect model?
\r\nFrom the graph, the perfect model should have a gini of 100 or the top left corner.
", "rawMarkdown": "", "votes": null }, { "id": "4578", "postDate": "07/13/2011 20:05:39", "content": "[quote=Sashi;4577]
\r\nJeff, where do you get the perfect model?
\r\n[/quote]
\r\nThe perfect model is the solution itself.
", "rawMarkdown": "", "votes": null }, { "id": "4579", "postDate": "07/13/2011 20:29:32", "content": "@Jeff:
\r\nThanks for posting the code. I will wait for the test cases to verify my implementation.
\r\nHere is my C++ implementation.
\r\ndouble Gini(const std::vector &a, decltype(a) &p) {
assert(a.size() == p.size());
struct K {double a, p;} k[a.size()];
for (auto i = 0; i != a.size(); ++i) k[i] = {a[i], p[i]};
std::stable_sort(k, k+a.size(), [](const K &a, const K &b) {return a.p > b.p;});
double accPopPercSum=0, accLossPercSum=0, giniSum=0, sum=0;
for (auto &i: a) sum += i;
for (auto &i: k) {
accLossPercSum += i.a/sum;
accPopPercSum += 1.0/a.size();
giniSum += accLossPercSum-accPopPercSum;
}
return giniSum/a.size();
}
double GiniNormalized(const std::vector &a, decltype(a) &p) {
return Gini(a, p)/Gini(a, a);
}\r\nstd::cout << GiniNormalized({3,1,2,4,0}, {3,2,0,4,1}) //returns 0.7
", "rawMarkdown": "", "votes": null }, { "id": "4580", "postDate": "07/13/2011 23:00:06", "content": "Here are some test vectors in unit test style:
\r\n[Test]", "rawMarkdown": "", "votes": null }, { "id": "4593", "postDate": "07/14/2011 09:28:01", "content": "
public void GiniTest() {
// Sanity check:
AssertGini(new[] { 1.0, 2, 3 }, new[] { 10.0, 20, 30 }, 0.111111111111111, 1.0);
AssertGini(new[] { 1.0, 2, 3 }, new[] { 30.0, 20, 10 }, -0.111111111111111, -1.0);
// Verify primary sort is descending on prediction and secondary sort is ascending based on index of original list
AssertGini(new[] { 1.0, 2, 3 }, new[] { 0.0, 0, 0 }, -0.111111111111111, -1.0);
AssertGini(new[] { 3.0, 2, 1 }, new[] { 0.0, 0, 0 }, 0.111111111111111, 1.0);
AssertGini(new[] { 1.0, 2, 4, 3 }, new[] { 0.0, 0, 0, 0 }, -0.1, -0.8);
AssertGini(new[] { 2.0, 1, 4, 3 }, new[] { 0.0, 0.0, 2, 1 }, 0.125, 1.0);
// Verified against Excel:
AssertGini(new[] { 0.0, 20, 40, 0, 10}, new[] {40.0, 40.0, 10.0, 5, 5}, 0.0, 0.0);
AssertGini(new[] { 40.0, 0, 20, 0, 10 }, new[] { 1000000.0, 40, 40, 5, 5 }, 0.17142857, 0.60 );
AssertGini(new[] { 40.0, 20, 10, 0, 0 }, new[] { 40.0, 20, 10, 0, 0 }, 0.28571429, 1.00);
AssertGini(new[] { 1.0, 1.0, 0.0, 1.0}, new[] {0.86, 0.26, 0.52, 0.32}, -0.04166667, -0.33333333);
}
private const double ErrorTolerance = 1e-6;
private static void AssertGini(IList<double> actuals, IList<double> predictions, double expectedGini, double expectedNormalizedGini ) {
var gini = actuals.Gini(predictions);
Assert.AreEqual(expectedGini, gini, ErrorTolerance);
var normalizedGini = actuals.GiniNormalized(predictions);
Assert.AreEqual(expectedNormalizedGini, normalizedGini, ErrorTolerance);
}
So my implementation was correct. Here is a C++ version for the test code:
\r\nvoid GiniTest() {
auto T = [](const std::vector<double> &a, decltype(a) p, double g, double n) {
auto E = [](double a, double b, double e=1e-6) {return fabs(a-b) <= e;};
assert(E(Gini(a, p), g) && E(GiniNormalized(a, p), n));
};
T({1, 2, 3}, {10, 20, 30}, 0.111111, 1);
T({1, 2, 3}, {30, 20, 10}, -0.111111, -1);
T({1, 2, 3}, {0, 0, 0}, -0.111111, -1);
T({3, 2, 1}, {0, 0, 0}, 0.111111, 1);
T({1, 2, 4, 3}, {0, 0, 0, 0}, -0.1, -0.8);
T({2, 1, 4, 3}, {0, 0, 2, 1}, 0.125, 1);
T({0, 20, 40, 0, 10}, {40, 40, 10, 5, 5}, 0, 0);
T({40, 0, 20, 0, 10}, {1000000, 40, 40, 5, 5}, 0.171428, 0.6);
T({40, 20, 10, 0, 0}, {40, 20, 10, 0, 0}, 0.285714, 1);
T({1, 1, 0, 1}, {0.86, 0.26, 0.52, 0.32}, -0.041666, -0.333333);
}",
"rawMarkdown": "",
"votes": null
},
{
"id": "4622",
"postDate": "07/15/2011 16:10:57",
"content": "Don't know if I understand this correctly, but can't you get rid of the accumulatedPopulationPercentageSum term and just subtract 0.5 from the final gini in Gini() function ? Intuitively it makes sense because you're removing the 0.5 area under the diagonal\r\n line.
", "rawMarkdown": "", "votes": null }, { "id": "4623", "postDate": "07/15/2011 16:29:28", "content": "[quote=B Yang;4622]
\r\nDon't know if I understand this correctly, but can't you get rid of the accumulatedPopulationPercentageSum term and just subtract 0.5 from the final gini in Gini() function ? Intuitively it makes sense because you're removing the 0.5 area under the diagonal\r\n line.
\r\n[/quote]
\r\nGood observation! The inner loop is subtracting each term of this sum:
\r\n\\[ \\frac{1}{n} + \\frac{2}{n} + \\ldots + \\frac{n-2}{n} + \\frac{n-1}{n} + \\frac{n}{n} = \\sum_{k=1}^n \\frac{k}{n} = \\frac{n+1}{2} \\]
\r\nThus, if you got rid of that term in the inner loop, you'd have to subtract out \\\\(\\frac{n+1}{2}\\\\) which would give you this equivalent code:
\r\npublic static double Gini(this IList<double> a, IList<double> p) {
if(a.Count != p.Count) {
throw new ArgumentException();
}
var all =
p.Zip(a, (pred, actual) => new { actualValue = actual, predictedValue = pred }) // (actual, prediction)
.Zip(Enumerable.Range(1, a.Count), (ap, i) => new { ap.actualValue, ap.predictedValue, originalIndex = i })
.OrderByDescending(ap => ap.predictedValue) // important to sort descending by prediction
.ThenBy(ap => ap.originalIndex); // secondary sorts to ensure unambigious orders
var totalActualLosses = a.Sum();
double accumulatedLossPercentageSum = 0;
double giniSum = 0.0;
foreach(var currentPair in all) {
accumulatedLossPercentageSum += (currentPair.actualValue/totalActualLosses);
giniSum += accumulatedLossPercentageSum;
}
// Subtract out the null model
giniSum -= (a.Count + 1.0)/2;
var gini = giniSum/a.Count;
return gini;
}
public static double GiniNormalized(this IList<double> a, IList<double> p) {
return a.Gini(p)/(a.Gini(a));
}",
"rawMarkdown": "",
"votes": null
},
{
"id": "4677",
"postDate": "07/21/2011 08:22:30",
"content": "Here is Chris' R function updated with Jeff's feedback. Thanks to you both.
\r\nnormalizedGini <- function(aa, pp) {", "rawMarkdown": "", "votes": null }, { "id": "4766", "postDate": "07/27/2011 13:40:19", "content": "
Gini <- function(a, p) {
if (length(a) != length(p)) stop("Actual and Predicted need to be equal lengths!")
temp.df <- data.frame(actual = a, pred = p, range=c(1:length(a)))
temp.df <- temp.df[order(-temp.df$pred, temp.df$range),]
population.delta <- 1 / length(a)
total.losses <- sum(a)
null.losses <- rep(population.delta, length(a)) # Hopefully is similar to accumulatedPopulationPercentageSum
accum.losses <- temp.df$actual / total.losses # Hopefully is similar to accumulatedLossPercentageSum
gini.sum <- cumsum(accum.losses - null.losses) # Not sure if this is having the same effect or not
sum(gini.sum) / length(a)
}
Gini(aa,pp) / Gini(aa,aa)
}
Hi Sami,
\r\nI was wondering if you could share the whole C++ file (with a main) ? I should probably fill this gap by myself, but I haven't written C++ in more years than I want to acknowledge :P
\r\nThanks,
\r\nJose
", "rawMarkdown": "", "votes": null }, { "id": "4767", "postDate": "07/27/2011 13:45:10", "content": "Jeff, what is a and p? :-)
", "rawMarkdown": "", "votes": null }, { "id": "4769", "postDate": "07/27/2011 14:13:02", "content": "[quote=Jose ;4767]
\r\nJeff, what is a and p? :-)
\r\n[/quote]
\r\nactual values (solution) and predicted values (your submission) respectively
", "rawMarkdown": "", "votes": null }, { "id": "5183", "postDate": "08/19/2011 16:16:42", "content": "which version of Gcc did you use to compile your code?
\r\nI could not compile yours with gcc4.5
\r\n\r\n
Thanks!
", "rawMarkdown": "", "votes": null }, { "id": "5344", "postDate": "08/28/2011 20:18:32", "content": "#include <vector>
\r\n#include <algorithm>
\r\n#include <assert.h>
\r\n
\r\ndouble Gini(const std::vector<double> &a, decltype(a) &p) {
\r\n assert(a.size() == p.size());
\r\n
\r\n // k[i] = { a[i], p[i] };
\r\n struct K {double a, p;} k[a.size()];
\r\n for (auto i = 0; i != a.size(); ++i) k[i] = {a[i], p[i]};
\r\n
\r\n // sort(k) by descending p
\r\n std::stable_sort(k, k+a.size(), [](const K &a, const K &b) {return a.p > b.p;});
\r\n
\r\n double accPopPercSum=0, accLossPercSum=0, giniSum=0, sum=0;
\r\n
\r\n // sum = accum(a);
\r\n // or total actual sum
\r\n for (auto &i: a) sum += i;
\r\n
\r\n // accLossPercSum = accum(i.a / sum);
\r\n// for (auto &i: k) {
\r\n for (unsigned int fu = 0; fu < a.size(); ++fu) {
\r\n struct K& i = k[fu];
\r\n accLossPercSum += i.a/sum;
\r\n accPopPercSum += 1.0/a.size();
\r\n giniSum += accLossPercSum - accPopPercSum;
\r\n }
\r\n
\r\n return giniSum/a.size();
\r\n}
\r\n
\r\ndouble GiniNormalized(const std::vector<double> &a, decltype(a) &p) {
\r\n return Gini(a, p)/Gini(a, a);
\r\n}
void GiniTest() {
\r\n auto T = [](const std::vector<double> &a, decltype(a) p, double g, double n) {
\r\n auto E = [](double a, double b, double e=1e-6) {return fabs(a-b) <= e;};
\r\n assert(E(Gini(a, p), g) && E(GiniNormalized(a, p), n));
\r\n };
\r\n T({1, 2, 3}, {10, 20, 30}, 0.111111, 1);
\r\n T({1, 2, 3}, {30, 20, 10}, -0.111111, -1);
\r\n T({1, 2, 3}, {0, 0, 0}, -0.111111, -1);
\r\n T({3, 2, 1}, {0, 0, 0}, 0.111111, 1);
\r\n T({1, 2, 4, 3}, {0, 0, 0, 0}, -0.1, -0.8);
\r\n T({2, 1, 4, 3}, {0, 0, 2, 1}, 0.125, 1);
\r\n T({0, 20, 40, 0, 10}, {40, 40, 10, 5, 5}, 0, 0);
\r\n T({40, 0, 20, 0, 10}, {1000000, 40, 40, 5, 5}, 0.171428, 0.6);
\r\n T({40, 20, 10, 0, 0}, {40, 20, 10, 0, 0}, 0.285714, 1);
\r\n T({1, 1, 0, 1}, {0.86, 0.26, 0.52, 0.32}, -0.041666, -0.333333);
\r\n}
\r\n
\r\nint main(int ac, char** av)
\r\n{
\r\n GiniTest();
\r\n}
compile with:
\r\ng++-4.6 -std=c++0x gini.cc
", "rawMarkdown": "", "votes": null }, { "id": "5345", "postDate": "08/28/2011 20:38:23", "content": "note that you can move the division outside the loop:
\r\n for (unsigned int fu = 0; fu < a.size(); ++fu) {
\r\n struct K& i = k[fu];
\r\n accLossPercSum += i.a;
\r\n giniSum += accLossPercSum;
\r\n }
\r\n
\r\n giniSum = giniSum / sum - (a.size() + 1.0) / 2;
Anyone have a Java version they care to share? I've tried to understand these "foreign" language examples but not getting to far. I fear I will produce my solution but be unable to submit it for consideration....
", "rawMarkdown": "", "votes": null }, { "id": "5897", "postDate": "10/04/2011 21:25:59", "content": "\r\n\r\ndef gini(actual, pred, cmpcol = 0, sortcol = 1):\r\n
assert( len(actual) == len(pred) )
all = np.asarray(np.c_[ actual, pred, np.arange(len(actual)) ], dtype=np.float)
all = all[ np.lexsort((all[:,2], -1*all[:,1])) ]
totalLosses = all[:,0].sum()
giniSum = all[:,0].cumsum().sum() / totalLosses
giniSum -= (len(actual) + 1) / 2.
return giniSum / len(actual)
def gini_normalized(a, p):
return gini(a, p) / gini(a, a)
def test_gini():
def fequ(a,b):
return abs( a -b) < 1e-6
def T(a, p, g, n):
assert( fequ(gini(a,p), g) )
assert( fequ(gini_normalized(a,p), n) )
T([1, 2, 3], [10, 20, 30], 0.111111, 1)
T([1, 2, 3], [30, 20, 10], -0.111111, -1)
T([1, 2, 3], [0, 0, 0], -0.111111, -1)
T([3, 2, 1], [0, 0, 0], 0.111111, 1)
T([1, 2, 4, 3], [0, 0, 0, 0], -0.1, -0.8)
T([2, 1, 4, 3], [0, 0, 2, 1], 0.125, 1)
T([0, 20, 40, 0, 10], [40, 40, 10, 5, 5], 0, 0)
T([40, 0, 20, 0, 10], [1000000, 40, 40, 5, 5], 0.171428,
0.6)
T([40, 20, 10, 0, 0], [40, 20, 10, 0, 0], 0.285714, 1)
T([1, 1, 0, 1], [0.86, 0.26, 0.52, 0.32], -0.041666,
-0.333333)
Python code for anyone interested. It assumes you're working in numpy, because if you're not it would probably take forever and a day to finish.
", "rawMarkdown": "", "votes": null }, { "id": "44646", "postDate": "05/16/2014 01:09:13", "content": "Has anyone in the community figured out R code to create the plot of the Gini. I have been trying like crazy but I am not sure if I am doing it right.
", "rawMarkdown": "", "votes": null }, { "id": "226658", "postDate": "10/02/2017 22:14:09", "content": "", "rawMarkdown": "", "votes": null }, { "id": "235814", "postDate": "10/26/2017 04:40:09", "content": "You likely would want to use a class to accumulate the values but here is a working Java ver:
\n\n public static double getGini2(List<Double> a, List<Double> p) {\n\n if(a.size() != p.size())\n throw new IllegalArgumentException(\"a.sz=\"+a.size()+\" != p.sz=\"+p.size());\n\n List<double[]> list=new ArrayList<>();\n\n int sz=a.size();\n double sum=0;\n for (int i = 0; i != sz; ++i) {\n list.add(new double[] {a.get(i), p.get(i)}); // add tuple {act, pred}\n sum+=a.get(i); // while we're at it...\n }\n\n Comparator<double[]> compEstDsc=new Comparator<double[]>() {\n @Override\n public int compare(double[] o1, double[] o2) {\n double v1=o1[1]; // only use prediction\n double v2=o2[1];\n return UtilComparator.compareDoubleDesc(v1, v2);\n }\n };\n list.sort(compEstDsc);\n\n double accPopPercSum=0;\n double accLossPercSum=0;\n double giniSum=0;\n\n for (int i = 0; i < sz; ++i) {\n\n double[] a0p1 = list.get(i);\n\n accLossPercSum += a0p1[0]/sum; // only use act\n\n accPopPercSum += 1.0/sz;\n\n giniSum += accLossPercSum - accPopPercSum;\n }\n\n return giniSum/sz;\n}\n\npublic static double getGini2Normalized(List<Double> a, List<Double> p) {\n return getGini2(a, p)/getGini2(a, a);\n}\nThanks for posting this! Looks like the parameters cmpcol sortcol are not being used ?
", "rawMarkdown": "Thanks for posting this! Looks like the parameters cmpcol sortcol are not being used ?", "votes": null }, { "id": "247248", "postDate": "11/22/2017 16:47:25", "content": "Here's a Python implementation with TensorFlow. Expects actual and pred to be a (1, N) tensor.
def gini_normalized(actual, pred):\n n = tf.shape(actual)[1]\n indices = tf.nn.top_k(pred, k=n)[1][0]\n actual_sorted = tf.gather(actual[0], indices)\n cost = tf.reduce_sum(actual_sorted)\n loss_proportion = tf.cumsum(actual_sorted) / cost\n null_model = tf.to_float(tf.range(1, n + 1)) / tf.to_float(n)\n g = tf.subtract(loss_proportion, null_model)\n g = tf.reduce_sum(g) / tf.to_float(n)\n g /= (1.0 - tf.reduce_mean(actual)) / 2.0\n return g\nHi Jeff!
\r\nI don't know C code, but I tried to read it and made some educated guesses in order to try and make an R function (good practice for me - as I generally hate reading code).
\r\nAnyway - can you tell me if I am on the right track:
\r\nnormalizedGini <- function(aa, pp){
\r\nGini <- function(a, p){
\r\n if(length(a) != length(p)) stop("Actual and Predicted need to be equal lengths!")
\r\n temp.df <- data.frame(actual = a, pred = p, range=c(1:length(a)))
\r\n temp.df <- temp.df[order(temp.df$pred, temp.df$range, decreasing=TRUE),]
\r\n population.delta <- 1 / length(a)
\r\n total.losses <- sum(a)
\r\n null.losses <- rep(population.delta, length(a)) # Hopefully is similar to accumulatedPopulationPercentageSum
\r\n accum.losses <- temp.df$actual / total.losses # Hopefully is similar to accumulatedLossPercentageSum
\r\n gini.sum <- cumsum(accum.losses - null.losses) # Not sure if this is having the same effect or not
\r\n sum(gini.sum)
\r\n}
\r\nGini(aa,pp) / Gini(aa,aa)
\r\n}
I get:
\r\n> normalizedGini(1:1000, 1:1000)", "votes": null, "replies": [] }, { "id": 4574, "author_name": "jeffmoser", "author_url": "", "post_date": "07/13/2011 18:58:11", "content": "
[1] 1\r\n> normalizedGini(1:1000, 99001:100000)
[1] 1\r\n> normalizedGini(1:1000, 1000:1)
[1] -1\r\n> normalizedGini(1:10000, runif(10000)) # Should be close to zero
[1] 0.01671304\r\n> normalizedGini(cars$Claim_Amount[1:1000000], cars$Var1[1:1000000]) #First million rows of training set
[1] -0.03384947
Thanks!
Hi Chris,
\r\nTwo observations:
\r\nYou may find this doc helpful.
\r\nhttp://www.rhinorisk.com/Publications/Gini%20Coefficients.pdf
Jeff, where do you get the perfect model?
\r\nFrom the graph, the perfect model should have a gini of 100 or the top left corner.
", "votes": null, "replies": [] }, { "id": 4578, "author_name": "jeffmoser", "author_url": "", "post_date": "07/13/2011 20:05:39", "content": "[quote=Sashi;4577]
\r\nJeff, where do you get the perfect model?
\r\n[/quote]
\r\nThe perfect model is the solution itself.
", "votes": null, "replies": [] }, { "id": 4579, "author_name": "sami6019", "author_url": "", "post_date": "07/13/2011 20:29:32", "content": "@Jeff:
\r\nThanks for posting the code. I will wait for the test cases to verify my implementation.
\r\nHere is my C++ implementation.
\r\ndouble Gini(const std::vector &a, decltype(a) &p) {
assert(a.size() == p.size());
struct K {double a, p;} k[a.size()];
for (auto i = 0; i != a.size(); ++i) k[i] = {a[i], p[i]};
std::stable_sort(k, k+a.size(), [](const K &a, const K &b) {return a.p > b.p;});
double accPopPercSum=0, accLossPercSum=0, giniSum=0, sum=0;
for (auto &i: a) sum += i;
for (auto &i: k) {
accLossPercSum += i.a/sum;
accPopPercSum += 1.0/a.size();
giniSum += accLossPercSum-accPopPercSum;
}
return giniSum/a.size();
}
double GiniNormalized(const std::vector &a, decltype(a) &p) {
return Gini(a, p)/Gini(a, a);
}\r\nstd::cout << GiniNormalized({3,1,2,4,0}, {3,2,0,4,1}) //returns 0.7
", "votes": null, "replies": [] }, { "id": 4580, "author_name": "jeffmoser", "author_url": "", "post_date": "07/13/2011 23:00:06", "content": "Here are some test vectors in unit test style:
\r\n[Test]", "votes": null, "replies": [] }, { "id": 4593, "author_name": "sami6019", "author_url": "", "post_date": "07/14/2011 09:28:01", "content": "
public void GiniTest() {
// Sanity check:
AssertGini(new[] { 1.0, 2, 3 }, new[] { 10.0, 20, 30 }, 0.111111111111111, 1.0);
AssertGini(new[] { 1.0, 2, 3 }, new[] { 30.0, 20, 10 }, -0.111111111111111, -1.0);
// Verify primary sort is descending on prediction and secondary sort is ascending based on index of original list
AssertGini(new[] { 1.0, 2, 3 }, new[] { 0.0, 0, 0 }, -0.111111111111111, -1.0);
AssertGini(new[] { 3.0, 2, 1 }, new[] { 0.0, 0, 0 }, 0.111111111111111, 1.0);
AssertGini(new[] { 1.0, 2, 4, 3 }, new[] { 0.0, 0, 0, 0 }, -0.1, -0.8);
AssertGini(new[] { 2.0, 1, 4, 3 }, new[] { 0.0, 0.0, 2, 1 }, 0.125, 1.0);
// Verified against Excel:
AssertGini(new[] { 0.0, 20, 40, 0, 10}, new[] {40.0, 40.0, 10.0, 5, 5}, 0.0, 0.0);
AssertGini(new[] { 40.0, 0, 20, 0, 10 }, new[] { 1000000.0, 40, 40, 5, 5 }, 0.17142857, 0.60 );
AssertGini(new[] { 40.0, 20, 10, 0, 0 }, new[] { 40.0, 20, 10, 0, 0 }, 0.28571429, 1.00);
AssertGini(new[] { 1.0, 1.0, 0.0, 1.0}, new[] {0.86, 0.26, 0.52, 0.32}, -0.04166667, -0.33333333);
}
private const double ErrorTolerance = 1e-6;
private static void AssertGini(IList<double> actuals, IList<double> predictions, double expectedGini, double expectedNormalizedGini ) {
var gini = actuals.Gini(predictions);
Assert.AreEqual(expectedGini, gini, ErrorTolerance);
var normalizedGini = actuals.GiniNormalized(predictions);
Assert.AreEqual(expectedNormalizedGini, normalizedGini, ErrorTolerance);
}
So my implementation was correct. Here is a C++ version for the test code:
\r\nvoid GiniTest() {
auto T = [](const std::vector<double> &a, decltype(a) p, double g, double n) {
auto E = [](double a, double b, double e=1e-6) {return fabs(a-b) <= e;};
assert(E(Gini(a, p), g) && E(GiniNormalized(a, p), n));
};
T({1, 2, 3}, {10, 20, 30}, 0.111111, 1);
T({1, 2, 3}, {30, 20, 10}, -0.111111, -1);
T({1, 2, 3}, {0, 0, 0}, -0.111111, -1);
T({3, 2, 1}, {0, 0, 0}, 0.111111, 1);
T({1, 2, 4, 3}, {0, 0, 0, 0}, -0.1, -0.8);
T({2, 1, 4, 3}, {0, 0, 2, 1}, 0.125, 1);
T({0, 20, 40, 0, 10}, {40, 40, 10, 5, 5}, 0, 0);
T({40, 0, 20, 0, 10}, {1000000, 40, 40, 5, 5}, 0.171428, 0.6);
T({40, 20, 10, 0, 0}, {40, 20, 10, 0, 0}, 0.285714, 1);
T({1, 1, 0, 1}, {0.86, 0.26, 0.52, 0.32}, -0.041666, -0.333333);
}",
"votes": null,
"replies": []
},
{
"id": 4622,
"author_name": "byang1",
"author_url": "",
"post_date": "07/15/2011 16:10:57",
"content": "Don't know if I understand this correctly, but can't you get rid of the accumulatedPopulationPercentageSum term and just subtract 0.5 from the final gini in Gini() function ? Intuitively it makes sense because you're removing the 0.5 area under the diagonal\r\n line.
", "votes": null, "replies": [] }, { "id": 4623, "author_name": "jeffmoser", "author_url": "", "post_date": "07/15/2011 16:29:28", "content": "[quote=B Yang;4622]
\r\nDon't know if I understand this correctly, but can't you get rid of the accumulatedPopulationPercentageSum term and just subtract 0.5 from the final gini in Gini() function ? Intuitively it makes sense because you're removing the 0.5 area under the diagonal\r\n line.
\r\n[/quote]
\r\nGood observation! The inner loop is subtracting each term of this sum:
\r\n\\[ \\frac{1}{n} + \\frac{2}{n} + \\ldots + \\frac{n-2}{n} + \\frac{n-1}{n} + \\frac{n}{n} = \\sum_{k=1}^n \\frac{k}{n} = \\frac{n+1}{2} \\]
\r\nThus, if you got rid of that term in the inner loop, you'd have to subtract out \\\\(\\frac{n+1}{2}\\\\) which would give you this equivalent code:
\r\npublic static double Gini(this IList<double> a, IList<double> p) {
if(a.Count != p.Count) {
throw new ArgumentException();
}
var all =
p.Zip(a, (pred, actual) => new { actualValue = actual, predictedValue = pred }) // (actual, prediction)
.Zip(Enumerable.Range(1, a.Count), (ap, i) => new { ap.actualValue, ap.predictedValue, originalIndex = i })
.OrderByDescending(ap => ap.predictedValue) // important to sort descending by prediction
.ThenBy(ap => ap.originalIndex); // secondary sorts to ensure unambigious orders
var totalActualLosses = a.Sum();
double accumulatedLossPercentageSum = 0;
double giniSum = 0.0;
foreach(var currentPair in all) {
accumulatedLossPercentageSum += (currentPair.actualValue/totalActualLosses);
giniSum += accumulatedLossPercentageSum;
}
// Subtract out the null model
giniSum -= (a.Count + 1.0)/2;
var gini = giniSum/a.Count;
return gini;
}
public static double GiniNormalized(this IList<double> a, IList<double> p) {
return a.Gini(p)/(a.Gini(a));
}",
"votes": null,
"replies": []
},
{
"id": 4677,
"author_name": "markwaddle",
"author_url": "",
"post_date": "07/21/2011 08:22:30",
"content": "Here is Chris' R function updated with Jeff's feedback. Thanks to you both.
\r\nnormalizedGini <- function(aa, pp) {", "votes": null, "replies": [] }, { "id": 4766, "author_name": "tropics", "author_url": "", "post_date": "07/27/2011 13:40:19", "content": "
Gini <- function(a, p) {
if (length(a) != length(p)) stop("Actual and Predicted need to be equal lengths!")
temp.df <- data.frame(actual = a, pred = p, range=c(1:length(a)))
temp.df <- temp.df[order(-temp.df$pred, temp.df$range),]
population.delta <- 1 / length(a)
total.losses <- sum(a)
null.losses <- rep(population.delta, length(a)) # Hopefully is similar to accumulatedPopulationPercentageSum
accum.losses <- temp.df$actual / total.losses # Hopefully is similar to accumulatedLossPercentageSum
gini.sum <- cumsum(accum.losses - null.losses) # Not sure if this is having the same effect or not
sum(gini.sum) / length(a)
}
Gini(aa,pp) / Gini(aa,aa)
}
Hi Sami,
\r\nI was wondering if you could share the whole C++ file (with a main) ? I should probably fill this gap by myself, but I haven't written C++ in more years than I want to acknowledge :P
\r\nThanks,
\r\nJose
", "votes": null, "replies": [] }, { "id": 4767, "author_name": "tropics", "author_url": "", "post_date": "07/27/2011 13:45:10", "content": "Jeff, what is a and p? :-)
", "votes": null, "replies": [] }, { "id": 4769, "author_name": "jeffmoser", "author_url": "", "post_date": "07/27/2011 14:13:02", "content": "[quote=Jose ;4767]
\r\nJeff, what is a and p? :-)
\r\n[/quote]
\r\nactual values (solution) and predicted values (your submission) respectively
", "votes": null, "replies": [] }, { "id": 5183, "author_name": "woshialex", "author_url": "", "post_date": "08/19/2011 16:16:42", "content": "which version of Gcc did you use to compile your code?
\r\nI could not compile yours with gcc4.5
\r\n\r\n
Thanks!
", "votes": null, "replies": [] }, { "id": 5344, "author_name": "texane", "author_url": "", "post_date": "08/28/2011 20:18:32", "content": "#include <vector>
\r\n#include <algorithm>
\r\n#include <assert.h>
\r\n
\r\ndouble Gini(const std::vector<double> &a, decltype(a) &p) {
\r\n assert(a.size() == p.size());
\r\n
\r\n // k[i] = { a[i], p[i] };
\r\n struct K {double a, p;} k[a.size()];
\r\n for (auto i = 0; i != a.size(); ++i) k[i] = {a[i], p[i]};
\r\n
\r\n // sort(k) by descending p
\r\n std::stable_sort(k, k+a.size(), [](const K &a, const K &b) {return a.p > b.p;});
\r\n
\r\n double accPopPercSum=0, accLossPercSum=0, giniSum=0, sum=0;
\r\n
\r\n // sum = accum(a);
\r\n // or total actual sum
\r\n for (auto &i: a) sum += i;
\r\n
\r\n // accLossPercSum = accum(i.a / sum);
\r\n// for (auto &i: k) {
\r\n for (unsigned int fu = 0; fu < a.size(); ++fu) {
\r\n struct K& i = k[fu];
\r\n accLossPercSum += i.a/sum;
\r\n accPopPercSum += 1.0/a.size();
\r\n giniSum += accLossPercSum - accPopPercSum;
\r\n }
\r\n
\r\n return giniSum/a.size();
\r\n}
\r\n
\r\ndouble GiniNormalized(const std::vector<double> &a, decltype(a) &p) {
\r\n return Gini(a, p)/Gini(a, a);
\r\n}
void GiniTest() {
\r\n auto T = [](const std::vector<double> &a, decltype(a) p, double g, double n) {
\r\n auto E = [](double a, double b, double e=1e-6) {return fabs(a-b) <= e;};
\r\n assert(E(Gini(a, p), g) && E(GiniNormalized(a, p), n));
\r\n };
\r\n T({1, 2, 3}, {10, 20, 30}, 0.111111, 1);
\r\n T({1, 2, 3}, {30, 20, 10}, -0.111111, -1);
\r\n T({1, 2, 3}, {0, 0, 0}, -0.111111, -1);
\r\n T({3, 2, 1}, {0, 0, 0}, 0.111111, 1);
\r\n T({1, 2, 4, 3}, {0, 0, 0, 0}, -0.1, -0.8);
\r\n T({2, 1, 4, 3}, {0, 0, 2, 1}, 0.125, 1);
\r\n T({0, 20, 40, 0, 10}, {40, 40, 10, 5, 5}, 0, 0);
\r\n T({40, 0, 20, 0, 10}, {1000000, 40, 40, 5, 5}, 0.171428, 0.6);
\r\n T({40, 20, 10, 0, 0}, {40, 20, 10, 0, 0}, 0.285714, 1);
\r\n T({1, 1, 0, 1}, {0.86, 0.26, 0.52, 0.32}, -0.041666, -0.333333);
\r\n}
\r\n
\r\nint main(int ac, char** av)
\r\n{
\r\n GiniTest();
\r\n}
compile with:
\r\ng++-4.6 -std=c++0x gini.cc
", "votes": null, "replies": [] }, { "id": 5345, "author_name": "texane", "author_url": "", "post_date": "08/28/2011 20:38:23", "content": "note that you can move the division outside the loop:
\r\n for (unsigned int fu = 0; fu < a.size(); ++fu) {
\r\n struct K& i = k[fu];
\r\n accLossPercSum += i.a;
\r\n giniSum += accLossPercSum;
\r\n }
\r\n
\r\n giniSum = giniSum / sum - (a.size() + 1.0) / 2;
Anyone have a Java version they care to share? I've tried to understand these "foreign" language examples but not getting to far. I fear I will produce my solution but be unable to submit it for consideration....
", "votes": null, "replies": [] }, { "id": 5897, "author_name": "patrick7", "author_url": "", "post_date": "10/04/2011 21:25:59", "content": "\r\n\r\ndef gini(actual, pred, cmpcol = 0, sortcol = 1):\r\n
assert( len(actual) == len(pred) )
all = np.asarray(np.c_[ actual, pred, np.arange(len(actual)) ], dtype=np.float)
all = all[ np.lexsort((all[:,2], -1*all[:,1])) ]
totalLosses = all[:,0].sum()
giniSum = all[:,0].cumsum().sum() / totalLosses
giniSum -= (len(actual) + 1) / 2.
return giniSum / len(actual)
def gini_normalized(a, p):
return gini(a, p) / gini(a, a)
def test_gini():
def fequ(a,b):
return abs( a -b) < 1e-6
def T(a, p, g, n):
assert( fequ(gini(a,p), g) )
assert( fequ(gini_normalized(a,p), n) )
T([1, 2, 3], [10, 20, 30], 0.111111, 1)
T([1, 2, 3], [30, 20, 10], -0.111111, -1)
T([1, 2, 3], [0, 0, 0], -0.111111, -1)
T([3, 2, 1], [0, 0, 0], 0.111111, 1)
T([1, 2, 4, 3], [0, 0, 0, 0], -0.1, -0.8)
T([2, 1, 4, 3], [0, 0, 2, 1], 0.125, 1)
T([0, 20, 40, 0, 10], [40, 40, 10, 5, 5], 0, 0)
T([40, 0, 20, 0, 10], [1000000, 40, 40, 5, 5], 0.171428,
0.6)
T([40, 20, 10, 0, 0], [40, 20, 10, 0, 0], 0.285714, 1)
T([1, 1, 0, 1], [0.86, 0.26, 0.52, 0.32], -0.041666,
-0.333333)
Python code for anyone interested. It assumes you're working in numpy, because if you're not it would probably take forever and a day to finish.
", "votes": null, "replies": [ { "id": 242481, "author_name": "ptolemy", "author_url": "", "post_date": "11/11/2017 17:33:17", "content": "Thanks for posting this! Looks like the parameters cmpcol sortcol are not being used ?
", "votes": null, "replies": [] } ] }, { "id": 44646, "author_name": "davidbess0", "author_url": "", "post_date": "05/16/2014 01:09:13", "content": "Has anyone in the community figured out R code to create the plot of the Gini. I have been trying like crazy but I am not sure if I am doing it right.
", "votes": null, "replies": [] }, { "id": 226658, "author_name": "jacekpoplawski", "author_url": "", "post_date": "10/02/2017 22:14:09", "content": "", "votes": null, "replies": [] }, { "id": 235814, "author_name": "rgwood", "author_url": "", "post_date": "10/26/2017 04:40:09", "content": "You likely would want to use a class to accumulate the values but here is a working Java ver:
\n\n public static double getGini2(List<Double> a, List<Double> p) {\n\n if(a.size() != p.size())\n throw new IllegalArgumentException(\"a.sz=\"+a.size()+\" != p.sz=\"+p.size());\n\n List<double[]> list=new ArrayList<>();\n\n int sz=a.size();\n double sum=0;\n for (int i = 0; i != sz; ++i) {\n list.add(new double[] {a.get(i), p.get(i)}); // add tuple {act, pred}\n sum+=a.get(i); // while we're at it...\n }\n\n Comparator<double[]> compEstDsc=new Comparator<double[]>() {\n @Override\n public int compare(double[] o1, double[] o2) {\n double v1=o1[1]; // only use prediction\n double v2=o2[1];\n return UtilComparator.compareDoubleDesc(v1, v2);\n }\n };\n list.sort(compEstDsc);\n\n double accPopPercSum=0;\n double accLossPercSum=0;\n double giniSum=0;\n\n for (int i = 0; i < sz; ++i) {\n\n double[] a0p1 = list.get(i);\n\n accLossPercSum += a0p1[0]/sum; // only use act\n\n accPopPercSum += 1.0/sz;\n\n giniSum += accLossPercSum - accPopPercSum;\n }\n\n return giniSum/sz;\n}\n\npublic static double getGini2Normalized(List<Double> a, List<Double> p) {\n return getGini2(a, p)/getGini2(a, a);\n}\nHere's a Python implementation with TensorFlow. Expects actual and pred to be a (1, N) tensor.
def gini_normalized(actual, pred):\n n = tf.shape(actual)[1]\n indices = tf.nn.top_k(pred, k=n)[1][0]\n actual_sorted = tf.gather(actual[0], indices)\n cost = tf.reduce_sum(actual_sorted)\n loss_proportion = tf.cumsum(actual_sorted) / cost\n null_model = tf.to_float(tf.range(1, n + 1)) / tf.to_float(n)\n g = tf.subtract(loss_proportion, null_model)\n g = tf.reduce_sum(g) / tf.to_float(n)\n g /= (1.0 - tf.reduce_mean(actual)) / 2.0\n return g\n