Hi Everyone! Thank you for your participation! From March 9-13, we’ll be hosting an AMA with the competition hosts, here in this thread. The Environmental Insights Explorer team will be judging your submissions, and this AMA is an excellent opportunity to get to know the team behind the competition and ask any questions that you might have.
\n\nFeel free to ask questions about:\n- Earth Engine\n- Environmental Insights Explorer\n- Emissions factors, and other related subject matters
\n\nPlease ask your questions in the thread below!
\n\nThe team will pop in on March 16 to answer as many as they can.
\n\nHi Everyone! Thank you for your participation! From March 9-13, we’ll be hosting an AMA with the competition hosts, here in this thread. The Environmental Insights Explorer team will be judging your submissions, and this AMA is an excellent opportunity to get to know the team behind the competition and ask any questions that you might have.
\n\nFeel free to ask questions about:\n- Earth Engine\n- Environmental Insights Explorer\n- Emissions factors, and other related subject matters
\n\nPlease ask your questions in the thread below!
\n\nThe team will pop in on March 16 to answer as many as they can.
\n\nHi,
\n\ni'm not very familiar with Puerto Rico. I assume there is little industrial activity and few other sources of NO2 emission other then power plants. The satellite images of troposphere indicate there is also emission at other locations then fossil fuel power plants. Any suggestions how to look at this?
how accurate would you want a prediction to be to be useful for your purpose? Any thoughts on this?
Thanks,
", "rawMarkdown": "Hi,\n\n- i'm not very familiar with Puerto Rico. I assume there is little industrial activity and few other sources of NO2 emission other then power plants. The satellite images of troposphere indicate there is also emission at other locations then fossil fuel power plants. Any suggestions how to look at this? \n\n- how accurate would you want a prediction to be to be useful for your purpose? Any thoughts on this? \n\nThanks, ", "votes": 3, "replies": [ { "id": 769296, "postDate": "2020-03-11T18:33:10.803Z", "content": "A lot of emissions come for traffic too
", "rawMarkdown": "A lot of emissions come for traffic too" }, { "id": 769520, "postDate": "2020-03-12T01:41:57.967Z", "content": "I have the same question, consider anomalies values, Can I prioritize some industrial activity or region ?
", "rawMarkdown": "I have the same question, consider anomalies values, Can I prioritize some industrial activity or region ?" }, { "id": 781106, "postDate": "2020-03-20T23:27:41.397Z", "content": "Sorry, I think the IEEE article you mentioned is missing, isn't it?
", "rawMarkdown": "Sorry, I think the IEEE article you mentioned is missing, isn't it?" }, { "id": 783895, "postDate": "2020-03-23T19:13:08.150Z", "content": "Thanks for catching that - my mistake. I updated the post. Here's a link to the article for completeness.
", "rawMarkdown": "Thanks for catching that - my mistake. I updated the post. Here's a [link](https://spectrum.ieee.org/energy/policy/rebuilding-puerto-ricos-power-grid-the-inside-story) to the article for completeness. " } ] }, { "id": 767789, "postDate": "2020-03-10T05:01:49.567Z", "content": "How to convert NO2 column number density (mol/m2) to NOx emission (ton)?
", "rawMarkdown": "How to convert NO2 column number density (mol/m2) to NOx emission (ton)?", "votes": 4, "replies": [ { "id": 768035, "postDate": "2020-03-10T11:47:57.713Z", "content": "Hey, I think it is actually pretty much straight forward. You need to calculate the factor to convert pixel(image shape) into longitudinal and latitudinal units (image bounding box) and then use the haversine formula (check the link below) to convert these into meters.
\n\nhttps://en.wikipedia.org/wiki/Haversine_formula
\n\nNow you have the height/width in meters per pixel (haven't calculated it yet so I cannot tell if the pixels are square).
\n\nNow you get the area (pixels) you are interested in, take the sum of the n02 column number density and multiply it by the area (m^2). Now you have the sum of n02 in mol for the whole area. The last step ist to convert mol to gram (or ton). The molar mass of n02 is 46,0055 g/mol (at least that's what google says).
\n\nI'm currently writing a function for that issue, when I'm done I will share it here. \nIf I'm mistaken somewhere please correct me :).
", "rawMarkdown": "Hey, I think it is actually pretty much straight forward. You need to calculate the factor to convert pixel(image shape) into longitudinal and latitudinal units (image bounding box) and then use the haversine formula (check the link below) to convert these into meters.\n\nhttps://en.wikipedia.org/wiki/Haversine_formula\n\nNow you have the height/width in meters per pixel (haven't calculated it yet so I cannot tell if the pixels are square).\n\nNow you get the area (pixels) you are interested in, take the sum of the n02 column number density and multiply it by the area (m^2). Now you have the sum of n02 in mol for the whole area. The last step ist to convert mol to gram (or ton). The molar mass of n02 is 46,0055 g/mol (at least that's what google says).\n\nI'm currently writing a function for that issue, when I'm done I will share it here. \nIf I'm mistaken somewhere please correct me :). ", "votes": 1 }, { "id": 769315, "postDate": "2020-03-11T18:57:48.447Z", "content": "A couple of weeks ago I would have said something similar (less accurate, probably). However, with just the slight idea I have now, the (big) problem in my opinion is that NOx chemistry is very dynamic and satellites only can take a picture (the NO2 in mol/m2) at a given moment of what happened a while ago far down (the lb or ton NOx emitted at the power plant); for instance, NO2 has a variable chemical lifetime of hours, and is also affected by wind among other things.\nThus, I subscribe the question to see if the hosts give us some clue.
", "rawMarkdown": "A couple of weeks ago I would have said something similar (less accurate, probably). However, with just the slight idea I have now, the (big) problem in my opinion is that NOx chemistry is very dynamic and satellites only can take a picture (the NO2 in mol/m2) at a given moment of what happened a while ago far down (the lb or ton NOx emitted at the power plant); for instance, NO2 has a variable chemical lifetime of hours, and is also affected by wind among other things.\nThus, I subscribe the question to see if the hosts give us some clue.", "votes": 2 }, { "id": 781104, "postDate": "2020-03-20T23:23:41.100Z", "content": "This is one of the cruxes of the challenge, and we are excited to learn about how the Kaggle community approaches this question. There are several ways to correlate this data. A correlation between fuel consumption at a power plant, electricity produced and airborne pollution might produce a useful model of emissions. NO2 only lasts for about 2 days in the lower atmosphere before breaking down into other products. The actual rate of breakdown will depend on weather and other factors. You may want to use that assumption in your model.
", "rawMarkdown": "This is one of the cruxes of the challenge, and we are excited to learn about how the Kaggle community approaches this question. There are several ways to correlate this data. A correlation between fuel consumption at a power plant, electricity produced and airborne pollution might produce a useful model of emissions. NO2 only lasts for about 2 days in the lower atmosphere before breaking down into other products. The actual rate of breakdown will depend on weather and other factors. You may want to use that assumption in your model.", "votes": 1 } ] }, { "id": 775096, "postDate": "2020-03-16T09:46:56.267Z", "content": "Hello team,
\n\nI have one more question for you:
\n\nIn the reference information on EIE I have found that for emission values of NO2 per kWh different values by factor of 10 to 15 are reported for coal, oil and gas-plants. See: http://data.ec.gc.ca/data/substances/monitor/canada-s-official-greenhouse-gas-inventory/E-Tables-Electricity-Canada-Provinces-Territories/?lang=en referenced on EIE page https://insights.sustainability.google/methodology#some-notes)
\n\nFrom the satellite images I can calculate emission values which show that local emission factors for coal, oil and gas plants are more or less the same.
\n\nWhich of the emission factor are you looking for? The absolute emission factor of a plant or the diluted value in the direct vicinity of the plant, in which NO2 emissions which have travelled to other parts of the environment (higher atmosphere or distances of kilometers away) are not taken into account?
", "rawMarkdown": "Hello team,\n\nI have one more question for you: \n\nIn the reference information on EIE I have found that for emission values of NO2 per kWh different values by factor of 10 to 15 are reported for coal, oil and gas-plants. See: http://data.ec.gc.ca/data/substances/monitor/canada-s-official-greenhouse-gas-inventory/E-Tables-Electricity-Canada-Provinces-Territories/?lang=en referenced on EIE page https://insights.sustainability.google/methodology#some-notes) \n\nFrom the satellite images I can calculate emission values which show that local emission factors for coal, oil and gas plants are more or less the same. \n\nWhich of the emission factor are you looking for? The absolute emission factor of a plant or the diluted value in the direct vicinity of the plant, in which NO2 emissions which have travelled to other parts of the environment (higher atmosphere or distances of kilometers away) are not taken into account?\n ", "votes": 1, "replies": [ { "id": 781105, "postDate": "2020-03-20T23:26:30.177Z", "content": "Emissions from burning a particular fuel could vary based on the quantity of the fuel used and the process used to burn the fuel. Some power plants have catalytic convertors that reduce emissions of certain pollutants. It might be helpful to more deeply research the power plants to understand whether there are efforts to reduce pollutants from coal or oil. In general gas should produce significantly fewer emissions than the other sources. The goal is to develop an absolute emission factor for electricity produced; if it is possible to produce lower spatial detail (at a power plant level), then that is even more useful.
", "rawMarkdown": "Emissions from burning a particular fuel could vary based on the quantity of the fuel used and the process used to burn the fuel. Some power plants have catalytic convertors that reduce emissions of certain pollutants. It might be helpful to more deeply research the power plants to understand whether there are efforts to reduce pollutants from coal or oil. In general gas should produce significantly fewer emissions than the other sources. The goal is to develop an absolute emission factor for electricity produced; if it is possible to produce lower spatial detail (at a power plant level), then that is even more useful. ", "votes": 2 } ] }, { "id": 767759, "postDate": "2020-03-10T04:11:35.420Z", "content": "HI, is the troposphere vertical column density of NO2 the same as the NO2 emissions? a bit confused about it thankx
", "rawMarkdown": "HI, is the troposphere vertical column density of NO2 the same as the NO2 emissions? a bit confused about it thankx", "votes": 1, "replies": [ { "id": 781103, "postDate": "2020-03-20T23:19:25.997Z", "content": "NO2 in the atmosphere comes from several sources, burning fossil fuels is one, but forest fires are another source. The Tropomi instrument on the Sentinel 5p satellite measures the amount of NO2 in between the ground and the troposphere. Some mixing of NO2 from physically close sources will occur, but in Puerto Rico the winds and mountain range may separate population-based sources (like traffic or factories) from the generation sources on the South coast. This challenge investigates whether it is possible to model the emissions from the vertical column density, knowledge of gas dispersion, and weather patterns.
", "rawMarkdown": "NO2 in the atmosphere comes from several sources, burning fossil fuels is one, but forest fires are another source. The Tropomi instrument on the Sentinel 5p satellite measures the amount of NO2 in between the ground and the troposphere. Some mixing of NO2 from physically close sources will occur, but in Puerto Rico the winds and mountain range may separate population-based sources (like traffic or factories) from the generation sources on the South coast. This challenge investigates whether it is possible to model the emissions from the vertical column density, knowledge of gas dispersion, and weather patterns. ", "votes": 1 } ] }, { "id": 770825, "postDate": "2020-03-13T13:07:06.507Z", "content": "I have a couple of questions regarding resolution (I have written a Notebook to avoid overwriting here: https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama):
\n\nQuestion 1.1: In some places NO2 image resolution is said to be 7 x 7 km, and 0.01 arc degrees in others. But a 0.01 arc degrees pixel seems to be 1 x 1.1 km in Puerto Rico, sure I am loosing something.
\n\nQuestion 1.2: Resolution in TIFF images provided seems to be 0.004, according to metadata. How does it match with EE info?
", "rawMarkdown": "I have a couple of questions regarding resolution (I have written a Notebook to avoid overwriting here: [https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama](https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama)):\n\n**Question 1.1**: In some places NO2 image resolution is said to be 7 x 7 km, and 0.01 arc degrees in others. But a 0.01 arc degrees pixel seems to be 1 x 1.1 km in Puerto Rico, sure I am loosing something.\n\n**Question 1.2**: Resolution in TIFF images provided seems to be 0.004, according to metadata. How does it match with EE info?", "votes": 2 }, { "id": 770341, "postDate": "2020-03-12T20:47:15.760Z", "content": "It appears that there may be an erroneous value in the GPPD dataset (for more details see Link #1, Link #2, and Link #3). Do you agree that the data point is erroneous? Do you have a specific recommendation for how participants should correct this value? Or do you think that maybe it is better to leave that up to each individual participant -- to identify and correct any potential errors in the data -- and to explain and justify their rationale?
", "rawMarkdown": "It appears that there may be an erroneous value in the GPPD dataset (for more details see [Link #1](https://www.kaggle.com/c/ds4g-environmental-insights-explorer/discussion/130537), [Link #2](https://www.kaggle.com/ajulian/gppd-120-pr-csv-and-the-capacity-factor), and [Link #3](https://github.com/wri/global-power-plant-database/issues/21)). Do you agree that the data point is erroneous? Do you have a specific recommendation for how participants should correct this value? Or do you think that maybe it is better to leave that up to each individual participant -- to identify and correct any potential errors in the data -- and to [explain and justify](https://www.kaggle.com/c/ds4g-environmental-insights-explorer/discussion/134825#770155) their rationale?\t", "votes": 2, "replies": [ { "id": 781112, "postDate": "2020-03-20T23:43:02.753Z", "content": "The goal of the GPPD is to take “data...from a variety of sources, including government ministries, utility companies, equipment manufacturers, crowd-sourced databases, financial reports, and more” and match and integrate them into a single, open-source set. Unfortunately, global power plant data can be incomplete or erroneous (in fact, as the database notes, “many cases there are conflicting values for the same attribute of the same power plant from different data sources”), and errors like the ones here are not unlikely. The goal with remote sensing is to be able to corroborate self-reported data using new measurement systems. \nThe underlying GPPD dataset, and further information on the sources can be found here: https://github.com/wri/global-power-plant-database\nIdentifying, offering a solution, and justifying the approach is the best way to approach any potential errors in the data. Ideally the approach scales to other areas in the world where power sector data is even more limited.
", "rawMarkdown": "The goal of the GPPD is to take “data...from a variety of sources, including government ministries, utility companies, equipment manufacturers, crowd-sourced databases, financial reports, and more” and match and integrate them into a single, open-source set. Unfortunately, global power plant data can be incomplete or erroneous (in fact, as the database notes, “many cases there are conflicting values for the same attribute of the same power plant from different data sources”), and errors like the ones here are not unlikely. The goal with remote sensing is to be able to corroborate self-reported data using new measurement systems. \nThe underlying GPPD dataset, and further information on the sources can be found here: https://github.com/wri/global-power-plant-database\nIdentifying, offering a solution, and justifying the approach is the best way to approach any potential errors in the data. Ideally the approach scales to other areas in the world where power sector data is even more limited. \n", "votes": 2 } ] }, { "id": 771395, "postDate": "2020-03-14T05:40:54.673Z", "content": "Also a question about the NO2 dataset: according to the documentation, two of the gas measurement bands (\"NO2 column number density\" and \"tropospheric NO2 column number density\") have a negative minimum value, but how can a gas measurement be negative?
", "rawMarkdown": "Also a question about the NO2 dataset: according to [the documentation](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S5P_OFFL_L3_NO2), two of the gas measurement bands (\"NO2 column number density\" and \"tropospheric NO2 column number density\") have a negative minimum value, but how can a gas measurement be negative?" }, { "id": 771386, "postDate": "2020-03-14T05:30:48.680Z", "content": "Some questions regarding Emissions Factors:\n- as the minimum output, are we expected to provide annual Emissions Factors for each Power Plant or for the whole Puerto Rico grid?\n- my understanding is that Emissions Factors at the Power Plant level (in case they are required) should be quite constant in time, assuming no changes are made at the Power Plant; if a windy month shows a lower Emissions Factor for a Power Plant, it means my model has to be fixed, is it right?
", "rawMarkdown": "Some questions regarding Emissions Factors:\n- as the minimum output, are we expected to provide annual Emissions Factors for each Power Plant or for the whole Puerto Rico grid?\n- my understanding is that Emissions Factors at the Power Plant level (in case they are required) should be quite constant in time, assuming no changes are made at the Power Plant; if a windy month shows a lower Emissions Factor for a Power Plant, it means my model has to be fixed, is it right?", "replies": [ { "id": 781108, "postDate": "2020-03-20T23:29:56.850Z", "content": "I have also questions regarding exporting EE data to numpy (same Notebook: https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama):
\n\nQuestion 2.1: How can I catch non-empty Images in an ImageCollection?
\n\nQuestion 2.2: Why the (supposed to be) minimum pixel has an array size of (2, 2) rather than (1, 1)? Might it be just a rounding problem?
\n\nThanks, it's being a wonderful experience!
", "rawMarkdown": "I have also questions regarding exporting EE data to numpy (same Notebook: [https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama](https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama)):\n\n**Question 2.1**: How can I catch non-empty Images in an ImageCollection? \n\n**Question 2.2**: Why the (supposed to be) minimum pixel has an array size of (2, 2) rather than (1, 1)? Might it be just a rounding problem?\n\nThanks, it's being a wonderful experience!", "replies": [ { "id": 783767, "postDate": "2020-03-23T16:40:22.683Z", "content": "If you're looking for pixels in a region, then the only way is to count the pixels in the region. The functional way to do this is by mapping a reducer over the collection and setting a property of each image which is the output of a count reducer. Then filter out all images for which count=0 and continue on the images that have pixels. However, something here seems funny. Why are so many images \"empty\" in this region? In code there's:
\n\nrectangle = ee.Geometry.Rectangle([long1, lat1, long2, lat2]) # the pixel
If your \"region\" is a single pixel, then don't use sampleRectangle(), use reduceRegion().get(). If you return reduceRegion().get() from a mapped function, set the second argument of map() to true such that nulls can be returned and automatically dropped.
Thanks for your answer. I am quite new to EE; I am not using sampleRectangle anymore. However, there is still something I do not understand (I may be doing something wrong); I finally could check those images and they are not empty, they just do not cover Puerto Rico, but the strange thing is that filterBounds does not filter them.
\n\nI wrote a script to show this: filterBounds. The selected image is supposed to be filtered out by filterBounds but is not.
", "rawMarkdown": "Thanks for your answer. I am quite new to EE; I am not using sampleRectangle anymore. However, there is still something I do not understand (I may be doing something wrong); I finally could check those images and they are not empty, they just do not cover Puerto Rico, but the strange thing is that filterBounds does not filter them.\n\nI wrote a script to show this: [filterBounds](https://code.earthengine.google.com/?scriptPath=users%2Fajulianes%2FDS4G_EIE1%3AfilterBounds). The selected image is supposed to be filtered out by filterBounds but is not.\n" }, { "id": 784961, "postDate": "2020-03-24T16:32:36.097Z", "content": "The footprint of the image may include areas without pixels. To more aggressively filter images with only a small part of the footprint in the area of interest, you could require the centroid to be in the region. Or you could do an intersection and reject images that have only a small part of the footprint in the region. See https://developers.google.com/earth-engine/joins_spatial.
", "rawMarkdown": "The footprint of the image may include areas without pixels. To more aggressively filter images with only a small part of the footprint in the area of interest, you could require the centroid to be in the region. Or you could do an intersection and reject images that have only a small part of the footprint in the region. See https://developers.google.com/earth-engine/joins_spatial.", "votes": 1 }, { "id": 785793, "postDate": "2020-03-25T11:55:27.453Z", "content": "Thanks for the link, it's nice to know how to join partial images. However, when I said those images \"do not cover Puerto Rico\" what I really meant is the image is several THOUSAND kilometers away. In the script I mentioned (https://code.earthengine.google.com/?scriptPath=users%2Fajulianes%2FDS4G_EIE1%3AfilterBounds) I apply filterBounds with one point, but you have to zoom out five times to begin seeing the NO2 image. It would be nice if you could take a look. I guess the person who saved the TIFF images for the competition had the same issue. Thanks!
", "rawMarkdown": "Thanks for the link, it's nice to know how to join partial images. However, when I said those images \"do not cover Puerto Rico\" what I really meant is the image is several THOUSAND kilometers away. In the script I mentioned (https://code.earthengine.google.com/?scriptPath=users%2Fajulianes%2FDS4G_EIE1%3AfilterBounds) I apply filterBounds with one point, but you have to zoom out five times to begin seeing the NO2 image. It would be nice if you could take a look. I guess the person who saved the TIFF images for the competition had the same issue. Thanks!", "votes": 1 } ] }, { "id": 781529, "postDate": "2020-03-21T11:51:49.337Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 769005, "author_name": "Vincent Verhagen", "author_url": "", "post_date": "2020-03-11T12:26:53.657000", "content": "Hi,
\n\ni'm not very familiar with Puerto Rico. I assume there is little industrial activity and few other sources of NO2 emission other then power plants. The satellite images of troposphere indicate there is also emission at other locations then fossil fuel power plants. Any suggestions how to look at this?
how accurate would you want a prediction to be to be useful for your purpose? Any thoughts on this?
Thanks,
", "votes": 3, "replies": [ { "id": 769296, "author_name": "Viktoriia Kobenko", "author_url": "", "post_date": "2020-03-11T18:33:10.803000", "content": "A lot of emissions come for traffic too
", "votes": 0, "replies": [] }, { "id": 769520, "author_name": "Caesar Lupum", "author_url": "", "post_date": "2020-03-12T01:41:57.967000", "content": "I have the same question, consider anomalies values, Can I prioritize some industrial activity or region ?
", "votes": 0, "replies": [] }, { "id": 781106, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-20T23:27:41.397000", "content": "Sorry, I think the IEEE article you mentioned is missing, isn't it?
", "votes": 0, "replies": [] }, { "id": 783895, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-23T19:13:08.150000", "content": "Thanks for catching that - my mistake. I updated the post. Here's a link to the article for completeness.
", "votes": 0, "replies": [] } ] }, { "id": 767789, "author_name": "Rasyid Ridha", "author_url": "", "post_date": "2020-03-10T05:01:49.567000", "content": "How to convert NO2 column number density (mol/m2) to NOx emission (ton)?
", "votes": 4, "replies": [ { "id": 768035, "author_name": "Jonathan", "author_url": "", "post_date": "2020-03-10T11:47:57.713000", "content": "Hey, I think it is actually pretty much straight forward. You need to calculate the factor to convert pixel(image shape) into longitudinal and latitudinal units (image bounding box) and then use the haversine formula (check the link below) to convert these into meters.
\n\nhttps://en.wikipedia.org/wiki/Haversine_formula
\n\nNow you have the height/width in meters per pixel (haven't calculated it yet so I cannot tell if the pixels are square).
\n\nNow you get the area (pixels) you are interested in, take the sum of the n02 column number density and multiply it by the area (m^2). Now you have the sum of n02 in mol for the whole area. The last step ist to convert mol to gram (or ton). The molar mass of n02 is 46,0055 g/mol (at least that's what google says).
\n\nI'm currently writing a function for that issue, when I'm done I will share it here. \nIf I'm mistaken somewhere please correct me :).
", "votes": 1, "replies": [] }, { "id": 769315, "author_name": "alberto julian", "author_url": "", "post_date": "2020-03-11T18:57:48.447000", "content": "A couple of weeks ago I would have said something similar (less accurate, probably). However, with just the slight idea I have now, the (big) problem in my opinion is that NOx chemistry is very dynamic and satellites only can take a picture (the NO2 in mol/m2) at a given moment of what happened a while ago far down (the lb or ton NOx emitted at the power plant); for instance, NO2 has a variable chemical lifetime of hours, and is also affected by wind among other things.\nThus, I subscribe the question to see if the hosts give us some clue.
", "votes": 2, "replies": [] }, { "id": 781104, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-20T23:23:41.100000", "content": "This is one of the cruxes of the challenge, and we are excited to learn about how the Kaggle community approaches this question. There are several ways to correlate this data. A correlation between fuel consumption at a power plant, electricity produced and airborne pollution might produce a useful model of emissions. NO2 only lasts for about 2 days in the lower atmosphere before breaking down into other products. The actual rate of breakdown will depend on weather and other factors. You may want to use that assumption in your model.
", "votes": 1, "replies": [] } ] }, { "id": 775096, "author_name": "Vincent Verhagen", "author_url": "", "post_date": "2020-03-16T09:46:56.267000", "content": "Hello team,
\n\nI have one more question for you:
\n\nIn the reference information on EIE I have found that for emission values of NO2 per kWh different values by factor of 10 to 15 are reported for coal, oil and gas-plants. See: http://data.ec.gc.ca/data/substances/monitor/canada-s-official-greenhouse-gas-inventory/E-Tables-Electricity-Canada-Provinces-Territories/?lang=en referenced on EIE page https://insights.sustainability.google/methodology#some-notes)
\n\nFrom the satellite images I can calculate emission values which show that local emission factors for coal, oil and gas plants are more or less the same.
\n\nWhich of the emission factor are you looking for? The absolute emission factor of a plant or the diluted value in the direct vicinity of the plant, in which NO2 emissions which have travelled to other parts of the environment (higher atmosphere or distances of kilometers away) are not taken into account?
", "votes": 1, "replies": [ { "id": 781105, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-20T23:26:30.177000", "content": "Emissions from burning a particular fuel could vary based on the quantity of the fuel used and the process used to burn the fuel. Some power plants have catalytic convertors that reduce emissions of certain pollutants. It might be helpful to more deeply research the power plants to understand whether there are efforts to reduce pollutants from coal or oil. In general gas should produce significantly fewer emissions than the other sources. The goal is to develop an absolute emission factor for electricity produced; if it is possible to produce lower spatial detail (at a power plant level), then that is even more useful.
", "votes": 2, "replies": [] } ] }, { "id": 767759, "author_name": "ian cecil akoto", "author_url": "", "post_date": "2020-03-10T04:11:35.420000", "content": "HI, is the troposphere vertical column density of NO2 the same as the NO2 emissions? a bit confused about it thankx
", "votes": 1, "replies": [ { "id": 781103, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-20T23:19:25.997000", "content": "NO2 in the atmosphere comes from several sources, burning fossil fuels is one, but forest fires are another source. The Tropomi instrument on the Sentinel 5p satellite measures the amount of NO2 in between the ground and the troposphere. Some mixing of NO2 from physically close sources will occur, but in Puerto Rico the winds and mountain range may separate population-based sources (like traffic or factories) from the generation sources on the South coast. This challenge investigates whether it is possible to model the emissions from the vertical column density, knowledge of gas dispersion, and weather patterns.
", "votes": 1, "replies": [] } ] }, { "id": 770825, "author_name": "alberto julian", "author_url": "", "post_date": "2020-03-13T13:07:06.507000", "content": "I have a couple of questions regarding resolution (I have written a Notebook to avoid overwriting here: https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama):
\n\nQuestion 1.1: In some places NO2 image resolution is said to be 7 x 7 km, and 0.01 arc degrees in others. But a 0.01 arc degrees pixel seems to be 1 x 1.1 km in Puerto Rico, sure I am loosing something.
\n\nQuestion 1.2: Resolution in TIFF images provided seems to be 0.004, according to metadata. How does it match with EE info?
", "votes": 2, "replies": [] }, { "id": 770341, "author_name": "Paul Mooney", "author_url": "", "post_date": "2020-03-12T20:47:15.760000", "content": "It appears that there may be an erroneous value in the GPPD dataset (for more details see Link #1, Link #2, and Link #3). Do you agree that the data point is erroneous? Do you have a specific recommendation for how participants should correct this value? Or do you think that maybe it is better to leave that up to each individual participant -- to identify and correct any potential errors in the data -- and to explain and justify their rationale?
", "votes": 2, "replies": [ { "id": 781112, "author_name": "Saleem Van Groenou", "author_url": "", "post_date": "2020-03-20T23:43:02.753000", "content": "The goal of the GPPD is to take “data...from a variety of sources, including government ministries, utility companies, equipment manufacturers, crowd-sourced databases, financial reports, and more” and match and integrate them into a single, open-source set. Unfortunately, global power plant data can be incomplete or erroneous (in fact, as the database notes, “many cases there are conflicting values for the same attribute of the same power plant from different data sources”), and errors like the ones here are not unlikely. The goal with remote sensing is to be able to corroborate self-reported data using new measurement systems. \nThe underlying GPPD dataset, and further information on the sources can be found here: https://github.com/wri/global-power-plant-database\nIdentifying, offering a solution, and justifying the approach is the best way to approach any potential errors in the data. Ideally the approach scales to other areas in the world where power sector data is even more limited.
", "votes": 2, "replies": [] } ] }, { "id": 771395, "author_name": "alberto julian", "author_url": "", "post_date": "2020-03-14T05:40:54.673000", "content": "Also a question about the NO2 dataset: according to the documentation, two of the gas measurement bands (\"NO2 column number density\" and \"tropospheric NO2 column number density\") have a negative minimum value, but how can a gas measurement be negative?
", "votes": 0, "replies": [] }, { "id": 771386, "author_name": "alberto julian", "author_url": "", "post_date": "2020-03-14T05:30:48.680000", "content": "Some questions regarding Emissions Factors:\n- as the minimum output, are we expected to provide annual Emissions Factors for each Power Plant or for the whole Puerto Rico grid?\n- my understanding is that Emissions Factors at the Power Plant level (in case they are required) should be quite constant in time, assuming no changes are made at the Power Plant; if a windy month shows a lower Emissions Factor for a Power Plant, it means my model has to be fixed, is it right?
", "votes": 0, "replies": [ { "id": 781108, "author_name": "Ollie Guinan", "author_url": "", "post_date": "2020-03-20T23:29:56.850000", "content": "I have also questions regarding exporting EE data to numpy (same Notebook: https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama):
\n\nQuestion 2.1: How can I catch non-empty Images in an ImageCollection?
\n\nQuestion 2.2: Why the (supposed to be) minimum pixel has an array size of (2, 2) rather than (1, 1)? Might it be just a rounding problem?
\n\nThanks, it's being a wonderful experience!
", "votes": 0, "replies": [ { "id": 783767, "author_name": "Nicholas Clinton", "author_url": "", "post_date": "2020-03-23T16:40:22.683000", "content": "If you're looking for pixels in a region, then the only way is to count the pixels in the region. The functional way to do this is by mapping a reducer over the collection and setting a property of each image which is the output of a count reducer. Then filter out all images for which count=0 and continue on the images that have pixels. However, something here seems funny. Why are so many images \"empty\" in this region? In code there's:
\n\nrectangle = ee.Geometry.Rectangle([long1, lat1, long2, lat2]) # the pixel
If your \"region\" is a single pixel, then don't use sampleRectangle(), use reduceRegion().get(). If you return reduceRegion().get() from a mapped function, set the second argument of map() to true such that nulls can be returned and automatically dropped.
Thanks for your answer. I am quite new to EE; I am not using sampleRectangle anymore. However, there is still something I do not understand (I may be doing something wrong); I finally could check those images and they are not empty, they just do not cover Puerto Rico, but the strange thing is that filterBounds does not filter them.
\n\nI wrote a script to show this: filterBounds. The selected image is supposed to be filtered out by filterBounds but is not.
", "votes": 0, "replies": [] }, { "id": 784961, "author_name": "Nicholas Clinton", "author_url": "", "post_date": "2020-03-24T16:32:36.097000", "content": "The footprint of the image may include areas without pixels. To more aggressively filter images with only a small part of the footprint in the area of interest, you could require the centroid to be in the region. Or you could do an intersection and reject images that have only a small part of the footprint in the region. See https://developers.google.com/earth-engine/joins_spatial.
", "votes": 1, "replies": [] }, { "id": 785793, "author_name": "alberto julian", "author_url": "", "post_date": "2020-03-25T11:55:27.453000", "content": "Thanks for the link, it's nice to know how to join partial images. However, when I said those images \"do not cover Puerto Rico\" what I really meant is the image is several THOUSAND kilometers away. In the script I mentioned (https://code.earthengine.google.com/?scriptPath=users%2Fajulianes%2FDS4G_EIE1%3AfilterBounds) I apply filterBounds with one point, but you have to zoom out five times to begin seeing the NO2 image. It would be nice if you could take a look. I guess the person who saved the TIFF images for the competition had the same issue. Thanks!
", "votes": 1, "replies": [] } ] }, { "id": 781529, "author_name": "", "author_url": "", "post_date": "2020-03-21T11:51:49.337000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "767722": "Hi Everyone! Thank you for your participation! From March 9-13, we’ll be hosting an AMA with the competition hosts, here in this thread. The Environmental Insights Explorer team will be judging your submissions, and this AMA is an excellent opportunity to get to know the team behind the competition and ask any questions that you might have. \n\nFeel free to ask questions about:\n- [Earth Engine](https://earthengine.google.com/)\n- [Environmental Insights Explorer](https://insights.sustainability.google/)\n- Emissions factors, and other related subject matters\n\nPlease ask your questions in the thread below!\n\nThe team will pop in on March 16 to answer as many as they can.\n\n- Paul\n", "769005": "Hi,\n\n- i'm not very familiar with Puerto Rico. I assume there is little industrial activity and few other sources of NO2 emission other then power plants. The satellite images of troposphere indicate there is also emission at other locations then fossil fuel power plants. Any suggestions how to look at this? \n\n- how accurate would you want a prediction to be to be useful for your purpose? Any thoughts on this? \n\nThanks, ", "767789": "How to convert NO2 column number density (mol/m2) to NOx emission (ton)?", "775096": "Hello team,\n\nI have one more question for you: \n\nIn the reference information on EIE I have found that for emission values of NO2 per kWh different values by factor of 10 to 15 are reported for coal, oil and gas-plants. See: http://data.ec.gc.ca/data/substances/monitor/canada-s-official-greenhouse-gas-inventory/E-Tables-Electricity-Canada-Provinces-Territories/?lang=en referenced on EIE page https://insights.sustainability.google/methodology#some-notes) \n\nFrom the satellite images I can calculate emission values which show that local emission factors for coal, oil and gas plants are more or less the same. \n\nWhich of the emission factor are you looking for? The absolute emission factor of a plant or the diluted value in the direct vicinity of the plant, in which NO2 emissions which have travelled to other parts of the environment (higher atmosphere or distances of kilometers away) are not taken into account?\n ", "767759": "HI, is the troposphere vertical column density of NO2 the same as the NO2 emissions? a bit confused about it thankx", "770825": "I have a couple of questions regarding resolution (I have written a Notebook to avoid overwriting here: [https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama](https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama)):\n\n**Question 1.1**: In some places NO2 image resolution is said to be 7 x 7 km, and 0.01 arc degrees in others. But a 0.01 arc degrees pixel seems to be 1 x 1.1 km in Puerto Rico, sure I am loosing something.\n\n**Question 1.2**: Resolution in TIFF images provided seems to be 0.004, according to metadata. How does it match with EE info?", "770341": "It appears that there may be an erroneous value in the GPPD dataset (for more details see [Link #1](https://www.kaggle.com/c/ds4g-environmental-insights-explorer/discussion/130537), [Link #2](https://www.kaggle.com/ajulian/gppd-120-pr-csv-and-the-capacity-factor), and [Link #3](https://github.com/wri/global-power-plant-database/issues/21)). Do you agree that the data point is erroneous? Do you have a specific recommendation for how participants should correct this value? Or do you think that maybe it is better to leave that up to each individual participant -- to identify and correct any potential errors in the data -- and to [explain and justify](https://www.kaggle.com/c/ds4g-environmental-insights-explorer/discussion/134825#770155) their rationale?\t", "771395": "Also a question about the NO2 dataset: according to [the documentation](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S5P_OFFL_L3_NO2), two of the gas measurement bands (\"NO2 column number density\" and \"tropospheric NO2 column number density\") have a negative minimum value, but how can a gas measurement be negative?", "771386": "Some questions regarding Emissions Factors:\n- as the minimum output, are we expected to provide annual Emissions Factors for each Power Plant or for the whole Puerto Rico grid?\n- my understanding is that Emissions Factors at the Power Plant level (in case they are required) should be quite constant in time, assuming no changes are made at the Power Plant; if a windy month shows a lower Emissions Factor for a Power Plant, it means my model has to be fixed, is it right?", "770830": "I have also questions regarding exporting EE data to numpy (same Notebook: [https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama](https://www.kaggle.com/ajulian/questions-for-ds4g-eie-ama)):\n\n**Question 2.1**: How can I catch non-empty Images in an ImageCollection? \n\n**Question 2.2**: Why the (supposed to be) minimum pixel has an array size of (2, 2) rather than (1, 1)? Might it be just a rounding problem?\n\nThanks, it's being a wonderful experience!", "781529": "" } }