{ "id": 5850, "title": "R NetCDF4 reading input files but always return almost duplicate data", "url": "/competitions/ams-2014-solar-energy-prediction-contest/discussion/5850", "author_name": "", "post_date": "2013-09-23T06:16:52.717Z", "votes": null, "comment_count": 6, "views": 1974, "content": "

Hi all,

\n

I'm mostly a Python user so I'm using the R to read NetCD4 for data exploration. However, I met some odd issues:

\n

Despite I read from two different NC files which are clearly different files, the data returned is almost the same - except the dataset$filename attribute.

\n

require(ncdf4)

\n

d1 <- nc_open('ulwrf_tatm_latlon_subset_19940101_20071231.nc')

\n

d2 <- nc_open('tmin_2m_latlon_subset_19940101_20071231.nc')

\n

v1 <- ncvar_get(d1)

\n

v2 <- ncvar_get(d2)

\n

cat(d1$filename) # ulwrf_tatm_latlon_subset_19940101_20071231.nc

\n

cat(attr(d1$var, 'name'), '\\n') # intTime intValidTime Upward_Long-Wave_Rad_Flux

\n

cat(sum(v1), '\\n') # 10327017841

\n

cat(d2$filename) # tmin_2m_latlon_subset_19940101_20071231.nc

\n

cat(attr(d2$var, 'name'), '\\n') # intTime intValidTime Upward_Long-Wave_Rad_Flux

\n

cat(sum(v2), '\\n') # 10327017841

\n

 

\n

Now, from the above, I would expect these two return different values. However, except the filename attribute, the rest are exactly duplicated. I am a little puzzled, what did I do wrong here?

", "messages": [ { "id": "31376", "postDate": "09/23/2013 06:16:52", "content": "

Hi all,

\n

I'm mostly a Python user so I'm using the R to read NetCD4 for data exploration. However, I met some odd issues:

\n

Despite I read from two different NC files which are clearly different files, the data returned is almost the same - except the dataset$filename attribute.

\n

require(ncdf4)

\n

d1 <- nc_open('ulwrf_tatm_latlon_subset_19940101_20071231.nc')

\n

d2 <- nc_open('tmin_2m_latlon_subset_19940101_20071231.nc')

\n

v1 <- ncvar_get(d1)

\n

v2 <- ncvar_get(d2)

\n

cat(d1$filename) # ulwrf_tatm_latlon_subset_19940101_20071231.nc

\n

cat(attr(d1$var, 'name'), '\\n') # intTime intValidTime Upward_Long-Wave_Rad_Flux

\n

cat(sum(v1), '\\n') # 10327017841

\n

cat(d2$filename) # tmin_2m_latlon_subset_19940101_20071231.nc

\n

cat(attr(d2$var, 'name'), '\\n') # intTime intValidTime Upward_Long-Wave_Rad_Flux

\n

cat(sum(v2), '\\n') # 10327017841

\n

 

\n

Now, from the above, I would expect these two return different values. However, except the filename attribute, the rest are exactly duplicated. I am a little puzzled, what did I do wrong here?

", "rawMarkdown": "", "votes": null }, { "id": "31377", "postDate": "09/23/2013 06:19:16", "content": "

In case someone is suspecting the files are wrong... I've made submissions based on these files.

\n

The file sizes are as follows:

\n

"63,238,020 ulwrf_tatm_latlon_subset_19940101_20071231.nc"

\n

"93,599,518 tmin_2m_latlon_subset_19940101_20071231.nc"

", "rawMarkdown": "", "votes": null }, { "id": "31407", "postDate": "09/23/2013 15:18:46", "content": "

Nevermind above. It seems I thought I have invoked nc_close (in the larger script, not this repro script) but didn't.

", "rawMarkdown": "", "votes": null }, { "id": "31409", "postDate": "09/23/2013 15:48:22", "content": "

I experienced the same issues and I solved it by opening one file at a time. You have to call the function nc_close(...) before opening the next file.

", "rawMarkdown": "", "votes": null }, { "id": "31411", "postDate": "09/23/2013 15:58:27", "content": "

What ' s the meaning of sum(v1) ?

", "rawMarkdown": "", "votes": null }, { "id": "31413", "postDate": "09/23/2013 16:32:20", "content": "

Thanks for response. I was just checking the values if they are equal or not. No problem... I noticed my mistake already. =]

", "rawMarkdown": "", "votes": null }, { "id": "31414", "postDate": "09/23/2013 16:32:56", "content": "

Indeed Azerty. I thought I invoked it but apparently I missed it, and missed it during debugging. Thanks for your reply.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 31377, "author_name": "ericchio", "author_url": "", "post_date": "09/23/2013 06:19:16", "content": "

In case someone is suspecting the files are wrong... I've made submissions based on these files.

\n

The file sizes are as follows:

\n

"63,238,020 ulwrf_tatm_latlon_subset_19940101_20071231.nc"

\n

"93,599,518 tmin_2m_latlon_subset_19940101_20071231.nc"

", "votes": null, "replies": [] }, { "id": 31407, "author_name": "ericchio", "author_url": "", "post_date": "09/23/2013 15:18:46", "content": "

Nevermind above. It seems I thought I have invoked nc_close (in the larger script, not this repro script) but didn't.

", "votes": null, "replies": [] }, { "id": 31409, "author_name": "lpkronek", "author_url": "", "post_date": "09/23/2013 15:48:22", "content": "

I experienced the same issues and I solved it by opening one file at a time. You have to call the function nc_close(...) before opening the next file.

", "votes": null, "replies": [] }, { "id": 31411, "author_name": "herimanitra", "author_url": "", "post_date": "09/23/2013 15:58:27", "content": "

What ' s the meaning of sum(v1) ?

", "votes": null, "replies": [] }, { "id": 31413, "author_name": "ericchio", "author_url": "", "post_date": "09/23/2013 16:32:20", "content": "

Thanks for response. I was just checking the values if they are equal or not. No problem... I noticed my mistake already. =]

", "votes": null, "replies": [] }, { "id": 31414, "author_name": "ericchio", "author_url": "", "post_date": "09/23/2013 16:32:56", "content": "

Indeed Azerty. I thought I invoked it but apparently I missed it, and missed it during debugging. Thanks for your reply.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "31376": "", "31377": "", "31407": "", "31409": "", "31411": "", "31413": "", "31414": "" }, "source": "meta" }