{ "id": 5686, "title": "Interpolation with R", "url": "/competitions/ams-2014-solar-energy-prediction-contest/discussion/5686", "author_name": "", "post_date": "2013-09-07T07:41:07.763Z", "votes": 1, "comment_count": 1, "views": 1271, "content": "

For R-users, here is a function which loads data with specified grid in order to avoid memory issue (because I've never been able to run python translated code into R provided by some competitors):

\n

datagrid_selection= function (dataname='apcp',filename='apcp_sfc_latlon_subset_19940101_20071231', lat=1:16, long=1:9, hours=1:5, gefs=1:11, nrow=5113)  {
setwd("your working directory/")
mygrid=expand.grid(lat, long, hours, gefs)
dataname <- nc_open(paste(filename,".nc",sep=""))
for ( g in 1:nrow(mygrid) ) {
cat( paste("GRID N°",g,sep="") , '\\n')
if ( g!=1) {
data= cbind(data, ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) ) )

\n

}  else  {

data=ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) )

}
}
return (data)

\n

}

\n

 

\n

But now, how to interpolate?

\n

I'm not far from the solution because one can access latitude and longitude using:

\n

ncvar_get(dataname, 'lon') ; ncvar_get(dataname, 'lat')

\n

 but some fragments of code remain unclear to me

", "messages": [ { "id": "30410", "postDate": "09/07/2013 07:41:07", "content": "

For R-users, here is a function which loads data with specified grid in order to avoid memory issue (because I've never been able to run python translated code into R provided by some competitors):

\n

datagrid_selection= function (dataname='apcp',filename='apcp_sfc_latlon_subset_19940101_20071231', lat=1:16, long=1:9, hours=1:5, gefs=1:11, nrow=5113)  {
setwd("your working directory/")
mygrid=expand.grid(lat, long, hours, gefs)
dataname <- nc_open(paste(filename,".nc",sep=""))
for ( g in 1:nrow(mygrid) ) {
cat( paste("GRID N°",g,sep="") , '\\n')
if ( g!=1) {
data= cbind(data, ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) ) )

\n

}  else  {

data=ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) )

}
}
return (data)

\n

}

\n

 

\n

But now, how to interpolate?

\n

I'm not far from the solution because one can access latitude and longitude using:

\n

ncvar_get(dataname, 'lon') ; ncvar_get(dataname, 'lat')

\n

 but some fragments of code remain unclear to me

", "rawMarkdown": "", "votes": null }, { "id": "31406", "postDate": "09/23/2013 15:07:34", "content": "

[quote=Herimanitra;30410]

\n

For R-users, here is a function which loads data with specified grid in order to avoid memory issue (because I've never been able to run python translated code into R provided by some competitors):

\n

datagrid_selection= function (dataname='apcp',filename='apcp_sfc_latlon_subset_19940101_20071231', lat=1:16, long=1:9, hours=1:5, gefs=1:11, nrow=5113)  {
setwd("your working directory/")
mygrid=expand.grid(lat, long, hours, gefs)
dataname <- nc_open(paste(filename,".nc",sep=""))
for ( g in 1:nrow(mygrid) ) {
cat( paste("GRID N°",g,sep="") , '\\n')
if ( g!=1) {
data= cbind(data, ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) ) )

\n

}  else  {

data=ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) )

}
}
return (data)

\n

}

\n

 

\n

But now, how to interpolate?

\n

I'm not far from the solution because one can access latitude and longitude using:

\n

ncvar_get(dataname, 'lon') ; ncvar_get(dataname, 'lat')

\n

 but some fragments of code remain unclear to me

\n

[/quote]

\n

Hi Herimanitra,

\n

I am meeting some problems even just reading in the datasets in this thread : http://www.kaggle.com/c/ams-2014-solar-energy-prediction-contest/forums/t/5850/r-netcdf4-reading-input-files-but-always-return-almost-duplicate-data . I was wondering if you could offer some advice, if you have met and dealt with it already?

\n

Thank you.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 31406, "author_name": "ericchio", "author_url": "", "post_date": "09/23/2013 15:07:34", "content": "

[quote=Herimanitra;30410]

\n

For R-users, here is a function which loads data with specified grid in order to avoid memory issue (because I've never been able to run python translated code into R provided by some competitors):

\n

datagrid_selection= function (dataname='apcp',filename='apcp_sfc_latlon_subset_19940101_20071231', lat=1:16, long=1:9, hours=1:5, gefs=1:11, nrow=5113)  {
setwd("your working directory/")
mygrid=expand.grid(lat, long, hours, gefs)
dataname <- nc_open(paste(filename,".nc",sep=""))
for ( g in 1:nrow(mygrid) ) {
cat( paste("GRID N°",g,sep="") , '\\n')
if ( g!=1) {
data= cbind(data, ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) ) )

\n

}  else  {

data=ncvar_get( dataname,dataname$var[[3]],start=c(mygrid[g,1],mygrid[g,2],mygrid[g,3],mygrid[g,4],1), count=c(1,1,1,1,nrow) )

}
}
return (data)

\n

}

\n

 

\n

But now, how to interpolate?

\n

I'm not far from the solution because one can access latitude and longitude using:

\n

ncvar_get(dataname, 'lon') ; ncvar_get(dataname, 'lat')

\n

 but some fragments of code remain unclear to me

\n

[/quote]

\n

Hi Herimanitra,

\n

I am meeting some problems even just reading in the datasets in this thread : http://www.kaggle.com/c/ams-2014-solar-energy-prediction-contest/forums/t/5850/r-netcdf4-reading-input-files-but-always-return-almost-duplicate-data . I was wondering if you could offer some advice, if you have met and dealt with it already?

\n

Thank you.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "30410": "", "31406": "" }, "source": "meta" }