{ "id": 10997, "title": "view large csv file.", "url": "/competitions/avazu-ctr-prediction/discussion/10997", "author_name": "", "post_date": "2014-11-20T18:28:35.773Z", "votes": null, "comment_count": 8, "views": 4226, "content": "

What tools people use to view/edit large csv file? I am using reCsvEditor for the first glance of data file but am wondering if there are alternatives.

", "messages": [ { "id": "58558", "postDate": "11/20/2014 18:28:35", "content": "

What tools people use to view/edit large csv file? I am using reCsvEditor for the first glance of data file but am wondering if there are alternatives.

", "rawMarkdown": "", "votes": null }, { "id": "58560", "postDate": "11/20/2014 19:05:53", "content": "

R + ff package

", "rawMarkdown": "", "votes": null }, { "id": "58567", "postDate": "11/20/2014 19:49:19", "content": "

Try csvstat (part of csvkit) as a first pass on a csv file to get some column statistics.

\n

https://csvkit.readthedocs.org/en/0.9.0/

", "rawMarkdown": "", "votes": null }, { "id": "59678", "postDate": "12/06/2014 14:22:19", "content": "

head / tail / more should be good enough if you use linux.

", "rawMarkdown": "", "votes": null }, { "id": "59979", "postDate": "12/11/2014 05:13:58", "content": "

I think head and tail work with | and  >  will be much better !

\n

for example :

\n

 head -n1 XXX.csv | tail -n2  >> testfile

\n

We can see every line in testfile with this way  no matter your file is too big or small ! 

", "rawMarkdown": "", "votes": null }, { "id": "59999", "postDate": "12/11/2014 13:50:36", "content": "

Or "cut" if you want to look at just a few columns from the entire dataset

\n

cut -d "," -f2,6,7,9,10 train.csv > site_or_app.csv

", "rawMarkdown": "", "votes": null }, { "id": "60000", "postDate": "12/11/2014 14:14:43", "content": "

你要想看某一列或几列上面的值的话，可以直接在刚才的命令上面

\n

head -n1 XXX.csv | tail -n2 | awk -F ',' '{print $1,$2,$X}' >> testfile.csv 

\n

这样可以直接查看某一列或者某几列！

", "rawMarkdown": "", "votes": null }, { "id": "60127", "postDate": "12/13/2014 02:26:29", "content": "

i wrote a fancy ASCII viewer of categorical variables in csv files.

\n

you can use it like this:

\n

csvcut -c click,site_domain,app_domain train | ./ctviewer.py -2 20 20

\n

csvcut is included in csvkit.

", "rawMarkdown": "", "votes": null }, { "id": "89545", "postDate": "08/17/2015 07:50:10", "content": "

We can use command 'less' in Linux to view content of a big file, e.g. "less train.csv".

", "rawMarkdown": "We can use command 'less' in Linux to view content of a big file, e.g. \"less train.csv\".", "votes": null } ], "comments": [ { "id": 58560, "author_name": "konradb", "author_url": "", "post_date": "11/20/2014 19:05:53", "content": "

R + ff package

", "votes": null, "replies": [] }, { "id": 58567, "author_name": "huibers", "author_url": "", "post_date": "11/20/2014 19:49:19", "content": "

Try csvstat (part of csvkit) as a first pass on a csv file to get some column statistics.

\n

https://csvkit.readthedocs.org/en/0.9.0/

", "votes": null, "replies": [] }, { "id": 59678, "author_name": "alisonbw", "author_url": "", "post_date": "12/06/2014 14:22:19", "content": "

head / tail / more should be good enough if you use linux.

", "votes": null, "replies": [] }, { "id": 59979, "author_name": "njumdl2014", "author_url": "", "post_date": "12/11/2014 05:13:58", "content": "

I think head and tail work with | and  >  will be much better !

\n

for example :

\n

 head -n1 XXX.csv | tail -n2  >> testfile

\n

We can see every line in testfile with this way  no matter your file is too big or small ! 

", "votes": null, "replies": [] }, { "id": 59999, "author_name": "cartographic", "author_url": "", "post_date": "12/11/2014 13:50:36", "content": "

Or "cut" if you want to look at just a few columns from the entire dataset

\n

cut -d "," -f2,6,7,9,10 train.csv > site_or_app.csv

", "votes": null, "replies": [] }, { "id": 60000, "author_name": "njumdl2014", "author_url": "", "post_date": "12/11/2014 14:14:43", "content": "

你要想看某一列或几列上面的值的话，可以直接在刚才的命令上面

\n

head -n1 XXX.csv | tail -n2 | awk -F ',' '{print $1,$2,$X}' >> testfile.csv 

\n

这样可以直接查看某一列或者某几列！

", "votes": null, "replies": [] }, { "id": 60127, "author_name": "takagiwa", "author_url": "", "post_date": "12/13/2014 02:26:29", "content": "

i wrote a fancy ASCII viewer of categorical variables in csv files.

\n

you can use it like this:

\n

csvcut -c click,site_domain,app_domain train | ./ctviewer.py -2 20 20

\n

csvcut is included in csvkit.

", "votes": null, "replies": [] }, { "id": 89545, "author_name": "ducnguyen1911", "author_url": "", "post_date": "08/17/2015 07:50:10", "content": "

We can use command 'less' in Linux to view content of a big file, e.g. "less train.csv".

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "58558": "", "58560": "", "58567": "", "59678": "", "59979": "", "59999": "", "60000": "", "60127": "", "89545": "We can use command 'less' in Linux to view content of a big file, e.g. \"less train.csv\"." }, "source": "meta" }