Getting a Cross-Section from Two CSV Files

Posted by Jonathan Sampson on Super User See other posts from Super User or by Jonathan Sampson
Published on 2010-05-17T18:24:41Z Indexed on 2010/05/17 18:31 UTC
Read the original article Hit count: 204

Filed under:

csv

|

openoffice.org

|

mysql

|

best-practices

I have two CSV files that I am working with. One is massive, with about 200,000 rows. The other is much smaller, having about 12,000 rows. Both fit the same format of names, and email addresses (everything is legit here, no worries). Basically I'm trying to get only a subset of the second list by removing all values that presently exist in the larger file.

So, List A has ~200k rows, and List B has ~12k. These lists overlap a bit, and I'd like to remove all entries from List B if they also exist in List A, leaving me with new and unique values only in List B. I've got a few tooks at my disposal that I can use. Open Office is loaded on this machine, along with MySQL (queries are alright).

What's the easiest way to create a third CSV with the intersection of data?

© Super User or respective owner

Related posts about csv

DBD::CSV: How can I generate different behavior with the two f_ext-options ".csv" and ".csv/r"?

as seen on Stack Overflow - Search for 'Stack Overflow'
This is from the DBD::File-documentation: f_ext This attribute is used for setting the file extension where (CSV) files are opened. There are several possibilities. DBI:CSV:f_dir=data;f_ext=.csv In this case, DBD::File will open only table.csv if both table.csv and table exist… >>> More
SUPER CSV write bean to CSV.

as seen on Stack Overflow - Search for 'Stack Overflow'
Here is my class, public class FreebasePeopleResults { public String intendedSearch; public String weight; public Double heightMeters; public Integer age; public String type; public String parents; public String profession; public String alias; public String children; public String siblings; public… >>> More
export to csv using fastercsv and CSV::Writer (Ruby on Rails)

as seen on Stack Overflow - Search for 'Stack Overflow'
What am I trying to do: export data to csv. I have a form which allows user to select the format (from a drop down menu). So based on the selection of the format the ouput is displayed using a ajax call. Works fine for html but when I select the format as csv I don't see any pop up on the screen… >>> More
How can i make changes to this file Encoding?

as seen on Super User - Search for 'Super User'
I have these 3 files 21/08/2014 07:15 PM 122 Tw2AWK.csv 21/08/2014 07:15 PM 125 Tw2Notepad.csv 21/08/2014 07:15 PM 119 Tw2REPL.csv C:\myfilesfile Tw2AWK.csv TwREPL.csv Tw2Notepad.csv Tw2AWK.csv; UTF-8 Unicode text, with CRLF line terminators Tw2REPL… >>> More
Unix sort keys cause performance problems

as seen on Super User - Search for 'Super User'
My data: It's a 71 MB file with 1.5 million rows. It has 6 fields All six fields combine to form a unique key - so that's what I need to sort on. Sort statement: sort -t ',' -k1,1 -k2,2 -k3,3 -k4,4 -k5,5 -k6,6 -o output.csv input.csv The problem: If I sort without keys, it takes 30 seconds… >>> More

Related posts about openoffice.org

apt-get is broken

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I Cannot install any package in the server, As I am newbie in Server. In Morning I found that some, I am not able to install any package from command line in the server,Now every package is now manually downloaded packages and then installed in the server. Can any one Please tell me what is the issue… >>> More
Install OpenOffice for headless use

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
The question very simple and short. I am following this article to install OpenOffice headless: sudo apt-get install openoffice.org-headless openoffice.org-writer openoffice.org-draw And I get the following error : Reading package lists... Done Building dependency tree Reading state information… >>> More
Install OOo for headless use

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
The question very simple and short. I am following this article to install OpenOfficle headless: sudo apt-get install openoffice.org-headless openoffice.org-writer openoffice.org-draw And I get the following error : Reading package lists... Done Building dependency tree Reading state information… >>> More
How to recover dpkg from corrupted downloads?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I just upgraded to the 10.10 RC earlier and had a few problems with graphic drivers (x didnt start) But i have remedied that now. When i run 'sudo apt-get install -f' i get this: will@UbuntuBox:/mnt/slax$ sudo apt-get install -f [sudo] password for will: Reading package lists... Done Building dependency… >>> More
Programmatically convert *.odt file to MS Word *.doc file using an OpenOffice.org basic macro

as seen on Stack Overflow - Search for 'Stack Overflow'
I am trying to build a reStructuredText to MS Word document tool-chain, so I will be able to save only the rst sources in version control. So far I -- Have rst2odt.py to convert reStructuredText to OpenOffice.org Writer format. Next I want to use the most recent OpenOffice.org (currently 3.1) that… >>> More