Search Results

Search found 1639 results on 66 pages for 'csv'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 18/66 | < Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >

  • importing csv file into pgsql

    - by running4surival
    ok im trying to upload this csv file onto my table in pgsql but im getting this error ERROR: invalid input syntax for integer: "mlname,mfname,slname,sfname,address,postalcode,membershiptype,hphone,email" CONTEXT: COPY members2, line 1, column id: "mlname,mfname,slname,sfname,address,postalcode,membershiptype,hphone,email" i really understand why im getting this error, both my table and my csv file have the same column names

    Read the article

  • Programmatically import CSV data to Access

    - by FrustratedWithFormsDesigner
    I have an Access database and the source of data comes from generated CSV files. I'd like to have an easy way for the users to simply select the data file and import it. Import should append the existing data to the data already in the data table. Is there a way in Access to create a file selector and import using saved CSV import settings that are already in the file?

    Read the article

  • How to import this data set into excel? (column headings on each row delimited by a colon)

    - by Anonymous
    I'm trying to import the following data set into Excel. I've had no luck with the text import wizard. I'd like Excel to make id, name, street, etc the column names and insert each record onto a new row. , id: sdfg:435-345, name: Some Name, type: , street: Address Line 1, Some Place, postalcode: DN2 5FF, city: Cityhere, telephoneNumber: 01234 567890, mobileNumber: 01234 567890, faxNumber: /, url: http://www.website.co.uk, email: [email protected], remark: , geocode: 526.2456;-0.8520, category: some, more, info , id: sdfg:435-345f, name: Some Name, type: , street: Address Line 1, Some Place, postalcode: DN2 5FF, city: Cityhere, telephoneNumber: 01234 567890, mobileNumber: 01234 567890, faxNumber: /, url: http://www.website.co.uk, email: [email protected], remark: , geocode: 526.2456;-0.8520, category: some, more, info Is there any easy way to do this with Excel? I'm struggling to think of a way to convert this to a conventional CSV easily. As far as I can think, I'd have to remove the labels from each line, enclose each line in quotes, then delimit them with commas. Obviously that's made a little more difficult to script though seeing as some fields (address, for instance) contain comma-delimited data. I'm not good with regex at all. What's the best way to tackle this?

    Read the article

  • Saving a Django form to a csv file

    - by Oli
    I have a Django form that is working fine. I'd like to save the data it submits to a CSV file. Is there a "best practice" way to do this? I need to include blank fields in the CSV file where the user has not filled in a "required=False" field

    Read the article

  • Optimizing python code performance when importing zipped csv to a mongo collection

    - by mark
    I need to import a zipped csv into a mongo collection, but there is a catch - every record contains a timestamp in Pacific Time, which must be converted to the local time corresponding to the (longitude,latitude) pair found in the same record. The code looks like so: def read_csv_zip(path, timezones): with ZipFile(path) as z, z.open(z.namelist()[0]) as input: csv_rows = csv.reader(input) header = csv_rows.next() check,converters = get_aux_stuff(header) for csv_row in csv_rows: if check(csv_row): row = { converter[0]:converter[1](value) for converter, value in zip(converters, csv_row) if allow_field(converter) } ts = row['ts'] lng, lat = row['loc'] found_tz_entry = timezones.find_one(SON({'loc': {'$within': {'$box': [[lng-tz_lookup_radius, lat-tz_lookup_radius],[lng+tz_lookup_radius, lat+tz_lookup_radius]]}}})) if found_tz_entry: tz_name = found_tz_entry['tz'] local_ts = ts.astimezone(timezone(tz_name)).replace(tzinfo=None) row['tz'] = tz_name else: local_ts = (ts.astimezone(utc) + timedelta(hours = int(lng/15))).replace(tzinfo = None) row['local_ts'] = local_ts yield row def insert_documents(collection, source, batch_size): while True: items = list(itertools.islice(source, batch_size)) if len(items) == 0: break; try: collection.insert(items) except: for item in items: try: collection.insert(item) except Exception as exc: print("Failed to insert record {0} - {1}".format(item['_id'], exc)) def main(zip_path): with Connection() as connection: data = connection.mydb.data timezones = connection.timezones.data insert_documents(data, read_csv_zip(zip_path, timezones), 1000) The code proceeds as follows: Every record read from the csv is checked and converted to a dictionary, where some fields may be skipped, some titles be renamed (from those appearing in the csv header), some values may be converted (to datetime, to integers, to floats. etc ...) For each record read from the csv, a lookup is made into the timezones collection to map the record location to the respective time zone. If the mapping is successful - that timezone is used to convert the record timestamp (pacific time) to the respective local timestamp. If no mapping is found - a rough approximation is calculated. The timezones collection is appropriately indexed, of course - calling explain() confirms it. The process is slow. Naturally, having to query the timezones collection for every record kills the performance. I am looking for advises on how to improve it. Thanks. EDIT The timezones collection contains 8176040 records, each containing four values: > db.data.findOne() { "_id" : 3038814, "loc" : [ 1.48333, 42.5 ], "tz" : "Europe/Andorra" } EDIT2 OK, I have compiled a release build of http://toblerity.github.com/rtree/ and configured the rtree package. Then I have created an rtree dat/idx pair of files corresponding to my timezones collection. So, instead of calling collection.find_one I call index.intersection. Surprisingly, not only there is no improvement, but it works even more slowly now! May be rtree could be fine tuned to load the entire dat/idx pair into RAM (704M), but I do not know how to do it. Until then, it is not an alternative. In general, I think the solution should involve parallelization of the task. EDIT3 Profile output when using collection.find_one: >>> p.sort_stats('cumulative').print_stats(10) Tue Apr 10 14:28:39 2012 ImportDataIntoMongo.profile 64549590 function calls (64549180 primitive calls) in 1231.257 seconds Ordered by: cumulative time List reduced from 730 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.012 0.012 1231.257 1231.257 ImportDataIntoMongo.py:1(<module>) 1 0.001 0.001 1230.959 1230.959 ImportDataIntoMongo.py:187(main) 1 853.558 853.558 853.558 853.558 {raw_input} 1 0.598 0.598 370.510 370.510 ImportDataIntoMongo.py:165(insert_documents) 343407 9.965 0.000 359.034 0.001 ImportDataIntoMongo.py:137(read_csv_zip) 343408 2.927 0.000 287.035 0.001 c:\python27\lib\site-packages\pymongo\collection.py:489(find_one) 343408 1.842 0.000 274.803 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:699(next) 343408 2.542 0.000 271.212 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:644(_refresh) 343408 4.512 0.000 253.673 0.001 c:\python27\lib\site-packages\pymongo\cursor.py:605(__send_message) 343408 0.971 0.000 242.078 0.001 c:\python27\lib\site-packages\pymongo\connection.py:871(_send_message_with_response) Profile output when using index.intersection: >>> p.sort_stats('cumulative').print_stats(10) Wed Apr 11 16:21:31 2012 ImportDataIntoMongo.profile 41542960 function calls (41542536 primitive calls) in 2889.164 seconds Ordered by: cumulative time List reduced from 778 to 10 due to restriction <10> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.028 0.028 2889.164 2889.164 ImportDataIntoMongo.py:1(<module>) 1 0.017 0.017 2888.679 2888.679 ImportDataIntoMongo.py:202(main) 1 2365.526 2365.526 2365.526 2365.526 {raw_input} 1 0.766 0.766 502.817 502.817 ImportDataIntoMongo.py:180(insert_documents) 343407 9.147 0.000 491.433 0.001 ImportDataIntoMongo.py:152(read_csv_zip) 343406 0.571 0.000 391.394 0.001 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:384(intersection) 343406 379.957 0.001 390.824 0.001 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:435(_intersection_obj) 686513 22.616 0.000 38.705 0.000 c:\python27\lib\site-packages\rtree-0.7.0-py2.7.egg\rtree\index.py:451(_get_objects) 343406 6.134 0.000 33.326 0.000 ImportDataIntoMongo.py:162(<dictcomp>) 346 0.396 0.001 30.665 0.089 c:\python27\lib\site-packages\pymongo\collection.py:240(insert) EDIT4 I have parallelized the code, but the results are still not very encouraging. I am convinced it could be done better. See my own answer to this question for details.

    Read the article

  • MySQL: Blank row in table after LOAD DATA INFILE

    - by Tom
    Hi, I'm uploading a large amount of data from a CSV (I'm doing it via MySQL Workbench): LOAD DATA INFILE 'C:/development/mydoc.csv' INTO TABLE mydatabase.mytable CHARACTER SET utf8 FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\r'; However, I'm noticing that it keeps adding an empty line full of nulls/zeros after the last record. I'm guessing it's because of the "LINES TERMINATED" command. However, I need that to load the data in correctly. Is there some way around this / some better SQL to avoid the blank row in the table? Thanks

    Read the article

  • Import data from .xls File

    - by tek3
    HI, I am developing an application in which i have to get data from .xls file. I am fairly new to iPhone development so any pointers in direction to getting started will be very much helpful. The steps that i am thinking are : 1) First i need to convert .xls to .csv Format. 2)Import the data from .csv file to SQlite Databse or Core Data.(I am not familier with any one of them, so kindly suggest which one to choose. I am looking forward to use Core Data) Am I thinking in the right direction. Will be greatful for any kind of assistance.. Thanx in advance..

    Read the article

  • Using excel import wizard through code?

    - by baron
    I am creating a Generate Report button for some data in my application. The report will just generate a CSV file, which I then automatically launch in excel. Problem is with one of the data fields, is text and has leading zeroes (which I need to keep). When the file is automatically opened in excel, excel automatically trims these. Is there anyway I can get it so that it opens with the data in the format I want it. I know you achieve this normally by opening the CSV file from within excel and can do the import wizard. So one potential solution would be to auto open the import wizard when the app launches this file. I don't know if this is possible. Thanks for reading.

    Read the article

  • Reading a text file in MATLAB line by line

    - by Hossein
    Hi, I have a CSV file, I want to read this file and do some pre-calculations on each row to see for example that row is useful for me or not and if yes I save it to a new CSV file. can someone give me an example? in more details this is how my data looks like: (string,float,float) the numbers are coordinates. ABC,51.9358183333333,4.183255 ABC,51.9353866666667,4.1841 ABC,51.9351716666667,4.184565 ABC,51.9343083333333,4.186425 ABC,51.9343083333333,4.186425 ABC,51.9340916666667,4.18688333333333 basically i want to save the rows that have for distances more than 50 or 50 in a new file.the string field should also be copied. thanks

    Read the article

  • Oracle UTL_FILE - multiple rows in excel for a single cell

    - by user1879150
    I have a requirement to display the below in a csv file which is generated using UTL_FILE package of PL/SQL. Heading1 Heading2 Heading3 ABCDE 123 987641213 xxx street yyyy For each value in 'Heading1', I get 3 rows for address from a particular table. In the output csv file, I need to display the address in a single cell with data separated by a new line. I tried using a cursor and appending data using chr(10), chr(12), chr(15) but in vain. Kindly help me in getting the output.

    Read the article

  • formatting fgetcsv output

    - by Patrick
    Im trying to figure out how to take the data returned by fgetcsv, and format it into a readable/editable table, and then use fputcsv to save that table so far i have this <?php $row = 1; $handle = fopen("csv.csv", "r"); while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) { $num = count($data); $row++; for ($c=0; $c < $num; $c++) { echo $data[$c] . "\n"; } } fclose($handle); ?>

    Read the article

  • Only strip commas surrounded by numbers in a string

    - by overlook77
    I am using StreamReader to read each line of a CSV file into a string. As I am processing each line, I need to strip out any commas that ONLY are surrounded by other numbers. For example, if the string was: "textfield1", "textfield2", "100.00", "1,070.00" I would need to only take the comma out of "1,070.00" from this entire string, resulting in: "textfield1", "textfield2", "100.00", "1070.00" Each string read from the CSV file can differ in number of fields, length, etc., so I need to use something (Regular Expressions maybe?) that looks at the entire string without hardcoding a location or blanket removal of all commas. Here is the approach I have been trying: StreamReader sr = new StreamReader(strInputFile); string nextLine = sr.ReadLine(); try { while ((nextLine = sr.ReadLine()) != null) { string rawtext = nextLine.Replace("[0-9]"+","+"[0-9]" , "[0-9]"+"[0-9]"); // ....rest of code } } This obviously doesn't work because I don't understand how to do this :) I am new at C# and inexperienced in Regex, so hopefully this is relatively simple.

    Read the article

  • Converting Outlook Express csv adress book and dbx files into Thunderbird on W7

    - by PiotrK
    Recently I changed my OS from XP to W7. I made backup of any Outlook Express messages (the dbx files and adress book as CSV). On W7 I want to import that data into Thunderbird. There is option for importing from Outlook Express, but it is looking for live application data (I can't specify directory with real files myself) and there is no Outlook Express installed on W7 so I can't just import it back to it and then into Thunderbird. How can I import that data into Thunderbird?

    Read the article

  • Importing from CSV and sorting by Date

    - by Andrew Rice
    I have the following script that parses an HR output file looking for employees and outputs information such as Hire Dare, First Name, Last Name, Supervisor etc. The problem I have is that in the current format I think the Hire Date column is being treated as a string so in effect it orders the output by month (i.e. 1/1/01 comes before 2/2/98). Is there a way to map that column to a date/time so it sorts properly? Import-CSV -delimiter "`t" Output.tab | Where-Object {$_.'First Nae' -like '*And*'} | Sort-Object 'Hire Date' | ft 'Hire Date', 'First Name'

    Read the article

  • Script or Utility to convert .nab to .csv without importing double entries in Outlook

    - by Chris
    Currently our environment is migrating from Groupwise 7 to Outlook 2003 and we have multiple users with mission critical outside contacts in their frequent contacts that will have to be imported in Outlook. Currently our only solution is to export GW contacts to a .nab, import to excel to scrub out the contacts in our own domain (to avoid double entry) and convert to .csv. This current solution will require a lot of man hours for hand holding because most of our users are not technically savvy AT ALL and are frankly too busy to do this themselves. Anyone know of any kind of tool or script to assist with this?

    Read the article

  • Conditional Regular Expression testing of a CSV

    - by Alastair Pitts
    I am doing some client side validation in ASP.NET MVC and I found myself trying to do conditional validation on a set of items (ie, if the checkbox is checked then validate and visa versa). This was problematic, to say the least. To get around this, I figured that I could "cheat" by having a hidden element that would contain all of the information for each set, thus the idea of a CSV string containing this information. I already use a custom [HiddenRequired] attribute to validate if the hidden input contains a value, with success, but I thought as I will need to validate each piece of data in the csv, that a regular expression would solve this. My regular expression work is extremely weak and after a good 2 hours I've almost given up. This is an example of the csv string: true,3,24,over,0.5 to explain: true denotes if I should validate the rest. I need to conditionally switch in the regex using this 3 and 24 are integers and will only ever fall in the range 0-24. over is a string and will either be over or under 0.5 is a decimal value, of unknown precision. In the validation, all values should be present and at least of the correct type Is there someone who can either provide such a regex or at least provide some hints, i'm really stuck!

    Read the article

  • process csv in scala

    - by portoalet
    I am using scala 2.7.7, and wanted to parse CSV file and store the data in SQLite database. I ended up using OpenCSV java library to parse the CSV file, and using sqlitejdbc library. Using these java libraries makes my scala code looks almost identical to that of Java code (sans semicolon and with val/var) As I am dealing with java objects, I can't use scala list, map, etc, unless I do scala2java conversion or upgrade to scala 2.8 Is there a way I can simplify my code further using scala bits that I don't know? val filename = "file.csv"; val reader = new CSVReader(new FileReader(filename)) var aLine = new Array[String](10) var lastSymbol = "" while( (aLine = reader.readNext()) != null ) { if( aLine != null ) { val symbol = aLine(0) if( !symbol.equals(lastSymbol)) { try { val rs = stat.executeQuery("select name from sqlite_master where name='" + symbol + "';" ) if( !rs.next() ) { stat.executeUpdate("drop table if exists '" + symbol + "';") stat.executeUpdate("create table '" + symbol + "' (symbol,data,open,high,low,close,vol);") } } catch { case sqle : java.sql.SQLException => println(sqle) } lastSymbol = symbol } val prep = conn.prepareStatement("insert into '" + symbol + "' values (?,?,?,?,?,?,?);") prep.setString(1, aLine(0)) //symbol prep.setString(2, aLine(1)) //date prep.setString(3, aLine(2)) //open prep.setString(4, aLine(3)) //high prep.setString(5, aLine(4)) //low prep.setString(6, aLine(5)) //close prep.setString(7, aLine(6)) //vol prep.addBatch() prep.executeBatch() } } conn.close()

    Read the article

  • scraping text from multiple html files into a single csv file

    - by Lulu
    I have just over 1500 html pages (1.html to 1500.html). I have written a code using Beautiful Soup that extracts most of the data I need but "misses" out some of the data within the table. My Input: e.g file 1500.html My Code: #!/usr/bin/env python import glob import codecs from BeautifulSoup import BeautifulSoup with codecs.open('dump2.csv', "w", encoding="utf-8") as csvfile: for file in glob.glob('*html*'): print 'Processing', file soup = BeautifulSoup(open(file).read()) rows = soup.findAll('tr') for tr in rows: cols = tr.findAll('td') #print >> csvfile,"#".join(col.string for col in cols) #print >> csvfile,"#".join(td.find(text=True)) for col in cols: print >> csvfile, col.string print >> csvfile, "===" print >> csvfile, "***" Output: One CSV file, with 1500 lines of text and columns of data. For some reason my code does not pull out all the required data but "misses" some data, e.g the Address1 and Address 2 data at the start of the table do not come out. I modified the code to put in * and === separators, I then use perl to put into a clean csv file, unfortunately I'm not sure how to work my code to get all the data I'm looking for!

    Read the article

  • Creating multiple csv files from data within a csv file.

    - by S1syphus
    System OSX or Linux I'm trying to automate my work flow at work, each week I receive an excel file, which I convert to a csv. An example is: ,,L1,,,L2,,,L3,,,L4,,,L5,,,L6,,,L7,,,L8,,,L9,,,L10,,,L11, Title,r/t,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,neede d,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst EXAMPLEfoo,60,6,6,6,0,0,0,0,0,0,6,6,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLEbar,30,6,6,12,6,7,14,6,6,12,6,6,12,6,8,16,6,7,14,6,7.5,15,6,6,12,6,8,16,6,0,0,6,7,14 EXAMPLE1,60,3,3,3,3,5,5,3,4,4,3,3,3,3,6,6,3,4,4,3,3,3,3,4,4,3,8,8,3,0,0,3,4,4 EXAMPLE2,120,6,6,3,0,0,0,6,8,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLE3,60,6,6,6,6,8,8,6,6,6,6,6,6,0,0,0,0,0,0,6,8,8,6,6,6,0,0,0,0,0,0,0,10,10 EXAMPLE4,30,6,6,12,6,7,14,6,6,12,6,6,12,3,5.5,11,6,7.5,15,6,6,12,6,0,0,6,9,18,6,0,0,6,6.5,13 And so you can get a picture of how it looks in excel: What I need to do, is create multiple csv files for each instance in row 1, so L1, L2, L3, L4... And within that each csv file it needs to contain the title, r/t, needed So for L1 an example out put would look like: EXAMPLEfoo,60,6 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,6 EXAMPLE3,60,6 EXAMPLE4,30,6 And for L2: EXAMPLEfoo,60,0 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,0 EXAMPLE3,60,6 EXAMPLE4,30,6 And so on. I have tried playing around with sed and awk and hit google but I have found nothing that really solves the issue. I'd imagine perl would be particular suited to this or maybe python, so I would be more than happy to accept suggestions from users. So, any suggestions? Thanks in advance.

    Read the article

  • Storing header and data sections in a CSV file

    - by morpheous
    This should be relatively easy to do, but after several hours straight programming my mind seems a bit frazzled and could do with some help. I have a C++ class which I am currently using to store read/write data to file. I was initially using binary data, but have decided to store the data as CSV in order to let programs written in other languages be able to load the data. The C++ class looks a bit like this: class BinaryData { public: BinaryData(); void serialize(std::ostream& output) const; void deserialize(std::istream& input); private: Header m_hdr; std::vector<Row> m_rows; }; I am simply rewriting the serialize/deserialize methods to write to a CSV file. I am not sure on the "best" way to store a header section and a "data" section in a "flat" CSV file though - any suggestions on the most sensible way to do this?

    Read the article

  • Export sheet from Excel to CSV

    - by Mike Wills
    I am creating a spread sheet to help ease the entry of data into one of our systems. They are entering inventory items into this spread sheet to calculate the unit cost of the item (item cost + tax + S&H). The software we purchased cannot do this. Aan invoice can have one or more lines (duh!) and I calculate the final unit cost. This is working fine. I then want to take that data and create a CSV from that so they can load it into our inventory system. I currently have a second tab that is laid out like I want the CSV, and I do an equal cell (=Sheet!A3) to get the values on the "export sheet". The problem is when they save this to a CSV, there are many blank lines that need to be deleted before they can upload it. I want a file that only contains the data that is needed. I am sure this could be done in VBA, but I don't know where to start or know how to search for an example to start. Any direction or other options would be appreciated.

    Read the article

  • Strange behavior while returning csv file from spring controller

    - by Fanooos
    I working in a spring application which have an action method that returns a CSV file. This action works fine but in some cases it throws a predefined exception (MyAppException). I have another method that is annotated @ExceptionHandler(MyAppException.class) In the exception handler method I return another csv file but with different contents. The code that returns the csv file is almost the same in the two methods. List<String[]> list= new ArrayList<String[]>(); list.add(new String[]{ integrationRequestErrorLog.getErrorMessage(), Long.toString(integrationRequestErrorLog.getId()), Integer.toString(integrationRequestErrorLog.getErrorCode()) }); CSVWriter writer = new CSVWriter(response.getWriter(), ','); writer.writeAll(list); writer.close(); the difference between the two method is the list of contents. In the first method the file is returned normally while in the exception handler method I have a strange behavior. The exception handler method works fine with Opera browser while it gives me a 404 with FireFox. Opera browser give me 404 also but it download the file while firefox does not? Really I do not understand what is the difference here.

    Read the article

  • How to update csv column names with database table header

    - by user1523311
    I am facing this problem some past days and now frustrate because I have to do it. I need to update my CSV file columns name with database table header. My database table fields are different from CSV file. Now the problem is that first I want to update column name of CSV file with database table headers and then import its data with field mapping into database. Please help me I don't know how I can solve this. This is my php code: $file = $_POST['fileName']; $filename = "../files/" . $file; $list = $_POST['str']; $array_imp = implode(',', $list); $array_exp = explode(',', $array_imp); $fp = fopen("../files/" . $file, "w"); $num = count($fp); for ($c = 0; $c < $num; $c++) { if ($fp[c] !== '') { fputcsv($fp, $array_exp); } } fclose($fp);

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 14 15 16 17 18 19 20 21 22 23 24 25  | Next Page >