Search Results

Search found 15000 results on 600 pages for 'python csv'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 3/600 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • Excel CSV into Nested Dictionary; List Comprehensions

    - by victorhooi
    heya, I have a Excel CSV files with employee records in them. Something like this: mail,first_name,surname,employee_id,manager_id,telephone_number [email protected],john,smith,503422,503423,+65(2)3423-2433 [email protected],george,brown,503097,503098,+65(2)3423-9782 .... I'm using DictReader to put this into a nested dictionary: import csv gd_extract = csv.DictReader(open('filename 20100331 original.csv'), dialect='excel') employees = dict([(row['employee_id'], row) for row in gp_extract]) Is the above the proper way to do it - it does work, but is it the Right Way? Something more efficient? Also, the funny thing is, in IDLE, if I try to print out "employees" at the shell, it seems to cause IDLE to crash (there's approximately 1051 rows). 2. Remove employee_id from inner dict The second issue issue, I'm putting it into a dictionary indexed by employee_id, with the value as a nested dictionary of all the values - however, employee_id is also a key:value inside the nested dictionary, which is a bit redundant? Is there any way to exclude it from the inner dictionary? 3. Manipulate data in comprehension Thirdly, we need do some manipulations to the imported data - for example, all the phone numbers are in the wrong format, so we need to do some regex there. Also, we need to convert manager_id to an actual manager's name, and their email address. Most managers are in the same file, while others are in an external_contractors CSV, which is similar but not quite the same format - I can import that to a separate dict though. Are these two items things that can be done within the single list comprehension, or should I use a for loop? Or does multiple comprehensions work? (sample code would be really awesome here). Or is there a smarter way in Python do it? Cheers, Victor

    Read the article

  • writing header in csv python with DictWriter

    - by user248237
    assume I have a csv.DictReader object and I want to write it out as a csv file. How can I do this? I thought of the following: dr = csv.DictReader(open(f), delimiter='\t') # process my dr object # ... # write out object output = csv.DictWriter(open(f2, 'w'), delimiter='\t') for item in dr: output.writerow(item) Is that the best way? More importantly, how can I make it so a header is written out too, in this case the object "dr"s .fieldnames property? thanks.

    Read the article

  • Export Foxbase database tables to csv

    - by RKS
    I have no experience with Foxbase whatsoever and I'm used to working with MySQL via phpmyadmin or interfaces like that. My company has a third party database we're trying to move away from, but we have no support from the company. The database is on our servers, but in a foxbase format. What kind tools do I need to convert these into other formats, or is there any type of admin UI I can tell them to use to export to csv or anything like that? Basically I'm asking how to export the foxbase tables to csv. Sorry if this question isn't clear or you need more information. I will edit with anything else you need.

    Read the article

  • CSV files not being written

    - by Kamalpreet
    I worked on a small project in which the data entered in HTML forms was saved in a CSV file which was used subsequently. The files were run on Apache2. It worked fine. After about 25 days, when I reopened the project, the data entered in forms was not saved in CSV file. I checked all the permissions. I even sent a zip file of my files to one of m friends. It worked well on his system. So should I figure it out there's some problem in the system. I am using Ubuntu 13.04. Kindly suggest me something so that I am able to figure out the problem.

    Read the article

  • Upload and parse csv file with "universal newline" in python on Google App Engine

    - by greg
    Hi, I'm uploading a csv/tsv file from a form in GAE, and I try to parse the file with python csv module. Like describe here, uploaded files in GAE are strings. So I treat my uploaded string a file-like object : file = self.request.get('catalog') catalog = csv.reader(StringIO.StringIO(file),dialect=csv.excel_tab) But new lines in my files are not necessarily '\n' (thanks to excel..), and it generated an error : Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode? Does anyone know how to use StringIO.StringIO to treat strings like files open in universal-newline?

    Read the article

  • How can i add encoding to the python generated CSV file

    - by user1958218
    I am following this post http://stackoverflow.com/a/9016545 and i want to know that how can i do that in Python. I don't know how can i insert BOM data in there This is my current code response = HttpResponse(content_type='text/csv') response['Content-Type'] = 'application/octet-stream' response['Content-Disposition'] = 'attachment; filename="results.csv"' writer = UnicodeWriter(response, quoting=csv.QUOTE_ALL, encoding="utf-8") I want to convert to utf -16 . BOm data is this but don't know how to insert it From here http://stackoverflow.com/a/4440143 echo "\xEF\xBB\xBF"; // UTF-8 BOM But i want it for python and utf-16 I tried opening that csv in notepad and insert \xef\xbb\xb in beginning and excel displayed that correctly. But it is also visible before first column. How can i hide that because user wont like that

    Read the article

  • 50 million+ Rows of Data - CSV or MySQL

    - by eWizardII
    Hello, I have a CSV file which is about 1GB big and contains about 50million rows of data, I am wondering is it better to keep it as a CSV file or store it as some form of a database. I don't know a great deal about MySQL to argue for why I should use it or another database framework over just keeping it as a CSV file. I am basically doing a Breadth-First Search with this dataset, so once I get the initial "seed" set the 50million I use this as the first values in my queue. Thanks,

    Read the article

  • Clarification needed about Python CSV file format parsing

    - by HH
    Format is like: CHINA;2002-06-25 00:00:00.000;5,60 CHINA;2002-06-26 00:00:00.000;5,32 CHINA;2002-06-27 00:00:00.000;5,31 and I try to use Python's CSV tools to parse it but cannot understand the paragraph, source: And while the module doesn’t directly support parsing strings, it can easily be done: import csv for row in csv.reader(['one,two,three']): print row Could someone clarify the line ['one,two,three']? How would you use it with format A;B;C?

    Read the article

  • How to install python modules for specific python version

    - by Zayatzz
    I needed to install UCS2 python next to UCS4 python. So I went to comp.lang.python and asked them about it. Probably not the best place to ask it, but they answered https://groups.google.com/forum/?fromgroups#!topic/comp.lang.python/bGuAfqa76W8 and now i have brand new python 2.7.3 ucs2 installed in /opt/bin/python What I need now is - how can I install all other python modules that I have installed for that python version also. Basically stuff like PIL and postgresql and mod_wsgi - basically everything needed to run Django for that python version. Is this the right the place to ask for it?

    Read the article

  • call python with system() in R to run a python script emulating the python console

    - by Yihui
    I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R? >>> print 'hello world' hello world This only shows the output: > system("python -c 'print \"hello world\"'") hello world Thanks! BTW, I asked in r-help but have not got a response yet (if I do, I'll post the answer here).

    Read the article

  • How to parse a CSV file containing serialized PHP? [migrated]

    - by garbetjie
    I've just started dabbling in Perl, to try and gain some exposure to different programming languages - so forgive me if some of the following code is horrendous. I needed a quick and dirty CSV parser that could receive a CSV file, and split it into file batches containing "X" number of CSV lines (taking into account that entries could contain embedded newlines). I came up with a working solution, and it was going along just fine. However, as one of the CSV files that I'm trying to split, I came across one that contains serialized PHP code. This seems to break the CSV parsing. As soon as I remove the serialization - the CSV file is parsed correctly. Are there any tricks I need to know when it comes to parsing serialized data in CSV files? Here is a shortened sample of the code: use strict; use warnings; my $csv = Text::CSV_XS->new({ eol => $/, always_quote => 1, binary => 1 }); my $out; my $in; open $in, "<:encoding(utf8)", "infile.csv" or die("cannot open input file $inputfile"); open $out, ">outfile.000"; binmode($out, ":utf8"); while (my $line = $csv->getline($in)) { $lines++; $csv->print($out, $line); } I'm never able to get into the while loop shown above. As soon as I remove the serialized data, I suddenly am able to get into the loop. Edit: An example of a line that is causing me trouble (taken straight from Vim - hence the ^M): "26","other","1","20,000 Subscriber Plan","Some text here.^M\ Some more text","on","","18","","0","","0","0","recurring","0","","payment","totalsend","0","tsadmin","R34bL9oq","37","0","0","","","","","","","","","","","","","","","","","","","","","","","0","0","0","a:18:{i:0;s:1:\"3\";i:1;s:1:\"2\";i:2;s:2:\"59\";i:3;s:2:\"60\";i:4;s:2:\"61\";i:5;s:2:\"62\";i:6;s:2:\"63\";i:7;s:2:\"64\";i:8;s:2:\"65\";i:9;s:2:\"66\";i:10;s:2:\"67\";i:11;s:2:\"68\";i:12;s:2:\"69\";i:13;s:2:\"70\";i:14;s:2:\"71\";i:15;s:2:\"72\";i:16;s:2:\"73\";i:17;s:2:\"74\";}","","","0","0","","0","0","0.0000","0.0000","0","","","0.00","","6","1" "27","other","1","35,000 Subscriber Plan","Some test here.^M\ Some more text","on","","18","","0","","0","0","recurring","0","","payment","totalsend","0","tsadmin","R34bL9oq","38","0","0","","","","","","","","","","","","","","","","","","","","","","","0","0","0","a:18:{i:0;s:1:\"3\";i:1;s:1:\"2\";i:2;s:2:\"59\";i:3;s:2:\"60\";i:4;s:2:\"61\";i:5;s:2:\"62\";i:6;s:2:\"63\";i:7;s:2:\"64\";i:8;s:2:\"65\";i:9;s:2:\"66\";i:10;s:2:\"67\";i:11;s:2:\"68\";i:12;s:2:\"69\";i:13;s:2:\"70\";i:14;s:2:\"71\";i:15;s:2:\"72\";i:16;s:2:\"73\";i:17;s:2:\"74\";}","","","0","0","","0","0","0.0000","0.0000","0","","","0.00","","7","1" "28","other","1","50,000 Subscriber Plan","Some text here.^M\ Some more text","on","","18","","0","","0","0","recurring","0","","payment","totalsend","0","tsadmin","R34bL9oq","39","0","0","","","","","","","","","","","","","","","","","","","","","","","0","0","0","a:18:{i:0;s:1:\"3\";i:1;s:1:\"2\";i:2;s:2:\"59\";i:3;s:2:\"60\";i:4;s:2:\"61\";i:5;s:2:\"62\";i:6;s:2:\"63\";i:7;s:2:\"64\";i:8;s:2:\"65\";i:9;s:2:\"66\";i:10;s:2:\"67\";i:11;s:2:\"68\";i:12;s:2:\"69\";i:13;s:2:\"70\";i:14;s:2:\"71\";i:15;s:2:\"72\";i:16;s:2:\"73\";i:17;s:2:\"74\";}","","","0","0","","0","0","0.0000","0.0000","0","","","0.00","","8","1""73","other","8","10,000,000","","","","0","","0","","0","0","recurring","0","","payment","","0","","","75","0","10000000","","","","","","","","","","","","","","","","","","","","","","","0","0","0","a:17:{i:0;s:1:\"3\";i:1;s:1:\"2\";i:2;s:2:\"59\";i:3;s:2:\"60\";i:4;s:2:\"61\";i:5;s:2:\"62\";i:6;s:2:\"63\";i:7;s:2:\"64\";i:8;s:2:\"65\";i:9;s:2:\"66\";i:10;s:2:\"67\";i:11;s:2:\"68\";i:12;s:2:\"69\";i:13;s:2:\"70\";i:14;s:2:\"71\";i:15;s:2:\"72\";i:16;s:2:\"74\";}","","","0","0","","0","0","0.0000","0.0000","0","","","0.00","","14","0"

    Read the article

  • Installing PIL (Python Imaging Library) in Win7 64 bits, Python 2.6.4

    - by Rafael Almeida
    I'm trying to install said library for use with Python. I tried downloading the executable installer for Windows, which runs, but says it doesn't find a Python installation. Then tried registering (http://effbot.org/zone/python-register.htm) Python, but the script says it can't register (although the keys appear in my register). Then I tried downloading the source package: I run the setup.py build and it works, but when I run setup.py install it says the following: running install running build running build_py running build_ext building '_imaging' extension error: Unable to find vcvarsall.bat What can I do?

    Read the article

  • CSV engine on MySQL server

    - by Jeff
    I don't think that this is a programming question so I am going to ask it here - Reading the book high performance mysql, I read about the CSV engine. The paragraph says: The CSV engine can treat comma-separated values (CSV) files as table, but it does not support indexes on them. This engine lets you copy files in and out of the database while the server is running. If you export a CSV file from a spreadsheet and save it in the MySQL server's data directory, the server can read it immediately. Similary, if you write data to a CSV table, an external program can read it right away. CSV tables are especially useful as a data interchange format and for certain kinds of logging. What I get from this paragraph is that I can copy a .CSV file into the data directory of database, and it should show as a table that is able to be read from. However, whenever I copy a test .csv file into the directory, it does not appear as a table. I can't access it. I am using MySQL 5.5 also Does anyone know why this is not working, or what I am doing wrong? Thanks

    Read the article

  • SUPER CSV write bean to CSV.

    - by ButtersB
    Here is my class, public class FreebasePeopleResults { public String intendedSearch; public String weight; public Double heightMeters; public Integer age; public String type; public String parents; public String profession; public String alias; public String children; public String siblings; public String spouse; public String degree; public String institution; public String wikipediaId; public String guid; public String id; public String gender; public String name; public String ethnicity; public String articleText; public String dob; public String getWeight() { return weight; } public void setWeight(String weight) { this.weight = weight; } public Double getHeightMeters() { return heightMeters; } public void setHeightMeters(Double heightMeters) { this.heightMeters = heightMeters; } public String getParents() { return parents; } public void setParents(String parents) { this.parents = parents; } public Integer getAge() { return age; } public void setAge(Integer age) { this.age = age; } public String getProfession() { return profession; } public void setProfession(String profession) { this.profession = profession; } public String getAlias() { return alias; } public void setAlias(String alias) { this.alias = alias; } public String getChildren() { return children; } public void setChildren(String children) { this.children = children; } public String getSpouse() { return spouse; } public void setSpouse(String spouse) { this.spouse = spouse; } public String getDegree() { return degree; } public void setDegree(String degree) { this.degree = degree; } public String getInstitution() { return institution; } public void setInstitution(String institution) { this.institution = institution; } public String getWikipediaId() { return wikipediaId; } public void setWikipediaId(String wikipediaId) { this.wikipediaId = wikipediaId; } public String getGuid() { return guid; } public void setGuid(String guid) { this.guid = guid; } public String getId() { return id; } public void setId(String id) { this.id = id; } public String getGender() { return gender; } public void setGender(String gender) { this.gender = gender; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getEthnicity() { return ethnicity; } public void setEthnicity(String ethnicity) { this.ethnicity = ethnicity; } public String getArticleText() { return articleText; } public void setArticleText(String articleText) { this.articleText = articleText; } public String getDob() { return dob; } public void setDob(String dob) { this.dob = dob; } public String getType() { return type; } public void setType(String type) { this.type = type; } public String getSiblings() { return siblings; } public void setSiblings(String siblings) { this.siblings = siblings; } public String getIntendedSearch() { return intendedSearch; } public void setIntendedSearch(String intendedSearch) { this.intendedSearch = intendedSearch; } } Here is my CSV writer method import java.io.FileWriter; import java.io.IOException; import java.util.ArrayList; import org.supercsv.io.CsvBeanWriter; import org.supercsv.prefs.CsvPreference; public class CSVUtils { public static void writeCSVFromList(ArrayList<FreebasePeopleResults> people, boolean writeHeader) throws IOException{ //String[] header = new String []{"title","acronym","globalId","interfaceId","developer","description","publisher","genre","subGenre","platform","esrb","reviewScore","releaseDate","price","cheatArticleId"}; FileWriter file = new FileWriter("/brian/brian/Documents/people-freebase.csv", true); // write the partial data CsvBeanWriter writer = new CsvBeanWriter(file, CsvPreference.EXCEL_PREFERENCE); for(FreebasePeopleResults person:people){ writer.write(person); } writer.close(); // show output } } I keep getting output errors. Here is the error: There is no content to write for line 2 context: Line: 2 Column: 0 Raw line: null Now, I know it is now totally null, so I am confused.

    Read the article

  • Python csv reader acting weird

    - by PylonsN00b
    So OK if I run this wrong code: csvReader1 = csv.reader(file('new_categories.csv', "rU"), delimiter=',') for row1 in csvReader1: print row1[0] print row1[8] category_sku = str(row[8]) if category_sku == sku: classifications["Craft"] = row[0] classifications["Theme"] = row[1] I get: Knitting 391 Traceback (most recent call last): File "upload_all_inventory_ebay.py", line 403, in <module> inventory_item_list = get_item_list(product) File "upload_all_inventory_ebay.py", line 294, in get_item_list category_sku = str(row[8]) NameError: global name 'row' is not defined Where Knitting and 391 are exactly right, of course I need to refer to row[8] as row1[8]...k so I do this: csvReader1 = csv.reader(file('new_categories.csv', "rU"), delimiter=',') for row1 in csvReader1: print row1[0] print row1[8] category_sku = str(row1[8]) if category_sku == sku: classifications["Craft"] = row1[0] classifications["Theme"] = row1[1] And I get this: ........... Crochet 107452 Knitting 107454 Knitting 107455 Knitting 107456 Knitting 107457 Crochet 108200 Crochet 108201 Crochet 108205 Crochet 108213 Crochet 108214 Crochet 108217 108432 Quilt 108451 108482 108488 Scrapbooking 108711 Knitting 122363 Needlework Beading Crafts & Decorating Crochet Crochet Crochet Traceback (most recent call last): File "upload_all_inventory_ebay.py", line 403, in <module> inventory_item_list = get_item_list(product) File "upload_all_inventory_ebay.py", line 292, in get_item_list print row1[0] IndexError: list index out of range Where the output you see there is every effing thing in column 0 and column 1 !!!!!!!!!! Why? And WHY is row1[0] out of range if it wasn't before. YAY Fridays!

    Read the article

  • How to become a solid python web developer [closed]

    - by Estarius
    Possible Duplicate: How do I learn Python from zero to web development? I have started Python recently with the goal to become a solid developer to make a web application eventually. However, as time goes by I am wondering if I am being optimal about how I will achieve my goal. I would compare it to a game for example, to be better you must spend time playing and trying new things... However, if you just log in and sit in the lobby chatting you are most likely not progressing. So far, this is my plan (feel free to comment or judge it): Review basic programmation concepts Start coding slowly in Python Once comfortable in Python, learn about web development in Python Learn about those things we heard about: SQLAlchemy, MVC, TDD, Git, Agile (Group project) To achieve these things, I started the Learn python the hard way exercises, which I am doing at the rate of 5 per days. I also started to read Think Python at the same time and planning to move on with Dive into python. As far as my research goes, these documentations along with Python documentation is usually what is the most recommended to learn Python. I consider this to get my point 1 and 2 done. While learning Python is really great, my goal remains to do quality web development. I know there are books about Django etc. however I would like to become comfortable with any Python web development. This means without Framework and with Framework... Any framework, then be able to choose the one which best fits our needs. For this I would like to know if some people have suggestions. Should I just get a book on Django and it should apply to everything ? What would be the best method to go from Python to Web Python and not end up creating crappy code which would turn into nightmares for other programmers ? Then finally, those "things we hear about". While I understand what they all do basically, I am fairly sure that like everything, there are good and wrong ways of making use of them. Should I go through at least a whole book on each before starting to use them or keep it at their respective online documentation ? Are there some kind of documentation which links their use to Python ? Also, from looking at Django and Pyramid they seems to use something else than MVC, while the Django model looks similar, the Pyramid one seems to cut a whole part of it... Is learning MVC still worth it ? Sorry for the wall of text, Thanks in advance !

    Read the article

  • Why do people hesitate using Python 3?

    - by Ham
    Python 3 has been released in December 2008. A lot of time has passed since then but still today many developers hesitate using Python 3. Even popular frameworks like Django are not compatible with Python 3 yet but still rely on Python 2. Sure, Python 3 has some incompatibilities to Python 2 and some people need to rely on backwards-compatibility. But hasn't Python 3 been around long enough now for most projects to switch or start with Python 3? Having two competiting versions has so many drawbacks; two branches need to be maintained, confusion for learners and so on, so why is there such a big hesitation throughout the Python community in switching to Python 3?

    Read the article

  • reading csv file without for

    - by Abruzzo Forte e Gentile
    Hi All I need to read a CSV file in python. Since for last row I receive a 'NULL byte' error I would like to avoid using for keyword but the while. Do you know how to do that? reader = csv.reader( file ) for row in reader # I have an error at this line # do whatever with row I want to substitute the for-loop with a while-loop so that I can check if the row is NULL or not. What is the function for reading a single row in the CSV module? Thanks Thanks p.S. below the traceback Traceback (most recent call last): File "FetchNeuro_TodayTrades.py", line 189, in for row in reader: _csv.Error: line contains NULL byte

    Read the article

  • Problems opening large csv file

    - by John Tyler
    I have a csv file that is 100mb in size. I need to parse some data out of it into a new format. I tried PHP, but keep running into memory issues. After around the first 150 "rows" or so, the script poops out. This is even on the localhost, and doing everything I can to tune the PHP settings, including max_memory and script_execution_time. Now before I continue, I'd like to know if Python will poop out on me too. Or if I will have to use C++. Can someone name good csv libraries for for these programmin langueage? The file is quoted csv. I mean scheiza I can't even open this text file in OpenOffice without it dying on me. (then again, Java sux as bad as PHP)

    Read the article

  • Replacing emty csv column values with a zero

    - by homerjay
    Hey, So I'm dealing with a csv file that has missing values. What I want my script to is: #!/usr/bin/python import csv import sys #1. Place each record of a file in a list. #2. Iterate thru each element of the list and get its length. #3. If the length is less than one replace with value x. reader = csv.reader(open(sys.argv[1], "rb")) for row in reader: for x in row[:]: if len(x)< 1: x = 0 print x print row Here is an example of data, I trying it on, ideally it should work on any column lenghth Before: actnum,col2,col4 xxxxx , , xxxxx , 845 , xxxxx , ,545 After actnum,col2,col4 xxxxx , 0 , 0 xxxxx , 845, 0 xxxxx , 0 ,545 Any guidance would be appreciated

    Read the article

  • Update existing columns and rows within csv file using Python

    - by wilbev
    So I've been attempting to use the csv module in Python to add data to existing rows and columns, but only specific columns of each row. So for examples let's say my existing csv file has the following: id, name, city, age 1, Ed,, 34 2, Pat,, 23 So basically the city of each person is missing, so I would like to update each row with that person's city. However, the writerow method only seems replace the existing data within the csv file. Changing the open file to append mode just adds the data to a new row. Is there any way to skip the existing data, and only add the city to each row? Thanks

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >