Search Results

Search found 38 results on 2 pages for 'dassouki'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 2/2 | < Previous Page | 1 2 

  • Python - CSV: Large file with rows of different lengths

    - by dassouki
    In short, I have a 20,000,000 line csv file that has different row lengths. This is due to archaic data loggers and proprietary formats. We get the end result as a csv file in the following format. MY goal is to insert this file into a postgres database. How Can I do the following: Keep the first 8 columns and my last 2 columns, to have a consistent CSV file Add a Column to the file. 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0, img_id.jpg, -50 1, 2, 3, 4, 5, 0,0,0,0,0,0,0,0,0,0,0 img_id.jpg, -50

    Read the article

  • Beginner - C# iteration through directory to produce a file list

    - by dassouki
    The end goal is to have some form of a data structure that stores a hierarchal structure of a directory to be stored in a txt file. I'm using the following code and so far, and I'm struggling with combining dirs, subdirs, and files. /// <summary> /// code based on http://msdn.microsoft.com/en-us/library/bb513869.aspx /// </summary> /// <param name="strFolder"></param> public static void TraverseTree ( string strFolder ) { // Data structure to hold names of subfolders to be // examined for files. Stack<string> dirs = new Stack<string>( 20 ); if ( !System.IO.Directory.Exists( strFolder ) ) { throw new ArgumentException(); } dirs.Push( strFolder ); while ( dirs.Count > 0 ) { string currentDir = dirs.Pop(); string[] subDirs; try { subDirs = System.IO.Directory.GetDirectories( currentDir ); } catch ( UnauthorizedAccessException e ) { MessageBox.Show( "Error: " + e.Message ); continue; } catch ( System.IO.DirectoryNotFoundException e ) { MessageBox.Show( "Error: " + e.Message ); continue; } string[] files = null; try { files = System.IO.Directory.GetFiles( currentDir ); } catch ( UnauthorizedAccessException e ) { MessageBox.Show( "Error: " + e.Message ); continue; } catch ( System.IO.DirectoryNotFoundException e ) { MessageBox.Show( "Error: " + e.Message ); continue; } // Perform the required action on each file here. // Modify this block to perform your required task. /* foreach ( string file in files ) { try { // Perform whatever action is required in your scenario. System.IO.FileInfo fi = new System.IO.FileInfo( file ); Console.WriteLine( "{0}: {1}, {2}", fi.Name, fi.Length, fi.CreationTime ); } catch ( System.IO.FileNotFoundException e ) { // If file was deleted by a separate application // or thread since the call to TraverseTree() // then just continue. MessageBox.Show( "Error: " + e.Message ); continue; } } */ // Push the subdirectories onto the stack for traversal. // This could also be done before handing the files. foreach ( string str in subDirs ) dirs.Push( str ); foreach ( string str in files ) MessageBox.Show( str ); }

    Read the article

  • Computing complex math equations in python

    - by dassouki
    Are there any libraries or techniques that simplify computing equations ? Take the following two examples: F = B * { [ a * b * sumOf (A / B ''' for all i ''' ) ] / [ sumOf(c * d * j) ] } where: F = cost from i to j B, a, b, c, d, j are all vectors in the format [ [zone_i, zone_j, cost_of_i_to_j], [..]] This should produce a vector F [ [1,2, F_1_2], ..., [i,j, F_i_j] ] T_ij = [ P_i * A_i * F_i_j] / [ SumOf [ Aj * F_i_j ] // j = 1 to j = n ] where: n is the number of zones T = vector [ [1, 2, A_1_2, P_1_2], ..., [i, j, A_i_j, P_i_j] ] F = vector [1, 2, F_1_2], ..., [i, j, F_i_j] so P_i would be the sum of all P_i_j for all j and Aj would be sum of all P_j for all i I'm not sure what I'm looking for, but perhaps a parser for these equations or methods to deal with multiple multiplications and products between vectors? To calculate some of the factors, for example A_j, this is what i use from collections import defaultdict A_j_dict = defaultdict(float) for A_item in TG: A_j_dict[A_item[1]] += A_item[3] Although this works fine, I really feel that it is a brute force / hacking method and unmaintainable in the case we want to add more variables or parameters. Are there any math equation parsers you'd recommend? Side Note: These equations are used to model travel. Currently I use excel to solve a lot of these equations; and I find that process to be daunting. I'd rather move to python where it pulls the data directly from our database (postgres) and outputs the results into the database. All that is figured out. I'm just struggling with evaluating the equations themselves. Thanks :)

    Read the article

  • AJAX Beginner: If then statement

    - by dassouki
    In menu.html, I have my menu items(a href links), let's call them, menu_1, menu_2, .... In map.js displays the contents of a map, it calls an API to display some layers and maps. I have a lot of maps to show but i only want to call the api once. Using AJAX, I was wondering if it's possible to have an if/then or select/case in my js, so that if menu_1 is clicked, then implement map_1 etc. without having the page to reload. The only way I thought I could do this is using Post/get .. so when you click on menu_1 it takes you to page.php?page_id=1 . JS will read that and display the map. I'm new to JS and AJAX so please feel free to butcher me

    Read the article

  • Generate an image / thumbnail of a webpage using X/Gui-less linux

    - by dassouki
    There are a lot of solutions out there on generating a thumbnail or an image preview of a webpage. Some of these are websites like websnapshots, windows libraries such as PHP's imagegrabscreen (only works on windows), and KDE's wkhtml. Many more do exist. However, I'm looking for a GUI-less solution. Something I can create an API around and link it to php or python. I'm comfortable with python, php, C, and shell. This is a personal project, so I'm not interested in commercial applications as I'm aware of their existence. Any ideas?

    Read the article

  • Python Ephem / Datetime calculation

    - by dassouki
    the output should process the first date as "day" and second as "night". I've been playing with this for a few hours now and can't figure out what I'm doing wrong. Any ideas? Edit I assume that the problem is due to my date comparison implementation Output: $ python time_of_day.py * should be day: event date: 2010/4/6 16:00:59 prev rising: 2010/4/6 09:24:24 prev setting: 2010/4/5 23:33:03 next rise: 2010/4/7 09:22:27 next set: 2010/4/6 23:34:27 day * should be night: event date: 2010/4/6 00:01:00 prev rising: 2010/4/5 09:26:22 prev setting: 2010/4/5 23:33:03 next rise: 2010/4/6 09:24:24 next set: 2010/4/6 23:34:27 day time_of_day.py import datetime import ephem # install from http://pypi.python.org/pypi/pyephem/ #event_time is just a date time corresponding to an sql timestamp def type_of_light(latitude, longitude, event_time, utc_time, horizon): o = ephem.Observer() o.lat, o.long, o.date, o.horizon = latitude, longitude, event_time, horizon print "event date ", o.date print "prev rising: ", o.previous_rising(ephem.Sun()) print "prev setting: ", o.previous_setting(ephem.Sun()) print "next rise: ", o.next_rising(ephem.Sun()) print "next set: ", o.next_setting(ephem.Sun()) if o.previous_rising(ephem.Sun()) <= o.date <= o.next_setting(ephem.Sun()): return "day" elif o.previous_setting(ephem.Sun()) <= o.date <= o.next_rising(ephem.Sun()): return "night" else: return "error" print "should be day: ", type_of_light('45.959','-66.6405','2010/4/6 16:01','-4', '-6') print "should be night: ", type_of_light('45.959','-66.6405','2010/4/6 00:01','-4', '-6')

    Read the article

  • Simple numpy question

    - by dassouki
    I can't get this snippet to work: #base code A = array([ [ 1, 2, 10 ], [ 1, 3, 20 ], [ 1, 4, 30 ], [ 2, 1, 15 ], [ 2, 3, 25 ], [ 2, 4, 35 ], [ 3, 1, 17 ], [ 3, 2, 27 ], [ 3, 4, 37 ], [ 4, 1, 13 ], [ 4, 2, 23 ], [ 4, 3, 33 ] ]) # Number of zones zones = unique1d(A[:,0]) for origin in zones: for destination in zones: if origin != destination: A_ik = A[(A[:,0] == origin & A[:,1] == destination), 2]

    Read the article

  • Creating an Excel Template for different data size

    - by dassouki
    I created an excel template for a file i've done for a routine work calculation. The file takes data from the data logger and does some analysis on it and outputs one number regardless of the input size. The problem I'm having is i have to modify the sheet to suit the number of rows, as everyday the data logger outputs a different number of rows. there are about 15 sheets in the workbook and it's annoying to have to change everyone of them everyday. What i'd like to do input the data logger csv, and boom the result gets outputted. Is there a way through vba or not to ahieve

    Read the article

  • Statistical analysis on large data set to be published on the web

    - by dassouki
    I have a non-computer related data logger, that collects data from the field. This data is stored as text files, and I manually lump the files together and organize them. The current format is through a csv file per year per logger. Each file is around 4,000,000 lines x 7 loggers x 5 years = a lot of data. some of the data is organized as bins item_type, item_class, item_dimension_class, and other data is more unique, such as item_weight, item_color, date_collected, and so on ... Currently, I do statistical analysis on the data using a python/numpy/matplotlib program I wrote. It works fine, but the problem is, I'm the only one who can use it, since it and the data live on my computer. I'd like to publish the data on the web using a postgres db; however, I need to find or implement a statistical tool that'll take a large postgres table, and return statistical results within an adequate time frame. I'm not familiar with python for the web; however, I'm proficient with PHP on the web side, and python on the offline side. users should be allowed to create their own histograms, data analysis. For example, a user can search for all items that are blue shipped between week x and week y, while another user can search for sort the weight distribution of all items by hour for all year long. I was thinking of creating and indexing my own statistical tools, or automate the process somehow to emulate most queries. This seemed inefficient. I'm looking forward to hearing your ideas Thanks

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2