Search Results

Search found 15 results on 1 pages for 'conradlee'.

Page 1/1 | 1 

• Unlock all private keys on Ubuntu, entering password only once at login

  - by conradlee

  I login to Ubuntu 12.04 using a password. Later on, when I use my browser(Chrome), I'm asked for a password to unlock the keychain so that the browser can access my saved credentials for various websites (it's the same password). Also, whenever I use SSH to connect to other computers using my private key, I am prompted for the same password to unlock my private key. How can I make it so that I am asked for my password exactly once per login (given that my login password is the same as the one I use for all my private keys)? Probably someone will try to label this question as a duplicate of this question, this question, or this question. While these questions are similar, none of them explicitly say that there still needs to be a password entered on login, as I am demanding here. As a result, the accepted solutions just say "set your passwords to blank"--I don't want that, it's dangerous! So I am aware of the similar questions, but none of them has received the correct answer yet, because they are slightly different.

  Read the article

• [C++] Write connected components of a graph using Boost Graph

  - by conradlee

  I have an file that is a long list of weighted edges, in the following form node1_id node2_id weight node1_id node3_id weight and so on. So one weighted edge per line. I want to load this file into boost graph and find the connected components in the graph. Each of these connected components is a subgraph. For each of these component subgraphs, I want to write the edges in the above-described format. I want to do all this using boost graph. This problem is in principle simple, it's just I'm not sure how to implement it neatly because I don't know my way around Boost Graph. I have already spent some hours and have code that will find the connected components, but my version is surely much longer and more complicated that necessary---I'm hoping there's a boost-graph ninja out there that can show me the right, easy way.

  Read the article

• [Python] Detect destination of shortened, or "tiny" url

  - by conradlee

  I have just scraped a bunch of Google Buzz data, and I want to know which Buzz posts reference the same news articles. The problem is that many of the links in these posts have been modified by URL shorteners, so it could be the case that many distinct shortened URLs actually all point to the same news article. Given that I have millions of posts, what is the most efficient way (preferably in python) for me to detect whether a url is a shortened URL (from any of the many URL shortening services, or at least the largest) Find the "destination" of the shortened url, i.e., the long, original version of the shortened URL. Does anyone know if the URL shorteners impose strict request rate limits? If I keep this down to 100/second (all coming form the same IP address), do you think I'll run into trouble?

  Read the article

• Quickest algorithm for finding sets with high intersection

  - by conradlee

  I have a large number of user IDs (integers), potentially millions. These users all belong to various groups (sets of integers), such that there are on the order of 10 million groups. To simplify my example and get to the essence of it, let's assume that all groups contain 20 user IDs (i.e., all integer sets have a cardinality of 100). I want to find all pairs of integer sets that have an intersection of 15 or greater. Should I compare every pair of sets? (If I keep a data structure that maps userIDs to set membership, this would not be necessary.) What is the quickest way to do this? That is, what should my underlying data structure be for representing the integer sets? Sorted sets, unsorted---can hashing somehow help? And what algorithm should I use to compute set intersection)? I prefer answers that relate C/C++ (especially STL), but also any more general, algorithmic insights are welcome. Update Also, note that I will be running this in parallel in a shared memory environment, so ideas that cleanly extend to a parallel solution are preferred.

  Read the article

• [UNIX] Sort lines of massive file by number of words on line (ideally in parallel)

  - by conradlee

  I am working on a community detection algorithm for analyzing social network data from Facebook. The first task, detecting all cliques in the graph, can be done efficiently in parallel, and leaves me with an output like this: 17118 17136 17392 17064 17093 17376 17118 17136 17356 17318 12345 17118 17136 17356 17283 17007 17059 17116 Each of these lines represents a unique clique (a collection of node ids), and I want to sort these lines in descending order by the number of ids per line. In the case of the example above, here's what the output should look like: 17118 17136 17356 17318 12345 17118 17136 17356 17283 17118 17136 17392 17064 17093 17376 17007 17059 17116 (Ties---i.e., lines with the same number of ids---can be sorted arbitrarily.) What is the most efficient way of sorting these lines. Keep the following points in mind: The file I want to sort could be larger than the physical memory of the machine Most of the machines that I'm running this on have several processors, so a parallel solution would be ideal An ideal solution would just be a shell script (probably using sort), but I'm open to simple solutions in python or perl (or any language, as long as it makes the task simple) This task is in some sense very easy---I'm not just looking for any old solution, but rather for a simple and above all efficient solution

  Read the article

• [Python] How can I speed up unpickling large objects if I have plenty of RAM?

  - by conradlee

  It's taking me up to an hour to read a 1-gigabyte NetworkX graph data structure using cPickle (its 1-GB when stored on disk as a binary pickle file). Note that the file quickly loads into memory. In other words, if I run: import cPickle as pickle f = open("bigNetworkXGraph.pickle","rb") binary_data = f.read() # This part doesn't take long graph = pickle.loads(binary_data) # This takes ages How can I speed this last operation up? Note that I have tried pickling the data both in using both binary protocols (1 and 2), and it doesn't seem to make much difference which protocol I use. Also note that although I am using the "loads" (meaning "load string") function above, it is loading binary data, not ascii-data. I have 128gb of RAM on the system I'm using, so I'm hoping that somebody will tell me how to increase some read buffer buried in the pickle implementation.

  Read the article

• [Python] How do I read binary pickle data first, then unpickle it?

  - by conradlee

  I'm unpickling a NetworkX object that's about 1GB in size on disk. Although I saved it in the binary format (using protocol 2), it is taking a very long time to unpickle this file---at least half an hour. The system I'm running on has plenty of system memory (128 GB), so that's not the bottleneck. I've read here that pickling can be sped up by first reading the entire file into memory, and then unpickling it (that particular thread refers to python 3.0, which I'm not using, but the point should still be true in python 2.6). How do I first read the binary file, and then unpickle it? I have tried: import cPickle as pickle f = open("big_networkx_graph.pickle","rb") bin_data = f.read() graph_data = pickle.load(bin_data) But this returns: TypeError: argument must have 'read' and 'readline' attributes Any ideas?

  Read the article

• [C++][OpenMP] Proper use of "atomic directive" to lock STL container

  - by conradlee

  I have a large number of sets of integers, which I have, in turn, put into a vector of pointers. I need to be able to update these sets of integers in parallel without causing a race condition. More specifically. I am using OpenMP's "parallel for" construct. For dealing with shared resources, OpenMP offers a handy "atomic directive," which allows one to avoid a race condition on a specific piece of memory without using locks. It would be convenient if I could use the "atomic directive" to prevent simultaneous updating to my integer sets, however, I'm not sure whether this is possible. Basically, I want to know whether the following code could lead to a race condition vector< set<int>* > membershipDirectory(numSets, new set<int>); #pragma omp for schedule(guided,expandChunksize) for(int i=0; i<100; i++) { set<int>* sp = membershipDirectory[5]; #pragma omp atomic sp->insert(45); } (Apologies for any syntax errors in the code---I hope you get the point) I have seen a similar example of this for incrementing an integer, but I'm not sure whether it works when working with a pointer to a container as in my case.

  Read the article

• [UNIX] Is it safe to pipe the output of several parallel processes to one file using >> ?

  - by conradlee

  I'm scraping data from the web, and I have several processes of my scraper running in parallel. I want the output of each of these processes to end up in the same file. As long as lines of text remain intact, I the order of the lines does not matter. In UNIX, can I just pipe the output of each process to the same file using the operator?

  Read the article

• How to create a named temporary file in memory?

  - by conradlee

  I would like to use Python's tempfile module to create a temporary file that I will use for communication between processes (use of pipes is awkward). The documentation I've linked to above shows two functions that almost do what I want: tempfile.NamedTemporaryFile # For creating named tempfiles tempfile.SpooledTemporaryFile # For creating tempfiles in memory but actually I want a tempfile that is both named AND in memory. Any ideas?

  Read the article

• What is a simple C library for a set of integer sets?

  - by conradlee

  I've got to modify a C program and I need to include a set of unsigned integer sets. That is, I have millions of sets of integers (each of these integer sets contains between 3 and 100 integers), and I need to store these in some structure, lets call it the directory, that can in logarithmic time tell me whether a given integer set already exists in the directory. The only operations that need to be defined on the directory is lookup and insert. This would be easy in languages with built-in support for useful data structures, but I'm a foreigner to C and looking around on Google did (surprisingly) not answer my question satisfactorily. This project looks about right: http://uthash.sourceforge.net/ but I would need to come up with my own hash key generator. This is a standard, simple problem, so I hope there is a standard and simple solution.

  Read the article

• Simple C++ container class that is thread-safe for writing

  - by conradlee

  I am writing a multi-threaded program using OpenMP in C++. At one point my program forks into many threads, each of which need to add "jobs" to some container that keeps track of all added jobs. Each job can just be a pointer to some object. Basically, I just need the add pointers to some container from several threads at the same time. Is there a simple solution that performs well? After some googling, I found that STL containers are not thread-safe. Some stackoverflow threads address this question, but none that forms a consensus on a simple solution.

  Read the article

• [C++] Needed: A simple C++ container (stack, linked list) that is thread-safe for writing

  - by conradlee

  I am writing a multi-threaded program using OpenMP in C++. At one point my program forks into many threads, each of which need to add "jobs" to some container that keeps track of all added jobs. Each job can just be a pointer to some object. Basically, I just need the add pointers to some container from several threads at the same time. Is there a simple solution that performs well? After some googling, I found that STL containers are not thread-safe. Some stackoverflow threads address this question, but none form a consensus on a simple solution.

  Read the article

• [Python] How to create a named temporary file in memory?

  - by conradlee

  I would like to use Python's tempfile module to create a temporary file that I will use for communication between processes (use of pipes is awkward). The documentation I've linked to above shows two functions that almost do what I want: tempfile.NamedTemporaryFile # For creating named tempfiles tempfile.SpooledTemporaryFile # For creating tempfiles in memory but actually I want a tempfile that is both named AND in memory. Any ideas?

  Read the article

• [C++] How should I compare pairs of pointers (for sort predicate)

  - by conradlee

  I have a STL container full of billions of the following objects pair I need some function of the following form /*returns items sorted biggest first */ bool sortPredicate (SomeClass *two, SomeClass *one) { return ???; } Is there some trick I can use to very quickly compare pairs of pointers?

  Read the article

1