Search Results

Search found 69 results on 3 pages for 'semaphores'.

Page 2/3 | < Previous Page | 1 2 3  | Next Page >

  • How to set shmall, shmmax, shmni, etc ... in general and for postgresql

    - by jpic
    I've used the documentation from PostgreSQL to set it for example this config: >>> cat /proc/meminfo MemTotal: 16345480 kB MemFree: 1770128 kB Buffers: 382184 kB Cached: 10432632 kB SwapCached: 0 kB Active: 9228324 kB Inactive: 4621264 kB Active(anon): 7019996 kB Inactive(anon): 548528 kB Active(file): 2208328 kB Inactive(file): 4072736 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 3432 kB Writeback: 0 kB AnonPages: 3034588 kB Mapped: 4243720 kB Shmem: 4533752 kB Slab: 481728 kB SReclaimable: 440712 kB SUnreclaim: 41016 kB KernelStack: 1776 kB PageTables: 39208 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 8172740 kB Committed_AS: 14935216 kB VmallocTotal: 34359738367 kB VmallocUsed: 399340 kB VmallocChunk: 34359334908 kB HardwareCorrupted: 0 kB AnonHugePages: 456704 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 12288 kB DirectMap2M: 16680960 kB >>> ipcs -l ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 4316816 max total shared memory (kbytes) = 4316816 min seg size (bytes) = 1 ------ Semaphore Limits -------- max number of arrays = 128 max semaphores per array = 250 max semaphores system wide = 32000 max ops per semop call = 32 semaphore max value = 32767 ------ Messages Limits -------- max queues system wide = 31918 max size of message (bytes) = 8192 default max size of queue (bytes) = 16384 sysctl.conf extract: kernel.shmall = 1079204 kernel.shmmax = 4420419584 postgresql.conf non defaults: max_connections = 60 # (change requires restart) shared_buffers = 4GB # min 128kB work_mem = 4MB # min 64kB wal_sync_method = open_sync # the default is the first option checkpoint_segments = 16 # in logfile segments, min 1, 16MB each checkpoint_completion_target = 0.9 # checkpoint target duration, 0.0 - 1.0 effective_cache_size = 6GB Is this appropriate ? If not (or not necessarily), in which case would it be appropriate ? We did note nice performance improvements with this config, how would you improve it ? How should kernel memory management parameters be set ? Can anybody explain how to really set them from the ground up ?

    Read the article

  • Apache will not stop/start gracefully

    - by ddjammin
    CentOs 6 64bit running apache 2.2.15-29.el6.centos. When I try to stop/start or restart httpd I get an error that says it has failed. A tail of the error log is below. I also noticed that a httpd.pid file is not created even though it is configured in the main conf file. If I set selinux to permissive, it works just fine. I do not want to run it with selinux disabled. If I delete the SSL_Mutex file it will start. HTTPD was running fine until I tried to add the ssl configuration. I copied over the ssl.conf file from a working server into the conf.d folder. I also copied a sslcert folder into the conf folder. It contains the certs, key, csr and password file. I think the problem has to do with the selinux context for the sslcert folder that was copied but I am not certain and not sure how to fix it. Below is the security context for the sslcert folder after executing restorecon -R sslcert ls -Z -rw-r--r--. root root system_u:object_r:httpd_config_t:s0 httpd.conf -rw-r--r--. root root system_u:object_r:httpd_config_t:s0 magic **drwxr-xr-x. root root system_u:object_r:httpd_config_t:s0 sslcert** tail -f /var/log/httpd/error_log [Thu Oct 17 13:33:19 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Thu Oct 17 13:33:20 2013] [notice] Digest: generating secret for digest authentication ... [Thu Oct 17 13:33:20 2013] [notice] Digest: done [Thu Oct 17 13:33:20 2013] [warn] pid file /etc/httpd/logs/ssl.pid overwritten -- Unclean shutdown of previous Apache run? [Thu Oct 17 13:33:20 2013] [notice] Apache/2.2.15 (Unix) DAV/2 mod_ssl/2.2.15 OpenSSL/1.0.0-fips configured -- resuming normal operations [Thu Oct 17 21:04:48 2013] [notice] caught SIGTERM, shutting down [Thu Oct 17 21:06:42 2013] [notice] **SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0** [Thu Oct 17 21:06:42 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Thu Oct 17 21:06:42 2013] [error] (17)File exists: Cannot create SSLMutex with file `/etc/httpd/logs/ssl_mutex' I also saw mention of possible issues with semaphores. Below is the output of the current semaphores and apache is currently not running. ipcs -s ------ Semaphore Arrays -------- key semid owner perms nsems 0x00000000 0 root 600 1 0x00000000 65537 root 600 1 Finally selinux reports the following error. `sealert -a /var/log/audit/audit.log` 0% donetype=AVC msg=audit(1382034755.118:420400): avc: denied { write } for pid=3393 comm="httpd" name="ssl_mutex" dev=dm-0 ino=9513484 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:httpd_log_t:s0 tclass=file **** Invalid AVC allowed in current policy *** 100% doneERROR: failed to read complete file, 1044649 bytes read out of total 1043317 bytes (/var/log/audit/audit.log) found 1 alerts in /var/log/audit/audit.log -------------------------------------------------------------------------------- SELinux is preventing /usr/sbin/httpd from remove_name access on the directory ssl_mutex.

    Read the article

  • What's the current best practice for C multithreading?

    - by Dexter
    I've been a Java (and PHP) developer for most of my life, but will need to use C for my next project. I remember some basic stuff we did back in school with fork, pthread, semaphores, etc. but I'm not sure if that's still the "way to go" here, or if there have been any recent developments/improvements etc. Google's results seem to be fairly dated. Or does that just mean that this is still the current "state of the art"? Or will I have to use a (system-independent) libary like glib if I want higher-level abstractions, like synchronized queues? What other libraries are there? (The programm will only have to run on Linux though)

    Read the article

  • Macbook 8.1 overheating

    - by timse201
    I have a macbook 8.1 with ubuntu 12.04 installed. But my cpu is getting very hot. On Mac my CPU is 50-60°C warm. But on ubuntu my mac is getting very hot and is by about 60°C but with min 3000rpm instead of 2000 on mac and the fan is getting very loud with 4500rpm on ubuntu when im browsing (without flash) or doing something else. i set it to 3000rpm because it is not getting so noisy instead of 2000rpm minimum. But thats not that what im expected. What ive done: i installed lm-sensors to see the temperatures and started the sensors-detect i installed macfancld, jupiter, the newest drivers from x-updates and installed the i965-va-driver oh and i installed mesa - with the default version my sandbridge was displayed as unknown i added GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi=force drm.vblankoffdelay=1 pcie_aspm=force drm.vblankoffdelay=1 i915.semaphores=1 i915.i915_enable_rc6=1 i915.i915_enable_fbc=1" ive added rfkill block bluetooth to /etc/rc.local to switch of bluetooth by default on boot my mac is not as noisy as before but it is noisy and sometimes very hot i hope you can help me

    Read the article

  • Plagued by multithreaded bugs

    - by koncurrency
    On my new team that I manage, the majority of our code is platform, TCP socket, and http networking code. All C++. Most of it originated from other developers that have left the team. The current developers on the team are very smart, but mostly junior in terms of experience. Our biggest problem: multi-threaded concurrency bugs. Most of our class libraries are written to be asynchronous by use of some thread pool classes. Methods on the class libraries often enqueue long running taks onto the thread pool from one thread and then the callback methods of that class get invoked on a different thread. As a result, we have a lot of edge case bugs involving incorrect threading assumptions. This results in subtle bugs that go beyond just having critical sections and locks to guard against concurrency issues. What makes these problems even harder is that the attempts to fix are often incorrect. Some mistakes I've observed the team attempting (or within the legacy code itself) includes something like the following: Common mistake #1 - Fixing concurrency issue by just put a lock around the shared data, but forgetting about what happens when methods don't get called in an expected order. Here's a very simple example: void Foo::OnHttpRequestComplete(statuscode status) { m_pBar->DoSomethingImportant(status); } void Foo::Shutdown() { m_pBar->Cleanup(); delete m_pBar; m_pBar=nullptr; } So now we have a bug in which Shutdown could get called while OnHttpNetworkRequestComplete is occuring on. A tester finds the bug, captures the crash dump, and assigns the bug to a developer. He in turn fixes the bug like this. void Foo::OnHttpRequestComplete(statuscode status) { AutoLock lock(m_cs); m_pBar->DoSomethingImportant(status); } void Foo::Shutdown() { AutoLock lock(m_cs); m_pBar->Cleanup(); delete m_pBar; m_pBar=nullptr; } The above fix looks good until you realize there's an even more subtle edge case. What happens if Shutdown gets called before OnHttpRequestComplete gets called back? The real world examples my team has are even more complex, and the edge cases are even harder to spot during the code review process. Common Mistake #2 - fixing deadlock issues by blindly exiting the lock, wait for the other thread to finish, then re-enter the lock - but without handling the case that the object just got updated by the other thread! Common Mistake #3 - Even though the objects are reference counted, the shutdown sequence "releases" it's pointer. But forgets to wait for the thread that is still running to release it's instance. As such, components are shutdown cleanly, then spurious or late callbacks are invoked on an object in an state not expecting any more calls. There are other edge cases, but the bottom line is this: Multithreaded programming is just plain hard, even for smart people. As I catch these mistakes, I spend time discussing the errors with each developer on developing a more appropriate fix. But I suspect they are often confused on how to solve each issue because of the enormous amount of legacy code that the "right" fix will involve touching. We're going to be shipping soon, and I'm sure the patches we're applying will hold for the upcoming release. Afterwards, we're going to have some time to improve the code base and refactor where needed. We won't have time to just re-write everything. And the majority of the code isn't all that bad. But I'm looking to refactor code such that threading issues can be avoided altogether. One approach I am considering is this. For each significant platform feature, have a dedicated single thread where all events and network callbacks get marshalled onto. Similar to COM apartment threading in Windows with use of a message loop. Long blocking operations could still get dispatched to a work pool thread, but the completion callback is invoked on on the component's thread. Components could possibly even share the same thread. Then all the class libraries running inside the thread can be written under the assumption of a single threaded world. Before I go down that path, I am also very interested if there are other standard techniques or design patterns for dealing with multithreaded issues. And I have to emphasize - something beyond a book that describes the basics of mutexes and semaphores. What do you think? I am also interested in any other approaches to take towards a refactoring process. Including any of the following: Literature or papers on design patterns around threads. Something beyond an introduction to mutexes and semaphores. We don't need massive parallelism either, just ways to design an object model so as to handle asynchronous events from other threads correctly. Ways to diagram the threading of various components, so that it will be easy to study and evolve solutions for. (That is, a UML equivalent for discussing threads across objects and classes) Educating your development team on the issues with multithreaded code. What would you do?

    Read the article

  • FIFO semaphore test

    - by L4N0
    Hello everyone, I have implemented FIFO semaphores but now I need a way to test/prove that they are working properly. A simple test would be to create some threads that try to wait on a semaphore and then print a message with a number and if the numbers are in order it should be FIFO, but this is not good enough to prove it because that order could have occurred by chance. Thus, I need a better way of testing it. If necessary locks or condition variables can be used too. Thanks

    Read the article

  • Cleaning hanging IPCS in UNIX

    - by Sachin Chourasiya
    I knew that IPCRM is used to clean hanging IPCS and semaphores for a particumar user by passing the segment id or the semaphore id in either -m or -s option. WE NEED TO PASS INDIVIDUAL SEGMENT ID/ SEMAPHORE ID IN -m OPTION. Is there any way to clean ipcs that belongs to a particular user in just one move. I think shell script could be the way but not sure. Please help

    Read the article

  • Linux synchronization with FIFO waiting queue

    - by EpsilonVector
    Are there locks in Linux where the waiting queue is FIFO? This seems like such an obvious thing, and yet I just discovered that pthread mutexes aren't FIFO, and semaphores apparently aren't FIFO either (I'm working on kernel 2.4 (homework))... Does Linux have a lock with FIFO waiting queue, or is there an easy way to make one with existing mechanisms?

    Read the article

  • Should mutex.WaitOne() inside or before the try/finally block.

    - by chillitom
    Hi Guys, I was wondering which of the following was the suggested pattern when using Mutex (or Semaphores or ReadWriteLockSlims etc.). Should the initial lock happen inside or outside of the try statement? Is it unimportant? _mutex.WaitOne() try { // critical code } finally { _mutex.ReleaseMutex(); } or try { _mutex.WaitOne() // critical code } finally { _mutex.ReleaseMutex(); }

    Read the article

  • Real World Examples of read-write in concurrent software

    - by Richard Fabian
    I'm looking for real world examples of needing read and write access to the same value in concurrent systems. In my opinion, many semaphores or locks are present because there's no known alternative (to the implementer,) but do you know of any patterns where mutexes seem to be a requirement? In a way I'm asking for candidates for the standard set of HARD problems for concurrent software in the real world.

    Read the article

  • Installing nGinX Reverse Proxy on CentOS 5

    - by heavymark
    I'm trying to install nGinX as a reverse proxy on CentOS 5 with apache. The instructions to do this are here: http://wiki.mediatemple.net/w/(dv):Configure_nginx_as_reverse_proxy_web_server Note- in the instructions, for the url to get nginx I'm using the following: http://nginx.org/download/nginx-1.0.10.tar.gz Now here is my problem. After installing the required packages and running .configure I get the following: checking for OS + Linux 2.6.18-028stab094.3 x86_64 checking for C compiler ... found + using GNU C compiler + gcc version: 4.1.2 20080704 (Red Hat 4.1.2-51) checking for gcc -pipe switch ... found checking for gcc builtin atomic operations ... found checking for C99 variadic macros ... found checking for gcc variadic macros ... found checking for unistd.h ... found checking for inttypes.h ... found checking for limits.h ... found checking for sys/filio.h ... not found checking for sys/param.h ... found checking for sys/mount.h ... found checking for sys/statvfs.h ... found checking for crypt.h ... found checking for Linux specific features checking for epoll ... found checking for sendfile() ... found checking for sendfile64() ... found checking for sys/prctl.h ... found checking for prctl(PR_SET_DUMPABLE) ... found checking for sched_setaffinity() ... found checking for crypt_r() ... found checking for sys/vfs.h ... found checking for nobody group ... found checking for poll() ... found checking for /dev/poll ... not found checking for kqueue ... not found checking for crypt() ... not found checking for crypt() in libcrypt ... found checking for F_READAHEAD ... not found checking for posix_fadvise() ... found checking for O_DIRECT ... found checking for F_NOCACHE ... not found checking for directio() ... not found checking for statfs() ... found checking for statvfs() ... found checking for dlopen() ... not found checking for dlopen() in libdl ... found checking for sched_yield() ... found checking for SO_SETFIB ... not found checking for SO_ACCEPTFILTER ... not found checking for TCP_DEFER_ACCEPT ... found checking for accept4() ... not found checking for int size ... 4 bytes checking for long size ... 8 bytes checking for long long size ... 8 bytes checking for void * size ... 8 bytes checking for uint64_t ... found checking for sig_atomic_t ... found checking for sig_atomic_t size ... 4 bytes checking for socklen_t ... found checking for in_addr_t ... found checking for in_port_t ... found checking for rlim_t ... found checking for uintptr_t ... uintptr_t found checking for system endianess ... little endianess checking for size_t size ... 8 bytes checking for off_t size ... 8 bytes checking for time_t size ... 8 bytes checking for setproctitle() ... not found checking for pread() ... found checking for pwrite() ... found checking for sys_nerr ... found checking for localtime_r() ... found checking for posix_memalign() ... found checking for memalign() ... found checking for mmap(MAP_ANON|MAP_SHARED) ... found checking for mmap("/dev/zero", MAP_SHARED) ... found checking for System V shared memory ... found checking for POSIX semaphores ... not found checking for POSIX semaphores in libpthread ... found checking for struct msghdr.msg_control ... found checking for ioctl(FIONBIO) ... found checking for struct tm.tm_gmtoff ... found checking for struct dirent.d_namlen ... not found checking for struct dirent.d_type ... found checking for PCRE library ... found checking for system md library ... not found checking for system md5 library ... not found checking for OpenSSL md5 crypto library ... found checking for sha1 in system md library ... not found checking for OpenSSL sha1 crypto library ... found checking for zlib library ... found creating objs/Makefile Configuration summary + using system PCRE library + OpenSSL library is not used + md5: using system crypto library + sha1: using system crypto library + using system zlib library nginx path prefix: "/usr/local/nginx" nginx binary file: "/usr/local/nginx/sbin/nginx" nginx configuration prefix: "/usr/local/nginx/conf" nginx configuration file: "/usr/local/nginx/conf/nginx.conf" nginx pid file: "/usr/local/nginx/logs/nginx.pid" nginx error log file: "/usr/local/nginx/logs/error.log" nginx http access log file: "/usr/local/nginx/logs/access.log" nginx http client request body temporary files: "client_body_temp" nginx http proxy temporary files: "proxy_temp" nginx http fastcgi temporary files: "fastcgi_temp" nginx http uwsgi temporary files: "uwsgi_temp" nginx http scgi temporary files: "scgi_temp" It says if you get errors to stop and make sure packages are installed. I didn't get errors but as you can see I got several "not founds". Are those considered errors? If so how do I resolve that. And as noted in the link, I cannot install through yum, because it wont work with plesk then. Thanks!

    Read the article

  • Thread Synchronisation 101

    - by taspeotis
    Previously I've written some very simple multithreaded code, and I've always been aware that at any time there could be a context switch right in the middle of what I'm doing, so I've always guarded access the shared variables through a CCriticalSection class that enters the critical section on construction and leaves it on destruction. I know this is fairly aggressive and I enter and leave critical sections quite frequently and sometimes egregiously (e.g. at the start of a function when I could put the CCriticalSection inside a tighter code block) but my code doesn't crash and it runs fast enough. At work my multithreaded code needs to be a tighter, only locking/synchronising at the lowest level needed. At work I was trying to debug some multithreaded code, and I came across this: EnterCriticalSection(&m_Crit4); m_bSomeVariable = true; LeaveCriticalSection(&m_Crit4); Now, m_bSomeVariable is a Win32 BOOL (not volatile), which as far as I know is defined to be an int, and on x86 reading and writing these values is a single instruction, and since context switches occur on an instruction boundary then there's no need for synchronising this operation with a critical section. I did some more research online to see whether this operation did not need synchronisation, and I came up with two scenarios it did: The CPU implements out of order execution or the second thread is running on a different core and the updated value is not written into RAM for the other core to see; and The int is not 4-byte aligned. I believe number 1 can be solved using the "volatile" keyword. In VS2005 and later the C++ compiler surrounds access to this variable using memory barriers, ensuring that the variable is always completely written/read to the main system memory before using it. Number 2 I cannot verify, I don't know why the byte alignment would make a difference. I don't know the x86 instruction set, but does mov need to be given a 4-byte aligned address? If not do you need to use a combination of instructions? That would introduce the problem. So... QUESTION 1: Does using the "volatile" keyword (implicity using memory barriers and hinting to the compiler not to optimise this code) absolve a programmer from the need to synchronise a 4-byte/8-byte on x86/x64 variable between read/write operations? QUESTION 2: Is there the explicit requirement that the variable be 4-byte/8-byte aligned? I did some more digging into our code and the variables defined in the class: class CExample { private: CRITICAL_SECTION m_Crit1; // Protects variable a CRITICAL_SECTION m_Crit2; // Protects variable b CRITICAL_SECTION m_Crit3; // Protects variable c CRITICAL_SECTION m_Crit4; // Protects variable d // ... }; Now, to me this seems excessive. I thought critical sections synchronised threads between a process, so if you've got one you can enter it and no other thread in that process can execute. There is no need for a critical section for each variable you want to protect, if you're in a critical section then nothing else can interrupt you. I think the only thing that can change the variables from outside a critical section is if the process shares a memory page with another process (can you do that?) and the other process starts to change the values. Mutexes would also help here, named mutexes are shared across processes, or only processes of the same name? QUESTION 3: Is my analysis of critical sections correct, and should this code be rewritten to use mutexes? I have had a look at other synchronisation objects (semaphores and spinlocks), are they better suited here? QUESTION 4: Where are critical sections/mutexes/semaphores/spinlocks best suited? That is, which synchronisation problem should they be applied to. Is there a vast performance penalty for choosing one over the other? And while we're on it, I read that spinlocks should not be used in a single-core multithreaded environment, only a multi-core multithreaded environment. So, QUESTION 5: Is this wrong, or if not, why is it right? Thanks in advance for any responses :)

    Read the article

  • Oracle RDBMS Server 11gR2 Pre-Install RPM for Oracle Linux 6 has been released

    - by Lenz Grimmer
    Now that the certification of the Oracle Database 11g R2 with Oracle Linux 6 and the Unbreakable Enterprise Kernel has been announced, we are glad to announce the availability of oracle-rdbms-server-11gR2-preinstall, the Oracle RDBMS Server 11gR2 Pre-install RPM package (formerly known as oracle-validated). Designed specifically for Oracle Linux 6, this RPM aids in the installation of the Oracle Database. In order to install the Oracle Database 11g R2 on Oracle Linux 6, your system needs to meet a few prerequisites, as outlined in the Linux Installation Guides. Using the Oracle RDBMS Server 11gR2 Pre-install RPM, you can complete most of the pre-installation configuration tasks. which is now available from the Unbreakable Linux Network, or via the Oracle public yum repository. The pre-install package is available for x86_64 only. Specifically, the package: Causes the download and installation of various software packages and specific versions needed for database installation, with package dependencies resolved via yum Creates the user oracle and the groups oinstall and dba, which are the defaults used during database installation Modifies kernel parameters in /etc/sysctl.conf to change settings for shared memory, semaphores, the maximum number of file descriptors, and so on Sets hard and soft shell resource limits in /etc/security/limits.conf, such as the number of open files, the number of processes, and stack size to the minimum required based on the Oracle Database 11g Release 2 Server installation requirements Sets numa=off in the kernel boot parameters for x86_64 machines Please see the release announcement for further details and instructions. Also take a look at Ginny Henningsen's "How I Simplified Oracle Database Installation on Oracle Linux" article on the Oracle Technology Network for a general description on how to perform the installation of the Oracle Database on Oracle Linux. While the article refers to Oracle Linux 5 and the former "oracle-validated" package, the steps for Oracle Linux 6 are still very similar (we're looking into updating that article for Oracle Linux 6).

    Read the article

  • Need theoretical help, how to comprehend an if-else dependency net

    - by macbie
    I am going to face a following issue: I'm writing a program that manages some properties, some of them are general and some are specific. Each property is a pair of key and value, and for example: if it is given a general property and other specific property with exactly the same key and value has been existed before then the general property will swap the specific one in the register. If there are two the same general properties - both will remain in the register. And so on; it is like a net of dependencies. In my case I can handle with it intuitively and foresee all cases, but only because the system is not too vast. What if it would? I have met such problems a few times in many different programs and languages (i.e working with C semaphores) and my question is: How to approach this kind of problem? Is this connected with finite state machine, graph theory or something similar? How to be sure that I have considered the whole system and each possible case? Could you recommend some resources (books, sites) to learn from?

    Read the article

  • Using Clojure instead of Python for scalability (multi core) reasons, good idea?

    - by Vandell
    After reading http://clojure.org/rationale and other performance comparisons between Clojure and many languages, I started to think that apart from ease of use, I shouldn't be coding in Python anymore, but in Clojure instead. Actually, I began to fill irresponsisble for not learning clojure seeing it's benefits. Does it make sense? Can't I make really efficient use of all cores using a more imperative language like Python, than a lisp dialect or other functional language? It seems that all the benefits of it come from using immutable data, can't I do just that in Python and have all the benefits? I once started to learn some Common Lisp, read and done almost all exercices from a book I borrowod from my university library (I found it to be pretty good, despite it's low popularity on Amazon). But, after a while, I got myself struggling to much to do some simple things. I think there's somethings that are more imperative in their nature, that makes it difficult to model those thins in a functional way, I guess. The thing is, is Python as powerful as Clojure for building applications that takes advantages of this new multi core future? Note that I don't think that using semaphores, lock mechanisms or other similar concurrency mechanism are good alternatives to Clojure 'automatic' parallelization.

    Read the article

  • Threading across multiple files

    - by Zach M.
    My program is reading in files and using thread to compute the highest prime number, when I put a print statement into the getNum() function my numbers are printing out. However, it seems to just lag no matter how many threads I input. Each file has 1 million integers in it. Does anyone see something apparently wrong with my code? Basically the code is giving each thread 1000 integers to check before assigning a new thread. I am still a C noobie and am just learning the ropes of threading. My code is a mess right now because I have been switching things around constantly. #include <stdio.h> #include <stdlib.h> #include <time.h> #include <string.h> #include <pthread.h> #include <math.h> #include <semaphore.h> //Global variable declaration char *file1 = "primes1.txt"; char *file2 = "primes2.txt"; char *file3 = "primes3.txt"; char *file4 = "primes4.txt"; char *file5 = "primes5.txt"; char *file6 = "primes6.txt"; char *file7 = "primes7.txt"; char *file8 = "primes8.txt"; char *file9 = "primes9.txt"; char *file10 = "primes10.txt"; char **fn; //file name variable int numberOfThreads; int *highestPrime = NULL; int fileArrayNum = 0; int loop = 0; int currentFile = 0; sem_t semAccess; sem_t semAssign; int prime(int n)//check for prime number, return 1 for prime 0 for nonprime { int i; for(i = 2; i <= sqrt(n); i++) if(n % i == 0) return(0); return(1); } int getNum(FILE* file) { int number; char* tempS = malloc(20 *sizeof(char)); fgets(tempS, 20, file); tempS[strlen(tempS)-1] = '\0'; number = atoi(tempS); free(tempS);//free memory for later call return(number); } void* findPrimality(void *threadnum) //main thread function to find primes { int tNum = (int)threadnum; int checkNum; char *inUseFile = NULL; int x=1; FILE* file; while(currentFile < 10){ if(inUseFile == NULL){//inUseFIle being used to check if a file is still being read sem_wait(&semAccess);//critical section inUseFile = fn[currentFile]; sem_post(&semAssign); file = fopen(inUseFile, "r"); while(!feof(file)){ if(x % 1000 == 0 && tNum !=1){ //go for 1000 integers and then wait sem_wait(&semAssign); } checkNum = getNum(file); /* * * * * I think the issue is here * * * */ if(checkNum > highestPrime[tNum]){ if(prime(checkNum)){ highestPrime[tNum] = checkNum; } } x++; } fclose(file); inUseFile = NULL; } currentFile++; } } int main(int argc, char* argv[]) { if(argc != 2){ //checks for number of arguements being passed printf("To many ARGS\n"); return(-1); } else{//Sets thread cound to user input checking for correct number of threads numberOfThreads = atoi(argv[1]); if(numberOfThreads < 1 || numberOfThreads > 10){ printf("To many threads entered\n"); return(-1); } time_t preTime, postTime; //creating time variables int i; fn = malloc(10 * sizeof(char*)); //create file array and initialize fn[0] = file1; fn[1] = file2; fn[2] = file3; fn[3] = file4; fn[4] = file5; fn[5] = file6; fn[6] = file7; fn[7] = file8; fn[8] = file9; fn[9] = file10; sem_init(&semAccess, 0, 1); //initialize semaphores sem_init(&semAssign, 0, numberOfThreads); highestPrime = malloc(numberOfThreads * sizeof(int)); //create an array to store each threads highest number for(loop = 0; loop < numberOfThreads; loop++){//set initial values to 0 highestPrime[loop] = 0; } pthread_t calculationThread[numberOfThreads]; //thread to do the work preTime = time(NULL); //start the clock for(i = 0; i < numberOfThreads; i++){ pthread_create(&calculationThread[i], NULL, findPrimality, (void *)i); } for(i = 0; i < numberOfThreads; i++){ pthread_join(calculationThread[i], NULL); } for(i = 0; i < numberOfThreads; i++){ printf("this is a prime number: %d \n", highestPrime[i]); } postTime= time(NULL); printf("Wall time: %ld seconds\n", (long)(postTime - preTime)); } } Yes I am trying to find the highest number over all. So I have made some head way the last few hours, rescucturing the program as spudd said, currently I am getting a segmentation fault due to my use of structures, I am trying to save the largest individual primes in the struct while giving them the right indices. This is the revised code. So in short what the first thread is doing is creating all the threads and giving them access points to a very large integer array which they will go through and find prime numbers, I want to implement semaphores around the while loop so that while they are executing every 2000 lines or the end they update a global prime number. #include <stdio.h> #include <stdlib.h> #include <time.h> #include <string.h> #include <pthread.h> #include <math.h> #include <semaphore.h> //Global variable declaration char *file1 = "primes1.txt"; char *file2 = "primes2.txt"; char *file3 = "primes3.txt"; char *file4 = "primes4.txt"; char *file5 = "primes5.txt"; char *file6 = "primes6.txt"; char *file7 = "primes7.txt"; char *file8 = "primes8.txt"; char *file9 = "primes9.txt"; char *file10 = "primes10.txt"; int numberOfThreads; int entries[10000000]; int entryIndex = 0; int fileCount = 0; char** fileName; int largestPrimeNumber = 0; //Register functions int prime(int n); int getNum(FILE* file); void* findPrimality(void *threadNum); void* assign(void *num); typedef struct package{ int largestPrime; int startingIndex; int numberCount; }pack; //Beging main code block int main(int argc, char* argv[]) { if(argc != 2){ //checks for number of arguements being passed printf("To many threads!!\n"); return(-1); } else{ //Sets thread cound to user input checking for correct number of threads numberOfThreads = atoi(argv[1]); if(numberOfThreads < 1 || numberOfThreads > 10){ printf("To many threads entered\n"); return(-1); } int threadPointer[numberOfThreads]; //Pointer array to point to entries time_t preTime, postTime; //creating time variables int i; fileName = malloc(10 * sizeof(char*)); //create file array and initialize fileName[0] = file1; fileName[1] = file2; fileName[2] = file3; fileName[3] = file4; fileName[4] = file5; fileName[5] = file6; fileName[6] = file7; fileName[7] = file8; fileName[8] = file9; fileName[9] = file10; FILE* filereader; int currentNum; for(i = 0; i < 10; i++){ filereader = fopen(fileName[i], "r"); while(!feof(filereader)){ char* tempString = malloc(20 *sizeof(char)); fgets(tempString, 20, filereader); tempString[strlen(tempString)-1] = '\0'; entries[entryIndex] = atoi(tempString); entryIndex++; free(tempString); } } //sem_init(&semAccess, 0, 1); //initialize semaphores //sem_init(&semAssign, 0, numberOfThreads); time_t tPre, tPost; pthread_t coordinate; tPre = time(NULL); pthread_create(&coordinate, NULL, assign, (void**)numberOfThreads); pthread_join(coordinate, NULL); tPost = time(NULL); } } void* findPrime(void* pack_array) { pack* currentPack= pack_array; int lp = currentPack->largestPrime; int si = currentPack->startingIndex; int nc = currentPack->numberCount; int i; int j = 0; for(i = si; i < nc; i++){ while(j < 2000 || i == (nc-1)){ if(prime(entries[i])){ if(entries[i] > lp) lp = entries[i]; } j++; } } return (void*)currentPack; } void* assign(void* num) { int y = (int)num; int i; int count = 10000000/y; int finalCount = count + (10000000%y); int sIndex = 0; pack pack_array[(int)num]; pthread_t workers[numberOfThreads]; //thread to do the workers for(i = 0; i < y; i++){ if(i == (y-1)){ pack_array[i].largestPrime = 0; pack_array[i].startingIndex = sIndex; pack_array[i].numberCount = finalCount; } pack_array[i].largestPrime = 0; pack_array[i].startingIndex = sIndex; pack_array[i].numberCount = count; pthread_create(&workers[i], NULL, findPrime, (void *)&pack_array[i]); sIndex += count; } for(i = 0; i< y; i++) pthread_join(workers[i], NULL); } //Functions int prime(int n)//check for prime number, return 1 for prime 0 for nonprime { int i; for(i = 2; i <= sqrt(n); i++) if(n % i == 0) return(0); return(1); }

    Read the article

  • Thread scheduling Round Robin / scheduling dispatch

    - by MRP
    #include <pthread.h> #include <stdio.h> #include <stdlib.h> #include <semaphore.h> #define NUM_THREADS 4 #define COUNT_LIMIT 13 int done = 0; int count = 0; int quantum = 2; int thread_ids[4] = {0,1,2,3}; int thread_runtime[4] = {0,5,4,7}; pthread_mutex_t count_mutex; pthread_cond_t count_threshold_cv; void * inc_count(void * arg); static sem_t count_sem; int quit = 0; ///////// Inc_Count//////////////// void *inc_count(void *t) { long my_id = (long)t; int i; sem_wait(&count_sem); /////////////CRIT SECTION////////////////////////////////// printf("run_thread = %d\n",my_id); printf("%d \n",thread_runtime[my_id]); for( i=0; i < thread_runtime[my_id];i++) { printf("runtime= %d\n",thread_runtime[my_id]); pthread_mutex_lock(&count_mutex); count++; if (count == COUNT_LIMIT) { pthread_cond_signal(&count_threshold_cv); printf("inc_count(): thread %ld, count = %d Threshold reached.\n", my_id, count); } printf("inc_count(): thread %ld, count = %d, unlocking mutex\n",my_id, count); pthread_mutex_unlock(&count_mutex); sleep(1) ; }//End For sem_post(&count_sem); // Next Thread Enters Crit Section pthread_exit(NULL); } /////////// Count_Watch //////////////// void *watch_count(void *t) { long my_id = (long)t; printf("Starting watch_count(): thread %ld\n", my_id); pthread_mutex_lock(&count_mutex); if (count<COUNT_LIMIT) { pthread_cond_wait(&count_threshold_cv, &count_mutex); printf("watch_count(): thread %ld Condition signal received.\n", my_id); printf("watch_count(): thread %ld count now = %d.\n", my_id, count); } pthread_mutex_unlock(&count_mutex); pthread_exit(NULL); } ////////////////// Main //////////////// int main (int argc, char *argv[]) { int i; long t1=0, t2=1, t3=2, t4=3; pthread_t threads[4]; pthread_attr_t attr; sem_init(&count_sem, 0, 1); /* Initialize mutex and condition variable objects */ pthread_mutex_init(&count_mutex, NULL); pthread_cond_init (&count_threshold_cv, NULL); /* For portability, explicitly create threads in a joinable state */ pthread_attr_init(&attr); pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE); pthread_create(&threads[0], &attr, watch_count, (void *)t1); pthread_create(&threads[1], &attr, inc_count, (void *)t2); pthread_create(&threads[2], &attr, inc_count, (void *)t3); pthread_create(&threads[3], &attr, inc_count, (void *)t4); /* Wait for all threads to complete */ for (i=0; i<NUM_THREADS; i++) { pthread_join(threads[i], NULL); } printf ("Main(): Waited on %d threads. Done.\n", NUM_THREADS); /* Clean up and exit */ pthread_attr_destroy(&attr); pthread_mutex_destroy(&count_mutex); pthread_cond_destroy(&count_threshold_cv); pthread_exit(NULL); } I am trying to learn thread scheduling, there is a lot of technical coding that I don't know. I do know in theory how it should work, but having trouble getting started in code... I know, at least I think, this program is not real time and its not meant to be. Some how I need to create a scheduler dispatch to control the threads in the order they should run... RR FCFS SJF ect. Right now I don't have a dispatcher. What I do have is semaphores/ mutex to control the threads. This code does run FCFS... and I have been trying to use semaphores to create a RR.. but having a lot of trouble. I believe it would be easier to create a dispatcher but I dont know how. I need help, I am not looking for answers just direction.. some sample code will help to understand a bit more. Thank you.

    Read the article

  • Multiple threads or process with threads

    - by sergiobuj
    Hi, this is for an assignment so I'm not looking for code. I have to simulate a game where each player has turns and needs to 'pay attention' to what's going on. So far, i know I'll need two threads for each player, one that will sleep until the player's turn and the other paying attention. My question is: Should I work each player as a 'fork' and the threads on the fork, or just create some threads for the player and associate them somehow? It's the first time I've worked with concurrency, semaphores and threads so I'm not sure about the good practices and programming style. Thanks!

    Read the article

  • How can I replace this semaphore with a monitor?

    - by Kurru
    Hi In previous question of mine, someone had meantioned that using Semaphores were expensive in C# compared to using a monitor. So I ask this, how can I replace the semaphore in this code with a monitor? I need function1 to return its value after function2 (in a separate thread) has been completed. I had replaced the Semaphore.WaitOne with a Monitor.Wait and the Semaphore.Release with a Monitor.PulseAll but the PulseAll was being triggered before the Wait causing the program to hang. Any idea how to avoid that race condition? Semaphore semaphore = new Semaphore(0,1); byte b; public byte Function1() { // new thread starting in Function2; semaphore.WaitOne(); return b; } public void Function2() { // do some thing b = 0; semaphore.Release(); }

    Read the article

  • How is thread synchronization implemented, at the assembly language level?

    - by Martin
    While I'm familiar with concurrent programming concepts such as mutexes and semaphores, I have never understood how they are implemented at the assembly language level. I imagine there being a set of memory "flags" saying: lock A is held by thread 1 lock B is held by thread 3 lock C is not held by any thread etc But how is access to these flags synchronized between threads? Something like this naive example would only create a race condition: mov edx, [myThreadId] wait: cmp [lock], 0 jne wait mov [lock], edx ; I wanted an exclusive lock but the above ; three instructions are not an atomic operation :(

    Read the article

  • linux thread synchronization

    - by johnnycrash
    I am new to linux and linux threads. I have spent some time googling to try to understand the differences between all the functions available for thread synchronization. I still have some questions. I have found all of these different types of synchronizations, each with a number of functions for locking, unlocking, testing the lock, etc. gcc atomic operations futexes mutexes spinlocks seqlocks rculocks conditions semaphores My current (but probably flawed) understanding is this: semaphores are process wide, involve the filesystem (virtually I assume), and are probably the slowest. Futexes might be the base locking mechanism used by mutexes, spinlocks, seqlocks, and rculocks. Futexes might be faster than the locking mechanisms that are based on them. Spinlocks dont block and thus avoid context swtiches. However they avoid the context switch at the expense of consuming all the cycles on a CPU until the lock is released (spinning). They should only should be used on multi processor systems for obvious reasons. Never sleep in a spinlock. The seq lock just tells you when you finished your work if a writer changed the data the work was based on. You have to go back and repeat the work in this case. Atomic operations are the fastest synch call, and probably are used in all the above locking mechanisms. You do not want to use atomic operations on all the fields in your shared data. You want to use a lock (mutex, futex, spin, seq, rcu) or a single atomic opertation on a lock flag when you are accessing multiple data fields. My questions go like this: Am I right so far with my assumptions? Does anyone know the cpu cycle cost of the various options? I am adding parallelism to the app so we can get better wall time response at the expense of running fewer app instances per box. Performances is the utmost consideration. I don't want to consume cpu with context switching, spinning, or lots of extra cpu cycles to read and write shared memory. I am absolutely concerned with number of cpu cycles consumed. Which (if any) of the locks prevent interruption of a thread by the scheduler or interrupt...or am I just an idiot and all synchonization mechanisms do this. What kinds of interruption are prevented? Can I block all threads or threads just on the locking thread's CPU? This question stems from my fear of interrupting a thread holding a lock for a very commonly used function. I expect that the scheduler might schedule any number of other workers who will likely run into this function and then block because it was locked. A lot of context switching would be wasted until the thread with the lock gets rescheduled and finishes. I can re-write this function to minimize lock time, but still it is so commonly called I would like to use a lock that prevents interruption...across all processors. I am writing user code...so I get software interrupts, not hardware ones...right? I should stay away from any functions (spin/seq locks) that have the word "irq" in them. Which locks are for writing kernel or driver code and which are meant for user mode? Does anyone think using an atomic operation to have multiple threads move through a linked list is nuts? I am thinking to atomicly change the current item pointer to the next item in the list. If the attempt works, then the thread can safely use the data the current item pointed to before it was moved. Other threads would now be moved along the list. futexes? Any reason to use them instead of mutexes? Is there a better way than using a condition to sleep a thread when there is no work? When using gcc atomic ops, specifically the test_and_set, can I get a performance increase by doing a non atomic test first and then using test_and_set to confirm? *I know this will be case specific, so here is the case. There is a large collection of work items, say thousands. Each work item has a flag that is initialized to 0. When a thread has exclusive access to the work item, the flag will be one. There will be lots of worker threads. Any time a thread is looking for work, they can non atomicly test for 1. If they read a 1, we know for certain that the work is unavailable. If they read a zero, they need to perform the atomic test_and_set to confirm. So if the atomic test_and_set is 500 cpu cycles because it is disabling pipelining, causes cpu's to communicate and L2 caches to flush/fill .... and a simple test is 1 cycle .... then as long as I had a better ratio of 500 to 1 when it came to stumbling upon already completed work items....this would be a win.* I hope to use mutexes or spinlocks to sparilngly protect sections of code that I want only one thread on the SYSTEM (not jsut the CPU) to access at a time. I hope to sparingly use gcc atomic ops to select work and minimize use of mutexes and spinlocks. For instance: a flag in a work item can be checked to see if a thread has worked it (0=no, 1=yes or in progress). A simple test_and_set tells the thread if it has work or needs to move on. I hope to use conditions to wake up threads when there is work. Thanks!

    Read the article

  • Unknown error in Producer/Consumer program, believe it to be an infinite loop.

    - by ray2k
    Hello, I am writing a program that is solving the producer/consumer problem, specifically the bounded-buffer version(i believe they mean the same thing). The producer will be generating x number of random numbers, where x is a command line parameter to my program. At the current moment, I believe my program is entering an infinite loop, but I'm not sure why it is occurring. I believe I am executing the semaphores correctly. You compile it like this: gcc -o prodcon prodcon.cpp -lpthread -lrt Then to run, ./prodcon 100(the number of randum nums to produce) This is my code. typedef int buffer_item; #include <stdlib.h> #include <stdio.h> #include <pthread.h> #include <semaphore.h> #include <unistd.h> #define BUFF_SIZE 10 #define RAND_DIVISOR 100000000 #define TRUE 1 //two threads void *Producer(void *param); void *Consumer(void *param); int insert_item(buffer_item item); int remove_item(buffer_item *item); int returnRandom(); //the global semaphores sem_t empty, full, mutex; //the buffer buffer_item buf[BUFF_SIZE]; //buffer counter int counter; //number of random numbers to produce int numRand; int main(int argc, char** argv) { /* thread ids and attributes */ pthread_t pid, cid; pthread_attr_t attr; pthread_attr_init(&attr); pthread_attr_setscope(&attr, PTHREAD_SCOPE_SYSTEM); numRand = atoi(argv[1]); sem_init(&empty,0,BUFF_SIZE); sem_init(&full,0,0); sem_init(&mutex,0,0); printf("main started\n"); pthread_create(&pid, &attr, Producer, NULL); pthread_create(&cid, &attr, Consumer, NULL); printf("main gets here"); pthread_join(pid, NULL); pthread_join(cid, NULL); printf("main done\n"); return 0; } //generates a randum number between 1 and 100 int returnRandom() { int num; srand(time(NULL)); num = rand() % 100 + 1; return num; } //begin producing items void *Producer(void *param) { buffer_item item; int i; for(i = 0; i < numRand; i++) { //sleep for a random period of time int rNum = rand() / RAND_DIVISOR; sleep(rNum); //generate a random number item = returnRandom(); //acquire the empty lock sem_wait(&empty); //acquire the mutex lock sem_wait(&mutex); if(insert_item(item)) { fprintf(stderr, " Producer report error condition\n"); } else { printf("producer produced %d\n", item); } /* release the mutex lock */ sem_post(&mutex); /* signal full */ sem_post(&full); } return NULL; } /* Consumer Thread */ void *Consumer(void *param) { buffer_item item; int i; for(i = 0; i < numRand; i++) { /* sleep for a random period of time */ int rNum = rand() / RAND_DIVISOR; sleep(rNum); /* aquire the full lock */ sem_wait(&full); /* aquire the mutex lock */ sem_wait(&mutex); if(remove_item(&item)) { fprintf(stderr, "Consumer report error condition\n"); } else { printf("consumer consumed %d\n", item); } /* release the mutex lock */ sem_post(&mutex); /* signal empty */ sem_post(&empty); } return NULL; } /* Add an item to the buffer */ int insert_item(buffer_item item) { /* When the buffer is not full add the item and increment the counter*/ if(counter < BUFF_SIZE) { buf[counter] = item; counter++; return 0; } else { /* Error the buffer is full */ return -1; } } /* Remove an item from the buffer */ int remove_item(buffer_item *item) { /* When the buffer is not empty remove the item and decrement the counter */ if(counter > 0) { *item = buf[(counter-1)]; counter--; return 0; } else { /* Error buffer empty */ return -1; } }

    Read the article

  • Is there a limit on the number of mutex objects that can be created in a Windows process?

    - by young-phillip
    I'm writing a c# application that can create a series of request messages. Each message could have a response, that needs to be waited on by a consumer. Where the number of outstanding request messages is constrained, I have used the windows EVENT to solve this problem. However, I know there is a limit on how many EVENT objects can be created in a single process, and in this instance, its possible I might exceed that limit. Does anyone know if there is a similar limit on creation of mutex objects or semaphores? I know this can be solved by some sort of pool of shared resources, that are grabbed by consumers when they need to wait, but it would be more convenient if each request message could have its own sync object.

    Read the article

  • Reflection: take values from an unknown running alpplication

    - by Dr.Lesh
    I'm writing an application that searchs for Semaphore types in the fields of unknown classes in unknown application (passed by user). I made it using Reflection and it worked. Now I want to fill up these semaphores with values, taking them from a running instance of this unknown application. So i took the class with "main" method of this unknown application, made a newInstance, and passed it when invoking the main method to start the application: Class mainClass = getItSomeWhere(); Object instance = mainClass.newInstance(); Method mainMethod = mainClass.getDeclaredMethod("main", new Class[]{String[].class}); mainMethod.invoke(instance, new Object[]{args}); and it worked fine. Now, how can I get the semaphore values, taking them from the classes of this running application, when I only have an instance of the main class? Many thanks for the answers.

    Read the article

< Previous Page | 1 2 3  | Next Page >