Search Results

Search found 5954 results on 239 pages for 'cpu cores'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 35/239 | < Previous Page | 31 32 33 34 35 36 37 38 39 40 41 42  | Next Page >

  • Best CPUs for speeding up compiling times of C++ w/ DistGCC

    - by Jay
    I'm putting together a distributed build farm with DistGCC to speed up our teams compile times and just looking for thoughts on which processors to use in the hosts. Are we going to get a noticeable decrease in time using 8 cores vs. 4-hyperthreaded cores? Big difference in time between i7 and Xeon? etc, etc. Just need advice from people who've put together kick-a build clusters. We've got a majority of the normal things to speed up builds in place (pre-compiled headers, ccache, local gigabit connections between them, tons of ram, etc) so please just give advice on the best processor to use. And money is a factor, but anythings doable if the performance increase is noticeable. Thanks. Jay EDIT: Although any advice IS welcome, please refrain from "Do this first" posts as we're not planning on skimping on things like SSD, maxed out RAM, etc. My personal system is a iMac Quad-core i5 with 8GB of RAM. When I build our project locally, my processor floats around 99-100% a majority of the time, which makes me assume it is a bottleneck, even if you made everything else faster. My ram on the other hand doesn't even get close to maxing out. It's also worth noting that I did research this, however every discussion I could find was primarily for gaming machines, which is obviously a different beast in usage. These machines won't even have monitors or anything but integrated graphics since they have one purpose: Build freakin fast. (hopefully)

    Read the article

  • Best CPUs for speeding up compiling times of C++ w/ DistGCC

    - by Jay
    I'm putting together a distributed build farm with DistGCC to speed up our teams compile times and just looking for thoughts on which processors to use in the hosts. Are we going to get a noticeable decrease in time using 8 cores vs. 4-hyperthreaded cores? Big difference in time between i7 and Xeon? etc, etc. Just need advice from people who've put together kick-a build clusters. We've got a majority of the normal things to speed up builds in place (pre-compiled headers, ccache, local gigabit connections between them, tons of ram, etc) so please just give advice on the best processor to use. And money is a factor, but anythings doable if the performance increase is noticeable. Thanks. Jay EDIT: Although any advice IS welcome, please refrain from "Do this first" posts as we're not planning on skimping on things like SSD, maxed out RAM, etc. My personal system is a iMac Quad-core i5 with 8GB of RAM. When I build our project locally, my processor floats around 99-100% a majority of the time, which makes me assume it is a bottleneck, even if you made everything else faster. My ram on the other hand doesn't even get close to maxing out. It's also worth noting that I did research this, however every discussion I could find was primarily for gaming machines, which is obviously a different beast in usage. These machines won't even have monitors or anything but integrated graphics since they have one purpose: Build freakin fast. (hopefully)

    Read the article

  • Best CPUs for speeding up compiling times of C++ w/ DistGCC

    - by Jay
    I'm putting together a distributed build farm with DistGCC to speed up our teams compile times and just looking for thoughts on which processors to use in the hosts. Are we going to get a noticeable decrease in time using 8 cores vs. 4-hyperthreaded cores? Big difference in time between i7 and Xeon? etc, etc. Just need advice from people who've put together kick-a build clusters. We've got a majority of the normal things to speed up builds in place (pre-compiled headers, ccache, local gigabit connections between them, tons of ram, etc) so please just give advice on the best processor to use. And money is a factor, but anythings doable if the performance increase is noticeable. Thanks. Jay EDIT: Although any advice IS welcome, please refrain from "Do this first" posts as we're not planning on skimping on things like SSD, maxed out RAM, etc. My personal system is a iMac Quad-core i5 with 8GB of RAM. When I build our project locally, my processor floats around 99-100% a majority of the time, which makes me assume it is a bottleneck, even if you made everything else faster. My ram on the other hand doesn't even get close to maxing out. It's also worth noting that I did research this, however every discussion I could find was primarily for gaming machines, which is obviously a different beast in usage. These machines won't even have monitors or anything but integrated graphics since they have one purpose: Build freakin fast. (hopefully)

    Read the article

  • procdump on w3wp.exe: Only part of a ReadProcessMemory or WriteProcessMemory request was completed.

    - by JakeS
    I'm having a problem with an IIS application that occasionally spikes up in CPU usage, and am trying to use procdump to get a memory dump for examination. I'm running "procdump.exe -64 -mA 9999" where 9999 is the pid of the process. But every time I do it, I get an error: Only part of a ReadProcessMemory or WriteProcessMemory request was completed. Doing this also recycles the apppool, relieving the CPU spike, so I can't keep trying until I get it right. Does anyone know what is going wrong?

    Read the article

  • Intel® Core™2 Duo Desktop Processor vs Intel® Core™ i3 Desktop Processor?

    - by metal gear solid
    Intel® Core™2 Duo Desktop Processor vs Intel® Core™ i3 Desktop Processor? Which CPU is better to buy ? Intel® Core™ i3-530 Processor (4M Cache, 2.93 GHz) (it supports DDR3 also) or Intel® Core™2 Duo Processor E7500 (3M Cache, 2.93 GHz, 1066 MHz FSB) (it supports DDR2 only ) Although I do not play games on my PC but I need good performance in Adobe Photoshop, Watching Full HD Movies. I need good performance in Multitasking. Along with any of these CPU I would purchase 2 GB x 2 stick of RAM. and I will use Windows 7. and I will use Microsoft VPC images also with MS Virtual PC.

    Read the article

  • Intel Core i7 clockspeeds

    - by skiwi
    I've got an Intel i7 3770 CPU and am generally quite happy about it. However when I am just browsing around without programs running in background it seems to underclock itself quite a lot. This is not a real issue, however there seems to be a small hiccup (0.1s maybe) when it clocks itself higher again. Does anyone have a similar issue? Can I change the thresholds that are being used to change the clocks? I absolutely like that it underclocks the CPU whenever I am not around, but it shouldn't start to annoy me. It might also be related to Windows 8.1, but I am not sure about that, just saying that it's a possibility. I ran LatencyMon as suggested and got this (worrying?) result:

    Read the article

  • The meaning of thermal throttle counters and package power limit notifications in Linux

    - by Trustin Lee
    Whenever I do some performance testing on my Linux-installed MacBook Pro, I often see the following messages in dmesg: Aug 8 09:29:31 infinity kernel: [79791.789404] CPU1: Package power limit notification (total events = 40365) Aug 8 09:29:31 infinity kernel: [79791.789408] CPU3: Package power limit notification (total events = 40367) Aug 8 09:29:31 infinity kernel: [79791.789411] CPU2: Package power limit notification (total events = 40453) Aug 8 09:29:31 infinity kernel: [79791.789414] CPU0: Package power limit notification (total events = 40453) I also see the throttle counters in the sysfs increases over time: trustin@infinity:/sys/devices/system/cpu/cpu0/thermal_throttle $ ls core_power_limit_count package_power_limit_count core_throttle_count package_throttle_count $ cat core_power_limit_count 0 $ cat core_throttle_count 41912 $ cat package_power_limit_count 67945 $ cat package_throttle_count 67565 What do these counters mean? Do they affect the performance of CPU or system? Do they result in increased deviation of performance numbers? (i.e. Do they prevent me from getting reliable performance numbers?) If so, how do I avoid these messages and increasing counters? Would running the performance tests on a well-cooled desktop system help?

    Read the article

  • Processor not running at max speed

    - by Andrew Hampton
    My laptop has an Intel Core 2 Duo T9300 which should be running at 2.5GHz, however CPU-Z consistently reports my Core speed at right under 1.6Ghz (8x multiplier and ~200MHz Bus Speed). Even when I'm doing heavy development work and the processor is running at 100% for extended periods of time the core speed reported by CPU-Z never goes up to 2.5GHz. My understanding is that this reduction in speed is to save power, but this happens even when I'm plugged into the outlet. Does anyone know why this is happening or how to fix it?

    Read the article

  • Processor always at max speed

    - by Pratyush Nalam
    I am running Windows 8 Pro on an Apple MacBook Pro 9,1 Mid 2012 15 inch non retina. It has a Core i7-3720 QM CPU @ 2.6 GHz. For the past few days, I have noticed that it is constantly running at max speed which is 2.59 GHz. Before, it used to run at 1.5 - 1.8 GHz on normal usage. And, the weirder thing is that CPU usage is minimal. Screenshot: So, what is the reason for this? And is it harmful?

    Read the article

  • please assist me debug a stack trace

    - by Eds
    I am having an issue on a windows 2003 terminal server, where the system process is using a constantly high amount of CPU. I believe that it is the srv.sys that is causing an issue, but not quite sure how to fully diagnose the problem. I have looked at the stack for the srv.sys!workerthread, which is what seems to be using the CPU. The stack is as follows: 0 ntoskrnl.exe!KeSetBasePriorityThread+0xf7 1 ntoskrnl.exe!MiDeleteAddressesInWorkingSet+0x103 2 srv.sys!WorkerThread+0x7c 3 ntoskrnl.exe!FsRtNotifyFilterReportChange+0x10 4 ntoskrnl.exe!RtClearBits+0x24 Can anyone offer any advice based on the above, or some other things I could look at in order to get to the bottom of this? Many Thanks, Eds

    Read the article

  • CPU Utilization LAMP stack

    - by Max
    We've got an ec2 m2.4xlarge running Magento (centos 5.6, httpd 2.2, php 5.2.17 with eaccelerator 0.9.5.3, mysql 5.1.52). Right now we're getting a large traffic spike, and our top looks like this: top - 09:41:29 up 31 days, 1:12, 1 user, load average: 120.01, 129.03, 113.23 Tasks: 1190 total, 18 running, 1172 sleeping, 0 stopped, 0 zombie Cpu(s): 97.3%us, 1.8%sy, 0.0%ni, 0.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.4%st Mem: 71687720k total, 36898928k used, 34788792k free, 49692k buffers Swap: 880737784k total, 0k used, 880737784k free, 1586524k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2433 mysql 15 0 23.6g 4.5g 7112 S 564.7 6.6 33607:34 mysqld 24046 apache 16 0 411m 65m 28m S 26.4 0.1 0:09.05 httpd 24360 apache 15 0 410m 60m 25m S 26.4 0.1 0:03.65 httpd 24993 apache 16 0 410m 57m 21m S 26.1 0.1 0:01.41 httpd 24838 apache 16 0 428m 74m 20m S 24.8 0.1 0:02.37 httpd 24359 apache 16 0 411m 62m 26m R 22.3 0.1 0:08.12 httpd 23850 apache 15 0 411m 64m 27m S 16.8 0.1 0:14.54 httpd 25229 apache 16 0 404m 46m 17m R 10.2 0.1 0:00.71 httpd 14594 apache 15 0 404m 63m 34m S 8.4 0.1 1:10.26 httpd 24955 apache 16 0 404m 50m 21m R 8.4 0.1 0:01.66 httpd 24313 apache 16 0 399m 46m 22m R 8.1 0.1 0:02.30 httpd 25119 apache 16 0 411m 59m 23m S 6.8 0.1 0:01.45 httpd Questions: Would giving msyqld more memory help it cache queries and react faster? If so, how? Other than splitting mysql and php to separate servers (which we're about to do) is there anything else we could/should be doing? Thanks! UPDATE: Here's our my.cnf along with the output of mysqltuner. It looks like a cache problem. Thanks again! # cat /etc/my.cnf [client] port = **** socket = /var/lib/mysql/mysql.sock [mysqld] datadir=/mnt/persistent/mysql port=**** socket=/var/lib/mysql/mysql.sock key_buffer = 512M max_allowed_packet = 64M table_cache = 1024 sort_buffer_size = 8M read_buffer_size = 4M read_rnd_buffer_size = 2M myisam_sort_buffer_size = 64M thread_cache_size = 128M tmp_table_size = 128M join_buffer_size = 1M query_cache_limit = 2M query_cache_size= 64M query_cache_type = 1 max_connections = 1000 thread_stack = 128K thread_concurrency = 48 log-bin=mysql-bin server-id = 1 wait_timeout = 300 innodb_data_home_dir = /mnt/persistent/mysql/ innodb_data_file_path = ibdata1:10M:autoextend innodb_buffer_pool_size = 20G innodb_additional_mem_pool_size = 20M innodb_log_file_size = 64M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit = 1 innodb_lock_wait_timeout = 50 innodb_thread_concurrency = 48 ft_min_word_len=3 [myisamchk] ft_min_word_len=3 key_buffer = 128M sort_buffer_size = 128M read_buffer = 2M write_buffer = 2M # ./mysqltuner.pl >> MySQLTuner 1.2.0 - Major Hayden <[email protected]> >> Bug reports, feature requests, and downloads at http://mysqltuner.com/ >> Run with '--help' for additional options and output filtering -------- General Statistics -------------------------------------------------- [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.1.52-log [OK] Operating on 64-bit architecture -------- Storage Engine Statistics ------------------------------------------- [--] Status: +Archive -BDB +Federated +InnoDB -ISAM -NDBCluster [--] Data in MyISAM tables: 2G (Tables: 26) [--] Data in InnoDB tables: 749M (Tables: 250) [!!] Total fragmented tables: 262 -------- Security Recommendations ------------------------------------------- -------- Performance Metrics ------------------------------------------------- [--] Up for: 31d 2h 30m 38s (680M q [253.371 qps], 2M conn, TX: 4825B, RX: 236B) [--] Reads / Writes: 89% / 11% [--] Total buffers: 20.6G global + 15.1M per thread (1000 max threads) [OK] Maximum possible memory usage: 35.4G (51% of installed RAM) [OK] Slow queries: 0% (35K/680M) [OK] Highest usage of available connections: 53% (537/1000) [OK] Key buffer size / total MyISAM indexes: 512.0M/457.2M [OK] Key buffer hit rate: 100.0% (9B cached / 264K reads) [OK] Query cache efficiency: 42.3% (260M cached / 615M selects) [!!] Query cache prunes per day: 4384652 [OK] Sorts requiring temporary tables: 0% (1K temp sorts / 38M sorts) [!!] Joins performed without indexes: 100404 [OK] Temporary tables created on disk: 17% (7M on disk / 45M total) [OK] Thread cache hit rate: 99% (537 created / 2M connections) [!!] Table cache hit rate: 0% (1K open / 946K opened) [OK] Open file limit used: 9% (453/5K) [OK] Table locks acquired immediately: 99% (758M immediate / 758M locks) [OK] InnoDB data size / buffer pool: 749.3M/20.0G -------- Recommendations ----------------------------------------------------- General recommendations: Run OPTIMIZE TABLE to defragment tables for better performance Enable the slow query log to troubleshoot bad queries Adjust your join queries to always utilize indexes Increase table_cache gradually to avoid file descriptor limits Variables to adjust: query_cache_size (> 64M) join_buffer_size (> 1.0M, or always use indexes with joins) table_cache (> 1024)

    Read the article

  • AMD FX 8350 potentially overheating

    - by rhughes
    I am worried that my CPU is overheating when running at maximum capacity. I have not overclocked the machine. The machine often powers down after a couple of minutes of max CPU usage. I get the following events in the Event Log after the crash: Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 11/10/2013 12:05:40 Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: home-pc Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly. What can I do to confirm this or further narrow down this issue? Due to the sudden nature of the crash, no MEMORY.DMP is created I believe. I am happy to post any extra information that is needed.

    Read the article

  • CPU/JVM/JBoss 7 slows down over time

    - by lukas
    I'm experiencing performance slow down on JBoss 7.1.1 Final. I wrote simple program that demostrates this behavior. I generate an array of 100,000 of random integers and run bubble sort on it. @Model public class PerformanceTest { public void proceed() { long now = System.currentTimeMillis(); int[] arr = new int[100000]; for(int i = 0; i < arr.length; i++) { arr[i] = (int) (Math.random() * 200000); } long now2 = System.currentTimeMillis(); System.out.println((now2 - now) + "ms took to generate array"); now = System.currentTimeMillis(); bubbleSort(arr); now2 = System.currentTimeMillis(); System.out.println((now2 - now) + "ms took to bubblesort array"); } public void bubbleSort(int[] arr) { boolean swapped = true; int j = 0; int tmp; while (swapped) { swapped = false; j++; for (int i = 0; i < arr.length - j; i++) { if (arr[i] > arr[i + 1]) { tmp = arr[i]; arr[i] = arr[i + 1]; arr[i + 1] = tmp; swapped = true; } } } } } Just after I start the server, it takes approximately 22 seconds to run this code. After few days of JBoss 7.1.1. running, it takes 330 sec to run this code. In both cases, I launch the code when the CPU utilization is very low (say, 1%). Any ideas why? I run the server with following arguments: -Xms1280m -Xmx2048m -XX:MaxPermSize=2048m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Duser.timezone=UTC -Djboss.server.default.config=standalone-full.xml -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n I'm running it on Linux 2.6.32-279.11.1.el6.x86_64 with java version "1.7.0_07". It's within J2EE applicaiton. I use CDI so I have a button on JSF page that will call method "proceed" on @RequestScoped component PerformanceTest. I deploy this as separate war file and even if I undeploy other applications, it doesn't change the performance. It's a virtual machine that is sharing CPUs with another machine but that one doesn't consume anything. Here's yet another observation: when the server is after fresh start and I run the bubble sort, It utilizes 100% of one processor core. It never switches to another core or drops utilization below 95%. However after some time the server is running and I'm experiencing the performance problems, the method above is utilizing CPU core usually 100%, however I just found out from htop that this task is being switched very often to other cores. That is, at the beginning it's running on core #1, after say 2 seconds it's running on #5 then after say 2 seconds #8 etc. Furthermore, the utilization is not kept at 100% at the core but sometimes drops to 80% or even lower. For the server after fresh start, even though If I simulate a load, it never switches the task to another core.

    Read the article

  • Laptop turns off after 20 minutes of use

    - by Christoph
    My laptop a sony vaio VGN-NW11S http://www.trustedreviews.com/Sony-VAIO-VGN-NW11S-S---15-5in-Laptop_Laptop_review. Everytime i turn it on, in safe mode or not, if i try to open an application i.e. run a process such as google chrome or event viewer, defrag, virus scan, it completely turns off without warning, nor giving a trace of events the next time I switch it on. Apart from that, I had worries it might be my battery or power supply but I dont think it is that, I took the laptop apart cleaning fans etc. and have ordered some cpu paste as I checked to see the condition of the processor. I will post to see if re-applying the paste works. One more thing, when the heavy processes kick in, the fan starts to make a lot of noise, maybe trying to cool down the CPU? Any ideas on what else it could be and what I could do to test what is wrong?

    Read the article

  • PC won't post after replacing mobo

    - by Dan
    I am trying to fix an older HP desktop (~2 yrs old) that didn't seem to have video working. The video card on the machine was integrated. After some debugging, it looked to be the mobo, so I went out and got a new one (also with integrated video). Now, it pretty much does the same thing. Here's the symptoms: no video, doesn't post at all, fans DO spin (both chasis and CPU). Any other ideas? I suppose it could be the CPU, but I would think if it was, it would at least post.

    Read the article

  • What happens when you add a Graphics card to a i7 with build in graphics (e.g. HD 4000)

    - by Matt
    I'm thinking of upgrading my computer from an AMD Phenom II X4 955BE with a AMD Radeon HD 6800 Graphics card (not integrated) to using a Intel Core i7 3770. As I have no knowledge of integrated graphics, my question is, what happens to the computing power when not using the HD4000 integrated graphics of the CPU (does it mean the CPU will run faster then it would if I relied on it?) Also what is better the CPUs inbuilt HD4000 or my Radeon Graphics card? I am mostly interested in terms of content creation: Using Adobe After Effects, 3D Rendering etc. Not too bothered about gaming performance. I will be using the spare parts of this build and older systems to make a second computer for network renders so what will be the advantages of keeping the Radeon with the current system for that?

    Read the article

  • what's POST code 18 / can I run an ASRock P55 Pro + Core i3 530?

    - by Michael Borgwardt
    When I switch my newly built PC on, the fans start up, but I get nothing on the monitor, and the POST display on the motherboard runs quickly through various codes and then stops at code 18, which does not appear in the manual (the list there seems identical to this one). This lasts about 10 seconds, after which the machine shuts down. After a pause (also about 10 seconds) it starts up again, and this repeats until I cut the power. Interestingly, when I push the reset button, it stops at POST code 16 (which is also not listed in the manual). Does anyone have information about the meaning of those codes? Motherboard: ASRock P55 Pro CPU: Intel Core i3 530 Graphics Card: Sapphire HHD 5750 Does anyone have experience with that Motherboard/CPU combination? It says on the packaging and in the manual that it's only compatible with Core i5 and i7 (no mention of i3), but on the maker's product page, the i3 is listed as compatible as well.

    Read the article

  • Can a processor upgrade cause BSOD?

    - by Daniel
    I Had the following hardware setup: Phenom II X4 945 Asus M4A97 4GB DDR2 OCZ Radeon HD5850 OCZ Agility 2 120 GB Windows 7 x64 Pro Fully updated (latest drivers and windows update patches) Then I bought an used Phenom II X6 1090T and installed it without formatting. Since that my computer started BSODing almost every time I'm playing any game and with different error messages, like: page fault in non-paged area bad pool header the video memory manager found a problem error in dxgmm1.sys(or something like that) And when it doesn't BSOD the game simply crashes. I have tried: Updating BIOS Reseting BIOS to defaults Reinstalling Video Drivers Installing the latest DirectX All that's left is to do a full format and I don't want to do that since it's going to be a lot of work to fine tune windows to my preferences again. So is my "new" processor defective or do I really need to format the computer? Update: I use a (properly installed) custom cooler from Coolermaster and both the BIOS and the Open Hardware Monitor(Application) attest the CPU is not overheating, so I guess the CPU its defective and since I bought it from a guy over the Internet I'm probably screwed

    Read the article

  • Should I bother upgrading my Opteron 270 Server?

    - by MousePad
    I have an Opteron Server machine (in a large workstation class case) running on the Tyan 2895 motherboard. It's a dual CPU socket board, but I only have one 270 in there. I have 4GB of RAM, but less than 3GB is addressable, even in 64bit mode, due to the way the board is designed. Is it worth spending a few hundred on an additional CPU and maybe some more RAM? The other problem is that one of the two SATA ports on the board had its wire socket break off. So only one drive can be run as of now. I could have it repaired, but at what cost? Add in the fact that the power supply is gunked up with dust and it's a bit of a nightmare. I actually work about it getting too hot. Seems that for the money I could buy a new server rack from Dell, but it also seems a shame to waste an otherwise working, and for my needs still very fast machine.

    Read the article

  • How does one calculate CPU utilization programmatically ?

    - by Scott Davies
    Hi, I have a benchmarking program that calculates the time (in milliseconds and ticks), for a persistance to Entity Framework 4.0. Is there a way to calculate CPU load ? I am guessing that I would need to query Windows to find out my CPU frequency, how many cores, etc. Does this sound right ? If so, what part of the .NET framework relates to querying the system ? I am guessing System.Diagnostics ? Thanks, Scott

    Read the article

  • code::blocks CPU spike

    - by User1
    I'm using Code::Blocks 8.02 in Ubuntu 9.10. If I leave Code::Blocks open for several hours, I notice that it's consuming a LOT of CPU. If I actually click on the application, the CPU usage immediately drops. Is anyone else having this problem? Is there a solution?

    Read the article

  • Optimizing tasks to reduce CPU in a trading application

    - by Joel
    Hello, I have designed a trading application that handles customers stocks investment portfolio. I am using two datastore kinds: Stocks - Contains unique stock name and its daily percent change. UserTransactions - Contains information regarding a specific purchase of a stock made by a user : the value of the purchase along with a reference to Stock for the current purchase. db.Model python modules: class Stocks (db.Model): stockname = db.StringProperty(multiline=True) dailyPercentChange=db.FloatProperty(default=1.0) class UserTransactions (db.Model): buyer = db.UserProperty() value=db.FloatProperty() stockref = db.ReferenceProperty(Stocks) Once an hour I need to update the database: update the daily percent change in Stocks and then update the value of all entities in UserTransactions that refer to that stock. The following python module iterates over all the stocks, update the dailyPercentChange property, and invoke a task to go over all UserTransactions entities which refer to the stock and update their value: Stocks.py # Iterate over all stocks in datastore for stock in Stocks.all(): # update daily percent change in datastore db.run_in_transaction(updateStockTxn, stock.key()) # create a task to update all user transactions entities referring to this stock taskqueue.add(url='/task', params={'stock_key': str(stock.key(), 'value' : self.request.get ('some_val_for_stock') }) def updateStockTxn(stock_key): #fetch the stock again - necessary to avoid concurrency updates stock = db.get(stock_key) stock.dailyPercentChange= data.get('some_val_for_stock') # I get this value from outside ... some more calculations here ... stock.put() Task.py (/task) # Amount of transaction per task amountPerCall=10 stock=db.get(self.request.get("stock_key")) # Get all user transactions which point to current stock user_transaction_query=stock.usertransactions_set cursor=self.request.get("cursor") if cursor: user_transaction_query.with_cursor(cursor) # Spawn another task if more than 10 transactions are in datastore transactions = user_transaction_query.fetch(amountPerCall) if len(transactions)==amountPerCall: taskqueue.add(url='/task', params={'stock_key': str(stock.key(), 'value' : self.request.get ('some_val_for_stock'), 'cursor': user_transaction_query.cursor() }) # Iterate over all transaction pointing to stock and update their value for transaction in transactions: db.run_in_transaction(updateUserTransactionTxn, transaction.key()) def updateUserTransactionTxn(transaction_key): #fetch the transaction again - necessary to avoid concurrency updates transaction = db.get(transaction_key) transaction.value= transaction.value* self.request.get ('some_val_for_stock') db.put(transaction) The problem: Currently the system works great, but the problem is that it is not scaling well… I have around 100 Stocks with 300 User Transactions, and I run the update every hour. In the dashboard, I see that the task.py takes around 65% of the CPU (Stock.py takes around 20%-30%) and I am using almost all of the 6.5 free CPU hours given to me by app engine. I have no problem to enable billing and pay for additional CPU, but the problem is the scaling of the system… Using 6.5 CPU hours for 100 stocks is very poor. I was wondering, given the requirements of the system as mentioned above, if there is a better and more efficient implementation (or just a small change that can help with the current implemntation) than the one presented here. Thanks!! Joel

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 31 32 33 34 35 36 37 38 39 40 41 42  | Next Page >