Search Results

Search found 2667 results on 107 pages for 'peopletools strategy'.

Page 84/107 | < Previous Page | 80 81 82 83 84 85 86 87 88 89 90 91  | Next Page >

  • split virtualization design based on environment or server role?

    - by Dan
    I'm setting up the server environment for a new software development group, which will include 4 test environments. These are web applications, so each environment will have an application server and a database server. I'm planning on buying two physical servers (e.g. 6-core CPU each with 12GB or so of RAM), and I'm thinking virtualization is appropriate here. With that in mind, I've thought of a couple ways that I could organize the virtualization strategy: - Separated by server role: Server 1 has all the application servers, each in their own guest VM. Server 2 has all the databases. OR - Separated by environment: Server 1 has a VM for two of the environments, with the VM containing both the app server and the database server. Server 2 would also contain two test environments, with the same style (app server and database in same VM). The advantages I see with all the app servers on one server and all the databases on another server is that I could probably be more efficient with the database server (one instance running multiple databases). But the other option seems easier to manage (archives/restorations would be contained in a single VM). Any recommendations? TIA.

    Read the article

  • About to go live: virtual dedicated server or cloud?

    - by morpheous
    I am about to launch my startup company, and we will be going live in a few weeks time. We have really tight budgetary constraints, since we are bootstrapping - and would prefer not to raise external capital. I cant use shared hosting because I need more control of the server machine (for technical reasons - e.g. using proprietary extensions to PHP, Apache and in the database layer as well) - but want to control costs and dont want to go fully private server route, until we have determined the market size etc. So the only real alternatives AFAIK is between virtual server and the cloud. At the moment, cloud services seem a bit "vague" to me. My understanding is that they allow an entity to outsource its IT infrastructure, which in my mind (at least), is indistinguishable from what a hosting provider provides (at least from a functional point of view) - I would like to seek some clarification on exactly what the difference between the two is. Back to my original question, my requirements are: IT infrastructure that can scale with growth Ability to have control of the machine (for e.g. to install our internally developed libraries etc) Backup software that is flexible and comprehensive enough (yet simple to use), that allows a (secured) backup strategy to be implemented. On this issue, I have always wondered where the actual backed up data was stored (since the physical machines are remote, and one cant get access to any actual tapes etc backed onto). I would also like some advice and recommendations in this area. Regarding data size, I am expecting the dataset to be increasing by a few megabytes of data (originally, say 10Mb, in about a years time, possibly 50Mb) every day. As an aside, I have decided to deploy on a Debian server (most of my additional libraries etc were compiled and built on a Debian machine). Mindful of all of the above, I would like some advice (and reason) as to which route to take. I would also like some advice on which backup software to use, from people who have walked a similar path.

    Read the article

  • Windows 7 doesn't start anymore

    - by martani_net
    Hi, I've experienced some BSOD on windows 7 RC, and some freezing when startup, but today was big surprise, it doesn't start anymore. I tried to start on safe mode and no results too, it shows the starting animation, then a blue screen for less than a second and turns off immediately. The only thing I remember did today is update flash player under Firefox, then chrome stopped working even after logging off, and once restarting, it doesn't start anymore. Anyone experienced the same issue? any hints? [EDIT 3] Solved : Windows 7 have a very smart repair strategy, it works automatically, and it tried every possible fix, what fixed my problem was the system restore to a previous date, all this happened automatically. [EDIT2] these are the last lines in the ntbtlog.txt file Did not load driver \SystemRoot\System32\drivers\vga.sys Loaded driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Loaded driver \SystemRoot\system32\drivers\CHDRT32.sys Loaded driver \SystemRoot\system32\DRIVERS\VSTAZL3.SYS Loaded driver \SystemRoot\system32\DRIVERS\VSTDPV3.SYS Loaded driver \SystemRoot\system32\DRIVERS\VSTCNXT3.SYS Loaded driver \SystemRoot\system32\drivers\modem.sys Loaded driver \SystemRoot\system32\DRIVERS\usbccgp.sys Loaded driver \SystemRoot\System32\Drivers\usbvideo.sys [edit] this is the BSOD I get : http://twitpic.com/i87cx Thank you.

    Read the article

  • Setting up a network where packets are traced

    - by Marcus
    My situation is the following: I have an internet connection, which is shared between people. More or less obviously, people is using it to download illegal stuff. Since I'm the owner of the connection, I want to avoid being sued. I don't want to prevent the people from doing the things they want, but I want to be legally safe. Now, I have relatively little competences in network administration, so I was wondering: is it possible to setup a network, where the source and destination of the packets are logged? I would use this to prove, in case of lawsuit, that the traffic was coming from a given machine. if the idea is feasible, is there any wireless router on which I can install linux, where I can install the packet sniffer? how much space could the logs take (containing only the timestamp/source/destination), per GB of traffic? a very rough estimation would be very helpful. if a machine on my network is sending bittorrent packets to a certain IP, would this log be able to reflect the time, source ip and destination ip? I assume that obviously the torrent data would be encrypted and un-decryptable. Am I missing something? Is there a better strategy? Any pointer to documentation would be helpful as well - in that case, I would use this as starting point.

    Read the article

  • Production deployment to EC2 with minimal downtime

    - by jensendarren
    I have a simple web application deployed on a large instance with EC2. I now want to deploy the latest code to this server but I want to do this in a way which minimizes downtime and is a smooth as possible for the end user. Here is my plan: Fire up another large instance Install all the software layers on that instance Restore and attach an EBS drive to the instance Deploy our latest production ready code on the new instance Run all tests (including manual testing of the application) (If tests pass) Put a "Site Under Maintenance" notice on the live site. Backup the EBS instance on the live site Detach the EBS instance from the new server and replace with the latest backup Use ec2-associate-address to move the IP address to the new instance Sit back and wait for traffic to start flowing though the new instance Terminate the old instance Does this seem like a good strategy? Are there any tutorials or books that might cover this topic? I have already read Cloud Application Architectures by George Reese, which is an excellent book, but does not cover deployment. Additionally, I know that there are tools that can help with this like RightScale or enStratus which I will use when I start using more than one instance.

    Read the article

  • Performance data collection for short-running, ephemeral servers

    - by ErikA
    We're building a medical image processing software stack, currently hosted on various AWS resources. As part of this application, we have a handful of long-running servers (database, load balancers, web application, etc.). Collecting performance data on those servers is quite simple - my go-to- recipe of Nagios (for monitoring/notifications) and Munin (for collection of performance data and displaying trends) will work just fine. However - as part of this application, we are constantly starting up and terminating compute instances on EC2. In typical usage, these compute instances start up, configure themselves, receive a job from a message queue, and then get to work processing that job, which takes anywhere from 15 minutes to over 8 hours. After job completion, these instances get terminated, never to be heard from again. What is a decent strategy for collecting performance data on these short-lived instances? I don't necessarily need monitoring on them - if they fail for whatever reason, our application will detect this and handle re-starting the job on another instance or raising the flag so an administrator can take a look at things. However, it still would be useful to collect information like CPU (user, idle, iowait, etc.), memory usage, network traffic, disk read/write data, etc. In our internal database, we track the instance ID of the machine that runs each job, and it would be quite helpful to be able to look up performance data for a specific instance ID for troubleshooting and profiling. Munin doesn't seem like a great candidate, as it requires maintaining a list of munin nodes in a text file - far from ideal for an environment with a high amount of churn, and for the short amount of time each node will be running, I'd rather keep the full-resolution data indefinitely than have RRD water down the data over time. In the end, my guess is that this will require a monitoring engine that: uses a database (MySQL, SQLite, etc.) for configuration and data storage exposes an API for adding/removing hosts and services Are there other things I should be thinking about when evaluating options? Perhaps I'm over-thinking this, though, and just ought to run sar at 1-minute intervals on these short-lived instances and collect the sar db files prior to termination.

    Read the article

  • Synchronize two directories on linux pc

    - by Gab
    I need a distributed filesystem (or a synchronization tool) that is capable of keeping a directory synchronized across 4 pc. My requirements are: offline access (data must be available offline on each pc) preserve execution rights: some files are marked executable on a linux partition. This flag should be replicated. efficient sync strategy: some of my files are 20GB, they are changed quite often, but in very little parts (Virtualbox images). Delta transmissions are welcome. efficient handling of space: no history for files, files shouldn't be copied to temp directories "just in case you break it". it must propagate deletions of files modification can happen in any of the 4 pcs, they should be propagated when other pc are connected. Other specs of my solution are: Sync is over a lan, the total amount of data to be synced is around 180GB, in some ten thousand files. Changes are small, but can happen in big files. At the moment i'm interested in a linux only solution. conflicts either don't happen or are solved with "last one wins" I haven't found any good solution. I've been trying: unison: it is the only one working at the moment, but during the hashing phase it hangs my pc for some minute, disk light steady on. Sparkleshare doesn't handle large files nicely. It keeps an history of all your changes that grows up indefinitely. They promise it will be fixed in next releases, but at the moment it still doesn't fit my needs. owncloud (keeps history of each file i change) coda ? (help! i couldn't set it up correctly!) git-annex assistant transforms all your files in symlinks and mark the original file as read only ("just in case you make a mistake while you modify it"!). Before you edit a file you have to issue a special command "git-annex unlock", that creates a local copy of the file, and you have to remember to lock it again if you want it synchronized. What to try next?

    Read the article

  • Limiting bandwith on an Windows 7 machine

    - by Mihai Damian
    I need to limit the bandwidth on my Windows 7 x64 machine. In the past (on XP) I've been able to use NetLimiter for similar tasks. However for some reason I can't get it to work anymore. For lower limits the bandwidth tests are able to exceed the limit by 10-50%; higher limits seem to be ignored completely and the bandwidth tests report download speeds of over 10 times the speed I set. I'm using speedtest.net and some similar service from my ISP for these tests. Anyway, I don't necessarily need a program as complex as NetLimiter since I only need to throttle my machine's bandwidth, not a specific program's. In case you are wondering why in the world I'd want to cripple my Internet speed, there is a funny story behind this. Long story short, my modem gets random disconnects. Tech support comes in, says my Internet speed is abnormally high and I must be using some tools to somehow make it go faster than it's supposed to and this messes up my modem. I check the connection with another computer and it seems that my PC is the only one in my network that gets abnormal speeds. I reinstall my OS, speed looks normal at first, after I install the batch of 50 or so updates, it goes back to abnormally high speeds and the disconnect problems are not solved. Now I don't have a clue if the explanation the tech team gave me was just a strategy to lay the blame on someone else, but I was trying to give them the benefit of the doubt and see what happens if I really reduce my speed to their specification. Any help appreciated.

    Read the article

  • toshiba a200 laptop hangs/freezes when plugged in

    - by tapan
    The subject says it all. It is the exact same problem as described here. I had asked a similar question before here. My system specs etc, everything is here. My problem is just that I cannot afford to get it fixed right now. I just need for the laptop to be kind of usable for 2-3 more months. It worked perfectly for 3 months or so in ubuntu recovery mode. Now it hangs/freezes every hour. I spend 8 hours a day on an average on my laptop. So i just want it to last at least that long. If the laptop is not plugged in in recovery mode / regular ubuntu mode, it freezes in 5 mins and the screen becomes all weird. With my windows 7 OS, it works perfectly on battery but hangs instantly as soon as it is plugged in. Also now, a high cpu usage doesn't help anymore. Any strategy or suggestions i could employ to extend the duration for which it doesn't freeze ?

    Read the article

  • virtual machines, dual booting and data disks on SSD

    - by stevemarvell
    This is in planning, so if I've got the strategy wrong, please let me know. There are multiple questions here, but I think they all degenerate to the same answers. The hardware is a laptop with a single SSD. I'm trying to not lose the performance of the SSD. I plan a native dual booting Windows (plus cygwin) and Linux machine which is my BYOD and represents the development environment. I keep the codebase on a shared partition (though sometimes this is an external thunderbolt SSD) which can be natively "mounted" by whichever OS is in operation. I boot into one or the other environments depending on the task in hand. Sometime I have to develop with windows tools, but generally, Linux is my preferred development environment. It would be ideal if I could VM the other OS and run either in either. I'm going to assume, because I've not found a sensible VM based solution, that I have get samba involved to share the code partition between VMs. Is this going to blow my SSD performance in the VM? The client also supplies me with a VM for the target environment, usually linux. This is not often suited to development and is used for testing only. I normally keep two copies of this, one as a sandbox and one which I deploy to using the client's preferred method. I keep these VM snapshots on the shared partition. The latter is interacted with over the network and so has no disk sharing requirements. However, it would be useful for the sandbox to be able to "mount" the code base from the natively running OS. Is this samba or nfs again, depending on the native OS? Am I missing a trick which allows this to all work smoothly with all four environments running at once without loosing the SSD performance?

    Read the article

  • How do I permanently delete /var/log/lastlog?

    - by GregB
    My /var/log/lastlog file is huge. I know it's really only a few kilobytes, but tar isn't smart enough to know that, so when I image a virtual machine, my restore fails because it thinks I'm trying to load more data than I have capacity on my disk. I want to delete /var/log/lastlog and stop any and all logging to the file. I'm aware of the security implications. This logging needs to stop to preserve my backup strategy. I've made a change to /etc/pam.d/login which I was told would disable logging to /var/log/lastlog, but it does not appear to work as /var/log/lastlog keeps growing. # Prints the last login info upon succesful login # (Replaces the `LASTLOG_ENAB' option from login.defs) #session optional pam_lastlog.so Any ideas? EDIT For anyone interested, I use Centrify Express to authenticate my users via LDAP. Centrify Express is "free", but one of the drawbacks is that I can't manage user UIDs via LDAP, so they are given a dynamic UID when they login to a server. Centrify picks some crazy high UID values (so they don't conflict with local users on the server, presumably). /var/log/lastlog is indexed by UID, and grows to accommodate the largest UID on the system. This means that when a Centrify user logs in, they get a UID in the upper-end of the UID range, which causes lastlog to allocate an obscene amount of space, according to the file system. ~$ ll /var/log/lastlog -rw-rw-r-- 1 root root 291487675780 Apr 10 16:37 /var/log/lastlog ~$ du -h /var/log/lastlog 20K /var/log/lastlog More Into --- Sparse Files

    Read the article

  • LXC, Port forwarding and iptables

    - by Roberto Aloi
    I have a LXC container (10.0.3.2) running on a host. A service is running inside the container on port 7000. From the host (10.0.3.1, lxcbr0), I can reach the service: $ telnet 10.0.3.2 7000 Trying 10.0.3.2... Connected to 10.0.3.2. Escape character is '^]'. I'd love to make the service running inside the container accessible to the outer world. Therefore, I want to forward port 7002 on the host to port 7000 on the container: iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000 Which results in (iptables -t nat -L): DNAT tcp -- anywhere anywhere tcp dpt:afs3-prserver to:10.0.3.2:7000 Still, I cannot access the service from the host using the forwarded port: $ telnet 10.0.3.1 7002 Trying 10.0.3.1... telnet: Unable to connect to remote host: Connection refused I feel like I'm missing something stupid here. What things should I check? What's a good strategy to debug these situations? For completeness, here is how iptables are set on the host: iptables -F iptables -F -t nat iptables -F -t mangle iptables -X iptables -P INPUT DROP iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT iptables -A INPUT -p tcp --dport 22 -j ACCEPT iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE iptables -t nat -A POSTROUTING -o lxcbr0 -j MASQUERADE iptables -t nat -A PREROUTING -p tcp --dport 7002 -j DNAT --to 10.0.3.2:7000

    Read the article

  • Ubuntu 12.04 partioning an external drive without lossing data

    - by Menelaos Perdikeas
    I have an Ubuntu 12.04 with an external 1.5T disk (just for data). It is /dev/sdc1 seen below: $ df -T Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/sda1 ext4 1451144932 27722584 1350794536 3% / udev devtmpfs 6199460 4 6199456 1% /dev tmpfs tmpfs 2482692 988 2481704 1% /run none tmpfs 5120 0 5120 0% /run/lock none tmpfs 6206724 284 6206440 1% /run/shm /dev/sdc1 fuseblk 1465135100 172507664 1292627436 12% /media/Elements The thing is I would like to implement this rsync-based backup strategy and I want to use my /dev/sdc1 external drive for that. Since the guide mentioned above recommends placing the backup directory in a separate partition I want to repartition the /dev/sdc1 external hard disk but retain existing data in a separate partition. E.g. split /dev/sdc1 into two partitions: (i) one to be used exclusively for the rsync-based backup and (ii) the other for the existing miscellaneous data. How should I go about partitioning with minimal risk to my existing data and what kind of filesystem do you recommend? I would prefer a console-based guide but unfortunately all the material I found on the web is oriented towards partitioning the main (bootable) disk and not an external fuseblk filesystem used only for passive data.

    Read the article

  • saving data from a failing drive

    - by intuited
    An external 3½" HDD seems to be in danger of failing — it's making ticking sounds when idle. I've acquired a replacement drive, and want to know the best strategy to get the data off of the dubious drive with the best chance of saving as much as possible. There are some directories that are more important than others. However, I'm guessing that picking and choosing directories is going to reduce my chances of saving the whole thing. I would also have to mount it, dump a file listing, and then unmount it in order to be able to effectively prioritize directories. Adding in the fact that it's time-consuming to do this, I'm leaning away from this approach. I've considered just using dd, but I'm not sure how it would handle read errors or other problems that might prevent only certain parts of the data from being rescued, or which could be overcome with some retries, but not so many that they endanger other parts of the drive from being saved. I guess ideally it would do a single pass to get as much as possible and then go back to retry anything that was missed due to errors. Is it possible that copying more slowly — e.g. pausing every x MB/GB — would be better than just running the operation full tilt, for example to avoid any overheating issues? For the "where is your backup" crowd: this actually is my backup drive, but it also contains some non-critical and bulky stuff, like music, that aren't backups, i.e. aren't backed up. The drive has not exhibited any clear signs of failure other than this somewhat ominous sound. I did have to fsck a few errors recently — orphaned inodes, incorrect free blocks/inodes counts, inode bitmap differences, zero dtime on deleted inodes; about 20 errors in all. The filesystem of the partition is ext3.

    Read the article

  • Windows 7 doesn't start anymore

    - by 0xFF
    I've experienced some BSOD on windows 7 RC, and some freezing when startup, but today was big surprise, it doesn't start anymore. I tried to start on safe mode and no results too, it shows the starting animation, then a blue screen for less than a second and turns off immediately. The only thing I remember did today is update flash player under Firefox, then chrome stopped working even after logging off, and once restarting, it doesn't start anymore. Anyone experienced the same issue? any hints? [EDIT 3] Solved : Windows 7 have a very smart repair strategy, it works automatically, and it tried every possible fix, what fixed my problem was the system restore to a previous date, all this happened automatically. [EDIT2] these are the last lines in the ntbtlog.txt file Did not load driver \SystemRoot\System32\drivers\vga.sys Loaded driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Did not load driver \SystemRoot\System32\Drivers\NDProxy.SYS Loaded driver \SystemRoot\system32\drivers\CHDRT32.sys Loaded driver \SystemRoot\system32\DRIVERS\VSTAZL3.SYS Loaded driver \SystemRoot\system32\DRIVERS\VSTDPV3.SYS Loaded driver \SystemRoot\system32\DRIVERS\VSTCNXT3.SYS Loaded driver \SystemRoot\system32\drivers\modem.sys Loaded driver \SystemRoot\system32\DRIVERS\usbccgp.sys Loaded driver \SystemRoot\System32\Drivers\usbvideo.sys [edit] this is the BSOD I get : http://twitpic.com/i87cx Thank you.

    Read the article

  • thought about shared storage (NFS, Lustre) [closed]

    - by user134880
    Possible Duplicate: Can you help me with my capacity planning? Now I habe small cluster with total of 8 nodes. 6 of them are computing nodes (apache and vmware) and 2 nodes are for storage. 2 storage nodes are identical. Each storage server is linux box with 8 x 1Tb WD RE4 in soft raid 10. 1st box is master and 2nd is slave. Data is mirrored with DRDB. We export NFSv4 shares to Apache (for document root) and iSCSI to Vmware. Now all is working pretty good and stable. But it will be soon time to upgrade our system. I have been thinking of Lustre. Does some one has any real experience with Lustre or NFS medium clusters? Will it be good idea just to upgrade server and change hdd's to 3Tb ? With NFS we will always have only 2 servers to maintain (one primary and one slave). Thanks. QUESTIONS: 1) Does some one used Lustre? In production? I have seen a lot of info about how it is hard to setup Lustre because you need to compile own kernel and patches. It's answers from newbies. Is there some one who has used Lustre for some period of time? 2) About disk upgrades - it's only description of strategy. I'm not asking if it is enough 3Tb or not. I just ask if it is right just to replace hdds instead of adding new server (like with Lustre) Thanks again.

    Read the article

  • Service redirection on same network

    - by Unode
    I have a network on which I run multiple servers each dedicated to a given service. Because most services run on distinct ports I'm currently looking for a way of unifying "all" services into a single "proxy" machine. The idea is to abstract which machine is being accessed but still allow direct connection if needed/requested. This "proxy" machine has only one network interface which is part of the same network as all the other service providing machines. I've looked into Routing and NAT but I've so far failed to figure out how to make it work. I tried to achieve this using shorewall but couldn't find clear examples. However I'm not entirely sure this is the best/simplest strategy. With that said, what would be the best way of achieving this result? Example case: Proxy IP - Listening port - Send requests to 192.168.0.50 80 192.168.0.1:80 " 22 192.168.0.2:2222 " 3306 192.168.0.3:3000 " 5432 192.168.0.4:5432 " 5222 192.168.0.5:5222 PS: I'm not concerned with the single-point-of-failure nature of the proxy. Thanks

    Read the article

  • What other ways can I load balance EC2 servers without using Elastic Load Balancing?

    - by undefined
    I have a web application that consists of a web server managed by a web hosting firm, a set of EC2 instances in amazons cloud and a MySQL database (hosted on the webserver). MySQL is behind a firewall and is set to allow access from Localhost and from a single IP address which is an Amazon Elastic IP address that is attached to the EC2 instance I have been running up to now. The problem is that I want to look at my scaling up and load balancing strategy for my EC2 instance. To this end I have been investigating the Elastic Load Balancers and Autoscaling tools that Amazon provides and have managed to set this up fine but for one thing - connecting to the MySQL database running on my webserver. I realised (thanks to answers on Serverfault) that I needed to check firewall settings and add the IP address for the load balancer, however Elastic Load Balancers provide you with a DNS name, not an IP address and infact the IP addresses change over time so this will not work. I have been told by the company hosting the database that the way the firewall works is to look up the IP address of the DNS name and store the IP rather than the DNS name. so basically this will not work and the only way to allow access would be to open up the SQL port to allow access from anyone! Is this a viable idea? Should I look at moving my database into the cloud? Is there another firewall that the server company can use? Should I find another way of load balancing (if so what?) tricky one eh? any help appreciated!

    Read the article

  • How to diagnose a spontaneous reboot?

    - by Spectralist
    My computer reboots, seemingly completely at random, about once every week to two weeks but has occasionally gone months. It just goes from running fine to the POST with no error messages or anything and doesn't seem to be due to heat or usage as it's happened a couple of times when the computer has booted just a few moments ago and is idling. It's been happening for as long as I've had this computer, almost two years. It's happened in both Vista and Windows7. I strongly suspect it's a hardware problem. But due to the rareish and random nature of the crashes my normal strategy of just removing hardware until the problem stops isn't really practical. My guess would be Power Supply, Ram, or Motherboard. But I just don't know how to test an issue this random and want to figure out how to confirm which it is before I go replacing things. So is there some software or hardware that can be used to test these sorts of errors? I did run memtest86 for about 8 hours without finding any issues. And the power supply is more than capable of running my system.

    Read the article

  • Mysql InnoDB and quickly applying large updates

    - by Tim
    Basically my problem is that I have a large table of about 17,000,000 products that I need to apply a bunch of updates to really quickly. The table has 30 columns with the id set as int(10) AUTO_INCREMENT. I have another table which all of the updates for this table are stored in, these updates have to be pre-calculated as they take a couple of days to calculate. This table is in the format of [ product_id int(10), update_value int(10) ]. The strategy I'm taking to issue these 17 million updates quickly is to load all of these updates into memory in a ruby script and group them in a hash of arrays so that each update_value is a key and each array is a list of sorted product_id's. { 150: => [1,2,3,4,5,6], 160: => [7,8,9,10] } Updates are then issued in the format of UPDATE product SET update_value = 150 WHERE product_id IN (1,2,3,4,5,6); UPDATE product SET update_value = 160 WHERE product_id IN (7,8,9,10); I'm pretty sure I'm doing this correctly in the sense that issuing the updates on sorted batches of product_id's should be the optimal way to do it with mysql / innodb. I'm hitting a weird issue though where when I was testing with updating ~13 million records, this only took around 45 minutes. Now I'm testing with more data, ~17 million records and the updates are taking closer to 120 minutes. I would have expected some sort of speed decrease here but not to the degree that I'm seeing. Any advice on how I can speed this up or what could be slowing me down with this larger record set? As far as server specs go they're pretty good, heaps of memory / cpu, the whole DB should fit into memory with plenty of room to grow.

    Read the article

  • I am getting an error with a oneToMany association when using annotations with gilead for hibernate

    - by user286630
    Hello Guys I'm using Gilead to persist my entities in my GWT project, im using hibernate annotations aswell. my problem is on my onetomany association.this is my User class that holds a reference to a list of FileLocations @Entity @Table(name = "yf_user_table") public class YFUser implements Serializable { @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "user_id",nullable = false) private int userId; @Column(name = "username") private String username; @Column(name = "password") private String password; @Column(name = "email") private String email; @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY) @JoinColumn(name="USER_ID") private List fileLocations = new ArrayList(); This is my file location class @Entity @Table(name = "fileLocationTable") public class FileLocation implements Serializable { @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "locationId", updatable = false, nullable = false) private int ieId; @Column (name = "location") private String location; @ManyToOne(fetch=FetchType.LAZY) @JoinColumn(name="USER_ID", nullable=false) private YFUser uploadedUser; When i persist this data in a normal desktop application, it works fine , creates the tables and i can add and store data to it. but when i try to persist the data in my gwt application i get errors i will show them lower. this is my ServiceImpl class that extends PersistentRemoteService. public class TestServiceImpl extends PersistentRemoteService implements TestService { public static final String SESSION_USER = "UserWithinSession"; public TestServiceImpl(){ HibernateUtil util = new HibernateUtil(); util.setSessionFactory(com.example.server.HibernateUtil .getSessionFactory()); PersistentBeanManager pbm = new PersistentBeanManager(); pbm.setPersistenceUtil(util); pbm.setProxyStore(new StatelessProxyStore()); setBeanManager(pbm); } @Override public String registerUser(String username, String password, String email) { Session session = com.example.server.HibernateUtil.getSessionFactory().getCurrentSession(); session.beginTransaction(); YFUser newUser = new YFUser(); newUser.setUsername(username);newUser.setPassword(password);newUser.setEmail(email); session.save(newUser); session.getTransaction().commit(); return "Thank you For registering "+ username; } this is the error that im am getting. the error goes away when i remove my onetoManyRelationship and builds my session factory when i put it in , it on the line of buildsessionfactory in hibernate Util that it throws this exception. my hibernate util class is ok also. this is the error java.lang.NoSuchMethodError: javax.persistence.OneToMany.orphanRemoval()Z Mar 24, 2010 10:03:22 PM org.hibernate.cfg.annotations.Version <clinit> INFO: Hibernate Annotations 3.5.0-Beta-4 Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Environment <clinit> INFO: Hibernate 3.5.0-Beta-4 Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Environment <clinit> INFO: hibernate.properties not found Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Environment buildBytecodeProvider INFO: Bytecode provider name : javassist Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Environment <clinit> INFO: using JDK 1.4 java.sql.Timestamp handling Mar 24, 2010 10:03:22 PM org.hibernate.annotations.common.Version <clinit> INFO: Hibernate Commons Annotations 3.2.0-SNAPSHOT Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Configuration configure INFO: configuring from resource: /hibernate.cfg.xml Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Configuration getConfigurationInputStream INFO: Configuration resource: /hibernate.cfg.xml Mar 24, 2010 10:03:22 PM org.hibernate.cfg.Configuration doConfigure INFO: Configured SessionFactory: null Mar 24, 2010 10:03:22 PM org.hibernate.cfg.search.HibernateSearchEventListenerRegister enableHibernateSearch INFO: Unable to find org.hibernate.search.event.FullTextIndexEventListener on the classpath. Hibernate Search is not enabled. Mar 24, 2010 10:03:22 PM org.hibernate.cfg.AnnotationBinder bindClass INFO: Binding entity from annotated class: com.example.client.Entity1 Mar 24, 2010 10:03:22 PM org.hibernate.cfg.annotations.EntityBinder bindTable INFO: Bind entity com.example.client.Entity1 on table entityTable1 Mar 24, 2010 10:03:22 PM org.hibernate.cfg.AnnotationBinder bindClass INFO: Binding entity from annotated class: com.example.client.YFUser Mar 24, 2010 10:03:22 PM org.hibernate.cfg.annotations.EntityBinder bindTable INFO: Bind entity com.example.client.YFUser on table yf_user_table Initial SessionFactory creation failed.java.lang.NoSuchMethodError: javax.persistence.OneToMany.orphanRemoval()Z [WARN] Nested in javax.servlet.ServletException: init: java.lang.ExceptionInInitializerError at com.example.server.HibernateUtil.<clinit>(HibernateUtil.java:38) at com.example.server.TestServiceImpl.<init>(TestServiceImpl.java:29) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39 ) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308) at org.mortbay.jetty.servlet.Holder.newInstance(Holder.java:153) at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:339) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:463) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:647) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488) Caused by: java.lang.NoSuchMethodError: javax.persistence.OneToMany.orphanRemoval()Z at org.hibernate.cfg.AnnotationBinder.processElementAnnotations(AnnotationBinder.java:1642) at org.hibernate.cfg.AnnotationBinder.bindClass(AnnotationBinder.java:772) at org.hibernate.cfg.AnnotationConfiguration.processArtifactsOfType(AnnotationConfiguration.java:629) at org.hibernate.cfg.AnnotationConfiguration.secondPassCompile(AnnotationConfiguration.java:350) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1373) at org.hibernate.cfg.AnnotationConfiguration.buildSessionFactory(AnnotationConfiguration.java:973) at com.example.server.HibernateUtil.<clinit>(HibernateUtil.java:33) ... 26 more [WARN] Nested in java.lang.ExceptionInInitializerError: java.lang.NoSuchMethodError: javax.persistence.OneToMany.orphanRemoval()Z at org.hibernate.cfg.AnnotationBinder.processElementAnnotations(AnnotationBinder.java:1642) at org.hibernate.cfg.AnnotationBinder.bindClass(AnnotationBinder.java:772) at org.hibernate.cfg.AnnotationConfiguration.processArtifactsOfType(AnnotationConfiguration.java:629) at org.hibernate.cfg.AnnotationConfiguration.secondPassCompile(AnnotationConfiguration.java:350) at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1373) at org.hibernate.cfg.AnnotationConfiguration.buildSessionFactory(AnnotationConfiguration.java:973) at com.example.server.HibernateUtil.<clinit>(HibernateUtil.java:33) at com.example.server.TestServiceImpl.<init>(TestServiceImpl.java:29) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308) at org.mortbay.jetty.servlet.Holder.newInstance(Holder.java:153) at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:339) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:463) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:647) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

    Read the article

  • RESTful services architecture question

    - by abovesun
    This is question more about service architecture strategy, we are building big web system based on rest services on back end. And we are currently trying to build some standard internal to follow while developing rest services. Some queries returns list of entities, for example lets consider we have image galleries retrieving service: /gell_all_galeries, returning next response: <galleries> <gallery> <id>some_gallery_id</id> <name>my photos</name> <photos> <photo> <id>123</id> <name>my photo</name> <location>http://mysite/photo/show/123</location> ...... <author> <id>some_id</id> <name>some name</name> ....... <author> </photo> <photo> ..... </photo> <photo> ..... </photo> <photo> ..... </photo> <photo> ..... </photo> </photos> </gallery> <gallery> .... </gallery> <gallery> .... </gallery> <gallery> .... </gallery> <gallery> .... </gallery> </galleries> As you see here, response quite big and heavy, and not always we need such deep info level. Usual solution is to use or http://ru.wikipedia.org/wiki/Atom elements for each gallery instead of full gallery data: <galleries> <gallery> <id>some_gallery_id</id> <link href="http://mysite/gallery/some_gallery_id"/> </gallery> <gallery> <id>second_gallery_id</id> <link href="http://mysite/gallery/second_gallery_id"/> </gallery> <gallery> .... </gallery> <gallery> .... </gallery> <gallery> .... </gallery> <gallery> .... </gallery> </galleries> The first question, is next: maybe instead we shouldn't even use and types, and just use generic and for all resources that return list objects: <list> <item><link href="http://mysite/gallery/some_gallery_id"/></item> <item><link href="http://mysite/gallery/other_gallery_id"/></item> <item>....</item> </list> And the second question, after user try to retrieve info about some concrete gallery, he'll use for example http://mysite/gallery/some_gallery_id link, what should he see as results? Should it be: <gallery> <id>some_gallery_id</id> <name>my photos</name> <photos> <photo> <id>123</id> <name>my photo</name> <location>http://mysite/photo/show/123</location> ...... <author> <id>some_id</id> <name>some name</name> ....... <author> </photo> <photo> ..... </photo> <photo> ..... </photo> <photo> ..... </photo> <photo> ..... </photo> </photos> </gallery> or : <gallery> <id>some_gallery_id</id> <name>my photos</name> <photos> <photo><link href="http://mysite/photo/11111"/></photo> <photo><link href="http://mysite/photo/22222"/></photo> <photo><link href="http://mysite/photo/33333"/> </photo> <photo> ..... </photo> </photos> </gallery> or <gallery> <id>some_gallery_id</id> <name>my photos</name> <photos> <photo> <link href="http://mysite/photo/11111"/> <author> <link href="http://mysite/author/11111"/> </author> </photo> <photo> <link href="http://mysite/photo/22222"/> <author> <link href="http://mysite/author/11111"/> </author> </photo> <photo> <link href="http://mysite/photo/33333"/> <author> <link href="http://mysite/author/11111"/> </author> </photo> <photo> ..... </photo> </photos> </gallery> I mean if we use link instead of full object info, how deep we should go there? Should I show an author inside photo and so on. Probably my question ambiguous, but what I'm trying to do is create general strategy in such cases for all team members to follow in future.

    Read the article

  • .LazyInitializationException when adding to a list that is held within a entity class using hibernat

    - by molleman
    Right so i am working with hibernate gilead and gwt to persist my data on users and files of a website. my users have a list of file locations. i am using annotations to map my classes to the database. i am getting a org.hibernate.LazyInitializationException when i try to add file locations to the list that is held in the user class. this is a method below that is overridden from a external file upload servlet class that i am using. when the file uploads it calls this method. the user1 is loaded from the database elsewhere. the exception occurs at user1.getFileLocations().add(fileLocation); . i dont understand it really at all.! any help would be great. the stack trace of the error is below public String executeAction(HttpServletRequest request, List<FileItem> sessionFiles) throws UploadActionException { for (FileItem item : sessionFiles) { if (false == item.isFormField()) { try { YFUser user1 = (YFUser)getSession().getAttribute(SESSION_USER); // This is the location where a file will be stored String fileLocationString = "/Users/Stefano/Desktop/UploadedFiles/" + user1.getUsername(); File fl = new File(fileLocationString); fl.mkdir(); // so here i will create the a file container for my uploaded file File file = File.createTempFile("upload-", ".bin",fl); // this is where the file is written to disk item.write(file); // the FileLocation object is then created FileLocation fileLocation = new FileLocation(); fileLocation.setLocation(fileLocationString); //test System.out.println("file path = "+file.getPath()); user1.getFileLocations().add(fileLocation); //the line above is where the exception occurs } catch (Exception e) { throw new UploadActionException(e.getMessage()); } } removeSessionFileItems(request); } return null; } //This is the class file for a Your Files User @Entity @Table(name = "yf_user_table") public class YFUser implements Serializable,ILightEntity { @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "user_id",nullable = false) private int userId; @Column(name = "username") private String username; @Column(name = "password") private String password; @Column(name = "email") private String email; @ManyToMany(cascade = CascadeType.ALL) @JoinTable(name = "USER_FILELOCATION", joinColumns = { @JoinColumn(name = "user_id") }, inverseJoinColumns = { @JoinColumn(name = "locationId") }) private List<FileLocation> fileLocations = new ArrayList<FileLocation>() ; public YFUser(){ } public int getUserId() { return userId; } private void setUserId(int userId) { this.userId = userId; } public String getUsername() { return username; } public void setUsername(String username) { this.username = username; } public String getPassword() { return password; } public void setPassword(String password) { this.password = password; } public String getEmail() { return email; } public void setEmail(String email) { this.email = email; } public List<FileLocation> getFileLocations() { if(fileLocations ==null){ fileLocations = new ArrayList<FileLocation>(); } return fileLocations; } public void setFileLocations(List<FileLocation> fileLocations) { this.fileLocations = fileLocations; } /* public void addFileLocation(FileLocation location){ fileLocations.add(location); }*/ @Override public void addProxyInformation(String property, Object proxyInfo) { // TODO Auto-generated method stub } @Override public String getDebugString() { // TODO Auto-generated method stub return null; } @Override public Object getProxyInformation(String property) { // TODO Auto-generated method stub return null; } @Override public boolean isInitialized(String property) { // TODO Auto-generated method stub return false; } @Override public void removeProxyInformation(String property) { // TODO Auto-generated method stub } @Override public void setInitialized(String property, boolean initialised) { // TODO Auto-generated method stub } @Override public Object getValue() { // TODO Auto-generated method stub return null; } } @Entity @Table(name = "fileLocationTable") public class FileLocation implements Serializable { @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column(name = "locationId", updatable = false, nullable = false) private int ieId; @Column (name = "location") private String location; /* private List uploadedUsers = new ArrayList(); */ public FileLocation(){ } public int getIeId() { return ieId; } private void setIeId(int ieId) { this.ieId = ieId; } public String getLocation() { return location; } public void setLocation(String location) { this.location = location; } /* public List getUploadedUsers() { return uploadedUsers; } public void setUploadedUsers(List<YFUser> uploadedUsers) { this.uploadedUsers = uploadedUsers; } public void addUploadedUser(YFUser user){ uploadedUsers.add(user); } */ } Apr 2, 2010 11:33:12 PM org.hibernate.LazyInitializationException <init> SEVERE: failed to lazily initialize a collection of role: com.example.client.YFUser.fileLocations, no session or session was closed org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.client.YFUser.fileLocations, no session or session was closed at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:380) at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:372) at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:365) at org.hibernate.collection.AbstractPersistentCollection.write(AbstractPersistentCollection.java:205) at org.hibernate.collection.PersistentBag.add(PersistentBag.java:297) at com.example.server.TestServiceImpl.saveFileLocation(TestServiceImpl.java:132) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at net.sf.gilead.gwt.PersistentRemoteService.processCall(PersistentRemoteService.java:174) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:224) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:729) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.RequestLogHandler.handle(RequestLogHandler.java:49) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:505) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:843) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:647) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:380) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488) Apr 2, 2010 11:33:12 PM net.sf.gilead.core.PersistentBeanManager clonePojo INFO: Third party instance, not cloned : org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.example.client.YFUser.fileLocations, no session or session was closed

    Read the article

  • selectOneMenu - java.lang.NullPointerException when adding record to the database (JSF2 and JPA2-OpenJPA)

    - by rogie
    Good day to all; I'm developing a program using JSF2 and JPA2 (OpenJPA). Im also using IBM Rapid App Dev't v8 with WebSphere App Server v8 test server. I have two simple entities, Employee and Department. Each Department has many Employees and each Employee belongs to a Department (using deptno and workdept). My problem occurs when i tried to add a new employee and selecting a department from a combo box (using selectOneMenu - populated from Department table): when i run the program, the following error messages appeared: An Error Occurred: java.lang.NullPointerException Caused by: java.lang.NullPointerException - java.lang.NullPointerException I also tried to make another program using Deptno and Workdept as String instead of integer, still doesn't work. Pls help. Im also a newbie. Tnx and God bless. Below are my codes, configurations and setup. Just tell me if there are some codes that I forgot to include. Im also using Derby v10.5 as my database: CREATE SCHEMA RTS; CREATE TABLE RTS.DEPARTMENT (DEPTNO INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1), DEPTNAME VARCHAR(30)); ALTER TABLE RTS.DEPARTMENT ADD CONSTRAINT PK_DEPARTMNET PRIMARY KEY (DEPTNO); CREATE TABLE RTS.EMPLOYEE (EMPNO INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1), NAME VARCHAR(50), WORKDEPT INTEGER); ALTER TABLE RTS.EMPLOYEE ADD CONSTRAINT PK_EMPLOYEE PRIMARY KEY (EMPNO); Employee and Department Entities package rts.entities; import java.io.Serializable; import javax.persistence.*; @Entity public class Employee implements Serializable { private static final long serialVersionUID = 1L; @Id @GeneratedValue(strategy=GenerationType.IDENTITY) private int empno; private String name; //bi-directional many-to-one association to Department @ManyToOne @JoinColumn(name="WORKDEPT") private Department department; ....... getter and setter methods package rts.entities; import java.io.Serializable; import javax.persistence.*; import java.util.List; @Entity public class Department implements Serializable { private static final long serialVersionUID = 1L; @Id @GeneratedValue(strategy=GenerationType.IDENTITY) private int deptno; private String deptname; //bi-directional many-to-one association to Employee @OneToMany(mappedBy="department") private List<Employee> employees; ....... getter and setter methods JSF 2 snipet using combo box (populated from Department table) <tr> <td align="left">Department</td> <td style="width: 5px">&#160;</td> <td><h:selectOneMenu styleClass="selectOneMenu" id="department1" value="#{pc_EmployeeAdd.employee.department}"> <f:selectItems value="#{DepartmentManager.departmentSelectList}" id="selectItems1"></f:selectItems> </h:selectOneMenu></td> </tr> package rts.entities.controller; import com.ibm.jpa.web.JPAManager; import javax.persistence.EntityManager; import javax.persistence.EntityManagerFactory; import com.ibm.jpa.web.NamedQueryTarget; import com.ibm.jpa.web.Action; import javax.persistence.PersistenceUnit; import javax.annotation.Resource; import javax.transaction.UserTransaction; import rts.entities.Department; import java.util.List; import javax.persistence.Query; import java.util.ArrayList; import java.text.MessageFormat; import javax.faces.model.SelectItem; @SuppressWarnings("unchecked") @JPAManager(targetEntity = rts.entities.Department.class) public class DepartmentManager { ....... public List<SelectItem> getDepartmentSelectList() { List<Department> departmentList = getDepartment(); List<SelectItem> selectList = new ArrayList<SelectItem>(); MessageFormat mf = new MessageFormat("{0}"); for (Department department : departmentList) { selectList.add(new SelectItem(department, mf.format( new Object[] { department.getDeptname() }, new StringBuffer(), null).toString())); } return selectList; } Converter: package rts.entities.converter; import javax.faces.component.UIComponent; import javax.faces.context.FacesContext; import javax.faces.convert.Converter; import rts.entities.Department; import rts.entities.controller.DepartmentManager; import com.ibm.jpa.web.TypeCoercionUtility; public class DepartmentConverter implements Converter { public Object getAsObject(FacesContext facesContext, UIComponent arg1, String entityId) { DepartmentManager departmentManager = (DepartmentManager) facesContext .getApplication().createValueBinding("#{DepartmentManager}") .getValue(facesContext); int deptno = (Integer) TypeCoercionUtility.coerceType("int", entityId); Department result = departmentManager.findDepartmentByDeptno(deptno); return result; } public String getAsString(FacesContext arg0, UIComponent arg1, Object object) { if (object instanceof Department) { return "" + ((Department) object).getDeptno(); } else { throw new IllegalArgumentException("Invalid object type:" + object.getClass().getName()); } } } Method for Add button: public String createEmployeeAction() { EmployeeManager employeeManager = (EmployeeManager) getManagedBean("EmployeeManager"); try { employeeManager.createEmployee(employee); } catch (Exception e) { logException(e); } return ""; } faces-conf.xml <converter> <converter-for-class>rts.entities.Department</converter-for-class> <converter-class>rts.entities.converter.DepartmentConverter</converter-class> </converter> Stack trace javax.faces.FacesException: java.lang.NullPointerException at org.apache.myfaces.shared_impl.context.ExceptionHandlerImpl.wrap(ExceptionHandlerImpl.java:241) at org.apache.myfaces.shared_impl.context.ExceptionHandlerImpl.handle(ExceptionHandlerImpl.java:156) at org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:258) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:191) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1147) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:722) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:449) at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1020) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:87) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:886) at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1655) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:195) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:511) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:305) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:276) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214) at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1650) Caused by: java.lang.NullPointerException at rts.entities.converter.DepartmentConverter.getAsString(DepartmentConverter.java:29) at org.apache.myfaces.shared_impl.renderkit.RendererUtils.getConvertedStringValue(RendererUtils.java:656) at org.apache.myfaces.shared_impl.renderkit.html.HtmlRendererUtils.getSubmittedOrSelectedValuesAsSet(HtmlRendererUtils.java:444) at org.apache.myfaces.shared_impl.renderkit.html.HtmlRendererUtils.internalRenderSelect(HtmlRendererUtils.java:421) at org.apache.myfaces.shared_impl.renderkit.html.HtmlRendererUtils.renderMenu(HtmlRendererUtils.java:359) at org.apache.myfaces.shared_impl.renderkit.html.HtmlMenuRendererBase.encodeEnd(HtmlMenuRendererBase.java:76) at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:519) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:626) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:622) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:622) at javax.faces.component.UIComponent.encodeAll(UIComponent.java:622) at org.apache.myfaces.view.facelets.FaceletViewDeclarationLanguage.renderView(FaceletViewDeclarationLanguage.java:1320) at org.apache.myfaces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:263) at org.apache.myfaces.lifecycle.RenderResponseExecutor.execute(RenderResponseExecutor.java:85) at org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:239) ... 24 more

    Read the article

  • F# - Facebook Hacker Cup - Double Squares

    - by Jacob
    I'm working on strengthening my F#-fu and decided to tackle the Facebook Hacker Cup Double Squares problem. I'm having some problems with the run-time and was wondering if anyone could help me figure out why it is so much slower than my C# equivalent. There's a good description from another post; Source: Facebook Hacker Cup Qualification Round 2011 A double-square number is an integer X which can be expressed as the sum of two perfect squares. For example, 10 is a double-square because 10 = 3^2 + 1^2. Given X, how can we determine the number of ways in which it can be written as the sum of two squares? For example, 10 can only be written as 3^2 + 1^2 (we don't count 1^2 + 3^2 as being different). On the other hand, 25 can be written as 5^2 + 0^2 or as 4^2 + 3^2. You need to solve this problem for 0 = X = 2,147,483,647. Examples: 10 = 1 25 = 2 3 = 0 0 = 1 1 = 1 My basic strategy (which I'm open to critique on) is to; Create a dictionary (for memoize) of the input numbers initialzed to 0 Get the largest number (LN) and pass it to count/memo function Get the LN square root as int Calculate squares for all numbers 0 to LN and store in dict Sum squares for non repeat combinations of numbers from 0 to LN If sum is in memo dict, add 1 to memo Finally, output the counts of the original numbers. Here is the F# code (See code changes at bottom) I've written that I believe corresponds to this strategy (Runtime: ~8:10); open System open System.Collections.Generic open System.IO /// Get a sequence of values let rec range min max = seq { for num in [min .. max] do yield num } /// Get a sequence starting from 0 and going to max let rec zeroRange max = range 0 max /// Find the maximum number in a list with a starting accumulator (acc) let rec maxNum acc = function | [] -> acc | p::tail when p > acc -> maxNum p tail | p::tail -> maxNum acc tail /// A helper for finding max that sets the accumulator to 0 let rec findMax nums = maxNum 0 nums /// Build a collection of combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) let rec combos range = seq { let count = ref 0 for inner in range do for outer in Seq.skip !count range do yield (inner, outer) count := !count + 1 } let rec squares nums = let dict = new Dictionary<int, int>() for s in nums do dict.[s] <- (s * s) dict /// Counts the number of possible double squares for a given number and keeps track of other counts that are provided in the memo dict. let rec countDoubleSquares (num: int) (memo: Dictionary<int, int>) = // The highest relevent square is the square root because it squared plus 0 squared is the top most possibility let maxSquare = System.Math.Sqrt((float)num) // Our relevant squares are 0 to the highest possible square; note the cast to int which shouldn't hurt. let relSquares = range 0 ((int)maxSquare) // calculate the squares up front; let calcSquares = squares relSquares // Build up our square combinations; ie [1,2,3] = (1,1), (1,2), (1,3), (2,2), (2,3), (3,3) for (sq1, sq2) in combos relSquares do let v = calcSquares.[sq1] + calcSquares.[sq2] // Memoize our relevant results if memo.ContainsKey(v) then memo.[v] <- memo.[v] + 1 // return our count for the num passed in memo.[num] // Read our numbers from file. //let lines = File.ReadAllLines("test2.txt") //let nums = [ for line in Seq.skip 1 lines -> Int32.Parse(line) ] // Optionally, read them from straight array let nums = [1740798996; 1257431873; 2147483643; 602519112; 858320077; 1048039120; 415485223; 874566596; 1022907856; 65; 421330820; 1041493518; 5; 1328649093; 1941554117; 4225; 2082925; 0; 1; 3] // Initialize our memoize dictionary let memo = new Dictionary<int, int>() for num in nums do memo.[num] <- 0 // Get the largest number in our set, all other numbers will be memoized along the way let maxN = findMax nums // Do the memoize let maxCount = countDoubleSquares maxN memo // Output our results. for num in nums do printfn "%i" memo.[num] // Have a little pause for when we debug let line = Console.Read() And here is my version in C# (Runtime: ~1:40: using System; using System.Collections.Generic; using System.Diagnostics; using System.IO; using System.Linq; using System.Text; namespace FBHack_DoubleSquares { public class TestInput { public int NumCases { get; set; } public List<int> Nums { get; set; } public TestInput() { Nums = new List<int>(); } public int MaxNum() { return Nums.Max(); } } class Program { static void Main(string[] args) { // Read input from file. //TestInput input = ReadTestInput("live.txt"); // As example, load straight. TestInput input = new TestInput { NumCases = 20, Nums = new List<int> { 1740798996, 1257431873, 2147483643, 602519112, 858320077, 1048039120, 415485223, 874566596, 1022907856, 65, 421330820, 1041493518, 5, 1328649093, 1941554117, 4225, 2082925, 0, 1, 3, } }; var maxNum = input.MaxNum(); Dictionary<int, int> memo = new Dictionary<int, int>(); foreach (var num in input.Nums) { if (!memo.ContainsKey(num)) memo.Add(num, 0); } DoMemoize(maxNum, memo); StringBuilder sb = new StringBuilder(); foreach (var num in input.Nums) { //Console.WriteLine(memo[num]); sb.AppendLine(memo[num].ToString()); } Console.Write(sb.ToString()); var blah = Console.Read(); //File.WriteAllText("out.txt", sb.ToString()); } private static int DoMemoize(int num, Dictionary<int, int> memo) { var highSquare = (int)Math.Floor(Math.Sqrt(num)); var squares = CreateSquareLookup(highSquare); var relSquares = squares.Keys.ToList(); Debug.WriteLine("Starting - " + num.ToString()); Debug.WriteLine("RelSquares.Count = {0}", relSquares.Count); int sum = 0; var index = 0; foreach (var square in relSquares) { foreach (var inner in relSquares.Skip(index)) { sum = squares[square] + squares[inner]; if (memo.ContainsKey(sum)) memo[sum]++; } index++; } if (memo.ContainsKey(num)) return memo[num]; return 0; } private static TestInput ReadTestInput(string fileName) { var lines = File.ReadAllLines(fileName); var input = new TestInput(); input.NumCases = int.Parse(lines[0]); foreach (var lin in lines.Skip(1)) { input.Nums.Add(int.Parse(lin)); } return input; } public static Dictionary<int, int> CreateSquareLookup(int maxNum) { var dict = new Dictionary<int, int>(); int square; foreach (var num in Enumerable.Range(0, maxNum)) { square = num * num; dict[num] = square; } return dict; } } } Thanks for taking a look. UPDATE Changing the combos function slightly will result in a pretty big performance boost (from 8 min to 3:45): /// Old and Busted... let rec combosOld range = seq { let rangeCache = Seq.cache range let count = ref 0 for inner in rangeCache do for outer in Seq.skip !count rangeCache do yield (inner, outer) count := !count + 1 } /// The New Hotness... let rec combos maxNum = seq { for i in 0..maxNum do for j in i..maxNum do yield i,j }

    Read the article

< Previous Page | 80 81 82 83 84 85 86 87 88 89 90 91  | Next Page >