Search Results

Search found 1351 results on 55 pages for 'aws s3'.

Page 15/55 | < Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >

• Large file download for a Rails project

  - by Horace Ho

  One client project will be online two months later. One of the requirements changed is to support large files (10 to 15MB per RAW camera file, expected 1000 to 5000 files download per day) download worldwide for their customers. The process will be: there is upload screen via paperclip to the rails local public folder a hourly task to upload to web storage (S3?) update the download url from paperclip url to the web url Questions: is there a gem/plug-in for this purpose? if no, any gem/plug-in for S3 to recommend? Questions about the storage provider: is S3 recommended? or other service to recommend? The baseline is: the client's web server does not and will not have the bandwidth to handle the downloads. Thanks

  Read the article

• Map Reduce job on Amazon: argument for custom jar

  - by zero51

  Hi all, This is one of my first try with Map Reduce on AWS in its Management Console. Hi have uploaded on AWS S3 my runnable jar developed on Hadoop 0.18, and it works on my local machine. As described on documentation, I have passed the S3 paths for input and output as argument of the jar: all right, but the problem is the third argument that is another path (as string) to a file that I need to load while the job is in execution. That file resides on S3 bucket too, but it seems that my jar doesn't recognize the path and I got a FileNotFound Exception while it tries to load it. That is strange because this is a path exactly like the other two... Anyone have any idea? Thank you Luca

  Read the article

• Ubuntu: Move fsbackup backups to Amazon S3

  - by Alexander Gladysh

  I have a legacy server (Ubuntu 9.10 Karmic x86), where previous admin set up backups with fsbackup. This server lives in a VPS (under some kind of Xen), and it is low on HDD space (16 GB total). Now it came to a point, where fsbackup backups take more space than the rest of data in the system. The filesystem is 100% filled, and I already cleaned up all that I could, aside from actual backups. I do not have any experience managing fsbackup, and I do not want to break or lose the backups. Googling fsbackup gives surprisingly low quality results... Here is how my backups look like: $ sudo ls -lh /var/archives total 8.1G -rw-rw---- 1 root root 318 2011-01-06 06:26 myserver-20110106.md5 -rw-rw---- 1 root root 258 2011-01-07 06:26 myserver-20110107.md5 -rw-rw---- 1 root root 318 2011-01-08 06:26 myserver-20110108.md5 -rw-rw---- 1 root root 318 2011-01-09 06:26 myserver-20110109.md5 -rw-rw---- 1 root root 346 2011-01-10 06:43 myserver-20110110.md5 -rw-rw---- 1 root root 14M 2011-01-06 06:26 myserver-all-mysql-databases.20110106.sql.bz2 -rw-rw---- 1 root root 14M 2011-01-07 06:26 myserver-all-mysql-databases.20110107.sql.bz2 -rw-rw---- 1 root root 14M 2011-01-08 06:26 myserver-all-mysql-databases.20110108.sql.bz2 -rw-rw---- 1 root root 14M 2011-01-09 06:26 myserver-all-mysql-databases.20110109.sql.bz2 -rw-rw---- 1 root root 862 2011-01-10 06:43 myserver-all-mysql-databases.20110110.sql.bz2 -rw-rw---- 1 root root 827K 2011-01-03 06:25 myserver-etc.20110103.master.tar.gz -rw-rw---- 1 root root 16K 2011-01-06 06:25 myserver-etc.20110106.tar.gz -rw-rw---- 1 root root 16K 2011-01-07 06:25 myserver-etc.20110107.tar.gz -rw-rw---- 1 root root 16K 2011-01-08 06:25 myserver-etc.20110108.tar.gz -rw-rw---- 1 root root 16K 2011-01-09 06:25 myserver-etc.20110109.tar.gz -rw-rw---- 1 root root 827K 2011-01-10 06:25 myserver-etc.20110110.master.tar.gz -rw------- 1 root root 36K 2011-01-10 06:25 myserver-etc.incremental.bin -rw-rw---- 1 root root 29M 2011-01-03 06:25 myserver-home.20110103.master.tar.gz -rw-rw---- 1 root root 11K 2011-01-06 06:25 myserver-home.20110106.tar.gz -rw-rw---- 1 root root 14K 2011-01-07 06:25 myserver-home.20110107.tar.gz -rw-rw---- 1 root root 11K 2011-01-08 06:25 myserver-home.20110108.tar.gz -rw-rw---- 1 root root 11K 2011-01-09 06:25 myserver-home.20110109.tar.gz -rw-rw---- 1 root root 2.0M 2011-01-10 06:25 myserver-home.20110110.master.tar.gz -rw------- 1 root root 27K 2011-01-10 06:25 myserver-home.incremental.bin -rw-rw---- 1 root root 1.5G 2011-01-03 06:29 myserver-opt.20110103.master.tar.gz -rw-rw---- 1 root root 1.5M 2011-01-06 06:25 myserver-opt.20110106.tar.gz -rw-rw---- 1 root root 1.5M 2011-01-07 06:25 myserver-opt.20110107.tar.gz -rw-rw---- 1 root root 1.5M 2011-01-08 06:25 myserver-opt.20110108.tar.gz -rw-rw---- 1 root root 1.5M 2011-01-09 06:25 myserver-opt.20110109.tar.gz -rw-rw---- 1 root root 1.5G 2011-01-10 06:30 myserver-opt.20110110.master.tar.gz -rw------- 1 root root 201K 2011-01-10 06:30 myserver-opt.incremental.bin -rw-rw---- 1 root root 2.3G 2011-01-03 06:41 myserver-srv.20110103.master.tar.gz -rw-rw---- 1 root root 44M 2011-01-06 06:26 myserver-srv.20110106.tar.gz -rw-rw---- 1 root root 27M 2011-01-07 06:25 myserver-srv.20110107.tar.gz -rw-rw---- 1 root root 39M 2011-01-08 06:26 myserver-srv.20110108.tar.gz -rw-rw---- 1 root root 2.0M 2011-01-09 06:25 myserver-srv.20110109.tar.gz -rw-rw---- 1 root root 2.7G 2011-01-10 06:42 myserver-srv.20110110.master.tar.gz -rw------- 1 root root 3.4M 2011-01-10 06:42 myserver-srv.incremental.bin I'm thinking about moving backups to Amazon S3, but before that I have to free some space, so the server can work. Perhaps I can mount /var/archives to an Amazon S3 bucket somehow... Any advice?

  Read the article

• AWS RDS connection count

  - by wmarbut

  I am using AWS RDS with MySQL for a project and have a "large" instance. The documentation is clear on what this means as far as compute resources and RAM goes, but I can't find anything that documents how many open database connections that I can have. The app that I am using is PHP and it utilizes PDO with persistent connections. This means that the number of open connections could reach the maximum number of PHP child processes running at any given point. How do I ensure that my RDS instance has a max connections setting high enough to be comfortable with this?

  Read the article

• AWS Large Instance: /mnt does not show all the space that should be available

  - by Emile Baizel

  I just created a Large (m1.large) 64 bit instance which comes with 850 GB instance storage. Look at the Large Instance http://aws.amazon.com/ec2/instance-types/ A 'df -h' from the root folder gives me the output below. The /mnt is where I'm thinking the instance storage is but here it is only showing me 414G. I have set up two servers and both are showing the same numbers. root@ip-11-11-11-11:/# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 7.9G 1.1G 6.5G 14% / none 3.7G 112K 3.7G 1% /dev none 3.7G 0 3.7G 0% /dev/shm none 3.7G 48K 3.7G 1% /var/run none 3.7G 0 3.7G 0% /var/lock /dev/sdb 414G 199M 393G 1% /mnt

  Read the article

• GitHub push to AWS Elastic Beanstalk

  - by nute

  I am using GitHub for code management. I am using Amazon AWS Elastic Beanstalk as a server. Amazon announced that you can use Git to push code to the application server. However, to do this I'd have to let go of GitHub as they are essentially replacing the git server. Is there any way to have the best of both worlds? I don't necessarily need to "deploy" everytime I push, but I'd like to have it uploaded as a "Version", and then I can deploy the version I want anytime.

  Read the article

• AWS forwarding email to a gmail account

  - by user2433617

  So I registered a domain name. I then set up a static webpage using aws (S3 and Rout53). Now what I want to do is forward any email I get from that custom domain name to a personal email address I have set up. I can't seem to figure out how to do this. I have these record sets already: A NS SOA CNAME I believe I have to set up an MX record but not sure how. say I have the custom domain [email protected] and I want to redirect all email to [email protected]. The personal email account is a gmail (google accounts) email address. Thanks.

  Read the article

• AWS EC2 can't execute user-data script

  - by Bloodnut

  I'm pretty new to AWS and EC2 but I want to run instances with a user script after it's booted from another instance. I have installed ec2 tools and ran the command as it's explained in various examples like here http://www.turnkeylinux.org/blog/ec2-userdata and Eric Hammond's tutorials. however when I actually use the command: "ec2-run-instances --key my-key --user-data-file myscript my-ami" it only runs the new instance but doesn't execute the script myscript contains: #!/bin/bash echo "hello" ~/output.txt I'm running ubuntu server 12.04 AMIs. the target AMIs are duplicates of the initiating instance. if I run curl http:// 169.254.169.254/latest/user-data the imported script is there.

  Read the article

• Automatically Snapshoting AWS instances (or other back up strategy)

  - by user1172468

  I just realized that my aws instance count has risen into the double digits. I'm currently backing portions of my folders and dbs and moving them off to a backup instance. What I think I should be doing is taking a snapshot of the instances (automatically) and persisting them on S3 so I have a running 7 day collection of daily backups. There is a question asking the same thing here, however the answers don't go into depth. So the closest answer seems to be: use a cron job to snapshot the instance. So do I run the cron job on the instance itself? or do I have a micro instance to run these snapshots? Could I get an example script or the command for say a linux flavor? what software must I have installed to get this to run? Thanks.

  Read the article

• What ports to open in aws security group for aws?

  - by HarrisonJackson

  I am building the backend for a turn based gamed. My experience is mostly with a lamp stack; I've dabbled in nginx on a node side project. I just read Scaling PHP Applications by Stephen Corona of Twit Pic. He recommends an nginx server over apache. He says that his ubuntu machine has 32768-61000 ports open. On AWS do I need to modify my security to group to allow access to those ports? How do I ensure nginx is taking full advantage of this configuration?

  Read the article

• best filesystem for an aws s3 like service

  - by gucki

  Hi! I need to build a fault tolerant, highly available key/value storage (no posix, only same functionaluty as S3) using cheap existing hardware. The storage should be able to handle several billions of items. The maximum size of items is around 1GB, most are only several KB. What's the best software/ filesystem for this task? I already had a brief look at mogilefs, mongodb (grid-fs) & glusterfs but I'm not really sure which is stable & fault tolerant enough. The simpler the setup and later expansion the better :). Corin

  Read the article

• Windows Server 2012 VPN Server on AWS VPC EC2 Instance

  - by abran

  I'd like to use window server 2012 VPN on a AWS VPC EC2 instance. The VPC has one public subnet and the EC2 instance has one network adapter. I've taken the following steps, but have been unsuccessful; am I missing a step or configuration? Thanks. Configured an elastic IP for the VPC Enabled protocols 47, 50, & 51 Added the RRAS role to the (EC2 instance) server Configured the RRAS for vpn only. Note: I'm able to RDP to the EC2 instance, but not able to ping the external IP.

  Read the article

• AWS Autoscaling issue with existing nodes in ELB

  - by Ram Prasad

  I already have a ELB setup called MyLoadBalancer. I already have 2 nodes running on it with health checks (that checks a URL on the node to see if they are up) Created an autoscaling group (min 2, Max 10) Associated launchconfig mylaunchconfig that provisions a node using an AMI Created a trigger, that checks for avg min connections of 100 and Max of 500 (checks the load balancer and it is support to increase the node count by 1, if avg connections are 500 and decrease by one if less than 100) as-create-or-update-trigger MyTrigger --auto-scaling-group MyAutoScalingGroup --namespace "AWS/ELB" --measure RequestCount --statistic Average --dimensions "LoadBalancerName=MyLoadBalancer" --period 60 --lower-threshold 500 --upper-threshold 800 --lower-breach-increment=-1 --upper-breach-increment=1 --breach-duration 600 Now the issue is, as soon as I put in the trigger, it start 2 nodes .... but there are already two nodes in the LB. So, why is it provisioning 2 more nodes, when the nodes are there ? is it because it is not recognizing the existing 2 nodes ? then how do I add the existing nodes to the AutoScaling group ?

  Read the article

• Need advise on choosing aws EC2

  - by Mayank

  I'm planning to host a website where in the first phase I would target 30,000 users. It is in php and runs on Apache server. I'm assuming 8,000 users can be online in worst case scenario and 1000 of them will be uploading photographs. A photograph will be resized to around 1MB at client side and one HTTP request is uploading only one photograph. My plan: 2 Small EC2 instances to run Apache httpd 2 Small EC2 instances to DB (Postgresql). I to write data and other its read replica. EBS volumes for DBs Last, Amazon S3 for uploaded photographs. My question here Is Small EC2 instance more than what I require. I mean should I go for micro Is 8000 simultaneous user a right no. (to decide what EC2 instance to choose) for a new website Or should I go for Small instance so to make it capable of spikes

  Read the article

• AWS ELB as backend for Varnish Accelerator

  - by addisonj

  I am working on a large deployment on AWS that has high uptime requirements and variable loads throughout the day. Obviously, this is the perfect use case for ELB (Elastic Load Balancer) and autoscaling. However, we also rely on varnish for caching of API calls. My initial instinct was to structure the stack so that varnish uses ELB as a backend which in turn hits an appGroup. Varnish -> ELB -> AppServers However, according to a few sources that isn't possible as ELB constantly changes the IP address of its DNS hostname, which varnish caches on start, meaning changes to the IP won't be picked up by varnish. Reading around however, it looks like people are doing this so I am wondering what workarounds exist? Perhaps a script to reload the vcl periodically? In the case of where this is really just not a good idea, any idea of other solutions?

  Read the article

• paperclip overwrites / resets S3 permissions for non-bucket-owners

  - by adriandz

  I have opened this as an issue on Github (http://github.com/thoughtbot/paperclip/issues/issue/225) but on the chance that I'm just doing this wrong, I thought I'd also ask about it here. If someone can tell me where I'm going wrong, I can close the issue and save the Paperclip guys some trouble. Issue: When using S3 for storage, and you wish your bucket to allow access to other users to whom you have granted access, Paperclip appears to overwrite the permissions on the bucket, removing access to these users. Process for duplication: Create a bucket in S3 and set up a Rails app with Paperclip to use this bucket for storage Add a user (for example, [email protected], the user for the video encoding service Zencoder) to the bucket, and grant this user List and Read/Write permissions. Upload a file. Refresh the permissions. The user you added will be gone. As well, a user "Everyone" with read permissions will have been added. The end result is that you cannot, so far as I can tell, retain desired permissions on your bucket when using Paperclip and S3. Can anyone help?

  Read the article

• Jungledisk - choosing between Amazon S3 and Rackspace ?

  - by Dietmar

  Hi, jungledisk is owned by rackspace and offers an option to choose Amazon S3 which charges for traffic whereby if data is stored on rackspace, no charges for traffic. For a small company, to just backup files with jungle disk from ubuntu linux OS computers, i wonder what to choose. Commercial psychologically the price on traffic and the Amazon brand make it appear or maybe it is, of better quality than the cheaper Rackspace offer which is when it comes for just backing up ok ? but if its about backing up around 30 GB which updates from time to time, what would you consider a good choice ?

  Read the article

• Using AWS SES with Sendmail

  - by Abs

  I am trying to send mail via AWS SES uisng Sendmail. I have Sendmail version 8.14.4 installed and I followed the first section of this useful tutorial by Amazon. However, I get this: root@:/etc/mail# echo "Subject: test" | sendmail -v [email protected] [email protected]... Connecting to [127.0.0.1] via relay... [email protected]... Deferred: Connection timed out with [127.0.0.1] Can anyone help me get this working? The logs have the following: Dec 14 10:35:21 ip-10-xx-xx-181 sm-msp-queue[17910]: qBE8K1Lu016411: to=root, delay=00:21:24, xdelay=00:06:19, mailer=relay, pri=121806, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection timed out with [127.0.0.1]

  Read the article

• Sticky Load Balancing with AWS

  - by John Wheal

  I have just setup a load balancer with AWS for a few instances as search engine crawlers were bringing down the site (it has millions of pages). Parts of the site allow you to login so I selected: Enable Application Generated Cookie Stickiness and everything works fine. I now wonder how this will effect my SEO and the crawlers. As I selected sticky load balancing does this mean that a crawler will be stuck on one server and therefore defeat the point in the load balancer? Any recommendations will be appreciated.

  Read the article

• How to Architect a system on AWS for scaling (with a MySQL back-end)

  - by Edan Maor

  I'm trying to understand how to architect an Amazon Web Services application. As I understand it, the whole point of using something like AWS is to make the eventual scaling easier, so I'm trying to understand how to do that. I have an instance, running off of EBS (EBS-based instance, not a regular instance). My application (a Django app) uses MySQL as a back-end. So the question is, where am I supposed to install the MySQL? Do I install it on the same instance? In which case, as far as I can tell, I can't simply create more server instances from that image. Or am I supposed to simply spin up another server as a DB server, and run off of that? Thanks for any help!

  Read the article

• Sharing / replicating EBS across AWS nodes

  - by skrat

  I would like to use single EBS storage across multiple EC2 nodes (web/app servers). I've read some articles on snapshot sharing, but that doesn't suit well for what we need. We use filesystem for storing DB record attachments, so if one such attachment gets created, we need it to be immediately available to all nodes (to serve). So far only NFS seem to be viable, but it's a pain to configure and maintain. Another option could be storing those attachments on S3 instead, but that would cut us of doing any analysis on that data. This must be quite common problem when scaling in AWS, what solutions are there?

  Read the article

• Load testing nginx inside AWS

  - by andy

  I'm trying to load test nginx running on AWS. I need to try to optimise it to handle 1Gbps of inbound traffic. Currently I've got it to peak at 85Mbit/s by running nginx on an m1.large with 4 other machines hitting it by using ab with -i (for head requests) -k (keepalives) -r (ignore failed requests) -n 500000 -c 20000. I'm struggling to generate more than 85 Mbit/s traffic from 4 machines, yet when I do scp a large file I get nearly 0.25Gbit/s of traffic going over the network. Are there any tools or approaches that I could use to load test nginx that might generate more load? I'm only interested in inbound traffic, so perhaps a DoS tool could help if it chucks away responses? I'm hitting a very small (40 byte) static asset, and have peaked at handling 50K concurrent connections and getting 25k reqs/s when just using a single load generator machine.

  Read the article

• create a CNAME record for AWS LoadBalancer DNS name

  - by t q

  I am trying to setup a loadBalancer on AWS. The A-Record it gave me looks like myLoadBalancer-**********.us-east-1.elb.amazonaws.com however when i try to put that in my domain registrars A-Record, i get an errorIP address is not valid. Must be of type x.x.x.x where x is 0-255. amazons solution is you should create a CNAME record for the LoadBalancer DNS name, or use Amazon Route 53 to create a hosted zone. route 53 gives me DNS numbers but if i use that then my email doesnt work from the registrar. question: is there a way to use route 53 and retain my emails? or should i create a CNAME record for the LoadBalancer DNS name, if so how do i do this ... not sure what this means?

  Read the article

• Amazon AWS EC2 instance pricing

  - by iss42

  It seems to me that Amazon obfuscate their pricing somewhat. So if I were to create a "m1.medium" instance and run it say 4 hours a day what would I be charged per month? (assume nothing else needed) You'd think this would be straight forward to answer, but I don't see a clear answer anywhere. This tool: http://calculator.s3.amazonaws.com/calc5.html Gives: $22.21 If I set it to 1 hour is gives: $5.65 But is this correct? The m1.medium has a Local storage of 410gb... I assume that this is persistent, but is it charged for? In the tool from Amazon this is not implied or suggested or hinted at. Any guidance would be most appreciated!

  Read the article

• Uptime concerns in case of AWS outage

  - by Aditya Patawari

  I am running an Elastic Load Balancer backup by 2 instances in different Availability Zones in US East. I am using Multi-AZ RDS as well. Ideally this should ensure that if one AZ goes down, it should not effect the app because everything is spread across multiple AZs. But the recent AWS outage took the app down for a long time. I am not sure how this can happen. It would be great if someone can point out what went wrong. Major question here I have is how can I avoid this in future? I can setup app servers across different regions or even providers and use DNS for load balancing but what do I do with MySQL? Read Replicas will introduce some lag which I would want to avoid.

  Read the article

< Previous Page | 11 12 13 14 15 16 17 18 19 20 21 22  | Next Page >