Search Results

Search found 1351 results on 55 pages for 'aws s3'.

Page 17/55 | < Previous Page | 13 14 15 16 17 18 19 20 21 22 23 24  | Next Page >

  • best practice? Consumer data in MySQL on Amazon EBS (Elastic block store)

    - by jeff7091
    This is a consumer app, so I will care about storage costs - I don't want to have 5x copies of data lying about. The app shards very well, so I can use MySQL and not have scaling issues. Amazon EBS has a nice baseline+snapshot backup capability that uses S3. This should have a light footprint (in terms of storage cost). BUT: the magnolia.com story scares the crap out of me: basically flawless block-level backup of a corrupt DB or filesystem. Is there anything that is nearly as storage efficient as EBS at the MySQL level?

    Read the article

  • s3fs: how to force remount on errors?

    - by Alexander Gladysh
    I use s3fs 1.33 on Ubuntu 9.10. Regularily it gives me errors like this: rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32) rsync: close failed on "/mnt/s3/mybucket/filename": Software caused connection abort (103) rsync error: error in file IO (code 11) at receiver.c(731) [receiver=3.0.6] rsync: connection unexpectedly closed (86 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.6] Any attempt to work with mounted directory after that gives this error: Transport endpoint is not connected To get rid of this, I have to remount. Is there a way to force a remount automatically?

    Read the article

  • Reducing latency for different geographic regions on Amazon Cloud

    - by Shoaibi
    I have got an application which has three components Application code : Amazon EC2 US-EAST-1 instance Application images, and other static data : Amazon S3 with CloudFront Application Database : Amazon RDS In short i need something like Cloud Front for EC2. In long, people using this application from a different region say middle east will have faster static content downloading due to Cloud Front but there would be a lot of latency in communicating to EC2 instance. I want to use a budget friendly way of enhancing this. Launching Amazon Instances in every region that offer is sure a choice, but isn't really cheap, so would try to avoid it unless its last resort. Also say if my clients also need to communicate to the RDS database directly, is there some kind of solution which gives that kind of functionality mentioned above, but for RDS?

    Read the article

  • Amazon EC2 migration from one region to the other

    - by Gnanam
    I'm using the following Amazon EC2 resources in the US East (Virginia) region: 1 Running Instance 1 Elastic IP 2 EBS Volumes 100 EBS Snapshots 1 Key Pair 2 Security Groups 5 My Own AMIs (customized based on my application stack) My instance is based on Linux distribution (CentOS) and my AMIs are S3-backed. Both EBS volumes are mounted on this running instance. We're planning to migrate our deployment to US-West region. Because Amazon EC2 resources are not shared across regions, my questions are: What are all the factors that I need to consider in advance? What are all the recommended & different ways of migrating each EC2 resources from one region to the other? Are there any hidden risks involved during and/or after the migration? Experts ideas/suggestions/recommendations on this are highly appreciated.

    Read the article

  • How can I limit CloudFront downloads

    - by Alex Crouzen
    I'm looking to use Amazon's CloudFront to host some content in the near future. Currently, I'm keeping it very simple and I'm just uploading my content to S3 and then making a distribution available via Cloudfront. However, because I have a limited budget, I'd like to be able to limit the number of downloads or the money spent on bandwidth. As far as I can see, I can't set any quotas or budgets like you can in Google's App Engine, so I'm looking for another way of doing this. Has anyone had any experience doing this? One approach I'm thinking of is having to place a webserver with redirects in between, but that kind of defeats the simplicity of CF for me.

    Read the article

  • One Way Sync of a Bucket With Local Directory

    - by user48651
    I have a local directory that I would like to synchronize with an S3 bucket. I have two specific requirements: If local file is the same as the remote, do not re-transfer it to the bucket. If some files or directories exist in the bucket but do not exist on local, delete them. Basically the bucket should mirror the local copy and not vice-versa. I looked into s3cmd sync command, but unfortunately requirement 2 is not fulfilled. If files exists in the bucket but not on local copy, they will be copied to the local instead of being deleted.

    Read the article

  • Challenges w.r.t. proximity between application hosted outside Amazon and Amazon persistence service

    - by Kabeer
    Hello. This is about hosting a web portal. Earlier my topology was entirely based on Amazon AWS but the price factor (especially for EC2) now makes me re-think. I'll now quickly come to what I have finally arrived at. I'll launch the portal that'll be hosted on Godaddy (unlimited plan on Windows). The portal uses SimpleDB for storing metadata and S3 for blobs. Locally available MySQL will be used for the ASP.Net provider services. Once the portal is profitable, I intent to move to Amazon in totality. Now considering the proximity between Godaddy & Amazon, would I face 'substantial' performance problems? Are there any suggestions to improve upon my topology.

    Read the article

  • Debugging logrotate postrotate script

    - by robert
    Following is my logrotate conf. /mnt/je/logs/apache/jesites/web/*.log" { missingok rotate 0 size 5M copytruncate notifempty sharedscripts postrotate /home/bitnami/.conf/compress-and-upload.sh /mnt/je/logs/apache/jesites/web/ web endscript } And compress-and-upload.sh script, #!/bin/sh # Perform Rotated Log File Compression tar -czPf $1/log.gz $1/*.1 # Fetch the instance id from the instance EC2_INSTANCE_ID="`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`" if [ -z $EC2_INSTANCE_ID ]; then echo "Error: Couldn't fetch Instance ID .. Exiting .." exit; else /usr/local/bin/s3cmd put $1/log.gz s3://xxxx/logs/$(date +%Y)/$(date +%m)/$(date +%d)/$2/$EC2_INSTANCE_ID-$(date +%H:%M:%S)-$2.gz fi # Removing Rotated Compressed Log File rm -f $1/log.gz The files are rotated, but shell script is not executed. I don't know how to debug the postscript. Is there any logfile I chek to see if there is any permission issues. If i directly execute the script from commandline file upload works. Thanks.

    Read the article

  • Setting up scripts in Amazon EC2 Cloud

    - by racket99
    Hello, I am currently running a few perl and python scripts on a windows pc and would like to port over to the Amazon EC2 servers running 64-bit LINUX. The scripts are basic web scrapers that go to a variety of websites, get data and then save daily as csv files. I would like to install these in the cloud and get them running in an automated way so that they will run without my intervention. Also given that I don't want to lose all the data if the instance crashes, I should also upload the csv files to Amazon S3. Any idea how I can do this? I am not terribly versed in LINUX nor do I know Perl/Python well. What is the best way for me to tackle thi

    Read the article

  • AWS autoscaling. Launch Config/Auto Scaling Group and VPC instance with two ifaces

    - by icalvete
    I want create an Launch Config/Auto Scaling Group to build instances inside an VPC with two subnets ("frontend" and "backend") I need that this instances have two ifaces. One in "frontend" subnet and one in "backend" subnet. I can't see how do that. It's no posible from AWS console and neither with aws cli. http://docs.aws.amazon.com/cli/latest/reference/autoscaling/create-launch-configuration.html http://docs.aws.amazon.com/cli/latest/reference/autoscaling/create-auto-scaling-group.html Launch Config don't say nothing about this. http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/create-lc-with-instanceID.html Ideas? Thanks!!!

    Read the article

  • Problems with the backup

    - by marcodv
    I did a script which run around 4 o'clock in the morning, for backup all the mysql databases and the config file for 250 linux vm. The problem is that it tooks ages for complete and more than 50% of these vm, need more than 8 hours for complete. More or less all the vm had the same configuration,I mean Same amount of ram same amount of disk space same number of cpu Debian 6.0.5 I am saving these backup on amazon s3, because is the cheapest solutions that I've found. Now my questions is: Has anyone some solutions or suggestions about that? On one blog I've read that probably the ionice and nice combination could be good work around about that. any thought?

    Read the article

  • Backup software for Ubuntu - which one?

    - by Industrial
    Hi everybody, I have spent some time testing out different backup solutions for my small home office during the last weeks, but still haven't found anything that have been working out too well yet. We can definitely work with a non-GUI script if that's what it takes, if only the requirements are fulfilled: Upload to Amazon S3 Europe. We get unbelievable slow uploading speed to US, so uploading 400+ GB of data will not be happening anytime this year... Incremental backups - only changed files shall be uploaded or we will have a big bill from Amazon in the end of each month.. Files should not be uploaded in one big per-folder archive. This is not efficient at all, since if we change one file in a subfolder, a huge two-digit GB sized file would have to be uploaded during next backup. Not good for economy again, or traffic overhead on our internet connection. What options are available to us? Thanks!

    Read the article

  • Backing Up User Data when data is not in use. Should I be concerned?

    - by jberryman
    This may be a dumb question. I would like to use duplicity to make backups to Amazon S3 of directories, each of which contains a different user's data. Each directory could be written to at any time. So I have two questions: Should I be concerned that a scheduled backup of a directory might occur in the middle of data being written to files in the directory, resulting in a corrupted backup? And if that is a valid concern, how would I go about temporarily delaying an operation while IO was happening, to try to minimize that effect. Thanks for the advice

    Read the article

  • mirror sql server 2008 to AWS instance from our datacenter?

    - by Alex
    We are currenlty running on hosted pos system locally and would like to mirror to AWS. We are new to AWS and would like to know the most cost effective way to do this? We have 2 DB and 2 web servers right now in one cabinet in CA. One tape drive, one firewall, one SNA. We are thinking to replicate our system in AWS (using sql server 2008) and just mirror both systems and use a witness server between them to keep the data in sync? The goal is, if CA datacenter goes down, AWS keeps running. User see no downtime. All data is synced. Is anyone doing something similar? Would this be practical to just use AWS in this fashion? Thanks

    Read the article

  • Whats the best cloud backup solution for a small scale server envoirnment?

    - by nbv4
    I have a server that runs a postgres database that contains about 200MB of data. Currently I have a cron job setup on my home computer which: ssh's into my server runs a remote script which makes a backup of the database scp's that dump over to my local hard drive for storage. Each dump is 20MiB. does this every six hours (one months of backups is roughly 2GiB) The problem with this setup is that if my local machine goes down for whatever reason, no backups will be made. Also, I can't have the cron run from the server, because I can't have it scp'd to my local machine from my server (firewalls and all that crap). My local machine is running Ubuntu 10.04, and my server is Ubuntu 9.10 server edition. I looked into Ubuntu One, but currently it's gui-only. I also looked into dropbox, but it's a pain in the ass to get setup in linux without gui support. Amazon S3 looks good but it's not free (yet dirt cheap). Is there any other alternative that I should look into? I'd prefer something where I can just have my script dump the database into a directory, and have the backup service 'watch' that folder and sync accordingly. I can maybe also have my local machine sync to the cloud backup so I have even more redundancy, plus easy access to my backups for use in testing.

    Read the article

  • Whats the best cloud backup solution for a small scale server environment?

    - by nbv4
    I have a server that runs a postgres database that contains about 200MB of data. Currently I have a cron job setup on my home computer which: ssh's into my server runs a remote script which makes a backup of the database scp's that dump over to my local hard drive for storage. Each dump is 20MiB. does this every six hours (one months of backups is roughly 2GiB) The problem with this setup is that if my local machine goes down for whatever reason, no backups will be made. Also, I can't have the cron run from the server, because I can't have it scp'd to my local machine from my server (firewalls and all that crap). My local machine is running Ubuntu 10.04, and my server is Ubuntu 9.10 server edition. I looked into Ubuntu One, but currently it's gui-only. I also looked into dropbox, but it's a pain in the ass to get setup in linux without gui support. Amazon S3 looks good but it's not free (yet dirt cheap). Is there any other alternative that I should look into? I'd prefer something where I can just have my script dump the database into a directory, and have the backup service 'watch' that folder and sync accordingly. I can maybe also have my local machine sync to the cloud backup so I have even more redundancy, plus easy access to my backups for use in testing. Edit: My server is a VPS, so what solution I end up using has to be 100% software based.

    Read the article

  • 404 redirect with cloud storage

    - by Jeremy DeGroot
    I'm hoping to reach someone with some experience using a service like Amazon's S3 with this question. On my site we have a dedicated image server. And on this server, we have an automatic 404 redirect through Apapche so that, if a user tries to access an image that doesn't exist, they'll see a snazzy "Image Not Available" image. We're looking to move the hosting of these images to a cloud storage solution (S3 or Rackspace's CloudFiles), and I'm wondering if anyone's had any success replicating this behavior on a cloud storage service and if so how they did it.

    Read the article

  • Online file storage similar to Amazon S3

    - by Joel G
    I am looking to code a file storage application in perl similar to amazon s3. I already have a amazon s3 clone that I found online called parkplace but its in ruby and is old also isn't built for high loads. I am not really sure what modules and programs I should use so id like some help picking them out. My requirements are listed below (yes I know there are lots but I could start simple then add more once I get it going): Easy API implementation for client side apps. (maybe RESTful but extras like mkdir and cp (?) Centralized database server for the USERDB (maybe PostgreSQL (?). Logging of all connections, bandwidth used, well pretty much everything to a centralized server (maybe PostgreSQL again (?). Easy server side configuration (config file(s) stored on the servers). Web based control panel for admin(s) and user(s) to show logs. (could work just running queries from the databases) Fast High Uptime Low memory usage Some sort of load distribution/load balancer (maybe a dns based or pound or perlbal or something else (?). Maybe a cache of some sort (memcached or parlbal or something else (?). Thanks in advance

    Read the article

  • How to implement message queuing and handling in AWS with NServiceBus

    - by Pete Lunenfeld
    I am creating a new ASP MVC order application in the Amazon (AWS) cloud with the persistence layer at my local datacenter. I will be using the CQRS pattern. The goal of the project is high availability using Queue(s) to store and forward writes (commands/events) that can be picked up and handled asynchronously at my local datacenter. Then, ff the WAN or my local datacenter fails, my cloud MVC app can still take orders and just queue them up until processing can resume. My first thought was to use AWS SQS for the queuing and create my own queue consumer/dispatcher/handler in my own c# application to process the incoming messages/events. MVC (@ Amazon) -- Event/POCO -- SQS -- QueueReader (@ my datacenter) -- DB Then I found NServiceBus. NSB seems to handle lots of details very nicely: message handling, retries, error handling, etc. I hate to reinvent the wheel, and NServiceBus seems like a full featured and mature product that would be perfect for me. But on further research, it does NOT look like NServiceBus is really meant to be used over the WAN in physically separated environments (Cloud to my Datacenter). Google and SO don't really paint a good picture of using NServiceBus across the WAN like I need. How can I use NServiceBus across the WAN? Or is there a better solution to handle queuing and message handling between Amazon an my local datacenter?

    Read the article

  • CSS not displayed depending on page

    - by Kanjiroushi
    I have a friend that has a really strange issue with my website. When he clicks on http://www.copeo.fr/ the page displays fine but when he clicks on a link like www.copeo.fr/user/ the CSS is not applied even after a refresh. The raw html does display. I asked him to display the CSS that is hosted on amazon S3 hcopeoressources.s3.amazonaws.com/style/futurvert/style.css and it displays fine. The code validates on W3C validator so does the CSS. I am lost what could be the origin of the issue. Could it be its enterprise cache? configuration of IE7 on his machine? If it happens to someone else who could explain the issue to me, I am all hears. Thanks

    Read the article

  • Bitmap issue in Samsung Galaxy S3

    - by user1531240
    I wrote a method to change Bitmap from camera shot : public Bitmap bitmapChange(Bitmap bm) { /* get original image size */ int w = bm.getWidth(); int h = bm.getHeight(); /* check the image's orientation */ float scale = w / h; if(scale < 1) { /* if the orientation is portrait then scaled and show on the screen*/ float scaleWidth = (float) 90 / (float) w; float scaleHeight = (float) 130 / (float) h; Matrix mtx = new Matrix(); mtx.postScale(scaleWidth, scaleHeight); Bitmap rotatedBMP = Bitmap.createBitmap(bm, 0, 0, w, h, mtx, true); return rotatedBMP; } else { /* if the orientation is landscape then rotate 90 */ float scaleWidth = (float) 130 / (float) w; float scaleHeight = (float) 90 / (float) h; Matrix mtx = new Matrix(); mtx.postScale(scaleWidth, scaleHeight); mtx.postRotate(90); Bitmap rotatedBMP = Bitmap.createBitmap(bm, 0, 0, w, h, mtx, true); return rotatedBMP; } } It works fine in another Android device, even Galaxy Nexus but in Samsung Galaxy S3, the scaled image doesn't show on screen. I tried to mark the bitmapChange method , let it show the original size Bitmap on screen but S3 also show nothing on screen. The information of variables in eclipse is here. The information of sony xperia is here. xperia and other device is working fine.

    Read the article

  • Amazon AMIs and Oracle VM templates

    - by llaszews
    I have worked with Oracle VM templates and most recently with Amazon Machine Images (AMI). The similarities in the functionality and capabilities they provide are striking. Just take a look a the definitions: An Amazon Machine Image (AMI) is a special type of pre-configured operating system and virtual application software which is used to create a virtual machine within the Amazon Elastic Compute Cloud (EC2). It serves as the basic unit of deployment for services delivered using EC2. AWS AMIs Oracle VM Templates provide an innovative approach to deploying a fully configured software stack by offering pre-installed and pre-configured software images. Use of Oracle VM Templates eliminates the installation and configuration costs, and reduces the ongoing maintenance costs helping organizations achieve faster time to market and lower cost of operations. Oracle VM Templates Other things they have in common: 1. Both have 35 Oracle images or templates: AWS AMI pre-built images Oracle pre-built VM Templates 2. Both allow to build your own images or templates: A. OVM template builder - OVM Template Builder - Oracle VM Template Builder, an open source, graphical utility that makes it easy to use Oracle Enterprise Linux “Just enough OS” (JeOS)–based scripts for developing pre-packaged virtual machines for Oracle VM. B. AMI 'builder' - AMI builder However, AWS has the added feature/benefit of adding your own AMI to the AWS AMI catalog: AMI - Adding to the AWS AMI catalog Another plus with AWS and AMI is there are hundreds of MySQL AMIs (AWS MySQL AMIs ). A benefit of Oracle VM templates is they can run on any public or private cloud environment, not just AWS EC2. However, with Oracle VM templates they first need to be images as AMIs before they can run in the AWS cloud.

    Read the article

  • Amazon EC2 spot instances - is there a catch ?

    - by gareth_bowles
    I needed to start a new EC2 instance today and decided to try out the new spot instances, where you can reduce your instance cost by bidding on the maximum per-hour price you're prepared to pay. Since today's spot price was only 35c / hour, compared with 85c / hour for an on-demand instance, I was wondering: if I just bid a really high price, say $1 / hour, can I effectively be sure of getting a much cheaper long-running instance than an on-demand instance (since the spot instances are only charged by the current spot price) ? I suppose it's theoretically possible for the spot price to go over the on-demand price, but as far as I can tell from the data on the AWS site, the spot price has always been well below that.

    Read the article

  • Amazon EC2 spot instances - is there a catch ?

    - by gareth_bowles
    I needed to start a new EC2 instance today and decided to try out the new spot instances, where you can reduce your instance cost by bidding on the maximum per-hour price you're prepared to pay. Since today's spot price was only 3.5c / hour, compared with 8.5c / hour for an on-demand instance, I was wondering: if I just bid a really high price, say 10c / hour, can I effectively be sure of getting a much cheaper long-running instance than an on-demand instance (since the spot instances are only charged by the current spot price) ? I suppose it's theoretically possible for the spot price to go over the on-demand price, but as far as I can tell from the data on the AWS site, the spot price has always been well below that.

    Read the article

< Previous Page | 13 14 15 16 17 18 19 20 21 22 23 24  | Next Page >