Search Results

Search found 1351 results on 55 pages for 'aws s3'.

Page 3/55 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • Should I persist images on EBS or S3?

    - by javanes
    I am migrating my Java,Tomcat, Mysql server to AWS EC2. I have already attached EBS volume for storing MySql data. In my web application people may upload images. So I should persist them. There are 2 alternatives in my mind: Save uploaded images to EBS volume. Use the S3 service. The followings are my notes, please be skeptic about them, as my expertise is not on servers, but software development. EBS plus: S3 storage is more expensive. (0.15 $/Gb 0.1$/Gb) S3 plus: Serving statics from EBS may influence my web server's performance negatively. Is this true? Does Serving images affect server performance notably? For S3 my server will not be responsible for serving statics. S3 plus: Serving statics from EBS may result I/O cost, probably it will be minor. EBS plus: People say EBS is faster. S3 plus: People say S3 is more safe for persistence. EBS plus: No need to learn API, it is straight forward to save the images to EBS volume. Namely I can not decide, will be happy if you guide. Thanks

    Read the article

  • Amazon Web Services (AWS) Plug-in for Oracle Enterprise Manager

    - by Anand Akela
    v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} Normal 0 false false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-bidi-font-family:"Times New Roman";} Contributed by Sunil Kunisetty and Daniel Chan Introduction and ArchitectureAs more and more enterprises deploy some of their non-critical workload on Amazon Web Services (AWS), it’s becoming critical to monitor those public AWS resources along side with their on-premise resources. Oracle recently announced Oracle Enterprise Manager Plug-in for Amazon Web Services (AWS) allows you to achieve that goal. The on-premise Oracle Enterprise Manager (EM12c) acts as a single tool to get a comprehensive view of your public AWS resources as well as your private cloud resources.  By deploying the plug-in within your Cloud Control environment, you gain the following management features: Monitor EBS, EC2 and RDS instances on Amazon Web Services Gather performance metrics and configuration details for AWS instances Raise alerts and violations based on thresholds set on monitoring Generate reports based on the gathered data Users of this Plug-in can leverage the rich Enterprise Manager features such as system promotion, incident generation based on thresholds, integration with 3rd party ticketing applications etc. AWS Monitoring via this Plug-in is enabled via Amazon CloudWatch API and the users of this Plug-in are responsible for supplying credentials for accessing AWS and the CloudWatch API. This Plug-in can only be deployed on an EM12C R2 platform and agent version should be at minimum 12c R2.Here is a pictorial view of the overall architecture: Amazon Elastic Block Store (EBS) Amazon Elastic Compute Cloud (EC2) Amazon Relational Database Service (RDS) Here are a few key features: Rich and exhaustive list of metrics. Metrics can be gathered from an Agent running outside AWS. Critical configuration information. Custom Home Pages with charts and AWS configuration information. Generate incidents based on thresholds set on monitoring data. Discovery and Monitoring AWS instances can be added to EM12C either via the EM12c User Interface (UI) or the EM12c Command Line Interface ( EMCLI)  by providing the AWS credentials (Secret Key and Access Key Id) as well as resource specific properties as target properties. Here is a quick mapping of target types and properties for each AWS resources AWS Resource Type Target Type Resource specific properties EBS Resource Amazon EBS Service CloudWatch base URI, EC2 Base URI, Period, Volume Id, Proxy Server and Port EC2 Resource Amazon EC2 Service CloudWatch base URI, EC2 Base URI, Period, Instance  Id, Proxy Server and Port RDS Resource Amazon RDS Service CloudWatch base URI, RDS Base URI, Period, Instance  Id, Proxy Server and Port Proxy server and port are optional and are only needed if the agent is within the firewall. Here is an emcli example to add an EC2 target. Please read the Installation and Readme guide for more details and step-by-step instructions to deploy  the plugin and adding the AWS the instances. ./emcli add_target \       -name="<target name>" \       -type="AmazonEC2Service" \       -host="<host>" \       -properties="ProxyHost=<proxy server>;ProxyPort=<proxy port>;EC2_BaseURI=http://ec2.<region>.amazonaws.com;BaseURI=http://monitoring.<region>.amazonaws.com;InstanceId=<EC2 instance Id>;Period=<data point periond>"  \     -subseparator=properties="=" ./emcli set_monitoring_credential \                 -set_name="AWSKeyCredentialSet"  \                 -target_name="<target name>"  \                 -target_type="AmazonEC2Service" \                 -cred_type="AWSKeyCredential"  \                 -attributes="AccessKeyId:<access key id>;SecretKey:<secret key>" Emcli utility is found under the ORACLE_HOME of EM12C install. Once the instance is discovered, the target will show up under the ‘All Targets’ list under “Amazon EC2 Service’. Once the instances are added, one can navigate to the custom homepages for these resource types. The custom home pages not only include critical metrics, but also vital configuration parameters and incidents raised for these instances.  By mapping the configuration parameters as instance properties, we can slice-and-dice and group various AWS instance by leveraging the EM12C Config search feature. The following configuration properties and metrics are collected for these Resource types. Resource Type Configuration Properties Metrics EBS Resource Volume Id, Volume Type, Device Name, Size, Availability Zone Response: Status Utilization: QueueLength, IdleTime Volume Statistics: ReadBrandwith, WriteBandwidth, ReadThroughput, WriteThroughput Operation Statistics: ReadSize, WriteSize, ReadLatency, WriteLatency EC2 Resource Instance ID, Owner Id, Root Device type, Instance Type. Availability Zone Response: Status CPU Utilization: CPU Utilization Disk I/O:  DiskReadBytes, DiskWriteBytes, DiskReadOps, DiskWriteOps, DiskReadRate, DiskWriteRate, DiskIOThroughput, DiskReadOpsRate, DiskWriteOpsRate, DiskOperationThroughput Network I/O : NetworkIn, NetworkOut, NetworkInRate, NetworkOutRate, NetworkThroughput RDS Resource Instance ID, Database Engine Name, Database Engine Version, Database Instance Class, Allocated Storage Size, Availability Zone Response: Status Disk I/O:  ReadIOPS, WriteIOPS, ReadLatency, WriteLatency, ReadThroughput, WriteThroughput DB Utilization:  BinLogDiskUsage, CPUUtilization, DatabaseConnections, FreeableMemory, ReplicaLag, SwapUsage Custom Home Pages As mentioned above, we have custom home pages for these target types that include basic configuration information,  last 24 hours availability, top metrics and the incidents generated. Here are few snapshots. EBS Instance Home Page: EC2 Instance Home Page: RDS Instance Home Page: Further Reading: 1)      AWS Plugin download 2)      Installation and  Read Me. 3)      Screenwatch on SlideShare 4)      Extensibility Programmer's Guide 5)      Amazon Web Services

    Read the article

  • Alternative to Amazon's S3 service?

    - by Cory
    Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher. I don't mind prepaying or having monthly payment in order to reduce the bandwidth cost greatly. Thank you for any help

    Read the article

  • Amazon S3 bucket - download only certain files

    - by mottey
    Hi I have an Amazon S3 bucket with 10,000 images sitting in it with a standard naming convention: 001_small.jpg 001_large.jpg 002_small.jpg 002_large.jpg Because there are such a large amount of files I don't want to download ALL of them and I don't want to sit there for a couple of hours to select just the *_large.jpg files... Can someone suggest an S3 file manager that can let me select only the *_large.jpg files to download?? Thanks!

    Read the article

  • How to move S3 bucket to different location

    - by skrat
    We use S3 for storing millions of entries in our webapp, now we move the whole thing to EC2, EU servers, and we also want to move that S3 data to EU. But the bucket we use is in US, and there seem to be no tool to move whole bucket content to different bucket. There is also problem on how to synchronize the data later on when we switch to EU bucket, the data that will be created meanwhile while the migration was running.

    Read the article

  • What's your opinion in Amazon S3 ?

    - by Space Cracker
    i searching for best online file storage and i found a lot with different features ... i feel that Amazon S3 is the best ... Could any who try such a service give me his opinion on it and if is there any others that are most valuable Amazon S3 ?

    Read the article

  • Some questions about setting up Amazon S3 with Ruby on Rails

    - by ben
    I'm trying to setup Amazon S3 hosting with my Ruby on Rails 3 app, which is hosted on Heroku. After reading these instructions in the Heroku docs, I'm trying to use the aws-s3 gem. The instructions say to put the S3 account details in config/amazon_s3.yml, but the aws-s3 Github page says you create a connection like this: AWS::S3::Base.establish_connection!( :access_key_id => 'abc', :secret_access_key => '123' ) Why is the connection created by providing the details if they're already provided in the config file? Is that not the correct way to establish a connection? Do I have to establish a connection for each user everytime an upload is about to occur, or is a connection established for the application as a whole? Thanks for reading.

    Read the article

  • Uploading or writing to files contained within s3

    - by Chase
    We have a widget that needs to be hosted on s3 due to a bunch of hoopla that isn't relevant to the issue. I'd explain if it was worth the time. Problem is it calls an external XML file for part of its content. The XML file is accessible if it is local on s3 but I cannot get my php file to write to it on s3. The widget could also be useful if I could access the xml file elsewhere but s3 does not seem to permit this. Suggestions? Here's the error I'm getting when trying to write to S3. [function.fopen]: failed to open stream: HTTP wrapper does not support writeable connections.

    Read the article

  • php convert images and upload to amazon s3

    - by faraklit
    I am looking for a best practice while uploading images to amazon s3 server and serving from there. We need four different sizes of an image. So just after image upload we convert the image and scale in 4 different widths and heights. And then we send them to the amazon s3 using official php api. // ... // image conversions, bucket setting, s3 initialization etc. $sizes= array("", "48", "64", "128"); foreach($sizes as $size) { $filename = $upload_path.$dest_file.$size.$ext; $s3->batch()->create_object($bucket, , array( 'fileUpload' => $filename, 'acl' => AmazonS3::ACL_PUBLIC, )); } But for a 1M image the client sometimes wait up to 30 seconds which is a very long time. Instead of sending images immediately to S3, it may be better to add them to a job queue. But the user should see the uploaded image immediately.

    Read the article

  • Paperclip and Amazon S3 Issue

    - by Jimmy
    Hey everyone, I have a rails app running on Heroku. I am using paperclip for some simple image uploads for user avatars and some other things, I have S3 set as my backend and everything seems to be working fine except when trying to push to S3 I get the following error: The AWS Access Key Id you provided does not exist in our records. Thinking I mis-pasted my access key and secret key, I tried again, still no luck. Thinking maybe it was just a buggy key I deactivated it and generated a new one. Still no luck. Now for both keys I have used the S3 browser app on OS X and have been able to connect to each and view my current buckets and add/delete buckets. Is there something I should be looking out for? I have my application's S3 and paperclip setup like so development: bucket: (unique name) access_key_id: ENV['S3_KEY'] secret_access_key: ENV['S3_SECRET'] test: bucket: (unique name) access_key_id: ENV['S3_KEY'] secret_access_key: ENV['S3_SECRET'] production: bucket: (unique_name) access_key_id: ENV['S3_KEY'] secret_access_key: ENV['S3_SECRET'] has_attached_file :cover, :styles => { :thumb => "50x50" }, :storage => :s3, :s3_credentials => "#{RAILS_ROOT}/config/s3.yml", :path => ":class/:id/:style/:filename" Note: I just added the (unique name) bits, those aren't actually there--I have also verified bucket names, but I don't even think this is getting that far. I also have my heroku environment vars setup correctly and have them setup on dev

    Read the article

  • Amazon S3 and swfaddress

    - by justinbach
    I recently migrated a large AS3 site (lots of swfs, lots of flvs) to Amazon S3. Pretty much everything but HTML and JS files is being stored/served from Amazon, and it's working well. The only problem I'm having is that I built the site using SWFaddress (actually, via the Gaia framework which uses SWFaddress), and for some reason, SWFaddress is no longer updating the address bar correctly as users navigate from page to page. In other words, the URL persistently remains http://www.mysite.com, not http://www.mysite.com/#/section as would be the case were SWFaddress functioning correctly (and as it was functioning prior to the migration). Stranger yet, if I go to (e.g.) http://www.mysite.com/#/section directly, the deeplinking functions as you'd expect--I arrive directly at the correct section. However, navigating away from that section doesn't have any effect on the address bar, despite the fact that it should be dynamically updated. I've got a crossdomain.xml file set up on the site that allows access from all domains, so that's not the issue, and I don't know what else might be. Any ideas would be greatly appreciated! P.S. I integrated S3 by putting pretty much the entire site in an S3 bucket and then just changing the initial swfobject embed to point to the S3 instance of main.swf, passing in the S3 path as the "base" param to the embedded swf so that all dynamically loaded assets and swfs would also be sourced from s3. Dunno if that's related to the troubles I'm having.

    Read the article

  • S3 Backup Memory Usage in Python

    - by danpalmer
    I currently use WebFaction for my hosting with the basic package that gives us 80MB of RAM. This is more than adequate for our needs at the moment, apart from our backups. We do our own backups to S3 once a day. The backup process is this: dump the database, tar.gz all the files into one backup named with the correct date of the backup, upload to S3 using the python library provided by Amazon. Unfortunately, it appears (although I don't know this for certain) that either my code for reading the file or the S3 code is loading the entire file in to memory. As the file is approximately 320MB (for today's backup) it is using about 320MB just for the backup. This causes WebFaction to quit all our processes meaning the backup doesn't happen and our site goes down. So this is the question: Is there any way to not load the whole file in to memory, or are there any other python S3 libraries that are much better with RAM usage. Ideally it needs to be about 60MB at the most! If this can't be done, how can I split the file and upload separate parts? Thanks for your help. This is the section of code (in my backup script) that caused the processes to be quit: filedata = open(filename, 'rb').read() content_type = mimetypes.guess_type(filename)[0] if not content_type: content_type = 'text/plain' print 'Uploading to S3...' response = connection.put(BUCKET_NAME, 'daily/%s' % filename, S3.S3Object(filedata), {'x-amz-acl': 'public-read', 'Content-Type': content_type})

    Read the article

  • Using .htaccess to server files from Amazon S3 CloudFront

    - by Adrian A.
    My ideal setup would be to take a current clients site, upload a .htaccess with a regex inside, that would match the URI, and if it finds a certain file extension, it would use the same path, but with an altered domain. ie. Normal path: http://www.domain.com/something/images/someimage.jpeg http://www.domain.com/assets/js/jquery.js .htaccess translated would turn the above into: http://mycdn.other.com/something/images/someimage.jpeg http://mycdn.other.com/assets/js/jquery.js I googled this for hours in a row, no luck. Again, this is for actually making use of Amazon's CloudFront. S3 is already mounted to the website for backups and storing files using s3fs, but this doesn't solve the issue since it's using S3 directly, not using the CloudFront.

    Read the article

  • How can I send SMS from my PC using my Android Phone (Galaxy S3)

    - by Dean Rather
    I've been a long-time Nokia fan, but finally succumbed to the Samsung Galaxy S3's amazing screen, & Android's Open community. One feature I use a lot (about a hundred times a day...) is Nokia PC Suite's ability to send SMS via the phone. It: Uses my phone's contact list. Notifies me when I get an sms. Allows me to send SMS using my PC's keyboard. Essentially turning SMS into just another IM medium I use on the PC. How will I do this with the Samsung Galaxy S3 + Windows 7?

    Read the article

  • Sync local files to S3 similar to Robocopy

    - by Yuck
    I am looking for a way to synchronize an entire local folder structure to Amazon S3, similar to how one might synchronize two folders using Robocopy. Whatever solution I come up with needs to be scheduled to run periodically from the Windows Task Scheduler. So anything that requires a GUI to perform the synchronization is not a viable solution. Standalone Windows .EXE command line utility for Amazon S3 & EC2 looked promising, but seems to have been abandoned and would not work when I tried to use it. Possibly a difference in the way that security is handled now compared to that software's most recent release.

    Read the article

  • Move files from ftp server to s3

    - by lev
    I would like to set up an ftp server, where users will upload files, and for each file, put it on s3 storage, and delete it from the ftp server. (the server runs on ec2 ubuntu) Here are the stuff I already tried, with no success.. Mount s3 bucket using s3fs. I followed those instructions, but there is a bug in the latest version of s3fs, that prevents it from working. The bug was fixed on the develop branch, but I don't want to use unstable version on my production. Use vsftpd and using s3cmd sync via cron to sync the files periodically. The problem with that approach, is that s3cmd can start running in the middle of a file upload, and start synching the incomplete file. Also s3cmd doesn't give any feedback it the sync fails, so I have no way of knowing if I can delete the files after the sync command finished running. Use pure-ftpd's upload script feature (which allows to run a script after a file is finished uploading), but I noticed that if the file upload was failed in the middle, the script will run anyway, and I have no way of knowing if the upload was successful or not. I've been at it for a few days now, and I'm at a loss here. Any suggestions will be welcomed.

    Read the article

  • How to use SSL on AWS EC2

    - by Aubada Taljo
    Hello I have an AWS EC2 account and I am running an instance that serves as a web host for my PHP website... This is a private website that has no UI but only URLs to be requested by my other software to get some response from the server... I want the requests (that I send to the server) to be secured so I want to use https instead of http... so what should I do to achieve that? PS: I found this link while searching... but I don't know how useful it's in my situation http://matt-darby.com/posts/690-aws-ec2-and-ssl Thanks in advance Good luck

    Read the article

  • Do you I think I should integrate something like AWS or other cloud service from the initial phases of my project?

    - by Kareem Ergawy
    Do you I think I should integrate something like AWS or other cloud service from the initial phases of my project or I should be working on the front and back end components regularly and integrate AWS later? I am starting to work on a mobile service. From day one, I wish to make sure that my service will be scalable and able to handle large loads of requests. This is my first time in architecting a large scale system from the beginning so I can't decide what is best.

    Read the article

  • Controlling number of downloads on Amazon S3

    - by m7d
    Is there a way to control number of downloads of digital content on Amazon S3 or via some middle man software that talks to S3? I already use their timed links, but I would like to control number of downloads also. Any ideas of how to accomplish this using S3 or suggestions about alternative services that could? Thanks!

    Read the article

  • Serving GZipped files from s3 using the Asset Pipeline

    - by kmurph79
    I have a Rails 3.2.3 app on Heroku and I'm using the asset_sync gem to serve my assets from s3, via these instructions. It works great, except s3 is not serving up the gzipped css/js files (just the uncompressed version). I've enabled gzip compression, to no avail: config.gzip_compression = true According to Using GZIP with html pages served from Amazon S3 I need to add meta-data to the s3 object for uploading. How would I do this in concert with the Asset Pipeline? Thank you for any help.

    Read the article

  • Reading in gzipped data from S3 in Ruby

    - by Evan Zamir
    My company has data messages (json) stored in gzipped files on Amazon S3. I want to use Ruby to iterate through the files and do some analytics. I started to use the 'aws/s3' gem, and get get each file as an object: #<AWS::S3::S3Object:0x4xxx4760 '/my.company.archive/data/msg/20131030093336.json.gz'> But once I have this object, I do not know how to unzip it or even access the data inside of it.

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >