Search Results

Search found 2978 results on 120 pages for 'amazon aws'.

Page 23/120 | < Previous Page | 19 20 21 22 23 24 25 26 27 28 29 30  | Next Page >

• "Permission denied (publickey)" error when ssh'ing to Amazon EC2 Debian AMI 05f3ef71

  - by user193537

  I have launched a Debian system using AMI 05f3ef71 in Amazon EC2, but I have no lock connecting to it using SSH as suggested in "Connecting to Your Linux/UNIX Instances Using SSH". I tried several user names: ec2-user, root, debian... None of them worked. I always get a Permission denied (publickey) error message. Using ec2-get-console-output instance_id as suggested doesn't work either, it requires option "-K". If I supply it, I get the error message Required option '-C, --cert CERT' missing, but I have no idea what to supply there. Port 22 is opened on the affected instance. Does anyone have an idea what I could try to log in to my instance?

  Read the article

• Amazon EC2 as load balanced/failover solution

  - by sugiggs

  Hi All, I'm thinking of an idea but not sure the pros/cons of it. At the moment, we are hosting our website on a dedicated server. As a failover/load balanced solution, I'm thinking to use Amazon EC2+EBS. The files can be rsync and mysql can be setup as master-master replication When the load is high, I can up the machine, given sometime to "sync" and load balanced the traffic there. is it do-able? any link I can read more on this?

  Read the article

• Amazon EC2 - Reserved Private Addresses

  - by reach4thelasers

  I've got a private DNS Server running on Amazon EC2. I don't need a public IP Address because its only used for private addressing web1.xxx.internal database1.xxx.internal Problem is I had to terminate the instance recently and start a new one. This meant that the private IP address of the DNS server changed and I had to log in to each of my other 15 server one-by-one and change the DNS address to point to the new DNS server. There must be a better way to do this, if so, what is it?

  Read the article

• Mounting an Amazon EC2 instance on Mac OS X

  - by hinghoo

  I've got public key authentication working between my Mac OS X and an Amazon EC2 instance so that from the command-line I can just type the following and it works: ssh root@[IPAddressOfEC2Instance] The strange thing is that I can't seem to mount the instance using "Connect to Server" in the Finder. I've tried typing the following server addresses into the "Connect to Server" dialog: ftps://[IPAddressOfEC2Instance] ftps://root@[IPAddressOfEC2Instance] But all I get is You entered an invalid username or password. Please try again. The root user on the EC2 instance has a blank password and I'm wondering if it has to do with that. However, I can't change the password for the root user. I can use an SFTP client to connect to the machine, I just can't mount it with "Connect to server". It asks for a username and password (for a registered user) and it's root/[blank] which it doesn't accept. The other option is "Guest" which brings up an empty folder in the Finder.

  Read the article

• Use Amazon EC2 as a backup server

  - by MikeMurko

  I would like to use Amazon EC2 as an emergency backup database+web server in the event our primary host becomes unavailable. I feel like I wouldn't have trouble setting up a Windows instance, install SQL Server and get the web server up and running (would take a few hours, plus installing various libraries, our source code, etc). My question relates to pricing. If I simply "stop" the instance rather than "terminate" it, does that stop counting "instance-hours"? I would prefer not to terminate the instance and lose all that work I spent setting it up. If I must "terminate" in order to stop the billing - is it possible to make an image of the server after I have set it all up, then save that image somewhere (S3?) Is this something that people do regularly? Ideally this instance would just be waiting in the wings for an issue with our host, but costing us nothing except perhaps data storage costs.

  Read the article

• Amazon S3 - Storage Class and Server Side Encryption

  - by Steven

  Ahhh! I am using Amazon S3 for some low price storage to clear down out SAN. I created the bucket and created a root folder. I set the storage class to standard and server side encryption AES. I started a copy job to move the files, some files copied over and i checked the files: Reduced Redundancy Encryption set to none WTF? So i deleted all files and folders. I manuallyed created the folder structure and again set the storage class and encryption level. I coped some files and bamm, still showing (at a file level as Reduced and no encryption). So my question is this, is it really raid'd and encrypted just not showing it properly (as the root folder is, how can the file not be??) or (b) am i being a huge tool and missing something?

  Read the article

• Is there a way of using HTTPS with Amazon's CloudFront CDN and CNAMEs?

  - by Metalshark

  We use Amazon's CloudFront CDN with custom CNAMEs hanging under the main domain (static1.example.com). Although we can break this uniform appearance and use the original whatever123wigglyw00.cloudfront.net URLs to utilise HTTPS, is there another way? Do Amazon or any other similar provider offer HTTPS CDN hosting? Is TLS and its selective encryption available for use somewhere (SNI: Server Name Indication)? Foot note: assuming that the answer is no, but just in the hope someone knows. EDIT: Now using Google App Engine https://developers.google.com/appengine/docs/ssl for CDN hosting with SSL support.

  Read the article

• can not connect to SQL running on amazon ec2 machine

  - by njj56

  I am using SQL managment studio 2008 running on an Amazon EC2 machine. I am unable to connect to the database in my asp.net application. The EC2 instance has been set to accept connections over the SQL port. I am also able to remote the machine as well as view websites hosted on the server. Listed below is part of the connection string relating to this instance. When the program is ran and this connection string is called, it returns tcp error 0 - no return response. it just times out. <add name="ProjectServer" connectionString="Data Source=*IP ADDRESS HERE*,1433;Initial Catalog=*Catalog Name*;User ID=IP-0A6ED514\Administrator;"/> I removed the ip and the catalog name for the example, but I am sure they are correct. The only thing that I could think may cause an error, is the differences in names between the user id and the server name - the server name is ip-0A6ED514\sharepoint but the user name is ip-0A6ED514\administrator when I log into the sql server manager on the EC2 instance. A password is not used. Not sure if I would need to leave in a blank string for password - also not sure if the difference between server name and user id to log in makes a difference. Any help is appreciated. Thank you. update - when this connection string is used with out the port, i get tcp provider error 40 - when the port is in there, i get error 0 edit- the sql server is using windows authentication - does this make a difference? Usually I always use SQL server authentication

  Read the article

• Enabling mod_rewrite on Amazon Linux

  - by L. De Leo

  I'm trying to enable mod_rewrite on an Amazon Linux instance. My Directory directives look like this: <Directory /> Order deny,allow Allow from all Options None AllowOverride None </Directory> <Directory "/var/www/vhosts"> Order allow,deny Allow from all Options None AllowOverride All </Directory> And then further down in httpd.conf I have the LoadModule directive: ... other modules... #LoadModule substitute_module modules/mod_substitute.so LoadModule rewrite_module modules/mod_rewrite.so #LoadModule proxy_module modules/mod_proxy.so ... other modules... I have commented out all the Apache modules not needed by Wordpress. Still when I issue http restart and then check the loaded modules with /usr/sbin/httpd -l I get only: [root@foobar]# /usr/sbin/httpd -l Compiled in modules: core.c prefork.c http_core.c mod_so.c Inside the virtual host containing the Wordpress site I have an .htaccess containing: # BEGIN WordPress <IfModule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </IfModule> # END WordPress The .htaccess is owned by apache which is the user apache runs under. The apachectl -t command returns Syntax OK What am I doing wrong? What should I check?

  Read the article

• Hadoop or Hadoop Streaming for MapReduce on AWS

  - by aeolist

  I'm about to start a mapreduce project which will run on AWS and I am presented with a choice, to either use Java or C++. I understand that writing the project in Java would make more functionality available to me, however C++ could pull it off too, through Hadoop Streaming. Mind you, I have little background in either language. A similar project has been done in C++ and the code is available to me. So my question: is this extra functionality available through AWS or is it only relevant if you have more control over the cloud? Is there anything else I should bear in mind in order to make a decision, like availability of plugins for hadoop that work better with one language or the other? Thanks in advance

  Read the article

• Amazon VPC NAT not working

  - by rpkelly

  I'm trying to create a NAT instance for my VPC to allow instances on private subnets connect to the internet (most importantly, S3). I tried following the instructions here: http://docs.amazonwebservices.com/AmazonVPC/2011-07-15/UserGuide/index.html?VPC_NAT_Instance.html . Unfortunately, the instances in the private subnet (call it 10.10.2.0/24) cannot reach the internet. I have done the following: Create a NAT instance (Amazon's ami-vpc-nat-1.0.0-beta.i386-ebs (ami-d8699bb1)) in public subnet (call it 10.10.1.0/24). Changed "Source / Dest Check" to disabled. Created a new entry in the default routing table (which is used by 10.10.2.0/24) and had it point to the ID of the newly created instance. Associated an Elastic IP address with the NAT instance. Allowed all outbound traffic on the security group of the NAT instance. Ensured that all traffic could pass between the two subnets. I've tried also doing this with an existing instance using iptables, but had no luck. And I have verified that sys.net.ipv4.ip_forward is 1, just in case anyone was wondering. And I still have no internet connectivity from the instances on 10.10.2.0/24. Does anyone have any suggestions?

  Read the article

• Serving and caching content from Amazon S3 with Tomcat

  - by Rob

  Hi all, We're looking to serve a range of content using Amazon S3 as a store for the content and Tomcat to host the web application. The content is divided into free and paid for content. We intend to authenticate the users when they access the web application running in Tomcat. Based around their authentication we are able to tell if the user has access to paid for content or simply free stuff. So I envision the flow of a request being something like this: Authenticated request to Tomcat If user is "paid" user, display links to premium content Direct requests for paid content back through Tomcat to prevent direct access to it by non-paying users. Tomcat makes request to S3 through a web cache to keep our costs down Content is returned to user. As we have to pay for each request to S3, I'd ideally like to cache content locally to the Tomcat instance after it has been requested for the first time to keep costs to a minimum and to speed things up. I would also like to be able to invalidate this cache if we publish fresh content to S3. So to confirm my proposal: Client Request - Tomcat - Web Cache - S3 To invalidate the cache, I was thinking of using something like PubSubHubbub with the cache waiting for updates to the feed for content that it should invalidate. I'd appreciate some general feedback on this approach as I've no real experience of caching and I'm sure I've made some invalid assumptions. I'd also appreciate any recommendations for caching technologies. Thanks.

  Read the article

• Managing persistent data on an Amazon EC2 web server

  - by Derek

  I've just started trying out Amazon's EC2 service for running an asp.net web app which uses a SQL Server 2005 Express database. I have some questions about how to configure and operate it best for reliability, and I'm hoping to tap into some collective wisdom here as this is my first foray into EC2. Here's how I have it configured currently: OS: Windows 2003 SQL Server Express 2005 Web content stored on an EBS Volume (E Drive) Database Data stored on an EBS Volume (E Drive) Database backups to "C Drive" and then copied off to S3. Elastic IP Address attached to the production instance. Now when I make a change to the OS configuration, I make a new AMI using the bundle feature. Unfortunately, I found that this results in significant downtime. While the bundle is created and the new instance is started. It seems that when I'm ready to make a new AMI, I should: Start up a new temporary instance. Detach the EBS volume from the production instance. Detach the IP Address from the production instance. Attach the IP Address to the temporary instance. Attach the EBS volume to the temporary instance. Create an AMI from the production instance. After the production instance restarts, reverse the attach/detach steps to put it back in production. Is this the right order of events to prevent any chance to corrupt the EBS volume? Will the EBS volume become corrupt if I detach it while a database Write is taking place? Should I snapshot the EBS volume of the production instance and attach it to the temporary instance instead? Or could taking a snapshot of the EBS volume while it's in use cause corruption? Any suggestions to improve the reliability and operations?

  Read the article

• Ruby: Streaming large AWS S3 object freezes

  - by Peter

  Hi, I am using the ruby aws/s3 library to retrieve files from Amazon S3. I stream an object and write it to file as per the documentation (with debug every 100 chunks to confirm progress) This works for small files, but randomly freezes downloading large (150MB) files on VPS Ubuntu. Fetching the same files (150MB) from my mac on a much slower connection works just fine. When it hangs there is no error thrown and the last line of debug output is the 'Finished chunk'. I've seen it write between 100 and 10,000 chunks before freezing. Anyone come across this or have ideas on what the cause might be? Thanks The code that hangs: i=1 open(local_file, 'w') do |f| AWS::S3::S3Object.value(key, @s3_bucket) do |chunk| puts("Writing chunk #{i}") f.write chunk.read_body puts("Finished chunk #{i}") i=i+1 end end

  Read the article

• beanstalk using php-git on windows client

  - by ntidote

  I am trying to install beanstalk for php using git. I am using a Windows Client machine. I am done with the prerequisite installations , credentials setup. I am following the link http://docs.amazonwebservices.com/elasticbeanstalk/latest/dg/create_deploy_PHP.sdlc.html The following step does not workout (i use git bash for git related commands) From your Git repository directory, type the following command. git aws.config This gives the error git : 'aws.config' is not a git command. Please suggest how to deal with the issue.

  Read the article

• Output on namespaced xpath in java

  - by user347928

  I have the following code and have had some trouble with a specific field and it's output. The namespace is connected but doesn't seem to be outputting on the required field. Any info on this would be great. import org.w3c.dom.Document; import org.xml.sax.SAXException; import javax.xml.parsers.DocumentBuilderFactory; import javax.xml.parsers.DocumentBuilder; import javax.xml.parsers.ParserConfigurationException; import javax.xml.xpath.XPathFactory; import javax.xml.xpath.XPath; import javax.xml.xpath.XPathExpressionException; import java.io.ByteArrayInputStream; import java.io.IOException; public class test { public static void main(String args[]) { String xmlStr = "<aws:UrlInfoResponse xmlns:aws=\"http://alexa.amazonaws.com/doc/2005-10-05/\">\n" + " <aws:Response xmlns:aws=\"http://awis.amazonaws.com/doc/2005-07-11\">\n" + " <aws:OperationRequest>\n" + " <aws:RequestId>blah</aws:RequestId>\n" + " </aws:OperationRequest>\n" + " <aws:UrlInfoResult>\n" + " <aws:Alexa>\n" + " <aws:TrafficData>\n" + " <aws:DataUrl type=\"canonical\">harvard.edu/</aws:DataUrl>\n" + " <aws:Rank>1635</aws:Rank>\n" + " </aws:TrafficData>\n" + " </aws:Alexa>\n" + " </aws:UrlInfoResult>\n" + " <aws:ResponseStatus xmlns:aws=\"http://alexa.amazonaws.com/doc/2005-10-05/\">\n" + " <aws:StatusCode>Success</aws:StatusCode>\n" + " </aws:ResponseStatus>\n" + " </aws:Response>\n" + "</aws:UrlInfoResponse>"; DocumentBuilderFactory xmlFact = DocumentBuilderFactory.newInstance(); xmlFact.setNamespaceAware(true); DocumentBuilder builder = null; try { builder = xmlFact.newDocumentBuilder(); } catch (ParserConfigurationException e) { e.printStackTrace(); } Document doc = null; try { doc = builder.parse( new ByteArrayInputStream( xmlStr.getBytes())); } catch (SAXException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } System.out.println(doc.getDocumentElement().getNamespaceURI()); System.out.println(xmlFact.isNamespaceAware()); String xpathStr = "//aws:OperationRequest"; XPathFactory xpathFact = XPathFactory.newInstance(); XPath xpath = xpathFact.newXPath(); String result = null; try { result = xpath.evaluate(xpathStr, doc); } catch (XPathExpressionException e) { e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates. } System.out.println("XPath result is \"" + result + "\""); } }

  Read the article

• Output on namespaced xpath

  - by user347928

  Hi there, I have the following code and have had some trouble with a specific field and it's output. The namespace is connected but doesn't seem to be outputting on the required field. Any info on this would be great. import org.w3c.dom.Document; import org.xml.sax.SAXException; import javax.xml.parsers.DocumentBuilderFactory; import javax.xml.parsers.DocumentBuilder; import javax.xml.parsers.ParserConfigurationException; import javax.xml.xpath.XPathFactory; import javax.xml.xpath.XPath; import javax.xml.xpath.XPathExpressionException; import java.io.ByteArrayInputStream; import java.io.IOException; public class test { public static void main(String args[]) { String xmlStr = "<aws:UrlInfoResponse xmlns:aws=\"http://alexa.amazonaws.com/doc/2005-10-05/\">\n" + " <aws:Response xmlns:aws=\"http://awis.amazonaws.com/doc/2005-07-11\">\n" + " <aws:OperationRequest>\n" + " <aws:RequestId>blah</aws:RequestId>\n" + " </aws:OperationRequest>\n" + " <aws:UrlInfoResult>\n" + " <aws:Alexa>\n" + " <aws:TrafficData>\n" + " <aws:DataUrl type=\"canonical\">harvard.edu/</aws:DataUrl>\n" + " <aws:Rank>1635</aws:Rank>\n" + " </aws:TrafficData>\n" + " </aws:Alexa>\n" + " </aws:UrlInfoResult>\n" + " <aws:ResponseStatus xmlns:aws=\"http://alexa.amazonaws.com/doc/2005-10-05/\">\n" + " <aws:StatusCode>Success</aws:StatusCode>\n" + " </aws:ResponseStatus>\n" + " </aws:Response>\n" + "</aws:UrlInfoResponse>"; DocumentBuilderFactory xmlFact = DocumentBuilderFactory.newInstance(); xmlFact.setNamespaceAware(true); DocumentBuilder builder = null; try { builder = xmlFact.newDocumentBuilder(); } catch (ParserConfigurationException e) { e.printStackTrace(); } Document doc = null; try { doc = builder.parse( new ByteArrayInputStream( xmlStr.getBytes())); } catch (SAXException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } System.out.println(doc.getDocumentElement().getNamespaceURI()); System.out.println(xmlFact.isNamespaceAware()); String xpathStr = "//aws:OperationRequest"; XPathFactory xpathFact = XPathFactory.newInstance(); XPath xpath = xpathFact.newXPath(); String result = null; try { result = xpath.evaluate(xpathStr, doc); } catch (XPathExpressionException e) { e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates. } System.out.println("XPath result is \"" + result + "\""); } } Thanks Tony

  Read the article

• How to implement message queuing and handling in AWS with NServiceBus

  - by Pete Lunenfeld

  I am creating a new ASP MVC order application in the Amazon (AWS) cloud with the persistence layer at my local datacenter. I will be using the CQRS pattern. The goal of the project is high availability using Queue(s) to store and forward writes (commands/events) that can be picked up and handled asynchronously at my local datacenter. Then, ff the WAN or my local datacenter fails, my cloud MVC app can still take orders and just queue them up until processing can resume. My first thought was to use AWS SQS for the queuing and create my own queue consumer/dispatcher/handler in my own c# application to process the incoming messages/events. MVC (@ Amazon) -- Event/POCO -- SQS -- QueueReader (@ my datacenter) -- DB Then I found NServiceBus. NSB seems to handle lots of details very nicely: message handling, retries, error handling, etc. I hate to reinvent the wheel, and NServiceBus seems like a full featured and mature product that would be perfect for me. But on further research, it does NOT look like NServiceBus is really meant to be used over the WAN in physically separated environments (Cloud to my Datacenter). Google and SO don't really paint a good picture of using NServiceBus across the WAN like I need. How can I use NServiceBus across the WAN? Or is there a better solution to handle queuing and message handling between Amazon an my local datacenter?

  Read the article

• Integrating Amazon S3 in Java via NetBeans IDE

  - by Geertjan

  To continue from yesterday, let's set up a scenario that enables us to make use of this drag/drop service in NetBeans IDE: The above service is applicable to Amazon S3, an Amazon storage provider that is typically used to store large binary files. In Amazon S3, every object stored is contained in a bucket. Buckets partition the namespace of objects stored in Amazon S3. More on buckets here. Let's use the tools in NetBeans IDE to create a Java application that accesses our Amazon S3 buckets. Create a Java application named "AmazonBuckets" with a main class named "AmazonBuckets". Open the main class and then drag the above service into the main method of the class. Now, NetBeans IDE will create all the other classes and the properties file that you see in the screenshot below. The first thing to do is to open the properties file above and enter the access key and secret: access_key=SOMETHINGsecret=SOMETHINGELSE Now you're all set up. Make sure to, of course, actually have some buckets available: Then rewrite the Java class to parse the XML that is returned via the generated code: package amazonbuckets;import java.io.ByteArrayInputStream;import java.io.IOException;import javax.xml.parsers.DocumentBuilder;import javax.xml.parsers.DocumentBuilderFactory;import javax.xml.parsers.ParserConfigurationException;import org.netbeans.saas.amazon.AmazonS3Service;import org.netbeans.saas.RestResponse;import org.w3c.dom.DOMException;import org.w3c.dom.Document;import org.w3c.dom.Node;import org.w3c.dom.NodeList;import org.xml.sax.InputSource;import org.xml.sax.SAXException;public class AmazonBuckets {    public static void main(String[] args) {        try {            RestResponse result = AmazonS3Service.getBuckets();            String dataAsString = result.getDataAsString();            DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();            DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();            Document doc = dBuilder.parse(                    new InputSource(new ByteArrayInputStream(dataAsString.getBytes("utf-8"))));            NodeList bucketList = doc.getElementsByTagName("Bucket");            for (int i = 0; i < bucketList.getLength(); i++) {                Node node = bucketList.item(i);                System.out.println("Bucket Name: " + node.getFirstChild().getTextContent());            }        } catch (IOException | ParserConfigurationException | SAXException | DOMException ex) {        }    }}That's all. This is simpler to setup than the scenario described yesterday. Also notice that there are other Amazon S3 services you can interact with from your Java code, again after generating a heap of code after drag/drop into a Java source file: I tried the above, e.g., I created a new Amazon S3 bucket after dragging "createBucket", adding my credentials in the properties file, and then running the code that had been created. I.e., without adding a single line of code I was able to programmatically create new buckets. The above outlines a handy set of tools and techniques to use if you want to let your users store and access data in Amazon S3 buckets directly from the application you've created for them.

  Read the article

• SQL Express 2008 R2 on Amazon EC2 instance: tons of free memory, poor performance

  - by gravyface

  The old SQL Express 2005 was running on a low-end single Xeon CPU Dell server, RAID 5 7200 disks, 2 GB RAM (SBS 2003). I have not done any baseline measurements on the old physical server, but the Web app is used by half a dozen people (maybe 2 concurrently), so I figured "how bad can an Amazon EC2 instance be?". It's pretty horrible: a difference of 8 seconds of load time on one screen. First of all, I'm not a SQL guru, but here's what I've tried: Had a Small Instance, now running a c1.medium (High Cpu Medium) Windows 2008 32-bit R2 EBS-backed instance running IIS 7.5 and SQL Express 2008 R2. No noticeable improvement. Changed Page File from fixed 256 to Automatic. Setup a Striped Mirror from within Disk Management with two attached 1 GB EBS volumes. Moved database and transaction log, left everything else on the boot EBS volume. No noticeable change. Looked at memory, ~1000 MB of physical memory free (1.7 GB total). Changed SQL instance to use a minimum of 1024 RAM; restarted server, no change in memory usage. SQL still only using ~28MB of RAM(!). So I'm thinking: this database is tiny (28MB), why isn't the whole thing cached in RAM? Surely that would speed up performance. The transaction log is 241 MB. Seems kind of large in comparison -- has this not been committed? Is it a cause of performance degradation? I recall something about Recovery Models and log sizes somewhere in my travels, but not positive. Another thing: the old server was running SQL Express 2005. Not sure if that has any impact, but I tried changing the compatibility level from SQL 2000 to 2008, but that had no effect. Anyways, what else can I try here? Seems ridiculous to throw more virtual hardware at this thing. I know I/O is going to be rough on EBS volumes, but surely others are successfully running small .NET/SQL apps on reasonably priced instances?

  Read the article

• amazon web services

  - by user285825

  Hi, Has anyone of you worked on amazon web services? I just wanted to retrieve the citations of a given book. Is this possible using the aws? Is aws free? Thanks.

  Read the article

• Accessing Amazon E-Commerce Services through Cocoa Touch

  - by Gregory Hill

  Has anyone successfully accessed AWS (Amazon E-Commerce Services) through Cocoa Touch? I've been digging around, and haven't seen the issue addressed directly. I've created an AWS account, and have seen some sample code for accessing web services through Cocoa, but I'm not quite sure how to tie it all together. If anyone has any sample code, I'd love to see it. I'm just trying to get my head around the concepts.

  Read the article

• simple and reliable centralized logging inside Amazon VPC

  - by Nakedible

  I need to set up centralized logging for a set of servers (10-20) in an Amazon VPC. The logging should be as to not lose any log messages in case any single server goes offline - or in the case that an entire availability zone goes offline. It should also tolerate packet loss and other normal network conditions without losing or duplicating messages. It should store the messages durably, at the minimum on two different EBS volumes in two availability zones, but S3 is a good place as well. It should also be realtime so that the messages arrive within seconds of their generation to two different availability zones. I also need to sync logfiles not generated via syslog, so a syslog-only centralized logging solution would not fulfill all the needs, although I guess that limitation could be worked around. I have already reviewed a few solutions, and I will list them here: Flume to Flume to S3: I could set up two logservers as Flume hosts which would store log messages either locally or in S3, and configure all the servers with Flume to send all messages to both servers, using the end-to-end reliability options. That way the loss of a single server shouldn't cause lost messages and all messages would arrive in two availability zones in realtime. However, there would need to be some way to join the logs of the two servers, deduplicating all the messages delivered to both. This could be done by adding a unique id on the sending side to each message and then write some manual deduplication runs on the logfiles. I haven't found an easy solution to the duplication problem. Logstash to Logstash to ElasticSearch: I could install Logstash on the servers and have them deliver to a central server via AMQP, with the durability options turned on. However, for this to work I would need to use some of the clustering capable AMQP implementations, or fan out the deliver just as in the Flume case. AMQP seems to be a yet another moving part with several implementations and no real guidance on what works best this sort of setup. And I'm not entirely convinced that I could get actual end-to-end durability from logstash to elasticsearch, assuming crashing servers in between. The fan-out solutions run in to the deduplication problem again. The best solution that would seem to handle all the cases, would be Beetle, which seems to provide high availability and deduplication via a redis store. However, I haven't seen any guidance on how to set this up with Logstash and Redis is one more moving part again for something that shouldn't be terribly difficult. Logstash to ElasticSearch: I could run Logstash on all the servers, have all the filtering and processing rules in the servers themselves and just have them log directly to a removet ElasticSearch server. I think this should bring me reliable logging and I can use the ElasticSearch clustering features to share the database transparently. However, I am not sure if the setup actually survives Logstash restarts and intermittent network problems without duplicating messages in a failover case or similar. But this approach sounds pretty promising. rsync: I could just rsync all the relevant log files to two different servers. The reliability aspect should be perfect here, as the files should be identical to the source files after a sync is done. However, doing an rsync several times per second doesn't sound fun. Also, I need the logs to be untamperable after they have been sent, so the rsyncs would need to be in append-only mode. And log rotations mess things up unless I'm careful. rsyslog with RELP: I could set up rsyslog to send messages to two remote hosts via RELP and have a local queue to store the messages. There is the deduplication problem again, and RELP itself might also duplicate some messages. However, this would only handle the things that log via syslog. None of these solutions seem terribly good, and they have many unknowns still, so I am asking for more information here from people who have set up centralized reliable logging as to what are the best tools to achieve that goal.

  Read the article

• DNS with name.com and Amazon S3

  - by aledalgrande

  I have a website on a bucket in Amazon S3, and recently started to get emails from Google "Googlebot can't access your site". When I go to Webmaster Tools and I try to fetch in fact it doesn't work. Also people in locations different from mine sometimes reported they could not access the website. Now for curiosity I tried from my terminal: $ host xxx xxx is an alias for xxx.s3-website-us-west-1.amazonaws.com. xxx.s3-website-us-west-1.amazonaws.com is an alias for s3-website-us-west-1.amazonaws.com. s3-website-us-west-1.amazonaws.com has address yyy.yyy.yyy.yyy And when I try with dig: $ dig xxx ; <<>> DiG 9.8.3-P1 <<>> xxx ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17860 ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;xxx. IN A ;; ANSWER SECTION: xxx. 300 IN CNAME xxx.s3-website-us-west-1.amazonaws.com. xxx.s3-website-us-west-1.amazonaws.com. 60 IN CNAME s3-website-us-west-1.amazonaws.com. s3-website-us-west-1.amazonaws.com. 60 IN A yyy ;; Query time: 1514 msec ;; SERVER: 75.75.75.75#53(75.75.75.75) ;; WHEN: Fri Aug 22 12:32:13 2014 ;; MSG SIZE rcvd: 127 It seems OK to me. Why would Google tell me there is a DNS error? UPDATE: Google also cannot fetch robots.txt, but I can fetch it from my browser. UPDATE 2: I have a forwarding on the root to the www.* hostname: $ dig thenifty.me ; <<>> DiG 9.8.3-P1 <<>> thenifty.me ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49286 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 ;; QUESTION SECTION: ;thenifty.me. IN A ;; AUTHORITY SECTION: thenifty.me. 300 IN SOA ns1hwy.name.com. support.name.com. 1 10800 3600 604800 300 ;; Query time: 148 msec ;; SERVER: 75.75.75.75#53(75.75.75.75) ;; WHEN: Fri Aug 22 13:32:56 2014 ;; MSG SIZE rcvd: 88

  Read the article

• Amazon API ItemSearch returns (400) Bad Request.

  - by BuzzBubba

  I'm using a simple example from Amazon documentation for ItemSearch and I get a strange error: "The remote server returned an unexpected response: (400) Bad Request." This is the code: public static void Main() { //Remember to create an instance of the amazon service, including you Access ID. AWSECommerceServicePortTypeClient service = new AWSECommerceServicePortTypeClient(new BasicHttpBinding(), new EndpointAddress( "http://webservices.amazon.com/onca/soap?Service=AWSECommerceService")); AWSECommerceServicePortTypeClient client = new AWSECommerceServicePortTypeClient( new BasicHttpBinding(), new EndpointAddress("http://webservices.amazon.com/onca/soap?Service=AWSECommerceService")); // prepare an ItemSearch request ItemSearchRequest request = new ItemSearchRequest(); request.SearchIndex = "Books"; request.Title = "Harry+Potter"; request.ResponseGroup = new string[] { "Small" }; ItemSearch itemSearch = new ItemSearch(); itemSearch.Request = new ItemSearchRequest[] { request }; itemSearch.AWSAccessKeyId = accessKeyId; // issue the ItemSearch request try { ItemSearchResponse response = client.ItemSearch(itemSearch); // write out the results foreach (var item in response.Items[0].Item) { Console.WriteLine(item.ItemAttributes.Title); } } catch(Exception e) { Console.ForegroundColor = ConsoleColor.Red; Console.WriteLine(e.Message); Console.ForegroundColor = ConsoleColor.White; Console.WriteLine("Press any key to quit..."); Clipboard.SetText(e.Message); } Console.ReadKey(); What is wrong?

  Read the article

< Previous Page | 19 20 21 22 23 24 25 26 27 28 29 30  | Next Page >