Search Results

Search found 27 results on 2 pages for 'sqs'.

Page 1/2 | 1 2  | Next Page >

  • Send an XML message to Amazon SQS

    - by bartligthart
    I am a newbie to Amazon SQS and ruby on rails. And i am working on a project that some XML messages must be send to SQS. How do i do that? Now i have this in the controller after the .save def create @thing = Thing.new(params[:thing]) respond_to do |format| if @thing.save message = @thing.to_xml and in the model inputqueue.send_message(message) Is this the way i can send an XML file to SQS or??

    Read the article

  • Sqs vs SqsGen2 using RightScale right_aws GEM

    - by Fitter Man
    I'm trying to use the right_aws (1.10.0) GEM with Rails, and I've reduced my problem to a 3-line irb session. The following works require 'rubygems' require 'right_aws' sqs = RightAws::Sqs.new("xxxxxxxxxxxxxxx", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx") while this fails require 'rubygems' require 'right_aws' sqs = RightAws::SqsGen2.new("xxxxxxxxxxxxxxx", "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx") with NameError: uninitialized constant RightAws::SqsGen2. I see the class definition in the GEM source, the documentation is old but seems accurate, but I can't figure out what I'm doing wrong. And while you're at it, is there any reason if I'm building something new, I'd want to use the older interface?

    Read the article

  • Amazon SQS invalid binary character in message body

    - by letronje
    I have a web app that sends messages to an Amazon SQS Queue. Amazon sqs lib throws a 'AmazonSQSException' since the message contained invalid binary character. The message is the referrer obtained from an incoming http request. This is what it looks like: http://ads.vrx.adbrite.com/adserver/display_iab_ads.php?sid=1220459&title_color=0000FF&text_color=000000&background_color=FFFFFF&border_color=CCCCCC&url_color=008000&newwin=0&zs=3330305f323530&width=300&height=250&url=http%3A%2F%2Funblockorkutproxy.com%2Fsearch.php%2FOi8vZG93%2FbmxvYWRz%2FLnppZGR1%2FLmNvbS9k%2Fb3dubG9h%2FZGZpbGUv%2FNTY5MTQ3%2FNi9NeUN1%2FdGVHaXJs%2FZnJpZW5k%2FWmFoaXJh%2FLndtdi5o%2FdG1s%2Fb0%2F^Fô}úÃ<99ë)j Looks like the characters in bold are the invalid characters. Is there an easy way to filter out characters characters that are not accepted by amazon ? Here are the characters allowed by amazon in message body. I am not sure what regex i should use to replace invalid characters by ''

    Read the article

  • Ruby1.9 and Amazon SQS?

    - by fields
    Is there a good library/gem for accessing Amazon SQS from ruby1.9? The Amazon ruby example and right_aws do not work as-is with ruby1.9. I'd strongly prefer something that's known to work under reasonably heavy load (a few hundred thousand queue items or more per day).

    Read the article

  • Emulating Amazon SQS during development

    - by pyo
    I'm quite interested in beginning some development using Amazon SQS, perhaps SimpleDB too, my question is this, are there any open source solutions that mimic the functionality, just for the purposes of development. I've already encountered the Eucalyptus project (http://open.eucalyptus.com) for creating an EC-esque cloud. I've not had any success with google, I suspect it's because the cost of entry is so inexpensive, but still, does anyone know of anything like this?

    Read the article

  • Best practices for using Amazon SQS - Polling the queue

    - by alex
    I'm designing a service for sending out emails for our eCommerce site (order confirmations, alerts etc...) The plan is to have a "SendEmail" method, that generates a chunk of XML representing the email to be sent, and sticks it on an Amazon SQS queue. My web app(s) and other applications will use this to "send" emails. I then require a way of checking the queue, and physically sending out the email messages. (I know how I'm going to be dispatching emails) I'm curious as to what the best way to "poll" the queue would be? Should I create a windows service, and use something like Quartz.net to schedule it to check the queue every x number of minutes for example? Is there a better way of doing this?

    Read the article

  • Is Amazon SQS the right choice here? Rails performance issue.

    - by ole_berlin
    I'm close to releasing a rails app with the common networking features (messaging, wall, etc.). I want to use some kind of background processing (most likely Bj) for off-loading tasks from the request/response cycle. This would happen when users invite friends via email to join and for email notifications. I'm not sure if I should just drop these invites and notifications in my Database, using a model and then just process it with a worker process every x minutes or if I should go for Amazon SQS, storing the messages and invites there and let my worker retrieve it from Amazon SQS for processing (sending the invites / notifications). The Amazon approach would get load off my Database but I guess it is slower to retrieve messages from there. What do you think?

    Read the article

  • AWS .NET SDK v2: setting up queues and topics

    - by Elton Stoneman
    Originally posted on: http://geekswithblogs.net/EltonStoneman/archive/2013/10/13/aws-.net-sdk-v2-setting-up-queues-and-topics.aspxFollowing on from my last post, reading from SQS queues with the new SDK is easy stuff, but linking a Simple Notification Service topic to an SQS queue is a bit more involved. The AWS model for topics and subscriptions is a bit more advanced than in Azure Service Bus. SNS lets you have subscribers on multiple different channels, so you can send a message which gets relayed to email address, mobile apps and SQS queues all in one go. As the topic owner, when you request a subscription on any channel, the owner needs to confirm they’re happy for you to send them messages. With email subscriptions, the user gets a confirmation request from Amazon which they need to reply to before they start getting messages. With SQS, you need to grant the topic permission to write to the queue. If you own both the topic and the queue, you can do it all in code with the .NET SDK. Let’s say you want to create a new topic, a new queue as a topic subscriber, and link the two together. Creating the topic is easy with the SNS client (which has an expanded name, AmazonSimpleNotificationServiceClient, compare to the SQS class which is just called QueueClient): var request = new CreateTopicRequest(); request.Name = TopicName; var response = _snsClient.CreateTopic(request); TopicArn = response.TopicArn; In the response from AWS (which I’m assuming is successful), you get an ARN – Amazon Resource Name – which is the unique identifier for the topic. We create the queue using the same code from my last post, AWS .NET SDK v2: the message-pump pattern, and then we need to subscribe the queue to the topic. The topic creates the subscription request: var response = _snsClient.Subscribe(new SubscribeRequest { TopicArn = TopicArn, Protocol = "sqs", Endpoint = _queueClient.QueueArn }); That response will give you an ARN for the subscription, which you’ll need if you want to set attributes like RawMessageDelivery. Then the SQS client needs to confirm the subscription by allowing the topic to send messages to it. The SDK doesn’t give you a nice mechanism for doing that, so I’ve extended my AWS wrapper with a method that encapsulates it: internal void AllowSnsToSendMessages(TopicClient topicClient) { var policy = Policies.AllowSendFormat.Replace("%QueueArn%", QueueArn).Replace("%TopicArn%", topicClient.TopicArn); var request = new SetQueueAttributesRequest(); request.Attributes.Add("Policy", policy); request.QueueUrl = QueueUrl; var response = _sqsClient.SetQueueAttributes(request); } That builds up a policy statement, which gets added to the queue as an attribute, and specifies that the topic is allowed to send messages to the queue. The statement itself is a JSON block which contains the ARN of the queue, the ARN of the topic, and an Allow effect for the sqs:SendMessage action: public const string AllowSendFormat= @"{ ""Statement"": [ { ""Sid"": ""MySQSPolicy001"", ""Effect"": ""Allow"", ""Principal"": { ""AWS"": ""*"" }, ""Action"": ""sqs:SendMessage"", ""Resource"": ""%QueueArn%"", ""Condition"": { ""ArnEquals"": { ""aws:SourceArn"": ""%TopicArn%"" } } } ] }"; There’s a new gist with an updated QueueClient and a new TopicClient here: Wrappers for the SQS and SNS clients in the AWS SDK for .NET v2. Both clients have an Ensure() method which creates the resource, so if you want to create a topic and a subscription you can use:  var topicClient = new TopicClient(“BigNews”, “ImListening”); And the topic client has a Subscribe() method, which calls into the message pump on the queue client: topicClient.Subscribe(x=>Log.Debug(x.Body)); var message = {}; //etc. topicClient.Publish(message); So you can isolate all the fiddly bits and use SQS and SNS with a similar interface to the Azure SDK.

    Read the article

  • architecture and tools for a remote control application?

    - by slothbear
    I'm working on the design of a remote control application. From my iPhone or a web browser, I'll send a few commands. Soon my home computer will perform the commands and send back results. I know there are remote desktop apps, but I want something programmable, something simpler, and something that I wrote. My current direction is to use Amazon Simple Queue Service (SQS) as the message bus. The iPhone places some messages in a queue. My local Java/JRuby program notices the messages on the queue, performs the work and sends back status via a different queue. This will be a very low-volume application. At $1.00 for a million requests (plus a handful of data transfer charges), Amazon SQS looks a lot more affordable than having my own server of any type. And super reliable, that's important for me too. Are there better/standard toolkits or architectures for this kind of remote control? Cost is not a big issue, but I prefer the tons I learn by doing it myself. I'm moderately concerned about security, but doubt it will be a problem. The list of commands recognized will be very short, and only recognized in specific contexts. No "erase hard drive" stuff. update: I'll probably distribute these programs to some other people who want the same function, but who don't have Amazon SQS accounts. For now, they'll use anonymous access to my queues, with random 80-character queue names.

    Read the article

  • How to implement message queuing and handling in AWS with NServiceBus

    - by Pete Lunenfeld
    I am creating a new ASP MVC order application in the Amazon (AWS) cloud with the persistence layer at my local datacenter. I will be using the CQRS pattern. The goal of the project is high availability using Queue(s) to store and forward writes (commands/events) that can be picked up and handled asynchronously at my local datacenter. Then, ff the WAN or my local datacenter fails, my cloud MVC app can still take orders and just queue them up until processing can resume. My first thought was to use AWS SQS for the queuing and create my own queue consumer/dispatcher/handler in my own c# application to process the incoming messages/events. MVC (@ Amazon) -- Event/POCO -- SQS -- QueueReader (@ my datacenter) -- DB Then I found NServiceBus. NSB seems to handle lots of details very nicely: message handling, retries, error handling, etc. I hate to reinvent the wheel, and NServiceBus seems like a full featured and mature product that would be perfect for me. But on further research, it does NOT look like NServiceBus is really meant to be used over the WAN in physically separated environments (Cloud to my Datacenter). Google and SO don't really paint a good picture of using NServiceBus across the WAN like I need. How can I use NServiceBus across the WAN? Or is there a better solution to handle queuing and message handling between Amazon an my local datacenter?

    Read the article

  • How can I poll different aws sqs in the same process?

    - by Luccas
    What is the right way to poll from differents AWS SQS in the same process? Suppose I have a ruby script: listen_queues.rb and run it. Should I need to create threads to wrap each SQS poll or start sub processes? t1 = Thread.new do queue1.poll do |msg| .... end t2 = Thread.new do queue2.poll do |msg| .... end t2.join I tried this code, but the poll is not receiving any of the messages available. When I run only one of them (t1 or t2), it works. But I need the 2 running. What is going on? Thanks!!

    Read the article

  • Should I have a heroku worker dyno for poll a AWS SQS?

    - by Luccas
    Im confusing about where should I have a script polling an Aws Sqs inside a Rails application. If I use a thread inside the web app probably it will use cpu cycles to listen this queue forever and then affecting performance. And if I reserve a single heroku worker dyno it costs $34.50 per month. It makes sense to pay this price for it for a single queue poll? Or it's not the case to use a worker for it? The script code: queue = AWS::SQS::Queue.new(SQSADDR['my_queue']) queue.poll(:idle_timeout => 20) do |msg| # code here end I need help!! Thanks

    Read the article

  • AWS .NET SDK v2: the message-pump pattern

    - by Elton Stoneman
    Originally posted on: http://geekswithblogs.net/EltonStoneman/archive/2013/10/11/aws-.net-sdk-v2--the-message-pump-pattern.aspxVersion 2 of the AWS SDK for .NET has had a few pre-release iterations on NuGet and is stable, if a bit lacking in step-by-step guides. There’s at least one big reason to try it out: the SQS queue client now supports asynchronous reads, so you don’t need a clumsy polling mechanism to retrieve messages. The new approach  is easy to use, and lets you work with AWS queues in a similar way to the message-pump pattern used in the latest Azure SDK for Service Bus queues and topics. I’ve posted a simple wrapper class for subscribing to an SQS hub on gist here: A wrapper for the SQS client in the AWS SDK for.NET v2, which uses the message-pump pattern. Here’s the core functionality in the subscribe method: private async void Subscribe() { if (_isListening) { var request = new ReceiveMessageRequest { MaxNumberOfMessages = 10 }; request.QueueUrl = QueueUrl; var result = await _sqsClient.ReceiveMessageAsync(request, _cancellationTokenSource.Token); if (result.Messages.Count > 0) { foreach (var message in result.Messages) { if (_receiveAction != null && message != null) { _receiveAction(message.Body); DeleteMessage(message.ReceiptHandle); } } } } if (_isListening) { Subscribe(); } } which you call with something like this: client.Subscribe(x=>Log.Debug(x.Body)); The async SDK call returns when there is something in the queue, and will run your receive action for every message it gets in the batch (defaults to the maximum size of 10 messages per call). The listener will sit there awaiting messages until you stop it with: client.Unsubscribe(); Internally it has a cancellation token which it sets when you call unsubscribe, which cancels any in-flight call to SQS and stops the pump. The wrapper will also create the queue if it doesn’t exist at runtime. The Ensure() method gets called in the constructor so when you first use the client for a queue (sending or subscribing), it will set itself up: if (!Exists()) { var request = new CreateQueueRequest(); request.QueueName = QueueName; var response = _sqsClient.CreateQueue(request); QueueUrl = response.QueueUrl; } The Exists() check has to do make a call to ListQueues on the SNS client, as it doesn’t provide its own method to check if a queue exists. That call also populates the Amazon Resource Name, the unique identifier for this queue, which will be useful later. To use the wrapper, just instantiate and go: var queueClient = new QueueClient(“ProcessWorkflow”); queueClient.Subscribe(x=>Log.Debug(x.Body)); var message = {}; //etc. queueClient.Send(message);

    Read the article

  • Two threads in initializer on rails not working

    - by Luccas
    Initially I was using one thread to listen a queue from amazon and works perfectly. aws.rb Thread.new do my_queue = AWS::SQS::Queue.new(SQSADDR['my_queue']) my_queue.poll do |msg| ... but now I appended another thread to listen another queue: ... Thread.new do my_another_queue = AWS::SQS::Queue.new(SQSADDR['my_another_queue']) my_another_queue.poll do |msg| ... and now it seems to not work. Only the last one receives response... What is going on?

    Read the article

  • Django Haystack exact filtering

    - by blackrobot
    I have a haystack search which has the following SearchIndex: class GrantIndex(indexes.SearchIndex): """ This provides the search index for the Grant application. """ text = indexes.CharField(document=True, use_template=True) year = indexes.IntegerField(model_attr='year__year') date = indexes.DateField(model_attr='date') program = indexes.CharField(model_attr='program__area') grantee = indexes.CharField(model_attr='grantee') amount = indexes.IntegerField(model_attr='amount') site.register(Grant, GrantIndex) If I want to search filtering out any programs that ARE NOT 'Health', I run the following query: from haystack.query import SearchQuerySet sqs = SearchQuerySet() sqs = sqs.filter(program='Health') Unfortunately, this also produces objects from the program 'Health\Other' and 'Health\Cardiovascular'. How do I stop the search from allowing those other programs in? I run Ubuntu 9.10 with Xapian as my search back-end.

    Read the article

  • ez components and AWS PHP SDK makes ez components freak out

    - by David
    Hi, I try to work with ez Components and AWS PHP SDK at the same time. I have a file called resize.php which is just handling resizing images using the ez Components ImageTransition tools. I queue the image for resize in Amazon AWS SQS. If I load the AWS PHP SDK and ez Components in the same file, PHP always complains about not finding the ez Components classes. Code looks something like this: amazonSQS.php: require 'modules/resize.php'; require 'modules/aws/sdk.class.php'; $sqs = new AmazonSQS(); $response = $sqs->send_message($queue_url, $message); resize.php: function resize_image($filename) { $settings = new ezcImageConverterSettings( array( //new ezcImageHandlerSettings( 'GD', 'ezcImageGdHandler' ), new ezcImageHandlerSettings( 'ImageMagick', 'ezcImageImagemagickHandler' ), ) ); Error message: Fatal error: Class 'ezcImageConverterSettings' not found in /home/www.com/public_html/modules/resize.php on line 10 If I call resize.php from another PHP file which has AWS not included, it works fine. I load ezComponents like this: require 'ezc/Base/ezc_bootstrap.php'; It is installed as a PEAR package. Any idea someone?

    Read the article

  • Two threads in a rails initializer file seems to not run them properly

    - by Luccas
    Initially I was using one thread to listen a queue from amazon and works perfectly. aws.rb Thread.new do queue1 = AWS::SQS::Queue.new(SQSADDR['my_queue']) queue1.poll do |msg| ... but now I appended another thread to listen another queue: ... Thread.new do queue2 = AWS::SQS::Queue.new(SQSADDR['my_another_queue']) queue2.poll do |msg| ... and now it seems to not work. Only the last one receives response... I have to join the threads? I can't understand What is going on?

    Read the article

  • Clojure agents consuming from a queue

    - by erikcw
    I'm trying to figure out the best way to use agents to consume items from a Message Queue (Amazon SQS). Right now I have a function (process-queue-item) that grabs an items from the queue, and processes it. I want to process these items concurrently, but I can't wrap my head around how to control the agents. Basically I want to keep all of the agents busy as much as possible without pulling to many items from the Queue and developing a backlog (I'll have this running on a couple of machines, so items need to be left in the queue until they are really needed). Can anyone give me some pointers on improving my implementation? (def active-agents (ref 0)) (defn process-queue-item [_] (dosync (alter active-agents inc)) ;retrieve item from Message Queue (Amazon SQS) and process (dosync (alter active-agents dec))) (defn -main [] (def agents (for [x (range 20)] (agent x))) (loop [loop-count 0] (if (< @active-agents 20) (doseq [agent agents] (if (agent-errors agent) (clear-agent-errors agent)) ;should skip this agent until later if it is still busy processing (not sure how) (send-off agent process-queue-item))) ;(apply await-for (* 10 1000) agents) (Thread/sleep 10000) (logging/info (str "ACTIVE AGENTS " @active-agents)) (if (> 10 loop-count) (do (logging/info (str "done, let's cleanup " count)) (doseq [agent agents] (if (agent-errors agent) (clear-agent-errors agent))) (apply await agents) (shutdown-agents)) (recur (inc count)))))

    Read the article

  • How can I script an alert for when my Amazon Web Service usage goes above a certain amount?

    - by frabcus
    We're using S3, SimpleDB and SQS on quite a complicated project. I'd like to be able to automatically track their usage, to be sure we don't suddenly spend large amounts of money when we didn't intend to (perhaps because of a bug). Is there a way of reading the usage figures of all Amazon Web Services and/or the current real time dollar cost of an account from a script? Or any service or script which provides alerts based on that?

    Read the article

  • Architecting Python application consisting of many small scripts

    - by Duke Dougal
    I am building an application which, at the moment, consists of many small Python scripts. Each Python script processes items from one Amazon SQS queue. Emails come into an initial queue and are processed by a script and typically the script will do a small unit of processing (for example, parse email and store some database fields), then an item will be placed on the next queue for further processing, until eventually the email has finished going through the various scripts and queues. What I like about this approach is that it is very loosely coupled. However, I'm not sure how I should implement live. Should I make each script a daemon which is constantly polling it's inbound queue for things to do? Or should there be some overarching orchestration program or process? Or maybe I should not have lots of small Python scripts but one large application? Specific questions: How should I run each of these scripts - as a daemon with some sort or restart monitor to restart them in case they stop for any reason? If yes, should I have some program which orchestrates this? Or is the idea of many small script not a good one, would it make more sense to have a larger python program which contains all the functionality and does all the queue polling and execution of functionality for each queue? What is the current preferred approach to daemonising Python scripts? Broadly I would welcome any comments or opinions on any aspect of this. thanks

    Read the article

  • Eclipse and python: library will import in interprer, but not in IDE

    - by John
    I'm running Windows 7, Python 2.6.4 and the latest version of Eclipse. I downloaded the boto library (http://code.google.com/p/boto/) and ran python setup.py install, which created boto-1.9b-py2.6.egg in C:\Python26\Lib\site-packages. Importing a class - say, by doing 'from boto.sqs.connection import SQSConnection' - works fine from the python command line tool. But Eclipse will not find boto, despite the fact that it is using the same python interpreter as I am using when at the command line. I added the library as an external source folder, but that didn't work either. How can I properly import the boto library into Eclipse? Thanks.

    Read the article

  • use java-ffmpeg wrapper, or simply use java runtime to execute ffmpeg?

    - by user156153
    I'm pretty new to Java, need to write a program that listen to video conversion instructions and convert the video once an new instruction arrives (instructions is stored in Amazon SQS, but it's irrelevant to my question) I'm facing a choice, either use Java RunTime to exec 'ffmpeg' conversion (like from command line), or I can use a ffmpeg wrapper written inJava http://fmj-sf.net/ffmpeg-java/getting%5Fstarted.php I'd much prefer using Java Runtime to exec ffmpeg directly, and avoid using java-ffmpeg wrapper as I have to learn the library. so my question is are there any benefits using java-ffmpeg wrapper over exec ffmpeg directly using Runtime? I don't need ffmpeg to play videos, just convert videos Thanks

    Read the article

  • AWS EC2 Oracle RDB - Storing and managing my data

    - by llaszews
    When create an Oracle Database on the Amazon cloud you will need to store you database files somewhere on the EC2 cloud. There are basically three places where database files can be stored: 1. Local drive - This is the local drive that is part of the virtual server EC2 instance. 2. Elastic Block Storage (EBS) - Network attached storage that appears as a local drive. 3. Simple Storage Server (S3) - 'Storage for the Internet'. S3 is not high speed and intended for store static document type files. S3 can also be used for storing static web page files. Local drives are ephemeral so not appropriate to be used as a database storage device. The leaves EBS which is the best place to store database files. EBS volumes appear as local disk drives. They are actually network-attached to an Amazon EC2 instance. In addition, EBS persists independently from the running life of a single Amazon EC2 instance. If you use an EBS backed instance for your database data, it will remain available after reboot but not after terminate. In many cases you would not need to terminate your instance but only stop it, which is equivalent of shutdown. In order to save your database data before you terminate an instance, you can snapshot the EBS to S3. Using EBS as a data store you can move your Oracle data files from one instance to another. This allows you to move your database from one region or or zone to another. Unfortunately, to scale out your Oracle RDS on AWS you can not have read only replicas. This is only possible with the other Oracle relational database - MySQL. The free micro instances use EBS as its storage. This is a very good white paper that has more details: AWS Storage Options This white paper also discusses: SQS, SimpleDB, and Amazon RDS in the context of storage devices. However, these are not storage devices you would use to store an Oracle database. This slide deck discusses a lot of information that is in the white paper: AWS Storage Options slideshow

    Read the article

  • "cloud architecture" concepts in a system architecture diagrams

    - by markus
    If you design a distributed application for easy scale-out, or you just want to make use of any of the new “cloud computing” offerings by Amazon, Google or Microsoft, there are some typical concepts or components you usually end up using: distributed blob storage (aka S3) asynchronous, durable message queues (aka SQS) non-Relational-/non-transactional databases (like SimpleDB, Google BigTable, Azure SQL Services) distributed background worker pool load-balanced, edge-service processes handling user requests (often virtualized) distributed caches (like memcached) CDN (content delivery network like Akamai) Now when it comes to design and sketch an architecture that makes use of such patterns, are there any commonly used symbols I could use? Or even a download with some cool Visio stencils? :) It doesn’t have to be a formal system like UML but I think it would be great if there were symbols that everyone knows and understands, like we have commonly used shapes for databases or a documents, for example. I think it would be important to not mix it up with traditional concepts like a normal file system (local or network server/SAN), or a relational database. Simply speaking, I want to be able to draw some conclusions about an application’s scalability or data consistency issues by just looking at the system architecture overview diagram. Update: Thank you very much for your answers. I like the idea of putting a small "cloud symbol" on the traditional symbols. However I leave this thread open just in case someone will find specific symbols (maybe in a book or so) - or uploaded some pimped up Visio stencils ;)

    Read the article

1 2  | Next Page >