blogspot - Page 20 - Developer IT

Using BPEL Performance Statistics to Diagnose Performance Bottlenecks

- by fip

Tuning performance of Oracle SOA 11G applications could be challenging. Because SOA is a platform for you to build composite applications that connect many applications and "services", when the overall performance is slow, the bottlenecks could be anywhere in the system: the applications/services that SOA connects to, the infrastructure database, or the SOA server itself.How to quickly identify the bottleneck becomes crucial in tuning the overall performance. Fortunately, the BPEL engine in Oracle SOA 11G (and 10G, for that matter) collects BPEL Engine Performance Statistics, which show the latencies of low level BPEL engine activities. The BPEL engine performance statistics can make it a bit easier for you to identify the performance bottleneck. Although the BPEL engine performance statistics are always available, the access to and interpretation of them are somewhat obscure in the early and current (PS5) 11G versions. This blog attempts to offer instructions that help you to enable, retrieve and interpret the performance statistics, before the future versions provides a more pleasant user experience. Overview of BPEL Engine Performance Statistics SOA BPEL has a feature of collecting some performance statistics and store them in memory. One MBean attribute, StatLastN, configures the size of the memory buffer to store the statistics. This memory buffer is a "moving window", in a way that old statistics will be flushed out by the new if the amount of data exceeds the buffer size. Since the buffer size is limited by StatLastN, impacts of statistics collection on performance is minimal. By default StatLastN=-1, which means no collection of performance data. Once the statistics are collected in the memory buffer, they can be retrieved via another MBean oracle.as.soainfra.bpel:Location=[Server Name],name=BPELEngine,type=BPELEngine.> My friend in Oracle SOA development wrote this simple 'bpelstat' web app that looks up and retrieves the performance data from the MBean and displays it in a human readable form. It does not have beautiful UI but it is fairly useful. Although in Oracle SOA 11.1.1.5 onwards the same statistics can be viewed via a more elegant UI under "request break down" at EM -> SOA Infrastructure -> Service Engines -> BPEL -> Statistics, some unsophisticated minds like mine may still prefer the simplicity of the 'bpelstat' JSP. One thing that simple JSP does do well is that you can save the page and send it to someone to further analyze Follows are the instructions of how to install and invoke the BPEL statistic JSP. My friend in SOA Development will soon blog about interpreting the statistics. Stay tuned. Step1: Enable BPEL Engine Statistics for Each SOA Servers via Enterprise Manager First st you need to set the StatLastN to some number as a way to enable the collection of BPEL Engine Performance Statistics EM Console -> soa-infra(Server Name) -> SOA Infrastructure -> SOA Administration -> BPEL Properties Click on "More BPEL Configuration Properties" Click on attribute "StatLastN", set its value to some integer number. Typically you want to set it 1000 or more. Step 2: Download and Deploy bpelstat.war File to Admin Server, Note: the WAR file contains a JSP that does NOT have any security restriction. You do NOT want to keep in your production server for a long time as it is a security hazard. Deactivate the war once you are done. Download the bpelstat.war to your local PC At WebLogic Console, Go to Deployments -> Install Click on the "upload your file(s)" Click the "Browse" button to upload the deployment to Admin Server Accept the uploaded file as the path, click next Check the default option "Install this deployment as an application" Check "AdminServer" as the target server Finish the rest of the deployment with default settings Console -> Deployments Check the box next to "bpelstat" application Click on the "Start" button. It will change the state of the app from "prepared" to "active" Step 3: Invoke the BPEL Statistic Tool The BPELStat tool merely call the MBean of BPEL server and collects and display the in-memory performance statics. You usually want to do that after some peak loads. Go to http://<admin-server-host>:<admin-server-port>/bpelstat Enter the correct admin hostname, port, username and password Enter the SOA Server Name from which you want to collect the performance statistics. For example, SOA_MS1, etc. Click Submit Keep doing the same for all SOA servers. Step 3: Interpret the BPEL Engine Statistics You will see a few categories of BPEL Statistics from the JSP Page. First it starts with the overall latency of BPEL processes, grouped by synchronous and asynchronous processes. Then it provides the further break down of the measurements through the life time of a BPEL request, which is called the "request break down". 1. Overall latency of BPEL processes The top of the page shows that the elapse time of executing the synchronous process TestSyncBPELProcess from the composite TestComposite averages at about 1543.21ms, while the elapse time of executing the asynchronous process TestAsyncBPELProcess from the composite TestComposite2 averages at about 1765.43ms. The maximum and minimum latency were also shown. Synchronous process statistics <statistics> <stats key="default/TestComposite!2.0.2-ScopedJMSOSB*soa_bfba2527-a9ba-41a7-95c5-87e49c32f4ff/TestSyncBPELProcess" min="1234" max="4567" average="1543.21" count="1000"> </stats> </statistics> Asynchronous process statistics <statistics> <stats key="default/TestComposite2!2.0.2-ScopedJMSOSB*soa_bfba2527-a9ba-41a7-95c5-87e49c32f4ff/TestAsyncBPELProcess" min="2234" max="3234" average="1765.43" count="1000"> </stats> </statistics> 2. Request break down Under the overall latency categorized by synchronous and asynchronous processes is the "Request breakdown". Organized by statistic keys, the Request breakdown gives finer grain performance statistics through the life time of the BPEL requests.It uses indention to show the hierarchy of the statistics. Request breakdown <statistics> <stats key="eng-composite-request" min="0" max="0" average="0.0" count="0"> <stats key="eng-single-request" min="22" max="606" average="258.43" count="277"> <stats key="populate-context" min="0" max="0" average="0.0" count="248"> Please note that in SOA 11.1.1.6, the statistics under Request breakdown is aggregated together cross all the BPEL processes based on statistic keys. It does not differentiate between BPEL processes. If two BPEL processes happen to have the statistic that share same statistic key, the statistics from two BPEL processes will be aggregated together. Keep this in mind when we go through more details below. 2.1 BPEL process activity latencies A very useful measurement in the Request Breakdown is the performance statistics of the BPEL activities you put in your BPEL processes: Assign, Invoke, Receive, etc. The names of the measurement in the JSP page directly come from the names to assign to each BPEL activity. These measurements are under the statistic key "actual-perform" Example 1: Follows is the measurement for BPEL activity "AssignInvokeCreditProvider_Input", which looks like the Assign activity in a BPEL process that assign an input variable before passing it to the invocation: <stats key="AssignInvokeCreditProvider_Input" min="1" max="8" average="1.9" count="153"> <stats key="sensor-send-activity-data" min="0" max="1" average="0.0" count="306"> </stats> <stats key="sensor-send-variable-data" min="0" max="0" average="0.0" count="153"> </stats> <stats key="monitor-send-activity-data" min="0" max="0" average="0.0" count="306"> </stats> </stats> Note: because as previously mentioned that the statistics cross all BPEL processes are aggregated together based on statistic keys, if two BPEL processes happen to name their Invoke activity the same name, they will show up at one measurement (i.e. statistic key). Example 2: Follows is the measurement of BPEL activity called "InvokeCreditProvider". You can not only see that by average it takes 3.31ms to finish this call (pretty fast) but also you can see from the further break down that most of this 3.31 ms was spent on the "invoke-service". <stats key="InvokeCreditProvider" min="1" max="13" average="3.31" count="153"> <stats key="initiate-correlation-set-again" min="0" max="0" average="0.0" count="153"> </stats> <stats key="invoke-service" min="1" max="13" average="3.08" count="153"> <stats key="prep-call" min="0" max="1" average="0.04" count="153"> </stats> </stats> <stats key="initiate-correlation-set" min="0" max="0" average="0.0" count="153"> </stats> <stats key="sensor-send-activity-data" min="0" max="0" average="0.0" count="306"> </stats> <stats key="sensor-send-variable-data" min="0" max="0" average="0.0" count="153"> </stats> <stats key="monitor-send-activity-data" min="0" max="0" average="0.0" count="306"> </stats> <stats key="update-audit-trail" min="0" max="2" average="0.03" count="153"> </stats> </stats> 2.2 BPEL engine activity latency Another type of measurements under Request breakdown are the latencies of underlying system level engine activities. These activities are not directly tied to a particular BPEL process or process activity, but they are critical factors in the overall engine performance. These activities include the latency of saving asynchronous requests to database, and latency of process dehydration. My friend Malkit Bhasin is working on providing more information on interpreting the statistics on engine activities on his blog (https://blogs.oracle.com/malkit/). I will update this blog once the information becomes available. Update on 2012-10-02: My friend Malkit Bhasin has published the detail interpretation of the BPEL service engine statistics at his blog http://malkit.blogspot.com/2012/09/oracle-bpel-engine-soa-suite.html.

Read the article

Users using Perl script to bypass Squid Proxy

- by mk22

The users on our network have been using a perl script to bypass our Squid proxy restrictions. Is there any way we can block this script from working?? #!/usr/bin/perl ######################################################################## # (c) 2008 Indika Bandara Udagedara # [email protected] # http://indikabandara19.blogspot.com # # ---------- # LICENCE # ---------- # This work is protected under GNU GPL # It simply says # " you are hereby granted to do whatever you want with this # except claiming you wrote this." # # # ---------- # README # ---------- # A simple tool to download via http proxies which enforce a download # size limit. Requires curl. # This is NOT a hack. This uses the absolutely legal HTTP/1.1 spec # Tested only for squid-2.6. Only squids will work with this(i think) # Please read the verbose README provided kindly by Rahadian Pratama # if u r on cygwin and think this documentation is not enough :) # # The newest version of pget is available at # http://indikabandara.no-ip.com/~indika/pget # # ---------- # USAGE # ---------- # + Edit below configurations(mainly proxy) # + First run with -i <file> giving a sample file of same type that # you are going to download. Doing this once is enough. # eg. to download '.tar' files first run with # pget -i my.tar ('my.tar' should be a real file) # + Run with # pget -g <URL> # # ######################################################################## ######################################################################## # CONFIGURATIONS - CHANGE THESE FREELY ######################################################################## # *magic* file # pls set absolute path if in cygwin my $_extFile = "./pget.ext" ; # download in chunks of below size my $_chunkSize = 1024*1024; # in Bytes # the proxy that troubles you my $_proxy = "192.168.0.2:3128"; # proxy URL:port my $_proxy_auth = "user:pass"; # proxy user:pass # whereis curl # pls set absolute path if in cygwin my $_curl = "/usr/bin/curl"; ######################################################################## # EDIT BELOW ONLY IF YOU KNOW WHAT YOU ARE DOING ######################################################################## use warnings; my $_version = "0.1.0"; PrintBanner(); if (@ARGV == 0) { PrintHelp(); exit; } PrimaryValidations(); my $val; while(scalar(@ARGV)) { my $arg = shift(@ARGV); if($arg eq '-h') { PrintHelp(); } elsif($arg eq '-i') { $val = shift(@ARGV); if (!defined($val)) { printf("-i option requires a filename\n"); exit; } Init($val); } elsif($arg eq '-g') { $val = shift(@ARGV); if (!defined($val)) { printf("-g option requires a URL\n"); exit; } GetURL($val); } elsif($arg eq '-c') { $val = shift(@ARGV); if (!defined($val)) { printf("-c option requires a URL\n"); exit; } ContinueURL($val); } else { printf ("Unknown option %s\n", $arg); PrintHelp(); } } sub GetURL { my ($URL) = @_; chomp($URL); my $fileName = GetFileName($URL); my %mapExt; my $first; my $readLen; my $ext = GetExt($fileName); ReadMap($_extFile, \%mapExt); if ( exists($mapExt{$ext})) { $first = $mapExt{$ext}; GetFile($URL, $first, $fileName, 0); } else { die "Unknown ext in $fileName. Rerun with -i <fileName>"; } } sub ContinueURL { my ($URL) = @_; chomp($URL); my $fileName = GetFileName($URL); my $fileSize = 0; $fileSize = -s $fileName; printf("Size = %d\n", $fileSize); my $first = -1; if ( $fileSize > 0 ) { $fileSize -= 1; GetFile($URL, $first, $fileName, $fileSize); } else { GetURL($URL); } } sub Init { my ($fileName) = @_; my ($key, $value); my %mapExt; my $ext = GetExt($fileName); if ( $ext eq "") { die "Cannot get ext of \'$fileName\'"; } ReadMap($_extFile, \%mapExt); my $b = GetFirst($fileName); $mapExt{$ext} = $b; WriteMap($_extFile, \%mapExt); print "I handle\n"; while ( ($key, $value) = each(%mapExt) ) { print "\t$key -> $value\n"; } } sub GetExt { my ($name) = @_; my @x = split(/\./, $name); my $ext = ""; if (@x != 1) { $ext = pop @x; } return $ext; } sub ReadMap { my($fileName, $mapRef) = @_; my $f; my @arr; open($f, '<', $fileName) or die "Couldn't open $fileName"; my %map = %{$mapRef}; while (<$f>) { my $line = $_; chomp($line); @arr = split(/[ \t]+/, $line, 2); $mapRef->{ $arr[0]} = $arr[1]; } printf("known ext\n"); while (($key, $value) = each(%$mapRef)) { print("$key, $value\n"); } close($f); } sub WriteMap { my ($fileName, $mapRef) = @_; my $f; my @arr; open($f, '>', $fileName) or die "Couldn't open $fileName"; my ($k, $v); while( ($k, $v) = each(%{$mapRef})) { print $f "$k" . "\t$v\n"; } close($f); } sub PrintHelp { print "usage: -h Print this help -i <filename> Initialize for this filetype -g <URL> Get this URL\n -c <URL> Continue this URL\n" } sub GetFirst { my ($fileName) = @_; my $f; open($f, "<$fileName") or die "Couldn't open $fileName"; my $buffer = ""; my $first = -1; binmode($f); sysread($f, $buffer, 1, 0); close($f); $first = ord($buffer); return $first; } sub GetFirstFromMap { } sub GetFileName { my ($URL) = @_; my @x = split(/\//, $URL); my $fileName = pop @x; return $fileName; } sub GetChunk { my ($URL, $file, $offset, $readLen) = @_; my $end = $offset + $_chunkSize - 1; my $curlCmd = "$_curl -x $_proxy -u $_proxy_auth -r $offset-$end -# \"$URL\""; print "$curlCmd\n"; my $buff = `$curlCmd`; ${$readLen} = syswrite($file, $buff, length($buff)); } sub GetFile { my ($URL, $first, $outFile, $fileSize) = @_; my $readLen = 0; my $start = $fileSize + 1; my $file; open($file, "+>>$outFile") or die "Couldn't open $outFile to write"; if ($fileSize <= 0) { my $uc = pack("C", $first); syswrite ($file, $uc, 1); } do { GetChunk($URL, $file, $start ,\$readLen); $start = $start + $_chunkSize; $fileSize += $readLen; }while ($readLen == $_chunkSize); printf("Downloaded %s(%d bytes).\n", $outFile, $fileSize); close($file); } sub PrintBanner { printf ("pget version %s\n", $_version); printf ("There is absolutely NO WARRANTY for pget.\n"); printf ("Use at your own risk. You have been warned.\n\n"); } sub PrimaryValidations { unless( -e "$_curl") { printf("ERROR:curl is not at %s. Pls install or provide correct path.\n", $_curl); exit; } unless( -e "$_extFile") { printf("extFile is not at %s. Creating one\n", $_extFile); `touch $_extFile`; } if ( $_chunkSize <= 0) { printf ("Invalid chunk size. Using 1Mb as default.\n"); $_chunkSize = 1024*1024; } }

Read the article

Changing the action of a form with javascript/jquery

- by Micah

I'm having an issue that is driving me crazy. I'm trying to modify the openid-selector to support facebook. I'm using RPXNow as my provider so it requires the form to be submitted to a different url than the standard. For example. RpxNow requires me to setup my form like this: <form action="https://wikipediamaze.rpxnow.com/openid/start?token_url=..."> This works for every provider except for facebook and myspace. Those require the form to be posted to a different url like this: <form action="https://wikipediamaze.rpxnow.com/facebook/start?token_url=..."> and <form action="https://wikipediamaze.rpxnow.com/myspace/start?token_url=..."> The open id selector has a bunch of buttons on the form each representing the openid providers. What I'm trying to do is detect when the facebook or myspace button is clicked and changed the action on the form before submitting. However it's not working. Here is my code. I've tried several variations all with the same "not supported" exception $("#openid_form").attr("action", form_url) document.forms[0].action = form_url Any suggestions? Update Here are more details on the code. I've ommitted some for brevity. The only thing i've done is added the facebook section to the "providers_large" object (which successfully adds the logo to the website), and instead of supply a url identifying the user, I'm creating a property called "form_url" which is what I want to set the action of my form to. If you look at the section title "Provider image click" you'll see where I'm checking for the presence of the property "form_url" and using jquery to change the action and submit the form. However when I step through the javascript in debug mode it tells me it's an ivalid operation. var providers_large = { google: { name: 'Google', url: 'https://www.google.com/accounts/o8/id' }, facebook: { name: 'Facebook', form_url: 'http://wikipediamaze.rpxnow.com/facebook/start?token_url=http://www.wikipediamaze.com/Accounts/Logon' }, }; var providers_small = { myopenid: { name: 'MyOpenID', label: 'Enter your MyOpenID username.', url: 'http://{username}.myopenid.com/' }, livejournal: { name: 'LiveJournal', label: 'Enter your Livejournal username.', url: 'http://{username}.livejournal.com/' }, flickr: { name: 'Flickr', label: 'Enter your Flickr username.', url: 'http://flickr.com/{username}/' }, technorati: { name: 'Technorati', label: 'Enter your Technorati username.', url: 'http://technorati.com/people/technorati/{username}/' }, wordpress: { name: 'Wordpress', label: 'Enter your Wordpress.com username.', url: 'http://{username}.wordpress.com/' }, blogger: { name: 'Blogger', label: 'Your Blogger account', url: 'http://{username}.blogspot.com/' }, verisign: { name: 'Verisign', label: 'Your Verisign username', url: 'http://{username}.pip.verisignlabs.com/' }, vidoop: { name: 'Vidoop', label: 'Your Vidoop username', url: 'http://{username}.myvidoop.com/' }, verisign: { name: 'Verisign', label: 'Your Verisign username', url: 'http://{username}.pip.verisignlabs.com/' }, claimid: { name: 'ClaimID', label: 'Your ClaimID username', url: 'http://claimid.com/{username}' } }; var providers = $.extend({}, providers_large, providers_small); var openid = { cookie_expires: 6*30, // 6 months. cookie_name: 'openid_provider', cookie_path: '/', img_path: 'images/', input_id: null, provider_url: null, init: function(input_id) { var openid_btns = $('#openid_btns'); this.input_id = input_id; $('#openid_choice').show(); $('#openid_input_area').empty(); // add box for each provider for (id in providers_large) { openid_btns.append(this.getBoxHTML(providers_large[id], 'large', '.gif')); } if (providers_small) { openid_btns.append('<br/>'); for (id in providers_small) { openid_btns.append(this.getBoxHTML(providers_small[id], 'small', '.ico')); } } $('#openid_form').submit(this.submit); var box_id = this.readCookie(); if (box_id) { this.signin(box_id, true); } }, getBoxHTML: function(provider, box_size, image_ext) { var box_id = provider["name"].toLowerCase(); return '<a title="'+provider["name"]+'" href="javascript: openid.signin(\''+ box_id +'\');"' + ' style="background: #FFF url(' + this.img_path + box_id + image_ext+') no-repeat center center" ' + 'class="' + box_id + ' openid_' + box_size + '_btn"></a>'; }, /* Provider image click */ signin: function(box_id, onload) { var provider = providers[box_id]; if (! provider) { return; } this.highlight(box_id); this.setCookie(box_id); // prompt user for input? if (provider['label']) { this.useInputBox(provider); this.provider_url = provider['url']; } else if(provider['form_url']) { $('#openid_form').attr("action", provider['form_url']); $('#openid_form').submit(); } else { this.setOpenIdUrl(provider['url']); if (! onload) { $('#openid_form').submit(); } } }, /* Sign-in button click */ submit: function() { var url = openid.provider_url; if (url) { url = url.replace('{username}', $('#openid_username').val()); openid.setOpenIdUrl(url); } return true; }, setOpenIdUrl: function (url) { var hidden = $('#'+this.input_id); if (hidden.length > 0) { hidden.value = url; } else { $('#openid_form').append('<input type="hidden" id="' + this.input_id + '" name="' + this.input_id + '" value="'+url+'"/>'); } }, highlight: function (box_id) { // remove previous highlight. var highlight = $('#openid_highlight'); if (highlight) { highlight.replaceWith($('#openid_highlight a')[0]); } // add new highlight. $('.'+box_id).wrap('<div id="openid_highlight"></div>'); }, setCookie: function (value) { var date = new Date(); date.setTime(date.getTime()+(this.cookie_expires*24*60*60*1000)); var expires = "; expires="+date.toGMTString(); document.cookie = this.cookie_name+"="+value+expires+"; path=" + this.cookie_path; }, readCookie: function () { var nameEQ = this.cookie_name + "="; var ca = document.cookie.split(';'); for(var i=0;i < ca.length;i++) { var c = ca[i]; while (c.charAt(0)==' ') c = c.substring(1,c.length); if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length); } return null; }, useInputBox: function (provider) { var input_area = $('#openid_input_area'); var html = ''; var id = 'openid_username'; var value = ''; var label = provider['label']; var style = ''; if (label) { html = '<p>' + label + '</p>'; } if (provider['name'] == 'OpenID') { id = this.input_id; value = 'http://'; style = 'background:#FFF url('+this.img_path+'openid-inputicon.gif) no-repeat scroll 0 50%; padding-left:18px;'; } html += '<input id="'+id+'" type="text" style="'+style+'" name="'+id+'" value="'+value+'" />' + '<input id="openid_submit" type="submit" value="Sign-In"/>'; input_area.empty(); input_area.append(html); $('#'+id).focus(); } };

Developer IT