just my correct opinion - Page 704

Strange problem with Random Access Filing in C++

- by sam

This is a simple random access filing program . The problem arises where i want to write data randomly. If I write any where in the file the previous records are set to 0. the last 1 which is being entered currently holds the correct value all others =0. This is the code #include <iostream> #include<fstream> #include<string> using namespace std; class name { int id; int pass; public: void writeBlank(); void writedata(); void readdata(); void readall(); int getid() { return id; } int getpass() { return pass; } void setid(int i) { id=i; } void setpass(int p) { pass=p; } }; void name::writeBlank() { name person; person.setid(0); person.setpass(0); int i; ofstream out("pass.txt",ios::binary); if ( !out ) { cout << "File could not be opened." << endl; } for(i=0;i<10;i++) //make 10 records { cout<<"Put pointer is at: "<<out.tellp()<<endl; cout<<"Blank record "<<i<<" is: "<<person.getid()<<" "<<person.getpass()<<" and size: "<<sizeof(person)<<endl; cout<<"Put pointer is at: "<<out.tellp()<<endl; out.write(reinterpret_cast< const char * >(&person),sizeof(name)); } } void name::writedata() { ofstream out("pass.txt",ios::binary|ios::out); name n1; int iD,p; cout<<"ID?"; cin>>iD; n1.setid(iD); cout<<"Enter password"; cin>>p; n1.setpass(p); if (!out ) { cout << "File could not be opened." << endl; } out.seekp((n1.getid()-1)*sizeof(name),ios::beg); //pointer moves to desired location where we have to store password according to its ID(index) cout<<"File pointer is at: "<<out.tellp()<<endl; out.write(reinterpret_cast<const char*> (&n1), sizeof(name)); //write on that pointed location } void name::readall() { name n1; ifstream in("pass.txt",ios::binary); if ( !in ) { cout << "File could not be opened." << endl; } in.read( reinterpret_cast<char *>(&n1), sizeof(name) ); while ( !in.eof() ) { // display record cout<<endl<<"password at this index is:"<<n1.getpass()<<endl; cout<<"File pointer is at: "<<in.tellg()<<endl; // read next from file in.read( reinterpret_cast< char * >(&n1), sizeof(name)); } // end while } void name::readdata() { ifstream in("pass.txt",ios::binary); if ( !in ) { cout << "File could not be opened." << endl; } in.seekg((getid()-1)*sizeof(name)); //pointer moves to desired location where we have to read password according to its ID(index) cout<<"File pointer is at: "<<in.tellg()<<endl; in.read((char* )this,sizeof(name)); //reads from that pointed location cout<<endl<<"password at this index is:"<<getpass()<<endl; } int main() { name n1; cout<<"Enter 0 to write blank records"<<endl; cout<<"Enter 1 for new account"<<endl; cout<<"Enter 2 to login"<<endl; cout<<"Enter 3 to read all"<<endl; cout<<"Enter 9 to exit"<<endl; int option; cin>>option; while(option==0 || option==1 || option==2 || option==3) { if (option == 0) n1.writeBlank(); if(option==1) { /*int iD,p; cout<<"ID?"; cin>>iD; n1.setid(iD); cout<<"Enter password"; cin>>p; n1.setpass(p);*/ n1.writedata(); } int ind; if(option==2) { cout<<"Index?"; cin>>ind; n1.setid(ind); n1.readdata(); } if(option == 3) n1.readall(); cout<<"Enter 0 to write blank records"<<endl; cout<<"Enter 1 for new account"<<endl; cout<<"Enter 2 to login"<<endl; cout<<"Enter 3 to read all"<<endl; cout<<"Enter 9 to exit"<<endl; cin>>option; } } I Cant understand Y the previous records turn 0.

Read the article

php + MySQL editing table data.

- by Jacksta

This question is relating to 2 php scripts. The first script is called pick_modcontact.php where I choose a contact (from a contact book like phone book), then posts to the script show_modcontact.php When I click the submit button on the form on pick.modcontact.php. As a result of submitting the form I am then taken to show_modcontact.php. As the variables are not present the user is directed back to pick_modcontact.php I can not work out how to correct the code so that it will show the results of the script show_modcontact.php This script shows all contacts in a database which is an "address book" this part works fine. please see below. Name:pick_modcontact.php if ($_SESSION['valid'] != "yes") { header( "Location: contact_menu.php"); exit; } $db_name = "testDB"; $table_name = "my_contacts"; $connection = @mysql_connect("localhost", "admin", "user") or die(mysql_error()); $db = @mysql_select_db($db_name, $connection) or die(mysql_error()); $sql = "SELECT id, f_name, l_name FROM $table_name ORDER BY f_name"; $result = @mysql_query($sql, $connection) or die(mysql_error()); $num = @mysql_num_rows($result); if ($num < 1) { $display_block = "<p><em>Sorry No Results!</em></p>"; } else { while ($row = mysql_fetch_array($result)) { $id = $row['id']; $f_name = $row['f_name']; $l_name = $row['l_name']; $option_block .= "<option value\"$id\">$f_name, $l_name</option>"; } $display_block = "<form method=\"POST\" action=\"show_modcontact.php\"> <p><strong>Contact:</strong> <select name=\"id\">$option_block</select> <input type=\"submit\" name=\"submit\" value=\"Select This Contact\"></p> </form>"; } ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Modify A Contact</title> </head> <body> <h1>My Contact Management System</h1> <h2><em>Modify a Contact</em></h2> <p>Select a contact from the list below, to modify the contact's record.</p> <? echo "$display_block"; ?> <br> <p><a href="contact_menu.php">Return to Main Menu</a></p> </body> </html> This script is for modifying the contact: named show_modcontact.php <?php if (!$_POST['id']) { header( "Location: pick_modcontact.php"); exit; } else { session_start(); } if ($_SESSION['valid'] != "yes") { header( "Location: pick_modcontact.php"); exit; } $db_name = "testDB"; $table_name = "my_contacts"; $connection = @mysql_connect("localhost", "admin", "pass") or die(mysql_error()); $db = @mysql_select_db($db_name, $connection) or die(mysql_error()); $sql = "SELECT f_name, l_name, address1, address2, address3, postcode, prim_tel, sec_tel, email, birthday FROM $table_name WHERE id = '" . $_POST['id'] . "'"; $result = @mysql_query($sql, $connection) or die(mysql_error()); while ($row = mysql_fetch_array($result)) { $f_name = $row['f_name']; $l_name = $row['l_name']; $address1 = $row['address1']; $address2 = $row['address2']; $address3 = $row['address3']; $country = $row['country']; $prim_tel = $row['prim_tel']; $sec_tel = $row['sec_tel']; $email = $row['email']; $birthday = $row['birthday']; } ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Modify A Contact</title> </head> <body> <form action="do_modcontact.php" method="post"> <input type="text" name="id" value="<? echo $_POST['id']; ?>" /> <table cellpadding="5" cellspacing="3"> <tr> <th>Name & Address Information</th> <th> Other Contact / Personal Information</th> </tr> <tr> <td align="top"> <p><strong>First Name:</strong><br /> <input type="text" name="f_name" value="<? echo "$f_name"; ?>" size="35" maxlength="75" /></p> <p><strong>Last Name:</strong><br /> <input type="text" name="l_name" value="<? echo "$l_name"; ?>" size="35" maxlength="75" /></p> <p><strong>Address1:</strong><br /> <input type="text" name="f_name" value="<? echo "$address1"; ?>" size="35" maxlength="75" /></p> <p><strong>Address2:</strong><br /> <input type="text" name="f_name" value="<? echo "$address2"; ?>" size="35" maxlength="75" /></p> <p><strong>Address3:</strong><br /> <input type="text" name="f_name" value="<? echo "$address3"; ?>" size="35" maxlength="75" /> </p> <p><strong>Postcode:</strong><br /> <input type="text" name="f_name" value="<? echo "$postcode"; ?>" size="35" maxlength="75" /></p> <p><strong>Country:</strong><br /> <input type="text" name="f_name" value="<? echo "$country"; ?>" size="35" maxlength="75" /> </p> <p><strong>First Name:</strong><br /> <input type="text" name="f_name" value="<? echo "$f_name"; ?>" size="35" maxlength="75" /></p> </td> <td align="top"> <p><strong>Prim Tel:</strong><br /> <input type="text" name="f_name" value="<? echo "$prim_tel"; ?>" size="35" maxlength="75" /></p> <p><strong>Sec Tel:</strong><br /> <input type="text" name="f_name" value="<? echo "$sec_tel"; ?>" size="35" maxlength="75" /></p> <p><strong>Email:</strong><br /> <input type="text" name="f_name" value="<? echo "$email;" ?>" size="35" maxlength="75" /> </p> <p><strong>Birthday:</strong><br /> <input type="text" name="f_name" value="<? echo "$birthday"; ?>" size="35" maxlength="75" /> </p> </td> </tr> <tr> <td align="center"> <p><input type="submit" name="submit" value="Update Contact" /></p> <br /> <p><a href="contact_menu.php">Retuen To Menu</a></p> </td> </tr> </table> </form> </body> </html> note for site admin, I am re posting this question with the hope of someone else reading over it. older questions seem to go dead after a while.

Read the article

Logic error for Gauss elimination

- by iwanttoprogram

Logic error problem with the Gaussian Elimination code...This code was from my Numerical Methods text in 1990's. The code is typed in from the book- not producing correct output... Sample Run: SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS USING GAUSSIAN ELIMINATION This program uses Gaussian Elimination to solve the system Ax = B, where A is the matrix of known coefficients, B is the vector of known constants and x is the column matrix of the unknowns. Number of equations: 3 Enter elements of matrix [A] A(1,1) = 0 A(1,2) = -6 A(1,3) = 9 A(2,1) = 7 A(2,2) = 0 A(2,3) = -5 A(3,1) = 5 A(3,2) = -8 A(3,3) = 6 Enter elements of [b] vector B(1) = -3 B(2) = 3 B(3) = -4 SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS The solution is x(1) = 0.000000 x(2) = -1.#IND00 x(3) = -1.#IND00 Determinant = -1.#IND00 Press any key to continue . . . The code as copied from the text... //Modified Code from C Numerical Methods Text- June 2009 #include <stdio.h> #include <math.h> #define MAXSIZE 20 //function prototype int gauss (double a[][MAXSIZE], double b[], int n, double *det); int main(void) { double a[MAXSIZE][MAXSIZE], b[MAXSIZE], det; int i, j, n, retval; printf("\n \t SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS"); printf("\n \t USING GAUSSIAN ELIMINATION \n"); printf("\n This program uses Gaussian Elimination to solve the"); printf("\n system Ax = B, where A is the matrix of known"); printf("\n coefficients, B is the vector of known constants"); printf("\n and x is the column matrix of the unknowns."); //get number of equations n = 0; while(n <= 0 || n > MAXSIZE) { printf("\n Number of equations: "); scanf ("%d", &n); } //read matrix A printf("\n Enter elements of matrix [A]\n"); for (i = 0; i < n; i++) for (j = 0; j < n; j++) { printf(" A(%d,%d) = ", i + 1, j + 1); scanf("%lf", &a[i][j]); } //read {B} vector printf("\n Enter elements of [b] vector\n"); for (i = 0; i < n; i++) { printf(" B(%d) = ", i + 1); scanf("%lf", &b[i]); } //call Gauss elimination function retval = gauss(a, b, n, &det); //print results if (retval == 0) { printf("\n\t SOLUTION OF SIMULTANEOUS LINEAR EQUATIONS\n"); printf("\n\t The solution is"); for (i = 0; i < n; i++) printf("\n \t x(%d) = %lf", i + 1, b[i]); printf("\n \t Determinant = %lf \n", det); } else printf("\n \t SINGULAR MATRIX \n"); return 0; } /* Solves the system of equations [A]{x} = {B} using */ /* the Gaussian elimination method with partial pivoting. */ /* Parameters: */ /* n - number of equations */ /* a[n][n] - coefficient matrix */ /* b[n] - right-hand side vector */ /* *det - determinant of [A] */ int gauss (double a[][MAXSIZE], double b[], int n, double *det) { double tol, temp, mult; int npivot, i, j, l, k, flag; //initialization *det = 1.0; tol = 1e-30; //initial tolerance value npivot = 0; //mult = 0; //forward elimination for (k = 0; k < n; k++) { //search for max coefficient in pivot row- a[k][k] pivot element for (i = k + 1; i < n; i++) { if (fabs(a[i][k]) > fabs(a[k][k])) { //interchange row with maxium element with pivot row npivot++; for (l = 0; l < n; l++) { temp = a[i][l]; a[i][l] = a[k][l]; a[k][l] = temp; } temp = b[i]; b[i] = b[k]; b[k] = temp; } } //test for singularity if (fabs(a[k][k]) < tol) { //matrix is singular- terminate flag = 1; return flag; } //compute determinant- the product of the pivot elements *det = *det * a[k][k]; //eliminate the coefficients of X(I) for (i = k; i < n; i++) { mult = a[i][k] / a[k][k]; b[i] = b[i] - b[k] * mult; //compute constants for (j = k; j < n; j++) //compute coefficients a[i][j] = a[i][j] - a[k][j] * mult; } } //adjust the sign of the determinant if(npivot % 2 == 1) *det = *det * (-1.0); //backsubstitution b[n] = b[n] / a[n][n]; for(i = n - 1; i > 1; i--) { for(j = n; j > i + 1; j--) b[i] = b[i] - a[i][j] * b[j]; b[i] = b[i] / a[i - 1][i]; } flag = 0; return flag; } The solution should be: 1.058824, 1.823529, 0.882353 with det as -102.000000 Any insight is appreciated...

Read the article

Error with my Android Application httpGet

- by Coombes

Basically I'm getting a strange issue with my Android application, it's supposed to grab a JSON Array and print out some values, the class looks like this: ShowComedianActivity.class package com.example.connecttest; public class ShowComedianActivity extends Activity{ TextView name; TextView add; TextView email; TextView tel; String id; // Progress Dialog private ProgressDialog pDialog; //JSON Parser class JSONParser jsonParser = new JSONParser(); // Single Comedian url private static final String url_comedian_details = "http://86.9.71.17/connect/get_comedian_details.php"; // JSON Node names private static final String TAG_SUCCESS = "success"; private static final String TAG_COMEDIAN = "comedian"; private static final String TAG_ID = "id"; private static final String TAG_NAME = "name"; private static final String TAG_ADDRESS = "address"; private static final String TAG_EMAIL = "email"; private static final String TAG_TEL = "tel"; public void onCreate(Bundle savedInstanceState){ super.onCreate(savedInstanceState); setContentView(R.layout.show_comedian); // Getting Comedian Details from intent Intent i = getIntent(); // Getting id from intent id = i.getStringExtra(TAG_ID); new GetComedianDetails().execute(); } class GetComedianDetails extends AsyncTask<String, String, String>{ protected void onPreExecute(){ super.onPreExecute(); pDialog = new ProgressDialog(ShowComedianActivity.this); pDialog.setMessage("Fetching Comedian details. Please wait..."); pDialog.setIndeterminate(false); pDialog.setCancelable(true); pDialog.show(); } @Override protected String doInBackground(String... params) { runOnUiThread(new Runnable(){ public void run(){ int success; try{ //Building parameters List<NameValuePair> params = new ArrayList<NameValuePair>(); params.add(new BasicNameValuePair("id",id)); // Getting comedian details via HTTP request // Uses a GET request JSONObject json = jsonParser.makeHttpRequest(url_comedian_details, "GET", params); // Check Log for json response Log.d("Single Comedian details", json.toString()); //JSON Success tag success = json.getInt(TAG_SUCCESS); if(success == 1){ // Succesfully received product details JSONArray comedianObj = json.getJSONArray(TAG_COMEDIAN); //JSON Array // get first comedian object from JSON Array JSONObject comedian = comedianObj.getJSONObject(0); // comedian with id found name = (TextView) findViewById(R.id.name); add = (TextView) findViewById(R.id.add); email = (TextView) findViewById(R.id.email); tel = (TextView) findViewById(R.id.tel); // Set text to details name.setText(comedian.getString(TAG_NAME)); add.setText(comedian.getString(TAG_ADDRESS)); email.setText(comedian.getString(TAG_EMAIL)); tel.setText(comedian.getString(TAG_TEL)); } } catch (JSONException e){ e.printStackTrace(); } } }); return null; } } } And my JSON Parser class looks like: package com.example.connecttest; public class JSONParser { static InputStream is = null; static JSONObject jObj = null; static String json = ""; // constructor public JSONParser() { } // function get json from url // by making HTTP POST or GET method public JSONObject makeHttpRequest(String url, String method, List<NameValuePair> params) { // Making HTTP request try { // check for request method if(method == "POST"){ // request method is POST // defaultHttpClient DefaultHttpClient httpClient = new DefaultHttpClient(); HttpPost httpPost = new HttpPost(url); httpPost.setEntity(new UrlEncodedFormEntity(params)); HttpResponse httpResponse = httpClient.execute(httpPost); HttpEntity httpEntity = httpResponse.getEntity(); is = httpEntity.getContent(); }else if(method == "GET"){ // request method is GET DefaultHttpClient httpClient = new DefaultHttpClient(); String paramString = URLEncodedUtils.format(params, "utf-8"); url += "?" + paramString; HttpGet httpGet = new HttpGet(url); HttpResponse httpResponse = httpClient.execute(httpGet); HttpEntity httpEntity = httpResponse.getEntity(); is = httpEntity.getContent(); } } catch (UnsupportedEncodingException e) { e.printStackTrace(); } catch (ClientProtocolException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } try { BufferedReader reader = new BufferedReader(new InputStreamReader( is, "iso-8859-1"), 8); StringBuilder sb = new StringBuilder(); String line = null; while ((line = reader.readLine()) != null) { sb.append(line + "\n"); } is.close(); json = sb.toString(); } catch (Exception e) { Log.e("Buffer Error", "Error converting result " + e.toString()); } // try parse the string to a JSON object try { jObj = new JSONObject(json); } catch (JSONException e) { Log.e("JSON Parser", "Error parsing data " + e.toString()); } // return JSON String return jObj; } } Now when I run a debug it's querying the correct address with ?id=1 on the end of the URL, and when I navigate to that url I get the following JSON Array: {"success":1,"comedian":[{"id":"1","name":"Michael Coombes","address":"5 Trevethenick Road","email":"[email protected]","tel":"xxxxxxxxxxxx"}]} However my app just crashes, the log-cat report looks like this: 03-22 02:05:02.140: E/Trace(3776): error opening trace file: No such file or directory (2) 03-22 02:05:04.590: E/AndroidRuntime(3776): FATAL EXCEPTION: main 03-22 02:05:04.590: E/AndroidRuntime(3776): android.os.NetworkOnMainThreadException 03-22 02:05:04.590: E/AndroidRuntime(3776): at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1117) 03-22 02:05:04.590: E/AndroidRuntime(3776): at libcore.io.BlockGuardOs.connect(BlockGuardOs.java:84) 03-22 02:05:04.590: E/AndroidRuntime(3776): at libcore.io.IoBridge.connectErrno(IoBridge.java:127) 03-22 02:05:04.590: E/AndroidRuntime(3776): at libcore.io.IoBridge.connect(IoBridge.java:112) 03-22 02:05:04.590: E/AndroidRuntime(3776): at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:192) 03-22 02:05:04.590: E/AndroidRuntime(3776): at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:459) 03-22 02:05:04.590: E/AndroidRuntime(3776): at java.net.Socket.connect(Socket.java:842) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:119) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:144) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:164) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:119) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:360) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487) 03-22 02:05:04.590: E/AndroidRuntime(3776): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465) 03-22 02:05:04.590: E/AndroidRuntime(3776): at com.example.connecttest.JSONParser.makeHttpRequest(JSONParser.java:62) 03-22 02:05:04.590: E/AndroidRuntime(3776): at com.example.connecttest.ShowComedianActivity$GetComedianDetails$1.run(ShowComedianActivity.java:89) 03-22 02:05:04.590: E/AndroidRuntime(3776): at android.os.Handler.handleCallback(Handler.java:615) 03-22 02:05:04.590: E/AndroidRuntime(3776): at android.os.Handler.dispatchMessage(Handler.java:92) 03-22 02:05:04.590: E/AndroidRuntime(3776): at android.os.Looper.loop(Looper.java:137) 03-22 02:05:04.590: E/AndroidRuntime(3776): at android.app.ActivityThread.main(ActivityThread.java:4745) 03-22 02:05:04.590: E/AndroidRuntime(3776): at java.lang.reflect.Method.invokeNative(Native Method) 03-22 02:05:04.590: E/AndroidRuntime(3776): at java.lang.reflect.Method.invoke(Method.java:511) 03-22 02:05:04.590: E/AndroidRuntime(3776): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786) 03-22 02:05:04.590: E/AndroidRuntime(3776): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553) 03-22 02:05:04.590: E/AndroidRuntime(3776): at dalvik.system.NativeStart.main(Native Method) From this I'm guessing the error is in the jsonParser.makeHttpRequest however I can't for the life of me figure out what's going wrong and was hoping someone brighter than I could illuminate me.

Read the article

Problems with real-valued deep belief networks (of RBMs)

- by Junier

I am trying to recreate the results reported in Reducing the dimensionality of data with neural networks of autoencoding the olivetti face dataset with an adapted version of the MNIST digits matlab code, but am having some difficulty. It seems that no matter how much tweaking I do on the number of epochs, rates, or momentum the stacked RBMs are entering the fine-tuning stage with a large amount of error and consequently fail to improve much at the fine-tuning stage. I am also experiencing a similar problem on another real-valued dataset. For the first layer I am using a RBM with a smaller learning rate (as described in the paper) and with negdata = poshidstates*vishid' + repmat(visbiases,numcases,1); I'm fairly confident I am following the instructions found in the supporting material but I cannot achieve the correct errors. Is there something I am missing? See the code I'm using for real-valued visible unit RBMs below, and for the whole deep training. The rest of the code can be found here. rbmvislinear.m: epsilonw = 0.001; % Learning rate for weights epsilonvb = 0.001; % Learning rate for biases of visible units epsilonhb = 0.001; % Learning rate for biases of hidden units weightcost = 0.0002; initialmomentum = 0.5; finalmomentum = 0.9; [numcases numdims numbatches]=size(batchdata); if restart ==1, restart=0; epoch=1; % Initializing symmetric weights and biases. vishid = 0.1*randn(numdims, numhid); hidbiases = zeros(1,numhid); visbiases = zeros(1,numdims); poshidprobs = zeros(numcases,numhid); neghidprobs = zeros(numcases,numhid); posprods = zeros(numdims,numhid); negprods = zeros(numdims,numhid); vishidinc = zeros(numdims,numhid); hidbiasinc = zeros(1,numhid); visbiasinc = zeros(1,numdims); sigmainc = zeros(1,numhid); batchposhidprobs=zeros(numcases,numhid,numbatches); end for epoch = epoch:maxepoch, fprintf(1,'epoch %d\r',epoch); errsum=0; for batch = 1:numbatches, if (mod(batch,100)==0) fprintf(1,' %d ',batch); end %%%%%%%%% START POSITIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% data = batchdata(:,:,batch); poshidprobs = 1./(1 + exp(-data*vishid - repmat(hidbiases,numcases,1))); batchposhidprobs(:,:,batch)=poshidprobs; posprods = data' * poshidprobs; poshidact = sum(poshidprobs); posvisact = sum(data); %%%%%%%%% END OF POSITIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% poshidstates = poshidprobs > rand(numcases,numhid); %%%%%%%%% START NEGATIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);% + randn(numcases,numdims) if not using mean neghidprobs = 1./(1 + exp(-negdata*vishid - repmat(hidbiases,numcases,1))); negprods = negdata'*neghidprobs; neghidact = sum(neghidprobs); negvisact = sum(negdata); %%%%%%%%% END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% err= sum(sum( (data-negdata).^2 )); errsum = err + errsum; if epoch>5, momentum=finalmomentum; else momentum=initialmomentum; end; %%%%%%%%% UPDATE WEIGHTS AND BIASES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% vishidinc = momentum*vishidinc + ... epsilonw*( (posprods-negprods)/numcases - weightcost*vishid); visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact); hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact); vishid = vishid + vishidinc; visbiases = visbiases + visbiasinc; hidbiases = hidbiases + hidbiasinc; %%%%%%%%%%%%%%%% END OF UPDATES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% end fprintf(1, '\nepoch %4i error %f \n', epoch, errsum); end dofacedeepauto.m: clear all close all maxepoch=200; %In the Science paper we use maxepoch=50, but it works just fine. numhid=2000; numpen=1000; numpen2=500; numopen=30; fprintf(1,'Pretraining a deep autoencoder. \n'); fprintf(1,'The Science paper used 50 epochs. This uses %3i \n', maxepoch); load fdata %makeFaceData; [numcases numdims numbatches]=size(batchdata); fprintf(1,'Pretraining Layer 1 with RBM: %d-%d \n',numdims,numhid); restart=1; rbmvislinear; hidrecbiases=hidbiases; save mnistvh vishid hidrecbiases visbiases; maxepoch=50; fprintf(1,'\nPretraining Layer 2 with RBM: %d-%d \n',numhid,numpen); batchdata=batchposhidprobs; numhid=numpen; restart=1; rbm; hidpen=vishid; penrecbiases=hidbiases; hidgenbiases=visbiases; save mnisthp hidpen penrecbiases hidgenbiases; fprintf(1,'\nPretraining Layer 3 with RBM: %d-%d \n',numpen,numpen2); batchdata=batchposhidprobs; numhid=numpen2; restart=1; rbm; hidpen2=vishid; penrecbiases2=hidbiases; hidgenbiases2=visbiases; save mnisthp2 hidpen2 penrecbiases2 hidgenbiases2; fprintf(1,'\nPretraining Layer 4 with RBM: %d-%d \n',numpen2,numopen); batchdata=batchposhidprobs; numhid=numopen; restart=1; rbmhidlinear; hidtop=vishid; toprecbiases=hidbiases; topgenbiases=visbiases; save mnistpo hidtop toprecbiases topgenbiases; backpropface; Thanks for your time

Read the article

Httpsession with Spring 3 MVC

- by vipul12389

I want to use httpsession in Spring 3 MVC..i have searched all the web and got this solution..at http://forum.springsource.org/showthread.php?98850-Adding-to-stuff-to-the-session-while-using-ResponseBody Basically, My application auto authenticates user by getting winId and authorizes through LDAP..(Its a intranet site) Here is the flow of the application, 1. User enters Aplication url (http://localhost:8082/eIA_Mock_5) it has a welcome page (index.jsp) Index.jsp gets winId through jQuery and hits login.html (through Ajax) and passes windowsId login.html (Controller) authenticates through LDAP and gives back 'Valid' String as a response javascript, upon getting the correct response, redirects/loads welcome page i.e. goes to localhost:8082/eIA_Mock_5/welcome.html Now, i have filter associated with it..which checks for is session valid for each incoming request..Now the problem is even though i set data on to httpsession, yet the filter or any other controller fails to get the data through session as a result it doesnt proceeds further.. here is the code..and could you suggest what is wrong actually ?? Home_Controller.java @Controller public class Home_Controller { public static Log logger = LogFactory.getLog(Home_Controller.class); @RequestMapping(value={"/welcome"}) public ModelAndView loadWelcomePage(HttpServletRequest request,HttpServletResponse response) { ModelAndView mdv = new ModelAndView(); try{ /*HttpSession session = request.getSession(); UserMasterBean userBean = (UserMasterBean)session.getAttribute("userBean"); String userName=userBean.getWindowsId(); if(userName==null || userName.equalsIgnoreCase("")) { mdv.setViewName("homePage"); System.out.println("Unable to authenticate user "); logger.debug("Unable to authenticate user "); } else { System.out.println("Welcome User "+userName); logger.debug("Welcome User "+userName); */ mdv.setViewName("homePage"); /*}*/ } catch(Exception e){ logger.debug("inside authenticateUser ",e); e.printStackTrace(); } return mdv; } @RequestMapping(value = "/login", method = RequestMethod.GET) public @ResponseBody String authenticateUser(@RequestParam String userName,HttpSession session) { logger.debug("inside authenticateUser"); String returnResponse=new String(); try{ logger.debug("userName for Authentication "+userName); System.out.println("userName for Authentication "+userName); //HttpSession session = request.getSession(); if(userName==null || userName.trim().equalsIgnoreCase("")) returnResponse="Invalid"; else { System.out.println("uname "+userName); String ldapResponse = LDAPConnectUtil.isValidActiveDirectoryUser(userName, ""); if(ldapResponse.equalsIgnoreCase("true")) { returnResponse="Valid"; System.out.println(userName+" Authenticated"); logger.debug(userName+" Authenticated"); UserMasterBean userBean = new UserMasterBean(); userBean.setWindowsId(userName); //if(session.getAttribute("userBean")==null) session.setAttribute("userBean", userBean); } else { returnResponse="Invalid"; //session.setAttribute("userBean", null); System.out.println("Unable to Authenticate the user through Ldap"); logger.debug("Unable to Authenticate the user through Ldap"); } System.out.println("ldapResponse "+ldapResponse); logger.debug("ldapResponse "+ldapResponse); System.out.println("returnResponse "+returnResponse); } UserMasterBean u = (UserMasterBean)session.getAttribute("userBean"); System.out.println("winId "+u.getWindowsId()); } catch(Exception e){ e.printStackTrace(); logger.debug("Exception in authenticateUser ",e); } return returnResponse; } Filter public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) { System.out.println("in PageFilter"); boolean flag = false; HttpServletRequest objHttpServletRequest = (HttpServletRequest)request; HttpServletResponse objHttpServletResponse = (HttpServletResponse)response; HttpSession session = objHttpServletRequest.getSession(); String contextPath = objHttpServletRequest.getContextPath(); String servletPath = objHttpServletRequest.getSession().getServletContext().getRealPath(objHttpServletRequest.getServletPath()); logger.debug("contextPath :" + contextPath); logger.debug("servletPath :" + servletPath); System.out.println("in PageFilter, contextPath :" + contextPath); System.out.println("in PageFilter, servletPath :" + servletPath); if (servletPath.endsWith("\\") || servletPath.endsWith("/") || servletPath.indexOf("css") > 0 || servletPath.indexOf("jsp") > 0 || servletPath.indexOf("images") > 0 || servletPath.indexOf("js") > 0 || servletPath.endsWith("index.jsp") || servletPath.indexOf("xls") > 0 || servletPath.indexOf("ini") > 0 || servletPath.indexOf("login.html") > 0 || /*servletPath.endsWith("welcome.html") ||*/ servletPath.endsWith("logout.do") ) { System.out.println("User is trying to access allowed pages like Login.jsp, errorPage.jsp, js, images, css"); logger.debug("User is trying to access allowed pages like Login.jsp, errorPage.jsp, js, images, css"); flag = true; } if (flag== false) { System.out.println("flag = false"); if(session.getAttribute("userBean") == null) System.out.println("yes session.userbean is null"); if ((session != null) && (session.getAttribute("userBean") != null)) { System.out.println("session!=null && session.getAttribute(userId)!=null"); logger.debug("IF Part"); UserMasterBean userBean = (UserMasterBean)session.getAttribute("userBean"); String windowsId = userBean.getWindowsId(); logger.debug("User Id " + windowsId + " allowed access"); System.out.println("User Id " + windowsId + " allowed access"); flag = true; } else { System.out.println("else .....session!=null && session.getAttribute(userId)!=null"); logger.debug("Else Part"); flag = false; } } if (flag == true) { try { System.out.println("before chain.doFilter(request, response)"); chain.doFilter(request, response); } catch (Exception e) { e.printStackTrace(); try { objHttpServletResponse.sendRedirect(contextPath + "/logout.do"); } catch (Exception ex) { ex.printStackTrace(); } } } else { try { System.out.println("before sendRedirect"); objHttpServletResponse.sendRedirect(contextPath + "/jsp/errorPage.jsp"); } catch (Exception ex) { ex.printStackTrace(); } } System.out.println("end of PageFilter"); } Index.jsp <script type="text/javascript"> //alert("inside s13"); var WinNetwork = new ActiveXObject("WScript.Network"); var userName=WinNetwork.UserName; alert(userName); $.ajax({ url : "login.html", data : "userName="+userName, success : function(result) { alert("result == "+result); if(result=="Valid") window.location = "http://10.160.118.200:8082/eIA_Mock_5/welcome.html"; } }); </script> web.xml has a filter entry with URL pattern as * I am using spring 3 mvc

Read the article

Avoid Jquery Plugin Conflict

- by user1511579

on the same page i'm using this plugin: $g=jQuery.noConflict(); $g(function() { /* number of fieldsets */ var fieldsetCount = $g('#formElem').children().length; /* current position of fieldset / navigation link */ var current = 1; /* sum and save the widths of each one of the fieldsets set the final sum as the total width of the steps element */ var stepsWidth = 0; var widths = new Array(); $g('#steps .step').each(function(i){ var $step = $g(this); widths[i] = stepsWidth; stepsWidth += $step.width(); }); $g('#steps').width(stepsWidth); /* to avoid problems in IE, focus the first input of the form */ $g('#formElem').children(':first').find(':input:first').focus(); /* show the navigation bar */ $g('#navigation_form').show(); /* when clicking on a navigation link the form slides to the corresponding fieldset */ $g('#navigation_form a').bind('click',function(e){ var $this = $g(this); var prev = current; $this.closest('ul').find('li').removeClass('selected'); $this.parent().addClass('selected'); /* we store the position of the link in the current variable */ current = $this.parent().index() + 1; /* animate / slide to the next or to the corresponding fieldset. The order of the links in the navigation is the order of the fieldsets. Also, after sliding, we trigger the focus on the first input element of the new fieldset If we clicked on the last link (confirmation), then we validate all the fieldsets, otherwise we validate the previous one before the form slided */ $g('#steps').stop().animate({ marginLeft: '-' + widths[current-1] + 'px' },500,function(){ if(current == fieldsetCount) validateSteps(); else validateStep(prev); $g('#formElem').children(':nth-child('+ parseInt(current) +')').find(':input:first').focus(); }); e.preventDefault(); }); /* clicking on the tab (on the last input of each fieldset), makes the form slide to the next step */ $g('#formElem > fieldset').each(function(){ var $fieldset = $g(this); $fieldset.children(':last').find(':input').keydown(function(e){ if (e.which == 9){ $g('#navigation_form li:nth-child(' + (parseInt(current)+1) + ') a').click(); /* force the blur for validation */ $g(this).blur(); e.preventDefault(); } }); }); /* validates errors on all the fieldsets records if the Form has errors in $('#formElem').data() */ function validateSteps(){ var FormErrors = false; for(var i = 1; i < fieldsetCount; ++i){ var error = validateStep(i); if(error == -1) FormErrors = true; } $g('#formElem').data('errors',FormErrors); } /* validates one fieldset and returns -1 if errors found, or 1 if not */ function validateStep(step){ if(step == fieldsetCount) return; var error = 1; var hasError = false; $g('#formElem').children(':nth-child('+ parseInt(step) +')').find(':input:not(button)').each(function(){ var $this = $g(this); var valueLength = jQuery.trim($this.val()).length; if(valueLength == ''){ hasError = true; $this.css('background-color','#FFEDEF'); } else $this.css('background-color','#FFFFFF'); }); var $link = $g('#navigation_form li:nth-child(' + parseInt(step) + ') a'); $link.parent().find('.error,.checked').remove(); var valclass = 'checked'; if(hasError){ error = -1; valclass = 'error'; } $g('<span class="'+valclass+'"></span>').insertAfter($link); return error; } /* if there are errors don't allow the user to submit */ $g('#registerButton').bind('click',function(){ if($g('#formElem').data('errors')){ alert('Please correct the errors in the Form'); return false; } }); }); and this one: (function($){ $countCursos = 1; $countFormsA = 1; $countFormsB = 1; $.fn.addForms = function(idform){ var adicionar_curso = "<p>"+ " <label for='nome_curso'>Nome do Curso</label>"+ " <input id='nome_curso' name='nome_curso["+$countCursos+"]' type='text' />"+ " </p>"; var myform2 = "<table>"+ " <tr>"+ " <td>Field C</td>"+ " <td><input type='text' name='fieldc["+$countFormsA+"]'></td>"+ " <td>Field D ("+$countFormsA+"):</td>"+ " <td><textarea name='fieldd["+$countFormsA+"]'></textarea></td>"+ " <td><button>remove</button></td>"+ " </tr>"+ "</table>"; var myform3 = "<table>"+ " <tr>"+ " <td>Field C</td>"+ " <td><input type='text' name='fieldc["+$countFormsB+"]'></td>"+ " <td>Field D ("+$countFormsB+"):</td>"+ " <td><textarea name='fieldd["+$countFormsB+"]'></textarea></td>"+ " <td><button>remove</button></td>"+ " </tr>"+ "</table>"; if(idform=='novo_curso'){ alert(idform); adicionar_curso = $("<div>"+adicionar_curso+"</div>"); $("button", $(adicionar_curso)).click(function(){ $(this).parent().parent().remove(); }); $(this).append(adicionar_curso); $countCursos++; } if(idform=='mybutton1'){ alert(idform); myform2 = $("<div>"+myform2+"</div>"); $("button", $(myform2)).click(function(){ $(this).parent().parent().remove(); }); $(this).append(myform2); $countFormsA++; } if(idform=='mybutton2'){ alert(idform); myform3 = $("<div>"+myform3+"</div>"); $("button", $(myform3)).click(function(){ $(this).parent().parent().remove(); }); $(this).append(myform3); $countFormsB++; } }; })(jQuery); $(function(){ $("#mybutton1").bind("click", function(e){ e.preventDefault(); var idform=this.id; if($countFormsA<3){ $("#container1").addForms(idform); } }); }); $(function(){ $("#novo_curso").bind("click", function(e){ e.preventDefault(); var idform=this.id; alert(idform); if($countCursos<3){ $("#outro_curso").addForms(idform); } }); }); $(function(){ $("#mybutton2").bind("click", function(e){ e.preventDefault(); var idform=this.id; if($countFormsB<3){ $("#container2").addForms(idform); } }); }); My problem is the two are making conflict: I added previously the $g on the first to avoid conflict, but the truth is they don't work together, any hint how can i configure the second one to avoid this? Thanks in advance!

Read the article

MySQL Config File for Large System

- by Jonathon

We are running MySQL on a Windows 2003 Server Enterpise Edition box. MySQL is about the only program running on the box. We have approx. 8 slaves replicated to it, but my understanding is that having multiple slaves connecting to the same master does not significantly slow down performance, if at all. The master server has 16G RAM, 10 Terabyte drives in RAID 10, and four dual-core processors. From what I have seen from other sites, we have a really robust machine as our master db server. We just upgraded from a machine with only 4G RAM, but with similar hard drives, RAID, etc. It also ran Apache on it, so it was our db server and our application server. It was getting a little slow, so we split the db server onto this new machine and kept the application server on the first machine. We also distributed the application load amongst a few of our other slave servers, which also run the application. The problem is the new db server has mysqld.exe consuming 95-100% of CPU almost all the time and is really causing the app to run slowly. I know we have several queries and table structures that could be better optimized, but since they worked okay on the older, smaller server, I assume that our my.ini (MySQL config) file is not properly configured. Most of what I see on the net is for setting config files on small machines, so can anyone help me get the my.ini file correct for a large dedicated machine like ours? I just don't see how mysqld could get so bogged down! FYI: We have about 100 queries per second. We only use MyISAM tables, so skip-innodb is set in the ini file. And yes, I know it is reading the ini file correctly because I can change some settings (like the server-id and it will kill the server at startup). Here is the my.ini file: #MySQL Server Instance Configuration File # ---------------------------------------------------------------------- # Generated by the MySQL Server Instance Configuration Wizard # # # Installation Instructions # ---------------------------------------------------------------------- # # On Linux you can copy this file to /etc/my.cnf to set global options, # mysql-data-dir/my.cnf to set server-specific options # (@localstatedir@ for this installation) or to # ~/.my.cnf to set user-specific options. # # On Windows you should keep this file in the installation directory # of your server (e.g. C:\Program Files\MySQL\MySQL Server X.Y). To # make sure the server reads the config file use the startup option # "--defaults-file". # # To run run the server from the command line, execute this in a # command line shell, e.g. # mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # To install the server as a Windows service manually, execute this in a # command line shell, e.g. # mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini" # # And then execute this in a command line shell to start the server, e.g. # net start MySQLXY # # # Guildlines for editing this file # ---------------------------------------------------------------------- # # In this file, you can use all long options that the program supports. # If you want to know the options a program supports, start the program # with the "--help" option. # # More detailed information about the individual options can also be # found in the manual. # # # CLIENT SECTION # ---------------------------------------------------------------------- # # The following options will be read by MySQL client applications. # Note that only client applications shipped by MySQL are guaranteed # to read this section. If you want your own MySQL client program to # honor these values, you need to specify it as an option during the # MySQL client library initialization. # [client] port=3306 [mysql] default-character-set=latin1 # SERVER SECTION # ---------------------------------------------------------------------- # # The following options will be read by the MySQL Server. Make sure that # you have installed the server correctly (see above) so it reads this # file. # [mysqld] # The TCP/IP Port the MySQL Server will listen on port=3306 #Path to installation directory. All paths are usually resolved relative to this. basedir="D:/MySQL/" #Path to the database root datadir="D:/MySQL/data" # The default character set that will be used when a new schema or table is # created and no character set is defined default-character-set=latin1 # The default storage engine that will be used when create new tables when default-storage-engine=MYISAM # Set the SQL mode to strict #sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" # we changed this because there are a couple of queries that can get blocked otherwise sql-mode="" #performance configs skip-locking max_allowed_packet = 1M table_open_cache = 512 # The maximum amount of concurrent sessions the MySQL server will # allow. One of these connections will be reserved for a user with # SUPER privileges to allow the administrator to login even if the # connection limit has been reached. max_connections=1510 # Query cache is used to cache SELECT results and later return them # without actual executing the same query once again. Having the query # cache enabled may result in significant speed improvements, if your # have a lot of identical queries and rarely changing tables. See the # "Qcache_lowmem_prunes" status variable to check if the current value # is high enough for your load. # Note: In case your tables change very often or if your queries are # textually different every time, the query cache may result in a # slowdown instead of a performance improvement. query_cache_size=168M # The number of open tables for all threads. Increasing this value # increases the number of file descriptors that mysqld requires. # Therefore you have to make sure to set the amount of open files # allowed to at least 4096 in the variable "open-files-limit" in # section [mysqld_safe] table_cache=3020 # Maximum size for internal (in-memory) temporary tables. If a table # grows larger than this value, it is automatically converted to disk # based table This limitation is for a single table. There can be many # of them. tmp_table_size=30M # How many threads we should keep in a cache for reuse. When a client # disconnects, the client's threads are put in the cache if there aren't # more than thread_cache_size threads from before. This greatly reduces # the amount of thread creations needed if you have a lot of new # connections. (Normally this doesn't give a notable performance # improvement if you have a good thread implementation.) thread_cache_size=64 #*** MyISAM Specific options # The maximum size of the temporary file MySQL is allowed to use while # recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE. # If the file-size would be bigger than this, the index will be created # through the key cache (which is slower). myisam_max_sort_file_size=100G # If the temporary file used for fast index creation would be bigger # than using the key cache by the amount specified here, then prefer the # key cache method. This is mainly used to force long character keys in # large tables to use the slower key cache method to create the index. myisam_sort_buffer_size=64M # Size of the Key Buffer, used to cache index blocks for MyISAM tables. # Do not set it larger than 30% of your available memory, as some memory # is also required by the OS to cache rows. Even if you're not using # MyISAM tables, you should still set it to 8-64M as it will also be # used for internal temporary disk tables. key_buffer_size=3072M # Size of the buffer used for doing full table scans of MyISAM tables. # Allocated per thread, if a full scan is needed. read_buffer_size=2M read_rnd_buffer_size=8M # This buffer is allocated when MySQL needs to rebuild the index in # REPAIR, OPTIMZE, ALTER table statements as well as in LOAD DATA INFILE # into an empty table. It is allocated per thread so be careful with # large settings. sort_buffer_size=2M #*** INNODB Specific options *** innodb_data_home_dir="D:/MySQL InnoDB Datafiles/" # Use this option if you have a MySQL server with InnoDB support enabled # but you do not plan to use it. This will save memory and disk space # and speed up some things. skip-innodb # Additional memory pool that is used by InnoDB to store metadata # information. If InnoDB requires more memory for this purpose it will # start to allocate it from the OS. As this is fast enough on most # recent operating systems, you normally do not need to change this # value. SHOW INNODB STATUS will display the current amount used. innodb_additional_mem_pool_size=11M # If set to 1, InnoDB will flush (fsync) the transaction logs to the # disk at each commit, which offers full ACID behavior. If you are # willing to compromise this safety, and you are running small # transactions, you may set this to 0 or 2 to reduce disk I/O to the # logs. Value 0 means that the log is only written to the log file and # the log file flushed to disk approximately once per second. Value 2 # means the log is written to the log file at each commit, but the log # file is only flushed to disk approximately once per second. innodb_flush_log_at_trx_commit=1 # The size of the buffer InnoDB uses for buffering log data. As soon as # it is full, InnoDB will have to flush it to disk. As it is flushed # once per second anyway, it does not make sense to have it very large # (even with long transactions). innodb_log_buffer_size=6M # InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and # row data. The bigger you set this the less disk I/O is needed to # access data in tables. On a dedicated database server you may set this # parameter up to 80% of the machine physical memory size. Do not set it # too large, though, because competition of the physical memory may # cause paging in the operating system. Note that on 32bit systems you # might be limited to 2-3.5G of user level memory per process, so do not # set it too high. innodb_buffer_pool_size=500M # Size of each log file in a log group. You should set the combined size # of log files to about 25%-100% of your buffer pool size to avoid # unneeded buffer pool flush activity on log file overwrite. However, # note that a larger logfile size will increase the time needed for the # recovery process. innodb_log_file_size=100M # Number of threads allowed inside the InnoDB kernel. The optimal value # depends highly on the application, hardware as well as the OS # scheduler properties. A too high value may lead to thread thrashing. innodb_thread_concurrency=10 #replication settings (this is the master) log-bin=log server-id = 1 Thanks for all the help. It is greatly appreciated.

Read the article

SINGLE SIGN ON SECURITY THREAT! FACEBOOK access_token broadcast in the open/clear

- by MOKANA

Subsequent to my posting there was a remark made that this was not really a question but I thought I did indeed postulate one. So that there is no ambiquity here is the question with a lead in: Since there is no data sent from Facebook during the Canvas Load process that is not at some point divulged, including the access_token, session and other data that could uniquely identify a user, does any one see any other way other than adding one more layer, i.e., a password, sent over the wire via HTTPS along with the access_toekn, that will insure unique untampered with security by the user? Using Wireshark I captured the local broadcast while loading my Canvas Application page. I was hugely surprised to see the access_token broadcast in the open, viewable for any one to see. This access_token is appended to any https call to the Facebook OpenGraph API. Using facebook as a single click log on has now raised huge concerns for me. It is stored in a session object in memory and the cookie is cleared upon app termination and after reviewing the FB.Init calls I saw a lot of HTTPS calls so I assumed the access_token was always encrypted. But last night I saw in the status bar a call from what was simply an http call that included the App ID so I felt I should sniff the Application Canvas load sequence. Today I did sniff the broadcast and in the attached image you can see that there are http calls with the access_token being broadcast in the open and clear for anyone to gain access to. Am I missing something, is what I am seeing and my interpretation really correct. If any one can sniff and get the access_token they can theorically make calls to the Graph API via https, even though the call back would still need to be the site established in Facebook's application set up. But what is truly a security threat is anyone using the access_token for access to their own site. I do not see the value of a single sign on via Facebook if the only thing that was established as secure was the access_token - becuase for what I can see it clearly is not secure. Access tokens that never have an expire date do not change. Access_tokens are different for every user, to access to another site could be held tight to just a single user, but compromising even a single user's data is unacceptable. http://www.creatingstory.com/images/InTheOpen.png Went back and did more research on this: FINDINGS: Went back an re ran the canvas application to verify that it was not any of my code that was not broadcasting. In this call: HTTP GET /connect.php/en_US/js/CacheData HTTP/1.1 The USER ID is clearly visible in the cookie. So USER_ID's are fully visible, but they are already. Anyone can go to pretty much any ones page and hover over the image and see the USER ID. So no big threat. APP_ID are also easily obtainable - but . . . http://www.creatingstory.com/images/InTheOpen2.png The above file clearly shows the FULL ACCESS TOKEN clearly in the OPEN via a Facebook initiated call. Am I wrong. TELL ME I AM WRONG because I want to be wrong about this. I have since reset my app secret so I am showing the real sniff of the Canvas Page being loaded. Additional data 02/20/2011: @ifaour - I appreciate the time you took to compile your response. I am pretty familiar with the OAuth process and have a pretty solid understanding of the signed_request unpacking and utilization of the access_token. I perform a substantial amount of my processing on the server and my Facebook server side flows are all complete and function without any flaw that I know of. The application secret is secure and never passed to the front end application and is also changed regularly. I am being as fanatical about security as I can be, knowing there is so much I don’t know that could come back and bite me. Two huge access_token issues: The issues concern the possible utilization of the access_token from the USER AGENT (browser). During the FB.INIT() process of the Facebook JavaScript SDK, a cookie is created as well as an object in memory called a session object. This object, along with the cookie contain the access_token, session, a secret, and uid and status of the connection. The session object is structured such that is supports both the new OAuth and the legacy flows. With OAuth, the access_token and status are pretty much al that is used in the session object. The first issue is that the access_token is used to make HTTPS calls to the GRAPH API. If you had the access_token, you could do this from any browser: https://graph.facebook.com/220439?access_token=... and it will return a ton of information about the user. So any one with the access token can gain access to a Facebook account. You can also make additional calls to any info the user has granted access to the application tied to the access_token. At first I thought that a call into the GRAPH had to have a Callback to the URL established in the App Setup, but I tested it as mentioned below and it will return info back right into the browser. Adding that callback feature would be a good idea I think, tightens things up a bit. The second issue is utilization of some unique private secured data that identifies the user to the third party data base, i.e., like in my case, I would use a single sign on to populate user information into my database using this unique secured data item (i.e., access_token which contains the APP ID, the USER ID, and a hashed with secret sequence). None of this is a problem on the server side. You get a signed_request, you unpack it with secret, make HTTPS calls, get HTTPS responses back. When a user has information entered via the USER AGENT(browser) that must be stored via a POST, this unique secured data element would be sent via HTTPS such that they are validated prior to data base insertion. However, If there is NO secured piece of unique data that is supplied via the single sign on process, then there is no way to guarantee unauthorized access. The access_token is the one piece of data that is utilized by Facebook to make the HTTPS calls into the GRAPH API. it is considered unique in regards to BOTH the USER and the APPLICATION and is initially secure via the signed_request packaging. If however, it is subsequently transmitted in the clear and if I can sniff the wire and obtain the access_token, then I can pretend to be the application and gain the information they have authorized the application to see. I tried the above example from a Safari and IE browser and it returned all of my information to me in the browser. In conclusion, the access_token is part of the signed_request and that is how the application initially obtains it. After OAuth authentication and authorization, i.e., the USER has logged into Facebook and then runs your app, the access_token is stored as mentioned above and I have sniffed it such that I see it stored in a Cookie that is transmitted over the wire, resulting in there being NO UNIQUE SECURED IDENTIFIABLE piece of information that can be used to support interaction with the database, or in other words, unless there were one more piece of secure data sent along with the access_token to my database, i.e., a password, I would not be able to discern if it is a legitimate call. Luckily I utilized secure AJAX via POST and the call has to come from the same domain, but I am sure there is a way to hijack that. I am totally open to any ideas on this topic on how to uniquely identify my USERS other than adding another layer (password) via this single sign on process or if someone would just share with me that I read and analyzed my data incorrectly and that the access_token is always secure over the wire. Mahalo nui loa in advance.

Read the article

Javascript not working in IE but works in Firefox chrome

- by user1290528

So i have the following php page with a java script that gets the total of items based on their quatity, then inputs the total into a text box for each item. In ie the text boxes are being filled with $NaN. While in firefox, chrome the text boxes are filled with the correct values. Any help would be graatly appreciated. <?php echo $_SESSION['SESS_MEMBER_ID']; require_once('auth.php'); ?> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <title>Breakfast Menu</title> <link href="loginmodule.css" rel="stylesheet" type="text/css"> <script type='text/javascript'> var totalarray=new Array(); var totalarray2= new Array(); var runningtotal = 0; var runningtotal2 = 0; var discount = .2; var discounttotal = 0; var discount1 = 0; runningtotal = runningtotal * 1; runningtotal2 = runningtotal2 * 1; function displayResult(price,init) { var newstring = "quantity"+init; var totstring = "total"+init; var quantity = document.getElementById(newstring).value; var quantity = parseFloat(quantity); var test = price * quantity; var test = test.toFixed(2); document.getElementById(newstring).value = quantity; document.getElementById(totstring).value = "$" + test; totalarray[init] = test; getTotal(); } function getTotal(){ runningtotal = 0; var i=0; for (i=0;i<totalarray.length;i++){ totalarray[i] = totalarray[i] *1; runningtotal = runningtotal + totalarray[i]; discounttotal = totalarray[i] * discount; discounttotal = totalarray[i] - discounttotal; This line is where IE shows its first error document.getElementById('totalcost').value="$" + runningtotal.toFixed(2); } var orderpart1 = document.getElementById('totalcost').value; var orderpart1 = orderpart1.substr(1); var orderpart1 = orderpart1 * 1; var orderpart2 = document.getElementById('totalcost2').value; var orderpart2 = orderpart2.substr(1); var orderpart2 = orderpart2 * 1; var ordertot = orderpart1 + orderpart2; document.getElementById('ordertotal').value ="$"+ ordertot.toFixed(2) } function displayResult2(price2,init2) { var newstring2 = "quantity2"+init2; var totstring2 = "total2"+init2; var quantity2 = document.getElementById(newstring2).value; var quantity2 = parseFloat(quantity2); var test2 = price2 * quantity2; var test2 = test2.toFixed(2); document.getElementById(newstring2).value = quantity2; document.getElementById(totstring2).value = "$" + test2; totalarray2[init2] = test2; getTotal2(); } function getTotal2(){ runningtotal2 = 0; var i=0; for (i=0;i<totalarray2.length;i++){ totalarray2[i] = totalarray2[i] *1; runningtotal2 = runningtotal2+ totalarray2[i]; This is where IE shows its second error document.getElementById('totalcost2').value="$" + runningtotal2.toFixed(2); }//IE Shows Second error here var orderpart1 = document.getElementById('totalcost').value; var orderpart1 = orderpart1.substr(1); var orderpart1 = orderpart1 * 1; var orderpart2 = document.getElementById('totalcost2').value; var orderpart2 = orderpart2.substr(1); var orderpart2 = orderpart2 * 1; var ordertot = orderpart1 + orderpart2; document.getElementById('ordertotal').value ="$"+ ordertot.toFixed(2); } </script> </head> <body> <?php include("newnew.php"); ?> <td style="vertical-align: top; width: 80%; height:80%;"><br> <div style="text-align: center;"> <form action="testplaceorder.php" method="post" onSubmit="return confirm('Are you sure?');"> <h4>Employee Breakfast Order Form</h4> <h1 align="left">Breakfest Foods</h1> <table border='0' cellpadding='0' cellspacing='0'> <tr> <td> <table width="100%" border="1"> <tr> <th>Item&nbsp&nbsp&nbsp&nbsp&nbsp</th> <th>Price&nbsp&nbsp&nbsp&nbsp&nbsp </th> <th>Quantity&nbsp&nbsp&nbsp&nbsp&nbsp</th> <th>Total&nbsp&nbsp&nbsp&nbsp&nbsp</th> </tr> <?php mysql_connect("localhost", "seniorproject", "farmingdale123") or die(mysql_error()); mysql_select_db("fsenior") or die(mysql_error()); $result = mysql_query("SELECT name, price,foodid FROM Food where foodtype='br'") or die(mysql_error()); $init = 0; while(list($name, $price, $brId) = mysql_fetch_row($result)) { echo "<tr> <td>$name</td> <td>\$$price</td> <td><select name='quantity$init' id='quantity$init' onchange='displayResult($price,$init)'><option>0</option><option>1</option><option>2</option><option>3</option><option>4</option><option>5</option><option>6</option><option>7</option><option>8</option><option>9</option></td> <td><input name='total$init' type='text' id='total$init' readonly='readonly' value='\$0.00'></td> </tr>" ; echo "<script type='text/javascript'>displayResult($price,$init);</script>"; $foodname = "'SESS_FOODNAME_" . $init . "'"; $foodid = "'SESS_FOODID_" . $init."'"; $_SESSION[$foodname] = $name; $_SESSION[$foodid] = $brId; $init = $init+1; } $_SESSION['SESS_INIT'] = $init; ?> <tr> <td></td> <td></td> <td>Total Cost</td> <td><input name='totalcost' type='text' id='totalcost' readonly='readonly' value='$0.00'></td> </tr> <tr><td></td><td></td><td>Discount</td><td><input name='discountvalue1' id ='discountvalue1' type='text' readonly='readonly' value='20%'></td> </tr> <tr><td></td><td></td><td>Total After Discount</td><td><input name='discounttotal1' id ='discounttotal1' type='text' readonly='readonly' value='$0.00'></td></tr> </table> <tr> <td><br></td> </tr> </table> <h1 align="left">Breakfest Drinks</h1> <table border='0' cellpadding='0' cellspacing='0'> <tr> <td> <table width="100%" border="1"> <tr> <th>Item&nbsp&nbsp&nbsp&nbsp&nbsp</th> <th>Price&nbsp&nbsp&nbsp&nbsp&nbsp </th> <th>Quantity&nbsp&nbsp&nbsp&nbsp&nbsp</th> <th>Total&nbsp&nbsp&nbsp&nbsp&nbsp</th> </tr> <?php mysql_connect("localhost", "****", "***") or die(mysql_error()); mysql_select_db("fsenior") or die(mysql_error()); $result2 = mysql_query("SELECT drinkname, price,drinkid FROM Drinks where drinktype='br'") or die(mysql_error()); $init2 = 0; while(list($name2, $price2, $brId2) = mysql_fetch_row($result2)) { echo "<tr> <td>$name2</td> <td>\$$price2</td> <td><select name='quantity2$init2' id='quantity2$init2' onchange='displayResult2($price2,$init2)'><option>0</option><option>1</option><option>2</option><option>3</option><option>4</option><option>5</option><option>6</option><option>7</option><option>8</option><option>9</option></td> <td><input name='total2$init2' type='text' id='total2$init2' readonly='readonly' value='\$0.00'></td> </tr>" ; echo "<script type='text/javascript'>displayResult2($price2,$init2);</script>"; $drinkname = "'SESS_DRINKNAME_" . $init2 . "'"; $drinkid = "'SESS_DRINKID_" . $init2."'"; $_SESSION[$drinkname] = $name2; $_SESSION[$drinkid] = $brId2; $init2 = $init2+1; } $_SESSION['SESS_INIT2'] = $init2; ?> <tr> <td></td> <td></td> <td>Total Cost</td> <td><input name='totalcost2' type='text' id='totalcost2' readonly='readonly' value='$0.00'></td> </tr> </table> <tr> <td><br></td> </tr> </table> <table border="2"> <tr><td>Total Order Cost:</td><td> <?php echo "<input name='ordertotal' type='text' id='ordertotal' readonly='readonly' value='\$0.00'></td></table>"; ?> <p align="left"><input type='submit' name='submit' value='Submit'/></p> </form> </div></td> </tr> </tbody> </table></td> </tr> </tbody> </table> </body> </html>

Read the article

graph with database

- by Flip_novidade

I need to make two graphs with data coming from the database. I do not know where I am going wrong. If someone can show me the correct way, or provide any examples. must be two graphs, a graph of a specific student another graph of all students thank you public class NotasBean { private Notas notas; private Notas selectedNotas; private List<Notas> filtroNotass; public Notas getNotas() { return notas; } public void setNotas(Notas notas) { this.notas = notas; } public Notas getSelectedNotas() { return selectedNotas; } public void setSelectedNotas(Notas selectedNotas) { this.selectedNotas = selectedNotas; } public List<Notas> getFiltroNotass() { return filtroNotass; } public void setFiltroNotass(List<Notas> filtroNotass) { this.filtroNotass = filtroNotass; } public void prepararAdicionarNotas(){ notas = new Notas(); } public void adicionarNotas(){ dao.NotasDao obj_dao = new dao.NotasDao(); obj_dao.save(notas); } } package dao; import java.sql.PreparedStatement; import java.sql.ResultSet; import java.sql.Statement; import java.util.ArrayList; import java.util.List; import javax.faces.application.FacesMessage; import javax.faces.context.FacesContext; import model.Aluno; import model.Notas; import model.Notas; public class NotasDao { private Conexao obj_conexao; public NotasDao() { obj_conexao = new Conexao(); } public List<Notas> list() { List<Notas> array_registros = new ArrayList<Notas>(); try { String sql = "Select alu_in_ra, dis_st_sigla, ald_fl_p1, ald_fl_p2, ald_fl_p3, ald_fl_trab1, ald_fl_trab2 from cad_aluno_disciplina"; Statement comando_sql = (Statement) obj_conexao.getConexao() .createStatement(); ResultSet obj_result = comando_sql.executeQuery(sql); while (obj_result.next()) { Notas obj_notas = new Notas(); obj_notas.setAlura(obj_result.getInt("alu_in_ra")); obj_notas.setDiscsigla(obj_result.getString("dis_st_sigla")); obj_notas.setP1(obj_result.getInt("ald_fl_p1")); obj_notas.setP2(obj_result.getInt("ald_fl_p2")); obj_notas.setP3(obj_result.getInt("ald_fl_p3")); obj_notas.setTrb1(obj_result.getInt("ald_fl_trab1")); obj_notas.setTrb2(obj_result.getInt("ald_fl_trab2")); array_registros.add(obj_notas); } } catch (Exception e) { System.out.println("Erro no select" + e.getMessage()); } finally { obj_conexao.fecharConexao(); } return array_registros; } public void select(Aluno obj_aluno){ FacesContext mensagem = FacesContext.getCurrentInstance(); try{ String comando_sql = "Select alu_in_ra, dis_st_sigla, ald_fl_p1, ald_fl_p2, ald_fl_p3, ald_fl_trab1, ald_fl_trab2 from cad_aluno_disciplina where alu_in_ra=?"; PreparedStatement obj_sql = (PreparedStatement) obj_conexao.getConexao().prepareStatement(comando_sql); obj_sql.setInt(1, obj_aluno.getRa()); obj_sql.executeUpdate(); mensagem.addMessage(null, new FacesMessage(FacesMessage.SEVERITY_WARN, "Erro ao selecionar aluno!","Snif")); }catch(Exception e){ mensagem.addMessage(null, new FacesMessage(FacesMessage.SEVERITY_FATAL, "Erro na inclusão: "+e.getMessage()," Ocoreu o erro: "+e.getMessage())); }finally{ obj_conexao.fecharConexao(); } return; } }//fecha a classe package model; import java.io.Serializable; public class Notas implements Serializable{ private static final long serialVersionUID = 1L; private int alura; private String discsigla; private float p1; private float p2; private float p3; private float trb1; private float trb2; public Notas() { } public Notas (int alura, String discsigla, float p1, float p2, float p3, float trb1, float trb2){ super(); this.alura=alura; this.discsigla=discsigla; this.p1=p1; this.p2=p2; this.p3=p3; this.trb1=trb1; this.trb2=trb2; } public int getAlura() { return alura; } public void setAlura(int alura) { this.alura = alura; } public String getDiscsigla() { return discsigla; } public void setDiscsigla(String discsigla) { this.discsigla = discsigla; } public float getP1() { return p1; } public void setP1(float p1) { this.p1 = p1; } public float getP2() { return p2; } public void setP2(float p2) { this.p2 = p2; } public float getP3() { return p3; } public void setP3(float p3) { this.p3 = p3; } public float getTrb1() { return trb1; } public void setTrb1(float trb1) { this.trb1 = trb1; } public float getTrb2() { return trb2; } public void setTrb2(float trb2) { this.trb2 = trb2; } } <p:panel header="Grafico Notas Aluno" style="width: 550px"> <p:lineChart id="linear" value="#{notasDao.aluno.alura}" var="notas" xfield="#{notas.alura}" height="300px" width="500px" style="chartStyle"> <p:chartSeries label="Prova 1" value="#{notas.p1}" /> <p:chartSeries label="Prova 2" value="#{notas.p2}" /> <p:chartSeries label="Prova 3" value="#{notas.p3}" /> <p:chartSeries label="Trabalho 1" value="#{notas.trb1}" /> <p:chartSeries label="Trabalho 2" value="#{notas.trb2}" /> </p:lineChart> </p:panel> <p:panel header="Grafico Notas" style="width: 550px"> <p:lineChart id="linear" value="#{notasDao.natas}" var="notas" xfield="#{notas.p1}" height="300px" width="500px" style="chartStyle"> <p:chartSeries label="Prova 1" value="#{notas.p1}" /> <p:chartSeries label="Prova 2" value="#{notas.p2}" /> <p:chartSeries label="Prova 3" value="#{notas.p3}" /> <p:chartSeries label="Trabalho 1" value="#{notas.trb1}" /> <p:chartSeries label="Trabalho 2" value="#{notas.trb2}" /> </p:lineChart> </p:panel>

Read the article

Is there a Telecommunications Reference Architecture?

- by raul.goycoolea

@font-face { font-family: "Arial"; }@font-face { font-family: "Courier New"; }@font-face { font-family: "Wingdings"; }@font-face { font-family: "Cambria"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }div.Section1 { page: Section1; }ol { margin-bottom: 0cm; }ul { margin-bottom: 0cm; } Abstract Reference architecture provides needed architectural information that can be provided in advance to an enterprise to enable consistent architectural best practices. Enterprise Reference Architecture helps business owners to actualize their strategies, vision, objectives, and principles. It evaluates the IT systems, based on Reference Architecture goals, principles, and standards. It helps to reduce IT costs by increasing functionality, availability, scalability, etc. Telecom Reference Architecture provides customers with the flexibility to view bundled service bills online with the provision of multiple services. It provides real-time, flexible billing and charging systems, to handle complex promotions, discounts, and settlements with multiple parties. This paper attempts to describe the Reference Architecture for the Telecom Enterprises. It lays the foundation for a Telecom Reference Architecture by articulating the requirements, drivers, and pitfalls for telecom service providers. It describes generic reference architecture for telecom enterprises and moves on to explain how to achieve Enterprise Reference Architecture by using SOA. Introduction A Reference Architecture provides a methodology, set of practices, template, and standards based on a set of successful solutions implemented earlier. These solutions have been generalized and structured for the depiction of both a logical and a physical architecture, based on the harvesting of a set of patterns that describe observations in a number of successful implementations. It helps as a reference for the various architectures that an enterprise can implement to solve various problems. It can be used as the starting point or the point of comparisons for various departments/business entities of a company, or for the various companies for an enterprise. It provides multiple views for multiple stakeholders. Major artifacts of the Enterprise Reference Architecture are methodologies, standards, metadata, documents, design patterns, etc. Purpose of Reference Architecture In most cases, architects spend a lot of time researching, investigating, defining, and re-arguing architectural decisions. It is like reinventing the wheel as their peers in other organizations or even the same organization have already spent a lot of time and effort defining their own architectural practices. This prevents an organization from learning from its own experiences and applying that knowledge for increased effectiveness. Reference architecture provides missing architectural information that can be provided in advance to project team members to enable consistent architectural best practices. Enterprise Reference Architecture helps an enterprise to achieve the following at the abstract level: · Reference architecture is more of a communication channel to an enterprise · Helps the business owners to accommodate to their strategies, vision, objectives, and principles. · Evaluates the IT systems based on Reference Architecture Principles · Reduces IT spending through increasing functionality, availability, scalability, etc · A Real-time Integration Model helps to reduce the latency of the data updates Is used to define a single source of Information · Provides a clear view on how to manage information and security · Defines the policy around the data ownership, product boundaries, etc. · Helps with cost optimization across project and solution portfolios by eliminating unused or duplicate investments and assets · Has a shorter implementation time and cost Once the reference architecture is in place, the set of architectural principles, standards, reference models, and best practices ensure that the aligned investments have the greatest possible likelihood of success in both the near term and the long term (TCO). Common pitfalls for Telecom Service Providers Telecom Reference Architecture serves as the first step towards maturity for a telecom service provider. During the course of our assignments/experiences with telecom players, we have come across the following observations – Some of these indicate a lack of maturity of the telecom service provider: · In markets that are growing and not so mature, it has been observed that telcos have a significant amount of in-house or home-grown applications. In some of these markets, the growth has been so rapid that IT has been unable to cope with business demands. Telcos have shown a tendency to come up with workarounds in their IT applications so as to meet business needs. · Even for core functions like provisioning or mediation, some telcos have tried to manage with home-grown applications. · Most of the applications do not have the required scalability or maintainability to sustain growth in volumes or functionality. · Applications face interoperability issues with other applications in the operator's landscape. Integrating a new application or network element requires considerable effort on the part of the other applications. · Application boundaries are not clear, and functionality that is not in the initial scope of that application gets pushed onto it. This results in the development of the multiple, small applications without proper boundaries. · Usage of Legacy OSS/BSS systems, poor Integration across Multiple COTS Products and Internal Systems. Most of the Integrations are developed on ad-hoc basis and Point-to-Point Integration. · Redundancy of the business functions in different applications • Fragmented data across the different applications and no integrated view of the strategic data • Lot of performance Issues due to the usage of the complex integration across OSS and BSS systems However, this is where the maturity of the telecom industry as a whole can be of help. The collaborative efforts of telcos to overcome some of these problems have resulted in bodies like the TM Forum. They have come up with frameworks for business processes, data, applications, and technology for telecom service providers. These could be a good starting point for telcos to clean up their enterprise landscape. Industry Trends in Telecom Reference Architecture Telecom reference architectures are evolving rapidly because telcos are facing business and IT challenges. “The reality is that there probably is no killer application, no silver bullet that the telcos can latch onto to carry them into a 21st Century.... Instead, there are probably hundreds – perhaps thousands – of niche applications.... And the only way to find which of these works for you is to try out lots of them, ramp up the ones that work, and discontinue the ones that fail.” – Martin Creaner President & CTO TM Forum. The following trends have been observed in telecom reference architecture: · Transformation of business structures to align with customer requirements · Adoption of more Internet-like technical architectures. The Web 2.0 concept is increasingly being used. · Virtualization of the traditional operations support system (OSS) · Adoption of SOA to support development of IP-based services · Adoption of frameworks like Service Delivery Platforms (SDPs) and IP Multimedia Subsystem · (IMS) to enable seamless deployment of various services over fixed and mobile networks · Replacement of in-house, customized, and stove-piped OSS/BSS with standards-based COTS products · Compliance with industry standards and frameworks like eTOM, SID, and TAM to enable seamless integration with other standards-based products Drivers of Reference Architecture The drivers of the Reference Architecture are Reference Architecture Goals, Principles, and Enterprise Vision and Telecom Transformation. The details are depicted below diagram. @font-face { font-family: "Cambria"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoCaption, li.MsoCaption, div.MsoCaption { margin: 0cm 0cm 10pt; font-size: 9pt; font-family: "Times New Roman"; color: rgb(79, 129, 189); font-weight: bold; }div.Section1 { page: Section1; } Figure 1. Drivers for Reference Architecture @font-face { font-family: "Arial"; }@font-face { font-family: "Courier New"; }@font-face { font-family: "Wingdings"; }@font-face { font-family: "Cambria"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }div.Section1 { page: Section1; }ol { margin-bottom: 0cm; }ul { margin-bottom: 0cm; } Today’s telecom reference architectures should seamlessly integrate traditional legacy-based applications and transition to next-generation network technologies (e.g., IP multimedia subsystems). This has resulted in new requirements for flexible, real-time billing and OSS/BSS systems and implications on the service provider’s organizational requirements and structure. Telecom reference architectures are today expected to: · Integrate voice, messaging, email and other VAS over fixed and mobile networks, back end systems · Be able to provision multiple services and service bundles • Deliver converged voice, video and data services · Leverage the existing Network Infrastructure · Provide real-time, flexible billing and charging systems to handle complex promotions, discounts, and settlements with multiple parties. · Support charging of advanced data services such as VoIP, On-Demand, Services (e.g. Video), IMS/SIP Services, Mobile Money, Content Services and IPTV. · Help in faster deployment of new services • Serve as an effective platform for collaboration between network IT and business organizations · Harness the potential of converging technology, networks, devices and content to develop multimedia services and solutions of ever-increasing sophistication on a single Internet Protocol (IP) · Ensure better service delivery and zero revenue leakage through real-time balance and credit management · Lower operating costs to drive profitability Enterprise Reference Architecture The Enterprise Reference Architecture (RA) fills the gap between the concepts and vocabulary defined by the reference model and the implementation. Reference architecture provides detailed architectural information in a common format such that solutions can be repeatedly designed and deployed in a consistent, high-quality, supportable fashion. This paper attempts to describe the Reference Architecture for the Telecom Application Usage and how to achieve the Enterprise Level Reference Architecture using SOA. • Telecom Reference Architecture • Enterprise SOA based Reference Architecture Telecom Reference Architecture Tele Management Forum’s New Generation Operations Systems and Software (NGOSS) is an architectural framework for organizing, integrating, and implementing telecom systems. NGOSS is a component-based framework consisting of the following elements: · The enhanced Telecom Operations Map (eTOM) is a business process framework. · The Shared Information Data (SID) model provides a comprehensive information framework that may be specialized for the needs of a particular organization. · The Telecom Application Map (TAM) is an application framework to depict the functional footprint of applications, relative to the horizontal processes within eTOM. · The Technology Neutral Architecture (TNA) is an integrated framework. TNA is an architecture that is sustainable through technology changes. NGOSS Architecture Standards are: · Centralized data · Loosely coupled distributed systems · Application components/re-use · A technology-neutral system framework with technology specific implementations · Interoperability to service provider data/processes · Allows more re-use of business components across multiple business scenarios · Workflow automation The traditional operator systems architecture consists of four layers, · Business Support System (BSS) layer, with focus toward customers and business partners. Manages order, subscriber, pricing, rating, and billing information. · Operations Support System (OSS) layer, built around product, service, and resource inventories. · Networks layer – consists of Network elements and 3rd Party Systems. · Integration Layer – to maximize application communication and overall solution flexibility. Reference architecture for telecom enterprises is depicted below. @font-face { font-family: "Arial"; }@font-face { font-family: "Courier New"; }@font-face { font-family: "Wingdings"; }@font-face { font-family: "Cambria"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoCaption, li.MsoCaption, div.MsoCaption { margin: 0cm 0cm 10pt; font-size: 9pt; font-family: "Times New Roman"; color: rgb(79, 129, 189); font-weight: bold; }p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }div.Section1 { page: Section1; }ol { margin-bottom: 0cm; }ul { margin-bottom: 0cm; } Figure 2. Telecom Reference Architecture The major building blocks of any Telecom Service Provider architecture are as follows: 1. Customer Relationship Management CRM encompasses the end-to-end lifecycle of the customer: customer initiation/acquisition, sales, ordering, and service activation, customer care and support, proactive campaigns, cross sell/up sell, and retention/loyalty. CRM also includes the collection of customer information and its application to personalize, customize, and integrate delivery of service to a customer, as well as to identify opportunities for increasing the value of the customer to the enterprise. The key functionalities related to Customer Relationship Management are · Manage the end-to-end lifecycle of a customer request for products. · Create and manage customer profiles. · Manage all interactions with customers – inquiries, requests, and responses. · Provide updates to Billing and other south bound systems on customer/account related updates such as customer/ account creation, deletion, modification, request bills, final bill, duplicate bills, credit limits through Middleware. · Work with Order Management System, Product, and Service Management components within CRM. · Manage customer preferences – Involve all the touch points and channels to the customer, including contact center, retail stores, dealers, self service, and field service, as well as via any media (phone, face to face, web, mobile device, chat, email, SMS, mail, the customer's bill, etc.). · Support single interface for customer contact details, preferences, account details, offers, customer premise equipment, bill details, bill cycle details, and customer interactions. CRM applications interact with customers through customer touch points like portals, point-of-sale terminals, interactive voice response systems, etc. The requests by customers are sent via fulfillment/provisioning to billing system for ordering processing. 2. Billing and Revenue Management Billing and Revenue Management handles the collection of appropriate usage records and production of timely and accurate bills – for providing pre-bill usage information and billing to customers; for processing their payments; and for performing payment collections. In addition, it handles customer inquiries about bills, provides billing inquiry status, and is responsible for resolving billing problems to the customer's satisfaction in a timely manner. This process grouping also supports prepayment for services. The key functionalities provided by these applications are · To ensure that enterprise revenue is billed and invoices delivered appropriately to customers. · To manage customers’ billing accounts, process their payments, perform payment collections, and monitor the status of the account balance. · To ensure the timely and effective fulfillment of all customer bill inquiries and complaints. · Collect the usage records from mediation and ensure appropriate rating and discounting of all usage and pricing. · Support revenue sharing; split charging where usage is guided to an account different from the service consumer. · Support prepaid and post-paid rating. · Send notification on approach / exceeding the usage thresholds as enforced by the subscribed offer, and / or as setup by the customer. · Support prepaid, post paid, and hybrid (where some services are prepaid and the rest of the services post paid) customers and conversion from post paid to prepaid, and vice versa. · Support different billing function requirements like charge prorating, promotion, discount, adjustment, waiver, write-off, account receivable, GL Interface, late payment fee, credit control, dunning, account or service suspension, re-activation, expiry, termination, contract violation penalty, etc. · Initiate direct debit to collect payment against an invoice outstanding. · Send notification to Middleware on different events; for example, payment receipt, pre-suspension, threshold exceed, etc. Billing systems typically get usage data from mediation systems for rating and billing. They get provisioning requests from order management systems and inquiries from CRM systems. Convergent and real-time billing systems can directly get usage details from network elements. 3. Mediation Mediation systems transform/translate the Raw or Native Usage Data Records into a general format that is acceptable to billing for their rating purposes. The following lists the high-level roles and responsibilities executed by the Mediation system in the end-to-end solution. · Collect Usage Data Records from different data sources – like network elements, routers, servers – via different protocol and interfaces. · Process Usage Data Records – Mediation will process Usage Data Records as per the source format. · Validate Usage Data Records from each source. · Segregates Usage Data Records coming from each source to multiple, based on the segregation requirement of end Application. · Aggregates Usage Data Records based on the aggregation rule if any from different sources. · Consolidates multiple Usage Data Records from each source. · Delivers formatted Usage Data Records to different end application like Billing, Interconnect, Fraud Management, etc. · Generates audit trail for incoming Usage Data Records and keeps track of all the Usage Data Records at various stages of mediation process. · Checks duplicate Usage Data Records across files for a given time window. 4. Fulfillment This area is responsible for providing customers with their requested products in a timely and correct manner. It translates the customer's business or personal need into a solution that can be delivered using the specific products in the enterprise's portfolio. This process informs the customers of the status of their purchase order, and ensures completion on time, as well as ensuring a delighted customer. These processes are responsible for accepting and issuing orders. They deal with pre-order feasibility determination, credit authorization, order issuance, order status and tracking, customer update on customer order activities, and customer notification on order completion. Order management and provisioning applications fall into this category. The key functionalities provided by these applications are · Issuing new customer orders, modifying open customer orders, or canceling open customer orders; · Verifying whether specific non-standard offerings sought by customers are feasible and supportable; · Checking the credit worthiness of customers as part of the customer order process; · Testing the completed offering to ensure it is working correctly; · Updating of the Customer Inventory Database to reflect that the specific product offering has been allocated, modified, or cancelled; · Assigning and tracking customer provisioning activities; · Managing customer provisioning jeopardy conditions; and · Reporting progress on customer orders and other processes to customer. These applications typically get orders from CRM systems. They interact with network elements and billing systems for fulfillment of orders. 5. Enterprise Management This process area includes those processes that manage enterprise-wide activities and needs, or have application within the enterprise as a whole. They encompass all business management processes that · Are necessary to support the whole of the enterprise, including processes for financial management, legal management, regulatory management, process, cost, and quality management, etc.; · Are responsible for setting corporate policies, strategies, and directions, and for providing guidelines and targets for the whole of the business, including strategy development and planning for areas, such as Enterprise Architecture, that are integral to the direction and development of the business; · Occur throughout the enterprise, including processes for project management, performance assessments, cost assessments, etc. (i) Enterprise Risk Management: Enterprise Risk Management focuses on assuring that risks and threats to the enterprise value and/or reputation are identified, and appropriate controls are in place to minimize or eliminate the identified risks. The identified risks may be physical or logical/virtual. Successful risk management ensures that the enterprise can support its mission critical operations, processes, applications, and communications in the face of serious incidents such as security threats/violations and fraud attempts. Two key areas covered in Risk Management by telecom operators are: · Revenue Assurance: Revenue assurance system will be responsible for identifying revenue loss scenarios across components/systems, and will help in rectifying the problems. The following lists the high-level roles and responsibilities executed by the Revenue Assurance system in the end-to-end solution. o Identify all usage information dropped when networks are being upgraded. o Interconnect bill verification. o Identify where services are routinely provisioned but never billed. o Identify poor sales policies that are intensifying collections problems. o Find leakage where usage is sent to error bucket and never billed for. o Find leakage where field service, CRM, and network build-out are not optimized. · Fraud Management: Involves collecting data from different systems to identify abnormalities in traffic patterns, usage patterns, and subscription patterns to report suspicious activity that might suggest fraudulent usage of resources, resulting in revenue losses to the operator. The key roles and responsibilities of the system component are as follows: o Fraud management system will capture and monitor high usage (over a certain threshold) in terms of duration, value, and number of calls for each subscriber. The threshold for each subscriber is decided by the system and fixed automatically. o Fraud management will be able to detect the unauthorized access to services for certain subscribers. These subscribers may have been provided unauthorized services by employees. The component will raise the alert to the operator the very first time of such illegal calls or calls which are not billed. o The solution will be to have an alarm management system that will deliver alarms to the operator/provider whenever it detects a fraud, thus minimizing fraud by catching it the first time it occurs. o The Fraud Management system will be capable of interfacing with switches, mediation systems, and billing systems (ii) Knowledge Management This process focuses on knowledge management, technology research within the enterprise, and the evaluation of potential technology acquisitions. Key responsibilities of knowledge base management are to · Maintain knowledge base – Creation and updating of knowledge base on ongoing basis. · Search knowledge base – Search of knowledge base on keywords or category browse · Maintain metadata – Management of metadata on knowledge base to ensure effective management and search. · Run report generator. · Provide content – Add content to the knowledge base, e.g., user guides, operational manual, etc. (iii) Document Management It focuses on maintaining a repository of all electronic documents or images of paper documents relevant to the enterprise using a system. (iv) Data Management It manages data as a valuable resource for any enterprise. For telecom enterprises, the typical areas covered are Master Data Management, Data Warehousing, and Business Intelligence. It is also responsible for data governance, security, quality, and database management. Key responsibilities of Data Management are · Using ETL, extract the data from CRM, Billing, web content, ERP, campaign management, financial, network operations, asset management info, customer contact data, customer measures, benchmarks, process data, e.g., process inputs, outputs, and measures, into Enterprise Data Warehouse. · Management of data traceability with source, data related business rules/decisions, data quality, data cleansing data reconciliation, competitors data – storage for all the enterprise data (customer profiles, products, offers, revenues, etc.) · Get online update through night time replication or physical backup process at regular frequency. · Provide the data access to business intelligence and other systems for their analysis, report generation, and use. (v) Business Intelligence It uses the Enterprise Data to provide the various analysis and reports that contain prospects and analytics for customer retention, acquisition of new customers due to the offers, and SLAs. It will generate right and optimized plans – bolt-ons for the customers. The following lists the high-level roles and responsibilities executed by the Business Intelligence system at the Enterprise Level: · It will do Pattern analysis and reports problem. · It will do Data Analysis – Statistical analysis, data profiling, affinity analysis of data, customer segment wise usage patterns on offers, products, service and revenue generation against services and customer segments. · It will do Performance (business, system, and forecast) analysis, churn propensity, response time, and SLAs analysis. · It will support for online and offline analysis, and report drill down capability. · It will collect, store, and report various SLA data. · It will provide the necessary intelligence for marketing and working on campaigns, etc., with cost benefit analysis and predictions. It will advise on customer promotions with additional services based on loyalty and credit history of customer · It will Interface with Enterprise Data Management system for data to run reports and analysis tasks. It will interface with the campaign schedules, based on historical success evidence. (vi) Stakeholder and External Relations Management It manages the enterprise's relationship with stakeholders and outside entities. Stakeholders include shareholders, employee organizations, etc. Outside entities include regulators, local community, and unions. Some of the processes within this grouping are Shareholder Relations, External Affairs, Labor Relations, and Public Relations. (vii) Enterprise Resource Planning It is used to manage internal and external resources, including tangible assets, financial resources, materials, and human resources. Its purpose is to facilitate the flow of information between all business functions inside the boundaries of the enterprise and manage the connections to outside stakeholders. ERP systems consolidate all business operations into a uniform and enterprise wide system environment. The key roles and responsibilities for Enterprise System are given below: · It will handle responsibilities such as core accounting, financial, and management reporting. · It will interface with CRM for capturing customer account and details. · It will interface with billing to capture the billing revenue and other financial data. · It will be responsible for executing the dunning process. Billing will send the required feed to ERP for execution of dunning. · It will interface with the CRM and Billing through batch interfaces. Enterprise management systems are like horizontals in the enterprise and typically interact with all major telecom systems. E.g., an ERP system interacts with CRM, Fulfillment, and Billing systems for different kinds of data exchanges. 6. External Interfaces/Touch Points The typical external parties are customers, suppliers/partners, employees, shareholders, and other stakeholders. External interactions from/to a Service Provider to other parties can be achieved by a variety of mechanisms, including: · Exchange of emails or faxes · Call Centers · Web Portals · Business-to-Business (B2B) automated transactions These applications provide an Internet technology driven interface to external parties to undertake a variety of business functions directly for themselves. These can provide fully or partially automated service to external parties through various touch points. Typical characteristics of these touch points are · Pre-integrated self-service system, including stand-alone web framework or integration front end with a portal engine · Self services layer exposing atomic web services/APIs for reuse by multiple systems across the architectural environment · Portlets driven connectivity exposing data and services interoperability through a portal engine or web application These touch points mostly interact with the CRM systems for requests, inquiries, and responses. 7. Middleware The component will be primarily responsible for integrating the different systems components under a common platform. It should provide a Standards-Based Platform for building Service Oriented Architecture and Composite Applications. The following lists the high-level roles and responsibilities executed by the Middleware component in the end-to-end solution. · As an integration framework, covering to and fro interfaces · Provide a web service framework with service registry. · Support SOA framework with SOA service registry. · Each of the interfaces from / to Middleware to other components would handle data transformation, translation, and mapping of data points. · Receive data from the caller / activate and/or forward the data to the recipient system in XML format. · Use standard XML for data exchange. · Provide the response back to the service/call initiator. · Provide a tracking until the response completion. · Keep a store transitional data against each call/transaction. · Interface through Middleware to get any information that is possible and allowed from the existing systems to enterprise systems; e.g., customer profile and customer history, etc. · Provide the data in a common unified format to the SOA calls across systems, and follow the Enterprise Architecture directive. · Provide an audit trail for all transactions being handled by the component. 8. Network Elements The term Network Element means a facility or equipment used in the provision of a telecommunications service. Such terms also includes features, functions, and capabilities that are provided by means of such facility or equipment, including subscriber numbers, databases, signaling systems, and information sufficient for billing and collection or used in the transmission, routing, or other provision of a telecommunications service. Typical network elements in a GSM network are Home Location Register (HLR), Intelligent Network (IN), Mobile Switching Center (MSC), SMS Center (SMSC), and network elements for other value added services like Push-to-talk (PTT), Ring Back Tone (RBT), etc. Network elements are invoked when subscribers use their telecom devices for any kind of usage. These elements generate usage data and pass it on to downstream systems like mediation and billing system for rating and billing. They also integrate with provisioning systems for order/service fulfillment. 9. 3rd Party Applications 3rd Party systems are applications like content providers, payment gateways, point of sale terminals, and databases/applications maintained by the Government. Depending on applicability and the type of functionality provided by 3rd party applications, the integration with different telecom systems like CRM, provisioning, and billing will be done. 10. Service Delivery Platform A service delivery platform (SDP) provides the architecture for the rapid deployment, provisioning, execution, management, and billing of value added telecom services. SDPs are based on the concept of SOA and layered architecture. They support the delivery of voice, data services, and content in network and device-independent fashion. They allow application developers to aggregate network capabilities, services, and sources of content. SDPs typically contain layers for web services exposure, service application development, and network abstraction. SOA Reference Architecture SOA concept is based on the principle of developing reusable business service and building applications by composing those services, instead of building monolithic applications in silos. It’s about bridging the gap between business and IT through a set of business-aligned IT services, using a set of design principles, patterns, and techniques. In an SOA, resources are made available to participants in a value net, enterprise, line of business (typically spanning multiple applications within an enterprise or across multiple enterprises). It consists of a set of business-aligned IT services that collectively fulfill an organization’s business processes and goals. We can choreograph these services into composite applications and invoke them through standard protocols. SOA, apart from agility and reusability, enables: · The business to specify processes as orchestrations of reusable services · Technology agnostic business design, with technology hidden behind service interface · A contractual-like interaction between business and IT, based on service SLAs · Accountability and governance, better aligned to business services · Applications interconnections untangling by allowing access only through service interfaces, reducing the daunting side effects of change · Reduced pressure to replace legacy and extended lifetime for legacy applications, through encapsulation in services · A Cloud Computing paradigm, using web services technologies, that makes possible service outsourcing on an on-demand, utility-like, pay-per-usage basis The following section represents the Reference Architecture of logical view for the Telecom Solution. The new custom built application needs to align with this logical architecture in the long run to achieve EA benefits. Packaged implementation applications, such as ERP billing applications, need to expose their functions as service providers (as other applications consume) and interact with other applications as service consumers. COT applications need to expose services through wrappers such as adapters to utilize existing resources and at the same time achieve Enterprise Architecture goal and objectives. The following are the various layers for Enterprise level deployment of SOA. This diagram captures the abstract view of Enterprise SOA layers and important components of each layer. Layered architecture means decomposition of services such that most interactions occur between adjacent layers. However, there is no strict rule that top layers should not directly communicate with bottom layers. The diagram below represents the important logical pieces that would result from overall SOA transformation. @font-face { font-family: "Arial"; }@font-face { font-family: "Courier New"; }@font-face { font-family: "Wingdings"; }@font-face { font-family: "Cambria"; }p.MsoNormal, li.MsoNormal, div.MsoNormal { margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoCaption, li.MsoCaption, div.MsoCaption { margin: 0cm 0cm 10pt; font-size: 9pt; font-family: "Times New Roman"; color: rgb(79, 129, 189); font-weight: bold; }p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast { margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: "Times New Roman"; }div.Section1 { page: Section1; }ol { margin-bottom: 0cm; }ul { margin-bottom: 0cm; } Figure 3. Enterprise SOA Reference Architecture 1. Operational System Layer: This layer consists of all packaged applications like CRM, ERP, custom built applications, COTS based applications like Billing, Revenue Management, Fulfilment, and the Enterprise databases that are essential and contribute directly or indirectly to the Enterprise OSS/BSS Transformation. ERP holds the data of Asset Lifecycle Management, Supply Chain, and Advanced Procurement and Human Capital Management, etc. CRM holds the data related to Order, Sales, and Marketing, Customer Care, Partner Relationship Management, Loyalty, etc. Content Management handles Enterprise Search and Query. Billing application consists of the following components: · Collections Management, Customer Billing Management, Invoices, Real-Time Rating, Discounting, and Applying of Charges · Enterprise databases will hold both the application and service data, whether structured or unstructured. MDM - Master data majorly consists of Customer, Order, Product, and Service Data. 2. Enterprise Component Layer: This layer consists of the Application Services and Common Services that are responsible for realizing the functionality and maintaining the QoS of the exposed services. This layer uses container-based technologies such as application servers to implement the components, workload management, high availability, and load balancing. Application Services: This Service Layer enables application, technology, and database abstraction so that the complex accessing logic is hidden from the other service layers. This is a basic service layer, which exposes application functionalities and data as reusable services. The three types of the Application access services are: · Application Access Service: This Service Layer exposes application level functionalities as a reusable service between BSS to BSS and BSS to OSS integration. This layer is enabled using disparate technology such as Web Service, Integration Servers, and Adaptors, etc. · Data Access Service: This Service Layer exposes application data services as a reusable reference data service. This is done via direct interaction with application data. and provides the federated query. · Network Access Service: This Service Layer exposes provisioning layer as a reusable service from OSS to OSS integration. This integration service emphasizes the need for high performance, stateless process flows, and distributed design. Common Services encompasses management of structured, semi-structured, and unstructured data such as information services, portal services, interaction services, infrastructure services, and security services, etc. 3. Integration Layer: This consists of service infrastructure components like service bus, service gateway for partner integration, service registry, service repository, and BPEL processor. Service bus will carry the service invocation payloads/messages between consumers and providers. The other important functions expected from it are itinerary based routing, distributed caching of routing information, transformations, and all qualities of service for messaging-like reliability, scalability, and availability, etc. Service registry will hold all contracts (wsdl) of services, and it helps developers to locate or discover service during design time or runtime. • BPEL processor would be useful in orchestrating the services to compose a complex business scenario or process. • Workflow and business rules management are also required to support manual triggering of certain activities within business process. based on the rules setup and also the state machine information. Application, data, and service mediation layer typically forms the overall composite application development framework or SOA Framework. 4. Business Process Layer: These are typically the intermediate services layer and represent Shared Business Process Services. At Enterprise Level, these services are from Customer Management, Order Management, Billing, Finance, and Asset Management application domains. 5. Access Layer: This layer consists of portals for Enterprise and provides a single view of Enterprise information management and dashboard services. 6. Channel Layer: This consists of various devices; applications that form part of extended enterprise; browsers through which users access the applications. 7. Client Layer: This designates the different types of users accessing the enterprise applications. The type of user typically would be an important factor in determining the level of access to applications. 8. Vertical pieces like management, monitoring, security, and development cut across all horizontal layers Management and monitoring involves all aspects of SOA-like services, SLAs, and other QoS lifecycle processes for both applications and services surrounding SOA governance. 9. EA Governance, Reference Architecture, Roadmap, Principles, and Best Practices: EA Governance is important in terms of providing the overall direction to SOA implementation within the enterprise. This involves board-level involvement, in addition to business and IT executives. At a high level, this involves managing the SOA projects implementation, managing SOA infrastructure, and controlling the entire effort through all fine-tuned IT processes in accordance with COBIT (Control Objectives for Information Technology). Devising tools and techniques to promote reuse culture, and the SOA way of doing things needs competency centers to be established in addition to training the workforce to take up new roles that are suited to SOA journey. Conclusions Reference Architectures can serve as the basis for disparate architecture efforts throughout the organization, even if they use different tools and technologies. Reference architectures provide best practices and approaches in the independent way a vendor deals with technology and standards. Reference Architectures model the abstract architectural elements for an enterprise independent of the technologies, protocols, and products that are used to implement an SOA. Telecom enterprises today are facing significant business and technology challenges due to growing competition, a multitude of services, and convergence. Adopting architectural best practices could go a long way in meeting these challenges. The use of SOA-based architecture for communication to each of the external systems like Billing, CRM, etc., in OSS/BSS system has made the architecture very loosely coupled, with greater flexibility. Any change in the external systems would be absorbed at the Integration Layer without affecting the rest of the ecosystem. The use of a Business Process Management (BPM) tool makes the management and maintenance of the business processes easy, with better performance in terms of lead time, quality, and cost. Since the Architecture is based on standards, it will lower the cost of deploying and managing OSS/BSS applications over their lifecycles.

Read the article

Oracle Support Master Note for Troubleshooting Advanced Queuing and Oracle Streams Propagation Issues (Doc ID 233099.1)

- by faye.todd(at)oracle.com

Master Note for Troubleshooting Advanced Queuing and Oracle Streams Propagation Issues (Doc ID 233099.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this Document Purpose Last Review Date Instructions for the Reader Troubleshooting Details 1. Scope and Application 2. Definitions and Classifications 3. How to Use This Guide 4. Basic AQ Propagation Troubleshooting 5. Additional Troubleshooting Steps for AQ Propagation of User-Enqueued and Dequeued Messages 6. Additional Troubleshooting Steps for Propagation in an Oracle Streams Environment 7. Performance Issues References Applies to: Oracle Server - Enterprise Edition - Version: 8.1.7.0 to 11.2.0.2 - Release: 8.1.7 to 11.2Information in this document applies to any platform. Purpose This document presents a step-by-step methodology for troubleshooting and resolving problems with Advanced Queuing Propagation in both Streams and basic Advanced Queuing environments. It also serves as a master reference for other more specific notes on Oracle Streams Propagation and Advanced Queuing Propagation issues. Last Review Date December 20, 2010 Instructions for the Reader A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting. Troubleshooting Details 1. Scope and Application This note is intended for Database Administrators of Oracle databases where issues are being encountered with propagating messages between advanced queues, whether the queues are used for user-created messaging systems or for Oracle Streams. It contains troubleshooting steps and links to notes for further problem resolution.It can also be used a template to document a problem when it is necessary to engage Oracle Support Services. Knowing what is NOT happening can frequently speed up the resolution process by focusing solely on the pertinent problem area. This guide is divided into five parts: Section 2: Definitions and Classifications (discusses the different types and features of propagations possible - helpful for understanding the rest of the guide) Section 3: How to Use this Guide (to be used as a start part for determining the scope of the problem and what sections to consult) Section 4. Basic AQ propagation troubleshooting (applies to both AQ propagation of user enqueued and dequeued messages as well as Oracle Streams propagations) Section 5. Additional troubleshooting steps for AQ propagation of user enqueued and dequeued messages Section 6. Additional troubleshooting steps for Oracle Streams propagation Section 7. Performance issues 2. Definitions and Classifications Given the potential scope of issues that can be encountered with AQ propagation, the first recommended step is to do some basic diagnosis to determine the type of problem that is being encountered. 2.1. What Type of Propagation is Being Used? 2.1.1. Buffered Messaging For an advanced queue, messages can be maintained on disk (persistent messaging) or in memory (buffered messaging). To determine if a queue is buffered or not, reference the GV_$BUFFERED_QUEUES view. If the queue does not appear in this view, it is persistent. 2.1.2. Propagation mode - queue-to-dblink vs queue-to-queue As of 10.2, an AQ propagation can also be defined as queue-to-dblink, or queue-to-queue: queue-to-dblink: The propagation delivers messages or events from the source queue to all subscribing queues at the destination database identified by the dblink. A single propagation schedule is used to propagate messages to all subscribing queues. Hence any changes made to this schedule will affect message delivery to all the subscribing queues. This mode does not support multiple propagations from the same source queue to the same target database. queue-to-queue: Added in 10.2, this propagation mode delivers messages or events from the source queue to a specific destination queue identified on the database link. This allows the user to have fine-grained control on the propagation schedule for message delivery. This new propagation mode also supports transparent failover when propagating to a destination Oracle RAC system. With queue-to-queue propagation, you are no longer required to re-point a database link if the owner instance of the queue fails on Oracle RAC. This mode supports multiple propagations to the same target database if the target queues are different. The default is queue-to-dblink. To verify if queue-to-queue propagation is being used, in non-Streams environments query DBA_QUEUE_SCHEDULES.DESTINATION - if a remote queue is listed along with the remote database link, then queue-to-queue propagation is being used. For Streams environments, the DBA_PROPAGATION.QUEUE_TO_QUEUE column can be checked.See the following note for a method to switch between the two modes:Document 827473.1 How to alter propagation from queue-to-queue to queue-to-dblink 2.1.3. Combined Capture and Apply (CCA) for Streams In 11g Oracle Streams environments, an optimization called Combined Capture and Apply (CCA) is implemented by default when possible. Although a propagation is configured in this case, Streams does not use it; instead it passes information directly from capture to an apply receiver. To see if CCA is in use: COLUMN CAPTURE_NAME HEADING 'Capture Name' FORMAT A30COLUMN OPTIMIZATION HEADING 'CCA Mode?' FORMAT A10SELECT CAPTURE_NAME, DECODE(OPTIMIZATION,0, 'No','Yes') OPTIMIZATIONFROM V$STREAMS_CAPTURE; Also, see the following note:Document 463820.1 Streams Combined Capture and Apply in 11g 2.2. Queue Table Compatibility There are three types of queue table compatibility. In more recent databases, queue tables may be present in all three modes of compatibility: 8.0 - earliest version, deprecated in 10.2 onwards 8.1 - support added for RAC, asynchronous notification, secure queues, queue level access control, rule-based subscribers, separate storage of history information 10.0 - if the database is in 10.1-compatible mode, then the default value for queue table compatibility is 10.0 2.3. Single vs Multiple Consumer Queue Tables If more than one recipient can dequeue a message from a queue, then its queue table is multiple consumer. You can propagate messages from a multiple-consumer queue to a single-consumer queue. Propagation from a single-consumer queue to a multiple-consumer queue is not possible. 3. How to Use This Guide 3.1. Are Messages Being Propagated at All, or is the Propagation Just Slow? Run the following query on the source database for the propagation (assuming that it is running): select TOTAL_NUMBER from DBA_QUEUE_SCHEDULES where QNAME='<source_queue_name>'; If TOTAL_NUMBER is increasing, then propagation is most likely functioning, although it may be slow. For performance issues, see Section 7. 3.2. Propagation Between Persistent User-Created Queues See Sections 4 and 5 (and optionally Section 6 if performance is an issue). 3.3. Propagation Between Buffered User-Created Queues See Sections 4, 5, and 6 (and optionally Section 7 if performance is an issue). 3.4. Propagation between Oracle Streams Queues (without Combined Capture and Apply (CCA) Optimization) See Sections 4 and 6 (and optionally Section 7 if performance is an issue). 3.5. Propagation between Oracle Streams Queues (with Combined Capture and Apply (CCA) Optimization) Although an AQ propagation is not used directly in this case, some characteristics of the message transfer are inferred from the propagation parameters used. Some parts of Sections 4 and 6 still apply. 3.6. Messaging Gateway Propagations This note does not apply to Messaging Gateway propagations. 4. Basic AQ Propagation Troubleshooting 4.1. Double-check Your Code Make sure that you are consistent in your usage of the database link(s) names, queue names, etc. It may be useful to plot a diagram of which queues are connected via which database links to make sure that the logical structure is correct. 4.2. Verify that Job Queue Processes are Running 4.2.1. Versions 10.2 and Lower - DBA_JOBS Package For versions 10.2 and lower, a scheduled propagation is managed by DBMS_JOB package. The propagation is performed by job queue process background processes. Therefore we need to verify that there are sufficient processes available for the propagation process. We should have at least 4 job queue processes running and preferably more depending on the number of other jobs running in the database. It should be noted that for AQ specific work, AQ will only ever use half of the job queue processes available.An issue caused by an inadequate job queue processes parameter setting is described in the following note:Document 298015.1 Kwqjswproc:Excep After Loop: Assigning To Self 4.2.1.1. Job Queue Processes in Initalization Parameter File The parameter JOB_QUEUE_PROCESSES in the init.ora/spfile should be > 0. The value can be changed dynamically via connect / as sysdbaalter system set JOB_QUEUE_PROCESSES=10; 4.2.1.2. Job Queue Processes in Memory The following command will show how many job queue processes are currentlyin use by this instance (this may be different than what is in the init.ora/spfile): connect / as sysdbashow parameter job; 4.2.1.3. OS PIDs Corresponding to Job Queue Processes Identify the operating system process ids (spids) of job queue processes involved in propagation via select p.SPID, p.PROGRAM from V$PROCESS p, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j where s.SID=jr.SID and s.PADDR=p.ADDR and jr.JOB=j.JOBand j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%'; and these SPIDs can be used to check at the operating system level that they exist.In 8i a job queue process will have a name similar to: ora_snp1_<instance_name>.In 9i onwards you will see a coordinator process: ora_cjq0_ and multiple slave processes: ora_jnnn_<instance_name>, where nnn is an integer between 1 and 999. 4.2.2. Version 11.1 and Above - Oracle Scheduler In version 11.1 and above, Oracle Scheduler is used to perform AQ and Streams propagations. Oracle Scheduler automatically tunes the number of slave processes for these jobs based on the load on the computer system, and the JOB_QUEUE_PROCESSES initialization parameter is only used to specify the maximum number of slave processes. Therefore, the JOB_QUEUE_PROCESSES initialization parameter does not need to be set (it defaults to a very high number), unless you want to limit the number of slaves that can be created. If JOB_QUEUE_PROCESSES = 0, no propagation jobs will run.See the following note for a discussion of Oracle Streams 11g and Oracle Scheduler:Document 1083608.1 11g Streams and Oracle Scheduler 4.2.2.1. Job Queue Processes in Initalization Parameter File The parameter JOB_QUEUE_PROCESSES in the init.ora/spfile should be > 0, and preferably be left at its default value. The value can be changed dynamically via connect / as sysdbaalter system set JOB_QUEUE_PROCESSES=10; To set the JOB_QUEUE_PROCESSES parameter to its default value, run: connect / as sysdbaalter system reset JOB_QUEUE_PROCESSES; and then bounce the instance. 4.2.2.2. Job Queue Processes in Memory The following command will show how many job queue processes are currently in use by this instance (this may be different than what is in the init.ora/spfile): connect / as sysdbashow parameter job; 4.2.2.3. OS PIDs Corresponding to Job Queue Processes Identify the operating system process ids (SPIDs) of job queue processes involved in propagation via col PROGRAM for a30select p.SPID, p.PROGRAM, j.JOB_namefrom v$PROCESS p, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j where s.SID=jr.SESSION_ID and s.PADDR=p.ADDRand jr.JOB_name=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%'; and these SPIDs can be used to check at the operating system level that they exist.You will see a coordinator process: ora_cjq0_ and multiple slave processes: ora_jnnn_<instance_name>, where nnn is an integer between 1 and 999. 4.3. Check the Alert Log and Any Associated Trace Files The first place to check for propagation failures is the alert logs at all sites (local and if relevant all remote sites). When a job queue process attempts to execute a schedule and fails it will always write an error stack to the alert log. This error stack will also be written in a job queue process trace file, which will be written to the BACKGROUND_DUMP_DEST location for 10.2 and below, and in the DIAGNOSTIC_DEST location for 11g. The fact that errors are written to the alert log demonstrates that the schedule is executing. This means that the problem could be with the set up of the schedule. In this example the ORA-02068 demonstrates that the failure was at the remote site. Further investigation revealed that the remote database was not open, hence the ORA-03114 error. Starting the database resolved the problem. Thu Feb 14 10:40:05 2002 Propagation Schedule for (AQADM.MULTIPLEQ, SHANE816.WORLD) encountered following error:ORA-04052: error occurred when looking up Remote object [email protected]: error occurred at recursive SQL level 4ORA-02068: following severe error from SHANE816ORA-03114: not connected to ORACLEORA-06512: at "SYS.DBMS_AQADM_SYS", line 4770ORA-06512: at "SYS.DBMS_AQADM", line 548ORA-06512: at line 1 Other potential errors that may be written to the alert log can be found in the following notes:Document 827184.1 AQ Propagation with CLOB data types Fails with ORA-22990 (11.1)Document 846297.1 AQ Propagation Fails : ORA-00600[kope2upic2954] or Ora-00600[Kghsstream_copyn] (10.2, 11.1)Document 731292.1 ORA-25215 Reported on Local Propagation When Using Transformation with ANYDATA queue tables (10.2, 11.1, 11.2)Document 365093.1 ORA-07445 [kwqppay2aqe()+7360] Reported on Propagation of a Transformed Message (10.1, 10.2)Document 219416.1 Advanced Queuing Propagation Fails with ORA-22922 (9.0)Document 1203544.1 AQ Propagation Aborted with ORA-600 [ociksin: invalid status] on SYS.DBMS_AQADM_SYS.AQ$_PROPAGATION_PROCEDURE After Upgrade (11.1, 11.2)Document 1087324.1 ORA-01405 ORA-01422 reported by Advanced Queuing Propagation schedules after RAC reconfiguration (10.2)Document 1079577.1 Advanced Queuing Propagation Fails With "ORA-22370 incorrect usage of method" (9.2, 10.2, 11.1, 11.2)Document 332792.1 ORA-04061 error relating to SYS.DBMS_PRVTAQIP reported when setting up Statspack (8.1, 9.0, 9.2, 10.1)Document 353325.1 ORA-24056: Internal inconsistency for QUEUE <queue_name> and destination <dblink> (8.1, 9.0, 9.2, 10.1, 10.2, 11.1, 11.2)Document 787367.1 ORA-22275 reported on Propagating Messages with LOB component when propagating between 10.1 and 10.2 (10.1, 10.2)Document 566622.1 ORA-22275 when propagating >4K AQ$_JMS_TEXT_MESSAGEs from 9.2.0.8 to 10.2.0.1 (9.2, 10.1)Document 731539.1 ORA-29268: HTTP client error 401 Unauthorized Error when the AQ Servlet attempts to Propagate a message via HTTP (9.0, 9.2, 10.1, 10.2, 11.1)Document 253131.1 Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space Management (ORA-1555) (9.2)Document 118884.1 How to unschedule a propagation schedule stuck in pending stateDocument 222992.1 DBMS_AQADM.DISABLE_PROPAGATION_SCHEDULE Returns ORA-24082Document 282987.1 Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueDocument 1204080.1 AQ Propagation Failing With ORA-25329 After Upgraded From 8i or 9i to 10g or 11g.Document 1233675.1 AQ Propagation stops after upgrade to 11.2.0.1 ORA-30757 4.3.1. Errors Related to Incorrect Network Configuration The most common propagation errors result from an incorrect network configuration. The list below contains common errors caused by tnsnames.ora file or database links being configured incorrectly: - ORA-12154: TNS:could not resolve service name- ORA-12505: TNS:listener does not currently know of SID given in connect descriptor- ORA-12514: TNS:listener could not resolve SERVICE_NAME - ORA-12541: TNS-12541 TNS:no listener 4.4. Check the Database Links Exist and are Functioning Correctly For schedules to remote databases confirm the database link exists via. SQL> col DBLINK for a45SQL> select QNAME, NVL(REGEXP_SUBSTR(DESTINATION, '[^@]+', 1, 2), DESTINATION) dblink2 from DBA_QUEUE_SCHEDULES3 where MESSAGE_DELIVERY_MODE = 'PERSISTENT';QNAME DBLINK------------------------------ ---------------------------------------------MY_QUEUE ORCL102B.WORLD Connect as the owner of the link and select across it to verify it works and connects to the database we expect. i.e. select * from ALL_QUEUES@ ORCL102B.WORLD; You need to ensure that the userid that scheduled the propagation (using DBMS_AQADM.SCHEDULE_PROPAGATION or DBMS_PROPAGATION_ADM.CREATE_PROPAGATION if using Streams) has access to the database link for the destination. 4.5. Has Propagation Been Correctly Scheduled? Check that the propagation schedule has been created and that a job queue process has been assigned. Look for the entry in DBA_QUEUE_SCHEDULES and SYS.AQ$_SCHEDULES for your schedule. For 10g and below, check that it has a JOBNO entry in SYS.AQ$_SCHEDULES, and that there is an entry in DBA_JOBS with that JOBNO. For 11g and above, check that the schedule has a JOB_NAME entry in SYS.AQ$_SCHEDULES, and that there is an entry in DBA_SCHEDULER_JOBS with that JOB_NAME. Check the destination is as intended and spelled correctly. SQL> select SCHEMA, QNAME, DESTINATION, SCHEDULE_DISABLED, PROCESS_NAME from DBA_QUEUE_SCHEDULES;SCHEMA QNAME DESTINATION S PROCESS------- ---------- ------------------ - -----------AQADM MULTIPLEQ AQ$_LOCAL N J000 AQ$_LOCAL in the destination column shows that the queue to which we are propagating to is in the same database as the source queue. If the propagation was to a remote (different) database, a database link will be in the DESTINATION column. The entry in the SCHEDULE_DISABLED column, N, means that the schedule is NOT disabled. If Y (yes) appears in this column, propagation is disabled and the schedule will not be executed. If not using Oracle Streams, propagation should resume once you have enabled the schedule by invoking DBMS_AQADM.ENABLE_PROPAGATION_SCHEDULE (for 10.2 Oracle Streams and above, the DBMS_PROPAGATION_ADM.START_PROPAGATION procedure should be used). The PROCESS_NAME is the name of the job queue process currently allocated to execute the schedule. This process is allocated dynamically at execution time. If the PROCESS_NAME column is null (empty) the schedule is not currently executing. You may need to execute this statement a number of times to verify if a process is being allocated. If a process is at some time allocated to the schedule, it is attempting to execute. SQL> select SCHEMA, QNAME, LAST_RUN_DATE, NEXT_RUN_DATE from DBA_QUEUE_SCHEDULES;SCHEMA QNAME LAST_RUN_DATE NEXT_RUN_DATE------ ----- ----------------------- ----------------------- AQADM MULTIPLEQ 13-FEB-2002 13:18:57 13-FEB-2002 13:20:30 In 11g, these dates are expressed in TIMESTAMP WITH TIME ZONE datatypes. If the NEXT_RUN_DATE and NEXT_RUN_TIME columns are null when this statement is executed, the scheduled propagation is currently in progress. If they never change it would suggest that the schedule itself is never executing. If the next scheduled execution is too far away, change the NEXT_TIME parameter of the schedule so that schedules are executed more frequently (assuming that the window is not set to be infinite). Parameters of a schedule can be changed using the DBMS_AQADM.ALTER_PROPAGATION_SCHEDULE call. In 10g and below, scheduling propagation posts a job in the DBA_JOBS view. The columns are more or less the same as DBA_QUEUE_SCHEDULES so you just need to recognize the job and verify that it exists. SQL> select JOB, WHAT from DBA_JOBS where WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%';JOB WHAT---- ----------------- 720 next_date := sys.dbms_aqadm.aq$_propaq(job); For 11g, scheduling propagation posts a job in DBA_SCHEDULER_JOBS instead: SQL> select JOB_NAME from DBA_SCHEDULER_JOBS where JOB_NAME like 'AQ_JOB$_%';JOB_NAME------------------------------AQ_JOB$_41 If no job exists, check DBA_QUEUE_SCHEDULES to make sure that the schedule has not been disabled. For 10g and below, the job number is dynamic for AQ propagation schedules. The procedure that is executed to expedite a propagation schedule runs, removes itself from DBA_JOBS, and then reposts a new job for the next scheduled propagation. The job number should therefore always increment unless the schedule has been set up to run indefinitely. 4.6. Is the Schedule Executing but Failing to Complete? Run the following query: SQL> select FAILURES, LAST_ERROR_MSG from DBA_QUEUE_SCHEDULES;FAILURES LAST_ERROR_MSG------------ -----------------------1 ORA-25207: enqueue failed, queue AQADM.INQ is disabled from enqueueingORA-02063: preceding line from SHANE816 The failures column shows how many times we have attempted to execute the schedule and failed. Oracle will attempt to execute the schedule 16 times after which it will be removed from the DBA_JOBS or DBA_SCHEDULER_JOBS view and the schedule will become disabled. The column DBA_QUEUE_SCHEDULES.SCHEDULE_DISABLED will show 'Y'. For 11g and above, the DBA_SCHEDULER_JOBS.STATE column will show 'BROKEN' for the job corresponding to DBA_QUEUE_SCHEDULES.JOB_NAME. Prior to 10g the back off algorithm for failures was exponential, whereas from 10g onwards it is linear. The propagation will become disabled on the 17th attempt. Only the last execution failure will be reflected in the LAST_ERROR_MSG column. That is, if the schedule fails 5 times for 5 different reasons, only the last set of errors will be recorded in DBA_QUEUE_SCHEDULES. Any errors need to be resolved to allow propagation to continue. If propagation has also become disabled due to 17 failures, first resolve the reason for the error and then re-enable the schedule using the DBMS_AQADM.ENABLE_PROPAGATION_SCHEDULE procedure, or DBMS_PROPAGATION_ADM.START_PROPAGATION if using 10.2 or above Oracle Streams. As soon as the schedule executes successfully the error message entries will be deleted. Oracle does not keep a history of past failures. However, when using Oracle Streams, the errors will be retained in the DBA_PROPAGATION view even after the schedule resumes successfully. See the following note for instructions on how to clear out the errors from the DBA_PROPAGATION view:Document 808136.1 How to clear the old errors from DBA_PROPAGATION view?If a schedule is active and no errors are being reported then the source queue may not have any messages to be propagated. 4.7. Do the Propagation Notification Queue Table and Queue Exist? Check to see that the propagation notification queue table and queue exist and are enabled for enqueue and dequeue. Propagation makes use of the propagation notification queue for handling propagation run-time events, and the messages in this queue are stored in a SYS-owned queue table. This queue should never be stopped or dropped and the corresponding queue table never be dropped. 10g and belowThe propagation notification queue table is of the format SYS.AQ$_PROP_TABLE_n, where 'n' is the RAC instance number, i.e. '1' for a non-RAC environment. This queue and queue table are created implicitly when propagation is first scheduled. If propagation has been scheduled and these objects do not exist, try unscheduling and rescheduling propagation. If they still do not exist contact Oracle Support. SQL> select QUEUE_TABLE from DBA_QUEUE_TABLES2 where QUEUE_TABLE like '%PROP_TABLE%' and OWNER = 'SYS';QUEUE_TABLE------------------------------AQ$_PROP_TABLE_1SQL> select NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED2 from DBA_QUEUES where owner='SYS'3 and QUEUE_TABLE like '%PROP_TABLE%';NAME ENQUEUE DEQUEUE------------------------------ ------- -------AQ$_PROP_NOTIFY_1 YES YESAQ$_AQ$_PROP_TABLE_1_E NO NO If the AQ$_PROP_NOTIFY_1 queue is not enabled for enqueue or dequeue, it should be so enabled using DBMS_AQADM.START_QUEUE. However, the exception queue AQ$_AQ$_PROP_TABLE_1_E should not be enabled for enqueue or dequeue.11g and aboveThe propagation notification queue table is of the format SYS.AQ_PROP_TABLE, and is created when the database is created. If they do not exist, contact Oracle Support. SQL> select QUEUE_TABLE from DBA_QUEUE_TABLES2 where QUEUE_TABLE like '%PROP_TABLE%' and OWNER = 'SYS';QUEUE_TABLE------------------------------AQ_PROP_TABLESQL> select NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED2 from DBA_QUEUES where owner='SYS'3 and QUEUE_TABLE like '%PROP_TABLE%';NAME ENQUEUE DEQUEUE------------------------------ ------- -------AQ_PROP_NOTIFY YES YESAQ$_AQ_PROP_TABLE_E NO NO If the AQ_PROP_NOTIFY queue is not enabled for enqueue or dequeue, it should be so enabled using DBMS_AQADM.START_QUEUE. However, the exception queue AQ$_AQ$_PROP_TABLE_E should not be enabled for enqueue or dequeue. 4.8. Does the Remote Queue Exist and is it Enabled for Enqueueing? Check that the remote queue the propagation is transferring messages to exists and is enabled for enqueue: SQL> select DESTINATION from USER_QUEUE_SCHEDULES where QNAME = 'OUTQ';DESTINATION-----------------------------------------------------------------------------"AQADM"."INQ"@M2V102.ESSQL> select OWNER, NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED from [email protected];OWNER NAME ENQUEUE DEQUEUE-------- ------ ----------- -----------AQADM INQ YES YES 4.9. Do the Target and Source Database Charactersets Differ? If a message fails to propagate, check the database charactersets of the source and target databases. Investigate whether the same message can propagate between the databases with the same characterset or it is only a particular combination of charactersets which causes a problem. 4.10. Check the Queue Table Type Agreement Propagation is not possible between queue tables which have types that differ in some respect. One way to determine if this is the case is to run the DBMS_AQADM.VERIFY_QUEUE_TYPES procedure for the two queues that the propagation operates on. If the types do not agree, DBMS_AQADM.VERIFY_QUEUE_TYPES will return '0'.For AQ propagation between databases which have different NLS_LENGTH_SEMANTICS settings, propagation will not work, unless the queues are Oracle Streams ANYDATA queues.See the following notes for issues caused by lack of type agreement:Document 1079577.1 Advanced Queuing Propagation Fails With "ORA-22370: incorrect usage of method"Document 282987.1 Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueDocument 353754.1 Streams Messaging Propagation Fails between Single and Multi-byte Charactersets when using Chararacter Length Semantics in the ADT 4.11. Enable Propagation Tracing 4.11.1. System Level This is set it in the init.ora/spfile as follows: event="24040 trace name context forever, level 10" and restart the instanceThis event cannot be set dynamically with an alter system command until version 10.2: SQL> alter system set events '24040 trace name context forever, level 10'; To unset the event: SQL> alter system set events '24040 trace name context off'; Debugging information will be logged to job queue trace file(s) (jnnn) as propagation takes place. You can check the trace file for errors, and for statements indicating that messages have been sent. For the most part the trace information is understandable. This trace should also be uploaded to Oracle Support if a service request is created. 4.11.2. Attaching to a Specific Process We can also attach to an existing job queue processes that is running a propagation schedule and trace it individually using the oradebug utility, as follows:10.2 and below connect / as sysdbaselect p.SPID, p.PROGRAM from v$PROCESS p, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j where s.SID=jr.SID and s.PADDR=p.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%';-- For the process id (SPID) attach to it via oradebug and generate the following traceoradebug setospid <SPID>oradebug unlimitoradebug Event 10046 trace name context forever, level 12oradebug Event 24040 trace name context forever, level 10-- Trace the process for 5 minutesoradebug Event 10046 trace name context offoradebug Event 24040 trace name context off-- The following command returns the pathname/filename to the file being written tooradebug tracefile_name 11g connect / as sysdbacol PROGRAM for a30select p.SPID, p.PROGRAM, j.JOB_NAMEfrom v$PROCESS p, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j where s.SID=jr.SESSION_ID and s.PADDR=p.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%';-- For the process id (SPID) attach to it via oradebug and generate the following traceoradebug setospid <SPID>oradebug unlimitoradebug Event 10046 trace name context forever, level 12oradebug Event 24040 trace name context forever, level 10-- Trace the process for 5 minutesoradebug Event 10046 trace name context offoradebug Event 24040 trace name context off-- The following command returns the pathname/filename to the file being written tooradebug tracefile_name 4.11.3. Further Tracing The previous tracing steps only trace the job queue process executing the propagation on the source. At times it is useful to trace the propagation receiver process (the session which is enqueueing the messages into the target queue) on the target database which is associated with the job queue process on the source database.These following queries provide ways of identifying the processes involved in propagation so that you can attach to them via oradebug to generate trace information.In order to identify the propagation receiver process you need to execute the query as a user with privileges to access the v$ views in both the local and remote databases so the database link must connect as a user with those privileges in the remote database. The <DBLINK> in the queries should be replaced by the appropriate database link.The queries have two forms due to the differences between operating systems. The value returned by 'Rem Process' is the operating system identifier of the propagation receiver on the remote database. Once identified, this process can be attached to and traced on the remote database using the commands given in Section 4.11.2.10.2 and below - Windows select pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from v$PROCESS pl, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SID and s.PADDR=pl.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%' and pl.SPID=substr(sr.PROCESS, instr(sr.PROCESS,':')+1); 10.2 and below - Unix select pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SID and s.PADDR=pl.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%' and pl.SPID=sr.PROCESS; 11g - Windows select pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SESSION_ID and s.PADDR=pl.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%%' and pl.SPID=substr(sr.PROCESS, instr(sr.PROCESS,':')+1); 11g - Unix select pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SESSION_ID and s.PADDR=pl.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%%' and pl.SPID=sr.PROCESS; 5. Additional Troubleshooting Steps for AQ Propagation of User-Enqueued and Dequeued Messages 5.1. Check the Privileges of All Users Involved Ensure that the owner of the database link has the necessary privileges on the aq packages. SQL> select TABLE_NAME, PRIVILEGE from USER_TAB_PRIVS;TABLE_NAME PRIVILEGE------------------------------ ----------------------------------------DBMS_LOCK EXECUTEDBMS_AQ EXECUTEDBMS_AQADM EXECUTEDBMS_AQ_BQVIEW EXECUTEQT52814_BUFFER SELECT Note that when queue table is created, a view called QT<nnn>_BUFFER is created in the SYS schema, and the queue table owner is given SELECT privileges on it. The <nnn> corresponds to the object_id of the associated queue table. SQL> select * from USER_ROLE_PRIVS;USERNAME GRANTED_ROLE ADM DEF OS_------------------------------ ------------------------------ ---- ---- ---AQ_USER1 AQ_ADMINISTRATOR_ROLE NO YES NOAQ_USER1 CONNECT NO YES NOAQ_USER1 RESOURCE NO YES NO It is good practice to configure central AQ administrative user. All admin and processing jobs are created, executed and administered as this user. This configuration is not mandatory however, and the database link can be owned by any existing queue user. If this latter configuration is used, ensure that the connecting user has the necessary privileges on the AQ packages and objects involved. Privileges for an AQ Administrative user Execute on DBMS_AQADM Execute on DBMS_AQ Granted the AQ_ADMINISTRATOR_ROLE Privileges for an AQ user Execute on DBMS_AQ Execute on the message payload Enqueue privileges on the remote queue Dequeue privileges on the originating queue Privileges need to be confirmed on both sites when propagation is scheduled to remote destinations. Verify that the user ID used to login to the destination through the database link has been granted privileges to use AQ. 5.2. Verify Queue Payload Types AQ will not propagate messages from one queue to another if the payload types of the two queues are not verified to be equivalent. An AQ administrator can verify if the source and destination's payload types match by executing the DBMS_AQADM.VERIFY_QUEUE_TYPES procedure. The results of the type checking will be stored in the SYS.AQ$_MESSAGE_TYPES table. This table can be accessed using the object identifier OID of the source queue and the address database link of the destination queue, i.e. [schema.]queue_name[@destination]. Prior to Oracle 9i the payload (message type) had to be the same for all the queue tables involved in propagation. From Oracle9i onwards a transformation can be used so that payloads can be converted from one type to another. The following procedural call made on the source database can verify whether we can propagate between the source and the destination queue tables. connect aq_user1/[email protected] serverout onDECLARErc_value number;BEGINDBMS_AQADM.VERIFY_QUEUE_TYPES(src_queue_name => 'AQ_USER1.Q_1', dest_queue_name => 'AQ_USER2.Q_2',destination => 'dbl_aq_user2.es',rc => rc_value);dbms_output.put_line('rc_value code is '||rc_value);END;/ If propagation is possible then the return code value will be 1. If it is 0 then propagation is not possible and further investigation of the types and transformations used by and in conjunction with the queue tables is required. With regard to comparison of the types the following sql can be used to extract the DDL for a specific type with' %' changed appropriately on the source and target. This can then be compared for the source and target. SET LONG 20000 set pagesize 50 EXECUTE DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM, 'STORAGE',false); SELECT DBMS_METADATA.GET_DDL('TYPE',t.type_name) from user_types t WHERE t.type_name like '%'; EXECUTE DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM, 'DEFAULT'); 5.3. Check Message State and Destination The first step in this process is to identify the queue table associated with the problem source queue. Although you schedule propagation for a specific queue, most of the meta-data associated with that queue is stored in the underlying queue table. The following statement finds the queue table for a given queue (note that this is a multiple-consumer queue table). SQL> select QUEUE_TABLE from DBA_QUEUES where NAME = 'MULTIPLEQ';QUEUE_TABLE --------------------MULTIPLEQTABLE For a small amount of messages in a multiple-consumer queue table, the following query can be run: SQL> select MSG_STATE, CONSUMER_NAME, ADDRESS from AQ$MULTIPLEQTABLE where QUEUE = 'MULTIPLEQ';MSG_STATE CONSUMER_NAME ADDRESS-------------- ----------------------- -------------READY AQUSER2 [email protected] AQUSER1READY AQUSER3 AQADM.INQ In this example we see 2 messages ready to be propagated to remote queues and 1 that is not. If the address column is blank, the message is not scheduled for propagation and can only be dequeued from the queue upon which it was enqueued. The MSG_STATE column values are discussed in Document 102330.1 Advanced Queueing MSG_STATE Values and their Interpretation. If the address column has a value, the message has been enqueued for propagation to another queue. The first row in the example includes a database link (@M2V102.ES). This demonstrates that the message should be propagated to a queue at a remote database. The third row does not include a database link so will be propagated to a queue that resides on the same database as the source queue. The consumer name is the intended recipient at the target queue. Note that we are not querying the base queue table directly; rather, we are querying a view that is available on top of every queue table, AQ$<queue_table_name>.A more realistic query in an environment where the queue table contains thousands of messages is8.0.3-compatible multiple-consumer queue table and all compatibility single-consumer queue tables select count(*), MSG_STATE, QUEUE from AQ$<queue_table_name> group by MSG_STATE, QUEUE; 8.1.3 and 10.0-compatible queue tables select count(*), MSG_STATE, QUEUE, CONSUMER_NAME from AQ$<queue_table_name>group by MSG_STATE, QUEUE, CONSUMER_NAME; For multiple-consumer queue tables, if you did not see the expected CONSUMER_NAME , check the syntax of the enqueue code and verify the recipients are declared correctly. If a recipients list is not used on enqueue, check the subscriber list in the AQ$_<queue_table_name>_S view (note that a single-consumer queue table does not have a subscriber view. This view records all members of the default subscription list which were added using the DBMS_AQADM.ADD_SUBSCRIBER procedure and also those enqueued using a recipient list. SQL> select QUEUE, NAME, ADDRESS from AQ$MULTIPLEQTABLE_S;QUEUE NAME ADDRESS---------- ----------- -------------MULTIPLEQ AQUSER2 [email protected] AQUSER1 In this example we have 2 subscribers registered with the queue. We have a local subscriber AQUSER1, and a remote subscriber AQUSER2, on the queue INQ, owned by AQADM, at M2V102.ES. Unless overridden with a recipient list during enqueue every message enqueued to this queue will be propagated to INQ at M2V102.ES.For 8.1 style and above multiple consumer queue tables, you can also check the following information at the target: select CONSUMER_NAME, DEQ_TXN_ID, DEQ_TIME, DEQ_USER_ID, PROPAGATED_MSGID from AQ$<queue_table_name> where QUEUE = '<QUEUE_NAME>'; For 8.0 style queues, if the queue table supports multiple consumers you can obtain the same information from the history column of the queue table: select h.CONSUMER, h.TRANSACTION_ID, h.DEQ_TIME, h.DEQ_USER, h.PROPAGATED_MSGIDfrom AQ$<queue_table_name> t, table(t.history) h where t.Q_NAME = '<QUEUE_NAME>'; A non-NULL TRANSACTION_ID indicates that the message was successfully propagated. Further, the DEQ_TIME indicates the time of propagation, the DEQ_USER indicates the userid used for propagation, and the PROPAGATED_MSGID indicates the message ID of the message that was enqueued at the destination. 6. Additional Troubleshooting Steps for Propagation in an Oracle Streams Environment 6.1. Is the Propagation Enabled? For a propagation job to propagate messages, the propagation must be enabled. For Streams, a special view called DBA_PROPAGATION exists to convey information about Streams propagations. If messages are not being propagated by a propagation as expected, then the propagation might not be enabled. To query for this: SELECT p.PROPAGATION_NAME, DECODE(s.SCHEDULE_DISABLED, 'Y', 'Disabled','N', 'Enabled') SCHEDULE_DISABLED, s.PROCESS_NAME, s.FAILURES, s.LAST_ERROR_MSGFROM DBA_QUEUE_SCHEDULES s, DBA_PROPAGATION pWHERE p.DESTINATION_DBLINK = NVL(REGEXP_SUBSTR(s.DESTINATION, '[^@]+', 1, 2), s.DESTINATION) AND s.SCHEMA = p.SOURCE_QUEUE_OWNER AND s.QNAME = p.SOURCE_QUEUE_NAME AND MESSAGE_DELIVERY_MODE = 'PERSISTENT' order by PROPAGATION_NAME; At times, the propagation job may become "broken" or fail to start after an error has been encountered or after a database restart. If an error is indicated by the above query, an attempt to disable the propagation and then re-enable it can be made. In the examples below, for the propagation named STRMADMIN_PROPAGATE where the queue name is STREAMS_QUEUE owned by STRMADMIN and the destination database link is ORCL2.WORLD, the commands would be:10.2 and above exec dbms_propagation_adm.stop_propagation('STRMADMIN_PROPAGATE'); exec dbms_propagation_adm.start_propagation('STRMADMIN_PROPAGATE'); If the above does not fix the problem, stop the propagation specifying the force parameter (2nd parameter on stop_propagation) as TRUE: exec dbms_propagation_adm.stop_propagation('STRMADMIN_PROPAGATE',true); exec dbms_propagation_adm.start_propagation('STRMADMIN_PROPAGATE'); The statistics for the propagation as well as any old error messages are cleared when the force parameter is set to TRUE. Therefore if the propagation schedule is stopped with FORCE set to TRUE, and upon restart there is still an error message in DBA_PROPAGATION, then the error message is current.9.2 or 10.1 exec dbms_aqadm.disable_propagation_schedule('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); exec dbms.aqadm.enable_propagation_schedule('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); If the above does not fix the problem, perform an unschedule of propagation and then schedule_propagation: exec dbms_aqadm.unschedule_propagation('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); exec dbms_aqadm.schedule_propagation('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); Typically if the error from the first query in Section 6.1 recurs after restarting the propagation as shown above, further troubleshooting of the error is needed. 6.2. Check Propagation Rule Sets and Transformations Inspect the configuration of the rules in the rule set that is associated with the propagation process to make sure that they evaluate to TRUE as expected. If not, then the object or schema will not be propagated. Remember that when a negative rule evaluates to TRUE, the specified object or schema will not be propagated. Finally inspect any rule-based transformations that are implemented with propagation to make sure they are changing the data in the intended way.The following query shows what rule sets are assigned to a propagation: select PROPAGATION_NAME, RULE_SET_OWNER||'.'||RULE_SET_NAME "Positive Rule Set",NEGATIVE_RULE_SET_OWNER||'.'||NEGATIVE_RULE_SET_NAME "Negative Rule Set"from DBA_PROPAGATION; The next two queries list the propagation rules and their conditions. The first is for the positive rule set, the second is for the negative rule set: set long 4000select rsr.RULE_SET_OWNER||'.'||rsr.RULE_SET_NAME RULE_SET ,rsr.RULE_OWNER||'.'||rsr.RULE_NAME RULE_NAME,r.RULE_CONDITION CONDITION fromDBA_RULE_SET_RULES rsr, DBA_RULES rwhere rsr.RULE_NAME = r.RULE_NAME and rsr.RULE_OWNER = r.RULE_OWNER and RULE_SET_NAME in(select RULE_SET_NAME from DBA_PROPAGATION) order by rsr.RULE_SET_OWNER, rsr.RULE_SET_NAME; set long 4000select c.PROPAGATION_NAME, rsr.RULE_SET_OWNER||'.'||rsr.RULE_SET_NAME RULE_SET ,rsr.RULE_OWNER||'.'||rsr.RULE_NAME RULE_NAME,r.RULE_CONDITION CONDITION fromDBA_RULE_SET_RULES rsr, DBA_RULES r ,DBA_PROPAGATION cwhere rsr.RULE_NAME = r.RULE_NAME and rsr.RULE_OWNER = r.RULE_OWNER andrsr.RULE_SET_OWNER=c.NEGATIVE_RULE_SET_OWNER and rsr.RULE_SET_NAME=c.NEGATIVE_RULE_SET_NAMEand rsr.RULE_SET_NAME in(select NEGATIVE_RULE_SET_NAME from DBA_PROPAGATION) order by rsr.RULE_SET_OWNER, rsr.RULE_SET_NAME; 6.3. Determining the Total Number of Messages and Bytes Propagated As in Section 3.1, determining if messages are flowing can be instructive to see whether the propagation is entirely hung or just slow. If the propagation is not in flow control (see Section 6.5.2), but the statistics are incrementing slowly, there may be a performance issue. For Streams implementations two views are available that can assist with this that can show the number of messages sent by a propagation, as well as the number of acknowledgements being returned from the target site: the V$PROPAGATION_SENDER view at the Source site and the V$PROPAGATION_RECEIVER view at the destination site. It is helpful to query both to determine if messages are being delivered to the target. Look for the statistics to increase.Source: select QUEUE_SCHEMA, QUEUE_NAME, DBLINK,HIGH_WATER_MARK, ACKNOWLEDGEMENT, TOTAL_MSGS, TOTAL_BYTESfrom V$PROPAGATION_SENDER; Target: select SRC_QUEUE_SCHEMA, SRC_QUEUE_NAME, SRC_DBNAME, DST_QUEUE_SCHEMA, DST_QUEUE_NAME, HIGH_WATER_MARK, ACKNOWLEDGEMENT, TOTAL_MSGS from V$PROPAGATION_RECEIVER; 6.4. Check Buffered Subscribers The V$BUFFERED_SUBSCRIBERS view displays information about subscribers for all buffered queues in the instance. This view can be queried to make sure that the site that the propagation is propagating to is listed as a subscriber address for the site being propagated from: select QUEUE_SCHEMA, QUEUE_NAME, SUBSCRIBER_ADDRESS from V$BUFFERED_SUBSCRIBERS; The SUBSCRIBER_ADDRESS column will not be populated when the propagation is local (between queues on the same database). 6.5. Common Streams Propagation Errors 6.5.1. ORA-02082: A loopback database link must have a connection qualifier. This error can occur if you use the Streams Setup Wizard in Oracle Enterprise Manager without first configuring the GLOBAL_NAME for your database. 6.5.2. ORA-25307: Enqueue rate too high. Enable flow control DBA_QUEUE_SCHEDULES will display this informational message for propagation when the automatic flow control (10g feature of Streams) has been invoked.Similar to Streams capture processes, a Streams propagation process can also go into a state of 'flow control. This is an informative message that indicates flow control has been automatically enabled to reduce the rate at which messages are being enqueued into at target queue.This typically occurs when the target site is unable to keep up with the rate of messages flowing from the source site. Other than checking that the apply process is running normally on the target site, usually no action is required by the DBA. Propagation and the capture process will be resumed automatically when the target site is able to accept more messages.The following document contains more information:Document 302109.1 Streams Propagation Error: ORA-25307 Enqueue rate too high. Enable flow controlSee the following document for one potential cause of this situation:Document 1097115.1 Oracle Streams Apply Reader is in 'Paused' State 6.5.3. ORA-25315 unsupported configuration for propagation of buffered messages This error typically occurs when the target database is RAC and usually indicates that an attempt was made to propagate buffered messages with the database link pointing to an instance in the destination database which is not the owner instance of the destination queue. To resolve the problem, use queue-to-queue propagation for buffered messages. 6.5.4. ORA-600 [KWQBMCRCPTS101] after dropping / recreating propagation For cause/fixes refer to:Document 421237.1 ORA-600 [KWQBMCRCPTS101] reported by a Qmon slave process after dropping a Streams Propagation 6.5.5. Stopping or Dropping a Streams Propagation Hangs See the following note:Document 1159787.1 Troubleshooting Streams Propagation When It is Not Functioning and Attempts to Stop It Hang 6.6. Streams Propagation-Related Notes for Common Issues Document 437838.1 Streams Specific PatchesDocument 749181.1 How to Recover Streams After Dropping PropagationDocument 368912.1 Queue to Queue Propagation Schedule encountered ORA-12514 in a RAC environmentDocument 564649.1 ORA-02068/ORA-03114/ORA-03113 Errors From Streams Propagation Process - Remote Database is Available and Unschedule/Reschedule Does Not ResolveDocument 553017.1 Stream Propagation Process Errors Ora-4052 Ora-6554 From 11g To 10201Document 944846.1 Streams Propagation Fails Ora-7445 [kohrsmc]Document 745601.1 ORA-23603 'STREAMS enqueue aborted due to low SGA' Error from Streams Propagation, and V$STREAMS_CAPTURE.STATE Hanging on 'Enqueuing Message'Document 333068.1 ORA-23603: Streams Enqueue Aborted Eue To Low SGADocument 363496.1 Ora-25315 Propagating on RAC StreamsDocument 368237.1 Unable to Unschedule Propagation. Streams Queue is InvalidDocument 436332.1 dbms_propagation_adm.stop_propagation hangsDocument 727389.1 Propagation Fails With ORA-12528Document 730911.1 ORA-4063 Is Reported After Dropping Negative Prop.RulesetDocument 460471.1 Propagation Blocked by Qmon Process - Streams_queue_table / 'library cache lock' waitsDocument 1165583.1 ORA-600 [kwqpuspse0-ack] In Streams EnvironmentDocument 1059029.1 Combined Capture and Apply (CCA) : Capture aborts : ORA-1422 after schedule_propagationDocument 556309.1 Changing Propagation/ queue_to_queue : false -> true does does not work; no LCRs propagatedDocument 839568.1 Propagation failing with error: ORA-01536: space quota exceeded for tablespace ''Document 311021.1 Streams Propagation Process : Ora 12154 After Reboot with Transparent Application Failover TAF configuredDocument 359971.1 STREAMS propagation to Primary of physical Standby configuation errors with Ora-01033, Ora-02068Document 1101616.1 DBMS_PROPAGATION_ADM.DROP_PROPAGATION FAILS WITH ORA-1747 7. Performance Issues A propagation may seem to be slow if the queries from Sections 3.1 and 6.3 show that the message statistics are not changing quickly. In Oracle Streams, this more usually is due to a slow apply process at the target rather than a slow propagation. Propagation could be inferred to be slow if the message statistics are changing, and the state of a capture process according to V$STREAMS_CAPTURE.STATE is PAUSED FOR FLOW CONTROL, but an ORA-25307 'Enqueue rate too high. Enable flow control' warning is NOT observed in DBA_QUEUE_SCHEDULES per Section 6.5.2. If this is the case, see the following notes / white papers for suggestions to increase performance:Document 335516.1 Master Note for Streams Performance RecommendationsDocument 730036.1 Overview for Troubleshooting Streams Performance IssuesDocument 780733.1 Streams Propagation Tuning with Network ParametersWhite Paper: http://www.oracle.com/technetwork/database/features/availability/maa-wp-10gr2-streams-performance-130059.pdfWhite Paper: Oracle Streams Configuration Best Practices: Oracle Database 10g Release 10.2, http://www.oracle.com/technetwork/database/features/availability/maa-10gr2-streams-configuration-132039.pdf, See APPENDIX A: USING STREAMS CONFIGURATIONS OVER A NETWORKFor basic AQ propagation, the network tuning in the aforementioned Appendix A of the white paper 'Oracle Streams Configuration Best Practices: Oracle Database 10g Release 10.2' is applicable. References NOTE:102330.1 - Advanced Queueing MSG_STATE Values and their InterpretationNOTE:102771.1 - Advanced Queueing Propagation using PL/SQLNOTE:1059029.1 - Combined Capture and Apply (CCA) : Capture aborts : ORA-1422 after schedule_propagationNOTE:1079577.1 - Advanced Queuing Propagation Fails With "ORA-22370: incorrect usage of method"NOTE:1083608.1 - 11g Streams and Oracle SchedulerNOTE:1087324.1 - ORA-01405 ORA-01422 reported by Adavanced Queueing Propagation schedules after RAC reconfigurationNOTE:1097115.1 - Oracle Streams Apply Reader is in 'Paused' StateNOTE:1101616.1 - DBMS_PROPAGATION_ADM.DROP_PROPAGATION FAILS WITH ORA-1747NOTE:1159787.1 - Troubleshooting Streams Propagation When It is Not Functioning and Attempts to Stop It HangNOTE:1165583.1 - ORA-600 [kwqpuspse0-ack] In Streams EnvironmentNOTE:118884.1 - How to unschedule a propagation schedule stuck in pending stateNOTE:1203544.1 - AQ PROPAGATION ABORTED WITH ORA-600[OCIKSIN: INVALID STATUS] ON SYS.DBMS_AQADM_SYS.AQ$_PROPAGATION_PROCEDURE AFTER UPGRADENOTE:1204080.1 - AQ Propagation Failing With ORA-25329 After Upgraded From 8i or 9i to 10g or 11g.NOTE:219416.1 - Advanced Queuing Propagation fails with ORA-22922NOTE:222992.1 - DBMS_AQADM.DISABLE_PROPAGATION_SCHEDULE Returns ORA-24082NOTE:253131.1 - Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space Management (ORA-1555)NOTE:282987.1 - Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueNOTE:298015.1 - Kwqjswproc:Excep After Loop: Assigning To SelfNOTE:302109.1 - Streams Propagation Error: ORA-25307 Enqueue rate too high. Enable flow controlNOTE:311021.1 - Streams Propagation Process : Ora 12154 After Reboot with Transparent Application Failover TAF configuredNOTE:332792.1 - ORA-04061 error relating to SYS.DBMS_PRVTAQIP reported when setting up StatspackNOTE:333068.1 - ORA-23603: Streams Enqueue Aborted Eue To Low SGANOTE:335516.1 - Master Note for Streams Performance RecommendationsNOTE:353325.1 - ORA-24056: Internal inconsistency for QUEUE and destination NOTE:353754.1 - Streams Messaging Propagation Fails between Single and Multi-byte Charactersets when using Chararacter Length Semantics in the ADT.NOTE:359971.1 - STREAMS propagation to Primary of physical Standby configuation errors with Ora-01033, Ora-02068NOTE:363496.1 - Ora-25315 Propagating on RAC StreamsNOTE:365093.1 - ORA-07445 [kwqppay2aqe()+7360] reported on Propagation of a Transformed MessageNOTE:368237.1 - Unable to Unschedule Propagation. Streams Queue is InvalidNOTE:368912.1 - Queue to Queue Propagation Schedule encountered ORA-12514 in a RAC environmentNOTE:421237.1 - ORA-600 [KWQBMCRCPTS101] reported by a Qmon slave process after dropping a Streams PropagationNOTE:436332.1 - dbms_propagation_adm.stop_propagation hangsNOTE:437838.1 - Streams Specific PatchesNOTE:460471.1 - Propagation Blocked by Qmon Process - Streams_queue_table / 'library cache lock' waitsNOTE:463820.1 - Streams Combined Capture and Apply in 11gNOTE:553017.1 - Stream Propagation Process Errors Ora-4052 Ora-6554 From 11g To 10201NOTE:556309.1 - Changing Propagation/ queue_to_queue : false -> true does does not work; no LCRs propagatedNOTE:564649.1 - ORA-02068/ORA-03114/ORA-03113 Errors From Streams Propagation Process - Remote Database is Available and Unschedule/Reschedule Does Not ResolveNOTE:566622.1 - ORA-22275 when propagating >4K AQ$_JMS_TEXT_MESSAGEs from 9.2.0.8 to 10.2.0.1NOTE:727389.1 - Propagation Fails With ORA-12528NOTE:730036.1 - Overview for Troubleshooting Streams Performance IssuesNOTE:730911.1 - ORA-4063 Is Reported After Dropping Negative Prop.RulesetNOTE:731292.1 - ORA-25215 Reported On Local Propagation When Using Transformation with ANYDATA queue tablesNOTE:731539.1 - ORA-29268: HTTP client error 401 Unauthorized Error when the AQ Servlet attempts to Propagate a message via HTTPNOTE:745601.1 - ORA-23603 'STREAMS enqueue aborted due to low SGA' Error from Streams Propagation, and V$STREAMS_CAPTURE.STATE Hanging on 'Enqueuing Message'NOTE:749181.1 - How to Recover Streams After Dropping PropagationNOTE:780733.1 - Streams Propagation Tuning with Network ParametersNOTE:787367.1 - ORA-22275 reported on Propagating Messages with LOB component when propagating between 10.1 and 10.2NOTE:808136.1 - How to clear the old errors from DBA_PROPAGATION view ?NOTE:827184.1 - AQ Propagation with CLOB data types Fails with ORA-22990NOTE:827473.1 - How to alter propagation from queue_to_queue to queue_to_dblinkNOTE:839568.1 - Propagation failing with error: ORA-01536: space quota exceeded for tablespace ''NOTE:846297.1 - AQ Propagation Fails : ORA-00600[kope2upic2954] or Ora-00600[Kghsstream_copyn]NOTE:944846.1 - Streams Propagation Fails Ora-7445 [kohrsmc]

Read the article

CodePlex Daily Summary for Saturday, August 09, 2014

CodePlex Daily Summary for Saturday, August 09, 2014Popular ReleasesSEToolbox: SEToolbox 01.042.019 Release 1: Added RadioAntenna broadcast name to ship name detail. Added two additional columns for Asteroid material generation for Asteroid Fields. Added Mass and Block number columns to main display. Added Ellipsis to some columns on main display to reduce name confusion. Added correct SE version number in file when saving. Re-added in reattaching Motor when drag/dropping or importing ships (KeenSH have added RotorEntityId back in after removing it months ago). Added option to export and r...QuickDAL: Initial Public Build: We recommend simply forking the source into your project. But you can use this DLL and PDB (debug symbols) if you'd rather not add clutter to your solution.N-Tier Entity Framework: N-Tier Entity Framework - 1.5 Beta 2: This is a pre-release version. Contents: N-Tier Entity Framework (VS2010).1.5.0-beta002.vsix N-Tier Entity Framework (VS2012).1.5.0-beta002.vsix N-Tier Entity Framework (VS2013).1.5.0-beta002.vsixThe Freemwork - An open game framework in C# .NET: v0.2 - Vortigaunt: Added space partitionning support Added depth support on CompositeSpriteHolder Added multiple components Added camera strategies Added extensions and math methods Added another constructor to Identity2D Updated Tools.TuplesFromT64 to support multiple layers of tiles Updated Identity2D.GlobalTransform property : now returns the screen position, regardless of Identity2D.DependsOnCamera Fixed Rectangle.Transform method Fixed sprite depth support Fixed wrong mathematic functions Multiple bugfixesLanMngmtXL: LanMngmtXL-1.0.61: This release includes binary files for 64 bit OS, source code and manual. Changes- host arp tables loaded - if more than 1 million rows then data saved to several excel tabs Quick How-To (using binary file)Follow these steps to use the software. Check included documentation for more details: Unpack ZIP file Download aditional tools for retrieving configurations like Plink (save to C:\Putty\plink.exe) or WVT (read documentation) Create a device list to get configurations (ex. devices.txt...Remote Linq: Demo - RemoteQueryableToEntityFramework: Demo application to show dynamic queries (joins, aggregations, groupings, projections, etc.) over WCF using EF on server sideTransMock: TransMock 0.9.1 (Beta): Minor fixes of the assembly name with the BizUnit steps Updated the name of the isntaller to indicate that it is for BizTalk server 2010. Tested and working with BizTalk 2013 as well, though assemblies are compiled for .NET 4.0.SharePoint 2013 Lync Presence using jQuery: jQuery Lync Presence: I have fixed the same user multiple times on page issue in this release (Issue # 1506)Dynamics AX IEIDE Project Explorer: IEIDE.1.1.40803.1: Installing the project: 1. Install the ax model file; do this by running the following commands on the machine having the AX Management Tools: 1.a. cmd (depending if you have UAC enabled, you may want to Run as administrator); 1.b. net stop aos60$01 (wait until you have your AOS stopped); 1.c. cd "c:\Program Files\Microsoft Dynamics AX\60\ManagementUtili ties\" 1.d. axutil import /file:c:\IEIDE.1.1.40806.1.axmodel /verbose 1.e. net start aos60$01 1.f. start the client and select Skip 2. Per...Facebook Graph Toolkit: Facebook Graph Toolkit 5.1: Updated to 100 Graph Api endpoint mappings with 258 object propertiesJSLint.NET: JSLint.NET 1.6.4: Bugs: #38: MSBuild task support for linked settings file. #40: Explicitly typed imports / exports required in some Visual Studio environments.Instant Beautiful Browsing: IBB 14.3 Alpha: An alpha release of IBB. After 3 years of the last release this version is made from scratch, with tons of new features like: Make your own IBB aps. HTML 5. Better UI. Extreme Windows 8 resemblance. Photos. Store. Movement TONS of times smother compared to previous versions. Remember that this is AN ALPHA release, I hope I will have "IBB 14" finished by December. The documentation on how to create a new application for IBB will come next monthjQuery List DragSort: jQuery List DragSort 0.5.2: Fixed scrollContainer removing deprecated use of $.browser so should now work with latest version of jQuery. Added the ability to return false in dragEnd to revert sort order Project changes Added nuget package for dragsort https://www.nuget.org/packages/dragsort Converted repository from SVN to MercurialLexisnexis directory of corporate affiliates Text Analyzer: Lexisnexis Text Analyzer: This version has functions below.Standards to analyze, columns, keywords editing Import of document Export to CSV and Microsoft Excel fileWix# (WixSharp) - managed interface for WiX: Release 1.0.0.0: Release 1.0.0.0 Custom UI Custom MSI Dialog Custom CLR Dialog External UIRecaptcha for .NET: Recaptcha for .NET v1.6.0: What's New?Bug fixes Optimized codeMath.NET Numerics: Math.NET Numerics v3.2.0: Linear Algebra: Vector.Map2 (map2 in F#), storage-optimized Linear Algebra: fix RemoveColumn/Row early index bound check (was not strict enough) Statistics: Entropy ~Jeff Mastry Interpolation: use Array.BinarySearch instead of local implementation ~Candy Chiu Resources: fix a corrupted exception message string Portable Build: support .Net 4.0 as well by using profile 328 instead of 344. .Net 3.5: F# extensions now support .Net 3.5 as well .Net 3.5: NuGet package now contains pro...Virto Commerce Enterprise Open Source eCommerce Platform (asp.net mvc): Virto Commerce 1.11: Virto Commerce Community Edition version 1.11. To install the SDK package, please refer to SDK getting started documentation To configure source code package, please refer to Source code getting started documentation This release includes many bug fixes and minor improvements. More details about this release can be found on our blog at http://blog.virtocommerce.com.Json.NET: Json.NET 6.0 Release 4: New feature - Added Merge to LINQ to JSON New feature - Added JValue.CreateNull and JValue.CreateUndefined New feature - Added Windows Phone 8.1 support to .NET 4.0 portable assembly New feature - Added OverrideCreator to JsonObjectContract New feature - Added support for overriding the creation of interfaces and abstract types New feature - Added support for reading UUID BSON binary values as a Guid New feature - Added MetadataPropertyHandling.Ignore New feature - Improv...VidCoder: 1.5.24 Beta: Added NL-Means denoiser. Updated HandBrake core to SVN 6254. Added extra error handling to DVD player code to avoid a crash when the player was moved.New Projects2113110030: ten: pham van long mon: oop 2113110282: Mon: OPP Ten: Tran Thanh Danh2113110294: Mon: OOP Ten: Vo Ngoc Thai Lop:CCQ1311LAarena: TESTSTSTBowling Game: Program that makes you the follow-up to a game of bowling scoresCTP API for .NET: Comprehensive Trading Platform Application Programming Interface for the .NET Framework CTP API Version: V6.3.0Dashboard Chart Exporter for Microsoft Dynamics CRM 2013: Tool to export Dashboard Charts as Image files in Microsoft Dynamics CRM 2013MergeHosts: A python script to merge hosts file together for those enjoying a clean browsing experienceMFCBDAINF: BDA DTV topology information applicationNetFlux Tunneler Client: This client is a simple Tunneler that creates a TCP connection to the server and sends a custom designed header (originally to bypass firewalls) to the server.NetFlux Tunneler Server: This server is a simple Tunneler that accepts a TCP connection from the client and forwards it to another address, intercepting the custom header. oDesk .NET Library: oDesk .NET Portable Library developed by Andrew SorochakOooPlayer: Lightweight music player that supports mp3, aac, ogg, opus, flac, alac, ape, mpc, tta, wv, wma, ac3OOP-2113110292: class :OOP Name : Nguyen Minh TanOOP-2113110299: Mon: OOP Ten: Nguyen Minh Xuan Lop: CCQ1311LA Truong: Cao dang Cong Thuong - Tp. Ho Chi MinhSFML Tower Defence: SFML Tower defence game that was meant to demonstrate the principals taught in the NWU Potchefstroom EERI 314 course. The game is created using SFML and C++.The Happy Birthday Dad Project: A happy birthday message for my dad???????: ???????QQ:2281595668，?????，????，????　　????????????????，?????????????????????，?????????????????????????!???????????????????????????????????????????????、??????????????: ?????QQ:2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。???????????????????: ????????????: ???????????: ?????QQ:2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: ?????QQ:2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ:2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、???????: ???????????: ?????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，?????????????????: ?????????????: ????????????: ????????????: ???????????: ???????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ??????QQ：2281595668，?????，????，????。????????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，???????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,???????: ???????QQ：2281595668，?????，????，????。????????????????，?????????????????????，?????????????????????????!???????????????????????????????????????????????、?????????、??????: ??????QQ:2281595668，?????，????，????　　???????????????，?????????????????????，????????????????????????!??????????????????????????????????????????????、?????????、????????: ??????????: ??????????: ?????QQ:2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、???????: ?????????????: ????????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ???????QQ：2281595668，?????，????，????。????????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????: ????????????: ??????QQ:2281595668，?????，????，????　　???????????????????，????????????，???????98???????????，????，??????????，?????????. 　　?????????,????????????????，???????????，???????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，???????: ????????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ:2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。???????????????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，???????????????: ?????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”?????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ:2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ:2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ??????QQ:2281595668，?????，????，????　　????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，???????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、???????: ??????QQ：2281595668，?????，????，????。???????????????，??????????????????????????，????????????!??????????????????????????????????????????、???????、????、??????、??????????: ?????QQ:2281595668，?????，????，????　　??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;?????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ:2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ??????QQ：2281595668，?????，????，????。????????????????????，??????????????，????，?????????????????????，??????。????????，??????????????????????。????????，????????????，?????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，??????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，????????????????????????????????: ?????QQ:2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，???????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，????????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，???????????????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ:2281595668，?????，????，????　　??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;?????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ????????????: ???????????: ?????QQ:2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ????????????: ???????????: 5454??????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,??????????????????: ???????????: ????? QQ:2281595668，?????，????，????。????? ??????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;???????: ?????QQ:2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，???????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，???????????????: ?????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”?????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ??????????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ:2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ????? QQ:2281595668，?????，????，????。????? ?????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，???????????????: ???????????: ?????QQ:2281595668，?????，????，????　　???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，?????????????????: jhfiwegb?????: ???????????: ??????QQ:2281595668，?????，????，????　　???????????????????，????????????，???????98???????????，????，??????????，?????????. 　　?????????,????????????????，???????????，??????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????????: ?????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”?????????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，????????????????: ??????QQ:2281595668，?????，????，????　　???????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，??????????????????????????????: ???????????: ???????????: ???????????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ???????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????????: ???????????: ?????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,????????????????????: ?????????????: ????????????: ??????QQ：2281595668，?????，????，????。??????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???? ?????，?????????????: ???????????: ???????????: ????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，????????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，?????????: ??????QQ:2281595668，?????，????，????　　???????????????????，????????????，???????98???????????，????，??????????，?????????. 　　?????????,????????????????，???????????，???????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，???????????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，????????????????????????????????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ????????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，??????????: ?????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，???????????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ????????????: ??????QQ:2281595668，?????，????，????　　???????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，?????????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ???????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ????????????: ????????????: ????????????: ???????????: ??????????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ??????????: ??????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，???????????: ??????QQ:2281595668，?????，????，????　　???????????????，?????????????????????，????????????????????????!??????????????????????????????????????????????、?????????、????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ??????????: ?????QQ:2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，????????: ??????????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，??????????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，???????????: ??????QQ:2281595668，?????，????，????　　???????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????: ?????QQ:2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ????????????: ???????????: ?????QQ:2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。???????????????????: ????????????: ???????????: ?????????????: ??????????????: ??????QQ：2281595668，?????，????，????。??????????????????，????????。????????????????、???????。??????????????。?????????????????,??????,??????,??????,??????????????，???????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ:2281595668，?????，????，????　　??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，???????????????????????????????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，??????????: ????? QQ:2281595668，?????，????，????。????? ????????????，????????。????????????????、???????。??????????????。????? ???????????,??????,??????,??????,??????????????，????????: ??????QQ:2281595668，?????，????，????　　???????????????，?????????????????????，????????????????????????!??????????????????????????????????????????????、?????????、????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、???????????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ????????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ????????????: ????????????????: ?????????????????: ?????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;???????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。??????????????????: ?????QQ：2281595668，?????，????，????。　　??????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，???????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????????: ??????QQ：2281595668，?????，????，????。???????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，???????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????????: ???????????: ???????????: ??????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ????? QQ：2281595668，?????，????，????。?????????? ???????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ:2281595668，?????，????，????　　??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;?????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ:2281595668，?????，????，????　　??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;?????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，?????????: ??????QQ：2281595668，?????，????，????。???????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，????????????????????????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、???????????????: ???????????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: ???????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ??????QQ:2281595668，?????，????，????　　???????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ:2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ???????????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，????????????????????????????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，???????: ????????????: ????????????: ???????????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，???????: ?????????????: ???????????: ??????????: ??????????: ????????????: ??????????????: ????????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ????????????: ????????????: ???????????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，?????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，?????????: ????????????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ:2281595668，?????，????，????　　??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、???????: ????????????: ?????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,??????????????????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ??????QQ：2281595668，?????，????，????。???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、???????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ???????????: ?????QQ：2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ??????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ????????????: ????????????: ????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: xtbjza??????: ??????QQ：2281595668，?????，????，????。???????????????，??????????????????????????，????????????!??????????????????????????????????????????、???????、????、??????、????????????: ???????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ?????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ:2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,????????: ?????????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。???????????????????: ??????QQ：2281595668，?????，????，????。???????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????????: ?????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，???????????????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ????? QQ:2281595668，?????，????，????。????? ??????????????，??????????????，????，?????????????????????，??????。????? ??，??????????????????????。????????，????????????，??????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ????????????: ??????QQ：2281595668，?????，????，????。??????????????????，????????。????????????????、???????。??????????????。?????????????????,??????,??????,??????,??????????????，?????????: ????????????: ?????QQ:2281595668??????: ???????????: ????? QQ：2281595668，?????，????，????。????? ????????????，????????。????????????????、???????。??????????????。????? ???????????,??????,??????,??????,??????????????，????????: ????????????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????????: ?????QQ:2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、???????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，?????????????: ??????????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????: ??????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”????????: ??????QQ:2281595668，?????，????，????　　????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，???????????: ?????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ?????，?????: ???????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ??????????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，???????????: ????????????: ??????????????: ????????QQ:2281595668，?????，????，????　　??????????????????????，??1998?7??????，??????????，?????????，?????????????，?????????????????????????. ????????????????????，??????: ??????QQ：2281595668，?????，????，????。???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ??????????: ?????QQ:2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，??????: ????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，????????????: ??????QQ：2281595668，?????，????，????。???????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ：2281595668，?????，????，????。?????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”???? ?????，???????????????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,??????????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ?????????????: ?????????????: ???????????: ????? QQ：2281595668，?????，????，????。????? ???????????????????，????????????，????? ?98???????????，????，??????????，?????????. 　　?????????,????????????????，??????????????: ???????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，?????????: ????????????: ???????????: ??????????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,??????: ???????????: 　?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，???????????????????????????????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，????????????????????????????????: ??????????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，????????????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”???????: ?????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”??????????: ??????QQ:2281595668，?????，????，????　　???????????????????，????????????，???????98???????????，????，??????????，?????????. 　　?????????,????????????????，???????????，??????: ?????????????: ????????????????: ????????????????: ?????????????: ????????????: ???????QQ:2281595668，?????，????，????　　???????????????????，????????????，????????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ??????QQ:2281595668，?????，????，????　　???????????????????，????????????，???????98???????????，????，??????????，?????????. 　　?????????,????????????????，???????????，???????: ????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????????: ?????????????: ???????QQ:2281595668，?????，????，????　　???????????????????，????????????，????????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，????????: ?????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,??????????????????: ?????QQ:2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，??????: ?????QQ：2281595668，?????，????，????。?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????。???????????????????: ????????????: ??????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，????????????，????，??“??????”????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;???????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,??????????????????: ????????????：　　?????：??????、??、??、????、????(??、??、????)???、???????、?????、?????????????????(???、??、???、??、???); 　　?????：???、?????、???、?????(??、??、??、???)、?????(??????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、??????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ:2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ:2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ：2281595668，?????，????，????。?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，????????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，?????????????????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ??????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;??????: ???????????: ???????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。???????，??????????????????????。????????，????????????，????????: ?????QQ:2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ???????????: ????????????: ????????????: ?????????????: ???????QQ:2281595668，?????，????，????　　????????????????????????????,????????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;?????????: ?????QQ：2281595668，?????，????，????。????????????????????????，????????????，??????98???????????，????，??????????，?????????. 　　?????????,????????????????，????????????????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，???????????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、???????????????: ?????????????: ?????????????: ?????????????: ???????QQ：2281595668，?????，????，????。　　????????98???????????，???????????????????，????????????，????，??????????，?????????，?????????,????????????????，???????????，???????: ???????????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，????????????????: ?????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ?????，?????: ???????????: ????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;???????: ?????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，???????????，????，??“??????”?????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ:2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。?????????????????，????????。????????????????、???????。??????????????。????????????????,??????,??????,??????,??????????????，????,?????: ?????QQ:2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，???????????????: ?????QQ：2281595668，?????，????，????。???????????????????，??????????????，????，?????????????????????，??????。??，??????????????????????。????????，????????????，??????，???????: ??????QQ:2281595668，?????，????，????　　???????????????，?????????????????????，????????????????????????!??????????????????????????????????????????????、?????????、?????????: ??????QQ:2281595668，?????，????，????　　????????????????????，????????。????????????????、???????。??????????????。???????????????????,??????,??????,??????,?????????????????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，???????: ???????QQ：2281595668，?????，????，????。???????????，??????????????????。?????? ?????，????????????????，????????????。????，?????????????，????，??“??????”???? ?????，?????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????????，????????????!?????????????????????????????????????????、???????、????、??????、????????????????: ???????QQ:2281595668，?????，????，????　　?????2004?，??????????????，??????????????????。?????? ?????，????????????????，????????????。????，?????????????，????，??“??????”???????: ???????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ????????????: ?????????????: ??????????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、????????: ?????????????: ????????????: ????????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ：2281595668，?????，????，????。??????????????，?????????????????????，???????????????????????!?????????????????????????????????????????????、?????????、??????、??????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ????????????: ???????QQ：2281595668，?????，????，????。???????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，??????????????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ???????????: ??????QQ：2281595668，?????，????，????。??????????????????，????????。????????????????、???????。??????????????。?????????????????,??????,??????,??????,??????????????，???????: ????? QQ：2281595668，?????，????，????。????? ????????????，????????。????????????????、???????。??????????????。????? ???????????,??????,??????,??????,??????????????，???????: ?????QQ:2281595668，?????，????，????。?????????????????!????????????????????????????????????????????、?????????、??????、????????、????????????。??????????，???????????，?????: ?????QQ：2281595668，?????，????，????。??????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????????: ???????????: ?????QQ:2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ???????????: ??????QQ：2281595668，?????，????，????。????????????????????，??1998?7??????，??????????，?????????，?????????????，???????????????????????. ????????????????????，????????????: ?????QQ:2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，?????: ?????QQ:2281595668，?????，????，????。　　?????????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，?????????????，??????: ?????QQ：2281595668，?????，????，????。???????????????????，??1998?7??????，??????????，?????????，?????????????，??????????????????????. ????????????????????，???????????????: ?????QQ：2281595668，?????，????，????。??????????????，??????、????、?????????????????????????，???????????? ??。???????????，??????????，???????????，????2000?，??????????，??????: ????????????: ???????????: ???????????: ???????????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ????????????: ??????QQ:2281595668，?????，????，????　　???????????????????????????,???????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;???????: ?????QQ：2281595668，?????，????，????。??????????????????????????,??????????,???????????????????????,???????:????,????;?????????????;??;??;????;????????;????;????;???????: ??????QQ:2281595668，?????，????，????　　????? ????????、??、??、??????????????????，???????，???????，?????????????，??????????????，?????、??????????、??、???，???????? ???????????: ??????QQ:2281595668，?????，????，????　　???????????1998?????????。????????????，???????????????，?????????????，???????????，?2003?????????????，?????????????????????????????: ???????????: ?????????????: ???????

Read the article

How John Got 15x Improvement Without Really Trying

- by rchrd

The following article was published on a Sun Microsystems website a number of years ago by John Feo. It is still useful and worth preserving. So I'm republishing it here. How I Got 15x Improvement Without Really Trying John Feo, Sun Microsystems Taking ten "personal" program codes used in scientific and engineering research, the author was able to get from 2 to 15 times performance improvement easily by applying some simple general optimization techniques. Introduction Scientific research based on computer simulation depends on the simulation for advancement. The research can advance only as fast as the computational codes can execute. The codes' efficiency determines both the rate and quality of results. In the same amount of time, a faster program can generate more results and can carry out a more detailed simulation of physical phenomena than a slower program. Highly optimized programs help science advance quickly and insure that monies supporting scientific research are used as effectively as possible. Scientific computer codes divide into three broad categories: ISV, community, and personal. ISV codes are large, mature production codes developed and sold commercially. The codes improve slowly over time both in methods and capabilities, and they are well tuned for most vendor platforms. Since the codes are mature and complex, there are few opportunities to improve their performance solely through code optimization. Improvements of 10% to 15% are typical. Examples of ISV codes are DYNA3D, Gaussian, and Nastran. Community codes are non-commercial production codes used by a particular research field. Generally, they are developed and distributed by a single academic or research institution with assistance from the community. Most users just run the codes, but some develop new methods and extensions that feed back into the general release. The codes are available on most vendor platforms. Since these codes are younger than ISV codes, there are more opportunities to optimize the source code. Improvements of 50% are not unusual. Examples of community codes are AMBER, CHARM, BLAST, and FASTA. Personal codes are those written by single users or small research groups for their own use. These codes are not distributed, but may be passed from professor-to-student or student-to-student over several years. They form the primordial ocean of applications from which community and ISV codes emerge. Government research grants pay for the development of most personal codes. This paper reports on the nature and performance of this class of codes. Over the last year, I have looked at over two dozen personal codes from more than a dozen research institutions. The codes cover a variety of scientific fields, including astronomy, atmospheric sciences, bioinformatics, biology, chemistry, geology, and physics. The sources range from a few hundred lines to more than ten thousand lines, and are written in Fortran, Fortran 90, C, and C++. For the most part, the codes are modular, documented, and written in a clear, straightforward manner. They do not use complex language features, advanced data structures, programming tricks, or libraries. I had little trouble understanding what the codes did or how data structures were used. Most came with a makefile. Surprisingly, only one of the applications is parallel. All developers have access to parallel machines, so availability is not an issue. Several tried to parallelize their applications, but stopped after encountering difficulties. Lack of education and a perception that parallelism is difficult prevented most from trying. I parallelized several of the codes using OpenMP, and did not judge any of the codes as difficult to parallelize. Even more surprising than the lack of parallelism is the inefficiency of the codes. I was able to get large improvements in performance in a matter of a few days applying simple optimization techniques. Table 1 lists ten representative codes [names and affiliation are omitted to preserve anonymity]. Improvements on one processor range from 2x to 15.5x with a simple average of 4.75x. I did not use sophisticated performance tools or drill deep into the program's execution character as one would do when tuning ISV or community codes. Using only a profiler and source line timers, I identified inefficient sections of code and improved their performance by inspection. The changes were at a high level. I am sure there is another factor of 2 or 3 in each code, and more if the codes are parallelized. The study’s results show that personal scientific codes are running many times slower than they should and that the problem is pervasive. Computational scientists are not sloppy programmers; however, few are trained in the art of computer programming or code optimization. I found that most have a working knowledge of some programming language and standard software engineering practices; but they do not know, or think about, how to make their programs run faster. They simply do not know the standard techniques used to make codes run faster. In fact, they do not even perceive that such techniques exist. The case studies described in this paper show that applying simple, well known techniques can significantly increase the performance of personal codes. It is important that the scientific community and the Government agencies that support scientific research find ways to better educate academic scientific programmers. The inefficiency of their codes is so bad that it is retarding both the quality and progress of scientific research. # cacheperformance redundantoperations loopstructures performanceimprovement 1 x x 15.5 2 x 2.8 3 x x 2.5 4 x 2.1 5 x x 2.0 6 x 5.0 7 x 5.8 8 x 6.3 9 2.2 10 x x 3.3 Table 1 — Area of improvement and performance gains of 10 codes The remainder of the paper is organized as follows: sections 2, 3, and 4 discuss the three most common sources of inefficiencies in the codes studied. These are cache performance, redundant operations, and loop structures. Each section includes several examples. The last section summaries the work and suggests a possible solution to the issues raised. Optimizing cache performance Commodity microprocessor systems use caches to increase memory bandwidth and reduce memory latencies. Typical latencies from processor to L1, L2, local, and remote memory are 3, 10, 50, and 200 cycles, respectively. Moreover, bandwidth falls off dramatically as memory distances increase. Programs that do not use cache effectively run many times slower than programs that do. When optimizing for cache, the biggest performance gains are achieved by accessing data in cache order and reusing data to amortize the overhead of cache misses. Secondary considerations are prefetching, associativity, and replacement; however, the understanding and analysis required to optimize for the latter are probably beyond the capabilities of the non-expert. Much can be gained simply by accessing data in the correct order and maximizing data reuse. 6 out of the 10 codes studied here benefited from such high level optimizations. Array Accesses The most important cache optimization is the most basic: accessing Fortran array elements in column order and C array elements in row order. Four of the ten codes—1, 2, 4, and 10—got it wrong. Compilers will restructure nested loops to optimize cache performance, but may not do so if the loop structure is too complex, or the loop body includes conditionals, complex addressing, or function calls. In code 1, the compiler failed to invert a key loop because of complex addressing do I = 0, 1010, delta_x IM = I - delta_x IP = I + delta_x do J = 5, 995, delta_x JM = J - delta_x JP = J + delta_x T1 = CA1(IP, J) + CA1(I, JP) T2 = CA1(IM, J) + CA1(I, JM) S1 = T1 + T2 - 4 * CA1(I, J) CA(I, J) = CA1(I, J) + D * S1 end do end do In code 2, the culprit is conditionals do I = 1, N do J = 1, N If (IFLAG(I,J) .EQ. 0) then T1 = Value(I, J-1) T2 = Value(I-1, J) T3 = Value(I, J) T4 = Value(I+1, J) T5 = Value(I, J+1) Value(I,J) = 0.25 * (T1 + T2 + T5 + T4) Delta = ABS(T3 - Value(I,J)) If (Delta .GT. MaxDelta) MaxDelta = Delta endif enddo enddo I fixed both programs by inverting the loops by hand. Code 10 has three-dimensional arrays and triply nested loops. The structure of the most computationally intensive loops is too complex to invert automatically or by hand. The only practical solution is to transpose the arrays so that the dimension accessed by the innermost loop is in cache order. The arrays can be transposed at construction or prior to entering a computationally intensive section of code. The former requires all array references to be modified, while the latter is cost effective only if the cost of the transpose is amortized over many accesses. I used the second approach to optimize code 10. Code 5 has four-dimensional arrays and loops are nested four deep. For all of the reasons cited above the compiler is not able to restructure three key loops. Assume C arrays and let the four dimensions of the arrays be i, j, k, and l. In the original code, the index structure of the three loops is L1: for i L2: for i L3: for i for l for l for j for k for j for k for j for k for l So only L3 accesses array elements in cache order. L1 is a very complex loop—much too complex to invert. I brought the loop into cache alignment by transposing the second and fourth dimensions of the arrays. Since the code uses a macro to compute all array indexes, I effected the transpose at construction and changed the macro appropriately. The dimensions of the new arrays are now: i, l, k, and j. L3 is a simple loop and easily inverted. L2 has a loop-carried scalar dependence in k. By promoting the scalar name that carries the dependence to an array, I was able to invert the third and fourth subloops aligning the loop with cache. Code 5 is by far the most difficult of the four codes to optimize for array accesses; but the knowledge required to fix the problems is no more than that required for the other codes. I would judge this code at the limits of, but not beyond, the capabilities of appropriately trained computational scientists. Array Strides When a cache miss occurs, a line (64 bytes) rather than just one word is loaded into the cache. If data is accessed stride 1, than the cost of the miss is amortized over 8 words. Any stride other than one reduces the cost savings. Two of the ten codes studied suffered from non-unit strides. The codes represent two important classes of "strided" codes. Code 1 employs a multi-grid algorithm to reduce time to convergence. The grids are every tenth, fifth, second, and unit element. Since time to convergence is inversely proportional to the distance between elements, coarse grids converge quickly providing good starting values for finer grids. The better starting values further reduce the time to convergence. The downside is that grids of every nth element, n > 1, introduce non-unit strides into the computation. In the original code, much of the savings of the multi-grid algorithm were lost due to this problem. I eliminated the problem by compressing (copying) coarse grids into continuous memory, and rewriting the computation as a function of the compressed grid. On convergence, I copied the final values of the compressed grid back to the original grid. The savings gained from unit stride access of the compressed grid more than paid for the cost of copying. Using compressed grids, the loop from code 1 included in the previous section becomes do j = 1, GZ do i = 1, GZ T1 = CA(i+0, j-1) + CA(i-1, j+0) T4 = CA1(i+1, j+0) + CA1(i+0, j+1) S1 = T1 + T4 - 4 * CA1(i+0, j+0) CA(i+0, j+0) = CA1(i+0, j+0) + DD * S1 enddo enddo where CA and CA1 are compressed arrays of size GZ. Code 7 traverses a list of objects selecting objects for later processing. The labels of the selected objects are stored in an array. The selection step has unit stride, but the processing steps have irregular stride. A fix is to save the parameters of the selected objects in temporary arrays as they are selected, and pass the temporary arrays to the processing functions. The fix is practical if the same parameters are used in selection as in processing, or if processing comprises a series of distinct steps which use overlapping subsets of the parameters. Both conditions are true for code 7, so I achieved significant improvement by copying parameters to temporary arrays during selection. Data reuse In the previous sections, we optimized for spatial locality. It is also important to optimize for temporal locality. Once read, a datum should be used as much as possible before it is forced from cache. Loop fusion and loop unrolling are two techniques that increase temporal locality. Unfortunately, both techniques increase register pressure—as loop bodies become larger, the number of registers required to hold temporary values grows. Once register spilling occurs, any gains evaporate quickly. For multiprocessors with small register sets or small caches, the sweet spot can be very small. In the ten codes presented here, I found no opportunities for loop fusion and only two opportunities for loop unrolling (codes 1 and 3). In code 1, unrolling the outer and inner loop one iteration increases the number of result values computed by the loop body from 1 to 4, do J = 1, GZ-2, 2 do I = 1, GZ-2, 2 T1 = CA1(i+0, j-1) + CA1(i-1, j+0) T2 = CA1(i+1, j-1) + CA1(i+0, j+0) T3 = CA1(i+0, j+0) + CA1(i-1, j+1) T4 = CA1(i+1, j+0) + CA1(i+0, j+1) T5 = CA1(i+2, j+0) + CA1(i+1, j+1) T6 = CA1(i+1, j+1) + CA1(i+0, j+2) T7 = CA1(i+2, j+1) + CA1(i+1, j+2) S1 = T1 + T4 - 4 * CA1(i+0, j+0) S2 = T2 + T5 - 4 * CA1(i+1, j+0) S3 = T3 + T6 - 4 * CA1(i+0, j+1) S4 = T4 + T7 - 4 * CA1(i+1, j+1) CA(i+0, j+0) = CA1(i+0, j+0) + DD * S1 CA(i+1, j+0) = CA1(i+1, j+0) + DD * S2 CA(i+0, j+1) = CA1(i+0, j+1) + DD * S3 CA(i+1, j+1) = CA1(i+1, j+1) + DD * S4 enddo enddo The loop body executes 12 reads, whereas as the rolled loop shown in the previous section executes 20 reads to compute the same four values. In code 3, two loops are unrolled 8 times and one loop is unrolled 4 times. Here is the before for (k = 0; k < NK[u]; k++) { sum = 0.0; for (y = 0; y < NY; y++) { sum += W[y][u][k] * delta[y]; } backprop[i++]=sum; } and after code for (k = 0; k < KK - 8; k+=8) { sum0 = 0.0; sum1 = 0.0; sum2 = 0.0; sum3 = 0.0; sum4 = 0.0; sum5 = 0.0; sum6 = 0.0; sum7 = 0.0; for (y = 0; y < NY; y++) { sum0 += W[y][0][k+0] * delta[y]; sum1 += W[y][0][k+1] * delta[y]; sum2 += W[y][0][k+2] * delta[y]; sum3 += W[y][0][k+3] * delta[y]; sum4 += W[y][0][k+4] * delta[y]; sum5 += W[y][0][k+5] * delta[y]; sum6 += W[y][0][k+6] * delta[y]; sum7 += W[y][0][k+7] * delta[y]; } backprop[k+0] = sum0; backprop[k+1] = sum1; backprop[k+2] = sum2; backprop[k+3] = sum3; backprop[k+4] = sum4; backprop[k+5] = sum5; backprop[k+6] = sum6; backprop[k+7] = sum7; } for one of the loops unrolled 8 times. Optimizing for temporal locality is the most difficult optimization considered in this paper. The concepts are not difficult, but the sweet spot is small. Identifying where the program can benefit from loop unrolling or loop fusion is not trivial. Moreover, it takes some effort to get it right. Still, educating scientific programmers about temporal locality and teaching them how to optimize for it will pay dividends. Reducing instruction count Execution time is a function of instruction count. Reduce the count and you usually reduce the time. The best solution is to use a more efficient algorithm; that is, an algorithm whose order of complexity is smaller, that converges quicker, or is more accurate. Optimizing source code without changing the algorithm yields smaller, but still significant, gains. This paper considers only the latter because the intent is to study how much better codes can run if written by programmers schooled in basic code optimization techniques. The ten codes studied benefited from three types of "instruction reducing" optimizations. The two most prevalent were hoisting invariant memory and data operations out of inner loops. The third was eliminating unnecessary data copying. The nature of these inefficiencies is language dependent. Memory operations The semantics of C make it difficult for the compiler to determine all the invariant memory operations in a loop. The problem is particularly acute for loops in functions since the compiler may not know the values of the function's parameters at every call site when compiling the function. Most compilers support pragmas to help resolve ambiguities; however, these pragmas are not comprehensive and there is no standard syntax. To guarantee that invariant memory operations are not executed repetitively, the user has little choice but to hoist the operations by hand. The problem is not as severe in Fortran programs because in the absence of equivalence statements, it is a violation of the language's semantics for two names to share memory. Codes 3 and 5 are C programs. In both cases, the compiler did not hoist all invariant memory operations from inner loops. Consider the following loop from code 3 for (y = 0; y < NY; y++) { i = 0; for (u = 0; u < NU; u++) { for (k = 0; k < NK[u]; k++) { dW[y][u][k] += delta[y] * I1[i++]; } } } Since dW[y][u] can point to the same memory space as delta for one or more values of y and u, assignment to dW[y][u][k] may change the value of delta[y]. In reality, dW and delta do not overlap in memory, so I rewrote the loop as for (y = 0; y < NY; y++) { i = 0; Dy = delta[y]; for (u = 0; u < NU; u++) { for (k = 0; k < NK[u]; k++) { dW[y][u][k] += Dy * I1[i++]; } } } Failure to hoist invariant memory operations may be due to complex address calculations. If the compiler can not determine that the address calculation is invariant, then it can hoist neither the calculation nor the associated memory operations. As noted above, code 5 uses a macro to address four-dimensional arrays #define MAT4D(a,q,i,j,k) (double *)((a)->data + (q)*(a)->strides[0] + (i)*(a)->strides[3] + (j)*(a)->strides[2] + (k)*(a)->strides[1]) The macro is too complex for the compiler to understand and so, it does not identify any subexpressions as loop invariant. The simplest way to eliminate the address calculation from the innermost loop (over i) is to define a0 = MAT4D(a,q,0,j,k) before the loop and then replace all instances of *MAT4D(a,q,i,j,k) in the loop with a0[i] A similar problem appears in code 6, a Fortran program. The key loop in this program is do n1 = 1, nh nx1 = (n1 - 1) / nz + 1 nz1 = n1 - nz * (nx1 - 1) do n2 = 1, nh nx2 = (n2 - 1) / nz + 1 nz2 = n2 - nz * (nx2 - 1) ndx = nx2 - nx1 ndy = nz2 - nz1 gxx = grn(1,ndx,ndy) gyy = grn(2,ndx,ndy) gxy = grn(3,ndx,ndy) balance(n1,1) = balance(n1,1) + (force(n2,1) * gxx + force(n2,2) * gxy) * h1 balance(n1,2) = balance(n1,2) + (force(n2,1) * gxy + force(n2,2) * gyy)*h1 end do end do The programmer has written this loop well—there are no loop invariant operations with respect to n1 and n2. However, the loop resides within an iterative loop over time and the index calculations are independent with respect to time. Trading space for time, I precomputed the index values prior to the entering the time loop and stored the values in two arrays. I then replaced the index calculations with reads of the arrays. Data operations Ways to reduce data operations can appear in many forms. Implementing a more efficient algorithm produces the biggest gains. The closest I came to an algorithm change was in code 4. This code computes the inner product of K-vectors A(i) and B(j), 0 = i < N, 0 = j < M, for most values of i and j. Since the program computes most of the NM possible inner products, it is more efficient to compute all the inner products in one triply-nested loop rather than one at a time when needed. The savings accrue from reading A(i) once for all B(j) vectors and from loop unrolling. for (i = 0; i < N; i+=8) { for (j = 0; j < M; j++) { sum0 = 0.0; sum1 = 0.0; sum2 = 0.0; sum3 = 0.0; sum4 = 0.0; sum5 = 0.0; sum6 = 0.0; sum7 = 0.0; for (k = 0; k < K; k++) { sum0 += A[i+0][k] * B[j][k]; sum1 += A[i+1][k] * B[j][k]; sum2 += A[i+2][k] * B[j][k]; sum3 += A[i+3][k] * B[j][k]; sum4 += A[i+4][k] * B[j][k]; sum5 += A[i+5][k] * B[j][k]; sum6 += A[i+6][k] * B[j][k]; sum7 += A[i+7][k] * B[j][k]; } C[i+0][j] = sum0; C[i+1][j] = sum1; C[i+2][j] = sum2; C[i+3][j] = sum3; C[i+4][j] = sum4; C[i+5][j] = sum5; C[i+6][j] = sum6; C[i+7][j] = sum7; }} This change requires knowledge of a typical run; i.e., that most inner products are computed. The reasons for the change, however, derive from basic optimization concepts. It is the type of change easily made at development time by a knowledgeable programmer. In code 5, we have the data version of the index optimization in code 6. Here a very expensive computation is a function of the loop indices and so cannot be hoisted out of the loop; however, the computation is invariant with respect to an outer iterative loop over time. We can compute its value for each iteration of the computation loop prior to entering the time loop and save the values in an array. The increase in memory required to store the values is small in comparison to the large savings in time. The main loop in Code 8 is doubly nested. The inner loop includes a series of guarded computations; some are a function of the inner loop index but not the outer loop index while others are a function of the outer loop index but not the inner loop index for (j = 0; j < N; j++) { for (i = 0; i < M; i++) { r = i * hrmax; R = A[j]; temp = (PRM[3] == 0.0) ? 1.0 : pow(r, PRM[3]); high = temp * kcoeff * B[j] * PRM[2] * PRM[4]; low = high * PRM[6] * PRM[6] / (1.0 + pow(PRM[4] * PRM[6], 2.0)); kap = (R > PRM[6]) ? high * R * R / (1.0 + pow(PRM[4]*r, 2.0) : low * pow(R/PRM[6], PRM[5]); < rest of loop omitted > }} Note that the value of temp is invariant to j. Thus, we can hoist the computation for temp out of the loop and save its values in an array. for (i = 0; i < M; i++) { r = i * hrmax; TEMP[i] = pow(r, PRM[3]); } [N.B. – the case for PRM[3] = 0 is omitted and will be reintroduced later.] We now hoist out of the inner loop the computations invariant to i. Since the conditional guarding the value of kap is invariant to i, it behooves us to hoist the computation out of the inner loop, thereby executing the guard once rather than M times. The final version of the code is for (j = 0; j < N; j++) { R = rig[j] / 1000.; tmp1 = kcoeff * par[2] * beta[j] * par[4]; tmp2 = 1.0 + (par[4] * par[4] * par[6] * par[6]); tmp3 = 1.0 + (par[4] * par[4] * R * R); tmp4 = par[6] * par[6] / tmp2; tmp5 = R * R / tmp3; tmp6 = pow(R / par[6], par[5]); if ((par[3] == 0.0) && (R > par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * tmp5; } else if ((par[3] == 0.0) && (R <= par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * tmp4 * tmp6; } else if ((par[3] != 0.0) && (R > par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * TEMP[i] * tmp5; } else if ((par[3] != 0.0) && (R <= par[6])) { for (i = 1; i <= imax1; i++) KAP[i] = tmp1 * TEMP[i] * tmp4 * tmp6; } for (i = 0; i < M; i++) { kap = KAP[i]; r = i * hrmax; < rest of loop omitted > } } Maybe not the prettiest piece of code, but certainly much more efficient than the original loop, Copy operations Several programs unnecessarily copy data from one data structure to another. This problem occurs in both Fortran and C programs, although it manifests itself differently in the two languages. Code 1 declares two arrays—one for old values and one for new values. At the end of each iteration, the array of new values is copied to the array of old values to reset the data structures for the next iteration. This problem occurs in Fortran programs not included in this study and in both Fortran 77 and Fortran 90 code. Introducing pointers to the arrays and swapping pointer values is an obvious way to eliminate the copying; but pointers is not a feature that many Fortran programmers know well or are comfortable using. An easy solution not involving pointers is to extend the dimension of the value array by 1 and use the last dimension to differentiate between arrays at different times. For example, if the data space is N x N, declare the array (N, N, 2). Then store the problem’s initial values in (_, _, 2) and define the scalar names new = 2 and old = 1. At the start of each iteration, swap old and new to reset the arrays. The old–new copy problem did not appear in any C program. In programs that had new and old values, the code swapped pointers to reset data structures. Where unnecessary coping did occur is in structure assignment and parameter passing. Structures in C are handled much like scalars. Assignment causes the data space of the right-hand name to be copied to the data space of the left-hand name. Similarly, when a structure is passed to a function, the data space of the actual parameter is copied to the data space of the formal parameter. If the structure is large and the assignment or function call is in an inner loop, then copying costs can grow quite large. While none of the ten programs considered here manifested this problem, it did occur in programs not included in the study. A simple fix is always to refer to structures via pointers. Optimizing loop structures Since scientific programs spend almost all their time in loops, efficient loops are the key to good performance. Conditionals, function calls, little instruction level parallelism, and large numbers of temporary values make it difficult for the compiler to generate tightly packed, highly efficient code. Conditionals and function calls introduce jumps that disrupt code flow. Users should eliminate or isolate conditionls to their own loops as much as possible. Often logical expressions can be substituted for if-then-else statements. For example, code 2 includes the following snippet MaxDelta = 0.0 do J = 1, N do I = 1, M < code omitted > Delta = abs(OldValue ? NewValue) if (Delta > MaxDelta) MaxDelta = Delta enddo enddo if (MaxDelta .gt. 0.001) goto 200 Since the only use of MaxDelta is to control the jump to 200 and all that matters is whether or not it is greater than 0.001, I made MaxDelta a boolean and rewrote the snippet as MaxDelta = .false. do J = 1, N do I = 1, M < code omitted > Delta = abs(OldValue ? NewValue) MaxDelta = MaxDelta .or. (Delta .gt. 0.001) enddo enddo if (MaxDelta) goto 200 thereby, eliminating the conditional expression from the inner loop. A microprocessor can execute many instructions per instruction cycle. Typically, it can execute one or more memory, floating point, integer, and jump operations. To be executed simultaneously, the operations must be independent. Thick loops tend to have more instruction level parallelism than thin loops. Moreover, they reduce memory traffice by maximizing data reuse. Loop unrolling and loop fusion are two techniques to increase the size of loop bodies. Several of the codes studied benefitted from loop unrolling, but none benefitted from loop fusion. This observation is not too surpising since it is the general tendency of programmers to write thick loops. As loops become thicker, the number of temporary values grows, increasing register pressure. If registers spill, then memory traffic increases and code flow is disrupted. A thick loop with many temporary values may execute slower than an equivalent series of thin loops. The biggest gain will be achieved if the thick loop can be split into a series of independent loops eliminating the need to write and read temporary arrays. I found such an occasion in code 10 where I split the loop do i = 1, n do j = 1, m A24(j,i)= S24(j,i) * T24(j,i) + S25(j,i) * U25(j,i) B24(j,i)= S24(j,i) * T25(j,i) + S25(j,i) * U24(j,i) A25(j,i)= S24(j,i) * C24(j,i) + S25(j,i) * V24(j,i) B25(j,i)= S24(j,i) * U25(j,i) + S25(j,i) * V25(j,i) C24(j,i)= S26(j,i) * T26(j,i) + S27(j,i) * U26(j,i) D24(j,i)= S26(j,i) * T27(j,i) + S27(j,i) * V26(j,i) C25(j,i)= S27(j,i) * S28(j,i) + S26(j,i) * U28(j,i) D25(j,i)= S27(j,i) * T28(j,i) + S26(j,i) * V28(j,i) end do end do into two disjoint loops do i = 1, n do j = 1, m A24(j,i)= S24(j,i) * T24(j,i) + S25(j,i) * U25(j,i) B24(j,i)= S24(j,i) * T25(j,i) + S25(j,i) * U24(j,i) A25(j,i)= S24(j,i) * C24(j,i) + S25(j,i) * V24(j,i) B25(j,i)= S24(j,i) * U25(j,i) + S25(j,i) * V25(j,i) end do end do do i = 1, n do j = 1, m C24(j,i)= S26(j,i) * T26(j,i) + S27(j,i) * U26(j,i) D24(j,i)= S26(j,i) * T27(j,i) + S27(j,i) * V26(j,i) C25(j,i)= S27(j,i) * S28(j,i) + S26(j,i) * U28(j,i) D25(j,i)= S27(j,i) * T28(j,i) + S26(j,i) * V28(j,i) end do end do Conclusions Over the course of the last year, I have had the opportunity to work with over two dozen academic scientific programmers at leading research universities. Their research interests span a broad range of scientific fields. Except for two programs that relied almost exclusively on library routines (matrix multiply and fast Fourier transform), I was able to improve significantly the single processor performance of all codes. Improvements range from 2x to 15.5x with a simple average of 4.75x. Changes to the source code were at a very high level. I did not use sophisticated techniques or programming tools to discover inefficiencies or effect the changes. Only one code was parallel despite the availability of parallel systems to all developers. Clearly, we have a problem—personal scientific research codes are highly inefficient and not running parallel. The developers are unaware of simple optimization techniques to make programs run faster. They lack education in the art of code optimization and parallel programming. I do not believe we can fix the problem by publishing additional books or training manuals. To date, the developers in questions have not studied the books or manual available, and are unlikely to do so in the future. Short courses are a possible solution, but I believe they are too concentrated to be much use. The general concepts can be taught in a three or four day course, but that is not enough time for students to practice what they learn and acquire the experience to apply and extend the concepts to their codes. Practice is the key to becoming proficient at optimization. I recommend that graduate students be required to take a semester length course in optimization and parallel programming. We would never give someone access to state-of-the-art scientific equipment costing hundreds of thousands of dollars without first requiring them to demonstrate that they know how to use the equipment. Yet the criterion for time on state-of-the-art supercomputers is at most an interesting project. Requestors are never asked to demonstrate that they know how to use the system, or can use the system effectively. A semester course would teach them the required skills. Government agencies that fund academic scientific research pay for most of the computer systems supporting scientific research as well as the development of most personal scientific codes. These agencies should require graduate schools to offer a course in optimization and parallel programming as a requirement for funding. About the Author John Feo received his Ph.D. in Computer Science from The University of Texas at Austin in 1986. After graduate school, Dr. Feo worked at Lawrence Livermore National Laboratory where he was the Group Leader of the Computer Research Group and principal investigator of the Sisal Language Project. In 1997, Dr. Feo joined Tera Computer Company where he was project manager for the MTA, and oversaw the programming and evaluation of the MTA at the San Diego Supercomputer Center. In 2000, Dr. Feo joined Sun Microsystems as an HPC application specialist. He works with university research groups to optimize and parallelize scientific codes. Dr. Feo has published over two dozen research articles in the areas of parallel parallel programming, parallel programming languages, and application performance.

Read the article

StackOverFlowError while creating Mac object on AS400/Java

- by Prasanna K Rao

Hello all, I am a newbie to AS400-Java programming. I am trying to create my first program to test the implementation of Message Authentication Code (MAC). I am trying to use the HMACSHA1 hash function. My (Java 1.4) program runs fine on a dev box (V5R4).But fails terribly on the QA box (V5R3). My program is as below: ===================================================== import java.security.InvalidKeyException; import java.security.NoSuchAlgorithmException; import java.security.Security; import java.security.Provider; import javax.crypto.Mac; import javax.crypto.spec.SecretKeySpec; import javax.crypto.SecretKey; public class Test01 { private static final String HMAC_SHA1_ALGORITHM = "HmacSHA1"; public static void main (String [] arguments) { byte[] key = { 1,2,3,4,5,6,7,8}; SecretKeySpec SHA1key = new SecretKeySpec(key, "HmacSHA1"); Mac hmac; String strFinalRslt = ""; try { hmac = Mac.getInstance("HmacSHA1"); hmac.init(SHA1key); byte[] result = hmac.doFinal(); strFinalRslt = toHexString(result); }catch (NoSuchAlgorithmException e) { // TODO Auto-generated catch block e.printStackTrace(); }catch (InvalidKeyException e) { // TODO Auto-generated catch block e.printStackTrace(); }catch(StackOverflowError e){ e.printStackTrace(); } System.out.println(strFinalRslt); System.out.println("All done!!!"); } public static byte[] fromHexString ( String s ) { int stringLength = s.length(); if ( (stringLength & 0x1) != 0 ) { throw new IllegalArgumentException ( "fromHexString requires an even number of hex characters" ); } byte[] b = new byte[stringLength / 2]; for ( int i=0,j=0; i 4] ); //look up low nibble char sb.append( hexChar [b[i] & 0x0f] ); } return sb.toString(); } static char[] hexChar = { '0' , '1' , '2' , '3' , '4' , '5' , '6' , '7' , '8' , '9' , 'a' , 'b' , 'c' , 'd' , 'e' , 'f'}; } This program compiles fine and gets the correct response on my win-xp client and also my dev box. But, fails with the following error on the QA box: java.lang.StackOverflowError at java.lang.Throwable.(Throwable.java:180) at java.lang.Error.(Error.java:37) at java.lang.StackOverflowError.(StackOverflowError.java:24) at java.io.Os400FileSystem.list(Native method) at java.io.File.list(File.java:922) at javax.crypto.b.e(Unknown source) at javax.crypto.b.a(Unknown source) at javax.crypto.b.c(Unknown source) at javax.crypto.b£0.run(Unknown source) at javax.crypto.b.(Unknown source) at javax.crypto.Mac.getInstance(Unknown source) I have verified the java.security file and entry corresponding to the jce files are all ok. The DMPJVM command gives me the following response: Thu Jun 03 12:25:34 E Java Virtual Machine Information 016822/QPGMR/11111 ........................................................................ . Classpath . ........................................................................ java.version=1.4 sun.boot.class.path=/QIBM/ProdData/OS400/Java400/jdk/lib/jdkptf14.zip:/QIBM /ProdData/OS400/Java400/ext/ibmjssefw.jar:/QIBM/ProdData/CAP/ibmjsseprovide r.jar:/QIBM/ProdData/OS400/Java400/ext/ibmjsseprovider2.jar:/QIBM/ProdData/ OS400/Java400/ext/ibmpkcs11impl.jar:/QIBM/ProdData/CAP/ibmjssefips.jar:/QIB M/ProdData/OS400/Java400/jdk/lib/IBMiSeriesJSSE.jar:/QIBM/ProdData/OS400/Ja va400/jdk/lib/jce.jar:/QIBM/ProdData/OS400/Java400/jdk/lib/jaas.jar:/QIBM/P rodData/OS400/Java400/jdk/lib/ibmcertpathfw.jar:/QIBM/ProdData/OS400/Java40 0/jdk/lib/ibmcertpathprovider.jar:/QIBM/ProdData/OS400/Java400/ext/ibmpkcs. jar:/QIBM/ProdData/OS400/Java400/jdk/lib/ibmjgssfw.jar:/QIBM/ProdData/OS400 /Java400/jdk/lib/ibmjgssprovider.jar:/QIBM/ProdData/OS400/Java400/jdk/lib/s ecurity.jar:/QIBM/ProdData/OS400/Java400/jdk/lib/charsets.jar:/QIBM/ProdDat a/OS400/Java400/jdk/lib/resources.jar:/QIBM/ProdData/OS400/Java400/jdk/lib/ rt.jar:/QIBM/ProdData/OS400/Java400/jdk/lib/sunrsasign.jar:/QIBM/ProdData/O S400/Java400/ext/IBMmisc.jar:/QIBM/ProdData/Java400/ java.class.path=/myhome/lib/commons-codec-1.3.jar:/myhome/lib/commons-httpc lient-3.1.jar:/myhome/lib/commons-logging-1.1.jar:/myhome/lib/log4j-1.2.15.jar:/myhome/lib/log4j-core.jar ; java.ext.dirs=/QIBM/ProdData/OS400/Java400/jdk/lib/ext:/QIBM/UserData/Java4 00/ext:/QIBM/ProdData/Java400/jdk14/lib/ext java.library.path=/QSYS.LIB/ROBOTLIB.LIB:/QSYS.LIB/QTEMP.LIB:/QSYS.LIB/ODIP GM.LIB:/QSYS.LIB/QGPL.LIB ........................................................................ . Garbage Collection . ........................................................................ Garbage collector parameters Initial size: 16384 K Max size: 240000000 K Current values Heap size: 437952 K Garbage collections: 58 Additional values JIT heap size: 53824 K JVM heap size: 55752 K Last GC cycle time: 1333 ms ........................................................................ . Thread information . ........................................................................ Information for 4 thread(s) of 4 thread(s) processed Thread: 00000004 Thread-0 TDE: B00380000BAA0000 Thread priority: 5 Thread status: Running Thread group: main Runnable: java/lang/Thread Stack: java/io/Os400FileSystem.list(Ljava/io/File;)[Ljava/lang/String;+0 (Os400FileSystem.java:0) java/io/File.list()[Ljava/lang/String;+19 (File.java:922) javax/crypto/b.e()[B+127 (:0) javax/crypto/b.a(Ljava/security/cert/X509Certificate;)V+7 (:0) javax/crypto/b.access$500(Ljava/security/cert/X509Certificate;)V+1 (:0) javax/crypto/b$0.run()Ljava/lang/Object;+98 (:0) javax/crypto/b.()V+507 (:0) javax/crypto/Mac.getInstance(Ljava/lang/String;)Ljavax/crypto/Mac;+10 (:0) Locks: None Thread: 00000007 jitcompilethread TDE: B00380000BD58000 Thread priority: 5 Thread status: Java wait Thread group: system Runnable: java/lang/Thread Stack: None Locks: None Thread: 00000005 Reference Handler TDE: B00380000BAAC000 Thread priority: 10 Thread status: Waiting Wait object: java/lang/ref/Reference$Lock Thread group: system Runnable: java/lang/ref/Reference$ReferenceHandler Stack: java/lang/Object.wait()V+1 (Object.java:452) java/lang/ref/Reference$ReferenceHandler.run()V+47 (Reference.java:169) Locks: None Thread: 00000006 Finalizer TDE: B00380000BAB3000 Thread priority: 8 Thread status: Waiting Wait object: java/lang/ref/ReferenceQueue$Lock Thread group: system Runnable: java/lang/ref/Finalizer$FinalizerThread Stack: java/lang/ref/ReferenceQueue.remove(J)Ljava/lang/ref/Reference;+43 (ReferenceQueue.java:111) java/lang/ref/ReferenceQueue.remove()Ljava/lang/ref/Reference;+1 (ReferenceQueue.java:127) java/lang/ref/Finalizer$FinalizerThread.run()V+3 (Finalizer.java:171) Locks: None ........................................................................ . Class loader information . ........................................................................ 0 Default class loader 1 sun/reflect/DelegatingClassLoader 2 sun/misc/Launcher$ExtClassLoader ........................................................................ . GC heap information . ........................................................................ Loader Objects Class name ------ ------- ---------- 0 1493 [C 0 2122181 java/lang/String 0 47 [Ljava/util/Hashtable$Entry; 0 68 [Ljava/lang/Object; 0 1016 java/lang/Class 0 31 java/util/HashMap 0 37 java/util/Hashtable 0 2 java/lang/ThreadGroup 0 2 java/lang/RuntimePermission 0 2 java/lang/ref/ReferenceQueue$Null 0 5 java/lang/ref/ReferenceQueue 0 50 java/util/Vector 0 4 java/util/Stack 0 3 sun/misc/SoftCache 0 1 [Ljava/lang/ThreadGroup; 0 5 [Ljava/io/ObjectStreamField; 0 1 sun/reflect/ReflectionFactory 0 7 java/lang/ref/ReferenceQueue$Lock 0 10 java/lang/Object 0 1 java/lang/String$CaseInsensitiveComparator 0 1 java/util/Hashtable$EmptyEnumerator 0 1 java/util/Hashtable$EmptyIterator 0 33 [Ljava/util/HashMap$Entry; 0 19210 [J 0 1 sun/nio/cs/StandardCharsets 0 5 java/util/TreeMap 0 1075 java/util/TreeMap$Entry 0 469 [Ljava/lang/String; 0 1 java/lang/StringBuffer 0 2 java/io/FileInputStream 0 2 java/io/FileOutputStream 0 2 java/io/BufferedOutputStream 0 1 java/lang/reflect/ReflectPermission 0 1 [[Ljava/lang/ref/SoftReference; 0 2 [Ljava/lang/ref/SoftReference; 0 2 sun/nio/cs/Surrogate$Parser 0 3 sun/misc/Signal 0 1 [Ljava/io/File; 0 6 java/io/File 0 1 java/util/BitSet 0 17 sun/reflect/NativeConstructorAccessorImpl 0 2 java/net/URLClassLoader$ClassFinder 0 12 java/util/ArrayList 0 32 java/io/RandomAccessFile 0 16 java/lang/Thread 0 1 java/lang/ref/Reference$ReferenceHandler 0 1 java/lang/ref/Finalizer$FinalizerThread 0 266 [B 0 2 java/util/Properties 0 71 java/lang/ref/Finalizer 0 2 com/ibm/nio/cs/DirectEncoder 0 38 java/lang/reflect/Constructor 0 33 java/util/jar/JarFile 0 19200 java/lang/StackOverflowError 0 5 java/security/AccessControlContext 0 2 [Ljava/lang/Thread; 0 4 java/lang/OutOfMemoryError 0 1065 java/util/Hashtable$Entry 0 1 java/io/BufferedInputStream 0 2 java/io/PrintStream 0 2 java/io/OutputStreamWriter 0 428 [I 0 3 java/lang/ClassLoader$NativeLibrary 0 25 java/util/Locale 0 3 sun/misc/URLClassPath 0 30 java/util/zip/Inflater 0 612 java/util/HashMap$Entry 0 2 java/io/FilePermission 0 10 java/io/ObjectStreamField 0 1 java/security/BasicPermissionCollection 0 2 java/security/ProtectionDomain 0 1 java/lang/Integer$1 0 1 java/lang/ref/Reference$Lock 0 1 java/lang/Shutdown$Lock 0 1 java/lang/Runtime 0 36 java/io/FileDescriptor 0 1 java/lang/Long$1 0 202 java/lang/Long 0 3 java/lang/ThreadLocal 0 3 java/nio/charset/CodingErrorAction 0 2 java/nio/charset/CoderResult 0 1 java/nio/charset/CoderResult$1 0 1 java/nio/charset/CoderResult$2 0 1 sun/misc/Unsafe 0 2 java/nio/ByteOrder 0 1 java/io/Os400FileSystem 0 3 java/lang/Boolean 0 1 java/lang/Terminator$1 0 23 java/lang/Integer 0 2 sun/misc/NativeSignalHandler 0 1 sun/misc/Launcher$Factory 0 1 sun/misc/Launcher 0 53 [Ljava/lang/Class; 0 1 java/lang/reflect/ReflectAccess 0 18 sun/reflect/DelegatingConstructorAccessorImpl 0 1 sun/net/www/protocol/file/Handler 0 3 java/util/HashSet 0 3 sun/net/www/protocol/jar/Handler 0 1 java/util/jar/JavaUtilJarAccessImpl 0 1 java/net/UnknownContentHandler 0 2 [Ljava/security/Principal; 0 10 [Ljava/security/cert/Certificate; 0 2 sun/misc/AtomicLongCSImpl 0 3 sun/reflect/DelegatingMethodAccessorImpl 0 1 sun/security/util/ByteArrayLexOrder 0 1 sun/security/util/ByteArrayTagOrder 0 7 sun/security/x509/CertificateVersion 0 7 sun/security/x509/CertificateSerialNumber 0 7 sun/security/x509/SerialNumber 0 7 sun/security/x509/CertificateAlgorithmId 0 7 sun/security/x509/CertificateIssuerName 0 60 sun/security/x509/RDN 0 60 [Lsun/security/x509/AVA; 0 67 sun/security/util/DerInputStream 0 3 [Ljava/math/BigInteger; 0 2 com/ibm/nio/cs/Converter 0 2 sun/nio/cs/StreamEncoder$CharsetSE 0 35 java/lang/ref/SoftReference 0 2 java/nio/HeapByteBuffer 0 2 java/io/BufferedWriter 0 33 sun/misc/URLClassPath$JarLoader 0 4 java/lang/ThreadLocal$ThreadLocalMap$Entry 0 76 java/net/URL 0 1 sun/misc/Launcher$ExtClassLoader 0 1 sun/misc/Launcher$AppClassLoader 0 4 java/lang/Throwable 0 7 java/lang/reflect/Method 0 2 sun/misc/URLClassPath$FileLoader 0 2 java/security/CodeSource 0 2 java/security/Permissions 0 2 java/io/FilePermissionCollection 0 1 java/lang/ThreadLocal$ThreadLocalMap 0 1 javax/crypto/spec/SecretKeySpec 0 17 java/util/jar/Attributes$Name 0 1 [Ljava/lang/ThreadLocal$ThreadLocalMap$Entry; 0 1 java/security/SecureRandom 0 2 sun/security/provider/Sun 0 1 java/util/jar/JarFile$JarFileEntry 0 1 java/util/jar/JarVerifier 0 3 sun/reflect/NativeMethodAccessorImpl 0 116 sun/security/util/ObjectIdentifier 0 1 java/lang/Package 0 2 [S 0 104 java/math/BigInteger 0 20 sun/security/x509/AlgorithmId 0 14 sun/security/x509/X500Name 0 14 [Lsun/security/x509/RDN; 0 60 sun/security/x509/AVA 0 67 sun/security/util/DerValue 0 67 sun/security/util/DerInputBuffer 0 21 sun/security/x509/AVAKeyword 0 6 sun/security/x509/X509CertImpl 0 7 sun/security/x509/X509CertInfo 0 1 [Lsun/security/util/ObjectIdentifier; 0 1 [[Ljava/lang/Byte; 0 3 [[B 0 7 sun/security/provider/DSAPublicKey 0 7 sun/security/x509/AuthorityKeyIdentifierExtension 0 12 [Ljava/lang/Byte; 0 14 java/lang/Byte 0 7 sun/security/x509/CertificateSubjectName 0 7 sun/security/x509/CertificateX509Key 0 14 sun/security/x509/KeyIdentifier 0 4 [Z 0 5 sun/text/Normalizer$Mode 0 7 sun/security/x509/CertificateValidity 0 14 java/util/Date 0 7 sun/security/provider/DSAParameters 0 7 sun/security/util/BitArray 0 7 sun/security/x509/CertificateExtensions 0 7 java/security/AlgorithmParameters 0 7 sun/security/x509/SubjectKeyIdentifierExtension 0 5 sun/security/x509/BasicConstraintsExtension 0 2 sun/security/x509/KeyUsageExtension 0 1 sun/text/CompactCharArray 0 1 sun/text/CompactByteArray 0 1 sun/net/www/protocol/jar/JarFileFactory 0 1 java/util/Collections$EmptySet 0 1 java/util/Collections$EmptyList 0 1 java/util/Collections$ReverseComparator 0 1 com/ibm/security/jgss/i18n/PropertyResource 0 1 javax/crypto/b$0 0 1 sun/security/provider/X509Factory 0 1 sun/reflect/BootstrapConstructorAccessorImpl 1 1 sun/reflect/GeneratedConstructorAccessor3202134454 2 1 com/ibm/crypto/provider/IBMJCE 0 6 java/util/ResourceBundle$LoaderReference 0 1 [Lsun/security/x509/NetscapeCertTypeExtension$MapEntry; 0 1 com/sun/rsajca/Provider 0 1 com/ibm/security/cert/IBMCertPath 0 1 com/ibm/as400/ibmonly/net/ssl/Provider 0 1 com/ibm/jsse/IBMJSSEProvider 0 1 com/ibm/security/jgss/IBMJGSSProvider 0 5 org/ietf/jgss/Oid 0 1 java/util/PropertyResourceBundle 0 7 java/util/ResourceBundle$ResourceCacheKey 0 2 sun/net/www/protocol/jar/URLJarFile 0 6 sun/misc/SoftCache$ValueCell 0 1 java/util/Random 0 1 java/util/Collections$EmptyMap 0 112 com/ibm/security/util/ObjectIdentifier 0 5 java/security/Security$ProviderProperty 0 1 java/security/cert/CertificateFactory 0 1 sun/security/provider/SecureRandom 0 2 java/security/MessageDigest$Delegate 0 2 sun/security/provider/SHA 0 1 sun/util/calendar/ZoneInfo 0 4 com/ibm/security/x509/X500Name 0 2 [Ljava/security/cert/X509Certificate; 0 1 sun/reflect/DelegatingClassLoader 0 1 sun/security/x509/NetscapeCertTypeExtension 0 7 sun/security/x509/NetscapeCertTypeExtension$MapEntry 0 3 [[Ljava/lang/String; 0 3 java/util/Arrays$ArrayList 0 7 com/ibm/security/x509/NetscapeCertTypeExtension$MapEntry 0 1 com/ibm/security/validator/EndEntityChecker 0 1 java/util/AbstractList$Itr 0 1 com/ibm/security/util/ByteArrayLexOrder 0 1 com/ibm/security/util/ByteArrayTagOrder 0 18 [Lcom/ibm/security/x509/AVA; 0 18 com/ibm/security/util/DerInputStream 0 5 com/ibm/security/util/text/Normalizer$Mode 0 1 com/ibm/security/validator/SimpleValidator 0 1 [Lcom/ibm/security/x509/NetscapeCertTypeExtension$MapEntry; 0 4 [Lcom/ibm/security/x509/RDN; 0 1 java/util/Hashtable$Enumerator 0 4 java/util/LinkedHashMap$Entry 0 1 sun/text/resources/LocaleElements 0 1 sun/text/resources/LocaleElements_en 0 22 com/ibm/security/x509/AVAKeyword 0 4 javax/security/auth/x500/X500Principal 0 18 com/ibm/security/x509/RDN 0 18 com/ibm/security/x509/AVA 0 18 com/ibm/security/util/DerInputBuffer 0 18 com/ibm/security/util/DerValue 0 1 com/ibm/security/util/text/CompactCharArray 0 1 com/ibm/security/util/text/CompactByteArray 0 2 java/util/LinkedHashMap 0 1 java/net/InetAddress$1 0 2 [Ljava/net/InetAddress; 0 2 java/net/InetAddress$Cache 0 1 java/net/Inet4AddressImpl 0 3 java/net/Inet4Address 0 2 java/net/InetAddress$CacheEntry ........................................................................ . Global registry information . ........................................................................ Loader Objects Class name ------ ------- ---------- 0 23 [C 0 1017 java/lang/Class 0 1 java/lang/ref/Reference$ReferenceHandler 0 1 java/lang/ref/Finalizer$FinalizerThread 0 1 sun/misc/Launcher$AppClassLoader 0 32 java/io/RandomAccessFile 0 32 [B Can someone please advise me? Thanks a lot, Prasanna

Read the article

SQLite, python, unicode, and non-utf data

- by Nathan Spears

I started by trying to store strings in sqlite using python, and got the message: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. Ok, I switched to Unicode strings. Then I started getting the message: sqlite3.OperationalError: Could not decode to UTF-8 column 'tag_artist' with text 'Sigur Rós' when trying to retrieve data from the db. More research and I started encoding it in utf8, but then 'Sigur Rós' starts looking like 'Sigur RÃ³s' note: My console was set to display in 'latin_1' as @John Machin pointed out. What gives? After reading this, describing exactly the same situation I'm in, it seems as if the advice is to ignore the other advice and use 8-bit bytestrings after all. I didn't know much about unicode and utf before I started this process. I've learned quite a bit in the last couple hours, but I'm still ignorant of whether there is a way to correctly convert 'ó' from latin-1 to utf-8 and not mangle it. If there isn't, why would sqlite 'highly recommend' I switch my application to unicode strings? I'm going to update this question with a summary and some example code of everything I've learned in the last 24 hours so that someone in my shoes can have an easy(er) guide. If the information I post is wrong or misleading in any way please tell me and I'll update, or one of you senior guys can update. Summary of answers Let me first state the goal as I understand it. The goal in processing various encodings, if you are trying to convert between them, is to understand what your source encoding is, then convert it to unicode using that source encoding, then convert it to your desired encoding. Unicode is a base and encodings are mappings of subsets of that base. utf_8 has room for every character in unicode, but because they aren't in the same place as, for instance, latin_1, a string encoded in utf_8 and sent to a latin_1 console will not look the way you expect. In python the process of getting to unicode and into another encoding looks like: str.decode('source_encoding').encode('desired_encoding') or if the str is already in unicode str.encode('desired_encoding') For sqlite I didn't actually want to encode it again, I wanted to decode it and leave it in unicode format. Here are four things you might need to be aware of as you try to work with unicode and encodings in python. The encoding of the string you want to work with, and the encoding you want to get it to. The system encoding. The console encoding. The encoding of the source file Elaboration: (1) When you read a string from a source, it must have some encoding, like latin_1 or utf_8. In my case, I'm getting strings from filenames, so unfortunately, I could be getting any kind of encoding. Windows XP uses UCS-2 (a Unicode system) as its native string type, which seems like cheating to me. Fortunately for me, the characters in most filenames are not going to be made up of more than one source encoding type, and I think all of mine were either completely latin_1, completely utf_8, or just plain ascii (which is a subset of both of those). So I just read them and decoded them as if they were still in latin_1 or utf_8. It's possible, though, that you could have latin_1 and utf_8 and whatever other characters mixed together in a filename on Windows. Sometimes those characters can show up as boxes, other times they just look mangled, and other times they look correct (accented characters and whatnot). Moving on. (2) Python has a default system encoding that gets set when python starts and can't be changed during runtime. See here for details. Dirty summary ... well here's the file I added: \# sitecustomize.py \# this file can be anywhere in your Python path, \# but it usually goes in ${pythondir}/lib/site-packages/ import sys sys.setdefaultencoding('utf_8') This system encoding is the one that gets used when you use the unicode("str") function without any other encoding parameters. To say that another way, python tries to decode "str" to unicode based on the default system encoding. (3) If you're using IDLE or the command-line python, I think that your console will display according to the default system encoding. I am using pydev with eclipse for some reason, so I had to go into my project settings, edit the launch configuration properties of my test script, go to the Common tab, and change the console from latin-1 to utf-8 so that I could visually confirm what I was doing was working. (4) If you want to have some test strings, eg test_str = "ó" in your source code, then you will have to tell python what kind of encoding you are using in that file. (FYI: when I mistyped an encoding I had to ctrl-Z because my file became unreadable.) This is easily accomplished by putting a line like so at the top of your source code file: # -*- coding: utf_8 -*- If you don't have this information, python attempts to parse your code as ascii by default, and so: SyntaxError: Non-ASCII character '\xf3' in file _redacted_ on line 81, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details Once your program is working correctly, or, if you aren't using python's console or any other console to look at output, then you will probably really only care about #1 on the list. System default and console encoding are not that important unless you need to look at output and/or you are using the builtin unicode() function (without any encoding parameters) instead of the string.decode() function. I wrote a demo function I will paste into the bottom of this gigantic mess that I hope correctly demonstrates the items in my list. Here is some of the output when I run the character 'ó' through the demo function, showing how various methods react to the character as input. My system encoding and console output are both set to utf_8 for this run: '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Now I will change the system and console encoding to latin_1, and I get this output for the same input: 'ó' = original char <type 'str'> repr(char)='\xf3' 'ó' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Notice that the 'original' character displays correctly and the builtin unicode() function works now. Now I change my console output back to utf_8. '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' '?' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Here everything still works the same as last time but the console can't display the output correctly. Etc. The function below also displays more information that this and hopefully would help someone figure out where the gap in their understanding is. I know all this information is in other places and more thoroughly dealt with there, but I hope that this would be a good kickoff point for someone trying to get coding with python and/or sqlite. Ideas are great but sometimes source code can save you a day or two of trying to figure out what functions do what. Disclaimers: I'm no encoding expert, I put this together to help my own understanding. I kept building on it when I should have probably started passing functions as arguments to avoid so much redundant code, so if I can I'll make it more concise. Also, utf_8 and latin_1 are by no means the only encoding schemes, they are just the two I was playing around with because I think they handle everything I need. Add your own encoding schemes to the demo function and test your own input. One more thing: there are apparently crazy application developers making life difficult in Windows. #!/usr/bin/env python # -*- coding: utf_8 -*- import os import sys def encodingDemo(str): validStrings = () try: print "str =",str,"{0} repr(str) = {1}".format(type(str), repr(str)) validStrings += ((str,""),) except UnicodeEncodeError as ude: print "Couldn't print the str itself because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print ude try: x = unicode(str) print "unicode(str) = ",x validStrings+= ((x, " decoded into unicode by the default system encoding"),) except UnicodeDecodeError as ude: print "ERROR. unicode(str) couldn't decode the string because the system encoding is set to an encoding that doesn't understand some character in the string." print "\tThe system encoding is set to {0}. See error:\n\t".format(sys.getdefaultencoding()), print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the unicode(str) because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('latin_1') print "str.decode('latin_1') =",x validStrings+= ((x, " decoded with latin_1 into unicode"),) try: print "str.decode('latin_1').encode('utf_8') =",str.decode('latin_1').encode('utf_8') validStrings+= ((x, " decoded with latin_1 into unicode and encoded into utf_8"),) except UnicodeDecodeError as ude: print "The string was decoded into unicode using the latin_1 encoding, but couldn't be encoded into utf_8. See error:\n\t", print ude except UnicodeDecodeError as ude: print "Something didn't work, probably because the string wasn't latin_1 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('latin_1') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('utf_8') print "str.decode('utf_8') =",x validStrings+= ((x, " decoded with utf_8 into unicode"),) try: print "str.decode('utf_8').encode('latin_1') =",str.decode('utf_8').encode('latin_1') except UnicodeDecodeError as ude: print "str.decode('utf_8').encode('latin_1') didn't work. The string was decoded into unicode using the utf_8 encoding, but couldn't be encoded into latin_1. See error:\n\t", validStrings+= ((x, " decoded with utf_8 into unicode and encoded into latin_1"),) print ude except UnicodeDecodeError as ude: print "str.decode('utf_8') didn't work, probably because the string wasn't utf_8 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('utf_8') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t",uee print print "Printing information about each character in the original string." for char in str: try: print "\t'" + char + "' = original char {0} repr(char)={1}".format(type(char), repr(char)) except UnicodeDecodeError as ude: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), ude) except UnicodeEncodeError as uee: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), uee) print uee try: x = unicode(char) print "\t'" + x + "' = unicode(char) {1} repr(unicode(char))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = unicode(char) ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = unicode(char) {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('latin_1') print "\t'" + x + "' = char.decode('latin_1') {1} repr(char.decode('latin_1'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('latin_1') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('latin_1') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('utf_8') print "\t'" + x + "' = char.decode('utf_8') {1} repr(char.decode('utf_8'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('utf_8') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('utf_8') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) print x = 'ó' encodingDemo(x) Much thanks for the answers below and especially to @John Machin for answering so thoroughly.

Read the article

SQLite, python, unicode, and non-utf data

- by Nathan Spears

I started by trying to store strings in sqlite using python, and got the message: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. Ok, I switched to Unicode strings. Then I started getting the message: sqlite3.OperationalError: Could not decode to UTF-8 column 'tag_artist' with text 'Sigur Rós' when trying to retrieve data from the db. More research and I started encoding it in utf8, but then 'Sigur Rós' starts looking like 'Sigur RÃ³s' note: My console was set to display in 'latin_1' as @John Machin pointed out. What gives? After reading this, describing exactly the same situation I'm in, it seems as if the advice is to ignore the other advice and use 8-bit bytestrings after all. I didn't know much about unicode and utf before I started this process. I've learned quite a bit in the last couple hours, but I'm still ignorant of whether there is a way to correctly convert 'ó' from latin-1 to utf-8 and not mangle it. If there isn't, why would sqlite 'highly recommend' I switch my application to unicode strings? I'm going to update this question with a summary and some example code of everything I've learned in the last 24 hours so that someone in my shoes can have an easy(er) guide. If the information I post is wrong or misleading in any way please tell me and I'll update, or one of you senior guys can update. Summary of answers Let me first state the goal as I understand it. The goal in processing various encodings, if you are trying to convert between them, is to understand what your source encoding is, then convert it to unicode using that source encoding, then convert it to your desired encoding. Unicode is a base and encodings are mappings of subsets of that base. utf_8 has room for every character in unicode, but because they aren't in the same place as, for instance, latin_1, a string encoded in utf_8 and sent to a latin_1 console will not look the way you expect. In python the process of getting to unicode and into another encoding looks like: str.decode('source_encoding').encode('desired_encoding') or if the str is already in unicode str.encode('desired_encoding') For sqlite I didn't actually want to encode it again, I wanted to decode it and leave it in unicode format. Here are four things you might need to be aware of as you try to work with unicode and encodings in python. The encoding of the string you want to work with, and the encoding you want to get it to. The system encoding. The console encoding. The encoding of the source file Elaboration: (1) When you read a string from a source, it must have some encoding, like latin_1 or utf_8. In my case, I'm getting strings from filenames, so unfortunately, I could be getting any kind of encoding. Windows XP uses UCS-2 (a Unicode system) as its native string type, which seems like cheating to me. Fortunately for me, the characters in most filenames are not going to be made up of more than one source encoding type, and I think all of mine were either completely latin_1, completely utf_8, or just plain ascii (which is a subset of both of those). So I just read them and decoded them as if they were still in latin_1 or utf_8. It's possible, though, that you could have latin_1 and utf_8 and whatever other characters mixed together in a filename on Windows. Sometimes those characters can show up as boxes, other times they just look mangled, and other times they look correct (accented characters and whatnot). Moving on. (2) Python has a default system encoding that gets set when python starts and can't be changed during runtime. See here for details. Dirty summary ... well here's the file I added: \# sitecustomize.py \# this file can be anywhere in your Python path, \# but it usually goes in ${pythondir}/lib/site-packages/ import sys sys.setdefaultencoding('utf_8') This system encoding is the one that gets used when you use the unicode("str") function without any other encoding parameters. To say that another way, python tries to decode "str" to unicode based on the default system encoding. (3) If you're using IDLE or the command-line python, I think that your console will display according to the default system encoding. I am using pydev with eclipse for some reason, so I had to go into my project settings, edit the launch configuration properties of my test script, go to the Common tab, and change the console from latin-1 to utf-8 so that I could visually confirm what I was doing was working. (4) If you want to have some test strings, eg test_str = "ó" in your source code, then you will have to tell python what kind of encoding you are using in that file. (FYI: when I mistyped an encoding I had to ctrl-Z because my file became unreadable.) This is easily accomplished by putting a line like so at the top of your source code file: # -*- coding: utf_8 -*- If you don't have this information, python attempts to parse your code as ascii by default, and so: SyntaxError: Non-ASCII character '\xf3' in file _redacted_ on line 81, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details Once your program is working correctly, or, if you aren't using python's console or any other console to look at output, then you will probably really only care about #1 on the list. System default and console encoding are not that important unless you need to look at output and/or you are using the builtin unicode() function (without any encoding parameters) instead of the string.decode() function. I wrote a demo function I will paste into the bottom of this gigantic mess that I hope correctly demonstrates the items in my list. Here is some of the output when I run the character 'ó' through the demo function, showing how various methods react to the character as input. My system encoding and console output are both set to utf_8 for this run: '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Now I will change the system and console encoding to latin_1, and I get this output for the same input: 'ó' = original char <type 'str'> repr(char)='\xf3' 'ó' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Notice that the 'original' character displays correctly and the builtin unicode() function works now. Now I change my console output back to utf_8. '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' '?' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Here everything still works the same as last time but the console can't display the output correctly. Etc. The function below also displays more information that this and hopefully would help someone figure out where the gap in their understanding is. I know all this information is in other places and more thoroughly dealt with there, but I hope that this would be a good kickoff point for someone trying to get coding with python and/or sqlite. Ideas are great but sometimes source code can save you a day or two of trying to figure out what functions do what. Disclaimers: I'm no encoding expert, I put this together to help my own understanding. I kept building on it when I should have probably started passing functions as arguments to avoid so much redundant code, so if I can I'll make it more concise. Also, utf_8 and latin_1 are by no means the only encoding schemes, they are just the two I was playing around with because I think they handle everything I need. Add your own encoding schemes to the demo function and test your own input. One more thing: there are apparently crazy application developers making life difficult in Windows. #!/usr/bin/env python # -*- coding: utf_8 -*- import os import sys def encodingDemo(str): validStrings = () try: print "str =",str,"{0} repr(str) = {1}".format(type(str), repr(str)) validStrings += ((str,""),) except UnicodeEncodeError as ude: print "Couldn't print the str itself because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print ude try: x = unicode(str) print "unicode(str) = ",x validStrings+= ((x, " decoded into unicode by the default system encoding"),) except UnicodeDecodeError as ude: print "ERROR. unicode(str) couldn't decode the string because the system encoding is set to an encoding that doesn't understand some character in the string." print "\tThe system encoding is set to {0}. See error:\n\t".format(sys.getdefaultencoding()), print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the unicode(str) because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('latin_1') print "str.decode('latin_1') =",x validStrings+= ((x, " decoded with latin_1 into unicode"),) try: print "str.decode('latin_1').encode('utf_8') =",str.decode('latin_1').encode('utf_8') validStrings+= ((x, " decoded with latin_1 into unicode and encoded into utf_8"),) except UnicodeDecodeError as ude: print "The string was decoded into unicode using the latin_1 encoding, but couldn't be encoded into utf_8. See error:\n\t", print ude except UnicodeDecodeError as ude: print "Something didn't work, probably because the string wasn't latin_1 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('latin_1') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('utf_8') print "str.decode('utf_8') =",x validStrings+= ((x, " decoded with utf_8 into unicode"),) try: print "str.decode('utf_8').encode('latin_1') =",str.decode('utf_8').encode('latin_1') except UnicodeDecodeError as ude: print "str.decode('utf_8').encode('latin_1') didn't work. The string was decoded into unicode using the utf_8 encoding, but couldn't be encoded into latin_1. See error:\n\t", validStrings+= ((x, " decoded with utf_8 into unicode and encoded into latin_1"),) print ude except UnicodeDecodeError as ude: print "str.decode('utf_8') didn't work, probably because the string wasn't utf_8 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('utf_8') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t",uee print print "Printing information about each character in the original string." for char in str: try: print "\t'" + char + "' = original char {0} repr(char)={1}".format(type(char), repr(char)) except UnicodeDecodeError as ude: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), ude) except UnicodeEncodeError as uee: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), uee) print uee try: x = unicode(char) print "\t'" + x + "' = unicode(char) {1} repr(unicode(char))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = unicode(char) ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = unicode(char) {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('latin_1') print "\t'" + x + "' = char.decode('latin_1') {1} repr(char.decode('latin_1'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('latin_1') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('latin_1') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('utf_8') print "\t'" + x + "' = char.decode('utf_8') {1} repr(char.decode('utf_8'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('utf_8') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('utf_8') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) print x = 'ó' encodingDemo(x) Much thanks for the answers below and especially to @John Machin for answering so thoroughly.

Read the article

IIRF reverse proxy problem

- by Sergei

Hi everyone, We have a java application ( Atlassian Bamboo) running on port 8085 on Windows 2003. It is accessile as http: //bamboo:8085. I am trying to setup reverse proxy for IIS6 using IIRF so content is accessible via http: //bamboo. It seems that I set it ip correctly, and I can retrieve Status page. This is how my IIRF.ini looks like: RewriteLog c:\temp\iirf RewriteLogLevel 2 StatusUrl /iirfStatus RewriteCond %{HTTP_HOST} ^bambooi$ [I] #This setup works #ProxyPass ^/(.*)$ http://othersite/$1 #This does not ProxyPass ^/(.*)$ http://bamboo:8085/$1 However when I type in http: //bamboo in IE, I get 'page cannot be displayed ' message. FF does not return anything at all. I made Wireshark network dump, selected 'follow TCPstream' and it seems like correct page is being retrieved.Why cannot I see it then? I also noticed that I can retrieve http: //bamboo/favicon.ico so I must be very close to the solution.. This is the Wireshark output: GET / HTTP/1.1 Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-ms-application, application/x-ms-xbap, application/vnd.ms-xpsdocument, application/xaml+xml, */* Accept-Language: en-gb User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) Accept-Encoding: gzip, deflate Host: bamboo Connection: Keep-Alive Cookie: JSESSIONID=wpsse0zyo4g5 HTTP/1.1 200 200 OK Date: Sat, 30 Jan 2010 09:19:46 GMT Server: Microsoft-IIS/6.0 Via: 1.1 DESTINATION_IP (IIRF 2.0) Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Dashboard</title> <meta http-equiv="content-type" content="text/html; charset=UTF-8" /> <meta name="robots" content="all" /> <meta name="MSSmartTagsPreventParsing" content="true" /> <meta http-equiv="Pragma" content="no-cache" /> <meta http-equiv="Expires" content="-1" /> <link type="text/css" rel="stylesheet" href="/s/1206/1/_/scripts/yui-2.6.0/build/grids/grids.css" />  <link rel="stylesheet" href="/s/1206/1/_/styles/main.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/main2.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/global-static.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/widePlanList.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/forms.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/yui-support/yui-custom.css" type="text/css" /> <link rel="shortcut icon" href="/s/1206/1/_/images/icons/favicon.ico" type="image/x-icon"/> <link rel="icon" href="/s/1206/1/_/images/icons/favicon.png" type="image/png" /> <link rel="stylesheet" href="/s/1206/1/_/styles/bamboo-tabs.css" type="text/css" />  <link rel="stylesheet" type="text/css" href="/s/1206/1/_/scripts/yui-2.6.0/build/tabview/assets/tabview-core.css"> <link rel="stylesheet" type="text/css" href="/s/1206/1/_/scripts/yui-2.6.0/build/tabview/assets/skins/sam/tabview-skin.css"> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/yahoo/yahoo-min.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/event/event-min.js" ></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/dom/dom-min.js" ></script>   <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/container/container-min.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/connection/connection-min.js"></script> <link type="text/css" rel="stylesheet" href="/s/1206/1/_/scripts/yui-2.6.0/build/container/assets/container.css" />  <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/menu/menu-min.js"></script> <link type="text/css" rel="stylesheet" href="/s/1206/1/_/scripts/yui-2.6.0/build/menu/assets/menu.css" />   <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/yahoo-dom-event/yahoo-dom-event.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/element/element-beta-min.js"></script>  <link rel="stylesheet" href="/s/1206/1/_/styles/yui-support/tabview.css" type="text/css" /> <link rel="stylesheet" href="/s/1206/1/_/styles/yui-support/round_tabs.css" type="text/css" /> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/tabview/tabview-min.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-2.6.0/build/json/json-min.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/yui-ext/yui-ext-nogrid.js"></script> <script type="text/javascript" src="/s/1206/1/_/scripts/bamboo.js"></script> <script type="text/javascript"> YAHOO.namespace('bamboo'); YAHOO.bamboo.tooltips = new Object(); YAHOO.bamboo.contextPath = ''; YAHOO.ext.UpdateManager.defaults.loadScripts = true; YAHOO.ext.UpdateManager.defaults.indicatorText = '<div class="loading-indicator">Currently loading...</div>'; YAHOO.ext.UpdateManager.defaults.timeout = 60; addUniversalOnload(addConfirmationToLinks); </script> <link rel="alternate" type="application/rss+xml" title="Bamboo RSS feed" href="/rss/createAllBuildsRssFeed.action?feedType=rssAll" /> </head> <body> <ul id="top"> <li id="skipNav"> <a href="#menu">Skip to navigation</a> </li> <li> <a href="#content">Skip to content</a> </li> </ul> <div id="nonFooter"> <div id="hd"> <div id="header"> <div id="logo"> <a href="/start.action"><img src="/images/bamboo_header_logo.gif" alt="Atlassian Bamboo" height="36" width="118" /></a> </div> <ul id="userOptions"> <li id="loginLink"> <a id="login" href="/userlogin!default.action?os_destination=%2Fstart.action">Log in</a> </li> <li id="signupLink"> <a id="signup" href="/signupUser!default.action">Signup</a> </li> <li id="helpLink"> <a id="help" href="http://confluence.atlassian.com/display/BAMBOO">Help</a> </li> </ul> </div>  <div id="menu"> <ul> <li><a id="home" href="/start.action" title="Atlassian Bamboo" accesskey="H"> <u>H</u>ome</a></li> <li><a id="authors" href="/authors/gotoAuthorReport.action" accesskey="U">A<u>u</u>thors</a></li> <li><a id="reports" href="/reports/viewReport.action" accesskey="R"> <u>R</u>eports</a></li> </ul> </div>  </div>  <div id="bd"> <div id="content"> <h1>Header here</h1> <div class="topMarginned"> <div id='buildSummaryTabs' class='dashboardTab'> </div> <script type="text/javascript"> function initUI(){ var jtabs = new YAHOO.ext.TabPanel('buildSummaryTabs'); YAHOO.bamboo.tabPanel = jtabs; // Use setUrl for Ajax loading var tab3 = jtabs.addTab('allTab', "All Plans"); tab3.setUrl('/ajax/displayAllBuildSummaries.action', null, true); var tab4 = jtabs.addTab("currentTab", "Current Activity"); tab4.setUrl('/ajax/displayCurrentActivity.action', null, true); var handleTabChange = function(e, activePanel) { saveCookie('atlassian.bamboo.dashboard.tab.selected', activePanel.id, 365); }; jtabs.on('tabchange', handleTabChange); var selectedCookie = getCookieValue('atlassian.bamboo.dashboard.tab.selected'); if (jtabs.getTab(selectedCookie)) { jtabs.activate(selectedCookie); } else { jtabs.activate('allTab'); } } YAHOO.util.Event.onContentReady('buildSummaryTabs', initUI); </script> </div> <script type="text/javascript"> setTimeout( "window.location.reload()", 1800*1000 ); </script> <div class="clearer" ></div> </div>  </div>  </div>  <div id="ft"> <div id="footer"> <p> Powered by <a href="http://www.atlassian.com/software/bamboo/">Atlassian Bamboo</a> version 2.2.1 build 1206 - <span title="15:59:44 17 Mar 2009">17 Mar 09</span> </p> <ul> <li class="first"> <a href="https://support.atlassian.com/secure/CreateIssue.jspa?pid=10060&issuetype=1">Report a problem</a> </li> <li> <a href="http://jira.atlassian.com/secure/CreateIssue.jspa?pid=11011&issuetype=4">Request a feature</a> </li> <li> <a href="http://forums.atlassian.com/forum.jspa?forumID=103">Contact Atlassian</a> </li> <li> <a href="/viewAdministrators.action">Contact Administrators</a> </li> </ul> </div>  </div>

Read the article

Fitting an Image to Screen on Rotation iPhone / iPad ?

- by user356937

I have been playing around with one of the iPhone examples from Apple' web site (ScrollViewSuite) . I am trying to tweak it a bit so that when I rotate the the iPad the image will fit into the screen in landscape mode vertical. I have been successful in getting the image to rotate, but the image is larger than the height of the landscape screen, so the bottom is below the screen. I would like to image to scale to the height of the landscape screen. I have been playing around with various autoSizingMask attributes without success. The imageView is called "zoomView" this is the actual image which loads into a scrollView called imageScrollView. I am trying to achieve the screen to rotate and look like this.... olsonvox.com/photos/correct.png However, this is what My screen is looking like. olsonvox.com/photos/incorrect.png I would really appreciate some advice or guidance. Below is the RootViewController.m for the project. Blade # import "RootViewController.h" #define ZOOM_VIEW_TAG 100 #define ZOOM_STEP 1.5 #define THUMB_HEIGHT 150 #define THUMB_V_PADDING 25 #define THUMB_H_PADDING 25 #define CREDIT_LABEL_HEIGHT 25 #define AUTOSCROLL_THRESHOLD 30 @interface RootViewController (ViewHandlingMethods) - (void)toggleThumbView; - (void)pickImageNamed:(NSString *)name; - (NSArray *)imageNames; - (void)createThumbScrollViewIfNecessary; - (void)createSlideUpViewIfNecessary; @end @interface RootViewController (AutoscrollingMethods) - (void)maybeAutoscrollForThumb:(ThumbImageView *)thumb; - (void)autoscrollTimerFired:(NSTimer *)timer; - (void)legalizeAutoscrollDistance; - (float)autoscrollDistanceForProximityToEdge:(float)proximity; @end @interface RootViewController (UtilityMethods) - (CGRect)zoomRectForScale:(float)scale withCenter:(CGPoint)center; @end @implementation RootViewController - (void)loadView { [super loadView]; imageScrollView = [[UIScrollView alloc] initWithFrame:[[self view]bounds]]; // this code makes the image resize to the width and height properly. imageScrollView.autoresizingMask = UIViewAutoresizingFlexibleHeight | UIViewAutoresizingFlexibleLeftMargin | UIViewAutoresizingFlexibleRightMargin| UIViewAutoresizingFlexibleBottomMargin| UIViewAutoresizingFlexibleBottomMargin; // TRY SETTNG CENTER HERE SOMEHOW>.... [imageScrollView setBackgroundColor:[UIColor blackColor]]; [imageScrollView setDelegate:self]; [imageScrollView setBouncesZoom:YES]; [[self view] addSubview:imageScrollView]; [self toggleThumbView]; // intitializes with the first image. [self pickImageNamed:@"lookbook1"]; } - (void)dealloc { [imageScrollView release]; [slideUpView release]; [thumbScrollView release]; [super dealloc]; } #pragma mark UIScrollViewDelegate methods - (UIView *)viewForZoomingInScrollView:(UIScrollView *)scrollView { UIView *view = nil; if (scrollView == imageScrollView) { view = [imageScrollView viewWithTag:ZOOM_VIEW_TAG]; } return view; } /************************************** NOTE **************************************/ /* The following delegate method works around a known bug in zoomToRect:animated: */ /* In the next release after 3.0 this workaround will no longer be necessary */ /**********************************************************************************/ - (void)scrollViewDidEndZooming:(UIScrollView *)scrollView withView:(UIView *)view atScale:(float)scale { [scrollView setZoomScale:scale+0.01 animated:NO]; [scrollView setZoomScale:scale animated:NO]; } #pragma mark TapDetectingImageViewDelegate methods - (void)tapDetectingImageView:(TapDetectingImageView *)view gotSingleTapAtPoint:(CGPoint)tapPoint { // Single tap shows or hides drawer of thumbnails. [self toggleThumbView]; } - (void)tapDetectingImageView:(TapDetectingImageView *)view gotDoubleTapAtPoint:(CGPoint)tapPoint { // double tap zooms in float newScale = [imageScrollView zoomScale] * ZOOM_STEP; CGRect zoomRect = [self zoomRectForScale:newScale withCenter:tapPoint]; [imageScrollView zoomToRect:zoomRect animated:YES]; } - (void)tapDetectingImageView:(TapDetectingImageView *)view gotTwoFingerTapAtPoint:(CGPoint)tapPoint { // two-finger tap zooms out float newScale = [imageScrollView zoomScale] / ZOOM_STEP; CGRect zoomRect = [self zoomRectForScale:newScale withCenter:tapPoint]; [imageScrollView zoomToRect:zoomRect animated:YES]; } #pragma mark ThumbImageViewDelegate methods - (void)thumbImageViewWasTapped:(ThumbImageView *)tiv { [self pickImageNamed:[tiv imageName]]; [self toggleThumbView]; } - (void)thumbImageViewStartedTracking:(ThumbImageView *)tiv { [thumbScrollView bringSubviewToFront:tiv]; } // CONTROLS DRAGGING AND DROPPING THUMBNAILS... - (void)thumbImageViewMoved:(ThumbImageView *)draggingThumb { // check if we've moved close enough to an edge to autoscroll, or far enough away to stop autoscrolling [self maybeAutoscrollForThumb:draggingThumb]; /* The rest of this method handles the reordering of thumbnails in the thumbScrollView. See */ /* ThumbImageView.h and ThumbImageView.m for more information about how this works. */ // we'll reorder only if the thumb is overlapping the scroll view if (CGRectIntersectsRect([draggingThumb frame], [thumbScrollView bounds])) { BOOL draggingRight = [draggingThumb frame].origin.x > [draggingThumb home].origin.x ? YES : NO; /* we're going to shift over all the thumbs who live between the home of the moving thumb */ /* and the current touch location. A thumb counts as living in this area if the midpoint */ /* of its home is contained in the area. */ NSMutableArray *thumbsToShift = [[NSMutableArray alloc] init]; // get the touch location in the coordinate system of the scroll view CGPoint touchLocation = [draggingThumb convertPoint:[draggingThumb touchLocation] toView:thumbScrollView]; // calculate minimum and maximum boundaries of the affected area float minX = draggingRight ? CGRectGetMaxX([draggingThumb home]) : touchLocation.x; float maxX = draggingRight ? touchLocation.x : CGRectGetMinX([draggingThumb home]); // iterate through thumbnails and see which ones need to move over for (ThumbImageView *thumb in [thumbScrollView subviews]) { // skip the thumb being dragged if (thumb == draggingThumb) continue; // skip non-thumb subviews of the scroll view (such as the scroll indicators) if (! [thumb isMemberOfClass:[ThumbImageView class]]) continue; float thumbMidpoint = CGRectGetMidX([thumb home]); if (thumbMidpoint >= minX && thumbMidpoint <= maxX) { [thumbsToShift addObject:thumb]; } } // shift over the other thumbs to make room for the dragging thumb. (if we're dragging right, they shift to the left) float otherThumbShift = ([draggingThumb home].size.width + THUMB_H_PADDING) * (draggingRight ? -1 : 1); // as we shift over the other thumbs, we'll calculate how much the dragging thumb's home is going to move float draggingThumbShift = 0.0; // send each of the shifting thumbs to its new home for (ThumbImageView *otherThumb in thumbsToShift) { CGRect home = [otherThumb home]; home.origin.x += otherThumbShift; [otherThumb setHome:home]; [otherThumb goHome]; draggingThumbShift += ([otherThumb frame].size.width + THUMB_H_PADDING) * (draggingRight ? 1 : -1); } // change the home of the dragging thumb, but don't send it there because it's still being dragged CGRect home = [draggingThumb home]; home.origin.x += draggingThumbShift; [draggingThumb setHome:home]; } } - (void)thumbImageViewStoppedTracking:(ThumbImageView *)tiv { // if the user lets go of the thumb image view, stop autoscrolling [autoscrollTimer invalidate]; autoscrollTimer = nil; } #pragma mark Autoscrolling methods - (void)maybeAutoscrollForThumb:(ThumbImageView *)thumb { autoscrollDistance = 0; // only autoscroll if the thumb is overlapping the thumbScrollView if (CGRectIntersectsRect([thumb frame], [thumbScrollView bounds])) { CGPoint touchLocation = [thumb convertPoint:[thumb touchLocation] toView:thumbScrollView]; float distanceFromLeftEdge = touchLocation.x - CGRectGetMinX([thumbScrollView bounds]); float distanceFromRightEdge = CGRectGetMaxX([thumbScrollView bounds]) - touchLocation.x; if (distanceFromLeftEdge < AUTOSCROLL_THRESHOLD) { autoscrollDistance = [self autoscrollDistanceForProximityToEdge:distanceFromLeftEdge] * -1; // if scrolling left, distance is negative } else if (distanceFromRightEdge < AUTOSCROLL_THRESHOLD) { autoscrollDistance = [self autoscrollDistanceForProximityToEdge:distanceFromRightEdge]; } } // if no autoscrolling, stop and clear timer if (autoscrollDistance == 0) { [autoscrollTimer invalidate]; autoscrollTimer = nil; } // otherwise create and start timer (if we don't already have a timer going) else if (autoscrollTimer == nil) { autoscrollTimer = [NSTimer scheduledTimerWithTimeInterval:(1.0 / 60.0) target:self selector:@selector(autoscrollTimerFired:) userInfo:thumb repeats:YES]; } } - (float)autoscrollDistanceForProximityToEdge:(float)proximity { // the scroll distance grows as the proximity to the edge decreases, so that moving the thumb // further over results in faster scrolling. return ceilf((AUTOSCROLL_THRESHOLD - proximity) / 5.0); } - (void)legalizeAutoscrollDistance { // makes sure the autoscroll distance won't result in scrolling past the content of the scroll view float minimumLegalDistance = [thumbScrollView contentOffset].x * -1; float maximumLegalDistance = [thumbScrollView contentSize].width - ([thumbScrollView frame].size.width + [thumbScrollView contentOffset].x); autoscrollDistance = MAX(autoscrollDistance, minimumLegalDistance); autoscrollDistance = MIN(autoscrollDistance, maximumLegalDistance); } - (void)autoscrollTimerFired:(NSTimer*)timer { [self legalizeAutoscrollDistance]; // autoscroll by changing content offset CGPoint contentOffset = [thumbScrollView contentOffset]; contentOffset.x += autoscrollDistance; [thumbScrollView setContentOffset:contentOffset]; // adjust thumb position so it appears to stay still ThumbImageView *thumb = (ThumbImageView *)[timer userInfo]; [thumb moveByOffset:CGPointMake(autoscrollDistance, 0)]; } #pragma mark View handling methods - (void)toggleThumbView { [self createSlideUpViewIfNecessary]; // no-op if slideUpView has already been created CGRect frame = [slideUpView frame]; if (thumbViewShowing) { frame.origin.y = 0; } else { frame.origin.y = -225; } [UIView beginAnimations:nil context:nil]; [UIView setAnimationDuration:0.3]; [slideUpView setFrame:frame]; [UIView commitAnimations]; thumbViewShowing = !thumbViewShowing; } - (void)pickImageNamed:(NSString *)name { // first remove previous image view, if any [[imageScrollView viewWithTag:ZOOM_VIEW_TAG] removeFromSuperview]; UIImage *image = [UIImage imageNamed:[NSString stringWithFormat:@"%@.jpg", name]]; TapDetectingImageView *zoomView = [[TapDetectingImageView alloc] initWithImage:image]; zoomView.autoresizingMask = UIViewAutoresizingFlexibleWidth ; [zoomView setDelegate:self]; [zoomView setTag:ZOOM_VIEW_TAG]; [imageScrollView addSubview:zoomView]; [imageScrollView setContentSize:[zoomView frame].size]; [zoomView release]; // choose minimum scale so image width fits screen float minScale = [imageScrollView frame].size.width / [zoomView frame].size.width; [imageScrollView setMinimumZoomScale:minScale]; [imageScrollView setZoomScale:minScale]; [imageScrollView setContentOffset:CGPointZero]; } - (NSArray *)imageNames { // the filenames are stored in a plist in the app bundle, so create array by reading this plist NSString *path = [[NSBundle mainBundle] pathForResource:@"Images" ofType:@"plist"]; NSData *plistData = [NSData dataWithContentsOfFile:path]; NSString *error; NSPropertyListFormat format; NSArray *imageNames = [NSPropertyListSerialization propertyListFromData:plistData mutabilityOption:NSPropertyListImmutable format:&format errorDescription:&error]; if (!imageNames) { NSLog(@"Failed to read image names. Error: %@", error); [error release]; } return imageNames; } - (void)createSlideUpViewIfNecessary { if (!slideUpView) { [self createThumbScrollViewIfNecessary]; CGRect bounds = [[self view] bounds]; float thumbHeight = [thumbScrollView frame].size.height; float labelHeight = CREDIT_LABEL_HEIGHT; // create label giving credit for images UILabel *creditLabel = [[UILabel alloc] initWithFrame:CGRectMake(0, thumbHeight, bounds.size.width, labelHeight)]; [creditLabel setBackgroundColor:[UIColor clearColor]]; [creditLabel setTextColor:[UIColor whiteColor]]; // [creditLabel setFont:[UIFont fontWithName:@"Helvetica" size:16]]; // [creditLabel setText:@"SAMPLE TEXT"]; [creditLabel setTextAlignment:UITextAlignmentCenter]; // create container view that will hold scroll view and label CGRect frame = CGRectMake(0.0, -225.00, bounds.size.width+256, thumbHeight + labelHeight); slideUpView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleTopMargin; slideUpView = [[UIView alloc] initWithFrame:frame]; [slideUpView setBackgroundColor:[UIColor blackColor]]; [slideUpView setOpaque:NO]; [slideUpView setAlpha:.75]; [[self view] addSubview:slideUpView]; // add subviews to container view [slideUpView addSubview:thumbScrollView]; [slideUpView addSubview:creditLabel]; [creditLabel release]; } } - (void)createThumbScrollViewIfNecessary { if (!thumbScrollView) { float scrollViewHeight = THUMB_HEIGHT + THUMB_V_PADDING; float scrollViewWidth = [[self view] bounds].size.width; thumbScrollView = [[UIScrollView alloc] initWithFrame:CGRectMake(0, 0, scrollViewWidth, scrollViewHeight)]; [thumbScrollView setCanCancelContentTouches:NO]; [thumbScrollView setClipsToBounds:NO]; // now place all the thumb views as subviews of the scroll view // and in the course of doing so calculate the content width float xPosition = THUMB_H_PADDING; for (NSString *name in [self imageNames]) { UIImage *thumbImage = [UIImage imageNamed:[NSString stringWithFormat:@"%@_thumb.jpg", name]]; if (thumbImage) { ThumbImageView *thumbView = [[ThumbImageView alloc] initWithImage:thumbImage]; [thumbView setDelegate:self]; [thumbView setImageName:name]; CGRect frame = [thumbView frame]; frame.origin.y = THUMB_V_PADDING; frame.origin.x = xPosition; [thumbView setFrame:frame]; [thumbView setHome:frame]; [thumbScrollView addSubview:thumbView]; [thumbView release]; xPosition += (frame.size.width + THUMB_H_PADDING); } } [thumbScrollView setContentSize:CGSizeMake(xPosition, scrollViewHeight)]; } } #pragma mark Utility methods - (CGRect)zoomRectForScale:(float)scale withCenter:(CGPoint)center { CGRect zoomRect; // the zoom rect is in the content view's coordinates. // At a zoom scale of 1.0, it would be the size of the imageScrollView's bounds. // As the zoom scale decreases, so more content is visible, the size of the rect grows. zoomRect.size.height = [imageScrollView frame].size.height / scale; zoomRect.size.width = [imageScrollView frame].size.width / scale; // choose an origin so as to get the right center. zoomRect.origin.x = center.x - (zoomRect.size.width / 2.0); zoomRect.origin.y = center.y - (zoomRect.size.height / 2.0); return zoomRect; } #pragma mark - #pragma mark Rotation support // Ensure that the view controller supports rotation and that the split view can therefore show in both portrait and landscape. - (BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)interfaceOrientation { return YES; } @end

Read the article

Python Glade could not create GladeXML Object

- by Peter

Hey, I've created a simple window GUI in Glade 3.6.7 and I am trying to import it into Python. Every time I try to do so I get the following error: (queryrelevanceevaluation.py:8804): libglade-WARNING **: Expected <glade-interface>. Got <interface>. (queryrelevanceevaluation.py:8804): libglade-WARNING **: did not finish in PARSER_FINISH state Traceback (most recent call last): File "queryrelevanceevaluation.py", line 17, in <module> app = QueryRelevanceEvaluationApp() File "queryrelevanceevaluation.py", line 10, in __init__ self.widgets = gtk.glade.XML(gladefile) RuntimeError: could not create GladeXML object My Python Code: #!/usr/bin/env python import gtk import gtk.glade class QueryRelevanceEvaluationApp: def __init__(self): gladefile = "foo.glade" self.widgets = gtk.glade.XML(gladefile) dic = {"on_buttonGenerate_clicked" : self.on_buttonGenerate_clicked} self.widgets.signal_autoconnect(dic) def on_buttonGenerate_clicked(self, widget): print "You clicked the button" app = QueryRelevanceEvaluationApp() gtk.main() And the foo.glade file: <?xml version="1.0"?> <interface> <requires lib="gtk+" version="2.16"/>  <object class="GtkWindow" id="windowRelevanceEvaluation"> <property name="visible">True</property> <property name="title" translatable="yes">Query Result Relevance Evaluation</property> <child> <object class="GtkVBox" id="vbox1"> <property name="visible">True</property> <property name="orientation">vertical</property> <child> <object class="GtkHBox" id="hbox2"> <property name="visible">True</property> <child> <object class="GtkLabel" id="labelQuery"> <property name="visible">True</property> <property name="label" translatable="yes">Query:</property> </object> <packing> <property name="expand">False</property> <property name="padding">4</property> <property name="position">0</property> </packing> </child> <child> <object class="GtkEntry" id="entry1"> <property name="visible">True</property> <property name="can_focus">True</property> <property name="invisible_char">●</property> </object> <packing> <property name="padding">4</property> <property name="position">1</property> </packing> </child> </object> <packing> <property name="position">0</property> </packing> </child> <child> <object class="GtkFrame" id="frameSource"> <property name="visible">True</property> <property name="label_xalign">0</property> <child> <object class="GtkAlignment" id="alignment1"> <property name="visible">True</property> <property name="left_padding">12</property> <child> <object class="GtkHButtonBox" id="hbuttonbox1"> <property name="visible">True</property> <child> <object class="GtkRadioButton" id="radiobuttonGoogle"> <property name="label" translatable="yes">Google</property> <property name="visible">True</property> <property name="can_focus">True</property> <property name="receives_default">False</property> <property name="active">True</property> <property name="draw_indicator">True</property> </object> <packing> <property name="expand">False</property> <property name="fill">False</property> <property name="position">0</property> </packing> </child> <child> <object class="GtkRadioButton" id="radiobuttonBing"> <property name="label" translatable="yes">Bing</property> <property name="visible">True</property> <property name="can_focus">True</property> <property name="receives_default">False</property> <property name="active">True</property> <property name="draw_indicator">True</property> </object> <packing> <property name="expand">False</property> <property name="fill">False</property> <property name="position">1</property> </packing> </child> <child> <object class="GtkRadioButton" id="radiobuttonBoden"> <property name="label" translatable="yes">Boden</property> <property name="visible">True</property> <property name="can_focus">True</property> <property name="receives_default">False</property> <property name="active">True</property> <property name="draw_indicator">True</property> </object> <packing> <property name="expand">False</property> <property name="fill">False</property> <property name="position">2</property> </packing> </child> <child> <object class="GtkRadioButton" id="radiobuttonCSV"> <property name="label" translatable="yes">CSV</property> <property name="visible">True</property> <property name="can_focus">True</property> <property name="receives_default">False</property> <property name="active">True</property> <property name="draw_indicator">True</property> </object> <packing> <property name="expand">False</property> <property name="fill">False</property> <property name="position">3</property> </packing> </child> </object> </child> </object> </child> <child type="label"> <object class="GtkLabel" id="labelFrameSource"> <property name="visible">True</property> <property name="label" translatable="yes"><b>Source</b></property> <property name="use_markup">True</property> </object> </child> </object> <packing> <property name="position">1</property> </packing> </child> <child> <object class="GtkFrame" id="frame1"> <property name="visible">True</property> <property name="label_xalign">0</property> <child> <object class="GtkHBox" id="hbox3"> <property name="visible">True</property> <child> <object class="GtkLabel" id="labelResults"> <property name="visible">True</property> <property name="label" translatable="yes">Number Results:</property> </object> <packing> <property name="expand">False</property> <property name="position">0</property> </packing> </child> <child> <object class="GtkSpinButton" id="spinbuttonResults"> <property name="visible">True</property> <property name="can_focus">True</property> <property name="invisible_char">●</property> </object> <packing> <property name="padding">4</property> <property name="position">1</property> </packing> </child> </object> </child> <child type="label"> <object class="GtkLabel" id="labelFrameResults"> <property name="visible">True</property> <property name="label" translatable="yes"><b>Results</b></property> <property name="use_markup">True</property> </object> </child> </object> <packing> <property name="padding">2</property> <property name="position">2</property> </packing> </child> <child> <object class="GtkButton" id="buttonGenerateResults"> <property name="label" translatable="yes">Generate!</property> <property name="visible">True</property> <property name="can_focus">True</property> <property name="receives_default">True</property> </object> <packing> <property name="position">3</property> </packing> </child> </object> </child> </object> </interface> foo.glade and the above python script are in the same directory, and I have tried using a fully-qualified path but still get the same error (I am certain that the path is correct!). Any ideas? Cheers, Pete

Read the article

Reading input from a text file, omits the first and adds a nonsense value to the end?

- by Greenhouse Gases

Hi there When I input locations from a txt file I am getting a peculiar error where it seems to miss off the first entry, yet add a garbage entry to the end of the link list (it is designed to take the name, latitude and longitude for each location you will notice). I imagine this to be an issue with where it starts collecting the inputs and where it stops but I cant find the error!! It reads the first line correctly but then skips to the next before adding it because during testing for the bug it had no record of the first location Lisbon though whilst stepping into the method call it was reading it. Very bizarre but hopefully someone knows the issue. Here is firstly my header file: #include <string> struct locationNode { char nodeCityName [35]; double nodeLati; double nodeLongi; locationNode* Next; void CorrectCase() // Correct upper and lower case letters of input { int MAX_SIZE = 35; int firstLetVal = this->nodeCityName[0], letVal; int n = 1; // variable for name index from second letter onwards if((this->nodeCityName[0] >90) && (this->nodeCityName[0] < 123)) // First letter is lower case { firstLetVal = firstLetVal - 32; // Capitalise first letter this->nodeCityName[0] = firstLetVal; } while(n <= MAX_SIZE - 1) { if((this->nodeCityName[n] >= 65) && (this->nodeCityName[n] <= 90)) { letVal = this->nodeCityName[n] + 32; this->nodeCityName[n] = letVal; } n++; } //cityNameInput = this->nodeCityName; } }; class Locations { private: int size; public: Locations(){ }; // constructor for the class locationNode* Head; //int Add(locationNode* Item); }; And here is the file containing main: // U08221.cpp : main project file. #include "stdafx.h" #include "Locations.h" #include <iostream> #include <string> #include <fstream> using namespace std; int n = 0,x, locationCount = 0, MAX_SIZE = 35; string cityNameInput; char targetCity[35]; bool acceptedInput = false, userInputReq = true, match = false, nodeExists = false;// note: addLocation(), set to true to enable user input as opposed to txt file locationNode *start_ptr = NULL; // pointer to first entry in the list locationNode *temp, *temp2; // Part is a pointer to a new locationNode we can assign changing value followed by a call to Add locationNode *seek, *bridge; void setElementsNull(char cityParam[]) { int y=0, count =0; while(cityParam[y] != NULL) { y++; } while(y < MAX_SIZE) { cityParam[y] = NULL; y++; } } void addLocation() { temp = new locationNode; // declare the space for a pointer item and assign a temporary pointer to it if(!userInputReq) // bool that determines whether user input is required in adding the node to the list { cout << endl << "Enter the name of the location: "; cin >> temp->nodeCityName; temp->CorrectCase(); setElementsNull(temp->nodeCityName); cout << endl << "Please enter the latitude value for this location: "; cin >> temp->nodeLati; cout << endl << "Please enter the longitude value for this location: "; cin >> temp->nodeLongi; cout << endl; } temp->Next = NULL; //set to NULL as when one is added it is currently the last in the list and so can not point to the next if(start_ptr == NULL){ // if list is currently empty, start_ptr will point to this node start_ptr = temp; } else { temp2 = start_ptr; // We know this is not NULL - list not empty! while (temp2->Next != NULL) { temp2 = temp2->Next; // Move to next link in chain until reach end of list } temp2->Next = temp; } ++locationCount; // increment counter for number of records in list if(!userInputReq){ cout << "Location sucessfully added to the database! There are " << locationCount << " location(s) stored" << endl; } } void populateList(){ ifstream inputFile; inputFile.open ("locations.txt", ios::in); userInputReq = true; temp = new locationNode; // declare the space for a pointer item and assign a temporary pointer to it do { inputFile.get(temp->nodeCityName, 35, ' '); setElementsNull(temp->nodeCityName); inputFile >> temp->nodeLati; inputFile >> temp->nodeLongi; setElementsNull(temp->nodeCityName); if(temp->nodeCityName[0] == 10) //remove linefeed from input { for(int i = 0; temp->nodeCityName[i] != NULL; i++) { temp->nodeCityName[i] = temp->nodeCityName[i + 1]; } } addLocation(); } while(!inputFile.eof()); userInputReq = false; cout << "Successful!" << endl << "List contains: " << locationCount << " entries" << endl; cout << endl; inputFile.close(); } bool nodeExistTest(char targetCity[]) // see if entry is present in the database { match = false; seek = start_ptr; int letters = 0, letters2 = 0, x = 0, y = 0; while(targetCity[y] != NULL) { letters2++; y++; } while(x <= locationCount) // locationCount is number of entries currently in list { y=0, letters = 0; while(seek->nodeCityName[y] != NULL) // count letters in the current name { letters++; y++; } if(letters == letters2) // same amount of letters in the name { y = 0; while(y <= letters) // compare each letter against one another { if(targetCity[y] == seek->nodeCityName[y]) { match = true; y++; } else { match = false; y = letters + 1; // no match, terminate comparison } } } if(match) { x = locationCount + 1; //found match so terminate loop } else{ if(seek->Next != NULL) { bridge = seek; seek = seek->Next; x++; } else { x = locationCount + 1; // end of list so terminate loop } } } return match; } void deleteRecord() // complete this { int junction = 0; locationNode *place; cout << "Enter the name of the city you wish to remove" << endl; cin >> targetCity; setElementsNull(targetCity); if(nodeExistTest(targetCity)) //if this node does exist { if(seek == start_ptr) // if it is the first in the list { junction = 1; } if(seek != start_ptr && seek->Next == NULL) // if it is last in the list { junction = 2; } switch(junction) // will alter list accordingly dependant on where the searched for link is { case 1: start_ptr = start_ptr->Next; delete seek; --locationCount; break; case 2: place = seek; seek = bridge; delete place; --locationCount; break; default: bridge->Next = seek->Next; delete seek; --locationCount; break; } } else { cout << targetCity << "That entry does not currently exist" << endl << endl << endl; } } void searchDatabase() { char choice; cout << "Enter search term..." << endl; cin >> targetCity; if(nodeExistTest(targetCity)) { cout << "Entry: " << endl << endl; } else { cout << "Sorry, that city is not currently present in the list." << endl << "Would you like to add this city now Y/N?" << endl; cin >> choice; /*while(choice != ('Y' || 'N')) { cout << "Please enter a valid choice..." << endl; cin >> choice; }*/ switch(choice) { case 'Y': addLocation(); break; case 'N': break; default : cout << "Invalid choice" << endl; break; } } } void printDatabase() { temp = start_ptr; // set temp to the start of the list do { if (temp == NULL) { cout << "You have reached the end of the database" << endl; } else { // Display details for what temp points to at that stage cout << "Location : " << temp->nodeCityName << endl; cout << "Latitude : " << temp->nodeLati << endl; cout << "Longitude : " << temp->nodeLongi << endl; cout << endl; // Move on to next locationNode if one exists temp = temp->Next; } } while (temp != NULL); } void nameValidation(string name) { n = 0; // start from first letter x = name.size(); while(!acceptedInput) { if((name[n] >= 65) && (name[n] <= 122)) // is in the range of letters { while(n <= x - 1) { while((name[n] >=91) && (name[n] <=97)) // ERROR!! { cout << "Please enter a valid city name" << endl; cin >> name; } n++; } } else { cout << "Please enter a valid city name" << endl; cin >> name; } if(n <= x - 1) { acceptedInput = true; } } cityNameInput = name; } int main(array<System::String ^> ^args) { //main contains test calls to functions at present cout << "Populating list..."; populateList(); printDatabase(); deleteRecord(); printDatabase(); cin >> cityNameInput; } The text file contains this (ignore the names, they are just for testing!!): Lisbon 45 47 Fattah 45 47 Darius 42 49 Peter 45 27 Sarah 85 97 Michelle 45 47 John 25 67 Colin 35 87 Shiron 40 57 George 34 45 Sean 22 33 The output omits Lisbon, but adds on a garbage entry with nonsense values. Any ideas why? Thank you in advance.

Read the article

client problems - misaligned expectations & not following SDLC protocols

- by louism

hi guys, im having some serious problems with a client on a project - i could use some advice please the short version i have been working with this client now for almost 6 months without any problems (a classified website project in the range of 500 hours) over the last few days things have drastically deteriorated to the point where ive had to place the project on-hold whilst i work-out what to do (this has pissed the client off even more) to be simplistic, the root cause of the issue is this: the client doesnt read the specs i make for him, i code the feature, he than wants to change things, i tell him its not to the agreed spec and that that change will have to be postponed and possibly charged for, he gets upset and rants saying 'hes paid for the feature' and im not keeping to the agreement (<- misalignment of expectations) i think the root cause of the root cause is my clients failure to take my SDLC protocols seriously. i have a bug tracking system in place which he practically refuses to use (he still emails me bugs), he doesnt seem to care to much for the protocols i use for dealing with scope creep and change control the whole situation came to a head recently where he 'cracked it' (an aussie term for being fed-up). the more terms like 'postponed for post-launch implementation', 'costed feature addition', and 'not to agreed spec' i kept using, the worse it got finally, he began to bully me - basically insisting i shut-up and do the work im being paid for. i wrote a long-winded email explaining how wrong he was on all these different points, and explaining what all the SDLC protocols do to protect the success of the project. than i deleted that email and wrote a new one in the new email, i suggested as a solution i write up a list of grievances we both had. we than review the list and compromise on different points: he gets some things he wants, i get some things i want. sometimes youve got to give ground to get ground his response to this suggestion was flat-out refusal, and a restatement that i should just get on with the work ive been paid to do so there you have the very subjective short version. if you have the time and inclination, the long version may be a little less bias as it has the email communiques between me and my client the long version (with background) the long version works by me showing you the email communiques which lead to the situation coming to a head. so here it is, judge for yourself where the trouble started... 1. client asked me why something was missing from a feature i just uploaded, my response was to show him what was in the spec: it basically said the item he was looking for was never going to be included 2. [clients response...] Memo Louis, We are following your own title fields and keeping a consistent layout. Why the big fuss about not adding "Part". It simply replaces "model" and is consistent with your current title fields. 3. [my response...] hi [client], the 'part' field appeared to me as a redundancy / mistake. i requested clarification but never received any in a timely manner (about 2 weeks ago) the specification for this feature also indicated it wasnt going to be included: RE: "Why the big fuss about not adding "Part" " it may not appear so, but it would actually be a lot of work for me to now add a 'Part' field it could take me up to 15-20 minutes to properly explain why its such a big undertaking to do this, but i would prefer to use that time instead to work on completing your v1.1 features as a simplistic explanation - it connects to the change in paradigm from a 'generic classified ad' model to a 'specific attributes for specific categories' model basically, i am saying it is a big fuss, but i understand that it doesnt look that way - after all, it is just one ity-bitty field :) if you require a fuller explanation, please let me know and i will commit the time needed to write that out also, if you recall when we first started on the project, i said that with the effort/time required for features, you would likely not know off the top of your head. you may think something is really complex, but in reality its quite simple, you might think something is easy - but it could actually be a massive trauma to code (which is the case here with the 'Part' field). if you also recalled, i said the best course of action is to just ask, and i would let you know on a case-by-case basis 4. [email from me to client...] hi [client], the online catalogue page is now up live (see my email from a few days ago for information on how it works) note: the window of opportunity for input/revisions on what data the catalogue stores has now closed (as i have put the code up live now) RE: the UI/layout of the online catalogue page you may still do visual/ui tweaks to the page at the moment (this window for input/revisions will close in a couple of days time) 5. [email from client to me...] *(note: i had put up the feature & asked the client to review it, never heard back from them for a few days)* Memo Louis, Here you go again. CLOSED without a word of input from the customer. I don't think so. I will reply tomorrow regarding the content and functionality we require from this feature. 5. [from me to client...] hi [client]: RE: from my understanding, you are saying that the mini-sale yard control would change itself based on the fact someone was viewing for parts & accessories <- is that correct? this change is outside the scope of the v1.1 mini-spec and therefore will need to wait 'til post launch for costing/implementation 6. [email from client to me...] Memo Louis, Following your v1.1 mini-spec and all your time paid in full for the work selected. We need to make the situation clear. There will be no further items held for post-launch. Do not expect us to pay for any further items other than those we have agreed upon. You have undertaken to complete the Parts and accessories feature as follows. Obviously, as part of this process the "mini search" will be effected, and will require "adaption to make sense". 7. [email from me to client...] hi [client], RE: "There will be no further items held for post-launch. Do not expect us to pay for any further items other than those we have agreed upon." a few points to consider: 1) the specification for the 'parts & accessories' feature was as follows: (i.e. [what] "...we have agreed upon.") 2) you have received the 'parts & accessories' feature free of charge (you have paid $0 for it). ive spent two days coding that feature as a gesture of good will i would request that you please consider these two facts carefully and sincerely 8. [email from client to me...] Memo Louis, I don't see how you are giving us anything for free. From your original fee proposal you have deleted more than 30 hours of included features. Your title "shelved features". Further you have charged us twice by adding back into the site, at an addition cost, some of those "shelved features" features. See v1.1 mini-spec. Did include in your original fee proposal a change request budget but then charge without discussion items included in v1.1 mini-spec. Included a further Features test plan for a regression test, a fee of 10 hours that would not have been required if the "shelved features" were not left out of the agreed fee proposal. I have made every attempt to satisfy your your uneven business sense by offering you everything your heart desired, in the v1.1 mini-spec, to be left once again with your attitude of "its too hard, lets leave it for post launch". I am no longer accepting anything less than what we have contracted you to do. That is clearly defined in v1.1 mini-spec, and you are paid in advance for delivering those items as an acceptable function. a few notes about the above email... i had to cull features from the original spec because it didnt fit into the budget. i explained this to the client at the start of the project (he wanted more features than he had budget hours to do them all) nothing has been charged for twice, i didnt charge the client for culled features. im charging him to now do those culled features the draft version of the project schedule included a change request budget of 10 hours, but i had to remove that to meet the budget (the client may not have been aware of this to be fair to them) what the client refers to as my attitude of 'too hard/leave it for post-launch', i called a change request protocol and a method for keeping scope creep under control 9. [email from me to client...] hi [client], RE: "...all your grievances..." i had originally written out a long email response; it was fantastic, it had all these great points of how 'you were wrong' and 'i was right', you would of loved it (and by 'loved it', i mean it would of just infuriated you more) so, i decided to deleted it start over, for two reasons: 1) a long email is being disrespectful of your time (youre a busy businessman with things to do) 2) whos wrong or right gets us no closer to fixing the problems we are experiencing what i propose is this... i prepare a bullet point list of your grievances and my grievances (yes, im unhappy too about how things are going - and it has little to do with money) i submit this list to you for you to add to as necessary we then both take a good hard look at this list, and we decide which areas we are willing to give ground on as an example, the list may look something like this: "louis, you keep taking away features you said you would do" [your grievance 2] [your grievance 3] [your grievance ...] "[client], i feel you dont properly read the specs i prepare for you..." [my grievance 2] [my grievance 3] [my grievance ...] if you are willing to give this a try, let me know will it work? who knows. but if it doesnt, we can always go back to arguing some more :) obviously, this will only work if you are willing to give it a genuine try, and you can accept that you may have to 'give some ground to get some ground' what do you think? 10. [email from client to me ...] Memo Louis, Instead of wasting your time listing grievances, I would prefer you complete the items in v1.1 mini-spec, to a satisfactory conclusion. We almost had the website ready for launch until you brought the v1.1 mini-spec into the frame. Obviously I expected you could complete the v1.1 mini-spec in a two-week time frame as you indicated and give the site a more profession presentation. Most of the problems have been caused by you not following our instructions, but deciding to do what you feel like at the time. And then arguing with us how the missing information is not necessary. For instance "Parts and Accessories". Why on earth would you leave out the parts heading, when it ties-in with the fields you have already developed. It replaces "model" and is just as important in the context of information that appears in the "Details" panel. We are at a stage where the the v1.1 mini-spec needs to be completed without further time wasting and the site is complete (subject to all features working). We are on standby at this end to do just that. Let me know when you are back, working on the site and we will process and complete each v1.1 mini-spec, item by item, until the job is complete. 11. [last email from me to client...] hi [client], based on this reply, and your demonstrated unwillingness to compromise/give any ground on issues at hand, i have decided to place your project on-hold for the moment i will be considering further options on how to over-come our challenges over the next few days i will contact you by monday 17/may to discuss any new options i have come up with, and if i believe it is appropriate to restart work on your project at that point or not told you it was long... what do you think?

Read the article

Python script is exiting with no output and I have no idea why

- by Adam Tuttle

I'm attempting to debug a Subversion post-commit hook that calls some python scripts. What I've been able to determine so far is that when I run post-commit.bat manually (I've created a wrapper for it to make it easier) everything succeeds, but when SVN runs it one particular step doesn't work. We're using CollabNet SVNServe, which I know from the documentation removes all environment variables. This had caused some problems earlier, but shouldn't be an issue now. Before Subversion calls a hook script, it removes all variables - including $PATH on Unix, and %PATH% on Windows - from the environment. Therefore, your script can only run another program if you spell out that program's absolute name. The relevant portion of post-commit.bat is: echo -------------------------- >> c:\svn-repos\company\hooks\svn2ftp.out.log set SITENAME=staging set SVNPATH=branches/staging/wwwroot/ "C:\Python3\python.exe" C:\svn-repos\company\hooks\svn2ftp.py ^ --svnUser="svnusername" ^ --svnPass="svnpassword" ^ --ftp-user=ftpuser ^ --ftp-password=ftppassword ^ --ftp-remote-dir=/ ^ --access-url=svn://10.0.100.6/company ^ --status-file="C:\svn-repos\company\hooks\svn2ftp-%SITENAME%.dat" ^ --project-directory=%SVNPATH% "staging.company.com" %1 %2 >> c:\svn-repos\company\hooks\svn2ftp.out.log echo -------------------------- >> c:\svn-repos\company\hooks\svn2ftp.out.log When I run post-commit.bat manually, for example: post-commit c:\svn-repos\company 12345, I see output like the following in svn2ftp.out.log: -------------------------- args1: c:\svn-repos\company args0: staging.company.com abspath: c:\svn-repos\company project_dir: branches/staging/wwwroot/ local_repos_path: c:\svn-repos\company getting youngest revision... done, up-to-date -------------------------- However, when I commit something to the repo and it runs automatically, the output is: -------------------------- -------------------------- svn2ftp.py is a bit long, so I apologize but here goes. I'll have some notes/disclaimers about its contents below it. #!/usr/bin/env python """Usage: svn2ftp.py [OPTION...] FTP-HOST REPOS-PATH Upload to FTP-HOST changes committed to the Subversion repository at REPOS-PATH. Uses svn diff --summarize to only propagate the changed files Options: -?, --help Show this help message. -u, --ftp-user=USER The username for the FTP server. Default: 'anonymous' -p, --ftp-password=P The password for the FTP server. Default: '@' -P, --ftp-port=X Port number for the FTP server. Default: 21 -r, --ftp-remote-dir=DIR The remote directory that is expected to resemble the repository project directory -a, --access-url=URL This is the URL that should be used when trying to SVN export files so that they can be uploaded to the FTP server -s, --status-file=PATH Required. This script needs to store the last successful revision that was transferred to the server. PATH is the location of this file. -d, --project-directory=DIR If the project you are interested in sending to the FTP server is not under the root of the repository (/), set this parameter. Example: -d 'project1/trunk/' This should NOT start with a '/'. 2008.5.2 CKS Fixed possible Windows-related bug with tempfile, where the script didn't have permission to write to the tempfile. Replaced this with a open()-created file created in the CWD. 2008.5.13 CKS Added error logging. Added exception for file-not-found errors when deleting files. 2008.5.14 CKS Change file open to 'rb' mode, to prevent Python's universal newline support from stripping CR characters, causing later comparisons between FTP and SVN to report changes. """ try: import sys, os import logging logging.basicConfig( level=logging.DEBUG, format='%(asctime)s %(levelname)s %(message)s', filename='svn2ftp.debug.log', filemode='a' ) console = logging.StreamHandler() console.setLevel(logging.ERROR) logging.getLogger('').addHandler(console) import getopt, tempfile, smtplib, traceback, subprocess from io import StringIO import pysvn import ftplib import inspect except Exception as e: logging.error(e) #capture the location of the error frame = inspect.currentframe() stack_trace = traceback.format_stack(frame) logging.debug(stack_trace) print(stack_trace) #end capture sys.exit(1) #defaults host = "" user = "anonymous" password = "@" port = 21 repo_path = "" local_repos_path = "" status_file = "" project_directory = "" remote_base_directory = "" toAddrs = "[email protected]" youngest_revision = "" def email(toAddrs, message, subject, fromAddr='[email protected]'): headers = "From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n" % (fromAddr, toAddrs, subject) message = headers + message logging.info('sending email to %s...' % toAddrs) server = smtplib.SMTP('smtp.company.com') server.set_debuglevel(1) server.sendmail(fromAddr, toAddrs, message) server.quit() logging.info('email sent') def captureErrorMessage(e): sout = StringIO() traceback.print_exc(file=sout) errorMessage = '\n'+('*'*80)+('\n%s'%e)+('\n%s\n'%sout.getvalue())+('*'*80) return errorMessage def usage_and_exit(errmsg): """Print a usage message, plus an ERRMSG (if provided), then exit. If ERRMSG is provided, the usage message is printed to stderr and the script exits with a non-zero error code. Otherwise, the usage message goes to stdout, and the script exits with a zero errorcode.""" if errmsg is None: stream = sys.stdout else: stream = sys.stderr print(__doc__, file=stream) if errmsg: print("\nError: %s" % (errmsg), file=stream) sys.exit(2) sys.exit(0) def read_args(): global host global user global password global port global repo_path global local_repos_path global status_file global project_directory global remote_base_directory global youngest_revision try: opts, args = getopt.gnu_getopt(sys.argv[1:], "?u:p:P:r:a:s:d:SU:SP:", ["help", "ftp-user=", "ftp-password=", "ftp-port=", "ftp-remote-dir=", "access-url=", "status-file=", "project-directory=", "svnUser=", "svnPass=" ]) except getopt.GetoptError as msg: usage_and_exit(msg) for opt, arg in opts: if opt in ("-?", "--help"): usage_and_exit() elif opt in ("-u", "--ftp-user"): user = arg elif opt in ("-p", "--ftp-password"): password = arg elif opt in ("-SU", "--svnUser"): svnUser = arg elif opt in ("-SP", "--svnPass"): svnPass = arg elif opt in ("-P", "--ftp-port"): try: port = int(arg) except ValueError as msg: usage_and_exit("Invalid value '%s' for --ftp-port." % (arg)) if port < 1 or port > 65535: usage_and_exit("Value for --ftp-port must be a positive integer less than 65536.") elif opt in ("-r", "--ftp-remote-dir"): remote_base_directory = arg elif opt in ("-a", "--access-url"): repo_path = arg elif opt in ("-s", "--status-file"): status_file = os.path.abspath(arg) elif opt in ("-d", "--project-directory"): project_directory = arg if len(args) != 3: print(str(args)) usage_and_exit("host and/or local_repos_path not specified (" + len(args) + ")") host = args[0] print("args1: " + args[1]) print("args0: " + args[0]) print("abspath: " + os.path.abspath(args[1])) local_repos_path = os.path.abspath(args[1]) print('project_dir:',project_directory) youngest_revision = int(args[2]) if status_file == "" : usage_and_exit("No status file specified") def main(): global host global user global password global port global repo_path global local_repos_path global status_file global project_directory global remote_base_directory global youngest_revision read_args() #repository,fs_ptr #get youngest revision print("local_repos_path: " + local_repos_path) print('getting youngest revision...') #youngest_revision = fs.youngest_rev(fs_ptr) assert youngest_revision, "Unable to lookup youngest revision." last_sent_revision = get_last_revision() if youngest_revision == last_sent_revision: # no need to continue. we should be up to date. print('done, up-to-date') return if last_sent_revision or youngest_revision < 10: # Only compare revisions if the DAT file contains a valid # revision number. Otherwise we risk waiting forever while # we parse and uploading every revision in the repo in the case # where a repository is retroactively configured to sync with ftp. pysvn_client = pysvn.Client() pysvn_client.callback_get_login = get_login rev1 = pysvn.Revision(pysvn.opt_revision_kind.number, last_sent_revision) rev2 = pysvn.Revision(pysvn.opt_revision_kind.number, youngest_revision) summary = pysvn_client.diff_summarize(repo_path, rev1, repo_path, rev2, True, False) print('summary len:',len(summary)) if len(summary) > 0 : print('connecting to %s...' % host) ftp = FTPClient(host, user, password) print('connected to %s' % host) ftp.base_path = remote_base_directory print('set remote base directory to %s' % remote_base_directory) #iterate through all the differences between revisions for change in summary : #determine whether the path of the change is relevant to the path that is being sent, and modify the path as appropriate. print('change path:',change.path) ftp_relative_path = apply_basedir(change.path) print('ftp rel path:',ftp_relative_path) #only try to sync path if the path is in our project_directory if ftp_relative_path != "" : is_file = (change.node_kind == pysvn.node_kind.file) if str(change.summarize_kind) == "delete" : print("deleting: " + ftp_relative_path) try: ftp.delete_path("/" + ftp_relative_path, is_file) except ftplib.error_perm as e: if 'cannot find the' in str(e) or 'not found' in str(e): # Log, but otherwise ignore path-not-found errors # when deleting, since it's not a disaster if the file # we want to delete is already gone. logging.error(captureErrorMessage(e)) else: raise elif str(change.summarize_kind) == "added" or str(change.summarize_kind) == "modified" : local_file = "" if is_file : local_file = svn_export_temp(pysvn_client, repo_path, rev2, change.path) print("uploading file: " + ftp_relative_path) ftp.upload_path("/" + ftp_relative_path, is_file, local_file) if is_file : os.remove(local_file) elif str(change.summarize_kind) == "normal" : print("skipping 'normal' element: " + ftp_relative_path) else : raise str("Unknown change summarize kind: " + str(change.summarize_kind) + ", path: " + ftp_relative_path) ftp.close() #write back the last revision that was synced print("writing last revision: " + str(youngest_revision)) set_last_revision(youngest_revision) # todo: undo def get_login(a,b,c,d): #arguments don't matter, we're always going to return the same thing try: return True, "svnUsername", "svnPassword", True except Exception as e: logging.error(e) #capture the location of the error frame = inspect.currentframe() stack_trace = traceback.format_stack(frame) logging.debug(stack_trace) #end capture sys.exit(1) #functions for persisting the last successfully synced revision def get_last_revision(): if os.path.isfile(status_file) : f=open(status_file, 'r') line = f.readline() f.close() try: i = int(line) except ValueError: i = 0 else: i = 0 f = open(status_file, 'w') f.write(str(i)) f.close() return i def set_last_revision(rev) : f = open(status_file, 'w') f.write(str(rev)) f.close() #augmented ftp client class that can work off a base directory class FTPClient(ftplib.FTP) : def __init__(self, host, username, password) : self.base_path = "" self.current_path = "" ftplib.FTP.__init__(self, host, username, password) def cwd(self, path) : debug_path = path if self.current_path == "" : self.current_path = self.pwd() print("pwd: " + self.current_path) if not os.path.isabs(path) : debug_path = self.base_path + "<" + path path = os.path.join(self.current_path, path) elif self.base_path != "" : debug_path = self.base_path + ">" + path.lstrip("/") path = os.path.join(self.base_path, path.lstrip("/")) path = os.path.normpath(path) #by this point the path should be absolute. if path != self.current_path : print("change from " + self.current_path + " to " + debug_path) ftplib.FTP.cwd(self, path) self.current_path = path else : print("staying put : " + self.current_path) def cd_or_create(self, path) : assert os.path.isabs(path), "absolute path expected (" + path + ")" try: self.cwd(path) except ftplib.error_perm as e: for folder in path.split('/'): if folder == "" : self.cwd("/") continue try: self.cwd(folder) except: print("mkd: (" + path + "):" + folder) self.mkd(folder) self.cwd(folder) def upload_path(self, path, is_file, local_path) : if is_file: (path, filename) = os.path.split(path) self.cd_or_create(path) # Use read-binary to avoid universal newline support from stripping CR characters. f = open(local_path, 'rb') self.storbinary("STOR " + filename, f) f.close() else: self.cd_or_create(path) def delete_path(self, path, is_file) : (path, filename) = os.path.split(path) print("trying to delete: " + path + ", " + filename) self.cwd(path) try: if is_file : self.delete(filename) else: self.delete_path_recursive(filename) except ftplib.error_perm as e: if 'The system cannot find the' in str(e) or '550 File not found' in str(e): # Log, but otherwise ignore path-not-found errors # when deleting, since it's not a disaster if the file # we want to delete is already gone. logging.error(captureErrorMessage(e)) else: raise def delete_path_recursive(self, path): if path == "/" : raise "WARNING: trying to delete '/'!" for node in self.nlst(path) : if node == path : #it's a file. delete and return self.delete(path) return if node != "." and node != ".." : self.delete_path_recursive(os.path.join(path, node)) try: self.rmd(path) except ftplib.error_perm as msg : sys.stderr.write("Error deleting directory " + os.path.join(self.current_path, path) + " : " + str(msg)) # apply the project_directory setting def apply_basedir(path) : #remove any leading stuff (in this case, "trunk/") and decide whether file should be propagated if not path.startswith(project_directory) : return "" return path.replace(project_directory, "", 1) def svn_export_temp(pysvn_client, base_path, rev, path) : # Causes access denied error. Couldn't deduce Windows-perm issue. # It's possible Python isn't garbage-collecting the open file-handle in time for pysvn to re-open it. # Regardless, just generating a simple filename seems to work. #(fd, dest_path) = tempfile.mkstemp() dest_path = tmpName = '%s.tmp' % __file__ exportPath = os.path.join(base_path, path).replace('\\','/') print('exporting %s to %s' % (exportPath, dest_path)) pysvn_client.export( exportPath, dest_path, force=False, revision=rev, native_eol=None, ignore_externals=False, recurse=True, peg_revision=rev ) return dest_path if __name__ == "__main__": logging.info('svnftp.start') try: main() logging.info('svnftp.done') except Exception as e: # capture the location of the error for debug purposes frame = inspect.currentframe() stack_trace = traceback.format_stack(frame) logging.debug(stack_trace[:-1]) print(stack_trace) # end capture error_text = '\nFATAL EXCEPTION!!!\n'+captureErrorMessage(e) subject = "ALERT: SVN2FTP Error" message = """An Error occurred while trying to FTP an SVN commit. repo_path = %(repo_path)s\n local_repos_path = %(local_repos_path)s\n project_directory = %(project_directory)s\n remote_base_directory = %(remote_base_directory)s\n error_text = %(error_text)s """ % globals() email(toAddrs, message, subject) logging.error(e) Notes/Disclaimers: I have basically no python training so I'm learning as I go and spending lots of time reading docs to figure stuff out. The body of get_login is in a try block because I was getting strange errors saying there was an unhandled exception in callback_get_login. Never figured out why, but it seems fine now. Let sleeping dogs lie, right? The username and password for get_login are currently hard-coded (but correct) just to eliminate variables and try to change as little as possible at once. (I added the svnuser and svnpass arguments to the existing argument parsing.) So that's where I am. I can't figure out why on earth it's not printing anything into svn2ftp.out.log. If you're wondering, the output for one of these failed attempts in svn2ftp.debug.log is: 2012-09-06 15:18:12,496 INFO svnftp.start 2012-09-06 15:18:12,496 INFO svnftp.done And it's no different on a successful run. So there's nothing useful being logged. I'm lost. I've gone way down the rabbit hole on this one, and don't know where to go from here. Any ideas?

Read the article

Hi, I have a C hashing routine which is behaving strangely?

- by aks

Hi, In this hashing routine: 1.) I am able to add strings. 2.) I am able to view my added strings. 3.) When i try to add a duplicate string, it throws me an error saying already present. 4.) But, when i try to delete the same string which is already present in hash table, then the lookup_routine calls hash function to get an index. At this time, it throws a different hash index to the one it was already added. Hence, my delete routine is failing? I am able to understand the reason why for same string, hash fucntion calculates a different index each time (whereas the same logic works in view hash table routine)? Can someone help me? This is the Output, i am getting: $ ./a Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: 1 Please enter the string :gaura enters in add_string DEBUG purpose in hash function: str passed = gaura Hashval returned in hash func= 1 hashval = 1 enters in lookup_string str in lookup_string = gaura DEBUG purpose in hash function: str passed = gaura Hashval returned in hash func= 1 hashval = 1 DEBUG ERROR :element not found in lookup string DEBUG Purpose NULL Inserting... DEBUG1 : enters here hashval = 1 String added successfully Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: 1 Please enter the string :ayu enters in add_string DEBUG purpose in hash function: str passed = ayu Hashval returned in hash func= 1 hashval = 1 enters in lookup_string str in lookup_string = ayu DEBUG purpose in hash function: str passed = ayu Hashval returned in hash func= 1 hashval = 1 returns NULL in lookup_string DEBUG Purpose NULL Inserting... DEBUG2 : enters here hashval = 1 String added successfully Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: 1 Please enter the string :gaurava enters in add_string DEBUG purpose in hash function: str passed = gaurava Hashval returned in hash func= 7 hashval = 7 enters in lookup_string str in lookup_string = gaurava DEBUG purpose in hash function: str passed = gaurava Hashval returned in hash func= 7 hashval = 7 DEBUG ERROR :element not found in lookup string DEBUG Purpose NULL Inserting... DEBUG1 : enters here hashval = 7 String added successfully Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: 4 Index : i = 1 String = gaura ayu Index : i = 7 String = gaurava Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: 2 Please enter the string you want to delete :gaura String entered = gaura enters in delete_string DEBUG purpose in hash function: str passed = gaura Hashval returned in hash func= 0 hashval = 0 enters in lookup_string str in lookup_string = gaura DEBUG purpose in hash function: str passed = gaura Hashval returned in hash func= 0 hashval = 0 DEBUG ERROR :element not found in lookup string DEBUG Purpose Item not present. So, cannot be deleted Item found in list: Deletion failed Press 1 to add an element to the hashtable Press 2 to delete an element from the hashtable Press 3 to search the hashtable Press 4 to view the hashtable Press 5 to exit Please enter your choice: My routine is pasted below: include include struct list { char *string; struct list *next; }; struct hash_table { int size; /* the size of the table */ struct list *table; / the table elements */ }; struct hash_table * hashtable = NULL; struct hash_table *create_hash_table(int size) { struct hash_table *new_table; int i; if (size<1) return NULL; /* invalid size for table */ /* Attempt to allocate memory for the table structure */ if ((new_table = malloc(sizeof(struct hash_table))) == NULL) { return NULL; } /* Attempt to allocate memory for the table itself */ if ((new_table->table = malloc(sizeof(struct list *) * size)) == NULL) { return NULL; } /* Initialize the elements of the table */ for(i=0; i<size; i++) new_table->table[i] = '\0'; /* Set the table's size */ new_table->size = size; return new_table; } unsigned int hash(struct hash_table *hashtable, char *str) { printf("\n DEBUG purpose in hash function:\n"); printf("\n str passed = %s", str); unsigned int hashval = 0; int i = 0; for(; *str != '\0'; str++) { hashval += str[i]; i++; } hashval = hashval % 10; printf("\n Hashval returned in hash func= %d", hashval); return hashval; } struct list *lookup_string(struct hash_table *hashtable, char *str) { printf("\n enters in lookup_string \n"); printf("\n str in lookup_string = %s",str); struct list * new_list; unsigned int hashval = hash(hashtable, str); printf("\n hashval = %d \n", hashval); if(hashtable->table[hashval] == NULL) { printf("\n DEBUG ERROR :element not found in lookup string \n"); return NULL; } /* Go to the correct list based on the hash value and see if str is * in the list. If it is, return return a pointer to the list element. * If it isn't, the item isn't in the table, so return NULL. */ for(new_list = hashtable->table[hashval]; new_list != NULL;new_list = new_list->next) { if (strcmp(str, new_list->string) == 0) return new_list; } printf("\n returns NULL in lookup_string \n"); return NULL; } int add_string(struct hash_table *hashtable, char *str) { printf("\n enters in add_string \n"); struct list *new_list; struct list *current_list; unsigned int hashval = hash(hashtable, str); printf("\n hashval = %d", hashval); /* Attempt to allocate memory for list */ if ((new_list = malloc(sizeof(struct list))) == NULL) { printf("\n enters here \n"); return 1; } /* Does item already exist? */ current_list = lookup_string(hashtable, str); if (current_list == NULL) { printf("\n DEBUG Purpose \n"); printf("\n NULL \n"); } /* item already exists, don't insert it again. */ if (current_list != NULL) { printf("\n Item already present...\n"); return 2; } /* Insert into list */ printf("\n Inserting...\n"); new_list->string = strdup(str); new_list->next = NULL; //new_list->next = hashtable->table[hashval]; if(hashtable->table[hashval] == NULL) { printf("\n DEBUG1 : enters here \n"); printf("\n hashval = %d", hashval); hashtable->table[hashval] = new_list; } else { printf("\n DEBUG2 : enters here \n"); printf("\n hashval = %d", hashval); struct list * temp_list = hashtable->table[hashval]; while(temp_list->next!=NULL) temp_list = temp_list->next; temp_list->next = new_list; // hashtable->table[hashval] = new_list; } return 0; } int delete_string(struct hash_table *hashtable, char *str) { printf("\n enters in delete_string \n"); struct list *new_list; struct list *current_list; unsigned int hashval = hash(hashtable, str); printf("\n hashval = %d", hashval); /* Does item already exist? */ current_list = lookup_string(hashtable, str); if (current_list == NULL) { printf("\n DEBUG Purpose \n"); printf("\n Item not present. So, cannot be deleted \n"); return 1; } /* item exists, delete it. */ if (current_list != NULL) { struct list * temp = hashtable->table[hashval]; if(strcmp(temp->string,str) == 0) { if(temp->next == NULL) { hashtable->table[hashval] = NULL; free(temp); } else { hashtable->table[hashval] = temp->next; free(temp); } } else { struct list * temp1; while(temp->next != NULL) { temp1 = temp; if(strcmp(temp->string, str) == 0) { break; } else { temp = temp->next; } } if(temp->next == NULL) { temp1->next = NULL; free(temp); } else { temp1->next = temp->next; free(temp); } } } return 0; } void free_table(struct hash_table *hashtable) { int i; struct list *new_list, *temp_list; if (hashtable==NULL) return; /* Free the memory for every item in the table, including the * strings themselves. */ for(i=0; i<hashtable->size; i++) { new_list = hashtable->table[i]; while(new_list!=NULL) { temp_list = new_list; new_list = new_list->next; free(temp_list->string); free(temp_list); } } /* Free the table itself */ free(hashtable->table); free(hashtable); } void view_hashtable(struct hash_table * hashtable) { int i = 0; if(hashtable == NULL) return; for(i =0; i < hashtable->size; i++) { if((hashtable->table[i] == 0) || (strcmp(hashtable->table[i]->string, "*") == 0)) { continue; } printf(" Index : i = %d\t String = %s",i, hashtable->table[i]->string); struct list * temp = hashtable->table[i]->next; while(temp != NULL) { printf("\t %s",temp->string); temp = temp->next; } printf("\n"); } } int main() { hashtable = create_hash_table(10); if(hashtable == NULL) { printf("\n Memory allocation failure during creation of hash table \n"); return 0; } int flag = 1; while(flag) { int choice; printf("\n Press 1 to add an element to the hashtable\n"); printf("\n Press 2 to delete an element from the hashtable\n"); printf("\n Press 3 to search the hashtable\n"); printf("\n Press 4 to view the hashtable\n"); printf("\n Press 5 to exit \n"); printf("\n Please enter your choice: "); scanf("%d",&choice); if(choice == 5) flag = 0; else if(choice == 1) { char str[20]; printf("\n Please enter the string :"); scanf("%s",&str); int i; i = add_string(hashtable,str); if(i == 1) { printf("\n Memory allocation failure:Choice 1 \n"); return 0; } else if(i == 2) { printf("\n String already prsent in hash table : Couldnot add it again\n"); return 0; } else { printf("\n String added successfully \n"); } } else if(choice == 2) { int i; struct list * temp_list; char str[20]; printf("\n Please enter the string you want to delete :"); scanf("%s",&str); printf("\n String entered = %s", str); i = delete_string(hashtable,str); if(i == 0) { printf("\n Item found in list: Deletion success \n"); } else printf("\n Item found in list: Deletion failed \n"); } else if(choice == 3) { struct list * temp_list; char str[20]; printf("\n Please enter the string :"); scanf("%s",&str); temp_list = lookup_string(hashtable,str); if(!temp_list) { printf("\n Item not found in list: Deletion failed \n"); return 0; } printf("\n Item found: Present in Hash Table \n"); } else if(choice == 4) { view_hashtable(hashtable); } else if(choice == 5) { printf("\n Exiting ...."); return 0; } else printf("\n Invalid choice:"); }; free_table(hashtable); return 0; }

Search Results

Search found 17621 results on 705 pages for 'just my correct opinion'.

Page 704/705 | < Previous Page | 700 701 702 703 704 705 | Next Page >

- by sam

- by Jacksta

- by iwanttoprogram

- by Coombes

- by Junier

- by vipul12389

- by user1511579

- by Jonathon

- by MOKANA

- by user1290528

- by Flip_novidade

- by raul.goycoolea

- by faye.todd(at)oracle.com

- by rchrd

- by Prasanna K Rao

- by Nathan Spears

- by Nathan Spears

- by Sergei

- by user356937

- by Peter

- by Greenhouse Gases

- by louism

- by Adam Tuttle

- by aks

< Previous Page | 700 701 702 703 704 705 | Next Page >