Search Results

Search found 2457 results on 99 pages for 'uri cohen'.

Page 61/99 | < Previous Page | 57 58 59 60 61 62 63 64 65 66 67 68  | Next Page >

  • Receiving POST data in ASP.NET

    - by grast
    Hi, I want to use ASP for code generation in a C# desktop application. To achieve this, I set up a simple host (derived from System.MarshalByRefObject) that processes a System.Web.Hosting.SimpleWorkerRequest via HttpRuntime.ProcessRequest. This processes the ASPX script specified by the incoming request (using System.Net.HttpListener to wait for requests). The client-part is represented by a System.ComponentModel.BackgroundWorker that builds the System.Net.HttpWebRequest and receives the response from the server. A simplified version of my client-part-code looks like this: private void SendRequest(object sender, DoWorkEventArgs e) { // create request with GET parameter var uri = "http://localhost:9876/test.aspx?getTest=321"; var request = (HttpWebRequest)WebRequest.Create(uri); // append POST parameter request.Method = "POST"; request.ContentType = "application/x-www-form-urlencoded"; var postData = Encoding.Default.GetBytes("postTest=654"); var postDataStream = request.GetRequestStream(); postDataStream.Write(postData, 0, postData.Length); // send request, wait for response and store/print content using (var response = (HttpWebResponse)request.GetResponse()) { using (var reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8)) { _processsedContent = reader.ReadToEnd(); Debug.Print(_processsedContent); } } } My server-part-code looks like this (without exception-handling etc.): public void ProcessRequests() { // HttpListener at http://localhost:9876/ var listener = SetupListener(); // SimpleHost created by ApplicationHost.CreateApplicationHost var host = SetupHost(); while (_running) { var context = listener.GetContext(); using (var writer = new StreamWriter(context.Response.OutputStream)) { // process ASP script and send response back to client host.ProcessRequest(GetPage(context), GetQuery(context), writer); } context.Response.Close(); } } So far all this works fine as long as I just use GET parameters. But when it comes to receiving POST data in my ASPX script I run into trouble. For testing I use the following script: // GET parameters are working: var getTest = Request.QueryString["getTest"]; Response.Write("getTest: " + getTest); // prints "getTest: 321" // don't know how to access POST parameters: var postTest1 = Request.Form["postTest"]; // Request.Form is empty?! Response.Write("postTest1: " + postTest1); // so this prints "postTest1: " var postTest2 = Request.Params["postTest"]; // Request.Params is empty?! Response.Write("postTest2: " + postTest2); // so this prints "postTest2: " It seems that the System.Web.HttpRequest object I'm dealing with in ASP does not contain any information about my POST parameter "postTest". I inspected it in debug mode and none of the members did contain neither the parameter-name "postTest" nor the parameter-value "654". I also tried the BinaryRead method of Request, but unfortunately it is empty. This corresponds to Request.InputStream==null and Request.ContentLength==0. And to make things really confusing the Request.HttpMethod member is set to "GET"?! To isolate the problem I tested the code by using a PHP script instead of the ASPX script. This is very simple: print_r($_GET); // prints all GET variables print_r($_POST); // prints all POST variables And the result is: Array ( [getTest] = 321 ) Array ( [postTest] = 654 ) So with the PHP script it works, I can access the POST data. Why does the ASPX script don't? What am I doing wrong? Is there a special accessor or method in the Response object? Can anyone give a hint or even know how to solve this? Thanks in advance.

    Read the article

  • Code Access Security and Sharepoint WebParts

    - by Gordon Carpenter-Thompson
    I've got a vague handle on how Code Access Security works in Sharepoint. I have developed a custom webpart and setup a CAS policy in my Manifest <CodeAccessSecurity> <PolicyItem> <PermissionSet class="NamedPermissionSet" version="1" Description="Permission set for Okana"> <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" ObjectModel="True" Impersonate="True" /> <IPermission class="SecurityPermission" version="1" Flags="Assertion, Execution, ControlThread, ControlPrincipal, RemotingConfiguration" /> <IPermission class="AspNetHostingPermission" version="1" Level="Medium" /> <IPermission class="DnsPermission" version="1" Unrestricted="true" /> <IPermission class="EventLogPermission" version="1" Unrestricted="true"> <Machine name="localhost" access="Administer" /> </IPermission> <IPermission class="EnvironmentPermission" version="1" Unrestricted="true" /> <IPermission class="System.Configuration.ConfigurationPermission, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" version="1" Unrestricted="true"/> <IPermission class="System.Net.WebPermission, System, Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Unrestricted="true" /> <IPermission class="System.Net.WebPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" Unrestricted="true" /> <IPermission class="System.Security.Permissions.FileIOPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Unrestricted="true" PathDiscovery="*AllFiles*" /> <IPermission class="IsolatedStorageFilePermission" version="1" Allowed="AssemblyIsolationByUser" UserQuota="9223372036854775807" /> <IPermission class="PrintingPermission" version="1" Level="DefaultPrinting" /> <IPermission class="PerformanceCounterPermission" version="1"> <Machine name="localhost"> <Category name="Enterprise Library Caching Counters" access="Write"/> <Category name="Enterprise Library Cryptography Counters" access="Write"/> <Category name="Enterprise Library Data Counters" access="Write"/> <Category name="Enterprise Library Exception Handling Counters" access="Write"/> <Category name="Enterprise Library Logging Counters" access="Write"/> <Category name="Enterprise Library Security Counters" access="Write"/> </Machine> </IPermission> <IPermission class="ReflectionPermission" version="1" Unrestricted="true"/> <IPermission class="SecurityPermission" version="1" Flags="SerializationFormatter, UnmanagedCode, Infrastructure, Assertion, Execution, ControlThread, ControlPrincipal, RemotingConfiguration, ControlAppDomain,ControlDomainPolicy" /> <IPermission class="SharePointPermission" version="1" ObjectModel="True" /> <IPermission class="SmtpPermission" version="1" Access="Connect" /> <IPermission class="SqlClientPermission" version="1" Unrestricted="true"/> <IPermission class="WebPartPermission" version="1" Connections="True" /> <IPermission class="WebPermission" version="1"> <ConnectAccess> <URI uri="$OriginHost$"/> </ConnectAccess> </IPermission> </PermissionSet> <Assemblies> .... </Assemblies> This is correctly converted into a wss_custom_wss_minimaltrust.config when it is deployed onto the Sharepoint server and mostly works. To get the WebPart working fully, however I find that I need to modify the wss_custom_wss_minimaltrust.config by hand after deployment and set Unrestricted="true" on the permissions set <PermissionSet class="NamedPermissionSet" version="1" Description="Permission set for MyApp" Name="mywebparts.wsp-86d8cae1-7db2-4057-8c17-dc551adb17a2-1"> to <PermissionSet class="NamedPermissionSet" version="1" Description="Permission set for MyApp" Name="mywebparts.wsp-86d8cae1-7db2-4057-8c17-dc551adb17a2-1" Unrestricted="true"> It's all because I'm loading a User Control from the webpart. I don't believe there is a way to enable that using CAS but am willing to be proven wrong. Is there a way to set something in the manifest so I don't need to make this fix by hand? Thanks

    Read the article

  • Why do I get error "prefix [..] is not defined" when I try to use my jsf custom tag?

    - by Roman
    I created a jsf custom tag (I'm not sure that it's correct, I could miss something easily, so I attached code below). Now I'm trying to use this tag but I get an error: error on line 28 at column 49: Namespace prefix gc on ganttchart is not defined So, here is the xhtml-page: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xmlns:ui="http://java.sun.com/jsf/facelets" xmlns:h="http://java.sun.com/jsf/html" xmlns:f="http://java.sun.com/jsf/core" xmlns:gc="http://myganttchart.org"> <body> <ui:composition template="/masterpage.xhtml"> <ui:define name="title">Gantt chart test</ui:define> <ui:define name="content"> <f:view> <gc:ganttchart width="300" height="100" rendered="true"/> ... </f:view> </ui:define> </ui:composition> </body> </html> And here is tld-file (it's placed in WEB-INF/): <taglib xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-jsptaglibrary_2_1.xsd" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.1"> <tlib-version> 1.0 </tlib-version> <short-name> oext </short-name> <uri> http://myganttchart.org </uri> <tag> <name>ganttchart</name> <tag-class>usermanagement.support.ganttchart.GanttChartTag</tag-class> <body-content>empty</body-content> <attribute> <name>binding</name> <deferred-value> <type>javax.faces.component.UIComponent</type> </deferred-value> </attribute> ... </tag> </tablib> Here is a part of tag-class code: public class GanttChartTag extends UIComponentELTag { private ValueExpression width; private ValueExpression height; private ValueExpression styleClass; public String getComponentType () { return "org.myganttchart"; } public String getRendererType () { return null; } ... } Correspondent block from faces-config: <component> <component-type>org.myganttchart</component-type> <component-class>usermanagement.support.ganttchart.UIGanttChart</component-class> </component> And the last part if UIGanttChart: public class UIGanttChart extends UIOutput { public UIGanttChart() { setRendererType (null); } //some test code public void encodeBegin(FacesContext context) throws IOException { ResponseWriter writer = context.getResponseWriter (); writer.startElement("img", this); writer.writeAttribute("src", "no-img", "source"); writer.writeAttribute("width", getAttributes ().get ("width"), "width"); writer.writeAttribute("height", getAttributes ().get ("height"), "height"); writer.writeAttribute("class", ".someclass", "styleClass"); writer.endElement("img"); } } So, what did I miss?

    Read the article

  • WCF Service error on IIS with metadata

    - by Bruno Silva
    Hi, I'm trying to publish a service to IIS, it builds and runs OK on the ASP.NET dev server. When running in IIS I can get to the metadata by navigating to the service or by adding service reference in Visual Studio. But when I call a method from my client app it crashes with a internal server error. So I went to the Event Log and found this: WebHost failed to process a request. Sender Information: System.ServiceModel.Activation.HostedHttpRequestAsyncResult/8810861 Exception: System.Web.HttpException (0x80004005): There was no channel actively listening at 'http://mysite.net/soundhubservice.svc/$metadata'. This is often caused by an incorrect address URI. Ensure that the address to which the message is sent matches an address on which a service is listening. ---> System.ServiceModel.EndpointNotFoundException: There was no channel actively listening at 'http://mysite.net/soundhubservice.svc/$metadata'. This is often caused by an incorrect address URI. Ensure that the address to which the message is sent matches an address on which a service is listening. at System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(HostedHttpRequestAsyncResult result) at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest() at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest() at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result) at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.End(IAsyncResult result) Process Name: w3wp Process ID: 1080 My Web.Config looks something like this: <configuration> <system.web> <compilation debug="true" targetFramework="4.0" /> </system.web> <system.serviceModel> <services> <service name="SoundHub.Services.SoundHubService" behaviorConfiguration="StreamingServiceBehavior"> <host> <baseAddresses> <add baseAddress="http://localhost/SoundHubServive"/> </baseAddresses> </host> <endpoint address="service" binding="basicHttpBinding" bindingConfiguration="httpBuffering" contract="SoundHub.Services.ISoundHubService"/> <endpoint address="stream" binding="basicHttpBinding" bindingConfiguration="HttpStreaming" contract="SoundHub.Services.ISoundHubStreamService"/> <!--<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />--> </service> </services> <bindings> <basicHttpBinding> <binding name="HttpStreaming" maxReceivedMessageSize="67108864" transferMode="Streamed"/> <binding name="httpBuffering" transferMode="Buffered" /> </basicHttpBinding> </bindings> <behaviors> <serviceBehaviors> <behavior name="StreamingServiceBehavior"> <serviceMetadata httpGetEnabled="True"/> <serviceDebug includeExceptionDetailInFaults="False"/> </behavior> </serviceBehaviors> </behaviors> </system.serviceModel> <system.webServer> <modules runAllManagedModulesForAllRequests="true"/> </system.webServer> </configuration> Tried several combinations of settings I found while searching online but nothing helped, always the same error. Thanks Bruno

    Read the article

  • Launching a file using ACTION_VIEW Intent Action

    - by Sneha
    I have the following code to launch a file : try { path = fileJsonObject.getString("filePath"); if (path.indexOf("/") == 0) { path = path.substring(1, path.length()); } path = root + path; final File fileToOpen = new File(path); if (fileToOpen.exists()) { if (fileToOpen.isFile()) { Intent myIntent = new Intent(android.content.Intent.ACTION_VIEW); myIntent.setData(Uri.parse(path)); final String pathToCheck = new String(path); pathToCheck.toLowerCase(); if (pathToCheck.endsWith(".wav") || pathToCheck.endsWith(".ogg") || pathToCheck.endsWith(".mp3") || pathToCheck.endsWith(".mid") || pathToCheck.endsWith(".midi") || pathToCheck.endsWith(".amr")) { myIntent.setType("audio/*"); } else if (pathToCheck.endsWith(".mpg") || pathToCheck.endsWith(".mpeg") || pathToCheck.endsWith(".3gp") || pathToCheck.endsWith(".mp4")) { myIntent.setType("video/*"); } else if (pathToCheck.endsWith(".jpg") || pathToCheck.endsWith(".jpeg") || pathToCheck.endsWith(".gif") || pathToCheck.endsWith(".png") || pathToCheck.endsWith(".bmp")) { myIntent.setType("image/*"); } else if (pathToCheck.endsWith(".txt") || pathToCheck.endsWith(".csv") || pathToCheck.endsWith(".xml")) { Log.i("txt","Text fileeeeeeeeeeeeeeeeeeeeeeeeee"); myIntent.setType("text/*"); } else if (pathToCheck.endsWith(".gz") || pathToCheck.endsWith(".rar") || pathToCheck.endsWith(".zip")) { myIntent.setType("package/*"); } else if (pathToCheck.endsWith(".apk")) { myIntent.setType("application/vnd.android.package-archive"); } ((Activity) context).startActivityForResult(myIntent, RequestCodes.LAUNCH_FILE_CODE); } else { errUrl = resMsgHandler.errMsgResponse(fileJsonObject, "Incorrect path provided. please give correct path of file"); return errUrl; } } else { errUrl = resMsgHandler.errMsgResponse(fileJsonObject,"Incorrect path provided. please give correct path of file"); return errUrl; } } catch (Exception e) { e.printStackTrace(); Log.i("err","Unable to launch file" + " " + e.getMessage()); errUrl = resMsgHandler.errMsgResponse(fileJsonObject, "Unable to launch file" + " " + e.getMessage()); return errUrl; } @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { // TODO Auto-generated method stub super.onActivityResult(requestCode, resultCode, data); try { if (requestCode == RequestCodes.LAUNCH_FILE_CODE) { if (resultCode == RESULT_CANCELED) { Log.i("err","errrrrrrrrrrrrrrrrrrrrrrrrrrrrrr"); String errUrl = responseMsgHandler.errMsgResponse(FileHandler.fileJsonObject, "Unable to launch file"); mWebView.loadUrl(errUrl); } else if (resultCode == RESULT_OK) { String successUrl = responseMsgHandler.launchfileResponse(FileHandler.fileJsonObject); mWebView.loadUrl(successUrl); } Amd the result ctrl is at "if (resultCode == RESULT_CANCELED)". So how to successfully launch this? May be in short i am doing this: final File fileToOpen = new File(path); if (fileToOpen.exists()) { if (fileToOpen.isFile()) { Intent myIntent = new Intent(android.content.Intent.ACTION_VIEW); myIntent.setData(Uri.parse(path)); if (pathToCheck.endsWith(".txt") || pathToCheck.endsWith(".csv") || pathToCheck.endsWith(".xml")) { Log.i("txt","Text fileeeeeeeeeeeeeeeeeeeeeeeeee"); myIntent.setType("text/*"); startActivityForResult(myIntent, RequestCodes.LAUNCH_FILE_CODE); and @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { // TODO Auto-generated method stub super.onActivityResult(requestCode, resultCode, data); if (requestCode == RequestCodes.LAUNCH_FILE_CODE) { if (resultCode == RESULT_CANCELED) { Log.i ("err","errrrrrrrrrrrrrrrrrrrrrrrrrrrrrr"); String errUrl = responseMsgHandler.errMsgResponse(FileHandler.fileJsonObject, "Unable to launch file"); mWebView.loadUrl(errUrl); } else if (resultCode == RESULT_OK) { String successUrl = responseMsgHandler.launchfileResponse(FileHandler.fileJsonObject); mWebView.loadUrl(successUrl); } My err log: 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.tf.thinkdroid.sstablet/com.tf.thinkdroid.write.editor.WriteEditorActivity}: java.lang.NullPointerException 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2663) 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2679) ..... 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): Caused by: java.lang.NullPointerException 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): at com.tf.thinkdroid.common.app.TFActivity.storeDataToFileIfNecessary(Unknown Source) 04-04 10:53:57.077: ERROR/AndroidRuntime(6861): at com.tf.thinkdroid.common.app.TFActivity.onPostCreate(Unknown Source) ... Thanks Sneha

    Read the article

  • Load balancing using Mina example with Java DSL

    - by Flame_Phoenix
    So, recently I started learning Camel. As part of the process I decided to go through all the examples (listed HERE and available when you DOWNLOAD the package with all the examples and docs) and to see what I could learn. One of the examples, Load Balancing using Mina caught my attention because it uses a Mina in different JVM's and it simulates a load balancer with round robin. I have a few problems with this example. First it uses the Spring DSL, instead of the Java DSL which my project uses and which I find a lot easier to understand now (mainly also because I am used to it). So the first question: is there a version of this example using only the Java DSL instead of the Spring DSL for the routes and the beans? My second questions is code related. The description states, and I quote: Within this demo every ten seconds, a Report object is created from the Camel load balancer server. This object is sent by the Camel load balancer to a MINA server where the object is then serialized. One of the two MINA servers (localhost:9991 and localhost:9992) receives the object and enriches the message by setting the field reply of the Report object. The reply is sent back by the MINA server to the client, which then logs the reply on the console. So, from what I read, I understand that the MINA server 1 (per example) receives a report from the loadbalancer, changes it, and then it sends that report back to some invisible client. Upon checking the code, I see no client java class or XML and when I run, the server simply posts the results on the command line. Where is the client ?? What is this client? In the MINA 1server code presented here: <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:camel="http://camel.apache.org/schema/spring" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <bean id="service" class="org.apache.camel.example.service.Reporting"/> <camelContext xmlns="http://camel.apache.org/schema/spring"> <route id="mina1"> <from uri="mina:tcp://localhost:9991"/> <setHeader headerName="minaServer"> <constant>localhost:9991</constant> </setHeader> <bean ref="service" method="updateReport"/> </route> </camelContext> </beans> I don't understand how the updateReport method magically prints the object on my console. What if I wanted to send message to a third MINA server? How would I do it? (I would have to add a new route, and send it to the URI of the 3rd server correct?) I know most of these questions may sound dumb, but I would appreciate if anyone could help me. A Java DSL version of this would really help me.

    Read the article

  • GoogleTv video using VideoView is clipped, even though its in a separate layout of its own

    - by MYR
    Wrote an application for GoogleTV box to play a video. The video is a separate activity to the main activity. Wired up a button on my main layout/activity to to start the video activity up (using its own video.xml layout), the video activity loads and starts playing, but the video is clipped, only showing a few centimetres of the bottom. The clipped region looks like the views that occupied the previous layout (main.xml) . The intriguing thing is that if I push the back button, before returning to the main activity the full video frame is shown. Not sure what I am doing wrong here. Any suggestions welcome. Code: main.xml <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="fill_parent" android:layout_height="fill_parent" android:orientation="vertical" > <TextView android:id="@+id/hello_message" android:layout_width="fill_parent" android:layout_height="200dp" android:gravity="center" android:text="@string/hello_message" android:textSize="78sp" /> <LinearLayout android:layout_width="match_parent" android:layout_height="wrap_content" > <EditText android:id="@+id/edit_message" android:layout_width="0dip" android:layout_height="wrap_content" android:layout_weight="1" android:ems="10" android:hint="@string/edit_message" > <requestFocus /> </EditText> <Button android:id="@+id/button_send" android:layout_width="wrap_content" android:layout_height="wrap_content" android:onClick="displayMessage" android:text="@string/button_send" /> <Button android:id="@+id/button_send_a" android:layout_width="wrap_content" android:layout_height="wrap_content" android:onClick="sendMesssage" android:text="@string/button_send_a" /> </LinearLayout> <Button android:id="@+id/videobutton" android:layout_width="wrap_content" android:layout_height="wrap_content" android:onClick="startAVideo" android:text="Video Player" /> </LinearLayout> GoogleTVExActivity.java (excerpt): ... public class GoogleTVExActivity extends Activity { public final static String EXTRA_MESSAGE = "uk.co.bbc.googletvex.MESSAGE"; /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main); } public void displayMessage(View view) { TextView t = (TextView)findViewById(R.id.hello_message); EditText e =(EditText) findViewById(R.id.edit_message); t.setText(e.getText().toString()); e.setText(""); } public void sendMesssage(View view) { Intent intent = new Intent(this, DisplayMessageActivity.class); EditText editText = (EditText) findViewById(R.id.edit_message); String message = editText.getText().toString(); intent.putExtra(EXTRA_MESSAGE, message); startActivity(intent); } public void startAVideo(View view) { Intent intent = new Intent(this, VideoViewActivity.class); startActivity(intent); } } video.xml: <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" > <VideoView android:id="@+id/myvideoview" android:layout_width="fill_parent" android:layout_height="fill_parent" /> </LinearLayout> VideoViewActivity.java (excerpt) ... import android.app.Activity; import android.net.Uri; import android.os.Bundle; import android.widget.MediaController; import android.widget.VideoView; public class VideoViewActivity extends Activity { String SrcPath = "rtsp://v5.cache1.c.youtube.com/CjYLENy73wIaLQnhycnrJQ8qmRMYESARFEIJbXYtZ29vZ2xlSARSBXdhdGNoYPj_hYjnq6uUTQw=/0/0/0/video.3gp"; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.video); VideoView myVideoView = (VideoView)findViewById(R.id.myvideoview); myVideoView.setVideoURI(Uri.parse(SrcPath)); myVideoView.setMediaController(new MediaController(this)); myVideoView.requestFocus(); myVideoView.start(); } }

    Read the article

  • Android: Showing photos runs out of memory

    - by Tom Beech
    I'm using a dialog box to display images in my android project. The first one opens fine, but when I close it and do the process again to show a different one the app falls over with a memory error (it's running on a samsung galaxy s3 - so shouldnt be an issue). Error: 10-24 11:25:45.575: E/dalvikvm-heap(29194): Out of memory on a 31961104-byte allocation. 10-24 11:25:45.580: E/AndroidRuntime(29194): FATAL EXCEPTION: main 10-24 11:25:45.580: E/AndroidRuntime(29194): java.lang.OutOfMemoryError 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.graphics.BitmapFactory.nativeDecodeStream(Native Method) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:587) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:389) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.graphics.BitmapFactory.decodeFile(BitmapFactory.java:418) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.graphics.drawable.Drawable.createFromPath(Drawable.java:882) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.ImageView.resolveUri(ImageView.java:569) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.ImageView.setImageURI(ImageView.java:340) 10-24 11:25:45.580: E/AndroidRuntime(29194): at com.directenquiries.assessment.tool.AddAsset.loadPhoto(AddAsset.java:771) 10-24 11:25:45.580: E/AndroidRuntime(29194): at com.directenquiries.assessment.tool.AddAsset$11.onClick(AddAsset.java:748) 10-24 11:25:45.580: E/AndroidRuntime(29194): at com.android.internal.app.AlertController$AlertParams$3.onItemClick(AlertController.java:936) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.AdapterView.performItemClick(AdapterView.java:292) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.AbsListView.performItemClick(AbsListView.java:1359) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.AbsListView$PerformClick.run(AbsListView.java:2988) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.widget.AbsListView$1.run(AbsListView.java:3783) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.os.Handler.handleCallback(Handler.java:605) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.os.Handler.dispatchMessage(Handler.java:92) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.os.Looper.loop(Looper.java:137) 10-24 11:25:45.580: E/AndroidRuntime(29194): at android.app.ActivityThread.main(ActivityThread.java:4517) 10-24 11:25:45.580: E/AndroidRuntime(29194): at java.lang.reflect.Method.invokeNative(Native Method) 10-24 11:25:45.580: E/AndroidRuntime(29194): at java.lang.reflect.Method.invoke(Method.java:511) 10-24 11:25:45.580: E/AndroidRuntime(29194): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:993) 10-24 11:25:45.580: E/AndroidRuntime(29194): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:760) 10-24 11:25:45.580: E/AndroidRuntime(29194): at dalvik.system.NativeStart.main(Native Method) Loading code: public void loadPhotoList(){ Cursor f = db.rawQuery("select * from stationphotos where StationObjectID = '"+ checkStationObjectID + "'", null); final ArrayList<String> mHelperNames= new ArrayList<String>(); if(f.getCount() != 0) { f.moveToFirst(); f.moveToFirst(); while(!f.isAfterLast()) { mHelperNames.add(f.getString(f.getColumnIndex("FilePath"))); f.moveToNext(); } } f.close(); final String [] nameStrings = new String [mHelperNames.size()]; for(int i=0; i<mHelperNames.size(); i++) nameStrings[i] = mHelperNames.get(i).toString(); AlertDialog.Builder builder = new AlertDialog.Builder(this); builder.setTitle("Select Picture"); builder.setItems(nameStrings, new DialogInterface.OnClickListener() { public void onClick(DialogInterface dialog, int item) { loadPhoto(mHelperNames.get(item).toString()); } }); AlertDialog alert = builder.create(); alert.show(); } public void loadPhoto(String imagepath){ Dialog dialog = new Dialog(this); dialog.setContentView(R.layout.activity_show_image); dialog.setTitle("Image"); dialog.setCancelable(true); ImageView img = (ImageView) dialog.findViewById(R.id.imageView1); img.setImageResource(R.drawable.ico_partial); Uri imgUri = Uri.parse(imagepath); img.setImageURI(imgUri); dialog.show(); }

    Read the article

  • Element.appendChild() hosed in IE .. workaround? (related to innerText vs textContent)

    - by Rowe Morehouse
    I've heard that using el.innerText||el.textContent can yield unreliable cross-browswer results, so I'm walking the DOM tree to collect text nodes recursively, and write them into tags in the HTML body. What this script does is read hash substring valus from the window.location and write them into the HTML. This script is working for me in Chrome & Firefox, but choking in IE. I call the page with an URL syntax like this: http://example.com/pagename.html#dyntext=FOO&dynterm=BAR&dynimage=FRED UPDATE UPDATE UPDATE Solution: I moved the scripts to before </body> (where they should have been) then removed console.log(sPageURL); and now it's working in Chrome, Firefox, IE8 and IE9. This my workaround for the innerText vs textContent crossbrowser issue when you are just placing text rather than getting text. In this case, getting hash substring values from the window.location and writing them into the page. <html> <body> <span id="dyntext-span" style="font-weight: bold;"></span><br /> <span id="dynterm-span" style="font-style: italic;"></span><br /> <span id="dynimage-span" style="text-decoration: underline;"></span><br /> <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8/jquery.min.js"></script> <script> $(document).ready(function() { var tags = ["dyntext", "dynterm", "dynimage"]; for (var i = 0; i < tags.length; ++i) { var param = GetURLParameter(tags[i]); if (param) { var dyntext = GetURLParameter('dyntext'); var dynterm = GetURLParameter('dynterm'); var dynimage = GetURLParameter('dynimage'); } } var elem = document.getElementById("dyntext-span"); var text = document.createTextNode(dyntext); elem.appendChild(text); var elem = document.getElementById("dynterm-span"); var text = document.createTextNode(dynterm); elem.appendChild(text); var elem = document.getElementById("dynimage-span"); var text = document.createTextNode(dynimage); elem.appendChild(text); }); function GetURLParameter(sParam) { var sPageURL = window.location.hash.substring(1); var sURLVariables = sPageURL.split('&'); for (var i = 0; i < sURLVariables.length; i++) { var sParameterName = sURLVariables[i].split('='); if (sParameterName[0] == sParam) { return sParameterName[1]; } } } </script> </body> </html> FINAL UPDATE If your hash substring values require spaces (like a linguistic phrase with three words, for example) then separate the words with the + character in your URI, and replace the unicode \u002B character with a space when you create each text node, like this: var elem = document.getElementById("dyntext-span"); var text = document.createTextNode(dyntext.replace(/\u002B/g, " ")); elem.appendChild(text); var elem = document.getElementById("dynterm-span"); var text = document.createTextNode(dynterm.replace(/\u002B/g, " ")); elem.appendChild(text); var elem = document.getElementById("dynimage-span"); var text = document.createTextNode(dynimage.replace(/\u002B/g, " ")); elem.appendChild(text); Now form your URI like this: http://example.com/pagename.html#dyntext=FOO+MAN+CHU&dynterm=BAR+HOPPING&dynimage=FRED+IS+DEAD

    Read the article

  • Launching an Activity from a Service

    - by nldev
    When I am trying to launch a call activity from a Service, I get a NullPointerException. Here is my code: Intent callIntent = new Intent(Intent.ACTION_CALL); callIntent.setData(Uri.parse("tel:" + number)); callIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK); startActivity(callIntent); I get the exception on the startActivity line. I tried to use getApplication.startActivity and getApplicationContext.startActivity but no luck. Any ideas? edit : Maybe some usefull info: I am trying to create a service that will run on the background and scan sensor data, when a certain signal is given i would like to maken an automated call to a number. edit : full adb error code: 03-31 09:04:10.214: ERROR/AndroidRuntime(1896): Uncaught handler: thread main exiting due to uncaught exception 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): java.lang.RuntimeException: Unable to instantiate service dfz.epilepsiedetector.services.DetectionService: java.lang.NullPointerException 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.app.ActivityThread.handleCreateService(ActivityThread.java:2668) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.app.ActivityThread.access$3100(ActivityThread.java:116) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1846) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.os.Handler.dispatchMessage(Handler.java:99) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.os.Looper.loop(Looper.java:123) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.app.ActivityThread.main(ActivityThread.java:4203) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at java.lang.reflect.Method.invokeNative(Native Method) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at java.lang.reflect.Method.invoke(Method.java:521) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:791) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:549) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at dalvik.system.NativeStart.main(Native Method) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): Caused by: java.lang.NullPointerException 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.content.ContextWrapper.getPackageName(ContextWrapper.java:120) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.content.ComponentName.<init>(ComponentName.java:75) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.content.Intent.<init>(Intent.java:2302) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at dfz.epilepsiedetector.services.DetectionService.<init>(DetectionService.java:35) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at java.lang.Class.newInstanceImpl(Native Method) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at java.lang.Class.newInstance(Class.java:1472) 03-31 09:04:10.226: ERROR/AndroidRuntime(1896): at android.app.ActivityThread.handleCreateService(ActivityThread.java:2665) edit Trimmed class code: `public class DetectionService extends IntentService implements SensorEventListener { private SensorManager mSensorManager; private Sensor mAccelerometer; private boolean hasSeizure = false; private final int POLLS_PER_SECOND = 10; public DetectionService() { super("EpilepsionDetectionService"); //mSensorManager = (SensorManager)getSystemService(SENSOR_SERVICE); // mAccelerometer = mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER); /*Intent callIntent = new Intent(Intent.ACTION_CALL); callIntent.setData(Uri.parse("tel:" + "+31648363944")); callIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);*/ Intent callIntent = new Intent(DetectionService.this, InformationActivity.class); callIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK); getApplication().startActivity(callIntent); //((Activity) getContext()).startActivity(callIntent); }`

    Read the article

  • Java -Android. Parser problem

    - by Kano
    I am making a very simple app with an RSS reader. The reader works great, but it's only giving me the title, and i want the description too. I'am very new to android, and I have tried a lot of things, but I can't get it to work. I've found a lot of parsers but they are to complicated for me to understand, so I was hoping to find a simple solution, since it's only title and description i want. Can anyone help me? import java.io.IOException; import java.net.MalformedURLException; import java.net.URL; import javax.xml.parsers.ParserConfigurationException; import javax.xml.parsers.SAXParser; import javax.xml.parsers.SAXParserFactory; import org.xml.sax.Attributes; import org.xml.sax.InputSource; import org.xml.sax.SAXException; import org.xml.sax.XMLReader; import org.xml.sax.helpers.DefaultHandler; import android.app.Activity; import android.os.Bundle; import android.widget.TextView; public class NyhedActivity extends Activity { String streamTitle = ""; @Override protected void onCreate(Bundle savedInstanceState) { // TODO Auto-generated method stub super.onCreate(savedInstanceState); setContentView(R.layout.nyheder); TextView result = (TextView)findViewById(R.id.result); try { URL rssUrl = new URL("http://tv2sport.dk/rss/*/*/*/248/*/*"); SAXParserFactory mySAXParserFactory = SAXParserFactory.newInstance(); SAXParser mySAXParser = mySAXParserFactory.newSAXParser(); XMLReader myXMLReader = mySAXParser.getXMLReader(); RSSHandler myRSSHandler = new RSSHandler(); myXMLReader.setContentHandler(myRSSHandler); InputSource myInputSource = new InputSource(rssUrl.openStream()); myXMLReader.parse(myInputSource); result.setText(streamTitle); } catch (MalformedURLException e) { // TODO Auto-generated catch block e.printStackTrace(); result.setText("Cannot connect RSS!"); } catch (ParserConfigurationException e) { // TODO Auto-generated catch block e.printStackTrace(); result.setText("Cannot connect RSS!"); } catch (SAXException e) { // TODO Auto-generated catch block e.printStackTrace(); result.setText("Cannot connect RSS!"); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); result.setText("Cannot connect RSS!"); } } private class RSSHandler extends DefaultHandler { final int stateUnknown = 0; final int stateTitle = 1; int state = stateUnknown; int numberOfTitle = 0; String strTitle = ""; String strElement = ""; @Override public void startDocument() throws SAXException { // TODO Auto-generated method stub strTitle = "Nyheder fra "; } @Override public void endDocument() throws SAXException { // TODO Auto-generated method stub strTitle += ""; streamTitle = "" + strTitle; } @Override public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException { // TODO Auto-generated method stub if (localName.equalsIgnoreCase("title")) { state = stateTitle; strElement = ""; numberOfTitle++; } else { state = stateUnknown; } } @Override public void endElement(String uri, String localName, String qName) throws SAXException { // TODO Auto-generated method stub if (localName.equalsIgnoreCase("title")) { strTitle += strElement + "\n"+"\n"; } state = stateUnknown; } @Override public void characters(char[] ch, int start, int length) throws SAXException { // TODO Auto-generated method stub String strCharacters = new String(ch, start, length); if (state == stateTitle) { strElement += strCharacters; } } } }

    Read the article

  • JSF - Random Number using Beans (JAVA)

    - by Alex Encore Tr
    I am trying to create a jsf application which, upon page refresh increments the hit counter and generates two random numbers. What should be displayed on the window may look something like this: On your On your roll x you have thrown x and x For this program I decided to create two Beans, one to hold the page refresh counter and one to generate a random number. Those look like this for the moment: CounterBean.java package diceroll; public class CounterBean { int count=0; public CounterBean() { } public void setCount(int count) { this.count=count; } public int getCount() { count++; return count; } } RandomNumberBean.java package diceroll; import java.util.Random; public class RandomNumberBean { int rand=0; Random r = new Random(); public RandomNumberBean() { rand = r.nextInt(6); } public void setNextInt(int rand) { this.rand=rand; } public int getNextInt() { return rand; } } I have then created an index.jsp to display the above message. <html> <%@ taglib uri="http://java.sun.com/jsf/core" prefix="f"%> <%@ taglib uri="http://java.sun.com/jsf/html" prefix="h"%> <f:view> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <title>Roll the Dice</title> </head> <body> <h:form> <p> On your roll # <h:outputText value="#{CounterBean.count} " /> you have thrown <h:outputText value="#{RandomNumberBean.rand}" />and <h:outputText value="#{RandomNumberBean.rand} " /> </p> </h:form> </body> </f:view> </html> However, when I run the application, I get the following message: org.apache.jasper.el.JspPropertyNotFoundException: /index.jsp(14,20) '#{RandomNumberBean.rand}' Property 'rand' not found on type diceroll.RandomNumberBean Caused by: org.apache.jasper.el.JspPropertyNotFoundException - /index.jsp(14,20) '#{RandomNumberBean.rand}' Property 'rand' not found on type diceroll.RandomNumberBean I suppose there's a mistake with my faces-config.xml file, so I will post this here as well, see if somebody can provide some help: faces-config.xml <?xml version="1.0" encoding="UTF-8"?> <faces-config xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-facesconfig_2_0.xsd" version="2.0"> <managed-bean> <managed-bean-name>CounterBean</managed-bean-name> <managed-bean-class>diceroll.CounterBean</managed-bean-class> <managed-bean-scope>session</managed-bean-scope> </managed-bean> <managed-bean> <managed-bean-name>RandomNumberBean</managed-bean-name> <managed-bean-class>diceroll.RandomNumberBean</managed-bean-class> <managed-bean-scope>session</managed-bean-scope> </managed-bean> </faces-config>

    Read the article

  • Azure - Part 4 - Table Storage Service in Windows Azure

    - by Shaun
    In Windows Azure platform there are 3 storage we can use to save our data on the cloud. They are the Table, Blob and Queue. Before the Chinese New Year Microsoft announced that Azure SDK 1.1 had been released and it supports a new type of storage – Drive, which allows us to operate NTFS files on the cloud. I will cover it in the coming few posts but now I would like to talk a bit about the Table Storage.   Concept of Table Storage Service The most common development scenario is to retrieve, create, update and remove data from the data storage. In the normal way we communicate with database. When we attempt to move our application over to the cloud the most common requirement should be have a storage service. Windows Azure provides a in-build service that allow us to storage the structured data, which is called Windows Azure Table Storage Service. The data stored in the table service are like the collection of entities. And the entities are similar to rows or records in the tradtional database. An entity should had a partition key, a row key, a timestamp and set of properties. You can treat the partition key as a group name, the row key as a primary key and the timestamp as the identifer for solving the concurrency problem. Different with a table in a database, the table service does not enforce the schema for tables, which means you can have 2 entities in the same table with different property sets. The partition key is being used for the load balance of the Azure OS and the group entity transaction. As you know in the cloud you will never know which machine is hosting your application and your data. It could be moving based on the transaction weight and the number of the requests. If the Azure OS found that there are many requests connect to your Book entities with the partition key equals “Novel” it will move them to another idle machine to increase the performance. So when choosing the partition key for your entities you need to make sure they indecate the category or gourp information so that the Azure OS can perform the load balance as you wish.   Consuming the Table Although the table service looks like a database, you cannot access it through the way you are using now, neither ADO.NET nor ODBC. The table service exposed itself by ADO.NET Data Service protocol, which allows you can consume it through the RESTful style by Http requests. The Azure SDK provides a sets of classes for us to connect it. There are 2 classes we might need: TableServiceContext and TableServiceEntity. The TableServiceContext inherited from the DataServiceContext, which represents the runtime context of the ADO.NET data service. It provides 4 methods mainly used by us: CreateQuery: It will create a IQueryable instance from a given type of entity. AddObject: Add the specified entity into Table Service. UpdateObject: Update an existing entity in the Table Service. DeleteObject: Delete an entity from the Table Service. Beofre you operate the table service you need to provide the valid account information. It’s something like the connect string of the database but with your account name and the account key when you created the storage service on the Windows Azure Development Portal. After getting the CloudStorageAccount you can create the CloudTableClient instance which provides a set of methods for using the table service. A very useful method would be CreateTableIfNotExist. It will create the table container for you if it’s not exsited. And then you can operate the eneities to that table through the methods I mentioned above. Let me explain a bit more through an exmaple. We always like code rather than sentence.   Straightforward Accessing to the Table Here I would like to build a WCF service on the Windows Azure platform, and for now just one requirement: it would allow the client to create an account entity on the table service. The WCF service would have a method named Register and accept an instance of the account which the client wants to create. After perform some validation it will add the entity into the table service. So the first thing I should do is to create a Cloud Application on my VIstial Studio 2010 RC. (The Azure SDK 1.1 only supports VS2008 and VS2010 RC.) The solution should be like this below. Then I added a configuration items for the storage account through the Settings section under the cloud project. (Double click the Services file under Roles folder and navigate to the Setting section.) This setting will be used when to retrieve my storage account information. Since for now I just in the development phase I will select “UseDevelopmentStorage=true”. And then I navigated to the WebRole.cs file under my WCF project. If you have read my previous posts you would know that this file defines the process when the application start, and terminate on the cloud. What I need to do is to when the application start, set the configuration publisher to load my config file with the config name I specified. So the code would be like below. I removed the original service and contract created by the VS template and add my IAccountService contract and its implementation class - AccountService. And I add the service method Register with the parameters: email, password and it will return a boolean value to indicates the result which is very simple. At this moment if I press F5 the application will be established on my local development fabric and I can see my service runs well through the browser. Let’s implement the service method Rigister, add a new entity to the table service. As I said before the entities you want to store in the table service must have 3 properties: partition key, row key and timespan. You can create a class with these 3 properties. The Azure SDK provides us a base class for that named TableServiceEntity in Microsoft.WindowsAzure.StorageClient namespace. So what we need to do is more simply, create a class named Account and let it derived from the TableServiceEntity. And I need to add my own properties: Email, Password, DateCreated and DateDeleted. The DateDeleted is a nullable date time value to indecate whether this entity had been deleted and when. Do you notice that I missed something here? Yes it’s the partition key and row key I didn’t assigned. The TableServiceEntity base class defined 2 constructors one was a parameter-less constructor which will be used to fill values into the properties from the table service when retrieving data. The other was one with 2 parameters: partition key and row key. As I said below the partition key may affect the load balance and the row key must be unique so here I would like to use the email as the parition key and the email plus a Guid as the row key. OK now we finished the entity class we need to store onto the table service. The next step is to create a data access class for us to add it. Azure SDK gives us a base class for it named TableServiceContext as I mentioned below. So let’s create a class for operate the Account entities. The TableServiceContext need the storage account information for its constructor. It’s the combination of the storage service URI that we will create on Windows Azure platform, and the relevant account name and key. The TableServiceContext will use this information to find the related address and verify the account to operate the storage entities. Hence in my AccountDataContext class I need to override this constructor and pass the storage account into it. All entities will be saved in the table storage with one or many tables which we call them “table containers”. Before we operate an entity we need to make sure that the table container had been created on the storage. There’s a method we can use for that: CloudTableClient.CreateTableIfNotExist. So in the constructor I will perform it firstly to make sure all method will be invoked after the table had been created. Notice that I passed the storage account enpoint URI and the credentials to specify where my storage is located and who am I. Another advise is that, make your entity class name as the same as the table name when create the table. It will increase the performance when you operate it over the cloud especially querying. Since the Register WCF method will add a new account into the table service, here I will create a relevant method to add the account entity. Before implement, I should add a reference - System.Data.Services.Client to the project. This reference provides some common method within the ADO.NET Data Service which can be used in the Windows Azure Table Service. I will use its AddObject method to create my account entity. Since the table service are not fully implemented the ADO.NET Data Service, there are some methods in the System.Data.Services.Client that TableServiceContext doesn’t support, such as AddLinks, etc. Then I implemented the serivce method to add the account entity through the AccountDataContext. You can see in the service implmentation I load the storage account information through my configuration file and created the account table entity from the parameters. Then I created the AccountDataContext. If it’s my first time to invoke this method the constructor of the AccountDataContext will create a table container for me. Then I use Add method to add the account entity into the table. Next, let’s create a farely simple client application to test this service. I created a windows console application and added a service reference to my WCF service. The metadata information of the WCF service cannot be retrieved if it’s deployed on the Windows Azure even though the <serviceMetadata httpGetEnabled="true"/> had been set. If we need to get its metadata we can deploy it on the local development service and then changed the endpoint to the address which is on the cloud. In the client side app.config file I specified the endpoint to the local development fabric address. And the just implement the client to let me input an email and a password then invoke the WCF service to add my acocunt. Let’s run my application and see the result. Of course it should return TRUE to me. And in the local SQL Express I can see the data had been saved in the table.   Summary In this post I explained more about the Windows Azure Table Storage Service. I also created a small application for demostration of how to connect and consume it through the ADO.NET Data Service Managed Library provided within the Azure SDK. I only show how to create an eneity in the storage service. In the next post I would like to explain about how to query the entities with conditions thruogh LINQ. I also would like to refactor my AccountDataContext class to make it dyamic for any kinds of entities.   Hope this helps, Shaun   All documents and related graphics, codes are provided "AS IS" without warranty of any kind. Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

    Read the article

  • Firefox for NTLM secured sites

    - by Sarang
    Spent the last weekend fighting to get firefox to connect to a sharepoint portal hosting my homegrown TFS instance's TFS/WEB and team project portal from a friends' place. Firefox is THE favourite browser and I was hating to se it fail miserably with NTLM authentication. Fun part is it showed the login prompt accepted credentials and like a pestering young puppy came back for the same credentials. After banging my head and various "I don't know what I don't know" attempts I decided to play god and entered the firefox's advance config mode. And voila! there sits a nifty little option called network.automatic-ntlm-auth.trusted-uris. Assign your URI to it and you are through. No more shameface before those Chrome/Opera users. :). To enter FireFox's god mode, open a new tab and type about:config in the address bar. There is a search box which certainly comes handy shifting through hundreds of options. The root lies in firefox's default mechanism of not allowing NTLM passthrough authentication. Firefox defaults to digest credentials which are blatantly refused by a web app expecting NTLM which results in authentication failure and Firefox keeps asking for credentials over and over again. The steps listed above are same as adding a website to your trusted sites' list in Internet Explorer.

    Read the article

  • How to call Office365 web service in a Console application using WCF

    - by ybbest
    In my previous post, I showed you how to call the SharePoint web service using a console application. In this post, I’d like to show you how to call the same web service in the cloud, aka Office365.In office365, it uses claims authentication as opposed to windows authentication for normal in-house SharePoint Deployment. For Details of the explanation you can see Wictor’s post on this here. The key to make it work is to understand when you authenticate from Office365, you get your authentication token. You then need to pass this token to your HTTP request as cookie to make the web service call. Here is the code sample to make it work.I have modified Wictor’s by removing the client object references. static void Main(string[] args) { MsOnlineClaimsHelper claimsHelper = new MsOnlineClaimsHelper( "[email protected]", "YourPassword","https://ybbest.sharepoint.com/"); HttpRequestMessageProperty p = new HttpRequestMessageProperty(); var cookie = claimsHelper.CookieContainer; string cookieHeader = cookie.GetCookieHeader(new Uri("https://ybbest.sharepoint.com/")); p.Headers.Add("Cookie", cookieHeader); using (ListsSoapClient proxy = new ListsSoapClient()) { proxy.Endpoint.Address = new EndpointAddress("https://ybbest.sharepoint.com/_vti_bin/Lists.asmx"); using (new OperationContextScope(proxy.InnerChannel)) { OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = p; XElement spLists = proxy.GetListCollection(); foreach (var el in spLists.Descendants()) { //System.Console.WriteLine(el.Name); foreach (var attrib in el.Attributes()) { if (attrib.Name.LocalName.ToLower() == "title") { System.Console.WriteLine("> " + attrib.Name + " = " + attrib.Value); } } } } System.Console.ReadKey(); } } You can download the complete code from here. Reference: Managing shared cookies in WCF How to do active authentication to Office 365 and SharePoint Online

    Read the article

  • How to work RavenDB Id with ASP.NET MVC Routes

    - by shiju
    By default RavenDB's Id would be sperated by "/". Let's say that we have a category object, the Ids would be like "categories/1". This will make problems when working with ASP.NET MVC's route rule. For a route category/edit/id, the uri would be category/edit/categories/1. You can solve this problem in two waysSolution 1 - Change Id SeparatorWe can use different Id Separator for RavenDB Ids in order to working with ASP.NET MVC route rules. The following code specify that Ids would be seperated by "-" rather than the default "/"  documentStore = new DocumentStore { Url = "http://localhost:8080/" };  documentStore.Initialize();  documentStore.Conventions.IdentityPartsSeparator = "-"; The above IdentityPartsSeparator would be generate Ids like "categories-1"Solution 2 - Modify ASP.NET MVC Route Modify the ASP.NET MVC routes in the Global.asax.cs file, as shown in the following code  routes.MapRoute(     "WithParam",                                           // Route name     "{controller}/{action}/{*id}"                         // URL with parameters     );  We just put "*" in front of the id variable that will be working with the default Id separator of RavenDB

    Read the article

  • Restful Services, oData, and Rest Sharp

    - by jkrebsbach
    After a great presentation by Jason Sheehan at MDC about RestSharp, I decided to implement it. RestSharp is a .Net framework for consuming restful data sources via either Json or XML. My first step was to put together a Restful data source for RestSharp to consume.  Staying entirely withing .Net, I decided to use Microsoft's oData implementation, built on System.Data.Services.DataServices.  Natively, these support Json, or atom+pub xml.  (XML with a few bells and whistles added on) There are three main steps for creating an oData data source: 1)  override CreateDSPMetaData This is where the metadata data is returned.  The meta data defines the structure of the data to return.  The structure contains the relationships between data objects, along with what properties the objects expose.  The meta data can and should be somehow cached so that the structure is not rebuild with every data request. 2) override CreateDataSource The context contains the data the data source will publish.  This method is the conduit which will populate the metadata objects to be returned to the requestor. 3) implement static InitializeService At this point we can set up security, along with setting up properties of the web service (versioning, etc)   Here is a web service which publishes stock prices for various Products (stocks) in various Categories. namespace RestService {     public class RestServiceImpl : DSPDataService<DSPContext>     {         private static DSPContext _context;         private static DSPMetadata _metadata;         /// <summary>         /// Populate traversable data source         /// </summary>         /// <returns></returns>         protected override DSPContext CreateDataSource()         {             if (_context == null)             {                 _context = new DSPContext();                 Category utilities = new Category(0);                 utilities.Name = "Electric";                 Category financials = new Category(1);                 financials.Name = "Financial";                                 IList products = _context.GetResourceSetEntities("Products");                 Product electric = new Product(0, utilities);                 electric.Name = "ABC Electric";                 electric.Description = "Electric Utility";                 electric.Price = 3.5;                 products.Add(electric);                 Product water = new Product(1, utilities);                 water.Name = "XYZ Water";                 water.Description = "Water Utility";                 water.Price = 2.4;                 products.Add(water);                 Product banks = new Product(2, financials);                 banks.Name = "FatCat Bank";                 banks.Description = "A bank that's almost too big";                 banks.Price = 19.9; // This will never get to the client                 products.Add(banks);                 IList categories = _context.GetResourceSetEntities("Categories");                 categories.Add(utilities);                 categories.Add(financials);                 utilities.Products.Add(electric);                 utilities.Products.Add(electric);                 financials.Products.Add(banks);             }             return _context;         }         /// <summary>         /// Setup rules describing published data structure - relationships between data,         /// key field, other searchable fields, etc.         /// </summary>         /// <returns></returns>         protected override DSPMetadata CreateDSPMetadata()         {             if (_metadata == null)             {                 _metadata = new DSPMetadata("DemoService", "DataServiceProviderDemo");                 // Define entity type product                 ResourceType product = _metadata.AddEntityType(typeof(Product), "Product");                 _metadata.AddKeyProperty(product, "ProductID");                 // Only add properties we wish to share with end users                 _metadata.AddPrimitiveProperty(product, "Name");                 _metadata.AddPrimitiveProperty(product, "Description");                 EntityPropertyMappingAttribute att = new EntityPropertyMappingAttribute("Name",                     SyndicationItemProperty.Title, SyndicationTextContentKind.Plaintext, true);                 product.AddEntityPropertyMappingAttribute(att);                 att = new EntityPropertyMappingAttribute("Description",                     SyndicationItemProperty.Summary, SyndicationTextContentKind.Plaintext, true);                 product.AddEntityPropertyMappingAttribute(att);                 // Define products as a set of product entities                 ResourceSet products = _metadata.AddResourceSet("Products", product);                 // Define entity type category                 ResourceType category = _metadata.AddEntityType(typeof(Category), "Category");                 _metadata.AddKeyProperty(category, "CategoryID");                 _metadata.AddPrimitiveProperty(category, "Name");                 _metadata.AddPrimitiveProperty(category, "Description");                 // Define categories as a set of category entities                 ResourceSet categories = _metadata.AddResourceSet("Categories", category);                 att = new EntityPropertyMappingAttribute("Name",                     SyndicationItemProperty.Title, SyndicationTextContentKind.Plaintext, true);                 category.AddEntityPropertyMappingAttribute(att);                 att = new EntityPropertyMappingAttribute("Description",                     SyndicationItemProperty.Summary, SyndicationTextContentKind.Plaintext, true);                 category.AddEntityPropertyMappingAttribute(att);                 // A product has a category, a category has products                 _metadata.AddResourceReferenceProperty(product, "Category", categories);                 _metadata.AddResourceSetReferenceProperty(category, "Products", products);             }             return _metadata;         }         /// <summary>         /// Based on the requesting user, can set up permissions to Read, Write, etc.         /// </summary>         /// <param name="config"></param>         public static void InitializeService(DataServiceConfiguration config)         {             config.SetEntitySetAccessRule("*", EntitySetRights.All);             config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;             config.DataServiceBehavior.AcceptProjectionRequests = true;         }     } }     The objects prefixed with DSP come from the samples on the oData site: http://www.odata.org/developers The products and categories objects are POCO business objects with no special modifiers. Three main options are available for defining the MetaData of data sources in .Net: 1) Generate Entity Data model (Potentially directly from SQL Server database).  This requires the least amount of manual interaction, and uses the edmx WYSIWYG editor to generate a data model.  This can be directly tied to the SQL Server database and generated from the database if you want a data access layer tightly coupled with your database. 2) Object model decorations.  If you already have a POCO data layer, you can decorate your objects with properties to statically inform the compiler how the objects are related.  The disadvantage is there are now tags strewn about your business layer that need to be updated as the business rules change.  3) Programmatically construct metadata object.  This is the object illustrated above in CreateDSPMetaData.  This puts all relationship information into one central programmatic location.  Here business rules are constructed when the DSPMetaData response object is returned.   Once you have your service up and running, RestSharp is designed for XML / Json, along with the native Microsoft library.  There are currently some differences between how Jason made RestSharp expect XML with how atom+pub works, so I found better results currently with the Json implementation - modifying the RestSharp XML parser to make an atom+pub parser is fairly trivial though, so use what implementation works best for you. I put together a sample console app which calls the RestSvcImpl.svc service defined above (and assumes it to be running on port 2000).  I used both RestSharp as a client, and also the default Microsoft oData client tools. namespace RestConsole {     class Program     {         private static DataServiceContext _ctx;         private enum DemoType         {             Xml,             Json         }         static void Main(string[] args)         {             // Microsoft implementation             _ctx = new DataServiceContext(new System.Uri("http://localhost:2000/RestServiceImpl.svc"));             var msProducts = RunQuery<Product>("Products").ToList();             var msCategory = RunQuery<Category>("/Products(0)/Category").AsEnumerable().Single();             var msFilteredProducts = RunQuery<Product>("/Products?$filter=length(Name) ge 4").ToList();             // RestSharp implementation                          DemoType demoType = DemoType.Json;             var client = new RestClient("http://localhost:2000/RestServiceImpl.svc");             client.ClearHandlers(); // Remove all available handlers             // Set up handler depending on what situation dictates             if (demoType == DemoType.Json)                 client.AddHandler("application/json", new RestSharp.Deserializers.JsonDeserializer());             else if (demoType == DemoType.Xml)             {                 client.AddHandler("application/atom+xml", new RestSharp.Deserializers.XmlDeserializer());             }                          var request = new RestRequest();             if (demoType == DemoType.Json)                 request.RootElement = "d"; // service root element for json             else if (demoType == DemoType.Xml)             {                 request.XmlNamespace = "http://www.w3.org/2005/Atom";             }                              // Return all products             request.Resource = "/Products?$orderby=Name";             RestResponse<List<Product>> productsResp = client.Execute<List<Product>>(request);             List<Product> products = productsResp.Data;             // Find category for product with ProductID = 1             request.Resource = string.Format("/Products(1)/Category");             RestResponse<Category> categoryResp = client.Execute<Category>(request);             Category category = categoryResp.Data;             // Specialized queries             request.Resource = string.Format("/Products?$filter=ProductID eq {0}", 1);             RestResponse<Product> productResp = client.Execute<Product>(request);             Product product = productResp.Data;                          request.Resource = string.Format("/Products?$filter=Name eq '{0}'", "XYZ Water");             productResp = client.Execute<Product>(request);             product = productResp.Data;         }         private static IEnumerable<TElement> RunQuery<TElement>(string queryUri)         {             try             {                 return _ctx.Execute<TElement>(new Uri(queryUri, UriKind.Relative));             }             catch (Exception ex)             {                 throw ex;             }         }              } }   Feel free to step through the code a few times and to attach a debugger to the service as well to see how and where the context and metadata objects are constructed and returned.  Pay special attention to the response object being returned by the oData service - There are several properties of the RestRequest that can be used to help troubleshoot when the structure of the response is not exactly what would be expected.

    Read the article

  • How to call Office365 web service in a Console application using WCF

    - by ybbest
    In my previous post, I showed you how to call the SharePoint web service using a console application. In this post, I’d like to show you how to call the same web service in the cloud, aka Office365.In office365, it uses claims authentication as opposed to windows authentication for normal in-house SharePoint Deployment. For Details of the explanation you can see Wictor’s post on this here. The key to make it work is to understand when you authenticate from Office365, you get your authentication token. You then need to pass this token to your HTTP request as cookie to make the web service call. Here is the code sample to make it work.I have modified Wictor’s by removing the client object references. static void Main(string[] args) { MsOnlineClaimsHelper claimsHelper = new MsOnlineClaimsHelper( "[email protected]", "YourPassword","https://ybbest.sharepoint.com/"); HttpRequestMessageProperty p = new HttpRequestMessageProperty(); var cookie = claimsHelper.CookieContainer; string cookieHeader = cookie.GetCookieHeader(new Uri("https://ybbest.sharepoint.com/")); p.Headers.Add("Cookie", cookieHeader); using (ListsSoapClient proxy = new ListsSoapClient()) { proxy.Endpoint.Address = new EndpointAddress("https://ybbest.sharepoint.com/_vti_bin/Lists.asmx"); using (new OperationContextScope(proxy.InnerChannel)) { OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = p; XElement spLists = proxy.GetListCollection(); foreach (var el in spLists.Descendants()) { //System.Console.WriteLine(el.Name); foreach (var attrib in el.Attributes()) { if (attrib.Name.LocalName.ToLower() == "title") { System.Console.WriteLine("> " + attrib.Name + " = " + attrib.Value); } } } } System.Console.ReadKey(); } } You can download the complete code from here. Reference: Managing shared cookies in WCF How to do active authentication to Office 365 and SharePoint Online

    Read the article

  • Simple scan not working after upgrading to 12.10 (Xubuntu)

    - by mydoghasworms
    Since upgrading to 12.10 (Xubuntu), Simple Scan is not working anymore. I got scanning working with Xsane, but only if Simple Scan has not run before. Otherwise I have to restart the printer/scanner (HP OfficeJet J5783). In kernel.log I see: kernel: [ 1214.120964] usb 2-1.4: >usbfs: process 4412 (simple-scan) did not claim interface 2 before use and in syslog simple-scan: io/hpmud/dot4.c 172: unable to read Dot4ReverseCmd header: No data available simple-scan: io/hpmud/musb.c 1933: invalid Dot4Credit from peripheral simple-scan: io/hpmud/dot4.c 172: unable to read Dot4ReverseCmd header: No data available simple-scan: io/hpmud/musb.c 1933: invalid Dot4Credit from peripheral simple-scan: sane_hpaio_cancel: already cancelled! simple-scan: io/hpmud/dot4.c 172: unable to read Dot4ReverseCmd header: No data available simple-scan: io/hpmud/musb.c 1933: invalid Dot4Credit from peripheral simple-scan: io/hpmud/dot4.c 231: unable to read Dot4ReverseReply header: No data available bytesRead=0 simple-scan: io/hpmud/dot4.c 319: invalid DOT4InitReply retrying command... simple-scan: io/hpmud/dot4.c 172: unable to read Dot4ReverseCmd header: No data available simple-scan: io/hpmud/musb.c 1933: invalid Dot4Credit from peripheral simple-scan: io/hpmud/hpmud.c 342: device_cleanup: device uri=hp:/usb/Officejet_J5700_series?serial=CN81LCV0V604TC simple-scan: io/hpmud/hpmud.c 354: device_cleanup: close device dd=1... simple-scan: io/hpmud/hpmud.c 356: device_cleanup: done closing device dd=1 Any ideas?

    Read the article

  • Running a simple integration scenario using the Oracle Big Data Connectors on Hadoop/HDFS cluster

    - by hamsun
    Between the elephant ( the tradional image of the Hadoop framework) and the Oracle Iron Man (Big Data..) an english setter could be seen as the link to the right data Data, Data, Data, we are living in a world where data technology based on popular applications , search engines, Webservers, rich sms messages, email clients, weather forecasts and so on, have a predominant role in our life. More and more technologies are used to analyze/track our behavior, try to detect patterns, to propose us "the best/right user experience" from the Google Ad services, to Telco companies or large consumer sites (like Amazon:) ). The more we use all these technologies, the more we generate data, and thus there is a need of huge data marts and specific hardware/software servers (as the Exadata servers) in order to treat/analyze/understand the trends and offer new services to the users. Some of these "data feeds" are raw, unstructured data, and cannot be processed effectively by normal SQL queries. Large scale distributed processing was an emerging infrastructure need and the solution seemed to be the "collocation of compute nodes with the data", which in turn leaded to MapReduce parallel patterns and the development of the Hadoop framework, which is based on MapReduce and a distributed file system (HDFS) that runs on larger clusters of rather inexpensive servers. Several Oracle products are using the distributed / aggregation pattern for data calculation ( Coherence, NoSql, times ten ) so once that you are familiar with one of these technologies, lets says with coherence aggregators, you will find the whole Hadoop, MapReduce concept very similar. Oracle Big Data Appliance is based on the Cloudera Distribution (CDH), and the Oracle Big Data Connectors can be plugged on a Hadoop cluster running the CDH distribution or equivalent Hadoop clusters. In this paper, a "lab like" implementation of this concept is done on a single Linux X64 server, running an Oracle Database 11g Enterprise Edition Release 11.2.0.4.0, and a single node Apache hadoop-1.2.1 HDFS cluster, using the SQL connector for HDFS. The whole setup is fairly simple: Install on a Linux x64 server ( or virtual box appliance) an Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 server Get the Apache Hadoop distribution from: http://mir2.ovh.net/ftp.apache.org/dist/hadoop/common/hadoop-1.2.1. Get the Oracle Big Data Connectors from: http://www.oracle.com/technetwork/bdc/big-data-connectors/downloads/index.html?ssSourceSiteId=ocomen. Check the java version of your Linux server with the command: java -version java version "1.7.0_40" Java(TM) SE Runtime Environment (build 1.7.0_40-b43) Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode) Decompress the hadoop hadoop-1.2.1.tar.gz file to /u01/hadoop-1.2.1 Modify your .bash_profile export HADOOP_HOME=/u01/hadoop-1.2.1 export PATH=$PATH:$HADOOP_HOME/bin export HIVE_HOME=/u01/hive-0.11.0 export PATH=$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin (also see my sample .bash_profile) Set up ssh trust for Hadoop process, this is a mandatory step, in our case we have to establish a "local trust" as will are using a single node configuration copy the new public keys to the list of authorized keys connect and test the ssh setup to your localhost: We will run a "pseudo-Hadoop cluster", in what is called "local standalone mode", all the Hadoop java components are running in one Java process, this is enough for our demo purposes. We need to "fine tune" some Hadoop configuration files, we have to go at our $HADOOP_HOME/conf, and modify the files: core-site.xml hdfs-site.xml mapred-site.xml check that the hadoop binaries are referenced correctly from the command line by executing: hadoop -version As Hadoop is managing our "clustered HDFS" file system we have to create "the mount point" and format it , the mount point will be declared to core-site.xml as: The layout under the /u01/hadoop-1.2.1/data will be created and used by other hadoop components (MapReduce = /mapred/...) HDFS is using the /dfs/... layout structure format the HDFS hadoop file system: Start the java components for the HDFS system As an additional check, you can use the GUI Hadoop browsers to check the content of your HDFS configurations: Once our HDFS Hadoop setup is done you can use the HDFS file system to store data ( big data : )), and plug them back and forth to Oracle Databases by the means of the Big Data Connectors ( which is the next configuration step). You can create / use a Hive db, but in our case we will make a simple integration of "raw data" , through the creation of an External Table to a local Oracle instance ( on the same Linux box, we run the Hadoop HDFS one node cluster and one Oracle DB). Download some public "big data", I use the site: http://france.meteofrance.com/france/observations, from where I can get *.csv files for my big data simulations :). Here is the data layout of my example file: Download the Big Data Connector from the OTN (oraosch-2.2.0.zip), unzip it to your local file system (see picture below) Modify your environment in order to access the connector libraries , and make the following test: [oracle@dg1 bin]$./hdfs_stream Usage: hdfs_stream locationFile [oracle@dg1 bin]$ Load the data to the Hadoop hdfs file system: hadoop fs -mkdir bgtest_data hadoop fs -put obsFrance.txt bgtest_data/obsFrance.txt hadoop fs -ls /user/oracle/bgtest_data/obsFrance.txt [oracle@dg1 bg-data-raw]$ hadoop fs -ls /user/oracle/bgtest_data/obsFrance.txt Found 1 items -rw-r--r-- 1 oracle supergroup 54103 2013-10-22 06:10 /user/oracle/bgtest_data/obsFrance.txt [oracle@dg1 bg-data-raw]$hadoop fs -ls hdfs:///user/oracle/bgtest_data/obsFrance.txt Found 1 items -rw-r--r-- 1 oracle supergroup 54103 2013-10-22 06:10 /user/oracle/bgtest_data/obsFrance.txt Check the content of the HDFS with the browser UI: Start the Oracle database, and run the following script in order to create the Oracle database user, the Oracle directories for the Oracle Big Data Connector (dg1 it’s my own db id replace accordingly yours): #!/bin/bash export ORAENV_ASK=NO export ORACLE_SID=dg1 . oraenv sqlplus /nolog <<EOF CONNECT / AS sysdba; CREATE OR REPLACE DIRECTORY osch_bin_path AS '/u01/orahdfs-2.2.0/bin'; CREATE USER BGUSER IDENTIFIED BY oracle; GRANT CREATE SESSION, CREATE TABLE TO BGUSER; GRANT EXECUTE ON sys.utl_file TO BGUSER; GRANT READ, EXECUTE ON DIRECTORY osch_bin_path TO BGUSER; CREATE OR REPLACE DIRECTORY BGT_LOG_DIR as '/u01/BG_TEST/logs'; GRANT READ, WRITE ON DIRECTORY BGT_LOG_DIR to BGUSER; CREATE OR REPLACE DIRECTORY BGT_DATA_DIR as '/u01/BG_TEST/data'; GRANT READ, WRITE ON DIRECTORY BGT_DATA_DIR to BGUSER; EOF Put the following in a file named t3.sh and make it executable, hadoop jar $OSCH_HOME/jlib/orahdfs.jar \ oracle.hadoop.exttab.ExternalTable \ -D oracle.hadoop.exttab.tableName=BGTEST_DP_XTAB \ -D oracle.hadoop.exttab.defaultDirectory=BGT_DATA_DIR \ -D oracle.hadoop.exttab.dataPaths="hdfs:///user/oracle/bgtest_data/obsFrance.txt" \ -D oracle.hadoop.exttab.columnCount=7 \ -D oracle.hadoop.connection.url=jdbc:oracle:thin:@//localhost:1521/dg1 \ -D oracle.hadoop.connection.user=BGUSER \ -D oracle.hadoop.exttab.printStackTrace=true \ -createTable --noexecute then test the creation fo the external table with it: [oracle@dg1 samples]$ ./t3.sh ./t3.sh: line 2: /u01/orahdfs-2.2.0: Is a directory Oracle SQL Connector for HDFS Release 2.2.0 - Production Copyright (c) 2011, 2013, Oracle and/or its affiliates. All rights reserved. Enter Database Password:] The create table command was not executed. The following table would be created. CREATE TABLE "BGUSER"."BGTEST_DP_XTAB" ( "C1" VARCHAR2(4000), "C2" VARCHAR2(4000), "C3" VARCHAR2(4000), "C4" VARCHAR2(4000), "C5" VARCHAR2(4000), "C6" VARCHAR2(4000), "C7" VARCHAR2(4000) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY "BGT_DATA_DIR" ACCESS PARAMETERS ( RECORDS DELIMITED BY 0X'0A' CHARACTERSET AL32UTF8 STRING SIZES ARE IN CHARACTERS PREPROCESSOR "OSCH_BIN_PATH":'hdfs_stream' FIELDS TERMINATED BY 0X'2C' MISSING FIELD VALUES ARE NULL ( "C1" CHAR(4000), "C2" CHAR(4000), "C3" CHAR(4000), "C4" CHAR(4000), "C5" CHAR(4000), "C6" CHAR(4000), "C7" CHAR(4000) ) ) LOCATION ( 'osch-20131022081035-74-1' ) ) PARALLEL REJECT LIMIT UNLIMITED; The following location files would be created. osch-20131022081035-74-1 contains 1 URI, 54103 bytes 54103 hdfs://localhost:19000/user/oracle/bgtest_data/obsFrance.txt Then remove the --noexecute flag and create the external Oracle table for the Hadoop data. Check the results: The create table command succeeded. CREATE TABLE "BGUSER"."BGTEST_DP_XTAB" ( "C1" VARCHAR2(4000), "C2" VARCHAR2(4000), "C3" VARCHAR2(4000), "C4" VARCHAR2(4000), "C5" VARCHAR2(4000), "C6" VARCHAR2(4000), "C7" VARCHAR2(4000) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY "BGT_DATA_DIR" ACCESS PARAMETERS ( RECORDS DELIMITED BY 0X'0A' CHARACTERSET AL32UTF8 STRING SIZES ARE IN CHARACTERS PREPROCESSOR "OSCH_BIN_PATH":'hdfs_stream' FIELDS TERMINATED BY 0X'2C' MISSING FIELD VALUES ARE NULL ( "C1" CHAR(4000), "C2" CHAR(4000), "C3" CHAR(4000), "C4" CHAR(4000), "C5" CHAR(4000), "C6" CHAR(4000), "C7" CHAR(4000) ) ) LOCATION ( 'osch-20131022081719-3239-1' ) ) PARALLEL REJECT LIMIT UNLIMITED; The following location files were created. osch-20131022081719-3239-1 contains 1 URI, 54103 bytes 54103 hdfs://localhost:19000/user/oracle/bgtest_data/obsFrance.txt This is the view from the SQL Developer: and finally the number of lines in the oracle table, imported from our Hadoop HDFS cluster SQL select count(*) from "BGUSER"."BGTEST_DP_XTAB"; COUNT(*) ---------- 1151 In a next post we will integrate data from a Hive database, and try some ODI integrations with the ODI Big Data connector. Our simplistic approach is just a step to show you how these unstructured data world can be integrated to Oracle infrastructure. Hadoop, BigData, NoSql are great technologies, they are widely used and Oracle is offering a large integration infrastructure based on these services. Oracle University presents a complete curriculum on all the Oracle related technologies: NoSQL: Introduction to Oracle NoSQL Database Using Oracle NoSQL Database Big Data: Introduction to Big Data Oracle Big Data Essentials Oracle Big Data Overview Oracle Data Integrator: Oracle Data Integrator 12c: New Features Oracle Data Integrator 11g: Integration and Administration Oracle Data Integrator: Administration and Development Oracle Data Integrator 11g: Advanced Integration and Development Oracle Coherence 12c: Oracle Coherence 12c: New Features Oracle Coherence 12c: Share and Manage Data in Clusters Oracle Coherence 12c: Oracle GoldenGate 11g: Fundamentals for Oracle Oracle GoldenGate 11g: Fundamentals for SQL Server Oracle GoldenGate 11g Fundamentals for Oracle Oracle GoldenGate 11g Fundamentals for DB2 Oracle GoldenGate 11g Fundamentals for Teradata Oracle GoldenGate 11g Fundamentals for HP NonStop Oracle GoldenGate 11g Management Pack: Overview Oracle GoldenGate 11g Troubleshooting and Tuning Oracle GoldenGate 11g: Advanced Configuration for Oracle Other Resources: Apache Hadoop : http://hadoop.apache.org/ is the homepage for these technologies. "Hadoop Definitive Guide 3rdEdition" by Tom White is a classical lecture for people who want to know more about Hadoop , and some active "googling " will also give you some more references. About the author: Eugene Simos is based in France and joined Oracle through the BEA-Weblogic Acquisition, where he worked for the Professional Service, Support, end Education for major accounts across the EMEA Region. He worked in the banking sector, ATT, Telco companies giving him extensive experience on production environments. Eugen currently specializes in Oracle Fusion Middleware teaching an array of courses on Weblogic/Webcenter, Content,BPM /SOA/Identity-Security/GoldenGate/Virtualisation/Unified Comm Suite) throughout the EMEA region.

    Read the article

  • CodePlex Daily Summary for Friday, April 16, 2010

    CodePlex Daily Summary for Friday, April 16, 2010New Projects[C#] UML Touch: This project is my master thesis project for Msc Software Engineering at Oxford Brookes univesity. The goal of this project is to develop a UML Edi...3D Chart Reports, using ASP.NET 2.0: Project focuses on the visibility of internal supply chain management process. The objective is materialized, using MSChart and hence provides asse...Active Directory Health: ActiveDirectoryHealth (ADHealth) Aim: To create a suite of tools to allow an Active Directory administrator monitor and identify problems, particu...Art4Desktop: Aplicação desktop simplesAuto Increment field for any entity in MS CRM: Auto Increment field for any entity in MS CRM will add an attribute in any entity which will be auto increment type.Convection Game Engine (Basic Edition): A basic 2D game engine written in C# using XNA 3.1CSharp Intellisense: VS 2010 IntelliSense plug-in Provides custom IntelliSense for the CSharp Editor that is capable to filter out events, preoperties or methods. ...EP: Generic platform for enterprise applications developed on .NETFile Change Checker: A simple windows form application that checks and copies all modified files given a specified date. It aims to help developers that use Visual Stu...Folder: Folder is a puzzle platformer game, which provides you a whole new experience. You can fold objects in game play; a wall becomes a road, a high sti...Industrial Dashboard: WCF service that allows executing SQL Server stored procedures straight from javascript code, enabling sending and receiving structured data withou...kafrilion: Open Source Platform GameLogikBug.Injection: LogikBug.Injection is an IOC container and is very light weight. It is very simple to use and yet very roboust, there are many options and ways to ...MongoMvc: NoSql with MongoDB, NoRM and ASP.NET MVC. For more details check out the blog post http://weblogs.asp.net/shijuvarghese/archive/2010/04/16/nosql-wi...MOSSDAL: MOSS Data Access Layer for data from the Sharepoint Lists Service: MOSSDAL is a lightweight framework for working with Sharepoint MOSS List data using the List web Service. It can be used with silverlight or regul...Should: This is a set of test framework agnostic extensions assetion extensions. This project was born because test runners Should be independent of the...Software Localization Tool: summaryTIMETABLEASY: planning managerTR9: implementions of t9 technologies from mobile device to personal computer and laptopTrp net tools: net toolTwep: Twep is a JavaScript lib.Using PowerPivot to analyze MS Dynamics NAV: Project show how to prepare MS Dynamics NAV data for analyzing in PowerPivot for Excel. Project include Data Warehouse demo database, sql procedure...vmware virtual mashines management system for education: System for management many classes with PC and VMware virtual mashines (bad english,sorry)WebSite: Mr. John's projectsWebtrends DX and DC Services Libraries: This project is setup to provide assemblies to simplify access to the REST based DX and DC Web Services that Webtrends Provides. For access you ne...YetAnotherFileRenamer: Searches a directory and/or sub-directories for files matching a certain extension and copies and renames them to the same folder. The intent is a...New Releases3D Chart Reports, using ASP.NET 2.0: Beta iscm1.0.0.1: Its first release, might be a tolerant to bugs.A Guide to Parallel Programming: Drop 2 - Guide Chapters 2 and 5, accompanying code: This is Drop 2 with just the Guide Chapters 2 and 5 and the accompanying code samples. This drop requires Visual Studio 2010 Beta 2 or later in ord...AnimeJ: AnimeJ 1.1: This new release features reversible animations. It is fully backward compatible, and by simply specifying an additional parameter to Run() you can...AutoPoco: AutoPoco 0.4: A load of additions to the convention system Inheritance precedence added And a pile of extra data sourcesCSharp Intellisense: V1.5: CSharpIntellisensePresenter(Free) Created by: Bnaya Eshet (Bnaya Eshet (credit to Karl Shifflett)) Last Updated: Tuesday, April 13, 2010 Version...DevTreks -social budgeting that improves lives and livelihoods: Social Budgeting Web Software, DevTreks alpha 4: Alpha 4 upgrades all story-telling with one very basic, simple, 'story' data pattern, schema, and stylesheet. Basic, simple, stories and eBook pa...ESPEHA: Espeha 4.3: Deletion of categories and tasks via F8 Saving on CtrlS, CtrlShiftS Navigating to search on CtrlF Hiding on CtrlQ, CtrlX, CtrlH, Q, X, H + Drag ...Folder Bookmarks: Folder Bookmarks 1.4.4: This is the latest version of Folder Bookmarks (1.4.4), with general improvements. It has an installer - it will create a directory 'CPascoe' in My...HouseFly controls: HouseFly controls alpha 0.9.1: Version 0.9.1 alpha release of HouseFly controlsIndustrial Dashboard: 2.0 Beta: 2.0 Beta includes : IndustrialDashboard Framework Sample widgets : TabularReport, DropDownPicker, DatePicker, ScopePicker Sample html pageskdar: KDAR 0.0.20: KDAR - Kernel Debugger Anti Rootkit - signature's bases updated - many bugs fixedLogikBug.Injection: Initial Release: This project is dependent upon Microsoft.Practices.ServiceLocation.IServiceLocator and must be referenced when referencing LogikBug.Injection.Mesopotamia Experiment: Mesopotamia 1.2.72: New Features - Select organims to load in sim or in hw mode Fixes - fixed robotics engine not shutting down properly - selects new robotics port o...MobySharp: MobySharp 1.1: Fixed GetComments Added the new Likes featureMongoMvc: NoSQL with MongoDB, NoRM and ASP.NET MVC: A demo on NoSQL with MongoDB, NoRM and ASP.NET MVC. To run the demo application do the ollowing actions. 1. Create a directory call C:\data\...NetMassDownloader: Release 1.6.0.0: Mass Downloader For .Net Framework which allows you do download .Net Framework source code all at once.Offline debugging for VS2005 , VS2008 , Code...Nito.KitchenSink: Version 4: Features added in this release: PDBs are source-indexed to CodePlex, so it should be possible to step through the code (with on-demand downloads) w...Nito.KitchenSink: Version 5: The only feature for this release is support of the .NET 4.0 platform. Dependencies Nito.Linq 0.4 Beta (released 2010-04-15) Rx 1.0.2441.0 (rele...Nito.LINQ: Beta (v0.4): Rx version The "with Rx" versions of Nito.LINQ are built against Rx 1.0.2441.0, released 2010-04-15. Breaking changes IEnumerable<T>.Min and IEnum...N-Twill Twitter Client for VB.NET: NTWILL PROJECT 15-abril-2010: Este archivo es un Zip que contiene el proyecto hasta este punto editado... tiene algunas que otras funciones, pero lo cuelgo para tener un referen...PanBrowser: 1.2.0: added screensaver support, hit a key to exit, up/down keys cycle through images.PersianDateTimePicker, PersianMonthCalendar: PersianDatetimePicker,PersianMonthCalendar: PersianDatetimePicker,PersianMonthCalendar 1.1 releasesPokeIn Comet Ajax Library: PokeIn Library v03: New Features! Client MessageLog Management Added CrossBrowser Script Injector Added to work with non ajax supported browsers and activeX disabled...PokeIn Comet Ajax Library: PokeIn Sample VS09: Sample Project of PokeIn VS09. Contains version 0.3 of LibraryPokeIn Comet Ajax Library: PokeIn Sample VS10: Sample Visual Studio 10 project for PokeIn. Contains v03 Library of PokeInSilverlight Toolkit: April 2010: Suggestions? Features? Questions? Ask questions in our Silverlight Toolkit forum on Silverlight.net. The forum is the best resource for Silverlig...Software Localization Tool: SharpSLT 0.9: This is the first release of SharpSLT. Features Framework: .Net 3.5 UI language: English Works as standalone and External Tool for Visual C# Ex...Star Trooper for XNA 2D Tutorial: Lesson two content: Here is Lesson two original content for the StarTrooper 2D XNA tutorial. The blog tutorial has now started over on http://xna-uk.net/blogs/darkgen...StyleCop for ReSharper: StyleCop for ReSharper 5.0.14714.1: StyleCop for ReSharper 5.0.14714.1 o StyleCop 4.3.3.0 o ReSharper 5.0.1659.36 o Visual Studio 2008 / Visual Studio 2010 Installation Instructi...TaskUnZip for SSIS: TaskUnZip for SSIS 1.1.1.0 (beta 2): Ver. 1.1.1.0 (beta2) Bug: Correct installation bug. Add: Support SQL SERVER 2008 / 2005 Minor corrections Ver. 1.1.0.0 (Tnx to: Kevin Wendler)...TaskUnZip for SSIS: TaskUnZip for SSIS 1.2.0.0: Ver. 1.2.0.0 Add: Support SQL SERVER 2008 / 2005. Add: recursive compress.* Add: filter option for extract e compress file.* Add: Test archiv...Test Project (ignore): abcde: aaaaadaTR9: TR9 Beta: this setup is first release of the program.TR9: TR9 Source Code: TR9 Source CodeUsing PowerPivot to analyze MS Dynamics NAV: GL sql procedures and demo DB: For using sql procedures and creating DW database, please see Documentation.VivoSocial: VivoSocial 7.1.1: Version 7.1.1 of VivoSocial has been released. If you experienced any issues with the previous version, please update your modules to the 7.1.1 rel...WPF Inspirational Quote Management System: Release 1.1.1: - Changed to only allow one running instance a time. - Custom icon in system tray popup removed for now as this breaks when text size is set to gr...WPF ShaderEffect Generator: WPF ShaderEffect Generator 1.6.1: Just a minor release to fix the problem with resource Uri generation in the C# Shadereffect files. ChangesThe Uri should be correctly generated no...Most Popular ProjectsRawrWBFS ManagerAJAX Control ToolkitMicrosoft SQL Server Product Samples: DatabaseSilverlight ToolkitWindows Presentation Foundation (WPF)ASP.NETMicrosoft SQL Server Community & SamplesPHPExcelpatterns & practices – Enterprise LibraryMost Active ProjectsRawrpatterns & practices – Enterprise LibraryGMap.NET - Great Maps for Windows Forms & PresentationIndustrial DashboardFarseer Physics EngineIonics Isapi Rewrite FilterNB_Store - Free DotNetNuke Ecommerce Catalog ModuleBlogEngine.NETjQuery Library for SharePoint Web ServicesDotRas

    Read the article

  • Identity Claims Encoding for SharePoint

    - by Shawn Cicoria
    Just to remind myself, the list of claim types and their encodings are listed here at the bottom. http://msdn.microsoft.com/en-us/library/gg481769.aspx Where for example: i:0#.w|contoso\scicoria ‘i’ = identity, could be ‘c’ for others # == SPClaimTypes.UserLogonName . == Microsoft.IdentityModel.Claims.ClaimValueTypes.String Table for reference: Table 1. Claim types encoding Character Claim Type ! SPClaimTypes.IdentityProvider ” SPClaimTypes.UserIdentifier # SPClaimTypes.UserLogonName $ SPClaimTypes.DistributionListClaimType % SPClaimTypes.FarmId & SPClaimTypes.ProcessIdentitySID ‘ SPClaimTypes.ProcessIdentityLogonName ( SPClaimTypes.IsAuthenticated ) Microsoft.IdentityModel.Claims.ClaimTypes.PrimarySid * Microsoft.IdentityModel.Claims.ClaimTypes.PrimaryGroupSid + Microsoft.IdentityModel.Claims.ClaimTypes.GroupSid - Microsoft.IdentityModel.Claims.ClaimTypes.Role . System.IdentityModel.Claims.ClaimTypes.Anonymous / System.IdentityModel.Claims.ClaimTypes.Authentication 0 System.IdentityModel.Claims.ClaimTypes.AuthorizationDecision 1 System.IdentityModel.Claims.ClaimTypes.Country 2 System.IdentityModel.Claims.ClaimTypes.DateOfBirth 3 System.IdentityModel.Claims.ClaimTypes.DenyOnlySid 4 System.IdentityModel.Claims.ClaimTypes.Dns 5 System.IdentityModel.Claims.ClaimTypes.Email 6 System.IdentityModel.Claims.ClaimTypes.Gender 7 System.IdentityModel.Claims.ClaimTypes.GivenName 8 System.IdentityModel.Claims.ClaimTypes.Hash 9 System.IdentityModel.Claims.ClaimTypes.HomePhone < System.IdentityModel.Claims.ClaimTypes.Locality = System.IdentityModel.Claims.ClaimTypes.MobilePhone > System.IdentityModel.Claims.ClaimTypes.Name ? System.IdentityModel.Claims.ClaimTypes.NameIdentifier @ System.IdentityModel.Claims.ClaimTypes.OtherPhone [ System.IdentityModel.Claims.ClaimTypes.PostalCode \ System.IdentityModel.Claims.ClaimTypes.PPID ] System.IdentityModel.Claims.ClaimTypes.Rsa ^ System.IdentityModel.Claims.ClaimTypes.Sid _ System.IdentityModel.Claims.ClaimTypes.Spn ` System.IdentityModel.Claims.ClaimTypes.StateOrProvince a System.IdentityModel.Claims.ClaimTypes.StreetAddress b System.IdentityModel.Claims.ClaimTypes.Surname c System.IdentityModel.Claims.ClaimTypes.System d System.IdentityModel.Claims.ClaimTypes.Thumbprint e System.IdentityModel.Claims.ClaimTypes.Upn f System.IdentityModel.Claims.ClaimTypes.Uri g System.IdentityModel.Claims.ClaimTypes.Webpage Table 2. Claim value types encoding Character Claim Type ! Microsoft.IdentityModel.Claims.ClaimValueTypes.Base64Binary “ Microsoft.IdentityModel.Claims.ClaimValueTypes.Boolean # Microsoft.IdentityModel.Claims.ClaimValueTypes.Date $ Microsoft.IdentityModel.Claims.ClaimValueTypes.Datetime % Microsoft.IdentityModel.Claims.ClaimValueTypes.DaytimeDuration & Microsoft.IdentityModel.Claims.ClaimValueTypes.Double ‘ Microsoft.IdentityModel.Claims.ClaimValueTypes.DsaKeyValue ( Microsoft.IdentityModel.Claims.ClaimValueTypes.HexBinary ) Microsoft.IdentityModel.Claims.ClaimValueTypes.Integer * Microsoft.IdentityModel.Claims.ClaimValueTypes.KeyInfo + Microsoft.IdentityModel.Claims.ClaimValueTypes.Rfc822Name - Microsoft.IdentityModel.Claims.ClaimValueTypes.RsaKeyValue . Microsoft.IdentityModel.Claims.ClaimValueTypes.String / Microsoft.IdentityModel.Claims.ClaimValueTypes.Time 0 Microsoft.IdentityModel.Claims.ClaimValueTypes.X500Name 1 Microsoft.IdentityModel.Claims.ClaimValueTypes.YearMonthDuration

    Read the article

  • Logparser and Powershell

    - by Michel Klomp
    Logparser in powershell One of the few examples how to use logparser in powershell is from the Microsoft.com Operations blog. This script is a good base to create more advanced logparser scripts: $myQuery = new-object -com MSUtil.LogQuery $szQuery = “Select top 10 * from r:\ex07011210.log”; $recordSet = $myQuery.Execute($szQuery) for(; !$recordSet.atEnd(); $recordSet.moveNext()) {             $record=$recordSet.getRecord();             write-host ($record.GetValue(0) + “,”+ $record.GetValue(1)); } $recordSet.Close(); Logparser input formats The previous example uses the default logparser object, you can extent this with the logparser input formats. with this formats get information from the event-log, different types of logfiles, the Active Directory, the registry and XML files. Here are the different ProgId’s you can use. Input Format ProgId ADS MSUtil.LogQuery.ADSInputFormat BIN MSUtil.LogQuery.IISBINInputFormat CSV MSUtil.LogQuery.CSVInputFormat ETW MSUtil.LogQuery.ETWInputFormat EVT MSUtil.LogQuery.EventLogInputFormat FS MSUtil.LogQuery.FileSystemInputFormat HTTPERR MSUtil.LogQuery.HttpErrorInputFormat IIS MSUtil.LogQuery.IISIISInputFormat IISODBC MSUtil.LogQuery.IISODBCInputFormat IISW3C MSUtil.LogQuery.IISW3CInputFormat NCSA MSUtil.LogQuery.IISNCSAInputFormat NETMON MSUtil.LogQuery.NetMonInputFormat REG MSUtil.LogQuery.RegistryInputFormat TEXTLINE MSUtil.LogQuery.TextLineInputFormat TEXTWORD MSUtil.LogQuery.TextWordInputFormat TSV MSUtil.LogQuery.TSVInputFormat URLSCAN MSUtil.LogQuery.URLScanLogInputFormat W3C MSUtil.LogQuery.W3CInputFormat XML MSUtil.LogQuery.XMLInputFormat Using logparser to parse IIS logs if you use the IISW3CinputFormat you can use the field names instead of de row number to get the information from an IIS logfile, it also skips the comment rows in the logfile. $ObjLogparser = new-object -com MSUtil.LogQuery $objInputFormat = new-object -com MSUtil.LogQuery.IISW3CInputFormat $Query = “Select top 10 * from c:\temp\hb\ex071002.log”; $recordSet = $ObjLogparser.Execute($Query, $objInputFormat) for(; !$recordSet.atEnd(); $recordSet.moveNext()) {     $record=$recordSet.getRecord();     write-host ($record.GetValue(“s-ip”) + “,”+ $record.GetValue(“cs-uri-query”)); } $recordSet.Close();

    Read the article

  • Building Simple Workflows in Oozie

    - by dan.mcclary
    Introduction More often than not, data doesn't come packaged exactly as we'd like it for analysis. Transformation, match-merge operations, and a host of data munging tasks are usually needed before we can extract insights from our Big Data sources. Few people find data munging exciting, but it has to be done. Once we've suffered that boredom, we should take steps to automate the process. We want codify our work into repeatable units and create workflows which we can leverage over and over again without having to write new code. In this article, we'll look at how to use Oozie to create a workflow for the parallel machine learning task I described on Cloudera's site. Hive Actions: Prepping for Pig In my parallel machine learning article, I use data from the National Climatic Data Center to build weather models on a state-by-state basis. NCDC makes the data freely available as gzipped files of day-over-day observations stretching from the 1930s to today. In reading that post, one might get the impression that the data came in a handy, ready-to-model files with convenient delimiters. The truth of it is that I need to perform some parsing and projection on the dataset before it can be modeled. If I get more observations, I'll want to retrain and test those models, which will require more parsing and projection. This is a good opportunity to start building up a workflow with Oozie. I store the data from the NCDC in HDFS and create an external Hive table partitioned by year. This gives me flexibility of Hive's query language when I want it, but let's me put the dataset in a directory of my choosing in case I want to treat the same data with Pig or MapReduce code. CREATE EXTERNAL TABLE IF NOT EXISTS historic_weather(column 1, column2) PARTITIONED BY (yr string) STORED AS ... LOCATION '/user/oracle/weather/historic'; As new weather data comes in from NCDC, I'll need to add partitions to my table. That's an action I should put in the workflow. Similarly, the weather data requires parsing in order to be useful as a set of columns. Because of their long history, the weather data is broken up into fields of specific byte lengths: x bytes for the station ID, y bytes for the dew point, and so on. The delimiting is consistent from year to year, so writing SerDe or a parser for transformation is simple. Once that's done, I want to select columns on which to train, classify certain features, and place the training data in an HDFS directory for my Pig script to access. ALTER TABLE historic_weather ADD IF NOT EXISTS PARTITION (yr='2010') LOCATION '/user/oracle/weather/historic/yr=2011'; INSERT OVERWRITE DIRECTORY '/user/oracle/weather/cleaned_history' SELECT w.stn, w.wban, w.weather_year, w.weather_month, w.weather_day, w.temp, w.dewp, w.weather FROM ( FROM historic_weather SELECT TRANSFORM(...) USING '/path/to/hive/filters/ncdc_parser.py' as stn, wban, weather_year, weather_month, weather_day, temp, dewp, weather ) w; Since I'm going to prepare training directories with at least the same frequency that I add partitions, I should also add that to my workflow. Oozie is going to invoke these Hive actions using what's somewhat obviously referred to as a Hive action. Hive actions amount to Oozie running a script file containing our query language statements, so we can place them in a file called weather_train.hql. Starting Our Workflow Oozie offers two types of jobs: workflows and coordinator jobs. Workflows are straightforward: they define a set of actions to perform as a sequence or directed acyclic graph. Coordinator jobs can take all the same actions of Workflow jobs, but they can be automatically started either periodically or when new data arrives in a specified location. To keep things simple we'll make a workflow job; coordinator jobs simply require another XML file for scheduling. The bare minimum for workflow XML defines a name, a starting point, and an end point: <workflow-app name="WeatherMan" xmlns="uri:oozie:workflow:0.1"> <start to="ParseNCDCData"/> <end name="end"/> </workflow-app> To this we need to add an action, and within that we'll specify the hive parameters Also, keep in mind that actions require <ok> and <error> tags to direct the next action on success or failure. <action name="ParseNCDCData"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>localhost:8021</job-tracker> <name-node>localhost:8020</name-node> <configuration> <property> <name>oozie.hive.defaults</name> <value>/user/oracle/weather_ooze/hive-default.xml</value> </property> </configuration> <script>ncdc_parse.hql</script> </hive> <ok to="WeatherMan"/> <error to="end"/> </action> There are a couple of things to note here: I have to give the FQDN (or IP) and port of my JobTracker and NameNode. I have to include a hive-default.xml file. I have to include a script file. The hive-default.xml and script file must be stored in HDFS That last point is particularly important. Oozie doesn't make assumptions about where a given workflow is being run. You might submit workflows against different clusters, or have different hive-defaults.xml on different clusters (e.g. MySQL or Postgres-backed metastores). A quick way to ensure that all the assets end up in the right place in HDFS is just to make a working directory locally, build your workflow.xml in it, and copy the assets you'll need to it as you add actions to workflow.xml. At this point, our local directory should contain: workflow.xml hive-defaults.xml (make sure this file contains your metastore connection data) ncdc_parse.hql Adding Pig to the Ooze Adding our Pig script as an action is slightly simpler from an XML standpoint. All we do is add an action to workflow.xml as follows: <action name="WeatherMan"> <pig> <job-tracker>localhost:8021</job-tracker> <name-node>localhost:8020</name-node> <script>weather_train.pig</script> </pig> <ok to="end"/> <error to="end"/> </action> Once we've done this, we'll copy weather_train.pig to our working directory. However, there's a bit of a "gotcha" here. My pig script registers the Weka Jar and a chunk of jython. If those aren't also in HDFS, our action will fail from the outset -- but where do we put them? The Jython script goes into the working directory at the same level as the pig script, because pig attempts to load Jython files in the directory from which the script executes. However, that's not where our Weka jar goes. While Oozie doesn't assume much, it does make an assumption about the Pig classpath. Anything under working_directory/lib gets automatically added to the Pig classpath and no longer requires a REGISTER statement in the script. Anything that uses a REGISTER statement cannot be in the working_directory/lib directory. Instead, it needs to be in a different HDFS directory and attached to the pig action with an <archive> tag. Yes, that's as confusing as you think it is. You can get the exact rules for adding Jars to the distributed cache from Oozie's Pig Cookbook. Making the Workflow Work We've got a workflow defined and have collected all the components we'll need to run. But we can't run anything yet, because we still have to define some properties about the job and submit it to Oozie. We need to start with the job properties, as this is essentially the "request" we'll submit to the Oozie server. In the same working directory, we'll make a file called job.properties as follows: nameNode=hdfs://localhost:8020 jobTracker=localhost:8021 queueName=default weatherRoot=weather_ooze mapreduce.jobtracker.kerberos.principal=foo dfs.namenode.kerberos.principal=foo oozie.libpath=${nameNode}/user/oozie/share/lib oozie.wf.application.path=${nameNode}/user/${user.name}/${weatherRoot} outputDir=weather-ooze While some of the pieces of the properties file are familiar (e.g., JobTracker address), others take a bit of explaining. The first is weatherRoot: this is essentially an environment variable for the script (as are jobTracker and queueName). We're simply using them to simplify the directives for the Oozie job. The oozie.libpath pieces is extremely important. This is a directory in HDFS which holds Oozie's shared libraries: a collection of Jars necessary for invoking Hive, Pig, and other actions. It's a good idea to make sure this has been installed and copied up to HDFS. The last two lines are straightforward: run the application defined by workflow.xml at the application path listed and write the output to the output directory. We're finally ready to submit our job! After all that work we only need to do a few more things: Validate our workflow.xml Copy our working directory to HDFS Submit our job to the Oozie server Run our workflow Let's do them in order. First validate the workflow: oozie validate workflow.xml Next, copy the working directory up to HDFS: hadoop fs -put working_dir /user/oracle/working_dir Now we submit the job to the Oozie server. We need to ensure that we've got the correct URL for the Oozie server, and we need to specify our job.properties file as an argument. oozie job -oozie http://url.to.oozie.server:port_number/ -config /path/to/working_dir/job.properties -submit We've submitted the job, but we don't see any activity on the JobTracker? All I got was this funny bit of output: 14-20120525161321-oozie-oracle This is because submitting a job to Oozie creates an entry for the job and places it in PREP status. What we got back, in essence, is a ticket for our workflow to ride the Oozie train. We're responsible for redeeming our ticket and running the job. oozie -oozie http://url.to.oozie.server:port_number/ -start 14-20120525161321-oozie-oracle Of course, if we really want to run the job from the outset, we can change the "-submit" argument above to "-run." This will prep and run the workflow immediately. Takeaway So, there you have it: the somewhat laborious process of building an Oozie workflow. It's a bit tedious the first time out, but it does present a pair of real benefits to those of us who spend a great deal of time data munging. First, when new data arrives that requires the same processing, we already have the workflow defined and ready to run. Second, as we build up a set of useful action definitions over time, creating new workflows becomes quicker and quicker.

    Read the article

  • What is the best strategy for transforming unicode strings into filenames?

    - by David Cowden
    I have a bunch (thousands) of resources in an RDF/XML file. I am writing a certain subset of the resources to files -- one file for each, and I'm using the resource's title property as the file name. However, the titles are every day article, website, and blog post titles, so they contain characters unsafe for a URI (the necessary step for constructing a valid file path). I know of the Jersey UriBuilder but I can't quite get it to work for my needs as I detailed in a different question on SO. Some possibilities I have considered are: Since each resource should also have an associated URL, I could try to use the name of the file on the server. The down side of this is sometimes people don't name their content logically and I think the title of an article better reflects the content that will be in each text file. Construct a white list of valid characters and parse the string myself defining substitutions for unsafe characters. The downside of this is the result could be just as unreadable as the former solution because presumably the content creators went through a similar process when placing the files on their server. Choose a more generic naming scheme, place the title in the text file along with the other attributes, and tell my boss to live with it. So my question here is, what methods work well for dealing with a scenario where you need to construct file names out of strings with potentially unsafe characters? Is there a solution that better fills out my constraints?

    Read the article

< Previous Page | 57 58 59 60 61 62 63 64 65 66 67 68  | Next Page >