Search Results

Search found 59297 results on 2372 pages for 'data serialization'.

Page 9/2372 | < Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >

  • New Feature in ODI 11.1.1.6: ODI for Big Data

    - by Julien Testut
    Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-bidi-font-family:"Times New Roman";} By Ananth Tirupattur Starting with Oracle Data Integrator 11.1.1.6.0, ODI is offering a solution to process Big Data. This post provides an overview of this feature. With all the buzz around Big Data and before getting into the details of ODI for Big Data, I will provide a brief introduction to Big Data and Oracle Solution for Big Data. So, what is Big Data? Big data includes: structured data (this includes data from relation data stores, xml data stores), semi-structured data (this includes data from weblogs) unstructured data (this includes data from text blob, images) Traditionally, business decisions are based on the information gathered from transactional data. For example, transactional Data from CRM applications is fed to a decision system for analysis and decision making. Products such as ODI play a key role in enabling decision systems. However, with the emergence of massive amounts of semi-structured and unstructured data it is important for decision system to include them in the analysis to achieve better decision making capability. While there is an abundance of opportunities for business for gaining competitive advantages, process of Big Data has challenges. The challenges of processing Big Data include: Volume of data Velocity of data - The high Rate at which data is generated Variety of data In order to address these challenges and convert them into opportunities, we would need an appropriate framework, platform and the right set of tools. Hadoop is an open source framework which is highly scalable, fault tolerant system, for storage and processing large amounts of data. Hadoop provides 2 key services, distributed and reliable storage called Hadoop Distributed File System or HDFS and a framework for parallel data processing called Map-Reduce. Innovations in Hadoop and its related technology continue to rapidly evolve, hence therefore, it is highly recommended to follow information on the web to keep up with latest information. Oracle's vision is to provide a comprehensive solution to address the challenges faced by Big Data. Oracle is providing the necessary Hardware, software and tools for processing Big Data Oracle solution includes: Big Data Appliance Oracle NoSQL Database Cloudera distribution for Hadoop Oracle R Enterprise- R is a statistical package which is very popular among data scientists. ODI solution for Big Data Oracle Loader for Hadoop for loading data from Hadoop to Oracle. Further details can be found here: http://www.oracle.com/us/products/database/big-data-appliance/overview/index.html ODI Solution for Big Data: ODI’s goal is to minimize the need to understand the complexity of Hadoop framework and simplify the adoption of processing Big Data seamlessly in an enterprise. ODI is providing the capabilities for an integrated architecture for processing Big Data. This includes capability to load data in to Hadoop, process data in Hadoop and load data from Hadoop into Oracle. ODI is expanding its support for Big Data by providing the following out of the box Knowledge Modules (KMs). IKM File to Hive (LOAD DATA).Load unstructured data from File (Local file system or HDFS ) into Hive IKM Hive Control AppendTransform and validate structured data on Hive IKM Hive TransformTransform unstructured data on Hive IKM File/Hive to Oracle (OLH)Load processed data in Hive to Oracle RKM HiveReverse engineer Hive tables to generate models Using the Loading KM you can map files (local and HDFS files) to the corresponding Hive tables. For example, you can map weblog files categorized by date into a corresponding partitioned Hive table schema. Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-bidi-font-family:"Times New Roman";} Using the Hive control Append KM you can validate and transform data in Hive. In the below example, two source Hive tables are joined and mapped to a target Hive table. Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-bidi-font-family:"Times New Roman";} The Hive Transform KM facilitates processing of semi-structured data in Hive. In the below example, the data from weblog is processed using a Perl script and mapped to target Hive table. Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Calibri","sans-serif"; mso-bidi-font-family:"Times New Roman";} Using the Oracle Loader for Hadoop (OLH) KM you can load data from Hive table or HDFS to a corresponding table in Oracle. OLH is available as a standalone product. ODI greatly enhances OLH capability by generating the configuration and mapping files for OLH based on the configuration provided in the interface and KM options. ODI seamlessly invokes OLH when executing the scenario. In the below example, a HDFS file is mapped to a table in Oracle. Development and Deployment:The following diagram illustrates the development and deployment of ODI solution for Big Data. Using the ODI Studio on your development machine create and develop ODI solution for processing Big Data by connecting to a MySQL DB or Oracle database on a BDA machine or Hadoop cluster. Schedule the ODI scenarios to be executed on the ODI agent deployed on the BDA machine or Hadoop cluster. ODI Solution for Big Data provides several exciting new capabilities to facilitate the adoption of Big Data in an enterprise. You can find more information about the Oracle Big Data connectors on OTN. You can find an overview of all the new features introduced in ODI 11.1.1.6 in the following document: ODI 11.1.1.6 New Features Overview

    Read the article

  • Silverlight WCF serialization DataContract(IsReference=true) problem

    - by Ciaran
    Hi, I'm have a Silverlight 3 UI that access WCF services which in turn access respositories that use NHibernate. To overcome some NHibernate lazy loading issues with WCF I'm using my own DataContract surrogate as described here: http://timvasil.com/blog14/post/2008/02/WCF-serialization-with-NHibernate.aspx. In here I'm setting preserveObjectReferences = true My model contains cycles (i.e. Customer with IList[Order]) When I retrieve an object from my service it works fine, however when I try and send that same object back to the wcf service I get the error: System.ServiceModel.CommunicationException was unhandled by user code Message=There was an error while trying to serialize parameter http://tempuri.org/:searchCriteria. The InnerException message was 'Object graph ...' contains cycles and cannot be serialized if references are not tracked. Consider using the DataContractAttribute with the IsReference property set to true.' So cyclical references are now a problem in Silverlight, so I try change my DataContract to be [DataContract(IsReference=true)] but now when I try to retrieve an object from my service I get the following exception: System.ExecutionEngineException was unhandled Message=Exception of type 'System.ExecutionEngineException' was thrown. InnerException: Any ideas?

    Read the article

  • Silverlight WCF serialization [DataContract(IsReference=true)] problem

    - by Ciaran
    Hi, I'm have a Silverlight 3 UI that access WCF services which in turn access respositories that use NHibernate. To overcome some NHibernate lazy loading issues with WCF I'm using my own DataContract surrogate as described here: http://timvasil.com/blog14/post/2008/02/WCF-serialization-with-NHibernate.aspx. In here I'm setting preserveObjectReferences = true My model contains cycles (i.e. Customer with Collection). When I retrieve an object from my service it works fine, however when I try and send that same object back to the wcf service I get the error: System.ServiceModel.CommunicationException was unhandled by user code Message=There was an error while trying to serialize parameter http://tempuri.org/:searchCriteria. The InnerException message was 'Object graph ...' contains cycles and cannot be serialized if references are not tracked. Consider using the DataContractAttribute with the IsReference property set to true.' So cyclical references are now a problem in Silverlight, so I try change my DataContract to be [DataContract(IsReference=true)] but now when I try to retrieve an object from my service I get the following exception: System.ServiceModel.CommunicationException was unhandled by user code Message=The remote server returned an error: NotFound. It shouldn't be this hard to do something so trivial...

    Read the article

  • Data Contract Serialization Not Working For All Elements

    - by splatto
    I have an XML file that I'm trying to serialize into an object. Some elements are being ignored. My XML File: <?xml version="1.0" encoding="utf-8" ?> <License xmlns="http://schemas.datacontract.org/2004/07/MyApp.Domain"> <Guid>7FF07F74-CD5F-4369-8FC7-9BF50274A8E8</Guid> <Url>http://www.gmail.com</Url> <ValidKey>true</ValidKey> <CurrentDate>3/1/2010 9:39:28 PM</CurrentDate> <RegistrationDate>3/8/2010 9:39:28 PM</RegistrationDate> <ExpirationDate>3/8/2099 9:39:28 PM</ExpirationDate> </License> My class definition: [DataContract] public class License { [DataMember] public virtual int Id { get; set; } [DataMember] public virtual string Guid { get; set; } [DataMember] public virtual string ValidKey { get; set; } [DataMember] public virtual string Url { get; set; } [DataMember] public virtual string CurrentDate { get; set; } [DataMember] public virtual string RegistrationDate { get; set; } [DataMember] public virtual string ExpirationDate { get; set; } } And my Serialization attempt: XmlDocument Xmldoc = new XmlDocument(); Xmldoc.Load(string.Format(url)); string xml = Xmldoc.InnerXml; var serializer = new DataContractSerializer(typeof(License)); var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(xml)); License license = (License)serializer.ReadObject(memoryStream); memoryStream.Close(); The following elements are serialized: Guid ValidKey The following elements are not serialized: Url CurrentDate RegistrationDate ExpirationDate Replacing the string dates in the xml file with "blah" doesn't work either. What gives?

    Read the article

  • XML Serialization Not Working For All Elements (C#)

    - by splatto
    I have an XML file that I'm trying to serialize into an object. Some elements are being ignored. My XML File: <?xml version="1.0" encoding="utf-8" ?> <License xmlns="http://schemas.datacontract.org/2004/07/MyApp.Domain"> <Guid>7FF07F74-CD5F-4369-8FC7-9BF50274A8E8</Guid> <Url>http://www.gmail.com</Url> <ValidKey>true</ValidKey> <CurrentDate>3/1/2010 9:39:28 PM</CurrentDate> <RegistrationDate>3/8/2010 9:39:28 PM</RegistrationDate> <ExpirationDate>3/8/2099 9:39:28 PM</ExpirationDate> </License> My class definition: [DataContract] public class License { [DataMember] public virtual int Id { get; set; } [DataMember] public virtual string Guid { get; set; } [DataMember] public virtual string ValidKey { get; set; } [DataMember] public virtual string Url { get; set; } [DataMember] public virtual string CurrentDate { get; set; } [DataMember] public virtual string RegistrationDate { get; set; } [DataMember] public virtual string ExpirationDate { get; set; } } And my Serialization attempt: XmlDocument Xmldoc = new XmlDocument(); Xmldoc.Load(string.Format(url)); string xml = Xmldoc.InnerXml; var serializer = new DataContractSerializer(typeof(License)); var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(xml)); License license = (License)serializer.ReadObject(memoryStream); memoryStream.Close(); The following elements are serialized: Guid ValidKey The following elements are not serialized: Url CurrentDate RegistrationDate ExpirationDate Replacing the string dates in the xml file with "blah" doesn't work either. What gives?

    Read the article

  • java serialization problems with different JVMs

    - by Alberto
    I am having trouble using serialization in Java. I've searched the web for a solution but haven't found an answer yet. The problem is this - I have a Java library (I have the code and I export it to an archive prior to executing the code) which I need to use with two differents JVMs. One JVM is on the server (Ubuntu, running Java(TM) JRE SE Runtime Environment (build 1.7.0_09-b05)) and the other on Android 2.3.3. I compiled the library in Java 1.6. Now, I am trying to import to the client, an object exported from the server, but I receive this error: java.io.InvalidClassException: [Lweka.classifiers.functions.MultilayerPerceptron$NeuralEnd;; Incompatible class (SUID): [Lweka.classifiers.functions.MultilayerPerceptron$NeuralEnd;: static final long serialVersionUID =-359311387972759020L; but expected [Lweka.classifiers.functions.MultilayerPerceptron$NeuralEnd;: static final long serialVersionUID =1920571045915494592L; I do have an explicit serial version UID declared on the class MultilayerPerceptron$NeuralEnd, like this: protected class NeuralEnd extends NeuralConnection { private static final long serialVersionUID = 7305185603191183338L; } Where NeuralConnection implements the java.io.Serializable interface. If I do a serialver on MultilayerPerceptron$NeuralEnd I receive the serialVersionUID which I declared. So, why have both JVMs changed this value? Can you help me? Thanks, Alberto

    Read the article

  • Serialization Performance and Google Android

    - by Jomanscool2
    I'm looking for advice to speed up serialization performance, specifically when using the Google Android. For a project I am working on, I am trying to relay a couple hundred objects from a server to the Android app, and am going through various stages to get the performance I need. First I tried a terrible XML parser that I hacked together using Scanner specifically for this project, and that caused unbelievably slow performance when loading the objects (~5 minutes for a 300KB file). I then moved away from that and made my classes implement Serializable and wrote the ArrayList of objects I had to a file. Reading that file into the objects the Android, with the file already downloaded mind you, was taking ~15-30 seconds for the ~100KB serialized file. I still find this completely unacceptable for an Android app, as my app requires loading the data when starting the application. I have read briefly about Externalizable and how it can increase performance, but I am not sure as to how one implements it with nested classes. Right now, I am trying to store an ArrayList of the following class, with the nested classes below it. public class MealMenu implements Serializable{ private String commonsName; private long startMillis, endMillis, modMillis; private ArrayList<Venue> venues; private String mealName; } And the Venue class: public class Venue implements Serializable{ private String name; private ArrayList<FoodItem> foodItems; } And the FoodItem class: public class FoodItem implements Serializable{ private String name; private boolean vegan; private boolean vegetarian; } IF Externalizable is the way to go to increase performance, is there any information as to how java calls the methods in the objects when you try to write it out? I am not sure if I need to implement it in the parent class, nor how I would go about serializing the nested objects within each object.

    Read the article

  • Virtual properties duplicated during serialization when XmlElement attribute used

    - by Laramie
    The Goal: XML serialize an object that contains a list of objects of that and its derived types. The resulting XML should not use the xsi:type attribute to describe the type, to wit the names of the serialized XML elements would be an assigned name specific to the derived type, not always that of the base class, which is the default behavior. The Attempt: After exploring IXmlSerializable and IXmlSerializable with eerie XmlSchemaProvider methods and voodoo reflection to return specialized schemas and an XmlQualifiedName over the course of days, I found I was able to use the simple [XmlElement] attribute to accomplish the goal... almost. The Problem: Overridden properties appear twice when serializing. The exception reads "The XML element 'overriddenProperty' from namespace '' is already present in the current scope. Use XML attributes to specify another XML name or namespace for the element." I attempted using a *Specified property (see code), but it didn't work. Sample Code: Class Declaration using System; using System.Collections.Generic; using System.Xml.Serialization; [XmlInclude(typeof(DerivedClass))] public class BaseClass { public BaseClass() { } [XmlAttribute("virt")] public virtual string Virtual { get; set; } [XmlIgnore] public bool VirtualSpecified { get { return (this is BaseClass); } set { } } [XmlElement(ElementName = "B", Type = typeof(BaseClass), IsNullable = false)] [XmlElement(ElementName = "D", Type = typeof(DerivedClass), IsNullable = false)] public List<BaseClass> Children { get; set; } } public class DerivedClass : BaseClass { public DerivedClass() { } [XmlAttribute("virt")] public override string Virtual { get { return "always return spackle"; } set { } } } Driver: BaseClass baseClass = new BaseClass() { Children = new List<BaseClass>() }; BaseClass baseClass2 = new BaseClass(){}; DerivedClass derivedClass1 = new DerivedClass() { Children = new List<BaseClass>() }; DerivedClass derivedClass2 = new DerivedClass() { Children = new List<BaseClass>() }; baseClass.Children.Add(derivedClass1); baseClass.Children.Add(derivedClass2); derivedClass1.Children.Add(baseClass2); I've been wrestling with this on and off for weeks and can't find the answer anywhere.

    Read the article

  • Cross-platform and language (de)serialization

    - by fwgx
    I'm looking for a way to serialize a bunch of C++ structs in the most convenient way so that the serialization is portable across C++ and Java (at a minimum) and across 32bit/64bit, big/little endian platforms. The structures to be serialized just contain data, i.e. they're pure data objects with no state or behavior. The idea being that we serialize the structs into an octet blob that we can store in a database "generically" and be read out later on. Thus avoiding changing the database whenever a struct changes and also avoiding assigning each data member to a field - i.e. we only want one table to hold everything "generically" as a binary blob. This should make less work for developers and require less changes when structures change. I've looked at boost.serialize but don't think there's a way to enable compatibility with Java. And likewise for inheriting Serializable in Java. If there is a way to do it by starting with an IDL file that would be best as we already have IDL files that describe the structures. Cheers in advance!

    Read the article

  • Really simple JSON serialization in .NET

    - by Evgeny
    I have some simple .NET objects I'd like to serialize to JSON and back again. The set of objects to be serialized is quite small and I control the implementation, so I don't need a generic solution that will work for everything. Since my assembly will be distributed as a library I'd really like to avoid a dependency on some third-party DLL: I just want to give users one assembly that they can reference. I've read the other questions I could find on converting to and from JSON in .NET. The recommended solution of JSON.NET does work, of course, but it requires distributing an extra DLL. I don't need any of the fancy features of JSON.NET. I just need to handle a simple object (or even dictionary) that contains strings, integers, DateTimes and arrays of strings and bytes. On deserializing I'm happy to get back a dictionary - it doesn't need to create the object again. Is there some really simple code out there that I could compile into my assembly to do this simple job? I've also tried System.Web.Script.Serialization.JavaScriptSerializer, but where it falls down is the byte array: I want to base64-encode it and even registering a converter doesn't let me easily accomplish that due to the way that API works (it doesn't pass in the name of the field).

    Read the article

  • Looking for a fast, compact, streamable, multi-language, strongly typed serialization format

    - by sanity
    I'm currently using JSON (compressed via gzip) in my Java project, in which I need to store a large number of objects (hundreds of millions) on disk. I have one JSON object per line, and disallow linebreaks within the JSON object. This way I can stream the data off disk line-by-line without having to read the entire file at once. It turns out that parsing the JSON code (using http://www.json.org/java/) is a bigger overhead than either pulling the raw data off disk, or decompressing it (which I do on the fly). Ideally what I'd like is a strongly-typed serialization format, where I can specify "this object field is a list of strings" (for example), and because the system knows what to expect, it can deserialize it quickly. I can also specify the format just by giving someone else its "type". It would also need to be cross-platform. I use Java, but work with people using PHP, Python, and other languages. So, to recap, it should be: Strongly typed Streamable (ie. read a file bit by bit without having to load it all into RAM at once) Cross platform (including Java and PHP) Fast Free (as in speech) Any pointers?

    Read the article

  • django: control json serialization

    - by abolotnov
    Is there a way to control json serialization in django? Simple code below will return serialized object in json: co = Collection.objects.all() c = serializers.serialize('json',co) The json will look similar to this: [ { "pk": 1, "model": "picviewer.collection", "fields": { "urlName": "architecture", "name": "\u0413\u043e\u0440\u043e\u0434 \u0438 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0430", "sortOrder": 0 } }, { "pk": 2, "model": "picviewer.collection", "fields": { "urlName": "nature", "name": "\u041f\u0440\u0438\u0440\u043e\u0434\u0430", "sortOrder": 1 } }, { "pk": 3, "model": "picviewer.collection", "fields": { "urlName": "objects", "name": "\u041e\u0431\u044a\u0435\u043a\u0442\u044b \u0438 \u043d\u0430\u0442\u044e\u0440\u043c\u043e\u0440\u0442", "sortOrder": 2 } } ] You can see it's serializing it in a way that you are able to re-create the whole model, shall you want to do this at some point - fair enough, but not very handy for simple JS ajax in my case: I want bring the traffic to minimum and make the whole thing little clearer. What I did is I created a view that passes the object to a .json template and the template will do something like this to generate "nicer" json output: [ {% if collections %} {% for c in collections %} {"id": {{c.id}},"sortOrder": {{c.sortOrder}},"name": "{{c.name}}","urlName": "{{c.urlName}}"}{% if not forloop.last %},{% endif %} {% endfor %} {% endif %} ] This does work and the output is much (?) nicer: [ { "id": 1, "sortOrder": 0, "name": "????? ? ???????????", "urlName": "architecture" }, { "id": 2, "sortOrder": 1, "name": "???????", "urlName": "nature" }, { "id": 3, "sortOrder": 2, "name": "??????? ? ?????????", "urlName": "objects" } ] However, I'm bothered by the fast that my solution uses templates (an extra step in processing and possible performance impact) and it will take manual work to maintain shall I update the model, for example. I'm thinking json generating should be part of the model (correct me if I'm wrong) and done with either native python-json and django implementation but can't figure how to make it strip the bits that I don't want. One more thing - even when I restrict it to a set of fields to serialize, it will keep the id always outside the element container and instead present it as "pk" outside of it.

    Read the article

  • final transient fields and serialization

    - by doublep
    Is it possible to have final transient fields that are set to any non-default value after serialization in Java? My usecase is a cache variable — that's why it is transient. I also have a habit of making Map fields that won't be changed (i.e. contents of the map is changed, but object itself remains the same) final. However, these attributes seem to be contradictory — while compiler allows such a combination, I cannot have the field set to anything but null after unserialization. I tried the following, without success: simple field initialization (shown in the example): this is what I normally do, but the initialization doesn't seem to happen after unserialization; initialization in constructor (I believe this is semantically the same as above though); assigning the field in readObject() — cannot be done since the field is final. In the example cache is public only for testing. import java.io.*; import java.util.*; public class test { public static void main (String[] args) throws Exception { X x = new X (); System.out.println (x + " " + x.cache); ByteArrayOutputStream buffer = new ByteArrayOutputStream (); new ObjectOutputStream (buffer).writeObject (x); x = (X) new ObjectInputStream (new ByteArrayInputStream (buffer.toByteArray ())).readObject (); System.out.println (x + " " + x.cache); } public static class X implements Serializable { public final transient Map <Object, Object> cache = new HashMap <Object, Object> (); } } Output: test$X@1a46e30 {} test$X@190d11 null

    Read the article

  • .NET binary serialization conditionally without ISerializable

    - by SillyWhy
    I got 2 classes, for example: public class A { private B b; ... } public class B { ... } I need to serialize an object A using BinaryFormatter. When remoting it shall include the field b, but not when serialize to file. Here is what I added: [Serializable] public class A : MarshalByRefObject { private B b; [OnSerializing] private void OnSerializing(StreamingContext context) { if (context.State == StreamingContextStates.File) { this.b = null; } } ... } [Serializable] public class B : MarshalByRefObject { ... } I think this is a bad design because if another class C also contains B, in class C we must add the duplicate OnSerializing() logic as in A. Class B should decide what to do, not class A or C. I don't want to use ISerializable interface because there are too many variables in class B have to be added to SerializationInfo. I can create a SerializationSurrogate for class B, which perform nothing in GetObjectData() & SetObjectData(), then use it when serializing to file. However the same maintenance issue because whoever modify class B can't notice what going to happen during serialization & the existence of SerializationSurrogate. Is there a better alternative?

    Read the article

  • C#: Determine Type for (De-)Serialization

    - by dbemerlin
    Hi, i have a little problem implementing some serialization/deserialization logic. I have several classes that each take a different type of Request object, all implementing a common interface and inheriting from a default implementation: This is how i think it should be: Requests interface IRequest { public String Action {get;set;} } class DefaultRequest : IRequest { public String Action {get;set;} } class LoginRequest : DefaultRequest { public String User {get;set;} public String Pass {get;set;} } Handlers interface IHandler<T> { public Type GetRequestType(); public IResponse HandleRequest(IModel model, T request); } class DefaultHandler<T> : IHandler<T> // Used as fallback if the handler cannot be determined { public Type GetRequestType() { return /* ....... how to get the Type of T? ((new T()).GetType()) ? .......... */ } public IResponse HandleRequest(IModel model, T request) { /* ... */ } } class LoginHandler : DefaultHandler<LoginRequest> { public IResponse HandleRequest(IModel mode, LoginRequest request) { } } Calling class Controller { public ProcessRequest(String action, String serializedRequest) { IHandler handler = GetHandlerForAction(action); IRequest request = serializer.Deserialize<handler.GetRequestType()>(serializedRequest); handler(this.Model, request); } } Is what i think of even possible? My current Solution is that each handler gets the serialized String and is itself responsible for deserialization. This is not a good solution as it contains duplicate code, the beginning of each HandleRequest method looks the same (FooRequest request = Deserialize(serializedRequest); + try/catch and other Error Handling on failed deserialization). Embedding type information into the serialized Data is not possible and not intended. Thanks for any Hints.

    Read the article

  • Fast serialization/deserialization of structs

    - by user256890
    I have huge amont of geographic data represented in simple object structure consisting only structs. All of my fields are of value type. public struct Child { readonly float X; readonly float Y; readonly int myField; } public struct Parent { readonly int id; readonly int field1; readonly int field2; readonly Child[] children; } The data is chunked up nicely to small portions of Parent[]-s. Each array contains a few thousands Parent instances. I have way too much data to keep all in memory, so I need to swap these chunks to disk back and forth. (One file would result approx. 2-300KB). What would be the most efficient way of serializing/deserializing the Parent[] to a byte[] for dumpint to disk and reading back? Concerning speed, I am particularly interested in fast deserialization, write speed is not that critical. Would simple BinarySerializer good enough? Or should I hack around with StructLayout (see accepted answer)? I am not sure if that would work with array field of Parent.children. UPDATE: Response to comments - Yes, the objects are immutable (code updated) and indeed the children field is not value type. 300KB sounds not much but I have zillions of files like that, so speed does matter.

    Read the article

  • remove versioning on boost xml serialization

    - by cppanda
    hi, i just can't find a way to remove the version tracking from the boost xmlarchives. example <Settings class_id="0" tracking_level="0" version="1"> <px class_id="1" tracking_level="1" version="0" object_id="_0"> <TestInt>3</TestInt> <Resolution class_id="2" tracking_level="0" version="0"> <x>800</x> <y>600</y> </Resolution> <SomeStuff>0</SomeStuff> </px> </Settings> I want to get ride of the class_id="0" tracking_level="0" version="1" stuff, because for in this case i just don't need it and want a simple clean config like file code void serialize(Archive & ar, const unsigned int version) { ar & make_nvp("TestInt", TestInt); ar & make_nvp("Resolution", resolution); ar & make_nvp("SomeStuff", SomeStuff); } i found boost::serialization::track_never, but nowhere to use it

    Read the article

  • iPhone Serialization problem

    - by Jenicek
    Hi, I need to save my own created class to file, I found on the internet, that good approach is to use NSKeyedArchiver and NSKeyedUnarchiver My class definition looks like this: @interface Game : NSObject <NSCoding> { NSMutableString *strCompleteWord; NSMutableString *strWordToGuess; NSMutableArray *arGuessedLetters; //This array stores characters NSMutableArray *arGuessedLettersPos; //This array stores CGRects NSInteger iScore; NSInteger iLives; NSInteger iRocksFallen; BOOL bGameCompleted; BOOL bGameOver; } I've implemented methods initWithCoder: and encodeWithCoder: this way: - (id)initWithCoder:(NSCoder *)coder { if([coder allowsKeyedCoding]) { strCompleteWord = [[coder decodeObjectForKey:@"CompletedWord"] copy]; strWordToGuess = [[coder decodeObjectForKey:@"WordToGuess"] copy]; arGuessedLetters = [[coder decodeObjectForKey:@"GuessedLetters"] retain]; // arGuessedLettersPos = [[coder decodeObjectForKey:@"GuessedLettersPos"] retain]; iScore = [coder decodeIntegerForKey:@"Score"]; iLives = [coder decodeIntegerForKey:@"Lives"]; iRocksFallen = [coder decodeIntegerForKey:@"RocksFallen"]; bGameCompleted = [coder decodeBoolForKey:@"GameCompleted"]; bGameOver = [coder decodeBoolForKey:@"GameOver"]; } else { strCompleteWord = [[coder decodeObject] retain]; strWordToGuess = [[coder decodeObject] retain]; arGuessedLetters = [[coder decodeObject] retain]; // arGuessedLettersPos = [[coder decodeObject] retain]; [coder decodeValueOfObjCType:@encode(NSInteger) at:&iScore]; [coder decodeValueOfObjCType:@encode(NSInteger) at:&iLives]; [coder decodeValueOfObjCType:@encode(NSInteger) at:&iRocksFallen]; [coder decodeValueOfObjCType:@encode(BOOL) at:&bGameCompleted]; [coder decodeValueOfObjCType:@encode(BOOL) at:&bGameOver]; } return self; } - (void)encodeWithCoder:(NSCoder *)coder { if([coder allowsKeyedCoding]) { [coder encodeObject:strCompleteWord forKey:@"CompleteWord"]; [coder encodeObject:strWordToGuess forKey:@"WordToGuess"]; [coder encodeObject:arGuessedLetters forKey:@"GuessedLetters"]; //[coder encodeObject:arGuessedLettersPos forKey:@"GuessedLettersPos"]; [coder encodeInteger:iScore forKey:@"Score"]; [coder encodeInteger:iLives forKey:@"Lives"]; [coder encodeInteger:iRocksFallen forKey:@"RocksFallen"]; [coder encodeBool:bGameCompleted forKey:@"GameCompleted"]; [coder encodeBool:bGameOver forKey:@"GameOver"]; } else { [coder encodeObject:strCompleteWord]; [coder encodeObject:strWordToGuess]; [coder encodeObject:arGuessedLetters]; //[coder encodeObject:arGuessedLettersPos]; [coder encodeValueOfObjCType:@encode(NSInteger) at:&iScore]; [coder encodeValueOfObjCType:@encode(NSInteger) at:&iLives]; [coder encodeValueOfObjCType:@encode(NSInteger) at:&iRocksFallen]; [coder encodeValueOfObjCType:@encode(BOOL) at:&bGameCompleted]; [coder encodeValueOfObjCType:@encode(BOOL) at:&bGameOver]; } } And I use these methods to archive and unarchive data: [NSKeyedArchiver archiveRootObject:currentGame toFile:strPath]; Game *currentGame = [NSKeyedUnarchiver unarchiveObjectWithFile:strPath]; I have two problems. 1) As you can see, lines with arGuessedLettersPos is commented, it's because every time I try to encode this array, error comes up(this archiver cannot encode structs), and this array is used for storing CGRect structs. I've seen solution on the internet. The thing is, that every CGRect in the array is converted to an NSString (using NSStringFromCGRect()) and then saved. Is it a good approach? 2)This is bigger problem for me. Even if I comment this line and then run the code successfully, then save(archive) the data and then try to load (unarchive) them, no data is loaded. There aren't any error but currentGame object does not have data that should be loaded. Could you please give me some advice? This is first time I'm using archivers and unarchivers. Thanks a lot for every reply.

    Read the article

  • Data recovery on a data HDD (no OS)

    - by aCuria
    I am helping a family member with a dead hard disk. It is a seagate 200Gb 3.5" HDD in one of those old-school external enclosures. The problem was that windows failed to detect the hard disk when plugged in through USB. I removed the hard disk from its enclosure, and plugged it into my desktop PC. The BIOS does detect it upon POST, but unfortunately windows 7 would refuse to boot. It will get stuck on the loading screen with the glowing windows logo. Safe mode doesn't help either. What options do I have before going for some professional data recovery? edit: Someone modified the Title to something completely different from what I was asking, i just changed it back. 1) 2 HDD drives, DiskA(Dead), DiskB(my OS disk) 2) when B is connected to my system, everything works fine 3) when A AND B is connected, failure to boot. POSTs fine, but windows wont load 4) A has NO OS, its PURE data. It came from an EXTERNAL HDD enclosure which doesnt belong to me, and im trying to do data recovery.

    Read the article

  • How to implement Xml Serialization with inherited classes in C#

    - by liorafar
    I have two classes : base class name Component and inheritd class named DBComponent [Serializable] public class Component { private string name = string.Empty; private string description = string.Empty; } [Serializable] public class DBComponent : Component { private List<string> spFiles = new List<string>(); // Storage Procedure Files [XmlArrayItem("SPFile", typeof(string))] [XmlArray("SPFiles")] public List<string> SPFiles { get { return spFiles; } set { spFiles = value; } } public DBComponent(string name, string description) : base(name, description) { } } [Serializable] public class ComponentsCollection { private static ComponentsCollection instance = null; private List<Component> components = new List<Component>(); public List<Component> Components { get { return components; } set { components = value; } } public static ComponentsCollection GetInstance() { if (ccuInstance == null) { lock (lockObject) { if (instance == null) PopulateComponents(); } } return instance; } private static void PopulateComponents() { instance = new CCUniverse(); XmlSerializer xs = new XmlSerializer(instance.GetType()); instance = xs.Deserialize(XmlReader.Create("Components.xml")) as ComponentsCollection; } } } I want read\write from a Xml file. I know that I need to implement the Serialization for DBComponent class otherwise it will not read it.But i cannot find any simple article for that. all the articles that I found were too complex for this simple scenario. The Xml file looks like this: <?xml version="1.0" encoding="utf-8" ?> <ComponentsCollection> <Components> <DBComponent Name="Tenant Historical Database" Description="Tenant Historical Database"> <SPFiles> <SPFile>Setup\TenantHistoricalSP.sql</SPFile> </SPFiles> </DBComponent> <Component Name="Agent" Description="Desktop Agent" /> </Components> </ComponentsCollection> Can someone please give me a simple example of how to read this kind of xml file and what should be implemented ? Thanks Lior

    Read the article

  • C# Serialization Surrogate - Cannot access a disposed object

    - by crushhawk
    I have an image class (VisionImage) that is a black box to me. I'm attempting to serialize the image object to file using Serialization Surrogates as explained on this page. Below is my surrogate code. sealed class VisionImageSerializationSurrogate : ISerializationSurrogate { public void GetObjectData(Object obj, SerializationInfo info, StreamingContext context) { VisionImage image = (VisionImage)obj; byte[,] temp = image.ImageToArray().U8; info.AddValue("width", image.Width); info.AddValue("height", image.Height); info.AddValue("pixelvalues", temp, temp.GetType()); } public Object SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector) { VisionImage image = (VisionImage)obj; Int32 width = info.GetInt32("width"); Int32 height = info.GetInt32("height"); byte[,] temp = new byte[height, width]; temp = (byte[,])info.GetValue("pixelvalues", temp.GetType()); PixelValue2D tempPixels = new PixelValue2D(temp); image.ArrayToImage(tempPixels); return image; } } I've stepped through it to write to binary. It seems to be working fine (file is getting bigger as the images are captured). I tried to test it read the file back in. The values read back in are correct as far as the "info" object is concerned. When I get to the line image.ArrayToImage(tempPixels); It throws the "Cannot access a disposed object" exception. Upon further inspection, the object and the resulting image are both marked as disposed. My code behind the form spawns an "acquisitionWorker" and runs the following code. void acquisitionWorker_LoadImages(object sender, DoWorkEventArgs e) { // This is the main function of the acquisition background worker thread. // Perform image processing here instead of the UI thread to avoid a // sluggish or unresponsive UI. BackgroundWorker worker = (BackgroundWorker)sender; try { uint bufferNumber = 0; // Loop until we tell the thread to cancel or we get an error. When this // function completes the acquisitionWorker_RunWorkerCompleted method will // be called. while (!worker.CancellationPending) { VisionImage savedImage = (VisionImage) formatter.Deserialize(fs); CommonAlgorithms.Copy(savedImage, imageViewer.Image); // Update the UI by calling ReportProgress on the background worker. // This will call the acquisition_ProgressChanged method in the UI // thread, where it is safe to update UI elements. Do not update UI // elements directly in this thread as doing so could result in a // deadlock. worker.ReportProgress(0, bufferNumber); bufferNumber++; } } catch (ImaqException ex) { // If an error occurs and the background worker thread is not being // cancelled, then pass the exception along in the result so that // it can be handled in the acquisition_RunWorkerCompleted method. if (!worker.CancellationPending) e.Result = ex; } } What am I missing here? Why would the object be immediately disposed?

    Read the article

  • Serialization java to c#

    - by tom
    Hi, I have manged to serialize an arraylist in java using xstream, send it to my c# application via http and then write that to file (just for now). The serialized data was an arraylist in java. I want to try and re form this arraylist in c# from the xml i have. I have been looking at http://code.google.com/p/xstream-dot-net/ to do this. Does anyone know if that is the right way to go or is there a better way of reforming the serialized data in c#?

    Read the article

  • Data Governance 2010 Conference in San Diego

    - by Tony Ouk
    The Data Governance Annual Conference is one of the world's most authoritative and vendor neutral event on Data Governance and Data Quality.  The conference will focus on the "how-tos" from starting a data governance and stewardship program to attaining data governance maturity with specific topics on MDM.  This year's event will be hosted June 7 through June 10 in San Diego, California. For more information, including registration details, visit the Data Governance 2010 Conference website.

    Read the article

  • How to search for newline or linebreak characters in Excel?

    - by Highly Irregular
    I've imported some data into Excel (from a text file) and it contains some sort of newline characters. It looks like this initially: If I hit F2 (to edit) then Enter (to save changes) on each of the cells with a newline (without actually editing anything), Excel automatically changes the layout to look like this: I don't want these newlines characters here, as it messes up data processing further down the track. How can I do a search for these to detect more of them? The usual search function doesn't accept an enter character as a search character.

    Read the article

< Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >