Search Results

Search found 73 results on 3 pages for 'protobuf'.

Page 2/3 | < Previous Page | 1 2 3  | Next Page >

• Are there any Tutorials for Protobuf-net?

  - by AKRamkumar

  I have been reading about protobuf-net and it is amazing! Are there any tutorials that I could use? (More specifically for Dictionary<TKey,TValue> and contracts for generics) Are there any tips associated with it? Could I simply plug it into my current codebase or are there any changes I need to do?

  Read the article

• How to get structure of a Google Protobuf message without the definition

  - by dqminh

  I have to get the message structure of a protobuf message transfered to me without the message's definition. Using UnknownFieldSet methods, I was able to get a string representation of the message as below: 1: "a" 2: { 3:"b" 4:"c" } What data structure does field 2 represent ? Using UnknownFieldSet.Field.getGroupList i was able to get the content of field 3 and 4, does that means field 2 has the "deprecated" group structure ?

  Read the article

• how to serialize / deserialize classes defined in .proto (protobuf)

  - by make

  Hi, Could someone please help me with serialization/deserialization classes defined in .proto (protobuf). here is an exp that I am trying to build: file.proto message Data{ required string x1 = 1; required uint32 x2 = 2; required float x3 = 3; } message DataExge { repeated Data data = 1; } client.cpp ... void serialize(const DataExge &data_snd){ try { ofstream ofs("DataExge"); data_snd.SerializeToOstream(&ofs); } catch(exception &e) { cerr << "serialize/exception: " << e.what() << endl; exit(1); } } void deserialize(DataExge &data_rec){ try { ifstream ifs("DataExge"); data_rec.ParseFromIstream(&ifs); } catch(exception& e) { cerr << "deserialize/exception: " << e.what() << endl; exit(1); } } int main(){ ... DataExge dataexge; Data *dat = dataexge.add_data(); char *y1 = "operation1"; uint32_t y2 = 123 ; float y3 = 3.14; // assigning data to send() dat->set_set_x1(y1); dat->set_set_x2(y2); dat->set_set_x3(y3); //sending data to the client serialize(dataexge); if (send(socket, &dataexge, sizeof(dataexge), 0) < 0) { cerr << "send() failed" ; exit(1); } //receiving data from the server deserialize(dataexge); if (recv(socket, &dataexge, sizeof(dataexge), 0) < 0) { cerr << "recv() failed"; exit(1); } //printing received data cout << dat->x1() << "\n"; cout << dat->x2() << "\n"; cout << dat->x3() << "\n"; ... } server.cpp ... void serialize(const DataExge &data_snd){ try { ofstream ofs("DataExge"); data_snd.SerializeToOstream(&ofs); } catch(exception &e) { cerr << "serialize/exception: " << e.what() << endl; exit(1); } } void deserialize(DataExge &data_rec){ try { ifstream ifs("DataExge"); data_rec.ParseFromIstream(&ifs); } catch(exception& e) { cerr << "deserialize/exception: " << e.what() << endl; exit(1); } } int main(){ ... DataExge dataexge; Data *dat = dataexge.add_data(); //receiving data from the client deserialize(dataexge); if (recv(socket, &dataexge, sizeof(dataexge), 0) < 0) { cerr << "recv() failed"; exit(1); } //printing received data cout << dat->x1() << "\n"; cout << dat->x2() << "\n"; cout << dat->x3() << "\n"; // assigning data to send() dat->set_set_x1("operation2"); dat->set_set_x2(dat->x2() + 1); dat->set_set_x3(dat->x3() + 1.1); //sending data to the client serialize(dataexge); //error// I am getting error at this line ... if (send(socket, &dataexge, sizeof(dataexge), 0) < 0) { cerr << "send() failed" ; exit(1); } ... } Thanks for your help and replies -

  Read the article

• Protocol Buffers In C#: How Are Boxed Value Types Handled

  - by Greg Dean

  In the following examples: public class RowData { public object[] Values; } public class FieldData { public object Value; } I am curious as how either protobuf-net or dotnet-protobufs would handle such classes. I am more familiar with protobuf-net, so what I actually have is: [ProtoContract] public class RowData { [ProtoMember(1)] public object[] Values; } [ProtoContract] public class FieldData { [ProtoMember(1)] public object Value; } However I get an error saying "No suitable Default Object encoding found". Is there an easy way to treat these classes, that I am just not aware of? To elaborate more on the use case: This is a scaled down version of a data class used in remoting. So essentially it looks like this: FieldData data = new FieldData(); data.Value = 8; remoteObject.DoSomething(data); Note: I've omitted the ISerializable implementation for simplicity, but it is as you'd expect.

  Read the article

• Custom tool error: Command.proto:11:18: "Message2" is not defined.

  - by cornerback84

  I am using the latest version of protobuf-net with VS2008 integration. I have created 2 messages (Message1 and Message2), 1 message in 1 proto file. When I try to reference Message2 in Message1 the code generator tool gives me error "Message2" is not defined and won't generate the code file. The package for both messages is same. It look like that because of the integration of the tool with VS2008 it expects all the messages in same .proto file to compile. Currently I have to paste an empty message with same name in Message1.proto file, like this: message Message1 { optional Message2 message2 = 1; } message Message2 { } And then later on remove the declaration of class Message2 from Message1.cs. Are there any settings that I need to do so to avoid this error, or is providing an empty declaration the only way?

  Read the article

• Is it necessary to declare attribute [DataMember(Order=n)] on public method?

  - by veera

  In my solution, I have created public class to store value and already declare [DataContract/DataMember] attribute. For example, [DataContract] public class MeterSizeInfo { string _meterSizeId; [DataMember(Order = 1)] public string MeterSizeId { get { return this._meterSizeId; } set { this._meterSizeId = value; } } string _meterSizeName; [DataMember(Order = 2)] public string MeterSizeName { get { return this._meterSizeName; } set { this._meterSizeName = value; } } } Then I need to add another public method exposing to entire project. I wonder I have to add [DataMember(Order = 3)] for it or not. [DataMember(Order = 3)] //<--- must declare or not? public string DoSomething() { // do something... } I understand that if I want to use serializer in protobuf-net to serialize my data stored in, I have to declare those attribute. but I'm not sure about that on method. please help. Thank you in advance.

  Read the article

• WCF extensions without including the assembly version

  - by Marc Gravell

  As discussed here, I'm trying to add a WCF endpoint-extension; I've got it working, but I need to include the full assembly details: <extensions> <behaviorExtensions> <add name="protobuf" type="ProtoBuf.ServiceModel.ProtoBehaviorExtension, protobuf-net, Version=1.0.0.275, Culture=neutral, PublicKeyToken=257b51d87d2e4d67"/> </behaviorExtensions> </extensions> What I would like to do (to avoid issues when updating etc, especially for samples) is to include just the names: <add name="protobuf" type="ProtoBuf.ServiceModel.ProtoBehaviorExtension, protobuf-net"/> (which is what you might expect from standard assembly-naming conventions) However, if I do this, I get a big error: Parser Error Message: An error occurred creating the configuration section handler for system.serviceModel/behaviors: Extension element 'protobuf' cannot be added to this element. Verify that the extension is registered in the extension collection at system.serviceModel/extensions/behaviorExtensions. Is it possible (and if so: how) to add an extension without having to specify the assembly-version?

  Read the article

• WCF error service error message with shared classes

  - by sevenalive

  Source code: http://code.google.com/p/sevenupdate/source/browse/#hg/Source/SevenUpdate.Base SevenUpdate.Base.Sui cannot be used since it does not match imported DataContract. Need to exclude this type from referenced types. Now I tried unchecking reuse reference types and I was able to get my project to compile. but when sending a collection from the client it was never received or couldn't be deserialized on the server end. I really need this to work. Any help would be appreciated, the fullsource code is provided by google code.

  Read the article

• Does portobuf-net generated binary compatible with Google specs

  - by cornerback84

  Actually I want to serialize my data using Google's java implementation and then deserialize using C# implementation? I have chosen portobuf-net as it seems to be more stable (porto# is still v0.9 or I would have gone for it). Before I start working on it I wanted to be sure that I can achieve this (serializing data using java implementation and deserializing it using potobuf-net). Or is there any list of methods that are specific to portobuf-net implementation?

  Read the article

• WCF hosting in .NET compact framework

  - by jauwaad

  I would like to host a service on a WinCE device. The WinCE device is the host which can be accessed(control and data acquisition) by multiple clients (PC or WinCE) over serial port, TCP, USB etc. I was considering using Protocol Buffers for serialization on the wire. It would be ideal to use WCF or remoting but as far as I see they are not implement on the Compact Framework. Anyone have any ideas how this can be achieved.

  Read the article

• Is portobuf-net generated binary compatible with Google specs

  - by cornerback84

  Actually I want to serialize my data using Google's java implementation and then deserialize using C# implementation? I have chosen portobuf-net as it seems to be more stable (porto# is still v0.9 or I would have gone for it). Before I start working on it I wanted to be sure that I can achieve this (serializing data using java implementation and deserializing it using potobuf-net). Or is there any list of methods that are specific to portobuf-net implementation?

  Read the article

• ProtoInclude for fields ?

  - by Big

  I have a simple object [ProtoContract] public class DataChangedEventArgs<T> : EventArgs { private readonly object key; private readonly T data; private readonly DataChangeType changeType; ///<summary> /// Key to identify the data item ///</summary> public object Key { get { return key; } } [ProtoMember(2, IsRequired = true)] public T Data { get { return data; } } [ProtoMember(3, IsRequired = true)] public DataChangeType ChangeType { get { return changeType; } } and I have a problem with the key. Its type is object, but it can be either int, long or string. I would intuitively use a ProtoInclude attribute to say "expect these types" but unfortunately they are class only attribute. Does anybody has any idea how I could work around this ? For background, the public object Key is here for historical reasons (and all over the place) so I would very much like to avoid the mother of all refactorings ;-) Any chance I could get this to Serialize, even force it to Serialize as a string ?

  Read the article

• Protocol buffer deserialization and a dynamically loaded DLL in Compact Framework

  - by cloudraven

  I saw a question related to this on the full framework here. Since it seems to have stayed unresolved for quite a while and this is for the compact framework, I though it would be better to create a new question for it. I want to deserialize types for which I am loading assemblies dynamically (with Assembly.LoadFrom) and I am getting a "Unable to identify known-type for ProtoIncludeAttribute" error. In the related question I mentioned, it was hinted that hooking AppDomain.AssemblyResolve event would help solving the problem. It makes sense for the full framework, but that event is not available in the CF. I wonder if there is a way to do this with CF. The structures I am using look a lot like this and all the classes required for deserialization are loaded from the same Assembly. If the assembly is referenced instead of dynamically loaded it works fine, but fails if done dynamically.

  Read the article

• Google Protocol Buffers - Fixed size buffer?

  - by Roey

  Hi All. Using Google Protocol Buffers, can I set a maximum size for all messages I encode? if I know that what I encode is never larger than X bytes, then Google Protobuffs would always produce a buffer of size Y, and if I give it a smaller amount of data, pad it to size Y?

  Read the article

• How do I use Google protobuf to communicate over a serial port?

  - by rob

  I'm working on a project that uses RXTX and protobuf to communicate with an application on a development board and I've been running into issues which implies that I'm likely doing things the wrong way. Here's what I currently have for writing the request to the board (the read code is similar): public void write(CableCommandRequest request, OutputStream out) { CodedOutputStream outStream = CodedOutputStream.newInstance(out); request.writeTo(outStreatm); outStream.flush(); } The OutputStream that is used is prepared by RXTX and the development board seems to indicate that data is being received, but it is getting garbled or is otherwise not being understood. There seems to be little documentation on using protobuf over a serial connection so I'm assuming that passing the OutputStream should be sufficient. Is this in fact correct, or is this the wrong way of sending the response over the serial connection?

  Read the article

• Move from JSON to Protobuf. Is it worth it?

  - by katit

  We have REST webservices that can serve XML or JSON (WCF). I'm toying with idea of implementing Protobufs. Why? PROS Less load on servers. Smaller message size - less traffic. It is easier to switch now than later. CONS Need to be implemented Going to be harder to troubleshoot/sniff messages for debugging. I can enable GZip on server and JSON will consume as much traffic What is your suggestion and/or experience on this?

  Read the article

• Should I use a binary or a text file for storing protobuf messages?

  - by nbolton

  Using Google protobuf, I am saving my serialized messaged data to a file - in each file there are several messages. We have both C++ and Python versions of the code, so I need to use protobuf functions that are available in both languages. I have experimented with using SerializeToArray and SerializeAsString and there seems to be the following unfortunate conditions: SerializeToArray: As suggested in one answer, the best way to use this is to prefix each message with it's data size. This would work great for C++, but in Python it doesn't look like this is possible - am I wrong? SerializeAsString: This generates a serialized string equivalent to it's binary counterpart - which I can save to a file, but what happens if one of the characters in the serialization result is \n - how do we find line endings, or the ending of messages for that matter? Update: Please allow me to rephrase slightly. As I understand it, I cannot write binary data in C++ because then our Python application cannot read the data, since it can only parse string serialized messages. Should I then instead use SerializeAsString in both C++ and Python? If yes, then is it best practice to store such data in a text file rather than a binary file? My gut feeling is binary, but as you can see this doesn't look like an option.

  Read the article

• How can I fix this configure error?

  - by balor123

  I'm trying to build mosh from source on a SUSE10 machine and am getting the following error: checking for protobuf... no configure: error: Package requirements (protobuf) were not met: No package 'protobuf' found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables protobuf_CFLAGS and protobuf_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. I downloaded the source to protobuf and installed it in a custom path as well. I'm not using a package manager for any of this and cannot for various reasons outside the scope of the question. I added that custom path to my PATH and rehashed. Typically, this is enough for configure but in this case its not doing the trick. I added the prefix for protobuf to PKG_CONFIG_PATH but am still hitting this error. What should I do next to get past this error?

  Read the article

• Java Appengine APPSTATS causing java out of memory error

  - by aloo

  I have several servlets in my java appengine app that do in memory sorting and take on the order of seconds to complete. These complete error free. However, I recently enabled appstats for appengine and started receiving the following error: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Unknown Source) at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source) at java.lang.AbstractStringBuilder.append(Unknown Source) at java.lang.StringBuilder.append(Unknown Source) at java.lang.StringBuilder.append(Unknown Source) at java.lang.StringBuilder.append(Unknown Source) at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.write(TextFormat.java:344) at com.google.appengine.repackaged.com.google.protobuf.TextFormat$TextGenerator.print(TextFormat.java:332) at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printUnknownFields(TextFormat.java:249) at com.google.appengine.repackaged.com.google.protobuf.TextFormat.print(TextFormat.java:47) at com.google.appengine.repackaged.com.google.protobuf.TextFormat.printToString(TextFormat.java:73) at com.google.appengine.tools.appstats.Recorder.makeSummary(Recorder.java:157) at com.google.appengine.tools.appstats.Recorder.makeSyncCall(Recorder.java:239) at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java:54) at com.google.appengine.api.datastore.PreparedQueryImpl.runQuery(PreparedQueryImpl.java:127) at com.google.appengine.api.datastore.PreparedQueryImpl.asQueryResultList(PreparedQueryImpl.java:81) at org.datanucleus.store.appengine.query.DatastoreQuery.fulfillEntityQuery(DatastoreQuery.java:379) at org.datanucleus.store.appengine.query.DatastoreQuery.executeQuery(DatastoreQuery.java:289) at org.datanucleus.store.appengine.query.DatastoreQuery.performExecute(DatastoreQuery.java:239) at org.datanucleus.store.appengine.query.JDOQLQuery.performExecute(JDOQLQuery.java:89) at org.datanucleus.store.query.Query.executeQuery(Query.java:1489) at org.datanucleus.store.query.Query.executeWithArray(Query.java:1371) at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243) at com.poo.pooserver.dataaccess.DataAccessHelper.getPooStream(DataAccessHelper.java:204) at com.poo.pooserver.GetPooStreamServlet.doPost(GetPooStreamServlet.java:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:92) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)

  Read the article

• Performance Optimization &ndash; It Is Faster When You Can Measure It

  - by Alois Kraus

  Performance optimization in bigger systems is hard because the measured numbers can vary greatly depending on the measurement method of your choice. To measure execution timing of specific methods in your application you usually use Time Measurement Method Potential Pitfalls Stopwatch Most accurate method on recent processors. Internally it uses the RDTSC instruction. Since the counter is processor specific you can get greatly different values when your thread is scheduled to another core or the core goes into a power saving mode. But things do change luckily: Intel's Designer's vol3b, section 16.11.1 "16.11.1 Invariant TSC The time stamp counter in newer processors may support an enhancement, referred to as invariant TSC. Processor's support for invariant TSC is indicated by CPUID.80000007H:EDX[8]. The invariant TSC will run at a constant rate in all ACPI P-, C-. and T-states. This is the architectural behavior moving forward. On processors with invariant TSC support, the OS may use the TSC for wall clock timer services (instead of ACPI or HPET timers). TSC reads are much more efficient and do not incur the overhead associated with a ring transition or access to a platform resource." DateTime.Now Good but it has only a resolution of 16ms which can be not enough if you want more accuracy.   Reporting Method Potential Pitfalls Console.WriteLine Ok if not called too often. Debug.Print Are you really measuring performance with Debug Builds? Shame on you. Trace.WriteLine Better but you need to plug in some good output listener like a trace file. But be aware that the first time you call this method it will read your app.config and deserialize your system.diagnostics section which does also take time.   In general it is a good idea to use some tracing library which does measure the timing for you and you only need to decorate some methods with tracing so you can later verify if something has changed for the better or worse. In my previous article I did compare measuring performance with quantum mechanics. This analogy does work surprising well. When you measure a quantum system there is a lower limit how accurately you can measure something. The Heisenberg uncertainty relation does tell us that you cannot measure of a quantum system the impulse and location of a particle at the same time with infinite accuracy. For programmers the two variables are execution time and memory allocations. If you try to measure the timings of all methods in your application you will need to store them somewhere. The fastest storage space besides the CPU cache is the memory. But if your timing values do consume all available memory there is no memory left for the actual application to run. On the other hand if you try to record all memory allocations of your application you will also need to store the data somewhere. This will cost you memory and execution time. These constraints are always there and regardless how good the marketing of tool vendors for performance and memory profilers are: Any measurement will disturb the system in a non predictable way. Commercial tool vendors will tell you they do calculate this overhead and subtract it from the measured values to give you the most accurate values but in reality it is not entirely true. After falling into the trap to trust the profiler timings several times I have got into the habit to Measure with a profiler to get an idea where potential bottlenecks are. Measure again with tracing only the specific methods to check if this method is really worth optimizing. Optimize it Measure again. Be surprised that your optimization has made things worse. Think harder Implement something that really works. Measure again Finished! - Or look for the next bottleneck. Recently I have looked into issues with serialization performance. For serialization DataContractSerializer was used and I was not sure if XML is really the most optimal wire format. After looking around I have found protobuf-net which uses Googles Protocol Buffer format which is a compact binary serialization format. What is good for Google should be good for us. A small sample app to check out performance was a matter of minutes: using ProtoBuf; using System; using System.Diagnostics; using System.IO; using System.Reflection; using System.Runtime.Serialization; [DataContract, Serializable] class Data { [DataMember(Order=1)] public int IntValue { get; set; } [DataMember(Order = 2)] public string StringValue { get; set; } [DataMember(Order = 3)] public bool IsActivated { get; set; } [DataMember(Order = 4)] public BindingFlags Flags { get; set; } } class Program { static MemoryStream _Stream = new MemoryStream(); static MemoryStream Stream { get { _Stream.Position = 0; _Stream.SetLength(0); return _Stream; } } static void Main(string[] args) { DataContractSerializer ser = new DataContractSerializer(typeof(Data)); Data data = new Data { IntValue = 100, IsActivated = true, StringValue = "Hi this is a small string value to check if serialization does work as expected" }; var sw = Stopwatch.StartNew(); int Runs = 1000 * 1000; for (int i = 0; i < Runs; i++) { //ser.WriteObject(Stream, data); Serializer.Serialize<Data>(Stream, data); } sw.Stop(); Console.WriteLine("Did take {0:N0}ms for {1:N0} objects", sw.Elapsed.TotalMilliseconds, Runs); Console.ReadLine(); } } The results are indeed promising: Serializer Time in ms N objects protobuf-net   807 1000000 DataContract 4402 1000000 Nearly a factor 5 faster and a much more compact wire format. Lets use it! After switching over to protbuf-net the transfered wire data has dropped by a factor two (good) and the performance has worsened by nearly a factor two. How is that possible? We have measured it? Protobuf-net is much faster! As it turns out protobuf-net is faster but it has a cost: For the first time a type is de/serialized it does use some very smart code-gen which does not come for free. Lets try to measure this one by setting of our performance test app the Runs value not to one million but to 1. Serializer Time in ms N objects protobuf-net 85 1 DataContract 24 1 The code-gen overhead is significant and can take up to 200ms for more complex types. The break even point where the code-gen cost is amortized by its faster serialization performance is (assuming small objects) somewhere between 20.000-40.000 serialized objects. As it turned out my specific scenario involved about 100 types and 1000 serializations in total. That explains why the good old DataContractSerializer is not so easy to take out of business. The final approach I ended up was to reduce the number of types and to serialize primitive types via BinaryWriter directly which turned out to be a pretty good alternative. It sounded good until I measured again and found that my optimizations so far do not help much. After looking more deeper at the profiling data I did found that one of the 1000 calls did take 50% of the time. So how do I find out which call it was? Normal profilers do fail short at this discipline. A (totally undeserved) relatively unknown profiler is SpeedTrace which does unlike normal profilers create traces of your applications by instrumenting your IL code at runtime. This way you can look at the full call stack of the one slow serializer call to find out if this stack was something special. Unfortunately the call stack showed nothing special. But luckily I have my own tracing as well and I could see that the slow serializer call did happen during the serialization of a bool value. When you encounter after much analysis something unreasonable you cannot explain it then the chances are good that your thread was suspended by the garbage collector. If there is a problem with excessive GCs remains to be investigated but so far the serialization performance seems to be mostly ok.  When you do profile a complex system with many interconnected processes you can never be sure that the timings you just did measure are accurate at all. Some process might be hitting the disc slowing things down for all other processes for some seconds as well. There is a big difference between warm and cold startup. If you restart all processes you can basically forget the first run because of the OS disc cache, JIT and GCs make the measured timings very flexible. When you are in need of a random number generator you should measure cold startup times of a sufficiently complex system. After the first run you can try again getting different and much lower numbers. Now try again at least two times to get some feeling how stable the numbers are. Oh and try to do the same thing the next day. It might be that the bottleneck you found yesterday is gone today. Thanks to GC and other random stuff it can become pretty hard to find stuff worth optimizing if no big bottlenecks except bloatloads of code are left anymore. When I have found a spot worth optimizing I do make the code changes and do measure again to check if something has changed. If it has got slower and I am certain that my change should have made it faster I can blame the GC again. The thing is that if you optimize stuff and you allocate less objects the GC times will shift to some other location. If you are unlucky it will make your faster working code slower because you see now GCs at times where none were before. This is where the stuff does get really tricky. A safe escape hatch is to create a repro of the slow code in an isolated application so you can change things fast in a reliable manner. Then the normal profilers do also start working again. As Vance Morrison does point out it is much more complex to profile a system against the wall clock compared to optimize for CPU time. The reason is that for wall clock time analysis you need to understand how your system does work and which threads (if you have not one but perhaps 20) are causing a visible delay to the end user and which threads can wait a long time without affecting the user experience at all. Next time: Commercial profiler shootout.

  Read the article

• Domain model integration using JSON capable DTOs

  - by g-makulik

  I'm a bit confused about architectural choices for the java/web-applications world. The background is I have a system with certain hardware components (that introduce system immanent active behavior) and a configuration database for system meta and HW-components configuration data (these are even usually self contained, since the HW-components persist configuration data anyway). For realization of the configuration/status data exchange protocol with the HW-components we have chosen the Google Protobuf format, which works well for the directly wired communication with these components. Now we want to develop an abstract model (domain model) for those HW-components and I have the feeling that a plain Java class model would fit best for this (c++ implementation seems to have too much implementation/integration overhead with viable language-bridge interfaces). Google Protobuf message definitions could still serve well to describe DTO objects used to interact with a domain model API. But integrating Google Protobuf messages client side for e.g. data binding in the current view doesn't seem to be a good choice. I'm thinking about some extra serialization features, e.g. for JSON based data exchange with the views/controllers. Most lightweight solutions seem to involve a python based presentation layer using JSON based data transfer (I'm at least not sure to be fully informed about this). Is there some lightweight (applicable for a limited ARM Linux platform) framework available, supporting such architecture to realize a web-application?

  Read the article

• python protobufs - avoid the install step ?

  - by orion elenzil

  i'm writing a small python utility which will be consumed by moderately non-technical users and which needs to interface w/ some protobufs. ideally, i would like the only prerequisites to using this on a local machine to be: have python installed * have an SVN checkout of the repository * run a simple bash script to build the local proto .py definitions * run "python myutility" i'm running into trouble around importing descriptor_pb2.py, tho. i've seen Why do I see "cannot import name descriptor_pb2" error when using Google Protocol Buffers? , but would like to avoid adding the additional prerequisite of having run the proto SDK installer. i've modified the bash script to also generate descriptor_pb2.py in the local heirarchy, which works for the first level of imports from my other _pb2.py files, but it looks like descriptor_pb2.py itself tries to import descriptor_pb2 can't find it: $ python myutility.py Traceback (most recent call last): File "myutility.py", line 4, in <module> import protos.myProto_pb2 File "/myPath/protos/myProto_pb2.py", line 8, in <module> from google.protobuf import descriptor_pb2 File "/myPath/google/protobuf/descriptor_pb2.py", line 8, in <module> from google.protobuf import descriptor_pb2 ImportError: cannot import name descriptor_pb2 my local folder looks like: * myutility.py * google/ * protobuf/ * descriptor.py * descriptor_pb2.py * protos * myProto_ob2.py also, i'm a python n00b, so it's possible i'm overlooking something obvious. tia, orion

  Read the article

• Is it safe to convert a mysqlpp::sql_blob to a std::string?

  - by Runcible

  I'm grabbing some binary data out of my MySQL database. It comes out as a mysqlpp::sql_blob type. It just so happens that this BLOB is a serialized Google Protobuf. I need to de-serialize it so that I can access it normally. This gives a compile error, since ParseFromString() is not intended for mysqlpp:sql_blob types: protobuf.ParseFromString( record.data ); However, if I force the cast, it compiles OK: protobuf.ParseFromString( (std::string) record.data ); Is this safe? I'm particularly worried because of this snippet from the mysqlpp documentation: "Because C++ strings handle binary data just fine, you might think you can use std::string instead of sql_blob, but the current design of String converts to std::string via a C string. As a result, the BLOB data is truncated at the first embedded null character during population of the SSQLS. There’s no way to fix that without completely redesigning either String or the SSQLS mechanism." Thanks for your assistance!

  Read the article

• Web application / Domain model integration using JSON capable DTOs [on hold]

  - by g-makulik

  I'm a bit confused about architectural choices for the web-applications/java/python world. For c/c++ world the available (open source) choices to implement web applications is pretty limited to zero, involving java or python the choices explode to a,- hard to sort out -, mess of available 'frameworks' and application approaches. I want to sort out a clean MVC model, where the M stands for a fully blown (POCO, POJO driven) domain model (according M.Fowler's EAA pattern) using a mature OO language (Java,C++) for implementation. The background is: I have a system with certain hardware components (that introduce system immanent active behavior) and a configuration database for system meta and HW-components configuration data (these are even usually self contained, since the HW-components are capable to persist their configuration data anyway). For realization of the configuration/status data exchange protocol with the HW-components we have chosen the Google Protobuf format, which works well for the directly wired communication with these components. This protocol is already used successfully with a Java based GUI application via TCP/IP connection to the main system controlling HW-component. This application has some drawbacks and design flaws for historical reasons. Now we want to develop an abstract model (domain model) for configuration and monitoring those HW-components, that represents a more use case oriented view to the overall system behavior. I have the feeling that a plain Java class model would fit best for this (c++ implementation seems to have too much implementation/integration overhead with viable language-bridge interfaces). Google Protobuf message definitions could still serve well to describe DTO objects used to interact with a domain model API. But integrating Google Protobuf messages client side for e.g. data binding in the current view doesn't seem to be a good choice. I'm thinking about some extra serialization features, e.g. for JSON based data exchange with the views/controllers. Most lightweight solutions seem to involve a python based presentation layer using JSON based data transfer (I'm at least not sure to be fully informed about this). Is there some lightweight (applicable for a limited ARM Linux platform) framework available, supporting such architecture to realize a web-application? UPDATE: According to my recent research and comments of colleagues I've noticed that using Java (and some JVM) might not be the preferable choice for integration with python on a limited linux system as we have (running on ARM9 with hard to discuss memory and MCU costs), but C/C++ modules would do well for this (since this forms the native interface to python extensions, doesn't it?). I can imagine to provide a domain model from an appropriate C/C++ API (though I still think it's more efforts and higher skill requirements for the involved developers to do with these languages). Still I'm searching for a good approach that supports such architecture. I'll appreciate any pointers!

  Read the article

• What are good RPC frameworks between a Java server and C++ clients?

  - by Zwei Steinen

  Hi, I am looking for a RPC stack that can be used between a Java Server and C++ clients. My requirements are: Ease of integration (for both C++ and Java) Performance, especially number of concurrent connections and response time. Payload are mostly binaries (8-100kb) I found some like: http://code.google.com/p/protobuf-socket-rpc/ http://code.google.com/p/netty-protobuf-rpc/ Are there any other good alternatives?

  Read the article

< Previous Page | 1 2 3  | Next Page >