Search Results

Search found 531 results on 22 pages for 'datastore'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 1/22 | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

  • Datastore performance, my code or the datastore latency

    - by fredrik
    I had for the last month a bit of a problem with a quite basic datastore query. It involves 2 db.Models with one referring to the other with a db.ReferenceProperty. The problem is that according to the admin logs the request takes about 2-4 seconds to complete. I strip it down to a bare form and a list to display the results. The put works fine, but the get accumulates (in my opinion) way to much cpu time. #The get look like this: outputData['items'] = {} labelsData = Label.all() for label in labelsData: labelItem = label.item.name if labelItem not in outputData['items']: outputData['items'][labelItem] = { 'item' : labelItem, 'labels' : [] } outputData['items'][labelItem]['labels'].append(label.text) path = os.path.join(os.path.dirname(__file__), 'index.html') self.response.out.write(template.render(path, outputData)) #And the models: class Item(db.Model): name = db.StringProperty() class Label(db.Model): text = db.StringProperty() lang = db.StringProperty() item = db.ReferenceProperty(Item) I've tried to make it a number of different way ie. instead of ReferenceProperty storing all Label keys in the Item Model as a db.ListProperty. My test data is just 10 rows in Item and 40 in Label. So my questions: Is it a fools errand to try to optimize this since the high cpu usage is due to the problems with the datastore or have I just screwed up somewhere in the code? ..fredrik

    Read the article

  • GAE datastore querying integer fields

    - by ParanoidAndroid
    I notice strange behavior when querying the GAE datastore. Under certain circumstances Filter does not work for integer fields. The following java code reproduces the problem: log.info("start experiment"); DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); int val = 777; // create and store the first entity. Entity testEntity1 = new Entity(KeyFactory.createKey("Test", "entity1")); Object value = new Integer(val); testEntity1.setProperty("field", value); datastore.put(testEntity1); // create the second entity by using BeanUtils. Test test2 = new Test(); // just a regular bean with an int field test2.setField(val); Entity testEntity2 = new Entity(KeyFactory.createKey("Test", "entity2")); Map<String, Object> description = BeanUtilsBean.getInstance().describe(test2); for(Entry<String,Object> entry:description.entrySet()){ testEntity2.setProperty(entry.getKey(), entry.getValue()); } datastore.put(testEntity2); // now try to retrieve the entities from the database... Filter equalFilter = new FilterPredicate("field", FilterOperator.EQUAL, val); Query q = new Query("Test").setFilter(equalFilter); Iterator<Entity> iter = datastore.prepare(q).asIterator(); while (iter.hasNext()) { log.info("found entity: " + iter.next().getKey()); } log.info("experiment finished"); the log looks like this: INFO: start experiment INFO: found entity: Test("entity1") INFO: experiment finished For some reason it only finds the first entity even though both entities are actually stored in the datastore and both 'field' values are 777 (I see it in the Datastore Viewer)! Why does it matter how the entity is created? I would like to use BeanUtils, because it is convenient. The same problem occurs on the local devserver and when deployed to GAE.

    Read the article

  • vSphere - datastore falling off a host

    - by Chadddada
    Recently we have been running the vCheck powershell script daily in order to help in monitoring our vSphere ESX 4.0 environment. One of the oddities that we have been seeing is that some of the datastores on the SAN don't always show up on every host. Our hosts are connected redundantly, via FC, to some brocade FC switches, which then connect via fiber to our EMC Ax4 SAN. While all the datastores are presented to each host we have, and they see them initially, they sometimes seem to fall off and are no longer visible. It easy enough to rescan for datastores and add them back to the hosts the hosts but this seems to be an error. Has anyone else seen this or know why it may be happening? Responses to questions: 1. Is it always the same ESX servers that lose their connection? – Scott Warren No this happens randomly on random hosts. If a VM is running on a particular host, of which the VM's disks are on a SAN datastore, then that datastore won't disappear. It seems to happen if a host doesn't touch a datastore for a bit and it just forgets about it.

    Read the article

  • ESXi add datastore without partitioning

    - by Daniel
    hi all, I've recently started playing with ESXI and now want to move all my current data (movies etc) to an openfiler vm image. Currently I have esxi 4.1 running off a Patriot XT memory stick and a 500gb hdd for the VM datastore which I will put OF on. How do I go about adding in the other hard drives I have to make them available to the OF machine without losing the data on them? They are currently formatted as NTFS

    Read the article

  • App engine datastore - query on Enum fields.

    - by Gopi
    I am using GAE(Java) with JDO for persistence. I have an entity with a Enum field which is marked as @Persistent and gets saved correctly into the datastore (As observed from the Datastore viewer in Development Console). But when I query these entities putting a filter based on the Enum value, it is always returning me all the entities whatever value I specify for the enum field. I know GAE java supports enums being persisted just like basic datatypes. But does it also allow retrieving/querying based on them? Google search could not point me to any such example code. Details: I have printed the Query just before being executed. So in two cases the query looks like - SELECT FROM com.xxx.yyy.User WHERE role == super ORDER BY key desc RANGE 0,50 SELECT FROM com.xxx.yyy.User WHERE role == admin ORDER BY key desc RANGE 0,50 Both above queries return me all the User entities from datastore in spite of datastore viewer showing some Users are of type 'admin' and some are of type 'super'.

    Read the article

  • App Engine Datastore access

    - by Pablo Alvim
    Is it possible to query App Engine's Datastore from outside the cloud, i.e. a client application? I could possibly write an app to be housed within AppStore and query the Datastore returning XML-formatted data; I want to know, however, if there are any Datastore endpoints which would allow me to do it directly. Also, in case it is possible, am I able to do so via SSL?

    Read the article

  • Google App Engine - Cannot See Children in Datastore Viewer

    - by Taylor L
    I have the following kinds/relationships in my datastore: UserAccount 1-to-1 PersistentLogin 1-to-many PersistentLogins They are all in the same entity group and UserAccount is the parent. Should I expect to see the other kinds in the datastore viewer? Currently, I only see UserAccount entities, but I'm pretty confident the other entities are there because my code is working as expected. Is this just a nuance of the datastore viewer?

    Read the article

  • Data Synchronization between Enterprise DataStore and External WebSite DataStore

    - by Yoann. B
    Hi, I've an enterprise database store used by some rich applications and a website with it own database store. Enterprise application work with local data and some of these data (like orders,prices ...) have to be "synchronized" to the web site datastore. On the other side, internet customers are able to edit their profile which have to be "synchronized" to the enterprise datastore too. Basically i need this architecture : WebSite = WebSite Database <= || Internet || <= Enterprise Database <= Rich Applications

    Read the article

  • Intermittent Issue Writing to Google Appengine Datastore

    - by user242153
    Hi, I have a functioning app and recently have had intermittent problems writing to the datastore. I did not make any relevant code changes, however in the last few days my attempts to write to the datastore sometimes work and sometimes don't. I am trying to save an object that is in a many to one relationship with an existing persisted parent. So, the logic works like this: 1) Parent pulled from the datastore 2) Child created / instantiated using constructor 3) Parent.addSingleChild(child); // the "addSingleChild" method just adds the object argument to the collection of children 4) child.setParent(Parent); // sets the Parent object to the parent field I am using transactions as explained in the documentation ending with "finally {if (tx.isActive()) {tx.rollback(); } }" When the servlet is called, the parent is called from the datastore and the child object is created and added to the many to one mapping to the pre-existing parent. The child should automatically be persisted, since the parent is already persistent, and the child is added to the collection of children that map to the parent. And it worked this way in the past. However, to be sure, i did add a pm.makePersistent(child). Doesn't seem to help, still have the intermittent problem. Any suggestions would be appreciated, and if you need to see the actual code I can post. Thanks

    Read the article

  • .Net friendly, local, key-value pair, replicatable datastore

    - by Brad Mathews
    I am looking for a key/value type datastore with very specific requirements. Anyone know anything that will work? Needs to be a component of some sort. No additional installation needed. The datastore needs to be on the local hard drive. I am using VB.Net for a desktop app running Windows XP through 7 so it needs to callable by that environment. It needs to replicatable. If I have four copies of my app running on the network, each local copy of the datastore needs to replicate with the others. As close to real time as possible. The first three are easy, I can do that with ADO.Net out of the box. The last one, replication, is the one I do not have answer to. Does such an animal exist? Thanks, Brad

    Read the article

  • App Engine datastore does not support operator OR

    - by JohnIdol
    I am trying to query the google datastore for something like (with pm -- persistanceManager): String filters = "( field == 'value' || field == 'anotherValue' )"; Query query = pm.newQuery(myType.class, filters); When I execute - I am getting back: App Engine datastore does not support operator OR. What's the best approach in people experience for this kind of queries? Any help appreciated!

    Read the article

  • Appengine datastore phantom entity - inconsistent state?

    - by aloo
    Getting a weird error on java appengine code that used to work fine (nothing has changed but the data in the datastore). I'm trying to iterate over the results of a query and change a few properties of the entities. The query does return a set of results, however, when I try to access the first result in the list, it throws an exception when trying to access any of its properties (but its key). Here's the exception: org.datanucleus.state.JDOStateManagerImpl isLoaded: Exception thrown by StateManager.isLoaded Could not retrieve entity of kind OnTheCan with key OnTheCan(3204258) org.datanucleus.exceptions.NucleusObjectNotFoundException: Could not retrieve entity of kind OnTheCan with key OnTheCan(3204258) at org.datanucleus.store.appengine.DatastoreExceptionTranslator.wrapEntityNotFoundException(DatastoreExceptionTranslator.java:60) And here is my code: PersistenceManager pm = PMF.get().getPersistenceManager(); Query query = null; List<OnTheCan> cans; query = pm.newQuery("SELECT this FROM " + OnTheCan.class.getName() + " WHERE open == true ORDER BY onTheCanId ASC"); query.setRange(0, num); cans = (List<OnTheCan>) query.execute(); for (OnTheCan c : cans) { System.err.println(c.getOnTheCanId()); // this works fine! getting the key works c.setOpen(false); // failure here with the above exception c.setAutoClosed(true); c.setEndTime(new Date(c.getStartTime().getTime() + 600000/*10*60*1000*/)); } pm.close(); The code throws the exception when trying to execute c.setOpen(false) - thats the first time I'm accessing or setting a property that isnt the key. So it seems there is a phantom entity in my datastore with key 3204258. THis entity doesn't really exist (queried the datastore from admin console) but for some reason its being returned by the query. Could my data store be in an inconsistent state? I've managed the following workaround by placing it as the first line in my for loop. Clearly an ugly hack: if (c.getOnTheCanId() == 3204258) { continue; } Any ideas?

    Read the article

  • Copy an entity in Google App Engine datastore in Python without knowing property names at 'compile'

    - by Gordon Worley
    In a Python Google App Engine app I'm writing, I have an entity stored in the datastore that I need to retrieve, make an exact copy of it (with the exception of the key), and then put this entity back in. How should I do this? In particular, are there any caveats or tricks I need to be aware of when doing this so that I get a copy of the sort I expect and not something else. ETA: Well, I tried it out and I did run into problems. I would like to make my copy in such a way that I don't have to know the names of the properties when I write the code. My thinking was to do this: #theThing = a particular entity we pull from the datastore with model Thing copyThing = Thing(user = user) for thingProperty in theThing.properties(): copyThing.__setattr__(thingProperty[0], thingProperty[1]) This executes without any errors... until I try to pull copyThing from the datastore, at which point I discover that all of the properties are set to None (with the exception of the user and key, obviously). So clearly this code is doing something, since it's replacing the defaults with None (all of the properties have a default value set), but not at all what I want. Suggestions?

    Read the article

  • GAE Datastore: persisting referenced objects

    - by David
    Two "I'm sorries" to begin with: 1) I've looked for a solution (here, and elsewhere), and couldn't find the answer. 2) English is not my mother tongue, so I may have some typos and the sort - please ignore them. To the point: I am trying to persist Java objects to the GAE datastore. I am not sure as to how to persist object having ("non-trivial") referenced object. That is, assume I have the following. public class Father { String name; int age; Vector<Child> offsprings; //this is what I call "non-trivial" reference //ctor, getters, setters... } public class Child { String name; int age; Father father; //this is what I call "non-trivial" reference //ctor, getters, setters... } The name field is unique in each type domain, and is considered a Primary-Key. In order to persist the "trivial" (String, int) fields, all I need is to add the correct annotation. So far so good. However, I don't understand how should I persist the home-brewed (Child, Father) types referenced. Should I: Convert each such reference to hold the Primary-Key (a name String, in this example) instead of the "actual" object. So, Vector<Child> offsprings; becomes Vector<String> offspringsNames; ? If that is the case, how do I handle the object at run-time? Do I just query for the Primary-Key from Class.getName, to retrieve the refrenced objects? Convert each such reference to hold the actual Key provided to me by the Datastore upon the proper put() operation? So, Vector<Child> offsprings; becomes Vector<Key> offspringsHashKeys; ? I would very much appreciate all kinds of comments. I have read ALL the offical relevant GAE docs/example. Throughout, they always persist "trivial" references, natively supported by the Datastore (e.g. in the Guestbook example, only Strings, and Longs). Many thanks, David

    Read the article

  • Using Property Builtin with GAE Datastore's Model

    - by ejel
    I want to make attributes of GAE Model properties. The reason is for cases like to turn the value into uppercase before storing it. For a plain Python class, I would do something like: Foo(db.Model): def get_attr(self): return self.something def set_attr(self, value): self.something = value.upper() if value != None else None attr = property(get_attr, set_attr) However, GAE Datastore have their own concept of Property class, I looked into the documentation and it seems that I could override get_value_for_datastore(model_instance) to achieve my goal. Nevertheless, I don't know what model_instance is and how to extract the corresponding field from it. Is overriding GAE Property classes the right way to provides getter/setter-like functionality? If so, how to do it? Added: One potential issue of overriding get_value_for_datastore that I think of is it might not get called before the object was put into datastore. Hence getting the attribute before storing the object would yield an incorrect value.

    Read the article

  • Schema-less design guidelines for Google App Engine Datastore and other NoSQL DBs

    - by jamesaharvey
    Coming from a relational database background, as I'm sure many others are, I'm looking for some solid guidelines for setting up / designing my datastore on Google App Engine. Are there any good rules of thumb people have for setting up these kinds of schema-less data stores? I understand some of the basics such as denormalizing since you can't do joins, but I was wondering what other recommendations people had. The particular simple example I am working with concerns storing searches and their results. For example I have the following 2 models defined in my Google App Engine app using Python: class Search(db.Model): who = db.StringProperty() what = db.StringProperty() where = db.StringProperty() createDate = db.DateTimeProperty(auto_now_add=True) class SearchResult(db.Model): title = db.StringProperty() content = db.StringProperty() who = db.StringProperty() what = db.StringProperty() where = db.StringProperty() createDate = db.DateTimeProperty(auto_now_add=True) I'm duplicating a bunch of properties between the models for the sake of denormalization since I can't join Search and SearchResult together. Does this make sense? Or should I store a search ID in the SearchResult model and effectively 'join' the 2 models in code when I retrieve them from the datastore? Please keep in mind that this is a simple example. Both models will have a lot more properties and the way I'm approaching this right now, I would put any property I put in the Search model in the SearchResult model as well.

    Read the article

  • Google app engine: Poor Performance with JDO + Datastore

    - by Bosh
    I have a simple data model that includes USERS: store basic information (key, name, phone # etc) RELATIONS: describe, e.g. a friendship between two users (supplying a relationship_type + two user keys) I'm getting very poor performance, for instance, if I try to print the first names of all of a user's friends. Say the user has 500 friends: I can fetch the list of friend user_ids very easily in a single query. But then, to pull out first names, I have to do 500 back-and-forth trips to the Datastore, each of which seems to take on the order of 30 ms. If this were SQL, I'd just do a JOIN and get the answer out fast. I understand there are rudimentary facilities for performing joins across un-owned relations in a relaxed implementation of JDO (as described at http://gae-java-persistence.blogspot.com) but they sound experimental and non-standard (e.g. my code won't work in any other JDO implementation). Is this really my best bet? Otherwise, how do people extract satisfactory performance from JDO/Datastore in this kind of (very common) situation? -Bosh

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >