Search Results

Search found 74171 results on 2967 pages for 'file structure'.

Page 498/2967 | < Previous Page | 494 495 496 497 498 499 500 501 502 503 504 505  | Next Page >

  • Mercurial fails while commiting/updating/etc. using Mercuriual+TrueCrypt+MAC

    - by lukewar
    While trying to work with Mercurial on project located on TrueCrypt partition I always get en error as follows: ** unknown exception encountered, details follow ** report bug details to http://mercurial.selenic.com/bts/ ** or [email protected] ** Mercurial Distributed SCM (version 1.5.2+20100502) ** Extensions loaded: Traceback (most recent call last): File "/usr/local/bin/hg", line 27, in mercurial.dispatch.run() File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 16, in run sys.exit(dispatch(sys.argv[1:])) File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 30, in dispatch return _runcatch(u, args) File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 50, in _runcatch return _dispatch(ui, args) File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 470, in _dispatch return runcommand(lui, repo, cmd, fullargs, ui, options, d) File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 340, in runcommand ret = _runcommand(ui, options, cmd, d) File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 521, in _runcommand return checkargs() File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 475, in checkargs return cmdfunc() File "/Library/Python/2.6/site-packages/mercurial/dispatch.py", line 469, in d = lambda: util.checksignature(func)(ui, *args, **cmdoptions) File "/Library/Python/2.6/site-packages/mercurial/util.py", line 401, in check return func(*args, **kwargs) File "/Library/Python/2.6/site-packages/mercurial/commands.py", line 3332, in update return hg.update(repo, rev) File "/Library/Python/2.6/site-packages/mercurial/hg.py", line 362, in update stats = _merge.update(repo, node, False, False, None) File "/Library/Python/2.6/site-packages/mercurial/merge.py", line 495, in update _checkunknown(wc, p2) File "/Library/Python/2.6/site-packages/mercurial/merge.py", line 77, in _checkunknown for f in wctx.unknown(): File "/Library/Python/2.6/site-packages/mercurial/context.py", line 660, in unknown return self._status[4] File "/Library/Python/2.6/site-packages/mercurial/util.py", line 156, in get result = self.func(obj) File "/Library/Python/2.6/site-packages/mercurial/context.py", line 622, in _status return self._repo.status(unknown=True) File "/Library/Python/2.6/site-packages/mercurial/localrepo.py", line 1023, in status if (f not in ctx1 or ctx2.flags(f) != ctx1.flags(f) File "/Library/Python/2.6/site-packages/mercurial/context.py", line 694, in flags flag = findflag(self._parents[0]) File "/Library/Python/2.6/site-packages/mercurial/context.py", line 690, in findflag return ff(path) File "/Library/Python/2.6/site-packages/mercurial/dirstate.py", line 145, in f if 'x' in fallback(x): TypeError: argument of type 'NoneType' is not iterable It is worth mention that Mercurial works perfectly if project is not located on TrueCrypt partition. Configuration: MacOS X 10.6.3 Mercurial Distributed SCM (version 1.5.2+20100502) Python 2.6.5 Have anyone of you generous people able to help me? :)

    Read the article

  • Django Deploy trouble

    - by i-Malignus
    Well, i've walking around this for a couples of days now... I think is time to ask for some help, i think my installation is ok... Server OS: Centos 5 Python -v 2.6.5 Django -v (1, 1, 1, 'final', 0) my apache conf: <VirtualHost *:80> DocumentRoot /opt/workshop ServerName taller.antell.com.py WSGIScriptAlias / /opt/workshop/workshop.wsgi WSGIDaemonProcess taller.antell.com.py user=ignacio group=ignacio processes=2 threads=25 ErrorLog /opt/workshop/apache.error.log CustomLog /opt/workshop/apache.custom.log combined <Directory "/opt/workshop"> Options +ExecCGI +FollowSymLinks -Indexes -MultiViews AllowOverride All Order allow,deny Allow from all </Directory> </VirtualHost> my mod_wsgi conf: import os import sys sys.path.append('/opt/workshop') os.environ['DJANGO_SETTINGS_MODULE'] = 'workshop.settings' os.environ['PYTHON_EGG_CACHE'] = '/tmp/.python-eggs' import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler( ) the error that i'm getting on my apache error log is: [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] mod_wsgi (pid=11459): Exception occurred processing WSGI script '/opt/workshop/workshop.wsgi'. [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] Traceback (most recent call last): [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 241, in __call__ [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] response = self.get_response(request) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/handlers/base.py", line 134, in get_response [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return self.handle_uncaught_exception(request, resolver, exc_info) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/handlers/base.py", line 154, in handle_uncaught_exception [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return debug.technical_500_response(request, *exc_info) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/views/debug.py", line 40, in technical_500_response [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] html = reporter.get_traceback_html() [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/views/debug.py", line 114, in get_traceback_html [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return t.render(c) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/__init__.py", line 178, in render [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return self.nodelist.render(context) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/__init__.py", line 779, in render [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] bits.append(self.render_node(node, context)) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/debug.py", line 81, in render_node [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] raise wrapped [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] TemplateSyntaxError: Caught an exception while rendering: No module named vehicles [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] Original Traceback (most recent call last): [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/debug.py", line 71, in render_node [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] result = node.render(context) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/debug.py", line 87, in render [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] output = force_unicode(self.filter_expression.resolve(context)) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/__init__.py", line 572, in resolve [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] new_obj = func(obj, *arg_vals) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/template/defaultfilters.py", line 687, in date [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return format(value, arg) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/dateformat.py", line 269, in format [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return df.format(format_string) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/dateformat.py", line 30, in format [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] pieces.append(force_unicode(getattr(self, piece)())) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/dateformat.py", line 175, in r [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return self.format('D, j M Y H:i:s O') [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/dateformat.py", line 30, in format [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] pieces.append(force_unicode(getattr(self, piece)())) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/encoding.py", line 71, in force_unicode [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] s = unicode(s) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/functional.py", line 201, in __unicode_cast [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return self.__func(*self.__args, **self.__kw) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 62, in ugettext [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return real_ugettext(message) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 286, in ugettext [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return do_translate(message, 'ugettext') [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 276, in do_translate [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] _default = translation(settings.LANGUAGE_CODE) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 194, in translation [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] default_translation = _fetch(settings.LANGUAGE_CODE) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 180, in _fetch [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] app = import_module(appname) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] __import__(name) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] ImportError: No module named vehicles [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] mod_wsgi (pid=11463): Exception occurred processing WSGI script '/opt/workshop/workshop.wsgi'. [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] Traceback (most recent call last): [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 241, in __call__ [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] response = self.get_response(request) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/handlers/base.py", line 73, in get_response [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] response = middleware_method(request) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/middleware/common.py", line 56, in process_request [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] if (not _is_valid_path(request.path_info) and [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/middleware/common.py", line 142, in _is_valid_path [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] urlresolvers.resolve(path) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/urlresolvers.py", line 303, in resolve [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] return get_resolver(urlconf).resolve(path) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/urlresolvers.py", line 218, in resolve [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] sub_match = pattern.resolve(new_path) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/urlresolvers.py", line 216, in resolve [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] for pattern in self.url_patterns: [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/urlresolvers.py", line 245, in _get_url_patterns [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/core/urlresolvers.py", line 240, in _get_urlconf_module [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] self._urlconf_module = import_module(self.urlconf_name) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] File "/opt/python2.6/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] __import__(name) [Wed Apr 21 15:17:48 2010] [error] [client 190.128.226.122] ImportError: No module named vehicles.urls Please give my a hand, i stuck... Obviously is a problem with my vehicle module (the only one in the app), another thing is that when i try: [root@localhost workshop]# python manage.py runserver 0:8000 The app runs perfectly, i think that the problem is something near the wsgi conf, something is not clicking.... Tks... Update: workshop dir looks like... [root@localhost workshop]# ls -l total 504 -rw-r--r-- 1 root root 22706 Apr 21 15:17 apache.custom.log -rw-r--r-- 1 root root 408141 Apr 21 15:17 apache.error.log -rw-r--r-- 1 root root 0 Apr 17 10:56 __init__.py -rw-r--r-- 1 root root 124 Apr 21 11:09 __init__.pyc -rw-r--r-- 1 root root 542 Apr 17 10:56 manage.py -rw-r--r-- 1 root root 3326 Apr 17 10:56 settings.py -rw-r--r-- 1 root root 2522 Apr 21 11:09 settings.pyc drw-r--r-- 4 root root 4096 Apr 17 10:56 templates -rw-r--r-- 1 root root 381 Apr 21 13:42 urls.py -rw-r--r-- 1 root root 398 Apr 21 13:00 urls.pyc drw-r--r-- 2 root root 4096 Apr 21 13:44 vehicles -rw-r--r-- 1 root root 38912 Apr 17 10:56 workshop.db -rw-r--r-- 1 root root 263 Apr 21 15:30 workshop.wsgi vehicles dir [root@localhost vehicles]# ls -l total 52 -rw-r--r-- 1 root root 390 Apr 17 10:56 admin.py -rw-r--r-- 1 root root 967 Apr 21 13:00 admin.pyc -rw-r--r-- 1 root root 732 Apr 17 10:56 forms.py -rw-r--r-- 1 root root 2086 Apr 21 13:00 forms.pyc -rw-r--r-- 1 root root 0 Apr 17 10:56 __init__.py -rw-r--r-- 1 root root 133 Apr 21 11:36 __init__.pyc -rw-r--r-- 1 root root 936 Apr 17 10:56 models.py -rw-r--r-- 1 root root 1827 Apr 21 11:36 models.pyc -rw-r--r-- 1 root root 514 Apr 17 10:56 tests.py -rw-r--r-- 1 root root 989 Apr 21 13:44 tests.pyc -rw-r--r-- 1 root root 1035 Apr 17 10:56 urls.py -rw-r--r-- 1 root root 1935 Apr 21 13:00 urls.pyc -rw-r--r-- 1 root root 3164 Apr 17 10:56 views.py -rw-r--r-- 1 root root 4081 Apr 21 13:00 views.pyc Update 2: this is my settings.py # Django settings for workshop project. DEBUG = True TEMPLATE_DEBUG = DEBUG ADMINS = ( ('Ignacio Rojas', '[email protected]'), ('Fabian Biedermann', '[email protected]'), ) MANAGERS = ADMINS DATABASE_ENGINE = 'sqlite3' DATABASE_NAME = '/opt/workshop/workshop.db' DATABASE_USER = '' DATABASE_PASSWORD = '' DATABASE_HOST = '' DATABASE_PORT = '' # Local time zone for this installation. Choices can be found here: # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name # although not all choices may be available on all operating systems. # If running in a Windows environment this must be set to the same as your # system time zone. TIME_ZONE = 'America/Asuncion' # Language code for this installation. All choices can be found here: # http://www.i18nguy.com/unicode/language-identifiers.html LANGUAGE_CODE = 'es-py' SITE_ID = 1 # If you set this to False, Django will make some optimizations so as not # to load the internationalization machinery. USE_I18N = True # Absolute path to the directory that holds media. # Example: "/home/media/media.lawrence.com/" MEDIA_ROOT = '' # URL that handles the media served from MEDIA_ROOT. Make sure to use a # trailing slash if there is a path component (optional in other cases). # Examples: "http://media.lawrence.com", "http://example.com/media/" MEDIA_URL = '' # URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a # trailing slash. # Examples: "http://foo.com/media/", "/media/". ADMIN_MEDIA_PREFIX = '/media/' # Make this unique, and don't share it with anybody. SECRET_KEY = '11y0_jb=+b4^nq@2-fo#g$-ihk5*v&d5-8hg_y0i@*9$w8jalp' MIDDLEWARE_CLASSES = ( 'django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', ) ROOT_URLCONF = 'workshop.urls' TEMPLATE_DIRS = ( # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. "/opt/workshop/templates" ) INSTALLED_APPS = ( 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'workshop.vehicles', ) TEMPLATE_CONTEXT_PROCESSORS = ( 'django.core.context_processors.auth', 'django.core.context_processors.debug', 'django.core.context_processors.i18n', 'django.core.context_processors.media', )

    Read the article

  • what data structure should I use for hash lookup as well as binary search?

    - by zebraman
    I am working on a school homework. I have a list of names. I want to be able to perform binary search on these names (find all names between a lower and upper bound) for first name as well as last name, and perform keyword searches as well (this will be accomplished using hashing. For example, if I have the names Garfield Cat Snoopy Dog Captain Crunch Fat Cat then a binary search of first names (C,H) will return Captain Crunch, Fat Cat, and Garfield Cat. A binary search of last names (Cr,D) will return Captain Crunch. A keyword search of 'cat' will return Fat Cat and Garfield Cat. I understand binary search will only work on a sorted list, but since I am planning on searching two different criteria, I will have to sort the list by last name or first name depending on what I'm searching for. I feel like it will be too inefficient to have to resort the list each time I want to perform a new binary search. Would it just be better for me to set up and maintain two sorted lists (one for sorted by first name, one for sorted by last name)? Also, for hashing, will I have to set up a different table of names for that as well? I understand each keyword will hash to some value determined by a hash function, and this value (or key) is a table address where the corresponding names are stored. So I just want to know what would be the best way to solve this problem? Maintaining separate structures, or is there a way to efficiently do everything I want with just one data structure?

    Read the article

  • Is a red-black tree my ideal data structure?

    - by Hugo van der Sanden
    I have a collection of items (big rationals) that I'll be processing. In each case, processing will consist of removing the smallest item in the collection, doing some work, and then adding 0-2 new items (which will always be larger than the removed item). The collection will be initialised with one item, and work will continue until it is empty. I'm not sure what size the collection is likely to reach, but I'd expect in the range 1M-100M items. I will not need to locate any item other than the smallest. I'm currently planning to use a red-black tree, possibly tweaked to keep a pointer to the smallest item. However I've never used one before, and I'm unsure whether my pattern of use fits its characteristics well. 1) Is there a danger the pattern of deletion from the left + random insertion will affect performance, eg by requiring a significantly higher number of rotations than random deletion would? Or will delete and insert operations still be O(log n) with this pattern of use? 2) Would some other data structure give me better performance, either because of the deletion pattern or taking advantage of the fact I only ever need to find the smallest item? Update: glad I asked, the binary heap is clearly a better solution for this case, and as promised turned out to be very easy to implement. Hugo

    Read the article

  • Design suggestions for creating document management structure using hidden shares.

    - by focus.nz
    I need to add some document management functionality into my software. Documents will be grouped by company name and project name. The folders need to be accessed by the application using the id numbers of clients/projects, but also easily browsed by the end user using windows explorer. Clients and Projects will be stored in a database. I am thinking of having the software create the folders using the friendly name and then using a hidden share with the id number for the software to access the files. The folder structure would be something like this --Company 1 (Company-1234$) -- Project 101 (Project-101$) -- Project 102 (Project-102$) -- Project 103 (Project-103$) -- Company 2 (Company-5678$) -- Project 201 (Project-201$) -- Project 202 (Project-202$) -- Project 203 (Project-203$) So in the example above there would be a company called "Company 1" with a ID of "1234". When browsing the folders using windows explorer the user would see \\ServerName\Documents\Company1 and you could also access the same folder from \\ServerName\Documents\Company-1234$ By using the hidden share, if the company name changes or its renamed for some reason it doesn't break the link in the application because its using the hidden shared based on the ID that never changes. Will having hundreds (maybe thousands) or hidden shares on a server provide a huge performance hit? Does any one have any suggestions or alternatives to provide this feature?

    Read the article

  • What is a data structure for quickly finding non-empty intersections of a list of sets?

    - by Andrey Fedorov
    I have a set of N items, which are sets of integers, let's assume it's ordered and call it I[1..N]. Given a candidate set, I need to find the subset of I which have non-empty intersections with the candidate. So, for example, if: I = [{1,2}, {2,3}, {4,5}] I'm looking to define valid_items(items, candidate), such that: valid_items(I, {1}) == {1} valid_items(I, {2}) == {1, 2} valid_items(I, {3,4}) == {2, 3} I'm trying to optimize for one given set I and a variable candidate sets. Currently I am doing this by caching items_containing[n] = {the sets which contain n}. In the above example, that would be: items_containing = [{}, {1}, {1,2}, {2}, {3}, {3}] That is, 0 is contained in no items, 1 is contained in item 1, 2 is contained in itmes 1 and 2, 2 is contained in item 2, 3 is contained in item 2, and 4 and 5 are contained in item 3. That way, I can define valid_items(I, candidate) = union(items_containing[n] for n in candidate). Is there any more efficient data structure (of a reasonable size) for caching the result of this union? The obvious example of space 2^N is not acceptable, but N or N*log(N) would be.

    Read the article

  • Incorporating Devise Authentication into an already existing user structure?

    - by Kevin
    I have a fully functional authentication system with a user table that has over fifty columns. It's simple but it does hash encryption with salt, uses email instead of usernames, and has two separate kinds of users with an admin as well. I'm looking to incorporate Devise authentication into my application to beef up the extra parts like email validation, forgetting passwords, remember me tokens, etc... I just wanted to see if anyone has any advice or problems they've encountered when incorporating Devise into an already existing user structure. The essential fields in my user model are: t.string :first_name, :null => false t.string :last_name, :null => false t.string :email, :null => false t.string :hashed_password t.string :salt t.boolean :is_userA, :default => false t.boolean :is_userB, :default => false t.boolean :is_admin, :default => false t.boolean :active, :default => true t.timestamps For reference sake, here's the Devise fields from the migration: t.database_authenticatable :null => false t.confirmable t.recoverable t.rememberable t.trackable That eventually turn into these actual fields in the schema: t.string "email", :default => "", :null => false t.string "encrypted_password", :limit => 128, :default => "", :null => false t.string "password_salt", :default => "", :null => false t.string "confirmation_token" t.datetime "confirmed_at" t.datetime "confirmation_sent_at" t.string "reset_password_token" t.string "remember_token" t.datetime "remember_created_at" t.integer "sign_in_count", :default => 0 t.datetime "current_sign_in_at" t.datetime "last_sign_in_at" t.string "current_sign_in_ip" t.string "last_sign_in_ip" t.datetime "created_at" t.datetime "updated_at" What do you guys recommend? Do I just remove email, hashed_password, and salt from my migration and put in the 5 Devise migration fields and everything will be OK or do I need to do something else?

    Read the article

  • compiling and running java on windows

    - by artur grzesiak
    I thought the task should be fairly easy, but my understanding of javac and java in general is rather poor. I am using Windows8 and would like to compile a single file.java that makes use of other classes. I successfully (at least without any warnings) compiled my file by calling: javac -cp relative_path_to_needed_classes relative_path_to_file_java\file.java Now if I want to run the result of compilation (file.class) by calling: java -cp relative_path_to_needed_classes relative_path_to_file_java\file I get: Error: Could not find or load main class relative_path_to_file_java\file I was trying a lot of combination to modify the -cp (eg. by adding .\;relative_path_to_file;) but still the closest where I get is: Exception in thread "main" java.lang.NoClassDefFoundError : relative_path_to_file\file (wrong name: file) The file from (wrong name: file) is the name of the file and the name of the public class within the file. I was reading a lot on SO and other sources but could not figure out what is wrong. Btw. as a result of compilation some other classes were created: file$1.class file$name1.class file$name2.class (where name1 and name2 are names of private classes within file.java) My questions are: What is (the most) probably source of my error / what am I doing wrong? What is the purpose of file$1.class? What is the minimum I should specify in -cp? May encoding play any role? Do every java class have to reside in a package? (rather not directly related to my issue) Do order of paths specified in -cp play any role in runtime?

    Read the article

  • How to structure an index for type ahead for extremely large dataset using Lucene or similar?

    - by Pete
    I have a dataset of 200million+ records and am looking to build a dedicated backend to power a type ahead solution. Lucene is of interest given its popularity and license type, but I'm open to other open source suggestions as well. I am looking for advice, tales from the trenches, or even better direct instruction on what I will need as far as amount of hardware and structure of software. Requirements: Must have: The ability to do starts with substring matching (I type in 'st' and it should match 'Stephen') The ability to return results very quickly, I'd say 500ms is an upper bound. Nice to have: The ability to feed relevance information into the indexing process, so that, for example, more popular terms would be returned ahead of others and not just alphabetical, aka Google style. In-word substring matching, so for example ('st' would match 'bestseller') Note: This index will purely be used for type ahead, and does not need to serve standard search queries. I am not worried about getting advice on how to set up the front end or AJAX, as long as the index can be queried as a service or directly via Java code. Up votes for any useful information that allows me to get closer to an enterprise level type ahead solution

    Read the article

  • MediaFileUpload of HTML in UTF-8 encoding using Python and Google-Drive-SDK

    - by Victoria
    Looking for example using MediaFileUpload has a reference to the basic documentation for creating/uploading a file to Google Drive. However, while I have code that creates files, converting from HTML to Google Doc format. It works perfectly when they contain only ASCII characters, but when I add a non-ASCII character, it fails, with the following traceback: Traceback (most recent call last): File "d:\my\py\ckwort.py", line 949, in <module> rids, worker_documents = analyze( meta, gd ) File "d:\my\py\ckwort.py", line 812, in analyze gd.mkdir( **iy ) File "d:\my\py\ckwort.py", line 205, in mkdir self.create( **( kw['subop'])) File "d:\my\py\ckwort.py", line 282, in create media_body=kw['media_body'], File "D:\my\py\gdrive2\oauth2client\util.py", line 120, in positional_wrapper return wrapped(*args, **kwargs) File "D:\my\py\gdrive2\apiclient\http.py", line 676, in execute headers=self.headers) File "D:\my\py\gdrive2\oauth2client\util.py", line 120, in positional_wrapper return wrapped(*args, **kwargs) File "D:\my\py\gdrive2\oauth2client\client.py", line 420, in new_request redirections, connection_type) File "D:\my\py\gdrive2\httplib2\__init__.py", line 1597, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "D:\my\py\gdrive2\httplib2\__init__.py", line 1345, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "D:\my\py\gdrive2\httplib2\__init__.py", line 1282, in _conn_request conn.request(method, request_uri, body, headers) File "C:\Python27\lib\httplib.py", line 958, in request self._send_request(method, url, body, headers) File "C:\Python27\lib\httplib.py", line 992, in _send_request self.endheaders(body) File "C:\Python27\lib\httplib.py", line 954, in endheaders self._send_output(message_body) File "C:\Python27\lib\httplib.py", line 812, in _send_output msg += message_body UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 370: ordinal not in range(128) I don't find any parameter to use to specify what file encoding should be used by MediaFileUpload (My files are using UTF-8). Am I missing something?

    Read the article

  • What are simple instructions for creating a Python package structure and egg?

    - by froadie
    I just completed my first (minor) Python project, and my boss wants me to package it nicely so that it can be distributed and called from other programs easily. He suggested I look into eggs. I've been googling and reading, but I'm just getting confused. Most of the sites I'm looking at explain how to use Python eggs that were already created, or how to create an egg from a setup.py file (which I don't yet have). All I have now is an Eclipse pydev project with about 4 modules and a settings/configuration file. In easy steps, how do I go about structuring it into folders/packages and compiling it into an egg? And once it's an egg, what do I have to know about deploying/building/using it? I'm really starting from scratch here, so don't assume I know anything; simple step-by-step instructions would be really helpful... These are some of the sites that I've been looking at so far: http://peak.telecommunity.com/DevCenter/PythonEggs http://www.packtpub.com/article/writing-a-package-in-python http://www.ibm.com/developerworks/library/l-cppeak3.html#N10232 I've also browsed a few SO questions but haven't really found what I need. Thanks!

    Read the article

  • How to create XSD schema from XML with this kind of structure (in .net)?

    - by Mr. Brownstone
    Here's the problem: my input is XML file that looks something like: <BaseEntityClassInfo> <item> <key>BaseEntityClassInfo.SomeField</key> <value>valueData1</value> </item> <item> <key>BaseEntityClassInfo.AdditionalDataClass.SomeOtherField</key> <value>valueData2</value> </item> <item> <key>BaseEntityClassInfo.AdditionalDataClass.AnotherClassInfo.DisplayedText</key> <value>valueData3</value> </item> ... ... </BaseEntityClassInfo> The <key> element somehow describes entity classes fields and relationships (used in some other app that I don't have access to) and the <value> stores the actual data that I need. My goal is to programatically generate a typed Dataset from this XML that could then be used for creating reports. I thought of building some XSD schema from input XML file first and then use this schema to generate Dataset but I'm not sure how to do that. The problem is that I don't want all data in one table, I need several tables with relationships based on the <key> value so I guess I need to infer relational structure from XML <key> data in some way. So what do you think? How could this be done and what would be the best approach? Any advice, ideas, suggestions would be appreciated!

    Read the article

  • query structure - ignoring entries for the same event from multiple users?

    - by Andrew Heath
    One table in my MySQL database tracks game plays. It has the following structure: SCENARIO_VICTORIES [ID] [scenario_id] [game] [timestamp] [user_id] [winning_side] [play_date] ID is the autoincremented primary key. timestamp records the moment of submission for the record. winning_side has one of three possible values: 1, 2, or 0 (meaning a draw) One of the queries done on this table calculates the victory percentage for each scenario, when that scenario's page is viewed. The output is expressed as: Side 1 win % Side 2 win % Draw % and queried with: SELECT winning_side, COUNT(scenario_id) FROM scenario_victories WHERE scenario_id='$scenID' GROUP BY winning_side ORDER BY winning_side ASC and then processed into the percentages and such. Sorry for the long setup. My problem is this: several of my users play each other, and record their mutual results. So these battles are being doubly represented in the victory percentages and result counts. Though this happens infrequently, the userbase isn't large and the double entries do have a noticeable effect on the data. Given the table and query above - does anyone have any suggestions for how I can "collapse" records that have the same play_date & game & scenario_id & winning_side so that they're only counted once?

    Read the article

  • How to encapsulate a third party complex object structure?

    - by tangens
    Motivation Currently I'm using the java parser japa to create an abstract syntax tree (AST) of a java file. With this AST I'm doing some code generation (e.g.: if there's an annotation on a method, create some other source files, ...) Problem When my code generation becomes more complex, I've to dive deeper into the structure of the AST (e.g. I have to use visitors to extract some type information of method parameters). But I'm not sure if I want to stay with japa or if I will change the parser library later. Because my code generator uses freemarker (which isn't good at automatic refactoring) I want the interface that it uses to access the AST information to be stable, even if I decide to change the java parser. Question What's the best way to encapsulate complex datastructures of third party libraries? I could create my own datatypes and copy the parts of the AST that I need into these. I could create lots of specialized access methods that work with the AST and create exactly the infos I need (e.g. the fully qualified return type of a method as one string, or the first template parameter of a class). I could create wrapper classes for the japa datastructures I currently need and embed the japa types inside, so that I can delegate requests to the japa types and transform the resulting japa types to my wrapper classes again. Which solution should I take? Are there other (better) solutions to this problem?

    Read the article

  • How to structure the tables of a very simple blog in MySQL?

    - by Programmer
    I want to add a very simple blog feature on one of my existing LAMP sites. It would be tied to a user's existing profile, and they would be able to simply input a title and a body for each post in their blog, and the date would be automatically set upon submission. They would be allowed to edit and delete any blog post and title at any time. The blog would be displayed from most recent to oldest, perhaps 20 posts to a page, with proper pagination above that. Other users would be able to leave comments on each post, which the blog owner would be allowed to delete, but not pre-moderate. That's basically it. Like I said, very simple. How should I structure the MySQL tables for this? I'm assuming that since there will be blog posts and comments, I would need a separate table for each, is that correct? But then what columns would I need in each table, what data structures should I use, and how should I link the two tables together (e.g. any foreign keys)? I could not find any tutorials for something like this, and what I'm looking to do is really offer my users the simplest version of a blog possible. No tags, no moderation, no images, no fancy formatting, etc. Just a simple diary-type, pure-text blog with commenting by other users.

    Read the article

  • Typical SVN repo structure seems to be sub-optimal for continuous integration...

    - by Dave
    I've set up our SVN repository like the Subversion book suggests, and this is also how my previous companies have done it. It looks something like this: /trunk /branches /tags /extlibs /docs where the first three are pretty obvious, and extlibs is for 3rd party assemblies that we wouldn't typically recompile ourselves. All of this works great for the daily development stuff. Now I've installed TeamCity and have builds, unit tests, code coverage, and code analysis running. Everything is great, except for the fact that this code structure results in too much code getting downloaded. So here's the catch 22, in my opinion: it's silly to download all of aforementioned folders from the SVN repo when I only need /trunk and /extlibs. But I can only specify one repo folder to download in the TeamCity VCS settings. So then the other possibility is to put the /extlibs folder into /trunk, but in order to compile branches, /extlibs would have to go into all of those as well (since I usually branch the trunk, and not individual subfolders... and this would seem infinitely more evil since /extlibs could actually be larger than /trunk and /branches, with all of the binaries stored there... Do you guys have any suggestions for me? Thanks!

    Read the article

  • How to sort data in a table data structure in Java?

    - by rgksugan
    I need to sort data based on the third column of the table data structure. I tried based on the answers for the following question. But my sorting does not work. Please help me in this. Here goes my code. Object[] data = new Object[y]; rst.beforeFirst(); while (rst.next()) { int p_id = Integer.parseInt(rst.getString(1)); String sw2 = "select sum(quantity) from tbl_order_detail where product_id=" + p_id; rst1 = stmt1.executeQuery(sw2); rst1.next(); String sw3 = "select max(order_date) from tbl_order where tbl_order.`Order_ID` in (select tbl_order_detail.`Order_ID` from tbl_order_detail where product_id=" + p_id + ")"; rst2 = stmt2.executeQuery(sw3); rst2.next(); data[i] = new Object[]{new String(rst.getString(2)), new String(rst.getString(3)), new Integer(rst1.getString(1)), new String(rst2.getString(1))}; i++; } ColumnComparator cc = new ColumnComparator(2); Arrays.sort(data, cc); if (i == 0) { table.addCell(""); table.addCell(""); table.addCell(""); table.addCell(""); } else { for (int j = 0; j < y; j++) { Object[] theRow = (Object[]) data[j]; table.addCell((String) theRow[0]); table.addCell((String) theRow[1]); table.addCell((String) theRow[2]); table.addCell((String) theRow[3]); }

    Read the article

  • What the heck is the "Structure and Interpretation of Computer Programs" cover drawing about?

    - by Paul Reiners
    What the heck is the Structure and Interpretation of Computer Programs cover drawing about? I mean I know what "eval", "apply", and '?' all mean, but I'm having a hard time deciphering the rest of the picture. Who the heck is the maiden? Does she work for the wizard? Why the heck is she pointing at the table? Is she pointing at that little bowl-type thing? Or the books? Or the table in general? Is she trying to tell the wizard that he should apply some sort of Lisp wizardry to the table or the items on it? Or is she just telling him something prosaic, such as his food is getting cold? What the heck is the one leg on that table that looks like...a leg...with a foot at the end (as legs tend to have)? How does the table balance on one leg? (Or is that another leg in the shadows?) [Note: I'm waiting for a lengthy build to finish in case you were wondering.]

    Read the article

  • Slow query. Wrong database structure?

    - by Tin
    I have a database with table that contains tasks. Tasks have a lifecycle. The status of the task's lifecycle can change. These state transitions are stored in a separate table tasktransitions. Now I wrote a query to find all open/reopened tasks and recently changed tasks but I already see with a rather small number of tasks (<1000) that execution time has becoming very long (0.5s). Tasks +-------------+---------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+---------+------+-----+---------+----------------+ | taskid | int(11) | NO | PRI | NULL | auto_increment | | description | text | NO | | NULL | | +-------------+---------+------+-----+---------+----------------+ Tasktransitions +------------------+-----------+------+-----+-------------------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------------+-----------+------+-----+-------------------+----------------+ | tasktransitionid | int(11) | NO | PRI | NULL | auto_increment | | taskid | int(11) | NO | MUL | NULL | | | status | int(11) | NO | MUL | NULL | | | description | text | NO | | NULL | | | userid | int(11) | NO | | NULL | | | transitiondate | timestamp | NO | | CURRENT_TIMESTAMP | | +------------------+-----------+------+-----+-------------------+----------------+ Query SELECT tasks.taskid,tasks.description,tasklaststatus.status FROM tasks LEFT OUTER JOIN ( SELECT tasktransitions.taskid,tasktransitions.transitiondate,tasktransitions.status FROM tasktransitions INNER JOIN ( SELECT taskid,MAX(transitiondate) AS lasttransitiondate FROM tasktransitions GROUP BY taskid ) AS tasklasttransition ON tasklasttransition.lasttransitiondate=tasktransitions.transitiondate AND tasklasttransition.taskid=tasktransitions.taskid ) AS tasklaststatus ON tasklaststatus.taskid=tasks.taskid WHERE tasklaststatus.status IS NULL OR tasklaststatus.status=0 or tasklaststatus.transitiondate>'2013-09-01'; I'm wondering if the database structure is best choice performance wise. Could adding indexes help? I already tried to add some but I don't see great improvements. +-----------------+------------+----------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | +-----------------+------------+----------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ | tasktransitions | 0 | PRIMARY | 1 | tasktransitionid | A | 896 | NULL | NULL | | BTREE | | | | tasktransitions | 1 | taskid_date_ix | 1 | taskid | A | 896 | NULL | NULL | | BTREE | | | | tasktransitions | 1 | taskid_date_ix | 2 | transitiondate | A | 896 | NULL | NULL | | BTREE | | | | tasktransitions | 1 | status_ix | 1 | status | A | 3 | NULL | NULL | | BTREE | | | +-----------------+------------+----------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+ Any other suggestions?

    Read the article

  • Listening Port Permanently. if file is on my stream, Get file. How to?

    - by Phsika
    i writed 2 client and server program. client dend file also server listen port and than get file.But i need My server App must listen on 51124 port permanently. if any file on my stream, show my message "there is a file on your stream" and than show me savefile dialog. But my server app in "Infinite loop". 1) listen 51124 port every time 2) do i have a file on my stream, show me a messagebox. private void Form1_Load(object sender, EventArgs e) { TcpListener Dinle = new TcpListener(51124); try { Dinle.Start(); Socket Baglanti = Dinle.AcceptSocket(); if (!Baglanti.Connected) { MessageBox.Show("No Connection!"); } else { while (true) { byte[] Dizi = new byte[250000]; Baglanti.Receive(Dizi, Dizi.Length, 0); string Yol; saveFileDialog1.Title = "Save File"; saveFileDialog1.ShowDialog(); Yol = saveFileDialog1.FileName; FileStream Dosya = new FileStream(Yol, FileMode.Create); Dosya.Write(Dizi, 0, Dizi.Length - 20); Dosya.Close(); listBox1.Items.Add("dosya indirildi"); listBox1.Items.Add("Dosya Boyutu=" + Dizi.Length.ToString()); listBox1.Items.Add("Indirilme Tarihi=" + DateTime.Now); listBox1.Items.Add("--------------------------------"); } } } catch (Exception) { throw; } } My Algorithm: if(AnyFileonStream()==true) { GetFile() //Also continue to listening 51124 port... } How can i do that?

    Read the article

  • Is there a Java data structure that is effectively an ArrayList with double indicies and built-in in

    - by Bob Cross
    I am looking for a pre-built Java data structure with the following characteristics: It should look something like an ArrayList but should allow indexing via double-precision rather than integers. Note that this means that it's likely that you'll see indicies that don't line up with the original data points (i.e., asking for the value that corresponds to key "1.5"). As a consequence, the value returned will likely be interpolated. For example, if the key is 1.5, the value returned could be the average of the value at key 1.0 and the value at key 2.0. The keys will be sorted but the values are not ensured to be monotonically increasing. In fact, there's no assurance that the first derivative of the values will be continuous (making it a poor fit for certain types of splines). Freely available code only, please. For clarity, I know how to write such a thing. In fact, we already have an implementation of this and some related data structures in legacy code that I want to replace due to some performance and coding issues. What I'm trying to avoid is spending a lot of time rolling my own solution when there might already be such a thing in the JDK, Apache Commons or another standard library. Frankly, that's exactly the approach that got this legacy code into the situation that it's in right now.... Is there such a thing out there in a freely available library?

    Read the article

  • Is there command-line tool to extract typedef, structure, enumeration, variable, function from a C or C++ file?

    - by FooF
    I am desiring a command-line tool to extract a definition or declaration (typedef, structure, enumeration, variable, or function) from a C or C++ source file. Also a way to replace an existing definition/declaration would be handy (after transforming the extracted definition by a user-submitted script). Is there such generic tool available, or is some resonably close approximation of such a tool? Scriptability and ability to hook-up with user created scripts or programs is of importance here, although I am academically curious of GUI programs too. Open source solutions for Unix/Linux camp are preferred (although I am curious of Windows and OS X tools too). Primary language interests are C and C++ but more generic solution would be even better (I think we do not need super accurate parsing capabilities for finding, extracting and replacing a definition in a program source file). Sample Use Cases (extra - for the curious mind): Given deeply nested structs and variable (array) initializations of these types, suppose there is a need to change a struct definition by adding or reordering fields or rewriting the variable/array definitions in more readable format without introducing errors resulting from manual labor. This would work by extracting the old initializations, then using a script/program to write the new initializations to replace the old ones. For implementing a code browsing tool - extract a definition. Decorative code generation (e.g. logging function entries / returns). Scripted code structuring (e.g. extract this and that thing and put in different place without change - version control commit comment could document the command to perform this operation to make it evident and verifiable that nothing changed). Note about tags: More accurate tag than code-generation would be code-transformation but it does not exist.

    Read the article

  • How to structure Javascript programs in complex web applications?

    - by mixedpickles
    Hi there. I have a problem, which is not easily described. I'm writing a web application that makes heavy usage of jquery and ajax calls. Now I don't have experience in designing the architecture for javascript programms, but I realize that my program has not a good structure. I think I have to many identifiers referring to the same (at least more or less) thing. Let's have an exemplary look at an arbitrary UI widget: The eventhandlers use DOM elements as parameters. The DOM element represents a widget in the browser. A lot of times I use jQuery objects (I think they are basically a wrapper around DOM elements) to do something with the widget. Sometimes they are used transiently, sometimes they are stored in a variable for later purposes. The ajax function calls use strings identifiers for these widgets. They are processed server side. Beside that I have a widget class whose instances represents a widget. It is instantiated through the new operator. Now I have somehow four different object identifiers for the same thing, which needs to be kept in sync until the page is loaded anew. This seems not to be a good thing. Any advice?

    Read the article

  • Talend Enterprise Data Integration overperforms on Oracle SPARC T4

    - by Amir Javanshir
    The SPARC T microprocessor, released in 2005 by Sun Microsystems, and now continued at Oracle, has a good track record in parallel execution and multi-threaded performance. However it was less suited for pure single-threaded workloads. The new SPARC T4 processor is now filling that gap by offering a 5x better single-thread performance over previous generations. Following our long-term relationship with Talend, a fast growing ISV positioned by Gartner in the “Visionaries” quadrant of the “Magic Quadrant for Data Integration Tools”, we decided to test some of their integration components with the T4 chip, more precisely on a T4-1 system, in order to verify first hand if this new processor stands up to its promises. Several tests were performed, mainly focused on: Single-thread performance of the new SPARC T4 processor compared to an older SPARC T2+ processor Overall throughput of the SPARC T4-1 server using multiple threads The tests consisted in reading large amounts of data --ten's of gigabytes--, processing and writing them back to a file or an Oracle 11gR2 database table. They are CPU, memory and IO bound tests. Given the main focus of this project --CPU performance--, bottlenecks were removed as much as possible on the memory and IO sub-systems. When possible, the data to process was put into the ZFS filesystem cache, for instance. Also, two external storage devices were directly attached to the servers under test, each one divided in two ZFS pools for read and write operations. Multi-thread: Testing throughput on the Oracle T4-1 The tests were performed with different number of simultaneous threads (1, 2, 4, 8, 12, 16, 32, 48 and 64) and using different storage devices: Flash, Fibre Channel storage, two stripped internal disks and one single internal disk. All storage devices used ZFS as filesystem and volume management. Each thread read a dedicated 1GB-large file containing 12.5M lines with the following structure: customerID;FirstName;LastName;StreetAddress;City;State;Zip;Cust_Status;Since_DT;Status_DT 1;Ronald;Reagan;South Highway;Santa Fe;Montana;98756;A;04-06-2006;09-08-2008 2;Theodore;Roosevelt;Timberlane Drive;Columbus;Louisiana;75677;A;10-05-2009;27-05-2008 3;Andrew;Madison;S Rustle St;Santa Fe;Arkansas;75677;A;29-04-2005;09-02-2008 4;Dwight;Adams;South Roosevelt Drive;Baton Rouge;Vermont;75677;A;15-02-2004;26-01-2007 […] The following graphs present the results of our tests: Unsurprisingly up to 16 threads, all files fit in the ZFS cache a.k.a L2ARC : once the cache is hot there is no performance difference depending on the underlying storage. From 16 threads upwards however, it is clear that IO becomes a bottleneck, having a good IO subsystem is thus key. Single-disk performance collapses whereas the Sun F5100 and ST6180 arrays allow the T4-1 to scale quite seamlessly. From 32 to 64 threads, the performance is almost constant with just a slow decline. For the database load tests, only the best IO configuration --using external storage devices-- were used, hosting the Oracle table spaces and redo log files. Using the Sun Storage F5100 array allows the T4-1 server to scale up to 48 parallel JVM processes before saturating the CPU. The final result is a staggering 646K lines per second insertion in an Oracle table using 48 parallel threads. Single-thread: Testing the single thread performance Seven different tests were performed on both servers. Given the fact that only one thread, thus one file was read, no IO bottleneck was involved, all data being served from the ZFS cache. Read File ? Filter ? Write File: Read file, filter data, write the filtered data in a new file. The filter is set on the “Status” column: only lines with status set to “A” are selected. This limits each output file to about 500 MB. Read File ? Load Database Table: Read file, insert into a single Oracle table. Average: Read file, compute the average of a numeric column, write the result in a new file. Division & Square Root: Read file, perform a division and square root on a numeric column, write the result data in a new file. Oracle DB Dump: Dump the content of an Oracle table (12.5M rows) into a CSV file. Transform: Read file, transform, write the result data in a new file. The transformations applied are: set the address column to upper case and add an extra column at the end, which is the concatenation of two columns. Sort: Read file, sort a numeric and alpha numeric column, write the result data in a new file. The following table and graph present the final results of the tests: Throughput unit is thousand lines per second processed (K lines/second). Improvement is the % of improvement between the T5140 and T4-1. Test T4-1 (Time s.) T5140 (Time s.) Improvement T4-1 (Throughput) T5140 (Throughput) Read/Filter/Write 125 806 645% 100 16 Read/Load Database 195 1111 570% 64 11 Average 96 557 580% 130 22 Division & Square Root 161 1054 655% 78 12 Oracle DB Dump 164 945 576% 76 13 Transform 159 1124 707% 79 11 Sort 251 1336 532% 50 9 The improvement of single-thread performance is quite dramatic: depending on the tests, the T4 is between 5.4 to 7 times faster than the T2+. It seems clear that the SPARC T4 processor has gone a long way filling the gap in single-thread performance, without sacrifying the multi-threaded capability as it still shows a very impressive scaling on heavy-duty multi-threaded jobs. Finally, as always at Oracle ISV Engineering, we are happy to help our ISV partners test their own applications on our platforms, so don't hesitate to contact us and let's see what the SPARC T4-based systems can do for your application! "As describe in this benchmark, Talend Enterprise Data Integration has overperformed on T4. I was generally happy to see that the T4 gave scaling opportunities for many scenarios like complex aggregations. Row by row insertion in Oracle DB is faster with more than 650,000 rows per seconds without using any bulk Oracle capabilities !" Cedric Carbone, Talend CTO.

    Read the article

< Previous Page | 494 495 496 497 498 499 500 501 502 503 504 505  | Next Page >