Search Results

Search found 27655 results on 1107 pages for 'visual python'.

Page 494/1107 | < Previous Page | 490 491 492 493 494 495 496 497 498 499 500 501  | Next Page >

• Data Warehouse ETL slow - change primary key in dimension?

  - by Jubbles

  I have a working MySQL data warehouse that is organized as a star schema and I am using Talend Open Studio for Data Integration 5.1 to create the ETL process. I would like this process to run once per day. I have estimated that one of the dimension tables (dimUser) will have approximately 2 million records and 23 columns. I created a small test ETL process in Talend that worked, but given the amount of data that may need to be updated daily, the current performance will not cut it. It takes the ETL process four minutes to UPDATE or INSERT 100 records to dimUser. If I assumed a linear relationship between the count of records and the amount of time to UPDATE or INSERT, then there is no way the ETL can finish in 3-4 hours (my hope), let alone one day. Since I'm unfamiliar with Java, I wrote the ETL as a Python script and ran into the same problem. Although, I did discover that if I did only INSERT, the process went much faster. I am pretty sure that the bottleneck is caused by the UPDATE statements. The primary key in dimUser is an auto-increment integer. My friend suggested that I scrap this primary key and replace it with a multi-field primary key (in my case, 2-3 fields). Before I rip the test data out of my warehouse and change the schema, can anyone provide suggestions or guidelines related to the design of the data warehouse the ETL process how realistic it is to have an ETL process INSERT or UPDATE a few million records each day will my friend's suggestion significantly help If you need any further information, just let me know and I'll post it. UPDATE - additional information: mysql> describe dimUser; Field Type Null Key Default Extra user_key int(10) unsigned NO PRI NULL auto_increment id_A int(10) unsigned NO NULL id_B int(10) unsigned NO NULL field_4 tinyint(4) unsigned NO 0 field_5 varchar(50) YES NULL city varchar(50) YES NULL state varchar(2) YES NULL country varchar(50) YES NULL zip_code varchar(10) NO 99999 field_10 tinyint(1) NO 0 field_11 tinyint(1) NO 0 field_12 tinyint(1) NO 0 field_13 tinyint(1) NO 1 field_14 tinyint(1) NO 0 field_15 tinyint(1) NO 0 field_16 tinyint(1) NO 0 field_17 tinyint(1) NO 1 field_18 tinyint(1) NO 0 field_19 tinyint(1) NO 0 field_20 tinyint(1) NO 0 create_date datetime NO 2012-01-01 00:00:00 last_update datetime NO 2012-01-01 00:00:00 run_id int(10) unsigned NO 999 I used a surrogate key because I had read that it was good practice. Since, from a business perspective, I want to keep aware of potential fraudulent activity (say for 200 days a user is associated with state X and then the next day they are associated with state Y - they could have moved or their account could have been compromised), so that is why geographic data is kept. The field id_B may have a few distinct values of id_A associated with it, but I am interested in knowing distinct (id_A, id_B) tuples. In the context of this information, my friend suggested that something like (id_A, id_B, zip_code) be the primary key. For the large majority of daily ETL processes (80%), I only expect the following fields to be updated for existing records: field_10 - field_14, last_update, and run_id (this field is a foreign key to my etlLog table and is used for ETL auditing purposes).

  Read the article

• Django CMS - not able to upload images through cmsplugin_filer_image

  - by Luke

  i have a problem with a local installation on django cms 2.3.3: i've installed it trough pip, in a separated virtualenv. next i followed the tutorial for settings.py configuration, i started the server. Then in the admin i created an page (home), and i've tried to add an image in the placeholder through the cmsplugin_filer_image, but the upload seems that doesn't work. here's my settings.py: # Django settings for cms1 project. # -*- coding: utf-8 -*- import os gettext = lambda s: s PROJECT_PATH = os.path.abspath(os.path.dirname(__file__)) DEBUG = True TEMPLATE_DEBUG = DEBUG ADMINS = ( # ('Your Name', '[email protected]'), ) MANAGERS = ADMINS DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'. 'NAME': 'cms1', # Or path to database file if using sqlite3. 'USER': 'cms', # Not used with sqlite3. 'PASSWORD': 'cms', # Not used with sqlite3. 'HOST': '', # Set to empty string for localhost. Not used with sqlite3. 'PORT': '', # Set to empty string for default. Not used with sqlite3. } } # Local time zone for this installation. Choices can be found here: # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name # although not all choices may be available on all operating systems. # In a Windows environment this must be set to your system time zone. TIME_ZONE = 'Europe/Rome' # Language code for this installation. All choices can be found here: # http://www.i18nguy.com/unicode/language-identifiers.html LANGUAGE_CODE = 'it-it' SITE_ID = 1 # If you set this to False, Django will make some optimizations so as not # to load the internationalization machinery. USE_I18N = True # If you set this to False, Django will not format dates, numbers and # calendars according to the current locale. USE_L10N = True # If you set this to False, Django will not use timezone-aware datetimes. USE_TZ = True # Absolute filesystem path to the directory that will hold user-uploaded files. # Example: "/home/media/media.lawrence.com/media/" MEDIA_ROOT = os.path.join(PROJECT_PATH, "media") # URL that handles the media served from MEDIA_ROOT. Make sure to use a # trailing slash. # Examples: "http://media.lawrence.com/media/", "http://example.com/media/" MEDIA_URL = '/media/' # Absolute path to the directory static files should be collected to. # Don't put anything in this directory yourself; store your static files # in apps' "static/" subdirectories and in STATICFILES_DIRS. # Example: "/home/media/media.lawrence.com/static/" STATIC_ROOT = os.path.join(PROJECT_PATH, "static") STATIC_URL = "/static/" # Additional locations of static files STATICFILES_DIRS = ( os.path.join(PROJECT_PATH, "static_auto"), # Put strings here, like "/home/html/static" or "C:/www/django/static". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. ) # List of finder classes that know how to find static files in # various locations. STATICFILES_FINDERS = ( 'django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', # 'django.contrib.staticfiles.finders.DefaultStorageFinder', ) # Make this unique, and don't share it with anybody. SECRET_KEY = '^c2q3d8w)f#gk%5i)(#i*lwt%lm-!2=(*1d!1cf+rg&amp;-hqi_9u' # List of callables that know how to import templates from various sources. TEMPLATE_LOADERS = ( 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', # 'django.template.loaders.eggs.Loader', ) MIDDLEWARE_CLASSES = ( 'django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'cms.middleware.multilingual.MultilingualURLMiddleware', 'cms.middleware.page.CurrentPageMiddleware', 'cms.middleware.user.CurrentUserMiddleware', 'cms.middleware.toolbar.ToolbarMiddleware', # Uncomment the next line for simple clickjacking protection: # 'django.middleware.clickjacking.XFrameOptionsMiddleware', ) ROOT_URLCONF = 'cms1.urls' # Python dotted path to the WSGI application used by Django's runserver. WSGI_APPLICATION = 'cms1.wsgi.application' TEMPLATE_DIRS = ( os.path.join(PROJECT_PATH, "templates"), # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates". # Always use forward slashes, even on Windows. # Don't forget to use absolute paths, not relative paths. ) CMS_TEMPLATES = ( ('template_1.html', 'Template One'), ('template_2.html', 'Template Two'), ) TEMPLATE_CONTEXT_PROCESSORS = ( 'django.contrib.auth.context_processors.auth', 'django.core.context_processors.i18n', 'django.core.context_processors.request', 'django.core.context_processors.media', 'django.core.context_processors.static', 'cms.context_processors.media', 'sekizai.context_processors.sekizai', ) LANGUAGES = [ ('it', 'Italiano'), ('en', 'English'), ] INSTALLED_APPS = ( 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages', 'django.contrib.staticfiles', 'cms', #django CMS itself 'mptt', #utilities for implementing a modified pre-order traversal tree 'menus', #helper for model independent hierarchical website navigation 'south', #intelligent schema and data migrations 'sekizai', #for javascript and css management #'cms.plugins.file', 'cms.plugins.flash', 'cms.plugins.googlemap', 'cms.plugins.link', #'cms.plugins.picture', 'cms.plugins.snippet', 'cms.plugins.teaser', 'cms.plugins.text', #'cms.plugins.video', 'cms.plugins.twitter', 'filer', 'cmsplugin_filer_file', 'cmsplugin_filer_folder', 'cmsplugin_filer_image', 'cmsplugin_filer_teaser', 'cmsplugin_filer_video', 'easy_thumbnails', 'PIL', # Uncomment the next line to enable the admin: 'django.contrib.admin', # Uncomment the next line to enable admin documentation: # 'django.contrib.admindocs', ) # A sample logging configuration. The only tangible logging # performed by this configuration is to send an email to # the site admins on every HTTP 500 error when DEBUG=False. # See http://docs.djangoproject.com/en/dev/topics/logging for # more details on how to customize your logging configuration. LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'filters': { 'require_debug_false': { '()': 'django.utils.log.RequireDebugFalse' } }, 'handlers': { 'mail_admins': { 'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler' } }, 'loggers': { 'django.request': { 'handlers': ['mail_admins'], 'level': 'ERROR', 'propagate': True, }, } } when i try to upload an image, in the clipboard section i don't have the thumbnail, but just an 'undefined' message: and this is the runserver console while trying to upload: [20/Oct/2012 15:15:56] "POST /admin/filer/clipboard/operations/upload/?qqfile=29708_1306856312320_7706073_n.jpg HTTP/1.1" 500 248133 [20/Oct/2012 15:15:56] "GET /it/admin/filer/folder/unfiled_images/undefined HTTP/1.1" 301 0 [20/Oct/2012 15:15:56] "GET /it/admin/filer/folder/unfiled_images/undefined/ HTTP/1.1" 404 1739 Also, this is project filesystem: cms1 +-- cms1 ¦   +-- __init__.py ¦   +-- __init__.pyc ¦   +-- media ¦   ¦   +-- filer_public ¦   ¦   +-- 2012 ¦   ¦   +-- 10 ¦   ¦   +-- 20 ¦   ¦   +-- 29708_1306856312320_7706073_n_1.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_2.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_3.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_4.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_5.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_6.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n_7.jpg ¦   ¦   +-- 29708_1306856312320_7706073_n.jpg ¦   ¦   +-- torrent-client-macosx.jpg ¦   +-- settings.py ¦   +-- settings.pyc ¦   +-- static ¦   +-- static_auto ¦   +-- static_manual ¦   +-- templates ¦   ¦   +-- base.html ¦   ¦   +-- template_1.html ¦   ¦   +-- template_2.html ¦   +-- urls.py ¦   +-- urls.pyc ¦   +-- wsgi.py ¦   +-- wsgi.pyc +-- manage.py So files are uploaded, but they are not accessible to cms. there's a similar question here, but doens't help me so much. It would be very helpful any help on this issue to me. Thanks, luke

  Read the article

• "Exception: msg 'axis2:null', not-found" when using a suds client with an axis2 server

  - by konrad

  I am writing a Suds (Python) SOAP client for an Axis2 server I have no control over. Suds chokes on the WSDL file with the following exception: File "site-packages/suds/wsdl.py", line 494, in resolve raise Exception("msg '%s', not-found" % op.input) Exception: msg 'axis2:null', not-found This is the WSDL file (I have replaced the hostnames with localhost). Any clue on how to fix this with the ImportDoctor? <?xml version="1.0" encoding="UTF-8"?> <wsdl:definitions xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:axis2="http://ws.apache.org/axis2" xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" targetNamespace="http://ws.apache.org/axis2"> <wsdl:types/> <wsdl:portType name="__SynapseServicePortType"> <wsdl:operation name="mediate"> <wsdl:input message="axis2:null" wsaw:Action="urn:mediate"/> <wsdl:output message="axis2:null" wsaw:Action="urn:mediateResponse"/> </wsdl:operation> </wsdl:portType> <wsdl:binding name="__SynapseServiceSoap11Binding" type="axis2:__SynapseServicePortType"> <soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/> <wsdl:operation name="mediate"> <soap:operation soapAction="urn:mediate" style="document"/> <wsdl:input> <soap:body use="literal"/> </wsdl:input> <wsdl:output> <soap:body use="literal"/> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:binding name="__SynapseServiceSoap12Binding" type="axis2:__SynapseServicePortType"> <soap12:binding transport="http://schemas.xmlsoap.org/soap/http" style="document"/> <wsdl:operation name="mediate"> <soap12:operation soapAction="urn:mediate" style="document"/> <wsdl:input> <soap12:body use="literal"/> </wsdl:input> <wsdl:output> <soap12:body use="literal"/> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:binding name="__SynapseServiceHttpBinding" type="axis2:__SynapseServicePortType"> <http:binding verb="POST"/> <wsdl:operation name="mediate"> <http:operation location="mediate"/> <wsdl:input> <mime:content type="text/xml" part="mediate"/> </wsdl:input> <wsdl:output> <mime:content type="text/xml" part="mediate"/> </wsdl:output> </wsdl:operation> </wsdl:binding> <wsdl:service name="__SynapseService"> <wsdl:port name="__SynapseServiceHttpsSoap11Endpoint" binding="axis2:__SynapseServiceSoap11Binding"> <soap:address location="https://localhost:8843/services/__SynapseService.__SynapseServiceHttpsSoap11Endpoint"/> </wsdl:port> <wsdl:port name="__SynapseServiceHttpSoap11Endpoint" binding="axis2:__SynapseServiceSoap11Binding"> <soap:address location="http://localhost:8880/services/__SynapseService.__SynapseServiceHttpSoap11Endpoint"/> </wsdl:port> <wsdl:port name="__SynapseServiceHttpsSoap12Endpoint" binding="axis2:__SynapseServiceSoap12Binding"> <soap12:address location="https://localhost:8843/services/__SynapseService.__SynapseServiceHttpsSoap12Endpoint"/> </wsdl:port> <wsdl:port name="__SynapseServiceHttpSoap12Endpoint" binding="axis2:__SynapseServiceSoap12Binding"> <soap12:address location="http://localhost:8880/services/__SynapseService.__SynapseServiceHttpSoap12Endpoint"/> </wsdl:port> <wsdl:port name="__SynapseServiceHttpsEndpoint" binding="axis2:__SynapseServiceHttpBinding"> <http:address location="https://localhost:8843/services/__SynapseService.__SynapseServiceHttpsEndpoint"/> </wsdl:port> <wsdl:port name="__SynapseServiceHttpEndpoint" binding="axis2:__SynapseServiceHttpBinding"> <http:address location="http://localhost:8880/services/__SynapseService.__SynapseServiceHttpEndpoint"/> </wsdl:port> </wsdl:service> </wsdl:definitions>

  Read the article

• Plone: Creating new Page fails "AttributeError: 'str' object has no attribute 'dates'"

  - by paskster

  I just installed Plone on my Centos 5.5. I was able login via the admin-account and create new users. But when I try to create a new page I get the following error message: Traceback (innermost last): Module ZPublisher.Publish, line 127, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 47, in call_object Module Products.CMFPlone.FactoryTool, line 446, in __call__ Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 47, in call_object Module Products.CMFFormController.FSControllerPageTemplate, line 91, in __call__ Module Products.CMFFormController.BaseControllerPageTemplate, line 31, in _call Module Shared.DC.Scripts.Bindings, line 324, in __call__ Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec Module Products.CMFCore.FSPageTemplate, line 240, in _exec Module Products.CMFCore.FSPageTemplate, line 180, in pt_render Module Products.PageTemplates.PageTemplate, line 80, in pt_render Module zope.pagetemplate.pagetemplate, line 113, in pt_render Warning: Macro expansion failed Warning: <type 'exceptions.KeyError'>: 'macro' Module zope.tal.talinterpreter, line 271, in __call__ Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 954, in do_defineSlot Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 858, in do_defineMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 954, in do_defineSlot Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 946, in do_defineSlot Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 858, in do_defineMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 852, in do_condition Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 954, in do_defineSlot Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 852, in do_condition Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 821, in do_loop_tal Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 522, in do_optTag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 821, in do_loop_tal Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 852, in do_condition Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 852, in do_condition Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 946, in do_defineSlot Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 583, in do_setLocal_tal Module zope.tales.tales, line 696, in evaluate URL: file:/usr/local/Plone/buildout-cache/eggs/Plone-4.0.2-py2.6.egg/Products/CMFPlone/skins/plone_templates/calendar_macros.pt Line 12, Column 4 Expression: <PythonExpr date_components_support_view.result(inputvalue, 0, starting_year, ending_year, future_years)> Names: {'container': <PloneSite at /reeple>, 'context': <ATDocument at /reeple/portal_factory/Document/document.2010-12-24.9331499294 used for /reeple>, 'default': <object object at 0x2ad1b9a18ae0>, 'here': <ATDocument at /reeple/portal_factory/Document/document.2010-12-24.9331499294 used for /reeple>, 'loop': {u'field': <Products.PageTemplates.Expressions.PathIterator object at 0x1bc9b9d0>, u'fieldset': <Products.PageTemplates.Expressions.PathIterator object at 0x1d396c90>}, 'nothing': None, 'options': {'args': (), 'state': <Products.CMFFormController.ControllerState.ControllerState object at 0x1ccdb2d0>}, 'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0x1d452ec0>, 'request': <HTTPRequest, URL=http://82.165.145.121:8081/reeple/portal_factory/Document/document.2010-12-24.9331499294/atct_edit>, 'root': <Application at >, 'template': <FSControllerPageTemplate at /reeple/atct_edit used for /reeple/portal_factory/Document/document.2010-12-24.9331499294>, 'traverse_subpath': [], 'user': <PloneUser 'pascalklein'>} Module Products.PageTemplates.ZRPythonExpr, line 49, in __call__ __traceback_info__: date_components_support_view.result(inputvalue, 0, starting_year, ending_year, future_years) Module PythonExpr, line 1, in <expression> Module plone.app.form.widgets.datecomponents, line 50, in result AttributeError: 'str' object has no attribute 'dates' Any suggestions? CentOS 5.5 has the Python Version 2.4. So I'm not sure if that causes the issue.

  Read the article

• Global name not defined

  - by anteater7171

  I wrote a CPU monitoring program in Python. For some reason sometimes the the program will run without any problem. Then other times the program won't even start because of the following error. Traceback (most recent call last): File "", line 244, in run_nodebug File "C:\Python26\CPUR1.7.pyw", line 601, in app = simpleapp_tk(None) File "C:\Python26\CPUR1.7.pyw", line 26, in init self.initialize() File "C:\Python26\CPUR1.7.pyw", line 107, in initialize self.F() File "C:\Python26\CPUR1.7.pyw", line 517, in F S2 = TL.entryVariableS.get() NameError: global name 'TL' is not defined I can't seem to find the problem, maybe someone more experienced may assist me? Here is a snippet of the part giving me trouble: (The second to last line in the snippet is what's giving me trouble) def E(self): if self.selectedM.get() =='Options...': Setup global TL TL = Tkinter.Toplevel(self) menu = Tkinter.Menu(TL) TL.config(menu=menu) filemenu = Tkinter.Menu(menu) menu.add_cascade(label="| Menu |", menu=filemenu) filemenu.add_command(label="Instruction Manual...", command=self.helpmenu) filemenu.add_command(label="About...", command=self.aboutmenu) filemenu.add_separator() filemenu.add_command(label="Exit Options", command=TL.destroy) filemenu.add_command(label="Exit", command=self.destroy) helpmenu = Tkinter.Menu(menu) menu.add_cascade(label="| Help |", menu=helpmenu) helpmenu.add_command(label="Instruction Manual...", command=self.helpmenu) helpmenu.add_separator() helpmenu.add_command(label="Quick Help...", command=self.helpmenu) Title TL.label5 = Tkinter.Label(TL,text="CPU Usage: Options",anchor="center",fg="black",bg="lightgreen",relief="ridge",borderwidth=5,font=('Arial', 18, 'bold')) TL.label5.pack(padx=15,ipadx=5) X Y scale TL.separator = Tkinter.Frame(TL,height=7, bd=1, relief='ridge', bg='grey95') TL.separator.pack(pady=5,padx=5) # TL.sclX = Tkinter.Scale(TL.separator, from_=0, to=1500, orient='horizontal', resolution=1, command=self.A) TL.sclX.grid(column=1,row=0,ipadx=27, sticky='w') TL.label1 = Tkinter.Label(TL.separator,text="X",anchor="s",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label1.grid(column=0,row=0, pady=1, sticky='S') TL.sclY = Tkinter.Scale(TL.separator, from_=0, to=1500, resolution=1, command=self.A) TL.sclY.grid(column=2,row=1,rowspan=2,sticky='e', padx=4) TL.label3 = Tkinter.Label(TL.separator,text="Y",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label3.grid(column=2,row=0, padx=10, sticky='e') TL.entryVariable2 = Tkinter.StringVar() TL.entry2 = Tkinter.Entry(TL.separator,textvariable=TL.entryVariable2, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entry2.grid(column=1,row=1,ipadx=20, pady=10,sticky='EW') TL.entry2.bind("<Return>", self.B) TL.label2 = Tkinter.Label(TL.separator,text="X:",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label2.grid(column=0,row=1, ipadx=4, sticky='W') TL.entryVariable1 = Tkinter.StringVar() TL.entry1 = Tkinter.Entry(TL.separator,textvariable=TL.entryVariable1, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entry1.grid(column=1,row=2,sticky='EW') TL.entry1.bind("<Return>", self.B) TL.label4 = Tkinter.Label(TL.separator,text="Y:", anchor="center",fg="black",bg="grey95",font=('Arial', 8 ,'bold')) TL.label4.grid(column=0,row=2, ipadx=4, sticky='W') TL.label7 = Tkinter.Label(TL.separator,text="Text Colour:",fg="black",bg="grey95",font=('Arial', 8 ,'bold'),justify='left') TL.label7.grid(column=1,row=3, sticky='W',padx=10,ipady=10,ipadx=30) TL.selectedP = Tkinter.StringVar() TL.opt1 = Tkinter.OptionMenu(TL.separator, TL.selectedP,'Normal', 'White','Black', 'Blue', 'Steel Blue','Green','Light Green','Yellow','Orange' ,'Red',command=self.G) TL.opt1.config(fg="black",bg="grey90",activebackground="grey90",activeforeground="black", anchor="center",relief="raised",direction='right',font=('Arial', 10)) TL.opt1.grid(column=1,row=4,sticky='EW',padx=20,ipadx=20) TL.selectedP.set('Normal') TL.sclS = Tkinter.Scale(TL.separator, from_=10, to=2000, orient='horizontal', resolution=10, command=self.H) TL.sclS.grid(column=1,row=5,ipadx=27, sticky='w') TL.sclS.set(600) TL.entryVariableS = Tkinter.StringVar() TL.entryS = Tkinter.Entry(TL.separator,textvariable=TL.entryVariableS, fg="grey15",bg="grey90",relief="sunken",insertbackground="black",borderwidth=5,font=('Arial', 10)) TL.entryS.grid(column=1,row=6,ipadx=20, pady=10,sticky='EW') TL.entryS.bind("<Return>", self.I) TL.entryVariableS.set(600) # TL.resizable(False,False) TL.title('Options') geomPatt = re.compile(r"(\d+)?x?(\d+)?([+-])(\d+)([+-])(\d+)") s = self.wm_geometry() m = geomPatt.search(s) X = m.group(4) Y = m.group(6) TL.sclY.set(Y) TL.sclX.set(X) if self.selectedM.get() == 'Exit': self.destroy() def F (self): G = round(psutil.cpu_percent(), 1) G1 = str(G) + '%' self.labelVariable.set(G1) if G < 5: self.imageLabel.configure(image=self.image0) if G >= 5: self.imageLabel.configure(image=self.image5) if G >= 10: self.imageLabel.configure(image=self.image10) if G >= 15: self.imageLabel.configure(image=self.image15) if G >= 20: self.imageLabel.configure(image=self.image20) if G >= 25: self.imageLabel.configure(image=self.image25) if G >= 30: self.imageLabel.configure(image=self.image30) if G >= 35: self.imageLabel.configure(image=self.image35) if G >= 40: self.imageLabel.configure(image=self.image40) if G >= 45: self.imageLabel.configure(image=self.image45) if G >= 50: self.imageLabel.configure(image=self.image50) if G >= 55: self.imageLabel.configure(image=self.image55) if G >= 60: self.imageLabel.configure(image=self.image60) if G >= 65: self.imageLabel.configure(image=self.image65) if G >= 70: self.imageLabel.configure(image=self.image70) if G >= 75: self.imageLabel.configure(image=self.image75) if G >= 80: self.imageLabel.configure(image=self.image80) if G >= 85: self.imageLabel.configure(image=self.image85) if G >= 90: self.imageLabel.configure(image=self.image90) if 100> G >= 95: self.imageLabel.configure(image=self.image95) if G == 100: self.imageLabel.configure(image=self.image100) S2 = TL.entryVariableS.get() self.after(int(S2), self.F)

  Read the article

• How to access members of an rdf list with rdflib (or plain sparql)

  - by tjb

  What is the best way to access the members of an rdf list? I'm using rdflib (python) but an answer given in plain SPARQL is also ok (this type of answer can be used through rdfextras, a rdflib helper library). I'm trying to access the authors of a particular journal article in rdf produced by Zotero (some fields have been removed for brevity): <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:z="http://www.zotero.org/namespaces/export#" xmlns:dcterms="http://purl.org/dc/terms/" xmlns:bib="http://purl.org/net/biblio#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:prism="http://prismstandard.org/namespaces/1.2/basic/" xmlns:link="http://purl.org/rss/1.0/modules/link/"> <bib:Article rdf:about="http://www.ncbi.nlm.nih.gov/pubmed/18273724"> <z:itemType>journalArticle</z:itemType> <dcterms:isPartOf rdf:resource="urn:issn:0954-6634"/> <bib:authors> <rdf:Seq> <rdf:li> <foaf:Person> <foaf:surname>Lee</foaf:surname> <foaf:givenname>Hyoun Seung</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Lee</foaf:surname> <foaf:givenname>Jong Hee</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Ahn</foaf:surname> <foaf:givenname>Gun Young</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Lee</foaf:surname> <foaf:givenname>Dong Hun</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Shin</foaf:surname> <foaf:givenname>Jung Won</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Kim</foaf:surname> <foaf:givenname>Dong Hyun</foaf:givenname> </foaf:Person> </rdf:li> <rdf:li> <foaf:Person> <foaf:surname>Chung</foaf:surname> <foaf:givenname>Jin Ho</foaf:givenname> </foaf:Person> </rdf:li> </rdf:Seq> </bib:authors> <dc:title>Fractional photothermolysis for the treatment of acne scars: a report of 27 Korean patients</dc:title> <dcterms:abstract>OBJECTIVES: Atrophic post-acne scarring remains a therapeutically challe *CUT*, erythema and edema. CONCLUSIONS: The 1550-nm erbium-doped FP is associated with significant patient-reported improvement in the appearance of acne scars, with minimal downtime.</dcterms:abstract> <bib:pages>45-49</bib:pages> <dc:date>2008</dc:date> <z:shortTitle>Fractional photothermolysis for the treatment of acne scars</z:shortTitle> <dc:identifier> <dcterms:URI> <rdf:value>http://www.ncbi.nlm.nih.gov/pubmed/18273724</rdf:value> </dcterms:URI> </dc:identifier> <dcterms:dateSubmitted>2010-12-06 11:36:52</dcterms:dateSubmitted> <z:libraryCatalog>NCBI PubMed</z:libraryCatalog> <dc:description>PMID: 18273724</dc:description> </bib:Article> <bib:Journal rdf:about="urn:issn:0954-6634"> <dc:title>The Journal of Dermatological Treatment</dc:title> <prism:volume>19</prism:volume> <prism:number>1</prism:number> <dcterms:alternative>J Dermatolog Treat</dcterms:alternative> <dc:identifier>DOI 10.1080/09546630701691244</dc:identifier> <dc:identifier>ISSN 0954-6634</dc:identifier> </bib:Journal>

  Read the article

• : in node causing Keyerror in xmlparsing using ElementTree

  - by kguckian

  Hi I'm using ElementTree to parse out an xml feed from Kuler. I'm only beginning in python but am stuck here. The parsing works fine until I attempt to retrieve any nodes containing ':' e.g kuler:swatchHexColor Below is a cut down version of the full feed but same structure: <rss xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:kuler="http://kuler.adobe.com/kuler/API/rss/" xmlns:rss="http://blogs.law.harvard.edu/tech/rss" version="2.0"> <channel> <title>kuler popular themes</title> <item> <title>Theme Title: Fresh Money</title> <description> &lt;img src="http://kuler-api.adobe.com/kuler/themeImages/theme_808366.png" /&gt;&lt;br /&gt; Artist: thesylph005&lt;br /&gt; ThemeID: 808366&lt;br /&gt; Posted: 03/02/2010&lt;br /&gt; Hex: 2F400D, 8CBF26, A8CA65, E8E5B0, 419184 </description> <kuler:themeItem> <kuler:themeID>808366</kuler:themeID> <kuler:themeTitle>Fresh Money</kuler:themeTitle> <kuler:themeImage>http://kuler-api.adobe.com/kuler/themeImages/theme_808366.png</kuler:themeImage> <kuler:themeAuthor> <kuler:authorID>370750</kuler:authorID> <kuler:authorLabel>thesylph005</kuler:authorLabel> </kuler:themeAuthor> <kuler:themeTags/> <kuler:themeRating>4</kuler:themeRating> <kuler:themeDownloadCount>708</kuler:themeDownloadCount> <kuler:themeCreatedAt>20100302</kuler:themeCreatedAt> <kuler:themeEditedAt>20100302</kuler:themeEditedAt> <kuler:themeSwatches> <kuler:swatch> <kuler:swatchHexColor>2F400D</kuler:swatchHexColor> <kuler:swatchColorMode>rgb</kuler:swatchColorMode> <kuler:swatchChannel1>0.183333</kuler:swatchChannel1> <kuler:swatchChannel2>0.25</kuler:swatchChannel2> <kuler:swatchChannel3>0.05</kuler:swatchChannel3> <kuler:swatchChannel4>0.0</kuler:swatchChannel4> <kuler:swatchIndex>0</kuler:swatchIndex> </kuler:swatch> <kuler:swatch> <kuler:swatchHexColor>8CBF26</kuler:swatchHexColor> <kuler:swatchColorMode>rgb</kuler:swatchColorMode> <kuler:swatchChannel1>0.55</kuler:swatchChannel1> <kuler:swatchChannel2>0.75</kuler:swatchChannel2> <kuler:swatchChannel3>0.15</kuler:swatchChannel3> <kuler:swatchChannel4>0.0</kuler:swatchChannel4> <kuler:swatchIndex>1</kuler:swatchIndex> </kuler:swatch> <kuler:swatch> <kuler:swatchHexColor>A8CA65</kuler:swatchHexColor> <kuler:swatchColorMode>rgb</kuler:swatchColorMode> <kuler:swatchChannel1>0.659722</kuler:swatchChannel1> <kuler:swatchChannel2>0.791667</kuler:swatchChannel2> <kuler:swatchChannel3>0.395833</kuler:swatchChannel3> <kuler:swatchChannel4>0.0</kuler:swatchChannel4> <kuler:swatchIndex>2</kuler:swatchIndex> </kuler:swatch> <kuler:swatch> <kuler:swatchHexColor>E8E5B0</kuler:swatchHexColor> <kuler:swatchColorMode>rgb</kuler:swatchColorMode> <kuler:swatchChannel1>0.91</kuler:swatchChannel1> <kuler:swatchChannel2>0.898047</kuler:swatchChannel2> <kuler:swatchChannel3>0.688705</kuler:swatchChannel3> <kuler:swatchChannel4>0.0</kuler:swatchChannel4> <kuler:swatchIndex>3</kuler:swatchIndex> </kuler:swatch> <kuler:swatch> <kuler:swatchHexColor>419184</kuler:swatchHexColor> <kuler:swatchColorMode>rgb</kuler:swatchColorMode> <kuler:swatchChannel1>0.254901</kuler:swatchChannel1> <kuler:swatchChannel2>0.57</kuler:swatchChannel2> <kuler:swatchChannel3>0.519034</kuler:swatchChannel3> <kuler:swatchChannel4>0.0</kuler:swatchChannel4> <kuler:swatchIndex>4</kuler:swatchIndex> </kuler:swatch> </kuler:themeSwatches> Tue, 30 Mar 2010 11:27:12 PST So if I do a findall on say each item's description, I get that back fine. But the minute I try to retrieve anything with a : in the nodename I get Exception Type: KeyError Exception Value: ':' So this works from elementtree.ElementTree import Element, SubElement, dump, parse def xml(): kulerurl = 'http://kuler-api.adobe.com/rss/get.cfm?listType=popular&startIndex=0&itemsPerPage=5&timeSpan=30&key=mykey' rss = parse(urllib.urlopen(kulerurl)).getroot() for element in rss.findall('channel/item'): print(element.findtext('description')) dump (rss) but this doesn't def xml(): kulerurl = 'http://kuler-api.adobe.com/rss/get.cfm?listType=popular&startIndex=0&itemsPerPage=5&timeSpan=30&key=mykey' rss = parse(urllib.urlopen(kulerurl)).getroot() for element in rss.findall('channel/item/kuler:themeItem'): print(element.findtext('kuler:themeID')) dump (rss) I'm sure it's something simple if anyone could point me to what I'm doing wrong here I'd be most grateful thanks Kieran

  Read the article

• Trouble with copying dictionaries and using deepcopy on an SQLAlchemy ORM object

  - by Az

  Hi there, I'm doing a Simulated Annealing algorithm to optimise a given allocation of students and projects. This is language-agnostic pseudocode from Wikipedia: s ? s0; e ? E(s) // Initial state, energy. sbest ? s; ebest ? e // Initial "best" solution k ? 0 // Energy evaluation count. while k < kmax and e > emax // While time left & not good enough: snew ? neighbour(s) // Pick some neighbour. enew ? E(snew) // Compute its energy. if enew < ebest then // Is this a new best? sbest ? snew; ebest ? enew // Save 'new neighbour' to 'best found'. if P(e, enew, temp(k/kmax)) > random() then // Should we move to it? s ? snew; e ? enew // Yes, change state. k ? k + 1 // One more evaluation done return sbest // Return the best solution found. The following is an adaptation of the technique. My supervisor said the idea is fine in theory. First I pick up some allocation (i.e. an entire dictionary of students and their allocated projects, including the ranks for the projects) from entire set of randomised allocations, copy it and pass it to my function. Let's call this allocation aOld (it is a dictionary). aOld has a weight related to it called wOld. The weighting is described below. The function does the following: Let this allocation, aOld be the best_node From all the students, pick a random number of students and stick in a list Strip (DEALLOCATE) them of their projects ++ reflect the changes for projects (allocated parameter is now False) and lecturers (free up slots if one or more of their projects are no longer allocated) Randomise that list Try assigning (REALLOCATE) everyone in that list projects again Calculate the weight (add up ranks, rank 1 = 1, rank 2 = 2... and no project rank = 101) For this new allocation aNew, if the weight wNew is smaller than the allocation weight wOld I picked up at the beginning, then this is the best_node (as defined by the Simulated Annealing algorithm above). Apply the algorithm to aNew and continue. If wOld < wNew, then apply the algorithm to aOld again and continue. The allocations/data-points are expressed as "nodes" such that a node = (weight, allocation_dict, projects_dict, lecturers_dict) Right now, I can only perform this algorithm once, but I'll need to try for a number N (denoted by kmax in the Wikipedia snippet) and make sure I always have with me, the previous node and the best_node. So that I don't modify my original dictionaries (which I might want to reset to), I've done a shallow copy of the dictionaries. From what I've read in the docs, it seems that it only copies the references and since my dictionaries contain objects, changing the copied dictionary ends up changing the objects anyway. So I tried to use copy.deepcopy().These dictionaries refer to objects that have been mapped with SQLA. Questions: I've been given some solutions to the problems faced but due to my über green-ness with using Python, they all sound rather cryptic to me. Deepcopy isn't playing nicely with SQLA. I've been told thatdeepcopy on ORM objects probably has issues that prevent it from working as you'd expect. Apparently I'd be better off "building copy constructors, i.e. def copy(self): return FooBar(....)." Can someone please explain what that means? I checked and found out that deepcopy has issues because SQLAlchemy places extra information on your objects, i.e. an _sa_instance_state attribute, that I wouldn't want in the copy but is necessary for the object to have. I've been told: "There are ways to manually blow away the old _sa_instance_state and put a new one on the object, but the most straightforward is to make a new object with __init__() and set up the attributes that are significant, instead of doing a full deep copy." What exactly does that mean? Do I create a new, unmapped class similar to the old, mapped one? An alternate solution is that I'd have to "implement __deepcopy__() on your objects and ensure that a new _sa_instance_state is set up, there are functions in sqlalchemy.orm.attributes which can help with that." Once again this is beyond me so could someone kindly explain what it means? A more general question: given the above information are there any suggestions on how I can maintain the information/state for the best_node (which must always persist through my while loop) and the previous_node, if my actual objects (referenced by the dictionaries, therefore the nodes) are changing due to the deallocation/reallocation taking place? That is, without using copy?

  Read the article

• Wierd characters in exported csv files when converting

  - by Ahue

  Hey guys, I came across a problem I cannot solve on my own concerning the downloadable csv formatted trends data files from Google Insights for Search. I'm to lazy to reformat the files I4S gives me manually what means: Extracting the section with the actual trends data and reformatting the columns so that I can use it with a modelling program I do for school. So I wrote a tiny script the should do the work for me: Taking a file, do some magic and give me a new file in proper format. What it's supposed to do is reading the file contents, extracting the trends section, splitting it by newlines, splitting each line and then reorder the columns and maybe reformat them. When looking at a untouched I4S csv file it looks normal containing CR LF caracters at line breaks (maybe thats only because I'm using Windows). When just reading the contents and then writing them to a new file using the script wierd asian characters appear between CR and LF. I tried the script with a manually written similar looking file and even tried a csv file from Google Trends and it works fine. I use Python and the script (snippet) I used for the following example looks like this: # Read from an input file file = open(file,"r") contents = file.read() file.close() cfile = open("m.log","w+") cfile.write(contents) cfile.close() Has anybody an idea why those characters appear??? Thank you for you help! I'll give you and example: First few lines of I4S csv file: Web Search Interest: foobar Worldwide; 2004 - present Interest over time Week foobar 2004-01-04 - 2004-01-10 44 2004-01-11 - 2004-01-17 44 2004-01-18 - 2004-01-24 37 2004-01-25 - 2004-01-31 40 2004-02-01 - 2004-02-07 49 2004-02-08 - 2004-02-14 51 2004-02-15 - 2004-02-21 45 2004-02-22 - 2004-02-28 61 2004-02-29 - 2004-03-06 51 2004-03-07 - 2004-03-13 48 2004-03-14 - 2004-03-20 50 2004-03-21 - 2004-03-27 56 2004-03-28 - 2004-04-03 59 Output file when reading and writing contents: Web Search Interest: foobar ??????????? ? ? ? ????????? ????????? ???? ?????? Week foobar ?? ?? ?? ? ? ? ?? ??? ????? 2004-01-11 - 2004-01-17 44 ?? ?? ???? ? ? ?? ????????? 2004-01-25 - 2004-01-31 40 ?? ?? ?? ? ? ? ?? ?? ?????? 2004-02-08 - 2004-02-14 51 ?? ?? ???? ? ? ?? ????????? 2004-02-22 - 2004-02-28 61 ?? ?? ???? ? ? ?? ?? ?????? 2004-03-07 - 2004-03-13 48 ?? ?? ???? ? ? ?? ??? ?? ?? 2004-03-21 - 2004-03-27 56 ?? ?? ???? ? ? ?? ?? ?????? 2004-04-04 - 2004-04-10 69 ?? ?? ???? ? ? ?? ????????? 2004-04-18 - 2004-04-24 51 ?? ?? ???? ? ? ?? ?? ?????? 2004-05-02 - 2004-05-08 56 ?? ?? ?? ? ? ? ?? ????????? 2004-05-16 - 2004-05-22 54 ?? ?? ???? ? ? ?? ????????? 2004-05-30 - 2004-06-05 74 ?? ?? ?? ? ? ? ?? ????????? 2004-06-13 - 2004-06-19 50 ?? ?? ??? ? ? ?? ????????? 2004-06-27 - 2004-07-03 58 ?? ?? ?? ? ? ? ?? ??? ????? 2004-07-11 - 2004-07-17 59 ?? ?? ???? ? ? ?? ?????????

  Read the article

• Création d'une application « Hello World » dans Visual Studio Lightswitch, un article d'Avkash Chauhan, traduit par Deepin Prayag

  Citation: Visual Studio LightSwitch 2011 propose des « starter kits » et des options de déploiement flexibles qui vous aident à créer et à facilement publier des applications métier personnalisées à l'aspect professionnel et distingué, sans nécessité de code. Visual Studio LightSwitch propose une manière simple de développer des applications métier de type bureau ou Cloud. LightSwitch gère toute la plomberie pour vous, afin que vous puissiez vous concentrer sur la création de valeurs métier. Partie 1 :

  Read the article

• ASP.NET and Silverlight Conference with Scott Guthrie and Visual Studio 2010 Launch!

  Hear Scott Guthrie and key speakers talk about Silverlight 4, ASP.NET 4.0, MVC, AJAX etc - and see how it’s all supported in Visual Studio 2010! We'll even throw in the global Launch of Visual Studio 2010! Early bird ends March 8!

  Read the article

• Free training at Northwest Cadence

  - by Martin Hinshelwood

  Even though I have only been at Northwest Cadence for a short time I have already done so much. What I really wanted to do was let you guys know about a bunch of FREE training that NWC offers. These sessions are at a fantastic time for the UK as 9am PST (Seattle time) is around 5pm GMT. Its a fantastic way to finish off your Fridays and with the lack of love for developers in the UK set to continue I would love some of you guys to get some from the US instead. There are really two offerings. The first is something called Coffee talks that take you through an hours worth of detail in a specific category. Coffee Talks These coffee talks have some superb topics and you can get excellent interaction with the presenter as they are kind of informal. Date Day Time Topic Register Here 01/04/11 Tuesday 8:30AM – 9:30AM PST Real World Business and Technical Benefits of ALM with TFS 2010 150656 01/28/11 Friday 9:00AM - 10:00AM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152810 02/11/11 Friday 9:00AM - 10:00AM PST Visual Source Safe to Team Foundation Server 152844 02/25/11 Friday 2:00PM - 3:00PM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152816 03/11/11 Friday 9:00AM - 10:00AM PST Lab Manager The Ultimate “No More No Repro” Tool 152809 03/25/11 Friday 9:00AM - 10:00AM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152838 04/08/11 Friday 9:00AM - 10:00AM PST Visual Source Safe to Team Foundation Server 152846 04/22/11 Friday 9:00AM - 10:00AM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152839 05/06/11 Friday 2:00PM - 3:00PM PST Real World Business and Technical Benefits of ALM with TFS 2010 150657 05/20/11 Friday 9:00AM - 10:00AM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152842 06/03/11 Friday 9:00AM - 10:00AM PST Visual Source Safe to Team Foundation Server 152847 06/17/11 Friday 9:00AM - 10:00AM PST The Full Testing Experience Professional Quality Assurance with Visual Studio 2010 152843   ALM Training Engagement Program Microsoft has released a new program to bring free Visual Studio 2010 Training Sessions to select customers on Microsoft Visual Studio products and how Application Lifecycle Management (ALM) solutions can help drive greater business impact. For more details on this program, please see the process chart below.  To get started send an email to us; This training is paid for by Microsoft and you would need to commit to 4 sessions in order to get accepted into the program. So these have more hoops to jump through to get them, but the content is much more formal and centres around adoption.

  Read the article

• Visual Studio 2010 How to Videos for SharePoint 2010 Development.

  - by Harish Pavithran

  Nice blog post from Beth Massi talking about MSDN How do I video for SharePoint 2010 and Visual Studio, watch how easy it is to deploy a custom workflow and debug it right from VS 2010! I love it... enjoy http://blogs.msdn.com/bethmassi/archive/2010/04/27/more-how-do-i-videos-sharepoint-development-with-visual-studio-2010.aspx

  Read the article

• PySide devient un add-on Qt, le binding Python initié par Nokia rejoint le Qt Project et le modèle d'open gouvernance

  PySide devient un add-on Qt Le binding Python initié par Nokia est toujours disponible sous la même licence [IMG]http://ftp-developpez.com/gordon-fowler/PySide.png[/IMG] Le Qt Project étant arrivé depuis quelques mois, rien de plus normal que de voir le binding Python initié par Nokia le rejoindre : ce projet est maintenant plus aligné avec le framework Qt et bénéficie de toute l'infrastructure mise en place (nouvel emplacement pour la mailing list,

  Read the article

• How were some language communities (eg, Ruby and Python) able to prevent fragmentation while others (eg, Lisp or ML) were not?

  - by chrisaycock

  The term "Lisp" (or "Lisp-like") is an umbrella for lots of different languages, such as Common Lisp, Scheme, and Arc. There is similar fragmentation is other language communities, like in ML. However, Ruby and Python have both managed to avoid this fate, where innovation occurred more on the implementation (like PyPy or YARV) instead of making changes to the language itself. Did the Ruby and Python communities do something special to prevent language fragmentation?

  Read the article

• Le Service Pack 1 de Visual Studio 2010 est disponible, avec le Feature Pack de Team Foundation Server-Project Server Integration

  Le Service Pack 1 de Visual Studio 2010 est disponible Avec le Feature Pack de Team Foundation Server - Project Server Integration Mise à jour du 10/03/11 Disponible en version bêta depuis décembre 2010, le Service Pack 1 (SP1) de Visual Studio 2010 est désormais disponible en version finale pour les développeurs. Cette version corrige plusieurs bogues de la version beta et offre des fonctionnalités permettant un meilleur support ainsi que l'IntelliTrace pour SharePoint. L'IntelliTrace permet une amélioration du débogage en donnant la possibilité aux développeurs de suivre les événements lors du processus, au lieu d'avoir à les ...

  Read the article

• Découvrez les éditions de Visual Studio 2012 et leurs nouvelles fonctionnalités de design et de gestion du cycle de vie des applications

  Découvrez les éditions de Visual Studio 2012 ainsi que les fonctionnalités adaptées à vos besoins de la solution de gestion des projets logiciels Mise à jour du 24/09/2012 Visual Studio 2012 est disponible depuis le 12 septembre 2012. La solution propose une nouvelle génération d'outils de développement, de collaboration et de gestion du cycle de vie pour la conception des applications non seulement modernes et attrayantes, mais aussi accessibles n'importe où, n'importe quand, répondant aux attentes des clients. [IMG]http://rdonfack.developpez.com/images/vs20...

  Read the article

• Visual-Staff : un pôle de compétences 100% .NET qui se positionne en alternative aux SSII et aux cabinets de recrutement

  FLX lance Visual-Staff pour les développeurs .NET un pôle de compétences exclusivement spécialisé sur les technologies .NETLa pénurie de ressources dans le secteur du développement informatique est permanente. Il faut donc des solutions adaptées pour trouver des ressources compétentes et disponibles sur les technologies .NET.Après le succès et les enseignements tirés de Delphi-Staff pour les développeurs DELPHI, FLX lance Visual-Staff pour les développeurs .NET, la nouvelle Business Unit spécialisée sur le développement appliqué à la plateforme .NET.Ce service se positionne comme une alternative aux SSII et cabinets de recrutements généralistes pour tous ceux qui développent pour .NET ou recherch...

  Read the article

• Windows Azure SDK 1.6 disponible : simplification du déploiement avec Visual Studio et introduction de nouvelles bibliothèques .NET

  Windows Azure SDK 1.6 disponible : simplification du déploiement avec Visual Studio et introduction de nouvelles bibliothèques .NET Microsoft vient de publier une nouvelle version du kit de développement pour Windows Azure. Le SDK 1.6 pour Windows Azure intègre de nouveaux outils de la plateforme d'hébergement Cloud de Microsoft pour Visual Studio 2010, de nouvelles bibliothèques .NET pour Windows Azure ainsi que des améliorations de performance et corrections de bugs. L'outil réduit les contraintes liées au déploiement des applications sur Windows Azure, dont la configuration peut se faire désormais en seulement quelques clics, sans quitter l'environnement de développemen...

  Read the article

• « Pourquoi développer en MATLAB plutôt qu'en Java, Python ou C# ? », un séminaire gratuit de MathWorks le 13 décembre

  Pourquoi développer en MATLAB Plutôt qu'en Java, Python ou C# : un séminaire gratuit de MathWorks le 13 décembre MathWorks organise ce 13 décembre à La Défense un séminaire pour illustrer les atouts de MATLAB. « Lors de cette rencontre en matinée, vous découvrirez que MATLAB n'est pas simplement un langage de calcul scientifique mais un langage ouvert vous permettant de structurer vos développements d'applications d'entreprise », promet l'éditeur de MATLAB qui le compare aujourd'hui à Java, Python ou C#. La matinée illustrera l'utilisation d'interfaces externes (APIs) et la programmat...

  Read the article

• PHP introduit les générateurs, par un mécanisme similaire à celui de Python avec le mot-clé yield

  PHP introduit les générateurs Par un mécanisme similaire à celui de Python avec le mot-clé yield Les générateurs sont des moyens simples et puissants pour créer des itérateurs dans des langages tels que Python. Maintenant, c'est PHP qui fait le pas et s'approprie ce concept. Pour comprendre l'utilité et la puissance de ce dernier, on revoit l'exemple typique de lecture d'un fichier en entier : Code : Sélectionner tout - ...

  Read the article

• Error codes for C++

  - by billy

  #include <iostream> #include <iomanip> using namespace std; //Global constant variable declaration const int MaxRows = 8, MaxCols = 10, SEED = 10325; //Functions Declaration void PrintNameHeader(ostream& out); void Fill2DArray(double ary[][MaxCols]); void Print2DArray(const double ary[][MaxCols]); double GetTotal(const double ary[][MaxCols]); double GetAverage(const double ary[][MaxCols]); double GetRowTotal(const double ary[][MaxCols], int theRow); double GetColumnTotal(const double ary[][MaxCols], int theRow); double GetHighestInRow(const double ary[][MaxCols], int theRow); double GetLowestInRow(const double ary[][MaxCols], int theRow); double GetHighestInCol(const double ary[][MaxCols], int theCol); double GetLowestInCol(const double ary[][MaxCols], int theCol); double GetHighest(const double ary[][MaxCols], int& theRow, int& theCol); double GetLowest(const double ary[][MaxCols], int& theRow, int& theCol); int main() { int theRow; int theCol; PrintNameHeader(cout); cout << fixed << showpoint << setprecision(1); srand(static_cast<unsigned int>(SEED)); double ary[MaxRows][MaxCols]; cout << "The seed value for random number generator is: " << SEED << endl; cout << endl; Fill2DArray(ary); Print2DArray(ary); cout << " The Total for all the elements in this array is: " << setw(7) << GetTotal(ary) << endl; cout << "The Average of all the elements in this array is: " << setw(7) << GetAverage(ary) << endl; cout << endl; cout << "The sum of each row is:" << endl; for(int index = 0; index < MaxRows; index++) { cout << "Row " << (index + 1) << ": " << GetRowTotal(ary, theRow) << endl; } cout << "The highest and lowest of each row is: " << endl; for(int index = 0; index < MaxCols; index++) { cout << "Row " << (index + 1) << ": " << GetHighestInRow(ary, theRow) << " " << GetLowestInRow(ary, theRow) << endl; } cout << "The highest and lowest of each column is: " << endl; for(int index = 0; index < MaxCols; index++) { cout << "Col " << (index + 1) << ": " << GetHighestInCol(ary, theRow) << " " << GetLowestInCol(ary, theRow) << endl; } cout << "The highest value in all the elements in this array is: " << endl; cout << GetHighest(ary, theRow, theCol) << "[" << theRow << "]" << "[" << theCol << "]" << endl; cout << "The lowest value in all the elements in this array is: " << endl; cout << GetLowest(ary, theRow, theCol) << "[" << theRow << "]" << "[" << theCol << "]" << endl; return 0; } //Define Functions void PrintNameHeader(ostream& out) { out << "*******************************" << endl; out << "* *" << endl; out << "* C.S M10A Spring 2010 *" << endl; out << "* Programming Assignment 10 *" << endl; out << "* Due Date: Thurs. Mar. 25 *" << endl; out << "*******************************" << endl; out << endl; } void Fill2DArray(double ary[][MaxCols]) { for(int index1 = 0; index1 < MaxRows; index1++) { for(int index2= 0; index2 < MaxCols; index2++) { ary[index1][index2] = (rand()%1000)/10; } } } void Print2DArray(const double ary[][MaxCols]) { cout << " Column "; for(int index = 0; index < MaxCols; index++) { int column = index + 1; cout << " " << column << " "; } cout << endl; cout << " "; for(int index = 0; index < MaxCols; index++) { int column = index +1; cout << "----- "; } cout << endl; for(int index1 = 0; index1 < MaxRows; index1++) { cout << "Row " << (index1 + 1) << ":"; for(int index2= 0; index2 < MaxCols; index2++) { cout << setw(6) << ary[index1][index2]; } } } double GetTotal(const double ary[][MaxCols]) { double total = 0; for(int theRow = 0; theRow < MaxRows; theRow++) { total = total + GetRowTotal(ary, theRow); } return total; } double GetAverage(const double ary[][MaxCols]) { double total = 0, average = 0; total = GetTotal(ary); average = total / (MaxRows * MaxCols); return average; } double GetRowTotal(const double ary[][MaxCols], int theRow) { double sum = 0; for(int index = 0; index < MaxCols; index++) { sum = sum + ary[theRow][index]; } return sum; } double GetColumTotal(const double ary[][MaxCols], int theCol) { double sum = 0; for(int index = 0; index < theCol; index++) { sum = sum + ary[index][theCol]; } return sum; } double GetHighestInRow(const double ary[][MaxCols], int theRow) { double highest = 0; for(int index = 0; index < MaxCols; index++) { if(ary[theRow][index] > highest) highest = ary[theRow][index]; } return highest; } double GetLowestInRow(const double ary[][MaxCols], int theRow) { double lowest = 0; for(int index = 0; index < MaxCols; index++) { if(ary[theRow][index] < lowest) lowest = ary[theRow][index]; } return lowest; } double GetHighestInCol(const double ary[][MaxCols], int theCol) { double highest = 0; for(int index = 0; index < MaxRows; index++) { if(ary[index][theCol] > highest) highest = ary[index][theCol]; } return highest; } double GetLowestInCol(const double ary[][MaxCols], int theCol) { double lowest = 0; for(int index = 0; index < MaxRows; index++) { if(ary[index][theCol] < lowest) lowest = ary[index][theCol]; } return lowest; } double GetHighest(const double ary[][MaxCols], int& theRow, int& theCol) { theRow = 0; theCol = 0; double highest = ary[theRow][theCol]; for(int index = 0; index < MaxRows; index++) { for(int index1 = 0; index1 < MaxCols; index1++) { double highest = 0; if(ary[index1][theCol] > highest) { highest = ary[index][index1]; theRow = index; theCol = index1; } } } return highest; } double Getlowest(const double ary[][MaxCols], int& theRow, int& theCol) { theRow = 0; theCol = 0; double lowest = ary[theRow][theCol]; for(int index = 0; index < MaxRows; index++) { for(int index1 = 0; index1 < MaxCols; index1++) { double lowest = 0; if(ary[index1][theCol] < lowest) { lowest = ary[index][index1]; theRow = index; theCol = index1; } } } return lowest; } . 1>------ Build started: Project: teddy lab 10, Configuration: Debug Win32 ------ 1>Compiling... 1>lab 10.cpp 1>c:\users\owner\documents\visual studio 2008\projects\teddy lab 10\teddy lab 10\ lab 10.cpp(46) : warning C4700: uninitialized local variable 'theRow' used 1>c:\users\owner\documents\visual studio 2008\projects\teddy lab 10\teddy lab 10\ lab 10.cpp(62) : warning C4700: uninitialized local variable 'theCol' used 1>Linking... 1> lab 10.obj : error LNK2028: unresolved token (0A0002E0) "double __cdecl GetLowest(double const (* const)[10],int &,int &)" (?GetLowest@@$$FYANQAY09$$CBNAAH1@Z) referenced in function "int __cdecl main(void)" (?main@@$$HYAHXZ) 1> lab 10.obj : error LNK2019: unresolved external symbol "double __cdecl GetLowest(double const (* const)[10],int &,int &)" (?GetLowest@@$$FYANQAY09$$CBNAAH1@Z) referenced in function "int __cdecl main(void)" (?main@@$$HYAHXZ) 1>C:\Users\owner\Documents\Visual Studio 2008\Projects\ lab 10\Debug\ lab 10.exe : fatal error LNK1120: 2 unresolved externals 1>Build log was saved at "file://c:\Users\owner\Documents\Visual Studio 2008\Projects\ lab 10\teddy lab 10\Debug\BuildLog.htm" 1>teddy lab 10 - 3 error(s), 2 warning(s) ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

  Read the article

• Numpy zero rank array indexing/broadcasting

  - by Lemming

  I'm trying to write a function that supports broadcasting and is fast at the same time. However, numpy's zero-rank arrays are causing trouble as usual. I couldn't find anything useful on google, or by searching here. So, I'm asking you. How should I implement broadcasting efficiently and handle zero-rank arrays at the same time? This whole post became larger than anticipated, sorry. Details: To clarify what I'm talking about I'll give a simple example: Say I want to implement a Heaviside step-function. I.e. a function that acts on the real axis, which is 0 on the negative side, 1 on the positive side, and from case to case either 0, 0.5, or 1 at the point 0. Implementation Masking The most efficient way I found so far is the following. It uses boolean arrays as masks to assign the correct values to the corresponding slots in the output vector. from numpy import * def step_mask(x, limit=+1): """Heaviside step-function. y = 0 if x < 0 y = 1 if x > 0 See below for x == 0. Arguments: x Evaluate the function at these points. limit Which limit at x == 0? limit > 0: y = 1 limit == 0: y = 0.5 limit < 0: y = 0 Return: The values corresponding to x. """ b = broadcast(x, limit) out = zeros(b.shape) out[x>0] = 1 mask = (limit > 0) & (x == 0) out[mask] = 1 mask = (limit == 0) & (x == 0) out[mask] = 0.5 mask = (limit < 0) & (x == 0) out[mask] = 0 return out List Comprehension The following-the-numpy-docs way is to use a list comprehension on the flat iterator of the broadcast object. However, list comprehensions become absolutely unreadable for such complicated functions. def step_comprehension(x, limit=+1): b = broadcast(x, limit) out = empty(b.shape) out.flat = [ ( 1 if x_ > 0 else ( 0 if x_ < 0 else ( 1 if l_ > 0 else ( 0.5 if l_ ==0 else ( 0 ))))) for x_, l_ in b ] return out For Loop And finally, the most naive way is a for loop. It's probably the most readable option. However, Python for-loops are anything but fast. And hence, a really bad idea in numerics. def step_for(x, limit=+1): b = broadcast(x, limit) out = empty(b.shape) for i, (x_, l_) in enumerate(b): if x_ > 0: out[i] = 1 elif x_ < 0: out[i] = 0 elif l_ > 0: out[i] = 1 elif l_ < 0: out[i] = 0 else: out[i] = 0.5 return out Test First of all a brief test to see if the output is correct. >>> x = array([-1, -0.1, 0, 0.1, 1]) >>> step_mask(x, +1) array([ 0., 0., 1., 1., 1.]) >>> step_mask(x, 0) array([ 0. , 0. , 0.5, 1. , 1. ]) >>> step_mask(x, -1) array([ 0., 0., 0., 1., 1.]) It is correct, and the other two functions give the same output. Performance How about efficiency? These are the timings: In [45]: xl = linspace(-2, 2, 500001) In [46]: %timeit step_mask(xl) 10 loops, best of 3: 19.5 ms per loop In [47]: %timeit step_comprehension(xl) 1 loops, best of 3: 1.17 s per loop In [48]: %timeit step_for(xl) 1 loops, best of 3: 1.15 s per loop The masked version performs best as expected. However, I'm surprised that the comprehension is on the same level as the for loop. Zero Rank Arrays But, 0-rank arrays pose a problem. Sometimes you want to use a function scalar input. And preferably not have to worry about wrapping all scalars in at least 1-D arrays. >>> step_mask(1) Traceback (most recent call last): File "<ipython-input-50-91c06aa4487b>", line 1, in <module> step_mask(1) File "script.py", line 22, in step_mask out[x>0] = 1 IndexError: 0-d arrays can't be indexed. >>> step_for(1) Traceback (most recent call last): File "<ipython-input-51-4e0de4fcb197>", line 1, in <module> step_for(1) File "script.py", line 55, in step_for out[i] = 1 IndexError: 0-d arrays can't be indexed. >>> step_comprehension(1) array(1.0) Only the list comprehension can handle 0-rank arrays. The other two versions would need special case handling for 0-rank arrays. Numpy gets a bit messy when you want to use the same code for arrays and scalars. However, I really like to have functions that work on as arbitrary input as possible. Who knows which parameters I'll want to iterate over at some point. Question: What is the best way to implement a function as the one above? Is there a way to avoid if scalar then like special cases? I'm not looking for a built-in Heaviside. It's just a simplified example. In my code the above pattern appears in many places to make parameter iteration as simple as possible without littering the client code with for loops or comprehensions. Furthermore, I'm aware of Cython, or weave & Co., or implementation directly in C. However, the performance of the masked version above is sufficient for the moment. And for the moment I would like to keep things as simple as possible.

  Read the article

• pyqt QObject: Cannot create children for a parent that is in a different thread

  - by memomk

  QObject: Cannot create children for a parent that is in a different thread. (Parent is QTextDocument(0x9919018), parent's thread is QThread(0x97331e0), current thread is flooderthread(0x97b4c10) error means ? am sorry because am new to pyqt here is the code : i know the code is finished yet but it should work i guess the problem is with myfun.log function... #! /usr/bin/python # -*- coding: utf-8 -*- import urllib, urllib2, itertools, threading, cookielib, Cookie, sys, time, hashlib, os from PyQt4 import QtCore, QtGui try: _fromUtf8 = QtCore.QString.fromUtf8 except AttributeError: _fromUtf8 = lambda s: s gui=QtGui.QApplication.processEvents texttoset="" class fun(): global texttoset def checkpassword(self): if ui.passwordcheck.isChecked()==True: return 1 else : return 0 def log(self, text): if text != False: firsttext=str(ui.console.toPlainText()) secondtext=firsttext+text+"\n" ui.console.setText(secondtext) log=open("log.log", "a") log.write(text+"\n") log.close() else : firsttext=str(ui.console.toPlainText()) secondtext=firsttext+texttoset+"\n" ui.console.setText(secondtext) log=open("log.log", "a") log.write(texttoset+"\n") log.close() def disable(self): MainWindow.setEnabled(False) pass def enable(self): MainWindow.setEnabled(True) pass def checkmethod(self): if ui.get.isChecked()==True: return 1 elif ui.post.isChecked()==True: return 2 else : return 0 def main(self): connecter() gui() f1.start() gui() time.sleep(3) gui() f2.start() gui() time.sleep(3) gui() f3.start() gui() time.sleep(3) gui() f4.start() gui() time.sleep(3) gui() f5.start() gui() self.sleep(3) gui() f6.start() gui() def killer(self): f1.terminate() f2.terminate() f3.terminate() f4.terminate() f5.terminate() f6.terminate() def close(self): self.killer() os.abort() sys.exit() myfun=fun() def connecter(): QtCore.QObject.connect(f1, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f1, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f1, QtCore.SIGNAL("disable()"), myfun.disable) QtCore.QObject.connect(f2, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f2, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f2, QtCore.SIGNAL("disable()"), myfun.disable) QtCore.QObject.connect(f3, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f3, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f3, QtCore.SIGNAL("disable()"), myfun.disable) QtCore.QObject.connect(f4, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f4, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f4, QtCore.SIGNAL("disable()"), myfun.disable) QtCore.QObject.connect(f5, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f5, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f5, QtCore.SIGNAL("disable()"), myfun.disable) QtCore.QObject.connect(f6, QtCore.SIGNAL("log(bool)"), myfun.log) QtCore.QObject.connect(f6, QtCore.SIGNAL("enable()"), myfun.enable) QtCore.QObject.connect(f6, QtCore.SIGNAL("disable()"), myfun.disable) x=0 num=0 class flooderthread(QtCore.QThread): global texttoset def __init__(self, x, num): QtCore.QThread.__init__(self) self.x=x self.num=num def log(self, text): texttolog=str(text) time.sleep(1) self.emit(QtCore.SIGNAL("log(bool)"), False) time.sleep(2) def enable(self): time.sleep(1) self.emit(QtCore.SIGNAL("enable()")) def disable(self): time.sleep(1) self.emit(QtCore.SIGNAL("disable()")) def run(self): connecter() self.log("\n\n--------------------------------------------------new session-------------------------------------\n\n") itered=False gui() self.disable() gui() self.log("setting params...") param={ui.dataname1.text():ui.datavalue1.text(),ui.dataname3.text():ui.datavalue3.text(),ui.dataname3.text():ui.datavalue3.text(), } self.log("checking password...") if myfun.checkpassword()==1: itered=True self.log("password is true") else : self.log("password is null ") self.log("itered operation") self.log("setting url") url=str(ui.url.text()) if url[:4]!="http" and url[:3]!="ftp": self.log("url error exiting the whole function") self.log("please set a valide protocole!!") gui() self.enable() gui() return 1 pass else : self.log("valid url") gui() self.log("url is "+url) self.log("setting proxy") proxy="http://"+ui.proxyuser.text()+":"+ui.proxypass.text()+"@"+ui.proxyhost.text()+":"+ui.proxyport.text() self.log("proxy is "+proxy) gui() self.log("preparing params...") urlparam=urllib.urlencode(param) gui() self.log("params are "+urlparam) self.log("setting up headers...") header={'User-Agent':str(ui.useragent.toPlainText())} self.log("headers are "+ str(header)) self.log("setting up proxy handler..") proxyhandler=urllib2.ProxyHandler({"http":str(proxy)}) self.log("checking method") if myfun.checkmethod()==1: self.log("method is get..") self.log("setting request..") finalurl=url+urlparam gui() self.log("final url is"+finalurl) req=urllib2.Request(finalurl, None, headers) elif myfun.checkmethod()==2: self.log("method is post...") self.log("setting request..") finalurl=url gui() self.log("final url is "+finalurl) req=urllib2.Request(finalurl, urlparam, header) else : self.log("error has been accourded") self.log("please select a method!!") gui() self.log("exiting the whole functions") gui() self.enable() return 1 pass self.log("intilizing cookies..") c1=Cookie.SimpleCookie() c1[str(ui.cookiename1.text())]=str(ui.cookievalue1.text()) c1[str(ui.cookiename1.text())]['path']='/' c1[str(ui.cookiename2.text())]=str(ui.cookievalue2.text()) c1[str(ui.cookiename2.text())]['path']='/' c1[str(ui.cookiename3.text())]=str(ui.cookievalue3.text()) c1[str(ui.cookiename3.text())]['domain']=url c1[str(ui.cookiename3.text())]['path']='/' c1[str(ui.cookiename4.text())]=str(ui.cookievalue4.text()) c1[str(ui.cookiename4.text())]['domain']=url c1[str(ui.cookiename4.text())]['path']='/' self.log("cookies are.. :"+str(c1)) cj=cookielib.CookieJar() cj.set_cookie(c1) opener = urllib2.build_opener(proxyhandler, urllib2.HTTPCookieProcessor(cj)) self.log("insatlling opener") urllib2.install_opener(opener) self.log("setting the two operations....") if itered==Fasle: self.log("starting the flooding loop") gui() while true: try: gui() opener.open(req) except e: self.log("error connecting : "+e.reason) self.log("will continue....") continue gui() elif itered==True: pass f1=flooderthread(1, 1) f2=flooderthread(2, 2) f3=flooderthread(3, 3) f4=flooderthread(4, 4) f5=flooderthread(5, 5) f6=flooderthread(6, 6) class Ui_MainWindow(object): def setupUi(self, MainWindow): MainWindow.setObjectName(_fromUtf8("MainWindow")) MainWindow.setMinimumSize(QtCore.QSize(838, 500)) MainWindow.setMaximumSize(QtCore.QSize(838, 500)) MainWindow.setWindowTitle(QtGui.QApplication.translate("MainWindow", "memo flooder", None, QtGui.QApplication.UnicodeUTF8)) self.centralwidget = QtGui.QWidget(MainWindow) self.centralwidget.setObjectName(_fromUtf8("centralwidget")) self.console=QtGui.QTextEdit(self.centralwidget) self.console.setGeometry(10, 350, 800,130) self.console.setReadOnly(True) self.console.setObjectName("console") self.groupBox = QtGui.QGroupBox(self.centralwidget) self.groupBox.setGeometry(QtCore.QRect(30, 50, 71, 80)) self.groupBox.setTitle(QtGui.QApplication.translate("MainWindow", "method:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox.setObjectName(_fromUtf8("groupBox")) self.post = QtGui.QRadioButton(self.groupBox) self.post.setGeometry(QtCore.QRect(10, 20, 61, 22)) self.post.setText(QtGui.QApplication.translate("MainWindow", "post", None, QtGui.QApplication.UnicodeUTF8)) self.post.setChecked(True) self.post.setObjectName(_fromUtf8("post")) self.get = QtGui.QRadioButton(self.groupBox) self.get.setGeometry(QtCore.QRect(10, 50, 51, 22)) self.get.setText(QtGui.QApplication.translate("MainWindow", "get", None, QtGui.QApplication.UnicodeUTF8)) self.get.setObjectName(_fromUtf8("get")) self.url = QtGui.QLineEdit(self.centralwidget) self.url.setGeometry(QtCore.QRect(70, 20, 671, 27)) self.url.setInputMethodHints(QtCore.Qt.ImhUrlCharactersOnly) self.url.setObjectName(_fromUtf8("url")) self.groupBox_2 = QtGui.QGroupBox(self.centralwidget) self.groupBox_2.setGeometry(QtCore.QRect(110, 50, 371, 111)) self.groupBox_2.setTitle(QtGui.QApplication.translate("MainWindow", "data:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox_2.setObjectName(_fromUtf8("groupBox_2")) self.dataname1 = QtGui.QLineEdit(self.groupBox_2) self.dataname1.setGeometry(QtCore.QRect(20, 30, 101, 27)) self.dataname1.setObjectName(_fromUtf8("dataname1")) self.label = QtGui.QLabel(self.groupBox_2) self.label.setGeometry(QtCore.QRect(40, 10, 67, 17)) self.label.setText(QtGui.QApplication.translate("MainWindow", "name:", None, QtGui.QApplication.UnicodeUTF8)) self.label.setObjectName(_fromUtf8("label")) self.dataname2 = QtGui.QLineEdit(self.groupBox_2) self.dataname2.setGeometry(QtCore.QRect(130, 30, 113, 27)) self.dataname2.setObjectName(_fromUtf8("dataname2")) self.dataname3 = QtGui.QLineEdit(self.groupBox_2) self.dataname3.setGeometry(QtCore.QRect(250, 30, 113, 27)) self.dataname3.setObjectName(_fromUtf8("dataname3")) self.label_2 = QtGui.QLabel(self.groupBox_2) self.label_2.setGeometry(QtCore.QRect(40, 60, 67, 17)) self.label_2.setText(QtGui.QApplication.translate("MainWindow", "value:", None, QtGui.QApplication.UnicodeUTF8)) self.label_2.setObjectName(_fromUtf8("label_2")) self.datavalue1 = QtGui.QLineEdit(self.groupBox_2) self.datavalue1.setGeometry(QtCore.QRect(20, 80, 101, 27)) self.datavalue1.setObjectName(_fromUtf8("datavalue1")) self.datavalue2 = QtGui.QLineEdit(self.groupBox_2) self.datavalue2.setGeometry(QtCore.QRect(130, 80, 113, 27)) self.datavalue2.setObjectName(_fromUtf8("datavalue2")) self.datavalue3 = QtGui.QLineEdit(self.groupBox_2) self.datavalue3.setGeometry(QtCore.QRect(250, 80, 113, 27)) self.datavalue3.setObjectName(_fromUtf8("datavalue3")) self.groupBox_4 = QtGui.QGroupBox(self.centralwidget) self.groupBox_4.setGeometry(QtCore.QRect(670, 50, 151, 111)) self.groupBox_4.setTitle(QtGui.QApplication.translate("MainWindow", "password:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox_4.setObjectName(_fromUtf8("groupBox_4")) self.passname = QtGui.QLineEdit(self.groupBox_4) self.passname.setGeometry(QtCore.QRect(10, 30, 113, 27)) self.passname.setObjectName(_fromUtf8("passname")) self.passvalue = QtGui.QLineEdit(self.groupBox_4) self.passvalue.setGeometry(QtCore.QRect(10, 80, 113, 27)) self.passvalue.setObjectName(_fromUtf8("passvalue")) self.passwordcheck = QtGui.QCheckBox(self.centralwidget) self.passwordcheck.setGeometry(QtCore.QRect(670, 180, 97, 22)) self.passwordcheck.setText(QtGui.QApplication.translate("MainWindow", "password", None, QtGui.QApplication.UnicodeUTF8)) self.passwordcheck.setChecked(True) self.passwordcheck.setObjectName(_fromUtf8("passwordcheck")) self.groupBox_5 = QtGui.QGroupBox(self.centralwidget) self.groupBox_5.setGeometry(QtCore.QRect(29, 169, 441, 81)) self.groupBox_5.setTitle(QtGui.QApplication.translate("MainWindow", "proxy:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox_5.setObjectName(_fromUtf8("groupBox_5")) self.proxyhost = QtGui.QLineEdit(self.groupBox_5) self.proxyhost.setGeometry(QtCore.QRect(20, 30, 113, 27)) self.proxyhost.setObjectName(_fromUtf8("proxyhost")) self.proxyport = QtGui.QLineEdit(self.groupBox_5) self.proxyport.setGeometry(QtCore.QRect(140, 30, 51, 27)) self.proxyport.setInputMethodHints(QtCore.Qt.ImhDigitsOnly|QtCore.Qt.ImhPreferNumbers) self.proxyport.setObjectName(_fromUtf8("proxyport")) self.proxyuser = QtGui.QLineEdit(self.groupBox_5) self.proxyuser.setGeometry(QtCore.QRect(200, 30, 113, 27)) self.proxyuser.setObjectName(_fromUtf8("proxyuser")) self.proxypass = QtGui.QLineEdit(self.groupBox_5) self.proxypass.setGeometry(QtCore.QRect(320, 30, 113, 27)) self.proxypass.setObjectName(_fromUtf8("proxypass")) self.label_4 = QtGui.QLabel(self.groupBox_5) self.label_4.setGeometry(QtCore.QRect(100, 10, 67, 17)) self.label_4.setText(QtGui.QApplication.translate("MainWindow", "host", None, QtGui.QApplication.UnicodeUTF8)) self.label_4.setObjectName(_fromUtf8("label_4")) self.label_5 = QtGui.QLabel(self.groupBox_5) self.label_5.setGeometry(QtCore.QRect(150, 10, 67, 17)) self.label_5.setText(QtGui.QApplication.translate("MainWindow", "port", None, QtGui.QApplication.UnicodeUTF8)) self.label_5.setObjectName(_fromUtf8("label_5")) self.label_6 = QtGui.QLabel(self.groupBox_5) self.label_6.setGeometry(QtCore.QRect(200, 10, 67, 17)) self.label_6.setText(QtGui.QApplication.translate("MainWindow", "username", None, QtGui.QApplication.UnicodeUTF8)) self.label_6.setObjectName(_fromUtf8("label_6")) self.label_7 = QtGui.QLabel(self.groupBox_5) self.label_7.setGeometry(QtCore.QRect(320, 10, 67, 17)) self.label_7.setText(QtGui.QApplication.translate("MainWindow", "password", None, QtGui.QApplication.UnicodeUTF8)) self.label_7.setObjectName(_fromUtf8("label_7")) self.groupBox_6 = QtGui.QGroupBox(self.centralwidget) self.groupBox_6.setGeometry(QtCore.QRect(30, 260, 531, 91)) self.groupBox_6.setTitle(QtGui.QApplication.translate("MainWindow", "cookies:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox_6.setObjectName(_fromUtf8("groupBox_6")) self.cookiename1 = QtGui.QLineEdit(self.groupBox_6) self.cookiename1.setGeometry(QtCore.QRect(10, 20, 113, 27)) self.cookiename1.setObjectName(_fromUtf8("cookiename1")) self.cookiename2 = QtGui.QLineEdit(self.groupBox_6) self.cookiename2.setGeometry(QtCore.QRect(140, 20, 113, 27)) self.cookiename2.setObjectName(_fromUtf8("cookename2")) self.cookiename3 = QtGui.QLineEdit(self.groupBox_6) self.cookiename3.setGeometry(QtCore.QRect(270, 20, 113, 27)) self.cookiename3.setObjectName(_fromUtf8("cookiename3")) self.cookiename4 = QtGui.QLineEdit(self.groupBox_6) self.cookiename4.setGeometry(QtCore.QRect(390, 20, 113, 27)) self.cookiename4.setObjectName(_fromUtf8("cookiename4")) self.cookievalue1 = QtGui.QLineEdit(self.groupBox_6) self.cookievalue1.setGeometry(QtCore.QRect(10, 50, 113, 27)) self.cookievalue1.setObjectName(_fromUtf8("cookievalue1")) self.cookievalue2 = QtGui.QLineEdit(self.groupBox_6) self.cookievalue2.setGeometry(QtCore.QRect(140, 50, 113, 27)) self.cookievalue2.setObjectName(_fromUtf8("cookievalue2")) self.cookievalue3 = QtGui.QLineEdit(self.groupBox_6) self.cookievalue3.setGeometry(QtCore.QRect(270, 50, 113, 27)) self.cookievalue3.setObjectName(_fromUtf8("cookievalue3")) self.cookievalue4 = QtGui.QLineEdit(self.groupBox_6) self.cookievalue4.setGeometry(QtCore.QRect(390, 50, 113, 27)) self.cookievalue4.setObjectName(_fromUtf8("cookievalue4")) self.groupBox_7 = QtGui.QGroupBox(self.centralwidget) self.groupBox_7.setGeometry(QtCore.QRect(570, 260, 251, 80)) self.groupBox_7.setTitle(QtGui.QApplication.translate("MainWindow", "useragents:", None, QtGui.QApplication.UnicodeUTF8)) self.groupBox_7.setObjectName(_fromUtf8("groupBox_7")) self.useragent = QtGui.QTextEdit(self.groupBox_7) self.useragent.setGeometry(QtCore.QRect(10, 20, 211, 51)) self.useragent.setVerticalScrollBarPolicy(QtCore.Qt.ScrollBarAlwaysOn) self.useragent.setObjectName(_fromUtf8("useragent")) self.start = QtGui.QPushButton(self.centralwidget) self.start.setGeometry(QtCore.QRect(750, 20, 71, 27)) self.start.setText(QtGui.QApplication.translate("MainWindow", "start", None, QtGui.QApplication.UnicodeUTF8)) self.start.setObjectName(_fromUtf8("start")) self.label_3 = QtGui.QLabel(self.centralwidget) self.label_3.setGeometry(QtCore.QRect(30, 20, 67, 17)) self.label_3.setText(QtGui.QApplication.translate("MainWindow", "url :", None, QtGui.QApplication.UnicodeUTF8)) self.label_3.setObjectName(_fromUtf8("label_3")) MainWindow.setCentralWidget(self.centralwidget) QtCore.QObject.connect(self.start, QtCore.SIGNAL(_fromUtf8("clicked(bool)")), myfun.main) QtCore.QObject.connect(self.passwordcheck, QtCore.SIGNAL(_fromUtf8("clicked(bool)")), self.groupBox_4.setEnabled) QtCore.QMetaObject.connectSlotsByName(MainWindow) def __del__(): myfun.killer() os.abort() sys.exit() app = QtGui.QApplication(sys.argv) MainWindow = QtGui.QMainWindow() ui = Ui_MainWindow() ui.setupUi(MainWindow) myfun.log("\n\n--------------------------------------------------new session-------------------------------------\n\n") MainWindow.show() sys.exit(app.exec_())

  Read the article

• Reading Data from DDFS ValueError: No JSON object could be decoded

  - by secumind

  I'm running dozens of map reduce jobs for a number of different purposes using disco. My data has grown enormous and I thought I would try using DDFS for a change rather than standard txt files. I've followed the DISCO map/reduce example Counting Words as a map/reduce job, without to much difficulty and with the help of others, Reading JSON specific data into DISCO I've gotten past one of my latest problems. I'm trying to read data in/out of ddfs to better chunk and distribute it but am having a bit of trouble. Here's an example file: file.txt {"favorited": false, "in_reply_to_user_id": null, "contributors": null, "truncated": false, "text": "I'll call him back tomorrow I guess", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016843603968", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/305726905/FASHION-3.png", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1818996723/image_normal.jpg", "profile_sidebar_fill_color": "292727", "is_translator": false, "id": 113532729, "profile_text_color": "000000", "followers_count": 78, "protected": false, "location": "With My Niggas In Paris!", "default_profile_image": false, "listed_count": 0, "utc_offset": -21600, "statuses_count": 6733, "description": "Made in CHINA., Educated && Making My Own $$. Fear GOD && Put Him 1st. #TeamFollowBack #TeamiPhone\n", "friends_count": 74, "profile_link_color": "b03f3f", "profile_image_url": "http://a2.twimg.com/profile_images/1818996723/image_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "1f9199", "id_str": "113532729", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/305726905/FASHION-3.png", "name": "Bee'Jay", "lang": "en", "profile_background_tile": true, "favourites_count": 19, "screen_name": "OohMyBEEsNice", "url": "http://www.bitchimpaid.org", "created_at": "Fri Feb 12 03:32:54 +0000 2010", "contributors_enabled": false, "time_zone": "Central Time (US & Canada)", "profile_sidebar_border_color": "000000", "default_profile": false, "following": null}, "in_reply_to_screen_name": null, "retweet_count": 0, "geo": null, "id": 168931016843603968, "source": "<a href=\"http://twitter.com/#!/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"} {"favorited": false, "in_reply_to_user_id": 50940453, "contributors": null, "truncated": false, "text": "@LegaMrvica @MimozaBand makasi om artis :D kadoo kadoo", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": "168653037894770688", "coordinates": null, "in_reply_to_user_id_str": "50940453", "entities": {"user_mentions": [{"indices": [0, 11], "screen_name": "LegaMrvica", "id": 50940453, "name": "Lega_thePianis", "id_str": "50940453"}, {"indices": [12, 23], "screen_name": "MimozaBand", "id": 375128905, "name": "Mimoza", "id_str": "375128905"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": 168653037894770688, "id_str": "168931016868761600", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "profile_sidebar_fill_color": "DDFFCC", "is_translator": false, "id": 48293450, "profile_text_color": "333333", "followers_count": 182, "protected": false, "location": "\u00dcT: -6.906799,107.622383", "default_profile_image": false, "listed_count": 0, "utc_offset": -28800, "statuses_count": 3052, "description": "Fashion design maranatha '11 // traditional dancer (bali) at sanggar tampak siring & Natya Nataraja", "friends_count": 206, "profile_link_color": "0084B4", "profile_image_url": "http://a3.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "9AE4E8", "id_str": "48293450", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "name": "nana afiff", "lang": "en", "profile_background_tile": true, "favourites_count": 2, "screen_name": "hasnfebria", "url": null, "created_at": "Thu Jun 18 08:50:29 +0000 2009", "contributors_enabled": false, "time_zone": "Pacific Time (US & Canada)", "profile_sidebar_border_color": "BDDCAD", "default_profile": false, "following": null}, "in_reply_to_screen_name": "LegaMrvica", "retweet_count": 0, "geo": null, "id": 168931016868761600, "source": "<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry\u00ae</a>"} {"favorited": false, "in_reply_to_user_id": 27260086, "contributors": null, "truncated": false, "text": "@justinbieber u were born to be somebody, and u're super important in beliebers' life. thanks for all biebs. I love u. follow me? 84", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": "27260086", "entities": {"user_mentions": [{"indices": [0, 13], "screen_name": "justinbieber", "id": 27260086, "name": "Justin Bieber", "id_str": "27260086"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016856178688", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/416005864/Captura.JPG", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "profile_sidebar_fill_color": "f5e7f3", "is_translator": false, "id": 406750700, "profile_text_color": "333333", "followers_count": 1122, "protected": false, "location": "Adentro de una supra.", "default_profile_image": false, "listed_count": 0, "utc_offset": -14400, "statuses_count": 20966, "description": "Mi \u00eddolo es @justinbieber , si te gusta \u00a1genial!, si no, solo respetalo. El cambi\u00f3 mi vida completamente y mi sue\u00f1o es conocerlo #TrueBelieber . ", "friends_count": 1015, "profile_link_color": "9404b8", "profile_image_url": "http://a1.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "notifications": null, "show_all_inline_media": false, "geo_enabled": false, "profile_background_color": "f9fcfa", "id_str": "406750700", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/416005864/Captura.JPG", "name": "neversaynever,right?", "lang": "es", "profile_background_tile": false, "favourites_count": 22, "screen_name": "True_Belieebers", "url": "http://www.wehavebieber-fever.tumblr.com", "created_at": "Mon Nov 07 04:17:40 +0000 2011", "contributors_enabled": false, "time_zone": "Santiago", "profile_sidebar_border_color": "C0DEED", "default_profile": false, "following": null}, "in_reply_to_screen_name": "justinbieber", "retweet_count": 0, "geo": null, "id": 168931016856178688, "source": "<a href=\"http://yfrog.com\" rel=\"nofollow\">Yfrog</a>"} I load it into DDFS with: # ddfs chunk data:test1 ./file.txt created: disco://localhost/ddfs/vol0/blob/44/file_txt-0$549-db27b-125e1 I test that the file is indeed loaded into ddfs with: # ddfs xcat data:test1 {"favorited": false, "in_reply_to_user_id": null, "contributors": null, "truncated": false, "text": "I'll call him back tomorrow I guess", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016843603968", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/305726905/FASHION-3.png", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1818996723/image_normal.jpg", "profile_sidebar_fill_color": "292727", "is_translator": false, "id": 113532729, "profile_text_color": "000000", "followers_count": 78, "protected": false, "location": "With My Niggas In Paris!", "default_profile_image": false, "listed_count": 0, "utc_offset": -21600, "statuses_count": 6733, "description": "Made in CHINA., Educated && Making My Own $$. Fear GOD && Put Him 1st. #TeamFollowBack #TeamiPhone\n", "friends_count": 74, "profile_link_color": "b03f3f", "profile_image_url": "http://a2.twimg.com/profile_images/1818996723/image_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "1f9199", "id_str": "113532729", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/305726905/FASHION-3.png", "name": "Bee'Jay", "lang": "en", "profile_background_tile": true, "favourites_count": 19, "screen_name": "OohMyBEEsNice", "url": "http://www.bitchimpaid.org", "created_at": "Fri Feb 12 03:32:54 +0000 2010", "contributors_enabled": false, "time_zone": "Central Time (US & Canada)", "profile_sidebar_border_color": "000000", "default_profile": false, "following": null}, "in_reply_to_screen_name": null, "retweet_count": 0, "geo": null, "id": 168931016843603968, "source": "<a href=\"http://twitter.com/#!/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"} {"favorited": false, "in_reply_to_user_id": 50940453, "contributors": null, "truncated": false, "text": "@LegaMrvica @MimozaBand makasi om artis :D kadoo kadoo", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": "168653037894770688", "coordinates": null, "in_reply_to_user_id_str": "50940453", "entities": {"user_mentions": [{"indices": [0, 11], "screen_name": "LegaMrvica", "id": 50940453, "name": "Lega_thePianis", "id_str": "50940453"}, {"indices": [12, 23], "screen_name": "MimozaBand", "id": 375128905, "name": "Mimoza", "id_str": "375128905"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": 168653037894770688, "id_str": "168931016868761600", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "profile_sidebar_fill_color": "DDFFCC", "is_translator": false, "id": 48293450, "profile_text_color": "333333", "followers_count": 182, "protected": false, "location": "\u00dcT: -6.906799,107.622383", "default_profile_image": false, "listed_count": 0, "utc_offset": -28800, "statuses_count": 3052, "description": "Fashion design maranatha '11 // traditional dancer (bali) at sanggar tampak siring & Natya Nataraja", "friends_count": 206, "profile_link_color": "0084B4", "profile_image_url": "http://a3.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "9AE4E8", "id_str": "48293450", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "name": "nana afiff", "lang": "en", "profile_background_tile": true, "favourites_count": 2, "screen_name": "hasnfebria", "url": null, "created_at": "Thu Jun 18 08:50:29 +0000 2009", "contributors_enabled": false, "time_zone": "Pacific Time (US & Canada)", "profile_sidebar_border_color": "BDDCAD", "default_profile": false, "following": null}, "in_reply_to_screen_name": "LegaMrvica", "retweet_count": 0, "geo": null, "id": 168931016868761600, "source": "<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry\u00ae</a>"} {"favorited": false, "in_reply_to_user_id": 27260086, "contributors": null, "truncated": false, "text": "@justinbieber u were born to be somebody, and u're super important in beliebers' life. thanks for all biebs. I love u. follow me? 84", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": "27260086", "entities": {"user_mentions": [{"indices": [0, 13], "screen_name": "justinbieber", "id": 27260086, "name": "Justin Bieber", "id_str": "27260086"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016856178688", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/416005864/Captura.JPG", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "profile_sidebar_fill_color": "f5e7f3", "is_translator": false, "id": 406750700, "profile_text_color": "333333", "followers_count": 1122, "protected": false, "location": "Adentro de una supra.", "default_profile_image": false, "listed_count": 0, "utc_offset": -14400, "statuses_count": 20966, "description": "Mi \u00eddolo es @justinbieber , si te gusta \u00a1genial!, si no, solo respetalo. El cambi\u00f3 mi vida completamente y mi sue\u00f1o es conocerlo #TrueBelieber . ", "friends_count": 1015, "profile_link_color": "9404b8", "profile_image_url": "http://a1.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "notifications": null, "show_all_inline_media": false, "geo_enabled": false, "profile_background_color": "f9fcfa", "id_str": "406750700", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/416005864/Captura.JPG", "name": "neversaynever,right?", "lang": "es", "profile_background_tile": false, "favourites_count": 22, "screen_name": "True_Belieebers", "url": "http://www.wehavebieber-fever.tumblr.com", "created_at": "Mon Nov 07 04:17:40 +0000 2011", "contributors_enabled": false, "time_zone": "Santiago", "profile_sidebar_border_color": "C0DEED", "default_profile": false, "following": null}, "in_reply_to_screen_name": "justinbieber", "retweet_count": 0, "geo": null, "id": 168931016856178688, "source": "<a href=\"http://yfrog.com\" rel=\"nofollow\">Yfrog</a> At this point everything is great, I load up the script that resulted from a previous Stack Post: from disco.core import Job, result_iterator import gzip def map(line, params): import unicodedata import json r = json.loads(line).get('text') s = unicodedata.normalize('NFD', r).encode('ascii', 'ignore') for word in s.split(): yield word, 1 def reduce(iter, params): from disco.util import kvgroup for word, counts in kvgroup(sorted(iter)): yield word, sum(counts) if __name__ == '__main__': job = Job().run(input=["tag://data:test1"], map=map, reduce=reduce) for word, count in result_iterator(job.wait(show=True)): print word, count NOTE: That this script runs file if the input=["file.txt"], however when I run it with "tag://data:test1" I get the following error: # DISCO_EVENTS=1 python count_normal_words.py Job@549:db30e:25bd8: Status: [map] 0 waiting, 1 running, 0 done, 0 failed 2012/11/25 21:43:26 master New job initialized! 2012/11/25 21:43:26 master Starting job 2012/11/25 21:43:26 master Starting map phase 2012/11/25 21:43:26 master map:0 assigned to solice 2012/11/25 21:43:26 master ERROR: Job failed: Worker at 'solice' died: Traceback (most recent call last): File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/__init__.py", line 329, in main job.worker.start(task, job, **jobargs) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/__init__.py", line 290, in start self.run(task, job, **jobargs) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/classic/worker.py", line 286, in run getattr(self, task.mode)(task, params) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/classic/worker.py", line 299, in map for key, val in self['map'](entry, params): File "count_normal_words.py", line 12, in map File "/usr/lib64/python2.7/json/__init__.py", line 326, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded 2012/11/25 21:43:26 master WARN: Job killed Status: [map] 1 waiting, 0 running, 0 done, 1 failed Traceback (most recent call last): File "count_normal_words.py", line 28, in <module> for word, count in result_iterator(job.wait(show=True)): File "/usr/local/lib/python2.7/site-packages/disco/core.py", line 348, in wait timeout, poll_interval * 1000) File "/usr/local/lib/python2.7/site-packages/disco/core.py", line 309, in check_results raise JobError(Job(name=jobname, master=self), "Status %s" % status) disco.error.JobError: Job Job@549:db30e:25bd8 failed: Status dead The Error states: ValueError: No JSON object could be decoded. Again, this works fine using the text file as input but now DDFS. Any ideas, I'm open to suggestions?

  Read the article

< Previous Page | 490 491 492 493 494 495 496 497 498 499 500 501  | Next Page >