Search Results

Search found 8162 results on 327 pages for 'module packaging'.

Page 106/327 | < Previous Page | 102 103 104 105 106 107 108 109 110 111 112 113  | Next Page >

  • python protobufs - avoid the install step ?

    - by orion elenzil
    i'm writing a small python utility which will be consumed by moderately non-technical users and which needs to interface w/ some protobufs. ideally, i would like the only prerequisites to using this on a local machine to be: have python installed * have an SVN checkout of the repository * run a simple bash script to build the local proto .py definitions * run "python myutility" i'm running into trouble around importing descriptor_pb2.py, tho. i've seen Why do I see "cannot import name descriptor_pb2" error when using Google Protocol Buffers? , but would like to avoid adding the additional prerequisite of having run the proto SDK installer. i've modified the bash script to also generate descriptor_pb2.py in the local heirarchy, which works for the first level of imports from my other _pb2.py files, but it looks like descriptor_pb2.py itself tries to import descriptor_pb2 can't find it: $ python myutility.py Traceback (most recent call last): File "myutility.py", line 4, in <module> import protos.myProto_pb2 File "/myPath/protos/myProto_pb2.py", line 8, in <module> from google.protobuf import descriptor_pb2 File "/myPath/google/protobuf/descriptor_pb2.py", line 8, in <module> from google.protobuf import descriptor_pb2 ImportError: cannot import name descriptor_pb2 my local folder looks like: * myutility.py * google/ * protobuf/ * descriptor.py * descriptor_pb2.py * protos * myProto_ob2.py also, i'm a python n00b, so it's possible i'm overlooking something obvious. tia, orion

    Read the article

  • Titanium Appcelerator - After enabling cloud services I get UnicodeDecodeError when compiling for Android

    - by Shahar Zrihen
    I've got an app that has a UTF8 name (hebrew). I use the platform/android/AndroidManifest.xml file for this. I've managed to narrow it down to the Ti.cloudpush module. only when I enable this module I get the error. I used to be able to compile it to android without any issues but as soon as I enable cloud services I get this error - [ERROR] Exception occured while building Android project: [ERROR] Traceback (most recent call last): [ERROR] File "/Users/Shahar/Library/Application Support/Titanium/mobilesdk/osx/2.1.0.GA/android/builder.py", line 2218, in <module> [ERROR] s.build_and_run(True, None, key, password, alias, output_dir) [ERROR] File "/Users/Shahar/Library/Application Support/Titanium/mobilesdk/osx/2.1.0.GA/android/builder.py", line 1970, in build_and_run [ERROR] self.manifest_changed = self.generate_android_manifest(compiler) [ERROR] File "/Users/Shahar/Library/Application Support/Titanium/mobilesdk/osx/2.1.0.GA/android/builder.py", line 1195, in generate_android_manifest [ERROR] custom_manifest_contents = fill_manifest(custom_manifest_contents) [ERROR] File "/Users/Shahar/Library/Application Support/Titanium/mobilesdk/osx/2.1.0.GA/android/builder.py", line 1122, in fill_manifest [ERROR] manifest_source = manifest_source.replace(ti_permissions,permissions_required_xml) [ERROR] UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 501: ordinal not in range(128) and this is my manifest file with the part that causes the issues. If I remove the hebrew name, it compiles without any issues <application android:icon="@drawable/appicon" android:label="??????" android:name="QuestionnaireApplication" android:debuggable="false" > <activity android:name=".QuestionnaireActivity" android:label="??????" android:theme="@style/Theme.Titanium" android:screenOrientation="portrait" android:configChanges="keyboardHidden" > Any suggestions?

    Read the article

  • Node.js running under IIS Express Keeps Crashing

    - by PazoozaTest Pazman
    I recently resinstalled Windows 7 on my machine and went back to downloading and installing the tools to help me continue developing node.js windows azure web applications. I followed the instructions given on the node.js azure site: http://www.windowsazure.com/en-us/develop/nodejs/ and using web installer 4.0 it says I have successfully installed these tools: Windows Azure Powershell Windows Azure SDK for Node.js - June 2012 Windows Azure SDK for .Net (VS 2012 RC) - June 2012 IIS Recommend Configuration The problem I am experiencing is that when I run the site using powershell e.g: start-azureemulator -launch it goes ahead and runs IIS Express, and after several minutes IIS Express crashes with the following information: Problem signature: Problem Event Name: APPCRASH Application Name: iisexpress.exe Application Version: 8.0.8298.0 Application Timestamp: 4f620349 Fault Module Name: iiscore.dll Fault Module Version: 8.0.8298.0 Fault Module Timestamp: 4f63b65c Exception Code: c0000005 Exception Offset: 00021767 OS Version: 6.1.7601.2.1.0.256.28 Locale ID: 1033 Additional Information 1: f66d Additional Information 2: f66d807b515d6b2dc6f28f66db769a01 Additional Information 3: 7b2f Additional Information 4: 7b2f6797d07ebc2c23f2b227e779722e I am running 2 instances each time, and both of them crash one after the other. Is anyone experiencing something similar and fix this issue ? Is their an upgrade I need to do ? I've run windows update but it says I've got all the latest updates etc. Can I tell the powershell cmdlet to use IIS 7 instead of IIS Express? I'm guessing its something to do with IIS Express on my machine. I did some hunting around and found this person here who experienced a similar problem: https://github.com/tjanczuk/iisnode/issues/149 I've got a cron job running every 1 second, to check if any website totals need to be updated. Could this be causing IIS Express to crash? Cheers

    Read the article

  • mysql-python stopped working

    - by MAC
    This is a rather dumb question but i'am looking at a bizarre situation. I am running fedora and have python 2.6.5 installed. The other day i installed MySQL-python using yum (because i do not have the setuptools module so i cannot build it from source). Anyway yesterday i wrote my entire data access layer in python and it was running fine, i did test it. Today however it gives me an ImportError: No module named MySQLdb The only thing i ever changed was i installed eclipse and pyDev. Any ideas on what went wrong and how i fix it. I tried removing and re-installing MySql-python but that did not help. I did the following import sys print sys.path And it shows me all the paths which are basically pertaining to /usr/local/lib/python2.6 However i was trying to find where the MySQLdb module is installed and it seems that its installed in /usr/lib/python2.5/sitepackages Now i have no idea why it got installed there and why it was working earlier and why it stopped working now. Any ideas on how i should fix it. I did try copying the site-packages folder over to the python2.6 folder but that did not work Help!!

    Read the article

  • Why isn't module_filters filtering Mail::IspMailGate in CPAN::Mini?

    - by user304122
    Edited - Ummm - now have a module in schwigon giving same problem !! I am on a corporate PC that forces mcshield on everything that moves. I get blocked when trying to mirror on ... authors/id/J/JV/JV/EekBoek-2.00.01.tar.gz ... updated authors/id/J/JV/JV/CHECKSUMS ... updated Could not stat tmpfile '/cygdrive/t/cpan_mirror/authors/id/J/JW/JWIED/Mail-IspMailGate-1.1013.tar.gz-4712': No such file or directory at /usr/lib/perl5/site_perl/5.10/LWP/UserAgent.pm line 851. authors/id/J/JW/JWIED/Mail-IspMailGate-1.1013.tar.gz At this point, I get the Virus scanner mcshield sticking its Awr in. To maintain my mirror I execute:- #!/usr/bin/perl CPAN::Mini->update_mirror( remote => "http://mirror.eunet.fi/CPAN", local => "/cygdrive/t/cpan_mirror/", trace => 1, errors => 1, module_filters => [ qr/kjkjhkjhkjkj/i, qr/clamav/i, qr/ispmailgate/i, qr/IspMailGate/, qr/Mail-IspMailGate/, qr/mail-ispmailgate/i, ], path_filters => [ qr/ZZYYZZ/, #qr/WIED/, #qr/RJBS/, ] ); It skips OK if I enable the path_filter WIED. Just cannot get it to skip the module failing module to complete other WIED modules. Any ideas?

    Read the article

  • cx_Oracle makes subprocess give OSError

    - by Shrikant Sharat
    I am trying to use the cx_Oracle module with python 2.6.6 on ubuntu Maverick, with Oracle 11gR2 Enterprise edition. I am able to connect to my oracle db just fine, but once I do that, the subprocess module does not work anymore. Here is an iPython session that reproduces the problem... In [1]: import subprocess as sp, cx_Oracle as dbh In [2]: sp.call(['whoami']) sharat Out[2]: 0 In [3]: con = dbh.connect('system', 'password') In [4]: con.close() In [5]: sp.call(['whomai']) --------------------------------------------------------------------------- OSError Traceback (most recent call last) /home/sharat/desk/calypso-launcher/<ipython console> in <module>() /usr/lib/python2.6/subprocess.pyc in call(*popenargs, **kwargs) 468 retcode = call(["ls", "-l"]) 469 """ --> 470 return Popen(*popenargs, **kwargs).wait() 471 472 /usr/lib/python2.6/subprocess.pyc in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags) 621 p2cread, p2cwrite, 622 c2pread, c2pwrite, --> 623 errread, errwrite) 624 625 if mswindows: /usr/lib/python2.6/subprocess.pyc in _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite) 1134 1135 if data != "": -> 1136 _eintr_retry_call(os.waitpid, self.pid, 0) 1137 child_exception = pickle.loads(data) 1138 for fd in (p2cwrite, c2pread, errread): /usr/lib/python2.6/subprocess.pyc in _eintr_retry_call(func, *args) 453 while True: 454 try: --> 455 return func(*args) 456 except OSError, e: 457 if e.errno == errno.EINTR: OSError: [Errno 10] No child processes So, the call to sp.call works fine before connecting to oracle, but breaks after that. Even if I have closed the connection to the database. Looking around, I found http://bugs.python.org/issue1731717 as somewhat related to this issue, but I am not dealing with threads here. I don't know if cx_Oracle is. Moreover, the above issue mentions that adding a time.sleep(1) fixes it, but it didn't help me. Any help appreciated. Thanks.

    Read the article

  • Getting svn: E170000: Unrecognized URL scheme for my custom Svn Gradle plugin

    - by Ip Doh
    I wrote a custom gradle plugin using groovy to do basic svn tasks like, Checkout, Clean, Tag etc. The groovy class calls the svn command line client to do these operations, It works fine when i run it on my windows system but the same plugin gives the following error when i run it on a linux system (Centos). svn: E170000: Unrecognized URL scheme for '%22https://source.mycompany.net/svn/MyProject/trunk%22' Am able to make the same calls to the command line client through the command prompt or shell script without any issues. So what is the difference with Here is my code sample String command =String.format("svn co -r %d --non-interactive --trust-server-cert -- username %s --password %s --depth infinity \"%s\" \"%s\"", getRevision(), getUserName(), getUserPassword(), getSrcUrl(), getDir()); Process svnProcess = Runtime.getRuntime().exec(command); BufferedReader stdInput = new BufferedReader(new InputStreamReader(svnProcess.getInputStream())); BufferedReader stdError = new BufferedReader(new InputStreamReader(svnProcess.getErrorStream())); String statusOutputLine ="" while ((statusOutputLine = stdInput.readLine()) != null) { logger.quiet(" " + statusOutputLine); } while (( statusOutputLine = stdError.readLine()) != null) { logger.error(statusOutputLine) throw new Exception(statusOutputLine) } logger.quiet("Successfully Checked out the work space") i do have neon installed on the system -bash-4.1$ svn --version svn, version 1.6.11 (r934486) compiled Jun 25 2011, 11:30:15 Copyright (C) 2000-2009 CollabNet. Subversion is open source software, see http://subversion.tigris.org/ This product includes software developed by CollabNet (http://www.Collab.Net/). The following repository access (RA) modules are available: ra_neon : Module for accessing a repository via WebDAV protocol using Neon. handles 'http' scheme handles 'https' scheme ra_svn : Module for accessing a repository using the svn network protocol. with Cyrus SASL authentication handles 'svn' scheme ra_local : Module for accessing a repository on local disk. handles 'file' scheme

    Read the article

  • Maven webapp with Eclipse and WTP plugin deploy files in stranges ways in Tomcat

    - by hokkos
    Hi, I use Eclipse J2EE 3.5 with Maven and tomcat. To deploy my maven webapp with WTP I added a Dynamic Web Module facet and changed the "org.eclipse.wst.common.component" file of the project because the webapp is not in a WebContent directory, here is the content of the file: <?xml version="1.0" encoding="UTF-8"?> <project-modules id="moduleCoreId" project-version="1.5.0"> <wb-module deploy-name="tkey-ca-web"> <wb-resource deploy-path="/" source-path="/src/main/webapp"/> <wb-resource deploy-path="/WEB-INF/classes" source-path="/src/main/java"/> <wb-resource deploy-path="/WEB-INF/classes" source-path="/src/main/resources"/> <property name="context-root" value="toto"/> <property name="java-output-path" value="/toto/target/classes"/> </wb-module> </project-modules> But it never deploy the content correctly, in "workspace.metadata.plugins\org.eclipse.wst.server.core\tmp1\wtpwebapps\toto\" the directory structure is correct with WEB-INF and META-INF but empty, the jsp, html, css files are in "workspace.metadata.plugins\org.eclipse.wst.server.core\tmp1\wtpwebapps\toto\WEB-INF\classes\" with another WEB-INF and META-INF structure but with the files. I don't understand this at all, thanks.

    Read the article

  • Why isn't module_filters filetering Mail::IspMailGate in CPAN::Mini?

    - by user304122
    Edited - Ummm - now have a module in schwigon giving same problem !! I am on a corporate PC that forces mcshield on everything that moves. I get blocked when trying to mirror on ... authors/id/J/JV/JV/EekBoek-2.00.01.tar.gz ... updated authors/id/J/JV/JV/CHECKSUMS ... updated Could not stat tmpfile '/cygdrive/t/cpan_mirror/authors/id/J/JW/JWIED/Mail-IspMailGate-1.1013.tar.gz-4712': No such file or directory at /usr/lib/perl5/site_perl/5.10/LWP/UserAgent.pm line 851. authors/id/J/JW/JWIED/Mail-IspMailGate-1.1013.tar.gz At this point, I get the Virus scanner mcshield sticking its Awr in. To maintain my mirror I execute:- #!/usr/bin/perl CPAN::Mini->update_mirror( remote => "http://mirror.eunet.fi/CPAN", local => "/cygdrive/t/cpan_mirror/", trace => 1, errors => 1, module_filters => [ qr/kjkjhkjhkjkj/i, qr/clamav/i, qr/ispmailgate/i, qr/IspMailGate/, qr/Mail-IspMailGate/, qr/mail-ispmailgate/i, ], path_filters => [ qr/ZZYYZZ/, #qr/WIED/, #qr/RJBS/, ] ); It skips OK if I enable the path_filter WIED. Just cannot get it to skip the module failing module to complete other WIED modules. Any ideas?

    Read the article

  • virtualenv macosX --no-site-package ignored

    - by Tristram Gräbener
    Hello, I'm having problems with macOSX and virtualenv. It seems to ignore --no-site-package. Using exactly the same commands with linux (archlinux) it works. It it macOSX 10.5 with python 2.5 curl -o virtualenv.py 'http://bitbucket.org/ianb/virtualenv/raw/tip/virtualenv.py Create a new environment python virtualenv.py --no-site-packages foo New python executable in foo/bin/python Installing setuptools...........................done. Activate it source foo/bin/activate Try to install something in it. Despite virtualenv it looks for the system-wide install easy_install cherrypy Searching for cherrypy Best match: CherryPy 3.1.2 Adding CherryPy 3.1.2 to easy-install.pth file Using /Library/Python/2.5/site-packages Processing dependencies for cherrypy Finished processing dependencies for cherrypy Yet it doesn't find the module (foo)guidage-multimodal:~ tristram$ python Python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12) [GCC 4.0.1 (Apple Inc. build 5465)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import cherrypy Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named cherrypy I tried PIP after looking at http://stackoverflow.com/questions/1382925/virtualenv-no-site-packages-and-pip-still-finding-global-packages However it fails installing psycopg2 (some problems with gcc). Also I would like to be able to have a setup.py (from distribute) that does the whole woork

    Read the article

  • Import package problem in GWT

    - by Krt_Malta
    Hi! I'm developing an app using GWT Eclipse plug-in. (I'm also using GWT Designer but I don't think the problem is here). Previously when I wanted to communicate with a web service I created, I produced the "skeleton" classes from the WSDL url using Sun's wsimport tool. Then I would add the classes generated to a class folder in my Eclipse project. All worked well. However this doesn't seem to be working with GWT. I have these: VideoTutorialServiceService service = new VideoTutorialServiceService(); VideoTutorialService port = service.getVideoTutorialServicePort(); and I have VideoTutorialServiceService and VideoTutorialService underlined in red, the error saying videotutorialservice.VideoTutorialServiceService can not be found in source packages. Check the inheritance chain from your module; it may not be inheriting a required module or a module may not be adding its source path entries properly. .... I googled about it but I got confused. I'm a beginnier in GWT. How can I resolve this please? Thanks and regards, Krt_Malta

    Read the article

  • Debugging Django project problem.

    - by Wasim
    Hi all, I asked this question before, but had no replies, maybe I wasn't so clear. I'm trying to debug a django project using MySQL database. If I run the admin or trying to use the shell to communicate to the data base every thing is well and I can do every thing. I installed MySQLdb for Python 2.6. I installed PyDev on my Apatana studio. Configured the Debugging with runserver 8001 --noreload. When I start debugging , When I arrive to the following code in C:\Python26\Lib\site-packages\django\db\backends\mysql\base.py try: import MySQLdb as Database except ImportError, e: from django.core.exceptions import ImproperlyConfigured raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e) I get an import error : django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: DLL load failed: The specified module could not be found. I trying to ge more deeply with the import MySQLdb as Database line , it goes to the C:\Python26\Lib\site-packages\MySQLdb__init__.py and fail in the line import _mysql. I can't understand the problem. When running the Django admin every thing is ok, but with debugging it fails to work. Any help please. Thanks in advance.

    Read the article

  • Silverlight and WCF Ria Services

    - by Flex_Addicted
    Hi guys, I've created a new Silverlight 3 Business Application with VS 2008. The creation has completed correctly. When I try to open the xaml, it opens but in meanwhile this error is shown: Failed to load metadata assembly System.Windows.Controls.Data.Input.Design, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35. Exception message: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.. Stack Trace: at System.Reflection.Module._GetTypesInternal(StackCrawlMark& stackMark) at System.Reflection.Assembly.GetTypes() at MS.Internal.Package.MetadataLoader.RegisterDesignTimeMetadata(Assembly assembly, LogCallback logger)An exception of type ArgumentNullException was caught when calling IRegisterMetadata on type System.Windows.Controls.Data.Input.VisualStudio.Design.MetadataRegistration. Exception Message: Value cannot be null. Parameter name: type. Stack Trace: at Microsoft.Windows.Design.Metadata.AttributeTableBuilder.AddCallback(Type type, AttributeCallback callback) at System.Windows.Controls.Data.Input.VisualStudio.Design.MetadataRegistration.AddAttributes(AttributeTableBuilder builder) at System.Windows.Controls.Design.Common.MetadataRegistrationBase.BuildAttributeTable() at System.Windows.Controls.Data.Input.VisualStudio.Design.MetadataRegistration.Register() at MS.Internal.Package.MetadataLoader.RegisterDesignTimeMetadata(Assembly assembly, LogCallback logger)Failed to load metadata assembly System.Windows.Controls.Design, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35. Exception message: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.. Stack Trace: at System.Reflection.Module._GetTypesInternal(StackCrawlMark& stackMark) at System.Reflection.Assembly.GetTypes() at MS.Internal.Package.MetadataLoader.RegisterDesignTimeMetadata(Assembly assembly, LogCallback logger)Failed to load metadata assembly System.Windows.Controls.Navigation.Design, Version=2.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35. Exception message: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.. Stack Trace: at System.Reflection.Module._GetTypesInternal(StackCrawlMark& stackMark) at System.Reflection.Assembly.GetTypes() at MS.Internal.Package.MetadataLoader.RegisterDesignTimeMetadata(Assembly assembly, LogCallback logger) Why? Any solutions? Thank you in advance.

    Read the article

  • Asp.net MVC error with custom HttpModule

    - by Robert Koritnik
    I have a custom authentication HttpModule that is pretty strait forward. But I want it to run only for managed requests (and not for static ones). Asp.net MVC automatically adds configuration section for IIS7 web server: <system.webServer> <validation validateIntegratedModeConfiguration="false" /> <modules runAllManagedModulesForAllRequests="true"> <remove name="ScriptModule" /> <remove name="UrlRoutingModule" /> <add name="ScriptModule" preCondition="managedHandler" type="System.Web.Handlers.ScriptModule,..." /> <add name="UrlRoutingModule" type="System.Web.Routing.UrlRoutingModule,..." /> </modules> <handlers> ... </handlers> </system.webServer> When I add my own module I also set its preCondition="managedHandler", but since there's runAllManagedModulesForAllRequests="true" on parent <module> element my preCondition is ignored by design (as I read on MSDN). When I try to set though: <modules runAllManagedModulesForAllRequests="false"> I get this error: [image no longer valid] What else (which other module) do I have to set in web.config to make it work with this setting: <modules runAllManagedModulesForAllRequests="false">

    Read the article

  • Override ActiveRecord#save, Method Alias? Trying to mixin functionality into save method...

    - by viatropos
    Here's the situation: I have a User model, and two modules for authentication: Oauth and Openid. Both of them override ActiveRecord#save, and have a fair share of implementation logic. Given that I can tell when the user is trying to login via Oauth vs. Openid, but that both of them have overridden save, how do "finally" override save such that I can conditionally call one of the modules' implementations of it? Here is the base structure of what I'm describing: module UsesOauth def self.included(base) base.class_eval do def save puts "Saving with Oauth!" end def save_with_oauth save end end end end module UsesOpenid def self.included(base) base.class_eval do def save puts "Saving with OpenID!" end def save_with_openid save end end end end module Sequencer def save if using_oauth? save_with_oauth elsif using_openid? save_with_openid else super end end end class User < ActiveRecord::Base include UsesOauth include UsesOpenid include Sequencer end I was thinking about using alias_method like so, but that got too complicated, because I might have 1 or 2 more similar modules. I also tried using those save_with_oauth methods (shown above), which almost works. The only thing that's missing is that I also need to call ActiveRecord::Base#save (the super method), so something like this: def save_with_oauth # do this and that super.save # the rest end But I'm not allowed to do that in ruby. Any ideas for a clever solution to this?

    Read the article

  • Using Zend Framework and Doctrine with independend modular structures

    - by stefax
    I've seen a lot of articles about integrating ZF and Doctrine. There is also a proposal for ZF here but they have always two possible structures. Either they put all models into one top level model directory or they put it into a module related model directory. application |-- Bootstrap.php |-- configs |-- controllers |-- models - EITHER HERE |-- modules | -- examplemodule | |-- controllers | |-- models - OR HERE | |-- views |-- views For our projects I see problems for either of the two options: 1. One directory: application/models - in a complex system after a short time there will be hundreds of files, over all when you have the table classes two (e.g. User.php and UserTable.php). 2. Module based model directories: application/modules/examplemodule/models - in many cases we use models in multiple modules at the same time. So the "User" is required e.g. in the modules "game", "administration", ... Is there a way to use some kind of sub directories under the top level directory "models" to get some grouping. It should be completely independent of the module structure. application |-- Bootstrap.php ... |-- models | -- user | |-- User.php | |-- Friend.php | |-- other user related models | -- game | |-- Game.php | |-- Score.php | |-- ... ... Any solution should support autoloading and the class generation from yaml files. Any ideas, links or solutions? Thanks!

    Read the article

  • HTTP crawler in Erlang

    - by ctp
    I'm coding on a simple HTTP crawler but I have an issue running the code at the bottom. I'm requesting 50 URLs and get the content of 20+ back. I've generated few files with 150kB size each to test the crawler. So I think the 20+ responses are limited by the bandwidth? BUT: how to tell the Erlang snippet not to quit until the last file is not fetched? The test data server is online, so plz try the code out and any hints are welcome :) -module(crawler). -define(BASE_URL, "http://46.4.117.69/"). -export([start/0, send_reqs/0, do_send_req/1]). start() -> ibrowse:start(), proc_lib:spawn(?MODULE, send_reqs, []). to_url(Id) -> ?BASE_URL ++ integer_to_list(Id). fetch_ids() -> lists:seq(1, 50). send_reqs() -> spawn_workers(fetch_ids()). spawn_workers(Ids) -> lists:foreach(fun do_spawn/1, Ids). do_spawn(Id) -> proc_lib:spawn_link(?MODULE, do_send_req, [Id]). do_send_req(Id) -> io:format("Requesting ID ~p ... ~n", [Id]), Result = (catch ibrowse:send_req(to_url(Id), [], get, [], [], 10000)), case Result of {ok, Status, _H, B} -> io:format("OK -- ID: ~2..0w -- Status: ~p -- Content length: ~p~n", [Id, Status, length(B)]); Err -> io:format("ERROR -- ID: ~p -- Error: ~p~n", [Id, Err]) end. That's the output: Requesting ID 1 ... Requesting ID 2 ... Requesting ID 3 ... Requesting ID 4 ... Requesting ID 5 ... Requesting ID 6 ... Requesting ID 7 ... Requesting ID 8 ... Requesting ID 9 ... Requesting ID 10 ... Requesting ID 11 ... Requesting ID 12 ... Requesting ID 13 ... Requesting ID 14 ... Requesting ID 15 ... Requesting ID 16 ... Requesting ID 17 ... Requesting ID 18 ... Requesting ID 19 ... Requesting ID 20 ... Requesting ID 21 ... Requesting ID 22 ... Requesting ID 23 ... Requesting ID 24 ... Requesting ID 25 ... Requesting ID 26 ... Requesting ID 27 ... Requesting ID 28 ... Requesting ID 29 ... Requesting ID 30 ... Requesting ID 31 ... Requesting ID 32 ... Requesting ID 33 ... Requesting ID 34 ... Requesting ID 35 ... Requesting ID 36 ... Requesting ID 37 ... Requesting ID 38 ... Requesting ID 39 ... Requesting ID 40 ... Requesting ID 41 ... Requesting ID 42 ... Requesting ID 43 ... Requesting ID 44 ... Requesting ID 45 ... Requesting ID 46 ... Requesting ID 47 ... Requesting ID 48 ... Requesting ID 49 ... Requesting ID 50 ... OK -- ID: 49 -- Status: "200" -- Content length: 150000 OK -- ID: 47 -- Status: "200" -- Content length: 150000 OK -- ID: 50 -- Status: "200" -- Content length: 150000 OK -- ID: 17 -- Status: "200" -- Content length: 150000 OK -- ID: 48 -- Status: "200" -- Content length: 150000 OK -- ID: 45 -- Status: "200" -- Content length: 150000 OK -- ID: 46 -- Status: "200" -- Content length: 150000 OK -- ID: 10 -- Status: "200" -- Content length: 150000 OK -- ID: 09 -- Status: "200" -- Content length: 150000 OK -- ID: 19 -- Status: "200" -- Content length: 150000 OK -- ID: 13 -- Status: "200" -- Content length: 150000 OK -- ID: 21 -- Status: "200" -- Content length: 150000 OK -- ID: 16 -- Status: "200" -- Content length: 150000 OK -- ID: 27 -- Status: "200" -- Content length: 150000 OK -- ID: 03 -- Status: "200" -- Content length: 150000 OK -- ID: 23 -- Status: "200" -- Content length: 150000 OK -- ID: 29 -- Status: "200" -- Content length: 150000 OK -- ID: 14 -- Status: "200" -- Content length: 150000 OK -- ID: 18 -- Status: "200" -- Content length: 150000 OK -- ID: 01 -- Status: "200" -- Content length: 150000 OK -- ID: 30 -- Status: "200" -- Content length: 150000 OK -- ID: 40 -- Status: "200" -- Content length: 150000 OK -- ID: 05 -- Status: "200" -- Content length: 150000 Update: thanks stemm for the hint with the wait_workers. I've combined your and mine code but same behaviour :( -module(crawler). -define(BASE_URL, "http://46.4.117.69/"). -export([start/0, send_reqs/0, do_send_req/2]). start() -> ibrowse:start(), proc_lib:spawn(?MODULE, send_reqs, []). to_url(Id) -> ?BASE_URL ++ integer_to_list(Id). fetch_ids() -> lists:seq(1, 50). send_reqs() -> spawn_workers(fetch_ids()). spawn_workers(Ids) -> %% collect reference to each worker Refs = [ do_spawn(Id) || Id <- Ids ], %% wait for response from each worker wait_workers(Refs). wait_workers(Refs) -> lists:foreach(fun receive_by_ref/1, Refs). receive_by_ref(Ref) -> %% receive message only from worker with specific reference receive {Ref, done} -> done end. do_spawn(Id) -> Ref = make_ref(), proc_lib:spawn_link(?MODULE, do_send_req, [Id, {self(), Ref}]), Ref. do_send_req(Id, {Pid, Ref}) -> io:format("Requesting ID ~p ... ~n", [Id]), Result = (catch ibrowse:send_req(to_url(Id), [], get, [], [], 10000)), case Result of {ok, Status, _H, B} -> io:format("OK -- ID: ~2..0w -- Status: ~p -- Content length: ~p~n", [Id, Status, length(B)]), %% send message that work is done Pid ! {Ref, done}; Err -> io:format("ERROR -- ID: ~p -- Error: ~p~n", [Id, Err]), %% repeat request if there was error while fetching a page, do_send_req(Id, {Pid, Ref}) %% or - if you don't want to repeat request, put there: %% Pid ! {Ref, done} end. Running the crawler forks fine for a handful of files, but then the code even doesnt fetch the entire files (file size each 150000 bytes) - he crawler fetches some files partially, see the following web server log :( 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /10 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /1 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /3 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /8 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /39 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /7 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /6 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /2 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /5 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /50 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /9 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /44 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /38 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /47 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /49 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /43 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /37 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /46 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /48 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:00 +0200] "GET /36 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /42 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /41 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /45 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /17 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /35 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /16 HTTP/1.1" 200 150000 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /15 HTTP/1.1" 200 17020 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /21 HTTP/1.1" 200 120360 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /40 HTTP/1.1" 200 117600 "-" "-" 82.114.62.14 - - [13/Sep/2012:15:17:01 +0200] "GET /34 HTTP/1.1" 200 60660 "-" "-" Any hints are welcome. I have no clue what's going wrong there :(

    Read the article

  • How to recursive rake? -- or suitable alternatives

    - by TerryP
    I want my projects top level Rakefile to build things using rakefiles deeper in the tree; i.e. the top level rakefile says how to build the project (big picture) and the lower level ones build a specific module (local picture). There is of course a shared set of configuration for the minute details of doing that whenever it can be shared between tasks: so it is mostly about keeping the descriptions of what needs building, as close to the sources being built. E.g. /Source/Module/code.foo and cie should be built using the instructions in /Source/Module/Rakefile; and /Rakefile understands the dependencies between modules. I don't care if it uses multiple rake processes (ala recursive make), or just creates separate build environments. Either way it should be self-containable enough to be processed by a queue: so that non-dependent modules could be built simultaneously. The problem is, how the heck do you actually do something like that with Rake!? I haven't been able to find anything meaningful on the Internet, nor in the documentation. I tried creating a new Rake::Application object and setting it up, but whatever methods I try invoking, only exceptions or "Don't know how to build task ':default'" errors get thrown. (Yes, all rakefiles have a :default). Obviously one could just execute 'rake' in a sub directory for a :modulename task, but that would ditch the options given to the top level; e.g. think of $(MAKE) and $(MAKEFLAGS). Anyone have a clue on how to properly do something like a recursive rake?

    Read the article

  • JBoss: What does the warning EJBTHREE-1246 from the InterceptorRegistry mean?

    - by Simon Lehmann
    I am currently developing an EJB 3.0 based application on the JBoss AS 5.0.0.GA and just recently the following warning showed up in the server log: 09:50:19,735 WARN [InterceptorsFactory] EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container 09:50:19,735 WARN [InterceptorsFactory] EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container 09:50:19,735 WARN [InterceptorRegistry] applicable interceptors is non-existent for ... ... The warnings are generated as soon as an EJB (a stateless session bean) is injected into a backing bean of my JSF web-application. The beans do work without problems though, but I still would like to know where that warning comes from and what I can do to avoid it. I have already searched a bit around, but haven't found a good answer (some say, bean developers do not have to worry, but its a warning, so I'd like to have a better solution): http://www.jboss.com/index.html?module=bb&op=viewtopic&t=147292 http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4180366 http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4140136 JBoss JIRA Issue about the warning (Not that helpful in my opinion) So has anyone an explanation what causes the warning (in terms of a bean developer, not application server developer) and what to do to avoid it? Update: I've just upgraded JBoss to 5.0.1.GA (Download, Release-Notes) but unfortunatly the warning still appears.

    Read the article

  • ASP.Net HttpModule List<> content

    - by test
    We have created an http module that loads data on initialize into a list, and uses that list on every endrequest event of application. However we got a null reference exception on the module at endRequest event complaining that the content is null. Yet we are sure that at module initialization we have put non-null objects correctly. How is this possible? Any ideas? Thanks in advance. The place where i fill the data into the list what named as "Pages" : using (SqlDataReader reader = cmd.ExecuteReader()) { while (reader.Read()) { LandingPageDetails details = new LandingPageDetails(); details.DomainId = Convert.ToInt32(reader["DomainId"]); details.PageId = Convert.ToInt32(reader["TrackingPageId"]); details.PageAddress = Convert.ToString(reader["PageAddress"]); if (!string.IsNullOrEmpty(details.PageAddress)) { Pages.Add(details); } } } // Find the related record on EndRequesst event LandingPageDetails result = Pages.Find ( delegate(LandingPageDetails current) { return current.PageAddress.Equals(url, StringComparison.InvariantCultureIgnoreCase); } ); The line what we get the error return current.PageAddress.Equals(url, StringComparison.InvariantCultureIgnoreCase); "current" is the null object P.S : It works properly, but one day later contents become null. Possibly, there is something wrong on IIS. Our server is Windows 2003, IIS 6.0

    Read the article

  • Problem in migrating to LAMP from XAMPP.. Memory limit error

    - by Geshan
    I was using XAMPP for my local machine but as I wanted to run applications like mysql work bench and some test frameworks I decided to switch to LAMP self install. I'm using ubuntu and followed the instructions at: https://help.ubuntu.com/community/ApacheMySQLPHP But the problem is LAMP is consuming too much of my memory (RAM) I've allocated 124 MB currently but still it gives me memory exhausted error when I run Drush (Drupal command line). When I do drush cc to clear cache it give me the following: Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 122880 bytes) in /var/www/----/sites/all/modules/ubercart/uc_order/uc_order.order_pane.inc on line 150 Call Stack: 0.0020 185624 1. {main}() /opt/drush/drush.php:0 0.0254 1303672 2. drush_main() /opt/drush/drush.php:37 0.2674 5107784 3. drush_bootstrap() /opt/drush/drush.php:71 0.2676 5109872 4. _drush_bootstrap_drupal_full() /opt/drush/includes/environment.inc:173 0.2676 5151032 5. drupal_bootstrap() /opt/drush/includes/environment.inc:655 0.3030 7739048 6. _drupal_bootstrap() /var/www/missmoti/includes/bootstrap.inc:989 0.3122 8855792 7. _drupal_bootstrap_full() /var/www/missmoti/includes/bootstrap.inc:1078 0.3445 12387320 8. module_load_all() /var/www/missmoti/includes/common.inc:2608 0.5194 32586544 9. drupal_load() /var/www/missmoti/includes/module.inc:14 0.5251 33361112 10. include_once('/var/www/missmoti/sites/all/modules/ubercart/uc_order/uc_order.module') /var/www/-----/includes/bootstrap.inc:617 Drush command could not be completed. In each error it shows me a back trace and I guess this default debugger I'm not aware of in Apache or my PHP config it eating up the memory. If anyone can help I"d be glad. Another error below: Fatal error: Call to undefined function dsm() in /var/www/-----/sites/all/modules/custom/gtpath/gtpath.module on line 180 Call Stack # Time Memory Function Location 1 0.0002 120144 {main}( ) ../index.php:0 2 1.7604 68224112 theme( ) ../index.php:36 3 2.0188 77346112 call_user_func_array ( ) ../theme.inc:658 4 2.0188 77347024 gtpath_preprocess_page( ) ../theme.inc:0 how do I deal with this default debugger? how do I turn it off??

    Read the article

  • Is it possible to write an IIS URL Rewrite Rule that examines content of HTTP Post?

    - by JohnRudolfLewis
    I need to split a portion of functionality away from a legacy ISAPI dll onto another solution (ASP.NET MVC most likely). IIS7's URL Rewrite sounded like a perfect candidate for the job, but it turns out I cannot find a way to configure the rules the way I need. I need to write a rule that examines the content of the HTTP post for a particular value. i.e. <form method="post" action="legacy_isapi.dll"> <input name="foo" /> </form> if (Request.Form["foo"] == "bar") Context.RewritePath("/some_other_url/on_the_same_machine/foo/bar"); As a proof of concept, I was able to create an IHttpModule that examines context.Request.Form collection and performs a rewrite when certain parameters are present. I installed this module in my website, and it works. Rather than a custom module, however, I'd rather extend the existing URL Rewrite module to support examining the content of the HTTP Post as one of its rules. Is this possible?

    Read the article

  • How to manipulate a header and then continue with it in C#?

    - by Simon Linder
    Hi all, I want to replace an old ISAPI filter that ran on IIS6. This filter checks if the request is of a special kind, then manipulates the header and continues with the request. Two headers are added in the manipulating method that I need for calling another special ISAPI module. So I have ISAPI C++ code like: DWORD OnPreProc(HTTP_FILTER_CONTEXT *pfc, HTTP_FILTER_PREPROC_HEADERS *pHeaders) { if (ManipulateHeaderInSomeWay(pfc, pHeaders)) { return SF_STATUS_REQ_NEXT_NOTIFICATION; } return SF_STATUS_REQ_FINISHED; } I now want to rewrite this ISAPI filter as a managed module for the IIS7. So I have something like this: private void OnMapRequestHandler(HttpContext context) { ManipulateHeaderInSomeWay(context); } And now what? The request seems not to do what it should? I already wrote an IIS7 native module that implements the same method. But this method has a return value with which I can tell what to do next: REQUEST_NOTIFICATION_STATUS CMyModule::OnMapRequestHandler(IN IHttpContext *pHttpContext, OUT IMapHandlerProvider *pProvider) { if (DoSomething(pHttpContext)) { return RQ_NOTIFICATION_CONTINUE; } return RQ_NOTIFICATION_FINISH_REQUEST; } So is there a way to send my manipulated context again?

    Read the article

  • Compiled Haskell libraries with FFI imports are invalid when imported into GHCI

    - by John Millikin
    I am using GHC 6.12.1, in Ubuntu 10.04 When I try to use the FFI syntax for static storage, only modules running in interpreted mode (ie GHCI) work properly. Compiled modules have invalid pointers, and do not work. I'd like to know whether anybody can reproduce the problem, whether this an error in my code or GHC, and (if the latter) whether it's a known issue. I'm using sys_siglist because it's present in a standard library on my system, but I don't believe the actual storage used matters (I discovered this while writing a binding to libidn). If it helps, sys_siglist is defined in <signal.h> as: extern __const char *__const sys_siglist[_NSIG]; I thought this type might be the problem, so I also tried wrapping it in a plain C procedure: #include<stdio.h> const char **test_ffi_import() { printf("C think sys_siglist = %X\n", sys_siglist); return sys_siglist; } However, importing that doesn't change the result, and the printf() call prints the same pointer value as show siglist_a. My suspicion is that it's something to do with static and dynamic library loading. Update: somebody in #haskell suggested this might be 64-bit specific; if anybody tries to reproduce it, can you mention your architecture and whether it worked in a comment? Code as follows: -- A.hs {-# LANGUAGE ForeignFunctionInterface #-} module A where import Foreign import Foreign.C foreign import ccall "&sys_siglist" siglist_a :: Ptr CString -- -- B.hs {-# LANGUAGE ForeignFunctionInterface #-} module B where import Foreign import Foreign.C foreign import ccall "&sys_siglist" siglist_b :: Ptr CString -- -- Main.hs {-# LANGUAGE ForeignFunctionInterface #-} module Main where import Foreign import Foreign.C import A import B foreign import ccall "&sys_siglist" siglist_main :: Ptr CString main = do putStrLn $ "siglist_a = " ++ show siglist_a putStrLn $ "siglist_b = " ++ show siglist_b putStrLn $ "siglist_main = " ++ show siglist_main peekSiglist "a " siglist_a peekSiglist "b " siglist_b peekSiglist "main" siglist_main peekSiglist name siglist = do ptr <- peekElemOff siglist 2 str <- maybePeek peekCString ptr putStrLn $ "siglist_" ++ name ++ "[2] = " ++ show str I would expect something like this output, where all pointer values identical and valid: $ runhaskell Main.hs siglist_a = 0x00007f53a948fe00 siglist_b = 0x00007f53a948fe00 siglist_main = 0x00007f53a948fe00 siglist_a [2] = Just "Interrupt" siglist_b [2] = Just "Interrupt" siglist_main[2] = Just "Interrupt" However, if I compile A.hs (with ghc -c A.hs), then the output changes to: $ runhaskell Main.hs siglist_a = 0x0000000040378918 siglist_b = 0x00007fe7c029ce00 siglist_main = 0x00007fe7c029ce00 siglist_a [2] = Nothing siglist_b [2] = Just "Interrupt" siglist_main[2] = Just "Interrupt"

    Read the article

  • How can I apply a style to existing tikz node on specific slides

    - by Eugene Pimenov
    This is what I'm trying to do \begin{tikzpicture} [node distance = 1cm, auto,font=\footnotesize, % STYLES every node/.style={node distance=1.3cm}, comment/.style={rectangle, inner sep= 5pt, text width=4cm, node distance=0.25cm, font=}, module/.style={rectangle, drop shadow, draw, fill=black!10, inner sep=5pt, text width=3cm, text badly centered, minimum height=0.8cm, font=\bfseries\footnotesize\sffamily,rounded corners}, selected/.style={fill=red!40}] \node [module] (nodeA) {node A}; \node [module, below of=nodeA] (nodeA) {node B}; \only<1>{ \node [comment, text width=6cm, right=0.25 of nodeA] {short description of Node A}; \node [comment, text width=6cm, right=0.25 of nodeB] {short description of Node B}; } \only<2>{ \node [selected] (nodeA) {}; \node [comment, text width=6cm, right=0.25 of nodeA] {long description of node A}; } \only<3>{ \node [selected] (nodeB) {}; \node [comment, text width=6cm, right=0.25 of nodeA] {long description of node B}; } \end{tikzpicture} The problem is \node [selected] (nodeB) {}; creates a new node, but I want it to apply the style for the existing node. Is there any way to do so? Of course I could have copies of every node in selected state and not-selected state, but I really want to have a normal solution.

    Read the article

< Previous Page | 102 103 104 105 106 107 108 109 110 111 112 113  | Next Page >