Search Results

Search found 58272 results on 2331 pages for 'apache log files'.

Page 127/2331 | < Previous Page | 123 124 125 126 127 128 129 130 131 132 133 134  | Next Page >

• Installing php-module using port on osx lion. It works from command line but not from apache

  - by Lorraine Bernard

  I installed php5-intl on my osx lion. It seems to work properly on command line mode because if I try to run the following script (1): $fmt = datefmt_create( "en_US" ,IntlDateFormatter::FULL,IntlDateFormatter::FULL,'America/Los_Angeles',IntlDateFormatter::GREGORIAN ); echo "First Formatted output is ".datefmt_format( $fmt , 0)."\n"; > php -m |grep intl intl > php test.php First Formatted output is Wednesday, December 31, 1969 4:00:00 PM Pacific Standard Time But if I try to sun the same from my apache (after sudo apachectl restart) I get the following error: > tail /private/var/log/apache2/error_log PHP Fatal error: Call to undefined function datefmt_create() in P.S.: I also added the following line to my php.ini extension_dir = "/opt/local/lib/php/extensions/no-debug-non-zts-20090626" ll /opt/local/lib/php/extensions/no-debug-non-zts-20090626 total 848 -rwxr-xr-x 1 root 261712 7 May 10:54 xdebug.so -rwxr-xr-x 1 root 168912 31 May 19:53 intl.so

  Read the article

• Is there a way to get Apache to serve files with the question mark in their name?

  - by ldrg

  I scraped a bunch of pages using wget -m -k -E. The resulting files have names in the form foo.php?bar.html. Apache guesses everything after the ? is a query string, is there a way to tell it to ignore the ? as the query string delimiter (and see foo.php?bar.html as the requested file and not foo.php)? To save you a trip to wget manpage: -m : mirror recursively -E : foo.php?bar becomes foo.php?bar.html -k : convert links in pages (foo.php?bar now links to foo.php?bar.html inside of all the pages so they display properly)

  Read the article

• Apache rewrite to add a directory to REQUEST_URI not working on localhost running wamp.

  - by Brett Pontarelli

  I'm running wamp on Vista (Apache v2.2.11) and have projects setup such that http://localhost/projectx is the base directory for projectx. Now, I want that requests for http://localhost/projectx/somepage/extra will rewrite to http://localhost/projectx/PUBLIC/somepage/extra To that end I have a file in C:\wamp\www\projectx\.htacces that is this simple: RewriteEngine On RewriteBase /projectx RewriteCond %{REQUEST_URI} !^/PUBLIC RewriteRule ^(.*)$ /PUBLIC$1 [L] I can't for the life of me figure out why this doesn't work. The error I'm getting is "The requested URL /PUBLIC was not found on this server". Thanks.

  Read the article

• PHP/Apache: Permission settings for uploaded JPEG image files not correct.

  - by Mike

  I just setup a LAMP development server and am still trouble-shooting some things. The server is installed on one computer and I use a Windows laptop to write my code and test the site via the web browser. My file uploading script works in that JPEG image files are successfully uploaded to the server, but when I try to view the images in the web browser, permission is denied. I check the permissions on the file via the server and they are 600. I can fix the issue by chmod 777 theimage.jpg, but this doesn't seem like a good solution at all. Does the solution have something to do with Apache configuration? Or is there something else I should be doing. Thank-you, Mike

  Read the article

• Autoconf macros for Apache and conf.d install process?

  - by Stephen Burke

  I have a package that is using the autotools to build and install. Part of the package is a website that can be run on the local machine. So in the package there is a .conf file that is meant to be either copied or linked to the /etc/apache2/conf.d directory. What's the standard way that packages would do this? If possible, I'd like for the user not to have an extra step to make the website work. I'd like to have them install the package and then be able to browse to http://localhost/newpackage to get up and running. Also, is there a way that autoconf knows about the apache install or a standard way through then environment some how? If someone could point me in the right direction that would be great. Steve

  Read the article

• Custom Date Format with jax-rs in apache cxf?

  - by Oscar Chan

  Hi, I have been googling to figure out how I can customize the Date format when I use jax-rs on apache CXF. I looked at the codes, and it seems that it only support primitives, enum and a special hack that assume the type associated with @ForumParam has a constructor with a single string parameter. This force me to use String instead of Date if I want to use ForumParam. it is kind of ugly. Is there a better way to do it? @POST @Path("/xxx") public String addPackage(@FormParam("startDate") Date startDate) { ... } Thanks

  Read the article

• How to diganose an unlogged Error 500 on Apache?

  - by samuel morhaim

  We are running a very simple Symfony script, that randomly returns an Error 500. The system administrator says he can't find any trace of an Error 500 on the error logs, however using Curl or Firebug, it is obvious that an Error 500 is being returned. The script simply parses a POST request submitted to an URL on our server. We already checked for performance, memory etc but nothing seems to be the problem. How is this possible? We already enabled all debugging, logging, on Apache, PHP, etc and nothing.

  Read the article

• Help with replacing Mac factory install of apache with MAMP PRO so FileMaker will work with my other

  - by tannerSF

  I am running Mac 10.4 and have been using MAMP PRO to host several websites I manage. Now that I have installed FileMaker Pro Server, it forces me to turn Web Sharing on in System Preference. So is it possible to either replace Apache with MAMP PRO's version? Or is it possible to get FileMaker Pro to use MAMP instead of Web Sharing? If so, how can I make it happen? Also, with FileMaker Instant Web Publishing, how , how can I make my domain link directly to where FileMaker publishes it? So what I want to be able to do is type http:///mydomain.com and it direct to mydomain.com:591/FMI/IWP/

  Read the article

• Using Alias in Apache ONLY if local directory not present?

  - by Andrew E.

  Hey all, We're running a reseller web host, and aliasing a particular directory for ALL the sites within. This is all happening through WHM/Cpanel, which could possibly complicate my question. We want all hosts to run their "/concrete" directory through a shared location on the server. This is working fine by adding "Alias /concrete/ "/usr/local/share/concrete5/concrete-latest/concrete/" to the apache include editor in WHM. However, if a local concrete/ directory exists within the particular webroot of the virtual host, we'd rather use that than "/concrete" globally. It would also be nice if we could make this alias change in the context of the virtual host, within WHM (and I haven't found a great way to modify virtual host sections in WHM without hacking a file that I don't believe we're supposed to hack.) Thoughts?

  Read the article

• Which Happens First? Anyone Know Exactly How The Apache Server Will Handle This Request?

  - by user310594

  Hello, To keep things simple, please allow the "assumption" that some code requires the use of a full URL, even though the domain is on the same server, i.e. a simple file path cannot be used. TCP/IP?? Question: If a form action target = "http://this-full-URL.com/postdata" (for example) and that URL is also on the same server, then which happens first? A) Data is sent "out onto the web", and then returns to the same server, or B) Before sending any (possibly sensitive) data, the server (Linux, Apache, PHP), first "discovers" the target address is local, so (clearly) no data is sent over the net? Thank you.

  Read the article

• to get columns from Excel files using Apache POI??

  - by posdef

  Hi, In order to do some statistical analysis I need to extract values in a column of an Excel sheet. I have been using the Apache POI package to read from Excel files, and it works fine when one needs to iterate over rows. However I couldn't find anything about getting columns neither in the API (link text) nor through google searching. As I need to get max and min values of different columns and generate random numbers using these values, so without picking up individual columns, the only other option is to iterate over rows and columns to get the values and compare one by one, which doesn't sound all that time-efficient. Any ideas on how to tackle this problem? Thanks,

  Read the article

• Redirect to Apache built-in 404 page with mod_rewrite?

  - by Pekka

  Is there a way to actively serve Apache's default, built-in 404 page for a number of URLs using mod_rewrite? Not a custom error document, but a rule like RewriteCond %{REQUEST_URI} ^/dirname/pagename RewriteRule -- serve 404 page ----- I know how to build a PHP page that sends the 404 header and have mod_rewrite redirect all the URLs there but I would prefer a solution that is based on mod_rewrite only. I just had the idea of redirecting to a non-existent address: RewriteCond %{REQUEST_URI} ^/dirname/pagename RewriteRule .* /sflkadsölfkasdfölkasdflökasdf but that would give the user the message "/sflkadsölfkasdfölkasdflökasdf does not exist" on the error page, which looks a bit unprofessional.

  Read the article

• Apache Axis web service clients vs plain SOAP requests.

  - by Andy Pryor

  I'm looking for the best way to consume a Java web service that returns rather large and complex objects. I am currently using Apache Axis clients generated from the wsdl, (using eclipse "generate web service client" tool). We have concerns about performance of this. The service proxy objects are not thread safe, and they are rather heavy to instantiate, 2-3 MB on the JVM. The other alternative is making HTTP connections and building a String SOAP requests. I would have to interpret the response, and build objects from the XML. Would this be a better alternative to the heavy axis objects? I searched for good reading on this, if any one had any links I would greatly appreciate it.

  Read the article

• what is the responsibility of ngnix or apache in rails application?

  - by user2177410

  How rails applications actually work? Let's say we have nginx + passenger + Ubuntu, so my questions are: What is nginx actually doing? How does it transfer requests to the rails app? what is passenger responsible for? And what is rack? How actually rails app can work just on webrick without apache? Please dont give me answers like "nginx processes the requests"; I need something more, or may be you know the source where I can read about this.

  Read the article

• Will PHP script running on top of Apache be faster than C# stand alloun programm doing same thing (s

  - by Ole Jak

  I mean PHP scripts on Apache are oriented for many users to use tham at the same time. So will 1000 requests which came at the (relativly) same time be fully responsed faster than C# .Net programm perfoming algorithm 1000 times in while loop? So we input same data, we perform same algorithm, which is written in a wary same way (respecting language diferencis ofcourse), outputing same data (lat us say saving it to file for tham to be relativly equal) Who will be faster on some 1000 times of performing O(NN) algorithm, in which case (if it is possible) one will owerrun another?

  Read the article

• Does migrating a site that is 99% several megs of static HTML from Apache to Google App Enging make

  - by JonathanHayward

  I have a large site of mostly static content, and I have entertained migrating to Google App Engine. I am wondering, not so much if it is possible as whether that is cutting a steak with a screwdriver. I see a way to do it in Django that has a bad design smell. Does migrating a literature site that is largely static HTML from Apache to Google App Engine make sense? I'm not specifically asking for a comparison to Nginx or Cherokee; I am interested in whether migrating from a traditional web hosting solution to a more cloudy type of solution recommends itself. The site is JonathansCorner.com, and is presently unavailable ("the magic blue smoke has escaped").

  Read the article

• SharePoint OCR image files indexing

  Introduction This article describes how to setup indexing of the image files (including TIFF, PDF, JPEG, BMP...) using OCR technology. The indexing described below utilizes Microsoft IFilter technology and as such is not specific to SharePoint, but can be used with any product that uses Microsoft indexing: Microsoft Search, Desktop search, SQL Server search, and through the plug-ins with Google desktop search. I however use it with Microsoft Windows SharePoint Services 2003. For those other products, the registration may need to be slightly different. Background  One of the projects I was working on required a storage of old documents scanned into PDF files. Then there was a separate team of people responsible for providing a tags for a search engine so those image documents could be found. The whole process was clumsy, labor intensive, and error prone. That was what started me on my exploration path. OCR The first search I fired was for the Open Source OCR products. Pretty quickly, I narrowed it down to TESSERACT (http://code.google.com/p/tesseract-ocr/). Tesseract is an orphaned brain child of HP that worked on it from 1985 to 1995. Then it was moved to the Open Source, and now if I understand it correctly, Google is working on it. With credentials like that, it's no wonder that Tesseract scores one of the highest marks on OCR recognition and accuracy. After downloading and struggling just a bit, I got Tesseract to work. The struggling part was that the home page claims that its base input format is a TIFF file. May be my TIFFs were bad, but I was able to get it to work only for BMP files. Image files conversion So now that I have an OCR that can convert BMP files into text, how do I get text out of the image PDF files? One more search, and I settled down on ImageMagic (http://www.imagemagick.org/). This is another wonderful Open Source utility that can convert any file into image. It did work out of the box, converting any TIFF files into bitmaps, but to get PDF files converted, it requires a GhostScript (http://mirror.cs.wisc.edu/pub/mirrors/ghost/GPL/gs864/gs864w32.exe). Dealing with text PDFs With that utility installed, I was cooking - I can convert any file (in particular PDF and TIFF) into bitmap, and then I can extract the text out of the bitmap. The only consideration was to somehow treat PDF files containing text differently - after all, OCR is very computation intensive and somewhat error prone even with perfect image quality and resolution. So another quick search, and I have a PDFTOTEXT (ftp://ftp.foolabs.com/pub/xpdf/xpdf-3.02pl4-win32.zip) - thank God for Open Source! With these guys, I can pull text out of PDF in an eye blink. However, I would get nothing for pure image PDFs, but I already have a solution for that! Batch process It took another 15 minutes to setup a batch script to automate the process: Check the file extension If file is a PDF file try to extract text out of it if there is more than certain amount of text in the file - done! if there is no text, convert first page into bitmap run OCR on the bitmap For any other file type, convert file into bitmap Run OCR on the bitmap Once you unzip the attached project, check out the bin\OCR.BAT file. It will create a temporary file in the directory where your source file is with the same name + the '.txt' extension.Continue span.fullpost {display:none;}

  Read the article

• Convert .3GP and .3G2 Files to AVI / MPEG for Free

  - by DigitalGeekery

  3GP and .3G2 are common video capture formats used on many mobile phones, but they may not be supported by your favorite media player. Today we’ll show you a quick and easy way to convert those files to AVI or MPG format with the free Windows application, Pazera Free 3GP to AVI Converter. Download the Pazera Free 3GP to AVI Converter. You’ll have to unzip the download folder, but there is no need to install the application. Just double-click the 3gptoavi.exe file to run the application. To add your 3GP or 3G2 files to the queue to be converted, click on the Add files  button at the top left. Browse for your file, and click Open.   Your video will be added to the Queue. You can add multiple files to the queue and convert them all at one time.   Most users will find it preferable to use one of the pre-configured profiles for their conversion settings. To load a profile, choose one from the Profile drop down list and then click the Load button. You will see the profile update the settings in the panels at the bottom of the application. We tested Pazera Free 3GP to AVI Converter with 3GP files recorded on a Motorola Droid, and found the AVI H.264 Very High Q. profile to return the best results for AVI output, and the MPG – DVD NTSC: MPEG-2 the best results for MPG output. Other profiles produced smaller file sizes, but at a cost of reduced quality video output.   More advanced users may tweak video and audio settings to their liking in the lower panels. Click on the AVI button under Output file format / Video settings to adjust settings AVI… Or the MPG button to adjust the settings for MPG output. By default, the converted file will be output to the same location as the input directory. You can change it by clicking the text box input radio button and browsing for a different folder. When you’ve chosen your settings, click Convert to begin the conversion process.   A conversion output box will open and display the progress. When finished, click Close. Now you’re ready to enjoy your video in your favorite media player. Pazera Free 3GP to AVI Converter isn’t the most robust media conversion tool, but it does what it is intended to do. It handles the task of 3GP to AVI / MPG conversion very well. It’s easy enough for the beginner to manage without much trouble, but also has enough options to please more experienced users. Download Pazera Free 3GP to AVI Converter Similar Articles Productive Geek Tips How To Convert Video Files to MP3 with VLCEasily Change Audio File Formats with XRECODEConvert PDF Files to Word Documents and Other FormatsConvert Video and Remove Commercials in Windows 7 Media Center with MCEBuddy 1.1Compress Large Video Files with DivX / Xvid and AutoGK TouchFreeze Alternative in AutoHotkey The Icy Undertow Desktop Windows Home Server – Backup to LAN The Clear & Clean Desktop Use This Bookmarklet to Easily Get Albums Use AutoHotkey to Assign a Hotkey to a Specific Window Latest Software Reviews Tinyhacker Random Tips DVDFab 6 Revo Uninstaller Pro Registry Mechanic 9 for Windows PC Tools Internet Security Suite 2010 Install, Remove and HIDE Fonts in Windows 7 Need Help with Your Home Network? Awesome Lyrics Finder for Winamp & Windows Media Player Download Videos from Hulu Pixels invade Manhattan Convert PDF files to ePub to read on your iPad

  Read the article

• Copy New Files Only in .NET

  - by psheriff

  Recently I had a client that had a need to copy files from one folder to another. However, there was a process that was running that would dump new files into the original folder every minute or so. So, we needed to be able to copy over all the files one time, then also be able to go back a little later and grab just the new files. After looking into the System.IO namespace, none of the classes within here met my needs exactly. Of course I could build it out of the various File and Directory classes, but then I remembered back to my old DOS days (yes, I am that old!). The XCopy command in DOS (or the command prompt for you pure Windows people) is very powerful. One of the options you can pass to this command is to grab only newer files when copying from one folder to another. So instead of writing a ton of code I decided to simply call the XCopy command using the Process class in .NET. The command I needed to run at the command prompt looked like this: XCopy C:\Original\*.* D:\Backup\*.* /q /d /y What this command does is to copy all files from the Original folder on the C drive to the Backup folder on the D drive. The /q option says to do it quitely without repeating all the file names as it copies them. The /d option says to get any newer files it finds in the Original folder that are not in the Backup folder, or any files that have a newer date/time stamp. The /y option will automatically overwrite any existing files without prompting the user to press the "Y" key to overwrite the file. To translate this into code that we can call from our .NET programs, you can write the CopyFiles method presented below. C# using System.Diagnostics public void CopyFiles(string source, string destination){  ProcessStartInfo si = new ProcessStartInfo();  string args = @"{0}\*.* {1}\*.* /q /d /y";   args = string.Format(args, source, destination);   si.FileName = "xcopy";  si.Arguments = args;  Process.Start(si);} VB.NET Imports System.Diagnostics Public Sub CopyFiles(source As String, destination As String)  Dim si As New ProcessStartInfo()  Dim args As String = "{0}\*.* {1}\*.* /q /d /y"   args = String.Format(args, source, destination)   si.FileName = "xcopy"  si.Arguments = args  Process.Start(si)End Sub The CopyFiles method first creates a ProcessStartInfo object. This object is where you fill in name of the command you wish to run and also the arguments that you wish to pass to the command. I created a string with the arguments then filled in the source and destination folders using the string.Format() method. Finally you call the Start method of the Process class passing in the ProcessStartInfo object. That's all there is to calling any command in the operating system. Very simple, and much less code than it would have taken had I coded it using the various File and Directory classes. Good Luck with your Coding,Paul Sheriff ** SPECIAL OFFER FOR MY BLOG READERS **Visit http://www.pdsa.com/Event/Blog for a free video on Silverlight entitled Silverlight XAML for the Complete Novice - Part 1.  

  Read the article

• Kill all the project files!

  - by jamiet

  Like many folks I’m a keen podcast listener and yesterday my commute was filled by listening to Scott Hunter being interviewed on .Net Rocks about the next version of ASP.Net. One thing Scott said really struck a chord with me. I don’t remember the full quote but he was talking about how the ASP.Net project file (i.e. the .csproj file) is going away. The rationale being that the main purpose of that file is to list all the other files in the project, and that’s something that the file system is pretty good at. In Scott’s own words (that someone helpfully put in the comments): A file that lists files is really redundant when the OS already does this Romeliz Valenciano correctly pointed out on Twitter that there will still be a project.json file however no longer will there be a need to keep a list of files in a project file. I suspect project.json will simply contain a list of exclusions where necessary rather than the current approach where the project file is a list of inclusions. On the face of it this seems like a pretty good idea. I’ve long been a fan of convention over configuration and this is a great example of that. Instead of listing all the files in a separate file, just treat all the files in the directory as being part of the project. Ostensibly the approach is if its in the directory, its part of the project. Simple. Now I’m not an ASP.net developer, far from it, but it did occur to me that the same approach could be applied to the two Visual Studio project types that I am most familiar with, SSIS & SSDT. Like many people I’ve long been irritated by SSIS projects that display a faux file system inside Solution Explorer. As you can see in the screenshot below the project has Miscellaneous and Connection Managers folders but no such folders exist on the file system: This may seem like a minor thing but it means useful Solution Explorer features like Show All Files and Open Folder in Windows Explorer don’t work and quite frankly it makes me feel like a second class citizen in the Microsoft ecosystem. I’m a developer, treat me like one. Don’t try and hide the detail of how a project works under the covers, show it to me. I’m a big boy, I can handle it! Would it not be preferable to simply treat all the .dtsx files in a directory as being part of a project? I think it would, that’s pretty much all the .dtproj file does anyway (that, and present things in a non-alphabetic order – something else that wildly irritates me), so why not just get rid of the .dtproj file? In the case of SSDT the .sqlproj actually does a whole lot more than simply list files because it also states the BuildAction of each file (Build, NotInBuild, Post-Deployment, etc…) but I see no reason why the convention over configuration approach can’t help us there either. Want to know which is the Post-deployment script? Well, its the one called Post-DeploymentScript.sql! Simple! So that’s my new crusade. Let’s kill all the project files (well, the .dtproj & .sqlproj ones anyway). Are you with me? @Jamiet

  Read the article

• Server overloaded: DBCP issue on Mysql.

  - by taras

  Hi, I have 2 setups tomcat5.5.20 on Redhat and mysql 4.1.22 on another Redhat server. Recently i started getting the repeating error(each seconds) in catalina.out: DBCP object created 2010-12-22 13:33:12 by the following code was never closed: java.lang.Exception at org.apache.tomcat.dbcp.dbcp.AbandonedTrace.init(AbandonedTrace.java:96) at org.apache.tomcat.dbcp.dbcp.AbandonedTrace.(AbandonedTrace.java:79) at org.apache.tomcat.dbcp.dbcp.DelegatingResultSet.(DelegatingResultSet.java:71) at org.apache.tomcat.dbcp.dbcp.DelegatingResultSet.wrapResultSet(DelegatingResultSet.java:80) at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92) at org.apache.jsp.external_005fpage.signup_jsp._jspService(signup_jsp.java:1185) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:334) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:199) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:282) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595) i have to restart tomcat once a day when server load reaches 80-90%. Also catalina.out file is growing too fast which every few hours need to clear the logs. My datasource config: <bean id="myDataSource" class="org.apache.tomcat.dbcp.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName"> <value>com.mysql.jdbc.Driver</value> </property> jdbc:mysql://XXX/XXX?autoReconnect=true 20 20 <property name="maxIdle"> <value>50</value> </property> <property name="maxActive"> <value>50</value> </property> <property name="removeAbandoned"> <value>false</value> </property> <property name="removeAbandonedTimeout"> <value>2400</value> </property> <property name="username"> <value>XXX</value> </property> <property name="password"> <value>XXX</value> </property> </bean> Any idea what can be the issue ? Thanks for any direction.

  Read the article

• Homebrew large data cluster access for 2 user levels?

  - by Yegor

  The title probably makes little sense, so here is an example. I have a file hosting site, that serves a large amount of semi-randomly accessed files. The setup is as follows: High horsepower front-end +DB server that also does encoding for files that need encoding Fresh file server, which stores newly uploaded content, thats probably (and usually) rapidly accessible, which has 500GB of raided SSD storage, that can push over 3GBit of traffic. 3 cheap node servers, containing 2 x 750GB SATA drives in raid1, where files older than 2 weeks are archived, from the SSD server (mentioned above). Files on each server are accessed via subdomains (via modsec) in a straight forward fashion (server1.domain.com, server2.domain.com, etc) Where I have the problem is this. I introduced a "premium" service where people pay a small fee every month, and get ad-free, quick accesses to stuff on the site. Once they are logged in, they access same files via premium.server1.domain.com via a different modsec script, with a different pass phrase. That all works fine and dandy.... except the cheap node servers are all IO bound, so accessing the files on them via a different, unsaturated network makes no difference, since it cannot read off the drive fast enough. What would be a good way to make files on the site be accessible via 2 different network routes, 1 of which will be saturated (the "free network") while all other files are on an un-saturated "premium" network?

  Read the article

• Apache commons HTTPClient and log4j.xml

  - by java_pill

  I'm using Apache commons HTTPClient with Apache Axis 1.5 and I'm trying to log the messages exchanged when making Web Service calls by enabling org.apache.commons.httpclient to DEBUG and httpclient.wire to DEBUG. However, this doesn't work. Mentioned below is my log4j.xml - can someone help me? Thanks <?xml version="1.0" encoding="UTF-8" ?> <!DOCTYPE log4j:configuration SYSTEM "log4j.dtd"> <log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/"> <appender name="rolling" class="org.apache.log4j.DailyRollingFileAppender"> <param name="File" value="test.log" /> <layout class="org.apache.log4j.PatternLayout"> <param name="ConversionPattern" value="%d [%t] %-5p %c:%L - %m%n"/> </layout> </appender> <logger name="org.apache.commons.httpclient"> <level value="DEBUG"/> </logger> <logger name="httpclient.wire"> <level value="DEBUG"/> </logger> <root> <level value="DEBUG" /> <appender-ref ref="rolling"/> </root> </log4j:configuration>

  Read the article

• How to simulate inner join on very large files in java (without running out of memory)

  - by Constantin

  I am trying to simulate SQL joins using java and very large text files (INNER, RIGHT OUTER and LEFT OUTER). The files have already been sorted using an external sort routine. The issue I have is I am trying to find the most efficient way to deal with the INNER join part of the algorithm. Right now I am using two Lists to store the lines that have the same key and iterate through the set of lines in the right file once for every line in the left file (provided the keys still match). In other words, the join key is not unique in each file so would need to account for the Cartesian product situations ... left_01, 1 left_02, 1 right_01, 1 right_02, 1 right_03, 1 left_01 joins to right_01 using key 1 left_01 joins to right_02 using key 1 left_01 joins to right_03 using key 1 left_02 joins to right_01 using key 1 left_02 joins to right_02 using key 1 left_02 joins to right_03 using key 1 My concern is one of memory. I will run out of memory if i use the approach below but still want the inner join part to work fairly quickly. What is the best approach to deal with the INNER join part keeping in mind that these files may potentially be huge public class Joiner { private void join(BufferedReader left, BufferedReader right, BufferedWriter output) throws Throwable { BufferedReader _left = left; BufferedReader _right = right; BufferedWriter _output = output; Record _leftRecord; Record _rightRecord; _leftRecord = read(_left); _rightRecord = read(_right); while( _leftRecord != null && _rightRecord != null ) { if( _leftRecord.getKey() < _rightRecord.getKey() ) { write(_output, _leftRecord, null); _leftRecord = read(_left); } else if( _leftRecord.getKey() > _rightRecord.getKey() ) { write(_output, null, _rightRecord); _rightRecord = read(_right); } else { List<Record> leftList = new ArrayList<Record>(); List<Record> rightList = new ArrayList<Record>(); _leftRecord = readRecords(leftList, _leftRecord, _left); _rightRecord = readRecords(rightList, _rightRecord, _right); for( Record equalKeyLeftRecord : leftList ){ for( Record equalKeyRightRecord : rightList ){ write(_output, equalKeyLeftRecord, equalKeyRightRecord); } } } } if( _leftRecord != null ) { write(_output, _leftRecord, null); _leftRecord = read(_left); while(_leftRecord != null) { write(_output, _leftRecord, null); _leftRecord = read(_left); } } else { if( _rightRecord != null ) { write(_output, null, _rightRecord); _rightRecord = read(_right); while(_rightRecord != null) { write(_output, null, _rightRecord); _rightRecord = read(_right); } } } _left.close(); _right.close(); _output.flush(); _output.close(); } private Record read(BufferedReader reader) throws Throwable { Record record = null; String data = reader.readLine(); if( data != null ) { record = new Record(data.split("\t")); } return record; } private Record readRecords(List<Record> list, Record record, BufferedReader reader) throws Throwable { int key = record.getKey(); list.add(record); record = read(reader); while( record != null && record.getKey() == key) { list.add(record); record = read(reader); } return record; } private void write(BufferedWriter writer, Record left, Record right) throws Throwable { String leftKey = (left == null ? "null" : Integer.toString(left.getKey())); String leftData = (left == null ? "null" : left.getData()); String rightKey = (right == null ? "null" : Integer.toString(right.getKey())); String rightData = (right == null ? "null" : right.getData()); writer.write("[" + leftKey + "][" + leftData + "][" + rightKey + "][" + rightData + "]\n"); } public static void main(String[] args) { try { BufferedReader leftReader = new BufferedReader(new FileReader("LEFT.DAT")); BufferedReader rightReader = new BufferedReader(new FileReader("RIGHT.DAT")); BufferedWriter output = new BufferedWriter(new FileWriter("OUTPUT.DAT")); Joiner joiner = new Joiner(); joiner.join(leftReader, rightReader, output); } catch (Throwable e) { e.printStackTrace(); } } } After applying the ideas from the proposed answer, I changed the loop to this private void join(RandomAccessFile left, RandomAccessFile right, BufferedWriter output) throws Throwable { long _pointer = 0; RandomAccessFile _left = left; RandomAccessFile _right = right; BufferedWriter _output = output; Record _leftRecord; Record _rightRecord; _leftRecord = read(_left); _rightRecord = read(_right); while( _leftRecord != null && _rightRecord != null ) { if( _leftRecord.getKey() < _rightRecord.getKey() ) { write(_output, _leftRecord, null); _leftRecord = read(_left); } else if( _leftRecord.getKey() > _rightRecord.getKey() ) { write(_output, null, _rightRecord); _pointer = _right.getFilePointer(); _rightRecord = read(_right); } else { long _tempPointer = 0; int key = _leftRecord.getKey(); while( _leftRecord != null && _leftRecord.getKey() == key ) { _right.seek(_pointer); _rightRecord = read(_right); while( _rightRecord != null && _rightRecord.getKey() == key ) { write(_output, _leftRecord, _rightRecord ); _tempPointer = _right.getFilePointer(); _rightRecord = read(_right); } _leftRecord = read(_left); } _pointer = _tempPointer; } } if( _leftRecord != null ) { write(_output, _leftRecord, null); _leftRecord = read(_left); while(_leftRecord != null) { write(_output, _leftRecord, null); _leftRecord = read(_left); } } else { if( _rightRecord != null ) { write(_output, null, _rightRecord); _rightRecord = read(_right); while(_rightRecord != null) { write(_output, null, _rightRecord); _rightRecord = read(_right); } } } _left.close(); _right.close(); _output.flush(); _output.close(); } UPDATE While this approach worked, it was terribly slow and so I have modified this to create files as buffers and this works very well. Here is the update ... private long getMaxBufferedLines(File file) throws Throwable { long freeBytes = Runtime.getRuntime().freeMemory() / 2; return (freeBytes / (file.length() / getLineCount(file))); } private void join(File left, File right, File output, JoinType joinType) throws Throwable { BufferedReader leftFile = new BufferedReader(new FileReader(left)); BufferedReader rightFile = new BufferedReader(new FileReader(right)); BufferedWriter outputFile = new BufferedWriter(new FileWriter(output)); long maxBufferedLines = getMaxBufferedLines(right); Record leftRecord; Record rightRecord; leftRecord = read(leftFile); rightRecord = read(rightFile); while( leftRecord != null && rightRecord != null ) { if( leftRecord.getKey().compareTo(rightRecord.getKey()) < 0) { if( joinType == JoinType.LeftOuterJoin || joinType == JoinType.LeftExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, leftRecord, null); } leftRecord = read(leftFile); } else if( leftRecord.getKey().compareTo(rightRecord.getKey()) > 0 ) { if( joinType == JoinType.RightOuterJoin || joinType == JoinType.RightExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, null, rightRecord); } rightRecord = read(rightFile); } else if( leftRecord.getKey().compareTo(rightRecord.getKey()) == 0 ) { String key = leftRecord.getKey(); List<File> rightRecordFileList = new ArrayList<File>(); List<Record> rightRecordList = new ArrayList<Record>(); rightRecordList.add(rightRecord); rightRecord = consume(key, rightFile, rightRecordList, rightRecordFileList, maxBufferedLines); while( leftRecord != null && leftRecord.getKey().compareTo(key) == 0 ) { processRightRecords(outputFile, leftRecord, rightRecordFileList, rightRecordList, joinType); leftRecord = read(leftFile); } // need a dispose for deleting files in list } else { throw new Exception("DATA IS NOT SORTED"); } } if( leftRecord != null ) { if( joinType == JoinType.LeftOuterJoin || joinType == JoinType.LeftExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, leftRecord, null); } leftRecord = read(leftFile); while(leftRecord != null) { if( joinType == JoinType.LeftOuterJoin || joinType == JoinType.LeftExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, leftRecord, null); } leftRecord = read(leftFile); } } else { if( rightRecord != null ) { if( joinType == JoinType.RightOuterJoin || joinType == JoinType.RightExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, null, rightRecord); } rightRecord = read(rightFile); while(rightRecord != null) { if( joinType == JoinType.RightOuterJoin || joinType == JoinType.RightExclusiveJoin || joinType == JoinType.FullExclusiveJoin || joinType == JoinType.FullOuterJoin ) { write(outputFile, null, rightRecord); } rightRecord = read(rightFile); } } } leftFile.close(); rightFile.close(); outputFile.flush(); outputFile.close(); } public void processRightRecords(BufferedWriter outputFile, Record leftRecord, List<File> rightFiles, List<Record> rightRecords, JoinType joinType) throws Throwable { for(File rightFile : rightFiles) { BufferedReader rightReader = new BufferedReader(new FileReader(rightFile)); Record rightRecord = read(rightReader); while(rightRecord != null){ if( joinType == JoinType.LeftOuterJoin || joinType == JoinType.RightOuterJoin || joinType == JoinType.FullOuterJoin || joinType == JoinType.InnerJoin ) { write(outputFile, leftRecord, rightRecord); } rightRecord = read(rightReader); } rightReader.close(); } for(Record rightRecord : rightRecords) { if( joinType == JoinType.LeftOuterJoin || joinType == JoinType.RightOuterJoin || joinType == JoinType.FullOuterJoin || joinType == JoinType.InnerJoin ) { write(outputFile, leftRecord, rightRecord); } } } /** * consume all records having key (either to a single list or multiple files) each file will * store a buffer full of data. The right record returned represents the outside flow (key is * already positioned to next one or null) so we can't use this record in below while loop or * within this block in general when comparing current key. The trick is to keep consuming * from a List. When it becomes empty, re-fill it from the next file until all files have * been consumed (and the last node in the list is read). The next outside iteration will be * ready to be processed (either it will be null or it points to the next biggest key * @throws Throwable * */ private Record consume(String key, BufferedReader reader, List<Record> records, List<File> files, long bufferMaxRecordLines ) throws Throwable { boolean processComplete = false; Record record = records.get(records.size() - 1); while(!processComplete){ long recordCount = records.size(); if( record.getKey().compareTo(key) == 0 ){ record = read(reader); while( record != null && record.getKey().compareTo(key) == 0 && recordCount < bufferMaxRecordLines ) { records.add(record); recordCount++; record = read(reader); } } processComplete = true; // if record is null, we are done if( record != null ) { // if the key has changed, we are done if( record.getKey().compareTo(key) == 0 ) { // Same key means we have exhausted the buffer. // Dump entire buffer into a file. The list of file // pointers will keep track of the files ... processComplete = false; dumpBufferToFile(records, files); records.clear(); records.add(record); } } } return record; } /** * Dump all records in List of Record objects to a file. Then, add that * file to List of File objects * * NEED TO PLACE A LIMIT ON NUMBER OF FILE POINTERS (check size of file list) * * @param records * @param files * @throws Throwable */ private void dumpBufferToFile(List<Record> records, List<File> files) throws Throwable { String prefix = "joiner_" + files.size() + 1; String suffix = ".dat"; File file = File.createTempFile(prefix, suffix, new File("cache")); BufferedWriter writer = new BufferedWriter(new FileWriter(file)); for( Record record : records ) { writer.write( record.dump() ); } files.add(file); writer.flush(); writer.close(); }

  Read the article

• RoboCopy Log File Analysis

  - by BobJim

  Is it possible to analyse the log text file outputted from RoboCopy and extract the lines which are defined as "New Dir" and "Extra Dir"? I would like the line from the log contain all the details returned regarding this "New Dir" or "Extra Dir" The reason for completing this task is to understand how two folder structures have change over time. One version has been kept internally at the parent company, the second has been used by a consultancy. For your information I am using Windows 7.

  Read the article

< Previous Page | 123 124 125 126 127 128 129 130 131 132 133 134  | Next Page >