concat - Page 18 - Developer IT

XSLT split text and preserve HTML tags

- by Lycaon

I have an xml that has a description node: <config> <desc>A <b>first</b> sentence here. The second sentence with some link <a href="myurl">The link</a>. The <u>third</u> one.</desc> </config> I am trying to split the sentences using dot as separator but keeping in the same time in the HTML output the eventual HTML tags. What I have so far is a template that splits the description but the HTML tags are lost in the output due to the normalize-space and substring-before functions. My current template is given below: <xsl:template name="output-tokens"> <xsl:param name="sourceText" />  <xsl:variable name="newlist" select="concat(normalize-space($sourceText), ' ')" />  <xsl:choose> <xsl:when test ="contains($newlist, '.')">  <xsl:variable name="first" select="substring-before($newlist, '.')" />  <xsl:variable name="remaining" select="substring-after($newlist, '.')" />  <xsl:if test="normalize-space($first)!='.' and normalize-space($first)!=''"> <p><xsl:value-of select="normalize-space($first)" />.</p> </xsl:if>  <xsl:if test="$remaining"> <xsl:call-template name="output-tokens"> <xsl:with-param name="sourceText" select="$remaining" /> </xsl:call-template> </xsl:if> </xsl:when>  <xsl:otherwise> <p>  <xsl:if test="normalize-space($sourceText)!=''"> <xsl:value-of select="normalize-space($sourceText)" />. </xsl:if> </p> </xsl:otherwise> </xsl:choose> </xsl:template> and I am using it in the following manner: <xsl:template match="config"> <xsl:call-template name="output-tokens"> <xsl:with-param name="sourceText" select="desc" /> </xsl:call-template> </xsl:template> The expected output is: <p>A <b>first</b> sentence here.</p> <p>The second sentence with some link <a href="myurl">The link</a>.</p> <p>The <u>third</u> one.</p>

Read the article

Strange behaviour when simply adding strings in Lazarus - FreePascal

- by linkcharger

The program has several "encryption" algorithms. This one should blockwise reverse the input. "He|ll|o " becomes "o |ll|He" (block length of 2). I add two strings, in this case appending the result string to the current "block" string and making that the result. When I add the result first and then the block it works fine and gives me back the original string. But when i try to reverse the order it just gives me the the last "block". Several other functions that are used for "rotation" are above. //amount of blocks function amBl(i1:integer;i2:integer):integer; begin if (i1 mod i2) <> 0 then result := (i1 div i2) else result := (i1 div i2) - 1; end; //calculation of block length function calcBl(keyStr:string):integer; var i:integer; begin result := 0; for i := 1 to Length(keyStr) do begin result := (result + ord(keyStr[i])) mod 5; result := result + 2; end; end; //desperate try to add strings function append(s1,s2:string):string; begin insert(s2,s1,Length(s1)+1); result := s1; end; function rotation(inStr,keyStr:string):string; var //array of chars -> string block,temp:string; //position in block variable posB:integer; //block length and block count variable bl, bc:integer; //null character as placeholder n : ansiChar; begin //calculating block length 2..6 bl := calcBl(keyStr); setLength(block,bl); result := ''; temp := ''; {n := #00;} for bc := 0 to amBl(Length(inStr),bl) do begin //filling block with chars starting from back of virtual block (in inStr) for posB := 1 to bl do begin block[posB] := inStr[bc * bl + posB]; {if inStr[bc * bl + posB] = ' ' then block[posB] := n;} end; //adding the block in front of the existing result string temp := result; result := block + temp; //result := append(block,temp); //result := concat(block,temp); end; end; (full code http://pastebin.com/6Uarerhk) After all the loops "result" has the right value, but in the last step (between "result := block + temp" and the "end;" of the function) "block" replaces the content of "result" with itself completely, it doesn't add result at the end anymore. And as you can see I even used a temp variable to try to work around that.. doesnt change anything though.

Read the article

C++ Simple thread with parameter (no .net)

- by Marc Vollmer

I've searched the internet for a while now and found different solutions but then all don't really work or are to complicated for my use. I used C++ until 2 years ago so it might be a bit rusty :D I'm currently writing a program that posts data to an URL. It only posts the data nothing else. For posting the data I use curl, but it blocks the main thread and while the first post is still running there will be a second post that should start. In the end there are about 5-6 post operations running at the same time. Now I want to push the posting with curl into another thread. One thread per post. The thread should get a string parameter with the content what to push. I'm currently stuck on this. Tried the WINAPI for windows but that crashes on reading the parameter. (the second thread is still running in my example while the main thread ended (waiting on system("pause")). It would be nice to have a multi plattform solution, because it will run under windows and linux! Heres my current code: #define CURL_STATICLIB #include <curl/curl.h> #include <curl/easy.h> #include <cstdlib> #include <iostream> #include <stdio.h> #include <stdlib.h> #include <string> #if defined(WIN32) #include <windows.h> #else //#include <pthread.h> #endif using namespace std; void post(string post) { // Function to post it to url CURL *curl; // curl object CURLcode res; // CURLcode object curl = curl_easy_init(); // init curl if(curl) { // is curl init curl_easy_setopt(curl, CURLOPT_URL, "http://10.8.27.101/api.aspx"); // set url string data = "api=" + post; // concat post data strings curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data.c_str()); // post data res = curl_easy_perform(curl); // execute curl_easy_cleanup(curl); // cleanup } else { cerr << "Failed to create curl handle!\n"; } } #if defined(WIN32) DWORD WINAPI thread(LPVOID data) { // WINAPI Thread string pData = *((string*)data); // convert LPVOID to string [THIS FAILES] post(pData); // post it with curl } #else // Linux version #endif void startThread(string data) { // FUnction to start the thread string pData = data; // some Test #if defined(WIN32) CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thread, &pData, 0, NULL); // Start a Windows thread with winapi #else // Linux version #endif } int main(int argc, char *argv[]) { // The post data to send string postData = "test1234567890"; startThread(postData); // Start the thread system("PAUSE"); // Dont close the console window return EXIT_SUCCESS; } Has anyone a suggestion? Thanks for the help!

Read the article

A ToDynamic() Extension Method For Fluent Reflection

- by Dixin

Recently I needed to demonstrate some code with reflection, but I felt it inconvenient and tedious. To simplify the reflection coding, I created a ToDynamic() extension method. The source code can be downloaded from here. Problem One example for complex reflection is in LINQ to SQL. The DataContext class has a property Privider, and this Provider has an Execute() method, which executes the query expression and returns the result. Assume this Execute() needs to be invoked to query SQL Server database, then the following code will be expected: using (NorthwindDataContext database = new NorthwindDataContext()) { // Constructs the query. IQueryable<Product> query = database.Products.Where(product => product.ProductID > 0) .OrderBy(product => product.ProductName) .Take(2); // Executes the query. Here reflection is required, // because Provider, Execute(), and ReturnValue are not public members. IEnumerable<Product> results = database.Provider.Execute(query.Expression).ReturnValue; // Processes the results. foreach (Product product in results) { Console.WriteLine("{0}, {1}", product.ProductID, product.ProductName); } } Of course, this code cannot compile. And, no one wants to write code like this. Again, this is just an example of complex reflection. using (NorthwindDataContext database = new NorthwindDataContext()) { // Constructs the query. IQueryable<Product> query = database.Products.Where(product => product.ProductID > 0) .OrderBy(product => product.ProductName) .Take(2); // database.Provider PropertyInfo providerProperty = database.GetType().GetProperty( "Provider", BindingFlags.NonPublic | BindingFlags.GetProperty | BindingFlags.Instance); object provider = providerProperty.GetValue(database, null); // database.Provider.Execute(query.Expression) // Here GetMethod() cannot be directly used, // because Execute() is a explicitly implemented interface method. Assembly assembly = Assembly.Load("System.Data.Linq"); Type providerType = assembly.GetTypes().SingleOrDefault( type => type.FullName == "System.Data.Linq.Provider.IProvider"); InterfaceMapping mapping = provider.GetType().GetInterfaceMap(providerType); MethodInfo executeMethod = mapping.InterfaceMethods.Single(method => method.Name == "Execute"); IExecuteResult executeResult = executeMethod.Invoke(provider, new object[] { query.Expression }) as IExecuteResult; // database.Provider.Execute(query.Expression).ReturnValue IEnumerable<Product> results = executeResult.ReturnValue as IEnumerable<Product>; // Processes the results. foreach (Product product in results) { Console.WriteLine("{0}, {1}", product.ProductID, product.ProductName); } } This may be not straight forward enough. So here a solution will implement fluent reflection with a ToDynamic() extension method: IEnumerable<Product> results = database.ToDynamic() // Starts fluent reflection. .Provider.Execute(query.Expression).ReturnValue; C# 4.0 dynamic In this kind of scenarios, it is easy to have dynamic in mind, which enables developer to write whatever code after a dot: using (NorthwindDataContext database = new NorthwindDataContext()) { // Constructs the query. IQueryable<Product> query = database.Products.Where(product => product.ProductID > 0) .OrderBy(product => product.ProductName) .Take(2); // database.Provider dynamic dynamicDatabase = database; dynamic results = dynamicDatabase.Provider.Execute(query).ReturnValue; } This throws a RuntimeBinderException at runtime: 'System.Data.Linq.DataContext.Provider' is inaccessible due to its protection level. Here dynamic is able find the specified member. So the next thing is just writing some custom code to access the found member. .NET 4.0 DynamicObject, and DynamicWrapper<T> Where to put the custom code for dynamic? The answer is DynamicObject’s derived class. I first heard of DynamicObject from Anders Hejlsberg's video in PDC2008. It is very powerful, providing useful virtual methods to be overridden, like: TryGetMember() TrySetMember() TryInvokeMember() etc. (In 2008 they are called GetMember, SetMember, etc., with different signature.) For example, if dynamicDatabase is a DynamicObject, then the following code: dynamicDatabase.Provider will invoke dynamicDatabase.TryGetMember() to do the actual work, where custom code can be put into. Now create a type to inherit DynamicObject: public class DynamicWrapper<T> : DynamicObject { private readonly bool _isValueType; private readonly Type _type; private T _value; // Not readonly, for value type scenarios. public DynamicWrapper(ref T value) // Uses ref in case of value type. { if (value == null) { throw new ArgumentNullException("value"); } this._value = value; this._type = value.GetType(); this._isValueType = this._type.IsValueType; } public override bool TryGetMember(GetMemberBinder binder, out object result) { // Searches in current type's public and non-public properties. PropertyInfo property = this._type.GetTypeProperty(binder.Name); if (property != null) { result = property.GetValue(this._value, null).ToDynamic(); return true; } // Searches in explicitly implemented properties for interface. MethodInfo method = this._type.GetInterfaceMethod(string.Concat("get_", binder.Name), null); if (method != null) { result = method.Invoke(this._value, null).ToDynamic(); return true; } // Searches in current type's public and non-public fields. FieldInfo field = this._type.GetTypeField(binder.Name); if (field != null) { result = field.GetValue(this._value).ToDynamic(); return true; } // Searches in base type's public and non-public properties. property = this._type.GetBaseProperty(binder.Name); if (property != null) { result = property.GetValue(this._value, null).ToDynamic(); return true; } // Searches in base type's public and non-public fields. field = this._type.GetBaseField(binder.Name); if (field != null) { result = field.GetValue(this._value).ToDynamic(); return true; } // The specified member is not found. result = null; return false; } // Other overridden methods are not listed. } In the above code, GetTypeProperty(), GetInterfaceMethod(), GetTypeField(), GetBaseProperty(), and GetBaseField() are extension methods for Type class. For example: internal static class TypeExtensions { internal static FieldInfo GetBaseField(this Type type, string name) { Type @base = type.BaseType; if (@base == null) { return null; } return @base.GetTypeField(name) ?? @base.GetBaseField(name); } internal static PropertyInfo GetBaseProperty(this Type type, string name) { Type @base = type.BaseType; if (@base == null) { return null; } return @base.GetTypeProperty(name) ?? @base.GetBaseProperty(name); } internal static MethodInfo GetInterfaceMethod(this Type type, string name, params object[] args) { return type.GetInterfaces().Select(type.GetInterfaceMap).SelectMany(mapping => mapping.TargetMethods) .FirstOrDefault( method => method.Name.Split('.').Last().Equals(name, StringComparison.Ordinal) && method.GetParameters().Count() == args.Length && method.GetParameters().Select( (parameter, index) => parameter.ParameterType.IsAssignableFrom(args[index].GetType())).Aggregate( true, (a, b) => a && b)); } internal static FieldInfo GetTypeField(this Type type, string name) { return type.GetFields( BindingFlags.GetField | BindingFlags.Instance | BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic).FirstOrDefault( field => field.Name.Equals(name, StringComparison.Ordinal)); } internal static PropertyInfo GetTypeProperty(this Type type, string name) { return type.GetProperties( BindingFlags.GetProperty | BindingFlags.Instance | BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic).FirstOrDefault( property => property.Name.Equals(name, StringComparison.Ordinal)); } // Other extension methods are not listed. } So now, when invoked, TryGetMember() searches the specified member and invoke it. The code can be written like this: dynamic dynamicDatabase = new DynamicWrapper<NorthwindDataContext>(ref database); dynamic dynamicReturnValue = dynamicDatabase.Provider.Execute(query.Expression).ReturnValue; This greatly simplified reflection. ToDynamic() and fluent reflection To make it even more straight forward, A ToDynamic() method is provided: public static class DynamicWrapperExtensions { public static dynamic ToDynamic<T>(this T value) { return new DynamicWrapper<T>(ref value); } } and a ToStatic() method is provided to unwrap the value: public class DynamicWrapper<T> : DynamicObject { public T ToStatic() { return this._value; } } In the above TryGetMember() method, please notice it does not output the member’s value, but output a wrapped member value (that is, memberValue.ToDynamic()). This is very important to make the reflection fluent. Now the code becomes: IEnumerable<Product> results = database.ToDynamic() // Here starts fluent reflection. .Provider.Execute(query.Expression).ReturnValue .ToStatic(); // Unwraps to get the static value. With the help of TryConvert(): public class DynamicWrapper<T> : DynamicObject { public override bool TryConvert(ConvertBinder binder, out object result) { result = this._value; return true; } } ToStatic() can be omitted: IEnumerable<Product> results = database.ToDynamic() .Provider.Execute(query.Expression).ReturnValue; // Automatically converts to expected static value. Take a look at the reflection code at the beginning of this post again. Now it is much much simplified! Special scenarios In 90% of the scenarios ToDynamic() is enough. But there are some special scenarios. Access static members Using extension method ToDynamic() for accessing static members does not make sense. Instead, DynamicWrapper<T> has a parameterless constructor to handle these scenarios: public class DynamicWrapper<T> : DynamicObject { public DynamicWrapper() // For static. { this._type = typeof(T); this._isValueType = this._type.IsValueType; } } The reflection code should be like this: dynamic wrapper = new DynamicWrapper<StaticClass>(); int value = wrapper._value; int result = wrapper.PrivateMethod(); So accessing static member is also simple, and fluent of course. Change instances of value types Value type is much more complex. The main problem is, value type is copied when passing to a method as a parameter. This is why ref keyword is used for the constructor. That is, if a value type instance is passed to DynamicWrapper<T>, the instance itself will be stored in this._value of DynamicWrapper<T>. Without the ref keyword, when this._value is changed, the value type instance itself does not change. Consider FieldInfo.SetValue(). In the value type scenarios, invoking FieldInfo.SetValue(this._value, value) does not change this._value, because it changes the copy of this._value. I searched the Web and found a solution for setting the value of field: internal static class FieldInfoExtensions { internal static void SetValue<T>(this FieldInfo field, ref T obj, object value) { if (typeof(T).IsValueType) { field.SetValueDirect(__makeref(obj), value); // For value type. } else { field.SetValue(obj, value); // For reference type. } } } Here __makeref is a undocumented keyword of C#. But method invocation has problem. This is the source code of TryInvokeMember(): public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) { if (binder == null) { throw new ArgumentNullException("binder"); } MethodInfo method = this._type.GetTypeMethod(binder.Name, args) ?? this._type.GetInterfaceMethod(binder.Name, args) ?? this._type.GetBaseMethod(binder.Name, args); if (method != null) { // Oops! // If the returnValue is a struct, it is copied to heap. object resultValue = method.Invoke(this._value, args); // And result is a wrapper of that copied struct. result = new DynamicWrapper<object>(ref resultValue); return true; } result = null; return false; } If the returned value is of value type, it will definitely copied, because MethodInfo.Invoke() does return object. If changing the value of the result, the copied struct is changed instead of the original struct. And so is the property and index accessing. They are both actually method invocation. For less confusion, setting property and index are not allowed on struct. Conclusions The DynamicWrapper<T> provides a simplified solution for reflection programming. It works for normal classes (reference types), accessing both instance and static members. In most of the scenarios, just remember to invoke ToDynamic() method, and access whatever you want: StaticType result = someValue.ToDynamic()._field.Method().Property[index]; In some special scenarios which requires changing the value of a struct (value type), this DynamicWrapper<T> does not work perfectly. Only changing struct’s field value is supported. The source code can be downloaded from here, including a few unit test code.

Read the article

Threading extra state through a parser in Scala

- by Travis Brown

I'll give you the tl;dr up front I'm trying to use the state monad transformer in Scalaz 7 to thread extra state through a parser, and I'm having trouble doing anything useful without writing a lot of t m a -> t m b versions of m a -> m b methods. An example parsing problem Suppose I have a string containing nested parentheses with digits inside them: val input = "((617)((0)(32)))" I also have a stream of fresh variable names (characters, in this case): val names = Stream('a' to 'z': _*) I want to pull a name off the top of the stream and assign it to each parenthetical expression as I parse it, and then map that name to a string representing the contents of the parentheses, with the nested parenthetical expressions (if any) replaced by their names. To make this more concrete, here's what I'd want the output to look like for the example input above: val target = Map( 'a' -> "617", 'b' -> "0", 'c' -> "32", 'd' -> "bc", 'e' -> "ad" ) There may be either a string of digits or arbitrarily many sub-expressions at a given level, but these two kinds of content won't be mixed in a single parenthetical expression. To keep things simple, we'll assume that the stream of names will never contain either duplicates or digits, and that it will always contain enough names for our input. Using parser combinators with a bit of mutable state The example above is a slightly simplified version of the parsing problem in this Stack Overflow question. I answered that question with a solution that looked roughly like this: import scala.util.parsing.combinator._ class ParenParser(names: Iterator[Char]) extends RegexParsers { def paren: Parser[List[(Char, String)]] = "(" ~> contents <~ ")" ^^ { case (s, m) => (names.next -> s) :: m } def contents: Parser[(String, List[(Char, String)])] = "\\d+".r ^^ (_ -> Nil) | rep1(paren) ^^ ( ps => ps.map(_.head._1).mkString -> ps.flatten ) def parse(s: String) = parseAll(paren, s).map(_.toMap) } It's not too bad, but I'd prefer to avoid the mutable state. What I want Haskell's Parsec library makes adding user state to a parser trivially easy: import Control.Applicative ((*>), (<$>), (<*)) import Data.Map (fromList) import Text.Parsec paren = do (s, m) <- char '(' *> contents <* char ')' h : t <- getState putState t return $ (h, s) : m where contents = flip (,) [] <$> many1 digit <|> (\ps -> (map (fst . head) ps, concat ps)) <$> many1 paren main = print $ runParser (fromList <$> paren) ['a'..'z'] "example" "((617)((0)(32)))" This is a fairly straightforward translation of my Scala parser above, but without mutable state. What I've tried I'm trying to get as close to the Parsec solution as I can using Scalaz's state monad transformer, so instead of Parser[A] I'm working with StateT[Parser, Stream[Char], A]. I have a "solution" that allows me to write the following: import scala.util.parsing.combinator._ import scalaz._, Scalaz._ object ParenParser extends ExtraStateParsers[Stream[Char]] with RegexParsers { protected implicit def monadInstance = parserMonad(this) def paren: ESP[List[(Char, String)]] = (lift("(" ) ~> contents <~ lift(")")).flatMap { case (s, m) => get.flatMap( names => put(names.tail).map(_ => (names.head -> s) :: m) ) } def contents: ESP[(String, List[(Char, String)])] = lift("\\d+".r ^^ (_ -> Nil)) | rep1(paren).map( ps => ps.map(_.head._1).mkString -> ps.flatten ) def parse(s: String, names: Stream[Char]) = parseAll(paren.eval(names), s).map(_.toMap) } This works, and it's not that much less concise than either the mutable state version or the Parsec version. But my ExtraStateParsers is ugly as sin—I don't want to try your patience more than I already have, so I won't include it here (although here's a link, if you really want it). I've had to write new versions of every Parser and Parsers method I use above for my ExtraStateParsers and ESP types (rep1, ~>, <~, and |, in case you're counting). If I had needed to use other combinators, I'd have had to write new state transformer-level versions of them as well. Is there a cleaner way to do this? I'd love to see an example of a Scalaz 7's state monad transformer being used to thread state through a parser, but Scala 6 or Haskell examples would also be useful.

Read the article

How to get javascript object references or reference count?

- by Tauren

How to get reference count for an object Is it possible to determine if a javascript object has multiple references to it? Or if it has references besides the one I'm accessing it with? Or even just to get the reference count itself? Can I find this information from javascript itself, or will I need to keep track of my own reference counters. Obviously, there must be at least one reference to it for my code access the object. But what I want to know is if there are any other references to it, or if my code is the only place it is accessed. I'd like to be able to delete the object if nothing else is referencing it. If you know the answer, there is no need to read the rest of this question. Below is just an example to make things more clear. Use Case In my application, I have a Repository object instance called contacts that contains an array of ALL my contacts. There are also multiple Collection object instances, such as friends collection and a coworkers collection. Each collection contains an array with a different set of items from the contacts Repository. Sample Code To make this concept more concrete, consider the code below. Each instance of the Repository object contains a list of all items of a particular type. You might have a repository of Contacts and a separate repository of Events. To keep it simple, you can just get, add, and remove items, and add many via the constructor. var Repository = function(items) { this.items = items || []; } Repository.prototype.get = function(id) { for (var i=0,len=this.items.length; i<len; i++) { if (items[i].id === id) { return this.items[i]; } } } Repository.prototype.add = function(item) { if (toString.call(item) === "[object Array]") { this.items.concat(item); } else { this.items.push(item); } } Repository.prototype.remove = function(id) { for (var i=0,len=this.items.length; i<len; i++) { if (items[i].id === id) { this.removeIndex(i); } } } Repository.prototype.removeIndex = function(index) { if (items[index]) { if (/* items[i] has more than 1 reference to it */) { // Only remove item from repository if nothing else references it this.items.splice(index,1); return; } } } Note the line in remove with the comment. I only want to remove the item from my master repository of objects if no other objects have a reference to the item. Here's Collection: var Collection = function(repo,items) { this.repo = repo; this.items = items || []; } Collection.prototype.remove = function(id) { for (var i=0,len=this.items.length; i<len; i++) { if (items[i].id === id) { // Remove object from this collection this.items.splice(i,1); // Tell repo to remove it (only if no other references to it) repo.removeIndxe(i); return; } } } And then this code uses Repository and Collection: var contactRepo = new Repository([ {id: 1, name: "Joe"}, {id: 2, name: "Jane"}, {id: 3, name: "Tom"}, {id: 4, name: "Jack"}, {id: 5, name: "Sue"} ]); var friends = new Collection( contactRepo, [ contactRepo.get(2), contactRepo.get(4) ] ); var coworkers = new Collection( contactRepo, [ contactRepo.get(1), contactRepo.get(2), contactRepo.get(5) ] ); contactRepo.items; // contains item ids 1, 2, 3, 4, 5 friends.items; // contains item ids 2, 4 coworkers.items; // contains item ids 1, 2, 5 coworkers.remove(2); contactRepo.items; // contains item ids 1, 2, 3, 4, 5 friends.items; // contains item ids 2, 4 coworkers.items; // contains item ids 1, 5 friends.remove(4); contactRepo.items; // contains item ids 1, 2, 3, 5 friends.items; // contains item ids 2 coworkers.items; // contains item ids 1, 5 Notice how coworkers.remove(2) didn't remove id 2 from contactRepo? This is because it was still referenced from friends.items. However, friends.remove(4) causes id 4 to be removed from contactRepo, because no other collection is referring to it. Summary The above is what I want to do. I'm sure there are ways I can do this by keeping track of my own reference counters and such. But if there is a way to do it using javascript's built-in reference management, I'd like to hear about how to use it.

Read the article

Factorising program not working. Help required.

- by Ender

I am working on a factorisation problem using Fermat's Factorization and for small numbers it is working well. I've been able to calculate the factors (getting the answers from Wolfram Alpha) for small numbers, like the one on the Wikipedia page (5959). Just when I thought I had the problem licked I soon realised that my program was not working when it came to larger numbers. The program follows through the examples from the Wikipedia page, printing out the values a, b, a2 and b2; the results printed for large numbers are not correct. I've followed the pseudocode provided on the Wikipedia page, but am struggling to understand where to go next. Along with the Wikipedia page I have been following this guide. Once again, as my Math knowledge is pretty poor I cannot follow what I need to do next. The code I am using so far is as follows: import java.math.BigInteger; /** * * @author AlexT */ public class Fermat { private BigInteger a, b; private BigInteger b2; private static final BigInteger TWO = BigInteger.valueOf(2); public void fermat(BigInteger N) { // floor(sqrt(N)) BigInteger tmp = getIntSqrt(N); // a <- ceil(sqrt(N)) a = tmp.add(BigInteger.ONE); // b2 <- a*a-N b2 = (a.multiply(a)).subtract(N); final int bitLength = N.bitLength(); BigInteger root = BigInteger.ONE.shiftLeft(bitLength / 2); root = root.add(b2.divide(root)).divide(TWO); // while b2 not square root while(!(isSqrt(b2, root))) { // a <- a + 1 a = a.add(BigInteger.ONE); // b2 <- (a * a) - N b2 = (a.multiply(a)).subtract(N); root = root.add(b2.divide(root)).divide(TWO); } b = getIntSqrt(b2); BigInteger a2 = a.pow(2); // Wrong BigInteger sum = (a.subtract(b)).multiply((a.add(b))); //if(sum.compareTo(N) == 0) { System.out.println("A: " + a + "\nB: " + b); System.out.println("A^2: " + a2 + "\nB^2: " + b2); //} } /** * Is the number provided a perfect Square Root? * @param n * @param root * @return */ private static boolean isSqrt(BigInteger n, BigInteger root) { final BigInteger lowerBound = root.pow(2); final BigInteger upperBound = root.add(BigInteger.ONE).pow(2); return lowerBound.compareTo(n) <= 0 && n.compareTo(upperBound) < 0; } public BigInteger getIntSqrt(BigInteger x) { // It returns s where s^2 < x < (s+1)^2 BigInteger s; // final result BigInteger currentRes = BigInteger.valueOf(0); // init value is 0 BigInteger currentSum = BigInteger.valueOf(0); // init value is 0 BigInteger sum = BigInteger.valueOf(0); String xS = x.toString(); // change input x to a string xS int lengthOfxS = xS.length(); int currentTwoBits; int i=0; // index if(lengthOfxS % 2 != 0) {// if odd length, add a dummy bit xS = "0".concat(xS); // add 0 to the front of string xS lengthOfxS++; } while(i < lengthOfxS){ // go through xS two by two, left to right currentTwoBits = Integer.valueOf(xS.substring(i,i+2)); i += 2; // sum = currentSum*100 + currentTwoBits sum = currentSum.multiply(BigInteger.valueOf(100)); sum = sum.add(BigInteger.valueOf(currentTwoBits)); // subtraction loop do { currentSum = sum; // remember the value before subtract // in next 3 lines, we work out // currentRes = sum - 2*currentRes - 1 sum = sum.subtract(currentRes); // currentRes++ currentRes = currentRes.add(BigInteger.valueOf(1)); sum = sum.subtract(currentRes); } while(sum.compareTo(BigInteger.valueOf(0)) >= 0); // the loop stops when sum < 0 // go one step back currentRes = currentRes.subtract(BigInteger.valueOf(1)); currentRes = currentRes.multiply(BigInteger.valueOf(10)); } s = currentRes.divide(BigInteger.valueOf(10)); // go one step back return s; } /** * @param args the command line arguments */ public static void main(String[] args) { Fermat fermat = new Fermat(); //Works //fermat.fermat(new BigInteger("5959")); // Doesn't Work fermat.fermat(new BigInteger("90283")); } } If anyone can help me out with this problem I'll be eternally grateful.

Read the article

Zend_Form validation problem

- by GrumpyCanuck

I am having problems getting validation to work for a form built using Zend_Form. The idea is this: I have two dropdown. One is a list of players. The other is a list of free agents who play the same position as the player. I am using an onChange javascript callback to run some Ajax code that replaces the free agent list dropdown with a new one at the position of the player they've selected from the player dropdown. Now, perhaps this is the wrong way, but I built the form by creating an instance of Zend_Form and then creating all these setX methods that add elements to the form. My reasoning was that I wanted to display certain elements in specific places on the page, not just output $this-form on my template. The problem appears to be when I get the form post back, the validator seems to not know about the validation rule I set up for the free agent drop down. Here's some relevant code to look at. I'm a relative ZF n00b so feel free to tell me I am not doing things the ZF way if it leaps out at you. The action in the controller: public function indexAction() { if ($this->getRequest()->isPost()) { $form = new Baseball_Form_Transactions(); if ($form->isValid($this->_request->getPost())) { $data = $this->_request->getPost(); $leagueInfo = Doctrine::getTable('League')->findOneByShortName($data['shortLeagueName'])->toArray(); // Create the request top drop an existing player $transactionInfo = array( 'league_id' => $leagueInfo['id'], 'team_id' => $data['teamId'], 'player_id' => $data['players'], 'type' => 'drop', 'target_team_id' => 0, 'transaction_date' => date('Y-m-d H:m:s') ); $transaction = new Transaction(); $transaction->fromArray($transactionInfo); $transaction->save(); // Now we do the request to add a player $transactionInfo['team_id'] = 0; $transactionInfo['player_id'] = $data['freeAgents']; $transactionInfo['target_team_id'] = $data['teamId']; $transactionInfo['type'] = 'add'; $transaction = new Transaction(); $transaction->fromArray($transactionInfo); $transaction->save(); $this->_flashMessenger->addMessage('Added transaction'); } } $options = array( 'teamId' => $this->teamId, 'position' => 'C', 'leagueShortName' => $this->league ); $this->transactionForm->setMyPlayers($options); $this->transactionForm->setFreeAgents($options); $this->transactionForm->setTeamId($options); $this->transactionForm->setShortLeagueName($options); $this->view->transactionForm = $this->transactionForm; $this->view->messages = $this->_flashMessenger->getMessages(); $transaction = new Transaction(); $this->view->transactions = $transaction->byTeam($options); } Next we have the form itself public function setMyPlayers($options) { $data = Doctrine::getTable('Team')->find($options['teamId']); $players = array(); foreach ($data->Players->toArray() as $player) { $players[$player['id']] = "{$player['position']} - {$player['first_name']} {$player['last_name']}"; } $playersSelect = new Zend_Form_Element_Select( 'players', array( 'required' => true, 'label' => 'Players', 'multiOptions' => $players, ) ); $this->addElement($playersSelect); } public function setFreeAgents($options) { $q = Doctrine_Query::create() ->select('CONCAT(p.first_name, " ", p.last_name) as full_name, p.id, p.position') ->from('Player p') ->leftJoin('p.Teams t') ->leftJoin('t.League l ON l.short_name = ?', $options['leagueShortName']) ->where('t.id IS NULL') ->andWhere('p.position = ?', $options['position']) ->orderBy('p.last_name'); $q->setHydrationMode(Doctrine_Core::HYDRATE_ARRAY); $data = $q->execute(); $freeAgents = array(); foreach ($data as $player) { $freeAgents[$player['id']] = $player['full_name']; } $freeAgentsSelect = new Zend_Form_Element_Select( 'freeAgents', array( 'label' => 'Free Agents', 'multiOptions' => $freeAgents, 'size' => 15 ) ); $freeAgentsSelect->setRequired(true); $this->addElement($freeAgentsSelect); } public function setShortLeagueName($options) { $shortLeagueNameHidden = new Zend_Form_Element_Hidden( 'shortLeagueName', array('value' => $options['leagueShortName']) ); $this->addElement($shortLeagueNameHidden); } public function setTeamId($options) { $teamIdHidden = new Zend_Form_Element_Hidden( 'teamId', array('value' => $options['teamId']) ); $this->addElement($teamIdHidden); } There is no init or __construct() method in the form. My problem seems simple enough: reject the form contents as invalid if they have not selected someone from the free agent list. Right now, it sails through as valid. I've spent some considerable time searching online for an answer, and haven't been able to find it. Thanks in advance for any help.

Read the article

Java Array Index Out of Bounds Exception

- by user1302023

I need help debugging the following program: I'm getting a run time error that reads: Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1 at SearchEngine.main(SearchEngine.java:126) import java.util.*; import java.io.*; public class SearchEngine { public static int getNumberOfWords (File f) throws FileNotFoundException { int numWords = 0; Scanner scan = new Scanner(f); while (scan.hasNext()) { numWords++; scan.next(); } scan.close(); return numWords; } public static void readInWords (File input, String [] x) throws FileNotFoundException { Scanner scan = new Scanner(input); int i = 0; while (scan.hasNext() && i<x.length) { x[i] = scan.next(); i++; } scan.close(); } public static int getNumOfDistinctWords (File input, String [] x) throws FileNotFoundException { Scanner scan = new Scanner(input); int count = 0; int i = 1; while (scan.hasNext() && i<x.length) { if (!x[i].equals(x[i-1])) { count++; } i++; } scan.close(); return count; } public static void readInDistinctWords (String [] x, String [] y) { int i = 1; int k = 0; while (i<x.length) { if (!x[i].equals(x[i-1])) { y[k] = x[i]; k++; } i++; } } public static int getNumberOfLines (File input) throws FileNotFoundException { int numLines = 0; Scanner scan = new Scanner(input); while (scan.hasNextLine()) { numLines++; scan.nextLine(); } scan.close(); return numLines; } public static void readInLines (File input, String [] x) throws FileNotFoundException { Scanner scan = new Scanner(input); int i = 0; while (scan.hasNextLine() && i<x.length) { x[i] = scan.nextLine(); i++; } scan.close(); } public static void main(String [] args) { try { //gets file name System.out.println("Enter the name of the text file you wish to search"); Scanner kb = new Scanner(System.in); String fileName = kb.nextLine(); String TXT = ".txt"; if (!fileName.endsWith(TXT)) { fileName = fileName.concat(TXT); } File input = new File(fileName); //First part of creating index System.out.println("Creating vocabArray"); int NUM_WORDS = getNumberOfWords(input); //System.out.println(NUM_WORDS); String [] wordArray = new String[NUM_WORDS]; readInWords(input, wordArray); Arrays.sort(wordArray); int NUM_DISTINCT_WORDS = getNumOfDistinctWords(input, wordArray); String [] vocabArray = new String[NUM_DISTINCT_WORDS]; readInDistinctWords(wordArray, vocabArray); System.out.println("Finished creating vocabArray"); System.out.println("Creating concordanceArray"); int NUM_LINES = getNumberOfLines(input); String [] concordanceArray = new String[NUM_LINES]; readInLines(input, concordanceArray); System.out.println("Finished creating concordanceArray"); System.out.println("Creating invertedIndex"); int [][] invertedIndex = new int[NUM_DISTINCT_WORDS][10]; int [] wordCountArray = new int[NUM_DISTINCT_WORDS]; int lineNum = 0; while (lineNum<concordanceArray.length) { Scanner scan = new Scanner(concordanceArray[lineNum]); while (scan.hasNext()) { int wordPos = Arrays.binarySearch(vocabArray, scan.next()); wordCountArray[wordPos]+=1; for(int i = 0; i < invertedIndex.length; i++) { for(int j = 0; j < invertedIndex[i].length; j++) { if (invertedIndex[i][j] == 0) { invertedIndex[i][j] = lineNum; break; } } } } lineNum++; } System.out.println("Finished creating invertedIndex"); } catch (FileNotFoundException exception) { System.out.println("File Not Found"); } } //main } //class

Read the article

Symfony2 same form, different entities NOT related

- by user1381537

I'm trying to write one form for submitting against MySQL DB, but I can't get it working, I've tried a lot of things (separate forms, create an ->add('foo', new foo()) to a field, and trying to parse plain SQL with a normal HTML form is my only solution, which is obviously not the best. This is my DB structure: As you can see I need to insert the comments textarea to ticketcomments among the user who wrote it, etc. On crmentity the description field. Then on ticketcf the fields that I need to submit from form, are this (because you wont know if I don't tell you because of the field names): tcf.cf594 AS Type, tcf.cf675 AS Suscription, tcf.cf770 AS ID_PRODUCT, tcf.cf746 AS NotificationDate, tcf.cf747 AS ResponseDate, tcf.cf748 AS ResolutionDate, And, of course, every table needs to have the same ticketid id for the submitted form, so we can retrieve it with one simple query. It will be easy to do with plain SQL instead of using DQL and Symfony2 forms, but is not a good way to do it. Also, here's my "Ticket list" query, if you need it to have it more clear... SELECT t.ticketNo AS Ticket, t.title AS Asunto, t.status AS Estado, t.updateLog AS LOG, t.hours AS Horas, t.solution AS Solucion, t.priority AS Prioridad, tcf.cf594 AS Tipo, tcf.cf675 AS Suscripcion, tcf.cf770 AS IDPROD, tcf.cf746 AS F_Noti, tcf.cf747 AS F_Resp, tcf.cf748 AS F_Reso, CONCAT (cd.firstname, cd.lastname) AS Contacto, crm.description AS Descripcion, crm.crmid AS id FROM WbsGoclientsBundle:VtigerTroubletickets t INNER JOIN WbsGoclientsBundle:VtigerTicketcf tcf WITH t.ticketid = tcf.ticketid INNER JOIN WbsGoclientsBundle:VtigerContactdetails cd WITH t.parentId = cd.contactid INNER JOIN WbsGoclientsBundle:VtigerCrmentity crm WITH t.ticketid = crm.crmid WHERE t.parentId IN ( SELECT cd1.contactid FROM WbsGoclientsBundle:VtigerContactdetails cd1 WHERE cd1.accountid = ( SELECT cd2.accountid FROM WbsGoclientsBundle:VtigerContactdetails cd2 WHERE cd2.contactid = :contactid)) AND t.status <> \'Closed\' And also "Ticket details" query (which is not in DQL format yet, only SQL) is so simple, it only retrieve the comments field and createdtime from ticketcomments appended to this query so we have all the fields... Thank you. This is a test form, using troubletickets and ticketcomments, it's returning errores because I can't set a comments field because troubletickets doesn't has it, but I need that field to be submitted to ticketcomments ... VtigerTicketcommentsType <?php namespace WbsGo\clientsBundle\Form\Type; use Symfony\Component\Form\AbstractType, Symfony\Component\Form\FormBuilderInterface; use Symfony\Component\OptionsResolver\OptionsResolverInterface; class VtigerTicketcommentsType extends AbstractType { public function buildForm(FormBuilderInterface $builder, array $options) { $builder ->add('ticketid') ->add('comments') ->add('ownerid') ->add('ownertype') ->add('createdtype') ; } public function setDefaultOptions(OptionsResolverInterface $resolver) { $resolver->setDefaults(array( 'data_class' => 'WbsGo\clientsBundle\Entity\VtigerTicketcomments' )); } public function getName() { return 'comments'; } } OpenTicketType.php <?php namespace WbsGo\clientsBundle\Form; use Symfony\Component\Form\AbstractType, Symfony\Component\Form\FormBuilderInterface ; use WbsGo\clientsBundle\Form\Type\VtigerTicketcommentsType; use Symfony\Component\OptionsResolver\OptionsResolverInterface; class OpenTicketType extends AbstractType { public function buildForm(FormBuilderInterface $builder, array $options) { $builder ->add('title') ->add('priority') ->add('solution') ->add('comments', 'collection', array( 'type' => new VtigerTicketcommentsType() )) ; } public function setDefaultOptions(OptionsResolverInterface $resolver) { $resolver->setDefaults(array( 'data_class' => 'WbsGo\clientsBundle\Entity\VtigerTroubletickets' )); } public function getName() { return 'ticket'; } } TicketController.php <?php namespace WbsGo\clientsBundle\Controller; use Symfony\Bundle\FrameworkBundle\Controller\Controller; use WbsGo\clientsBundle\Entity\VtigerTroubletickets; use WbsGo\clientsBundle\Entity\VtigerTicketcomments; use WbsGo\clientsBundle\Form\OpenTicketType; use Symfony\Component\HttpFoundation\Request; class TicketController extends Controller { public function indexAction() { $em = $this->getDoctrine()->getManager(); $tickets = $em ->getRepository('WbsGoclientsBundle:VtigerTroubletickets') ->findAllOpenByCustomerId($this->getUser()->getId()); $userdata = $this->getDoctrine()->getManager() ->getRepository('WbsGoclientsBundle:VtigerContactdetails') ->findContact($this->getUser()->getId()); return $this ->render('WbsGoclientsBundle:Ticket:index.html.twig', array('tickets' => $tickets, 'userdata' => $userdata)); } public function addAction() { $assets = $this->getDoctrine()->getManager() ->getRepository('WbsGoclientsBundle:VtigerAssets') ->findAssetByAccountId($this->getUser()->getId()); $assetlist = array(); foreach ($assets as $key => $v) { $assetlist[$key] = $key; } $form = $this->createForm(new OpenTicketType(), new VtigerTroubletickets()); return $this ->render('WbsGoclientsBundle:Ticket:add.html.twig', array('form' => $form->createView(), 'assets' => $assets,)); } } This is the error Symfony2 is returning Neither the property "comments" nor one of the methods "getComments()", "isComments()", "hasComments()", "_get()" or "_call()" exist and have public access in class "WbsGo\clientsBundle\Entity\VtigerTroubletickets". EDIT 2 This code is actually rendering my forms, but I need help in order to submit each XXXType form to its corresponding table. public function buildForm(FormBuilderInterface $builder, array $options) { $builder ->add('descripcion') ->add('prioridad') ->add('solucion') ->add('comment', new VtigerTicketcommentsType() ) ->add('contacto') ->add('suscripcion') ->add('producto', 'entity', array( 'class' => 'WbsGo\clientsBundle\Entity\VtigerAssets', 'property' => 'assetname', 'empty_value' => '--SELECT--', 'query_builder' => function(\WbsGo\clientsBundle\Entity\VtigerAssetsRepository $repository) { //return $repository->findAssetByAccountId($this->customerId); return $repository->createQueryBuilder('a') ->select('a') ->where('a.account = (SELECT cd.accountid FROM WbsGoclientsBundle:VtigerContactdetails cd WHERE cd.contactid = ?1)') ->setParameter(1, $this->customerId); } ) ) ->add('hardware') ->add('backup') ->add('web') ->add('restore') ->add('customerId') ; } I also removed ->add('ticketid') from VtigerTicketcommentsType.php because it has relationship and is not needed. it's auto_incremental and must be generated once everything is submitted.

Read the article

jquery .append() not working for my html

- by user1056998

I have a program which appends an input(type="hidden") using jquery, to an html so that when I click the submit button, it passes the value to a php file and I can process it. However, it seems that the hidden type is not really being appended to the html nor it is being passed to the php file. I already used method="get" to see the values in the address bar and print_r to see the values being catched but there's nothing. To check if my form is actually passing a value, I added a <input type="hidden" name="absent[]" value="testing" /> in the HTML and the value got passed but the ones in the jquery aren't. Here are my files: jquery: $(function(){ $("td").click(function(){ if($(this).hasClass("on")) { alert("Already marked absent"); } else { $(this).addClass("on"); var currentCellText = $(this).text(); var temp = $(this).attr('id'); $("#collect").append("<input type='hidden' name='absent[]' value = '" + temp + "'/>" + currentCellText); alert(temp); } }); $("#clicky").click(function(){ $("td").removeClass("on"); $("#collect").text(''); $("#collect").append("Absentees: <br>") alert(temp); }); }); Here is the html part: <?php session_start(); include 'connectdb.php'; $classID = $_SESSION['csID']; $classQry = "SELECT e.csID, c.subjCode, c.section, b.subj_name, e.studentID, CONCAT(s.lname, ', ' , s.fname)name FROM ENROLLMENT e, CLASS_SCHEDULE c, STUDENT s, SUBJECT b WHERE e.csID = c.csID AND c.csID = '" . $classID . "' AND c.subjCode = b.subjCode AND e.studentID = s.studentID ORDER BY e.sort;"; $doClassQry = mysql_query($classQry); echo "<table id='tableone'>"; while($x = mysql_fetch_array($doClassQry)) { $subject = $x['subj_name']; $subjCode = $x['subjCode']; $section = $x['section']; $studentArr[] = $x['name']; $studentID[] = $x['studentID']; } echo "<thead>"; echo "<tr><th colspan = 7>" . "This is your class: " . $subjCode . " " . $section . " : " . $subject . "</th></tr>"; echo "</thead>"; echo "<tbody>"; echo "<tr>"; for($i = 0; $i < mysql_num_rows($doClassQry); $i++) { if($i % 7 == 0) { echo "</tr><tr><td id = '". $studentID[$i] . " '>" . $studentArr[$i] . "</td>"; } else { echo "<td id = '". $studentID[$i] . " '>" . $studentArr[$i] . "</td>"; } } echo "</tr>"; echo "</tbody>"; echo "</table>"; ?> Here's the html part with the form: <form name="save" action="saveTest.php" method="post"> <div id="submitt"> <input type="hidden" name="absent[]" value="testing"/> <input type="submit" value="submit"/> </div> </form> And here's the php part which processes the form (saveTest.php): <?php $absent = $_POST['absent']; //echo "absnt" . $absent[] . "<br>"; echo count($absent) . "<br>"; //print_r($_POST) . "<br>"; ?>

Read the article

Need help in setting lighttpd on Ubuntu 9.10

- by hap497

Hi, I am trying to run lighttpd on Ubuntu 9.10. I get the conf file from the doc directory of lighttpd source. $ sudo ./lighttpd -f lighttpd.conf $ ps -ef | grep lighttpd root 2094 1 0 19:40 ? 00:00:00 ./lighttpd -f lighttpd.conf This is my lighttpd.conf: $ more lighttpd.conf # lighttpd configuration file # # use it as a base for lighttpd 1.0.0 and above # # $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $ ############ Options you really have to take care of #################### ## modules to load # at least mod_access and mod_accesslog should be loaded # all other module should only be loaded if really neccesary # - saves some time # - saves memory server.modules = ( # "mod_rewrite", # "mod_redirect", # "mod_alias", "mod_access", # "mod_trigger_b4_dl", # "mod_auth", # "mod_status", # "mod_setenv", # "mod_fastcgi", # "mod_proxy", # "mod_simple_vhost", # "mod_evhost", # "mod_userdir", # "mod_cgi", # "mod_compress", # "mod_ssi", # "mod_usertrack", # "mod_expire", # "mod_secdownload", # "mod_rrdtool", "mod_accesslog" ) ## A static document-root. For virtual hosting take a look at the ## mod_simple_vhost module. server.document-root = "/srv/www/htdocs/" ## where to send error-messages to server.errorlog = "/var/log/lighttpd/error.log" # files to check for if .../ is requested index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) ## set the event-handler (read the performance section in the manual) # server.event-handler = "freebsd-kqueue" # needed on OS X # mimetype mapping mimetype.assign = ( ".pdf" => "application/pdf", ".sig" => "application/pgp-signature", ".spl" => "application/futuresplash", ".class" => "application/octet-stream", ".ps" => "application/postscript", ".torrent" => "application/x-bittorrent", ".dvi" => "application/x-dvi", ".gz" => "application/x-gzip", ".pac" => "application/x-ns-proxy-autoconfig", ".swf" => "application/x-shockwave-flash", ".tar.gz" => "application/x-tgz", ".tgz" => "application/x-tgz", ".tar" => "application/x-tar", ".zip" => "application/zip", ".mp3" => "audio/mpeg", ".m3u" => "audio/x-mpegurl", ".wma" => "audio/x-ms-wma", ".wax" => "audio/x-ms-wax", ".ogg" => "application/ogg", ".wav" => "audio/x-wav", ".gif" => "image/gif", ".jar" => "application/x-java-archive", ".jpg" => "image/jpeg", ".jpeg" => "image/jpeg", ".png" => "image/png", ".xbm" => "image/x-xbitmap", ".xpm" => "image/x-xpixmap", ".xwd" => "image/x-xwindowdump", ".css" => "text/css", ".html" => "text/html", ".htm" => "text/html", ".js" => "text/javascript", ".asc" => "text/plain", ".c" => "text/plain", ".cpp" => "text/plain", ".log" => "text/plain", ".conf" => "text/plain", ".text" => "text/plain", ".txt" => "text/plain", ".dtd" => "text/xml", ".xml" => "text/xml", ".mpeg" => "video/mpeg", ".mpg" => "video/mpeg", ".mov" => "video/quicktime", ".qt" => "video/quicktime", ".avi" => "video/x-msvideo", ".asf" => "video/x-ms-asf", ".asx" => "video/x-ms-asf", ".wmv" => "video/x-ms-wmv", ".bz2" => "application/x-bzip", ".tbz" => "application/x-bzip-compressed-tar", ".tar.bz2" => "application/x-bzip-compressed-tar", # default mime type "" => "application/octet-stream", ) # Use the "Content-Type" extended attribute to obtain mime type if possible #mimetype.use-xattr = "enable" ## send a different Server: header ## be nice and keep it at lighttpd # server.tag = "lighttpd" #### accesslog module accesslog.filename = "/var/log/lighttpd/access.log" ## deny access the file-extensions # # ~ is for backupfiles from vi, emacs, joe, ... # .inc is often used for code includes which should in general not be part # of the document-root url.access-deny = ( "~", ".inc" ) $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } ## # which extensions should not be handle via static-file transfer # # .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) ######### Options that are good to be but not neccesary to be changed ####### ## bind to port (default: 80) #server.port = 81 ## bind to localhost (default: all interfaces) #server.bind = "127.0.0.1" ## error-handler for status 404 #server.error-handler-404 = "/error-handler.html" #server.error-handler-404 = "/error-handler.php" ## to help the rc.scripts #server.pid-file = "/var/run/lighttpd.pid" ###### virtual hosts ## ## If you want name-based virtual hosting add the next three settings and load ## mod_simple_vhost ## ## document-root = ## virtual-server-root + virtual-server-default-host + virtual-server-docroot ## or ## virtual-server-root + http-host + virtual-server-docroot ## #simple-vhost.server-root = "/srv/www/vhosts/" #simple-vhost.default-host = "www.example.org" #simple-vhost.document-root = "/htdocs/" ## ## Format: <errorfile-prefix><status-code>.html ## -> ..../status-404.html for 'File not found' #server.errorfile-prefix = "/usr/share/lighttpd/errors/status-" #server.errorfile-prefix = "/srv/www/errors/status-" ## virtual directory listings #dir-listing.activate = "enable" ## select encoding for directory listings #dir-listing.encoding = "utf-8" ## enable debugging #debug.log-request-header = "enable" #debug.log-response-header = "enable" #debug.log-request-handling = "enable" #debug.log-file-not-found = "enable" ### only root can use these options # # chroot() to directory (default: no chroot() ) #server.chroot = "/" ## change uid to <uid> (default: don't care) #server.username = "wwwrun" ## change uid to <uid> (default: don't care) #server.groupname = "wwwrun" #### compress module #compress.cache-dir = "/var/cache/lighttpd/compress/" #compress.filetype = ("text/plain", "text/html") #### proxy module ## read proxy.txt for more info #proxy.server = ( ".php" => # ( "localhost" => # ( # "host" => "192.168.0.101", # "port" => 80 # ) # ) # ) #### fastcgi module ## read fastcgi.txt for more info ## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini #fastcgi.server = ( ".php" => # ( "localhost" => # ( # "socket" => "/var/run/lighttpd/php-fastcgi.s ocket", # "bin-path" => "/usr/local/bin/php-cgi" # ) # ) # ) #### CGI module #cgi.assign = ( ".pl" => "/usr/bin/perl", # ".cgi" => "/usr/bin/perl" ) # #### SSL engine #ssl.engine = "enable" #ssl.pemfile = "/etc/ssl/private/lighttpd.pem" #### status module #status.status-url = "/server-status" #status.config-url = "/server-config" #### auth module ## read authentication.txt for more info #auth.backend = "plain" #auth.backend.plain.userfile = "lighttpd.user" #auth.backend.plain.groupfile = "lighttpd.group" #auth.backend.ldap.hostname = "localhost" #auth.backend.ldap.base-dn = "dc=my-domain,dc=com" #auth.backend.ldap.filter = "(uid=$)" #auth.require = ( "/server-status" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "user=jan" # ), # "/server-config" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "valid-user" # ) # ) #### url handling modules (rewrite, redirect, access) #url.rewrite = ( "^/$" => "/server-status" ) #url.redirect = ( "^/wishlist/(.+)" => "http://www.123.org/$1" ) #### both rewrite/redirect support back reference to regex conditional using %n #$HTTP["host"] =~ "^www\.(.*)" { # url.redirect = ( "^/(.*)" => "http://%1/$1" ) #} # # define a pattern for the host url finding # %% => % sign # %0 => domain name + tld # %1 => tld # %2 => domain name without tld # %3 => subdomain 1 name # %4 => subdomain 2 name # #evhost.path-pattern = "/srv/www/vhosts/%3/htdocs/" #### expire module #expire.url = ( "/buggy/" => "access 2 hours", "/asdhas/" => "ac cess plus 1 seconds 2 minutes") #### ssi #ssi.extension = ( ".shtml" ) #### rrdtool #rrdtool.binary = "/usr/bin/rrdtool" #rrdtool.db-name = "/var/lib/lighttpd/lighttpd.rrd" #### setenv #setenv.add-request-header = ( "TRAV_ENV" => "mysql://user@host/db" ) #setenv.add-response-header = ( "X-Secret-Message" => "42" ) ## for mod_trigger_b4_dl # trigger-before-download.gdbm-filename = "/var/lib/lighttpd/trigger.db" # trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" ) # trigger-before-download.trigger-url = "^/trigger/" # trigger-before-download.download-url = "^/download/" # trigger-before-download.deny-url = "http://127.0.0.1/index.html" # trigger-before-download.trigger-timeout = 10 #### variable usage: ## variable name without "." is auto prefixed by "var." and becomes "var.bar" #bar = 1 #var.mystring = "foo" ## integer add #bar += 1 ## string concat, with integer cast as string, result: "www.foo1.com" #server.name = "www." + mystring + var.bar + ".com" ## array merge #index-file.names = (foo + ".php") + index-file.names #index-file.names += (foo + ".php") #### include #include /etc/lighttpd/lighttpd-inc.conf ## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf" #include "lighttpd-inc.conf" #### include_shell #include_shell "echo var.a=1" ## the above is same as: #var.a=1 When I go to browser and hit 'http://127.0.0.1', I get link not found. Any idea?

Read the article

I need advices: small memory footprint linux mail server with spam filtering

- by petermolnar

I have a VPS which is originally destined to be a webserver but some minimal mail capabilities are needed to be deployed as well, including sending and receiving as standalone server. The current setup is the following: Postfix reveices the mail, the users are in virtual tables, stored in MySQL on connection all servers are tested with policyd-weight service against some DNSBLs all mail is runs through SpamAssassin spamd with the help of spamc client the mail is then delivered with Dovecot 2' LDA (local delivery agent), virtual users as well As you saw... there's no virus scanner running, and that's for a reason: clamav eats all the memory possible and also, virus mails are all filtered out with this setup (I've tested the same with ClamAV enabled for 1,5 years, no virus mail ever got even to ClamAV) I don't use amavisd and I really don't want to. You only need that monster if you have plenty of memory and lots of simultaneous scanners. It's also a nightmare to fine tune by hand. I run policyd-weight instead of policyd and native DNSBLs in postfix. I don't like to send someone away because a single service listed them. Important statement: everything works fine. I receive very small amount of spam, nearly never get a false positive and most of the bad mail is stopped by policyd-weight. The only "problem" that I feel the services at total uses a bit much memory alltogether. I've already cut the modules of spamassassin (see below), but I'd really like to hear some advices how to cut the memory footprint as low as possible, mostly: what plugins SpamAssassin really needs and what are more or less useless, regarding to my current postfix & policyd-weight setup? SpamAssassin rules are also compiled with sa-compile (sa-update runs once a week from cron, compile runs right after that) These are some of the current configurations that may matter, please tell me if you need anything more. postfix/master.cf (parts only) dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -e /usr/lib/dovecot/deliver -d ${recipient} -f {sender} postfix/main.cf (parts only) smtpd_helo_required = yes smtpd_helo_restrictions = permit_mynetworks, reject_invalid_hostname, permit smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_invalid_hostname, reject_non_fqdn_hostname, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_pipelining, reject_unauth_destination, check_policy_service inet:127.0.0.1:12525, permit policyd-weight.conf (parts only) $REJECTMSG = "550 Mail appeared to be SPAM or forged. Ask your Mail/DNS-Administrator to correct HELO and DNS MX settings or to get removed from DNSBLs"; $REJECTLEVEL = 4; $DEFER_STRING = 'IN_SPAMCOP= BOGUS_MX='; $DEFER_ACTION = '450'; $DEFER_LEVEL = 5; $DNSERRMSG = '450 No DNS entries for your MTA, HELO and Domain. Contact YOUR administrator'; # 1: ON, 0: OFF (default) # If ON request that ALL clients are only checked against RBLs $dnsbl_checks_only = 0; # 1: ON (default), 0: OFF # When set to ON it logs only RBLs which affect scoring (positive or negative) $LOG_BAD_RBL_ONLY = 1; ## DNSBL settings @dnsbl_score = ( # host, hit, miss, log name 'dnsbl.ahbl.org', 3, -1, 'dnsbl.ahbl.org', 'dnsbl.njabl.org', 3, -1, 'dnsbl.njabl.org', 'dnsbl.sorbs.net', 3, -1, 'dnsbl.sorbs.net', 'bl.spamcop.net', 3, -1, 'bl.spamcop.net', 'zen.spamhaus.org', 3, -1, 'zen.spamhaus.org', 'pbl.spamhaus.org', 3, -1, 'pbl.spamhaus.org', 'cbl.abuseat.org', 3, -1, 'cbl.abuseat.org', 'list.dsbl.org', 3, -1, 'list.dsbl.org', ); # If Client IP is listed in MORE DNSBLS than this var, it gets REJECTed immediately $MAXDNSBLHITS = 3; # alternatively, if the score of DNSBLs is ABOVE this level, reject immediately $MAXDNSBLSCORE = 9; $MAXDNSBLMSG = '550 Az levelezoszerveruk IP cime tul sok spamlistan talahato, kerjuk ellenorizze! / Your MTA is listed in too many DNSBLs; please check.'; ## RHSBL settings @rhsbl_score = ( 'multi.surbl.org', 4, 0, 'multi.surbl.org', 'rhsbl.ahbl.org', 4, 0, 'rhsbl.ahbl.org', 'dsn.rfc-ignorant.org', 4, 0, 'dsn.rfc-ignorant.org', # 'postmaster.rfc-ignorant.org', 0.1, 0, 'postmaster.rfc-ignorant.org', # 'abuse.rfc-ignorant.org', 0.1, 0, 'abuse.rfc-ignorant.org' ); # skip a RBL if this RBL had this many continuous errors $BL_ERROR_SKIP = 2; # skip a RBL for that many times $BL_SKIP_RELEASE = 10; ## cache stuff # must be a directory (add trailing slash) $LOCKPATH = '/var/run/policyd-weight/'; # socket path for the cache daemon. $SPATH = $LOCKPATH.'/polw.sock'; # how many seconds the cache may be idle before starting maintenance routines #NOTE: standard maintenance jobs happen regardless of this setting. $MAXIDLECACHE = 60; # after this number of requests do following maintenance jobs: checking for config changes $MAINTENANCE_LEVEL = 5; # negative (i.e. SPAM) result cache settings ################################## # set to 0 to disable caching for spam results. To this level the cache will be cleaned. $CACHESIZE = 2000; # at this number of entries cleanup takes place $CACHEMAXSIZE = 4000; $CACHEREJECTMSG = '550 temporarily blocked because of previous errors'; # after NTTL retries the cache entry is deleted $NTTL = 1; # client MUST NOT retry within this seconds in order to decrease TTL counter $NTIME = 30; # positve (i.,e. HAM) result cache settings ################################### # set to 0 to disable caching of HAM. To this number of entries the cache will be cleaned $POSCACHESIZE = 1000; # at this number of entries cleanup takes place $POSCACHEMAXSIZE = 2000; $POSCACHEMSG = 'using cached result'; #after PTTL requests the HAM entry must succeed one time the RBL checks again $PTTL = 60; # after $PTIME in HAM Cache the client must pass one time the RBL checks again. #Values must be nonfractal. Accepted time-units: s, m, h, d $PTIME = '3h'; # The client must pass this time the RBL checks in order to be listed as hard-HAM # After this time the client will pass immediately for PTTL within PTIME $TEMP_PTIME = '1d'; ## DNS settings # Retries for ONE DNS-Lookup $DNS_RETRIES = 1; # Retry-interval for ONE DNS-Lookup $DNS_RETRY_IVAL = 5; # max error count for unresponded queries in a complete policy query $MAXDNSERR = 3; $MAXDNSERRMSG = 'passed - too many local DNS-errors'; # persistent udp connection for DNS queries. #broken in Net::DNS version 0.51. Works with Net::DNS 0.53; DEFAULT: off $PUDP= 0; # Force the usage of Net::DNS for RBL lookups. # Normally policyd-weight tries to use a faster RBL lookup routine instead of Net::DNS $USE_NET_DNS = 0; # A list of space separated NS IPs # This overrides resolv.conf settings # Example: $NS = '1.2.3.4 1.2.3.5'; # DEFAULT: empty $NS = ''; # timeout for receiving from cache instance $IPC_TIMEOUT = 2; # If set to 1 policyd-weight closes connections to smtpd clients in order to avoid too many #established connections to one policyd-weight child $TRY_BALANCE = 0; # scores for checks, WARNING: they may manipulate eachother # or be factors for other scores. # HIT score, MISS Score @client_ip_eq_helo_score = (1.5, -1.25 ); @helo_score = (1.5, -2 ); @helo_score = (0, -2 ); @helo_from_mx_eq_ip_score= (1.5, -3.1 ); @helo_numeric_score= (2.5, 0 ); @from_match_regex_verified_helo= (1,-2 ); @from_match_regex_unverified_helo = (1.6, -1.5 ); @from_match_regex_failed_helo = (2.5, 0 ); @helo_seems_dialup = (1.5, 0 ); @failed_helo_seems_dialup= (2, 0 ); @helo_ip_in_client_subnet= (0,-1.2 ); @helo_ip_in_cl16_subnet = (0,-0.41 ); #@client_seems_dialup_score = (3.75, 0 ); @client_seems_dialup_score = (0, 0 ); @from_multiparted = (1.09, 0 ); @from_anon= (1.17, 0 ); @bogus_mx_score = (2.1, 0 ); @random_sender_score = (0.25, 0 ); @rhsbl_penalty_score = (3.1, 0 ); @enforce_dyndns_score = (3, 0 ); spamassassin/init.pre (I've put the .pre files together) loadplugin Mail::SpamAssassin::Plugin::Hashcash loadplugin Mail::SpamAssassin::Plugin::SPF loadplugin Mail::SpamAssassin::Plugin::Pyzor loadplugin Mail::SpamAssassin::Plugin::Razor2 loadplugin Mail::SpamAssassin::Plugin::AutoLearnThreshold loadplugin Mail::SpamAssassin::Plugin::MIMEHeader loadplugin Mail::SpamAssassin::Plugin::ReplaceTags loadplugin Mail::SpamAssassin::Plugin::Check loadplugin Mail::SpamAssassin::Plugin::HTTPSMismatch loadplugin Mail::SpamAssassin::Plugin::URIDetail loadplugin Mail::SpamAssassin::Plugin::Bayes loadplugin Mail::SpamAssassin::Plugin::BodyEval loadplugin Mail::SpamAssassin::Plugin::DNSEval loadplugin Mail::SpamAssassin::Plugin::HTMLEval loadplugin Mail::SpamAssassin::Plugin::HeaderEval loadplugin Mail::SpamAssassin::Plugin::MIMEEval loadplugin Mail::SpamAssassin::Plugin::RelayEval loadplugin Mail::SpamAssassin::Plugin::URIEval loadplugin Mail::SpamAssassin::Plugin::WLBLEval loadplugin Mail::SpamAssassin::Plugin::VBounce loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody spamassassin/local.cf (parts) use_bayes 1 bayes_auto_learn 1 bayes_store_module Mail::SpamAssassin::BayesStore::MySQL bayes_sql_dsn DBI:mysql:db:127.0.0.1:3306 bayes_sql_username user bayes_sql_password pass bayes_ignore_header X-Bogosity bayes_ignore_header X-Spam-Flag bayes_ignore_header X-Spam-Status ### User settings user_scores_dsn DBI:mysql:db:127.0.0.1:3306 user_scores_sql_password user user_scores_sql_username pass user_scores_sql_custom_query SELECT preference, value FROM _TABLE_ WHERE username = _USERNAME_ OR username = '$GLOBAL' OR username = CONCAT('%',_DOMAIN_) ORDER BY username ASC # for better speed score DNS_FROM_AHBL_RHSBL 0 score __RFC_IGNORANT_ENVFROM 0 score DNS_FROM_RFC_DSN 0 score DNS_FROM_RFC_BOGUSMX 0 score __DNS_FROM_RFC_POST 0 score __DNS_FROM_RFC_ABUSE 0 score __DNS_FROM_RFC_WHOIS 0 UPDATE 01 As adaptr advised I remove policyd-weight and configured postfix postscreen, this resulted approximately -15-20 MB from RAM usage and a lot faster work. I'm not sure it's working at full capacity but it seems promising.

Read the article

How to configure fastcgi to work with ligttpd in ubuntu

- by michael

I am able to run lighttpd on ubuntu 9.10. But when i tried to setup fastcgi with lighttpd by putting this in the ligttpd.conf file: #### fastcgi module fastcgi.server = ( "/fastcgi_scripts/" => (( "host" => "127.0.0.1", "port" => "9098", "check-local" => "disable", "bin-path" => "/usr/local/bin/cgi-fcgi", "docroot" => "/" # remote server may use # it's own docroot )) ) This is what I get in the error.log in ligttpd: 2010-03-07 21:00:11: (log.c.166) server started 2010-03-07 21:00:11: (mod_fastcgi.c.1104) the fastcgi-backend /usr/local/bin/cgi-fcgi failed to start: 2010-03-07 21:00:11: (mod_fastcgi.c.1108) child exited with status 1 /usr/local/bin/cgi-fcgi 2010-03-07 21:00:11: (mod_fastcgi.c.1111) If you're trying to run your app as a FastCGI backend, make sure you're using the FastCGI-enabled version. If this is PHP on Gentoo, add 'fastcgi' to the USE flags. 2010-03-07 21:00:11: (mod_fastcgi.c.1399) [ERROR]: spawning fcgi failed. 2010-03-07 21:00:11: (server.c.931) Configuration of plugins failed. Going down. I do have cgi-fcgi in /usr/local/bin: $ which cgi-fcgi /usr/local/bin/cgi-fcgi '/usr/local/bin/cgi-fcgi' is the executable after I download and compile fast-cgi. Here is my lighttpd conf file: $ more lighttpd.conf # lighttpd configuration file # # use it as a base for lighttpd 1.0.0 and above # # $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $ ############ Options you really have to take care of #################### ## modules to load # at least mod_access and mod_accesslog should be loaded # all other module should only be loaded if really neccesary # - saves some time # - saves memory server.modules = ( # "mod_rewrite", # "mod_redirect", # "mod_alias", "mod_access", # "mod_trigger_b4_dl", # "mod_auth", # "mod_status", # "mod_setenv", "mod_fastcgi", # "mod_proxy", # "mod_simple_vhost", # "mod_evhost", # "mod_userdir", # "mod_cgi", # "mod_compress", # "mod_ssi", # "mod_usertrack", # "mod_expire", # "mod_secdownload", # "mod_rrdtool", "mod_accesslog" ) ## A static document-root. For virtual hosting take a look at the ## mod_simple_vhost module. server.document-root = "/srv/www/htdocs/" ## where to send error-messages to server.errorlog = "/var/log/lighttpd/error.log" # files to check for if .../ is requested index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) ## set the event-handler (read the performance section in the manual) # server.event-handler = "freebsd-kqueue" # needed on OS X # mimetype mapping mimetype.assign = ( ".pdf" => "application/pdf", ".sig" => "application/pgp-signature", ".spl" => "application/futuresplash", ".class" => "application/octet-stream", ".ps" => "application/postscript", ".torrent" => "application/x-bittorrent", ".dvi" => "application/x-dvi", ".gz" => "application/x-gzip", ".pac" => "application/x-ns-proxy-autoconfig", ".swf" => "application/x-shockwave-flash", ".tar.gz" => "application/x-tgz", ".tgz" => "application/x-tgz", ".tar" => "application/x-tar", ".zip" => "application/zip", ".mp3" => "audio/mpeg", ".m3u" => "audio/x-mpegurl", ".wma" => "audio/x-ms-wma", ".wax" => "audio/x-ms-wax", ".ogg" => "application/ogg", ".wav" => "audio/x-wav", ".gif" => "image/gif", ".jar" => "application/x-java-archive", ".jpg" => "image/jpeg", ".jpeg" => "image/jpeg", ".png" => "image/png", ".xbm" => "image/x-xbitmap", ".xpm" => "image/x-xpixmap", ".xwd" => "image/x-xwindowdump", ".css" => "text/css", ".html" => "text/html", ".htm" => "text/html", ".js" => "text/javascript", ".asc" => "text/plain", ".c" => "text/plain", ".cpp" => "text/plain", ".log" => "text/plain", ".conf" => "text/plain", ".text" => "text/plain", ".txt" => "text/plain", ".dtd" => "text/xml", ".xml" => "text/xml", ".mpeg" => "video/mpeg", ".mpg" => "video/mpeg", ".mov" => "video/quicktime", ".qt" => "video/quicktime", ".avi" => "video/x-msvideo", ".asf" => "video/x-ms-asf", ".asx" => "video/x-ms-asf", ".wmv" => "video/x-ms-wmv", ".bz2" => "application/x-bzip", ".tbz" => "application/x-bzip-compressed-tar", ".tar.bz2" => "application/x-bzip-compressed-tar", # default mime type "" => "application/octet-stream", ) # Use the "Content-Type" extended attribute to obtain mime type if possible #mimetype.use-xattr = "enable" ## send a different Server: header ## be nice and keep it at lighttpd # server.tag = "lighttpd" #### accesslog module accesslog.filename = "/var/log/lighttpd/access.log" ## deny access the file-extensions # # ~ is for backupfiles from vi, emacs, joe, ... # .inc is often used for code includes which should in general not be part # of the document-root url.access-deny = ( "~", ".inc" ) $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } ## # which extensions should not be handle via static-file transfer # # .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) ######### Options that are good to be but not neccesary to be changed ####### ## bind to port (default: 80) server.port = 9090 ## bind to localhost (default: all interfaces) server.bind = "127.0.0.1" ## error-handler for status 404 #server.error-handler-404 = "/error-handler.html" #server.error-handler-404 = "/error-handler.php" ## to help the rc.scripts #server.pid-file = "/var/run/lighttpd.pid" ###### virtual hosts ## ## If you want name-based virtual hosting add the next three settings and load ## mod_simple_vhost ## ## document-root = ## virtual-server-root + virtual-server-default-host + virtual-server-docroot ## or ## virtual-server-root + http-host + virtual-server-docroot ## #simple-vhost.server-root = "/srv/www/vhosts/" #simple-vhost.default-host = "www.example.org" #simple-vhost.document-root = "/htdocs/" ## ## Format: <errorfile-prefix><status-code>.html ## -> ..../status-404.html for 'File not found' #server.errorfile-prefix = "/usr/share/lighttpd/errors/status-" #server.errorfile-prefix = "/srv/www/errors/status-" ## virtual directory listings #dir-listing.activate = "enable" ## select encoding for directory listings #dir-listing.encoding = "utf-8" ## enable debugging #debug.log-request-header = "enable" #debug.log-response-header = "enable" #debug.log-request-handling = "enable" #debug.log-file-not-found = "enable" ### only root can use these options # # chroot() to directory (default: no chroot() ) #server.chroot = "/" ## change uid to <uid> (default: don't care) #server.username = "wwwrun" ## change uid to <uid> (default: don't care) #server.groupname = "wwwrun" #### compress module #compress.cache-dir = "/var/cache/lighttpd/compress/" #compress.filetype = ("text/plain", "text/html") #### proxy module ## read proxy.txt for more info #proxy.server = ( ".php" => # ( "localhost" => # ( # "host" => "192.168.0.101", # "port" => 80 # ) # ) # ) #### fastcgi module fastcgi.server = ( "/fastcgi_scripts/" => (( "host" => "127.0.0.1", "port" => 1026, "check-local" => "disable", "bin-path" => "/usr/local/bin/cgi-fcgi", #"docroot" => "/" # remote server may use # it's own docroot )) ) ## read fastcgi.txt for more info ## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini #fastcgi.server = ( ".php" => # ( "localhost" => # ( # "socket" => "/var/run/lighttpd/php-fastcgi.s ocket", # "bin-path" => "/usr/local/bin/php-cgi" # ) # ) # ) #### CGI module #cgi.assign = ( ".pl" => "/usr/bin/perl", # ".cgi" => "/usr/bin/perl" ) # #### SSL engine #ssl.engine = "enable" #ssl.pemfile = "/etc/ssl/private/lighttpd.pem" #### status module #status.status-url = "/server-status" #status.config-url = "/server-config" #### auth module ## read authentication.txt for more info #auth.backend = "plain" #auth.backend.plain.userfile = "lighttpd.user" #auth.backend.plain.groupfile = "lighttpd.group" #auth.backend.ldap.hostname = "localhost" #auth.backend.ldap.base-dn = "dc=my-domain,dc=com" #auth.backend.ldap.filter = "(uid=$)" #auth.require = ( "/server-status" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "user=jan" # ), # "/server-config" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "valid-user" # ) # ) #### url handling modules (rewrite, redirect, access) #url.rewrite = ( "^/$" => "/server-status" ) #url.redirect = ( "^/wishlist/(.+)" => "http://www.123.org/$1" ) #### both rewrite/redirect support back reference to regex conditional using %n #$HTTP["host"] =~ "^www\.(.*)" { # url.redirect = ( "^/(.*)" => "http://%1/$1" ) #} # # define a pattern for the host url finding # %% => % sign # %0 => domain name + tld # %1 => tld # %2 => domain name without tld # %3 => subdomain 1 name # %4 => subdomain 2 name # #evhost.path-pattern = "/srv/www/vhosts/%3/htdocs/" #### expire module #expire.url = ( "/buggy/" => "access 2 hours", "/asdhas/" => "ac cess plus 1 seconds 2 minutes") #### ssi #ssi.extension = ( ".shtml" ) #### rrdtool #rrdtool.binary = "/usr/bin/rrdtool" #rrdtool.db-name = "/var/lib/lighttpd/lighttpd.rrd" #### setenv #setenv.add-request-header = ( "TRAV_ENV" => "mysql://user@host/db" ) #setenv.add-response-header = ( "X-Secret-Message" => "42" ) ## for mod_trigger_b4_dl # trigger-before-download.gdbm-filename = "/var/lib/lighttpd/trigger.db" # trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" ) # trigger-before-download.trigger-url = "^/trigger/" # trigger-before-download.download-url = "^/download/" # trigger-before-download.deny-url = "http://127.0.0.1/index.html" # trigger-before-download.trigger-timeout = 10 #### variable usage: ## variable name without "." is auto prefixed by "var." and becomes "var.bar" #bar = 1 #var.mystring = "foo" ## integer add #bar += 1 ## string concat, with integer cast as string, result: "www.foo1.com" #server.name = "www." + mystring + var.bar + ".com" ## array merge #index-file.names = (foo + ".php") + index-file.names #index-file.names += (foo + ".php") #### include #include /etc/lighttpd/lighttpd-inc.conf ## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf" #include "lighttpd-inc.conf" #### include_shell #include_shell "echo var.a=1" ## the above is same as: #var.a=1 Thank you for your help.

Read the article

ServerRoot in my lighttpd.conf

- by michael

Hi, I have use the following example lighttpd.conf to launch my lighttpd. Can you please tell me where is my 'ServerRoot'? # lighttpd configuration file # # use it as a base for lighttpd 1.0.0 and above # # $Id: lighttpd.conf,v 1.7 2004/11/03 22:26:05 weigon Exp $ ############ Options you really have to take care of #################### ## modules to load # at least mod_access and mod_accesslog should be loaded # all other module should only be loaded if really neccesary # - saves some time # - saves memory server.modules = ( # "mod_rewrite", # "mod_redirect", # "mod_alias", "mod_access", # "mod_trigger_b4_dl", # "mod_auth", # "mod_status", # "mod_setenv", "mod_fastcgi", # "mod_proxy", # "mod_simple_vhost", # "mod_evhost", # "mod_userdir", # "mod_cgi", # "mod_compress", # "mod_ssi", # "mod_usertrack", # "mod_expire", # "mod_secdownload", # "mod_rrdtool", "mod_accesslog" ) ## A static document-root. For virtual hosting take a look at the ## mod_simple_vhost module. server.document-root = "/srv/www/htdocs/" ## where to send error-messages to server.errorlog = "/var/log/lighttpd/error.log" # files to check for if .../ is requested index-file.names = ( "index.php", "index.html", "index.htm", "default.htm" ) ## set the event-handler (read the performance section in the manual) # server.event-handler = "freebsd-kqueue" # needed on OS X # mimetype mapping mimetype.assign = ( ".pdf" => "application/pdf", ".sig" => "application/pgp-signature", ".spl" => "application/futuresplash", ".class" => "application/octet-stream", ".ps" => "application/postscript", ".torrent" => "application/x-bittorrent", ".dvi" => "application/x-dvi", ".gz" => "application/x-gzip", ".pac" => "application/x-ns-proxy-autoconfig", ".swf" => "application/x-shockwave-flash", ".tar.gz" => "application/x-tgz", ".tgz" => "application/x-tgz", ".tar" => "application/x-tar", ".zip" => "application/zip", ".mp3" => "audio/mpeg", ".m3u" => "audio/x-mpegurl", ".wma" => "audio/x-ms-wma", ".wax" => "audio/x-ms-wax", ".ogg" => "application/ogg", ".wav" => "audio/x-wav", ".gif" => "image/gif", ".jar" => "application/x-java-archive", ".jpg" => "image/jpeg", ".jpeg" => "image/jpeg", ".png" => "image/png", ".xbm" => "image/x-xbitmap", ".xpm" => "image/x-xpixmap", ".xwd" => "image/x-xwindowdump", ".css" => "text/css", ".html" => "text/html", ".htm" => "text/html", ".js" => "text/javascript", ".asc" => "text/plain", ".c" => "text/plain", ".cpp" => "text/plain", ".log" => "text/plain", ".conf" => "text/plain", ".text" => "text/plain", ".txt" => "text/plain", ".dtd" => "text/xml", ".xml" => "text/xml", ".mpeg" => "video/mpeg", ".mpg" => "video/mpeg", ".mov" => "video/quicktime", ".qt" => "video/quicktime", ".avi" => "video/x-msvideo", ".asf" => "video/x-ms-asf", ".asx" => "video/x-ms-asf", ".wmv" => "video/x-ms-wmv", ".bz2" => "application/x-bzip", ".tbz" => "application/x-bzip-compressed-tar", ".tar.bz2" => "application/x-bzip-compressed-tar", # default mime type "" => "application/octet-stream", ) # Use the "Content-Type" extended attribute to obtain mime type if possible #mimetype.use-xattr = "enable" ## send a different Server: header ## be nice and keep it at lighttpd # server.tag = "lighttpd" #### accesslog module accesslog.filename = "/var/log/lighttpd/access.log" ## deny access the file-extensions # # ~ is for backupfiles from vi, emacs, joe, ... # .inc is often used for code includes which should in general not be part # of the document-root url.access-deny = ( "~", ".inc" ) $HTTP["url"] =~ "\.pdf$" { server.range-requests = "disable" } ## # which extensions should not be handle via static-file transfer # # .php, .pl, .fcgi are most often handled by mod_fastcgi or mod_cgi static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" ) ######### Options that are good to be but not neccesary to be changed ####### ## bind to port (default: 80) server.port = 9090 ## bind to localhost (default: all interfaces) server.bind = "127.0.0.1" ## error-handler for status 404 #server.error-handler-404 = "/error-handler.html" #server.error-handler-404 = "/error-handler.php" ## to help the rc.scripts #server.pid-file = "/var/run/lighttpd.pid" ###### virtual hosts ## ## If you want name-based virtual hosting add the next three settings and load ## mod_simple_vhost ## ## document-root = ## virtual-server-root + virtual-server-default-host + virtual-server-docroot ## or ## virtual-server-root + http-host + virtual-server-docroot ## #simple-vhost.server-root = "/srv/www/vhosts/" #simple-vhost.default-host = "www.example.org" #simple-vhost.document-root = "/htdocs/" ## ## Format: <errorfile-prefix><status-code>.html ## -> ..../status-404.html for 'File not found' #server.errorfile-prefix = "/usr/share/lighttpd/errors/status-" #server.errorfile-prefix = "/srv/www/errors/status-" ## virtual directory listings #dir-listing.activate = "enable" ## select encoding for directory listings #dir-listing.encoding = "utf-8" ## enable debugging #debug.log-request-header = "enable" #debug.log-response-header = "enable" #debug.log-request-handling = "enable" #debug.log-file-not-found = "enable" ### only root can use these options # # chroot() to directory (default: no chroot() ) #server.chroot = "/" ## change uid to <uid> (default: don't care) #server.username = "wwwrun" ## change uid to <uid> (default: don't care) #server.groupname = "wwwrun" #### compress module #compress.cache-dir = "/var/cache/lighttpd/compress/" #compress.filetype = ("text/plain", "text/html") #### proxy module ## read proxy.txt for more info #proxy.server = ( ".php" => # ( "localhost" => # ( # "host" => "192.168.0.101", # "port" => 80 # ) # ) # ) #### fastcgi module fastcgi.server = ( "/fastcgi_scripts/" => (( "host" => "127.0.0.1", "port" => 1026, "check-local" => "disable", "bin-path" => "/usr/local/bin/cgi-fcgi", #"docroot" => "/" # remote server may use # it's own docroot )) ) ## read fastcgi.txt for more info ## for PHP don't forget to set cgi.fix_pathinfo = 1 in the php.ini #fastcgi.server = ( ".php" => # ( "localhost" => # ( # "socket" => "/var/run/lighttpd/php-fastcgi.socket", # "bin-path" => "/usr/local/bin/php-cgi" # ) # ) # ) #### CGI module #cgi.assign = ( ".pl" => "/usr/bin/perl", # ".cgi" => "/usr/bin/perl" ) # #### SSL engine #ssl.engine = "enable" #ssl.pemfile = "/etc/ssl/private/lighttpd.pem" #### status module #status.status-url = "/server-status" #status.config-url = "/server-config" #### auth module ## read authentication.txt for more info #auth.backend = "plain" #auth.backend.plain.userfile = "lighttpd.user" #auth.backend.plain.groupfile = "lighttpd.group" #auth.backend.ldap.hostname = "localhost" #auth.backend.ldap.base-dn = "dc=my-domain,dc=com" #auth.backend.ldap.filter = "(uid=$)" #auth.require = ( "/server-status" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "user=jan" # ), # "/server-config" => # ( # "method" => "digest", # "realm" => "download archiv", # "require" => "valid-user" # ) # ) #### url handling modules (rewrite, redirect, access) #url.rewrite = ( "^/$" => "/server-status" ) #url.redirect = ( "^/wishlist/(.+)" => "http://www.123.org/$1" ) #### both rewrite/redirect support back reference to regex conditional using %n #$HTTP["host"] =~ "^www\.(.*)" { # url.redirect = ( "^/(.*)" => "http://%1/$1" ) #} # # define a pattern for the host url finding # %% => % sign # %0 => domain name + tld # %1 => tld # %2 => domain name without tld # %3 => subdomain 1 name # %4 => subdomain 2 name # #evhost.path-pattern = "/srv/www/vhosts/%3/htdocs/" #### expire module #expire.url = ( "/buggy/" => "access 2 hours", "/asdhas/" => "access plus 1 seconds 2 minutes") #### ssi #ssi.extension = ( ".shtml" ) #### rrdtool #rrdtool.binary = "/usr/bin/rrdtool" #rrdtool.db-name = "/var/lib/lighttpd/lighttpd.rrd" #### setenv #setenv.add-request-header = ( "TRAV_ENV" => "mysql://user@host/db" ) #setenv.add-response-header = ( "X-Secret-Message" => "42" ) ## for mod_trigger_b4_dl # trigger-before-download.gdbm-filename = "/var/lib/lighttpd/trigger.db" # trigger-before-download.memcache-hosts = ( "127.0.0.1:11211" ) # trigger-before-download.trigger-url = "^/trigger/" # trigger-before-download.download-url = "^/download/" # trigger-before-download.deny-url = "http://127.0.0.1/index.html" # trigger-before-download.trigger-timeout = 10 #### variable usage: ## variable name without "." is auto prefixed by "var." and becomes "var.bar" #bar = 1 #var.mystring = "foo" ## integer add #bar += 1 ## string concat, with integer cast as string, result: "www.foo1.com" #server.name = "www." + mystring + var.bar + ".com" ## array merge #index-file.names = (foo + ".php") + index-file.names #index-file.names += (foo + ".php") #### include #include /etc/lighttpd/lighttpd-inc.conf ## same as above if you run: "lighttpd -f /etc/lighttpd/lighttpd.conf" #include "lighttpd-inc.conf" #### include_shell #include_shell "echo var.a=1" ## the above is same as: #var.a=1 Thank you.

Read the article

Understanding and Implementing a Force based graph layout algorithm

- by zcourts

I'm trying to implement a force base graph layout algorithm, based on http://en.wikipedia.org/wiki/Force-based_algorithms_(graph_drawing) My first attempt didn't work so I looked at http://blog.ivank.net/force-based-graph-drawing-in-javascript.html and https://github.com/dhotson/springy I changed my implementation based on what I thought I understood from those two but I haven't managed to get it right and I'm hoping someone can help? JavaScript isn't my strong point so be gentle... If you're wondering why write my own. In reality I have no real reason to write my own I'm just trying to understand how the algorithm is implemented. Especially in my first link, that demo is brilliant. This is what I've come up with //support function.bind - https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Function/bind#Compatibility if (!Function.prototype.bind) { Function.prototype.bind = function (oThis) { if (typeof this !== "function") { // closest thing possible to the ECMAScript 5 internal IsCallable function throw new TypeError("Function.prototype.bind - what is trying to be bound is not callable"); } var aArgs = Array.prototype.slice.call(arguments, 1), fToBind = this, fNOP = function () {}, fBound = function () { return fToBind.apply(this instanceof fNOP ? this : oThis || window, aArgs.concat(Array.prototype.slice.call(arguments))); }; fNOP.prototype = this.prototype; fBound.prototype = new fNOP(); return fBound; }; } (function() { var lastTime = 0; var vendors = ['ms', 'moz', 'webkit', 'o']; for(var x = 0; x < vendors.length && !window.requestAnimationFrame; ++x) { window.requestAnimationFrame = window[vendors[x]+'RequestAnimationFrame']; window.cancelAnimationFrame = window[vendors[x]+'CancelAnimationFrame'] || window[vendors[x]+'CancelRequestAnimationFrame']; } if (!window.requestAnimationFrame) window.requestAnimationFrame = function(callback, element) { var currTime = new Date().getTime(); var timeToCall = Math.max(0, 16 - (currTime - lastTime)); var id = window.setTimeout(function() { callback(currTime + timeToCall); }, timeToCall); lastTime = currTime + timeToCall; return id; }; if (!window.cancelAnimationFrame) window.cancelAnimationFrame = function(id) { clearTimeout(id); }; }()); function Graph(o){ this.options=o; this.vertices={}; this.edges={};//form {vertexID:{edgeID:edge}} } /** *Adds an edge to the graph. If the verticies in this edge are not already in the *graph then they are added */ Graph.prototype.addEdge=function(e){ //if vertex1 and vertex2 doesn't exist in this.vertices add them if(typeof(this.vertices[e.vertex1])==='undefined') this.vertices[e.vertex1]=new Vertex(e.vertex1); if(typeof(this.vertices[e.vertex2])==='undefined') this.vertices[e.vertex2]=new Vertex(e.vertex2); //add the edge if(typeof(this.edges[e.vertex1])==='undefined') this.edges[e.vertex1]={}; this.edges[e.vertex1][e.id]=e; } /** * Add a vertex to the graph. If a vertex with the same ID already exists then * the existing vertex's .data property is replaced with the @param v.data */ Graph.prototype.addVertex=function(v){ if(typeof(this.vertices[v.id])==='undefined') this.vertices[v.id]=v; else this.vertices[v.id].data=v.data; } function Vertex(id,data){ this.id=id; this.data=data?data:{}; //initialize to data.[x|y|z] or generate random number for each this.x = this.data.x?this.data.x:-100 + Math.random()*200; this.y = this.data.y?this.data.y:-100 + Math.random()*200; this.z = this.data.y?this.data.y:-100 + Math.random()*200; //set initial velocity to 0 this.velocity = new Point(0, 0, 0); this.mass=this.data.mass?this.data.mass:Math.random(); this.force=new Point(0,0,0); } function Edge(vertex1ID,vertex2ID){ vertex1ID=vertex1ID?vertex1ID:Math.random() vertex2ID=vertex2ID?vertex2ID:Math.random() this.id=vertex1ID+"->"+vertex2ID; this.vertex1=vertex1ID; this.vertex2=vertex2ID; } function Point(x, y, z) { this.x = x; this.y = y; this.z = z; } Point.prototype.plus=function(p){ this.x +=p.x this.y +=p.y this.z +=p.z } function ForceLayout(o){ this.repulsion = o.repulsion?o.repulsion:200; this.attraction = o.attraction?o.attraction:0.06; this.damping = o.damping?o.damping:0.9; this.graph = o.graph?o.graph:new Graph(); this.total_kinetic_energy =0; this.animationID=-1; } ForceLayout.prototype.draw=function(){ //vertex velocities initialized to (0,0,0) when a vertex is created //vertex positions initialized to random position when created cc=0; do{ this.total_kinetic_energy =0; //for each vertex for(var i in this.graph.vertices){ var thisNode=this.graph.vertices[i]; // running sum of total force on this particular node var netForce=new Point(0,0,0) //for each other node for(var j in this.graph.vertices){ if(thisNode!=this.graph.vertices[j]){ //net-force := net-force + Coulomb_repulsion( this_node, other_node ) netForce.plus(this.CoulombRepulsion( thisNode,this.graph.vertices[j])) } } //for each spring connected to this node for(var k in this.graph.edges[thisNode.id]){ //(this node, node its connected to) //pass id of this node and the node its connected to so hookesattraction //can update the force on both vertices and return that force to be //added to the net force this.HookesAttraction(thisNode.id, this.graph.edges[thisNode.id][k].vertex2 ) } // without damping, it moves forever // this_node.velocity := (this_node.velocity + timestep * net-force) * damping thisNode.velocity.x=(thisNode.velocity.x+thisNode.force.x)*this.damping; thisNode.velocity.y=(thisNode.velocity.y+thisNode.force.y)*this.damping; thisNode.velocity.z=(thisNode.velocity.z+thisNode.force.z)*this.damping; //this_node.position := this_node.position + timestep * this_node.velocity thisNode.x=thisNode.velocity.x; thisNode.y=thisNode.velocity.y; thisNode.z=thisNode.velocity.z; //normalize x,y,z??? //total_kinetic_energy := total_kinetic_energy + this_node.mass * (this_node.velocity)^2 this.total_kinetic_energy +=thisNode.mass*((thisNode.velocity.x+thisNode.velocity.y+thisNode.velocity.z)* (thisNode.velocity.x+thisNode.velocity.y+thisNode.velocity.z)) } cc+=1; }while(this.total_kinetic_energy >0.5) console.log(cc,this.total_kinetic_energy,this.graph) this.cancelAnimation(); } ForceLayout.prototype.HookesAttraction=function(v1ID,v2ID){ var a=this.graph.vertices[v1ID] var b=this.graph.vertices[v2ID] var force=new Point(this.attraction*(b.x - a.x),this.attraction*(b.y - a.y),this.attraction*(b.z - a.z)) // hook's attraction a.force.x += force.x; a.force.y += force.y; a.force.z += force.z; b.force.x += this.attraction*(a.x - b.x); b.force.y += this.attraction*(a.y - b.y); b.force.z += this.attraction*(a.z - b.z); return force; } ForceLayout.prototype.CoulombRepulsion=function(vertex1,vertex2){ //http://en.wikipedia.org/wiki/Coulomb's_law // distance squared = ((x1-x2)*(x1-x2)) + ((y1-y2)*(y1-y2)) + ((z1-z2)*(z1-z2)) var distanceSquared = ( (vertex1.x-vertex2.x)*(vertex1.x-vertex2.x)+ (vertex1.y-vertex2.y)*(vertex1.y-vertex2.y)+ (vertex1.z-vertex2.z)*(vertex1.z-vertex2.z) ); if(distanceSquared==0) distanceSquared = 0.001; var coul = this.repulsion / distanceSquared; return new Point(coul * (vertex1.x-vertex2.x),coul * (vertex1.y-vertex2.y), coul * (vertex1.z-vertex2.z)); } ForceLayout.prototype.animate=function(){ if(this.animating) this.animationID=requestAnimationFrame(this.animate.bind(this)); this.draw(); } ForceLayout.prototype.cancelAnimation=function(){ cancelAnimationFrame(this.animationID); this.animating=false; } ForceLayout.prototype.redraw=function(){ this.animating=true; this.animate(); } $(document).ready(function(){ var g= new Graph(); for(var i=0;i<=100;i++){ var v1=new Vertex(Math.random(), {}) var v2=new Vertex(Math.random(), {}) var e1= new Edge(v1.id,v2.id); g.addEdge(e1); } console.log(g); var l=new ForceLayout({ graph:g }); l.redraw(); });

Read the article

Asynchronous Webcrawling F#, something wrong ?

- by jlezard

Not quite sure if it is ok to do this but, my question is: Is there something wrong with my code ? It doesn't go as fast as I would like, and since I am using lots of async workflows maybe I am doing something wrong. The goal here is to build something that can crawl 20 000 pages in less than an hour. open System open System.Text open System.Net open System.IO open System.Text.RegularExpressions open System.Collections.Generic open System.ComponentModel open Microsoft.FSharp open System.Threading //This is the Parallel.Fs file type ComparableUri ( uri: string ) = inherit System.Uri( uri ) let elts (uri:System.Uri) = uri.Scheme, uri.Host, uri.Port, uri.Segments interface System.IComparable with member this.CompareTo( uri2 ) = compare (elts this) (elts(uri2 :?> ComparableUri)) override this.Equals(uri2) = compare this (uri2 :?> ComparableUri ) = 0 override this.GetHashCode() = 0 ///////////////////////////////////////////////Funtions to retreive html string////////////////////////////// let mutable error = Set.empty<ComparableUri> let mutable visited = Set.empty<ComparableUri> let getHtmlPrimitiveAsyncDelay (delay:int) (uri : ComparableUri) = async{ try let req = (WebRequest.Create(uri)) :?> HttpWebRequest // 'use' is equivalent to ‘using’ in C# for an IDisposable req.UserAgent<-"Mozilla" //Console.WriteLine("Waiting") do! Async.Sleep(delay * 250) let! resp = (req.AsyncGetResponse()) Console.WriteLine(uri.AbsoluteUri+" got response after delay "+string delay) use stream = resp.GetResponseStream() use reader = new StreamReader(stream) let html = reader.ReadToEnd() return html with | _ as ex -> Console.WriteLine( ex.ToString() ) lock error (fun () -> error<- error.Add uri ) lock visited (fun () -> visited<-visited.Add uri ) return "BadUri" } ///////////////////////////////////////////////Active Pattern Matching to retreive href////////////////////////////// let (|Matches|_|) (pat:string) (inp:string) = let m = Regex.Matches(inp, pat) // Note the List.tl, since the first group is always the entirety of the matched string. if m.Count > 0 then Some (List.tail [ for g in m -> g.Value ]) else None let (|Match|_|) (pat:string) (inp:string) = let m = Regex.Match(inp, pat) // Note the List.tl, since the first group is always the entirety of the matched string. if m.Success then Some (List.tail [ for g in m.Groups -> g.Value ]) else None ///////////////////////////////////////////////Find Bad href////////////////////////////// let isEmail (link:string) = link.Contains("@") let isMailto (link:string) = if Seq.length link >=6 then link.[0..5] = "mailto" else false let isJavascript (link:string) = if Seq.length link >=10 then link.[0..9] = "javascript" else false let isBadUri (link:string) = link="BadUri" let isEmptyHttp (link:string) = link="http://" let isFile (link:string)= if Seq.length link >=6 then link.[0..5] = "file:/" else false let containsPipe (link:string) = link.Contains("|") let isAdLink (link:string) = if Seq.length link >=6 then link.[0..5] = "adlink" elif Seq.length link >=9 then link.[0..8] = "http://adLink" else false ///////////////////////////////////////////////Find Bad href////////////////////////////// let getHref (htmlString:string) = let urlPat = "href=\"([^\"]+)" match htmlString with | Matches urlPat urls -> urls |> List.map( fun href -> match href with | Match (urlPat) (link::[]) -> link | _ -> failwith "The href was not in correct format, there was more than one match" ) | _ -> Console.WriteLine( "No links for this page" );[] |> List.filter( fun link -> not(isEmail link) ) |> List.filter( fun link -> not(isMailto link) ) |> List.filter( fun link -> not(isJavascript link) ) |> List.filter( fun link -> not(isBadUri link) ) |> List.filter( fun link -> not(isEmptyHttp link) ) |> List.filter( fun link -> not(isFile link) ) |> List.filter( fun link -> not(containsPipe link) ) |> List.filter( fun link -> not(isAdLink link) ) let treatAjax (href:System.Uri) = let link = href.ToString() let firstPart = (link.Split([|"#"|],System.StringSplitOptions.None)).[0] new Uri(firstPart) //only follow pages with certain extnsion or ones with no exensions let followHref (href:System.Uri) = let valid2 = set[".py"] let valid3 = set[".php";".htm";".asp"] let valid4 = set[".php3";".php4";".php5";".html";".aspx"] let arrLength = href.Segments |> Array.length let lastExtension = (href.Segments).[arrLength-1] let lengthLastExtension = Seq.length lastExtension if (lengthLastExtension <= 3) then not( lastExtension.Contains(".") ) else //test for the 2 case let last4 = lastExtension.[(lengthLastExtension-1)-3..(lengthLastExtension-1)] let isValid2 = valid2|>Seq.exists(fun validEnd -> last4.EndsWith( validEnd) ) if isValid2 then true else if lengthLastExtension <= 4 then not( last4.Contains(".") ) else let last5 = lastExtension.[(lengthLastExtension-1)-4..(lengthLastExtension-1)] let isValid3 = valid3|>Seq.exists(fun validEnd -> last5.EndsWith( validEnd) ) if isValid3 then true else if lengthLastExtension <= 5 then not( last5.Contains(".") ) else let last6 = lastExtension.[(lengthLastExtension-1)-5..(lengthLastExtension-1)] let isValid4 = valid4|>Seq.exists(fun validEnd -> last6.EndsWith( validEnd) ) if isValid4 then true else not( last6.Contains(".") ) && not(lastExtension.[0..5] = "mailto") //Create the correct links / -> add the homepage , make them a comparabel Uri let hrefLinksToUri ( uri:ComparableUri ) (hrefLinks:string list) = hrefLinks |> List.map( fun link -> try if Seq.length link <4 then Some(new Uri( uri, link )) else if link.[0..3] = "http" then Some(new Uri(link)) else Some(new Uri( uri, link )) with | _ as ex -> Console.WriteLine(link); lock error (fun () ->error<-error.Add uri) None ) |> List.filter( fun link -> link.IsSome ) |> List.map( fun o -> o.Value) |> List.map( fun uri -> new ComparableUri( string uri ) ) //Treat uri , removing ajax last part , and only following links specified b Benoit let linksToFollow (hrefUris:ComparableUri list) = hrefUris |>List.map( treatAjax ) |>List.filter( fun link -> followHref link ) |>List.map( fun uri -> new ComparableUri( string uri ) ) |>Set.ofList let needToVisit uri = ( lock visited (fun () -> not( visited.Contains uri) ) ) && (lock error (fun () -> not( error.Contains uri) )) let getLinksToFollowAsyncDelay (delay:int) ( uri: ComparableUri ) = async{ let! links = getHtmlPrimitiveAsyncDelay delay uri lock visited (fun () ->visited<-visited.Add uri) let linksToFollow = getHref links |> hrefLinksToUri uri |> linksToFollow |> Set.filter( needToVisit ) |> Set.map( fun link -> if uri.Authority=link.Authority then link else link ) return linksToFollow } //Add delays if visitng same authority let getDelay(uri:ComparableUri) (authorityDelay:Dictionary<string,int>) = let uriAuthority = uri.Authority let hasAuthority,delay = authorityDelay.TryGetValue(uriAuthority) if hasAuthority then authorityDelay.[uriAuthority] <-delay+1 delay else authorityDelay.Add(uriAuthority,1) 0 let rec getLinksToFollowFromSetAsync maxIteration ( uris: seq<ComparableUri> ) = let authorityDelay = Dictionary<string,int>() if maxIteration = 100 then Console.WriteLine("Finished") else //Unite by authority add delay for those we same authority others ignore let stopwatch= System.Diagnostics.Stopwatch() stopwatch.Start() let newLinks = uris |> Seq.map( fun uri -> let delay = lock authorityDelay (fun () -> getDelay uri authorityDelay ) getLinksToFollowAsyncDelay delay uri ) |> Async.Parallel |> Async.RunSynchronously |> Seq.concat stopwatch.Stop() Console.WriteLine("\n\n\n\n\n\n\nTimeElapse : "+string stopwatch.Elapsed+"\n\n\n\n\n\n\n\n\n") getLinksToFollowFromSetAsync (maxIteration+1) newLinks getLinksToFollowFromSetAsync 0 (seq[ComparableUri( "http://twitter.com/" )]) Console.WriteLine("Finished") Some feedBack would be great ! Thank you (note this is just something I am doing for fun)

Read the article

WordPress not resizing images with Nginx + php-fpm and other issues

- by Julian Fernandes

Recently i setup a Ubuntu 12.04 VPS with 512mb/1ghz CPU, Nginx + php-fpm + Varnish + APC + Percona's MySQL server + CloudFlare Pro for our Ubuntu LoCo Team's WordPress blog. The blog get about 3~4k daily hits, use about 180MB and 8~20% CPU. Everything seems to be working insanely fast... page load is really good and is about 16x faster than any of our competitors... but there is one problem. When we upload a image, WordPress don't resize it, so all we can do it insert the full image in the post. If the imagem have, let's say, 30kb, it resize fine... but if the image have 100kb+, it won't... In nginx error logs i see this: upstream timed out (110: Connection timed out) while reading response header from upstream, client: 150.162.216.64, server: www.ubuntubrsc.com, request: "POST /wp-admin/async-upload.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "www.ubuntubrsc.com", referrer: "http://www.ubuntubrsc.com/wp-admin/media-upload.php?post_id=2668&" It seems to be related with the issue, but i dunno. When that timeout happens, i started to get it when i'm trying to view a post too: upstream timed out (110: Connection timed out) while reading response header from upstream, client: 150.162.216.64, server: www.ubuntubrsc.com, request: "GET /tutoriais-gimp-6-adicionando-aplicando-novos-pinceis.html HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "www.ubuntubrsc.com", referrer: "http://www.ubuntubrsc.com/" And only a restart of php5-fpm fix it. I tryed increasing some timeouts and stuffs but it did not worked, so i guess it's some kind of limitation i did not figured yet. Could someone help me with it, please? /etc/nginx/nginx.conf: user www-data; worker_processes 1; pid /var/run/nginx.pid; events { worker_connections 1024; use epoll; multi_accept on; } http { ## # Basic Settings ## sendfile on; tcp_nopush on; tcp_nodelay off; keepalive_timeout 15; keepalive_requests 2000; types_hash_max_size 2048; server_tokens off; server_name_in_redirect off; open_file_cache max=1000 inactive=300s; open_file_cache_valid 360s; open_file_cache_min_uses 2; open_file_cache_errors off; server_names_hash_bucket_size 64; # server_name_in_redirect off; client_body_buffer_size 128K; client_header_buffer_size 1k; client_max_body_size 2m; large_client_header_buffers 4 8k; client_body_timeout 10m; client_header_timeout 10m; send_timeout 10m; include /etc/nginx/mime.types; default_type application/octet-stream; ## # Logging Settings ## error_log /var/log/nginx/error.log; access_log off; ## # CloudFlare's IPs (uncomment when site goes live) ## set_real_ip_from 204.93.240.0/24; set_real_ip_from 204.93.177.0/24; set_real_ip_from 199.27.128.0/21; set_real_ip_from 173.245.48.0/20; set_real_ip_from 103.22.200.0/22; set_real_ip_from 141.101.64.0/18; set_real_ip_from 108.162.192.0/18; set_real_ip_from 190.93.240.0/20; real_ip_header CF-Connecting-IP; set_real_ip_from 127.0.0.1/32; ## # Gzip Settings ## gzip on; gzip_disable "msie6"; gzip_vary on; gzip_proxied any; gzip_comp_level 9; gzip_min_length 1000; gzip_proxied expired no-cache no-store private auth; gzip_buffers 32 8k; # gzip_http_version 1.1; gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript; ## # nginx-naxsi config ## # Uncomment it if you installed nginx-naxsi ## #include /etc/nginx/naxsi_core.rules; ## # nginx-passenger config ## # Uncomment it if you installed nginx-passenger ## #passenger_root /usr; #passenger_ruby /usr/bin/ruby; ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; } /etc/nginx/fastcgi_params: fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; fastcgi_param SCRIPT_FILENAME $request_filename; fastcgi_param SCRIPT_NAME $fastcgi_script_name; fastcgi_param REQUEST_URI $request_uri; fastcgi_param DOCUMENT_URI $document_uri; fastcgi_param DOCUMENT_ROOT $document_root; fastcgi_param SERVER_PROTOCOL $server_protocol; fastcgi_param GATEWAY_INTERFACE CGI/1.1; fastcgi_param SERVER_SOFTWARE nginx/$nginx_version; fastcgi_param REMOTE_ADDR $remote_addr; fastcgi_param REMOTE_PORT $remote_port; fastcgi_param SERVER_ADDR $server_addr; fastcgi_param SERVER_PORT $server_port; fastcgi_param SERVER_NAME $server_name; fastcgi_param HTTPS $https; fastcgi_send_timeout 180; fastcgi_read_timeout 180; fastcgi_buffer_size 128k; fastcgi_buffers 256 4k; # PHP only, required if PHP was built with --enable-force-cgi-redirect fastcgi_param REDIRECT_STATUS 200; /etc/nginx/sites-avaiable/default: ## # DEFAULT HANDLER # ubuntubrsc.com ## server { listen 8080; # Make site available from main domain server_name www.ubuntubrsc.com; # Root directory root /var/www; index index.php index.html index.htm; include /var/www/nginx.conf; access_log off; location / { try_files $uri $uri/ /index.php?q=$uri&$args; } location = /favicon.ico { log_not_found off; access_log off; } location = /robots.txt { allow all; log_not_found off; access_log off; } location ~ /\. { deny all; access_log off; log_not_found off; } location ~* ^/wp-content/uploads/.*.php$ { deny all; access_log off; log_not_found off; } rewrite /wp-admin$ $scheme://$host$uri/ permanent; error_page 404 = @wordpress; log_not_found off; location @wordpress { include /etc/nginx/fastcgi_params; fastcgi_pass unix:/var/run/php5-fpm.sock; fastcgi_param SCRIPT_NAME /index.php; fastcgi_param SCRIPT_FILENAME $document_root/index.php; } location ~ \.php$ { try_files $uri =404; include /etc/nginx/fastcgi_params; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; if (-f $request_filename) { fastcgi_pass unix:/var/run/php5-fpm.sock; } } } server { listen 8080; server_name ubuntubrsc.* www.ubuntubrsc.net www.ubuntubrsc.org www.ubuntubrsc.com.br www.ubuntubrsc.info www.ubuntubrsc.in; return 301 $scheme://www.ubuntubrsc.com$request_uri; } /var/www/nginx.conf: # BEGIN W3TC Minify cache location ~ /wp-content/w3tc/min.*\.js$ { types {} default_type application/x-javascript; expires modified 31536000s; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; add_header Vary "Accept-Encoding"; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; } location ~ /wp-content/w3tc/min.*\.css$ { types {} default_type text/css; expires modified 31536000s; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; add_header Vary "Accept-Encoding"; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; } location ~ /wp-content/w3tc/min.*js\.gzip$ { gzip off; types {} default_type application/x-javascript; expires modified 31536000s; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; add_header Vary "Accept-Encoding"; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; add_header Content-Encoding gzip; } location ~ /wp-content/w3tc/min.*css\.gzip$ { gzip off; types {} default_type text/css; expires modified 31536000s; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; add_header Vary "Accept-Encoding"; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; add_header Content-Encoding gzip; } # END W3TC Minify cache # BEGIN W3TC Browser Cache gzip on; gzip_types text/css application/x-javascript text/x-component text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon; location ~ \.(css|js|htc)$ { expires 31536000s; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; } location ~ \.(html|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml)$ { expires 3600s; add_header Pragma "public"; add_header Cache-Control "max-age=3600, public, must-revalidate, proxy-revalidate"; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; try_files $uri $uri/ $uri.html /index.php?$args; } location ~ \.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$ { expires 31536000s; add_header Pragma "public"; add_header Cache-Control "max-age=31536000, public, must-revalidate, proxy-revalidate"; add_header X-Powered-By "W3 Total Cache/0.9.2.5b"; } # END W3TC Browser Cache # BEGIN W3TC Minify core rewrite ^/wp-content/w3tc/min/w3tc_rewrite_test$ /wp-content/w3tc/min/index.php?w3tc_rewrite_test=1 last; set $w3tc_enc ""; if ($http_accept_encoding ~ gzip) { set $w3tc_enc .gzip; } if (-f $request_filename$w3tc_enc) { rewrite (.*) $1$w3tc_enc break; } rewrite ^/wp-content/w3tc/min/(.+\.(css|js))$ /wp-content/w3tc/min/index.php?file=$1 last; # END W3TC Minify core # BEGIN W3TC Skip 404 error handling by WordPress for static files if (-f $request_filename) { break; } if (-d $request_filename) { break; } if ($request_uri ~ "(robots\.txt|sitemap(_index)?\.xml(\.gz)?|[a-z0-9_\-]+-sitemap([0-9]+)?\.xml(\.gz)?)") { break; } if ($request_uri ~* \.(css|js|htc|htm|rtf|rtx|svg|svgz|txt|xsd|xsl|xml|asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|wav|wma|wri|xla|xls|xlsx|xlt|xlw|zip)$) { return 404; } # END W3TC Skip 404 error handling by WordPress for static files # BEGIN Better WP Security location ~ /\.ht { deny all; } location ~ wp-config.php { deny all; } location ~ readme.html { deny all; } location ~ readme.txt { deny all; } location ~ /install.php { deny all; } set $susquery 0; set $rule_2 0; set $rule_3 0; rewrite ^wp-includes/(.*).php /not_found last; rewrite ^/wp-admin/includes(.*)$ /not_found last; if ($request_method ~* "^(TRACE|DELETE|TRACK)"){ return 403; } set $rule_0 0; if ($request_method ~ "POST"){ set $rule_0 1; } if ($uri ~ "^(.*)wp-comments-post.php*"){ set $rule_0 2$rule_0; } if ($http_user_agent ~ "^$"){ set $rule_0 4$rule_0; } if ($rule_0 = "421"){ return 403; } if ($args ~* "\.\./") { set $susquery 1; } if ($args ~* "boot.ini") { set $susquery 1; } if ($args ~* "tag=") { set $susquery 1; } if ($args ~* "ftp:") { set $susquery 1; } if ($args ~* "http:") { set $susquery 1; } if ($args ~* "https:") { set $susquery 1; } if ($args ~* "(<|%3C).*script.*(>|%3E)") { set $susquery 1; } if ($args ~* "mosConfig_[a-zA-Z_]{1,21}(=|%3D)") { set $susquery 1; } if ($args ~* "base64_encode") { set $susquery 1; } if ($args ~* "(%24&x)") { set $susquery 1; } if ($args ~* "(\[|\]|$|$|<|>|ê|\"|;|\?|\*|=$)"){ set $susquery 1; } if ($args ~* "("|'|<|>|\|{|||%24&x)"){ set $susquery 1; } if ($args ~* "(%0|%A|%B|%C|%D|%E|%F|127.0)") { set $susquery 1; } if ($args ~* "(globals|encode|localhost|loopback)") { set $susquery 1; } if ($args ~* "(request|select|insert|concat|union|declare)") { set $susquery 1; } if ($http_cookie !~* "wordpress_logged_in_" ) { set $susquery "${susquery}2"; set $rule_2 1; set $rule_3 1; } if ($susquery = 12) { return 403; } # END Better WP Security /etc/php5/fpm/php-fpm.conf: pid = /var/run/php5-fpm.pid error_log = /var/log/php5-fpm.log emergency_restart_threshold = 3 emergency_restart_interval = 1m process_control_timeout = 10s events.mechanism = epoll /etc/php5/fpm/php.ini (only options i changed): open_basedir ="/var/www/" disable_functions = pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,dl,system,shell_exec,fsockopen,parse_ini_file,passthru,popen,proc_open,proc_close,shell_exec,show_source,symlink,proc_close,proc_get_status,proc_nice,proc_open,proc_terminate,shell_exec ,highlight_file,escapeshellcmd,define_syslog_variables,posix_uname,posix_getpwuid,apache_child_terminate,posix_kill,posix_mkfifo,posix_setpgid,posix_setsid,posix_setuid,escapeshellarg,posix_uname,ftp_exec,ftp_connect,ftp_login,ftp_get,ftp_put,ftp_nb_fput,ftp_raw,ftp_rawlist,ini_alter,ini_restore,inject_code,syslog,openlog,define_syslog_variables,apache_setenv,mysql_pconnect,eval,phpAds_XmlRpc,phpA ds_remoteInfo,phpAds_xmlrpcEncode,phpAds_xmlrpcDecode,xmlrpc_entity_decode,fp,fput,virtual,show_source,pclose,readfile,wget expose_php = off max_execution_time = 30 max_input_time = 60 memory_limit = 128M display_errors = Off post_max_size = 2M allow_url_fopen = off default_socket_timeout = 60 APC settings: [APC] apc.enabled = 1 apc.shm_segments = 1 apc.shm_size = 64M apc.optimization = 0 apc.num_files_hint = 4096 apc.ttl = 60 apc.user_ttl = 7200 apc.gc_ttl = 0 apc.cache_by_default = 1 apc.filters = "" apc.mmap_file_mask = "/tmp/apc.XXXXXX" apc.slam_defense = 0 apc.file_update_protection = 2 apc.enable_cli = 0 apc.max_file_size = 10M apc.stat = 1 apc.write_lock = 1 apc.report_autofilter = 0 apc.include_once_override = 0 apc.localcache = 0 apc.localcache.size = 512 apc.coredump_unmap = 0 apc.stat_ctime = 0 /etc/php5/fpm/pool.d/www.conf user = www-data group = www-data listen = /var/run/php5-fpm.sock listen.owner = www-data listen.group = www-data listen.mode = 0666 pm = ondemand pm.max_children = 5 pm.process_idle_timeout = 3s; pm.max_requests = 50 I also started to get 404 errors in front page if i use W3 Total Cache's Page Cache (Disk Enhanced). It worked fine untill somedays ago, and then, out of nowhere, it started to happen. Tonight i will disable my mobile plugin and activate only W3 Total Cache to see if it's a conflict with them... And to finish all this, i have been getting this error: PHP Warning: apc_store(): Unable to allocate memory for pool. in /var/www/wp-content/plugins/w3-total-cache/lib/W3/Cache/Apc.php on line 41 I already modifed my APC settings, but no sucess. So... could anyone help me with those issuees, please? Ooohh... if it helps, i instaled PHP like this: sudo apt-get install php5-fpm php5-suhosin php-apc php5-gd php5-imagick php5-curl And Nginx from the official PPA. Sorry for my bad english and thanks for your time people! (:

Search Results

Search found 443 results on 18 pages for 'concat'.

Page 18/18 | < Previous Page | 14 15 16 17 18

- by Lycaon

- by linkcharger

- by Marc Vollmer

- by Dixin

- by Travis Brown

- by Tauren

- by Ender

- by GrumpyCanuck

- by user1302023

- by user1381537

- by user1056998

- by hap497

- by petermolnar

- by michael

- by michael

- by zcourts

- by jlezard

- by Julian Fernandes

< Previous Page | 14 15 16 17 18