C#: Efficiently search a large string for occurences of other strings

Posted by Jon on Stack Overflow See other posts from Stack Overflow or by Jon
Published on 2009-06-23T08:19:08Z Indexed on 2010/04/13 15:03 UTC
Read the original article Hit count: 284

Filed under:

c#

|

strings

Hi,

I'm using C# to continuously search for multiple string "keywords" within large strings, which are >= 4kb. This code is constantly looping, and sleeps aren't cutting down CPU usage enough while maintaining a reasonable speed. The bog-down is the keyword matching method.

I've found a few possibilities, and all of them give similar efficiency.

1) http://tomasp.net/articles/ahocorasick.aspx -I do not have enough keywords for this to be the most efficient algorithm.

2) Regex. Using an instance level, compiled regex. -Provides more functionality than I require, and not quite enough efficiency.

3) String.IndexOf. -I would need to do a "smart" version of this for it provide enough efficiency. Looping through each keyword and calling IndexOf doesn't cut it.

Does anyone know of any algorithms or methods that I can use to attain my goal?

© Stack Overflow or respective owner

Related posts about c#

.NET WebRequest.PreAuthenticate not quite what it sounds like

as seen on West-Wind - Search for 'West-Wind'
I’ve run into the problem a few times now: How to pre-authenticate .NET WebRequest calls doing an HTTP call to the server – essentially send authentication credentials on the very first request instead of waiting for a server challenge first? At first glance this sound like it should be easy:… >>> More
HttpWebRequest and Ignoring SSL Certificate Errors

as seen on West-Wind - Search for 'West-Wind'
Man I can't believe this. I'm still mucking around with OFX servers and it drives me absolutely crazy how some these servers are just so unbelievably misconfigured. I've recently hit three different 3 major brokerages which fail HTTP validation with bad or corrupt certificates at least according to… >>> More
The dynamic Type in C# Simplifies COM Member Access from Visual FoxPro

as seen on West-Wind - Search for 'West-Wind'
I’ve written quite a bit about Visual FoxPro interoperating with .NET in the past both for ASP.NET interacting with Visual FoxPro COM objects as well as Visual FoxPro calling into .NET code via COM Interop. COM Interop with Visual FoxPro has a number of problems but one of them at least got a lot… >>> More
Dynamic Type to do away with Reflection

as seen on West-Wind - Search for 'West-Wind'
The dynamic type in C# 4.0 is a welcome addition to the language. One thing I’ve been doing a lot with it is to remove explicit Reflection code that’s often necessary when you ‘dynamically’ need to walk and object hierarchy. In the past I’ve had a number of ReflectionUtils that used string based expressions… >>> More
Finding a Relative Path in .NET

as seen on West-Wind - Search for 'West-Wind'
Here’s a nice and simple path utility that I’ve needed in a number of applications: I need to find a relative path based on a base path. So if I’m working in a folder called c:\temp\templates\ and I want to find a relative path for c:\temp\templates\subdir\test.txt I want to receive back subdir\test… >>> More

Related posts about strings

References to other strings in strings.xml ?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Is it possible to reference other strings inside of strings.xml ? Something of the form: My string Here it is: android:id="@string/string_one" (If it did exist, there would of course be problems of circular, infinite definitions, etc. to beware of). Thanks. >>> More
Passing List of Strings or Array of strings into Unity Injection Constructor (Config-Based)

as seen on Stack Overflow - Search for 'Stack Overflow'
I cannot seem to get unity working when attempting to pass in an array of strings into a constructor parameter list, while using XML configuration. When I try the following: <typeConfig ...> <constructor ...> <param ... parameterType="System.String[]"> <array>… >>> More
Strings in array are no longer strings after jQuery.each()

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm pretty confused with the behaviour of arrays of strings when I loop them through the jQuery.each() method. Apparently, the strings become jQuery objects inside the callback function. However, I cannot use the this.get() method to obtain the original string; doing so triggers a this.get is not… >>> More
Complete if statement with strings and array of strings

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a page that has 3 variables. They look like this: String[] Headers = new String[] { "Max Width", "Max Length", "Max Height" }; String currentHeader = (String)HttpContext.Current.Request.QueryString["ItemHas"] ?? ""; String checkString = (String)HttpContext.Current.Request.QueryString["ItemIn"]… >>> More
[C++] Index strings by other strings

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I need to index specific strings with other strings and I can't really find a good way to do so. I tried to use tr1::unordered_map, but I'm having some difficulties using it. If someone could tell me what is the best way to do that I'd be really grateful :) I also need to index objects by a… >>> More