Write file need to optimised for heavy traffic part 2

Posted by Clayton Leung on Stack Overflow See other posts from Stack Overflow or by Clayton Leung
Published on 2012-06-04T15:43:38Z Indexed on 2012/06/09 4:40 UTC
Read the original article Hit count: 167

Filed under:
|
|

For anyone interest to see where I come from you can refer to part 1, but it is not necessary.

write file need to optimised for heavy traffic

Below is a snippet of code I have written to capture some financial tick data from the broker API. The code will run without error. I need to optimize the code, because in peak hours the zf_TickEvent method will be call more than 10000 times a second. I use a memorystream to hold the data until it reaches a certain size, then I output it into a text file.

The broker API is only single threaded.

void zf_TickEvent(object sender, ZenFire.TickEventArgs e)
{

    outputString = string.Format("{0},{1},{2},{3},{4}\r\n",
                        e.TimeStamp.ToString(timeFmt),
                        e.Product.ToString(),
                        Enum.GetName(typeof(ZenFire.TickType), e.Type),
                        e.Price,
                        e.Volume);

    fillBuffer(outputString);

}

public class memoryStreamClass
{
    public static MemoryStream ms = new MemoryStream();
}

void fillBuffer(string outputString)
{

    byte[] outputByte = Encoding.ASCII.GetBytes(outputString);

    memoryStreamClass.ms.Write(outputByte, 0, outputByte.Length);

    if (memoryStreamClass.ms.Length > 8192)
    {
        emptyBuffer(memoryStreamClass.ms);
        memoryStreamClass.ms.SetLength(0);
        memoryStreamClass.ms.Position = 0;
    }
}

void emptyBuffer(MemoryStream ms)
{
    FileStream outStream = new FileStream("c:\\test.txt", FileMode.Append);

    ms.WriteTo(outStream);
    outStream.Flush();
    outStream.Close();
}

Question:

  1. Any suggestion to make this even faster? I will try to vary the buffer length but in terms of code structure, is this (almost) the fastest?

  2. When memorystream is filled up and I am emptying it to the file, what would happen to the new data coming in? Do I need to implement a second buffer to hold that data while I am emptying my first buffer? Or is c# smart enough to figure it out?

Thanks for any advice

© Stack Overflow or respective owner

Related posts about c#

Related posts about optimization