fast load big object graph from DB

Posted by Famos on Stack Overflow See other posts from Stack Overflow or by Famos
Published on 2010-12-21T18:15:46Z Indexed on 2010/12/22 16:54 UTC
Read the original article Hit count: 138

Filed under:
|
|

Hi
I have my own data structure written in C# like:

public class ElectricScheme
{
    public List<Element> Elements { get; set; }
    public List<Net> Nets { get; set; }
}

public class Element
{       
    public string IdName { get; set; }
    public string Func { get; set; }
    public string Name { get; set; }
    public BaseElementType Type { get; set; }
    public List<Pin> Pins { get; set; }
}

public class Pin
{
    public string IdName { get; set; }
    public string Name { get; set; }
    public BasePinType PinType { get; set; }
    public BasePinDirection PinDirection { get; set; }
}

public class Net
{
    public string IdName { get; set; }
    public string Name { get; set; }
    public List<Tuple<Element,Pin>> ConnectionPoints { get; set; }
}

Where Elements count ~19000, each element contain >=3 Pin,
Nets count ~20000, each net contain >=3 pair (Element, Pin)

Parse txt (file size ~17mb) file takes 5 minutes. Serilization / Deserialization by default serializer ~3 minutes. Load from DB 20 minutes and not loaded... I use Entity Framework like


  public ElectricScheme LoadScheme(int schemeId)
  {
    var eScheme = (from s in container.ElectricSchemesSet
                   where s.IdElectricScheme.Equals(schemeId)
                   select s).FirstOrDefault();
    if (eScheme == null) return null;
container.LoadProperty(eScheme, "Elements"); container.LoadProperty(eScheme, "Nets"); container.LoadProperty(eScheme, "Elements.Pins"); return eScheme; }

The problem is dependencies between Element and Pin... (for ~19000 elements ~95000 pins)

Any ideas?

© Stack Overflow or respective owner

Related posts about c#

Related posts about Performance