I have an appplication that does a XDocument.Load from a 20mb file and then gets passed to a form to view its contents:

        openFileDialog1.FileName = "";
        if (openFileDialog1.ShowDialog() == DialogResult.OK)
        {
            AuditFile = XDocument.Load(openFileDialog1.FileName);

            fmAuditLogViewer AuditViewer = new fmAuditLogViewer();

            AuditViewer.ReportDocument = AuditFile;
            AuditViewer.Init();
            AuditViewer.ShowDialog();
            AuditViewer.Dispose();

            AuditFile.RemoveNodes();
            AuditFile = null;
        }

In Task Manager I can see the memory being used by my application shoot up when I open this file.
When I have finished viewing this file in my application I call :

myXDocument.RemoveNodes();
myXDocument = null;

However the memory use in Task Manager is still pretty high against my app.

Is the XDocument still being held in memory and can I decrease the memory usage by my app?