VB.NET: Can the .EXE built by VS2005 be deployed as a standalone EXE?

When I change the mode in VS2005 to "Release" and build the solution, the bin\Release directory then contains the solution .EXE file, but also a .pdb, vshost.exe and .xml file. What are these extra files and are they necessary?

I copied the .exe file to another machine and it executed properly, but there was a significant delay when it first executed - thereafter it was like any other program. What is the reason for this, and can it be helped? Is it because the other 3 files in the Release folder are not there with it?