Search Results

Search found 3768 results on 151 pages for 'lite byte'.

Page 150/151 | < Previous Page | 146 147 148 149 150 151  | Next Page >

  • Why did my flash drive become "read only" and (how) can I fix it?

    - by Bob
    I have a brand new flash drive (one week old) that has become marked as read only, by Windows, Kubuntu and a bootable partitioner. Why did this happen? Is it fixable? If it is, how can I fix this? The problem Firstly, this drive is new. It's certainly not been used enough to die from normal wear and tear, though I would not discount defective components. The drive itself has somehow become locked in a read only state. Windows' Disk management: Diskpart: Generic Flash Disk USB Device Disk ID: 33FA33FA Type : USB Status : Online Path : 0 Target : 0 LUN ID : 0 Location Path : UNAVAILABLE Current Read-only State : Yes Read-only : No Boot Disk : No Pagefile Disk : No Hibernation File Disk : No Crashdump Disk : No Clustered Disk : No What really confuses me is Current Read-only State : Yes and Read-only : No. Attempted solutions So far, I've tried: Formatting it in Windows (in Disk management, the format options are greyed out when right clicking). DiskPart Clean (CLEAN - Clear the configuration information, or all information, off the disk.): DISKPART> clean DiskPart has encountered an error: The media is write protected. See the System Event Log for more information. There was nothing in the event log. Windows command line format >format G: Insert new disk for drive G: and press ENTER when ready... The type of the file system is FAT32. Verifying 7740M Cannot format. This volume is write protected. Windows chkdsk: see below for details Kubuntu fsck (through VirtualBox USB passthrough): see below for details Acronis True Image to format, to convert to GPT, to destroy and rebuild MBR, basically anything: failed (could not write to MBR) Details (and a nice story) Background This was a brand new, generic, 8GB flash drive I wanted to create a multiboot flash drive with. It came formatted as FAT32, though oddly a little larger than most 8 GIGAbyte flash drives I've come across. Approximately 127MB was listed as "used" by Windows. I never discovered why. The end usable space was about what I normally expect from a 8GB drive (approx 7.4 GIBIbytes). I had thrown quite a few Linux distros on, along with a copy of Hiren's. They would all boot perfectly. They were put on with YUMI. When I tried to put the Knoppix DVD on, YUMI added an odd video option to its boot comman which caused Knoppix to boot with a black screen on X. ttys 1 through 6 still worked as text only interfaces. A few days later, I took some time to take that odd video option off, making the boot command match the one that comes with Knoppix. On the attempt to boot, Knoppix reported some form of LZMA corruption. Leading up to the current issue I was thinking the Knoppix files may have been corrupted somehow, so I tried reloading it. The drive was nearly full (45MB free), so I deleted a generic ISO that also was not booting. That went fine. I then went through YUMI to 'uninstall' Knoppix, i.e. delete files and remove from the menus. The files went first, then the menus were cleared successfully. However, the free space was stuck at about 700MB, same as it was before removing Knoppix. In the old Knoppix folder, there was a 0 byte file named KNOPPIX that could not be deleted. I tried reinserting the drive to delete this file - without safely removing, if that made a difference (hey, first time for everything). Running the standard Windows chkdsk scan without /r or /f reported errors found. Running with /r just got it stuck. I decided to give fsck a shot, so I loaded up my Kubuntu VM and attached the drive to it with VirtualBox's USB 2.0 passthrough. I umounted it (/dev/sda1) and ran a fsck. There are differences between boot sector and its backup. I chose No action. It told me FATs differ and asked me to select either the first or second FAT. Whichever I selected, I got a notice of Free cluster summary wrong. If I chose Correct, it gave a list of incorrect file names. To try to fix something, at least, I ran it with the -p option. Halfway through fixing the files, the VM froze - I ended its process about ten minutes later. Cause? My next attempt was to use YUMI, again, to rebuild the whole drive. I used YUMI's built in reformat (to FAT32) option and installed a Kubuntu ISO (700MB). The format was successful, however, the extract and copy of Kubuntu (which YUMI uses a 7zip binary for) froze at about 60% done. After waiting for about fifteen minutes (longer than the 3.5GB Knoppix ISO took last time), I pulled the drive out. The drive at this point was already formatted, SYSLINUX already installed, just waiting on the unpacking of an ISO and the modifying of the boot menus. Plugging it back in, it came up as normal - however, any write action would fail. Disk management reported it as read only. On reconnect, it would come up as normal but a write operation would cause it to go read only again. After a few attempts, it started coming up as read only on insertion. Attempts to fix This is when I ran through the attempts listed above, to try and reformat it in case of a faulty format. However the inability to do so even on a bootable disk indicated something more serious is wrong. chkdsk now reports nothing is wrong, and fsck still reports MBR inconsistencies, but now always chooses first FAT automatically after telling me FATs differ. It still does the same Free cluster summary wrong afterwards. I cannot run with -p anymore because it is now marked as read only. It also managed to corrupt my VM's disk somehow on the first attempt (yes, I'm sure I chose sda, which is mapped to a 7.4GB drive - I triple checked). Thank god for snapshots? I'm just about out of ideas. To my inexperienced mind it looks like something in the drive's firmware set it to read only "permanently" somehow - is there any way to reset this? I don't particularly care about keeping data, considering I've reformatted it twice. Also, fixes that keep me in Windows are better; it reduces the risk of me accidentally nuking my main hard drive. Update 1: I pulled apart the drive out of curiosity. As you can see, there are no obvious write protect switches. There is an IC on the other side, ALCOR branded labelled AU6989HL, if that matters. If there appears to be no way to fix this, I'll probably pull out the (glued down) card and put it in a card reader to check if it's the card or the controller that died. Update 2: I've pulled the card off, Windows detects the drive as a card reader now. The contacts on the card don't appear to be used, and there are several rows of holes on the card itself. Putting it into the card reader only detects about 30MB total, RAW. It's probably either the reader incorrectly reporting the card as faulty (as if a real SD card's write protect was switched on) or a bad contact somewhere. If nothing else, I have a spare 8GB Micro SD card now... as soon as I figure out how to format it as 8GB.

    Read the article

  • How to diagnose failing 6Gbps SATA connection?

    - by whitequark
    I have a Samsung RC530 notebook and OCZ Vertex-3 6Gbps SATA SSD working in AHCI mode. # dmesg | grep DMI SAMSUNG ELECTRONICS CO., LTD. RC530/RC730/RC530/RC730, BIOS 03WD.M008.20110927.PSA 09/27/2011 # lspci -nn 00:1f.2 SATA controller [0106]: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller [8086:1c03] (rev 04) # sdparm -a /dev/sda /dev/sda: ATA OCZ-VERTEX3 2.15 At the boot, the following messages are present in dmesg (I am running Debian wheezy @ Linux 3.2.8): # dmesg | grep -iE '(ata|ahci)' [ 5.179783] ahci 0000:00:1f.2: version 3.0 [ 5.179802] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19 [ 5.179864] ahci 0000:00:1f.2: irq 42 for MSI/MSI-X [ 5.195424] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x5 impl SATA mode [ 5.195429] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst [ 5.195436] ahci 0000:00:1f.2: setting latency timer to 64 [ 5.204035] scsi0 : ahci [ 5.204301] scsi1 : ahci [ 5.204447] scsi2 : ahci [ 5.204592] scsi3 : ahci [ 5.204682] scsi4 : ahci [ 5.204799] scsi5 : ahci [ 5.204917] ata1: SATA max UDMA/133 abar m2048@0xf7c06000 port 0xf7c06100 irq 42 [ 5.204920] ata2: DUMMY [ 5.204923] ata3: SATA max UDMA/133 abar m2048@0xf7c06000 port 0xf7c06200 irq 42 [ 5.204924] ata4: DUMMY [ 5.204926] ata5: DUMMY [ 5.204927] ata6: DUMMY [ 5.523039] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 5.525911] ata3.00: ATAPI: TSSTcorp CDDVDW SN-208BB, SC00, max UDMA/100 [ 5.531006] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 5.533703] ata3.00: configured for UDMA/100 [ 5.542790] ata1.00: ATA-8: OCZ-VERTEX3, 2.15, max UDMA/133 [ 5.542800] ata1.00: 117231408 sectors, multi 16: LBA48 NCQ (depth 31/32), AA [ 5.552751] ata1.00: configured for UDMA/133 [ 5.553050] scsi 0:0:0:0: Direct-Access ATA OCZ-VERTEX3 2.15 PQ: 0 ANSI: 5 [ 5.559621] scsi 2:0:0:0: CD-ROM TSSTcorp CDDVDW SN-208BB SC00 PQ: 0 ANSI: 5 [ 5.564059] sd 0:0:0:0: [sda] 117231408 512-byte logical blocks: (60.0 GB/55.8 GiB) [ 5.564127] sd 0:0:0:0: [sda] Write Protect is off [ 5.564131] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 5.564158] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 5.564582] sda: sda1 [ 5.564810] sd 0:0:0:0: [sda] Attached SCSI disk [ 5.572006] sr0: scsi3-mmc drive: 16x/24x writer dvd-ram cd/rw xa/form2 cdda tray [ 5.572010] cdrom: Uniform CD-ROM driver Revision: 3.20 [ 5.572189] sr 2:0:0:0: Attached scsi CD-ROM sr0 [ 6.717181] ata1.00: exception Emask 0x50 SAct 0x1 SErr 0x280900 action 0x6 frozen [ 6.717238] ata1.00: irq_stat 0x08000000, interface fatal error [ 6.717291] ata1: SError: { UnrecovData HostInt 10B8B BadCRC } [ 6.717342] ata1.00: failed command: READ FPDMA QUEUED [ 6.717395] ata1.00: cmd 60/50:00:20:39:58/00:00:00:00:00/40 tag 0 ncq 40960 in [ 6.717396] res 40/00:00:20:39:58/00:00:00:00:00/40 Emask 0x50 (ATA bus error) [ 6.717503] ata1.00: status: { DRDY } [ 6.717553] ata1: hard resetting link [ 7.033417] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 7.055234] ata1.00: configured for UDMA/133 [ 7.055262] ata1: EH complete [ 7.147280] ata1.00: exception Emask 0x10 SAct 0xf8 SErr 0x280100 action 0x6 frozen [ 7.147340] ata1.00: irq_stat 0x08000000, interface fatal error [ 7.147393] ata1: SError: { UnrecovData 10B8B BadCRC } [ 7.147460] ata1.00: failed command: READ FPDMA QUEUED [ 7.147529] ata1.00: cmd 60/08:18:88:17:41/00:00:02:00:00/40 tag 3 ncq 4096 in [ 7.147531] res 40/00:38:50:99:64/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.147691] ata1.00: status: { DRDY } [ 7.147754] ata1.00: failed command: READ FPDMA QUEUED [ 7.147821] ata1.00: cmd 60/00:20:f8:42:4c/01:00:02:00:00/40 tag 4 ncq 131072 in [ 7.147822] res 40/00:38:50:99:64/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.147977] ata1.00: status: { DRDY } [ 7.148036] ata1.00: failed command: READ FPDMA QUEUED [ 7.148100] ata1.00: cmd 60/50:28:f8:43:4c/00:00:02:00:00/40 tag 5 ncq 40960 in [ 7.148101] res 40/00:38:50:99:64/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.148255] ata1.00: status: { DRDY } [ 7.148315] ata1.00: failed command: READ FPDMA QUEUED [ 7.148379] ata1.00: cmd 60/00:30:50:98:64/01:00:02:00:00/40 tag 6 ncq 131072 in [ 7.148380] res 40/00:38:50:99:64/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.148534] ata1.00: status: { DRDY } [ 7.148593] ata1.00: failed command: READ FPDMA QUEUED [ 7.148657] ata1.00: cmd 60/00:38:50:99:64/01:00:02:00:00/40 tag 7 ncq 131072 in [ 7.148658] res 40/00:38:50:99:64/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.148813] ata1.00: status: { DRDY } [ 7.148875] ata1: hard resetting link [ 7.464842] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 7.486794] ata1.00: configured for UDMA/133 [ 7.486822] ata1: EH complete [ 7.546395] ata1.00: exception Emask 0x10 SAct 0x2f SErr 0x280100 action 0x6 frozen [ 7.546470] ata1.00: irq_stat 0x08000000, interface fatal error [ 7.546531] ata1: SError: { UnrecovData 10B8B BadCRC } [ 7.546588] ata1.00: failed command: READ FPDMA QUEUED [ 7.546648] ata1.00: cmd 60/00:00:e0:4b:61/01:00:02:00:00/40 tag 0 ncq 131072 in [ 7.546649] res 40/00:28:e0:4c:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.546794] ata1.00: status: { DRDY } [ 7.546847] ata1.00: failed command: READ FPDMA QUEUED [ 7.546906] ata1.00: cmd 60/00:08:90:2f:48/01:00:02:00:00/40 tag 1 ncq 131072 in [ 7.546907] res 40/00:28:e0:4c:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.547053] ata1.00: status: { DRDY } [ 7.547106] ata1.00: failed command: READ FPDMA QUEUED [ 7.547165] ata1.00: cmd 60/00:10:90:30:48/01:00:02:00:00/40 tag 2 ncq 131072 in [ 7.547166] res 40/00:28:e0:4c:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.547310] ata1.00: status: { DRDY } [ 7.547363] ata1.00: failed command: READ FPDMA QUEUED [ 7.547422] ata1.00: cmd 60/00:18:50:c7:64/01:00:02:00:00/40 tag 3 ncq 131072 in [ 7.547423] res 40/00:28:e0:4c:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.547568] ata1.00: status: { DRDY } [ 7.547621] ata1.00: failed command: READ FPDMA QUEUED [ 7.547681] ata1.00: cmd 60/00:28:e0:4c:61/01:00:02:00:00/40 tag 5 ncq 131072 in [ 7.547682] res 40/00:28:e0:4c:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.547825] ata1.00: status: { DRDY } [ 7.547882] ata1: hard resetting link [ 7.864408] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 7.886351] ata1.00: configured for UDMA/133 [ 7.886375] ata1: EH complete [ 7.890012] ata1: limiting SATA link speed to 3.0 Gbps [ 7.890016] ata1.00: exception Emask 0x10 SAct 0x7 SErr 0x280100 action 0x6 frozen [ 7.890093] ata1.00: irq_stat 0x08000000, interface fatal error [ 7.890152] ata1: SError: { UnrecovData 10B8B BadCRC } [ 7.890210] ata1.00: failed command: READ FPDMA QUEUED [ 7.890272] ata1.00: cmd 60/00:00:90:33:48/01:00:02:00:00/40 tag 0 ncq 131072 in [ 7.890273] res 40/00:10:e0:4f:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.890418] ata1.00: status: { DRDY } [ 7.890472] ata1.00: failed command: READ FPDMA QUEUED [ 7.890530] ata1.00: cmd 60/00:08:90:34:48/01:00:02:00:00/40 tag 1 ncq 131072 in [ 7.890531] res 40/00:10:e0:4f:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.890672] ata1.00: status: { DRDY } [ 7.890724] ata1.00: failed command: READ FPDMA QUEUED [ 7.890781] ata1.00: cmd 60/78:10:e0:4f:61/00:00:02:00:00/40 tag 2 ncq 61440 in [ 7.890782] res 40/00:10:e0:4f:61/00:00:02:00:00/40 Emask 0x10 (ATA bus error) [ 7.890925] ata1.00: status: { DRDY } [ 7.890981] ata1: hard resetting link [ 8.208021] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320) [ 8.230100] ata1.00: configured for UDMA/133 [ 8.230124] ata1: EH complete Looks like the SATA interface tries to use 6Gbps link, then fails miserably and Linux fallbacks to 3Gbps. This is somewhat fine for me, as the system boots successfully each time and works under high load (cd linux-3.2.8; make -j16). I've also ran memtest86+ and it did not find any errors. What concerns me more is that Grub sometimes takes a long time to load the images and/or fails to load itself completely. The error is consistent and is probablistic: that is, each time I boot I have a certain chance to fail. Actually, I have a slight suspiction on the cause of the failure. Look at the cabling: What kind of engineer does it this way? Nah. Even 1Gbps Ethernet hardly tolerates cables bent over a small angle, and there you have 6Gbps SATA. How cound I determine and fix the cause of errors and/or switch the link to 3Gbps mode permanently?

    Read the article

  • File Access problems with SLES 10 SP2 OES2 SP1

    - by Blackhawk131
    We have identified a couple of repeatable, demonstrable scenarios with unexplained rejected folder access on our servers for Mac users. Hopefully, this can be presented to Novell for a solution. What we did to demonstrate scenario 1; 1. setup a PC and Mac side-by-side 2. login to our server and open up to a central location on both Mac and PC 3. on the PC in that central location create a folder 4. on the Mac in that central location drag the created folder to the Mac desktop, this should work fine, no problem 5. on the PC rename that folder 6. on the Mac drag a file to that renamed folder, this should error with the following message; a. You cannot copy some of these items to the destination because their names are too long for the destination. Do you want to skip copying these items and continue copying the other items? b. Select skip, response is the filename is copied to the location with zero or small byte size. Try opening it and you get file is corrupted error message. What we did to demonstrate scenario 2; 1. setup a PC and Mac side-by-side 2. login to our server and open up to a central location on both Mac and PC 3. on the PC in that central location create a folder then create a subfolder 4. copy some content into the subfolder 5. on the Mac in that central location drag the created top level folder to the Mac desktop, this should work fine, no problem 6. on the PC rename that subfolder 7. on the Mac drag that top level folder to the Mac desktop, this should error on the Mac with the following; a. The operation cannot be completed because you do not have sufficient privileges for b. The operation cannot be completed because you do not have sufficient privileges for 8. on the Mac, if you open that subfolder you can see the file copied in step 4 above but, you can not open that file, you get the following message if you try; a. There was an error opening this document. You do not have permission to open this file. 9. on the PC drag some content into the top level folder 10. on the Mac you can open that file directly from the server or copy it locally, no problem, however-the subfolder is still corrupted or locked, whichever 11. on the PC rename the top level folder 12. on the Mac that same file just opened in step 10 above is now not accessible, get the following message; a. The document could not be opened. I have observed some variances in the above. For instance, a change on the PC side may take a moment before you can observer or act on the Mac side - kind of like the server is slow to respond. Also, the error message may vary. However, the key is once a folder, or subfolder, gets renamed by a PC, Mac problems commence. The solution is to create a new folder from a PC and copy the contents of the corrupted folder to the new folder and not rename the folder name. This has to be done on a PC because the corrupted folder is not accessible by a Mac user. Another problem that dovetails with the above is that we know certain characters are not allowed for PC folder or filenames. If a Mac user creates a folder with a slash in the file name, from the PC the user does not see that slash in the name. As soon as the PC user copies a file to that folder, the Mac user is locked from that folder. Will get the following error message; - Sorry, the operation could not be completed because an unexpected error occurred. - (Error code - 50) In addition to the above mentioned character issue with folders, the problem is more evil with filenames. If, for example, you create a file with a slash in the filename on a Mac and copy it to the server you will get the following error message; - You cannot copy some of these items to the destination because their names are too long for the destination. Do you want to skip copying these items and continue copying the other items? Select either Stop or Skip buttons. It does not matter which button is selected. The file name gets copied to the destination location at a reduced size. Depending on the file type, the icon associated with the file may or may not be present. Furthermore, if you open that file on the server you will get the following message; - Couldnt open the file. It may be corrupt or a file format that doesnt recognize. From the users perspective, if they are not observant of the icon or file size, they may disregard the error message and think their file has copied as intended. Only later do they discover the file is corrupt if they open that file. I want to make a note on this problem. It is the PC causing the issue. You can change folder and file names all day on a MAC and you don't have a problem as long as a character is not the issue. Once you change the file name or folder name from a PC the entire folder structure from that level down is corrupted. But it has to be resolved from a PC by creating a new folder and copying the contents to the new folder like stated above. Is something not configured correctly? SUSE Linux Enterprise Server 10 (x86_64) VERSION = 10 PATCHLEVEL = 2 LSB_VERSION="core-2.0-noarch:core-3.0-noarch:core-2.0-x86_64:core-3.0-x86_64" Novell Open Enterprise Server 2.0.1 (x86_64) VERSION = 2.0.1 PATCHLEVEL = 1 BUILD Note: We use Novell clients on all windows systems to connect to the servers for file access and network storage. We use AFP to allow OSx systems to connect to servers.

    Read the article

  • Backing up my data causes my server to crash using Symantec Backup Exec 12, or How I Came to Loathe Irony

    - by Kyle Noland
    I have a Dell PowerEdge 2850 running Windows Server 2003. It is the primary file server for one of my clients. I have another server also running Windows Server 2003 that acts as the core media server for Symantec Backup Exec 12. I recently upgraded from Backup Exec 11d to 12. This upgrade was necessary because we also just upgraded from Exchange 2003 to Exchange 2007. After the upgrade I had to push-install the new version 12 Backup Exec Remote Agents to each of the servers I am backing up (about 6 total). 5 of my servers are doing just fine, faithfully completing backups every night. My file server routinely crashes. Observations: When the server crashes, it does not blue screen, it just locks up completely. Even the mouse is unresponsive. If you leave the server locked up long enough, it will eventually reboot itself and hang on the Windows splash screen. There is absolutely zero useful Event Viewer evidence of a problem. The logs go from routine logging to an Unexplained Shutdown Event the next morning when I have to hard reset the server to get it to boot. 90% of the time the server does not boot cleanly, it hangs on the Windows splash screen. I don't have any light to shed here. When the server hangs all I can do is hard reset it and try again. Even after a successful boot and chkdsk /r operation, if you reboot the machine, you have a 90% chance it won't back up again cleanly. The back story: This server started crashing during nightly backups about a month ago. I tried everything I could think of to troubleshoot the problem and eventually had to give up because I could not keep coming to the office at 4 AM to try to get the server back online. One Friday I got lucky and the server stayed up for its entire full backup. I took this opportunity to restore the full backup to a temporary server I set up and switched all my users to the temporary. Then I reloaded the ailing file server. I kept all my users on the temporary file server for about 3 weeks. I installed the same Backup Exec Remote Agent and Trend Micro A/V client on the temporary server that I was using on the regular file server. During this time, I had absolutely no problems backing up the temporary server. I tested the reloaded file server extensively. I rebooted the server once an hour every day for 3 weeks trying to make it fail. It never did. I felt confident that the reload was the answer to my problems. I moved all of the data from the temporary server back to the regular server. I got 3 nightly backups out of it before it locked up again and started the familiar failure to boot cleanly behavior. This weekend I decided to monitor the file server through the entire backup job. I RDPd into the file server and also into the server running Backup Exec. On the file server I opened the Task Manager so I could view the processes and watch CPU and memory usage. Everything was running smoothly for about 60GB worth of backup. Then I noticed that the byte count of the backup job in Backup Exec had stopped progressing. I looked back over at my RDP session into the file server, and I was getting real time updates about CPU and memory usage still - both nearly 0%, which is unusual. Backups usually hover around 40% usage for the duration of the backup job. Let me reiterate this point: The screen was refreshing and I was getting real time Task Manager updates - until I clicked on the Start menu. The screen went black and the server locked up. In truth, I think the server had already locked up, the video card just hadn't figured it out yet. I went back into my bag of trick: driving to the office and hard reseting the server over and over again when it hangs up at the Windows splash screen. I did this for 2 hours without getting a successful boot. I started panicking because I did not have a decent backup to use to get everything back onto the working temporary file server. Once I exhausted everything I knew to do, I took a deep breath, booted to the Windows Server 2003 CD and performed a repair installation of Windows. The server came back up fine, with all of my data intact. I can now reboot the server at will and it will come back up cleanly. The problem is that I'm afraid as soon as I try to back that data up again I will back at square one. So let me sum things up: Here is what I've done so far to troubleshoot this server: Deleted and recreated the RAID 5 sets. Initialized the drives. Reloaded the server with a fresh Server 2003 install. Confirmed with Dell that I have installed the latest, Dell approved BIOS and NIC drivers. Uninstalled / reinstalled the Backup Exec Remote Agent. Uninstalled the Trend Micro A/V client. Configured the server not to reboot itself after a blue screen so I can see any stop error. I used to think the server was blue screening, but since I enabled this setting I now know that the server just completely locks up. Run chkdsk /r from the Windows Recovery Console. Several errors were found and corrected, but did not help my problem. Help confirm or deny the following assumptions: There are two problems at work here. Why the server is locking up in the first place, and why the server won't boot cleanly after a lockup. This is ultimately a software problem. The server works fine and can be rebooted cleanly all day long - until the first lockup - following a fresh OS load or even a Repair installation. This is not a problem with Backup Exec in general. All of my other servers back up just fine. For the record, all of the other servers run Server 2003, and some of them house more data than the file server in question here. Any help is appreciated. The irony is almost too much to bear. Backing up my data is what is jeopardizing it.

    Read the article

  • WCF WS-Security and WSE Nonce Authentication

    - by Rick Strahl
    WCF makes it fairly easy to access WS-* Web Services, except when you run into a service format that it doesn't support. Even then WCF provides a huge amount of flexibility to make the service clients work, however finding the proper interfaces to make that happen is not easy to discover and for the most part undocumented unless you're lucky enough to run into a blog, forum or StackOverflow post on the matter. This is definitely true for the Password Nonce as part of the WS-Security/WSE protocol, which is not natively supported in WCF. Specifically I had a need to create a WCF message on the client that includes a WS-Security header that looks like this from their spec document:<soapenv:Header> <wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <wsse:UsernameToken wsu:Id="UsernameToken-8" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <wsse:Username>TeStUsErNaMe1</wsse:Username> <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >TeStPaSsWoRd1</wsse:Password> <wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" >f8nUe3YupTU5ISdCy3X9Gg==</wsse:Nonce> <wsu:Created>2011-05-04T19:01:40.981Z</wsu:Created> </wsse:UsernameToken> </wsse:Security> </soapenv:Header> Specifically, the Nonce and Created keys are what WCF doesn't create or have a built in formatting for. Why is there a nonce? My first thought here was WTF? The username and password are there in clear text, what does the Nonce accomplish? The Nonce and created keys are are part of WSE Security specification and are meant to allow the server to detect and prevent replay attacks. The hashed nonce should be unique per request which the server can store and check for before running another request thus ensuring that a request is not replayed with exactly the same values. Basic ServiceUtl Import - not much Luck The first thing I did when I imported this service with a service reference was to simply import it as a Service Reference. The Add Service Reference import automatically detects that WS-Security is required and appropariately adds the WS-Security to the basicHttpBinding in the config file:<?xml version="1.0" encoding="utf-8" ?> <configuration> <system.serviceModel> <bindings> <basicHttpBinding> <binding name="RealTimeOnlineSoapBinding"> <security mode="Transport" /> </binding> <binding name="RealTimeOnlineSoapBinding1" /> </basicHttpBinding> </bindings> <client> <endpoint address="https://notarealurl.com:443/services/RealTimeOnline" binding="basicHttpBinding" bindingConfiguration="RealTimeOnlineSoapBinding" contract="RealTimeOnline.RealTimeOnline" name="RealTimeOnline" /> </client> </system.serviceModel> </configuration> If if I run this as is using code like this:var client = new RealTimeOnlineClient(); client.ClientCredentials.UserName.UserName = "TheUsername"; client.ClientCredentials.UserName.Password = "ThePassword"; … I get nothing in terms of WS-Security headers. The request is sent, but the the binding expects transport level security to be applied, rather than message level security. To fix this so that a WS-Security message header is sent the security mode can be changed to: <security mode="TransportWithMessageCredential" /> Now if I re-run I at least get a WS-Security header which looks like this:<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <u:Timestamp u:Id="_0"> <u:Created>2012-11-24T02:55:18.011Z</u:Created> <u:Expires>2012-11-24T03:00:18.011Z</u:Expires> </u:Timestamp> <o:UsernameToken u:Id="uuid-18c215d4-1106-40a5-8dd1-c81fdddf19d3-1"> <o:Username>TheUserName</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >ThePassword</o:Password> </o:UsernameToken> </o:Security> </s:Header> Closer! Now the WS-Security header is there along with a timestamp field (which might not be accepted by some WS-Security expecting services), but there's no Nonce or created timestamp as required by my original service. Using a CustomBinding instead My next try was to go with a CustomBinding instead of basicHttpBinding as it allows a bit more control over the protocol and transport configurations for the binding. Specifically I can explicitly specify the message protocol(s) used. Using configuration file settings here's what the config file looks like:<?xml version="1.0"?> <configuration> <system.serviceModel> <bindings> <customBinding> <binding name="CustomSoapBinding"> <security includeTimestamp="false" authenticationMode="UserNameOverTransport" defaultAlgorithmSuite="Basic256" requireDerivedKeys="false" messageSecurityVersion="WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10"> </security> <textMessageEncoding messageVersion="Soap11"></textMessageEncoding> <httpsTransport maxReceivedMessageSize="2000000000"/> </binding> </customBinding> </bindings> <client> <endpoint address="https://notrealurl.com:443/services/RealTimeOnline" binding="customBinding" bindingConfiguration="CustomSoapBinding" contract="RealTimeOnline.RealTimeOnline" name="RealTimeOnline" /> </client> </system.serviceModel> <startup> <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> </startup> </configuration> This ends up creating a cleaner header that's missing the timestamp field which can cause some services problems. The WS-Security header output generated with the above looks like this:<s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <o:UsernameToken u:Id="uuid-291622ca-4c11-460f-9886-ac1c78813b24-1"> <o:Username>TheUsername</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText" >ThePassword</o:Password> </o:UsernameToken> </o:Security> </s:Header> This is closer as it includes only the username and password. The key here is the protocol for WS-Security:messageSecurityVersion="WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10" which explicitly specifies the protocol version. There are several variants of this specification but none of them seem to support the nonce unfortunately. This protocol does allow for optional omission of the Nonce and created timestamp provided (which effectively makes those keys optional). With some services I tried that requested a Nonce just using this protocol actually worked where the default basicHttpBinding failed to connect, so this is a possible solution for access to some services. Unfortunately for my target service that was not an option. The nonce has to be there. Creating Custom ClientCredentials As it turns out WCF doesn't have support for the Digest Nonce as part of WS-Security, and so as far as I can tell there's no way to do it just with configuration settings. I did a bunch of research on this trying to find workarounds for this, and I did find a couple of entries on StackOverflow as well as on the MSDN forums. However, none of these are particularily clear and I ended up using bits and pieces of several of them to arrive at a working solution in the end. http://stackoverflow.com/questions/896901/wcf-adding-nonce-to-usernametoken http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/4df3354f-0627-42d9-b5fb-6e880b60f8ee The latter forum message is the more useful of the two (the last message on the thread in particular) and it has most of the information required to make this work. But it took some experimentation for me to get this right so I'll recount the process here maybe a bit more comprehensively. In order for this to work a number of classes have to be overridden: ClientCredentials ClientCredentialsSecurityTokenManager WSSecurityTokenizer The idea is that we need to create a custom ClientCredential class to hold the custom properties so they can be set from the UI or via configuration settings. The TokenManager and Tokenizer are mainly required to allow the custom credentials class to flow through the WCF pipeline and eventually provide custom serialization. Here are the three classes required and their full implementations:public class CustomCredentials : ClientCredentials { public CustomCredentials() { } protected CustomCredentials(CustomCredentials cc) : base(cc) { } public override System.IdentityModel.Selectors.SecurityTokenManager CreateSecurityTokenManager() { return new CustomSecurityTokenManager(this); } protected override ClientCredentials CloneCore() { return new CustomCredentials(this); } } public class CustomSecurityTokenManager : ClientCredentialsSecurityTokenManager { public CustomSecurityTokenManager(CustomCredentials cred) : base(cred) { } public override System.IdentityModel.Selectors.SecurityTokenSerializer CreateSecurityTokenSerializer(System.IdentityModel.Selectors.SecurityTokenVersion version) { return new CustomTokenSerializer(System.ServiceModel.Security.SecurityVersion.WSSecurity11); } } public class CustomTokenSerializer : WSSecurityTokenSerializer { public CustomTokenSerializer(SecurityVersion sv) : base(sv) { } protected override void WriteTokenCore(System.Xml.XmlWriter writer, System.IdentityModel.Tokens.SecurityToken token) { UserNameSecurityToken userToken = token as UserNameSecurityToken; string tokennamespace = "o"; DateTime created = DateTime.Now; string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ"); // unique Nonce value - encode with SHA-1 for 'randomness' // in theory the nonce could just be the GUID by itself string phrase = Guid.NewGuid().ToString(); var nonce = GetSHA1String(phrase); // in this case password is plain text // for digest mode password needs to be encoded as: // PasswordAsDigest = Base64(SHA-1(Nonce + Created + Password)) // and profile needs to change to //string password = GetSHA1String(nonce + createdStr + userToken.Password); string password = userToken.Password; writer.WriteRaw(string.Format( "<{0}:UsernameToken u:Id=\"" + token.Id + "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" + "<{0}:Username>" + userToken.UserName + "</{0}:Username>" + "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\">" + password + "</{0}:Password>" + "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" + nonce + "</{0}:Nonce>" + "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace)); } protected string GetSHA1String(string phrase) { SHA1CryptoServiceProvider sha1Hasher = new SHA1CryptoServiceProvider(); byte[] hashedDataBytes = sha1Hasher.ComputeHash(Encoding.UTF8.GetBytes(phrase)); return Convert.ToBase64String(hashedDataBytes); } } Realistically only the CustomTokenSerializer has any significant code in. The code there deals with actually serializing the custom credentials using low level XML semantics by writing output into an XML writer. I can't take credit for this code - most of the code comes from the MSDN forum post mentioned earlier - I made a few adjustments to simplify the nonce generation and also added some notes to allow for PasswordDigest generation. Per spec the nonce is nothing more than a unique value that's supposed to be 'random'. I'm thinking that this value can be any string that's unique and a GUID on its own probably would have sufficed. Comments on other posts that GUIDs can be potentially guessed are highly exaggerated to say the least IMHO. To satisfy even that aspect though I added the SHA1 encryption and binary decoding to give a more random value that would be impossible to 'guess'. The original example from the forum post used another level of encoding and decoding to string in between - but that really didn't accomplish anything but extra overhead. The header output generated from this looks like this:<s:Header> <o:Security s:mustUnderstand="1" xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"> <o:UsernameToken u:Id="uuid-f43d8b0d-0ebb-482e-998d-f544401a3c91-1" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"> <o:Username>TheUsername</o:Username> <o:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">ThePassword</o:Password> <o:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" >PjVE24TC6HtdAnsf3U9c5WMsECY=</o:Nonce> <u:Created>2012-11-23T07:10:04.670Z</u:Created> </o:UsernameToken> </o:Security> </s:Header> which is exactly as it should be. Password Digest? In my case the password is passed in plain text over an SSL connection, so there's no digest required so I was done with the code above. Since I don't have a service handy that requires a password digest,  I had no way of testing the code for the digest implementation, but here is how this is likely to work. If you need to pass a digest encoded password things are a little bit trickier. The password type namespace needs to change to: http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#Digest and then the password value needs to be encoded. The format for password digest encoding is this: Base64(SHA-1(Nonce + Created + Password)) and it can be handled in the code above with this code (that's commented in the snippet above): string password = GetSHA1String(nonce + createdStr + userToken.Password); The entire WriteTokenCore method for digest code looks like this:protected override void WriteTokenCore(System.Xml.XmlWriter writer, System.IdentityModel.Tokens.SecurityToken token) { UserNameSecurityToken userToken = token as UserNameSecurityToken; string tokennamespace = "o"; DateTime created = DateTime.Now; string createdStr = created.ToString("yyyy-MM-ddThh:mm:ss.fffZ"); // unique Nonce value - encode with SHA-1 for 'randomness' // in theory the nonce could just be the GUID by itself string phrase = Guid.NewGuid().ToString(); var nonce = GetSHA1String(phrase); string password = GetSHA1String(nonce + createdStr + userToken.Password); writer.WriteRaw(string.Format( "<{0}:UsernameToken u:Id=\"" + token.Id + "\" xmlns:u=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\">" + "<{0}:Username>" + userToken.UserName + "</{0}:Username>" + "<{0}:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#Digest\">" + password + "</{0}:Password>" + "<{0}:Nonce EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\">" + nonce + "</{0}:Nonce>" + "<u:Created>" + createdStr + "</u:Created></{0}:UsernameToken>", tokennamespace)); } I had no service to connect to to try out Digest auth - if you end up needing it and get it to work please drop a comment… How to use the custom Credentials The easiest way to use the custom credentials is to create the client in code. Here's a factory method I use to create an instance of my service client:  public static RealTimeOnlineClient CreateRealTimeOnlineProxy(string url, string username, string password) { if (string.IsNullOrEmpty(url)) url = "https://notrealurl.com:443/cows/services/RealTimeOnline"; CustomBinding binding = new CustomBinding(); var security = TransportSecurityBindingElement.CreateUserNameOverTransportBindingElement(); security.IncludeTimestamp = false; security.DefaultAlgorithmSuite = SecurityAlgorithmSuite.Basic256; security.MessageSecurityVersion = MessageSecurityVersion.WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10; var encoding = new TextMessageEncodingBindingElement(); encoding.MessageVersion = MessageVersion.Soap11; var transport = new HttpsTransportBindingElement(); transport.MaxReceivedMessageSize = 20000000; // 20 megs binding.Elements.Add(security); binding.Elements.Add(encoding); binding.Elements.Add(transport); RealTimeOnlineClient client = new RealTimeOnlineClient(binding, new EndpointAddress(url)); // to use full client credential with Nonce uncomment this code: // it looks like this might not be required - the service seems to work without it client.ChannelFactory.Endpoint.Behaviors.Remove<System.ServiceModel.Description.ClientCredentials>(); client.ChannelFactory.Endpoint.Behaviors.Add(new CustomCredentials()); client.ClientCredentials.UserName.UserName = username; client.ClientCredentials.UserName.Password = password; return client; } This returns a service client that's ready to call other service methods. The key item in this code is the ChannelFactory endpoint behavior modification that that first removes the original ClientCredentials and then adds the new one. The ClientCredentials property on the client is read only and this is the way it has to be added.   Summary It's a bummer that WCF doesn't suport WSE Security authentication with nonce values out of the box. From reading the comments in posts/articles while I was trying to find a solution, I found that this feature was omitted by design as this protocol is considered unsecure. While I agree that plain text passwords are rarely a good idea even if they go over secured SSL connection as WSE Security does, there are unfortunately quite a few services (mosly Java services I suspect) that use this protocol. I've run into this twice now and trying to find a solution online I can see that this is not an isolated problem - many others seem to have struggled with this. It seems there are about a dozen questions about this on StackOverflow all with varying incomplete answers. Hopefully this post provides a little more coherent content in one place. Again I marvel at WCF and its breadth of support for protocol features it has in a single tool. And even when it can't handle something there are ways to get it working via extensibility. But at the same time I marvel at how freaking difficult it is to arrive at these solutions. I mean there's no way I could have ever figured this out on my own. It takes somebody working on the WCF team or at least being very, very intricately involved in the innards of WCF to figure out the interconnection of the various objects to do this from scratch. Luckily this is an older problem that has been discussed extensively online and I was able to cobble together a solution from the online content. I'm glad it worked out that way, but it feels dirty and incomplete in that there's a whole learning path that was omitted to get here… Man am I glad I'm not dealing with SOAP services much anymore. REST service security - even when using some sort of federation is a piece of cake by comparison :-) I'm sure once standards bodies gets involved we'll be right back in security standard hell…© Rick Strahl, West Wind Technologies, 2005-2012Posted in WCF  Web Services   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

    Read the article

  • PTLQueue : a scalable bounded-capacity MPMC queue

    - by Dave
    Title: Fast concurrent MPMC queue -- I've used the following concurrent queue algorithm enough that it warrants a blog entry. I'll sketch out the design of a fast and scalable multiple-producer multiple-consumer (MPSC) concurrent queue called PTLQueue. The queue has bounded capacity and is implemented via a circular array. Bounded capacity can be a useful property if there's a mismatch between producer rates and consumer rates where an unbounded queue might otherwise result in excessive memory consumption by virtue of the container nodes that -- in some queue implementations -- are used to hold values. A bounded-capacity queue can provide flow control between components. Beware, however, that bounded collections can also result in resource deadlock if abused. The put() and take() operators are partial and wait for the collection to become non-full or non-empty, respectively. Put() and take() do not allocate memory, and are not vulnerable to the ABA pathologies. The PTLQueue algorithm can be implemented equally well in C/C++ and Java. Partial operators are often more convenient than total methods. In many use cases if the preconditions aren't met, there's nothing else useful the thread can do, so it may as well wait via a partial method. An exception is in the case of work-stealing queues where a thief might scan a set of queues from which it could potentially steal. Total methods return ASAP with a success-failure indication. (It's tempting to describe a queue or API as blocking or non-blocking instead of partial or total, but non-blocking is already an overloaded concurrency term. Perhaps waiting/non-waiting or patient/impatient might be better terms). It's also trivial to construct partial operators by busy-waiting via total operators, but such constructs may be less efficient than an operator explicitly and intentionally designed to wait. A PTLQueue instance contains an array of slots, where each slot has volatile Turn and MailBox fields. The array has power-of-two length allowing mod/div operations to be replaced by masking. We assume sensible padding and alignment to reduce the impact of false sharing. (On x86 I recommend 128-byte alignment and padding because of the adjacent-sector prefetch facility). Each queue also has PutCursor and TakeCursor cursor variables, each of which should be sequestered as the sole occupant of a cache line or sector. You can opt to use 64-bit integers if concerned about wrap-around aliasing in the cursor variables. Put(null) is considered illegal, but the caller or implementation can easily check for and convert null to a distinguished non-null proxy value if null happens to be a value you'd like to pass. Take() will accordingly convert the proxy value back to null. An advantage of PTLQueue is that you can use atomic fetch-and-increment for the partial methods. We initialize each slot at index I with (Turn=I, MailBox=null). Both cursors are initially 0. All shared variables are considered "volatile" and atomics such as CAS and AtomicFetchAndIncrement are presumed to have bidirectional fence semantics. Finally T is the templated type. I've sketched out a total tryTake() method below that allows the caller to poll the queue. tryPut() has an analogous construction. Zebra stripping : alternating row colors for nice-looking code listings. See also google code "prettify" : https://code.google.com/p/google-code-prettify/ Prettify is a javascript module that yields the HTML/CSS/JS equivalent of pretty-print. -- pre:nth-child(odd) { background-color:#ff0000; } pre:nth-child(even) { background-color:#0000ff; } border-left: 11px solid #ccc; margin: 1.7em 0 1.7em 0.3em; background-color:#BFB; font-size:12px; line-height:65%; " // PTLQueue : Put(v) : // producer : partial method - waits as necessary assert v != null assert Mask = 1 && (Mask & (Mask+1)) == 0 // Document invariants // doorway step // Obtain a sequence number -- ticket // As a practical concern the ticket value is temporally unique // The ticket also identifies and selects a slot auto tkt = AtomicFetchIncrement (&PutCursor, 1) slot * s = &Slots[tkt & Mask] // waiting phase : // wait for slot's generation to match the tkt value assigned to this put() invocation. // The "generation" is implicitly encoded as the upper bits in the cursor // above those used to specify the index : tkt div (Mask+1) // The generation serves as an epoch number to identify a cohort of threads // accessing disjoint slots while s-Turn != tkt : Pause assert s-MailBox == null s-MailBox = v // deposit and pass message Take() : // consumer : partial method - waits as necessary auto tkt = AtomicFetchIncrement (&TakeCursor,1) slot * s = &Slots[tkt & Mask] // 2-stage waiting : // First wait for turn for our generation // Acquire exclusive "take" access to slot's MailBox field // Then wait for the slot to become occupied while s-Turn != tkt : Pause // Concurrency in this section of code is now reduced to just 1 producer thread // vs 1 consumer thread. // For a given queue and slot, there will be most one Take() operation running // in this section. // Consumer waits for producer to arrive and make slot non-empty // Extract message; clear mailbox; advance Turn indicator // We have an obvious happens-before relation : // Put(m) happens-before corresponding Take() that returns that same "m" for T v = s-MailBox if v != null : s-MailBox = null ST-ST barrier s-Turn = tkt + Mask + 1 // unlock slot to admit next producer and consumer return v Pause tryTake() : // total method - returns ASAP with failure indication for auto tkt = TakeCursor slot * s = &Slots[tkt & Mask] if s-Turn != tkt : return null T v = s-MailBox // presumptive return value if v == null : return null // ratify tkt and v values and commit by advancing cursor if CAS (&TakeCursor, tkt, tkt+1) != tkt : continue s-MailBox = null ST-ST barrier s-Turn = tkt + Mask + 1 return v The basic idea derives from the Partitioned Ticket Lock "PTL" (US20120240126-A1) and the MultiLane Concurrent Bag (US8689237). The latter is essentially a circular ring-buffer where the elements themselves are queues or concurrent collections. You can think of the PTLQueue as a partitioned ticket lock "PTL" augmented to pass values from lock to unlock via the slots. Alternatively, you could conceptualize of PTLQueue as a degenerate MultiLane bag where each slot or "lane" consists of a simple single-word MailBox instead of a general queue. Each lane in PTLQueue also has a private Turn field which acts like the Turn (Grant) variables found in PTL. Turn enforces strict FIFO ordering and restricts concurrency on the slot mailbox field to at most one simultaneous put() and take() operation. PTL uses a single "ticket" variable and per-slot Turn (grant) fields while MultiLane has distinct PutCursor and TakeCursor cursors and abstract per-slot sub-queues. Both PTL and MultiLane advance their cursor and ticket variables with atomic fetch-and-increment. PTLQueue borrows from both PTL and MultiLane and has distinct put and take cursors and per-slot Turn fields. Instead of a per-slot queues, PTLQueue uses a simple single-word MailBox field. PutCursor and TakeCursor act like a pair of ticket locks, conferring "put" and "take" access to a given slot. PutCursor, for instance, assigns an incoming put() request to a slot and serves as a PTL "Ticket" to acquire "put" permission to that slot's MailBox field. To better explain the operation of PTLQueue we deconstruct the operation of put() and take() as follows. Put() first increments PutCursor obtaining a new unique ticket. That ticket value also identifies a slot. Put() next waits for that slot's Turn field to match that ticket value. This is tantamount to using a PTL to acquire "put" permission on the slot's MailBox field. Finally, having obtained exclusive "put" permission on the slot, put() stores the message value into the slot's MailBox. Take() similarly advances TakeCursor, identifying a slot, and then acquires and secures "take" permission on a slot by waiting for Turn. Take() then waits for the slot's MailBox to become non-empty, extracts the message, and clears MailBox. Finally, take() advances the slot's Turn field, which releases both "put" and "take" access to the slot's MailBox. Note the asymmetry : put() acquires "put" access to the slot, but take() releases that lock. At any given time, for a given slot in a PTLQueue, at most one thread has "put" access and at most one thread has "take" access. This restricts concurrency from general MPMC to 1-vs-1. We have 2 ticket locks -- one for put() and one for take() -- each with its own "ticket" variable in the form of the corresponding cursor, but they share a single "Grant" egress variable in the form of the slot's Turn variable. Advancing the PutCursor, for instance, serves two purposes. First, we obtain a unique ticket which identifies a slot. Second, incrementing the cursor is the doorway protocol step to acquire the per-slot mutual exclusion "put" lock. The cursors and operations to increment those cursors serve double-duty : slot-selection and ticket assignment for locking the slot's MailBox field. At any given time a slot MailBox field can be in one of the following states: empty with no pending operations -- neutral state; empty with one or more waiting take() operations pending -- deficit; occupied with no pending operations; occupied with one or more waiting put() operations -- surplus; empty with a pending put() or pending put() and take() operations -- transitional; or occupied with a pending take() or pending put() and take() operations -- transitional. The partial put() and take() operators can be implemented with an atomic fetch-and-increment operation, which may confer a performance advantage over a CAS-based loop. In addition we have independent PutCursor and TakeCursor cursors. Critically, a put() operation modifies PutCursor but does not access the TakeCursor and a take() operation modifies the TakeCursor cursor but does not access the PutCursor. This acts to reduce coherence traffic relative to some other queue designs. It's worth noting that slow threads or obstruction in one slot (or "lane") does not impede or obstruct operations in other slots -- this gives us some degree of obstruction isolation. PTLQueue is not lock-free, however. The implementation above is expressed with polite busy-waiting (Pause) but it's trivial to implement per-slot parking and unparking to deschedule waiting threads. It's also easy to convert the queue to a more general deque by replacing the PutCursor and TakeCursor cursors with Left/Front and Right/Back cursors that can move either direction. Specifically, to push and pop from the "left" side of the deque we would decrement and increment the Left cursor, respectively, and to push and pop from the "right" side of the deque we would increment and decrement the Right cursor, respectively. We used a variation of PTLQueue for message passing in our recent OPODIS 2013 paper. ul { list-style:none; padding-left:0; padding:0; margin:0; margin-left:0; } ul#myTagID { padding: 0px; margin: 0px; list-style:none; margin-left:0;} -- -- There's quite a bit of related literature in this area. I'll call out a few relevant references: Wilson's NYU Courant Institute UltraComputer dissertation from 1988 is classic and the canonical starting point : Operating System Data Structures for Shared-Memory MIMD Machines with Fetch-and-Add. Regarding provenance and priority, I think PTLQueue or queues effectively equivalent to PTLQueue have been independently rediscovered a number of times. See CB-Queue and BNPBV, below, for instance. But Wilson's dissertation anticipates the basic idea and seems to predate all the others. Gottlieb et al : Basic Techniques for the Efficient Coordination of Very Large Numbers of Cooperating Sequential Processors Orozco et al : CB-Queue in Toward high-throughput algorithms on many-core architectures which appeared in TACO 2012. Meneghin et al : BNPVB family in Performance evaluation of inter-thread communication mechanisms on multicore/multithreaded architecture Dmitry Vyukov : bounded MPMC queue (highly recommended) Alex Otenko : US8607249 (highly related). John Mellor-Crummey : Concurrent queues: Practical fetch-and-phi algorithms. Technical Report 229, Department of Computer Science, University of Rochester Thomasson : FIFO Distributed Bakery Algorithm (very similar to PTLQueue). Scott and Scherer : Dual Data Structures I'll propose an optimization left as an exercise for the reader. Say we wanted to reduce memory usage by eliminating inter-slot padding. Such padding is usually "dark" memory and otherwise unused and wasted. But eliminating the padding leaves us at risk of increased false sharing. Furthermore lets say it was usually the case that the PutCursor and TakeCursor were numerically close to each other. (That's true in some use cases). We might still reduce false sharing by incrementing the cursors by some value other than 1 that is not trivially small and is coprime with the number of slots. Alternatively, we might increment the cursor by one and mask as usual, resulting in a logical index. We then use that logical index value to index into a permutation table, yielding an effective index for use in the slot array. The permutation table would be constructed so that nearby logical indices would map to more distant effective indices. (Open question: what should that permutation look like? Possibly some perversion of a Gray code or De Bruijn sequence might be suitable). As an aside, say we need to busy-wait for some condition as follows : "while C == 0 : Pause". Lets say that C is usually non-zero, so we typically don't wait. But when C happens to be 0 we'll have to spin for some period, possibly brief. We can arrange for the code to be more machine-friendly with respect to the branch predictors by transforming the loop into : "if C == 0 : for { Pause; if C != 0 : break; }". Critically, we want to restructure the loop so there's one branch that controls entry and another that controls loop exit. A concern is that your compiler or JIT might be clever enough to transform this back to "while C == 0 : Pause". You can sometimes avoid this by inserting a call to a some type of very cheap "opaque" method that the compiler can't elide or reorder. On Solaris, for instance, you could use :"if C == 0 : { gethrtime(); for { Pause; if C != 0 : break; }}". It's worth noting the obvious duality between locks and queues. If you have strict FIFO lock implementation with local spinning and succession by direct handoff such as MCS or CLH,then you can usually transform that lock into a queue. Hidden commentary and annotations - invisible : * And of course there's a well-known duality between queues and locks, but I'll leave that topic for another blog post. * Compare and contrast : PTLQ vs PTL and MultiLane * Equivalent : Turn; seq; sequence; pos; position; ticket * Put = Lock; Deposit Take = identify and reserve slot; wait; extract & clear; unlock * conceptualize : Distinct PutLock and TakeLock implemented as ticket lock or PTL Distinct arrival cursors but share per-slot "Turn" variable provides exclusive role-based access to slot's mailbox field put() acquires exclusive access to a slot for purposes of "deposit" assigns slot round-robin and then acquires deposit access rights/perms to that slot take() acquires exclusive access to slot for purposes of "withdrawal" assigns slot round-robin and then acquires withdrawal access rights/perms to that slot At any given time, only one thread can have withdrawal access to a slot at any given time, only one thread can have deposit access to a slot Permissible for T1 to have deposit access and T2 to simultaneously have withdrawal access * round-robin for the purposes of; role-based; access mode; access role mailslot; mailbox; allocate/assign/identify slot rights; permission; license; access permission; * PTL/Ticket hybrid Asymmetric usage ; owner oblivious lock-unlock pairing K-exclusion add Grant cursor pass message m from lock to unlock via Slots[] array Cursor performs 2 functions : + PTL ticket + Assigns request to slot in round-robin fashion Deconstruct protocol : explication put() : allocate slot in round-robin fashion acquire PTL for "put" access store message into slot associated with PTL index take() : Acquire PTL for "take" access // doorway step seq = fetchAdd (&Grant, 1) s = &Slots[seq & Mask] // waiting phase while s-Turn != seq : pause Extract : wait for s-mailbox to be full v = s-mailbox s-mailbox = null Release PTL for both "put" and "take" access s-Turn = seq + Mask + 1 * Slot round-robin assignment and lock "doorway" protocol leverage the same cursor and FetchAdd operation on that cursor FetchAdd (&Cursor,1) + round-robin slot assignment and dispersal + PTL/ticket lock "doorway" step waiting phase is via "Turn" field in slot * PTLQueue uses 2 cursors -- put and take. Acquire "put" access to slot via PTL-like lock Acquire "take" access to slot via PTL-like lock 2 locks : put and take -- at most one thread can access slot's mailbox Both locks use same "turn" field Like multilane : 2 cursors : put and take slot is simple 1-capacity mailbox instead of queue Borrow per-slot turn/grant from PTL Provides strict FIFO Lock slot : put-vs-put take-vs-take at most one put accesses slot at any one time at most one put accesses take at any one time reduction to 1-vs-1 instead of N-vs-M concurrency Per slot locks for put/take Release put/take by advancing turn * is instrumental in ... * P-V Semaphore vs lock vs K-exclusion * See also : FastQueues-excerpt.java dice-etc/queue-mpmc-bounded-blocking-circular-xadd/ * PTLQueue is the same as PTLQB - identical * Expedient return; ASAP; prompt; immediately * Lamport's Bakery algorithm : doorway step then waiting phase Threads arriving at doorway obtain a unique ticket number Threads enter in ticket order * In the terminology of Reed and Kanodia a ticket lock corresponds to the busy-wait implementation of a semaphore using an eventcount and a sequencer It can also be thought of as an optimization of Lamport's bakery lock was designed for fault-tolerance rather than performance Instead of spinning on the release counter, processors using a bakery lock repeatedly examine the tickets of their peers --

    Read the article

  • FreeBSD 8.1 unstable network connection

    - by frankcheong
    I have three FreeBSD 8.1 running on three different hardware and therefore consist of different network adapter as well (bce, bge and igb). I found that the network connection is kind of unstable which I have tried to scp some 10MB file and found that I cannot always get the files completed successfully. I have further checked with my network admin and he claim that the problem is being caused by the network driver which cannot support the load whereby he tried to ping using huge packet size (around 15k) and my server will drop packet consistently at a regular interval. I found that this statement may not be valid since the three server is using three different network drive and it would be quite impossible that the same problem is being caused by three different network adapter and thus different network driver. Since then I have tried to tune up the performance by playing around with the /etc/sysctl.conf figures with no luck. kern.ipc.somaxconn=1024 kern.ipc.shmall=3276800 kern.ipc.shmmax=1638400000 # Security net.inet.ip.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.icmp.maskrepl=0 net.inet.icmp.log_redirect=0 net.inet.icmp.drop_redirect=1 net.inet.tcp.drop_synfin=1 # Security net.inet.udp.blackhole=1 net.inet.tcp.blackhole=2 # Required by pf net.inet.ip.forwarding=1 #Network Performance Tuning kern.ipc.maxsockbuf=16777216 net.inet.tcp.rfc1323=1 net.inet.tcp.sendbuf_max=16777216 net.inet.tcp.recvbuf_max=16777216 # Setting specifically for 1 or even 10Gbps network net.local.stream.sendspace=262144 net.local.stream.recvspace=262144 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.mssdflt=1460 net.inet.tcp.sendbuf_auto=1 net.inet.tcp.sendbuf_inc=16384 net.inet.tcp.recvbuf_auto=1 net.inet.tcp.recvbuf_inc=524288 net.inet.tcp.sendspace=262144 net.inet.tcp.recvspace=262144 net.inet.udp.recvspace=262144 kern.ipc.maxsockbuf=16777216 kern.ipc.nmbclusters=32768 net.inet.tcp.delayed_ack=1 net.inet.tcp.delacktime=100 net.inet.tcp.slowstart_flightsize=179 net.inet.tcp.inflight.enable=1 net.inet.tcp.inflight.min=6144 # Reduce the cache size of slow start connection net.inet.tcp.hostcache.expire=1 Our network admin also claim that they see quite a lot of network up and down from their cisco switch log while I cannot find any up down message inside the dmesg. Have further checked the netstat -s but dont have concrete idea. tcp: 133695291 packets sent 39408539 data packets (3358837321 bytes) 61868 data packets (89472844 bytes) retransmitted 24 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 50756141 ack-only packets (2148 delayed) 0 URG only packets 0 window probe packets 4372385 window update packets 39781869 control packets 134898031 packets received 72339403 acks (for 3357601899 bytes) 190712 duplicate acks 0 acks for unsent data 59339201 packets (3647021974 bytes) received in-sequence 114 completely duplicate packets (135202 bytes) 27 old duplicate packets 0 packets with some dup. data (0 bytes duped) 42090 out-of-order packets (60817889 bytes) 0 packets (0 bytes) of data after window 0 window probes 3953896 window update packets 64181 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 45192 discarded due to memory problems 19945391 connection requests 1323420 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 21133581 connections established (including accepts) 21268724 connections closed (including 32737 drops) 207874 connections updated cached RTT on close 207874 connections updated cached RTT variance on close 132439 connections updated cached ssthresh on close 42392 embryonic connections dropped 72339338 segments updated rtt (of 69477829 attempts) 390871 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 13990 keepalive timeouts 2 keepalive probes sent 13988 connections dropped by keepalive 173044 correct ACK header predictions 36947371 correct data packet header predictions 1323420 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 1323420 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 1323420 cookies sent 0 cookies received 1864 SACK recovery episodes 18005 segment rexmits in SACK recovery episodes 26066896 byte rexmits in SACK recovery episodes 147327 SACK options (SACK blocks) received 87473 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 5141258 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 1 with no checksum 0 dropped due to no socket 129616 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 5011642 delivered 5016050 datagrams output 0 times multicast source filter matched sctp: 0 input packets 0 datagrams 0 packets that had data 0 input SACK chunks 0 input DATA chunks 0 duplicate DATA chunks 0 input HB chunks 0 HB-ACK chunks 0 input ECNE chunks 0 input AUTH chunks 0 chunks missing AUTH 0 invalid HMAC ids received 0 invalid secret ids received 0 auth failed 0 fast path receives all one chunk 0 fast path multi-part data 0 output packets 0 output SACKs 0 output DATA chunks 0 retransmitted DATA chunks 0 fast retransmitted DATA chunks 0 FR's that happened more than once to same chunk 0 intput HB chunks 0 output ECNE chunks 0 output AUTH chunks 0 ip_output error counter Packet drop statistics: 0 from middle box 0 from end host 0 with data 0 non-data, non-endhost 0 non-endhost, bandwidth rep only 0 not enough for chunk header 0 not enough data to confirm 0 where process_chunk_drop said break 0 failed to find TSN 0 attempt reverse TSN lookup 0 e-host confirms zero-rwnd 0 midbox confirms no space 0 data did not match TSN 0 TSN's marked for Fast Retran Timeouts: 0 iterator timers fired 0 T3 data time outs 0 window probe (T3) timers fired 0 INIT timers fired 0 sack timers fired 0 shutdown timers fired 0 heartbeat timers fired 0 a cookie timeout fired 0 an endpoint changed its cookiesecret 0 PMTU timers fired 0 shutdown ack timers fired 0 shutdown guard timers fired 0 stream reset timers fired 0 early FR timers fired 0 an asconf timer fired 0 auto close timer fired 0 asoc free timers expired 0 inp free timers expired 0 packet shorter than header 0 checksum error 0 no endpoint for port 0 bad v-tag 0 bad SID 0 no memory 0 number of multiple FR in a RTT window 0 RFC813 allowed sending 0 RFC813 does not allow sending 0 times max burst prohibited sending 0 look ahead tells us no memory in interface 0 numbers of window probes sent 0 times an output error to clamp down on next user send 0 times sctp_senderrors were caused from a user 0 number of in data drops due to chunk limit reached 0 number of in data drops due to rwnd limit reached 0 times a ECN reduced the cwnd 0 used express lookup via vtag 0 collision in express lookup 0 times the sender ran dry of user data on primary 0 same for above 0 sacks the slow way 0 window update only sacks sent 0 sends with sinfo_flags !=0 0 unordered sends 0 sends with EOF flag set 0 sends with ABORT flag set 0 times protocol drain called 0 times we did a protocol drain 0 times recv was called with peek 0 cached chunks used 0 cached stream oq's used 0 unread messages abandonded by close 0 send burst avoidance, already max burst inflight to net 0 send cwnd full avoidance, already max burst inflight to net 0 number of map array over-runs via fwd-tsn's ip: 137814085 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 1200 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 300 packets reassembled ok 137813009 packets for this host 530 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 61 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 137234598 packets sent from this host 0 packets sent with fabricated ip header 685307 output packets dropped due to no bufs, etc. 52 output packets discarded due to no route 300 output datagrams fragmented 1200 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message Output histogram: echo reply: 305 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored Input histogram: destination unreachable: 530 echo: 305 305 message responses generated 0 invalid return addresses 0 no return routes ICMP address mask responses are disabled igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 376748 ARP requests sent 3207 ARP replies sent 245245 ARP requests received 80845 ARP replies received 326090 ARP packets received 267712 total packets dropped due to no ARP entry 108876 ARP entrys timed out 0 Duplicate IPs seen ip6: 2226633 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 2226633 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 2226633 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 8 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Input histogram: UDP: 2226633 Mbuf statistics: 962679 one mbuf 1263954 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not continuous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output netstat -m 516/5124/5640 mbufs in use (current/cache/total) 512/1634/2146/32768 mbuf clusters in use (current/cache/total/max) 512/1536 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1303/1303/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1153K/9761K/10914K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/8/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines Anyone got an idea what might be the possible cause?

    Read the article

  • How do I set the encoding statement in the XML declaration when performing an XSL transformation usi

    - by aspiehler
    I wrote a simple package installer in WinBatch that needs to update an XML file with information about the package contents. My first stab at it involved loading the file with Msxml2.DOMDocument, adding nodes and data as required, then saving the data back to disk. This worked well enough, except that it would not create tab and CR/LF whitespace in the new data. The solution I came up with was writing an XSL stylesheet that would recreate the XML file with whitespace added back in. I'm doing this by: loading the XSL file into an Msxml2.FreeThreadedDOMDocument object setting that object as the stylesheet property of an Msxml2.XSLTemplate object creating an XSL processor via Msxml2.XSLTemplate.createProcessor() setting my original Msxml2.DOMDocument as the input property of the XSL processor Calling transform() method of the XSL processor, and saving the output to a file. This works as for as reformatting the XML file with tabs and carriage returns, but my XML declaration comes out either as <?xml version="1.0"?> or <?xml version="1.0" encoding="UTF-16"?> depending on whether I used Msxml2.*.6.0 or Msxml2.* objects (a fall back if the system doesn't have 6.0). If the encoding is set to UTF-16, Msxml12.DOMDocument complains about trying to convert UTF-16 to 1-byte encoding the next time I run my package installer. I've tried creating and adding an XML declaration using both createProcessingInstruction() to both the XML and XSL DOM objects, but neither one seems to affect the output of the XSLTemplate processor. I've also set encoding to UTF-8 in the <xsl:output/> tag in my XSL file. Here is the relevant code in my Winbatch script: xmlDoc = ObjectCreate("Msxml2.DOMDocument.6.0") if !xmlDoc then xmlDoc = ObjectCreate("Msxml2.DOMDocument") xmlDoc.async = @FALSE xmlDoc.validateOnParse = @TRUE xmlDoc.resolveExternals = @TRUE xmlDoc.preserveWhiteSpace = @TRUE xmlDoc.setProperty("SelectionLanguge", "XPath") xmlDoc.setProperty("SelectionNamespaces", "xmlns:fns='http://www.abc.com/f_namespace'") xmlDoc.load(xml_file_path) xslStyleSheet = ObjectCreate("Msxml2.FreeThreadedDOMDocument.6.0") if !xslStyleSheet then xslStyleSheet = ObjectCreate("Msxml2.FreeThreadedDOMDocument") xslStyleSheet.async = @FALSE xslStyleSheet.validateOnParse = @TRUE xslStyleSheet.load(xsl_style_sheet_path) xslTemplate = ObjectCreate("Msxml2.XSLTemplate.6.0") if !xslTemplate then xslTemplate = ObjectCreate("Msxml2.XSLTemplate") xslTemplate.stylesheet = xslStyleSheet processor = xslTemplate.createProcessor() processor.input = xmlDoc processor.transform() ; create a new file and write the XML processor output to it fh = FileOpen(output_file_path, "WRITE" , @FALSE) FileWrite(fh, processor.output) FileClose(fh) The style sheet, with some slight changes to protect the innocent: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.1"> <xsl:output method="xml" indent="yes" encoding="UTF-8"/> <xsl:template match="/"> <fns:test_station xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:fns="http://www.abc.com/f_namespace"> <xsl:for-each select="/fns:test_station/identification"> <xsl:text>&#x0A; </xsl:text> <identification> <xsl:for-each select="./*"> <xsl:text>&#x0A; </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>&#x0A; </xsl:text> </identification> </xsl:for-each> <xsl:for-each select="/fns:test_station/software"> <xsl:text>&#x0A; </xsl:text> <software> <xsl:for-each select="./package"> <xsl:text>&#x0A; </xsl:text> <package> <xsl:for-each select="./*"> <xsl:text>&#x0A; </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>&#x0A; </xsl:text> </package> </xsl:for-each> <xsl:text>&#x0A; </xsl:text> </software> </xsl:for-each> <xsl:for-each select="/fns:test_station/calibration"> <xsl:text>&#x0A; </xsl:text> <calibration> <xsl:for-each select="./item"> <xsl:text>&#x0A; </xsl:text> <item> <xsl:for-each select="./*"> <xsl:text>&#x0A; </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>&#x0A; </xsl:text> </item> </xsl:for-each> <xsl:text>&#x0A; </xsl:text> </calibration> </xsl:for-each> </fns:test_station> </xsl:template> </xsl:stylesheet> And this is a sample output file: <?xml version="1.0" encoding="UTF-16"?> <fns:test_station xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:fns="http://www.abc.com/f_namespace"> <software> <package> <part_number>123456789</part_number> <version>00</version> <test_category>1</test_category> <description>name of software package</description> <execution_path>c:\program files\test\test.exe</execution_path> <execution_arguments>arguments</execution_arguments> <crc_path>c:\ste_config\crc\123456789.lst</crc_path> <uninstall_path>c:\ste_config\uninstall\uninst_123456789.bat</uninstall_path> <install_timestamp>2009-11-09T14:00:44</install_timestamp> </package> </software> </fns:test_station>

    Read the article

  • Large Object Heap Fragmentation

    - by Paul Ruane
    The C#/.NET application I am working on is suffering from a slow memory leak. I have used CDB with SOS to try to determine what is happening but the data does not seem to make any sense so I was hoping one of you may have experienced this before. The application is running on the 64 bit framework. It is continuously calculating and serialising data to a remote host and is hitting the Large Object Heap (LOH) a fair bit. However, most of the LOH objects I expect to be transient: once the calculation is complete and has been sent to the remote host, the memory should be freed. What I am seeing, however, is a large number of (live) object arrays interleaved with free blocks of memory, e.g., taking a random segment from the LOH: 0:000> !DumpHeap 000000005b5b1000 000000006351da10 Address MT Size ... 000000005d4f92e0 0000064280c7c970 16147872 000000005e45f880 00000000001661d0 1901752 Free 000000005e62fd38 00000642788d8ba8 1056 <-- 000000005e630158 00000000001661d0 5988848 Free 000000005ebe6348 00000642788d8ba8 1056 000000005ebe6768 00000000001661d0 6481336 Free 000000005f214d20 00000642788d8ba8 1056 000000005f215140 00000000001661d0 7346016 Free 000000005f9168a0 00000642788d8ba8 1056 000000005f916cc0 00000000001661d0 7611648 Free 00000000600591c0 00000642788d8ba8 1056 00000000600595e0 00000000001661d0 264808 Free ... Obviously I would expect this to be the case if my application were creating long-lived, large objects during each calculation. (It does do this and I accept there will be a degree of LOH fragmentation but that is not the problem here.) The problem is the very small (1056 byte) object arrays you can see in the above dump which I cannot see in code being created and which are remaining rooted somehow. Also note that CDB is not reporting the type when the heap segment is dumped: I am not sure if this is related or not. If I dump the marked (<--) object, CDB/SOS reports it fine: 0:015> !DumpObj 000000005e62fd38 Name: System.Object[] MethodTable: 00000642788d8ba8 EEClass: 00000642789d7660 Size: 1056(0x420) bytes Array: Rank 1, Number of elements 128, Type CLASS Element Type: System.Object Fields: None The elements of the object array are all strings and the strings are recognisable as from our application code. Also, I am unable to find their GC roots as the !GCRoot command hangs and never comes back (I have even tried leaving it overnight). So, I would very much appreciate it if anyone could shed any light as to why these small (<85k) object arrays are ending up on the LOH: what situations will .NET put a small object array in there? Also, does anyone happen to know of an alternative way of ascertaining the roots of these objects? Thanks in advance. Update 1 Another theory I came up with late yesterday is that these object arrays started out large but have been shrunk leaving the blocks of free memory that are evident in the memory dumps. What makes me suspicious is that the object arrays always appear to be 1056 bytes long (128 elements), 128 * 8 for the references and 32 bytes of overhead. The idea is that perhaps some unsafe code in a library or in the CLR is corrupting the number of elements field in the array header. Bit of a long shot I know... Update 2 Thanks to Brian Rasmussen (see accepted answer) the problem has been identified as fragmentation of the LOH caused by the string intern table! I wrote a quick test application to confirm this: static void Main() { const int ITERATIONS = 100000; for (int index = 0; index < ITERATIONS; ++index) { string str = "NonInterned" + index; Console.Out.WriteLine(str); } Console.Out.WriteLine("Continue."); Console.In.ReadLine(); for (int index = 0; index < ITERATIONS; ++index) { string str = string.Intern("Interned" + index); Console.Out.WriteLine(str); } Console.Out.WriteLine("Continue?"); Console.In.ReadLine(); } The application first creates and dereferences unique strings in a loop. This is just to prove that the memory does not leak in this scenario. Obviously it should not and it does not. In the second loop, unique strings are created and interned. This action roots them in the intern table. What I did not realise is how the intern table is represented. It appears it consists of a set of pages -- object arrays of 128 string elements -- that are created in the LOH. This is more evident in CDB/SOS: 0:000> .loadby sos mscorwks 0:000> !EEHeap -gc Number of GC Heaps: 1 generation 0 starts at 0x00f7a9b0 generation 1 starts at 0x00e79c3c generation 2 starts at 0x00b21000 ephemeral segment allocation context: none segment begin allocated size 00b20000 00b21000 010029bc 0x004e19bc(5118396) Large object heap starts at 0x01b21000 segment begin allocated size 01b20000 01b21000 01b8ade0 0x00069de0(433632) Total Size 0x54b79c(5552028) ------------------------------ GC Heap Size 0x54b79c(5552028) Taking a dump of the LOH segment reveals the pattern I saw in the leaking application: 0:000> !DumpHeap 01b21000 01b8ade0 ... 01b8a120 793040bc 528 01b8a330 00175e88 16 Free 01b8a340 793040bc 528 01b8a550 00175e88 16 Free 01b8a560 793040bc 528 01b8a770 00175e88 16 Free 01b8a780 793040bc 528 01b8a990 00175e88 16 Free 01b8a9a0 793040bc 528 01b8abb0 00175e88 16 Free 01b8abc0 793040bc 528 01b8add0 00175e88 16 Free total 1568 objects Statistics: MT Count TotalSize Class Name 00175e88 784 12544 Free 793040bc 784 421088 System.Object[] Total 1568 objects Note that the object array size is 528 (rather than 1056) because my workstation is 32 bit and the application server is 64 bit. The object arrays are still 128 elements long. So the moral to this story is to be very careful interning. If the string you are interning is not known to be a member of a finite set then your application will leak due to fragmentation of the LOH, at least in version 2 of the CLR. In our application's case, there is general code in the deserialisation code path that interns entity identifiers during unmarshalling: I now strongly suspect this is the culprit. However, the developer's intentions were obviously good as they wanted to make sure that if the same entity is deserialised multiple times then only one instance of the identifier string will be maintained in memory.

    Read the article

  • Bogus InvalidOperationException (in a DataServiceRequestException)

    - by Andrei Rinea
    I am having a hard time with ADO.NET Data Services (formerly code-named Astoria) as it gives me a bogus exception when I try to insert a new entity from the silverlight client and trying in a clean project (the same code) doesn't. In both cases, however, data is correctly inserted into the database. Using Fiddler (an HTTP debugger I could see that there is no problem in the HTTP communication as I will show later in this question. The code : var ctx = new MyProject123Entities(new Uri("http://andreiri/MyProject.Data/Data.svc")); var i = new Zone() { Data = DateTime.Now, IdElement = 1 }; ctx.AddToZone(i); i.StareZone = new StareZone() { IdStareZone = 1 }; ctx.AttachTo("StareZone", i.StareZone); ctx.SetLink(i, "StareZone", i.StareZone); i.TipZone = new TipZone() { IdTipZone = 1 }; ctx.AttachTo("TipZone", i.TipZone); ctx.SetLink(i, "TipZone", i.TipZone); i.User = new User() { IdUser = 2 }; ctx.AttachTo("User", i.User); ctx.SetLink(i, "User", i.User); ctx.BeginSaveChanges(r =] ctx.EndSaveChanges(r), null); when run the last line (ctx.EndSaveChanges(r)) will throw the following exception : System.Data.Services.Client.DataServiceRequestException was unhandled by user code Message="An error occurred while processing this request." StackTrace: at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.HandleBatchResponse() at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.EndRequest() at System.Data.Services.Client.DataServiceContext.EndSaveChanges(IAsyncResult asyncResult) at MyProject.MainPage.[]c__DisplayClassd6.[]c__DisplayClassd8.[dashboard_PostZoneCurent]b__d5(IAsyncResult r) at System.Data.Services.Client.BaseAsyncResult.HandleCompleted() at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.HandleCompleted(PerRequest pereq) at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.AsyncEndRead(IAsyncResult asyncResult) at System.IO.Stream.BeginRead(Byte[] buffer, Int32 offset, Int32 count, AsyncCallback callback, Object state) at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.AsyncEndGetResponse(IAsyncResult asyncResult) InnerException: System.InvalidOperationException Message="The context is already tracking a different entity with the same resource Uri." StackTrace: at System.Data.Services.Client.DataServiceContext.AttachTo(Uri identity, Uri editLink, String etag, Object entity, Boolean fail) at System.Data.Services.Client.MaterializeAtom.MoveNext() at System.Data.Services.Client.DataServiceContext.HandleResponsePost(ResourceBox entry, MaterializeAtom materializer, Uri editLink, String etag) at System.Data.Services.Client.DataServiceContext.SaveAsyncResult.[HandleBatchResponse]d__1d.MoveNext() InnerException: (there is no further information regarding the exception although the ADo.NET Data Service is configured to return detailed informations) However the row is inserted correctly and completely in the database. Using fiddler I can see that the request : <?xml version="1.0" encoding="utf-8" standalone="yes"?> <entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"> <category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="MyProject123Model.Zone" /> <title /> <updated>2009-09-11T13:36:46.917157Z</updated> <author> <name /> </author> <id /> <link href="http://andreiri/MyProject.Data/Data.svc/StareZone(1)" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/StareZone" type="application/atom+xml;type=entry" /> <link href="http://andreiri/MyProject.Data/Data.svc/TipZone(4)" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/TipZone" type="application/atom+xml;type=entry" /> <link href="http://andreiri/MyProject.Data/Data.svc/User(4)" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/User" type="application/atom+xml;type=entry" /> <content type="application/xml"> <m:properties> <d:Data m:type="Edm.DateTime">2009-09-11T16:36:40.588951+03:00</d:Data> <d:Detalii>aslkdfjasldkfj</d:Detalii> <d:IdElement m:type="Edm.Int32">1</d:IdElement> <d:IdZone m:type="Edm.Int32">0</d:IdZone> <d:X_Post m:type="Edm.Decimal">587647.4705</d:X_Post> <d:X_Repost m:type="Edm.Decimal" m:null="true" /> <d:Y_Post m:type="Edm.Decimal">325783.077599999</d:Y_Post> <d:Y_Repost m:type="Edm.Decimal" m:null="true" /> </m:properties> </content> </entry> is well accepted and a successful response is returned : HTTP/1.1 201 Created Date: Fri, 11 Sep 2009 13:36:47 GMT Server: Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 DataServiceVersion: 1.0; Location: http://andreiri/MyProject.Data/Data.svc/Zone(75) Cache-Control: no-cache Content-Type: application/atom+xml;charset=utf-8 Content-Length: 2213 <?xml version="1.0" encoding="utf-8" standalone="yes"?> <entry xml:base="http://andreiri/MyProject.Data/Data.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"> <id>http://andreiri/MyProject.Data/Data.svc/Zone(75)</id> <title type="text"></title> <updated>2009-09-11T13:36:47Z</updated> <author> <name /> </author> <link rel="edit" title="Zone" href="Zone(75)" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/CenterZone" type="application/atom+xml;type=feed" title="CenterZone" href="Zone(75)/CenterZone" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ZoneMobil" type="application/atom+xml;type=feed" title="ZoneMobil" href="Zone(75)/ZoneMobil" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/StareZone" type="application/atom+xml;type=entry" title="StareZone" href="Zone(75)/StareZone" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/TipZone" type="application/atom+xml;type=entry" title="TipZone" href="Zone(75)/TipZone" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/User" type="application/atom+xml;type=entry" title="User" href="Zone(75)/User" /> <category term="MyProject123Model.Zone" scheme="http://schemas.microsoft.com ado/2007/08/dataservices/scheme" /> <content type="application/xml"> <m:properties> <d:IdZone m:type="Edm.Int32">75</d:IdZone> <d:X_Post m:type="Edm.Decimal">587647.4705</d:X_Post> <d:Y_Post m:type="Edm.Decimal">325783.077599999</d:Y_Post> <d:X_Repost m:type="Edm.Decimal" m:null="true" /> <d:Y_Repost m:type="Edm.Decimal" m:null="true" /> <d:Data m:type="Edm.DateTime">2009-09-11T16:36:40.588951+03:00</d:Data> <d:Detalii>aslkdfjasldkfj</d:Detalii> <d:IdElement m:type="Edm.Int32">1</d:IdElement> </m:properties> </content> </entry> Why do I get an exception? And, using this in a clean project does not throw the exception..

    Read the article

  • android app does not show up on my device or the emulator in eclipse

    - by Sam
    hey everyone, I have no errors in my app-code what so ever, but when i try to run in on either my cell or my emulator/the avd in eclipse i can't run it because it doesn't show up on either one. this is my console output: [2011-02-04 08:14:58 - Versuch] Uploading Versuch.apk onto device 'CB511L2WTB' [2011-02-04 08:14:58 - Versuch] Installing Versuch.apk... [2011-02-04 08:15:01 - Versuch] Success! [2011-02-04 08:15:01 - Versuch] \Versuch\bin\Versuch.apk installed on device [2011-02-04 08:15:01 - Versuch] Done! and this is my LogCat output, which tells me nothing, but you are the experts ;) 02-04 08:18:10.020: DEBUG/dalvikvm(22167): GC freed 2576 objects / 559120 bytes in 37ms 02-04 08:18:10.700: DEBUG/dalvikvm(6709): GC freed 7692 objects / 478912 bytes in 41ms 02-04 08:18:11.170: DEBUG/dalvikvm(31774): GC freed 3367 objects / 163464 bytes in 122ms 02-04 08:18:13.230: DEBUG/dalvikvm(22167): GC freed 2790 objects / 552328 bytes in 38ms 02-04 08:18:14.650: DEBUG/dalvikvm(6709): GC freed 8443 objects / 540440 bytes in 39ms 02-04 08:18:16.260: DEBUG/dalvikvm(31921): GC freed 214 objects / 9824 bytes in 216ms 02-04 08:18:16.670: DEBUG/dalvikvm(22167): GC freed 3232 objects / 561256 bytes in 40ms 02-04 08:18:18.600: DEBUG/dalvikvm(6709): GC freed 7718 objects / 481952 bytes in 39ms 02-04 08:18:19.210: DEBUG/dalvikvm(1129): GC freed 6898 objects / 275328 bytes in 109ms 02-04 08:18:19.690: DEBUG/dalvikvm(22167): GC freed 2968 objects / 571232 bytes in 39ms 02-04 08:18:21.440: DEBUG/dalvikvm(1212): GC freed 1020 objects / 49328 bytes in 395ms 02-04 08:18:22.570: DEBUG/dalvikvm(6709): GC freed 7893 objects / 495616 bytes in 40ms 02-04 08:18:23.060: DEBUG/dalvikvm(22167): GC freed 3117 objects / 561912 bytes in 41ms 02-04 08:18:25.860: DEBUG/dalvikvm(22167): GC freed 2924 objects / 558448 bytes in 36ms 02-04 08:18:26.350: DEBUG/dalvikvm(32098): GC freed 4662 objects / 495496 bytes in 290ms 02-04 08:18:26.410: DEBUG/dalvikvm(22167): GC freed 1077 objects / 130680 bytes in 33ms 02-04 08:18:27.080: DEBUG/dalvikvm(6709): GC freed 7912 objects / 485368 bytes in 40ms 02-04 08:18:28.190: DEBUG/dalvikvm(22167): GC freed 953 objects / 767272 bytes in 33ms 02-04 08:18:29.500: DEBUG/dalvikvm(1129): GC freed 6756 objects / 270480 bytes in 105ms 02-04 08:18:30.500: WARN/System.err(22536): java.lang.Exception: You must call com.mercuryintermedia.productconfiguration.initialize() first 02-04 08:18:30.670: WARN/System.err(22536): at com.mercuryintermedia.ProductConfiguration.getProductName(ProductConfiguration.java:136) 02-04 08:18:30.670: WARN/System.err(22536): at com.mercuryintermedia.api.rest.Item.getPublishingContainersItems(Item.java:15) 02-04 08:18:30.670: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.getContainerFromServer(ContainerHelper.java:68) 02-04 08:18:30.670: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.run(ContainerHelper.java:46) 02-04 08:18:31.090: DEBUG/dalvikvm(6709): GC freed 10545 objects / 682480 bytes in 49ms 02-04 08:18:31.120: DEBUG/dalvikvm(1813): GC freed 5970 objects / 310912 bytes in 60ms 02-04 08:18:31.320: DEBUG/dalvikvm(22167): GC freed 2468 objects / 539520 bytes in 39ms 02-04 08:18:34.110: DEBUG/dalvikvm(22167): GC freed 2879 objects / 569008 bytes in 35ms 02-04 08:18:34.920: DEBUG/dalvikvm(6709): GC freed 7029 objects / 424632 bytes in 35ms 02-04 08:18:36.150: DEBUG/dalvikvm(9060): GC freed 564 objects / 27840 bytes in 89ms 02-04 08:18:36.630: DEBUG/dalvikvm(22167): GC freed 2437 objects / 554000 bytes in 35ms 02-04 08:18:38.760: DEBUG/dalvikvm(6709): GC freed 8309 objects / 545032 bytes in 36ms 02-04 08:18:39.270: DEBUG/dalvikvm(1129): GC freed 6958 objects / 278352 bytes in 107ms 02-04 08:18:39.970: DEBUG/dalvikvm(22167): GC freed 2915 objects / 560312 bytes in 38ms 02-04 08:18:41.260: DEBUG/dalvikvm(6184): GC freed 373 objects / 26152 bytes in 205ms 02-04 08:18:42.780: DEBUG/dalvikvm(6709): GC freed 7212 objects / 447696 bytes in 36ms 02-04 08:18:43.160: DEBUG/dalvikvm(22167): GC freed 3106 objects / 561824 bytes in 39ms 02-04 08:18:46.310: DEBUG/dalvikvm(22167): GC freed 3110 objects / 564080 bytes in 45ms 02-04 08:18:46.650: DEBUG/dalvikvm(6709): GC freed 7508 objects / 468832 bytes in 36ms 02-04 08:18:48.820: DEBUG/dalvikvm(31712): GC freed 13795 objects / 828232 bytes in 203ms 02-04 08:18:49.040: DEBUG/dalvikvm(1129): GC freed 6918 objects / 276224 bytes in 109ms 02-04 08:18:49.640: DEBUG/dalvikvm(22167): GC freed 2952 objects / 562168 bytes in 37ms 02-04 08:18:50.630: DEBUG/dalvikvm(6709): GC freed 8332 objects / 549680 bytes in 35ms 02-04 08:18:52.770: DEBUG/dalvikvm(22167): GC freed 3108 objects / 563192 bytes in 37ms 02-04 08:18:54.400: DEBUG/dalvikvm(6709): GC freed 7509 objects / 469016 bytes in 35ms 02-04 08:18:55.900: DEBUG/dalvikvm(22167): GC freed 3121 objects / 572920 bytes in 38ms 02-04 08:18:58.150: DEBUG/dalvikvm(6709): GC freed 7408 objects / 465456 bytes in 35ms 02-04 08:18:58.710: DEBUG/dalvikvm(1129): GC freed 6908 objects / 276440 bytes in 107ms 02-04 08:18:59.190: DEBUG/dalvikvm(22167): GC freed 3160 objects / 563144 bytes in 38ms 02-04 08:19:02.080: DEBUG/dalvikvm(6709): GC freed 7436 objects / 468040 bytes in 36ms 02-04 08:19:02.380: DEBUG/dalvikvm(22167): GC freed 3104 objects / 557600 bytes in 39ms 02-04 08:19:05.050: DEBUG/dalvikvm(22167): GC freed 2860 objects / 570072 bytes in 35ms 02-04 08:19:05.810: DEBUG/dalvikvm(6709): GC freed 7508 objects / 469080 bytes in 35ms 02-04 08:19:06.500: DEBUG/skia(22167): --- decoder->decode returned false 02-04 08:19:07.960: DEBUG/dalvikvm(22167): GC freed 2747 objects / 520008 bytes in 36ms 02-04 08:19:08.180: DEBUG/dalvikvm(1129): GC freed 7866 objects / 317304 bytes in 107ms 02-04 08:19:09.540: DEBUG/dalvikvm(6709): GC freed 8220 objects / 539688 bytes in 36ms 02-04 08:19:10.810: DEBUG/dalvikvm(22167): GC freed 2898 objects / 596824 bytes in 37ms 02-04 08:19:13.360: DEBUG/dalvikvm(22167): GC freed 2503 objects / 398936 bytes in 35ms 02-04 08:19:13.370: INFO/dalvikvm-heap(22167): Grow heap (frag case) to 5.029MB for 570264-byte allocation 02-04 08:19:13.400: DEBUG/dalvikvm(22167): GC freed 702 objects / 24976 bytes in 31ms 02-04 08:19:13.400: DEBUG/skia(22167): --- decoder->decode returned false 02-04 08:19:13.540: DEBUG/dalvikvm(6709): GC freed 7481 objects / 466544 bytes in 36ms 02-04 08:19:15.600: DEBUG/WifiService(1129): got ACTION_DEVICE_IDLE 02-04 08:19:15.960: INFO/wpa_supplicant(2522): CTRL-EVENT-DRIVER-STATE STOPPED 02-04 08:19:15.960: VERBOSE/WifiMonitor(1129): Event [CTRL-EVENT-DRIVER-STATE STOPPED] 02-04 08:19:17.270: DEBUG/dalvikvm(22167): GC freed 2372 objects / 1266992 bytes in 36ms 02-04 08:19:17.520: DEBUG/dalvikvm(6709): GC freed 7996 objects / 519128 bytes in 37ms 02-04 08:19:18.150: DEBUG/dalvikvm(1129): GC freed 7110 objects / 285032 bytes in 108ms 02-04 08:19:20.460: DEBUG/dalvikvm(22167): GC freed 3327 objects / 565264 bytes in 36ms 02-04 08:19:21.250: DEBUG/dalvikvm(6709): GC freed 7632 objects / 486024 bytes in 37ms 02-04 08:19:26.470: DEBUG/dalvikvm(31774): GC freed 345 objects / 16160 bytes in 96ms 02-04 08:19:30.423: WARN/System.err(22536): java.lang.Exception: You must call com.mercuryintermedia.productconfiguration.initialize() first 02-04 08:19:30.423: WARN/System.err(22536): at com.mercuryintermedia.ProductConfiguration.getProductName(ProductConfiguration.java:136) 02-04 08:19:30.423: WARN/System.err(22536): at com.mercuryintermedia.api.rest.Item.getPublishingContainersItems(Item.java:15) 02-04 08:19:30.423: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.getContainerFromServer(ContainerHelper.java:68) 02-04 08:19:30.423: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.run(ContainerHelper.java:46) 02-04 08:20:05.280: DEBUG/dalvikvm(1813): GC freed 741 objects / 36840 bytes in 91ms 02-04 08:20:23.580: DEBUG/WifiService(1129): ACTION_BATTERY_CHANGED pluggedType: 2 02-04 08:20:30.423: WARN/System.err(22536): java.lang.Exception: You must call com.mercuryintermedia.productconfiguration.initialize() first 02-04 08:20:30.423: WARN/System.err(22536): at com.mercuryintermedia.ProductConfiguration.getProductName(ProductConfiguration.java:136) 02-04 08:20:30.423: WARN/System.err(22536): at com.mercuryintermedia.api.rest.Item.getPublishingContainersItems(Item.java:15) 02-04 08:20:30.423: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.getContainerFromServer(ContainerHelper.java:68) 02-04 08:20:30.423: WARN/System.err(22536): at com.mercuryintermedia.mflow.ContainerHelper.run(ContainerHelper.java:46) 02-04 08:20:53.970: INFO/FastDormancyManager(1129): Fast Dormant executed. ExecuteCount:2683 NonExecuteCount:25773 I really hope you can help me.

    Read the article

  • DirectShow: Video-Preview and Image (with working code)

    - by xsl
    Questions / Issues If someone can recommend me a good free hosting site I can provide the whole project file. As mentioned in the text below the TakePicture() method is not working properly on the HTC HD 2 device. It would be nice if someone could look at the code below and tell me if it is right or wrong what I'm doing. Introduction I recently asked a question about displaying a video preview, taking camera image and rotating a video stream with DirectShow. The tricky thing about the topic is, that it's very hard to find good examples and the documentation and the framework itself is very hard to understand for someone who is new to windows programming and C++ in general. Nevertheless I managed to create a class that implements most of this features and probably works with most mobile devices. Probably because the DirectShow implementation depends a lot on the device itself. I could only test it with the HTC HD and HTC HD2, which are known as quite incompatible. HTC HD Working: Video preview, writing photo to file Not working: Set video resolution (CRASH), set photo resolution (LOW quality) HTC HD 2 Working: Set video resolution, set photo resolution Problematic: Video Preview rotated Not working: Writing photo to file To make it easier for others by providing a working example, I decided to share everything I have got so far below. I removed all of the error handling for the sake of simplicity. As far as documentation goes, I can recommend you to read the MSDN documentation, after that the code below is pretty straight forward. void Camera::Init() { CreateComObjects(); _captureGraphBuilder->SetFiltergraph(_filterGraph); InitializeVideoFilter(); InitializeStillImageFilter(); } Dipslay a video preview (working with any tested handheld): void Camera::DisplayVideoPreview(HWND windowHandle) { IVideoWindow *_vidWin; _filterGraph->QueryInterface(IID_IMediaControl,(void **) &_mediaControl); _filterGraph->QueryInterface(IID_IVideoWindow, (void **) &_vidWin); _videoCaptureFilter->QueryInterface(IID_IAMVideoControl, (void**) &_videoControl); _captureGraphBuilder->RenderStream(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video, _videoCaptureFilter, NULL, NULL); CRect rect; long width, height; GetClientRect(windowHandle, &rect); _vidWin->put_Owner((OAHWND)windowHandle); _vidWin->put_WindowStyle(WS_CHILD | WS_CLIPSIBLINGS); _vidWin->get_Width(&width); _vidWin->get_Height(&height); height = rect.Height(); _vidWin->put_Height(height); _vidWin->put_Width(rect.Width()); _vidWin->SetWindowPosition(0,0, rect.Width(), height); _mediaControl->Run(); } HTC HD2: If set SetPhotoResolution() is called FindPin will return E_FAIL. If not, it will create a file full of null bytes. HTC HD: Works void Camera::TakePicture(WCHAR *fileName) { CComPtr<IFileSinkFilter> fileSink; CComPtr<IPin> stillPin; CComPtr<IUnknown> unknownCaptureFilter; CComPtr<IAMVideoControl> videoControl; _imageSinkFilter.QueryInterface(&fileSink); fileSink->SetFileName(fileName, NULL); _videoCaptureFilter.QueryInterface(&unknownCaptureFilter); _captureGraphBuilder->FindPin(unknownCaptureFilter, PINDIR_OUTPUT, &PIN_CATEGORY_STILL, &MEDIATYPE_Video, FALSE, 0, &stillPin); _videoCaptureFilter.QueryInterface(&videoControl); videoControl->SetMode(stillPin, VideoControlFlag_Trigger); } Set resolution: Works great on HTC HD2. HTC HD won't allow SetVideoResolution() and only offers one low resolution photo resolution: void Camera::SetVideoResolution(int width, int height) { SetResolution(true, width, height); } void Camera::SetPhotoResolution(int width, int height) { SetResolution(false, width, height); } void Camera::SetResolution(bool video, int width, int height) { IAMStreamConfig *config; config = NULL; if (video) { _captureGraphBuilder->FindInterface(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video, _videoCaptureFilter, IID_IAMStreamConfig, (void**) &config); } else { _captureGraphBuilder->FindInterface(&PIN_CATEGORY_STILL, &MEDIATYPE_Video, _videoCaptureFilter, IID_IAMStreamConfig, (void**) &config); } int resolutions, size; VIDEO_STREAM_CONFIG_CAPS caps; config->GetNumberOfCapabilities(&resolutions, &size); for (int i = 0; i < resolutions; i++) { AM_MEDIA_TYPE *mediaType; if (config->GetStreamCaps(i, &mediaType, reinterpret_cast<BYTE*>(&caps)) == S_OK ) { int maxWidth = caps.MaxOutputSize.cx; int maxHeigth = caps.MaxOutputSize.cy; if(maxWidth == width && maxHeigth == height) { VIDEOINFOHEADER *info = reinterpret_cast<VIDEOINFOHEADER*>(mediaType->pbFormat); info->bmiHeader.biWidth = maxWidth; info->bmiHeader.biHeight = maxHeigth; info->bmiHeader.biSizeImage = DIBSIZE(info->bmiHeader); config->SetFormat(mediaType); DeleteMediaType(mediaType); break; } DeleteMediaType(mediaType); } } } Other methods used to build the filter graph and create the COM objects: void Camera::CreateComObjects() { CoInitialize(NULL); CoCreateInstance(CLSID_CaptureGraphBuilder, NULL, CLSCTX_INPROC_SERVER, IID_ICaptureGraphBuilder2, (void **) &_captureGraphBuilder); CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER, IID_IGraphBuilder, (void **) &_filterGraph); CoCreateInstance(CLSID_VideoCapture, NULL, CLSCTX_INPROC, IID_IBaseFilter, (void**) &_videoCaptureFilter); CoCreateInstance(CLSID_IMGSinkFilter, NULL, CLSCTX_INPROC, IID_IBaseFilter, (void**) &_imageSinkFilter); } void Camera::InitializeVideoFilter() { _videoCaptureFilter->QueryInterface(&_propertyBag); wchar_t deviceName[MAX_PATH] = L"\0"; GetDeviceName(deviceName); CComVariant comName = deviceName; CPropertyBag propertyBag; propertyBag.Write(L"VCapName", &comName); _propertyBag->Load(&propertyBag, NULL); _filterGraph->AddFilter(_videoCaptureFilter, L"Video Capture Filter Source"); } void Camera::InitializeStillImageFilter() { _filterGraph->AddFilter(_imageSinkFilter, L"Still image filter"); _captureGraphBuilder->RenderStream(&PIN_CATEGORY_STILL, &MEDIATYPE_Video, _videoCaptureFilter, NULL, _imageSinkFilter); } void Camera::GetDeviceName(WCHAR *deviceName) { HRESULT hr = S_OK; HANDLE handle = NULL; DEVMGR_DEVICE_INFORMATION di; GUID guidCamera = { 0xCB998A05, 0x122C, 0x4166, 0x84, 0x6A, 0x93, 0x3E, 0x4D, 0x7E, 0x3C, 0x86 }; di.dwSize = sizeof(di); handle = FindFirstDevice(DeviceSearchByGuid, &guidCamera, &di); StringCchCopy(deviceName, MAX_PATH, di.szLegacyName); } Full header file: #ifndef __CAMERA_H__ #define __CAMERA_H__ class Camera { public: void Init(); void DisplayVideoPreview(HWND windowHandle); void TakePicture(WCHAR *fileName); void SetVideoResolution(int width, int height); void SetPhotoResolution(int width, int height); private: CComPtr<ICaptureGraphBuilder2> _captureGraphBuilder; CComPtr<IGraphBuilder> _filterGraph; CComPtr<IBaseFilter> _videoCaptureFilter; CComPtr<IPersistPropertyBag> _propertyBag; CComPtr<IMediaControl> _mediaControl; CComPtr<IAMVideoControl> _videoControl; CComPtr<IBaseFilter> _imageSinkFilter; void GetDeviceName(WCHAR *deviceName); void InitializeVideoFilter(); void InitializeStillImageFilter(); void CreateComObjects(); void SetResolution(bool video, int width, int height); }; #endif

    Read the article

  • png image store in database and retrieve in android 1.5

    - by hany
    hai, I am new to android. I have problem. This is my code but it will not work, the problem is in view binder. Please correct it. // this is my activity package com.android.Fruits2; import java.util.ArrayList; import java.util.HashMap; import android.app.ListActivity; import android.database.Cursor; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.os.Bundle; import android.widget.SimpleAdapter; import android.widget.SimpleCursorAdapter; import android.widget.SimpleAdapter.ViewBinder; public class Fruits2 extends ListActivity { private DBhelper mDB; /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // setContentView(R.layout.main); mDB = new DBhelper(this); mDB.Reset(); Bitmap img = BitmapFactory.decodeResource(getResources(), R.drawable.icon); mDB.createPersonEntry(new PersonData(img, "Harsha", 24,"mca")); String[] columns = {mDB.KEY_ID, mDB.KEY_IMG, mDB.KEY_NAME, mDB.KEY_AGE, mDB.KEY_STUDY}; String table = mDB.PERSON_TABLE; Cursor c = mDB.getHandle().query(table, columns, null, null, null, null, null); startManagingCursor(c); SimpleCursorAdapter adapter = new SimpleCursorAdapter(this, R.layout.data, c, new String[] {mDB.KEY_IMG, mDB.KEY_NAME, mDB.KEY_AGE, mDB.KEY_STUDY}, new int[] {R.id.img, R.id.name, R.id.age,R.id.study}); adapter.setViewBinder( new MyViewBinder()); setListAdapter(adapter); } } //my viewbinder package com.android.Fruits2; import android.database.Cursor; import android.graphics.BitmapFactory; import android.view.View; import android.widget.ImageView; import android.widget.SimpleCursorAdapter; public class MyViewBinder implements SimpleCursorAdapter.ViewBinder { public boolean setViewValue(View view, Cursor cursor, int columnIndex) { if( (view instanceof ImageView) ) { ImageView iv = (ImageView) view; byte[] img = cursor.getBlob(columnIndex); iv.setImageBitmap(BitmapFactory.decodeByteArray(img, 0, img.length)); return true; } return false; } } // data package com.android.Fruits2; import android.graphics.Bitmap; public class PersonData { private Bitmap bmp; private String name; private int age; private String study; public PersonData(Bitmap b, String n, int k, String v) { bmp = b; name = n; age = k; study = v; } public Bitmap getBitmap() { return bmp; } public String getName() { return name; } public int getAge() { return age; } public String getStudy() { return study; } } //dbhelper package com.android.Fruits2; import java.io.ByteArrayOutputStream; import android.content.ContentValues; import android.content.Context; import android.database.sqlite.SQLiteDatabase; import android.database.sqlite.SQLiteOpenHelper; import android.graphics.Bitmap; import android.provider.BaseColumns; public class DBhelper { public static final String KEY_ID = BaseColumns._ID; public static final String KEY_NAME = "name"; public static final String KEY_AGE = "age"; public static final String KEY_STUDY = "study"; public static final String KEY_IMG = "image"; private DatabaseHelper mDbHelper; private SQLiteDatabase mDb; private static final String DATABASE_NAME = "PersonalDB"; private static final int DATABASE_VERSION = 1; public static final String PERSON_TABLE = "Person"; private static final String CREATE_PERSON_TABLE = "create table "+PERSON_TABLE+" (" +KEY_ID+" integer primary key autoincrement, " +KEY_IMG+" blob not null, " +KEY_NAME+" text not null , " +KEY_AGE+" integer not null, " +KEY_STUDY+" text not null);"; private final Context mCtx; private boolean opened = false; private static class DatabaseHelper extends SQLiteOpenHelper { DatabaseHelper(Context context) { super(context, DATABASE_NAME, null, DATABASE_VERSION); } public void onCreate(SQLiteDatabase db) { db.execSQL(CREATE_PERSON_TABLE); } public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) { db.execSQL("DROP TABLE IF EXISTS "+PERSON_TABLE); onCreate(db); } } public void Reset() { openDB(); mDbHelper.onUpgrade(this.mDb, 1, 1); closeDB(); } public DBhelper(Context ctx) { mCtx = ctx; mDbHelper = new DatabaseHelper(mCtx); } private SQLiteDatabase openDB() { if(!opened) mDb = mDbHelper.getWritableDatabase(); opened = true; return mDb; } public SQLiteDatabase getHandle() { return openDB(); } private void closeDB() { if(opened) mDbHelper.close(); opened = false; } public void createPersonEntry(PersonData about) { openDB(); ByteArrayOutputStream out = new ByteArrayOutputStream(); about.getBitmap().compress(Bitmap.CompressFormat.PNG, 100, out); ContentValues cv = new ContentValues(); cv.put(KEY_IMG, out.toByteArray()); cv.put(KEY_NAME, about.getName()); cv.put(KEY_AGE, about.getAge()); cv.put(KEY_STUDY, about.getStudy()); mDb.insert(PERSON_TABLE, null, cv); closeDB(); } } //data.xml <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="fill_parent" android:layout_height="wrap_content"> <ImageView android:id = "@+id/img" android:layout_width = "wrap_content" android:layout_height = "wrap_content" > </ImageView> <TextView android:id = "@+id/name" android:layout_width = "wrap_content" android:layout_height = "wrap_content" android:textSize="15dp" android:textColor="#ff0000" > </TextView> <TextView android:id = "@+id/age" android:layout_width = "wrap_content" android:layout_height = "wrap_content" android:textSize="15dp" android:textColor="#ff0000" /> <TextView android:id = "@+id/study" android:layout_width = "wrap_content" android:layout_height = "wrap_content" android:textSize="15dp" android:textColor="#ff0000" /> </LinearLayout> When I run this in android 1.6 and 2.1, it works. But when I run in android 1.5, not work. My application is android 1.5. Please correct and send code to me. Thank you.

    Read the article

  • Binary data instead of actual image in C#

    - by acadia
    Hello, I am using the below mentioned library to create a barcode which is storing in a specified location as shown below: My question is, is there a way instead of saving it to a png file I get byte data? thanks Code39 code = new Code39("10090"); code.Paint().Save("c:/NewBARCODE.png", ImageFormat.Png); using System; using System.Collections.Generic; using System.Drawing; using System.Drawing.Imaging; using System.Diagnostics; namespace BarCode39 { public class Code39Settings { private int height = 60; public int BarCodeHeight { get { return height; } set { height = value; } } private bool drawText = true; public bool DrawText { get { return drawText; } set { drawText = value; } } private int leftMargin = 10; public int LeftMargin { get { return leftMargin; } set { leftMargin = value; } } private int rightMargin = 10; public int RightMargin { get { return rightMargin; } set { rightMargin = value; } } private int topMargin = 10; public int TopMargin { get { return topMargin; } set { topMargin = value; } } private int bottomMargin = 10; public int BottomMargin { get { return bottomMargin; } set { bottomMargin = value; } } private int interCharacterGap = 2; public int InterCharacterGap { get { return interCharacterGap; } set { interCharacterGap = value; } } private int wideWidth = 2; public int WideWidth { get { return wideWidth; } set { wideWidth = value; } } private int narrowWidth = 1; public int NarrowWidth { get { return narrowWidth; } set { narrowWidth = value; } } private Font font = new Font(FontFamily.GenericSansSerif, 12); public Font Font { get { return font; } set { font = value; } } private int codeToTextGapHeight = 10; public int BarCodeToTextGapHeight { get { return codeToTextGapHeight; } set { codeToTextGapHeight = value; } } } public class Code39 { #region Static initialization static Dictionary<char, Pattern> codes; static Code39() { object[][] chars = new object[][] { new object[] {'0', "n n n w w n w n n"}, new object[] {'1', "w n n w n n n n w"}, new object[] {'2', "n n w w n n n n w"}, new object[] {'3', "w n w w n n n n n"}, new object[] {'4', "n n n w w n n n w"}, new object[] {'5', "w n n w w n n n n"}, new object[] {'6', "n n w w w n n n n"}, new object[] {'7', "n n n w n n w n w"}, new object[] {'8', "w n n w n n w n n"}, new object[] {'9', "n n w w n n w n n"}, new object[] {'A', "w n n n n w n n w"}, new object[] {'B', "n n w n n w n n w"}, new object[] {'C', "w n w n n w n n n"}, new object[] {'D', "n n n n w w n n w"}, new object[] {'E', "w n n n w w n n n"}, new object[] {'F', "n n w n w w n n n"}, new object[] {'G', "n n n n n w w n w"}, new object[] {'H', "w n n n n w w n n"}, new object[] {'I', "n n w n n w w n n"}, new object[] {'J', "n n n n w w w n n"}, new object[] {'K', "w n n n n n n w w"}, new object[] {'L', "n n w n n n n w w"}, new object[] {'M', "w n w n n n n w n"}, new object[] {'N', "n n n n w n n w w"}, new object[] {'O', "w n n n w n n w n"}, new object[] {'P', "n n w n w n n w n"}, new object[] {'Q', "n n n n n n w w w"}, new object[] {'R', "w n n n n n w w n"}, new object[] {'S', "n n w n n n w w n"}, new object[] {'T', "n n n n w n w w n"}, new object[] {'U', "w w n n n n n n w"}, new object[] {'V', "n w w n n n n n w"}, new object[] {'W', "w w w n n n n n n"}, new object[] {'X', "n w n n w n n n w"}, new object[] {'Y', "w w n n w n n n n"}, new object[] {'Z', "n w w n w n n n n"}, new object[] {'-', "n w n n n n w n w"}, new object[] {'.', "w w n n n n w n n"}, new object[] {' ', "n w w n n n w n n"}, new object[] {'*', "n w n n w n w n n"}, new object[] {'$', "n w n w n w n n n"}, new object[] {'/', "n w n w n n n w n"}, new object[] {'+', "n w n n n w n w n"}, new object[] {'%', "n n n w n w n w n"} }; codes = new Dictionary<char, Pattern>(); foreach (object[] c in chars) codes.Add((char)c[0], Pattern.Parse((string)c[1])); } #endregion private static Pen pen = new Pen(Color.Black); private static Brush brush = Brushes.Black; private string code; private Code39Settings settings; public Code39(string code) : this(code, new Code39Settings()) { } public Code39(string code, Code39Settings settings) { foreach (char c in code) if (!codes.ContainsKey(c)) throw new ArgumentException("Invalid character encountered in specified code."); if (!code.StartsWith("*")) code = "*" + code; if (!code.EndsWith("*")) code = code + "*"; this.code = code; this.settings = settings; } public Bitmap Paint() { string code = this.code.Trim('*'); SizeF sizeCodeText = Graphics.FromImage(new Bitmap(1, 1)).MeasureString(code, settings.Font); int w = settings.LeftMargin + settings.RightMargin; foreach (char c in this.code) w += codes[c].GetWidth(settings) + settings.InterCharacterGap; w -= settings.InterCharacterGap; int h = settings.TopMargin + settings.BottomMargin + settings.BarCodeHeight; if (settings.DrawText) h += settings.BarCodeToTextGapHeight + (int)sizeCodeText.Height; Bitmap bmp = new Bitmap(w, h, PixelFormat.Format32bppArgb); Graphics g = Graphics.FromImage(bmp); int left = settings.LeftMargin; foreach (char c in this.code) left += codes[c].Paint(settings, g, left) + settings.InterCharacterGap; if (settings.DrawText) { int tX = settings.LeftMargin + (w - settings.LeftMargin - settings.RightMargin - (int)sizeCodeText.Width) / 2; if (tX < 0) tX = 0; int tY = settings.TopMargin + settings.BarCodeHeight + settings.BarCodeToTextGapHeight; g.DrawString(code, settings.Font, brush, tX, tY); } return bmp; } private class Pattern { private bool[] nw = new bool[9]; public static Pattern Parse(string s) { Debug.Assert(s != null); s = s.Replace(" ", "").ToLower(); Debug.Assert(s.Length == 9); Debug.Assert(s.Replace("n", "").Replace("w", "").Length == 0); Pattern p = new Pattern(); int i = 0; foreach (char c in s) p.nw[i++] = c == 'w'; return p; } public int GetWidth(Code39Settings settings) { int width = 0; for (int i = 0; i < 9; i++) width += (nw[i] ? settings.WideWidth : settings.NarrowWidth); return width; } public int Paint(Code39Settings settings, Graphics g, int left) { #if DEBUG Rectangle gray = new Rectangle(left, 0, GetWidth(settings), settings.BarCodeHeight + settings.TopMargin + settings.BottomMargin); g.FillRectangle(Brushes.Gray, gray); #endif int x = left; int w = 0; for (int i = 0; i < 9; i++) { int width = (nw[i] ? settings.WideWidth : settings.NarrowWidth); if (i % 2 == 0) { Rectangle r = new Rectangle(x, settings.TopMargin, width, settings.BarCodeHeight); g.FillRectangle(brush, r); } x += width; w += width; } return w; } } } }

    Read the article

  • Weblogic 10.0: SAMLSignedObject.verify() failed to validate signature value

    - by joshea
    I've been having this problem for a while and it's driving me nuts. I'm trying to create a client (in C# .NET 2.0) that will use SAML 1.1 to sign on to a WebLogic 10.0 server (i.e., a Single Sign-On scenario, using browser/post profile). The client is on a WinXP machine and the WebLogic server is on a RHEL 5 box. I based my client largely on code in the example here: http://www.codeproject.com/KB/aspnet/DotNetSamlPost.aspx (the source has a section for SAML 1.1). I set up WebLogic based on instructions for SAML Destination Site from here:http://www.oracle.com/technology/pub/articles/dev2arch/2006/12/sso-with-saml4.html I created a certificate using makecert that came with VS 2005. makecert -r -pe -n "CN=whatever" -b 01/01/2010 -e 01/01/2011 -sky exchange whatever.cer -sv whatever.pvk pvk2pfx.exe -pvk whatever.pvk -spc whatever.cer -pfx whatever.pfx Then I installed the .pfx to my personal certificate directory, and installed the .cer into the WebLogic SAML Identity Asserter V2. I read on another site that formatting the response to be readable (ie, adding whitespace) to the response after signing would cause this problem, so I tried various combinations of turning on/off .Indent XMLWriterSettings and turning on/off .PreserveWhiteSpace when loading the XML document, and none of it made any difference. I've printed the SignatureValue both before the message is is encoded/sent and after it arrives/gets decoded, and they are the same. So, to be clear: the Response appears to be formed, encoded, sent, and decoded fine (I see the full Response in the WebLogic logs). WebLogic finds the certificate I want it to use, verifies that a key was supplied, gets the signed info, and then fails to validate the signature. Code: public string createResponse(Dictionary<string, string> attributes){ ResponseType response = new ResponseType(); // Create Response response.ResponseID = "_" + Guid.NewGuid().ToString(); response.MajorVersion = "1"; response.MinorVersion = "1"; response.IssueInstant = System.DateTime.UtcNow; response.Recipient = "http://theWLServer/samlacs/acs"; StatusType status = new StatusType(); status.StatusCode = new StatusCodeType(); status.StatusCode.Value = new XmlQualifiedName("Success", "urn:oasis:names:tc:SAML:1.0:protocol"); response.Status = status; // Create Assertion AssertionType assertionType = CreateSaml11Assertion(attributes); response.Assertion = new AssertionType[] {assertionType}; //Serialize XmlSerializerNamespaces ns = new XmlSerializerNamespaces(); ns.Add("samlp", "urn:oasis:names:tc:SAML:1.0:protocol"); ns.Add("saml", "urn:oasis:names:tc:SAML:1.0:assertion"); XmlSerializer responseSerializer = new XmlSerializer(response.GetType()); StringWriter stringWriter = new StringWriter(); XmlWriterSettings settings = new XmlWriterSettings(); settings.OmitXmlDeclaration = true; settings.Indent = false;//I've tried both ways, for the fun of it settings.Encoding = Encoding.UTF8; XmlWriter responseWriter = XmlTextWriter.Create(stringWriter, settings); responseSerializer.Serialize(responseWriter, response, ns); responseWriter.Close(); string samlString = stringWriter.ToString(); stringWriter.Close(); // Sign the document XmlDocument doc = new XmlDocument(); doc.PreserveWhiteSpace = true; //also tried this both ways to no avail doc.LoadXml(samlString); X509Certificate2 cert = null; X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser); store.Open(OpenFlags.ReadOnly); X509Certificate2Collection coll = store.Certificates.Find(X509FindType.FindBySubjectDistinguishedName, "distName", true); if (coll.Count < 1) { throw new ArgumentException("Unable to locate certificate"); } cert = coll[0]; store.Close(); //this special SignDoc just overrides a function in SignedXml so //it knows to look for ResponseID rather than ID XmlElement signature = SamlHelper.SignDoc( doc, cert, "ResponseID", response.ResponseID); doc.DocumentElement.InsertBefore(signature, doc.DocumentElement.ChildNodes[0]); // Base64Encode and URL Encode byte[] base64EncodedBytes = Encoding.UTF8.GetBytes(doc.OuterXml); string returnValue = System.Convert.ToBase64String( base64EncodedBytes); return returnValue; } private AssertionType CreateSaml11Assertion(Dictionary<string, string> attributes){ AssertionType assertion = new AssertionType(); assertion.AssertionID = "_" + Guid.NewGuid().ToString(); assertion.Issuer = "madeUpValue"; assertion.MajorVersion = "1"; assertion.MinorVersion = "1"; assertion.IssueInstant = System.DateTime.UtcNow; //Not before, not after conditions ConditionsType conditions = new ConditionsType(); conditions.NotBefore = DateTime.UtcNow; conditions.NotBeforeSpecified = true; conditions.NotOnOrAfter = DateTime.UtcNow.AddMinutes(10); conditions.NotOnOrAfterSpecified = true; //Name Identifier to be used in Saml Subject NameIdentifierType nameIdentifier = new NameIdentifierType(); nameIdentifier.NameQualifier = domain.Trim(); nameIdentifier.Value = subject.Trim(); SubjectConfirmationType subjectConfirmation = new SubjectConfirmationType(); subjectConfirmation.ConfirmationMethod = new string[] { "urn:oasis:names:tc:SAML:1.0:cm:bearer" }; // // Create some SAML subject. SubjectType samlSubject = new SubjectType(); AttributeStatementType attrStatement = new AttributeStatementType(); AuthenticationStatementType authStatement = new AuthenticationStatementType(); authStatement.AuthenticationMethod = "urn:oasis:names:tc:SAML:1.0:am:password"; authStatement.AuthenticationInstant = System.DateTime.UtcNow; samlSubject.Items = new object[] { nameIdentifier, subjectConfirmation}; attrStatement.Subject = samlSubject; authStatement.Subject = samlSubject; IPHostEntry ipEntry = Dns.GetHostEntry(System.Environment.MachineName); SubjectLocalityType subjectLocality = new SubjectLocalityType(); subjectLocality.IPAddress = ipEntry.AddressList[0].ToString(); authStatement.SubjectLocality = subjectLocality; attrStatement.Attribute = new AttributeType[attributes.Count]; int i=0; // Create SAML attributes. foreach (KeyValuePair<string, string> attribute in attributes) { AttributeType attr = new AttributeType(); attr.AttributeName = attribute.Key; attr.AttributeNamespace= domain; attr.AttributeValue = new object[] {attribute.Value}; attrStatement.Attribute[i] = attr; i++; } assertion.Conditions = conditions; assertion.Items = new StatementAbstractType[] {authStatement, attrStatement}; return assertion; } private static XmlElement SignDoc(XmlDocument doc, X509Certificate2 cert2, string referenceId, string referenceValue) { // Use our own implementation of SignedXml SamlSignedXml sig = new SamlSignedXml(doc, referenceId); // Add the key to the SignedXml xmlDocument. sig.SigningKey = cert2.PrivateKey; // Create a reference to be signed. Reference reference = new Reference(); reference.Uri= String.Empty; reference.Uri = "#" + referenceValue; // Add an enveloped transformation to the reference. XmlDsigEnvelopedSignatureTransform env = new XmlDsigEnvelopedSignatureTransform(); reference.AddTransform(env); // Add the reference to the SignedXml object. sig.AddReference(reference); // Add an RSAKeyValue KeyInfo (optional; helps recipient find key to validate). KeyInfo keyInfo = new KeyInfo(); keyInfo.AddClause(new KeyInfoX509Data(cert2)); sig.KeyInfo = keyInfo; // Compute the signature. sig.ComputeSignature(); // Get the XML representation of the signature and save // it to an XmlElement object. XmlElement xmlDigitalSignature = sig.GetXml(); return xmlDigitalSignature; } To open the page in my client app, string postData = String.Format("SAMLResponse={0}&APID=ap_00001&TARGET={1}", System.Web.HttpUtility.UrlEncode(builder.buildResponse("http://theWLServer/samlacs/acs",attributes)), "http://desiredURL"); webBrowser.Navigate("http://theWLServer/samlacs/acs", "_self", Encoding.UTF8.GetBytes(postData), "Content-Type: application/x-www-form-urlencoded");

    Read the article

  • InputDispatcher Error

    - by StarDust
    INFO/ActivityManager(68): Process com.example (pid 390) has died. ERROR/InputDispatcher(68): channel '406ed580 com.example/com.example.afeTest (server)' ~ Consumer closed input channel or an error occurred. events=0x8 ERROR/InputDispatcher(68): channel '406ed580 com.example/com.example.afeTest (server)' ~ Channel is unrecoverably broken and will be disposed! ERROR/InputDispatcher(68): Received spurious receive callback for unknown input channel. fd=165, events=0x8 Can anyone tell what may be the reason behind this error? I've ported a native code on the Android-ndk. One thing I noticed regarding fd (that may be some reason :S) My code uses fd_sets which was defined in winsock2.h But I didn't find fd_sets defined in android-ndk. So I had included "select.h" where fd_set is a typedef in the android-ndk: typedef __kernel_fd_set fd_set; Here is the log cat: 04-06 11:15:32.405: INFO/DEBUG(31): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** 04-06 11:15:32.405: INFO/DEBUG(31): Build fingerprint: 'generic/sdk/generic:2.3.3/GRI34/101070:eng/test-keys' 04-06 11:15:32.415: INFO/DEBUG(31): pid: 335, tid: 348 >>> com.example <<< 04-06 11:15:32.426: INFO/DEBUG(31): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad 04-06 11:15:32.426: INFO/DEBUG(31): r0 deadbaad r1 0000000c r2 00000027 r3 00000000 04-06 11:15:32.445: INFO/DEBUG(31): r4 00000080 r5 afd46668 r6 0000a000 r7 00000078 04-06 11:15:32.445: INFO/DEBUG(31): r8 804ab00d r9 002a9778 10 00100000 fp 00000001 04-06 11:15:32.445: INFO/DEBUG(31): ip ffffffff sp 44295d10 lr afd19375 pc afd15ef0 cpsr 00000030 04-06 11:15:32.756: INFO/DEBUG(31): #00 pc 00015ef0 /system/lib/libc.so 04-06 11:15:32.756: INFO/DEBUG(31): #01 pc 00013852 /system/lib/libc.so 04-06 11:15:32.767: INFO/DEBUG(31): code around pc: 04-06 11:15:32.785: INFO/DEBUG(31): afd15ed0 68241c23 d1fb2c00 68dae027 d0042a00 04-06 11:15:32.785: INFO/DEBUG(31): afd15ee0 20014d18 6028447d 48174790 24802227 04-06 11:15:32.785: INFO/DEBUG(31): afd15ef0 f7f57002 2106eb56 ec92f7f6 0563aa01 04-06 11:15:32.796: INFO/DEBUG(31): afd15f00 60932100 91016051 1c112006 e818f7f6 04-06 11:15:32.807: INFO/DEBUG(31): afd15f10 2200a905 f7f62002 f7f5e824 2106eb42 04-06 11:15:32.815: INFO/DEBUG(31): code around lr: 04-06 11:15:32.815: INFO/DEBUG(31): afd19354 b0834a0d 589c447b 26009001 686768a5 04-06 11:15:32.825: INFO/DEBUG(31): afd19364 220ce008 2b005eab 1c28d003 47889901 04-06 11:15:32.836: INFO/DEBUG(31): afd19374 35544306 d5f43f01 2c006824 b003d1ee 04-06 11:15:32.836: INFO/DEBUG(31): afd19384 bdf01c30 000281a8 ffffff88 1c0fb5f0 04-06 11:15:32.846: INFO/DEBUG(31): afd19394 43551c3d a904b087 1c16ac01 604d9004 04-06 11:15:32.856: INFO/DEBUG(31): stack: 04-06 11:15:32.856: INFO/DEBUG(31): 44295cd0 00000408 04-06 11:15:32.867: INFO/DEBUG(31): 44295cd4 afd18407 /system/lib/libc.so 04-06 11:15:32.875: INFO/DEBUG(31): 44295cd8 afd4270c /system/lib/libc.so 04-06 11:15:32.875: INFO/DEBUG(31): 44295cdc afd426b8 /system/lib/libc.so 04-06 11:15:32.885: INFO/DEBUG(31): 44295ce0 00000000 04-06 11:15:32.896: INFO/DEBUG(31): 44295ce4 afd19375 /system/lib/libc.so 04-06 11:15:32.896: INFO/DEBUG(31): 44295ce8 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:32.896: INFO/DEBUG(31): 44295cec afd183d9 /system/lib/libc.so 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf0 00000078 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf4 00000000 04-06 11:15:32.906: INFO/DEBUG(31): 44295cf8 afd46668 04-06 11:15:32.906: INFO/DEBUG(31): 44295cfc 0000a000 [heap] 04-06 11:15:32.916: INFO/DEBUG(31): 44295d00 00000078 04-06 11:15:32.927: INFO/DEBUG(31): 44295d04 afd18677 /system/lib/libc.so 04-06 11:15:32.927: INFO/DEBUG(31): 44295d08 df002777 04-06 11:15:32.945: INFO/DEBUG(31): 44295d0c e3a070ad 04-06 11:15:32.945: INFO/DEBUG(31): #00 44295d10 002c43a0 [heap] 04-06 11:15:32.945: INFO/DEBUG(31): 44295d14 002a9900 [heap] 04-06 11:15:32.956: INFO/DEBUG(31): 44295d18 afd46608 04-06 11:15:32.966: INFO/DEBUG(31): 44295d1c afd11010 /system/lib/libc.so 04-06 11:15:32.976: INFO/DEBUG(31): 44295d20 002c4298 [heap] 04-06 11:15:32.976: INFO/DEBUG(31): 44295d24 fffffbdf 04-06 11:15:33.006: INFO/DEBUG(31): 44295d28 000000da 04-06 11:15:33.006: INFO/DEBUG(31): 44295d2c afd46450 04-06 11:15:33.006: INFO/DEBUG(31): 44295d30 000001b4 04-06 11:15:33.026: INFO/DEBUG(31): 44295d34 afd13857 /system/lib/libc.so 04-06 11:15:33.026: INFO/DEBUG(31): #01 44295d38 afd46450 04-06 11:15:33.035: INFO/DEBUG(31): 44295d3c afd13857 /system/lib/libc.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d40 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d44 44295e8c 04-06 11:15:33.056: INFO/DEBUG(31): 44295d48 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d4c 804bfec3 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.056: INFO/DEBUG(31): 44295d50 002c43a0 [heap] 04-06 11:15:33.066: INFO/DEBUG(31): 44295d54 44295e8c 04-06 11:15:33.066: INFO/DEBUG(31): 44295d58 804ab00d /data/data/com.example/lib/libAFE.so 04-06 11:15:33.076: INFO/DEBUG(31): 44295d5c 002a9778 [heap] 04-06 11:15:33.085: INFO/DEBUG(31): 44295d60 00000078 04-06 11:15:33.085: INFO/DEBUG(31): 44295d64 afd14769 /system/lib/libc.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d68 44295e8c 04-06 11:15:33.085: INFO/DEBUG(31): 44295d6c 805d9763 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d70 44295e8c 04-06 11:15:33.085: INFO/DEBUG(31): 44295d74 8051dc35 /data/data/com.example/lib/libAFE.so 04-06 11:15:33.085: INFO/DEBUG(31): 44295d78 0000003a 04-06 11:15:33.085: INFO/DEBUG(31): 44295d7c 002a9900 [heap] 04-06 11:15:37.126: DEBUG/Zygote(33): Process 335 terminated by signal (11) 04-06 11:15:37.146: INFO/ActivityManager(68): Process com.example (pid 335) has died. 04-06 11:15:37.178: ERROR/InputDispatcher(68): channel '406f03a0 com.example/com.example.afeTest (server)' ~ Consumer closed input channel or an error occurred. events=0x8 04-06 11:15:37.178: ERROR/InputDispatcher(68): channel '406f03a0 com.example/com.example.afeTest (server)' ~ Channel is unrecoverably broken and will be disposed! 04-06 11:15:37.185: INFO/BootReceiver(68): Copying /data/tombstones/tombstone_09 to DropBox (SYSTEM_TOMBSTONE) 04-06 11:15:37.576: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 266K, 47% free 4404K/8199K, external 3520K/3903K, paused 306ms 04-06 11:15:37.835: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 203K, 47% free 4457K/8391K, external 3520K/3903K, paused 120ms 04-06 11:15:37.886: INFO/WindowManager(68): WIN DEATH: Window{406f03a0 com.example/com.example.afeTest paused=false} 04-06 11:15:38.095: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 67K, 47% free 4518K/8391K, external 3511K/3903K, paused 94ms 04-06 11:15:38.095: INFO/dalvikvm-heap(68): Grow heap (frag case) to 10.575MB for 196628-byte allocation 04-06 11:15:38.126: DEBUG/dalvikvm(126): GC_EXPLICIT freed 110K, 51% free 2903K/5895K, external 4701K/5293K, paused 2443ms 04-06 11:15:38.217: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 1K, 46% free 4708K/8647K, external 3511K/3903K, paused 96ms 04-06 11:15:38.225: INFO/WindowManager(68): WIN DEATH: Window{406f72f8 com.example/com.example.afeTest paused=false} 04-06 11:15:38.405: DEBUG/dalvikvm(68): GC_FOR_MALLOC freed 492K, 50% free 4345K/8647K, external 3511K/3903K, paused 96ms 04-06 11:15:38.485: ERROR/InputDispatcher(68): Received spurious receive callback for unknown input channel. fd=164, events=0x8

    Read the article

  • Probelm with String.split() in java

    - by Matt
    What I am trying to do is read a .java file, and pick out all of the identifiers and store them in a list. My problem is with the .split() method. If you run this code the way it is, you will get ArrayOutOfBounds, but if you change the delimiter from "." to anything else, the code works. But I need to lines parsed by "." so is there another way I could accomplish this? import java.io.BufferedReader; import java.io.FileNotFoundException; import java.io.FileReader; import java.io.IOException; import java.util.*; public class MyHash { private static String[] reserved = new String[100]; private static List list = new LinkedList(); private static List list2 = new LinkedList(); public static void main (String args[]){ Hashtable hashtable = new Hashtable(997); makeReserved(); readFile(); String line; ListIterator itr = list.listIterator(); int listIndex = 0; while (listIndex < list.size()) { if (itr.hasNext()){ line = itr.next().toString(); //PROBLEM IS HERE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! String[] words = line.split("."); //CHANGE THIS AND IT WILL WORK System.out.println(words[0]); //TESTING TO SEE IF IT WORKED } listIndex++; } } public static void readFile() { String text; String[] words; BufferedReader in = null; try { in = new BufferedReader(new FileReader("MyHash.java")); //NAME OF INPUT FILE } catch (FileNotFoundException ex) { Logger.getLogger(MyHash.class.getName()).log(Level.SEVERE, null, ex); } try { while ((text = in.readLine()) != null){ text = text.trim(); words = text.split("\\s+"); for (int i = 0; i < words.length; i++){ list.add(words[i]); } for (int j = 0; j < reserved.length; j++){ if (list.contains(reserved[j])){ list.remove(reserved[j]); } } } } catch (IOException ex) { Logger.getLogger(MyHash.class.getName()).log(Level.SEVERE, null, ex); } try { in.close(); } catch (IOException ex) { Logger.getLogger(MyHash.class.getName()).log(Level.SEVERE, null, ex); } } public static int keyIt (int x) { int key = x % 997; return key; } public static int horner (String word){ int length = word.length(); char[] letters = new char[length]; for (int i = 0; i < length; i++){ letters[i]=word.charAt(i); } char[] alphabet = new char[26]; String abc = "abcdefghijklmnopqrstuvwxyz"; for (int i = 0; i < 26; i++){ alphabet[i]=abc.charAt(i); } int[] numbers = new int[length]; int place = 0; for (int i = 0; i < length; i++){ for (int j = 0; j < 26; j++){ if (alphabet[j]==letters[i]){ numbers[place]=j+1; place++; } } } int hornered = numbers[0] * 32; for (int i = 1; i < numbers.length; i++){ hornered += numbers[i]; if (i == numbers.length -1){ return hornered; } hornered = hornered % 997; hornered *= 32; } return hornered; } public static String[] makeReserved (){ reserved[0] = "abstract"; reserved[1] = "assert"; reserved[2] = "boolean"; reserved[3] = "break"; reserved[4] = "byte"; reserved[5] = "case"; reserved[6] = "catch"; reserved[7] = "char"; reserved[8] = "class"; reserved[9] = "const"; reserved[10] = "continue"; reserved[11] = "default"; reserved[12] = "do"; reserved[13] = "double"; reserved[14] = "else"; reserved[15] = "enum"; reserved[16] = "extends"; reserved[17] = "false"; reserved[18] = "final"; reserved[19] = "finally"; reserved[20] = "float"; reserved[21] = "for"; reserved[22] = "goto"; reserved[23] = "if"; reserved[24] = "implements"; reserved[25] = "import"; reserved[26] = "instanceof"; reserved[27] = "int"; reserved[28] = "interface"; reserved[29] = "long"; reserved[30] = "native"; reserved[31] = "new"; reserved[32] = "null"; reserved[33] = "package"; reserved[34] = "private"; reserved[35] = "protected"; reserved[36] = "public"; reserved[37] = "return"; reserved[38] = "short"; reserved[39] = "static"; reserved[40] = "strictfp"; reserved[41] = "super"; reserved[42] = "switch"; reserved[43] = "synchronize"; reserved[44] = "this"; reserved[45] = "throw"; reserved[46] = "throws"; reserved[47] = "trasient"; reserved[48] = "true"; reserved[49] = "try"; reserved[50] = "void"; reserved[51] = "volatile"; reserved[52] = "while"; reserved[53] = "="; reserved[54] = "=="; reserved[55] = "!="; reserved[56] = "+"; reserved[57] = "-"; reserved[58] = "*"; reserved[59] = "/"; reserved[60] = "{"; reserved[61] = "}"; return reserved; } }

    Read the article

  • How would I define "GetDataFromNumber" so that my class contains a definition?

    - by JB
    My code gets an error saying: 'Eagle_Eye_Class_Finder.GetSchedule' does not contain a definition for 'GetDataFromNumber' and no extension method 'GetDataFromNumber'. using System; using System.IO; using System.Data; using System.Text; using System.Drawing; using System.Data.OleDb; using System.Collections; using System.Windows.Forms; using System.ComponentModel; using System.Drawing.Printing; using System.Collections.Generic; namespace Eagle_Eye_Class_Finder { /// This form is the entry form, it is the first form the user will see when the app is run. /// public class Form1 : System.Windows.Forms.Form { private System.Windows.Forms.TextBox textBox1; private System.Windows.Forms.ProgressBar progressBar1; private System.Windows.Forms.PictureBox pictureBox1; private System.Windows.Forms.Button button2; private System.Windows.Forms.DateTimePicker dateTimePicker1; private IContainer components; private Timer timer1; private BindingSource form1BindingSource; public static Form Mainform = null; // creates new instance of second form YOURCLASSSCHEDULE SecondForm = new YOURCLASSSCHEDULE(); public Form1() { InitializeComponent(); // TODO: Add any constructor code after InitializeComponent call } /// Clean up any resources being used. protected override void Dispose(bool disposing) { if (disposing) { if (components != null) { components.Dispose(); } } base.Dispose(disposing); } #region Windows Form Designer generated code /// <summary> /// Required method for Designer support - do not modify /// the contents of this method with the code editor. /// </summary> private void InitializeComponent() { this.components = new System.ComponentModel.Container(); System.ComponentModel.ComponentResourceManager resources = new System.ComponentModel.ComponentResourceManager(typeof(Form1)); this.textBox1 = new System.Windows.Forms.TextBox(); this.progressBar1 = new System.Windows.Forms.ProgressBar(); this.pictureBox1 = new System.Windows.Forms.PictureBox(); this.button2 = new System.Windows.Forms.Button(); this.dateTimePicker1 = new System.Windows.Forms.DateTimePicker(); this.timer1 = new System.Windows.Forms.Timer(this.components); this.form1BindingSource = new System.Windows.Forms.BindingSource(this.components); ((System.ComponentModel.ISupportInitialize)(this.pictureBox1)).BeginInit(); ((System.ComponentModel.ISupportInitialize)(this.form1BindingSource)).BeginInit(); this.SuspendLayout(); // // textBox1 // this.textBox1.BackColor = System.Drawing.SystemColors.ActiveCaption; this.textBox1.DataBindings.Add(new System.Windows.Forms.Binding("Text", this.form1BindingSource, "Text", true, System.Windows.Forms.DataSourceUpdateMode.OnValidation, null, "900456317")); this.textBox1.Location = new System.Drawing.Point(328, 280); this.textBox1.Name = "textBox1"; this.textBox1.Size = new System.Drawing.Size(208, 20); this.textBox1.TabIndex = 2; this.textBox1.TextChanged += new System.EventHandler(this.textBox1_TextChanged); // // progressBar1 // this.progressBar1.Location = new System.Drawing.Point(258, 410); this.progressBar1.MarqueeAnimationSpeed = 10; this.progressBar1.Name = "progressBar1"; this.progressBar1.Size = new System.Drawing.Size(344, 8); this.progressBar1.TabIndex = 3; this.progressBar1.Click += new System.EventHandler(this.progressBar1_Click); // // pictureBox1 // this.pictureBox1.BackColor = System.Drawing.SystemColors.ControlLightLight; this.pictureBox1.BorderStyle = System.Windows.Forms.BorderStyle.Fixed3D; this.pictureBox1.Image = ((System.Drawing.Image)(resources.GetObject("pictureBox1.Image"))); this.pictureBox1.Location = new System.Drawing.Point(680, 400); this.pictureBox1.Name = "pictureBox1"; this.pictureBox1.Size = new System.Drawing.Size(120, 112); this.pictureBox1.TabIndex = 4; this.pictureBox1.TabStop = false; this.pictureBox1.Click += new System.EventHandler(this.pictureBox1_Click); // // button2 // this.button2.Font = new System.Drawing.Font("Mistral", 15.75F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0))); this.button2.Image = ((System.Drawing.Image)(resources.GetObject("button2.Image"))); this.button2.Location = new System.Drawing.Point(699, 442); this.button2.Name = "button2"; this.button2.Size = new System.Drawing.Size(78, 28); this.button2.TabIndex = 5; this.button2.Text = "OK"; this.button2.Click += new System.EventHandler(this.button2_Click); // // dateTimePicker1 // this.dateTimePicker1.Location = new System.Drawing.Point(336, 104); this.dateTimePicker1.Name = "dateTimePicker1"; this.dateTimePicker1.Size = new System.Drawing.Size(200, 20); this.dateTimePicker1.TabIndex = 6; this.dateTimePicker1.ValueChanged += new System.EventHandler(this.dateTimePicker1_ValueChanged); // // timer1 // this.timer1.Tick += new System.EventHandler(this.timer1_Tick); // // form1BindingSource // this.form1BindingSource.DataSource = typeof(Eagle_Eye_Class_Finder.Form1); // // Form1 // this.AcceptButton = this.button2; this.AutoScaleBaseSize = new System.Drawing.Size(5, 13); this.BackgroundImage = ((System.Drawing.Image)(resources.GetObject("$this.BackgroundImage"))); this.BackgroundImageLayout = System.Windows.Forms.ImageLayout.Stretch; this.ClientSize = new System.Drawing.Size(856, 556); this.Controls.Add(this.dateTimePicker1); this.Controls.Add(this.button2); this.Controls.Add(this.pictureBox1); this.Controls.Add(this.progressBar1); this.Controls.Add(this.textBox1); this.Name = "Form1"; this.Text = "Eagle Eye Class Finder"; this.Load += new System.EventHandler(this.Form1_Load); ((System.ComponentModel.ISupportInitialize)(this.pictureBox1)).EndInit(); ((System.ComponentModel.ISupportInitialize)(this.form1BindingSource)).EndInit(); this.ResumeLayout(false); this.PerformLayout(); } #endregion /// The main entry point for the application. [STAThread] static void Main() { Application.Run(new Form1()); } public void Form1_Load(object sender, System.EventArgs e) { } public void textBox1_TextChanged(object sender, System.EventArgs e) { //allows only numbers to be entered in textbox string Str = textBox1.Text.Trim(); double Num; bool isNum = double.TryParse(Str, out Num); if (isNum) Console.ReadLine(); else MessageBox.Show("Enter A Valid ID Number!"); } public void button2_Click(object sender, System.EventArgs e) { string text = textBox1.Text; Mainform = this; this.Hide(); GetSchedule myScheduleFinder = new GetSchedule(); string result = myScheduleFinder.GetDataFromNumber(text);<<<-----MY PROBLEM if (!string.IsNullOrEmpty(result)) { MessageBox.Show(result); } else { MessageBox.Show("Enter A Valid ID Number!"); } } public void dateTimePicker1_ValueChanged(object sender, System.EventArgs e) { } public void pictureBox1_Click(object sender, System.EventArgs e) { } public void progressBar1_Click(object sender, EventArgs e) { //this.progressBar1 = new System.progressBar1(); //progressBar1.Maximum = 200; //progressBar1.Minimum = 0; //progressBar1.Step = 20; } private void timer1_Tick(object sender, EventArgs e) { //if (progressBar1.Value >= 200 ) //{ //progressBar1.Value = 0; //} //return; //} //progressBar1.Value != 20; } } }

    Read the article

  • Get the current location of the Gps? Showing the default one

    - by Gagandeep
    Need help Urgent!!!!! Did changes with help but still unsuccessful... I have to request location updates, but I am unsuccessful in implementing that... i modified the code but need help so that i can see the current location. PLEASE look through my code and help please.. I am learning this and new to this concept and android.. any help would be appreciated here is my code: package com.GoogleMaps; import java.util.List; import com.google.android.maps.GeoPoint; import com.google.android.maps.MapActivity; import com.google.android.maps.MapController; import com.google.android.maps.MapView; import com.google.android.maps.Overlay; import android.content.Context; import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.graphics.Canvas; import android.graphics.Paint; import android.graphics.Point; import android.graphics.drawable.Drawable; import android.location.Location; import android.location.LocationListener; import android.location.LocationManager; import android.os.Bundle; import android.widget.Toast; public class MapsActivity extends MapActivity { /** Called when the activity is first created. */ private MapView mapView; private LocationManager lm; private LocationListener ll; private MapController mc; GeoPoint p = null; Drawable defaultMarker = null; @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.main); mapView = (MapView)findViewById(R.id.mapview); //show zoom in/out buttons mapView.setBuiltInZoomControls(true); //Standard view of the map(map/sat) mapView.setSatellite(false); // get zoom tool mapView.setBuiltInZoomControls(true); //get controller of the map for zooming in/out mc = mapView.getController(); // Zoom Level mc.setZoom(18); lm = (LocationManager)getSystemService(Context.LOCATION_SERVICE); ll = new MyLocationListener(); lm.requestLocationUpdates( LocationManager.GPS_PROVIDER, 0, 0, ll); //Get the current location in start-up lm = (LocationManager)getSystemService(Context.LOCATION_SERVICE); ll = new MyLocationListener(); lm.requestLocationUpdates( LocationManager.GPS_PROVIDER, 0, 0, ll); //Get the current location in start-up if (lm.getLastKnownLocation(LocationManager.GPS_PROVIDER) != null){ GeoPoint p = new GeoPoint( (int)(lm.getLastKnownLocation(LocationManager.GPS_PROVIDER).getLatitude()*1000000), (int)(lm.getLastKnownLocation(LocationManager.GPS_PROVIDER).getLongitude()*1000000)); mc.animateTo(p); } MyLocationOverlay myLocationOverlay = new MyLocationOverlay(); List<Overlay> list = mapView.getOverlays(); list.add(myLocationOverlay); } protected class MyLocationOverlay extends com.google.android.maps.Overlay { @Override public boolean draw(Canvas canvas, MapView mapView, boolean shadow, long when) { Paint paint = new Paint(); super.draw(canvas, mapView, shadow); GeoPoint p = null; // Converts lat/lng-Point to OUR coordinates on the screen. Point myScreenCoords = new Point(); mapView.getProjection().toPixels(p, myScreenCoords); paint.setStrokeWidth(1); paint.setARGB(255, 255, 255, 255); paint.setStyle(Paint.Style.STROKE); Bitmap bmp = BitmapFactory.decodeResource(getResources(), R.drawable.ic_launcher); canvas.drawBitmap(bmp, myScreenCoords.x, myScreenCoords.y, paint); canvas.drawText("I am here...", myScreenCoords.x, myScreenCoords.y, paint); return true; } } private class MyLocationListener implements LocationListener{ public void onLocationChanged(Location argLocation) { // TODO Auto-generated method stub p = new GeoPoint((int)(argLocation.getLatitude()*1000000), (int)(argLocation.getLongitude()*1000000)); Toast.makeText(getBaseContext(), "New location latitude [" +argLocation.getLatitude() + "] longitude [" + argLocation.getLongitude()+"]", Toast.LENGTH_SHORT).show(); mc.animateTo(p); mapView.invalidate(); // call this so UI of map was updated } public void onProviderDisabled(String provider) { // TODO Auto-generated method stub } public void onProviderEnabled(String provider) { // TODO Auto-generated method stub } public void onStatusChanged(String provider, int status, Bundle extras) { // TODO Auto-generated method stub } } protected boolean isRouteDisplayed() { return false; } } catlog: 11-29 17:40:42.699: D/dalvikvm(371): GC_FOR_MALLOC freed 6074 objects / 369952 bytes in 74ms 11-29 17:40:42.970: I/MapActivity(371): Handling network change notification:CONNECTED 11-29 17:40:42.980: E/MapActivity(371): Couldn't get connection factory client 11-29 17:40:43.190: D/AndroidRuntime(371): Shutting down VM 11-29 17:40:43.190: W/dalvikvm(371): threadid=1: thread exiting with uncaught exception (group=0x4001d800) 11-29 17:40:43.280: E/AndroidRuntime(371): FATAL EXCEPTION: main 11-29 17:40:43.280: E/AndroidRuntime(371): java.lang.NullPointerException 11-29 17:40:43.280: E/AndroidRuntime(371): at com.google.android.maps.PixelConverter.toPixels(PixelConverter.java:71) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.google.android.maps.PixelConverter.toPixels(PixelConverter.java:61) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.GoogleMaps.MapsActivity$MyLocationOverlay.draw(MapsActivity.java:106) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.google.android.maps.OverlayBundle.draw(OverlayBundle.java:42) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.google.android.maps.MapView.onDraw(MapView.java:494) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.View.draw(View.java:6740) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.drawChild(ViewGroup.java:1640) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.drawChild(ViewGroup.java:1638) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.View.draw(View.java:6743) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.widget.FrameLayout.draw(FrameLayout.java:352) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.drawChild(ViewGroup.java:1640) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.drawChild(ViewGroup.java:1638) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewGroup.dispatchDraw(ViewGroup.java:1367) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.View.draw(View.java:6743) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.widget.FrameLayout.draw(FrameLayout.java:352) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.android.internal.policy.impl.PhoneWindow$DecorView.draw(PhoneWindow.java:1842) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewRoot.draw(ViewRoot.java:1407) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewRoot.performTraversals(ViewRoot.java:1163) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.view.ViewRoot.handleMessage(ViewRoot.java:1727) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.os.Handler.dispatchMessage(Handler.java:99) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.os.Looper.loop(Looper.java:123) 11-29 17:40:43.280: E/AndroidRuntime(371): at android.app.ActivityThread.main(ActivityThread.java:4627) 11-29 17:40:43.280: E/AndroidRuntime(371): at java.lang.reflect.Method.invokeNative(Native Method) 11-29 17:40:43.280: E/AndroidRuntime(371): at java.lang.reflect.Method.invoke(Method.java:521) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:868) 11-29 17:40:43.280: E/AndroidRuntime(371): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626) 11-29 17:40:43.280: E/AndroidRuntime(371): at dalvik.system.NativeStart.main(Native Method) 11-29 17:40:45.779: D/dalvikvm(371): GC_FOR_MALLOC freed 5970 objects / 506624 bytes in 1179ms 11-29 17:40:45.779: I/dalvikvm-heap(371): Grow heap (frag case) to 3.147MB for 17858-byte allocation 11-29 17:40:45.870: D/dalvikvm(371): GC_FOR_MALLOC freed 56 objects / 2304 bytes in 92ms 11-29 17:40:45.960: D/dalvikvm(371): GC_EXPLICIT freed 3459 objects / 196432 bytes in 74ms 11-29 17:40:48.310: D/dalvikvm(371): GC_EXPLICIT freed 116 objects / 41448 bytes in 68ms 11-29 17:40:49.540: I/Process(371): Sending signal. PID: 371 SIG: 9

    Read the article

  • while uploading image I get errors in logcat

    - by al0ne evenings
    I am uploading image and also sending some parameters with it. I am getting errors in my logcat while doing so. Here is my code public class Camera extends Activity { ImageView ivUserImage; Button bUpload; Intent i; int CameraResult = 0; Bitmap bmp; public String globalUID; UserFunctions userFunctions; String photoName; InputStream is; int serverResponseCode = 0; ProgressDialog dialog = null; @Override protected void onCreate(Bundle savedInstanceState) { // TODO Auto-generated method stub super.onCreate(savedInstanceState); setContentView(R.layout.camera); userFunctions = new UserFunctions(); globalUID = getIntent().getExtras().getString("globalUID"); Toast.makeText(getApplicationContext(), globalUID, Toast.LENGTH_LONG).show(); ivUserImage = (ImageView)findViewById(R.id.ivUserImage); bUpload = (Button)findViewById(R.id.bUpload); openCamera(); } private void openCamera() { i = new Intent(android.provider.MediaStore.ACTION_IMAGE_CAPTURE); startActivityForResult(i, CameraResult); } @Override protected void onActivityResult(int requestCode, int resultCode, Intent data) { super.onActivityResult(requestCode, resultCode, data); if(resultCode == RESULT_OK) { //set image taken from camera on ImageView Bundle extras = data.getExtras(); bmp = (Bitmap) extras.get("data"); ivUserImage.setImageBitmap(bmp); //Create new Cursor to obtain the file Path for the large image String[] largeFileProjection = { MediaStore.Images.ImageColumns._ID, MediaStore.Images.ImageColumns.DATA }; String largeFileSort = MediaStore.Images.ImageColumns._ID + " DESC"; //final String photoName = MediaStore.Images.ImageColumns.BUCKET_DISPLAY_NAME; Cursor myCursor = this.managedQuery(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, largeFileProjection, null, null, largeFileSort); try { myCursor.moveToFirst(); //This will actually give you the file path location of the image. final String largeImagePath = myCursor.getString(myCursor.getColumnIndexOrThrow(MediaStore.Images.ImageColumns.DATA)); //final String photoName = myCursor.getString(myCursor.getColumnIndexOrThrow(MediaStore.Images.ImageColumns.BUCKET_DISPLAY_NAME)); //Log.e("URI", largeImagePath); File f = new File("" + largeImagePath); photoName = f.getName(); bUpload.setOnClickListener(new View.OnClickListener() { public void onClick(View v) { dialog = ProgressDialog.show(Camera.this, "", "Uploading file...", true); new Thread(new Runnable() { public void run() { runOnUiThread(new Runnable() { public void run() { //tv.setText("uploading started....."); Toast.makeText(getBaseContext(), "File is uploading...", Toast.LENGTH_LONG).show(); } }); if(imageUpload(globalUID, largeImagePath)) { Toast.makeText(getApplicationContext(), "Image uploaded", Toast.LENGTH_LONG).show(); } else { Toast.makeText(getApplicationContext(), "Error uploading image", Toast.LENGTH_LONG).show(); } //Log.e("Response: ", response); //System.out.println("RES : " + response); } }).start(); } }); } finally { myCursor.close(); } //Uri uriLargeImage = Uri.withAppendedPath(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, String.valueOf(imageId)); //String imageUrl = getRealPathFromURI(uriLargeImage); } } public boolean imageUpload(String uid, String imagepath) { boolean success = false; //Bitmap bitmapOrg = BitmapFactory.decodeResource(getResources(), R.drawable.ic_launcher); Bitmap bitmapOrg = BitmapFactory.decodeFile(imagepath); ByteArrayOutputStream bao = new ByteArrayOutputStream(); bitmapOrg.compress(Bitmap.CompressFormat.JPEG, 90, bao); byte [] ba = bao.toByteArray(); String ba1=Base64.encodeToString(ba, 0); ArrayList<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(); nameValuePairs.add(new BasicNameValuePair("image",ba1)); nameValuePairs.add(new BasicNameValuePair("uid", uid)); try { HttpClient httpclient = new DefaultHttpClient(); HttpPost httppost = new HttpPost("http://www.example.info/androidfileupload/index.php"); httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs)); HttpResponse response = httpclient.execute(httppost); HttpEntity entity = response.getEntity(); if(response != null) { success = true; } is = entity.getContent(); } catch(Exception e) { Log.e("log_tag", "Error in http connection "+e.toString()); } dialog.dismiss(); return success; } Here is my logcat 06-23 11:54:48.555: E/AndroidRuntime(30601): FATAL EXCEPTION: Thread-11 06-23 11:54:48.555: E/AndroidRuntime(30601): java.lang.OutOfMemoryError 06-23 11:54:48.555: E/AndroidRuntime(30601): at java.lang.String.<init>(String.java:513) 06-23 11:54:48.555: E/AndroidRuntime(30601): at java.lang.AbstractStringBuilder.toString(AbstractStringBuilder.java:650) 06-23 11:54:48.555: E/AndroidRuntime(30601): at java.lang.StringBuilder.toString(StringBuilder.java:664) 06-23 11:54:48.555: E/AndroidRuntime(30601): at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:170) 06-23 11:54:48.555: E/AndroidRuntime(30601): at org.apache.http.client.entity.UrlEncodedFormEntity.<init>(UrlEncodedFormEntity.java:71) 06-23 11:54:48.555: E/AndroidRuntime(30601): at com.zafar.login.Camera.imageUpload(Camera.java:155) 06-23 11:54:48.555: E/AndroidRuntime(30601): at com.zafar.login.Camera$1$1.run(Camera.java:119) 06-23 11:54:48.555: E/AndroidRuntime(30601): at java.lang.Thread.run(Thread.java:1019) 06-23 11:54:53.960: E/WindowManager(30601): Activity com.zafar.login.DashboardActivity has leaked window com.android.internal.policy.impl.PhoneWindow$DecorView@405db540 that was originally added here 06-23 11:54:53.960: E/WindowManager(30601): android.view.WindowLeaked: Activity com.zafar.login.DashboardActivity has leaked window com.android.internal.policy.impl.PhoneWindow$DecorView@405db540 that was originally added here 06-23 11:54:53.960: E/WindowManager(30601): at android.view.ViewRoot.<init>(ViewRoot.java:266) 06-23 11:54:53.960: E/WindowManager(30601): at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:174) 06-23 11:54:53.960: E/WindowManager(30601): at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:117) 06-23 11:54:53.960: E/WindowManager(30601): at android.view.Window$LocalWindowManager.addView(Window.java:424) 06-23 11:54:53.960: E/WindowManager(30601): at android.app.Dialog.show(Dialog.java:241) 06-23 11:54:53.960: E/WindowManager(30601): at android.app.ProgressDialog.show(ProgressDialog.java:107) 06-23 11:54:53.960: E/WindowManager(30601): at android.app.ProgressDialog.show(ProgressDialog.java:90) 06-23 11:54:53.960: E/WindowManager(30601): at com.zafar.login.Camera$1.onClick(Camera.java:110) 06-23 11:54:53.960: E/WindowManager(30601): at android.view.View.performClick(View.java:2538) 06-23 11:54:53.960: E/WindowManager(30601): at android.view.View$PerformClick.run(View.java:9152) 06-23 11:54:53.960: E/WindowManager(30601): at android.os.Handler.handleCallback(Handler.java:587) 06-23 11:54:53.960: E/WindowManager(30601): at android.os.Handler.dispatchMessage(Handler.java:92) 06-23 11:54:53.960: E/WindowManager(30601): at android.os.Looper.loop(Looper.java:130) 06-23 11:54:53.960: E/WindowManager(30601): at android.app.ActivityThread.main(ActivityThread.java:3691) 06-23 11:54:53.960: E/WindowManager(30601): at java.lang.reflect.Method.invokeNative(Native Method) 06-23 11:54:53.960: E/WindowManager(30601): at java.lang.reflect.Method.invoke(Method.java:507) 06-23 11:54:53.960: E/WindowManager(30601): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:907) 06-23 11:54:53.960: E/WindowManager(30601): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:665) 06-23 11:54:53.960: E/WindowManager(30601): at dalvik.system.NativeStart.main(Native Method) 06-23 11:54:55.190: I/Process(30601): Sending signal. PID: 30601 SIG: 9

    Read the article

  • Little more help with writing a o buffer with libjpeg

    - by Richard Knop
    So I have managed to find another question discussing how to use the libjpeg to compress an image to jpeg. I have found this code which is supposed to work: Compressing IplImage to JPEG using libjpeg in OpenCV Here's the code (it compiles ok): /* This a custom destination manager for jpeglib that enables the use of memory to memory compression. See IJG documentation for details. */ typedef struct { struct jpeg_destination_mgr pub; /* base class */ JOCTET* buffer; /* buffer start address */ int bufsize; /* size of buffer */ size_t datasize; /* final size of compressed data */ int* outsize; /* user pointer to datasize */ int errcount; /* counts up write errors due to buffer overruns */ } memory_destination_mgr; typedef memory_destination_mgr* mem_dest_ptr; /* ------------------------------------------------------------- */ /* MEMORY DESTINATION INTERFACE METHODS */ /* ------------------------------------------------------------- */ /* This function is called by the library before any data gets written */ METHODDEF(void) init_destination (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->pub.next_output_byte = dest->buffer; /* set destination buffer */ dest->pub.free_in_buffer = dest->bufsize; /* input buffer size */ dest->datasize = 0; /* reset output size */ dest->errcount = 0; /* reset error count */ } /* This function is called by the library if the buffer fills up I just reset destination pointer and buffer size here. Note that this behavior, while preventing seg faults will lead to invalid output streams as data is over- written. */ METHODDEF(boolean) empty_output_buffer (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->pub.next_output_byte = dest->buffer; dest->pub.free_in_buffer = dest->bufsize; ++dest->errcount; /* need to increase error count */ return TRUE; } /* Usually the library wants to flush output here. I will calculate output buffer size here. Note that results become incorrect, once empty_output_buffer was called. This situation is notified by errcount. */ METHODDEF(void) term_destination (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->datasize = dest->bufsize - dest->pub.free_in_buffer; if (dest->outsize) *dest->outsize += (int)dest->datasize; } /* Override the default destination manager initialization provided by jpeglib. Since we want to use memory-to-memory compression, we need to use our own destination manager. */ GLOBAL(void) jpeg_memory_dest (j_compress_ptr cinfo, JOCTET* buffer, int bufsize, int* outsize) { mem_dest_ptr dest; /* first call for this instance - need to setup */ if (cinfo->dest == 0) { cinfo->dest = (struct jpeg_destination_mgr *) (*cinfo->mem->alloc_small) ((j_common_ptr) cinfo, JPOOL_PERMANENT, sizeof (memory_destination_mgr)); } dest = (mem_dest_ptr) cinfo->dest; dest->bufsize = bufsize; dest->buffer = buffer; dest->outsize = outsize; /* set method callbacks */ dest->pub.init_destination = init_destination; dest->pub.empty_output_buffer = empty_output_buffer; dest->pub.term_destination = term_destination; } /* ------------------------------------------------------------- */ /* MEMORY SOURCE INTERFACE METHODS */ /* ------------------------------------------------------------- */ /* Called before data is read */ METHODDEF(void) init_source (j_decompress_ptr dinfo) { /* nothing to do here, really. I mean. I'm not lazy or something, but... we're actually through here. */ } /* Called if the decoder wants some bytes that we cannot provide... */ METHODDEF(boolean) fill_input_buffer (j_decompress_ptr dinfo) { /* we can't do anything about this. This might happen if the provided buffer is either invalid with regards to its content or just a to small bufsize has been given. */ /* fail. */ return FALSE; } /* From IJG docs: "it's not clear that being smart is worth much trouble" So I save myself some trouble by ignoring this bit. */ METHODDEF(void) skip_input_data (j_decompress_ptr dinfo, INT32 num_bytes) { /* There might be more data to skip than available in buffer. This clearly is an error, so screw this mess. */ if ((size_t)num_bytes > dinfo->src->bytes_in_buffer) { dinfo->src->next_input_byte = 0; /* no buffer byte */ dinfo->src->bytes_in_buffer = 0; /* no input left */ } else { dinfo->src->next_input_byte += num_bytes; dinfo->src->bytes_in_buffer -= num_bytes; } } /* Finished with decompression */ METHODDEF(void) term_source (j_decompress_ptr dinfo) { /* Again. Absolute laziness. Nothing to do here. Boring. */ } GLOBAL(void) jpeg_memory_src (j_decompress_ptr dinfo, unsigned char* buffer, size_t size) { struct jpeg_source_mgr* src; /* first call for this instance - need to setup */ if (dinfo->src == 0) { dinfo->src = (struct jpeg_source_mgr *) (*dinfo->mem->alloc_small) ((j_common_ptr) dinfo, JPOOL_PERMANENT, sizeof (struct jpeg_source_mgr)); } src = dinfo->src; src->next_input_byte = buffer; src->bytes_in_buffer = size; src->init_source = init_source; src->fill_input_buffer = fill_input_buffer; src->skip_input_data = skip_input_data; src->term_source = term_source; /* IJG recommend to use their function - as I don't know **** about how to do better, I follow this recommendation */ src->resync_to_restart = jpeg_resync_to_restart; } All I need to do is replace the jpeg_stdio_dest in my program with this code: int numBytes = 0; //size of jpeg after compression char * storage = new char[150000]; //storage buffer JOCTET *jpgbuff = (JOCTET*)storage; //JOCTET pointer to buffer jpeg_memory_dest(&cinfo,jpgbuff,150000,&numBytes); So I need some help to incorporate the above four lines into this function which now works but writes to a file instead of a memory: int write_jpeg_file( char *filename ) { struct jpeg_compress_struct cinfo; struct jpeg_error_mgr jerr; /* this is a pointer to one row of image data */ JSAMPROW row_pointer[1]; FILE *outfile = fopen( filename, "wb" ); if ( !outfile ) { printf("Error opening output jpeg file %s\n!", filename ); return -1; } cinfo.err = jpeg_std_error( &jerr ); jpeg_create_compress(&cinfo); jpeg_stdio_dest(&cinfo, outfile); /* Setting the parameters of the output file here */ cinfo.image_width = width; cinfo.image_height = height; cinfo.input_components = bytes_per_pixel; cinfo.in_color_space = color_space; /* default compression parameters, we shouldn't be worried about these */ jpeg_set_defaults( &cinfo ); /* Now do the compression .. */ jpeg_start_compress( &cinfo, TRUE ); /* like reading a file, this time write one row at a time */ while( cinfo.next_scanline < cinfo.image_height ) { row_pointer[0] = &raw_image[ cinfo.next_scanline * cinfo.image_width * cinfo.input_components]; jpeg_write_scanlines( &cinfo, row_pointer, 1 ); } /* similar to read file, clean up after we're done compressing */ jpeg_finish_compress( &cinfo ); jpeg_destroy_compress( &cinfo ); fclose( outfile ); /* success code is 1! */ return 1; } Anybody could help me out a bit with it? I've tried meddling with it but I am not sure how to do it. I I just replace this line: jpeg_stdio_dest(&cinfo, outfile); It's not going to work. There is more stuff that needs to be changed a bit in that function and I am being a little lost from all those pointers and memory management.

    Read the article

  • An Introduction to ASP.NET Web API

    - by Rick Strahl
    Microsoft recently released ASP.NET MVC 4.0 and .NET 4.5 and along with it, the brand spanking new ASP.NET Web API. Web API is an exciting new addition to the ASP.NET stack that provides a new, well-designed HTTP framework for creating REST and AJAX APIs (API is Microsoft’s new jargon for a service, in case you’re wondering). Although Web API ships and installs with ASP.NET MVC 4, you can use Web API functionality in any ASP.NET project, including WebForms, WebPages and MVC or just a Web API by itself. And you can also self-host Web API in your own applications from Console, Desktop or Service applications. If you're interested in a high level overview on what ASP.NET Web API is and how it fits into the ASP.NET stack you can check out my previous post: Where does ASP.NET Web API fit? In the following article, I'll focus on a practical, by example introduction to ASP.NET Web API. All the code discussed in this article is available in GitHub: https://github.com/RickStrahl/AspNetWebApiArticle [republished from my Code Magazine Article and updated for RTM release of ASP.NET Web API] Getting Started To start I’ll create a new empty ASP.NET application to demonstrate that Web API can work with any kind of ASP.NET project. Although you can create a new project based on the ASP.NET MVC/Web API template to quickly get up and running, I’ll take you through the manual setup process, because one common use case is to add Web API functionality to an existing ASP.NET application. This process describes the steps needed to hook up Web API to any ASP.NET 4.0 application. Start by creating an ASP.NET Empty Project. Then create a new folder in the project called Controllers. Add a Web API Controller Class Once you have any kind of ASP.NET project open, you can add a Web API Controller class to it. Web API Controllers are very similar to MVC Controller classes, but they work in any kind of project. Add a new item to this folder by using the Add New Item option in Visual Studio and choose Web API Controller Class, as shown in Figure 1. Figure 1: This is how you create a new Controller Class in Visual Studio   Make sure that the name of the controller class includes Controller at the end of it, which is required in order for Web API routing to find it. Here, the name for the class is AlbumApiController. For this example, I’ll use a Music Album model to demonstrate basic behavior of Web API. The model consists of albums and related songs where an album has properties like Name, Artist and YearReleased and a list of songs with a SongName and SongLength as well as an AlbumId that links it to the album. You can find the code for the model (and the rest of these samples) on Github. To add the file manually, create a new folder called Model, and add a new class Album.cs and copy the code into it. There’s a static AlbumData class with a static CreateSampleAlbumData() method that creates a short list of albums on a static .Current that I’ll use for the examples. Before we look at what goes into the controller class though, let’s hook up routing so we can access this new controller. Hooking up Routing in Global.asax To start, I need to perform the one required configuration task in order for Web API to work: I need to configure routing to the controller. Like MVC, Web API uses routing to provide clean, extension-less URLs to controller methods. Using an extension method to ASP.NET’s static RouteTable class, you can use the MapHttpRoute() (in the System.Web.Http namespace) method to hook-up the routing during Application_Start in global.asax.cs shown in Listing 1.using System; using System.Web.Routing; using System.Web.Http; namespace AspNetWebApi { public class Global : System.Web.HttpApplication { protected void Application_Start(object sender, EventArgs e) { RouteTable.Routes.MapHttpRoute( name: "AlbumVerbs", routeTemplate: "albums/{title}", defaults: new { symbol = RouteParameter.Optional, controller="AlbumApi" } ); } } } This route configures Web API to direct URLs that start with an albums folder to the AlbumApiController class. Routing in ASP.NET is used to create extensionless URLs and allows you to map segments of the URL to specific Route Value parameters. A route parameter, with a name inside curly brackets like {name}, is mapped to parameters on the controller methods. Route parameters can be optional, and there are two special route parameters – controller and action – that determine the controller to call and the method to activate respectively. HTTP Verb Routing Routing in Web API can route requests by HTTP Verb in addition to standard {controller},{action} routing. For the first examples, I use HTTP Verb routing, as shown Listing 1. Notice that the route I’ve defined does not include an {action} route value or action value in the defaults. Rather, Web API can use the HTTP Verb in this route to determine the method to call the controller, and a GET request maps to any method that starts with Get. So methods called Get() or GetAlbums() are matched by a GET request and a POST request maps to a Post() or PostAlbum(). Web API matches a method by name and parameter signature to match a route, query string or POST values. In lieu of the method name, the [HttpGet,HttpPost,HttpPut,HttpDelete, etc] attributes can also be used to designate the accepted verbs explicitly if you don’t want to follow the verb naming conventions. Although HTTP Verb routing is a good practice for REST style resource APIs, it’s not required and you can still use more traditional routes with an explicit {action} route parameter. When {action} is supplied, the HTTP verb routing is ignored. I’ll talk more about alternate routes later. When you’re finished with initial creation of files, your project should look like Figure 2.   Figure 2: The initial project has the new API Controller Album model   Creating a small Album Model Now it’s time to create some controller methods to serve data. For these examples, I’ll use a very simple Album and Songs model to play with, as shown in Listing 2. public class Song { public string AlbumId { get; set; } [Required, StringLength(80)] public string SongName { get; set; } [StringLength(5)] public string SongLength { get; set; } } public class Album { public string Id { get; set; } [Required, StringLength(80)] public string AlbumName { get; set; } [StringLength(80)] public string Artist { get; set; } public int YearReleased { get; set; } public DateTime Entered { get; set; } [StringLength(150)] public string AlbumImageUrl { get; set; } [StringLength(200)] public string AmazonUrl { get; set; } public virtual List<Song> Songs { get; set; } public Album() { Songs = new List<Song>(); Entered = DateTime.Now; // Poor man's unique Id off GUID hash Id = Guid.NewGuid().GetHashCode().ToString("x"); } public void AddSong(string songName, string songLength = null) { this.Songs.Add(new Song() { AlbumId = this.Id, SongName = songName, SongLength = songLength }); } } Once the model has been created, I also added an AlbumData class that generates some static data in memory that is loaded onto a static .Current member. The signature of this class looks like this and that's what I'll access to retrieve the base data:public static class AlbumData { // sample data - static list public static List<Album> Current = CreateSampleAlbumData(); /// <summary> /// Create some sample data /// </summary> /// <returns></returns> public static List<Album> CreateSampleAlbumData() { … }} You can check out the full code for the data generation online. Creating an AlbumApiController Web API shares many concepts of ASP.NET MVC, and the implementation of your API logic is done by implementing a subclass of the System.Web.Http.ApiController class. Each public method in the implemented controller is a potential endpoint for the HTTP API, as long as a matching route can be found to invoke it. The class name you create should end in Controller, which is how Web API matches the controller route value to figure out which class to invoke. Inside the controller you can implement methods that take standard .NET input parameters and return .NET values as results. Web API’s binding tries to match POST data, route values, form values or query string values to your parameters. Because the controller is configured for HTTP Verb based routing (no {action} parameter in the route), any methods that start with Getxxxx() are called by an HTTP GET operation. You can have multiple methods that match each HTTP Verb as long as the parameter signatures are different and can be matched by Web API. In Listing 3, I create an AlbumApiController with two methods to retrieve a list of albums and a single album by its title .public class AlbumApiController : ApiController { public IEnumerable<Album> GetAlbums() { var albums = AlbumData.Current.OrderBy(alb => alb.Artist); return albums; } public Album GetAlbum(string title) { var album = AlbumData.Current .SingleOrDefault(alb => alb.AlbumName.Contains(title)); return album; }} To access the first two requests, you can use the following URLs in your browser: http://localhost/aspnetWebApi/albumshttp://localhost/aspnetWebApi/albums/Dirty%20Deeds Note that you’re not specifying the actions of GetAlbum or GetAlbums in these URLs. Instead Web API’s routing uses HTTP GET verb to route to these methods that start with Getxxx() with the first mapping to the parameterless GetAlbums() method and the latter to the GetAlbum(title) method that receives the title parameter mapped as optional in the route. Content Negotiation When you access any of the URLs above from a browser, you get either an XML or JSON result returned back. The album list result for Chrome 17 and Internet Explorer 9 is shown Figure 3. Figure 3: Web API responses can vary depending on the browser used, demonstrating Content Negotiation in action as these two browsers send different HTTP Accept headers.   Notice that the results are not the same: Chrome returns an XML response and IE9 returns a JSON response. Whoa, what’s going on here? Shouldn’t we see the same result in both browsers? Actually, no. Web API determines what type of content to return based on Accept headers. HTTP clients, like browsers, use Accept headers to specify what kind of content they’d like to see returned. Browsers generally ask for HTML first, followed by a few additional content types. Chrome (and most other major browsers) ask for: Accept: text/html, application/xhtml+xml,application/xml; q=0.9,*/*;q=0.8 IE9 asks for: Accept: text/html, application/xhtml+xml, */* Note that Chrome’s Accept header includes application/xml, which Web API finds in its list of supported media types and returns an XML response. IE9 does not include an Accept header type that works on Web API by default, and so it returns the default format, which is JSON. This is an important and very useful feature that was missing from any previous Microsoft REST tools: Web API automatically switches output formats based on HTTP Accept headers. Nowhere in the server code above do you have to explicitly specify the output format. Rather, Web API determines what format the client is requesting based on the Accept headers and automatically returns the result based on the available formatters. This means that a single method can handle both XML and JSON results.. Using this simple approach makes it very easy to create a single controller method that can return JSON, XML, ATOM or even OData feeds by providing the appropriate Accept header from the client. By default you don’t have to worry about the output format in your code. Note that you can still specify an explicit output format if you choose, either globally by overriding the installed formatters, or individually by returning a lower level HttpResponseMessage instance and setting the formatter explicitly. More on that in a minute. Along the same lines, any content sent to the server via POST/PUT is parsed by Web API based on the HTTP Content-type of the data sent. The same formats allowed for output are also allowed on input. Again, you don’t have to do anything in your code – Web API automatically performs the deserialization from the content. Accessing Web API JSON Data with jQuery A very common scenario for Web API endpoints is to retrieve data for AJAX calls from the Web browser. Because JSON is the default format for Web API, it’s easy to access data from the server using jQuery and its getJSON() method. This example receives the albums array from GetAlbums() and databinds it into the page using knockout.js.$.getJSON("albums/", function (albums) { // make knockout template visible $(".album").show(); // create view object and attach array var view = { albums: albums }; ko.applyBindings(view); }); Figure 4 shows this and the next example’s HTML output. You can check out the complete HTML and script code at http://goo.gl/Ix33C (.html) and http://goo.gl/tETlg (.js). Figu Figure 4: The Album Display sample uses JSON data loaded from Web API.   The result from the getJSON() call is a JavaScript object of the server result, which comes back as a JavaScript array. In the code, I use knockout.js to bind this array into the UI, which as you can see, requires very little code, instead using knockout’s data-bind attributes to bind server data to the UI. Of course, this is just one way to use the data – it’s entirely up to you to decide what to do with the data in your client code. Along the same lines, I can retrieve a single album to display when the user clicks on an album. The response returns the album information and a child array with all the songs. The code to do this is very similar to the last example where we pulled the albums array:$(".albumlink").live("click", function () { var id = $(this).data("id"); // title $.getJSON("albums/" + id, function (album) { ko.applyBindings(album, $("#divAlbumDialog")[0]); $("#divAlbumDialog").show(); }); }); Here the URL looks like this: /albums/Dirty%20Deeds, where the title is the ID captured from the clicked element’s data ID attribute. Explicitly Overriding Output Format When Web API automatically converts output using content negotiation, it does so by matching Accept header media types to the GlobalConfiguration.Configuration.Formatters and the SupportedMediaTypes of each individual formatter. You can add and remove formatters to globally affect what formats are available and it’s easy to create and plug in custom formatters.The example project includes a JSONP formatter that can be plugged in to provide JSONP support for requests that have a callback= querystring parameter. Adding, removing or replacing formatters is a global option you can use to manipulate content. It’s beyond the scope of this introduction to show how it works, but you can review the sample code or check out my blog entry on the subject (http://goo.gl/UAzaR). If automatic processing is not desirable in a particular Controller method, you can override the response output explicitly by returning an HttpResponseMessage instance. HttpResponseMessage is similar to ActionResult in ASP.NET MVC in that it’s a common way to return an abstract result message that contains content. HttpResponseMessage s parsed by the Web API framework using standard interfaces to retrieve the response data, status code, headers and so on[MS2] . Web API turns every response – including those Controller methods that return static results – into HttpResponseMessage instances. Explicitly returning an HttpResponseMessage instance gives you full control over the output and lets you mostly bypass WebAPI’s post-processing of the HTTP response on your behalf. HttpResponseMessage allows you to customize the response in great detail. Web API’s attention to detail in the HTTP spec really shows; many HTTP options are exposed as properties and enumerations with detailed IntelliSense comments. Even if you’re new to building REST-based interfaces, the API guides you in the right direction for returning valid responses and response codes. For example, assume that I always want to return JSON from the GetAlbums() controller method and ignore the default media type content negotiation. To do this, I can adjust the output format and headers as shown in Listing 4.public HttpResponseMessage GetAlbums() { var albums = AlbumData.Current.OrderBy(alb => alb.Artist); // Create a new HttpResponse with Json Formatter explicitly var resp = new HttpResponseMessage(HttpStatusCode.OK); resp.Content = new ObjectContent<IEnumerable<Album>>( albums, new JsonMediaTypeFormatter()); // Get Default Formatter based on Content Negotiation //var resp = Request.CreateResponse<IEnumerable<Album>>(HttpStatusCode.OK, albums); resp.Headers.ConnectionClose = true; resp.Headers.CacheControl = new CacheControlHeaderValue(); resp.Headers.CacheControl.Public = true; return resp; } This example returns the same IEnumerable<Album> value, but it wraps the response into an HttpResponseMessage so you can control the entire HTTP message result including the headers, formatter and status code. In Listing 4, I explicitly specify the formatter using the JsonMediaTypeFormatter to always force the content to JSON.  If you prefer to use the default content negotiation with HttpResponseMessage results, you can create the Response instance using the Request.CreateResponse method:var resp = Request.CreateResponse<IEnumerable<Album>>(HttpStatusCode.OK, albums); This provides you an HttpResponse object that's pre-configured with the default formatter based on Content Negotiation. Once you have an HttpResponse object you can easily control most HTTP aspects on this object. What's sweet here is that there are many more detailed properties on HttpResponse than the core ASP.NET Response object, with most options being explicitly configurable with enumerations that make it easy to pick the right headers and response codes from a list of valid codes. It makes HTTP features available much more discoverable even for non-hardcore REST/HTTP geeks. Non-Serialized Results The output returned doesn’t have to be a serialized value but can also be raw data, like strings, binary data or streams. You can use the HttpResponseMessage.Content object to set a number of common Content classes. Listing 5 shows how to return a binary image using the ByteArrayContent class from a Controller method. [HttpGet] public HttpResponseMessage AlbumArt(string title) { var album = AlbumData.Current.FirstOrDefault(abl => abl.AlbumName.StartsWith(title)); if (album == null) { var resp = Request.CreateResponse<ApiMessageError>( HttpStatusCode.NotFound, new ApiMessageError("Album not found")); return resp; } // kinda silly - we would normally serve this directly // but hey - it's a demo. var http = new WebClient(); var imageData = http.DownloadData(album.AlbumImageUrl); // create response and return var result = new HttpResponseMessage(HttpStatusCode.OK); result.Content = new ByteArrayContent(imageData); result.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg"); return result; } The image retrieval from Amazon is contrived, but it shows how to return binary data using ByteArrayContent. It also demonstrates that you can easily return multiple types of content from a single controller method, which is actually quite common. If an error occurs - such as a resource can’t be found or a validation error – you can return an error response to the client that’s very specific to the error. In GetAlbumArt(), if the album can’t be found, we want to return a 404 Not Found status (and realistically no error, as it’s an image). Note that if you are not using HTTP Verb-based routing or not accessing a method that starts with Get/Post etc., you have to specify one or more HTTP Verb attributes on the method explicitly. Here, I used the [HttpGet] attribute to serve the image. Another option to handle the error could be to return a fixed placeholder image if no album could be matched or the album doesn’t have an image. When returning an error code, you can also return a strongly typed response to the client. For example, you can set the 404 status code and also return a custom error object (ApiMessageError is a class I defined) like this:return Request.CreateResponse<ApiMessageError>( HttpStatusCode.NotFound, new ApiMessageError("Album not found") );   If the album can be found, the image will be returned. The image is downloaded into a byte[] array, and then assigned to the result’s Content property. I created a new ByteArrayContent instance and assigned the image’s bytes and the content type so that it displays properly in the browser. There are other content classes available: StringContent, StreamContent, ByteArrayContent, MultipartContent, and ObjectContent are at your disposal to return just about any kind of content. You can create your own Content classes if you frequently return custom types and handle the default formatter assignments that should be used to send the data out . Although HttpResponseMessage results require more code than returning a plain .NET value from a method, it allows much more control over the actual HTTP processing than automatic processing. It also makes it much easier to test your controller methods as you get a response object that you can check for specific status codes and output messages rather than just a result value. Routing Again Ok, let’s get back to the image example. Using the original routing we have setup using HTTP Verb routing there's no good way to serve the image. In order to return my album art image I’d like to use a URL like this: http://localhost/aspnetWebApi/albums/Dirty%20Deeds/image In order to create a URL like this, I have to create a new Controller because my earlier routes pointed to the AlbumApiController using HTTP Verb routing. HTTP Verb based routing is great for representing a single set of resources such as albums. You can map operations like add, delete, update and read easily using HTTP Verbs. But you cannot mix action based routing into a an HTTP Verb routing controller - you can only map HTTP Verbs and each method has to be unique based on parameter signature. You can't have multiple GET operations to methods with the same signature. So GetImage(string id) and GetAlbum(string title) are in conflict in an HTTP GET routing scenario. In fact, I was unable to make the above Image URL work with any combination of HTTP Verb plus Custom routing using the single Albums controller. There are number of ways around this, but all involve additional controllers.  Personally, I think it’s easier to use explicit Action routing and then add custom routes if you need to simplify your URLs further. So in order to accommodate some of the other examples, I created another controller – AlbumRpcApiController – to handle all requests that are explicitly routed via actions (/albums/rpc/AlbumArt) or are custom routed with explicit routes defined in the HttpConfiguration. I added the AlbumArt() method to this new AlbumRpcApiController class. For the image URL to work with the new AlbumRpcApiController, you need a custom route placed before the default route from Listing 1.RouteTable.Routes.MapHttpRoute( name: "AlbumRpcApiAction", routeTemplate: "albums/rpc/{action}/{title}", defaults: new { title = RouteParameter.Optional, controller = "AlbumRpcApi", action = "GetAblums" } ); Now I can use either of the following URLs to access the image: Custom route: (/albums/rpc/{title}/image)http://localhost/aspnetWebApi/albums/PowerAge/image Action route: (/albums/rpc/action/{title})http://localhost/aspnetWebAPI/albums/rpc/albumart/PowerAge Sending Data to the Server To send data to the server and add a new album, you can use an HTTP POST operation. Since I’m using HTTP Verb-based routing in the original AlbumApiController, I can implement a method called PostAlbum()to accept a new album from the client. Listing 6 shows the Web API code to add a new album.public HttpResponseMessage PostAlbum(Album album) { if (!this.ModelState.IsValid) { // my custom error class var error = new ApiMessageError() { message = "Model is invalid" }; // add errors into our client error model for client foreach (var prop in ModelState.Values) { var modelError = prop.Errors.FirstOrDefault(); if (!string.IsNullOrEmpty(modelError.ErrorMessage)) error.errors.Add(modelError.ErrorMessage); else error.errors.Add(modelError.Exception.Message); } return Request.CreateResponse<ApiMessageError>(HttpStatusCode.Conflict, error); } // update song id which isn't provided foreach (var song in album.Songs) song.AlbumId = album.Id; // see if album exists already var matchedAlbum = AlbumData.Current .SingleOrDefault(alb => alb.Id == album.Id || alb.AlbumName == album.AlbumName); if (matchedAlbum == null) AlbumData.Current.Add(album); else matchedAlbum = album; // return a string to show that the value got here var resp = Request.CreateResponse(HttpStatusCode.OK, string.Empty); resp.Content = new StringContent(album.AlbumName + " " + album.Entered.ToString(), Encoding.UTF8, "text/plain"); return resp; } The PostAlbum() method receives an album parameter, which is automatically deserialized from the POST buffer the client sent. The data passed from the client can be either XML or JSON. Web API automatically figures out what format it needs to deserialize based on the content type and binds the content to the album object. Web API uses model binding to bind the request content to the parameter(s) of controller methods. Like MVC you can check the model by looking at ModelState.IsValid. If it’s not valid, you can run through the ModelState.Values collection and check each binding for errors. Here I collect the error messages into a string array that gets passed back to the client via the result ApiErrorMessage object. When a binding error occurs, you’ll want to return an HTTP error response and it’s best to do that with an HttpResponseMessage result. In Listing 6, I used a custom error class that holds a message and an array of detailed error messages for each binding error. I used this object as the content to return to the client along with my Conflict HTTP Status Code response. If binding succeeds, the example returns a string with the name and date entered to demonstrate that you captured the data. Normally, a method like this should return a Boolean or no response at all (HttpStatusCode.NoConent). The sample uses a simple static list to hold albums, so once you’ve added the album using the Post operation, you can hit the /albums/ URL to see that the new album was added. The client jQuery code to call the POST operation from the client with jQuery is shown in Listing 7. var id = new Date().getTime().toString(); var album = { "Id": id, "AlbumName": "Power Age", "Artist": "AC/DC", "YearReleased": 1977, "Entered": "2002-03-11T18:24:43.5580794-10:00", "AlbumImageUrl": http://ecx.images-amazon.com/images/…, "AmazonUrl": http://www.amazon.com/…, "Songs": [ { "SongName": "Rock 'n Roll Damnation", "SongLength": 3.12}, { "SongName": "Downpayment Blues", "SongLength": 4.22 }, { "SongName": "Riff Raff", "SongLength": 2.42 } ] } $.ajax( { url: "albums/", type: "POST", contentType: "application/json", data: JSON.stringify(album), processData: false, beforeSend: function (xhr) { // not required since JSON is default output xhr.setRequestHeader("Accept", "application/json"); }, success: function (result) { // reload list of albums page.loadAlbums(); }, error: function (xhr, status, p3, p4) { var err = "Error"; if (xhr.responseText && xhr.responseText[0] == "{") err = JSON.parse(xhr.responseText).message; alert(err); } }); The code in Listing 7 creates an album object in JavaScript to match the structure of the .NET Album class. This object is passed to the $.ajax() function to send to the server as POST. The data is turned into JSON and the content type set to application/json so that the server knows what to convert when deserializing in the Album instance. The jQuery code hooks up success and failure events. Success returns the result data, which is a string that’s echoed back with an alert box. If an error occurs, jQuery returns the XHR instance and status code. You can check the XHR to see if a JSON object is embedded and if it is, you can extract it by de-serializing it and accessing the .message property. REST standards suggest that updates to existing resources should use PUT operations. REST standards aside, I’m not a big fan of separating out inserts and updates so I tend to have a single method that handles both. But if you want to follow REST suggestions, you can create a PUT method that handles updates by forwarding the PUT operation to the POST method:public HttpResponseMessage PutAlbum(Album album) { return PostAlbum(album); } To make the corresponding $.ajax() call, all you have to change from Listing 7 is the type: from POST to PUT. Model Binding with UrlEncoded POST Variables In the example in Listing 7 I used JSON objects to post a serialized object to a server method that accepted an strongly typed object with the same structure, which is a common way to send data to the server. However, Web API supports a number of different ways that data can be received by server methods. For example, another common way is to use plain UrlEncoded POST  values to send to the server. Web API supports Model Binding that works similar (but not the same) as MVC's model binding where POST variables are mapped to properties of object parameters of the target method. This is actually quite common for AJAX calls that want to avoid serialization and the potential requirement of a JSON parser on older browsers. For example, using jQUery you might use the $.post() method to send a new album to the server (albeit one without songs) using code like the following:$.post("albums/",{AlbumName: "Dirty Deeds", YearReleased: 1976 … },albumPostCallback); Although the code looks very similar to the client code we used before passing JSON, here the data passed is URL encoded values (AlbumName=Dirty+Deeds&YearReleased=1976 etc.). Web API then takes this POST data and maps each of the POST values to the properties of the Album object in the method's parameter. Although the client code is different the server can both handle the JSON object, or the UrlEncoded POST values. Dynamic Access to POST Data There are also a few options available to dynamically access POST data, if you know what type of data you're dealing with. If you have POST UrlEncoded values, you can dynamically using a FormsDataCollection:[HttpPost] public string PostAlbum(FormDataCollection form) { return string.Format("{0} - released {1}", form.Get("AlbumName"),form.Get("RearReleased")); } The FormDataCollection is a very simple object, that essentially provides the same functionality as Request.Form[] in ASP.NET. Request.Form[] still works if you're running hosted in an ASP.NET application. However as a general rule, while ASP.NET's functionality is always available when running Web API hosted inside of an  ASP.NET application, using the built in classes specific to Web API makes it possible to run Web API applications in a self hosted environment outside of ASP.NET. If your client is sending JSON to your server, and you don't want to map the JSON to a strongly typed object because you only want to retrieve a few simple values, you can also accept a JObject parameter in your API methods:[HttpPost] public string PostAlbum(JObject jsonData) { dynamic json = jsonData; JObject jalbum = json.Album; JObject juser = json.User; string token = json.UserToken; var album = jalbum.ToObject<Album>(); var user = juser.ToObject<User>(); return String.Format("{0} {1} {2}", album.AlbumName, user.Name, token); } There quite a few options available to you to receive data with Web API, which gives you more choices for the right tool for the job. Unfortunately one shortcoming of Web API is that POST data is always mapped to a single parameter. This means you can't pass multiple POST parameters to methods that receive POST data. It's possible to accept multiple parameters, but only one can map to the POST content - the others have to come from the query string or route values. I have a couple of Blog POSTs that explain what works and what doesn't here: Passing multiple POST parameters to Web API Controller Methods Mapping UrlEncoded POST Values in ASP.NET Web API   Handling Delete Operations Finally, to round out the server API code of the album example we've been discussin, here’s the DELETE verb controller method that allows removal of an album by its title:public HttpResponseMessage DeleteAlbum(string title) { var matchedAlbum = AlbumData.Current.Where(alb => alb.AlbumName == title) .SingleOrDefault(); if (matchedAlbum == null) return new HttpResponseMessage(HttpStatusCode.NotFound); AlbumData.Current.Remove(matchedAlbum); return new HttpResponseMessage(HttpStatusCode.NoContent); } To call this action method using jQuery, you can use:$(".removeimage").live("click", function () { var $el = $(this).parent(".album"); var txt = $el.find("a").text(); $.ajax({ url: "albums/" + encodeURIComponent(txt), type: "Delete", success: function (result) { $el.fadeOut().remove(); }, error: jqError }); }   Note the use of the DELETE verb in the $.ajax() call, which routes to DeleteAlbum on the server. DELETE is a non-content operation, so you supply a resource ID (the title) via route value or the querystring. Routing Conflicts In all requests with the exception of the AlbumArt image example shown so far, I used HTTP Verb routing that I set up in Listing 1. HTTP Verb Routing is a recommendation that is in line with typical REST access to HTTP resources. However, it takes quite a bit of effort to create REST-compliant API implementations based only on HTTP Verb routing only. You saw one example that didn’t really fit – the return of an image where I created a custom route albums/{title}/image that required creation of a second controller and a custom route to work. HTTP Verb routing to a controller does not mix with custom or action routing to the same controller because of the limited mapping of HTTP verbs imposed by HTTP Verb routing. To understand some of the problems with verb routing, let’s look at another example. Let’s say you create a GetSortableAlbums() method like this and add it to the original AlbumApiController accessed via HTTP Verb routing:[HttpGet] public IQueryable<Album> SortableAlbums() { var albums = AlbumData.Current; // generally should be done only on actual queryable results (EF etc.) // Done here because we're running with a static list but otherwise might be slow return albums.AsQueryable(); } If you compile this code and try to now access the /albums/ link, you get an error: Multiple Actions were found that match the request. HTTP Verb routing only allows access to one GET operation per parameter/route value match. If more than one method exists with the same parameter signature, it doesn’t work. As I mentioned earlier for the image display, the only solution to get this method to work is to throw it into another controller. Because I already set up the AlbumRpcApiController I can add the method there. First, I should rename the method to SortableAlbums() so I’m not using a Get prefix for the method. This also makes the action parameter look cleaner in the URL - it looks less like a method and more like a noun. I can then create a new route that handles direct-action mapping:RouteTable.Routes.MapHttpRoute( name: "AlbumRpcApiAction", routeTemplate: "albums/rpc/{action}/{title}", defaults: new { title = RouteParameter.Optional, controller = "AlbumRpcApi", action = "GetAblums" } ); As I am explicitly adding a route segment – rpc – into the route template, I can now reference explicit methods in the Web API controller using URLs like this: http://localhost/AspNetWebApi/rpc/SortableAlbums Error Handling I’ve already done some minimal error handling in the examples. For example in Listing 6, I detected some known-error scenarios like model validation failing or a resource not being found and returning an appropriate HttpResponseMessage result. But what happens if your code just blows up or causes an exception? If you have a controller method, like this:[HttpGet] public void ThrowException() { throw new UnauthorizedAccessException("Unauthorized Access Sucka"); } You can call it with this: http://localhost/AspNetWebApi/albums/rpc/ThrowException The default exception handling displays a 500-status response with the serialized exception on the local computer only. When you connect from a remote computer, Web API throws back a 500  HTTP Error with no data returned (IIS then adds its HTML error page). The behavior is configurable in the GlobalConfiguration:GlobalConfiguration .Configuration .IncludeErrorDetailPolicy = IncludeErrorDetailPolicy.Never; If you want more control over your error responses sent from code, you can throw explicit error responses yourself using HttpResponseException. When you throw an HttpResponseException the response parameter is used to generate the output for the Controller action. [HttpGet] public void ThrowError() { var resp = Request.CreateResponse<ApiMessageError>( HttpStatusCode.BadRequest, new ApiMessageError("Your code stinks!")); throw new HttpResponseException(resp); } Throwing an HttpResponseException stops the processing of the controller method and immediately returns the response you passed to the exception. Unlike other Exceptions fired inside of WebAPI, HttpResponseException bypasses the Exception Filters installed and instead just outputs the response you provide. In this case, the serialized ApiMessageError result string is returned in the default serialization format – XML or JSON. You can pass any content to HttpResponseMessage, which includes creating your own exception objects and consistently returning error messages to the client. Here’s a small helper method on the controller that you might use to send exception info back to the client consistently:private void ThrowSafeException(string message, HttpStatusCode statusCode = HttpStatusCode.BadRequest) { var errResponse = Request.CreateResponse<ApiMessageError>(statusCode, new ApiMessageError() { message = message }); throw new HttpResponseException(errResponse); } You can then use it to output any captured errors from code:[HttpGet] public void ThrowErrorSafe() { try { List<string> list = null; list.Add("Rick"); } catch (Exception ex) { ThrowSafeException(ex.Message); } }   Exception Filters Another more global solution is to create an Exception Filter. Filters in Web API provide the ability to pre- and post-process controller method operations. An exception filter looks at all exceptions fired and then optionally creates an HttpResponseMessage result. Listing 8 shows an example of a basic Exception filter implementation.public class UnhandledExceptionFilter : ExceptionFilterAttribute { public override void OnException(HttpActionExecutedContext context) { HttpStatusCode status = HttpStatusCode.InternalServerError; var exType = context.Exception.GetType(); if (exType == typeof(UnauthorizedAccessException)) status = HttpStatusCode.Unauthorized; else if (exType == typeof(ArgumentException)) status = HttpStatusCode.NotFound; var apiError = new ApiMessageError() { message = context.Exception.Message }; // create a new response and attach our ApiError object // which now gets returned on ANY exception result var errorResponse = context.Request.CreateResponse<ApiMessageError>(status, apiError); context.Response = errorResponse; base.OnException(context); } } Exception Filter Attributes can be assigned to an ApiController class like this:[UnhandledExceptionFilter] public class AlbumRpcApiController : ApiController or you can globally assign it to all controllers by adding it to the HTTP Configuration's Filters collection:GlobalConfiguration.Configuration.Filters.Add(new UnhandledExceptionFilter()); The latter is a great way to get global error trapping so that all errors (short of hard IIS errors and explicit HttpResponseException errors) return a valid error response that includes error information in the form of a known-error object. Using a filter like this allows you to throw an exception as you normally would and have your filter create a response in the appropriate output format that the client expects. For example, an AJAX application can on failure expect to see a JSON error result that corresponds to the real error that occurred rather than a 500 error along with HTML error page that IIS throws up. You can even create some custom exceptions so you can differentiate your own exceptions from unhandled system exceptions - you often don't want to display error information from 'unknown' exceptions as they may contain sensitive system information or info that's not generally useful to users of your application/site. This is just one example of how ASP.NET Web API is configurable and extensible. Exception filters are just one example of how you can plug-in into the Web API request flow to modify output. Many more hooks exist and I’ll take a closer look at extensibility in Part 2 of this article in the future. Summary Web API is a big improvement over previous Microsoft REST and AJAX toolkits. The key features to its usefulness are its ease of use with simple controller based logic, familiar MVC-style routing, low configuration impact, extensibility at all levels and tight attention to exposing and making HTTP semantics easily discoverable and easy to use. Although none of the concepts used in Web API are new or radical, Web API combines the best of previous platforms into a single framework that’s highly functional, easy to work with, and extensible to boot. I think that Microsoft has hit a home run with Web API. Related Resources Where does ASP.NET Web API fit? Sample Source Code on GitHub Passing multiple POST parameters to Web API Controller Methods Mapping UrlEncoded POST Values in ASP.NET Web API Creating a JSONP Formatter for ASP.NET Web API Removing the XML Formatter from ASP.NET Web API Applications© Rick Strahl, West Wind Technologies, 2005-2012Posted in Web Api   Tweet !function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs"); (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })();

    Read the article

  • Logging Into a site that uses Live.com authentication with C#

    - by Josh
    I've been trying to automate a log in to a website I frequent, www.bungie.net. The site is associated with Microsoft and Xbox Live, and as such makes uses of the Windows Live ID API when people log in to their site. I am relatively new to creating web spiders/robots, and I worry that I'm misunderstanding some of the most basic concepts. I've simulated logins to other sites such as Facebook and Gmail, but live.com has given me nothing but trouble. Anyways, I've been using Wireshark and the Firefox addon Tamper Data to try and figure out what I need to post, and what cookies I need to include with my requests. As far as I know these are the steps one must follow to log in to this site. 1. Visit https: //login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268167141&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917 2. Recieve the cookies MSPRequ and MSPOK. 3. Post the values from the form ID "PPSX", the values from the form ID "PPFT", your username, your password all to a changing URL similar to: https: //login.live.com/ppsecure/post.srf?wa=wsignin1.0&rpsnv=11&ct= (there are a few numbers that change at the end of that URL) 4. Live.com returns the user a page with more hidden forms to post. The client then posts the values from the form "ANON", the value from the form "ANONExp" and the values from the form "t" to the URL: http ://www.bung ie.net/Default.aspx?wa=wsignin1.0 5. After posting that data, the user is returned a variety of cookies the most important of which is "BNGAuth" which is the log in cookie for the site. Where I am having trouble is on fifth step, but that doesn't neccesarily mean I've done all the other steps correctly. I post the data from "ANON", "ANONExp" and "t" but instead of being returned a BNGAuth cookie, I'm returned a cookie named "RSPMaybe" and redirected to the home page. When I review the Wireshark log, I noticed something that instantly stood out to me as different between the log when I logged in with Firefox and when my program ran. It could be nothing but I'll include the picture here for you to review. I'm being returned an HTTP packet from the site before I post the data in the fourth step. I'm not sure how this is happening, but it must be a side effect from something I'm doing wrong in the HTTPS steps. ![alt text][1] http://img391.imageshack.us/img391/6049/31394881.gif using System; using System.Collections.Generic; using System.Collections.Specialized; using System.Text; using System.Net; using System.IO; using System.IO.Compression; using System.Security.Cryptography; using System.Security.Cryptography.X509Certificates; using System.Web; namespace SpiderFromScratch { class Program { static void Main(string[] args) { CookieContainer cookies = new CookieContainer(); Uri url = new Uri("https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268167141&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917"); HttpWebRequest http = (HttpWebRequest)HttpWebRequest.Create(url); http.Timeout = 30000; http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "300"); http.Referer = "http://www.bungie.net/"; http.ContentType = "application/x-www-form-urlencoded"; http.CookieContainer = new CookieContainer(); http.Method = WebRequestMethods.Http.Get; HttpWebResponse response = (HttpWebResponse)http.GetResponse(); StreamReader readStream = new StreamReader(response.GetResponseStream()); string HTML = readStream.ReadToEnd(); readStream.Close(); //gets the cookies (they are set in the eighth header) string[] strCookies = response.Headers.GetValues(8); response.Close(); string name, value; Cookie manualCookie; for (int i = 0; i < strCookies.Length; i++) { name = strCookies[i].Substring(0, strCookies[i].IndexOf("=")); value = strCookies[i].Substring(strCookies[i].IndexOf("=") + 1, strCookies[i].IndexOf(";") - strCookies[i].IndexOf("=") - 1); manualCookie = new Cookie(name, "\"" + value + "\""); Uri manualURL = new Uri("http://login.live.com"); http.CookieContainer.Add(manualURL, manualCookie); } //stores the cookies to be used later cookies = http.CookieContainer; //Get the PPSX value string PPSX = HTML.Remove(0, HTML.IndexOf("PPSX")); PPSX = PPSX.Remove(0, PPSX.IndexOf("value") + 7); PPSX = PPSX.Substring(0, PPSX.IndexOf("\"")); //Get this random PPFT value string PPFT = HTML.Remove(0, HTML.IndexOf("PPFT")); PPFT = PPFT.Remove(0, PPFT.IndexOf("value") + 7); PPFT = PPFT.Substring(0, PPFT.IndexOf("\"")); //Get the random URL you POST to string POSTURL = HTML.Remove(0, HTML.IndexOf("https://login.live.com/ppsecure/post.srf?wa=wsignin1.0&rpsnv=11&ct=")); POSTURL = POSTURL.Substring(0, POSTURL.IndexOf("\"")); //POST with cookies http = (HttpWebRequest)HttpWebRequest.Create(POSTURL); http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "300"); http.CookieContainer = cookies; http.Referer = "https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268158321&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917"; http.ContentType = "application/x-www-form-urlencoded"; http.Method = WebRequestMethods.Http.Post; Stream ostream = http.GetRequestStream(); //used to convert strings into bytes System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding(); //Post information byte[] buffer = encoding.GetBytes("PPSX=" + PPSX +"&PwdPad=IfYouAreReadingThisYouHaveTooMuc&login=YOUREMAILGOESHERE&passwd=YOURWORDGOESHERE" + "&LoginOptions=2&PPFT=" + PPFT); ostream.Write(buffer, 0, buffer.Length); ostream.Close(); HttpWebResponse response2 = (HttpWebResponse)http.GetResponse(); readStream = new StreamReader(response2.GetResponseStream()); HTML = readStream.ReadToEnd(); response2.Close(); ostream.Dispose(); foreach (Cookie cookie in response2.Cookies) { Console.WriteLine(cookie.Name + ": "); Console.WriteLine(cookie.Value); Console.WriteLine(cookie.Expires); Console.WriteLine(); } //SET POSTURL value string POSTANON = "http://www.bungie.net/Default.aspx?wa=wsignin1.0"; //Get the ANON value string ANON = HTML.Remove(0, HTML.IndexOf("ANON")); ANON = ANON.Remove(0, ANON.IndexOf("value") + 7); ANON = ANON.Substring(0, ANON.IndexOf("\"")); ANON = HttpUtility.UrlEncode(ANON); //Get the ANONExp value string ANONExp = HTML.Remove(0, HTML.IndexOf("ANONExp")); ANONExp = ANONExp.Remove(0, ANONExp.IndexOf("value") + 7); ANONExp = ANONExp.Substring(0, ANONExp.IndexOf("\"")); ANONExp = HttpUtility.UrlEncode(ANONExp); //Get the t value string t = HTML.Remove(0, HTML.IndexOf("id=\"t\"")); t = t.Remove(0, t.IndexOf("value") + 7); t = t.Substring(0, t.IndexOf("\"")); t = HttpUtility.UrlEncode(t); //POST the Info and Accept the Bungie Cookies http = (HttpWebRequest)HttpWebRequest.Create(POSTANON); http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Encoding", "gzip,deflate"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "115"); http.CookieContainer = new CookieContainer(); http.ContentType = "application/x-www-form-urlencoded"; http.Method = WebRequestMethods.Http.Post; http.Expect = null; ostream = http.GetRequestStream(); int test = ANON.Length; int test1 = ANONExp.Length; int test2 = t.Length; buffer = encoding.GetBytes("ANON=" + ANON +"&ANONExp=" + ANONExp + "&t=" + t); ostream.Write(buffer, 0, buffer.Length); ostream.Close(); //Here lies the problem, I am not returned the correct cookies. HttpWebResponse response3 = (HttpWebResponse)http.GetResponse(); GZipStream gzip = new GZipStream(response3.GetResponseStream(), CompressionMode.Decompress); readStream = new StreamReader(gzip); HTML = readStream.ReadToEnd(); //gets both cookies string[] strCookies2 = response3.Headers.GetValues(11); response3.Close(); } } } This has given me problems and I've put many hours into learning about HTTP protocols so any help would be appreciated. If there is an article detailing a similar log in to live.com feel free to point the way. I've been looking far and wide for any articles with working solutions. If I could be clearer, feel free to ask as this is my first time using Stack Overflow. Cheers, --Josh

    Read the article

  • What is a Delphi version of the C++ header for the DVP7010B video card DLL?

    - by grzegorz1
    I need help with converting c++ header file to delphi. I spent several days on this problem without success. Below is the original header file and my Delphi translation. C++ header #if _MSC_VER > 1000 #pragma once #endif // _MSC_VER > 1000 #ifdef DVP7010BDLL_EXPORTS #define DVP7010BDLL_API __declspec(dllexport) #else #define DVP7010BDLL_API __declspec(dllimport) #endif #define MAXBOARDS 4 #define MAXDEVS 4 #define ID_NEW_FRAME 37810 #define ID_MUX0_NEW_FRAME 37800 #define ID_MUX1_NEW_FRAME 37801 #define ID_MUX2_NEW_FRAME 37802 #define ID_MUX3_NEW_FRAME 37803 typedef enum { SUCCEEDED = 1, FAILED = 0, SDKINITFAILED = -1, PARAMERROR = -2, NODEVICES = -3, NOSAMPLE = -4, DEVICENUMERROR = -5, INPUTERROR = -6, // VERIFYHWERROR = -7 } Res; typedef enum tagAnalogVideoFormat { Video_None = 0x00000000, Video_NTSC_M = 0x00000001, Video_NTSC_M_J = 0x00000002, Video_PAL_B = 0x00000010, Video_PAL_M = 0x00000200, Video_PAL_N = 0x00000400, Video_SECAM_B = 0x00001000 } AnalogVideoFormat; typedef enum { SIZEFULLPAL=0, SIZED1, SIZEVGA, SIZEQVGA, SIZESUBQVGA } VideoSize; typedef enum { STOPPED = 1, RUNNING = 2, UNINITIALIZED = -1, UNKNOWNSTATE = -2 } CapState; class IDVP7010BDLL { public: int AdvDVP_CreateSDKInstence(void **pp); virtual int AdvDVP_InitSDK() PURE; virtual int AdvDVP_CloseSDK() PURE; virtual int AdvDVP_GetNoOfDevices(int *pNoOfDevs) PURE; virtual int AdvDVP_Start(int nDevNum, int SwitchingChans, HWND Main, HWND hwndPreview) PURE; virtual int AdvDVP_Stop(int nDevNum) PURE; virtual int AdvDVP_GetCapState(int nDevNum) PURE; virtual int AdvDVP_IsVideoPresent(int nDevNum, BOOL* VPresent) PURE; virtual int AdvDVP_GetCurFrameBuffer(int nDevNum, int VMux, long* bufSize, BYTE* buf) PURE; virtual int AdvDVP_SetNewFrameCallback(int nDevNum, int callback) PURE; virtual int AdvDVP_GetVideoFormat(int nDevNum, AnalogVideoFormat* vFormat) PURE; virtual int AdvDVP_SetVideoFormat(int nDevNum, AnalogVideoFormat vFormat) PURE; virtual int AdvDVP_GetFrameRate(int nDevNum, int *nFrameRate) PURE; virtual int AdvDVP_SetFrameRate(int nDevNum, int SwitchingChans, int nFrameRate) PURE; virtual int AdvDVP_GetResolution(int nDevNum, VideoSize *Size) PURE; virtual int AdvDVP_SetResolution(int nDevNum, VideoSize Size) PURE; virtual int AdvDVP_GetVideoInput(int nDevNum, int* input) PURE; virtual int AdvDVP_SetVideoInput(int nDevNum, int input) PURE; virtual int AdvDVP_GetBrightness(int nDevNum, int input, long *pnValue) PURE; virtual int AdvDVP_SetBrightness(int nDevNum, int input, long nValue) PURE; virtual int AdvDVP_GetContrast(int nDevNum, int input, long *pnValue) PURE; virtual int AdvDVP_SetContrast(int nDevNum, int input, long nValue) PURE; virtual int AdvDVP_GetHue(int nDevNum, int input, long *pnValue) PURE; virtual int AdvDVP_SetHue(int nDevNum, int input, long nValue) PURE; virtual int AdvDVP_GetSaturation(int nDevNum, int input, long *pnValue) PURE; virtual int AdvDVP_SetSaturation(int nDevNum, int input, long nValue) PURE; virtual int AdvDVP_GPIOGetData(int nDevNum, int DINum, BOOL* value) PURE; virtual int AdvDVP_GPIOSetData(int nDevNum, int DONum, BOOL value) PURE; }; Delphi unit IDVP7010BDLL_h; interface uses Windows, Messages, SysUtils, Classes; //{$if _MSC_VER > 1000} //pragma once //{$endif} // _MSC_VER > 1000 {$ifdef DVP7010BDLL_EXPORTS} //const DVP7010BDLL_API = __declspec(dllexport); {$else} //const DVP7010BDLL_API = __declspec(dllimport); {$endif} const MAXDEVS = 4; MAXMUXS = 4; ID_NEW_FRAME = 37810; ID_MUX0_NEW_FRAME = 37800; ID_MUX1_NEW_FRAME = 37801; ID_MUX2_NEW_FRAME = 37802; ID_MUX3_NEW_FRAME = 37803; // TRec SUCCEEDED = 1; FAILED = 0; SDKINITFAILED = -1; PARAMERROR = -2; NODEVICES = -3; NOSAMPLE = -4; DEVICENUMERROR = -5; INPUTERROR = -6; // TRec // TAnalogVideoFormat Video_None = $00000000; Video_NTSC_M = $00000001; Video_NTSC_M_J = $00000002; Video_PAL_B = $00000010; Video_PAL_M = $00000200; Video_PAL_N = $00000400; Video_SECAM_B = $00001000; // TAnalogVideoFormat // TCapState STOPPED = 1; RUNNING = 2; UNINITIALIZED = -1; UNKNOWNSTATE = -2; // TCapState type TCapState = Longint; TRes = Longint; TtagAnalogVideoFormat = DWORD; TAnalogVideoFormat = TtagAnalogVideoFormat; PAnalogVideoFormat = ^TAnalogVideoFormat; TVideoSize = ( SIZEFULLPAL, SIZED1, SIZEVGA, SIZEQVGA, SIZESUBQVGA); PVideoSize = ^TVideoSize; P_Pointer = ^Pointer; TIDVP7010BDLL = class function AdvDVP_CreateSDKInstence(pp: P_Pointer): integer; virtual; stdcall; abstract; function AdvDVP_InitSDK():Integer; virtual; stdcall; abstract; function AdvDVP_CloseSDK():Integer; virtual; stdcall; abstract; function AdvDVP_GetNoOfDevices(pNoOfDevs : PInteger) :Integer; virtual; stdcall; abstract; function AdvDVP_Start(nDevNum : Integer; SwitchingChans : Integer; Main : HWND; hwndPreview: HWND ) :Integer; virtual; stdcall; abstract; function AdvDVP_Stop(nDevNum : Integer ):Integer; virtual; stdcall; abstract; function AdvDVP_GetCapState(nDevNum : Integer ):Integer; virtual; stdcall; abstract; function AdvDVP_IsVideoPresent(nDevNum : Integer; VPresent : PBool) :Integer; virtual; stdcall; abstract; function AdvDVP_GetCurFrameBuffer(nDevNum : Integer; VMux : Integer; bufSize : PLongInt; buf : PByte) :Integer; virtual; stdcall; abstract; function AdvDVP_SetNewFrameCallback(nDevNum : Integer; callback : Integer ) :Integer; virtual; stdcall; abstract; function AdvDVP_GetVideoFormat(nDevNum : Integer; vFormat : PAnalogVideoFormat) :Integer; virtual; stdcall; abstract; function AdvDVP_SetVideoFormat(nDevNum : Integer; vFormat : TAnalogVideoFormat ) :Integer; virtual; stdcall; abstract; function AdvDVP_GetFrameRate(nDevNum : Integer; nFrameRate : Integer) :Integer; virtual; stdcall; abstract; function AdvDVP_SetFrameRate(nDevNum : Integer; SwitchingChans : Integer; nFrameRate : Integer) :Integer; virtual; stdcall; abstract; function AdvDVP_GetResolution(nDevNum : Integer; Size : PVideoSize) :Integer; virtual; stdcall; abstract; function AdvDVP_SetResolution(nDevNum : Integer; Size : TVideoSize ) :Integer; virtual; stdcall; abstract; function AdvDVP_GetVideoInput(nDevNum : Integer; input : PInteger) :Integer; virtual; stdcall; abstract; function AdvDVP_SetVideoInput(nDevNum : Integer; input : Integer) :Integer; virtual; stdcall; abstract; function AdvDVP_GetBrightness(nDevNum : Integer; input: Integer; pnValue : PLongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_SetBrightness(nDevNum : Integer; input: Integer; nValue : LongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_GetContrast(nDevNum : Integer; input: Integer; pnValue : PLongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_SetContrast(nDevNum : Integer; input: Integer; nValue : LongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_GetHue(nDevNum : Integer; input: Integer; pnValue : PLongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_SetHue(nDevNum : Integer; input: Integer; nValue : LongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_GetSaturation(nDevNum : Integer; input: Integer; pnValue : PLongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_SetSaturation(nDevNum : Integer; input: Integer; nValue : LongInt) :Integer; virtual; stdcall; abstract; function AdvDVP_GPIOGetData(nDevNum : Integer; DINum:Integer; value : PBool) :Integer; virtual; stdcall; abstract; function AdvDVP_GPIOSetData(nDevNum : Integer; DONum:Integer; value : Boolean) :Integer; virtual; stdcall; abstract; end; function IDVP7010BDLL : TIDVP7010BDLL ; stdcall; implementation function IDVP7010BDLL; external 'DVP7010B.dll'; end.

    Read the article

  • Logging Into a site that uses Live.com authentication

    - by Josh
    I've been trying to automate a log in to a website I frequent, www.bungie.net. The site is associated with Microsoft and Xbox Live, and as such makes uses of the Windows Live ID API when people log in to their site. I am relatively new to creating web spiders/robots, and I worry that I'm misunderstanding some of the most basic concepts. I've simulated logins to other sites such as Facebook and Gmail, but live.com has given me nothing but trouble. Anyways, I've been using Wireshark and the Firefox addon Tamper Data to try and figure out what I need to post, and what cookies I need to include with my requests. As far as I know these are the steps one must follow to log in to this site. 1. Visit https: //login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268167141&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917 2. Recieve the cookies MSPRequ and MSPOK. 3. Post the values from the form ID "PPSX", the values from the form ID "PPFT", your username, your password all to a changing URL similar to: https: //login.live.com/ppsecure/post.srf?wa=wsignin1.0&rpsnv=11&ct= (there are a few numbers that change at the end of that URL) 4. Live.com returns the user a page with more hidden forms to post. The client then posts the values from the form "ANON", the value from the form "ANONExp" and the values from the form "t" to the URL: http ://www.bung ie.net/Default.aspx?wa=wsignin1.0 5. After posting that data, the user is returned a variety of cookies the most important of which is "BNGAuth" which is the log in cookie for the site. Where I am having trouble is on fifth step, but that doesn't neccesarily mean I've done all the other steps correctly. I post the data from "ANON", "ANONExp" and "t" but instead of being returned a BNGAuth cookie, I'm returned a cookie named "RSPMaybe" and redirected to the home page. When I review the Wireshark log, I noticed something that instantly stood out to me as different between the log when I logged in with Firefox and when my program ran. It could be nothing but I'll include the picture here for you to review. I'm being returned an HTTP packet from the site before I post the data in the fourth step. I'm not sure how this is happening, but it must be a side effect from something I'm doing wrong in the HTTPS steps. using System; using System.Collections.Generic; using System.Collections.Specialized; using System.Text; using System.Net; using System.IO; using System.IO.Compression; using System.Security.Cryptography; using System.Security.Cryptography.X509Certificates; using System.Web; namespace SpiderFromScratch { class Program { static void Main(string[] args) { CookieContainer cookies = new CookieContainer(); Uri url = new Uri("https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268167141&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917"); HttpWebRequest http = (HttpWebRequest)HttpWebRequest.Create(url); http.Timeout = 30000; http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "300"); http.Referer = "http://www.bungie.net/"; http.ContentType = "application/x-www-form-urlencoded"; http.CookieContainer = new CookieContainer(); http.Method = WebRequestMethods.Http.Get; HttpWebResponse response = (HttpWebResponse)http.GetResponse(); StreamReader readStream = new StreamReader(response.GetResponseStream()); string HTML = readStream.ReadToEnd(); readStream.Close(); //gets the cookies (they are set in the eighth header) string[] strCookies = response.Headers.GetValues(8); response.Close(); string name, value; Cookie manualCookie; for (int i = 0; i < strCookies.Length; i++) { name = strCookies[i].Substring(0, strCookies[i].IndexOf("=")); value = strCookies[i].Substring(strCookies[i].IndexOf("=") + 1, strCookies[i].IndexOf(";") - strCookies[i].IndexOf("=") - 1); manualCookie = new Cookie(name, "\"" + value + "\""); Uri manualURL = new Uri("http://login.live.com"); http.CookieContainer.Add(manualURL, manualCookie); } //stores the cookies to be used later cookies = http.CookieContainer; //Get the PPSX value string PPSX = HTML.Remove(0, HTML.IndexOf("PPSX")); PPSX = PPSX.Remove(0, PPSX.IndexOf("value") + 7); PPSX = PPSX.Substring(0, PPSX.IndexOf("\"")); //Get this random PPFT value string PPFT = HTML.Remove(0, HTML.IndexOf("PPFT")); PPFT = PPFT.Remove(0, PPFT.IndexOf("value") + 7); PPFT = PPFT.Substring(0, PPFT.IndexOf("\"")); //Get the random URL you POST to string POSTURL = HTML.Remove(0, HTML.IndexOf("https://login.live.com/ppsecure/post.srf?wa=wsignin1.0&rpsnv=11&ct=")); POSTURL = POSTURL.Substring(0, POSTURL.IndexOf("\"")); //POST with cookies http = (HttpWebRequest)HttpWebRequest.Create(POSTURL); http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "300"); http.CookieContainer = cookies; http.Referer = "https://login.live.com/login.srf?wa=wsignin1.0&rpsnv=11&ct=1268158321&rver=5.5.4177.0&wp=LBI&wreply=http:%2F%2Fwww.bungie.net%2FDefault.aspx&id=42917"; http.ContentType = "application/x-www-form-urlencoded"; http.Method = WebRequestMethods.Http.Post; Stream ostream = http.GetRequestStream(); //used to convert strings into bytes System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding(); //Post information byte[] buffer = encoding.GetBytes("PPSX=" + PPSX +"&PwdPad=IfYouAreReadingThisYouHaveTooMuc&login=YOUREMAILGOESHERE&passwd=YOURWORDGOESHERE" + "&LoginOptions=2&PPFT=" + PPFT); ostream.Write(buffer, 0, buffer.Length); ostream.Close(); HttpWebResponse response2 = (HttpWebResponse)http.GetResponse(); readStream = new StreamReader(response2.GetResponseStream()); HTML = readStream.ReadToEnd(); response2.Close(); ostream.Dispose(); foreach (Cookie cookie in response2.Cookies) { Console.WriteLine(cookie.Name + ": "); Console.WriteLine(cookie.Value); Console.WriteLine(cookie.Expires); Console.WriteLine(); } //SET POSTURL value string POSTANON = "http://www.bungie.net/Default.aspx?wa=wsignin1.0"; //Get the ANON value string ANON = HTML.Remove(0, HTML.IndexOf("ANON")); ANON = ANON.Remove(0, ANON.IndexOf("value") + 7); ANON = ANON.Substring(0, ANON.IndexOf("\"")); ANON = HttpUtility.UrlEncode(ANON); //Get the ANONExp value string ANONExp = HTML.Remove(0, HTML.IndexOf("ANONExp")); ANONExp = ANONExp.Remove(0, ANONExp.IndexOf("value") + 7); ANONExp = ANONExp.Substring(0, ANONExp.IndexOf("\"")); ANONExp = HttpUtility.UrlEncode(ANONExp); //Get the t value string t = HTML.Remove(0, HTML.IndexOf("id=\"t\"")); t = t.Remove(0, t.IndexOf("value") + 7); t = t.Substring(0, t.IndexOf("\"")); t = HttpUtility.UrlEncode(t); //POST the Info and Accept the Bungie Cookies http = (HttpWebRequest)HttpWebRequest.Create(POSTANON); http.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100202 Firefox/3.5.8 (.NET CLR 3.5.30729)"; http.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; http.Headers.Add("Accept-Language", "en-us,en;q=0.5"); http.Headers.Add("Accept-Encoding", "gzip,deflate"); http.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7"); http.Headers.Add("Keep-Alive", "115"); http.CookieContainer = new CookieContainer(); http.ContentType = "application/x-www-form-urlencoded"; http.Method = WebRequestMethods.Http.Post; http.Expect = null; ostream = http.GetRequestStream(); int test = ANON.Length; int test1 = ANONExp.Length; int test2 = t.Length; buffer = encoding.GetBytes("ANON=" + ANON +"&ANONExp=" + ANONExp + "&t=" + t); ostream.Write(buffer, 0, buffer.Length); ostream.Close(); //Here lies the problem, I am not returned the correct cookies. HttpWebResponse response3 = (HttpWebResponse)http.GetResponse(); GZipStream gzip = new GZipStream(response3.GetResponseStream(), CompressionMode.Decompress); readStream = new StreamReader(gzip); HTML = readStream.ReadToEnd(); //gets both cookies string[] strCookies2 = response3.Headers.GetValues(11); response3.Close(); } } } This has given me problems and I've put many hours into learning about HTTP protocols so any help would be appreciated. If there is an article detailing a similar log in to live.com feel free to point the way. I've been looking far and wide for any articles with working solutions. If I could be clearer, feel free to ask as this is my first time using Stack Overflow.

    Read the article

< Previous Page | 146 147 148 149 150 151  | Next Page >