Search Results

Search found 63114 results on 2525 pages for 'set identity insert'.

Page 328/2525 | < Previous Page | 324 325 326 327 328 329 330 331 332 333 334 335 | Next Page >

Boot log from remotely managed/hacked iPhone for analysis

- by user1319903

in reference to my other post. syslog captured immediately after a hard reset for analysis of foul play. Apr 8, 2012 10:08:36 PM - dataaccessd [53] (Notice): 137860|CoreDAV|Warn |Account "iCloud" couldn't reach the server at p03-contacts.icloud.com: Error Domain=NSURLErrorDomain Code=-1009 "The Internet connection appears to be offline." UserInfo=0xde63920 {NSErrorFailingURLStringKey=https://%[email protected]/159665024/principal/, NSErrorFailingURLKey=https://%[email protected]/ /principal/, NSLocalizedDescription=The Internet connection appears to be offline., NSUnderlyingError=0xde7dc00 "The Internet connection appears to be offline."} Apr 8, 2012 10:08:36 PM - UserEventAgent [12] (Warning): TRACE: connection interrupted Apr 8, 2012 10:08:36 PM - UserEventAgent [12] (Warning): DEBUG: disconnected Apr 8, 2012 10:08:36 PM - UserEventAgent [12] (Warning): TRACE: Canceling Apr 8, 2012 10:08:36 PM - UserEventAgent [12] (Warning): TRACE: connection invalid Apr 8, 2012 10:08:35 PM - kernel [0] (Debug): launchd[82] Builtin profile: container (sandbox) Apr 8, 2012 10:08:35 PM - kernel [0] (Debug): launchd[82] Container: /private/var/mobile/Applications/048D35CA-6427-4EC8-8B76-A194697A7CE9 [69] (sandbox) Apr 8, 2012 10:08:35 PM - wifid [29] (Error): WiFi:[355640915.904103]: Client dataaccessd set type to background application Apr 8, 2012 10:08:35 PM - dataaccessd [53] (Notice): 137860|DA|Warn |Delegate 5ADDBE3B-D5FD-43E1-87D4-C1153733EFAB finished a refresh but it is not registered with the refresh manager Apr 8, 2012 10:08:34 PM - timed [31] (Notice): (Note ) CoreTime: Not setting system time to 04/09/2012 05:08:34 from GPS because time is unchanged Apr 8, 2012 10:08:34 PM - timed [31] (Notice): (Note ) CoreTime: Not setting time zone to America/Los_Angeles from NITZ Apr 8, 2012 10:08:33 PM - kernel [0] (Debug): AppleKeyStore:cp_key_store_action(1) Apr 8, 2012 10:08:33 PM - kernel [0] (Debug): AppleKeyStore:Sending lock change Apr 8, 2012 10:08:32 PM - profiled [20] (Notice): (Note ) profiled: Device unlock notification received Apr 8, 2012 10:08:31 PM - softwareupdated [37] (Notice): 3e828d98 : Cleaning up unused prepared updates Apr 8, 2012 10:08:27 PM - mstreamd [43] (Warning): PSDLog: Can't return photoStreamsPublishStreamID because no Apple Account has Photo Streams enabled Apr 8, 2012 10:08:27 PM - mstreamd [43] (Notice): (Note ) mstreamd: Not listening to push notifications. Apr 8, 2012 10:08:27 PM - mstreamd [43] (Warning): PSDLog: Can't return photoStreamsPublishStreamID because no Apple Account has Photo Streams enabled Apr 8, 2012 10:08:27 PM - mstreamd [43] (Notice): (Note ) mstreamd: Not listening to push notifications. Apr 8, 2012 10:08:27 PM - mstreamd [43] (Notice): (Note ) mstreamd: Retrieved push tokens. Dev: 0, Prod: 0 Apr 8, 2012 10:08:27 PM - mstreamd [43] (Notice): (Note ) mstreamd: Media stream daemon starting... Apr 8, 2012 10:08:26 PM - SpringBoard [15] (Notice): SMSCTServer is available and ready to rock. Apr 8, 2012 10:08:26 PM - SpringBoard [15] (Error): mms: * isMmsConfigured = 1 Apr 8, 2012 10:08:26 PM - MobilePhone [79] (Warning): Connection lost, retrying with key exchange. Apr 8, 2012 10:08:26 PM - MobilePhone [79] (Warning): Connection lost, retrying with key exchange. Apr 8, 2012 10:08:26 PM - MobilePhone [79] (Warning): Connection lost, retrying with key exchange. Apr 8, 2012 10:08:26 PM - MobilePhone [79] (Warning): Connection lost, retrying with key exchange. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): BT: failed to get connectable state with error 111 Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Error): WiFi: Consulting "no-sdio-devices" property. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Error): WiFi: "no-sdio-devices" property not found. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): SMS Plugin initialized. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): Telephony plugin initialized Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): SIMToolkit plugin for SpringBoard initialized. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Error): WiFi: Consulting "no-sdio-devices" property. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Error): WiFi: "no-sdio-devices" property not found. Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): WiFi picker plugin initialized Apr 8, 2012 10:08:25 PM - SpringBoard [15] (Warning): EKAlarmEngine: Region monitoring not available or enabled. Trigger ignored! Apr 8, 2012 10:08:24 PM - kernel [0] (Debug): AppleH4CamIn::setPowerStateGated: 0 Apr 8, 2012 10:08:24 PM - kernel [0] (Debug): AppleH4CamIn::power_off_hardware Apr 8, 2012 10:08:24 PM - SpringBoard [15] (Notice): IOMobileFrameBufferGetMirroringCapability returning -536870201 via kIOMFBConnectMethod_GetMirroringCapability Apr 8, 2012 10:08:24 PM - aggregated [61] (Warning): PLAggregateState Error: Leaving state unplugged_screen_off even though we are not in it, doing nothing Apr 8, 2012 10:08:24 PM - aggregated [61] (Warning): PLAggregateState Error: Entering state unplugged_screen_on even though we are already in it, doing nothing Apr 8, 2012 10:08:24 PM - wifid [29] (Error): WiFi:[355640904.616440]: Disable WoW requested by "spd" Apr 8, 2012 10:08:24 PM - SpringBoard [15] (Warning): Application windows are expected to have a root view controller at the end of application launch Apr 8, 2012 10:08:23 PM - SpringBoard [15] (Warning): BTM: attaching to BTServer Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_LoadFirmware_gated: fw len=1232920 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_LoadFirmware_gated - firmware checksum: 0x05935019 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::power_on_hardware Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_Init - No set-file loaded for camera channel 0 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_Init - No set-file loaded for camera channel 1 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_InitialSensorDetection - found sensor on chan 0: 0x0145 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_InitialSensorDetection - found sensor on chan 1: 0x7736 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::power_off_hardware Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_LoadSetfile_gated (camChan=0) Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::ISP_LoadSetfile_gated (camChan=1) Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::setPowerStateGated: 1 Apr 8, 2012 10:08:23 PM - kernel [0] (Debug): AppleH4CamIn::power_on_hardware Apr 8, 2012 10:08:23 PM - profiled [20] (Notice): (Note ) profiled: Locking device Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): HighlandParkResourceMgr::AddFirmware() {'cdma', ' '} added to resources Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function PTP Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction all functions registered- we are ready to start usb stack Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::handleUSBCableDisconnect Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): HighlandParkResourceMgr::AddFirmware() {'gsm ', 'nb '} added to resources Apr 8, 2012 10:08:22 PM - kernel [0] (Debug): HighlandParkResourceMgr::AddFirmware() {'gsm ', 'wb '} added to resources Apr 8, 2012 10:08:22 PM - MRMLowDiskUEA [12] (Notice): MobileDelete: LowDisk Plugin: start Apr 8, 2012 10:08:22 PM - MRMLowDiskUEA [12] (Notice): kqueue registration successful Apr 8, 2012 10:08:22 PM - mediaserverd [44] (Error): 22:08:22.522867 com.apple.AVConference: /SourceCache/GameKitServices/GameKitServices-344.21/AVConference.subproj/Sources/AVConferenceServer.m:1867: AVConferenceServerStart Apr 8, 2012 10:08:22 PM - CommCenter [18] (Notice): Carrier bundle value for recipient address: 28818773 Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice - Configuration: PTP Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: PTP Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice - Configuration: iPod USB Interface Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: USBAudioControl Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: USBAudioStreaming Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: IapOverUsbHid Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice - Configuration: PTP + Apple Mobile Device Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: PTP Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: AppleUSBMux Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice - Configuration: PTP + Apple Mobile Device + Apple USB Ethernet Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: PTP Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: AppleUSBMux Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice Interface: AppleUSBEthernet Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): IOAccessoryPortUSB::start Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function USBAudioControl Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): virtual bool AppleUSBDeviceMux::start(IOService*) build: Feb 1 2012 23:16:46 Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): init_waste Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function AppleUSBMux Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function IapOverUsbHid Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function USBAudioStreaming Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleSynopsysOTGDevice::gated_registerFunction Register function AppleUSBEthernet Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleUSBEthernetDevice::start: Host MAC address = 02:(this Mac address does not physically exist) -edit Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): AppleUSBEthernetDevice: Ethernet address Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): BTServer[66] Builtin profile: BlueTool (sandbox) Apr 8, 2012 10:08:21 PM - kernel [0] (Debug): BTServer[66] Builtin profile: BlueTool (sandbox) Apr 8, 2012 10:08:21 PM - hpfd [50] (Notice): firmware resource loaded { 'cdma' ' ' } Apr 8, 2012 10:08:21 PM - wifid [29] (Error): WiFi:[355640901.282776]: Could not read APPLE80211_IOC_SUPPORTED_CHANNELS err=82 Apr 8, 2012 10:08:21 PM - wifid [29] (Error): WiFi:[355640901.312786]: Client itunesstored is background application Apr 8, 2012 10:08:21 PM - timed [31] (Notice): (Note ) CoreTime: Want active time in 38.24hrs. Need active time in 121.57hrs. Apr 8, 2012 10:08:21 PM - SpringBoard [15] (Notice): MultitouchHID: detection mode: 255-0 (deferring until bootloaded) Apr 8, 2012 10:08:21 PM - CLTM [12] (Error): CLTM: resetting temps: now = 1333948101, last update = -2147483648 Apr 8, 2012 10:08:21 PM - locationd [28] (Error): WiFi:[355640901.852993]: WiFiManager now available Apr 8, 2012 10:08:21 PM - OTACrashCopier [62] (Notice): (Warn ) Failed to read attributes from '/var/mobile/Library/OTALogging/.last_successful_submission_marker' Apr 8, 2012 10:08:21 PM - hpfd [50] (Notice): firmware resource loaded { 'gsm ' 'nb ' } Apr 8, 2012 10:08:21 PM - hpfd [50] (Notice): firmware resource loaded { 'gsm ' 'wb ' } Apr 8, 2012 10:08:20 PM - kernel [0] (Debug): AppleBCMWLANCore::initFirmware(): successful initialization Apr 8, 2012 10:08:20 PM - kernel [0] (Debug): AppleBCMWLANCore:initFirmware(): 2496 PropTxStatus feature is not enabled for this platform Apr 8, 2012 10:08:20 PM - kernel [0] (Debug): AppleBCMWLANCore::initDongle():: creating virtual interface with prefix = ap Apr 8, 2012 10:08:20 PM - kernel [0] (Debug): AppleBCMWLANCore::initDongle(): Core Driver Initialization Time 19.38798583 Apr 8, 2012 10:08:20 PM - kernel [0] (Debug): 000019.281423 hsic-baseband::safetyNet: port is not connected Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 _create_cesm_vault: try to create blob Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 load_activation_records: This is the default record Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 _create_cesm_vault: blob written Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 ping_configd: Not setting host name, it already has one: Pete's iPod Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 lookup_baseband_info_new: radio not ready: kCTPostponementStatusNotReady Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 load_activation_records: This is the default record Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Error): WiFi: Consulting "no-sdio-devices" property. Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Error): WiFi: "no-sdio-devices" property not found. Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: Original act. state: Activated Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: radio not ready, don't change activation status, wait for notification, status: kCTPostponementStatusNotReady Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: Activation state now is Activated Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Warning): lockdown says the device is: [Activated], state is 3 Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Warning): lockdown says we've previously registered: [1], state is 1 Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 notification_worker: now listening for CT notifications Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 notification_worker: we've registered for notifications, now make sure we didn't miss one... Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 load_activation_records: This is the default record Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: Original act. state: Activated Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: radio not ready, don't change activation status, wait for notification, status: kCTPostponementStatusNotReady Apr 8, 2012 10:08:20 PM - lockdownd [23] (Notice): 3e828d98 determine_activation_state_new: Activation state now is Activated Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Notice): Posting 'com.apple.iokit.hid.displayStatus' notifyState=1 Apr 8, 2012 10:08:20 PM - SpringBoard [15] (Notice): __IOHIDLoadBundles: Loaded 1 HID plugin Apr 8, 2012 10:08:19 PM - wifiFirmwareLoader [30] (Warning): [ 18.778 sec] Downloaded firmware, 192512 bytes Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleKeyStore:cp_key_store_action(0) Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleMultitouchN1SPI: downloaded 128 bytes of prox calibration data ("built-in") Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleMultitouchN1SPI: downloaded 1024 bytes of calibration data ("built-in") Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::attachBusGated(): Bus Driver Initialization Time 18.266927958 Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore:attachBusGated(): Starting with MAC Address: 00:f4:b9:2f:d9:8d Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANFirmwareManager::setNVRAMData(): received 778 bytes Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore: Ethernet address 00:f4:b9:2f:d9:8d Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): Loading syscfg. Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleMultitouchN1SPI: downloaded 56264 bytes of firmware data ("0x0084.bin") in 152ms. Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::apple80211_ioctl() Driver not yet initialized, cannot process ioctl Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::apple80211_ioctl() Driver not yet initialized, cannot process ioctl Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AirPort: Enabled AppleBCMWLANCore (link 0, sys 0, user 0) Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::apple80211_ioctl() Driver not yet initialized, cannot process ioctl Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::apple80211_ioctl() Driver not yet initialized, cannot process ioctl Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANBusInterfaceHSIC::loadFirmware(): DL Ver: chip 0x4330, chiprev 0x4 Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): BTServer[66] Builtin profile: BlueTool (sandbox) Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): BCMWLAN Firmware Version: wl0: Dec 22 2011 19:03:58 version 5.95.45 Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::initFirmware(): Firmware supports ap mode; enabling apsta feature (currently enabled) Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::initFirmware(): country code set to XX Apr 8, 2012 10:08:19 PM - configd [14] (Notice): network configuration changed. Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCmdManager::processResponse(): Firmware Error "BCOM Unsupported" on command "WLC_SET_VAR: bus:txglom" (263). Transaction ID 3, length 0 Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::initFirmware(): Glomming not supported on this device: BCOM Unsupported Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::initFirmware: apsta set to 1 Apr 8, 2012 10:08:19 PM - kernel [0] (Debug): AppleBCMWLANCore::handleEventPacket(): WLC_E_FIFO_CREDIT_MAP,length 6 [9 2 5 3 2] Apr 8, 2012 10:08:19 PM - iapd [49] (Error): Timed out trying to acquire capabilities data. Apr 8, 2012 10:08:19 PM - softwareupdated [37] (Notice): 3e828d98 : Cleaning up unused prepared updates Apr 8, 2012 10:08:19 PM - com.apple.misd [63] (Notice): allowing special port forwarding for test fixtures Apr 8, 2012 10:08:19 PM - hpfd [50] (Notice): resource request { 'N94 ', ' ' } Apr 8, 2012 10:08:19 PM - mstreamd [43] (Notice): (Note ) mstreamd: mstreamd starting up. Apr 8, 2012 10:08:18 PM - kernel [0] (Debug): launchd[44] Builtin profile: mediaserverd (sandbox) Apr 8, 2012 10:08:18 PM - kernel [0] (Debug): launchd[49] Builtin profile: iapd (sandbox) Apr 8, 2012 10:08:18 PM - kernel [0] (Debug): launchd[53] Builtin profile: dataaccessd (sandbox) Apr 8, 2012 10:08:18 PM - kernel [0] (Debug): launchd[60] Builtin profile: apsd (sandbox) Apr 8, 2012 10:08:18 PM - kernel [0] (Debug): launchd[66] Builtin profile: BTServer (sandbox) Apr 8, 2012 10:08:18 PM - mDNSResponder [46] (Error): mDNSResponder mDNSResponder-329.10 (Jan 15 2012 19:07:41) starting iOSVers 9 Apr 8, 2012 10:08:18 PM - mDNSResponder [46] (Error): Note: SetDomainSecrets: no keychain support Apr 8, 2012 10:08:18 PM - mDNSResponder [46] (Error): Note: Compiled without SnowLeopard Fine-Grained Power Management support Apr 8, 2012 10:08:18 PM - fseventsd [51] (Critical): event logs in /private/var/.fseventsd out of sync with volume. destroying old logs. (10083 7 10090) Apr 8, 2012 10:08:18 PM - fseventsd [51] (Critical): log dir: /private/var/.fseventsd getting new uuid: 8778E61A-0283-4067-B7DF-F75D109983D1 Apr 8, 2012 10:08:18 PM - fseventsd [51] (Error): failed to make the directory /.fseventsd (30/Read-only file system) Apr 8, 2012 10:08:18 PM - fseventsd [51] (Critical): could not open < (No such file or directory) Apr 8, 2012 10:08:18 PM - fseventsd [51] (Critical): log dir: /tmp getting new uuid: 3919EB54-A54F-4289-864A-5158A25EF9DA Apr 8, 2012 10:08:18 PM - wifid [29] (Error): WiFi:[355640898.328610]: WiFi Preferences is up to date Apr 8, 2012 10:08:18 PM - mDNSResponder [46] (Error): D2DInitialize succeeded Apr 8, 2012 10:08:18 PM - fairplayd.N94 [52] (Notice): Vroum Apr 8, 2012 10:08:18 PM - wifid [29] (Error): WiFi:[355640898.537219]: WiFiManager starting, version: WiFiManager-260.9 Feb 4 2012 13:25:16 Apr 8, 2012 10:08:18 PM - configd [14] (Error): WiFi:[355640898.539342]: WiFiManager now available Apr 8, 2012 10:08:18 PM - keybagd [39] (Error): 3e828d98 main: System Keybag loaded Apr 8, 2012 10:08:18 PM - wifiFirmwareLoader [30] (Warning): [ 18.268 sec] Found AppleBCMWLANBusInterface; downloading FW.. Apr 8, 2012 10:08:18 PM - wifiFirmwareLoader [30] (Warning): Loading "/usr/share/firmware/wifi/4330b2/bcm94330OlympicUNO3.txt", file size = 778 bytes Apr 8, 2012 10:08:18 PM - wifiFirmwareLoader [30] (Warning): [ 18.276 sec] Sending NVRAM, 778 bytes Apr 8, 2012 10:08:18 PM - wifiFirmwareLoader [30] (Warning): Loading "/usr/share/firmware/wifi/4330b2/n94.trx", file size = 192512 bytes Apr 8, 2012 10:08:18 PM - wifiFirmwareLoader [30] (Warning): [ 18.300 sec] Sending firmware, 192512 bytes Apr 8, 2012 10:08:18 PM - lockdownd [23] (Error): libMobileGestalt copyEthernetMacAddress: got 00:f4:b9:2f:d9:8f from syscfg Apr 8, 2012 10:08:18 PM - mediaserverd [44] (Notice): 2012-04-08 10:08:18.817015 PM [AirTunes] HAL plugin started Apr 8, 2012 10:08:18 PM - lockdownd [23] (Error): libMobileGestalt createCFStringWithCFData: Cannot convert NULL data to string Apr 8, 2012 10:08:18 PM - lockdownd [23] (Error): libMobileGestalt copyBasebandBoardSnum: Could not convert baseband board snum data to string Apr 8, 2012 10:08:18 PM - lockdownd [23] (Error): libMobileGestalt createCFStringWithCFData: Cannot convert NULL data to string Apr 8, 2012 10:08:18 PM - lockdownd [23] (Error): libMobileGestalt copyWirelessBoardSnum: Could not convert wireless board snum data to string Apr 8, 2012 10:08:18 PM - lockdownd [23] (Notice): 3e828d98 lockstart_local: Build= 9B179 Apr 8, 2012 10:08:18 PM - lockdownd [23] (Notice): 3e828d98 _load_product_type: using Raptor Certs Apr 8, 2012 10:08:17 PM - wifiFirmwareLoader [30] (Warning): [ 17.590 sec] wlan AppleUSBHSICDevice found Apr 8, 2012 10:08:17 PM - wifiFirmwareLoader [30] (Warning): [ 17.590 sec] WLAN Enumeration attempt 0 / 6: Apr 8, 2012 10:08:17 PM - wifiFirmwareLoader [30] (Warning): [ 17.591 sec] Waiting for AppleBCMWLANBusInterface to enumerate... Apr 8, 2012 10:08:16 PM - CommCenter [18] (Notice): MMS thread running Apr 8, 2012 10:08:16 PM - CommCenter [18] (Notice): Communications Center Started. Apr 8, 2012 10:08:16 PM - CommCenter [18] (Notice): STOP LOCATION UPDATE Apr 8, 2012 10:08:16 PM - locationd [28] (Error): WiFi:[355640896.704327]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:16 PM - locationd [28] (Error): WiFi:[355640896.705542]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:16 PM - locationd [28] (Error): WiFi:[355640896.706648]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:16 PM - locationd [28] (Error): WiFi:[355640896.707418]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:15 PM - kernel [0] (Debug): bool AppleRGBOUT::power_down_hardware(), RGB_CTRL (0x00000000) clk_down_ready is not set after 60 msecs Apr 8, 2012 10:08:14 PM - lockdownd [23] (Notice): 3e828d98 main: Starting Up Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): IOReturn AppleRGBOUT::set_display_device_gated(uint32_t), 1 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): virtual void AppleRGBOUT::do_power_state_change(): fSoft: 1 fHard: 1 swapBusy: 1 fController: 0 - 1 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): bool AppleRGBOUT::power_up_hardware() Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): set_crc_notification_state 0 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): virtual void AppleRGBOUT::do_power_state_change(): fSoft: 0 fHard: 1 swapBusy: 0 fController: 1 - 0 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): bool AppleRGBOUT::power_down_hardware() Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): IOReturn IOMobileFramebufferUserClient::set_hotplug_notify(void *, void *) 0x314b3f0d 0xe215600 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): IOReturn IOMobileFramebufferUserClient::set_hotplug_notify(void *, void *) 0x849d5000 0x876e8828 0x314b3f0d 0xe215600 Apr 8, 2012 10:08:14 PM - kernel [0] (Debug): bool AppleRGBOUT::power_down_hardware(), clock down RGBOUT Apr 8, 2012 10:08:14 PM - SpringBoard [15] (Notice): IOMobileFrameBufferGetMirroringCapability returning -536870201 via kIOMFBConnectMethod_GetMirroringCapability Apr 8, 2012 10:08:14 PM - backupd [21] (Warning): INFO: Account changed (enabled=0, accountID=159665024) Apr 8, 2012 10:08:13 PM - kernel [0] (Debug): launchd[17] Builtin profile: ptpd (sandbox) Apr 8, 2012 10:08:13 PM - UserEventAgent [12] (Warning): Factory called Apr 8, 2012 10:08:13 PM - configd [14] (Error): WiFi:[355640893.157493]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:13 PM - configd [14] (Error): WiFi:[355640893.158197]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:13 PM - configd [14] (Error): WiFi:[355640893.158878]: bootstrap_look_up of WiFiManager server failed Apr 8, 2012 10:08:13 PM - UserEventAgent [12] (Notice): (Note ) PIH: MCUEAPlugin initialized. Apr 8, 2012 10:08:13 PM - UserEventAgent [12] (Error): Querying interface Apr 8, 2012 10:08:13 PM - configd [14] (Error): ioctl(SIOCGIFCAP) failed: Device not configured Apr 8, 2012 10:08:13 PM - configd [14] (Error): ioctl(SIOCGIFCAP) failed: Device not configured Apr 8, 2012 10:08:13 PM - configd [14] (Notice): setting hostname to "Petes-iPod" Apr 8, 2012 10:08:13 PM - configd [14] (Notice): network configuration changed. Apr 8, 2012 10:08:13 PM - UserEventAgent [12] (Warning): TRACE: sending { command = kMBMessageAccountChanged; } Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) profiled: Service starting... Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) profiled: Performing boot time checks. Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) MC: Checking for MDM installation... Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) MC: ...finished checking for MDM installation. Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) profiled: Checking for new carrier profile... Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) profiled: Installing new carrier profile. Apr 8, 2012 10:08:13 PM - profiled [20] (Notice): (Note ) profiled: Carrier profile has already been installed. Apr 8, 2012 10:08:12 PM - com.apple.launchd [1] (Warning): (com.apple.ptpd) The exception server is already claimed! Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: mitigation behavior enabled Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: camera equations enabled Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: thermal monitoring enabled Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: registered for wake notification Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 0 to 16384 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 1 to 546 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 2 to 5461 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 3 to 6553 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 4 to 5461 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 5 to 5461 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 6 to 16384 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 9 to 5461 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set decay on sensor 10 to 5461 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: Set AppleARMPerformanceControllerDVDFactor1 dithering level to 101% Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: Set AppleARMPerformanceControllerDVDFactor0 dithering level to 100% Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: Set charge rate index to 0 Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: HID not ready cannot set BL Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: setting thermal status level to 0 (0) [-32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768] Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: set allowable transmit power limit to 24.000 dBm [-32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768, -32768] Apr 8, 2012 10:08:12 PM - CLTM [12] (Error): CLTM: Could not close relay file Apr 8, 2012 10:08:12 PM - CLTM [12] (Notice): CLTM: thermtgraphrelay is not present

Read the article
NoSQL with RavenDB and ASP.NET MVC - Part 1

- by shiju

A while back, I have blogged NoSQL with MongoDB, NoRM and ASP.NET MVC Part 1 and Part 2 on how to use MongoDB with an ASP.NET MVC application. The NoSQL movement is getting big attention and RavenDB is the latest addition to the NoSQL and document database world. RavenDB is an Open Source (with a commercial option) document database for the .NET/Windows platform developed by Ayende Rahien. Raven stores schema-less JSON documents, allow you to define indexes using Linq queries and focus on low latency and high performance. RavenDB is .NET focused document database which comes with a fully functional .NET client API and supports LINQ. RavenDB comes with two components, a server and a client API. RavenDB is a REST based system, so you can write your own HTTP cleint API. As a .NET developer, RavenDB is becoming my favorite document database. Unlike other document databases, RavenDB is supports transactions using System.Transactions. Also it's supports both embedded and server mode of database. You can access RavenDB site at http://ravendb.netA demo App with ASP.NET MVCLet's create a simple demo app with RavenDB and ASP.NET MVC. To work with RavenDB, do the following steps. Go to http://ravendb.net/download and download the latest build.Unzip the downloaded file.Go to the /Server directory and run the RavenDB.exe. This will start the RavenDB server listening on localhost:8080You can change the port of RavenDB by modifying the "Raven/Port" appSetting value in the RavenDB.exe.config file.When running the RavenDB, it will automatically create a database in the /Data directory. You can change the directory name data by modifying "Raven/DataDirt" appSetting value in the RavenDB.exe.config file.RavenDB provides a browser based admin tool. When the Raven server is running, You can be access the browser based admin tool and view and edit documents and index using your browser admin tool. The web admin tool available at http://localhost:8080The below is the some screen shots of web admin tool Working with ASP.NET MVC To working with RavenDB in our demo ASP.NET MVC application, do the following steps Step 1 - Add reference to Raven Cleint API In our ASP.NET MVC application, Add a reference to the Raven.Client.Lightweight.dll from the Client directory. Step 2 - Create DocumentStoreThe document store would be created once per application. Let's create a DocumentStore on application start-up in the Global.asax.cs. documentStore = new DocumentStore { Url = "http://localhost:8080/" }; documentStore.Initialise(); The above code will create a Raven DB document store and will be listening the server locahost at port 8080 Step 3 - Create DocumentSession on BeginRequest Let's create a DocumentSession on BeginRequest event in the Global.asax.cs. We are using the document session for every unit of work. In our demo app, every HTTP request would be a single Unit of Work (UoW). BeginRequest += (sender, args) => HttpContext.Current.Items[RavenSessionKey] = documentStore.OpenSession(); Step 4 - Destroy the DocumentSession on EndRequest EndRequest += (o, eventArgs) => { var disposable = HttpContext.Current.Items[RavenSessionKey] as IDisposable; if (disposable != null) disposable.Dispose(); }; At the end of HTTP request, we are destroying the DocumentSession object.The below code block shown all the code in the Global.asax.cs private const string RavenSessionKey = "RavenMVC.Session"; private static DocumentStore documentStore; protected void Application_Start() { //Create a DocumentStore in Application_Start //DocumentStore should be created once per application and stored as a singleton. documentStore = new DocumentStore { Url = "http://localhost:8080/" }; documentStore.Initialise(); AreaRegistration.RegisterAllAreas(); RegisterRoutes(RouteTable.Routes); //DI using Unity 2.0 ConfigureUnity(); } public MvcApplication() { //Create a DocumentSession on BeginRequest //create a document session for every unit of work BeginRequest += (sender, args) => HttpContext.Current.Items[RavenSessionKey] = documentStore.OpenSession(); //Destroy the DocumentSession on EndRequest EndRequest += (o, eventArgs) => { var disposable = HttpContext.Current.Items[RavenSessionKey] as IDisposable; if (disposable != null) disposable.Dispose(); }; } //Getting the current DocumentSession public static IDocumentSession CurrentSession { get { return (IDocumentSession)HttpContext.Current.Items[RavenSessionKey]; } } We have setup all necessary code in the Global.asax.cs for working with RavenDB. For our demo app, Let’s write a domain class public class Category { public string Id { get; set; } [Required(ErrorMessage = "Name Required")] [StringLength(25, ErrorMessage = "Must be less than 25 characters")] public string Name { get; set;} public string Description { get; set; } } We have created simple domain entity Category. Let's create repository class for performing CRUD operations against our domain entity Category. public interface ICategoryRepository { Category Load(string id); IEnumerable<Category> GetCategories(); void Save(Category category); void Delete(string id); } public class CategoryRepository : ICategoryRepository { private IDocumentSession session; public CategoryRepository() { session = MvcApplication.CurrentSession; } //Load category based on Id public Category Load(string id) { return session.Load<Category>(id); } //Get all categories public IEnumerable<Category> GetCategories() { var categories= session.LuceneQuery<Category>() .WaitForNonStaleResults() .ToArray(); return categories; } //Insert/Update category public void Save(Category category) { if (string.IsNullOrEmpty(category.Id)) { //insert new record session.Store(category); } else { //edit record var categoryToEdit = Load(category.Id); categoryToEdit.Name = category.Name; categoryToEdit.Description = category.Description; } //save the document session session.SaveChanges(); } //delete a category public void Delete(string id) { var category = Load(id); session.Delete<Category>(category); session.SaveChanges(); } } For every CRUD operations, we are taking the current document session object from HttpContext object. session = MvcApplication.CurrentSession; We are calling the static method CurrentSession from the Global.asax.cs public static IDocumentSession CurrentSession { get { return (IDocumentSession)HttpContext.Current.Items[RavenSessionKey]; } } Retrieve Entities The Load method get the single Category object based on the Id. RavenDB is working based on the REST principles and the Id would be like categories/1. The Id would be created by automatically when a new object is inserted to the document store. The REST uri categories/1 represents a single category object with Id representation of 1. public Category Load(string id) { return session.Load<Category>(id); } The GetCategories method returns all the categories calling the session.LuceneQuery method. RavenDB is using a lucen query syntax for querying. I will explain more details about querying and indexing in my future posts. public IEnumerable<Category> GetCategories() { var categories= session.LuceneQuery<Category>() .WaitForNonStaleResults() .ToArray(); return categories; } Insert/Update entityFor insert/Update a Category entity, we have created Save method in repository class. If the Id property of Category is null, we call Store method of Documentsession for insert a new record. For editing a existing record, we load the Category object and assign the values to the loaded Category object. The session.SaveChanges() will save the changes to document store. //Insert/Update category public void Save(Category category) { if (string.IsNullOrEmpty(category.Id)) { //insert new record session.Store(category); } else { //edit record var categoryToEdit = Load(category.Id); categoryToEdit.Name = category.Name; categoryToEdit.Description = category.Description; } //save the document session session.SaveChanges(); } Delete Entity In the Delete method, we call the document session's delete method and call the SaveChanges method to reflect changes in the document store. public void Delete(string id) { var category = Load(id); session.Delete<Category>(category); session.SaveChanges(); } Let’s create ASP.NET MVC controller and controller actions for handling CRUD operations for the domain class Category public class CategoryController : Controller { private ICategoryRepository categoyRepository; //DI enabled constructor public CategoryController(ICategoryRepository categoyRepository) { this.categoyRepository = categoyRepository; } public ActionResult Index() { var categories = categoyRepository.GetCategories(); if (categories == null) return RedirectToAction("Create"); return View(categories); } [HttpGet] public ActionResult Edit(string id) { var category = categoyRepository.Load(id); return View("Save",category); } // GET: /Category/Create [HttpGet] public ActionResult Create() { var category = new Category(); return View("Save", category); } [HttpPost] public ActionResult Save(Category category) { if (!ModelState.IsValid) { return View("Save", category); } categoyRepository.Save(category); return RedirectToAction("Index"); } [HttpPost] public ActionResult Delete(string id) { categoyRepository.Delete(id); var categories = categoyRepository.GetCategories(); return PartialView("CategoryList", categories); } } RavenDB is an awesome document database and I hope that it will be the winner in .NET space of document database world. The source code of demo application available at http://ravenmvc.codeplex.com/

Read the article
The remote server returned an unexpected response: (400) Bad Request while streaming

- by phenevo

Hi, I have problem with streaming. When I send small file like 1kb txt everything is ok, but when I send larger file like 100 kb jpg or 2gb psd I get: The remote server returned an unexpected response: (400) Bad Request. I'm using windows 7, VS 2010 and .net 3.5 and WCF Service library I lost all my weekend on this and I still see this error :/ Please help me Client: var client = new WpfApplication1.ServiceReference1.Service1Client("WSHttpBinding_IService1"); client.GetString("test"); string filename = @"d:\test.jpg"; FileStream fs = new FileStream(filename, FileMode.Open); try { client.ProcessStreamFromClient(fs); } catch (Exception exception) { Console.WriteLine(exception); } app.config: <?xml version="1.0" encoding="utf-8" ?> <configuration> <system.serviceModel> <bindings> <basicHttpBinding> <binding name="StreamedHttp" closeTimeout="10:01:00" openTimeout="10:01:00" receiveTimeout="10:10:00" sendTimeout="10:01:00" allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="65536000" maxBufferPoolSize="524288000" maxReceivedMessageSize="65536000" messageEncoding="Text" textEncoding="utf-8" transferMode="Streamed" useDefaultWebProxy="true"> <readerQuotas maxDepth="0" maxStringContentLength="0" maxArrayLength="0" maxBytesPerRead="0" maxNameTableCharCount="0" /> <security mode="None"> <transport clientCredentialType="None" proxyCredentialType="None" realm="" /> <message clientCredentialType="UserName" algorithmSuite="Default" /> </security> </binding> </basicHttpBinding> </bindings> <client> <endpoint address="http://localhost:8732/Design_Time_Addresses/WcfServiceLibrary2/Service1/" binding="basicHttpBinding" bindingConfiguration="StreamedHttp" contract="ServiceReference1.IService1" name="WSHttpBinding_IService1" /> </client> </system.serviceModel> </configuration> And Wcf ServiceLibrary: public void ProcessStreamFromClient(Stream str) { using (var outStream = new FileStream(@"e:\test.jpg", FileMode.Create)) { var buffer = new byte[4096]; int count; while ((count = str.Read(buffer, 0, buffer.Length)) > 0) { outStream.Write(buffer, 0, count); } } } App.config <?xml version="1.0" encoding="utf-8" ?> <configuration> <system.web> <compilation debug="true" /> </system.web>  <system.serviceModel> <bindings> <basicHttpBinding> <binding name="Binding1" hostNameComparisonMode="StrongWildcard" maxBufferSize="65536000" transferMode="Streamed" bypassProxyOnLocal="false" closeTimeout="10:01:00" openTimeout="10:01:00" receiveTimeout="10:10:00" sendTimeout="10:01:00" maxBufferPoolSize="524288000" maxReceivedMessageSize="65536000" messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false"> <security mode="None" /> </binding> </basicHttpBinding> </bindings> <client /> <services> <service name="WcfServiceLibrary2.Service1"> <host> <baseAddresses> <add baseAddress="http://localhost:8732/Design_Time_Addresses/WcfServiceLibrary2/Service1/" /> </baseAddresses> </host>   <endpoint address="" binding="basicHttpBinding" contract="WcfServiceLibrary2.IService1">  <identity> <dns value="localhost"/> </identity> </endpoint>    <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/> </service> </services> <behaviors> <serviceBehaviors> <behavior>  <serviceMetadata httpGetEnabled="True"/>  <dataContractSerializer maxItemsInObjectGraph="2147483647"/>  <serviceDebug includeExceptionDetailInFaults="false" /> </behavior> </serviceBehaviors> </behaviors> </system.serviceModel> </configuration>

Read the article
A Taxonomy of Numerical Methods v1

- by JoshReuben

Numerical Analysis – When, What, (but not how) Once you understand the Math & know C++, Numerical Methods are basically blocks of iterative & conditional math code. I found the real trick was seeing the forest for the trees – knowing which method to use for which situation. Its pretty easy to get lost in the details – so I’ve tried to organize these methods in a way that I can quickly look this up. I’ve included links to detailed explanations and to C++ code examples. I’ve tried to classify Numerical methods in the following broad categories: Solving Systems of Linear Equations Solving Non-Linear Equations Iteratively Interpolation Curve Fitting Optimization Numerical Differentiation & Integration Solving ODEs Boundary Problems Solving EigenValue problems Enjoy – I did ! Solving Systems of Linear Equations Overview Solve sets of algebraic equations with x unknowns The set is commonly in matrix form Gauss-Jordan Elimination http://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_elimination C++: http://www.codekeep.net/snippets/623f1923-e03c-4636-8c92-c9dc7aa0d3c0.aspx Produces solution of the equations & the coefficient matrix Efficient, stable 2 steps: · Forward Elimination – matrix decomposition: reduce set to triangular form (0s below the diagonal) or row echelon form. If degenerate, then there is no solution · Backward Elimination –write the original matrix as the product of ints inverse matrix & its reduced row-echelon matrix à reduce set to row canonical form & use back-substitution to find the solution to the set Elementary ops for matrix decomposition: · Row multiplication · Row switching · Add multiples of rows to other rows Use pivoting to ensure rows are ordered for achieving triangular form LU Decomposition http://en.wikipedia.org/wiki/LU_decomposition C++: http://ganeshtiwaridotcomdotnp.blogspot.co.il/2009/12/c-c-code-lu-decomposition-for-solving.html Represent the matrix as a product of lower & upper triangular matrices A modified version of GJ Elimination Advantage – can easily apply forward & backward elimination to solve triangular matrices Techniques: · Doolittle Method – sets the L matrix diagonal to unity · Crout Method - sets the U matrix diagonal to unity Note: both the L & U matrices share the same unity diagonal & can be stored compactly in the same matrix Gauss-Seidel Iteration http://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_method C++: http://www.nr.com/forum/showthread.php?t=722 Transform the linear set of equations into a single equation & then use numerical integration (as integration formulas have Sums, it is implemented iteratively). an optimization of Gauss-Jacobi: 1.5 times faster, requires 0.25 iterations to achieve the same tolerance Solving Non-Linear Equations Iteratively find roots of polynomials – there may be 0, 1 or n solutions for an n order polynomial use iterative techniques Iterative methods · used when there are no known analytical techniques · Requires set functions to be continuous & differentiable · Requires an initial seed value – choice is critical to convergence à conduct multiple runs with different starting points & then select best result · Systematic - iterate until diminishing returns, tolerance or max iteration conditions are met · bracketing techniques will always yield convergent solutions, non-bracketing methods may fail to converge Incremental method if a nonlinear function has opposite signs at 2 ends of a small interval x1 & x2, then there is likely to be a solution in their interval – solutions are detected by evaluating a function over interval steps, for a change in sign, adjusting the step size dynamically. Limitations – can miss closely spaced solutions in large intervals, cannot detect degenerate (coinciding) solutions, limited to functions that cross the x-axis, gives false positives for singularities Fixed point method http://en.wikipedia.org/wiki/Fixed-point_iteration C++: http://books.google.co.il/books?id=weYj75E_t6MC&pg=PA79&lpg=PA79&dq=fixed+point+method++c%2B%2B&source=bl&ots=LQ-5P_taoC&sig=lENUUIYBK53tZtTwNfHLy5PEWDk&hl=en&sa=X&ei=wezDUPW1J5DptQaMsIHQCw&redir_esc=y#v=onepage&q=fixed%20point%20method%20%20c%2B%2B&f=false Algebraically rearrange a solution to isolate a variable then apply incremental method Bisection method http://en.wikipedia.org/wiki/Bisection_method C++: http://numericalcomputing.wordpress.com/category/algorithms/ Bracketed - Select an initial interval, keep bisecting it ad midpoint into sub-intervals and then apply incremental method on smaller & smaller intervals – zoom in Adv: unaffected by function gradient à reliable Disadv: slow convergence False Position Method http://en.wikipedia.org/wiki/False_position_method C++: http://www.dreamincode.net/forums/topic/126100-bisection-and-false-position-methods/ Bracketed - Select an initial interval , & use the relative value of function at interval end points to select next sub-intervals (estimate how far between the end points the solution might be & subdivide based on this) Newton-Raphson method http://en.wikipedia.org/wiki/Newton's_method C++: http://www-users.cselabs.umn.edu/classes/Summer-2012/csci1113/index.php?page=./newt3 Also known as Newton's method Convenient, efficient Not bracketed – only a single initial guess is required to start iteration – requires an analytical expression for the first derivative of the function as input. Evaluates the function & its derivative at each step. Can be extended to the Newton MutiRoot method for solving multiple roots Can be easily applied to an of n-coupled set of non-linear equations – conduct a Taylor Series expansion of a function, dropping terms of order n, rewrite as a Jacobian matrix of PDs & convert to simultaneous linear equations !!! Secant Method http://en.wikipedia.org/wiki/Secant_method C++: http://forum.vcoderz.com/showthread.php?p=205230 Unlike N-R, can estimate first derivative from an initial interval (does not require root to be bracketed) instead of inputting it Since derivative is approximated, may converge slower. Is fast in practice as it does not have to evaluate the derivative at each step. Similar implementation to False Positive method Birge-Vieta Method http://mat.iitm.ac.in/home/sryedida/public_html/caimna/transcendental/polynomial%20methods/bv%20method.html C++: http://books.google.co.il/books?id=cL1boM2uyQwC&pg=SA3-PA51&lpg=SA3-PA51&dq=Birge-Vieta+Method+c%2B%2B&source=bl&ots=QZmnDTK3rC&sig=BPNcHHbpR_DKVoZXrLi4nVXD-gg&hl=en&sa=X&ei=R-_DUK2iNIjzsgbE5ID4Dg&redir_esc=y#v=onepage&q=Birge-Vieta%20Method%20c%2B%2B&f=false combines Horner's method of polynomial evaluation (transforming into lesser degree polynomials that are more computationally efficient to process) with Newton-Raphson to provide a computational speed-up Interpolation Overview Construct new data points for as close as possible fit within range of a discrete set of known points (that were obtained via sampling, experimentation) Use Taylor Series Expansion of a function f(x) around a specific value for x Linear Interpolation http://en.wikipedia.org/wiki/Linear_interpolation C++: http://www.hamaluik.com/?p=289 Straight line between 2 points à concatenate interpolants between each pair of data points Bilinear Interpolation http://en.wikipedia.org/wiki/Bilinear_interpolation C++: http://supercomputingblog.com/graphics/coding-bilinear-interpolation/2/ Extension of the linear function for interpolating functions of 2 variables – perform linear interpolation first in 1 direction, then in another. Used in image processing – e.g. texture mapping filter. Uses 4 vertices to interpolate a value within a unit cell. Lagrange Interpolation http://en.wikipedia.org/wiki/Lagrange_polynomial C++: http://www.codecogs.com/code/maths/approximation/interpolation/lagrange.php For polynomials Requires recomputation for all terms for each distinct x value – can only be applied for small number of nodes Numerically unstable Barycentric Interpolation http://epubs.siam.org/doi/pdf/10.1137/S0036144502417715 C++: http://www.gamedev.net/topic/621445-barycentric-coordinates-c-code-check/ Rearrange the terms in the equation of the Legrange interpolation by defining weight functions that are independent of the interpolated value of x Newton Divided Difference Interpolation http://en.wikipedia.org/wiki/Newton_polynomial C++: http://jee-appy.blogspot.co.il/2011/12/newton-divided-difference-interpolation.html Hermite Divided Differences: Interpolation polynomial approximation for a given set of data points in the NR form - divided differences are used to approximately calculate the various differences. For a given set of 3 data points , fit a quadratic interpolant through the data Bracketed functions allow Newton divided differences to be calculated recursively Difference table Cubic Spline Interpolation http://en.wikipedia.org/wiki/Spline_interpolation C++: https://www.marcusbannerman.co.uk/index.php/home/latestarticles/42-articles/96-cubic-spline-class.html Spline is a piecewise polynomial Provides smoothness – for interpolations with significantly varying data Use weighted coefficients to bend the function to be smooth & its 1st & 2nd derivatives are continuous through the edge points in the interval Curve Fitting A generalization of interpolating whereby given data points may contain noise à the curve does not necessarily pass through all the points Least Squares Fit http://en.wikipedia.org/wiki/Least_squares C++: http://www.ccas.ru/mmes/educat/lab04k/02/least-squares.c Residual – difference between observed value & expected value Model function is often chosen as a linear combination of the specified functions Determines: A) The model instance in which the sum of squared residuals has the least value B) param values for which model best fits data Straight Line Fit Linear correlation between independent variable and dependent variable Linear Regression http://en.wikipedia.org/wiki/Linear_regression C++: http://www.oocities.org/david_swaim/cpp/linregc.htm Special case of statistically exact extrapolation Leverage least squares Given a basis function, the sum of the residuals is determined and the corresponding gradient equation is expressed as a set of normal linear equations in matrix form that can be solved (e.g. using LU Decomposition) Can be weighted - Drop the assumption that all errors have the same significance –-> confidence of accuracy is different for each data point. Fit the function closer to points with higher weights Polynomial Fit - use a polynomial basis function Moving Average http://en.wikipedia.org/wiki/Moving_average C++: http://www.codeproject.com/Articles/17860/A-Simple-Moving-Average-Algorithm Used for smoothing (cancel fluctuations to highlight longer-term trends & cycles), time series data analysis, signal processing filters Replace each data point with average of neighbors. Can be simple (SMA), weighted (WMA), exponential (EMA). Lags behind latest data points – extra weight can be given to more recent data points. Weights can decrease arithmetically or exponentially according to distance from point. Parameters: smoothing factor, period, weight basis Optimization Overview Given function with multiple variables, find Min (or max by minimizing –f(x)) Iterative approach Efficient, but not necessarily reliable Conditions: noisy data, constraints, non-linear models Detection via sign of first derivative - Derivative of saddle points will be 0 Local minima Bisection method Similar method for finding a root for a non-linear equation Start with an interval that contains a minimum Golden Search method http://en.wikipedia.org/wiki/Golden_section_search C++: http://www.codecogs.com/code/maths/optimization/golden.php Bisect intervals according to golden ratio 0.618.. Achieves reduction by evaluating a single function instead of 2 Newton-Raphson Method Brent method http://en.wikipedia.org/wiki/Brent's_method C++: http://people.sc.fsu.edu/~jburkardt/cpp_src/brent/brent.cpp Based on quadratic or parabolic interpolation – if the function is smooth & parabolic near to the minimum, then a parabola fitted through any 3 points should approximate the minima – fails when the 3 points are collinear , in which case the denominator is 0 Simplex Method http://en.wikipedia.org/wiki/Simplex_algorithm C++: http://www.codeguru.com/cpp/article.php/c17505/Simplex-Optimization-Algorithm-and-Implemetation-in-C-Programming.htm Find the global minima of any multi-variable function Direct search – no derivatives required At each step it maintains a non-degenerative simplex – a convex hull of n+1 vertices. Obtains the minimum for a function with n variables by evaluating the function at n-1 points, iteratively replacing the point of worst result with the point of best result, shrinking the multidimensional simplex around the best point. Point replacement involves expanding & contracting the simplex near the worst value point to determine a better replacement point Oscillation can be avoided by choosing the 2nd worst result Restart if it gets stuck Parameters: contraction & expansion factors Simulated Annealing http://en.wikipedia.org/wiki/Simulated_annealing C++: http://code.google.com/p/cppsimulatedannealing/ Analogy to heating & cooling metal to strengthen its structure Stochastic method – apply random permutation search for global minima - Avoid entrapment in local minima via hill climbing Heating schedule - Annealing schedule params: temperature, iterations at each temp, temperature delta Cooling schedule – can be linear, step-wise or exponential Differential Evolution http://en.wikipedia.org/wiki/Differential_evolution C++: http://www.amichel.com/de/doc/html/ More advanced stochastic methods analogous to biological processes: Genetic algorithms, evolution strategies Parallel direct search method against multiple discrete or continuous variables Initial population of variable vectors chosen randomly – if weighted difference vector of 2 vectors yields a lower objective function value then it replaces the comparison vector Many params: #parents, #variables, step size, crossover constant etc Convergence is slow – many more function evaluations than simulated annealing Numerical Differentiation Overview 2 approaches to finite difference methods: · A) approximate function via polynomial interpolation then differentiate · B) Taylor series approximation – additionally provides error estimate Finite Difference methods http://en.wikipedia.org/wiki/Finite_difference_method C++: http://www.wpi.edu/Pubs/ETD/Available/etd-051807-164436/unrestricted/EAMPADU.pdf Find differences between high order derivative values - Approximate differential equations by finite differences at evenly spaced data points Based on forward & backward Taylor series expansion of f(x) about x plus or minus multiples of delta h. Forward / backward difference - the sums of the series contains even derivatives and the difference of the series contains odd derivatives – coupled equations that can be solved. Provide an approximation of the derivative within a O(h^2) accuracy There is also central difference & extended central difference which has a O(h^4) accuracy Richardson Extrapolation http://en.wikipedia.org/wiki/Richardson_extrapolation C++: http://mathscoding.blogspot.co.il/2012/02/introduction-richardson-extrapolation.html A sequence acceleration method applied to finite differences Fast convergence, high accuracy O(h^4) Derivatives via Interpolation Cannot apply Finite Difference method to discrete data points at uneven intervals – so need to approximate the derivative of f(x) using the derivative of the interpolant via 3 point Lagrange Interpolation Note: the higher the order of the derivative, the lower the approximation precision Numerical Integration Estimate finite & infinite integrals of functions More accurate procedure than numerical differentiation Use when it is not possible to obtain an integral of a function analytically or when the function is not given, only the data points are Newton Cotes Methods http://en.wikipedia.org/wiki/Newton%E2%80%93Cotes_formulas C++: http://www.siafoo.net/snippet/324 For equally spaced data points Computationally easy – based on local interpolation of n rectangular strip areas that is piecewise fitted to a polynomial to get the sum total area Evaluate the integrand at n+1 evenly spaced points – approximate definite integral by Sum Weights are derived from Lagrange Basis polynomials Leverage Trapezoidal Rule for default 2nd formulas, Simpson 1/3 Rule for substituting 3 point formulas, Simpson 3/8 Rule for 4 point formulas. For 4 point formulas use Bodes Rule. Higher orders obtain more accurate results Trapezoidal Rule uses simple area, Simpsons Rule replaces the integrand f(x) with a quadratic polynomial p(x) that uses the same values as f(x) for its end points, but adds a midpoint Romberg Integration http://en.wikipedia.org/wiki/Romberg's_method C++: http://code.google.com/p/romberg-integration/downloads/detail?name=romberg.cpp&can=2&q= Combines trapezoidal rule with Richardson Extrapolation Evaluates the integrand at equally spaced points The integrand must have continuous derivatives Each R(n,m) extrapolation uses a higher order integrand polynomial replacement rule (zeroth starts with trapezoidal) à a lower triangular matrix set of equation coefficients where the bottom right term has the most accurate approximation. The process continues until the difference between 2 successive diagonal terms becomes sufficiently small. Gaussian Quadrature http://en.wikipedia.org/wiki/Gaussian_quadrature C++: http://www.alglib.net/integration/gaussianquadratures.php Data points are chosen to yield best possible accuracy – requires fewer evaluations Ability to handle singularities, functions that are difficult to evaluate The integrand can include a weighting function determined by a set of orthogonal polynomials. Points & weights are selected so that the integrand yields the exact integral if f(x) is a polynomial of degree <= 2n+1 Techniques (basically different weighting functions): · Gauss-Legendre Integration w(x)=1 · Gauss-Laguerre Integration w(x)=e^-x · Gauss-Hermite Integration w(x)=e^-x^2 · Gauss-Chebyshev Integration w(x)= 1 / Sqrt(1-x^2) Solving ODEs Use when high order differential equations cannot be solved analytically Evaluated under boundary conditions RK for systems – a high order differential equation can always be transformed into a coupled first order system of equations Euler method http://en.wikipedia.org/wiki/Euler_method C++: http://rosettacode.org/wiki/Euler_method First order Runge–Kutta method. Simple recursive method – given an initial value, calculate derivative deltas. Unstable & not very accurate (O(h) error) – not used in practice A first-order method - the local error (truncation error per step) is proportional to the square of the step size, and the global error (error at a given time) is proportional to the step size In evolving solution between data points xn & xn+1, only evaluates derivatives at beginning of interval xn à asymmetric at boundaries Higher order Runge Kutta http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods C++: http://www.dreamincode.net/code/snippet1441.htm 2nd & 4th order RK - Introduces parameterized midpoints for more symmetric solutions à accuracy at higher computational cost Adaptive RK – RK-Fehlberg – estimate the truncation at each integration step & automatically adjust the step size to keep error within prescribed limits. At each step 2 approximations are compared – if in disagreement to a specific accuracy, the step size is reduced Boundary Value Problems Where solution of differential equations are located at 2 different values of the independent variable x à more difficult, because cannot just start at point of initial value – there may not be enough starting conditions available at the end points to produce a unique solution An n-order equation will require n boundary conditions – need to determine the missing n-1 conditions which cause the given conditions at the other boundary to be satisfied Shooting Method http://en.wikipedia.org/wiki/Shooting_method C++: http://ganeshtiwaridotcomdotnp.blogspot.co.il/2009/12/c-c-code-shooting-method-for-solving.html Iteratively guess the missing values for one end & integrate, then inspect the discrepancy with the boundary values of the other end to adjust the estimate Given the starting boundary values u1 & u2 which contain the root u, solve u given the false position method (solving the differential equation as an initial value problem via 4th order RK), then use u to solve the differential equations. Finite Difference Method For linear & non-linear systems Higher order derivatives require more computational steps – some combinations for boundary conditions may not work though Improve the accuracy by increasing the number of mesh points Solving EigenValue Problems An eigenvalue can substitute a matrix when doing matrix multiplication à convert matrix multiplication into a polynomial EigenValue For a given set of equations in matrix form, determine what are the solution eigenvalue & eigenvectors Similar Matrices - have same eigenvalues. Use orthogonal similarity transforms to reduce a matrix to diagonal form from which eigenvalue(s) & eigenvectors can be computed iteratively Jacobi method http://en.wikipedia.org/wiki/Jacobi_method C++: http://people.sc.fsu.edu/~jburkardt/classes/acs2_2008/openmp/jacobi/jacobi.html Robust but Computationally intense – use for small matrices < 10x10 Power Iteration http://en.wikipedia.org/wiki/Power_iteration For any given real symmetric matrix, generate the largest single eigenvalue & its eigenvectors Simplest method – does not compute matrix decomposition à suitable for large, sparse matrices Inverse Iteration Variation of power iteration method – generates the smallest eigenvalue from the inverse matrix Rayleigh Method http://en.wikipedia.org/wiki/Rayleigh's_method_of_dimensional_analysis Variation of power iteration method Rayleigh Quotient Method Variation of inverse iteration method Matrix Tri-diagonalization Method Use householder algorithm to reduce an NxN symmetric matrix to a tridiagonal real symmetric matrix vua N-2 orthogonal transforms Whats Next Outside of Numerical Methods there are lots of different types of algorithms that I’ve learned over the decades: Data Mining – (I covered this briefly in a previous post: http://geekswithblogs.net/JoshReuben/archive/2007/12/31/ssas-dm-algorithms.aspx ) Search & Sort Routing Problem Solving Logical Theorem Proving Planning Probabilistic Reasoning Machine Learning Solvers (eg MIP) Bioinformatics (Sequence Alignment, Protein Folding) Quant Finance (I read Wilmott’s books – interesting) Sooner or later, I’ll cover the above topics as well.

Read the article
SSH new connection begins to hang (not reject or terminate) after a day or so on Ubuntu 13.04 server

- by kross

Recently we upgraded the server from 12.04 LTS server to 13.04. All was well, including after a reboot. With all packages updated we began to see a strange issue, ssh works for a day or so (unclear on timing) then a later request for SSH hangs (cannot ctrl+c, nothing). It is up and serving webserver traffic etc. Port 22 is open (ips etc altered slightly for posting): nmap -T4 -A x.acme.com Starting Nmap 6.40 ( http://nmap.org ) at 2013-09-12 16:01 CDT Nmap scan report for x.acme.com (69.137.56.18) Host is up (0.026s latency). rDNS record for 69.137.56.18: c-69-137-56-18.hsd1.tn.provider.net Not shown: 998 filtered ports PORT STATE SERVICE VERSION 22/tcp open ssh OpenSSH 6.1p1 Debian 4 (protocol 2.0) | ssh-hostkey: 1024 54:d3:e3:38:44:f4:20:a4:e7:42:49:d0:a7:f1:3e:21 (DSA) | 2048 dc:21:77:3b:f4:4e:74:d0:87:33:14:40:04:68:33:a6 (RSA) |_256 45:69:10:79:5a:9f:0b:f0:66:15:39:87:b9:a1:37:f7 (ECDSA) 80/tcp open http Jetty 7.6.2.v20120308 | http-title: Log in as a Bamboo user - Atlassian Bamboo |_Requested resource was http://x.acme.com/userlogin!default.action;jsessionid=19v135zn8cl1tgso28fse4d50?os_destination=%2Fstart.action Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel Service detection performed. Please report any incorrect results at http://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 12.89 seconds Here is the ssh -vvv: ssh -vvv x.acme.com OpenSSH_5.9p1, OpenSSL 0.9.8x 10 May 2012 debug1: Reading configuration data /Users/tfergeson/.ssh/config debug1: Reading configuration data /etc/ssh_config debug1: /etc/ssh_config line 20: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to x.acme.com [69.137.56.18] port 22. debug1: Connection established. debug3: Incorrect RSA1 identifier debug3: Could not load "/Users/tfergeson/.ssh/id_rsa" as a RSA1 public key debug1: identity file /Users/tfergeson/.ssh/id_rsa type 1 debug1: identity file /Users/tfergeson/.ssh/id_rsa-cert type -1 debug1: identity file /Users/tfergeson/.ssh/id_dsa type -1 debug1: identity file /Users/tfergeson/.ssh/id_dsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_6.1p1 Debian-4 debug1: match: OpenSSH_6.1p1 Debian-4 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.9 debug2: fd 3 setting O_NONBLOCK debug3: load_hostkeys: loading entries for host "x.acme.com" from file "/Users/tfergeson/.ssh/known_hosts" debug3: load_hostkeys: found key type RSA in file /Users/tfergeson/.ssh/known_hosts:10 debug3: load_hostkeys: loaded 1 keys debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],ssh-rsa debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: [email protected],[email protected],ssh-rsa,[email protected],[email protected],ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected] debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected] debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,[email protected],hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,[email protected],hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,[email protected],zlib debug2: kex_parse_kexinit: none,[email protected],zlib debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256 debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected] debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected] debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,[email protected] debug2: kex_parse_kexinit: none,[email protected] debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-md5 debug1: kex: server->client aes128-ctr hmac-md5 none debug2: mac_setup: found hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug2: dh_gen_key: priv key bits set: 130/256 debug2: bits set: 503/1024 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Server host key: RSA dc:21:77:3b:f4:4e:74:d0:87:33:14:40:04:68:33:a6 debug3: load_hostkeys: loading entries for host "x.acme.com" from file "/Users/tfergeson/.ssh/known_hosts" debug3: load_hostkeys: found key type RSA in file /Users/tfergeson/.ssh/known_hosts:10 debug3: load_hostkeys: loaded 1 keys debug3: load_hostkeys: loading entries for host "69.137.56.18" from file "/Users/tfergeson/.ssh/known_hosts" debug3: load_hostkeys: found key type RSA in file /Users/tfergeson/.ssh/known_hosts:6 debug3: load_hostkeys: loaded 1 keys debug1: Host 'x.acme.com' is known and matches the RSA host key. debug1: Found key in /Users/tfergeson/.ssh/known_hosts:10 debug2: bits set: 493/1024 debug1: ssh_rsa_verify: signature correct debug2: kex_derive_keys debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /Users/tfergeson/.ssh/id_rsa (0x7ff189c1d7d0) debug2: key: /Users/tfergeson/.ssh/id_dsa (0x0) debug1: Authentications that can continue: publickey debug3: start over, passed a different list publickey debug3: preferred publickey,keyboard-interactive,password debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Offering RSA public key: /Users/tfergeson/.ssh/id_rsa debug3: send_pubkey_test debug2: we sent a publickey packet, wait for reply debug1: Server accepts key: pkalg ssh-rsa blen 277 debug2: input_userauth_pk_ok: fp 3c:e5:29:6c:9d:27:d1:7d:e8:09:a2:e8:8e:6e:af:6f debug3: sign_and_send_pubkey: RSA 3c:e5:29:6c:9d:27:d1:7d:e8:09:a2:e8:8e:6e:af:6f debug1: read PEM private key done: type RSA debug1: Authentication succeeded (publickey). Authenticated to x.acme.com ([69.137.56.18]:22). debug1: channel 0: new [client-session] debug3: ssh_session2_open: channel_new: 0 debug2: channel 0: send open debug1: Requesting [email protected] debug1: Entering interactive session. debug2: callback start debug2: client_session2_setup: id 0 debug2: fd 3 setting TCP_NODELAY debug2: channel 0: request pty-req confirm 1 debug1: Sending environment. debug3: Ignored env ATLAS_OPTS debug3: Ignored env rvm_bin_path debug3: Ignored env TERM_PROGRAM debug3: Ignored env GEM_HOME debug3: Ignored env SHELL debug3: Ignored env TERM debug3: Ignored env CLICOLOR debug3: Ignored env IRBRC debug3: Ignored env TMPDIR debug3: Ignored env Apple_PubSub_Socket_Render debug3: Ignored env TERM_PROGRAM_VERSION debug3: Ignored env MY_RUBY_HOME debug3: Ignored env TERM_SESSION_ID debug3: Ignored env USER debug3: Ignored env COMMAND_MODE debug3: Ignored env rvm_path debug3: Ignored env COM_GOOGLE_CHROME_FRAMEWORK_SERVICE_PROCESS/USERS/tfergeson/LIBRARY/APPLICATION_SUPPORT/GOOGLE/CHROME_SOCKET debug3: Ignored env JPDA_ADDRESS debug3: Ignored env APDK_HOME debug3: Ignored env SSH_AUTH_SOCK debug3: Ignored env Apple_Ubiquity_Message debug3: Ignored env __CF_USER_TEXT_ENCODING debug3: Ignored env rvm_sticky_flag debug3: Ignored env MAVEN_OPTS debug3: Ignored env LSCOLORS debug3: Ignored env rvm_prefix debug3: Ignored env PATH debug3: Ignored env PWD debug3: Ignored env JAVA_HOME debug1: Sending env LANG = en_US.UTF-8 debug2: channel 0: request env confirm 0 debug3: Ignored env JPDA_TRANSPORT debug3: Ignored env rvm_version debug3: Ignored env M2_HOME debug3: Ignored env HOME debug3: Ignored env SHLVL debug3: Ignored env rvm_ruby_string debug3: Ignored env LOGNAME debug3: Ignored env M2_REPO debug3: Ignored env GEM_PATH debug3: Ignored env AWS_RDS_HOME debug3: Ignored env rvm_delete_flag debug3: Ignored env EC2_PRIVATE_KEY debug3: Ignored env RUBY_VERSION debug3: Ignored env SECURITYSESSIONID debug3: Ignored env EC2_CERT debug3: Ignored env _ debug2: channel 0: request shell confirm 1 debug2: callback done debug2: channel 0: open confirm rwindow 0 rmax 32768 I can hard reboot (only mac monitors at that location) and it will again be accessible. This now happens every single time. It is imperative that I get it sorted. The strange thing is that it behaves initially then starts to hang after several hours. I perused logs previously and nothing stood out. From the auth.log, I can see that it has allowed me in, but still I get nothing back on the client side: Sep 20 12:47:50 cbear sshd[25376]: Accepted publickey for tfergeson from 10.1.10.14 port 54631 ssh2 Sep 20 12:47:50 cbear sshd[25376]: pam_unix(sshd:session): session opened for user tfergeson by (uid=0) UPDATES: Still occurring even after setting UseDNS no and commenting out #session optional pam_mail.so standard noenv This does not appear to be a network/dns related issue, as all services running on the machine are as responsive and accessible as ever, with the exception of sshd. Any thoughts on where to start?

Read the article
Beware Sneaky Reads with Unique Indexes

- by Paul White NZ

A few days ago, Sandra Mueller (twitter | blog) asked a question using twitter’s #sqlhelp hash tag: “Might SQL Server retrieve (out-of-row) LOB data from a table, even if the column isn’t referenced in the query?” Leaving aside trivial cases (like selecting a computed column that does reference the LOB data), one might be tempted to say that no, SQL Server does not read data you haven’t asked for. In general, that’s quite correct; however there are cases where SQL Server might sneakily retrieve a LOB column… Example Table Here’s a T-SQL script to create that table and populate it with 1,000 rows: CREATE TABLE dbo.LOBtest ( pk INTEGER IDENTITY NOT NULL, some_value INTEGER NULL, lob_data VARCHAR(MAX) NULL, another_column CHAR(5) NULL, CONSTRAINT [PK dbo.LOBtest pk] PRIMARY KEY CLUSTERED (pk ASC) ); GO DECLARE @Data VARCHAR(MAX); SET @Data = REPLICATE(CONVERT(VARCHAR(MAX), 'x'), 65540); WITH Numbers (n) AS ( SELECT ROW_NUMBER() OVER (ORDER BY (SELECT 0)) FROM master.sys.columns C1, master.sys.columns C2 ) INSERT LOBtest WITH (TABLOCKX) ( some_value, lob_data ) SELECT TOP (1000) N.n, @Data FROM Numbers N WHERE N.n <= 1000; Test 1: A Simple Update Let’s run a query to subtract one from every value in the some_value column: UPDATE dbo.LOBtest WITH (TABLOCKX) SET some_value = some_value - 1; As you might expect, modifying this integer column in 1,000 rows doesn’t take very long, or use many resources. The STATITICS IO and TIME output shows a total of 9 logical reads, and 25ms elapsed time. The query plan is also very simple: Looking at the Clustered Index Scan, we can see that SQL Server only retrieves the pk and some_value columns during the scan: The pk column is needed by the Clustered Index Update operator to uniquely identify the row that is being changed. The some_value column is used by the Compute Scalar to calculate the new value. (In case you are wondering what the Top operator is for, it is used to enforce SET ROWCOUNT). Test 2: Simple Update with an Index Now let’s create a nonclustered index keyed on the some_value column, with lob_data as an included column: CREATE NONCLUSTERED INDEX [IX dbo.LOBtest some_value (lob_data)] ON dbo.LOBtest (some_value) INCLUDE ( lob_data ) WITH ( FILLFACTOR = 100, MAXDOP = 1, SORT_IN_TEMPDB = ON ); This is not a useful index for our simple update query; imagine that someone else created it for a different purpose. Let’s run our update query again: UPDATE dbo.LOBtest WITH (TABLOCKX) SET some_value = some_value - 1; We find that it now requires 4,014 logical reads and the elapsed query time has increased to around 100ms. The extra logical reads (4 per row) are an expected consequence of maintaining the nonclustered index. The query plan is very similar to before (click to enlarge): The Clustered Index Update operator picks up the extra work of maintaining the nonclustered index. The new Compute Scalar operators detect whether the value in the some_value column has actually been changed by the update. SQL Server may be able to skip maintaining the nonclustered index if the value hasn’t changed (see my previous post on non-updating updates for details). Our simple query does change the value of some_data in every row, so this optimization doesn’t add any value in this specific case. The output list of columns from the Clustered Index Scan hasn’t changed from the one shown previously: SQL Server still just reads the pk and some_data columns. Cool. Overall then, adding the nonclustered index hasn’t had any startling effects, and the LOB column data still isn’t being read from the table. Let’s see what happens if we make the nonclustered index unique. Test 3: Simple Update with a Unique Index Here’s the script to create a new unique index, and drop the old one: CREATE UNIQUE NONCLUSTERED INDEX [UQ dbo.LOBtest some_value (lob_data)] ON dbo.LOBtest (some_value) INCLUDE ( lob_data ) WITH ( FILLFACTOR = 100, MAXDOP = 1, SORT_IN_TEMPDB = ON ); GO DROP INDEX [IX dbo.LOBtest some_value (lob_data)] ON dbo.LOBtest; Remember that SQL Server only enforces uniqueness on index keys (the some_data column). The lob_data column is simply stored at the leaf-level of the non-clustered index. With that in mind, we might expect this change to make very little difference. Let’s see: UPDATE dbo.LOBtest WITH (TABLOCKX) SET some_value = some_value - 1; Whoa! Now look at the elapsed time and logical reads: Scan count 1, logical reads 2016, physical reads 0, read-ahead reads 0, lob logical reads 36015, lob physical reads 0, lob read-ahead reads 15992. CPU time = 172 ms, elapsed time = 16172 ms. Even with all the data and index pages in memory, the query took over 16 seconds to update just 1,000 rows, performing over 52,000 LOB logical reads (nearly 16,000 of those using read-ahead). Why on earth is SQL Server reading LOB data in a query that only updates a single integer column? The Query Plan The query plan for test 3 looks a bit more complex than before: In fact, the bottom level is exactly the same as we saw with the non-unique index. The top level has heaps of new stuff though, which I’ll come to in a moment. You might be expecting to find that the Clustered Index Scan is now reading the lob_data column (for some reason). After all, we need to explain where all the LOB logical reads are coming from. Sadly, when we look at the properties of the Clustered Index Scan, we see exactly the same as before: SQL Server is still only reading the pk and some_value columns – so what’s doing the LOB reads? Updates that Sneakily Read Data We have to go as far as the Clustered Index Update operator before we see LOB data in the output list: [Expr1020] is a bit flag added by an earlier Compute Scalar. It is set true if the some_value column has not been changed (part of the non-updating updates optimization I mentioned earlier). The Clustered Index Update operator adds two new columns: the lob_data column, and some_value_OLD. The some_value_OLD column, as the name suggests, is the pre-update value of the some_value column. At this point, the clustered index has already been updated with the new value, but we haven’t touched the nonclustered index yet. An interesting observation here is that the Clustered Index Update operator can read a column into the data flow as part of its update operation. SQL Server could have read the LOB data as part of the initial Clustered Index Scan, but that would mean carrying the data through all the operations that occur prior to the Clustered Index Update. The server knows it will have to go back to the clustered index row to update it, so it delays reading the LOB data until then. Sneaky! Why the LOB Data Is Needed This is all very interesting (I hope), but why is SQL Server reading the LOB data? For that matter, why does it need to pass the pre-update value of the some_value column out of the Clustered Index Update? The answer relates to the top row of the query plan for test 3. I’ll reproduce it here for convenience: Notice that this is a wide (per-index) update plan. SQL Server used a narrow (per-row) update plan in test 2, where the Clustered Index Update took care of maintaining the nonclustered index too. I’ll talk more about this difference shortly. The Split/Sort/Collapse combination is an optimization, which aims to make per-index update plans more efficient. It does this by breaking each update into a delete/insert pair, reordering the operations, removing any redundant operations, and finally applying the net effect of all the changes to the nonclustered index. Imagine we had a unique index which currently holds three rows with the values 1, 2, and 3. If we run a query that adds 1 to each row value, we would end up with values 2, 3, and 4. The net effect of all the changes is the same as if we simply deleted the value 1, and added a new value 4. By applying net changes, SQL Server can also avoid false unique-key violations. If we tried to immediately update the value 1 to a 2, it would conflict with the existing value 2 (which would soon be updated to 3 of course) and the query would fail. You might argue that SQL Server could avoid the uniqueness violation by starting with the highest value (3) and working down. That’s fine, but it’s not possible to generalize this logic to work with every possible update query. SQL Server has to use a wide update plan if it sees any risk of false uniqueness violations. It’s worth noting that the logic SQL Server uses to detect whether these violations are possible has definite limits. As a result, you will often receive a wide update plan, even when you can see that no violations are possible. Another benefit of this optimization is that it includes a sort on the index key as part of its work. Processing the index changes in index key order promotes sequential I/O against the nonclustered index. A side-effect of all this is that the net changes might include one or more inserts. In order to insert a new row in the index, SQL Server obviously needs all the columns – the key column and the included LOB column. This is the reason SQL Server reads the LOB data as part of the Clustered Index Update. In addition, the some_value_OLD column is required by the Split operator (it turns updates into delete/insert pairs). In order to generate the correct index key delete operation, it needs the old key value. The irony is that in this case the Split/Sort/Collapse optimization is anything but. Reading all that LOB data is extremely expensive, so it is sad that the current version of SQL Server has no way to avoid it. Finally, for completeness, I should mention that the Filter operator is there to filter out the non-updating updates. Beating the Set-Based Update with a Cursor One situation where SQL Server can see that false unique-key violations aren’t possible is where it can guarantee that only one row is being updated. Armed with this knowledge, we can write a cursor (or the WHILE-loop equivalent) that updates one row at a time, and so avoids reading the LOB data: SET NOCOUNT ON; SET STATISTICS XML, IO, TIME OFF; DECLARE @PK INTEGER, @StartTime DATETIME; SET @StartTime = GETUTCDATE(); DECLARE curUpdate CURSOR LOCAL FORWARD_ONLY KEYSET SCROLL_LOCKS FOR SELECT L.pk FROM LOBtest L ORDER BY L.pk ASC; OPEN curUpdate; WHILE (1 = 1) BEGIN FETCH NEXT FROM curUpdate INTO @PK; IF @@FETCH_STATUS = -1 BREAK; IF @@FETCH_STATUS = -2 CONTINUE; UPDATE dbo.LOBtest SET some_value = some_value - 1 WHERE CURRENT OF curUpdate; END; CLOSE curUpdate; DEALLOCATE curUpdate; SELECT DATEDIFF(MILLISECOND, @StartTime, GETUTCDATE()); That completes the update in 1280 milliseconds (remember test 3 took over 16 seconds!) I used the WHERE CURRENT OF syntax there and a KEYSET cursor, just for the fun of it. One could just as well use a WHERE clause that specified the primary key value instead. Clustered Indexes A clustered index is the ultimate index with included columns: all non-key columns are included columns in a clustered index. Let’s re-create the test table and data with an updatable primary key, and without any non-clustered indexes: IF OBJECT_ID(N'dbo.LOBtest', N'U') IS NOT NULL DROP TABLE dbo.LOBtest; GO CREATE TABLE dbo.LOBtest ( pk INTEGER NOT NULL, some_value INTEGER NULL, lob_data VARCHAR(MAX) NULL, another_column CHAR(5) NULL, CONSTRAINT [PK dbo.LOBtest pk] PRIMARY KEY CLUSTERED (pk ASC) ); GO DECLARE @Data VARCHAR(MAX); SET @Data = REPLICATE(CONVERT(VARCHAR(MAX), 'x'), 65540); WITH Numbers (n) AS ( SELECT ROW_NUMBER() OVER (ORDER BY (SELECT 0)) FROM master.sys.columns C1, master.sys.columns C2 ) INSERT LOBtest WITH (TABLOCKX) ( pk, some_value, lob_data ) SELECT TOP (1000) N.n, N.n, @Data FROM Numbers N WHERE N.n <= 1000; Now here’s a query to modify the cluster keys: UPDATE dbo.LOBtest SET pk = pk + 1; The query plan is: As you can see, the Split/Sort/Collapse optimization is present, and we also gain an Eager Table Spool, for Halloween protection. In addition, SQL Server now has no choice but to read the LOB data in the Clustered Index Scan: The performance is not great, as you might expect (even though there is no non-clustered index to maintain): Table 'LOBtest'. Scan count 1, logical reads 2011, physical reads 0, read-ahead reads 0, lob logical reads 36015, lob physical reads 0, lob read-ahead reads 15992. Table 'Worktable'. Scan count 1, logical reads 2040, physical reads 0, read-ahead reads 0, lob logical reads 34000, lob physical reads 0, lob read-ahead reads 8000. SQL Server Execution Times: CPU time = 483 ms, elapsed time = 17884 ms. Notice how the LOB data is read twice: once from the Clustered Index Scan, and again from the work table in tempdb used by the Eager Spool. If you try the same test with a non-unique clustered index (rather than a primary key), you’ll get a much more efficient plan that just passes the cluster key (including uniqueifier) around (no LOB data or other non-key columns): A unique non-clustered index (on a heap) works well too: Both those queries complete in a few tens of milliseconds, with no LOB reads, and just a few thousand logical reads. (In fact the heap is rather more efficient). There are lots more fun combinations to try that I don’t have space for here. Final Thoughts The behaviour shown in this post is not limited to LOB data by any means. If the conditions are met, any unique index that has included columns can produce similar behaviour – something to bear in mind when adding large INCLUDE columns to achieve covering queries, perhaps. Paul White Email: [email protected] Twitter: @PaulWhiteNZ

Read the article
Transactions in codeigniter with multiple tables.

- by Ethan

Hey SO, I'm new to transactions in general, but especially with CodeIgniter. I'm using InnoDB and everything, but my transactions aren't rolling back when I want them to. Here's my code (slightly simplified). $dog_db = $this->load->database('dog', true); $dog_db->trans_begin(); $dog_id = $this->dogs->insert($new_dog); //Gets primary key of insert if(!$dog_id) { $dog_db->trans_rollback(); throw new Exception('We have had an error trying to add this dog. Please go back and try again.'); } $new_review['dog_id'] = $dog_id; $new_review['user_id'] = $user_id; $new_review['date_added'] = time(); if(!$this->reviews->insert($new_review)) //If the insert fails { $dog_db->trans_rollback(); throw new Exception('We have had an error trying to add this dog. Please go back and try again.'); } //ADD DESCRIPTION $new_description['description'] = $add_dog['description']; $new_description['dog_id'] = $dog_id; $new_description['user_id'] = $user_id; $new_description['date_added'] = time(); if(!$this->descriptions->insert($new_description)) { $dog_db->trans_rollback(); throw new Exception('We have had an error trying to add this dog. Please go back and try again.'); } $booze_db->trans_rollback(); //THIS IS JUST TO SEE IF IT WORKS throw new Exception('We have had an error trying to add this dog. Please go back and try again.'); $booze_db->trans_commit(); } catch(Exception $e) { echo $e->getMessage(); } I'm not getting any error messages, but it's not rolling back either. It should roll back at that final trans_rollback right before the commit. My models are all on the "dog" database, so I think that the transaction would carry into the models' functions. Maybe you just can't use models like this. Any help would be greatly appreciated! Thanks!

Read the article
Fluent NHibernate - How to map a non nullable foreign key that exists in two joined tables

- by vakman

I'm mapping a set of membership classes for my application using Fluent NHibernate. I'm mapping the classes to the asp.net membership database structure. The database schema relevant to the problem looks like this: ASPNET_USERS UserId PK ApplicationId FK NOT NULL other user columns ... ASPNET_MEMBERSHIP UserId PK,FK ApplicationID FK NOT NULL other membership columns... There is a one to one relationship between these two tables. I'm attempting to join the two tables together and map data from both tables to a single 'User' entity which looks like this: public class User { public virtual Guid Id { get; set; } public virtual Guid ApplicationId { get; set; } // other properties to be mapped from aspnetuser/membership tables ... My mapping file is as follows: public class UserMap : ClassMap<User> { public UserMap() { Table("aspnet_Users"); Id(user => user.Id).Column("UserId").GeneratedBy.GuidComb(); Map(user => user.ApplicationId); // other user mappings Join("aspnet_Membership", join => { join.KeyColumn("UserId"); join.Map(user => user.ApplicationId); // Map other things from membership to 'User' class } } } If I try to run with the code above I get a FluentConfiguration exception Tried to add property 'ApplicationId' when already added. If I remove the line "Map(user = user.ApplicationId);" or change it to "Map(user = user.ApplicationId).Not.Update().Not.Insert();" then the application runs but I get the following exception when trying to insert a new user: Cannot insert the value NULL into column 'ApplicationId', table 'ASPNETUsers_Dev.dbo.aspnet_Users'; column does not allow nulls. INSERT fails. The statement has been terminated. And if I leave the .Map(user = user.ApplicationId) as it originally was and make either of those changes to the join.Map(user = user.ApplicationId) then I get the same exception above except of course the exception is related to an insert into the aspnet_Membership table So... how do I do this kind of mapping assuming I can't change my database schema?

Read the article
Unable to connect to Samba printer

- by user127236

I have a headless Ubuntu 12.04 server for files and printers. It shares files via Samba just fine. However, the HP PSC-750xi connected to the server via USB is not accessible from my Ubuntu 12.04 laptop. I can browse for it in the Printing control panel, but any attempt to authenticate my ID to the printer with my user credentials results in the error "This print share is not accessible". I have included the Samba smb.conf file below. Any help appreciated. Thanks... JGB # # Sample configuration file for the Samba suite for Debian GNU/Linux. # # # This is the main Samba configuration file. You should read the # smb.conf(5) manual page in order to understand the options listed # here. Samba has a huge number of configurable options most of which # are not shown in this example # # Some options that are often worth tuning have been included as # commented-out examples in this file. # - When such options are commented with ";", the proposed setting # differs from the default Samba behaviour # - When commented with "#", the proposed setting is the default # behaviour of Samba but the option is considered important # enough to be mentioned here # # NOTE: Whenever you modify this file you should run the command # "testparm" to check that you have not made any basic syntactic # errors. # A well-established practice is to name the original file # "smb.conf.master" and create the "real" config file with # testparm -s smb.conf.master >smb.conf # This minimizes the size of the really used smb.conf file # which, according to the Samba Team, impacts performance # However, use this with caution if your smb.conf file contains nested # "include" statements. See Debian bug #483187 for a case # where using a master file is not a good idea. # #======================= Global Settings ======================= [global] log file = /var/log/samba/log.%m passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* . obey pam restrictions = yes map to guest = bad user encrypt passwords = true passwd program = /usr/bin/passwd %u passdb backend = tdbsam dns proxy = no writeable = yes server string = %h server (Samba, Ubuntu) unix password sync = yes workgroup = WORKGROUP syslog = 0 panic action = /usr/share/samba/panic-action %d usershare allow guests = yes max log size = 1000 pam password change = yes ## Browsing/Identification ### # Change this to the workgroup/NT-domain name your Samba server will part of # server string is the equivalent of the NT Description field # Windows Internet Name Serving Support Section: # WINS Support - Tells the NMBD component of Samba to enable its WINS Server # wins support = no # WINS Server - Tells the NMBD components of Samba to be a WINS Client # Note: Samba can be either a WINS Server, or a WINS Client, but NOT both ; wins server = w.x.y.z # This will prevent nmbd to search for NetBIOS names through DNS. # What naming service and in what order should we use to resolve host names # to IP addresses ; name resolve order = lmhosts host wins bcast #### Networking #### # The specific set of interfaces / networks to bind to # This can be either the interface name or an IP address/netmask; # interface names are normally preferred ; interfaces = 127.0.0.0/8 eth0 # Only bind to the named interfaces and/or networks; you must use the # 'interfaces' option above to use this. # It is recommended that you enable this feature if your Samba machine is # not protected by a firewall or is a firewall itself. However, this # option cannot handle dynamic or non-broadcast interfaces correctly. ; bind interfaces only = yes #### Debugging/Accounting #### # This tells Samba to use a separate log file for each machine # that connects # Cap the size of the individual log files (in KiB). # If you want Samba to only log through syslog then set the following # parameter to 'yes'. # syslog only = no # We want Samba to log a minimum amount of information to syslog. Everything # should go to /var/log/samba/log.{smbd,nmbd} instead. If you want to log # through syslog you should set the following parameter to something higher. # Do something sensible when Samba crashes: mail the admin a backtrace ####### Authentication ####### # "security = user" is always a good idea. This will require a Unix account # in this server for every user accessing the server. See # /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html # in the samba-doc package for details. # security = user # You may wish to use password encryption. See the section on # 'encrypt passwords' in the smb.conf(5) manpage before enabling. # If you are using encrypted passwords, Samba will need to know what # password database type you are using. # This boolean parameter controls whether Samba attempts to sync the Unix # password with the SMB password when the encrypted SMB password in the # passdb is changed. # For Unix password sync to work on a Debian GNU/Linux system, the following # parameters must be set (thanks to Ian Kahan <<[email protected]> for # sending the correct chat script for the passwd program in Debian Sarge). # This boolean controls whether PAM will be used for password changes # when requested by an SMB client instead of the program listed in # 'passwd program'. The default is 'no'. # This option controls how unsuccessful authentication attempts are mapped # to anonymous connections ########## Domains ########### # Is this machine able to authenticate users. Both PDC and BDC # must have this setting enabled. If you are the BDC you must # change the 'domain master' setting to no # ; domain logons = yes # # The following setting only takes effect if 'domain logons' is set # It specifies the location of the user's profile directory # from the client point of view) # The following required a [profiles] share to be setup on the # samba server (see below) ; logon path = \\%N\profiles\%U # Another common choice is storing the profile in the user's home directory # (this is Samba's default) # logon path = \\%N\%U\profile # The following setting only takes effect if 'domain logons' is set # It specifies the location of a user's home directory (from the client # point of view) ; logon drive = H: # logon home = \\%N\%U # The following setting only takes effect if 'domain logons' is set # It specifies the script to run during logon. The script must be stored # in the [netlogon] share # NOTE: Must be store in 'DOS' file format convention ; logon script = logon.cmd # This allows Unix users to be created on the domain controller via the SAMR # RPC pipe. The example command creates a user account with a disabled Unix # password; please adapt to your needs ; add user script = /usr/sbin/adduser --quiet --disabled-password --gecos "" %u # This allows machine accounts to be created on the domain controller via the # SAMR RPC pipe. # The following assumes a "machines" group exists on the system ; add machine script = /usr/sbin/useradd -g machines -c "%u machine account" -d /var/lib/samba -s /bin/false %u # This allows Unix groups to be created on the domain controller via the SAMR # RPC pipe. ; add group script = /usr/sbin/addgroup --force-badname %g ########## Printing ########## # If you want to automatically load your printer list rather # than setting them up individually then you'll need this # load printers = yes # lpr(ng) printing. You may wish to override the location of the # printcap file ; printing = bsd ; printcap name = /etc/printcap # CUPS printing. See also the cupsaddsmb(8) manpage in the # cupsys-client package. ; printing = cups ; printcap name = cups ############ Misc ############ # Using the following line enables you to customise your configuration # on a per machine basis. The %m gets replaced with the netbios name # of the machine that is connecting ; include = /home/samba/etc/smb.conf.%m # Most people will find that this option gives better performance. # See smb.conf(5) and /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/speed.html # for details # You may want to add the following on a Linux system: # SO_RCVBUF=8192 SO_SNDBUF=8192 # socket options = TCP_NODELAY # The following parameter is useful only if you have the linpopup package # installed. The samba maintainer and the linpopup maintainer are # working to ease installation and configuration of linpopup and samba. ; message command = /bin/sh -c '/usr/bin/linpopup "%f" "%m" %s; rm %s' & # Domain Master specifies Samba to be the Domain Master Browser. If this # machine will be configured as a BDC (a secondary logon server), you # must set this to 'no'; otherwise, the default behavior is recommended. # domain master = auto # Some defaults for winbind (make sure you're not using the ranges # for something else.) ; idmap uid = 10000-20000 ; idmap gid = 10000-20000 ; template shell = /bin/bash # The following was the default behaviour in sarge, # but samba upstream reverted the default because it might induce # performance issues in large organizations. # See Debian bug #368251 for some of the consequences of *not* # having this setting and smb.conf(5) for details. ; winbind enum groups = yes ; winbind enum users = yes # Setup usershare options to enable non-root users to share folders # with the net usershare command. # Maximum number of usershare. 0 (default) means that usershare is disabled. ; usershare max shares = 100 # Allow users who've been granted usershare privileges to create # public shares, not just authenticated ones #======================= Share Definitions ======================= # Un-comment the following (and tweak the other settings below to suit) # to enable the default home directory shares. This will share each # user's home director as \\server\username ;[homes] ; comment = Home Directories ; browseable = no # By default, the home directories are exported read-only. Change the # next parameter to 'no' if you want to be able to write to them. ; read only = yes # File creation mask is set to 0700 for security reasons. If you want to # create files with group=rw permissions, set next parameter to 0775. ; create mask = 0700 # Directory creation mask is set to 0700 for security reasons. If you want to # create dirs. with group=rw permissions, set next parameter to 0775. ; directory mask = 0700 # By default, \\server\username shares can be connected to by anyone # with access to the samba server. Un-comment the following parameter # to make sure that only "username" can connect to \\server\username # The following parameter makes sure that only "username" can connect # # This might need tweaking when using external authentication schemes ; valid users = %S # Un-comment the following and create the netlogon directory for Domain Logons # (you need to configure Samba to act as a domain controller too.) ;[netlogon] ; comment = Network Logon Service ; path = /home/samba/netlogon ; guest ok = yes ; read only = yes # Un-comment the following and create the profiles directory to store # users profiles (see the "logon path" option above) # (you need to configure Samba to act as a domain controller too.) # The path below should be writable by all users so that their # profile directory may be created the first time they log on ;[profiles] ; comment = Users profiles ; path = /home/samba/profiles ; guest ok = no ; browseable = no ; create mask = 0600 ; directory mask = 0700 [printers] comment = All Printers browseable = no path = /var/spool/samba printable = yes guest ok = no read only = yes create mask = 0700 # Windows clients look for this share name as a source of downloadable # printer drivers [print$] comment = Printer Drivers browseable = yes writeable = no path = /var/lib/samba/printers # Uncomment to allow remote administration of Windows print drivers. # You may need to replace 'lpadmin' with the name of the group your # admin users are members of. # Please note that you also need to set appropriate Unix permissions # to the drivers directory for these users to have write rights in it ; write list = root, @lpadmin # A sample share for sharing your CD-ROM with others. ;[cdrom] ; comment = Samba server's CD-ROM ; read only = yes ; locking = no ; path = /cdrom ; guest ok = yes # The next two parameters show how to auto-mount a CD-ROM when the # cdrom share is accesed. For this to work /etc/fstab must contain # an entry like this: # # /dev/scd0 /cdrom iso9660 defaults,noauto,ro,user 0 0 # # The CD-ROM gets unmounted automatically after the connection to the # # If you don't want to use auto-mounting/unmounting make sure the CD # is mounted on /cdrom # ; preexec = /bin/mount /cdrom ; postexec = /bin/umount /cdrom [mediafiles] path = /media/multimedia/

Read the article
Autoscaling in a modern world…. Part 3

- by Steve Loethen

The Wasabi Hands on Labs give you a good look at the basic mechanics, but I don’t find the setup too practical. Using a local console application to host the Autoscaler and rules files is probably the (IMHO) least likely architecture. Far more common would be hosting in a service on premise (if you want to have the Autoscaler local) or most likely, host it in a Azure role of it’s own. I chose to go the Azure route. First step was to get the rules.xml and the services.xml files into the cloud. I tend to be a “one step at a time” sort of guy, so running the console application with the rules sitting in a Azure hosted set of blobs seemed to be the logical first step. Here are the steps: 1) Create a container in the storage account you wish to use. Name does not matter, you will get a chance to set the container name (as well as the file names) in the app.config 2) Copy the two files from where you created them to your container. I used the same files I had locally. I made the container public to eliminate security issues, but in the final application, a bit of security needs to be applied (one problem at a time). The content type was set to text/xml. I found one reference claiming the importance of this step, and it makes sense. 3) Adjust the app.config to set the location of the files. This will let you set all the storage account and key information needed to reach into the cloud form your console application. The sections of your app.config will look like this: <rulesStores> <add name="Blob Rules Store" type="Microsoft.Practices.EnterpriseLibrary.WindowsAzure.Autoscaling.Rules.Configuration.BlobXmlFileRulesStore, Microsoft.Practices.EnterpriseLibrary.WindowsAzure.Autoscaling, Version=5.0.1118.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" blobContainerName="[ContainerName]" blobName="rules.xml" storageAccount="DefaultEndpointsProtocol=https;AccountName=[StorageAccount];AccountKey=[AccountKey]" monitoringRate="00:00:30" certificateThumbprint="" certificateStoreLocation="LocalMachine" checkCertificateValidity="false" /> </rulesStores> <serviceInformationStores> <add name="Blob Service Information Store" type="Microsoft.Practices.EnterpriseLibrary.WindowsAzure.Autoscaling.ServiceModel.Configuration.BlobXmlFileServiceInformationStore, Microsoft.Practices.EnterpriseLibrary.WindowsAzure.Autoscaling, Version=5.0.1118.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" blobContainerName="[ContainerName]" blobName="services.xml" storageAccount="DefaultEndpointsProtocol=https;AccountName=[StorageAccount];AccountKey=[AccountKey]" monitoringRate="00:00:30" certificateThumbprint="" certificateStoreLocation="LocalMachine" checkCertificateValidity="false" /> </serviceInformationStores> Once I had the files up in the sky, I renamed the local copies to just to make my self feel better about the application using the correct set of rules and services. Deploy the web role to the cloud. Once it is up and running, start the console application. You should find the application scales up and down in response to the buttons on the web site. Tune in next time for moving the hosting of the Autoscaler to a worker role, discussions on getting the logging information into diagnostics into storage, and a set of discussions about certs and how they play a role.

Read the article
Understanding G1 GC Logs

- by poonam

The purpose of this post is to explain the meaning of GC logs generated with some tracing and diagnostic options for G1 GC. We will take a look at the output generated with PrintGCDetails which is a product flag and provides the most detailed level of information. Along with that, we will also look at the output of two diagnostic flags that get enabled with -XX:+UnlockDiagnosticVMOptions option - G1PrintRegionLivenessInfo that prints the occupancy and the amount of space used by live objects in each region at the end of the marking cycle and G1PrintHeapRegions that provides detailed information on the heap regions being allocated and reclaimed. We will be looking at the logs generated with JDK 1.7.0_04 using these options. Option -XX:+PrintGCDetails Here's a sample log of G1 collection generated with PrintGCDetails. 0.522: [GC pause (young), 0.15877971 secs] [Parallel Time: 157.1 ms] [GC Worker Start (ms): 522.1 522.2 522.2 522.2 Avg: 522.2, Min: 522.1, Max: 522.2, Diff: 0.1] [Ext Root Scanning (ms): 1.6 1.5 1.6 1.9 Avg: 1.7, Min: 1.5, Max: 1.9, Diff: 0.4] [Update RS (ms): 38.7 38.8 50.6 37.3 Avg: 41.3, Min: 37.3, Max: 50.6, Diff: 13.3] [Processed Buffers : 2 2 3 2 Sum: 9, Avg: 2, Min: 2, Max: 3, Diff: 1] [Scan RS (ms): 9.9 9.7 0.0 9.7 Avg: 7.3, Min: 0.0, Max: 9.9, Diff: 9.9] [Object Copy (ms): 106.7 106.8 104.6 107.9 Avg: 106.5, Min: 104.6, Max: 107.9, Diff: 3.3] [Termination (ms): 0.0 0.0 0.0 0.0 Avg: 0.0, Min: 0.0, Max: 0.0, Diff: 0.0] [Termination Attempts : 1 4 4 6 Sum: 15, Avg: 3, Min: 1, Max: 6, Diff: 5] [GC Worker End (ms): 679.1 679.1 679.1 679.1 Avg: 679.1, Min: 679.1, Max: 679.1, Diff: 0.1] [GC Worker (ms): 156.9 157.0 156.9 156.9 Avg: 156.9, Min: 156.9, Max: 157.0, Diff: 0.1] [GC Worker Other (ms): 0.3 0.3 0.3 0.3 Avg: 0.3, Min: 0.3, Max: 0.3, Diff: 0.0] [Clear CT: 0.1 ms] [Other: 1.5 ms] [Choose CSet: 0.0 ms] [Ref Proc: 0.3 ms] [Ref Enq: 0.0 ms] [Free CSet: 0.3 ms] [Eden: 12M(12M)->0B(10M) Survivors: 0B->2048K Heap: 13M(64M)->9739K(64M)] [Times: user=0.59 sys=0.02, real=0.16 secs] This is the typical log of an Evacuation Pause (G1 collection) in which live objects are copied from one set of regions (young OR young+old) to another set. It is a stop-the-world activity and all the application threads are stopped at a safepoint during this time. This pause is made up of several sub-tasks indicated by the indentation in the log entries. Here's is the top most line that gets printed for the Evacuation Pause. 0.522: [GC pause (young), 0.15877971 secs] This is the highest level information telling us that it is an Evacuation Pause that started at 0.522 secs from the start of the process, in which all the regions being evacuated are Young i.e. Eden and Survivor regions. This collection took 0.15877971 secs to finish. Evacuation Pauses can be mixed as well. In which case the set of regions selected include all of the young regions as well as some old regions. 1.730: [GC pause (mixed), 0.32714353 secs] Let's take a look at all the sub-tasks performed in this Evacuation Pause. [Parallel Time: 157.1 ms] Parallel Time is the total elapsed time spent by all the parallel GC worker threads. The following lines correspond to the parallel tasks performed by these worker threads in this total parallel time, which in this case is 157.1 ms. [GC Worker Start (ms): 522.1 522.2 522.2 522.2Avg: 522.2, Min: 522.1, Max: 522.2, Diff: 0.1] The first line tells us the start time of each of the worker thread in milliseconds. The start times are ordered with respect to the worker thread ids – thread 0 started at 522.1ms and thread 1 started at 522.2ms from the start of the process. The second line tells the Avg, Min, Max and Diff of the start times of all of the worker threads. [Ext Root Scanning (ms): 1.6 1.5 1.6 1.9 Avg: 1.7, Min: 1.5, Max: 1.9, Diff: 0.4] This gives us the time spent by each worker thread scanning the roots (globals, registers, thread stacks and VM data structures). Here, thread 0 took 1.6ms to perform the root scanning task and thread 1 took 1.5 ms. The second line clearly shows the Avg, Min, Max and Diff of the times spent by all the worker threads. [Update RS (ms): 38.7 38.8 50.6 37.3 Avg: 41.3, Min: 37.3, Max: 50.6, Diff: 13.3] Update RS gives us the time each thread spent in updating the Remembered Sets. Remembered Sets are the data structures that keep track of the references that point into a heap region. Mutator threads keep changing the object graph and thus the references that point into a particular region. We keep track of these changes in buffers called Update Buffers. The Update RS sub-task processes the update buffers that were not able to be processed concurrently, and updates the corresponding remembered sets of all regions. [Processed Buffers : 2 2 3 2Sum: 9, Avg: 2, Min: 2, Max: 3, Diff: 1] This tells us the number of Update Buffers (mentioned above) processed by each worker thread. [Scan RS (ms): 9.9 9.7 0.0 9.7 Avg: 7.3, Min: 0.0, Max: 9.9, Diff: 9.9] These are the times each worker thread had spent in scanning the Remembered Sets. Remembered Set of a region contains cards that correspond to the references pointing into that region. This phase scans those cards looking for the references pointing into all the regions of the collection set. [Object Copy (ms): 106.7 106.8 104.6 107.9 Avg: 106.5, Min: 104.6, Max: 107.9, Diff: 3.3] These are the times spent by each worker thread copying live objects from the regions in the Collection Set to the other regions. [Termination (ms): 0.0 0.0 0.0 0.0 Avg: 0.0, Min: 0.0, Max: 0.0, Diff: 0.0] Termination time is the time spent by the worker thread offering to terminate. But before terminating, it checks the work queues of other threads and if there are still object references in other work queues, it tries to steal object references, and if it succeeds in stealing a reference, it processes that and offers to terminate again. [Termination Attempts : 1 4 4 6 Sum: 15, Avg: 3, Min: 1, Max: 6, Diff: 5] This gives the number of times each thread has offered to terminate. [GC Worker End (ms): 679.1 679.1 679.1 679.1 Avg: 679.1, Min: 679.1, Max: 679.1, Diff: 0.1] These are the times in milliseconds at which each worker thread stopped. [GC Worker (ms): 156.9 157.0 156.9 156.9 Avg: 156.9, Min: 156.9, Max: 157.0, Diff: 0.1] These are the total lifetimes of each worker thread. [GC Worker Other (ms): 0.3 0.3 0.3 0.3Avg: 0.3, Min: 0.3, Max: 0.3, Diff: 0.0] These are the times that each worker thread spent in performing some other tasks that we have not accounted above for the total Parallel Time. [Clear CT: 0.1 ms] This is the time spent in clearing the Card Table. This task is performed in serial mode. [Other: 1.5 ms] Time spent in the some other tasks listed below. The following sub-tasks (which individually may be parallelized) are performed serially. [Choose CSet: 0.0 ms] Time spent in selecting the regions for the Collection Set. [Ref Proc: 0.3 ms] Total time spent in processing Reference objects. [Ref Enq: 0.0 ms] Time spent in enqueuing references to the ReferenceQueues. [Free CSet: 0.3 ms] Time spent in freeing the collection set data structure. [Eden: 12M(12M)->0B(13M) Survivors: 0B->2048K Heap: 14M(64M)->9739K(64M)] This line gives the details on the heap size changes with the Evacuation Pause. This shows that Eden had the occupancy of 12M and its capacity was also 12M before the collection. After the collection, its occupancy got reduced to 0 since everything is evacuated/promoted from Eden during a collection, and its target size grew to 13M. The new Eden capacity of 13M is not reserved at this point. This value is the target size of the Eden. Regions are added to Eden as the demand is made and when the added regions reach to the target size, we start the next collection. Similarly, Survivors had the occupancy of 0 bytes and it grew to 2048K after the collection. The total heap occupancy and capacity was 14M and 64M receptively before the collection and it became 9739K and 64M after the collection. Apart from the evacuation pauses, G1 also performs concurrent-marking to build the live data information of regions. 1.416: [GC pause (young) (initial-mark), 0.62417980 secs] ….... 2.042: [GC concurrent-root-region-scan-start] 2.067: [GC concurrent-root-region-scan-end, 0.0251507] 2.068: [GC concurrent-mark-start] 3.198: [GC concurrent-mark-reset-for-overflow] 4.053: [GC concurrent-mark-end, 1.9849672 sec] 4.055: [GC remark 4.055: [GC ref-proc, 0.0000254 secs], 0.0030184 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 4.088: [GC cleanup 117M->106M(138M), 0.0015198 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 4.090: [GC concurrent-cleanup-start] 4.091: [GC concurrent-cleanup-end, 0.0002721] The first phase of a marking cycle is Initial Marking where all the objects directly reachable from the roots are marked and this phase is piggy-backed on a fully young Evacuation Pause. 2.042: [GC concurrent-root-region-scan-start] This marks the start of a concurrent phase that scans the set of root-regions which are directly reachable from the survivors of the initial marking phase. 2.067: [GC concurrent-root-region-scan-end, 0.0251507] End of the concurrent root region scan phase and it lasted for 0.0251507 seconds. 2.068: [GC concurrent-mark-start] Start of the concurrent marking at 2.068 secs from the start of the process. 3.198: [GC concurrent-mark-reset-for-overflow] This indicates that the global marking stack had became full and there was an overflow of the stack. Concurrent marking detected this overflow and had to reset the data structures to start the marking again. 4.053: [GC concurrent-mark-end, 1.9849672 sec] End of the concurrent marking phase and it lasted for 1.9849672 seconds. 4.055: [GC remark 4.055: [GC ref-proc, 0.0000254 secs], 0.0030184 secs] This corresponds to the remark phase which is a stop-the-world phase. It completes the left over marking work (SATB buffers processing) from the previous phase. In this case, this phase took 0.0030184 secs and out of which 0.0000254 secs were spent on Reference processing. 4.088: [GC cleanup 117M->106M(138M), 0.0015198 secs] Cleanup phase which is again a stop-the-world phase. It goes through the marking information of all the regions, computes the live data information of each region, resets the marking data structures and sorts the regions according to their gc-efficiency. In this example, the total heap size is 138M and after the live data counting it was found that the total live data size dropped down from 117M to 106M. 4.090: [GC concurrent-cleanup-start] This concurrent cleanup phase frees up the regions that were found to be empty (didn't contain any live data) during the previous stop-the-world phase. 4.091: [GC concurrent-cleanup-end, 0.0002721] Concurrent cleanup phase took 0.0002721 secs to free up the empty regions. Option -XX:G1PrintRegionLivenessInfo Now, let's look at the output generated with the flag G1PrintRegionLivenessInfo. This is a diagnostic option and gets enabled with -XX:+UnlockDiagnosticVMOptions. G1PrintRegionLivenessInfo prints the live data information of each region during the Cleanup phase of the concurrent-marking cycle. 26.896: [GC cleanup ### PHASE Post-Marking @ 26.896### HEAP committed: 0x02e00000-0x0fe00000 reserved: 0x02e00000-0x12e00000 region-size: 1048576 Cleanup phase of the concurrent-marking cycle started at 26.896 secs from the start of the process and this live data information is being printed after the marking phase. Committed G1 heap ranges from 0x02e00000 to 0x0fe00000 and the total G1 heap reserved by JVM is from 0x02e00000 to 0x12e00000. Each region in the G1 heap is of size 1048576 bytes. ### type address-range used prev-live next-live gc-eff### (bytes) (bytes) (bytes) (bytes/ms) This is the header of the output that tells us about the type of the region, address-range of the region, used space in the region, live bytes in the region with respect to the previous marking cycle, live bytes in the region with respect to the current marking cycle and the GC efficiency of that region. ### FREE 0x02e00000-0x02f00000 0 0 0 0.0 This is a Free region. ### OLD 0x02f00000-0x03000000 1048576 1038592 1038592 0.0 Old region with address-range from 0x02f00000 to 0x03000000. Total used space in the region is 1048576 bytes, live bytes as per the previous marking cycle are 1038592 and live bytes with respect to the current marking cycle are also 1038592. The GC efficiency has been computed as 0. ### EDEN 0x03400000-0x03500000 20992 20992 20992 0.0 This is an Eden region. ### HUMS 0x0ae00000-0x0af00000 1048576 1048576 1048576 0.0### HUMC 0x0af00000-0x0b000000 1048576 1048576 1048576 0.0### HUMC 0x0b000000-0x0b100000 1048576 1048576 1048576 0.0### HUMC 0x0b100000-0x0b200000 1048576 1048576 1048576 0.0### HUMC 0x0b200000-0x0b300000 1048576 1048576 1048576 0.0### HUMC 0x0b300000-0x0b400000 1048576 1048576 1048576 0.0### HUMC 0x0b400000-0x0b500000 1001480 1001480 1001480 0.0 These are the continuous set of regions called Humongous regions for storing a large object. HUMS (Humongous starts) marks the start of the set of humongous regions and HUMC (Humongous continues) tags the subsequent regions of the humongous regions set. ### SURV 0x09300000-0x09400000 16384 16384 16384 0.0 This is a Survivor region. ### SUMMARY capacity: 208.00 MB used: 150.16 MB / 72.19 % prev-live: 149.78 MB / 72.01 % next-live: 142.82 MB / 68.66 % At the end, a summary is printed listing the capacity, the used space and the change in the liveness after the completion of concurrent marking. In this case, G1 heap capacity is 208MB, total used space is 150.16MB which is 72.19% of the total heap size, live data in the previous marking was 149.78MB which was 72.01% of the total heap size and the live data as per the current marking is 142.82MB which is 68.66% of the total heap size. Option -XX:+G1PrintHeapRegions G1PrintHeapRegions option logs the regions related events when regions are committed, allocated into or are reclaimed. COMMIT/UNCOMMIT events G1HR COMMIT [0x6e900000,0x6ea00000]G1HR COMMIT [0x6ea00000,0x6eb00000] Here, the heap is being initialized or expanded and the region (with bottom: 0x6eb00000 and end: 0x6ec00000) is being freshly committed. COMMIT events are always generated in order i.e. the next COMMIT event will always be for the uncommitted region with the lowest address. G1HR UNCOMMIT [0x72700000,0x72800000]G1HR UNCOMMIT [0x72600000,0x72700000] Opposite to COMMIT. The heap got shrunk at the end of a Full GC and the regions are being uncommitted. Like COMMIT, UNCOMMIT events are also generated in order i.e. the next UNCOMMIT event will always be for the committed region with the highest address. GC Cycle events G1HR #StartGC 7G1HR CSET 0x6e900000G1HR REUSE 0x70500000G1HR ALLOC(Old) 0x6f800000G1HR RETIRE 0x6f800000 0x6f821b20G1HR #EndGC 7 This shows start and end of an Evacuation pause. This event is followed by a GC counter tracking both evacuation pauses and Full GCs. Here, this is the 7th GC since the start of the process. G1HR #StartFullGC 17G1HR UNCOMMIT [0x6ed00000,0x6ee00000]G1HR POST-COMPACTION(Old) 0x6e800000 0x6e854f58G1HR #EndFullGC 17 Shows start and end of a Full GC. This event is also followed by the same GC counter as above. This is the 17th GC since the start of the process. ALLOC events G1HR ALLOC(Eden) 0x6e800000 The region with bottom 0x6e800000 just started being used for allocation. In this case it is an Eden region and allocated into by a mutator thread. G1HR ALLOC(StartsH) 0x6ec00000 0x6ed00000G1HR ALLOC(ContinuesH) 0x6ed00000 0x6e000000 Regions being used for the allocation of Humongous object. The object spans over two regions. G1HR ALLOC(SingleH) 0x6f900000 0x6f9eb010 Single region being used for the allocation of Humongous object. G1HR COMMIT [0x6ee00000,0x6ef00000]G1HR COMMIT [0x6ef00000,0x6f000000]G1HR COMMIT [0x6f000000,0x6f100000]G1HR COMMIT [0x6f100000,0x6f200000]G1HR ALLOC(StartsH) 0x6ee00000 0x6ef00000G1HR ALLOC(ContinuesH) 0x6ef00000 0x6f000000G1HR ALLOC(ContinuesH) 0x6f000000 0x6f100000G1HR ALLOC(ContinuesH) 0x6f100000 0x6f102010 Here, Humongous object allocation request could not be satisfied by the free committed regions that existed in the heap, so the heap needed to be expanded. Thus new regions are committed and then allocated into for the Humongous object. G1HR ALLOC(Old) 0x6f800000 Old region started being used for allocation during GC. G1HR ALLOC(Survivor) 0x6fa00000 Region being used for copying old objects into during a GC. Note that Eden and Humongous ALLOC events are generated outside the GC boundaries and Old and Survivor ALLOC events are generated inside the GC boundaries. Other Events G1HR RETIRE 0x6e800000 0x6e87bd98 Retire and stop using the region having bottom 0x6e800000 and top 0x6e87bd98 for allocation. Note that most regions are full when they are retired and we omit those events to reduce the output volume. A region is retired when another region of the same type is allocated or we reach the start or end of a GC(depending on the region). So for Eden regions: For example: 1. ALLOC(Eden) Foo2. ALLOC(Eden) Bar3. StartGC At point 2, Foo has just been retired and it was full. At point 3, Bar was retired and it was full. If they were not full when they were retired, we will have a RETIRE event: 1. ALLOC(Eden) Foo2. RETIRE Foo top3. ALLOC(Eden) Bar4. StartGC G1HR CSET 0x6e900000 Region (bottom: 0x6e900000) is selected for the Collection Set. The region might have been selected for the collection set earlier (i.e. when it was allocated). However, we generate the CSET events for all regions in the CSet at the start of a GC to make sure there's no confusion about which regions are part of the CSet. G1HR POST-COMPACTION(Old) 0x6e800000 0x6e839858 POST-COMPACTION event is generated for each non-empty region in the heap after a full compaction. A full compaction moves objects around, so we don't know what the resulting shape of the heap is (which regions were written to, which were emptied, etc.). To deal with this, we generate a POST-COMPACTION event for each non-empty region with its type (old/humongous) and the heap boundaries. At this point we should only have Old and Humongous regions, as we have collapsed the young generation, so we should not have eden and survivors. POST-COMPACTION events are generated within the Full GC boundary. G1HR CLEANUP 0x6f400000G1HR CLEANUP 0x6f300000G1HR CLEANUP 0x6f200000 These regions were found empty after remark phase of Concurrent Marking and are reclaimed shortly afterwards. G1HR #StartGC 5G1HR CSET 0x6f400000G1HR CSET 0x6e900000G1HR REUSE 0x6f800000 At the end of a GC we retire the old region we are allocating into. Given that its not full, we will carry on allocating into it during the next GC. This is what REUSE means. In the above case 0x6f800000 should have been the last region with an ALLOC(Old) event during the previous GC and should have been retired before the end of the previous GC. G1HR ALLOC-FORCE(Eden) 0x6f800000 A specialization of ALLOC which indicates that we have reached the max desired number of the particular region type (in this case: Eden), but we decided to allocate one more. Currently it's only used for Eden regions when we extend the young generation because we cannot do a GC as the GC-Locker is active. G1HR EVAC-FAILURE 0x6f800000 During a GC, we have failed to evacuate an object from the given region as the heap is full and there is no space left to copy the object. This event is generated within GC boundaries and exactly once for each region from which we failed to evacuate objects. When Heap Regions are reclaimed ? It is also worth mentioning when the heap regions in the G1 heap are reclaimed. All regions that are in the CSet (the ones that appear in CSET events) are reclaimed at the end of a GC. The exception to that are regions with EVAC-FAILURE events. All regions with CLEANUP events are reclaimed. After a Full GC some regions get reclaimed (the ones from which we moved the objects out). But that is not shown explicitly, instead the non-empty regions that are left in the heap are printed out with the POST-COMPACTION events.

Read the article
Compass - Lucene Full text search. Structure and Best Practice.

- by Rob

Hi, I have played about with the tutorial and Compass itself for a bit now. I have just started to ramp up the use of it and have found that the performance slows drastically. I am certain that this is due to my mappings and the relationships that I have between entities and was looking for suggestions about how this should be best done. Also as a side question I wanted to know if a variable is in an @searchableComponent but is not defined as @searchable when the component object is pulled out of Compass results will you be able to access that variable? I have 3 main classes that I want to search on - Provider, Location and Activity. They are all inter-related - a Provider can have many locations and activites and has an address; A Location has 1 provider, many activities and an address; An activity has 1 provider and many locations. I have a join table between activity and Location called ActivityLocation that can later provider additional information about the relationship. My classes are mapped to compass as shown below for provider location activity and address. This works but gives a huge index and searches on it are comparatively slow, any advice would be great. Cheers, Rob @Searchable public class AbstractActivity extends LightEntity implements Serializable { /** * Serialisation ID */ private static final long serialVersionUID = 3445339493203407152L; @SearchableId (name="actID") private Integer activityId =-1; @SearchableComponent() private Provider provider; @SearchableComponent(prefix = "activity") private Category category; private String status; @SearchableProperty (name = "activityName") @SearchableMetaData (name = "activityshortactivityName") private String activityName; @SearchableProperty (name = "shortDescription") @SearchableMetaData (name = "activityshortDescription") private String shortDescription; @SearchableProperty (name = "abRating") @SearchableMetaData (name = "activityabRating") private Integer abRating; private String contactName; private String phoneNumber; private String faxNumber; private String email; private String url; @SearchableProperty (name = "completed") @SearchableMetaData (name = "activitycompleted") private Boolean completed= false; @SearchableProperty (name = "isprivate") @SearchableMetaData (name = "activityisprivate") private Boolean isprivate= false; private Boolean subs= false; private Boolean newsfeed= true; private Set news = new HashSet(0); private Set ActivitySession = new HashSet(0); private Set payments = new HashSet(0); private Set userclub = new HashSet(0); private Set ActivityOpeningTimes = new HashSet(0); private Set Events = new HashSet(0); private User creator; private Set userInterested = new HashSet(0); boolean freeEdit = false; private Integer activityType =0; @SearchableComponent (maxDepth=2) private Set activityLocations = new HashSet(0); private Double userRating = -1.00; Getters and Setters .... @Searchable public class AbstractLocation extends LightEntity implements Serializable { /** * Serialisation ID */ private static final long serialVersionUID = 3445339493203407152L; @SearchableId (name="locationID") private Integer locationId; @SearchableComponent (prefix = "location") private Category category; @SearchableComponent (maxDepth=1) private Provider provider; @SearchableProperty (name = "status") @SearchableMetaData (name = "locationstatus") private String status; @SearchableProperty private String locationName; @SearchableProperty (name = "shortDescription") @SearchableMetaData (name = "locationshortDescription") private String shortDescription; @SearchableProperty (name = "abRating") @SearchableMetaData (name = "locationabRating") private Integer abRating; private Integer boolUseProviderDetails; @SearchableProperty (name = "contactName") @SearchableMetaData (name = "locationcontactName") private String contactName; @SearchableComponent private Address address; @SearchableProperty (name = "phoneNumber") @SearchableMetaData (name = "locationphoneNumber") private String phoneNumber; @SearchableProperty (name = "faxNumber") @SearchableMetaData (name = "locationfaxNumber") private String faxNumber; @SearchableProperty (name = "email") @SearchableMetaData (name = "locationemail") private String email; @SearchableProperty (name = "url") @SearchableMetaData (name = "locationurl") private String url; @SearchableProperty (name = "completed") @SearchableMetaData (name = "locationcompleted") private Boolean completed= false; @SearchableProperty (name = "isprivate") @SearchableMetaData (name = "locationisprivate") private Boolean isprivate= false; @SearchableComponent private Set activityLocations = new HashSet(0); private Set LocationOpeningTimes = new HashSet(0); private Set events = new HashSet(0); @SearchableProperty (name = "adult_cost") @SearchableMetaData (name = "locationadult_cost") private String adult_cost =""; @SearchableProperty (name = "child_cost") @SearchableMetaData (name = "locationchild_cost") private String child_cost =""; @SearchableProperty (name = "family_cost") @SearchableMetaData (name = "locationfamily_cost") private String family_cost =""; private Double userRating = -1.00; private Set costs = new HashSet(0); private String cost_caveats =""; Getters and Setters .... @Searchable public class AbstractActivitylocations implements java.io.Serializable { /** * */ private static final long serialVersionUID = 1365110541466870626L; @SearchableId (name="id") private Integer id; @SearchableComponent private Activity activity; @SearchableComponent private Location location; Getters and Setters..... @Searchable public class AbstractProvider extends LightEntity implements Serializable { private static final long serialVersionUID = 3060354043429663058L; @SearchableId private Integer providerId = -1; @SearchableComponent (prefix = "provider") private Category category; @SearchableProperty (name = "businessName") @SearchableMetaData (name = "providerbusinessName") private String businessName; @SearchableProperty (name = "contactName") @SearchableMetaData (name = "providercontactName") private String contactName; @SearchableComponent private Address address; @SearchableProperty (name = "phoneNumber") @SearchableMetaData (name = "providerphoneNumber") private String phoneNumber; @SearchableProperty (name = "faxNumber") @SearchableMetaData (name = "providerfaxNumber") private String faxNumber; @SearchableProperty (name = "email") @SearchableMetaData (name = "provideremail") private String email; @SearchableProperty (name = "url") @SearchableMetaData (name = "providerurl") private String url; @SearchableProperty (name = "status") @SearchableMetaData (name = "providerstatus") private String status; @SearchableProperty (name = "notes") @SearchableMetaData (name = "providernotes") private String notes; @SearchableProperty (name = "shortDescription") @SearchableMetaData (name = "providershortDescription") private String shortDescription; private Boolean completed = false; private Boolean isprivate = false; private Double userRating = -1.00; private Integer ABRating = 1; @SearchableComponent private Set locations = new HashSet(0); @SearchableComponent private Set activities = new HashSet(0); private Set ProviderOpeningTimes = new HashSet(0); private User creator; boolean freeEdit = false; Getters and Setters... Thanks for reading!! Rob

Read the article
LINQ InsertOnSubmit Required Fields needed for debugging

- by Derek Hunziker

Hi All, I've been using the ADO.NET Strogly-Typed DataSet model for about 2 years now for handling CRUD and stored procedure executions. This past year I built my first MVC app and I really enjoyed the ease and flexibility of LINQ. Perhaps the biggest selling point for me was that with LINQ I didn't have to create "Insert" stored procedures that would return the SCOPE_IDENTITY anymore (The auto-generated insert statements in the DataSet model were not capable of this without modification). Currently, I'm using LINQ with ASP.NET 3.5 WebForms. My inserts are looking like this: ProductsDataContext dc = new ProductsDataContext(); product p = new product { Title = "New Product", Price = 59.99, Archived = false }; dc.products.InsertOnSubmit(p); dc.SubmitChanges(); int productId = p.Id; So, this product example is pretty basic, right, and in the future, I'll probably be adding more fields to the database such as "InStock", "Quantity", etc... The way I understand it, I will need to add those fields to the database table and then delete and re-add the tables to the LINQ to SQL Class design view in order to refresh the DataContext. Does that sound right? The problem is that any new fields that are non-null are NOT caught by the ASP.NET build processes. For example, if I added a non-null field of "Quantity" to the database, the code above would still build. In the DataSet model, the stored procedure method would accept a certain amount of parameters and would warn me that my Insert would fail if I didn't include a quantity value. The same goes for LINQ stored procedure methods, however, to my knowledge, LINQ doesn't offer a way to auto generate the insert statements and that means I'm back to where I started. The bottom line is if I used insert statements like the one above and I add a non-null field to my database, it would break my app in about 10-20 places and there would be no way for me to detect it. Is my only option to do a solution-side search for the keyword "products.InsertOnSubmit" and make sure the new field is getting assigned? Is there a better way? Thanks!

Read the article
EF4 generates invalid script

- by Jaxidian

When I right-click in a .EDMX file and click Generate Database From Model, the resulting script is obviously wrong because of the table names. What it generates is the following script. Note the table names in the DROP TABLE part versus the CREATE TABLE part. Why is this inconsistent? This is obviously not a reusable script. What I created was an Entity named "Address" and an Entity named "Company", etc (all singular). The EntitySet names are pluralized. The "Pluralize New Objects" boolean does not change this either. So what's the deal? For what it's worth, I originally generated the EDMX by pointing it to a database that had tables with non-pluralized names and now that I've made some changes, I want to go back the other way. I'd like to have the option to go back and forth as neither the db-first nor the model-first model is ideal in all scenarios, and I have the control to ensure that there will be no merging issues from multiple people going both ways at the same time. -- -------------------------------------------------- -- Dropping existing FOREIGN KEY constraints -- NOTE: if the constraint does not exist, an ignorable error will be reported. -- -------------------------------------------------- ALTER TABLE [Address] DROP CONSTRAINT [FK_Address_StateID-State_ID]; GO ALTER TABLE [Company] DROP CONSTRAINT [FK_Company_AddressID-Address_ID]; GO ALTER TABLE [Employee] DROP CONSTRAINT [FK_Employee_BossEmployeeID-Employee_ID]; GO ALTER TABLE [Employee] DROP CONSTRAINT [FK_Employee_CompanyID-Company_ID]; GO ALTER TABLE [Employee] DROP CONSTRAINT [FK_Employee_PersonID-Person_ID]; GO ALTER TABLE [Person] DROP CONSTRAINT [FK_Person_AddressID-Address_ID]; GO -- -------------------------------------------------- -- Dropping existing tables -- NOTE: if the table does not exist, an ignorable error will be reported. -- -------------------------------------------------- DROP TABLE [Address]; GO DROP TABLE [Company]; GO DROP TABLE [Employee]; GO DROP TABLE [Person]; GO DROP TABLE [State]; GO -- -------------------------------------------------- -- Creating all tables -- -------------------------------------------------- -- Creating table 'Addresses' CREATE TABLE [Addresses] ( [ID] int IDENTITY(1,1) NOT NULL, [StreetAddress] nvarchar(100) NOT NULL, [City] nvarchar(100) NOT NULL, [StateID] int NOT NULL, [Zip] nvarchar(10) NOT NULL ); GO -- Creating table 'Companies' CREATE TABLE [Companies] ( [ID] int IDENTITY(1,1) NOT NULL, [Name] nvarchar(100) NOT NULL, [AddressID] int NOT NULL ); GO -- Creating table 'People' CREATE TABLE [People] ( [ID] int IDENTITY(1,1) NOT NULL, [FirstName] nvarchar(100) NOT NULL, [LastName] nvarchar(100) NOT NULL, [AddressID] int NOT NULL ); GO -- Creating table 'States' CREATE TABLE [States] ( [ID] int IDENTITY(1,1) NOT NULL, [Name] nvarchar(100) NOT NULL, [Abbreviation] nvarchar(2) NOT NULL ); GO -- Creating table 'Employees' CREATE TABLE [Employees] ( [ID] int IDENTITY(1,1) NOT NULL, [PersonID] int NOT NULL, [CompanyID] int NOT NULL, [Position] nvarchar(100) NOT NULL, [BossEmployeeID] int NULL ); GO -- -------------------------------------------------- -- Creating all PRIMARY KEY constraints -- -------------------------------------------------- -- Creating primary key on [ID] in table 'Addresses' ALTER TABLE [Addresses] ADD CONSTRAINT [PK_Addresses] PRIMARY KEY ([ID] ); GO -- Creating primary key on [ID] in table 'Companies' ALTER TABLE [Companies] ADD CONSTRAINT [PK_Companies] PRIMARY KEY ([ID] ); GO -- Creating primary key on [ID] in table 'People' ALTER TABLE [People] ADD CONSTRAINT [PK_People] PRIMARY KEY ([ID] ); GO -- Creating primary key on [ID] in table 'States' ALTER TABLE [States] ADD CONSTRAINT [PK_States] PRIMARY KEY ([ID] ); GO -- Creating primary key on [ID] in table 'Employees' ALTER TABLE [Employees] ADD CONSTRAINT [PK_Employees] PRIMARY KEY ([ID] ); GO -- -------------------------------------------------- -- Creating all FOREIGN KEY constraints -- -------------------------------------------------- -- Creating foreign key on [StateID] in table 'Addresses' ALTER TABLE [Addresses] ADD CONSTRAINT [FK_Address_StateID_State_ID] FOREIGN KEY ([StateID]) REFERENCES [States] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Address_StateID_State_ID' CREATE INDEX [IX_FK_Address_StateID_State_ID] ON [Addresses] ([StateID]); GO -- Creating foreign key on [AddressID] in table 'Companies' ALTER TABLE [Companies] ADD CONSTRAINT [FK_Company_AddressID_Address_ID] FOREIGN KEY ([AddressID]) REFERENCES [Addresses] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Company_AddressID_Address_ID' CREATE INDEX [IX_FK_Company_AddressID_Address_ID] ON [Companies] ([AddressID]); GO -- Creating foreign key on [AddressID] in table 'People' ALTER TABLE [People] ADD CONSTRAINT [FK_Person_AddressID_Address_ID] FOREIGN KEY ([AddressID]) REFERENCES [Addresses] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Person_AddressID_Address_ID' CREATE INDEX [IX_FK_Person_AddressID_Address_ID] ON [People] ([AddressID]); GO -- Creating foreign key on [CompanyID] in table 'Employees' ALTER TABLE [Employees] ADD CONSTRAINT [FK_Employee_CompanyID_Company_ID] FOREIGN KEY ([CompanyID]) REFERENCES [Companies] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Employee_CompanyID_Company_ID' CREATE INDEX [IX_FK_Employee_CompanyID_Company_ID] ON [Employees] ([CompanyID]); GO -- Creating foreign key on [BossEmployeeID] in table 'Employees' ALTER TABLE [Employees] ADD CONSTRAINT [FK_Employee_BossEmployeeID_Employee_ID] FOREIGN KEY ([BossEmployeeID]) REFERENCES [Employees] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Employee_BossEmployeeID_Employee_ID' CREATE INDEX [IX_FK_Employee_BossEmployeeID_Employee_ID] ON [Employees] ([BossEmployeeID]); GO -- Creating foreign key on [PersonID] in table 'Employees' ALTER TABLE [Employees] ADD CONSTRAINT [FK_Employee_PersonID_Person_ID] FOREIGN KEY ([PersonID]) REFERENCES [People] ([ID]) ON DELETE NO ACTION ON UPDATE NO ACTION; -- Creating non-clustered index for FOREIGN KEY 'FK_Employee_PersonID_Person_ID' CREATE INDEX [IX_FK_Employee_PersonID_Person_ID] ON [Employees] ([PersonID]); GO -- -------------------------------------------------- -- Script has ended -- --------------------------------------------------

Read the article
SQL SERVER – Introduction to Function SIGN

- by pinaldave

Yesterday I received an email from a friend asking how do SIGN function works. Well SIGN Function is very fundamental function. It will return the value 1, -1 or 0. If your value is negative it will return you negative -1 and if it is positive it will return you positive +1. Let us start with a simple small example. DECLARE @IntVal1 INT, @IntVal2 INT,@IntVal3 INT DECLARE @NumVal1 DECIMAL(4,2), @NumVal2 DECIMAL(4,2),@NumVal3 DECIMAL(4,2) SET @IntVal1 = 9; SET @IntVal2 = -9; SET @IntVal3 = 0; SET @NumVal1 = 9.0; SET @NumVal2 = -9.0; SET @NumVal3 = 0.0; SELECT SIGN(@IntVal1) IntVal1,SIGN(@IntVal2) IntVal2,SIGN(@IntVal3) IntVal3 SELECT SIGN(@NumVal1) NumVal1,SIGN(@NumVal2) NumVal2,SIGN(@NumVal2) NumVal3 The above function will give us following result set. You will notice that when there is positive value the function gives positive values and if the values are negative it will return you negative values. Also you will notice that if the data type is INT the return value is INT and when the value passed to the function is Numeric the result also matches it. Not every datatype is compatible with this function. Here is the quick look up of the return types. bigint -> bigint int/smallint/tinyint -> int money/smallmoney -> money numeric/decimal -> numeric/decimal everybody else -> float What will be the best example of the usage of this function that you will not have to use the CASE Statement. Here is example of CASE Statement usage and the same replaced with SIGN function. USE tempdb GO CREATE TABLE TestTable (Date1 SMALLDATETIME, Date2 SMALLDATETIME) INSERT INTO TestTable (Date1, Date2) SELECT '2012-06-22 16:15', '2012-06-20 16:15' UNION ALL SELECT '2012-06-24 16:15', '2012-06-22 16:15' UNION ALL SELECT '2012-06-22 16:15', '2012-06-22 16:15' GO -- Using Case Statement SELECT CASE WHEN DATEDIFF(d,Date1,Date2) > 0 THEN 1 WHEN DATEDIFF(d,Date1,Date2) < 0 THEN -1 ELSE 0 END AS Col FROM TestTable GO -- Using SIGN Function SELECT SIGN(DATEDIFF(d,Date1,Date2)) AS Col FROM TestTable GO DROP TABLE TestTable GO This was interesting blog post for me to write. Let me know your opinion. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Function, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology

Read the article
How to handle lookup data in a C# ASP.Net MVC4 application?

- by Jim

I am writing an MVC4 application to track documents we have on file for our clients. I'm using code first, and have created models for my objects (Company, Document, etc...). I am now faced with the topic of document expiration. Business logic dictates certain documents will expire a set number of days past the document date. For example, Document A might expire in 180 days, Document 2 in 365 days, etc... I have a class for my documents as shown below (simplified for this example). What is the best way for me to create a lookup for expiration values? I want to specify documents of type DocumentA expire in 30 days, type DocumentB expire in 75 days, etc... I can think of a few ways to do this: Lookup table in the database I can query New property in my class (DaysValidFor) which has a custom getter that returns different values based on the DocumentType A method that takes in the document type and returns the number of days and I'm sure there are other ways I'm not even thinking of. My main concern is a) not violating any best practices and b) maintainability. Are there any pros/cons I need to be aware of for the above options, or is this a case of "just pick one and run with it"? One last thought, right now the number of days is a value that does not need to be stored anywhere on a per-document basis -- however, it is possible that business logic will change this (i.e., DocumentA's are 30 days expiration by default, but this DocumentA associated with Company XYZ will be 60 days because we like them). In that case, is a property in the Document class the best way to go, seeing as I need to add that field to the DB? namespace Models { // Types of documents to track public enum DocumentType { DocumentA, DocumentB, DocumentC // etc... } // Document model public class Document { public int DocumentID { get; set; } // Foreign key to companies public int CompanyID { get; set; } public DocumentType DocumentType { get; set; } // Helper to translate enum's value to an integer for DB storage [Column("DocumentType")] public int DocumentTypeInt { get { return (int)this.DocumentType; } set { this.DocumentType = (DocumentType)value; } } [DataType(DataType.Date)] [DisplayFormat(DataFormatString = "{0:MM-dd-yyyy}", ApplyFormatInEditMode = true)] public DateTime DocumentDate { get; set; } // Navigation properties public virtual Company Company { get; set; } } }

Read the article
Only draw visible objects to the camera in 2D

- by Deukalion

I have Map, each map has an array of Ground, each Ground consists of an array of VertexPositionTexture and a texture name reference so it renders a texture at these points (as a shape through triangulation). Now when I render my map I only want to get a list of all objects that are visible in the camera. (So I won't loop through more than I have to) Structs: public struct Map { public Ground[] Ground { get; set; } } public struct Ground { public int[] Indexes { get; set; } public VertexPositionNormalTexture[] Points { get; set; } public Vector3 TopLeft { get; set; } public Vector3 TopRight { get; set; } public Vector3 BottomLeft { get; set; } public Vector3 BottomRight { get; set; } } public struct RenderBoundaries<T> { public BoundingBox Box; public T Items; } when I load a map: foreach (Ground ground in CurrentMap.Ground) { Boundaries.Add(new RenderBoundaries<Ground>() { Box = BoundingBox.CreateFromPoints(new Vector3[] { ground.TopLeft, ground.TopRight, ground.BottomLeft, ground.BottomRight }), Items = ground }); } TopLeft, TopRight, BottomLeft, BottomRight are simply the locations of each corner that the shape make. A rectangle. When I try to loop through only the objects that are visible I do this in my Draw method: public int Draw(GraphicsDevice device, ICamera camera) { BoundingFrustum frustum = new BoundingFrustum(camera.View * camera.Projection); // Visible count int count = 0; EffectTexture.World = camera.World; EffectTexture.View = camera.View; EffectTexture.Projection = camera.Projection; foreach (EffectPass pass in EffectTexture.CurrentTechnique.Passes) { pass.Apply(); foreach (RenderBoundaries<Ground> render in Boundaries.Where(m => frustum.Contains(m.Box) != ContainmentType.Disjoint)) { // Draw ground count++; } } return count; } When I try adding just one ground, then moving the camera so the ground is out of frame it still returns 1 which means it still gets draw even though it's not within the camera's view. Am I doing something or wrong or can it be because of my Camera? Any ideas why it doesn't work?

Read the article
How to write this JavaScript code without eval?

- by karlthorwald

How to write this JavaScript code without eval? var typeOfString = eval("typeof " + that.modules[modName].varName); if (typeOfString !== "undefined") { doSomething(); } The point is that the name of the var that I want to check for is in a string. Maybe it is simple but I don't know how. Edit: Thank you for the very interesting answers so far. I will follow your suggestions and integrate this into my code and do some testing and report. Could take a while. Edit2: I had another look at the could and maybe itis better I show you a bigger picture. I am greatful for the experts to explain so beautiful, it is better with more code: MYNAMESPACE.Loader = ( function() { function C() { this.modules = {}; this.required = {}; this.waitCount = 0; this.appendUrl = ''; this.docHead = document.getElementsByTagName('head')[0]; } function insert() { var that = this; //insert all script tags to the head now! //loop over all modules: for (var modName in this.required) { if(this.required.hasOwnProperty(modName)){ if (this.required[modName] === 'required') { this.required[modName] = 'loading'; this.waitCount = this.waitCount + 1; this.insertModule(modName); } } } //now poll until everything is loaded or //until timout this.intervalId = 0; var checkFunction = function() { if (that.waitCount === 0) { clearInterval(that.intervalId); that.onSuccess(); return; } for (var modName in that.required) { if(that.required.hasOwnProperty(modName)){ if (that.required[modName] === 'loading') { var typeOfString = eval("typeof " + that.modules[modName].varName); if (typeOfString !== "undefined") { //module is loaded! that.required[modName] = 'ok'; that.waitCount = that.waitCount - 1; if (that.waitCount === 0) { clearInterval(that.intervalId); that.onSuccess(); return; } } } } } }; //execute the function twice a second to check if all is loaded: this.intervalId = setInterval(checkFunction, 500); //further execution will be in checkFunction, //so nothing left to do here } C.prototype.insert = insert; //there are more functions here... return C; }()); var myLoader = new MYNAMESPACE.Loader(); //some more lines here... myLoader.insert();

Read the article
Use CSS Selectors with HtmlUnit

- by kerry

HtmlUnit is a great library for performing web integration tests in Java. But sometimes node traversal can be somewhat cumbersome. Fear not fellow automated tester (good for you!). I found a great little project on Github that will allow you to query your document for elements via css selectors similar to jQuery. The project is located at https://github.com/chrsan/css-selectors. You can use Maven to build it, or download 1.0.2 here. Beware. I will not be updating this link so I suggest you download the latest code. In any case, you can use it like so: // from HtmlUnit getting started final WebClient webClient = new WebClient(); final HtmlPage page = webClient.getPage("http://htmlunit.sourceforge.net"); final DOMNodeSelector cssSelector = new DOMNodeSelector(page.getDocumentElement()); final Set elements = cssSelector.querySelectorAll("div.section h2"); final Node first = elements.iterator().next(); assertThat(first.getTextContent(), equalTo("HtmlUnit")); The only problem here is that the querySelectAll returns a Set<Node>. Not HtmlElement like we may want in some cases. However, if you were to reflect on the Set, you would find that it is indeed a Set of HtmlElement objects. Typically, I like to create a base class for my web tests. Just for fun, I am using the $ method similar to jQuery. public class WebTestBase { protected WebClient webClient; protected HtmlPage htmlPage; protected void goTo(final String url){ return (HtmlPage)webClient.getPage(url); } protected List $(final String cssSelector) { final DOMNodeSelector cssSelector = new DOMNodeSelector(htmlPage.getDocumentElement()); final Set nodes = cssSelector.querySelectorAll("div.section h2"); // for some reason Set cannot be cast to Set? final List elements = new ArrayList(nodes.size()); for (final Node node : nodes) { elements.add((HtmlElement)node); } return elements; } } Now we can write tests like this: public class LoginWebTest extends WebTestBase { @Test public void login_page_has_instructions() throws Exception { goTo(baseUrl + "/login") assertThat( $("p.instructions").size(), equalTo(1) ); } }

Read the article
scrolling lags in emacs 23.2 with GTK

- by mefiX

Hey there, I am using emacs 23.2 with the GTK toolkit. I built emacs from source using the following configure-params: ./configure --prefix=/usr --without-makeinfo --without-sound Which builds emacs with the following configuration: Where should the build process find the source code? /home/****/incoming/emacs-23.2 What operating system and machine description files should Emacs use? `s/gnu-linux.h' and `m/intel386.h' What compiler should emacs be built with? gcc -g -O2 -Wdeclaration-after-statement -Wno-pointer-sign Should Emacs use the GNU version of malloc? yes (Using Doug Lea's new malloc from the GNU C Library.) Should Emacs use a relocating allocator for buffers? yes Should Emacs use mmap(2) for buffer allocation? no What window system should Emacs use? x11 What toolkit should Emacs use? GTK Where do we find X Windows header files? Standard dirs Where do we find X Windows libraries? Standard dirs Does Emacs use -lXaw3d? no Does Emacs use -lXpm? yes Does Emacs use -ljpeg? yes Does Emacs use -ltiff? yes Does Emacs use a gif library? yes -lgif Does Emacs use -lpng? yes Does Emacs use -lrsvg-2? no Does Emacs use -lgpm? yes Does Emacs use -ldbus? yes Does Emacs use -lgconf? no Does Emacs use -lfreetype? yes Does Emacs use -lm17n-flt? no Does Emacs use -lotf? yes Does Emacs use -lxft? yes Does Emacs use toolkit scroll bars? yes When I'm scrolling within files of a common size (about 1000 lines) holding the up/down-keys, emacs almost hangs and produces about 50% CPU-load. I use the following plugins: ido linum tabbar auto-complete-config Starting emacs with -q fixes the problem, but then I don't have any plugins. I can't figure out, which part of my .emacs is responsible for this behaviour. Here's an excerpt of my .emacs-file: (require 'ido) (ido-mode 1) (require 'linum) (global-linum-mode 1) (require 'tabbar) (tabbar-mode 1) (tabbar-local-mode 0) (tabbar-mwheel-mode 0) (setq tabbar-buffer-groups-function (lambda () (list "All"))) (global-set-key [M-left] 'tabbar-backward) (global-set-key [M-right] 'tabbar-forward) ;; hide the toolbar (gtk etc.) (tool-bar-mode -1) ;; Mouse scrolling enhancements (setq mouse-wheel-progressive-speed nil) (setq mouse-wheel-scroll-amount '(5 ((shift) . 5) ((control) . nil))) ;; Smart-HOME (defun smart-beginning-of-line () "Forces the cursor to jump to the first none whitespace char of the current line when pressing HOME" (interactive) (let ((oldpos (point))) (back-to-indentation) (and (= oldpos (point)) (beginning-of-line)))) (put 'smart-beginning-of-line 'CUA 'move) (global-set-key [home] 'smart-beginning-of-line) (custom-set-variables ;; custom-set-variables was added by Custom. ;; If you edit it by hand, you could mess it up, so be careful. ;; Your init file should contain only one such instance. ;; If there is more than one, they won't work right. '(column-number-mode t) '(cua-mode t nil (cua-base)) '(custom-buffer-indent 4) '(delete-selection-mode nil) '(display-time-24hr-format t) '(display-time-day-and-date 1) '(display-time-mode t) '(global-font-lock-mode t nil (font-lock)) '(inhibit-startup-buffer-menu t) '(inhibit-startup-screen t) '(pc-select-meta-moves-sexps t) '(pc-select-selection-keys-only t) '(pc-selection-mode t nil (pc-select)) '(scroll-bar-mode (quote right)) '(show-paren-mode t) '(standard-indent 4) '(uniquify-buffer-name-style (quote forward) nil (uniquify))) (setq-default tab-width 4) (setq-default indent-tabs-mode t) (setq c-basic-offset 4) ;; Highlighting of the current line (global-hl-line-mode 1) (set-face-background 'hl-line "#E8F2FE") (defalias 'yes-or-no-p 'y-or-n-p) (display-time) (set-language-environment "Latin-1") ;; Change cursor color according to mode (setq djcb-read-only-color "gray") ;; valid values are t, nil, box, hollow, bar, (bar . WIDTH), hbar, ;; (hbar. HEIGHT); see the docs for set-cursor-type (setq djcb-read-only-cursor-type 'hbar) (setq djcb-overwrite-color "red") (setq djcb-overwrite-cursor-type 'box) (setq djcb-normal-color "black") (setq djcb-normal-cursor-type 'bar) (defun djcb-set-cursor-according-to-mode () "change cursor color and type according to some minor modes." (cond (buffer-read-only (set-cursor-color djcb-read-only-color) (setq cursor-type djcb-read-only-cursor-type)) (overwrite-mode (set-cursor-color djcb-overwrite-color) (setq cursor-type djcb-overwrite-cursor-type)) (t (set-cursor-color djcb-normal-color) (setq cursor-type djcb-normal-cursor-type)))) (add-hook 'post-command-hook 'djcb-set-cursor-according-to-mode) (define-key global-map '[C-right] 'forward-sexp) (define-key global-map '[C-left] 'backward-sexp) (define-key global-map '[s-left] 'windmove-left) (define-key global-map '[s-right] 'windmove-right) (define-key global-map '[s-up] 'windmove-up) (define-key global-map '[s-down] 'windmove-down) (define-key global-map '[S-down-mouse-1] 'mouse-stay-and-copy) (define-key global-map '[C-M-S-down-mouse-1] 'mouse-stay-and-swap) (define-key global-map '[S-mouse-2] 'mouse-yank-and-kill) (define-key global-map '[C-S-down-mouse-1] 'mouse-stay-and-kill) (define-key global-map "\C-a" 'mark-whole-buffer) (custom-set-faces ;; custom-set-faces was added by Custom. ;; If you edit it by hand, you could mess it up, so be careful. ;; Your init file should contain only one such instance. ;; If there is more than one, they won't work right. '(default ((t (:inherit nil :stipple nil :background "#f7f9fa" :foreground "#191919" :inverse-video nil :box nil :strike-through nil :overline nil :underline nil :slant normal :weight normal :height 98 :width normal :foundry "unknown" :family "DejaVu Sans Mono")))) '(font-lock-builtin-face ((((class color) (min-colors 88) (background light)) (:foreground "#642880" :weight bold)))) '(font-lock-comment-face ((((class color) (min-colors 88) (background light)) (:foreground "#3f7f5f")))) '(font-lock-constant-face ((((class color) (min-colors 88) (background light)) (:weight bold)))) '(font-lock-doc-face ((t (:inherit font-lock-string-face :foreground "#3f7f5f")))) '(font-lock-function-name-face ((((class color) (min-colors 88) (background light)) (:foreground "Black" :weight bold)))) '(font-lock-keyword-face ((((class color) (min-colors 88) (background light)) (:foreground "#7f0055" :weight bold)))) '(font-lock-preprocessor-face ((t (:inherit font-lock-builtin-face :foreground "#7f0055" :weight bold)))) '(font-lock-string-face ((((class color) (min-colors 88) (background light)) (:foreground "#0000c0")))) '(font-lock-type-face ((((class color) (min-colors 88) (background light)) (:foreground "#7f0055" :weight bold)))) '(font-lock-variable-name-face ((((class color) (min-colors 88) (background light)) (:foreground "Black")))) '(minibuffer-prompt ((t (:foreground "medium blue")))) '(mode-line ((t (:background "#222222" :foreground "White")))) '(tabbar-button ((t (:inherit tabbar-default :foreground "dark red")))) '(tabbar-button-highlight ((t (:inherit tabbar-default :background "white" :box (:line-width 2 :color "white"))))) '(tabbar-default ((t (:background "gray90" :foreground "gray50" :box (:line-width 3 :color "gray90") :height 100)))) '(tabbar-highlight ((t (:underline t)))) '(tabbar-selected ((t (:inherit tabbar-default :foreground "blue" :weight bold)))) '(tabbar-separator ((t nil))) '(tabbar-unselected ((t (:inherit tabbar-default))))) Any suggestions? Kind regards, mefiX

Read the article
NHibernate GenericADO Exception

- by Ris90

Hi, I'm trying to make simple many-to-one association, using NHibernate.. I have class Recruit with this mapping: <class name="Recruit" table="Recruits"> <id name="ID"> <generator class="native"/> </id> <property name="Lastname" column="lastname"/> <property name="Name" column="name"/> <property name="MedicalReport" column="medicalReport"/> <property name="DateOfBirth" column ="dateOfBirth" type="Date"/> <many-to-one name="AssignedOnRecruitmentOffice" column="assignedOnRecruitmentOffice" class="RecruitmentOffice"/> which is many-to-one connected to RecruitmentOffices: <class name="RecruitmentOffice" table="RecruitmentOffices"> <id name="ID" column="ID"> <generator class="native"/> </id> <property name="Chief" column="chief"/> <property name="Name" column="name"/> <property name ="Address" column="address"/> <set name="Recruits" cascade="save-update" inverse="true" lazy="true"> <key> <column name="AssignedOnRecruitmentOffice"/> </key> <one-to-many class="Recruit"/> </set> And create Repository class with method Insert: public void Insert(Recruit recruit) { using (ITransaction transaction = session.BeginTransaction()) { session.Save(recruit); transaction.Commit(); } } then I try to save new recrui to base: Recruit test = new Recruit(); RecruitmentOffice office = new RecruitmentOffice(); ofice.Name = "test"; office.Chief = "test"; test.AssignedOnRecruitmentOffice = office; test.Name = "test"; test.DateOfBirth = DateTime.Now; RecruitRepository testing = new RecruitRepository(); testing.Insert(test); And have this error GenericADOException could not insert: [OSiUBD.Models.DAO.Recruit][SQL: INSERT INTO Recruits (lastname, name, medicalReport, dateOfBirth, assignedOnRecruitmentOffice) VALUES (?, ?, ?, ?, ?); select SCOPE_IDENTITY()] on session.Save

Read the article
Tales of a corrupt SQL log

- by guybarrette

Warning: I’m a simple dev, not an all powerful DBA with godly powers. This morning, one of my sites was down and DNN reported a problem with the database. A quick series of tests revealed that the culprit was a corrupted log file. Easy fix I said, I have daily backups so it’s just a mater of restoring a good copy of the database and log files. Well, I found out that’s not exactly true. You see, for this database, I have daily file backups and these are not database backups created by SQL Server. So I restored a set of files from a couple of days ago, stopped the SQL service, copied the files over the bad ones, restarted the service only to find out that SQL doesn’t like when you do that. It suspects something fishy and marks the database as suspect. A database marked as suspect can’t be accessed at all. So now what? I searched throughout the tubes of the InterWeb and found that you can restore from a corrupted log file by creating a new database with the same name as the defective one, then copy the restored database file (the one with data) over the newly created one. Sweet! But you still end up with SQL marking the database as suspect but at least, the newly created log is OK. Well not true, it’s not corrupted but the lack of data makes it not OK for SQL so you need to rebuild the log. How can you do that when SQL blocks any action the database? First, you need to change the database status from suspect to emergency. Then you need to set the database for single access only. After that, you need to repair the log with DBCC and do the DBA dance. If you dance long enough, SQL should repair the log file. Now you need to set the access back to multi user. Here’s the T-SQL script: use master GO EXEC sp_resetstatus 'MyDatabase' ALTER DATABASE MyDatabase SET EMERGENCY Alter database MyDatabase set Single_User DBCC checkdb('MyDatabase') ALTER DATABASE MyDatabase SET SINGLE_USER WITH ROLLBACK IMMEDIATE DBCC CheckDB ('MyDatabase', REPAIR_ALLOW_DATA_LOSS) ALTER DATABASE MyDatabase SET MULTI_USER So I guess that I would have been a lot easier to restore a SQL backup. I can’t really say but the InterWeb seems to say so. Anyway, lessons learned: Vive la différence: File backups are different then SQL backups. Don’t touch me: SQL doesn’t like when you restore a file over a corrupted one. The more the merrier: You should do both SQL and file backups. WTF?: The InterWeb provides you with dozens of way to deal with the problem but many are SQL 2000 or SQL 2005 only, many are confusing and many are written in strange dialects only DBAs understand. var addthis_pub="guybarrette";

Read the article
sql server procedure error

- by Mohan

CREATE PROCEDURE USP_SEARCH_HOTELS ( @Text varchar(50), @Type varchar(40) ) AS BEGIN Declare @Query VARCHAR(60) IF @Type = 'By Country' BEGIN SET @Query = 'Hotel.countryName like '+ @Text+'%' END ELSE IF @Type = 'By State' BEGIN SET @Query = 'HOTEL.stateName like '+ @Text+'%' END ELSE IF @Type='By Property Name' BEGIN SET @Query='hotel.propertyname like'+ @Text+'%' End ELSE IF @Type='By Rating' BEGIN SET @Query='hotel.starRating='+ Cast(@Text as INT) END ELSE IF @Type='By City' BEGIN SET @Query='hotel.cityName like '+ @Text+'%' END begin select * from hotel,tbl_cust_info where hotel.agentID=Tbl_Cust_Info.Cust_ID and (@Query) end END WHAT IS THE ERROR IN THIS PROCEDURE PLEASE HELP.

Read the article
Execute SQL SP in Excel VBA

- by TheOCD

HI I am having problem with getting all the columns back when i execute following code in excel vba. I only get 6 out of 23 columns back. Connection, command etc works fine (i can see exec command in the SQL Profiler), data headers are created for all 23 columns but i only get data for 6 column. Side Note: it's not prod level code, have missed out error handling on purpose, sp works fine in SQL management studio, ASP.Net, C# win form app, it is for Excel 2003 connecting to SQL 2008. Can someone help me troubleshoot it? Dim connection As ADODB.connection Dim recordset As ADODB.recordset Dim command As ADODB.command Dim strProcName As String 'Stored Procedure name Dim strConn As String ' connection string. Dim selectedVal As String 'Set ADODB requirements Set connection = New ADODB.connection Set recordset = New ADODB.recordset Set command = New ADODB.command If Workbooks("Book2.xls").MultiUserEditing = True Then MsgBox "You do not have Exclusive access to the workbook at this time." & _ vbNewLine & "Please have all other users close the workbook and then try again.", vbOKOnly + vbExclamation Exit Sub Else On Error Resume Next ActiveWorkbook.ExclusiveAccess 'On Error GoTo No_Bugs End If 'set the active sheet Set oSht = Workbooks("Book2.xls").Sheets(1) 'get the connection string, if empty just exit strConn = ConnectionString() If strConn = "" Then Exit Sub End If ' selected value, if <NOTHING> just exit selectedVal = selectedValue() If selectedVal = "<NOTHING>" Then Exit Sub End If If Not oSht Is Nothing Then 'Open database connection connection.ConnectionString = strConn connection.Open ' set command stuff. command.ActiveConnection = connection command.CommandText = "GetAlbumByName" command.CommandType = adCmdStoredProc command.Parameters.Refresh command.Parameters(1).Value = selectedVal 'Execute stored procedure and return to a recordset Set recordset = command.Execute() If recordset.BOF = False And recordset.EOF = False Then Sheets("Sheet2").[A1].CopyFromRecordset recordset ' Create headers and copy data With Sheets("Sheet2") For Column = 0 To recordset.Fields.Count - 1 .Cells(1, Column + 1).Value = recordset.Fields(Column).Name Next .Range(.Cells(1, 1), .Cells(1, recordset.Fields.Count)).Font.Bold = True .Cells(2, 1).CopyFromRecordset recordset End With Else MsgBox "b4 BOF or after EOF.", vbOKOnly + vbExclamation End If 'Close database connection and clean up If CBool(recordset.State And adStateOpen) = True Then recordset.Close Set recordset = Nothing If CBool(connection.State And adStateOpen) = True Then connection.Close Set connection = Nothing Else MsgBox "oSheet2 is Nothing.", vbOKOnly + vbExclamation End If

Read the article
XML parsing and transforming (XSLT or otherwise)

- by observ

I have several xml files that are formated this way: <ROOT> <OBJECT> <identity> <id>123</id> </identity> <child2 attr = "aa">32</child2> <child3> <childOfChild3 att1="aaa" att2="bbb" att3="CCC">LN</childOfChild3> </child3> <child4> <child5> <child6>3ddf</child6> <child7> <childOfChild7 att31="RR">1231</childOfChild7> </child7> </child5> </child4> </OBJECT> <OBJECT> <identity> <id>124</id> </identity> <child2 attr = "bb">212</child2> <child3> <childOfChild3 att1="ee" att2="ccc" att3="EREA">OP</childOfChild3> </child3> <child4> <child5> <child6>213r</child6> <child7> <childOfChild7 att31="EE">1233</childOfChild7> </child7> </child5> </child4> </OBJECT> </ROOT> How can i format it this way?: <ROOT> <OBJECT> <id>123</id> <child2>32</child2> <attr>aa</attr> <child3></child3> <childOfChild3>LN</childOfChild3> <att1>aaa</att1> <att2>bbb</att2> <att3>CCC</att3> <child4></child4> <child5></child5> <child6>3ddf</child6> <child7></child7> <childOfChild7>1231</childOfChild7> <att31>RR</att31> </OBJECT> <OBJECT> <id>124</id> <child2>212</child2> <attr>bb</attr> <child3></child3> <childOfChild3>LN</childOfChild3> <att1>ee</att1> <att2>ccc</att2> <att3>EREA</att3> <child4></child4> <child5></child5> <child6>213r</child6> <child7></child7> <childOfChild7>1233</childOfChild7> <att31>EE</att31> </OBJECT> </ROOT> I know some C# so maybe a parser there? or some generic xslt? The xml files are some data received from a client, so i can't control the way they are sending it to me. L.E. Basically when i am trying to test this data in excel (for example i want to make sure that the attribute of childOfChild7 corresponds to the correct identity id) i am getting a lot of blank spaces. If i am importing in access to get only the data i want out, i have to do a thousands subqueries to get them all in a nice table. Basically i just want to see for one Object all its data (one object - One row) and then just delete/hide the columns i don't need.

Read the article

Search Results

Search found 63114 results on 2525 pages for 'set identity insert'.

Page 328/2525 | < Previous Page | 324 325 326 327 328 329 330 331 332 333 334 335 | Next Page >

- by user1319903

- by shiju

- by phenevo

- by JoshReuben

- by kross

- by Paul White NZ

- by Ethan

- by vakman

- by user127236

- by Steve Loethen

- by poonam

- by Rob

- by Derek Hunziker

- by Jaxidian

- by pinaldave

- by Jim

- by Deukalion

- by karlthorwald

- by kerry

- by mefiX

- by Ris90

- by guybarrette

- by Mohan

- by TheOCD

- by observ

< Previous Page | 324 325 326 327 328 329 330 331 332 333 334 335 | Next Page >