Search Results

Search found 2534 results on 102 pages for 'ascii art'.

Page 75/102 | < Previous Page | 71 72 73 74 75 76 77 78 79 80 81 82  | Next Page >

• Our own Daily WTF

  - by Dennis Vroegop

  Originally posted on: http://geekswithblogs.net/dvroegop/archive/2014/08/20/our-own-daily-wtf.aspxIf you're a developer, you've probably heard of the website the DailyWTF. If you haven't, head on over to http://www.thedailywtf.com and read. And laugh. I'll wait. Read it? Good. If you're a bit like me probably you've been wondering how on earth some people ever get hired as a software engineer. Most of the stories there seem to weird to be true: no developer would write software like that right? And then you run into a little nugget of code one of your co-workers wrote. And then you realize: "Hey, it happens everywhere!" Look at this piece of art I found in our codebase recently: public static decimal ToDecimal(this string input) {     System.Globalization.CultureInfo cultureInfo = System.Globalization.CultureInfo.InstalledUICulture;     var numberFormatInfo = (System.Globalization.NumberFormatInfo)cultureInfo.NumberFormat.Clone();     int dotIndex = input.IndexOf(".");     int commaIndex = input.IndexOf(",");     if (dotIndex > commaIndex)         numberFormatInfo.NumberDecimalSeparator = ".";     else if (commaIndex > dotIndex)         numberFormatInfo.NumberDecimalSeparator = ",";     decimal result;     if (decimal.TryParse(input, System.Globalization.NumberStyles.Float, numberFormatInfo, out result))         return result;     else         throw new Exception(string.Format("Invalid input for decimal parsing: {0}. Decimal separator: {1}.", input, numberFormatInfo.NumberDecimalSeparator)); }  Me and a collegue have been looking long and hard at this and what we concluded was the following: Apparently, we don't trust our users to be able to correctly set the culture in Windows. Users aren't able to determine if they should tell Windows to use a decimal point or a comma to display numbers. So what we've done here is make sure that whatever the user enters, we'll translate that into whatever the user WANTS to enter instead of what he actually did. So if you set your locale to US, since you're a US citizen, but you want to enter the number 12.34 in the Dutch style (because, you know, the Dutch are way cooler with numbers) so you enter 12,34 we will understand this and respect your wishes! Of course, if you change your mind and in the next input field you decide to use the decimal dot again, that's fine with us as well. We will do the hard work. Now, I am all for smart software. Software that can handle all sorts of input the user can think of. But this feels a little uhm, I don't know.. wrong.. Or am I too old fashioned?

  Read the article

• Oredev 2012: Summary and source code

  - by Laurent Bugnion

  This week, I had the pleasure to be invited to talk at Oredev, a really cool conference taking place in Malmo, Sweden. The whole event is awesome, including a very special dinner on Monday including sauna and swimming in a 6 degrees cold Baltic sea, and a reception with dinner at the town hall, including the mayor himself. Considering Malmo is a town of 300'000 inhabitants, it is a pretty nice occasion and the historical building itself is really worth seeing. For those interested, I placed my pictures on my Flickr account. I had a workshop on Tuesday morning about Windows 8 development with XAML/C#, and then a session on Wednesday about MVVM in Windows Phone 8 and Windows 8, of course using MVVM Light. I was very nervous because I reworked some of my demos as recently as this morning, in the wake of the Build conference last week and the release of both the Windows Phone SDK and MVVM Light V4.1. Everything went well however, and if I judge by the people I talked t after the talk, and Twitter, everything went pretty well. Before my talk on Tuesday, I had the pleasure to see a talk by Iris Classon (@irisclasson) on the challenges of being a "n00b" and a woman in software development. I especially appreciated her research and conclusions on the lack of women I our industry, a topic that is dear to my heart (because I want the best possible future for my two daughters, and also because I really enjoy working with women on projects, and getting a different insight on the art of software development. I really want to thank the excellent organization committee for their hard work and their fantastic welcome to Malmo. In particular Emily Holweck did a wonderful job and was super helpful throughout the preparation and the conference itself. I made a few pictures during my stay, all with the new Nokia Lumia 920, and hope you will enjoy them too. The source code and the slides… The source code is available for download from Skydrive. You will find the following: Windows 8 workshop slides. MVVM Applied slides Source code package with Win8Demo: The demo I built during the 4 hours workshop, with some light MVVM, web services (JSON), GridView, Design time data (Blend / Visual Studio designer), Bing maps integration, location sensor, Search pane integration. SemanticZoomSample: a sample I put together to demonstrate the SemanticZoom control, with two GridViews and of course full design time data for Blend work. Due to time constraints, I was not able to show this demo during the workshop, but I publish it anyway, hoping it will be useful to someone. PictureUploader: The demo I built during my 50 minutes session about MVVM Applied in Windows Phone 8 and Windows 8. Code sharing, design time data, MVVM Light are used in Windows Phone 8 and Windows 8 apps. And in video… You can also see the video of my MVVM talk thanks to the good services of the Oredev team! MVVM Applied in Windows Phone and Windows 8 from Øredev Conference on Vimeo.   Laurent Bugnion (GalaSoft) Subscribe | Twitter | Facebook | Flickr | LinkedIn

  Read the article

• Can Anything be Done to Make Improv (a 1993 Win 3.1 App) handle larger Files?

  - by user75185

  My very favorite spradsheet is Improv, a 1993 Windows 3.1 application. It still puts Excel to shame for building spreadsheets and writing formulas. The only problem is because Improv was written when 1 Meg of RAM was state of the art, it becomes unstable when working with larger spreadsheets and often crashes and/or corrupts the data file. I am working on a project that greatly exceeds Improv's limits. Although it will ultimately require more robust databasing capability, I could save a lot of critical time if I could delay that headache and continue working in Improv for now. To that end, I moved to the only product I could find that comes close, Quantrix, which is nothing more than Improv updated to handle large spreadsheets and utilize today's technologies. The problems with Quantrix are its speed (significantly slower than Improv) and its $1000 price (which I cannot afford). I have already had 3 15 day extensions after the initial 30 day trial, so my time to use Quantrix as a bridge is at its end. Searches for Improv over the years have gotten me nowhere and, not surprisingly after reading some posts on this site, I got nothing for the money and time invested to find a programmer to write code to "fix" this problem. Improv is freely available as "abandonware" at http://vetusware.com/download/LotusImprov2.1/?id=5797 , and the best background info can be found on Wikipedia and at "Moose's Greatest Software Products of All Time - Lotus Improv" http://moosevalley.fhost.com.au/mooses_review_page_lotus_improv.html It is critically urgent for me to focus on analyzing the data asap. Working in a stable Improv would, without question, be the fastest route. To that end, I am looking for answers to the following questions and anything else that might be helpful: 1) Is it lawful to hire someone to fix Improv for my own use? If so, 2) About how much should it cost? 3) About how long should it take? 4) What skills should I be looking for &/or how should a post be worded? 5) Is there a niche site where it should it be posted? 6) What questions can I ask to quickly screen candidates? Since I am not a programmer, I need questions the answers to which leave no room to confuse me, whether intentional or not. For example, what tools or players should someone with an acceptable competency level have knowledge of?

  Read the article

• SQLite, python, unicode, and non-utf data

  - by Nathan Spears

  I started by trying to store strings in sqlite using python, and got the message: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. Ok, I switched to Unicode strings. Then I started getting the message: sqlite3.OperationalError: Could not decode to UTF-8 column 'tag_artist' with text 'Sigur Rós' when trying to retrieve data from the db. More research and I started encoding it in utf8, but then 'Sigur Rós' starts looking like 'Sigur Rós' note: My console was set to display in 'latin_1' as @John Machin pointed out. What gives? After reading this, describing exactly the same situation I'm in, it seems as if the advice is to ignore the other advice and use 8-bit bytestrings after all. I didn't know much about unicode and utf before I started this process. I've learned quite a bit in the last couple hours, but I'm still ignorant of whether there is a way to correctly convert 'ó' from latin-1 to utf-8 and not mangle it. If there isn't, why would sqlite 'highly recommend' I switch my application to unicode strings? I'm going to update this question with a summary and some example code of everything I've learned in the last 24 hours so that someone in my shoes can have an easy(er) guide. If the information I post is wrong or misleading in any way please tell me and I'll update, or one of you senior guys can update. Summary of answers Let me first state the goal as I understand it. The goal in processing various encodings, if you are trying to convert between them, is to understand what your source encoding is, then convert it to unicode using that source encoding, then convert it to your desired encoding. Unicode is a base and encodings are mappings of subsets of that base. utf_8 has room for every character in unicode, but because they aren't in the same place as, for instance, latin_1, a string encoded in utf_8 and sent to a latin_1 console will not look the way you expect. In python the process of getting to unicode and into another encoding looks like: str.decode('source_encoding').encode('desired_encoding') or if the str is already in unicode str.encode('desired_encoding') For sqlite I didn't actually want to encode it again, I wanted to decode it and leave it in unicode format. Here are four things you might need to be aware of as you try to work with unicode and encodings in python. The encoding of the string you want to work with, and the encoding you want to get it to. The system encoding. The console encoding. The encoding of the source file Elaboration: (1) When you read a string from a source, it must have some encoding, like latin_1 or utf_8. In my case, I'm getting strings from filenames, so unfortunately, I could be getting any kind of encoding. Windows XP uses UCS-2 (a Unicode system) as its native string type, which seems like cheating to me. Fortunately for me, the characters in most filenames are not going to be made up of more than one source encoding type, and I think all of mine were either completely latin_1, completely utf_8, or just plain ascii (which is a subset of both of those). So I just read them and decoded them as if they were still in latin_1 or utf_8. It's possible, though, that you could have latin_1 and utf_8 and whatever other characters mixed together in a filename on Windows. Sometimes those characters can show up as boxes, other times they just look mangled, and other times they look correct (accented characters and whatnot). Moving on. (2) Python has a default system encoding that gets set when python starts and can't be changed during runtime. See here for details. Dirty summary ... well here's the file I added: \# sitecustomize.py \# this file can be anywhere in your Python path, \# but it usually goes in ${pythondir}/lib/site-packages/ import sys sys.setdefaultencoding('utf_8') This system encoding is the one that gets used when you use the unicode("str") function without any other encoding parameters. To say that another way, python tries to decode "str" to unicode based on the default system encoding. (3) If you're using IDLE or the command-line python, I think that your console will display according to the default system encoding. I am using pydev with eclipse for some reason, so I had to go into my project settings, edit the launch configuration properties of my test script, go to the Common tab, and change the console from latin-1 to utf-8 so that I could visually confirm what I was doing was working. (4) If you want to have some test strings, eg test_str = "ó" in your source code, then you will have to tell python what kind of encoding you are using in that file. (FYI: when I mistyped an encoding I had to ctrl-Z because my file became unreadable.) This is easily accomplished by putting a line like so at the top of your source code file: # -*- coding: utf_8 -*- If you don't have this information, python attempts to parse your code as ascii by default, and so: SyntaxError: Non-ASCII character '\xf3' in file _redacted_ on line 81, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details Once your program is working correctly, or, if you aren't using python's console or any other console to look at output, then you will probably really only care about #1 on the list. System default and console encoding are not that important unless you need to look at output and/or you are using the builtin unicode() function (without any encoding parameters) instead of the string.decode() function. I wrote a demo function I will paste into the bottom of this gigantic mess that I hope correctly demonstrates the items in my list. Here is some of the output when I run the character 'ó' through the demo function, showing how various methods react to the character as input. My system encoding and console output are both set to utf_8 for this run: '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Now I will change the system and console encoding to latin_1, and I get this output for the same input: 'ó' = original char <type 'str'> repr(char)='\xf3' 'ó' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Notice that the 'original' character displays correctly and the builtin unicode() function works now. Now I change my console output back to utf_8. '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' '?' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Here everything still works the same as last time but the console can't display the output correctly. Etc. The function below also displays more information that this and hopefully would help someone figure out where the gap in their understanding is. I know all this information is in other places and more thoroughly dealt with there, but I hope that this would be a good kickoff point for someone trying to get coding with python and/or sqlite. Ideas are great but sometimes source code can save you a day or two of trying to figure out what functions do what. Disclaimers: I'm no encoding expert, I put this together to help my own understanding. I kept building on it when I should have probably started passing functions as arguments to avoid so much redundant code, so if I can I'll make it more concise. Also, utf_8 and latin_1 are by no means the only encoding schemes, they are just the two I was playing around with because I think they handle everything I need. Add your own encoding schemes to the demo function and test your own input. One more thing: there are apparently crazy application developers making life difficult in Windows. #!/usr/bin/env python # -*- coding: utf_8 -*- import os import sys def encodingDemo(str): validStrings = () try: print "str =",str,"{0} repr(str) = {1}".format(type(str), repr(str)) validStrings += ((str,""),) except UnicodeEncodeError as ude: print "Couldn't print the str itself because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print ude try: x = unicode(str) print "unicode(str) = ",x validStrings+= ((x, " decoded into unicode by the default system encoding"),) except UnicodeDecodeError as ude: print "ERROR. unicode(str) couldn't decode the string because the system encoding is set to an encoding that doesn't understand some character in the string." print "\tThe system encoding is set to {0}. See error:\n\t".format(sys.getdefaultencoding()), print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the unicode(str) because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('latin_1') print "str.decode('latin_1') =",x validStrings+= ((x, " decoded with latin_1 into unicode"),) try: print "str.decode('latin_1').encode('utf_8') =",str.decode('latin_1').encode('utf_8') validStrings+= ((x, " decoded with latin_1 into unicode and encoded into utf_8"),) except UnicodeDecodeError as ude: print "The string was decoded into unicode using the latin_1 encoding, but couldn't be encoded into utf_8. See error:\n\t", print ude except UnicodeDecodeError as ude: print "Something didn't work, probably because the string wasn't latin_1 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('latin_1') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('utf_8') print "str.decode('utf_8') =",x validStrings+= ((x, " decoded with utf_8 into unicode"),) try: print "str.decode('utf_8').encode('latin_1') =",str.decode('utf_8').encode('latin_1') except UnicodeDecodeError as ude: print "str.decode('utf_8').encode('latin_1') didn't work. The string was decoded into unicode using the utf_8 encoding, but couldn't be encoded into latin_1. See error:\n\t", validStrings+= ((x, " decoded with utf_8 into unicode and encoded into latin_1"),) print ude except UnicodeDecodeError as ude: print "str.decode('utf_8') didn't work, probably because the string wasn't utf_8 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('utf_8') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t",uee print print "Printing information about each character in the original string." for char in str: try: print "\t'" + char + "' = original char {0} repr(char)={1}".format(type(char), repr(char)) except UnicodeDecodeError as ude: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), ude) except UnicodeEncodeError as uee: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), uee) print uee try: x = unicode(char) print "\t'" + x + "' = unicode(char) {1} repr(unicode(char))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = unicode(char) ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = unicode(char) {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('latin_1') print "\t'" + x + "' = char.decode('latin_1') {1} repr(char.decode('latin_1'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('latin_1') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('latin_1') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('utf_8') print "\t'" + x + "' = char.decode('utf_8') {1} repr(char.decode('utf_8'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('utf_8') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('utf_8') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) print x = 'ó' encodingDemo(x) Much thanks for the answers below and especially to @John Machin for answering so thoroughly.

  Read the article

• SQLite, python, unicode, and non-utf data

  - by Nathan Spears

  I started by trying to store strings in sqlite using python, and got the message: sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. Ok, I switched to Unicode strings. Then I started getting the message: sqlite3.OperationalError: Could not decode to UTF-8 column 'tag_artist' with text 'Sigur Rós' when trying to retrieve data from the db. More research and I started encoding it in utf8, but then 'Sigur Rós' starts looking like 'Sigur Rós' note: My console was set to display in 'latin_1' as @John Machin pointed out. What gives? After reading this, describing exactly the same situation I'm in, it seems as if the advice is to ignore the other advice and use 8-bit bytestrings after all. I didn't know much about unicode and utf before I started this process. I've learned quite a bit in the last couple hours, but I'm still ignorant of whether there is a way to correctly convert 'ó' from latin-1 to utf-8 and not mangle it. If there isn't, why would sqlite 'highly recommend' I switch my application to unicode strings? I'm going to update this question with a summary and some example code of everything I've learned in the last 24 hours so that someone in my shoes can have an easy(er) guide. If the information I post is wrong or misleading in any way please tell me and I'll update, or one of you senior guys can update. Summary of answers Let me first state the goal as I understand it. The goal in processing various encodings, if you are trying to convert between them, is to understand what your source encoding is, then convert it to unicode using that source encoding, then convert it to your desired encoding. Unicode is a base and encodings are mappings of subsets of that base. utf_8 has room for every character in unicode, but because they aren't in the same place as, for instance, latin_1, a string encoded in utf_8 and sent to a latin_1 console will not look the way you expect. In python the process of getting to unicode and into another encoding looks like: str.decode('source_encoding').encode('desired_encoding') or if the str is already in unicode str.encode('desired_encoding') For sqlite I didn't actually want to encode it again, I wanted to decode it and leave it in unicode format. Here are four things you might need to be aware of as you try to work with unicode and encodings in python. The encoding of the string you want to work with, and the encoding you want to get it to. The system encoding. The console encoding. The encoding of the source file Elaboration: (1) When you read a string from a source, it must have some encoding, like latin_1 or utf_8. In my case, I'm getting strings from filenames, so unfortunately, I could be getting any kind of encoding. Windows XP uses UCS-2 (a Unicode system) as its native string type, which seems like cheating to me. Fortunately for me, the characters in most filenames are not going to be made up of more than one source encoding type, and I think all of mine were either completely latin_1, completely utf_8, or just plain ascii (which is a subset of both of those). So I just read them and decoded them as if they were still in latin_1 or utf_8. It's possible, though, that you could have latin_1 and utf_8 and whatever other characters mixed together in a filename on Windows. Sometimes those characters can show up as boxes, other times they just look mangled, and other times they look correct (accented characters and whatnot). Moving on. (2) Python has a default system encoding that gets set when python starts and can't be changed during runtime. See here for details. Dirty summary ... well here's the file I added: \# sitecustomize.py \# this file can be anywhere in your Python path, \# but it usually goes in ${pythondir}/lib/site-packages/ import sys sys.setdefaultencoding('utf_8') This system encoding is the one that gets used when you use the unicode("str") function without any other encoding parameters. To say that another way, python tries to decode "str" to unicode based on the default system encoding. (3) If you're using IDLE or the command-line python, I think that your console will display according to the default system encoding. I am using pydev with eclipse for some reason, so I had to go into my project settings, edit the launch configuration properties of my test script, go to the Common tab, and change the console from latin-1 to utf-8 so that I could visually confirm what I was doing was working. (4) If you want to have some test strings, eg test_str = "ó" in your source code, then you will have to tell python what kind of encoding you are using in that file. (FYI: when I mistyped an encoding I had to ctrl-Z because my file became unreadable.) This is easily accomplished by putting a line like so at the top of your source code file: # -*- coding: utf_8 -*- If you don't have this information, python attempts to parse your code as ascii by default, and so: SyntaxError: Non-ASCII character '\xf3' in file _redacted_ on line 81, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details Once your program is working correctly, or, if you aren't using python's console or any other console to look at output, then you will probably really only care about #1 on the list. System default and console encoding are not that important unless you need to look at output and/or you are using the builtin unicode() function (without any encoding parameters) instead of the string.decode() function. I wrote a demo function I will paste into the bottom of this gigantic mess that I hope correctly demonstrates the items in my list. Here is some of the output when I run the character 'ó' through the demo function, showing how various methods react to the character as input. My system encoding and console output are both set to utf_8 for this run: '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Now I will change the system and console encoding to latin_1, and I get this output for the same input: 'ó' = original char <type 'str'> repr(char)='\xf3' 'ó' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' 'ó' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Notice that the 'original' character displays correctly and the builtin unicode() function works now. Now I change my console output back to utf_8. '?' = original char <type 'str'> repr(char)='\xf3' '?' = unicode(char) <type 'unicode'> repr(unicode(char))=u'\xf3' '?' = char.decode('latin_1') <type 'unicode'> repr(char.decode('latin_1'))=u'\xf3' '?' = char.decode('utf_8') ERROR: 'utf8' codec can't decode byte 0xf3 in position 0: unexpected end of data Here everything still works the same as last time but the console can't display the output correctly. Etc. The function below also displays more information that this and hopefully would help someone figure out where the gap in their understanding is. I know all this information is in other places and more thoroughly dealt with there, but I hope that this would be a good kickoff point for someone trying to get coding with python and/or sqlite. Ideas are great but sometimes source code can save you a day or two of trying to figure out what functions do what. Disclaimers: I'm no encoding expert, I put this together to help my own understanding. I kept building on it when I should have probably started passing functions as arguments to avoid so much redundant code, so if I can I'll make it more concise. Also, utf_8 and latin_1 are by no means the only encoding schemes, they are just the two I was playing around with because I think they handle everything I need. Add your own encoding schemes to the demo function and test your own input. One more thing: there are apparently crazy application developers making life difficult in Windows. #!/usr/bin/env python # -*- coding: utf_8 -*- import os import sys def encodingDemo(str): validStrings = () try: print "str =",str,"{0} repr(str) = {1}".format(type(str), repr(str)) validStrings += ((str,""),) except UnicodeEncodeError as ude: print "Couldn't print the str itself because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print ude try: x = unicode(str) print "unicode(str) = ",x validStrings+= ((x, " decoded into unicode by the default system encoding"),) except UnicodeDecodeError as ude: print "ERROR. unicode(str) couldn't decode the string because the system encoding is set to an encoding that doesn't understand some character in the string." print "\tThe system encoding is set to {0}. See error:\n\t".format(sys.getdefaultencoding()), print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the unicode(str) because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('latin_1') print "str.decode('latin_1') =",x validStrings+= ((x, " decoded with latin_1 into unicode"),) try: print "str.decode('latin_1').encode('utf_8') =",str.decode('latin_1').encode('utf_8') validStrings+= ((x, " decoded with latin_1 into unicode and encoded into utf_8"),) except UnicodeDecodeError as ude: print "The string was decoded into unicode using the latin_1 encoding, but couldn't be encoded into utf_8. See error:\n\t", print ude except UnicodeDecodeError as ude: print "Something didn't work, probably because the string wasn't latin_1 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('latin_1') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t", print uee try: x = str.decode('utf_8') print "str.decode('utf_8') =",x validStrings+= ((x, " decoded with utf_8 into unicode"),) try: print "str.decode('utf_8').encode('latin_1') =",str.decode('utf_8').encode('latin_1') except UnicodeDecodeError as ude: print "str.decode('utf_8').encode('latin_1') didn't work. The string was decoded into unicode using the utf_8 encoding, but couldn't be encoded into latin_1. See error:\n\t", validStrings+= ((x, " decoded with utf_8 into unicode and encoded into latin_1"),) print ude except UnicodeDecodeError as ude: print "str.decode('utf_8') didn't work, probably because the string wasn't utf_8 encoded. See error:\n\t", print ude except UnicodeEncodeError as uee: print "ERROR. Couldn't print the str.decode('utf_8') because the console is set to an encoding that doesn't understand some character in the string. See error:\n\t",uee print print "Printing information about each character in the original string." for char in str: try: print "\t'" + char + "' = original char {0} repr(char)={1}".format(type(char), repr(char)) except UnicodeDecodeError as ude: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), ude) except UnicodeEncodeError as uee: print "\t'?' = original char {0} repr(char)={1} ERROR PRINTING: {2}".format(type(char), repr(char), uee) print uee try: x = unicode(char) print "\t'" + x + "' = unicode(char) {1} repr(unicode(char))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = unicode(char) ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = unicode(char) {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('latin_1') print "\t'" + x + "' = char.decode('latin_1') {1} repr(char.decode('latin_1'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('latin_1') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('latin_1') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) try: x = char.decode('utf_8') print "\t'" + x + "' = char.decode('utf_8') {1} repr(char.decode('utf_8'))={2}".format(x, type(x), repr(x)) except UnicodeDecodeError as ude: print "\t'?' = char.decode('utf_8') ERROR: {0}".format(ude) except UnicodeEncodeError as uee: print "\t'?' = char.decode('utf_8') {0} repr(char)={1} ERROR PRINTING: {2}".format(type(x), repr(x), uee) print x = 'ó' encodingDemo(x) Much thanks for the answers below and especially to @John Machin for answering so thoroughly.

  Read the article

• Implementing Linked Lists in C#

  - by nijhawan.saurabh

  Why? The question is why you need Linked Lists and why it is the foundation of any Abstract Data Structure. Take any of the Data Structures - Stacks, Queues, Heaps, Trees; there are two ways to go about implementing them - Using Arrays Using Linked Lists Now you use Arrays when you know about the size of the Nodes in the list at Compile time and Linked Lists are helpful where you are free to add as many Nodes to the List as required at Runtime.   How? Now, let's see how we go about implementing a Simple Linked List in C#. Note: We'd be dealing with singly linked list for time being, there's also another version of linked lists - the Doubly Linked List which maintains two pointers (NEXT and BEFORE).   Class Diagram Let's see the Class Diagram first:     Code     1 // -----------------------------------------------------------------------     2 // <copyright file="Node.cs" company="">     3 // TODO: Update copyright text.     4 // </copyright>     5 // -----------------------------------------------------------------------     6      7 namespace CSharpAlgorithmsAndDS     8 {     9     using System;    10     using System.Collections.Generic;    11     using System.Linq;    12     using System.Text;    13     14     /// <summary>    15     /// TODO: Update summary.    16     /// </summary>    17     public class Node    18     {    19         public Object data { get; set; }    20     21         public Node Next { get; set; }    22     }    23 }    24         1 // -----------------------------------------------------------------------     2 // <copyright file="LinkedList.cs" company="">     3 // TODO: Update copyright text.     4 // </copyright>     5 // -----------------------------------------------------------------------     6      7 namespace CSharpAlgorithmsAndDS     8 {     9     using System;    10     using System.Collections.Generic;    11     using System.Linq;    12     using System.Text;    13     14     /// <summary>    15     /// TODO: Update summary.    16     /// </summary>    17     public class LinkedList    18     {    19         private Node Head;    20     21         public void AddNode(Node n)    22         {    23             n.Next = this.Head;    24             this.Head = n;    25     26         }    27     28         public void printNodes()    29         {    30     31             while (Head!=null)    32             {    33                 Console.WriteLine(Head.data);    34                 Head = Head.Next;    35     36             }    37     38         }    39     }    40 }    41          1 using System;     2 using System.Collections.Generic;     3 using System.Linq;     4 using System.Text;     5      6 namespace CSharpAlgorithmsAndDS     7 {     8     class Program     9     {    10         static void Main(string[] args)    11         {    12             LinkedList ll = new LinkedList();    13             Node A = new Node();    14             A.data = "A";    15     16             Node B = new Node();    17             B.data = "B";    18     19             Node C = new Node();    20             C.data = "C";    21             ll.AddNode(A);    22             ll.AddNode(B);    23             ll.AddNode(C);    24     25             ll.printNodes();    26         }    27     }    28 }    29        Final Words This is just a start, I will add more posts on Linked List covering more operations like Delete etc. and will also explore Doubly Linked List / Implementing Stacks/ Heaps/ Trees / Queues and what not using Linked Lists.   Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

  Read the article

• CodePlex Daily Summary for Friday, May 09, 2014

  CodePlex Daily Summary for Friday, May 09, 2014Popular ReleasesQuickMon: Version 3.9: First official release of the PowerShell script Collector. Corrective script can now also be PowerShell scripts! There are a couple of internal bugfixes to the core components as well. e.g. Overriding remote host setting now applies to ALL child collectors Main UI app now indicates (in Window title) if there are changes that needs to be saved. Polling frequency can be adjusted by 'slide bar' Note: If you have issues with the new PowerShell script collector please see my post about issu...51Degrees.mobi - Mobile Device Detection and Redirection: 2.1.21.4: One Click Install from NuGet Changes to Version 2.1.21.4The ReadStrings method of the binary reader will now convert strings straight from ASCII as they’re now being written in the data file as ASCII only. The ReadCollection method of the binary reader will now return arrays of integers rather than generic lists to reduce memory consumption. This change has required some public methods to use IList rather the List types. 3rd party source code that accesses the library via these methods may...Media Companion: Media Companion MC3.597b: Thank you for being patient, againThere are a number of fixes in place with this release. and some new features added. Most are self explanatory, so check out the options in Preferences. Couple of new Features:* Movie - Allow save Title and Sort Title in Title Case format. * Movie - Allow save fanart.jpg if movie in folder. * TV - display episode source. Get episode source from episode filename. Fixed:* Movie - Added Fill Tags from plot keywords to Batch Rescraper. * Movie - Fixed TMDB s...SimCityPak: SimCityPak 0.3.0.0: Contains several bugfixes, newly identified properties and some UI improvements. Main new features UI overhaul for the main index list: Icons for each different index, including icons for different property files Tooltips for all relevant fields Removed clutter Identified hundreds of additional properties (thanks to MaxisGuillaume) - this should make modding gameplay easierSeal Report: Seal Report 1.4: New Features: Report Designer: New option to convert a Report Source into a Repository Source. Report Designer: New contextual helper menus to select, remove, copy, prompt elements in a model. Web Server: New option to expand sub-folders displayed in the tree view. Web Server: Web Folder Description File can be a .cshtml file to display a MVC View. Views: additional CSS parameters for some DIVs. NVD3 Chart: Some default configuration values have been changed. Issues Addressed:16 ...Magick.NET: Magick.NET 6.8.9.002: Magick.NET linked with ImageMagick 6.8.9.0.VidCoder: 1.5.22 Beta: Added ability to burn SRT subtitles. Updated to HandBrake SVN 6169. Added checks to prevent VidCoder from running with a database version newer than it expects. Tooltips in the Advanced Video panel now trigger on the field labels as well as the fields themselves. Fixed updating preset/profile/tune/level settings on changing video encoder. This should resolve some problems with QSV encoding. Fixed tunes and profiles getting set to blank when switching between x264 and x265. Fixed co...NuGet: NuGet 2.8.2: We will be releasing a 2.8.2 version of our own NuGet packages and the NuGet.exe command-line tool. The 2.8.2 release will not include updated VS or WebMatrix extensions. NuGet.Server.Extensions.dll needs to be used alongside NuGet-Signed.exe to provide the NuGet.exe mirror functionality.babelua: 1.5.3: V1.5.3 - 2014.5.6Stability improvement: support relative path when debugging lua files; improve variable view function in debugger; fix a bug that when a file is loaded at runtime , its breakpoints would not take effect; some other bug fix; New feature: improve tool windows color scheme to look better with different vs themes; comment/uncomment code block easily; some other improve;SmartStore.NET - Free ASP.NET MVC Ecommerce Shopping Cart Solution: SmartStore.NET 2.0.2: SmartStore.NET 2.0.2 is primarily a maintenance release for version 2.0.0, which has been released on April 04 2014. It contains several improvements & important fixes. BugfixesIMPORTANT FIX: Memory leak leads to OutOfMemoryException in application after a while Installation fix: some varchar(MAX) columns get created as varchar(4000). Added a migration to fix the column specs. Installation fix: Setup fails with exception Value cannot be null. Parameter name: stream Bugfix for stock iss...Channel9's Absolute Beginner Series: Windows Phone 8.1: Entire source code for Windows Phone 8.1 Absolute Beginner Series.BIDS Helper: BIDS Helper 1.6.6: This BIDS Helper beta release brings support for SQL Server 2014 and SSDTBI for Visual Studio 2013. (Note that SSDTBI for Visual Studio 2013 is currently unavailable to download from Microsoft. We are releasing BIDS Helper support to help those who downloaded it before it became unavailable, and we will recheck BIDS Helper 2014 is compatible after SSDTBI becomes available to download again.) BIDS Helper 2014 Beta Limitations: SQL Server 2014 support for Biml is still in progress, so this bet...Windows Phone IsoStoreSpy (a cool WP8.1 + WP8 Isolated Storage Explorer): IsoStoreSpy WP8.1 3.0.0.0 (Win8 only): 3.0.0.0 + WP8.1 & WP8 device allowed + Local, Roaming or Temp directory Selector for WindowsRuntime apps + Version number in the title :)CS-Script for Notepad++ (C# intellisense and code execution): Release v1.0.24.0: ShortcutMapping panel now allows direct modification of the shortcuts. Custom updater dropped support for MSI installation but it still allows downloading of MSI http://www.csscript.net/npp/codeplex/non_msi_update.pngProDinner - ASP.NET MVC Sample (EF5, N-Tier, jQuery): 8.2: upgrade to mvc 5 upgrade to ASP.net MVC Awesome 4.0, EF 6.1, latest jcrop, Windsor Castle etc. ability to show/hide the dog using tinymce for the feedback page mobile friendly uiASP.net MVC Awesome - jQuery Ajax Helpers: 4.0: version 4.0 ========================== - added InitPopup, InitPopupForm helpers, open initialized popups using awe.open(name, params) - added Tag or Tags for all helpers, usable in mod code - added popup aweclose, aweresize events - popups have api accessible via .data('api'), with methods open, close and destroy - all popups have .awe-popup class - popups can be grouped using .Group(string), only 1 popup in the same group can exist at the same time - added awebeginload event for...SEToolbox: SEToolbox 01.028.008 Release 2: Fixes for drag-drop copying. Added ship joiner, to rejoin a broken ship! Installation of this version will replace older version.ScreenToGif: Release 1.0: What's new: • Small UI tweaks everywhere. • You can add Text, Subtitles and Title Frames. • Processing the gif now takes less time. • Languages: Spanish, Italian and Tamil added. • Single .exe multi language. • Takes less time to apply blur and pixelated efect. • Restart button. • Language picker. (If the user wants to select a diferent language than the system's) • "Encoding Finished" page with a link to open the file. • GreenScreen unchanged pixels to save kilobytes. • "Check for Updates" t...Touchmote: Touchmote 1.0 beta 11: Changes Support for multiple monitor setup Additional settings for analog sticks More reliable pairing Bug fixes and improvementsPowerShell App Deployment Toolkit: PowerShell App Deployment Toolkit v3.1.2: Added Get-IniValue / Set-IniValue functions. Replaces Get-IniContent / Set-IniContent which were a bit unwieldy. The new functions are much easier to use. Added -Value parameter to Get-RegistryKey so that a specific registry value can be retrieved instead of the entire Key as an object Fixed Test-Battery to work with machines that can have multiple batteries Fixed issue where BlockExecution would fail if a Custom Temp Path was specified in the config file Updated examples with latest ...New ProjectsAgriComer: Este proyecto es creado con el objetivo inicial de ofrecer una herramienta integral para el comercio agrícola.AST - a tool for exploring windows kernel: A project for exploring windows kernel. It is consist of a Winform for presentation(by C#), and a windows driver(by C). VS2013 + WDK8.1 are required.CondimarSW: Sistema de ventasDeco Pattern: This project shows how the deco design pattern works.Geïntegreerd Project GH5: Geïntegreerd project .NET voor groep GH5Groep_EF4: PXL Groep EF4HttpRequestor: HttpRequestor is a nifty little tool for web developers, developed using Windows Forms, to make different types of HTTP requests to a web endpoint. IBehaviorPortable: This project has a single goal: to make it possible to write behaviors in a PCLKalkulator: Unit Test dan Coded UI TestLorenOS: LorenOS, an operating system based on MikeOSModernUI Extend: ModernUI ExtendPetite Scheme auto-execute pipeline: This program was originally created to avoid the steep learning curve of Emacs, and allow any editor to be used with ease for Chez Petite Scheme.projectgroep3.net: .net project for schoolReadable Athens (Hold-Cold Interactive Map): Demo application for Readable Athens art project Pins JPGs on map (using GPS EXIF info), shows at pin hover, louder noise closer to more text (*.jpg.txt OCRed)Training_Network_Messenger: Training network messengerTsLinq: This is a linq like implementation for typescript. Because typescript gets compiled into javascript, you can use this library for common javascript too.TSQL Smells SSDT: TSQL 'smells' finders TSQL Code can smell, it may work just fine but there can be hidden dangers held within. ??????-??????【??】????????: ?????????????????????????????,??????????,????,????,?????????、??????,??????。??????-??????【??】????????: ????????????????,?????????????? ??。????????、????、????、?????????? ???????。????-????【??】??????: ??????,??????:?????,?????,??????,??????????,????????。????????!?????-?????【??】???????: ?????????????????????????,????,????,??????????。???????????????,??,??,??????????,??????...?????-?????【??】?????????: ??????????????????????:????、????、??????????????,????????。????????!???????-???????【??】?????????: ???????????????????,???????????,??????????????,??????????，??????????????！???????-???????????: ????????????????,???????????????。???????????,??????:????、????、???????！?????-?????【??】???????: ??????????????????,?????????????,????,??????????,?????????????,?????,?????!??????-??????【??】??????????: ??????????????、????、????、??????、????、????????,????????、?????????,?????。??????-??????【??】??????????: ?????????????????,??????????????、???????、???????、???????、?????！??????-??????【??】??????????: ?????????????????,?????????????。?????????????????,???????,???????,?????,?????。????-????【??】??????: ???? ???? ???? ???? ???? ???? ???? ??????? ?????，????????????????.????-????【??】????????: ????????????????????????,???????????,????????,?????????????????????。??????-??????【??】????????: ????????????????,????,????,??????,????“????、????、????、????”????????,??????.??????-??????【??】??????????: ??????????????????????,????“???????,???????”?????,????????????！?????-?????【??】???????: ????????????,?????,???????????,???????,????,????,????,?????。?????-?????【??】?????????: ???????????????????,????????:??、??、???,?????????????????????!?????-?????【??】???????: ???????????????????,????????????????,????????????????,??????！??????-??????【??】????????: ????????????????????,??????????,????????、????,??????????,??????????。??????-??????【??】??????????: ????????????????、????,??100%????,??????,????????????,???????????!??????-??????【??】??????????: ????????????????6?,???????????????????????????,??????????????,?????????！????-????【??】????????: ??????????、???、??、??????????????????????????????,????????????????! ????-????【??】????????: ???????????????????????????、????、????、???????????,????,????!?????-?????【??】???????: ???????????,????,??????????? ???? ???? ?????????,???,??,?????!

  Read the article

• Difference between Factory Method and Abstract Factory design patterns using C#.Net

  - by nijhawan.saurabh

  First of all I'll just put both these patterns in context and describe their intent as in the GOF book: Factory Method: Define an interface for creating an object, but let subclasses decide which class to instantiate. Factory Method lets a class defer instantiation to subclasses.   Abstract Factory: Provide an interface for creating families of related or dependent objects without specifying their concrete classes.   Points to note:   Abstract factory pattern adds a layer of abstraction to the factory method pattern. The type of factory is not known to the client at compile time, this information is passed to the client at runtime (How it is passed is again dependent on the system, you may store this information in configuration files and the client can read it on execution). While implementing Abstract factory pattern, the factory classes can have multiple factory methods. In Abstract factory, a factory is capable of creating more than one type of product (Simpilar products are grouped together in a factory)   Sample implementation of factory method pattern   Let's see the class diagram first:                   ProductFactory.cs // ----------------------------------------------------------------------- // <copyright file="ProductFactory.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public abstract class ProductFactory     {         /// <summary>         /// </summary>         /// <returns>         /// </returns>         public abstract Product CreateProductInstance();     } }     ProductAFactory.cs // ----------------------------------------------------------------------- // <copyright file="ProductAFactory.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public class ProductAFactory:ProductFactory     {         public override Product CreateProductInstance()         {             return new ProductA();         }     } }         // ----------------------------------------------------------------------- // <copyright file="ProductBFactory.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public class ProductBFactory:ProductFactory     {         public override Product CreateProductInstance()         {             return new ProductB();           }     } }     // ----------------------------------------------------------------------- // <copyright file="Product.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public abstract class Product     {         public abstract string Name { get; set; }     } }     // ----------------------------------------------------------------------- // <copyright file="ProductA.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public class ProductA:Product     {         public ProductA()         {               Name = "ProductA";         }           public override string Name { get; set; }     } }       // ----------------------------------------------------------------------- // <copyright file="ProductB.cs" company=""> // TODO: Update copyright text. // </copyright> // -----------------------------------------------------------------------   namespace FactoryMethod {     using System;     using System.Collections.Generic;     using System.Linq;     using System.Text;       /// <summary>     /// TODO: Update summary.     /// </summary>     public class ProductB:Product     {          public ProductB()         {               Name = "ProductA";         }         public override string Name { get; set; }     } }     Program.cs using System; using System.Collections.Generic; using System.Linq; using System.Text;   namespace FactoryMethod {     class Program     {         static void Main(string[] args)         {             ProductFactory pf = new ProductAFactory();               Product product = pf.CreateProductInstance();             Console.WriteLine(product.Name);         }     } }       Normal 0 false false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

  Read the article

• How to Load Oracle Tables From Hadoop Tutorial (Part 5 - Leveraging Parallelism in OSCH)

  - by Bob Hanckel

  Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 Using OSCH: Beyond Hello World In the previous post we discussed a “Hello World” example for OSCH focusing on the mechanics of getting a toy end-to-end example working. In this post we are going to talk about how to make it work for big data loads. We will explain how to optimize an OSCH external table for load, paying particular attention to Oracle’s DOP (degree of parallelism), the number of external table location files we use, and the number of HDFS files that make up the payload. We will provide some rules that serve as best practices when using OSCH. The assumption is that you have read the previous post and have some end to end OSCH external tables working and now you want to ramp up the size of the loads. Using OSCH External Tables for Access and Loading OSCH external tables are no different from any other Oracle external tables.  They can be used to access HDFS content using Oracle SQL: SELECT * FROM my_hdfs_external_table; or use the same SQL access to load a table in Oracle. INSERT INTO my_oracle_table SELECT * FROM my_hdfs_external_table; To speed up the load time, you will want to control the degree of parallelism (i.e. DOP) and add two SQL hints. ALTER SESSION FORCE PARALLEL DML PARALLEL  8; ALTER SESSION FORCE PARALLEL QUERY PARALLEL 8; INSERT /*+ append pq_distribute(my_oracle_table, none) */ INTO my_oracle_table SELECT * FROM my_hdfs_external_table; There are various ways of either hinting at what level of DOP you want to use.  The ALTER SESSION statements above force the issue assuming you (the user of the session) are allowed to assert the DOP (more on that in the next section).  Alternatively you could embed additional parallel hints directly into the INSERT and SELECT clause respectively. /*+ parallel(my_oracle_table,8) *//*+ parallel(my_hdfs_external_table,8) */ Note that the "append" hint lets you load a target table by reserving space above a given "high watermark" in storage and uses Direct Path load.  In other doesn't try to fill blocks that are already allocated and partially filled. It uses unallocated blocks.  It is an optimized way of loading a table without incurring the typical resource overhead associated with run-of-the-mill inserts.  The "pq_distribute" hint in this context unifies the INSERT and SELECT operators to make data flow during a load more efficient. Finally your target Oracle table should be defined with "NOLOGGING" and "PARALLEL" attributes.   The combination of the "NOLOGGING" and use of the "append" hint disables REDO logging, and its overhead.  The "PARALLEL" clause tells Oracle to try to use parallel execution when operating on the target table. Determine Your DOP It might feel natural to build your datasets in Hadoop, then afterwards figure out how to tune the OSCH external table definition, but you should start backwards. You should focus on Oracle database, specifically the DOP you want to use when loading (or accessing) HDFS content using external tables. The DOP in Oracle controls how many PQ slaves are launched in parallel when executing an external table. Typically the DOP is something you want to Oracle to control transparently, but for loading content from Hadoop with OSCH, it's something that you will want to control. Oracle computes the maximum DOP that can be used by an Oracle user. The maximum value that can be assigned is an integer value typically equal to the number of CPUs on your Oracle instances, times the number of cores per CPU, times the number of Oracle instances. For example, suppose you have a RAC environment with 2 Oracle instances. And suppose that each system has 2 CPUs with 32 cores. The maximum DOP would be 128 (i.e. 2*2*32). In point of fact if you are running on a production system, the maximum DOP you are allowed to use will be restricted by the Oracle DBA. This is because using a system maximum DOP can subsume all system resources on Oracle and starve anything else that is executing. Obviously on a production system where resources need to be shared 24x7, this can’t be allowed to happen. The use cases for being able to run OSCH with a maximum DOP are when you have exclusive access to all the resources on an Oracle system. This can be in situations when your are first seeding tables in a new Oracle database, or there is a time where normal activity in the production database can be safely taken off-line for a few hours to free up resources for a big incremental load. Using OSCH on high end machines (specifically Oracle Exadata and Oracle BDA cabled with Infiniband), this mode of operation can load up to 15TB per hour. The bottom line is that you should first figure out what DOP you will be allowed to run with by talking to the DBAs who manage the production system. You then use that number to derive the number of location files, and (optionally) the number of HDFS data files that you want to generate, assuming that is flexible. Rule 1: Find out the maximum DOP you will be allowed to use with OSCH on the target Oracle system Determining the Number of Location Files Let’s assume that the DBA told you that your maximum DOP was 8. You want the number of location files in your external table to be big enough to utilize all 8 PQ slaves, and you want them to represent equally balanced workloads. Remember location files in OSCH are metadata lists of HDFS files and are created using OSCH’s External Table tool. They also represent the workload size given to an individual Oracle PQ slave (i.e. a PQ slave is given one location file to process at a time, and only it will process the contents of the location file.) Rule 2: The size of the workload of a single location file (and the PQ slave that processes it) is the sum of the content size of the HDFS files it lists For example, if a location file lists 5 HDFS files which are each 100GB in size, the workload size for that location file is 500GB. The number of location files that you generate is something you control by providing a number as input to OSCH’s External Table tool. Rule 3: The number of location files chosen should be a small multiple of the DOP Each location file represents one workload for one PQ slave. So the goal is to keep all slaves busy and try to give them equivalent workloads. Obviously if you run with a DOP of 8 but have 5 location files, only five PQ slaves will have something to do and the other three will have nothing to do and will quietly exit. If you run with 9 location files, then the PQ slaves will pick up the first 8 location files, and assuming they have equal work loads, will finish up about the same time. But the first PQ slave to finish its job will then be rescheduled to process the ninth location file, potentially doubling the end to end processing time. So for this DOP using 8, 16, or 32 location files would be a good idea. Determining the Number of HDFS Files Let’s start with the next rule and then explain it: Rule 4: The number of HDFS files should try to be a multiple of the number of location files and try to be relatively the same size In our running example, the DOP is 8. This means that the number of location files should be a small multiple of 8. Remember that each location file represents a list of unique HDFS files to load, and that the sum of the files listed in each location file is a workload for one Oracle PQ slave. The OSCH External Table tool will look in an HDFS directory for a set of HDFS files to load.  It will generate N number of location files (where N is the value you gave to the tool). It will then try to divvy up the HDFS files and do its best to make sure the workload across location files is as balanced as possible. (The tool uses a greedy algorithm that grabs the biggest HDFS file and delegates it to a particular location file. It then looks for the next biggest file and puts in some other location file, and so on). The tools ability to balance is reduced if HDFS file sizes are grossly out of balance or are too few. For example suppose my DOP is 8 and the number of location files is 8. Suppose I have only 8 HDFS files, where one file is 900GB and the others are 100GB. When the tool tries to balance the load it will be forced to put the singleton 900GB into one location file, and put each of the 100GB files in the 7 remaining location files. The load balance skew is 9 to 1. One PQ slave will be working overtime, while the slacker PQ slaves are off enjoying happy hour. If however the total payload (1600 GB) were broken up into smaller HDFS files, the OSCH External Table tool would have an easier time generating a list where each workload for each location file is relatively the same.  Applying Rule 4 above to our DOP of 8, we could divide the workload into160 files that were approximately 10 GB in size.  For this scenario the OSCH External Table tool would populate each location file with 20 HDFS file references, and all location files would have similar workloads (approximately 200GB per location file.) As a rule, when the OSCH External Table tool has to deal with more and smaller files it will be able to create more balanced loads. How small should HDFS files get? Not so small that the HDFS open and close file overhead starts having a substantial impact. For our performance test system (Exadata/BDA with Infiniband), I compared three OSCH loads of 1 TiB. One load had 128 HDFS files living in 64 location files where each HDFS file was about 8GB. I then did the same load with 12800 files where each HDFS file was about 80MB size. The end to end load time was virtually the same. However when I got ridiculously small (i.e. 128000 files at about 8MB per file), it started to make an impact and slow down the load time. What happens if you break rules 3 or 4 above? Nothing draconian, everything will still function. You just won’t be taking full advantage of the generous DOP that was allocated to you by your friendly DBA. The key point of the rules articulated above is this: if you know that HDFS content is ultimately going to be loaded into Oracle using OSCH, it makes sense to chop them up into the right number of files roughly the same size, derived from the DOP that you expect to use for loading. Next Steps So far we have talked about OLH and OSCH as alternative models for loading. That’s not quite the whole story. They can be used together in a way that provides for more efficient OSCH loads and allows one to be more flexible about scheduling on a Hadoop cluster and an Oracle Database to perform load operations. The next lesson will talk about Oracle Data Pump files generated by OLH, and loaded using OSCH. It will also outline the pros and cons of using various load methods.  This will be followed up with a final tutorial lesson focusing on how to optimize OLH and OSCH for use on Oracle's engineered systems: specifically Exadata and the BDA. /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin;}

  Read the article

• Print raw data to a thermal-printer using c#

  - by blauesocke

  Hi, I'm trying to print out raw ascii data to a thermal printer. I do this by using this code example: http://support.microsoft.com/kb/322091 but my printer prints always only one character and this not until I press the form feed button. If I print something with notepad the printer will do a form feed automatically but without printing any text. The printer is connected via usb over a lpt2usb adapter and Windows 7 uses the "Generic - Generic / Text Only" driver. Anyone knows what is going wrong? How is it possible to print some words and do some form feeds? Are there some control characters I have to send? And if yes: How do I send them? Thanks!

  Read the article

• ASP.NET MVC FileContentResult IE 8.0 hangs on download

  - by marc.d

  some of my users are expieriencing problems when they try to download a report, the download just hangs on 0%, restarting IE usually fixes the problem. why does this happen? i am using ASP.NET MVC (v1), the my action looks like this <Authorize()> _ <AcceptVerbs(HttpVerbs.Get)> _ Function RenderReport(ByVal guid As Guid, ByVal anonym As Boolean) As FileContentResult ... Dim mimeType As String = String.Empty Dim renderedBytes() As Byte = EmployeePresentation.Render(guid, mimeType, Server.MapPath("~/Reports/..."), anonym) Return File(renderedBytes, mimeType, filename) End Function the filename is US-ASCII encoded, filesize is usally around 300Kb, mimeType is application/pdf tia

  Read the article

• WebClient The request was aborted: Could not create SSL/TLS secure channel

  - by Tomas

  I am using WebClient in ASP.NET app to call PayPal secured url to create payment button. While calling secured PayPal Url I get error below. How to solve this problem? Do I need to purchase certificate to just call secured url? The request was aborted: Could not create SSL/TLS secure channel. My code ServicePointManager.Expect100Continue = true; ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3; using (var client = new WebClient()) { var postBytes = Encoding.ASCII.GetBytes(param); client.Headers.Add("Content-Type", "application/x-www-form-urlencoded"); responseBytes = client.UploadData(_paymentProcessorCredentials.PayPalApiUrl, "POST", postBytes); }

  Read the article

• Objective C HTML escape/unescape

  - by Squeegy

  Wondering if there is an easy way to do a simple HTML escape/unescape in Objective C. What I want is something like this psuedo code: NSString *string = @"&lt;span&gt;Foo&lt;/span&gt;"; [string stringByUnescapingHTML]; Which returns <span>Foo</span> Hopefully unescaping all other HTML entities as well and even ASCII codes like Ӓ and the like. Is there any methods in Cocoa Touch/UIKit to do this?

  Read the article

• Powershell replace lose line breaks

  - by Fadrian Sudaman

  Hi, I am a newbie in powershell. I have a simple powershell script that just replace text but I found that the regex replace turn my multiline data source into a single line text when the output is produced. I want the line breaks to be preserved. Here is the dumb down version of the script. $source=(Get-Content textfile.txt) $process1 = [regex]::Replace($source, "line", "line2") $process1 | out-file -encoding ascii textfile2.txt You can create a test file call textfile.txt with simple lines like this to test it line line Some line More line here Have I missed something obvious? Thanks, Fadrian

  Read the article

• pysqlite2: ProgrammingError - You must not use 8-bit bytestrings

  - by rfkrocktk

  I'm currently persisting filenames in a sqlite database for my own purposes. Whenever I try to insert a file that has a special character (like é etc.), it throws the following error: pysqlite2.dbapi2.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. When I do "switch my application over to Unicode strings" by wrapping the value sent to pysqlite with the unicode method like: unicode(filename), it throws this error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 66: ordinal not in range(128) Is there something I can do to get rid of this? Modifying all of my files to conform isn't an option.

  Read the article

• IText Lines Per Page?

  - by exit174

  I have used IText(Sharp) to make PDFs from raster images but have never used IText to build text based PDF's. In this new scenario, I have N words of ASCII text that I need to add as text pages to a new PDF. Each page should be 8.5 x 11, .5 margins, using some monospaced font. Is IText(Sharp) smart enough to automatically apply line and page breaks where necessary or do I need to do this manually? Are their any good IText tutorials for this scenario? thanks! scottm

  Read the article

• Rendering formatted text in a direct3d application

  - by Fire Lancer

  I need to render some formatted text (colours, different font sizes, underlines, bold, etc) however I'm not sure how to go about doing it. D3DXFont only allows text of a single font/size/weight/colour/etc to be rendered at once, and I cant see a practical way to "combine" multiple calls to ID3DXFont::DrawText to do such things... I looked around and there doesn't seem to be any existing libraries that do these things, but I have no idea how to implement such a text renderer, and I couldn't even find any documentation on how such a text render would work, only rendering simple fixed width, ASCII bitmap fonts which looking at it is probably an entirely different approach that is only suitable for rendering simple blocks of text where Unicode is not important. If there's no direct3d font renders capable of doing this, is there any other renderers (eg for use in rendering rich text in a normal window), and would rendering those to a texture in RAM, then uploading that to the video card to render onto the back buffer yield reasonable performance?

  Read the article

• Python decoding issue with hashlib.digest() method

  - by Sorw

  Hello StackOverflow community, Using Google App Engine, I wrote a keyToSha256() method within a model class (extending db.Model) : class Car(db.Model): def keyToSha256(self): keyhash = hashlib.sha256(str(self.key())).digest() return keyhash When displaying the output (ultimately within a Django template), I get garbled text, for example : ?????_??!`?I?!?;?QeqN??Al?'2 I was expecting something more in line with this : 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 Am I missing something important ? Despite reading several guides on ASCII, Unicode, utf-8 and the like, I think I'm still far from mastering the secrets of string encoding/decoding. After browsing StackOverflow and searching for insights via Google, I figured out I should ask the question here. Any idea ? Thanks !

  Read the article

• Ruby Iconv works with irb and ruby debugger but not in a unit test

  - by Mark B

  I'm running Ruby 1.8.7 with Rails 2.3.5 on Ubuntu 10.04 64-bit. I've written a method that should take a string like this, "École À la Découverte" and output a file-system name like this "ecole_a_la_decouverte": (Iconv.new('US-ASCII//TRANSLIT', 'utf-8').iconv "École À la Découverte").gsub(/[^\w\s-\—]/,'').gsub(/[^\w]|[_]/,' ').split.join('_').downcase When I test this line in my code, the test always fails saying that "cole_la_dcouverte" is unequal to "ecole_a_la_decouverte". The odd thing is that if I insert a debugger line and use the debugger console the test passes. As well, running this line manually in irb seems to work. Does anyone know what's going on and why this test is failing? My only thought is that including the debugger or irb somehow adds more support for UTF-8 but I'm at a loss to figure out where to go next. Thanks in advance!

  Read the article

• How can I Unescape and Reescape strings in .net?

  - by firoso

  So here's my situation. I am working on an editor for a communications channel that works over an RS232 serial ASCII terminal. Let's not go into detail for that ;-) To simplify, I need a textbox on a WPF control that can take in text like "Commit\r\n" (which is the .net string "Commit\r\n") and convert it back to "Commit\r\n" as a .net string. I was hoping for a string.Unescape() and string.Escape() method pair, but it doesn't seem to exist. Am I going to have to write my own? or is there a more simple way to do this?

  Read the article

• Needed list of special characters classification with respective characters

  - by pravin

  I am working on one web application , It's related to machine translation support i.e. which takes source text for translation and translated in to user specified language Currently it's in unit testing phase. Here, i want to check that, whether my machine translation feature is fully working for all the special characters. Because of different test cases I stuck at one point where i need all the special characters with classification. I needed all the special characters listing with classification. e.g. 1st : class name : Punctuation Characters : !?,"| etc test cases : segment1? segment2! segment3. 2nd : Class name : HTML entities characters : all the characters which belong under this class test cases : respective test cases 3rd : Class name : Extended ASCII characters :all the characters which belong under this class test cases : respective test cases Please folks provide this, if anyone has any idea or links so that i can make product perfect Thanks a lot

  Read the article

• Forward all traffic through an ssh tunnel

  - by Eamorr

  I hope someone can follow this and I'll explain as best I can. I'm trying to forward all traffic from port 6999 on x.x.x.224, through an ssh tunnel, and onto port 7000 on x.x.x.218. Here is some ASCII art: |browser|-----|Squid on x.x.x.224|------|ssh tunnel|------<satellite link>-----|Squid on x.x.x.218|-----|www| 3128 6999 7000 80 When I remove the ssh tunnel, everything works fine. The idea is to turn off encryption on the ssh tunnel (to save bandwidth) and turn on maximum compression (to save more bandwidth). This is because it's a satellite link. Here's the ssh tunnel I've been using: ssh -C -f -C -o CompressionLevel=9 -o Cipher=none [email protected] -L 7000:172.16.1.224:6999 -N The trouble is, I don't know how to get data from Squid on x.x.x.224 into the ssh tunnel? Am I going about this the wrong way? Should I create an ssh tunnel on x.x.x.218? I use iptables to stop squid on x.x.x.224 from reading port 80, but to feed from port 6999 instead (i.e. via the ssh tunnel). Do I need another iptables rule? Any comments greatly appreciated. Many thanks in advance,

  Read the article

• Type attribute for text_field

  - by Koning Baard XIV

  I have this code: <%= f.text_field :email, :type => "email", :placeholder => "[email protected]" %> So people can enter their email on an iPhone with the email keyboard instead of the ASCII keyboard. However, the output is: <input id="user_email" name="user[email]" placeholder="[email protected]" size="30" type="text" /> which should be: <input id="user_email" name="user[email]" placeholder="[email protected]" size="30" type="email" /> Is there a way to force Rails to use the email type instead of text, or must I use HTML directly? Thanks

  Read the article

• while downloading filenames from non english languages are not getting displayed on the downloaded f

  - by pks83

  When i am trying to download a file whose name has characters from languages like chinese japanese etc...... non ascii... the downloaded file name is garbled. How to rectify it. I have tried to put charset=UTF-8 in the Content-type header property, but no success. Please help. Code below. header("Cache-Control: ");// leave blank to avoid IE errors header("Pragma: ");// leave blank to avoid IE errors header("Content-type: application/octet-stream"); header("Content-Disposition: attachment; filename=\"".$instance_name."\""); header("Content-length:".(string)(filesize($fileString))); sleep(1); fpassthru($fdl);

  Read the article

• Python: How would i write this 'if' statement for a word of arbitrary length?

  - by ElCarlos

  This is what I currently have: wordlist = [fox, aced, definite, ace] for word in wordlist: a = len(word) if (ord(word[a-(a-1)] - ord(word[(a-a)])) == ord(word[a-(a-2)])-ord(word[a-(a-1)]: print "success", word else: print "fail", word What I'm trying to do is calculate the ASCII values between each of the letters in the word. And check to see if the ord of the letters are increasing by the same value. so for fox, it would check if the difference between the ord of 2nd and 1st letters are equal to the ord difference of the 3rd and 2nd letters. However, with my current 'if' statement, only the first 3 letters of a word are compared. How can I rewrite this statement to cover every letter in a word of length greater than 3? Sorry if I can't present this clearly, thanks for your time.

  Read the article

< Previous Page | 71 72 73 74 75 76 77 78 79 80 81 82  | Next Page >