Search Results

Search found 1649 results on 66 pages for 'unicode normalization'.

Page 27/66 | < Previous Page | 23 24 25 26 27 28 29 30 31 32 33 34  | Next Page >

• Most Lite-Weight XML Parser with XPath and Wide-char Support

  - by Mystagogue

  I want a lite-weight C++ XML parser/DOM that: Can take UTF-8 as input, and parse into UTF-16. Maybe it does this directly (ideal!), or perhaps it provides a hook for the conversion (such as taking a custom stream object that does the conversion before parsing). Offers some XPath support. I've been looking at RapidXML, the Kranf xmlParser, and pugiXML. The first two of those might permit requirement #1 by way of a hook. The third, pugiXML, supports the #2 requirement. But none of those three fulfill both requirements. What is the smallest (free) library that can handle both requirements?

  Read the article

• Convert UTF-16 to UTF-8 under Windows and Linux, in C

  - by DooriBar

  I was wondering if there is a recommended 'cross' Windows and Linux method for the purpose of converting strings from UTF-16LE to UTF-8? or one should use different methods for each environment? I've managed to google few references to 'iconv' , but for somreason I can't find samples of basic conversions, such as - converting a wchar_t UTF-16 to UTF-8. Anybody can recommend a method that would be 'cross', and if you know of references or a guide with samples, would very appreciate it. Thanks, Doori Bar

  Read the article

• Culture Sensitive GetHashCode

  - by user114928

  Hi, I'm writing a c# application that will process some text and provide basic query functions. In order to ensure the best possible support for other languages, I am allowing the users of the application to specify the System.Globalization.CultureInfo (via the "en-GB" style code) and also the full range of collation options using the System.Globalization.CompareOptions flags enum. For regular string comparison I'm then using a combination of: a) String.Compare overload that accepts the culture and options b) For some bulk processes I'm caching the byte data (KeyData) from CompareInfo.GetSortKey (overload that accepts the options) and using a byte-by-byte comparison of the KeyData. This seemed fine (although please comment if you think these two methods shouldn't be mixed), but then I had reason to use the HashSet< class which only has an overload for IEqualityComparer<. MS documentation seems to suggest that I should use StringComparer (which implements both IEqualityComparer< and IComparer<), but this only seems to support the "IgnoreCase" option from CompareOptions and not "IgnoreKanaType", "IgnoreSymbols", "IgnoreWidth" etc. I'm assuming that a StringComparer that ignores these other options could produce different hashcodes for two strings that might be considered the same using my other comparison options. I'd therefore get incorrect results from my application. Only thought at the moment is to create my own IEqualityComparer< that generates a hashcode from the SortKey.KeyData and compares eqality be using the String.Compare overload. Any suggestions?

  Read the article

• Weird SQL Server 2005 Collation difference between varchar() and nvarchar()

  - by richardtallent

  Can someone please explain this: SELECT CASE WHEN CAST('iX' AS nvarchar(20)) > CAST('-X' AS nvarchar(20)) THEN 1 ELSE 0 END, CASE WHEN CAST('iX' AS varchar(20)) > CAST('-X' AS varchar(20)) THEN 1 ELSE 0 END Results: 0 1 SELECT CASE WHEN CAST('i' AS nvarchar(20)) > CAST('-' AS nvarchar(20)) THEN 1 ELSE 0 END, CASE WHEN CAST('i' AS varchar(20)) > CAST('-' AS varchar(20)) THEN 1 ELSE 0 END Results: 1 1 On the first query, the nvarchar() result is not what I'm expecting, and yet removing the X make the nvarchar() sort happen as expected. (My original queries used the '' and N'' literal syntax to distinguish varchar() and nvarchar() rather than CAST() and got the same result.) Collation setting for the database is SQL_Latin1_General_CP1_CI_AS.

  Read the article

• What's the difference between utf8_general_ci and utf8_unicode_ci

  - by KahWee

  Between utf8_general_ci and utf8_unicode_ci, are there any differences in terms of performance?

  Read the article

• Emailing HTML from within an iPhone app is stopping at special characters

  - by user141146

  Hi, I have an iPhone app that will let users email some pre-determined text as HTML. I'm having a problem in that if the text contains special characters within the text (e.g., ampersand &, , <), the NSString variable that I use for sending the body of the email gets truncated at the special character. I'm not sure how to fix this (I tried using the method stringByAddingPercentEscapesUsingEncoding…but this hasn't fixed the problems). Thoughts on what I'm doing wrong / how to fix it? Here is sample code showing what I'm trying to do Thanks!!! - (void)send_an_email:(id)sender { NSString *subject_string = [NSString stringWithFormat:@"Summary of %@", commercial_name]; NSString *body_string = [NSString stringWithFormat:@"%@<br /><br />", [self.dl email_message]]; // email_message returns the body of text that should be shipped as html. If email_message contains special characters, the text truncates at the special character NSString *full_string = [NSString stringWithFormat:@"mailto:?to=&subject=%@&body=%@", [subject_string stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding], [body_string stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]]; [[UIApplication sharedApplication] openURL:[[NSURL alloc] initWithString:full_string]]; }

  Read the article

• Using Markov models to convert all caps to mixed case and related problems

  - by hippietrail

  I've been thinking about using Markov techniques to restore missing information to natural language text. Restore mixed case to text in all caps Restore accents / diacritics to languages which should have them but have been converted to plain ASCII Convert rough phonetic transcriptions back into native alphabets That seems to be in order of least difficult to most difficult. Basically the problem is resolving ambiguities based on context. I can use Wiktionary as a dictionary and Wikipedia as a corpus using n-grams and Markov chains to resolve the ambiguities. Am I on the right track? Are there already some services, libraries, or tools for this sort of thing? Examples GEORGE LOST HIS SIM CARD IN THE BUSH - George lost his SIM card in the bush tantot il rit a gorge deployee - tantôt il rit à gorge déployée

  Read the article

• Comparing ICU sort keys (collator_get_sort_key) in PHP

  - by fsb

  Is strcmp() appropriate for comparing ICU collator sort keys in PHP? The sort keys I'm asking about are from collator_get_sort_key() which are described in ICU documentation.

  Read the article

• How to check if the word is Japanese or English using PHP

  - by bn

  I want to have different process for English word and Japanese word in this function function process_word($word) { if($word is english) { ///////// }else if($word is japanese) { //////// } } thank you

  Read the article

• How to send parameters with same encoding from javascript?

  - by nimcap

  I have a javascript file that lots of people have embedded to their pages. Since I am hosting the file, I have control over that javascript file; I cannot control the way it is embedded because lots of people is using it already. This javascript file sends GET requests to my servlets, and the parameters passed with the request are recorded to DB. For example, javascript sends a request to http://myserver.com/servlet?p1=123&p2=aString and then servlet records 123 and aString to DB somehow. Before sending strings I use encodeURIComponent() to encode it. But what I figured out is every client sends the same string with different encodings depending on either their browser or the site they are visiting. As a result, same strings are represented with different characters when it reaches servlet (so they are different strings). What I am trying to do is to convert the strings to one kind of encoding from javascript so when they reach the client same words are represented with same characters. How is this possible? PS. If there is a way to convert the encoding from Java it is also applicable.

  Read the article

• reading non-english html pages with c#

  - by Gal Miller

  I am trying to find a string in Hebrew in a website. The reading code is attached. Afterward I try to read the file using streamReader but I can't match strings in other languages. what am I suppose to do? // used on each read operation byte[] buf = new byte[8192]; // prepare the web page we will be asking for HttpWebRequest request = (HttpWebRequest) WebRequest.Create("http://www.webPage.co.il"); // execute the request HttpWebResponse response = (HttpWebResponse) request.GetResponse(); // we will read data via the response stream Stream resStream = response.GetResponseStream(); string tempString = null; int count = 0; FileStream fileDump = new FileStream(@"c:\dump.txt", FileMode.Create); do { count = resStream.Read(buf, 0, buf.Length); fileDump.Write(buf, 0, buf.Length); } while (count > 0); // any more data to read? fileDump.Close();

  Read the article

• In MySQL how can I tell what character set a particular table is using?

  - by muudscope

  I have a large mysql table that I think might be using the wrong character set. If so I'll need to change it using ALTER TABLE mytable CONVERT TO CHARACTER SET utf8 But since this is a very large table, I'd rather not run this command unless I have to. So my question is, how can I ask mysql what the character set is on a particular table? I can call status in mysql to see the database's character set, but that doesn't necessarily mean all the tables have the same character set, right?

  Read the article

• Why does anyone use an encoding other than UTF-8?

  - by mostafa

  I want to know why any developer would need to use an encoding other than UTF-8.

  Read the article

• Matching 'weird' characters in PHP regex

  - by Bill X

  I have some strings that need a-strippin': ÃœT: 9.996636,76.294363 Tons of long strings of location codes. A literal regex in PHP won't match them, IE $pattern = /ÃœT:/; echo preg_replace($pattern, "", $row['location']); Won't match/strip anything. (To know it's working, /T:/ does strip the last bit of that string). What's the encoding error doing on here? Alternately, I would accept a concise way to take out just the numbers.

  Read the article

• Matching non-[a-zA-Z] characters in PHP regex

  - by Bill X

  I have some strings that need a-strippin': ÃœT: 9.996636,76.294363 Tons of long strings of location codes. A literal regex in PHP won't match them, IE $pattern = /ÃœT:/; echo preg_replace($pattern, "", $row['location']); Won't match/strip anything. (To know it's working, /T:/ does strip the last bit of that string). What's the encoding error going on here? Alternately, I would accept a concise way to take out just the numbers.

  Read the article

• Other language string in SQL Server 2005

  - by balachandar

  I am trying you insert some string which is not in English (other language). when i fetch back they are not correct. They comes like "?????". But at the same time when I enter the string through the SQL Server UI (SSMS) to enter the string, it works OK. What could be the solution please?

  Read the article

• PHP regex question : how to match none-ascii letters in latin1_swedish_ci charset?

  - by Shawn

  I have this string : Verbesserungsvorschläge which I think is in German. Now I want to match it with a regex in php. To be more general, I want to match such characters like German which are not 100% in the ASCII set. Thanks.

  Read the article

• Facelets charset problem

  - by Vladimir

  Hi! In my earlier post there was a problem with JSF charset handling, but also the other part of the problem was MySQL connection parameters for inserting data into db. The problem was solved. But, I migrated the same application from JSP to facelets and the same problem happened again. Characters from input fields are replaced when inserting to database (c is replaced with Ä), but data inserted into db from sql scripts with proper charset are displayed correctly. I'm still using registered filter and page templates are used with head meta tag as following: <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-2"> If I insert into h:form tag the following attribute: acceptcharset="iso-8859-2" I get correct characters in Firefox, but not in IE7. Is there anything else I should do to make it work? Thanks in advance.

  Read the article

• Google app engine error when I login.

  - by zjm1126

  i am using http://code.google.com/p/gaema/source/browse/#hg/demos/webapp, and this is my traceback: Traceback (most recent call last): File "D:\Program Files\Google\google_appengine\google\appengine\ext\webapp\__init__.py", line 510, in __call__ handler.get(*groups) File "D:\gaema\demos\webapp\main.py", line 31, in get google_auth.get_authenticated_user(self._on_auth) File "D:\gaema\demos\webapp\gaema\auth.py", line 641, in get_authenticated_user OpenIdMixin.get_authenticated_user(self, callback) File "D:\gaema\demos\webapp\gaema\auth.py", line 83, in get_authenticated_user url = self._OPENID_ENDPOINT + "?" + urllib.urlencode(args) File "D:\Python25\lib\urllib.py", line 1250, in urlencode v = quote_plus(str(v)) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128) how to do this thanks updated i change the code from args = dict((k, v[-1]) for k, v in self.request.arguments.iteritems()) args["openid.mode"] = u"check_authentication" url = self._OPENID_ENDPOINT + "?" + urllib.urlencode(args) to args = dict((k, v[-1].encode('utf-8')) for k, v in self.request.arguments.iteritems()) args["openid.mode"] = u"check_authentication" url = self._OPENID_ENDPOINT + "?" + urllib.urlencode(args) but also error.

  Read the article

• What if I put two kinds of encoded strings, say utf-8 and utf-16, in one file?

  - by jonny

  In Python, for example: f = open('test','w') f.write('this is a test\n'.encode('utf-16')) f.write('another test\n'.encode('utf-8')) f.close() That file gets messy when I re-open it: f = open("test") print f.readline().decode('utf-16') # it leads to UnicodeDecodeError print f.readline().decode('utf-8') # it works fine However if I keep the texts encoded in one style (say utf-16 only), it could read back ok. So I'm guessing mixing two types of encoding in the same file is wrong and couldn't be decoded back, even if I do know the encoding rules of each specific string? Any suggestion is welcome, thank you!

  Read the article

• Python and hebrew encoding/decoding error

  - by user340495

  Hey, I have sqlite database which I would like to insert values in Hebrew to I am keep getting the following error : UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 0: ordinal not in range(128) my code is as following : runsql(u'INSERT into personal values(%(ID)d,%(name)s)' % {'ID':1,'name':fabricate_hebrew_name()}) def fabricate_hebrew_name(): hebrew_names = [u'????',u'???',u'???',u'???',u'????',u'???',u'????',u'???',u'????',u'?????',u'????',u'???',u'????'] return random.sample(names,1)[0].encode('utf-8') note: runsql executing the query on the sqlite database fabricate_hebrew_name() should return a string which could be used in my SQL query. any help is much appreciated.

  Read the article

• Encoding gives "'ascii' codec can't encode character … ordinal not in range(128)"

  - by user140314

  I am working through the Django RSS reader project here. The RSS feed will read something like "OKLAHOMA CITY (AP) — James Harden let". The RSS feed's encoding reads encoding="UTF-8" so I believe I am passing utf-8 to markdown in the code snippet below. The em dash is where it chokes. I get the Django error of "'ascii' codec can't encode character u'\u2014' in position 109: ordinal not in range(128)" which is an UnicodeEncodeError. In the variables being passed I see "OKLAHOMA CITY (AP) \u2014 James Harden". The code line that is not working is: content = content.encode(parsed_feed.encoding, "xmlcharrefreplace") I am using markdown 2.0, django 1.1, and python 2.4. What is the magic sequence of encoding and decoding that I need to do to make this work? Thanks.

  Read the article

• C# web service, MySql encoding problem

  - by Boban

  I use C# web service to insert, delete and get data from MySql database. The problem is some of the data is in Macedonian (Cyrilic). When I insert directly in the database, it inserts ok. For example: "???" is "???". When I insert throgh the service, it's not. For example: "???" is "???". When I try to get data throug the service, it gets it ok. What's the problem with the inserting? Here is part of my code for inserting: MySqlConnection connection = new MySqlConnection(MyConString); MySqlCommand command = connection.CreateCommand(); command.CommandText = "INSERT INTO user (id_user, name VALUES (NULL, ?name);"; command.Parameters.Add("?name", MySqlDbType.VarChar).Value = name; connection.Open(); command.ExecuteReader(); connection.Close(); return thisrow; Tnq U in advance!!!

  Read the article

• .net Regular Expression to match any kind of letter from any language

  - by pedro

  Which regular expression can I use to match (allow) any kind of letter from any language I need to match any letter including any diacritics (e.g. á, ü, ñ, etc.) and exlude any kind of symbol (math symbols, currency signs, dingbats, box-drawing characters, etc.) and punctuation characters. I've tried using \p{L}\p{M}* but it doesn't work.

  Read the article

• Using SHIFT_JIS text in PHP

  - by Joe Corkery

  I am building a form that needs to accept characters encoded in SHIFT_JIS and then send those results via email to a recipient. I've tried to simply capture the results from the $_POST variable and then to insert them into a block of text like this: $contents = <<<TEST Name: $NameJp ... TEST but that doesn't appear to work (which doesn't really surprise me). This is my first attempt at dealing with non-ASCII characters in PHP and I am hoping that people might have some suggestions. Perhaps I am missing a simple function to encode the text. Also, are there any other potential pitfalls that I might encounter? Thanks.

  Read the article

< Previous Page | 23 24 25 26 27 28 29 30 31 32 33 34  | Next Page >