Search Results

Search found 67 results on 3 pages for 'tokenizer'.

Page 2/3 | < Previous Page | 1 2 3  | Next Page >

• Tokenizing numbers for a parser

  - by René Nyffenegger

  I am writing my first parser and have a few questions conerning the tokenizer. Basically, my tokenizer exposes a nextToken() function that is supposed to return the next token. These tokens are distinguished by a token-type. I think it would make sense to have the following token-types: SYMBOL (such as <, :=, ( and the like REMARK (or a comment) NUMBER IDENT (such as the name of a function or a variable) STRING (Something enclosed between "....") Now, do you think this makes sense? Also, I am struggling with the NUMBER token-type. Do you think it makes more sense to further split it up into a NUMBER and a FLOAT token-type? Without a FLOAT token-type, I'd receive NUMBER (eg 402), a SYMBOL (.) followed by another NUMBER (eg 203) if I were about to parse a float. Finally, what do you think makes more sense for the tokenizer to return when it encounters a -909? Should it return the SYMBOL - first, followed by the NUMBER 909 or should it return a NUMBER -909 right away?

  Read the article

• problem with a string's format in c++ while doing tcp communication

  - by james t

  hi, i am building a simple c++ client, i am splitting the info i get from the server to frames, and pass each frame to a function that processes it, i split the frame into lines using Poco::StringTokenizer tokenizer(frame, "\n"); i take the first line of the tokenizer which represents the type of frame StmpCommand command(tokenizer[0]); a StmpCommand is an enum with the different types of messages and the constructor works as follows : StmpCommand(std::string command): commandType_() { bool x=command=="CONNECTED"; std::cout<<x<<std::endl; if ("SUBSCRIBE" == command) commandType_ = SUBSCRIBE; else if ("UNSUBSCRIBE" == command) commandType_ = UNSUBSCRIBE; else if ("SEND" == command) commandType_ = SEND; else if ("BEGIN" == command) commandType_ = BEGIN; else if ("COMMIT" == command) commandType_ = COMMIT; else if ("CONNECT" == command) commandType_ = CONNECT; else if ("MESSAGE" == command) commandType_ = MESSAGE; else if ("RECEIPT" == command) commandType_ = RECEIPT; else if ("CONNECTED" == command) commandType_ = CONNECTED; else if ("DISCONNECT" == command) commandType_ = DISCONNECT; else if ("ERROR" == command) commandType_ = ERROR; else { std::cerr<<"Error in building StmpCommand object, unknown type - "<<command<<std::endl; } } the first frame i am trying to proccess is a CONNECTED frame therefor i try to create a StmpCommand with CONNECTED as the constructor's only argument and for some reason i am getting an : Error in building StmpCommand object, unknown type - CONNECTED i am clearly passing a string containing CONNECTED but i'm guessing there is something else there that isn't allowing the condition else if ("CONNECTED" == command) to hap

  Read the article

• Eclipse Juno failing to start with Java returning 1 error report

  - by Richard Lewin

  I have been using Eclipse Juno on windows 8 without any issues until yesterday when, upon starting Eclipse it shows the initial splash screen and then shows the following message: Eclipse Java was started but returned exit code=1 C:\Windows\system32\javaw.exe -Dosgi.requiredJavaVersion=1.5 -Dhelp.lucene.tokenizer=standard -Xms40m -Xmx384m -XX:MaxPermSize=256m -jar C:\Program Files\eclipse\plugins/org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar -os win32 -ws win32 -arch x86_64 -showsplash C:\Program Files\eclipse\plugins\org.eclipse.platform_4.2.0.v201206081400\splash.bmp -launcher C:\Program Files\eclipse\eclipse.exe -name Eclipse --launcher.library C:\Program Files\eclipse\plugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20120522-1813\eclipse_1503.dll -startup C:\Program Files\eclipse\plugins/org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar --launcher.overrideVmargs -exitdata 13f4_5c -product org.eclipse.epp.package.java.product -vm C:\Windows\system32\javaw.exe -vmargs -Dosgi.requiredJavaVersion=1.5 -Dhelp.lucene.tokenizer=standard -Xms40m -Xmx384m -XX:MaxPermSize=256m -jar C:\Program Files\eclipse\plugins/org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar OK At this point Eclipse does not load any further. I have attempted to re-install the Java JDK and JRE without having any effect. I have also re-downloaded Eclipse Juno and run it as a 'fresh' startup which has until now worked initially yesterday

  Read the article

• Eclipse does not start on Windows 7

  - by van

  Suddenly Eclipse today has decides to stop working. The last thing I did was close all perspectives and close Eclipse. When loading eclipse from the command prompt, using: "eclipse.exe -clean" the splash screen loads for a split second then exits. When I run the command: eclipsec -consoleLog -debug it results in the following output: Start VM: -Dosgi.requiredJavaVersion=1.6 -Dhelp.lucene.tokenizer=standard -Xms4096m -Xmx4096m -XX:MaxPermSize=512m -Djava.class.path=d:\devtools\eclipse\\plugins/org.eclipse.equinox.launcher_1.3. 0.v20130327-1440.jar -os win32 -ws win32 -arch x86_64 -showsplash d:\devtools\eclipse\\plugins\org.eclipse.platform_4.3.0.v20130605-20 00\splash.bmp -launcher d:\devtools\eclipse\eclipsec.exe -name Eclipsec --launcher.library d:\devtools\eclipse\\plugins/org.eclipse.equinox.launcher.win 32.win32.x86_64_1.1.200.v20130521-0416\eclipse_1503.dll -startup d:\devtools\eclipse\\plugins/org.eclipse.equinox.launcher_1.3.0.v201303 27-1440.jar --launcher.appendVmargs -product org.eclipse.epp.package.standard.product -consoleLog -debug -vm C:/Program Files/Java/jdk1.6.0_37/bin\..\jre\bin\server\jvm.dll -vmargs -Dosgi.requiredJavaVersion=1.6 -Dhelp.lucene.tokenizer=standard -Xms4096m -Xmx4096m -XX:MaxPermSize=512m -Djava.class.path=d:\devtools\eclipse\\plugins/org.eclipse.equinox.launcher_1.3. 0.v20130327-1440.jar Error occurred during initialization of VM Incompatible minimum and maximum heap sizes specified Checking Task Manager shows no Java process running and both the CPU and memory usage are very low. I have tried: Re-installing Eclipse Re-starting my machine But running eclipsec -consoleLog -debug from the command prompt still results in the issue: Error occurred during initialization of VM Incompatible minimum and maximum heap sizes specified

  Read the article

• SOLR - wildcard search with capital letter

  - by Yurish

  I have a problem with SOLR searching. When i`am searching query: dog* everything is ok, but when query is Dog*(with first capital letter), i get no results. Any advice? My config: <fieldType name="text" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> </analyzer> </fieldType>

  Read the article

• Wildcard searching and highlighting with Solr 1.4

  - by andy

  Hey guys, I've got a pretty much vanilla install of SOLR 1.4 apart from a few small config and schema changes. <requestHandler name="standard" class="solr.SearchHandler" default="true"> <!-- default values for query parameters --> <lst name="defaults"> <str name="defType">dismax</str> <str name="echoParams">explicit</str> <str name="qf"> text </str> <str name="spellcheck.dictionary">default</str> <str name="spellcheck.onlyMorePopular">false</str> <str name="spellcheck.extendedResults">false</str> <str name="spellcheck.count">1</str> </lst> </requestHandler> The main field type I'm using for Indexing is this: <fieldType name="textNoHTML" class="solr.TextField" positionIncrementGap="100"> <analyzer type="index"> <charFilter class="solr.HTMLStripCharFilterFactory" /> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/> </analyzer> <analyzer type="query"> <tokenizer class="solr.WhitespaceTokenizerFactory"/> <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/> <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/> <filter class="solr.LowerCaseFilterFactory"/> <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/> </analyzer> </fieldType> now, when I perform a search using "q=search+term&hl=on" I get highlighting, and nice accurate scores. BUT, for wildcard, I'm assuming you need to use "q.alt"? Is that true? If so my query looks like this: "q.alt=search*&hl=on" When I use the above query, highlighting doesn't work, and all the scores are "1.0". What am I doing wrong? is what I want possible without bypassing some of the really cool SOLR optimizations. cheers!

  Read the article

• android error NoSuchElementException

  - by Alexander

  I have returned a cursor string but it contains a delimiter. The delimiter is . I have the string quest.setText(String.valueOf(c.getString(1)));I want to turn the into a new line. What is the best method to achieve this task in android. I understand there is a way to get the delimeter. I want this to achieved for each record. I can itterate through record like so. Cursor c = db.getContact(2); I tried using a string tokenizer but it doesnt seem to work. Here is the code for the tokenizer. I tested it in just plain java and it works without errors. String question = c.getString(1); // quest.setText(String.valueOf(c.getString(1))); //quest.setText(String.valueOf(question)); StringTokenizer st = new StringTokenizer(question,"<ENTER>"); //DisplayContact(c); // StringTokenizer st = new StringTokenizer(question, "=<ENTER>"); while(st.hasMoreTokens()) { String key = st.nextToken(); String val = st.nextToken(); System.out.println(key + "\n" + val); } I then tried running it in android. Here is the error log 06-06 22:31:55.251: E/AndroidRuntime(537): FATAL EXCEPTION: main 06-06 22:31:55.251: E/AndroidRuntime(537): java.util.NoSuchElementException 06-06 22:31:55.251: E/AndroidRuntime(537): at java.util.StringTokenizer.nextToken(StringTokenizer.java:208) 06-06 22:31:55.251: E/AndroidRuntime(537): at alex.android.test.database.quiz.TestdatabasequizActivity$1.onClick(TestdatabasequizActivity.java:95) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.view.View.performClick(View.java:3511) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.view.View$PerformClick.run(View.java:14105) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.os.Handler.handleCallback(Handler.java:605) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.os.Handler.dispatchMessage(Handler.java:92) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.os.Looper.loop(Looper.java:137) 06-06 22:31:55.251: E/AndroidRuntime(537): at android.app.ActivityThread.main(ActivityThread.java:4424) 06-06 22:31:55.251: E/AndroidRuntime(537): at java.lang.reflect.Method.invokeNative(Native Method) 06-06 22:31:55.251: E/AndroidRuntime(537): at java.lang.reflect.Method.invoke(Method.java:511) 06-06 22:31:55.251: E/AndroidRuntime(537): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784) 06-06 22:31:55.251: E/AndroidRuntime(537): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551) 06-06 22:31:55.251: E/AndroidRuntime(537): at dalvik.system.NativeStart.main(Native Method) This is the database query public Cursor getContact(long rowId) throws SQLException { Cursor mCursor = db.query(true, DATABASE_TABLE, new String[] {KEY_ROWID, question, possibleAnsOne,possibleAnsTwo, possibleAnsThree,realQuestion,UR}, KEY_ROWID + "=" + rowId, null, null, null, null, null); if (mCursor != null) { mCursor.moveToFirst(); }

  Read the article

• Rails in production environment not working,but it's working in development environment

  - by user1834759

  An ActionView::Template::Error occurred in posts#index: couldn't find file 'jquery' (in /opt/ruby_apps/bookdate-website/app/assets/javascripts/cpanel_app.coffee:1) sprockets (2.1.3) lib/sprockets/context.rb:100:in `resolve' An ActionView::Template::Error occurred in topics#show: cannot load such file -- html/tokenizer actionpack (3.2.8) lib/action_controller/vendor/html-scanner/html/sanitizer.rb:18:in `tokenize' sometimes there is an exception thrown like the one mentioned above,but sometime it works why? my ruby environment is ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux] Rails 3.2.8

  Read the article

• MySQLi Extension with phpMyAdmin

  - by user1062058

  I just unpacked the latest phpMyAdmin into /var/www/html/phpMyAdmin and it is giving me "The mysqli extension is missing." - how do I install this? I'm on Centos. I checked php.ini and it seems to be "unlocked". when I type in php -i |grep -i mysqli ... it looks to be enabled. Also this is PHP Version 5.3.8 php -m apc ctype curl date dom filter ftp gd hash iconv json libxml mcrypt memcache mysql mysqli openssl pcre PDO pdo_sqlite posix REflection session SimpleXML soap SPL SQLite standard tokenizer xml xmlreader xmlwriter zlib Thanks.

  Read the article

• Adding functionality to any TextReader

  - by strager

  I have a Location class which represents a location somewhere in a stream. (The class isn't coupled to any specific stream.) The location information will be used to match tokens to location in the input in my parser, to allow for nicer error reporting to the user. I want to add location tracking to a TextReader instance. This way, while reading tokens, I can grab the location (which is updated by the TextReader as data is read) and give it to the token during the tokenization process. I am looking for a good approach on accomplishing this goal. I have come up with several designs. Manual location tracking Every time I need to read from the TextReader, I call AdvanceString on the Location object of the tokenizer with the data read. Advantages Very simple. No class bloat. No need to rewrite the TextReader methods. Disadvantages Couples location tracking logic to tokenization process. Easy to forget to track something (though unit testing helps with this). Bloats existing code. Plain TextReader wrapper Create a LocatedTextReaderWrapper class which surrounds each method call, tracking a Location property. Example: public class LocatedTextReaderWrapper : TextReader { private TextReader source; public Location Location { get; set; } public LocatedTextReaderWrapper(TextReader source) : this(source, new Location()) { } public LocatedTextReaderWrapper(TextReader source, Location location) { this.Location = location; this.source = source; } public override int Read(char[] buffer, int index, int count) { int ret = this.source.Read(buffer, index, count); if(ret >= 0) { this.location.AdvanceString(string.Concat(buffer.Skip(index).Take(count))); } return ret; } // etc. } Advantages Tokenization doesn't know about Location tracking. Disadvantages User needs to create and dispose a LocatedTextReaderWrapper instance, in addition to their TextReader instance. Doesn't allow different types of tracking or different location trackers to be added without layers of wrappers. Event-based TextReader wrapper Like LocatedTextReaderWrapper, but decouples it from the Location object raising an event whenever data is read. Advantages Can be reused for other types of tracking. Tokenization doesn't know about Location tracking or other tracking. Can have multiple, independent Location objects (or other methods of tracking) tracking at once. Disadvantages Requires boilerplate code to enable location tracking. User needs to create and dispose the wrapper instance, in addition to their TextReader instance. Aspect-orientated approach Use AOP to perform like the event-based wrapper approach. Advantages Can be reused for other types of tracking. Tokenization doesn't know about Location tracking or other tracking. No need to rewrite the TextReader methods. Disadvantages Requires external dependencies, which I want to avoid. I am looking for the best approach in my situation. I would like to: Not bloat the tokenizer methods with location tracking. Not require heavy initialization in user code. Not have any/much boilerplate/duplicated code. (Perhaps) not couple the TextReader with the Location class. Any insight into this problem and possible solutions or adjustments are welcome. Thanks! (For those who want a specific question: What is the best way to wrap the functionality of a TextReader?) I have implemented the "Plain TextReader wrapper" and "Event-based TextReader wrapper" approaches and am displeased with both, for reasons mentioned in their disadvantages.

  Read the article

• Reflector error or optimisation?

  - by David_001

  Long story short: I used reflector on the System.Security.Util.Tokenizer class, and there's loads of goto statements in there. Here's a brief example snippet: Label_0026: if (this._inSavedCharacter != -1) { num = this._inSavedCharacter; this._inSavedCharacter = -1; } else { switch (this._inTokenSource) { case TokenSource.UnicodeByteArray: if ((this._inIndex + 1) < this._inSize) { break; } stream.AddToken(-1); return; case TokenSource.UTF8ByteArray: if (this._inIndex < this._inSize) { goto Label_00CF; } stream.AddToken(-1); return; case TokenSource.ASCIIByteArray: if (this._inIndex < this._inSize) { goto Label_023C; } stream.AddToken(-1); return; case TokenSource.CharArray: if (this._inIndex < this._inSize) { goto Label_0272; } stream.AddToken(-1); return; case TokenSource.String: if (this._inIndex < this._inSize) { goto Label_02A8; } stream.AddToken(-1); return; case TokenSource.NestedStrings: if (this._inNestedSize == 0) { goto Label_030D; } if (this._inNestedIndex >= this._inNestedSize) { goto Label_0306; } num = this._inNestedString[this._inNestedIndex++]; goto Label_0402; default: num = this._inTokenReader.Read(); if (num == -1) { stream.AddToken(-1); return; } goto Label_0402; } num = (this._inBytes[this._inIndex + 1] << 8) + this._inBytes[this._inIndex]; this._inIndex += 2; } goto Label_0402; Label_00CF: num = this._inBytes[this._inIndex++]; if ((num & 0x80) != 0) { switch (((num & 240) >> 4)) { case 8: case 9: case 10: case 11: throw new XmlSyntaxException(this.LineNo); case 12: case 13: num &= 0x1f; num3 = 2; break; case 14: num &= 15; num3 = 3; break; case 15: throw new XmlSyntaxException(this.LineNo); } if (this._inIndex >= this._inSize) { throw new XmlSyntaxException(this.LineNo, Environment.GetResourceString("XMLSyntax_UnexpectedEndOfFile")); } byte num2 = this._inBytes[this._inIndex++]; if ((num2 & 0xc0) != 0x80) { throw new XmlSyntaxException(this.LineNo); } num = (num << 6) | (num2 & 0x3f); if (num3 != 2) { if (this._inIndex >= this._inSize) { throw new XmlSyntaxException(this.LineNo, Environment.GetResourceString("XMLSyntax_UnexpectedEndOfFile")); } num2 = this._inBytes[this._inIndex++]; if ((num2 & 0xc0) != 0x80) { throw new XmlSyntaxException(this.LineNo); } num = (num << 6) | (num2 & 0x3f); } } goto Label_0402; Label_023C: num = this._inBytes[this._inIndex++]; goto Label_0402; Label_0272: num = this._inChars[this._inIndex++]; goto Label_0402; Label_02A8: num = this._inString[this._inIndex++]; goto Label_0402; Label_0306: this._inNestedSize = 0; I essentially wanted to know how the class worked, but the number of goto's makes it impossible. Arguably something like a Tokenizer class needs to be heavily optimised, so my question is: is Reflector getting it wrong, or is goto an optimisation for this class?

  Read the article

• Can the csv format be defined by a regex?

  - by Spencer Rathbun

  A colleague and I have recently argued over whether a pure regex is capable of fully encapsulating the csv format, such that it is capable of parsing all files with any given escape char, quote char, and separator char. The regex need not be capable of changing these chars after creation, but it must not fail on any other edge case. I have argued that this is impossible for just a tokenizer. The only regex that might be able to do this is a very complex PCRE style that moves beyond just tokenizing. I am looking for something along the lines of: ... the csv format is a context free grammar and as such, it is impossible to parse with regex alone ... Or am I wrong? Is it possible to parse csv with just a POSIX regex? For example, if both the escape char and the quote char are ", then these two lines are valid csv: """this is a test.""","" "and he said,""What will be, will be."", to which I replied, ""Surely not!""","moving on to the next field here..."

  Read the article

• Haskell: Best tools to validate textual input?

  - by Ana

  In Haskell, there are a few different options to "parsing text". I know of Alex & Happy, Parsec and Attoparsec. Probably some others. I'd like to put together a library where the user can input pieces of a URL (scheme e.g. HTTP, hostname, username, port, path, query, etc.) I'd like to validate the pieces according to the ABNF specified in RFC 3986. In other words, I'd like to put together a set of functions such as: validateScheme :: String -> Bool validateUsername :: String -> Bool validatePassword :: String -> Bool validateAuthority :: String -> Bool validatePath :: String -> Bool validateQuery :: String -> Bool What is the most appropriate tool to use to write these functions? Alex's regexps is very concise, but it's a tokenizer and doesn't straightforwardly allow you to parse using specific rules, so it's not quite what I'm looking for, but perhaps it can be wrangled into doing this easily. I've written Parsec code that does some of the above, but it looks very different from the original ABNF and unnecessarily long. So, there must be an easier and/or more appropriate way. Recommendations?

  Read the article

• how to install posix in php

  - by Nizzy

  posix does not appear when I run php -m cmd, however, I see it from the phpinfo() –enable-posix=shared on Linux with Plesk 9. Basically, I can't use posix_*() functions as described at http://www.php.net/manual/en/ref.posix.php this shows doesn't exists: if (function_exists(‘posix_getuid’)) { echo “posix_getuid available”; } else { echo “posix_getuid not available”; // this prints in my server. } could someone show me how to install it? thank you [PHP Modules] bz2 calendar ctype curl date dbase dom exif fileinfo filter ftp gd geoip gettext gmp hash iconv imap ionCube Loader json libxml mbstring mcrypt memcache mhash mysql mysqli openssl pcntl pcre PDO pdo_mysql pdo_sqlite readline Reflection session shmop SimpleXML sockets SPL sqlite standard tokenizer wddx xml xmlreader xmlwriter xsl zip zlib

  Read the article

• Lucene.Net support phrases?: What is best approach to tokenize comma-delimited data (atomically) in

  - by Pete Alvin

  I have a database with a column I wish to index that has comma-delimited names, e.g., User.FullNameList = "Helen Ready, Phil Collins, Brad Paisley" I prefer to tokenize each name atomically (name as a whole searchable entity). What is the best approach for this? Did I miss a simple option to set the tokenize delimiter? Do I have to subclass or write my own class that to roll my own tokenizer? Something else? ;) Or does Lucene.net not support phrases? Or is it smart enough to handle this use case automatically? I'm sure I'm not the first person to have to do this. Googling produced no noticeable solutions.

  Read the article

• Sharing memory among YACC, Lex, and C files

  - by sczizzo

  I have a YACC (Bison) grammar, a Lex (Flex) tokenizer, and a C program among which I need to share a struct (or really any variable). Currently, I declare the actual object in the grammar file and extern it wherever I need it (which is to say, my C source file), usually using a pointer to manipulate it. I have a shared header (and implementation) file between the C file and the grammar file with functions useful for manipulating my data structure. This works, but it feels a little uncomfortable. Is there a better way to share memory between the grammar and program?

  Read the article

• Strategy for developing namespaced and non-namespaced versions of same PHP code

  - by porneL

  I'm maintaining library written for PHP 5.2 and I'd like to create PHP 5.3-namespaced version of it. However, I'd also keep non-namespaced version up to date until PHP 5.3 becomes so old, that even Debian stable ships it ;) I've got rather clean code, about 80 classes following Project_Directory_Filename naming scheme (I'd change them to \Project\Directory\Filename of course) and only few functions and constants (also prefixed with project name). Question is: what's the best way to develop namespaced and non-namespaced versions in parallel? Should I just create fork in repository and keep merging changes between branches? Are there cases where backslash-sprinkled code becomes hard to merge? Should I write script that converts 5.2 version to 5.3 or vice-versa? Should I use PHP tokenizer? sed? C preprocessor? Is there a better way to use namespaces where available and keep backwards compatibility with older PHP?

  Read the article

• How do Ruby and Python implement their interactive consoles?

  - by sxa

  When implementing the interpreter for my programming language I first thought of a simple console window which allows the user to enter some code which is then executed as a standalone program as a shell. But there are severe problems: If every line of code the user enters is handled as a standalone program, it has to go through the tokenizer and parser and is then just executed by the interpreter - what about functions then? How can the Python/Ruby interactive consoles (IDLE, irb) "share" the code? How is the code entered handled? Example: >> def x: >> print("Blah") >> >> x() Where is the function stored so it can be called at any time again? How can the interactive console take everything entered as obviously one program without executing everything over and over again?

  Read the article

• StreamTokenizer Iterator Adapter help

  - by alpdog14

  I have this StreamTokenizer Iterator Adapter that is suppose to create a Tokenizer Iterator Index Builder then build the index from a STIA wrapped around a StreamTokenizer. I am having trouble implementing the hasNext and Next for my STIA, can anyone help me, here is my class: public class StreamTokenizerIteratorAdapter implements Iterator<Token> { DefaultIndexImpl index; StreamTokenizer source; public StreamTokenizerIteratorAdapter(final StreamTokenizer source) { if (source == null) throw new IllegalArgumentException("source == null"); } @Override public boolean hasNext() { return !index.isEmpty(); } public Token next() { if(!index.isEmpty()) return next(); else return null; } @Override public void remove() { throw new UnsupportedOperationException(); } } Should I be using the source element in the hasNext() and next()?

  Read the article

• Mutually exclusive regular expressions

  - by CaptnCraig

  If I have a list of regular expressions, is there an easy way to determine that no two of them will both return a match for the same string? That is, the list is valid if and only if for all strings a maximum of one item in the list will match the entire string. It seems like this will be very hard (maybe impossible?) to prove definitively, but I can't seem to find any work on the subject. The reason I ask is that I am working on a tokenizer that accepts regexes, and I would like to ensure only one token at a time can match the head of the input.

  Read the article

• Compiler: Translation to assembly

  - by sub

  I've written an interpreter for my experimental language and know I want to move on and write a small compiler for it. It will probably take the source, go through the same steps as the interpreter (tokenizer, parser) and then translate the source to assembly. Now my questions: Can I expect that every command in my language can be 1:1 translated to a bunch of assembly instructions? What I mean is if I will have to completely throw over the whole input program or if it is just translated to assembly per line. Which assembler should I use as output format?

  Read the article

• Testing what a String token holds

  - by Yiwei Gao

  I am making a calculator and part of this program takes in user String input and tokenizes it (using my own implementation of a Tokenizer class). So now I have a bunch of Token objects and I would like to test each one of them to see if they hold numbers or operators. Is there a way to test to see if they hold operators (ie. +, -, *, /, =, (, ), etc.) without using if (token.equals("+") || token.equals("-") || ... and so on, for each operator? These Token objects are all of type String.

  Read the article

• "Unable to initialize module" fileinfo php-pecl-Fileinfo.x86_64

  - by Myers Network

  I have a brand new server server that I am trying to get setup up. This is a 64 bit machine that I can not install "fileinfo" or "memcache". I have uninstalled these and reinstalled them using yum and pecl with no luck. Yum install fine "no error" but then get error when running php. pecl from what I can tell is only installing 32bit. Does not put anything in the lib64 directory. Here is my output from php -v: PHP Warning: PHP Startup: fileinfo: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP Warning: PHP Startup: memcache: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP 5.2.14 (cli) (built: Aug 12 2010 16:03:48) Copyright (c) 1997-2010 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies Here is some other system info incase you need it uname: Linux server.actham.us 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:20 EST 2010 x86_64 x86_64 x86_64 GNU/Linux php -m: PHP Warning: PHP Startup: fileinfo: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP Warning: PHP Startup: memcache: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 [PHP Modules] bz2 calendar ctype curl date dbase dom exif filter ftp gd gettext gmp hash iconv imap json ldap libxml mbstring mcrypt mysql mysqli openssl pcntl pcre PDO pdo_mysql pdo_sqlite readline Reflection session shmop SimpleXML sockets SPL standard tokenizer wddx xml xmlreader xmlrpc xmlwriter xsl zip zlib [Zend Modules] Any help would be greatly appreciated, thanks....

  Read the article

• PHP - Centos OpenSSL error

  - by mabbs

  i'm currently having a problem with OpenSSL on my Centos 6.5 Server. it ran perfectly fine until sunday. and i checked the error_log and i saw this error in the log PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/openssl.so' - /usr/lib64/php/modules/openssl.so: cannot open shared object file: No such file or directory in Unknown on line 0 i tried phpinfo(); and i found that openssl is enabled i tried php -m it returned [PHP Modules] bz2 calendar Core ctype curl date dom ereg exif fileinfo filter ftp gd gettext gmp hash iconv interbase json libxml mbstring mcrypt memcache mysql mysqli openssl pcntl pcre PDO PDO_Firebird pdo_mysql pdo_sqlite Phar pspell readline Reflection session shmop SimpleXML snmp sockets SPL sqlite3 standard tokenizer wddx xml xmlreader xmlrpc xmlwriter xsl zip zlib UPDATE this is what i got from rpm -qa | grep php just like what Mike Suggested php-php-gettext-1.0.11-3.el6.noarch php-mcrypt-5.3.3-3.el6.x86_64 php-interbase-5.3.3-3.el6.x86_64 php-pdo-5.3.3-27.el6_5.1.x86_64 php-5.3.3-27.el6_5.1.x86_64 php-mysql-5.3.3-27.el6_5.1.x86_64 php-snmp-5.3.3-27.el6_5.1.x86_64 php-gd-5.3.3-27.el6_5.1.x86_64 php-xml-5.3.3-27.el6_5.1.x86_64 php-pear-1.9.4-4.el6.noarch php-pecl-memcache-3.0.5-4.el6.x86_64 phpMyAdmin-3.5.8.2-1.el6.noarch php-common-5.3.3-27.el6_5.1.x86_64 php-cli-5.3.3-27.el6_5.1.x86_64 php-devel-5.3.3-27.el6_5.1.x86_64 php-mbstring-5.3.3-27.el6_5.1.x86_64 php-xmlrpc-5.3.3-27.el6_5.1.x86_64 php-pspell-5.3.3-27.el6_5.1.x86_64

  Read the article

• "Unable to initialize module" fileinfo php-pecl-Fileinfo.x86_64

  - by Myers Network

  I have a brand new server server that I am trying to get setup up. This is a 64 bit machine that I can not install "fileinfo" or "memcache". I have uninstalled these and reinstalled them using yum and pecl with no luck. Yum install fine "no error" but then get error when running php. pecl from what I can tell is only installing 32bit. Does not put anything in the lib64 directory. Here is my output from php -v: PHP Warning: PHP Startup: fileinfo: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP Warning: PHP Startup: memcache: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP 5.2.14 (cli) (built: Aug 12 2010 16:03:48) Copyright (c) 1997-2010 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies Here is some other system info incase you need it uname: Linux server.actham.us 2.6.18-194.26.1.el5 #1 SMP Tue Nov 9 12:54:20 EST 2010 x86_64 x86_64 x86_64 GNU/Linux php -m: PHP Warning: PHP Startup: fileinfo: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 PHP Warning: PHP Startup: memcache: Unable to initialize module Module compiled with module API=20050922, debug=0, thread-safety=0 PHP compiled with module API=20060613, debug=0, thread-safety=0 These options need to match in Unknown on line 0 [PHP Modules] bz2 calendar ctype curl date dbase dom exif filter ftp gd gettext gmp hash iconv imap json ldap libxml mbstring mcrypt mysql mysqli openssl pcntl pcre PDO pdo_mysql pdo_sqlite readline Reflection session shmop SimpleXML sockets SPL standard tokenizer wddx xml xmlreader xmlrpc xmlwriter xsl zip zlib [Zend Modules] Any help would be greatly appreciated, thanks....

  Read the article

< Previous Page | 1 2 3  | Next Page >