subject - Page 140 - Developer IT

How to deploy the advanced search page using Module in SharePoint 2013

- by ybbest

Today, I’d like to show you how to deploy your custom advanced search page using module in Visual Studio 2012.Using a module is the way how SharePoint deploy all the publishing pages to the search centre. Browse to the template under 15 hive of SharePoint2013, then go to the SearchCenterFiles under Features(as shown below).Then open the Files.xml it shows how SharePoint using module to deploy advanced search.You can download the solution here. Now I am going to show you how to deploy your custom advanced search page.The feature is located in the C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\TEMPLATE\FEATURES\SearchCenterFiles . To deploy SharePoint advanced Search pages, you need to do the following: 1. Create SharePoint2013 project and then create a module item. 2. Find how Out of box SharePoint deploy the Advanced Search Page from Files.xml and copy and paste it into the elements.xml <File Url="advanced.aspx" Type="GhostableInLibrary"> <Property Name="PublishingPageLayout" Value="~SiteCollection/_catalogs/masterpage/AdvancedSearchLayout.aspx, $Resources:Microsoft.Office.Server.Search,SearchCenterAdvancedSearchTitle;" /> <Property Name="Title" Value="$Resources:Microsoft.Office.Server.Search,Search_Advanced_Page_Title;" /> <Property Name="ContentType" Value="$Resources:Microsoft.Office.Server.Search,contenttype_welcomepage_name;" /> <AllUsersWebPart WebPartZoneID="MainZone" WebPartOrder="1"> <![CDATA[ <WebPart xmlns="http://schemas.microsoft.com/WebPart/v2"> <Assembly>Microsoft.Office.Server.Search, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c</Assembly> <TypeName>Microsoft.Office.Server.Search.WebControls.AdvancedSearchBox</TypeName> <Title>$Resources:Microsoft.Office.Server.Search,AdvancedSearch_Webpart_Title;</Title> <Description>$Resources:Microsoft.Office.Server.Search,AdvancedSearch_Webpart_Description;</Description> <FrameType>None</FrameType> <AllowMinimize>true</AllowMinimize> <AllowRemove>true</AllowRemove> <IsVisible>true</IsVisible> <SearchResultPageURL xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">results.aspx</SearchResultPageURL> <TextQuerySectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">$Resources:Microsoft.Office.Server.Search,AdvancedSearch_FindDocsWith_Title;</TextQuerySectionLabelText> <ShowAndQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowAndQueryTextBox> <ShowPhraseQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowPhraseQueryTextBox> <ShowOrQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowOrQueryTextBox> <ShowNotQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowNotQueryTextBox> <ScopeSectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">$Resources:Microsoft.Office.Server.Search,AdvancedSearch_NarrowSearch_Title;</ScopeSectionLabelText> <ShowLanguageOptions xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowLanguageOptions> <ShowResultTypePicker xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowResultTypePicker> <ShowPropertiesSection xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowPropertiesSection> <PropertiesSectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">$Resources:Microsoft.Office.Server.Search,AdvancedSearch_AddPropRestrictions_Title;</PropertiesSectionLabelText> </WebPart> ]]> </AllUsersWebPart> </File> 3. Customize your SharePoint advanced Search Page by modifying the Advanced Search Box and Export the webpart and copy the webpart file to the elements under module. 4. Export the web part and copy the content of the web part file to the elements.xml in the module. <File Path="AdvancedSearchPage\advanced.aspx" Url="employeeAdvanced.aspx" Type="GhostableInLibrary"> <Property Name="PublishingPageLayout" Value="~SiteCollection/_catalogs/masterpage/AdvancedSearchLayout.aspx, $Resources:Microsoft.Office.Server.Search,SearchCenterAdvancedSearchTitle;" /> <Property Name="Title" Value="$Resources:Microsoft.Office.Server.Search,Search_Advanced_Page_Title;" /> <Property Name="ContentType" Value="$Resources:Microsoft.Office.Server.Search,contenttype_welcomepage_name;" /> <AllUsersWebPart WebPartZoneID="MainZone" WebPartOrder="1"> <![CDATA[ <WebPart xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/WebPart/v2"> <Title>Advanced Search Box</Title> <FrameType>None</FrameType> <Description>Displays parameterized search options based on properties and combinations of words.</Description> <IsIncluded>true</IsIncluded> <ZoneID>MainZone</ZoneID> <PartOrder>1</PartOrder> <FrameState>Normal</FrameState> <Height /> <Width /> <AllowRemove>true</AllowRemove> <AllowZoneChange>true</AllowZoneChange> <AllowMinimize>true</AllowMinimize> <AllowConnect>true</AllowConnect> <AllowEdit>true</AllowEdit> <AllowHide>true</AllowHide> <IsVisible>true</IsVisible> <DetailLink /> <HelpLink /> <HelpMode>Modeless</HelpMode> <Dir>Default</Dir> <PartImageSmall /> <MissingAssembly>Cannot import this Web Part.</MissingAssembly> <PartImageLarge /> <IsIncludedFilter /> <Assembly>Microsoft.Office.Server.Search, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c</Assembly> <TypeName>Microsoft.Office.Server.Search.WebControls.AdvancedSearchBox</TypeName> <SearchResultPageURL xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">results.aspx</SearchResultPageURL> <TextQuerySectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">Find documents that have...</TextQuerySectionLabelText> <ShowAndQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowAndQueryTextBox> <AndQueryTextBoxLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ShowPhraseQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowPhraseQueryTextBox> <PhraseQueryTextBoxLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ShowOrQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowOrQueryTextBox> <OrQueryTextBoxLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ShowNotQueryTextBox xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowNotQueryTextBox> <NotQueryTextBoxLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ScopeSectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">Narrow the search...</ScopeSectionLabelText> <ShowScopes xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">false</ShowScopes> <ScopeLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <DisplayGroup xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">Advanced Search</DisplayGroup> <ShowLanguageOptions xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">false</ShowLanguageOptions> <LanguagesLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ShowResultTypePicker xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowResultTypePicker> <ResultTypeLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox" /> <ShowPropertiesSection xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">true</ShowPropertiesSection> <PropertiesSectionLabelText xmlns="urn:schemas-microsoft-com:AdvancedSearchBox">Add property restrictions...</PropertiesSectionLabelText> <Properties xmlns="urn:schemas-microsoft-com:AdvancedSearchBox"><root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <LangDefs> <LangDef DisplayName="Arabic" LangID="ar"/> <LangDef DisplayName="Bengali" LangID="bn"/> <LangDef DisplayName="Bulgarian" LangID="bg"/> <LangDef DisplayName="Catalan" LangID="ca"/> <LangDef DisplayName="Simplified Chinese" LangID="zh-cn"/> <LangDef DisplayName="Traditional Chinese" LangID="zh-tw"/> <LangDef DisplayName="Croatian" LangID="hr"/> <LangDef DisplayName="Czech" LangID="cs"/> <LangDef DisplayName="Danish" LangID="da"/> <LangDef DisplayName="Dutch" LangID="nl"/> <LangDef DisplayName="English" LangID="en"/> <LangDef DisplayName="Finnish" LangID="fi"/> <LangDef DisplayName="French" LangID="fr"/> <LangDef DisplayName="German" LangID="de"/> <LangDef DisplayName="Greek" LangID="el"/> <LangDef DisplayName="Gujarati" LangID="gu"/> <LangDef DisplayName="Hebrew" LangID="he"/> <LangDef DisplayName="Hindi" LangID="hi"/> <LangDef DisplayName="Hungarian" LangID="hu"/> <LangDef DisplayName="Icelandic" LangID="is"/> <LangDef DisplayName="Indonesian" LangID="id"/> <LangDef DisplayName="Italian" LangID="it"/> <LangDef DisplayName="Japanese" LangID="ja"/> <LangDef DisplayName="Kannada" LangID="kn"/> <LangDef DisplayName="Korean" LangID="ko"/> <LangDef DisplayName="Latvian" LangID="lv"/> <LangDef DisplayName="Lithuanian" LangID="lt"/> <LangDef DisplayName="Malay" LangID="ms"/> <LangDef DisplayName="Malayalam" LangID="ml"/> <LangDef DisplayName="Marathi" LangID="mr"/> <LangDef DisplayName="Norwegian" LangID="no"/> <LangDef DisplayName="Polish" LangID="pl"/> <LangDef DisplayName="Portuguese" LangID="pt"/> <LangDef DisplayName="Punjabi" LangID="pa"/> <LangDef DisplayName="Romanian" LangID="ro"/> <LangDef DisplayName="Russian" LangID="ru"/> <LangDef DisplayName="Slovak" LangID="sk"/> <LangDef DisplayName="Slovenian" LangID="sl"/> <LangDef DisplayName="Spanish" LangID="es"/> <LangDef DisplayName="Swedish" LangID="sv"/> <LangDef DisplayName="Tamil" LangID="ta"/> <LangDef DisplayName="Telugu" LangID="te"/> <LangDef DisplayName="Thai" LangID="th"/> <LangDef DisplayName="Turkish" LangID="tr"/> <LangDef DisplayName="Ukrainian" LangID="uk"/> <LangDef DisplayName="Urdu" LangID="ur"/> <LangDef DisplayName="Vietnamese" LangID="vi"/> </LangDefs> <Languages> <Language LangRef="en"/> <Language LangRef="fr"/> <Language LangRef="de"/> <Language LangRef="ja"/> <Language LangRef="zh-cn"/> <Language LangRef="es"/> <Language LangRef="zh-tw"/> </Languages> <PropertyDefs> <PropertyDef Name="Path" DataType="url" DisplayName="URL"/> <PropertyDef Name="Size" DataType="integer" DisplayName="Size (bytes)"/> <PropertyDef Name="Write" DataType="datetime" DisplayName="Last Modified Date"/> <PropertyDef Name="FileName" DataType="text" DisplayName="Name"/> <PropertyDef Name="Description" DataType="text" DisplayName="Description"/> <PropertyDef Name="Title" DataType="text" DisplayName="Title"/> <PropertyDef Name="Author" DataType="text" DisplayName="Author"/> <PropertyDef Name="DocSubject" DataType="text" DisplayName="Subject"/> <PropertyDef Name="DocKeywords" DataType="text" DisplayName="Keywords"/> <PropertyDef Name="DocComments" DataType="text" DisplayName="Comments"/> <PropertyDef Name="CreatedBy" DataType="text" DisplayName="Created By"/> <PropertyDef Name="ModifiedBy" DataType="text" DisplayName="Last Modified By"/> <PropertyDef Name="EmployeeNumber" DataType="text" DisplayName="EmployeeNumber"/> <PropertyDef Name="EmployeeId" DataType="text" DisplayName="EmployeeId"/> <PropertyDef Name="EmployeeFirstName" DataType="text" DisplayName="EmployeeFirstName"/> <PropertyDef Name="EmployeeLastName" DataType="text" DisplayName="EmployeeLastName"/> </PropertyDefs> <ResultTypes> <ResultType DisplayName="Employee Document" Name="default"> <KeywordQuery/> <PropertyRef Name="EmployeeNumber" /> <PropertyRef Name="EmployeeId" /> <PropertyRef Name="EmployeeFirstName" /> <PropertyRef Name="EmployeeLastName" /> </ResultType> <ResultType DisplayName="All Results"> <KeywordQuery/> <PropertyRef Name="Author" /> <PropertyRef Name="Description" /> <PropertyRef Name="FileName" /> <PropertyRef Name="Size" /> <PropertyRef Name="Path" /> <PropertyRef Name="Write" /> <PropertyRef Name="CreatedBy" /> <PropertyRef Name="ModifiedBy" /> </ResultType> <ResultType DisplayName="Documents" Name="documents"> <KeywordQuery>IsDocument="True"</KeywordQuery> <PropertyRef Name="Author" /> <PropertyRef Name="DocComments"/> <PropertyRef Name="Description" /> <PropertyRef Name="DocKeywords"/> <PropertyRef Name="FileName" /> <PropertyRef Name="Size" /> <PropertyRef Name="DocSubject"/> <PropertyRef Name="Path" /> <PropertyRef Name="Write" /> <PropertyRef Name="CreatedBy" /> <PropertyRef Name="ModifiedBy" /> <PropertyRef Name="Title"/> </ResultType> <ResultType DisplayName="Word Documents" Name="worddocuments"> <KeywordQuery>FileExtension="doc" OR FileExtension="docx" OR FileExtension="dot" OR FileExtension="docm" OR FileExtension="odt"</KeywordQuery> <PropertyRef Name="Author" /> <PropertyRef Name="DocComments"/> <PropertyRef Name="Description" /> <PropertyRef Name="DocKeywords"/> <PropertyRef Name="FileName" /> <PropertyRef Name="Size" /> <PropertyRef Name="DocSubject"/> <PropertyRef Name="Path" /> <PropertyRef Name="Write" /> <PropertyRef Name="CreatedBy" /> <PropertyRef Name="ModifiedBy" /> <PropertyRef Name="Title"/> </ResultType> <ResultType DisplayName="Excel Documents" Name="exceldocuments"> <KeywordQuery>FileExtension="xls" OR FileExtension="xlsx" OR FileExtension="xlsm" OR FileExtension="xlsb" OR FileExtension="ods"</KeywordQuery> <PropertyRef Name="Author" /> <PropertyRef Name="DocComments"/> <PropertyRef Name="Description" /> <PropertyRef Name="DocKeywords"/> <PropertyRef Name="FileName" /> <PropertyRef Name="Size" /> <PropertyRef Name="DocSubject"/> <PropertyRef Name="Path" /> <PropertyRef Name="Write" /> <PropertyRef Name="CreatedBy" /> <PropertyRef Name="ModifiedBy" /> <PropertyRef Name="Title"/> </ResultType> <ResultType DisplayName="PowerPoint Presentations" Name="presentations"> <KeywordQuery>FileExtension="ppt" OR FileExtension="pptx" OR FileExtension="pptm" OR FileExtension="odp"</KeywordQuery> <PropertyRef Name="Author" /> <PropertyRef Name="DocComments"/> <PropertyRef Name="Description" /> <PropertyRef Name="DocKeywords"/> <PropertyRef Name="FileName" /> <PropertyRef Name="Size" /> <PropertyRef Name="DocSubject"/> <PropertyRef Name="Path" /> <PropertyRef Name="Write" /> <PropertyRef Name="CreatedBy" /> <PropertyRef Name="ModifiedBy" /> <PropertyRef Name="Title"/> </ResultType> </ResultTypes></root></Properties> </WebPart> ]]> </AllUsersWebPart> </File> 5.Deploy your custom solution and you will have a custom advanced search page.

Read the article

Using R to Analyze G1GC Log Files

- by user12620111

Using R to Analyze G1GC Log Files body, td { font-family: sans-serif; background-color: white; font-size: 12px; margin: 8px; } tt, code, pre { font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console', Consolas, Monaco, monospace; } h1 { font-size:2.2em; } h2 { font-size:1.8em; } h3 { font-size:1.4em; } h4 { font-size:1.0em; } h5 { font-size:0.9em; } h6 { font-size:0.8em; } a:visited { color: rgb(50%, 0%, 50%); } pre { margin-top: 0; max-width: 95%; border: 1px solid #ccc; white-space: pre-wrap; } pre code { display: block; padding: 0.5em; } code.r, code.cpp { background-color: #F8F8F8; } table, td, th { border: none; } blockquote { color:#666666; margin:0; padding-left: 1em; border-left: 0.5em #EEE solid; } hr { height: 0px; border-bottom: none; border-top-width: thin; border-top-style: dotted; border-top-color: #999999; } @media print { * { background: transparent !important; color: black !important; filter:none !important; -ms-filter: none !important; } body { font-size:12pt; max-width:100%; } a, a:visited { text-decoration: underline; } hr { visibility: hidden; page-break-before: always; } pre, blockquote { padding-right: 1em; page-break-inside: avoid; } tr, img { page-break-inside: avoid; } img { max-width: 100% !important; } @page :left { margin: 15mm 20mm 15mm 10mm; } @page :right { margin: 15mm 10mm 15mm 20mm; } p, h2, h3 { orphans: 3; widows: 3; } h2, h3 { page-break-after: avoid; } } pre .operator, pre .paren { color: rgb(104, 118, 135) } pre .literal { color: rgb(88, 72, 246) } pre .number { color: rgb(0, 0, 205); } pre .comment { color: rgb(76, 136, 107); } pre .keyword { color: rgb(0, 0, 255); } pre .identifier { color: rgb(0, 0, 0); } pre .string { color: rgb(3, 106, 7); } var hljs=new function(){function m(p){return p.replace(/&/gm,"&").replace(/"}while(y.length||w.length){var v=u().splice(0,1)[0];z+=m(x.substr(q,v.offset-q));q=v.offset;if(v.event=="start"){z+=t(v.node);s.push(v.node)}else{if(v.event=="stop"){var p,r=s.length;do{r--;p=s[r];z+=("")}while(p!=v.node);s.splice(r,1);while(r'+M[0]+""}else{r+=M[0]}O=P.lR.lastIndex;M=P.lR.exec(L)}return r+L.substr(O,L.length-O)}function J(L,M){if(M.sL&&e[M.sL]){var r=d(M.sL,L);x+=r.keyword_count;return r.value}else{return F(L,M)}}function I(M,r){var L=M.cN?'':"";if(M.rB){y+=L;M.buffer=""}else{if(M.eB){y+=m(r)+L;M.buffer=""}else{y+=L;M.buffer=r}}D.push(M);A+=M.r}function G(N,M,Q){var R=D[D.length-1];if(Q){y+=J(R.buffer+N,R);return false}var P=q(M,R);if(P){y+=J(R.buffer+N,R);I(P,M);return P.rB}var L=v(D.length-1,M);if(L){var O=R.cN?"":"";if(R.rE){y+=J(R.buffer+N,R)+O}else{if(R.eE){y+=J(R.buffer+N,R)+O+m(M)}else{y+=J(R.buffer+N+M,R)+O}}while(L1){O=D[D.length-2].cN?"":"";y+=O;L--;D.length--}var r=D[D.length-1];D.length--;D[D.length-1].buffer="";if(r.starts){I(r.starts,"")}return R.rE}if(w(M,R)){throw"Illegal"}}var E=e[B];var D=[E.dM];var A=0;var x=0;var y="";try{var s,u=0;E.dM.buffer="";do{s=p(C,u);var t=G(s[0],s[1],s[2]);u+=s[0].length;if(!t){u+=s[1].length}}while(!s[2]);if(D.length1){throw"Illegal"}return{r:A,keyword_count:x,value:y}}catch(H){if(H=="Illegal"){return{r:0,keyword_count:0,value:m(C)}}else{throw H}}}function g(t){var p={keyword_count:0,r:0,value:m(t)};var r=p;for(var q in e){if(!e.hasOwnProperty(q)){continue}var s=d(q,t);s.language=q;if(s.keyword_count+s.rr.keyword_count+r.r){r=s}if(s.keyword_count+s.rp.keyword_count+p.r){r=p;p=s}}if(r.language){p.second_best=r}return p}function i(r,q,p){if(q){r=r.replace(/^((]+|\t)+)/gm,function(t,w,v,u){return w.replace(/\t/g,q)})}if(p){r=r.replace(/\n/g,"")}return r}function n(t,w,r){var x=h(t,r);var v=a(t);var y,s;if(v){y=d(v,x)}else{return}var q=c(t);if(q.length){s=document.createElement("pre");s.innerHTML=y.value;y.value=k(q,c(s),x)}y.value=i(y.value,w,r);var u=t.className;if(!u.match("(\\s|^)(language-)?"+v+"(\\s|$)")){u=u?(u+" "+v):v}if(/MSIE [678]/.test(navigator.userAgent)&&t.tagName=="CODE"&&t.parentNode.tagName=="PRE"){s=t.parentNode;var p=document.createElement("div");p.innerHTML=""+y.value+"";t=p.firstChild.firstChild;p.firstChild.cN=s.cN;s.parentNode.replaceChild(p.firstChild,s)}else{t.innerHTML=y.value}t.className=u;t.result={language:v,kw:y.keyword_count,re:y.r};if(y.second_best){t.second_best={language:y.second_best.language,kw:y.second_best.keyword_count,re:y.second_best.r}}}function o(){if(o.called){return}o.called=true;var r=document.getElementsByTagName("pre");for(var p=0;p|=||=||=|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~";this.ER="(?![\\s\\S])";this.BE={b:"\\\\.",r:0};this.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[this.BE],r:0};this.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[this.BE],r:0};this.CLCM={cN:"comment",b:"//",e:"$"};this.CBLCLM={cN:"comment",b:"/\\*",e:"\\*/"};this.HCM={cN:"comment",b:"#",e:"$"};this.NM={cN:"number",b:this.NR,r:0};this.CNM={cN:"number",b:this.CNR,r:0};this.BNM={cN:"number",b:this.BNR,r:0};this.inherit=function(r,s){var p={};for(var q in r){p[q]=r[q]}if(s){for(var q in s){p[q]=s[q]}}return p}}();hljs.LANGUAGES.cpp=function(){var a={keyword:{"false":1,"int":1,"float":1,"while":1,"private":1,"char":1,"catch":1,"export":1,virtual:1,operator:2,sizeof:2,dynamic_cast:2,typedef:2,const_cast:2,"const":1,struct:1,"for":1,static_cast:2,union:1,namespace:1,unsigned:1,"long":1,"throw":1,"volatile":2,"static":1,"protected":1,bool:1,template:1,mutable:1,"if":1,"public":1,friend:2,"do":1,"return":1,"goto":1,auto:1,"void":2,"enum":1,"else":1,"break":1,"new":1,extern:1,using:1,"true":1,"class":1,asm:1,"case":1,typeid:1,"short":1,reinterpret_cast:2,"default":1,"double":1,register:1,explicit:1,signed:1,typename:1,"try":1,"this":1,"switch":1,"continue":1,wchar_t:1,inline:1,"delete":1,alignof:1,char16_t:1,char32_t:1,constexpr:1,decltype:1,noexcept:1,nullptr:1,static_assert:1,thread_local:1,restrict:1,_Bool:1,complex:1},built_in:{std:1,string:1,cin:1,cout:1,cerr:1,clog:1,stringstream:1,istringstream:1,ostringstream:1,auto_ptr:1,deque:1,list:1,queue:1,stack:1,vector:1,map:1,set:1,bitset:1,multiset:1,multimap:1,unordered_set:1,unordered_map:1,unordered_multiset:1,unordered_multimap:1,array:1,shared_ptr:1}};return{dM:{k:a,i:"",k:a,r:10,c:["self"]}]}}}();hljs.LANGUAGES.r={dM:{c:[hljs.HCM,{cN:"number",b:"\\b0[xX][0-9a-fA-F]+[Li]?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+(?:[eE][+\\-]?\\d*)?L\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+\\.(?!\\d)(?:i\\b)?",e:hljs.IMMEDIATE_RE,r:1},{cN:"number",b:"\\b\\d+(?:\\.\\d*)?(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\.\\d+(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"keyword",b:"(?:tryCatch|library|setGeneric|setGroupGeneric)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\.",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\d+(?![\\w.])",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\b(?:function)",e:hljs.IMMEDIATE_RE,r:2},{cN:"keyword",b:"(?:if|in|break|next|repeat|else|for|return|switch|while|try|stop|warning|require|attach|detach|source|setMethod|setClass)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"literal",b:"(?:NA|NA_integer_|NA_real_|NA_character_|NA_complex_)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"literal",b:"(?:NULL|TRUE|FALSE|T|F|Inf|NaN)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"identifier",b:"[a-zA-Z.][a-zA-Z0-9._]*\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"operator",b:"|=|| Using R to Analyze G1GC Log Files Using R to Analyze G1GC Log Files Introduction Working in Oracle Platform Integration gives an engineer opportunities to work on a wide array of technologies. My team’s goal is to make Oracle applications run best on the Solaris/SPARC platform. When looking for bottlenecks in a modern applications, one needs to be aware of not only how the CPUs and operating system are executing, but also network, storage, and in some cases, the Java Virtual Machine. I was recently presented with about 1.5 GB of Java Garbage First Garbage Collector log file data. If you’re not familiar with the subject, you might want to review Garbage First Garbage Collector Tuning by Monica Beckwith. The customer had been running Java HotSpot 1.6.0_31 to host a web application server. I was told that the Solaris/SPARC server was running a Java process launched using a commmand line that included the following flags: -d64 -Xms9g -Xmx9g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=80 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCDateStamps -XX:+PrintFlagsFinal -XX:+DisableExplicitGC -XX:+UnlockExperimentalVMOptions -XX:ParallelGCThreads=8 Several sources on the internet indicate that if I were to print out the 1.5 GB of log files, it would require enough paper to fill the bed of a pick up truck. Of course, it would be fruitless to try to scan the log files by hand. Tools will be required to summarize the contents of the log files. Others have encountered large Java garbage collection log files. There are existing tools to analyze the log files: IBM’s GC toolkit The chewiebug GCViewer gchisto HPjmeter Instead of using one of the other tools listed, I decide to parse the log files with standard Unix tools, and analyze the data with R. Data Cleansing The log files arrived in two different formats. I guess that the difference is that one set of log files was generated using a more verbose option, maybe -XX:+PrintHeapAtGC, and the other set of log files was generated without that option. Format 1 In some of the log files, the log files with the less verbose format, a single trace, i.e. the report of a singe garbage collection event, looks like this: {Heap before GC invocations=12280 (full 61): garbage-first heap total 9437184K, used 7499918K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 1 young (4096K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. 2014-05-14T07:24:00.988-0700: 60586.353: [GC pause (young) 7324M->7320M(9216M), 0.1567265 secs] Heap after GC invocations=12281 (full 61): garbage-first heap total 9437184K, used 7496533K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 0 young (0K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. } A simple grep can be used to extract a summary: $ grep "\[ GC pause (young" g1gc.log 2014-05-13T13:24:35.091-0700: 3.109: [GC pause (young) 20M->5029K(9216M), 0.0146328 secs] 2014-05-13T13:24:35.440-0700: 3.459: [GC pause (young) 9125K->6077K(9216M), 0.0086723 secs] 2014-05-13T13:24:37.581-0700: 5.599: [GC pause (young) 25M->8470K(9216M), 0.0203820 secs] 2014-05-13T13:24:42.686-0700: 10.704: [GC pause (young) 44M->15M(9216M), 0.0288848 secs] 2014-05-13T13:24:48.941-0700: 16.958: [GC pause (young) 51M->20M(9216M), 0.0491244 secs] 2014-05-13T13:24:56.049-0700: 24.066: [GC pause (young) 92M->26M(9216M), 0.0525368 secs] 2014-05-13T13:25:34.368-0700: 62.383: [GC pause (young) 602M->68M(9216M), 0.1721173 secs] But that format wasn't easily read into R, so I needed to be a bit more tricky. I used the following Unix command to create a summary file that was easy for R to read. $ echo "SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime" $ grep "\[GC pause (young" g1gc.log | grep -v mark | sed -e 's/[A-SU-z,]/ /g' -e 's/->/ /' -e 's/: / /g' | more SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime 2014-05-13T13:24:35.091-0700 3.109 20 5029 9216 0.0146328 2014-05-13T13:24:35.440-0700 3.459 9125 6077 9216 0.0086723 2014-05-13T13:24:37.581-0700 5.599 25 8470 9216 0.0203820 2014-05-13T13:24:42.686-0700 10.704 44 15 9216 0.0288848 2014-05-13T13:24:48.941-0700 16.958 51 20 9216 0.0491244 2014-05-13T13:24:56.049-0700 24.066 92 26 9216 0.0525368 2014-05-13T13:25:34.368-0700 62.383 602 68 9216 0.1721173 Format 2 In some of the log files, the log files with the more verbose format, a single trace, i.e. the report of a singe garbage collection event, was more complicated than Format 1. Here is a text file with an example of a single G1GC trace in the second format. As you can see, it is quite complicated. It is nice that there is so much information available, but the level of detail can be overwhelming. I wrote this awk script (download) to summarize each trace on a single line. #!/usr/bin/env awk -f BEGIN { printf("SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize\n") } ###################### # Save count data from lines that are at the start of each G1GC trace. # Each trace starts out like this: # {Heap before GC invocations=14 (full 0): # garbage-first heap total 9437184K, used 325496K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) ###################### /{Heap.*full/{ gsub ( "\\)" , "" ); nf=split($0,a,"="); split(a[2],b," "); getline; if ( match($0, "first") ) { G1GC=1; IncrementalCount=b[1]; FullCount=substr( b[3], 1, length(b[3])-1 ); } else { G1GC=0; } } ###################### # Pull out time stamps that are in lines with this format: # 2014-05-12T14:02:06.025-0700: 94.312: [GC pause (young), 0.08870154 secs] ###################### /GC pause/ { DateTime=$1; SecondsSinceLaunch=substr($2, 1, length($2)-1); } ###################### # Heap sizes are in lines that look like this: # [ 4842M->4838M(9216M)] ###################### /\[ .*]$/ { gsub ( "\\[" , "" ); gsub ( "\ \]" , "" ); gsub ( "->" , " " ); gsub ( "\$ " , " " ); gsub ( "\ $" , " " ); split($0,a," "); if ( split(a[1],b,"M") > 1 ) {BeforeSize=b[1]*1024;} if ( split(a[1],b,"K") > 1 ) {BeforeSize=b[1];} if ( split(a[2],b,"M") > 1 ) {AfterSize=b[1]*1024;} if ( split(a[2],b,"K") > 1 ) {AfterSize=b[1];} if ( split(a[3],b,"M") > 1 ) {TotalSize=b[1]*1024;} if ( split(a[3],b,"K") > 1 ) {TotalSize=b[1];} } ###################### # Emit an output line when you find input that looks like this: # [Times: user=1.41 sys=0.08, real=0.24 secs] ###################### /\[Times/ { if (G1GC==1) { gsub ( "," , "" ); split($2,a,"="); UserTime=a[2]; split($3,a,"="); SysTime=a[2]; split($4,a,"="); RealTime=a[2]; print DateTime,SecondsSinceLaunch,IncrementalCount,FullCount,UserTime,SysTime,RealTime,BeforeSize,AfterSize,TotalSize; G1GC=0; } } The resulting summary is about 25X smaller that the original file, but still difficult for a human to digest. SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ... 2014-05-12T18:36:34.669-0700: 3985.744 561 0 0.57 0.06 0.16 1724416 1720320 9437184 2014-05-12T18:36:34.839-0700: 3985.914 562 0 0.51 0.06 0.19 1724416 1720320 9437184 2014-05-12T18:36:35.069-0700: 3986.144 563 0 0.60 0.04 0.27 1724416 1721344 9437184 2014-05-12T18:36:35.354-0700: 3986.429 564 0 0.33 0.04 0.09 1725440 1722368 9437184 2014-05-12T18:36:35.545-0700: 3986.620 565 0 0.58 0.04 0.17 1726464 1722368 9437184 2014-05-12T18:36:35.726-0700: 3986.801 566 0 0.43 0.05 0.12 1726464 1722368 9437184 2014-05-12T18:36:35.856-0700: 3986.930 567 0 0.30 0.04 0.07 1726464 1723392 9437184 2014-05-12T18:36:35.947-0700: 3987.023 568 0 0.61 0.04 0.26 1727488 1723392 9437184 2014-05-12T18:36:36.228-0700: 3987.302 569 0 0.46 0.04 0.16 1731584 1724416 9437184 Reading the Data into R Once the GC log data had been cleansed, either by processing the first format with the shell script, or by processing the second format with the awk script, it was easy to read the data into R. g1gc.df = read.csv("summary.txt", row.names = NULL, stringsAsFactors=FALSE,sep="") str(g1gc.df) ## 'data.frame': 8307 obs. of 10 variables: ## $ row.names : chr "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ... ## $ SecondsSinceLaunch: num 1.16 1.47 1.97 3.83 6.1 ... ## $ IncrementalCount : int 0 1 2 3 4 5 6 7 8 9 ... ## $ FullCount : int 0 0 0 0 0 0 0 0 0 0 ... ## $ UserTime : num 0.11 0.05 0.04 0.21 0.08 0.26 0.31 0.33 0.34 0.56 ... ## $ SysTime : num 0.04 0.01 0.01 0.05 0.01 0.06 0.07 0.06 0.07 0.09 ... ## $ RealTime : num 0.02 0.02 0.01 0.04 0.02 0.04 0.05 0.04 0.04 0.06 ... ## $ BeforeSize : int 8192 5496 5768 22528 24576 43008 34816 53248 55296 93184 ... ## $ AfterSize : int 1400 1672 2557 4907 7072 14336 16384 18432 19456 21504 ... ## $ TotalSize : int 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 ... head(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount ## 1 2014-05-12T14:00:32.868-0700: 1.161 0 ## 2 2014-05-12T14:00:33.179-0700: 1.472 1 ## 3 2014-05-12T14:00:33.677-0700: 1.969 2 ## 4 2014-05-12T14:00:35.538-0700: 3.830 3 ## 5 2014-05-12T14:00:37.811-0700: 6.103 4 ## 6 2014-05-12T14:00:41.428-0700: 9.720 5 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 1 0 0.11 0.04 0.02 8192 1400 9437184 ## 2 0 0.05 0.01 0.02 5496 1672 9437184 ## 3 0 0.04 0.01 0.01 5768 2557 9437184 ## 4 0 0.21 0.05 0.04 22528 4907 9437184 ## 5 0 0.08 0.01 0.02 24576 7072 9437184 ## 6 0 0.26 0.06 0.04 43008 14336 9437184 Basic Statistics Once the data has been read into R, simple statistics are very easy to generate. All of the numbers from high school statistics are available via simple commands. For example, generate a summary of every column: summary(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount FullCount ## Length:8307 Min. : 1 Min. : 0 Min. : 0.0 ## Class :character 1st Qu.: 9977 1st Qu.:2048 1st Qu.: 0.0 ## Mode :character Median :12855 Median :4136 Median : 12.0 ## Mean :12527 Mean :4156 Mean : 31.6 ## 3rd Qu.:15758 3rd Qu.:6262 3rd Qu.: 61.0 ## Max. :55484 Max. :8391 Max. :113.0 ## UserTime SysTime RealTime BeforeSize ## Min. :0.040 Min. :0.0000 Min. : 0.0 Min. : 5476 ## 1st Qu.:0.470 1st Qu.:0.0300 1st Qu.: 0.1 1st Qu.:5137920 ## Median :0.620 Median :0.0300 Median : 0.1 Median :6574080 ## Mean :0.751 Mean :0.0355 Mean : 0.3 Mean :5841855 ## 3rd Qu.:0.920 3rd Qu.:0.0400 3rd Qu.: 0.2 3rd Qu.:7084032 ## Max. :3.370 Max. :1.5600 Max. :488.1 Max. :8696832 ## AfterSize TotalSize ## Min. : 1380 Min. :9437184 ## 1st Qu.:5002752 1st Qu.:9437184 ## Median :6559744 Median :9437184 ## Mean :5785454 Mean :9437184 ## 3rd Qu.:7054336 3rd Qu.:9437184 ## Max. :8482816 Max. :9437184 Q: What is the total amount of User CPU time spent in garbage collection? sum(g1gc.df$UserTime) ## [1] 6236 As you can see, less than two hours of CPU time was spent in garbage collection. Is that too much? To find the percentage of time spent in garbage collection, divide the number above by total_elapsed_time*CPU_count. In this case, there are a lot of CPU’s and it turns out the the overall amount of CPU time spent in garbage collection isn’t a problem when viewed in isolation. When calculating rates, i.e. events per unit time, you need to ask yourself if the rate is homogenous across the time period in the log file. Does the log file include spikes of high activity that should be separately analyzed? Averaging in data from nights and weekends with data from business hours may alias problems. If you have a reason to suspect that the garbage collection rates include peaks and valleys that need independent analysis, see the “Time Series” section, below. Q: How much garbage is collected on each pass? The amount of heap space that is recovered per GC pass is surprisingly low: At least one collection didn’t recover any data. (“Min.=0”) 25% of the passes recovered 3MB or less. (“1st Qu.=3072”) Half of the GC passes recovered 4MB or less. (“Median=4096”) The average amount recovered was 56MB. (“Mean=56390”) 75% of the passes recovered 36MB or less. (“3rd Qu.=36860”) At least one pass recovered 2GB. (“Max.=2121000”) g1gc.df$Delta = g1gc.df$BeforeSize - g1gc.df$AfterSize summary(g1gc.df$Delta) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0 3070 4100 56400 36900 2120000 Q: What is the maximum User CPU time for a single collection? The worst garbage collection (“Max.”) is many standard deviations away from the mean. The data appears to be right skewed. summary(g1gc.df$UserTime) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.040 0.470 0.620 0.751 0.920 3.370 sd(g1gc.df$UserTime) ## [1] 0.3966 Basic Graphics Once the data is in R, it is trivial to plot the data with formats including dot plots, line charts, bar charts (simple, stacked, grouped), pie charts, boxplots, scatter plots histograms, and kernel density plots. Histogram of User CPU Time per Collection I don't think that this graph requires any explanation. hist(g1gc.df$UserTime, main="User CPU Time per Collection", xlab="Seconds", ylab="Frequency") Box plot to identify outliers When the initial data is viewed with a box plot, you can see the one crazy outlier in the real time per GC. Save this data point for future analysis and drop the outlier so that it’s not throwing off our statistics. Now the box plot shows many outliers, which will be examined later, using times series analysis. Notice that the scale of the x-axis changes drastically once the crazy outlier is removed. par(mfrow=c(2,1)) boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(dominated by a crazy outlier)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") crazy.outlier.df=g1gc.df[g1gc.df$RealTime > 400,] g1gc.df=g1gc.df[g1gc.df$RealTime < 400,] boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(crazy outlier excluded)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") box(which = "outer", lty = "solid") Here is the crazy outlier for future analysis: crazy.outlier.df ## row.names SecondsSinceLaunch IncrementalCount ## 8233 2014-05-12T23:15:43.903-0700: 20741 8316 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 8233 112 0.55 0.42 488.1 8381440 8235008 9437184 ## Delta ## 8233 146432 R Time Series Data To analyze the garbage collection as a time series, I’ll use Z’s Ordered Observations (zoo). “zoo is the creator for an S3 class of indexed totally ordered observations which includes irregular time series.” require(zoo) ## Loading required package: zoo ## ## Attaching package: 'zoo' ## ## The following objects are masked from 'package:base': ## ## as.Date, as.Date.numeric head(g1gc.df[,1]) ## [1] "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" ## [3] "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ## [5] "2014-05-12T14:00:37.811-0700:" "2014-05-12T14:00:41.428-0700:" options("digits.secs"=3) times=as.POSIXct( g1gc.df[,1], format="%Y-%m-%dT%H:%M:%OS%z:") g1gc.z = zoo(g1gc.df[,-c(1)], order.by=times) head(g1gc.z) ## SecondsSinceLaunch IncrementalCount FullCount ## 2014-05-12 17:00:32.868 1.161 0 0 ## 2014-05-12 17:00:33.178 1.472 1 0 ## 2014-05-12 17:00:33.677 1.969 2 0 ## 2014-05-12 17:00:35.538 3.830 3 0 ## 2014-05-12 17:00:37.811 6.103 4 0 ## 2014-05-12 17:00:41.427 9.720 5 0 ## UserTime SysTime RealTime BeforeSize AfterSize ## 2014-05-12 17:00:32.868 0.11 0.04 0.02 8192 1400 ## 2014-05-12 17:00:33.178 0.05 0.01 0.02 5496 1672 ## 2014-05-12 17:00:33.677 0.04 0.01 0.01 5768 2557 ## 2014-05-12 17:00:35.538 0.21 0.05 0.04 22528 4907 ## 2014-05-12 17:00:37.811 0.08 0.01 0.02 24576 7072 ## 2014-05-12 17:00:41.427 0.26 0.06 0.04 43008 14336 ## TotalSize Delta ## 2014-05-12 17:00:32.868 9437184 6792 ## 2014-05-12 17:00:33.178 9437184 3824 ## 2014-05-12 17:00:33.677 9437184 3211 ## 2014-05-12 17:00:35.538 9437184 17621 ## 2014-05-12 17:00:37.811 9437184 17504 ## 2014-05-12 17:00:41.427 9437184 28672 Example of Two Benchmark Runs in One Log File The data in the following graph is from a different log file, not the one of primary interest to this article. I’m including this image because it is an example of idle periods followed by busy periods. It would be uninteresting to average the rate of garbage collection over the entire log file period. More interesting would be the rate of garbage collect in the two busy periods. Are they the same or different? Your production data may be similar, for example, bursts when employees return from lunch and idle times on weekend evenings, etc. Once the data is in an R Time Series, you can analyze isolated time windows. Clipping the Time Series data Flashing back to our test case… Viewing the data as a time series is interesting. You can see that the work intensive time period is between 9:00 PM and 3:00 AM. Lets clip the data to the interesting period: par(mfrow=c(2,1)) plot(g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Complete Log File", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") clipped.g1gc.z=window(g1gc.z, start=as.POSIXct("2014-05-12 21:00:00"), end=as.POSIXct("2014-05-13 03:00:00")) plot(clipped.g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Limited to Benchmark Execution", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") box(which = "outer", lty = "solid") Cumulative Incremental and Full GC count Here is the cumulative incremental and full GC count. When the line is very steep, it indicates that the GCs are repeating very quickly. Notice that the scale on the Y axis is different for full vs. incremental. plot(clipped.g1gc.z[,c(2:3)], main="Cumulative Incremental and Full GC count", xlab="Time of Day", col="#1b9e77") GC Analysis of Benchmark Execution using Time Series data In the following series of 3 graphs: The “After Size” show the amount of heap space in use after each garbage collection. Many Java objects are still referenced, i.e. alive, during each garbage collection. This may indicate that the application has a memory leak, or may indicate that the application has a very large memory footprint. Typically, an application's memory footprint plateau's in the early stage of execution. One would expect this graph to have a flat top. The steep decline in the heap space may indicate that the application crashed after 2:00. The second graph shows that the outliers in real execution time, discussed above, occur near 2:00. when the Java heap seems to be quite full. The third graph shows that Full GCs are infrequent during the first few hours of execution. The rate of Full GC's, (the slope of the cummulative Full GC line), changes near midnight. plot(clipped.g1gc.z[,c("AfterSize","RealTime","FullCount")], xlab="Time of Day", col=c("#1b9e77","red","#1b9e77")) GC Analysis of heap recovered Each GC trace includes the amount of heap space in use before and after the individual GC event. During garbage coolection, unreferenced objects are identified, the space holding the unreferenced objects is freed, and thus, the difference in before and after usage indicates how much space has been freed. The following box plot and bar chart both demonstrate the same point - the amount of heap space freed per garbage colloection is surprisingly low. par(mfrow=c(2,1)) boxplot(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", horizontal = TRUE, col="red") hist(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", breaks=100, col="red") box(which = "outer", lty = "solid") This graph is the most interesting. The dark blue area shows how much heap is occupied by referenced Java objects. This represents memory that holds live data. The red fringe at the top shows how much data was recovered after each garbage collection. barplot(clipped.g1gc.z[,c("AfterSize","Delta")], col=c("#7570b3","#e7298a"), xlab="Time of Day", border=NA) legend("topleft", c("Live Objects","Heap Recovered on GC"), fill=c("#7570b3","#e7298a")) box(which = "outer", lty = "solid") When I discuss the data in the log files with the customer, I will ask for an explaination for the large amount of referenced data resident in the Java heap. There are two are posibilities: There is a memory leak and the amount of space required to hold referenced objects will continue to grow, limited only by the maximum heap size. After the maximum heap size is reached, the JVM will throw an “Out of Memory” exception every time that the application tries to allocate a new object. If this is the case, the aplication needs to be debugged to identify why old objects are referenced when they are no longer needed. The application has a legitimate requirement to keep a large amount of data in memory. The customer may want to further increase the maximum heap size. Another possible solution would be to partition the application across multiple cluster nodes, where each node has responsibility for managing a unique subset of the data. Conclusion In conclusion, R is a very powerful tool for the analysis of Java garbage collection log files. The primary difficulty is data cleansing so that information can be read into an R data frame. Once the data has been read into R, a rich set of tools may be used for thorough evaluation.

Read the article

Help with Boost Spirit ASTs

- by Decmac04

I am writing a small tool for analyzing simple B Machine substitutions as part of a college research work. The code successfully parse test inputs of the form mySubst := var1 + var2. However, I get a pop-up error message saying "This application has requested the Runtime to terminate it in an unusual way. " In the command prompt window, I get an "Assertion failed message". The main program is given below: // BMachineTree.cpp : Defines the entry point for the console application. // /*============================================================================= Copyright (c) 2010 Temitope Onunkun =============================================================================*/ /////////////////////////////////////////////////////////////////////////////// // // UUsing Boost Spririt Trees (AST) to parse B Machine Substitutions. // /////////////////////////////////////////////////////////////////////////////// #define BOOST_SPIRIT_DUMP_PARSETREE_AS_XML #include <boost/spirit/core.hpp> #include <boost/spirit/tree/ast.hpp> #include <boost/spirit/tree/tree_to_xml.hpp> #include "BMachineTreeGrammar.hpp" #include <iostream> #include <stack> #include <functional> #include <string> #include <cassert> #include <vector> #if defined(BOOST_SPIRIT_DUMP_PARSETREE_AS_XML) #include <map> #endif // Using AST to parse B Machine substitutions //////////////////////////////////////////////////////////////////////////// using namespace std; using namespace boost::spirit; typedef char const* iterator_t; typedef tree_match<iterator_t> parse_tree_match_t; typedef parse_tree_match_t::tree_iterator iter_t; //////////////////////////////////////////////////////////////////////////// string evaluate(parse_tree_match_t hit); string eval_machine(iter_t const& i); vector<string> dx; string evaluate(tree_parse_info<> info) { return eval_machine(info.trees.begin()); } string eval_machine(iter_t const& i) { cout << "In eval_machine. i->value = " << string(i->value.begin(), i->value.end()) << " i->children.size() = " << i->children.size() << endl; if (i->value.id() == substitution::leafValueID) { assert(i->children.size() == 0); // extract string tokens string leafValue(i->value.begin(), i->value.end()); dx.push_back(leafValue.c_str()); return leafValue.c_str(); } // else if (i->value.id() == substitution::termID) { if ( (*i->value.begin() == '*') || (*i->value.begin() == '/') ) { assert(i->children.size() == 2); dx.push_back( eval_machine(i->children.begin()) ); dx.push_back( eval_machine(i->children.begin()+1) ); return eval_machine(i->children.begin()) + " " + eval_machine(i->children.begin()+1); } // else assert(0); } else if (i->value.id() == substitution::expressionID) { if ( (*i->value.begin() == '+') || (*i->value.begin() == '-') ) { assert(i->children.size() == 2); dx.push_back( eval_machine(i->children.begin()) ); dx.push_back( eval_machine(i->children.begin()+1) ); return eval_machine(i->children.begin()) + " " + eval_machine(i->children.begin()+1); } else assert(0); } // else if (i->value.id() == substitution::simple_substID) { if (*i->value.begin() == (':' >> '=') ) { assert(i->children.size() == 2); dx.push_back( eval_machine(i->children.begin()) ); dx.push_back( eval_machine(i->children.begin()+1) ); return eval_machine(i->children.begin()) + "|->" + eval_machine(i->children.begin()+1); } else assert(0); } else { assert(0); // error } return 0; } //////////////////////////////////////////////////////////////////////////// int main() { // look in BMachineTreeGrammar for the definition of BMachine substitution BMach_subst; cout << "/////////////////////////////////////////////////////////\n\n"; cout << "\t\tB Machine Substitution...\n\n"; cout << "/////////////////////////////////////////////////////////\n\n"; cout << "Type an expression...or [q or Q] to quit\n\n"; string str; while (getline(cin, str)) { if (str.empty() || str[0] == 'q' || str[0] == 'Q') break; tree_parse_info<> info = ast_parse(str.c_str(), BMach_subst, space_p); if (info.full) { #if defined(BOOST_SPIRIT_DUMP_PARSETREE_AS_XML) // dump parse tree as XML std::map<parser_id, std::string> rule_names; rule_names[substitution::identifierID] = "identifier"; rule_names[substitution::leafValueID] = "leafValue"; rule_names[substitution::factorID] = "factor"; rule_names[substitution::termID] = "term"; rule_names[substitution::expressionID] = "expression"; rule_names[substitution::simple_substID] = "simple_subst"; tree_to_xml(cout, info.trees, str.c_str(), rule_names); #endif // print the result cout << "Variables in Vector dx: " << endl; for(vector<string>::iterator idx = dx.begin(); idx < dx.end(); ++idx) cout << *idx << endl; cout << "parsing succeeded\n"; cout << "result = " << evaluate(info) << "\n\n"; } else { cout << "parsing failed\n"; } } cout << "Bye... :-) \n\n"; return 0; } The grammar, defined in BMachineTreeGrammar.hpp file is given below: /*============================================================================= Copyright (c) 2010 Temitope Onunkun http://www.dcs.kcl.ac.uk/pg/onun Use, modification and distribution is subject to the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) =============================================================================*/ #ifndef BOOST_SPIRIT_BMachineTreeGrammar_HPP_ #define BOOST_SPIRIT_BMachineTreeGrammar_HPP_ using namespace boost::spirit; /////////////////////////////////////////////////////////////////////////////// // // Using Boost Spririt Trees (AST) to parse B Machine Substitutions. // /////////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////// // // B Machine Grammar // //////////////////////////////////////////////////////////////////////////// struct substitution : public grammar<substitution> { static const int identifierID = 1; static const int leafValueID = 2; static const int factorID = 3; static const int termID = 4; static const int expressionID = 5; static const int simple_substID = 6; template <typename ScannerT> struct definition { definition(substitution const& ) { // Start grammar definition identifier = alpha_p >> (+alnum_p | ch_p('_') ) ; leafValue = leaf_node_d[ lexeme_d[ identifier | +digit_p ] ] ; factor = leafValue | inner_node_d[ ch_p( '(' ) >> expression >> ch_p(')' ) ] ; term = factor >> *( (root_node_d[ch_p('*') ] >> factor ) | (root_node_d[ch_p('/') ] >> factor ) ); expression = term >> *( (root_node_d[ch_p('+') ] >> term ) | (root_node_d[ch_p('-') ] >> term ) ); simple_subst= leaf_node_d[ lexeme_d[ identifier ] ] >> root_node_d[str_p(":=")] >> expression ; // End grammar definition // turn on the debugging info. BOOST_SPIRIT_DEBUG_RULE(identifier); BOOST_SPIRIT_DEBUG_RULE(leafValue); BOOST_SPIRIT_DEBUG_RULE(factor); BOOST_SPIRIT_DEBUG_RULE(term); BOOST_SPIRIT_DEBUG_RULE(expression); BOOST_SPIRIT_DEBUG_RULE(simple_subst); } rule<ScannerT, parser_context<>, parser_tag<simple_substID> > simple_subst; rule<ScannerT, parser_context<>, parser_tag<expressionID> > expression; rule<ScannerT, parser_context<>, parser_tag<termID> > term; rule<ScannerT, parser_context<>, parser_tag<factorID> > factor; rule<ScannerT, parser_context<>, parser_tag<leafValueID> > leafValue; rule<ScannerT, parser_context<>, parser_tag<identifierID> > identifier; rule<ScannerT, parser_context<>, parser_tag<simple_substID> > const& start() const { return simple_subst; } }; }; #endif The output I get on running the program is: ///////////////////////////////////////////////////////// B Machine Substitution... ///////////////////////////////////////////////////////// Type an expression...or [q or Q] to quit mySubst := var1 - var2 parsing succeeded In eval_machine. i->value = := i->children.size() = 2 Assertion failed: 0, file c:\redmound\bmachinetree\bmachinetree\bmachinetree.cpp , line 114 I will appreciate any help in resolving this problem.

Read the article

Getting Selected Dropdown content to show in a form-generated email

- by fmz

I have a small contact form: <form method="post" action="contact.php" name="contactform" id="contactform"> <fieldset> <legend>Please fill in the following form to contact us</legend> <label for="name"><span class="required">*</span> Your Name</label> <input name="name" type="text" id="name" size="30" value="" /> <br /> <label for="company"><span class="required">*</span> Company</label> <input name="company" type="text" id="name" size="30" value="" /> <br /> <label for="email"><span class="required">*</span> Email</label> <input name="email" type="text" id="email" size="30" value="" /> <br /> <label for="phone"><span class="required">*</span> Phone</label> <input name="phone" type="text" id="phone" size="30" value="" /> <br /> <label for="purpose"><span class="required">*</span> Purpose</label> <select id="purpose" style="width: 300px; height:35px;"> <option value="I am interested in your services">I am interested in your services!</option> <option value="I am interested in a partnership">I am interested in a partnership!</option> <option value="I am interested in a job">I am interested in a job!</option> </select> <br /> <label for=comments><span class="required">*</span> Comments</label> <textarea name="comments" cols="40" rows="3" id="comments" style="width: 350px;"></textarea> <p><span class="required">*</span> Please help us control spam.</p> <label for=verify accesskey=V>   3 + 1 =</label> <input name="verify" type="text" id="verify" size="4" value="" style="width: 30px;" /><br /><br /> <input type="submit" class="submit" id="submit" value="Submit" /> </fieldset> </form> I want to send the results of the form in a php generated email. Everything is coming through except the selected contents of the "purpose" drop down. Here is the PHP: <?php if(!$_POST) exit; $name = $_POST['name']; $company = $_POST['company']; $email = $_POST['email']; $phone = $_POST['phone']; $purpose = $_POST['purpose']; $comments = $_POST['comments']; $verify = $_POST['verify']; if(trim($name) == '') { echo '<div class="error_message">Attention! You must enter your name.</div>'; exit(); } else if(trim($company) == '') { echo '<div class="error_message">Attention! Please enter your company name.</div>'; exit(); } else if(trim($email) == '') { echo '<div class="error_message">Attention! Please enter a valid email address.</div>'; exit(); } else if(trim($phone) == '') { echo '<div class="error_message">Attention! Please enter a valid phone number.</div>'; exit(); } else if(!isEmail($email)) { echo '<div class="error_message">Attention! You have enter an invalid e-mail address, try again.</div>'; exit(); } if(trim($comments) == '') { echo '<div class="error_message">Attention! Please enter your message.</div>'; exit(); } else if(trim($verify) == '') { echo '<div class="error_message">Attention! Please enter the verification number.</div>'; exit(); } else if(trim($verify) != '4') { echo '<div class="error_message">Attention! The verification number you entered is incorrect.</div>'; exit(); } if($error == '') { if(get_magic_quotes_gpc()) { $comments = stripslashes($comments); } // Configuration option. // Enter the email address that you want to emails to be sent to. // Example $address = "[email protected]"; $address = "[email protected]"; // Configuration option. // i.e. The standard subject will appear as, "You've been contacted by John Doe." // Example, $e_subject = '$name . ' has contacted you via Your Website.'; $e_subject = 'You\'ve been contacted by ' . $name . '.'; // Configuration option. // You can change this if you feel that you need to. // Developers, you may wish to add more fields to the form, in which case you must be sure to add them here. $e_body = "You have been contacted by $name.\r\n\n"; $e_content = "Comments: \"$comments\"\r\n\n"; $e_company = "Company: $company\r\n\n"; $e_purpose = "Reason for contact: $purpose\r\n"; $e_reply = "You can contact $name via email, $email or via phone $phone"; $msg = $e_body . $e_content . $e_company . $e_purpose . $e_reply; if(mail($address, $e_subject, $msg, "From: $email\r\nReply-To: $email\r\nReturn-Path: $email\r\n")) { // Email has sent successfully, echo a success page. echo "<fieldset>"; echo "<div id='success_page'>"; echo "<h1>Email Sent Successfully.</h1>"; echo "<p>Thank you <strong>$name</strong>, your message has been submitted to us.</p>"; echo "</div>"; echo "</fieldset>"; } else { echo 'ERROR!'; } } function isEmail($email) { // Email address verification, do not edit. return(preg_match("/^[-_.[:alnum:]]+@((([[:alnum:]]|[[:alnum:]][[:alnum:]-]*[[:alnum:]])\.)+(ad|ae|aero|af|ag|ai|al|am|an|ao|aq|ar|arpa|as|at|au|aw|az|ba|bb|bd|be|bf|bg|bh|bi|biz|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|com|coop|cr|cs|cu|cv|cx|cy|cz|de|dj|dk|dm|do|dz|ec|edu|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gh|gi|gl|gm|gn|gov|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|in|info|int|io|iq|ir|is|it|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|mg|mh|mil|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|museum|mv|mw|mx|my|mz|na|name|nc|ne|net|nf|ng|ni|nl|no|np|nr|nt|nu|nz|om|org|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|pro|ps|pt|pw|py|qa|re|ro|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|sk|sl|sm|sn|so|sr|st|su|sv|sy|sz|tc|td|tf|tg|th|tj|tk|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|um|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)$|(([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5])\.){3}([0-9][0-9]?|[0-1][0-9][0-9]|[2][0-4][0-9]|[2][5][0-5]))$/i",$email)); } ?> What am I missing? Thanks.

Read the article

FCKEditor doesn't set Value property on postback!

- by Shaul

I'm using FCKEditor on my asp.net web page. It appears beautifully, and the editor looks really good on the front end. Only problem is, the .Value property is not being set on the postback. No matter what changes the user makes to the value of the control on the page, when I click "Submit", the .Value property remains blank. I have Googled for other solutions, and most of them are of the variety where there's some conflict with Ajax, such as this and this. My problem is not solved by these solutions; it's much more fundamental than that. I'm not doing anything to do with Ajax; I'm just a simple asp.net newbie with a simple web form, and the value property is not being set on postback, not in IE and not in FF. It appears that at least one other person has had this problem, but no solution yet. Any ideas? Thanks! New information: I tried this out on a "hello world" test web site - and the test web site works 100%. There is obviously a problem on my page, but I have no idea where to begin tracking this down. Here's the markup of my page, in case anyone can see anything obvious that my newbie eyes can't: <%@ Page Language="vb" AutoEventWireup="false" CodeBehind="EmailTemplateEditForm.aspx.vb" Inherits="EEI_App.EmailTemplateEditForm" %> <%@ Register Assembly="FredCK.FCKeditorV2" Namespace="FredCK.FCKeditorV2" TagPrefix="FCKeditorV2" %> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head runat="server"> <title>EEI - Email Template</title> <link rel="stylesheet" href="EEI.css"> <script language="javascript" id="jssembleWare" src="sembleWare.js"></script> <style type="text/css"> .style1 { height: 251px; } .style2 { width: 2%; height: 251px; } .style3 { height: 490px; } </style> </head> <body> <form id="form1" runat="server"> <%@ register src="header.ascx" tagname="header" tagprefix="uc1" %> <%@ register src="footer.ascx" tagname="footer" tagprefix="uc1" %> <uc1:header ID="header1" runat="server" />  <div class="content">  <div class="boxheader"> Email Template </div> <div class="standardbox">   <div class="errorbox"> <asp:Label ID="lblError" CssClass="ErrorControlStyle" runat="server" EnableViewState="False" Width="100%"></asp:Label> </div> <table class="contenttable"> <tr> <td align="left" valign="top" class="style3"> <div class="actionbox"> <div class="navheadertitle"> Navigation</div> <ul> <li> <asp:LinkButton ID="btnSubmit" CssClass="LinkButtonStyle" runat="server">Submit</asp:LinkButton> </li> <li> <asp:LinkButton ID="btnCancel" CssClass="LinkButtonStyle" runat="server" CausesValidation="false">Cancel</asp:LinkButton> </li> </ul> </div> </td> <td align="left" valign="top" class="style3"> <p> </p> <table> <tr class="MCRSFieldRow"> <td class="MCRSFieldLabelCell"> <asp:Label ID="lblEmailTemplate_TemplateName" CssClass="LabelStyle" runat="server" Width="175">Template Name</asp:Label> </td> <td class="MCRSFieldEditCell"> <asp:TextBox ID="txtEmailTemplate_TemplateName" CssClass="TextBoxStyle" runat="server" Width="100%"></asp:TextBox> </td> <td class="MCRSFieldLabelCell"> <asp:Label ID="lblEmailTemplate_TemplateType" CssClass="LabelStyle" runat="server" Width="175">Template Type</asp:Label> </td> <td class="MCRSFieldEditCell"> <asp:RadioButtonList ID="rblEmailTemplate_TemplateType" CssClass="RadioButtonListStyle" runat="server" RepeatColumns="1" RepeatDirection="Horizontal" Width="135px"> <asp:ListItem Value="1">Cover Letter</asp:ListItem> <asp:ListItem Value="2">Email</asp:ListItem> </asp:RadioButtonList> </td> <td class="MCRSRowRightCell">   </td> </tr> <tr class="MCRSFieldRow"> <td class="MCRSFieldLabelCell"> Composition Date </td> <td class="MCRSFieldEditCell"> <asp:Label ID="lblEmailTemplate_CompositionDate" CssClass="ElementLabelStyle" runat="server" Width="175"></asp:Label> </td> <td class="MCRSFieldLabelCell"> Last Used Date </td> <td class="MCRSFieldEditCell"> <asp:Label ID="lblEmailTemplate_LastUsedDate" CssClass="ElementLabelStyle" runat="server" Width="175"></asp:Label> </td> <td class="MCRSRowRightCell">   </td> </tr> <tr class="MCRSFieldRow"> <td class="MCRSFieldLabelCell"> Composed By </td> <td class="MCRSFieldEditCell" colspan="3"> <asp:Label ID="lblPerson_FirstNames" CssClass="ElementLabelStyle" runat="server"></asp:Label> <asp:Label ID="lblPerson_LastName" CssClass="ElementLabelStyle" runat="server"></asp:Label> </td> <td class="MCRSRowRightCell">   </td> </tr> <tr class="MCRSFieldRow"> <td class="MCRSFieldLabelCell"> <asp:Label ID="lblEmailTemplate_Subject" CssClass="LabelStyle" runat="server" Width="175">Subject</asp:Label> </td> <td class="MCRSFieldEditCell" colspan="3"> <asp:TextBox ID="txtEmailTemplate_Subject" CssClass="TextBoxStyle" runat="server" Width="100%"></asp:TextBox> </td> <td class="MCRSRowRightCell">   </td> </tr> <tr class="MCRSFieldRow"> <td class="style1"> <asp:Label ID="lblEmailTemplate_Body" CssClass="LabelStyle" runat="server" Width="175">Body</asp:Label> </td> <td class="style1" colspan="3"> <FCKeditorV2:FCKeditor ID="FCKeditor1" runat="server" Height="500px"> </FCKeditorV2:FCKeditor> </td> <td class="style2">   </td> </tr> </table> </td> </tr> </table> </div> <p> <a class="InputButtonStyle" href="#_swTopOfPage">Top of Page</a> </p> </div> <uc1:footer ID="footer1" runat="server" /> <p> <asp:TextBox ID="txtEmailTemplate_Body" CssClass="TextAreaStyle" Rows="4" runat="server" Width="100%" Height="16px" Visible="False"></asp:TextBox> </p> </form> </body> </html>

Read the article

Component returned failure code: 0x80600011 [nsIXSLTProcessorObsolete.transformDocument]

- by Sean Ochoa

So, I'm using the XSLT plugin for JQuery, and here's my code: function AddPlotcardEventHandlers(){ // some code } function reportError(exception){ alert(exception.constructor.name + " Exception: " + ((exception.name) ? exception.name : "[unknown name]") + " - " + exception.message); } function GetPlotcards(){ $("#content").xslt("../xml/plotcards.xml","../xslt/plotcards.xsl", AddPlotcardEventHandlers,reportError); } Here's the modified jquery plugin. I say that its modified because I've added callbacks for success and error handling. /* * jquery.xslt.js * * Copyright (c) 2005-2008 Johann Burkard (<mailto:[email protected]>) * <http://eaio.com> * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included * in all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN * NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE * USE OR OTHER DEALINGS IN THE SOFTWARE. * */ /** * jQuery client-side XSLT plugins. * * @author <a href="mailto:[email protected]">Johann Burkard</a> * @version $Id: jquery.xslt.js,v 1.10 2008/08/29 21:34:24 Johann Exp $ */ (function($) { $.fn.xslt = function() { return this; } var str = /^\s*</; if (document.recalc) { // IE 5+ $.fn.xslt = function(xml, xslt, onSuccess, onError) { try{ var target = $(this); var change = function() { try{ var c = 'complete'; if (xm.readyState == c && xs.readyState == c) { window.setTimeout(function() { target.html(xm.transformNode(xs.XMLDocument)); if (onSuccess) onSuccess(); }, 50); } }catch(exception){ if (onError) onError(exception); } }; var xm = document.createElement('xml'); xm.onreadystatechange = change; xm[str.test(xml) ? "innerHTML" : "src"] = xml; var xs = document.createElement('xml'); xs.onreadystatechange = change; xs[str.test(xslt) ? "innerHTML" : "src"] = xslt; $('body').append(xm).append(xs); return this; }catch(exception){ if (onError) onError(exception); } }; } else if (window.DOMParser != undefined && window.XMLHttpRequest != undefined && window.XSLTProcessor != undefined) { // Mozilla 0.9.4+, Opera 9+ var processor = new XSLTProcessor(); var support = false; if ($.isFunction(processor.transformDocument)) { support = window.XMLSerializer != undefined; } else { support = true; } if (support) { $.fn.xslt = function(xml, xslt, onSuccess, onError) { try{ var target = $(this); var transformed = false; var xm = { readyState: 4 }; var xs = { readyState: 4 }; var change = function() { try{ if (xm.readyState == 4 && xs.readyState == 4 && !transformed) { var processor = new XSLTProcessor(); if ($.isFunction(processor.transformDocument)) { // obsolete Mozilla interface resultDoc = document.implementation.createDocument("", "", null); processor.transformDocument(xm.responseXML, xs.responseXML, resultDoc, null); target.html(new XMLSerializer().serializeToString(resultDoc)); } else { processor.importStylesheet(xs.responseXML); resultDoc = processor.transformToFragment(xm.responseXML, document); target.empty().append(resultDoc); } transformed = true; if (onSuccess) onSuccess(); } }catch(exception){ if (onError) onError(exception); } }; if (str.test(xml)) { xm.responseXML = new DOMParser().parseFromString(xml, "text/xml"); } else { xm = $.ajax({ dataType: "xml", url: xml}); xm.onreadystatechange = change; } if (str.test(xslt)) { xs.responseXML = new DOMParser().parseFromString(xslt, "text/xml"); change(); } else { xs = $.ajax({ dataType: "xml", url: xslt}); xs.onreadystatechange = change; } }catch(exception){ if (onError) onError(exception); }finally{ return this; } }; } } })(jQuery); And, here's my error msg: Object Exception: [unknown name] - Component returned failure code: 0x80600011 [nsIXSLTProcessorObsolete.transformDocument] Here's the info on the browser that I'm using for testing (with firebug v1.5.4 add-on installed): Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 Here's my XML: <?xml version="1.0" encoding="ISO-8859-1"?> <plotcardCollection sortby="order"> <plotcard order="2" id="1378"> <name><![CDATA[[placeholder for name of plotcard 1378]]]></name> <content><![CDATA[[placeholder for content of plotcard 1378]]]></content> <tagCollection> <tag id="3"><![CDATA[[placeholder for tag with id=3]]]></tag> <tag id="7"><![CDATA[[placeholder for tag with id=7]]]></tag> </tagCollection> </plotcard> <plotcard order="1" id="2156"> <name><![CDATA[[placeholder for name of plotcard 2156]]]></name> <content><![CDATA[[placeholder for content of plotcard 2156]]]></content> <tagCollection> <tag id="2"><![CDATA[[placeholder for tag with id=2]]]></tag> <tag id="9"><![CDATA[[placeholder for tag with id=9]]]></tag> </tagCollection> </plotcard> </plotcardCollection> Here's my XSLT: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:template match="/plotcardCollection"> <xsl:variable name="sortby" select="@sortby" /> <xsl:for-each select="plotcard"> <xsl:sort select="$sortby" data-type="number" order="ascending"/> <div>  <xsl:attribute name="class">Plotcard</xsl:attribute> <xsl:for-each select="@"> <xsl:value-of select="name()"/> <xsl:text>='</xsl:text> <xsl:if test="name() = 'id'"> <xsl:text>Plotcard-</xsl:text> </xsl:if> <xsl:value-of select="." /> <xsl:text>'</xsl:text> </xsl:for-each>  <div> <xsl:attribute name="class"> <xsl:text disable-output-escaping="yes">PlotcardName</xsl:text> </xsl:attribute> <xsl:value-of select="name/text()"/> </div>  <div> <xsl:attribute name="class"> <xsl:text disable-output-escaping="yes">PlotcardContent</xsl:text> </xsl:attribute> <xsl:value-of select="content/text()"/> </div> </div> </xsl:for-each> </xsl:template> </xsl:stylesheet> I'm really not sure what to do about this.... any thoughts?

Read the article

WordPress contact form email as PDF

- by lock

I am using the below code for my WordPress site which is emailing all the form details as an HTML text but I need the details to be written into a PDF first and then have to email the PDF as an attachment. How can I achieve this? This is not a PHP code to use PHP's writePDF modules. So, any idea or any code to implement this? <div style="padding-left: 100px;"> [raw] [contact-form subject="Best Aussie Broker" to="[email protected]"] <div id="main34" style="border: 1px solid black; border-radius: 15px; width: 720px; padding: 15px;">   <h2><span style="color: #ff6600;">Express Application</span></h2>   [contact-field label="First Name" type="name" required="true" /] [contact-field label="Last Name" type="text" /] [contact-field label="Email" type="email" required="true" /] [contact-field label="Purpose of Finance?" type="select" options="Home Loan,Refinance,Investment Loan,Debt Consolidation,Other" /] [contact-field label="Your deposit amount" type="text" /] [contact-field label="Amount you need to borrow?" type="text" /] [contact-field label="Brief description of the purpose for finance" type="textarea" required="true" /] <div><label></label> <input class="radio" type="radio" name="19" value="Single Application" onchange="showsingle();" /> <label class="radio">Single Application</label> <div class="clear-form"></div> <input class="radio" type="radio" name="19" value="Joint Application" onchange="showjoint();" /> <label class="radio">Joint Application</label> <div class="clear-form"></div> [contact-field label="Privacy Act" type="checkbox" required="true" /] I have read the Privacy Act 1988 (as Amended) and understand that by selecting the submit button I/we Authorize Best Aussie Broker to act on my/our behalf and manage personal information in relation to this application.<br> <a href="http://googleplex.com.au/pdf.pdf"><img src="http://googleplex.com.au/pdf.png" alt="" /> </a> </div> </div> <div id="single" style="display: none; width: 720px; border: 1px solid black; border-radius: 15px; padding: 15px; margin-top: 10px;"> <div style="padding-top: 10px; width: 720px; text-align: left;"> <h4><span style="color: #ff6600;">Last step then we will get all listed Australian vendors to fight it out for your best deal</span></h4> </div> <div> <label class="select" for="19-date-of-birth">Date of Birth</label> [contact-field label="Day" type="select" options="1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31" /] [contact-field label="Month" type="select" options="January,February,March,April,May,June,July,August,September,October,November,December" /] [contact-field label="Year" type="select" options="2000,1999,1998,1997,1996,1995,1994,1993,1992,1991,1990,1989,1988,1987,1986,1985,1984,1983,1982,1981,1980,1979,1978,197,1976,1975,1974,1973,1972,1971,1970,1969,1968,1967,1966,1965,1964,1963,1962,1961,1960,1959,1958,1957,1956,1955,1954,1953,1952,1951,1950,1949,1948,1947,1946,1945,1944,1943,1942,1941,1940,1939,1938,1937,1936,1935,1934,1933,1932,1931,1930,1929,1928,1927,1926,1925,1924,1923,1922,1921,1920, 1919,1918,1917,1916,1915,1914,1913,1912,1911,1910,1909" /] </div> [contact-field label="Address" type="text" /] [contact-field label="Suburb" type="text" /] [contact-field label="Postcode" type="text" /] <div> [contact-field label="State" type="select" options="VIC,NSW,QLD,SA,WA,TAS,NZ,Other" /] </div> [contact-field label="Best Contact" type="radio" options="Landline,Mobile" /] [contact-field label="Phone Number" type="text" /] [contact-field label="Marital Status" type="select" options="Married,Single,Other" /] [contact-field label="Residential Status" type="select" options="Renting, Home Owned, Home Mortgage, Board, Other" /] [contact-field label="Children/Dependents" type="select" options="0,1,2,3,4,5,6" /] <div></div> [contact-field label="Gross Yearly Income" type="text" /] [contact-field label="Current Employer" type="text" /] <div> <label class="select" for="19-year-of-empl">Time at this employer</label> [contact-field label="Year" type="select" options="0,1,2,3,4,5,6,7,8,9,10,More" /] [contact-field label="Month" type="select" options="0,1,2,3,4,5,6,7,8,9,10,11,12" /] </div> <div style="padding-right: 15px;"></div> </div> <div id="joint" style="display: none; width: 720px; border: 1px solid black; border-radius: 15px; padding: 15px; margin-top: 10px;"> <div style="padding-top: 10px; width: 720px; text-align: left;"> <h4><span style="color: #ff6600;">Last step then we will get all listed Australian vendors to fight it out for your best deal</span></h4> </div> <div style="float: left; width: 320px;"> <div> <label class="select" for="19-date-of-birth1">Date of Birth</label> [contact-field label="Day" type="select" options="1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31" /] [contact-field label="Month" type="select" options="January,February,March,April,May,June,July,August,September,October,November,December" /] [contact-field label="Year" type="select" options="2000,1999,1998,1997,1996,1995,1994,1993,1992,1991,1990,1989,1988,1987,1986,1985,1984,1983,1982,1981,1980,1979,1978,197,1976,1975,1974,1973,1972,1971,1970,1969,1968,1967,1966,1965,1964,1963,1962,1961,1960,1959,1958,1957,1956,1955,1954,1953,1952,1951,1950,1949,1948,1947,1946,1945,1944,1943,1942,1941,1940,1939,1938,1937,1936,1935,1934,1933,1932,1931,1930,1929,1928,1927,1926,1925,1924,1923,1922,1921,1920, 1919,1918,1917,1916,1915,1914,1913,1912,1911,1910,1909" /] </div> [contact-field label="Address" type="text" /] [contact-field label="Suburb" type="text" /] [contact-field label="Postcode" type="text" /] <div> [contact-field label="State" type="select" options="VIC,NSW,QLD,SA,WA,TAS,NZ,Other" /] </div> [contact-field label="Best Contact" type="radio" options="Landline,Mobile" /] [contact-field label="Phone Number" type="text" /] <div></div> <div></div> [contact-field label="Marital Status" type="select" options="Married,Single,Other" /] [contact-field label="Residential Status" type="select" options="Renting, Home Owned, Home Mortgage, Board, Other" /] [contact-field label="Children/Dependents" type="select" options="0,1,2,3,4,5,6" /] <div></div> <div><label class="text" for="netincome">Net Income</label> <input id="netincome" type="text" name="netincome" /> <select id="netincome-dropdown" name="netincome-dropdown"> <option>Monthly</option> <option>Yearly</option> </select></div> [contact-field label="Current Employer" type="text" /] <div> <label class="select" for="19-year-of-empl2">Time at this employer</label> [contact-field label="Year" type="select" options="0,1,2,3,4,5,6,7,8,9,10,More" /] [contact-field label="Month" type="select" options="0,1,2,3,4,5,6,7,8,9,10,11,12" /] </div> </div> <div style="float: right; width: 320px; padding-right: 50px;"> <div> <label class="select" for="19-date-of-birth3">Date of Birth</label> [contact-field label="Day" type="select" options="1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31" /] [contact-field label="Month" type="select" options="January,February,March,April,May,June,July,August,September,October,November,December" /] [contact-field label="Year" type="select" options="2000,1999,1998,1997,1996,1995,1994,1993,1992,1991,1990,1989,1988,1987,1986,1985,1984,1983,1982,1981,1980,1979,1978,197,1976,1975,1974,1973,1972,1971,1970,1969,1968,1967,1966,1965,1964,1963,1962,1961,1960,1959,1958,1957,1956,1955,1954,1953,1952,1951,1950,1949,1948,1947,1946,1945,1944,1943,1942,1941,1940,1939,1938,1937,1936,1935,1934,1933,1932,1931,1930,1929,1928,1927,1926,1925,1924,1923,1922,1921,1920, 1919,1918,1917,1916,1915,1914,1913,1912,1911,1910,1909" /] </div> [contact-field label="Address" type="text" /] [contact-field label="Suburb" type="text" /] [contact-field label="Postcode" type="text" /] <div> [contact-field label="State" type="select" options="VIC,NSW,QLD,SA,WA,TAS,NZ,Other" /] </div> [contact-field label="Best Contact" type="radio" options="Landline,Mobile" /] [contact-field label="Phone Number" type="text" /] <div></div> <div></div> [contact-field label="Marital Status" type="select" options="Married,Single,Other" /] [contact-field label="Residential Status" type="select" options="Renting, Home Owned, Home Mortgage, Board, Other" /] [contact-field label="Children/Dependents" type="select" options="0,1,2,3,4,5,6" /] <div></div> <div><label class="text" for="netincome">Net Income</label> <input id="netincome" type="text" name="netincome" /> <select id="netincome-dropdown" name="netincome-dropdown"> <option>Monthly</option> <option>Yearly</option> </select></div> [contact-field label="Current Employer" type="text" /] <div> <label class="select" for="19-year-of-empl">Time at this employer</label> [contact-field label="Year" type="select" options="0,1,2,3,4,5,6,7,8,9,10,More" /] [contact-field label="Month" type="select" options="0,1,2,3,4,5,6,7,8,9,10,11,12" /] </div> </div> <div style="clear: both;"></div> <div></div> </div>   [/contact-form][/raw] </div>

Read the article

Problems extracting information from RSS feed description field

- by Graeme

Hi, I've built an iPhone application using the parsing code from the TopSongs sample iPhone application. I've hit a problem though - the feed I'm trying to parse data from doesn't have a separate field for every piece of information (i.e. if it was for a feed about dogs, all the information such as dog type, dog age and dog price is contained in the feed. However, the TopSongs app relies on information having its own tags, so instead of using it uses and . So my question is this. How do I extract this information from the description field so that it can be parsed using the TopSongs parser? Can you somehow extract the dog age, price and type information using Yahoo Pipes and use that RSS feed for the feed? Or is there code that I can add to do it in application? Update: To view the code of my application parser (based on the TopSongs Core Data Apple provided application, see below. Here's a sample of one item from the the actual RSS feed I'm using (the description is longer, and has status,size, and a couple of other fields, but they're all formatted the same.: <item> <title>MOE, MARGRET STREET</title> <description> <b>District/Region:</b> REGION 09</br><b>Location:</b> MOE</br><b>Name:</b> MARGRET STREET</br></description> <pubDate>Thu,11 Mar 2010 05:43:03 GMT</pubDate> <guid>1266148</guid> </item> /* File: iTunesRSSImporter.m Abstract: Downloads, parses, and imports the iTunes top songs RSS feed into Core Data. Version: 1.1 Disclaimer: IMPORTANT: This Apple software is supplied to you by Apple Inc. ("Apple") in consideration of your agreement to the following terms, and your use, installation, modification or redistribution of this Apple software constitutes acceptance of these terms. If you do not agree with these terms, please do not use, install, modify or redistribute this Apple software. In consideration of your agreement to abide by the following terms, and subject to these terms, Apple grants you a personal, non-exclusive license, under Apple's copyrights in this original Apple software (the "Apple Software"), to use, reproduce, modify and redistribute the Apple Software, with or without modifications, in source and/or binary forms; provided that if you redistribute the Apple Software in its entirety and without modifications, you must retain this notice and the following text and disclaimers in all such redistributions of the Apple Software. Neither the name, trademarks, service marks or logos of Apple Inc. may be used to endorse or promote products derived from the Apple Software without specific prior written permission from Apple. Except as expressly stated in this notice, no other rights or licenses, express or implied, are granted by Apple herein, including but not limited to any patent rights that may be infringed by your derivative works or by other works in which the Apple Software may be incorporated. The Apple Software is provided by Apple on an "AS IS" basis. APPLE MAKES NO WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, REGARDING THE APPLE SOFTWARE OR ITS USE AND OPERATION ALONE OR IN COMBINATION WITH YOUR PRODUCTS. IN NO EVENT SHALL APPLE BE LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) ARISING IN ANY WAY OUT OF THE USE, REPRODUCTION, MODIFICATION AND/OR DISTRIBUTION OF THE APPLE SOFTWARE, HOWEVER CAUSED AND WHETHER UNDER THEORY OF CONTRACT, TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY OR OTHERWISE, EVEN IF APPLE HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Copyright (C) 2009 Apple Inc. All Rights Reserved. */ #import "iTunesRSSImporter.h" #import "Song.h" #import "Category.h" #import "CategoryCache.h" #import <libxml/tree.h> // Function prototypes for SAX callbacks. This sample implements a minimal subset of SAX callbacks. // Depending on your application's needs, you might want to implement more callbacks. static void startElementSAX(void *context, const xmlChar *localname, const xmlChar *prefix, const xmlChar *URI, int nb_namespaces, const xmlChar **namespaces, int nb_attributes, int nb_defaulted, const xmlChar **attributes); static void endElementSAX(void *context, const xmlChar *localname, const xmlChar *prefix, const xmlChar *URI); static void charactersFoundSAX(void *context, const xmlChar *characters, int length); static void errorEncounteredSAX(void *context, const char *errorMessage, ...); // Forward reference. The structure is defined in full at the end of the file. static xmlSAXHandler simpleSAXHandlerStruct; // Class extension for private properties and methods. @interface iTunesRSSImporter () @property BOOL storingCharacters; @property (nonatomic, retain) NSMutableData *characterBuffer; @property BOOL done; @property BOOL parsingASong; @property NSUInteger countForCurrentBatch; @property (nonatomic, retain) Song *currentSong; @property (nonatomic, retain) NSURLConnection *rssConnection; @property (nonatomic, retain) NSDateFormatter *dateFormatter; // The autorelease pool property is assign because autorelease pools cannot be retained. @property (nonatomic, assign) NSAutoreleasePool *importPool; @end static double lookuptime = 0; @implementation iTunesRSSImporter @synthesize iTunesURL, delegate, persistentStoreCoordinator; @synthesize rssConnection, done, parsingASong, storingCharacters, currentSong, countForCurrentBatch, characterBuffer, dateFormatter, importPool; - (void)dealloc { [iTunesURL release]; [characterBuffer release]; [currentSong release]; [rssConnection release]; [dateFormatter release]; [persistentStoreCoordinator release]; [insertionContext release]; [songEntityDescription release]; [theCache release]; [super dealloc]; } - (void)main { self.importPool = [[NSAutoreleasePool alloc] init]; if (delegate && [delegate respondsToSelector:@selector(importerDidSave:)]) { [[NSNotificationCenter defaultCenter] addObserver:delegate selector:@selector(importerDidSave:) name:NSManagedObjectContextDidSaveNotification object:self.insertionContext]; } done = NO; self.dateFormatter = [[[NSDateFormatter alloc] init] autorelease]; [dateFormatter setDateStyle:NSDateFormatterLongStyle]; [dateFormatter setTimeStyle:NSDateFormatterNoStyle]; // necessary because iTunes RSS feed is not localized, so if the device region has been set to other than US // the date formatter must be set to US locale in order to parse the dates [dateFormatter setLocale:[[[NSLocale alloc] initWithLocaleIdentifier:@"US"] autorelease]]; self.characterBuffer = [NSMutableData data]; NSURLRequest *theRequest = [NSURLRequest requestWithURL:iTunesURL]; // create the connection with the request and start loading the data rssConnection = [[NSURLConnection alloc] initWithRequest:theRequest delegate:self]; // This creates a context for "push" parsing in which chunks of data that are not "well balanced" can be passed // to the context for streaming parsing. The handler structure defined above will be used for all the parsing. // The second argument, self, will be passed as user data to each of the SAX handlers. The last three arguments // are left blank to avoid creating a tree in memory. context = xmlCreatePushParserCtxt(&simpleSAXHandlerStruct, self, NULL, 0, NULL); if (rssConnection != nil) { do { [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]]; } while (!done); } // Display the total time spent finding a specific object for a relationship NSLog(@"lookup time %f", lookuptime); // Release resources used only in this thread. xmlFreeParserCtxt(context); self.characterBuffer = nil; self.dateFormatter = nil; self.rssConnection = nil; self.currentSong = nil; [theCache release]; theCache = nil; NSError *saveError = nil; NSAssert1([insertionContext save:&saveError], @"Unhandled error saving managed object context in import thread: %@", [saveError localizedDescription]); if (delegate && [delegate respondsToSelector:@selector(importerDidSave:)]) { [[NSNotificationCenter defaultCenter] removeObserver:delegate name:NSManagedObjectContextDidSaveNotification object:self.insertionContext]; } if (self.delegate != nil && [self.delegate respondsToSelector:@selector(importerDidFinishParsingData:)]) { [self.delegate importerDidFinishParsingData:self]; } [importPool release]; self.importPool = nil; } - (NSManagedObjectContext *)insertionContext { if (insertionContext == nil) { insertionContext = [[NSManagedObjectContext alloc] init]; [insertionContext setPersistentStoreCoordinator:self.persistentStoreCoordinator]; } return insertionContext; } - (void)forwardError:(NSError *)error { if (self.delegate != nil && [self.delegate respondsToSelector:@selector(importer:didFailWithError:)]) { [self.delegate importer:self didFailWithError:error]; } } - (NSEntityDescription *)songEntityDescription { if (songEntityDescription == nil) { songEntityDescription = [[NSEntityDescription entityForName:@"Song" inManagedObjectContext:self.insertionContext] retain]; } return songEntityDescription; } - (CategoryCache *)theCache { if (theCache == nil) { theCache = [[CategoryCache alloc] init]; theCache.managedObjectContext = self.insertionContext; } return theCache; } - (Song *)currentSong { if (currentSong == nil) { currentSong = [[Song alloc] initWithEntity:self.songEntityDescription insertIntoManagedObjectContext:self.insertionContext]; } return currentSong; } #pragma mark NSURLConnection Delegate methods // Forward errors to the delegate. - (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error { [self performSelectorOnMainThread:@selector(forwardError:) withObject:error waitUntilDone:NO]; // Set the condition which ends the run loop. done = YES; } // Called when a chunk of data has been downloaded. - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data { // Process the downloaded chunk of data. xmlParseChunk(context, (const char *)[data bytes], [data length], 0); } - (void)connectionDidFinishLoading:(NSURLConnection *)connection { // Signal the context that parsing is complete by passing "1" as the last parameter. xmlParseChunk(context, NULL, 0, 1); context = NULL; // Set the condition which ends the run loop. done = YES; } #pragma mark Parsing support methods static const NSUInteger kImportBatchSize = 20; - (void)finishedCurrentSong { parsingASong = NO; self.currentSong = nil; countForCurrentBatch++; // Periodically purge the autorelease pool and save the context. The frequency of this action may need to be tuned according to the // size of the objects being parsed. The goal is to keep the autorelease pool from growing too large, but // taking this action too frequently would be wasteful and reduce performance. if (countForCurrentBatch == kImportBatchSize) { [importPool release]; self.importPool = [[NSAutoreleasePool alloc] init]; NSError *saveError = nil; NSAssert1([insertionContext save:&saveError], @"Unhandled error saving managed object context in import thread: %@", [saveError localizedDescription]); countForCurrentBatch = 0; } } /* Character data is appended to a buffer until the current element ends. */ - (void)appendCharacters:(const char *)charactersFound length:(NSInteger)length { [characterBuffer appendBytes:charactersFound length:length]; } - (NSString *)currentString { // Create a string with the character data using UTF-8 encoding. UTF-8 is the default XML data encoding. NSString *currentString = [[[NSString alloc] initWithData:characterBuffer encoding:NSUTF8StringEncoding] autorelease]; [characterBuffer setLength:0]; return currentString; } @end #pragma mark SAX Parsing Callbacks // The following constants are the XML element names and their string lengths for parsing comparison. // The lengths include the null terminator, to ensure exact matches. static const char *kName_Item = "item"; static const NSUInteger kLength_Item = 5; static const char *kName_Title = "title"; static const NSUInteger kLength_Title = 6; static const char *kName_Category = "category"; static const NSUInteger kLength_Category = 9; static const char *kName_Itms = "itms"; static const NSUInteger kLength_Itms = 5; static const char *kName_Artist = "description"; static const NSUInteger kLength_Artist = 7; static const char *kName_Album = "description"; static const NSUInteger kLength_Album = 6; static const char *kName_ReleaseDate = "releasedate"; static const NSUInteger kLength_ReleaseDate = 12; /* This callback is invoked when the importer finds the beginning of a node in the XML. For this application, out parsing needs are relatively modest - we need only match the node name. An "item" node is a record of data about a song. In that case we create a new Song object. The other nodes of interest are several of the child nodes of the Song currently being parsed. For those nodes we want to accumulate the character data in a buffer. Some of the child nodes use a namespace prefix. */ static void startElementSAX(void *parsingContext, const xmlChar *localname, const xmlChar *prefix, const xmlChar *URI, int nb_namespaces, const xmlChar **namespaces, int nb_attributes, int nb_defaulted, const xmlChar **attributes) { iTunesRSSImporter *importer = (iTunesRSSImporter *)parsingContext; // The second parameter to strncmp is the name of the element, which we known from the XML schema of the feed. // The third parameter to strncmp is the number of characters in the element name, plus 1 for the null terminator. if (prefix == NULL && !strncmp((const char *)localname, kName_Item, kLength_Item)) { importer.parsingASong = YES; } else if (importer.parsingASong && ( (prefix == NULL && (!strncmp((const char *)localname, kName_Title, kLength_Title) || !strncmp((const char *)localname, kName_Category, kLength_Category))) || ((prefix != NULL && !strncmp((const char *)prefix, kName_Itms, kLength_Itms)) && (!strncmp((const char *)localname, kName_Artist, kLength_Artist) || !strncmp((const char *)localname, kName_Album, kLength_Album) || !strncmp((const char *)localname, kName_ReleaseDate, kLength_ReleaseDate))) )) { importer.storingCharacters = YES; } } /* This callback is invoked when the parse reaches the end of a node. At that point we finish processing that node, if it is of interest to us. For "item" nodes, that means we have completed parsing a Song object. We pass the song to a method in the superclass which will eventually deliver it to the delegate. For the other nodes we care about, this means we have all the character data. The next step is to create an NSString using the buffer contents and store that with the current Song object. */ static void endElementSAX(void *parsingContext, const xmlChar *localname, const xmlChar *prefix, const xmlChar *URI) { iTunesRSSImporter *importer = (iTunesRSSImporter *)parsingContext; if (importer.parsingASong == NO) return; if (prefix == NULL) { if (!strncmp((const char *)localname, kName_Item, kLength_Item)) { [importer finishedCurrentSong]; } else if (!strncmp((const char *)localname, kName_Title, kLength_Title)) { importer.currentSong.title = importer.currentString; } else if (!strncmp((const char *)localname, kName_Category, kLength_Category)) { double before = [NSDate timeIntervalSinceReferenceDate]; Category *category = [importer.theCache categoryWithName:importer.currentString]; double delta = [NSDate timeIntervalSinceReferenceDate] - before; lookuptime += delta; importer.currentSong.category = category; } } else if (!strncmp((const char *)prefix, kName_Itms, kLength_Itms)) { if (!strncmp((const char *)localname, kName_Artist, kLength_Artist)) { NSString *string = importer.currentSong.artist; NSArray *strings = [string componentsSeparatedByString: @", "]; //importer.currentSong.artist = importer.currentString; } else if (!strncmp((const char *)localname, kName_Album, kLength_Album)) { importer.currentSong.album = importer.currentString; } else if (!strncmp((const char *)localname, kName_ReleaseDate, kLength_ReleaseDate)) { NSString *dateString = importer.currentString; importer.currentSong.releaseDate = [importer.dateFormatter dateFromString:dateString]; } } importer.storingCharacters = NO; } /* This callback is invoked when the parser encounters character data inside a node. The importer class determines how to use the character data. */ static void charactersFoundSAX(void *parsingContext, const xmlChar *characterArray, int numberOfCharacters) { iTunesRSSImporter *importer = (iTunesRSSImporter *)parsingContext; // A state variable, "storingCharacters", is set when nodes of interest begin and end. // This determines whether character data is handled or ignored. if (importer.storingCharacters == NO) return; [importer appendCharacters:(const char *)characterArray length:numberOfCharacters]; } /* A production application should include robust error handling as part of its parsing implementation. The specifics of how errors are handled depends on the application. */ static void errorEncounteredSAX(void *parsingContext, const char *errorMessage, ...) { // Handle errors as appropriate for your application. NSCAssert(NO, @"Unhandled error encountered during SAX parse."); } // The handler struct has positions for a large number of callback functions. If NULL is supplied at a given position, // that callback functionality won't be used. Refer to libxml documentation at http://www.xmlsoft.org for more information // about the SAX callbacks. static xmlSAXHandler simpleSAXHandlerStruct = { NULL, /* internalSubset */ NULL, /* isStandalone */ NULL, /* hasInternalSubset */ NULL, /* hasExternalSubset */ NULL, /* resolveEntity */ NULL, /* getEntity */ NULL, /* entityDecl */ NULL, /* notationDecl */ NULL, /* attributeDecl */ NULL, /* elementDecl */ NULL, /* unparsedEntityDecl */ NULL, /* setDocumentLocator */ NULL, /* startDocument */ NULL, /* endDocument */ NULL, /* startElement*/ NULL, /* endElement */ NULL, /* reference */ charactersFoundSAX, /* characters */ NULL, /* ignorableWhitespace */ NULL, /* processingInstruction */ NULL, /* comment */ NULL, /* warning */ errorEncounteredSAX, /* error */ NULL, /* fatalError //: unused error() get all the errors */ NULL, /* getParameterEntity */ NULL, /* cdataBlock */ NULL, /* externalSubset */ XML_SAX2_MAGIC, // NULL, startElementSAX, /* startElementNs */ endElementSAX, /* endElementNs */ NULL, /* serror */ }; Thanks.

Read the article

Re: Help with Boost Grammar

- by Decmac04

I have redesigned and extended the grammar I asked about earlier as shown below: // BIFAnalyser.cpp : Defines the entry point for the console application. // // /*============================================================================= Copyright (c) Temitope Jos Onunkun 2010 http://www.dcs.kcl.ac.uk/pg/onun/ Use, modification and distribution is subject to the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) =============================================================================*/ //////////////////////////////////////////////////////////////////////////// // // // B Machine parser using the Boost "Grammar" and "Semantic Actions". // // // //////////////////////////////////////////////////////////////////////////// include include include include include include //////////////////////////////////////////////////////////////////////////// using namespace std; using namespace boost::spirit; //////////////////////////////////////////////////////////////////////////// // // Semantic Actions // //////////////////////////////////////////////////////////////////////////// // // namespace { //semantic action function on individual lexeme void do_noint(char const* start, char const* end) { string str(start, end); if (str != "NAT1") cout << "PUSH(" << str << ')' << endl; } //semantic action function on addition of lexemes void do_add(char const*, char const*) { cout << "ADD" << endl; // for(vector::iterator vi = strVect.begin(); vi < strVect.end(); ++vi) // cout << *vi << " "; } //semantic action function on subtraction of lexemes void do_subt(char const*, char const*) { cout << "SUBTRACT" << endl; } //semantic action function on multiplication of lexemes void do_mult(char const*, char const*) { cout << "\nMULTIPLY" << endl; } //semantic action function on division of lexemes void do_div(char const*, char const*) { cout << "\nDIVIDE" << endl; } // // vector flowTable; //semantic action function on simple substitution void do_sSubst(char const* start, char const* end) { string str(start, end); //use boost tokenizer to break down tokens typedef boost::tokenizer Tokenizer; boost::char_separator sep(" -+/*:=()",0,boost::drop_empty_tokens); // char separator definition Tokenizer tok(str, sep); Tokenizer::iterator tok_iter = tok.begin(); pair dependency; //create a pair object for dependencies //create a vector object to store all tokens vector dx; // int counter = 0; // tracks token position for(tok.begin(); tok_iter != tok.end(); ++tok_iter) //save all tokens in vector { dx.push_back(*tok_iter ); } counter = dx.size(); // vector d_hat; //stores set of dependency pairs string dep; //pairs variables as string object // dependency.first = *tok.begin(); vector FV; for(int unsigned i=1; i < dx.size(); i++) { // if(!atoi(dx.at(i).c_str()) && (dx.at(i) !=" ")) { dependency.second = dx.at(i); dep = dependency.first + "|-" + dependency.second + " "; d_hat.push_back(dep); vector<string> row; row.push_back(dependency.first); //push x_hat into first column of each row for(unsigned int j=0; j<2; j++) { row.push_back(dependency.second);//push an element (column) into the row } flowTable.push_back(row); //Add the row to the main vector } } //displays internal representation of information flow table cout << "\n****************\nDependency Table\n****************\n"; cout << "X_Hat\tDx\tG_Hat\n"; cout << "-----------------------------\n"; for(unsigned int i=0; i < flowTable.size(); i++) { for(unsigned int j=0; j<2; j++) { cout << flowTable[i][j] << "\t "; } if (*tok.begin() != "WHILE" ) //if there are no global flows, cout << "\t{}"; //display empty set cout << "\n"; } cout << "***************\n\n"; for(int unsigned j=0; j < FV.size(); j++) { if(FV.at(j) != dependency.second) dep = dependency.first + "|-" + dependency.second + " "; d_hat.push_back(dep); } cout << "PUSH(" << str << ')' << endl; cout << "\n*******\nDependency pairs\n*******\n"; for(int unsigned i=0; i < d_hat.size(); i++) cout << d_hat.at(i) << "\n...\n"; cout << "\nSIMPLE SUBSTITUTION\n\n"; } //semantic action function on multiple substitution void do_mSubst(char const* start, char const* end) { string str(start, end); cout << "PUSH(" << str << ')' << endl; //cout << "\nMULTIPLE SUBSTITUTION\n\n"; } //semantic action function on unbounded choice substitution void do_mChoice(char const* start, char const* end) { string str(start, end); cout << "PUSH(" << str << ')' << endl; cout << "\nUNBOUNDED CHOICE SUBSTITUTION\n\n"; } void do_logicExpr(char const* start, char const* end) { string str(start, end); //use boost tokenizer to break down tokens typedef boost::tokenizer Tokenizer; boost::char_separator sep(" -+/*=:()<",0,boost::drop_empty_tokens); // char separator definition Tokenizer tok(str, sep); Tokenizer::iterator tok_iter = tok.begin(); //pair dependency; //create a pair object for dependencies //create a vector object to store all tokens vector dx; for(tok.begin(); tok_iter != tok.end(); ++tok_iter) //save all tokens in vector { dx.push_back(*tok_iter ); } for(unsigned int i=0; i cout << "PUSH(" << str << ')' << endl; cout << "\nPREDICATE\n\n"; } void do_predicate(char const* start, char const* end) { string str(start, end); cout << "PUSH(" << str << ')' << endl; cout << "\nMULTIPLE PREDICATE\n\n"; } void do_ifSelectPre(char const* start, char const* end) { string str(start, end); //if cout << "PUSH(" << str << ')' << endl; cout << "\nPROTECTED SUBSTITUTION\n\n"; } //semantic action function on machine substitution void do_machSubst(char const* start, char const* end) { string str(start, end); cout << "PUSH(" << str << ')' << endl; cout << "\nMACHINE SUBSTITUTION\n\n"; } } //////////////////////////////////////////////////////////////////////////// // // Machine Substitution Grammar // //////////////////////////////////////////////////////////////////////////// // Simple substitution grammar parser with integer values removed struct Substitution : public grammar { template struct definition { definition(Substitution const& ) { machine_subst = ( (simple_subst) | (multi_subst) | (if_select_pre_subst) | (unbounded_choice) )[&do_machSubst] ; unbounded_choice = str_p("ANY") ide_list str_p("WHERE") predicate str_p("THEN") machine_subst str_p("END") ; if_select_pre_subst = ( ( str_p("IF") predicate str_p("THEN") machine_subst *( str_p("ELSIF") predicate machine_subst ) !( str_p("ELSE") machine_subst) str_p("END") ) | ( str_p("SELECT") predicate str_p("THEN") machine_subst *( str_p("WHEN") predicate machine_subst ) !( str_p("ELSE") machine_subst) str_p("END")) | ( str_p("PRE") predicate str_p("THEN") machine_subst str_p("END") ) )[&do_ifSelectPre] ; multi_subst = ( (machine_subst) *( ( str_p("||") (machine_subst) ) | ( str_p("[]") (machine_subst) ) ) ) [&do_mSubst] ; simple_subst = (identifier str_p(":=") arith_expr) [&do_sSubst] ; expression = predicate | arith_expr ; predicate = ( (logic_expr) *( ( ch_p('&') (logic_expr) ) | ( str_p("OR") (logic_expr) ) ) )[&do_predicate] ; logic_expr = ( identifier (str_p("<") arith_expr) | (str_p("<") arith_expr) | (str_p("/:") arith_expr) | (str_p("<:") arith_expr) | (str_p("/<:") arith_expr) | (str_p("<<:") arith_expr) | (str_p("/<<:") arith_expr) | (str_p("<=") arith_expr) | (str_p("=") arith_expr) | (str_p("=") arith_expr) | (str_p("=") arith_expr) ) [&do_logicExpr] ; arith_expr = term *( ('+' term)[&do_add] | ('-' term)[&do_subt] ) ; term = factor ( ('' factor)[&do_mult] | ('/' factor)[&do_div] ) ; factor = lexeme_d[( identifier | +digit_p)[&do_noint]] | '(' expression ')' | ('+' factor) ; ide_list = identifier *( ch_p(',') identifier ) ; identifier = alpha_p +( alnum_p | ch_p('_') ) ; } rule machine_subst, unbounded_choice, if_select_pre_subst, multi_subst, simple_subst, expression, predicate, logic_expr, arith_expr, term, factor, ide_list, identifier; rule<ScannerT> const& start() const { return predicate; //return multi_subst; //return machine_subst; } }; }; //////////////////////////////////////////////////////////////////////////// // // Main program // //////////////////////////////////////////////////////////////////////////// int main() { cout << "*********************************\n\n"; cout << "\t\t...Machine Parser...\n\n"; cout << "*********************************\n\n"; // cout << "Type an expression...or [q or Q] to quit\n\n"; string str; int machineCount = 0; char strFilename[256]; //file name store as a string object do { cout << "Please enter a filename...or [q or Q] to quit:\n\n "; //prompt for file name to be input //char strFilename[256]; //file name store as a string object cin strFilename; if(*strFilename == 'q' || *strFilename == 'Q') //termination condition return 0; ifstream inFile(strFilename); // opens file object for reading //output file for truncated machine (operations only) if (inFile.fail()) cerr << "\nUnable to open file for reading.\n" << endl; inFile.unsetf(std::ios::skipws); Substitution elementary_subst; // Simple substitution parser object string next; while (inFile str) { getline(inFile, next); str += next; if (str.empty() || str[0] == 'q' || str[0] == 'Q') break; parse_info< info = parse(str.c_str(), elementary_subst !end_p, space_p); if (info.full) { cout << "\n-------------------------\n"; cout << "Parsing succeeded\n"; cout << "\n-------------------------\n"; } else { cout << "\n-------------------------\n"; cout << "Parsing failed\n"; cout << "stopped at: " << info.stop << "\"\n"; cout << "\n-------------------------\n"; } } } while ( (*strFilename != 'q' || *strFilename !='Q')); return 0; } However, I am experiencing the following unexpected behaviours on testing: The text files I used are: f1.txt, ... containing ...: debt:=(LoanRequest+outstandingLoan1)*20 . f2.txt, ... containing ...: debt:=(LoanRequest+outstandingLoan1)*20 || newDebt := loanammount-paidammount || price := purchasePrice + overhead + bb . f3.txt, ... containing ...: yy < (xx+7+ww) . f4.txt, ... containing ...: yy < (xx+7+ww) & yy : NAT . When I use multi_subst as start rule both files (f1 and f2) are parsed correctly; When I use machine_subst as start rule file f1 parse correctly, while file f2 fails, producing the error: “Parsing failed stopped at: || newDebt := loanammount-paidammount || price := purchasePrice + overhead + bb” When I use predicate as start symbol, file f3 parse correctly, but file f4 yields the error: “ “Parsing failed stopped at: & yy : NAT” Can anyone help with the grammar, please? It appears there are problems with the grammar that I have so far been unable to spot.

Read the article

client problems - misaligned expectations & not following SDLC protocols

- by louism

hi guys, im having some serious problems with a client on a project - i could use some advice please the short version i have been working with this client now for almost 6 months without any problems (a classified website project in the range of 500 hours) over the last few days things have drastically deteriorated to the point where ive had to place the project on-hold whilst i work-out what to do (this has pissed the client off even more) to be simplistic, the root cause of the issue is this: the client doesnt read the specs i make for him, i code the feature, he than wants to change things, i tell him its not to the agreed spec and that that change will have to be postponed and possibly charged for, he gets upset and rants saying 'hes paid for the feature' and im not keeping to the agreement (<- misalignment of expectations) i think the root cause of the root cause is my clients failure to take my SDLC protocols seriously. i have a bug tracking system in place which he practically refuses to use (he still emails me bugs), he doesnt seem to care to much for the protocols i use for dealing with scope creep and change control the whole situation came to a head recently where he 'cracked it' (an aussie term for being fed-up). the more terms like 'postponed for post-launch implementation', 'costed feature addition', and 'not to agreed spec' i kept using, the worse it got finally, he began to bully me - basically insisting i shut-up and do the work im being paid for. i wrote a long-winded email explaining how wrong he was on all these different points, and explaining what all the SDLC protocols do to protect the success of the project. than i deleted that email and wrote a new one in the new email, i suggested as a solution i write up a list of grievances we both had. we than review the list and compromise on different points: he gets some things he wants, i get some things i want. sometimes youve got to give ground to get ground his response to this suggestion was flat-out refusal, and a restatement that i should just get on with the work ive been paid to do so there you have the very subjective short version. if you have the time and inclination, the long version may be a little less bias as it has the email communiques between me and my client the long version (with background) the long version works by me showing you the email communiques which lead to the situation coming to a head. so here it is, judge for yourself where the trouble started... 1. client asked me why something was missing from a feature i just uploaded, my response was to show him what was in the spec: it basically said the item he was looking for was never going to be included 2. [clients response...] Memo Louis, We are following your own title fields and keeping a consistent layout. Why the big fuss about not adding "Part". It simply replaces "model" and is consistent with your current title fields. 3. [my response...] hi [client], the 'part' field appeared to me as a redundancy / mistake. i requested clarification but never received any in a timely manner (about 2 weeks ago) the specification for this feature also indicated it wasnt going to be included: RE: "Why the big fuss about not adding "Part" " it may not appear so, but it would actually be a lot of work for me to now add a 'Part' field it could take me up to 15-20 minutes to properly explain why its such a big undertaking to do this, but i would prefer to use that time instead to work on completing your v1.1 features as a simplistic explanation - it connects to the change in paradigm from a 'generic classified ad' model to a 'specific attributes for specific categories' model basically, i am saying it is a big fuss, but i understand that it doesnt look that way - after all, it is just one ity-bitty field :) if you require a fuller explanation, please let me know and i will commit the time needed to write that out also, if you recall when we first started on the project, i said that with the effort/time required for features, you would likely not know off the top of your head. you may think something is really complex, but in reality its quite simple, you might think something is easy - but it could actually be a massive trauma to code (which is the case here with the 'Part' field). if you also recalled, i said the best course of action is to just ask, and i would let you know on a case-by-case basis 4. [email from me to client...] hi [client], the online catalogue page is now up live (see my email from a few days ago for information on how it works) note: the window of opportunity for input/revisions on what data the catalogue stores has now closed (as i have put the code up live now) RE: the UI/layout of the online catalogue page you may still do visual/ui tweaks to the page at the moment (this window for input/revisions will close in a couple of days time) 5. [email from client to me...] *(note: i had put up the feature & asked the client to review it, never heard back from them for a few days)* Memo Louis, Here you go again. CLOSED without a word of input from the customer. I don't think so. I will reply tomorrow regarding the content and functionality we require from this feature. 5. [from me to client...] hi [client]: RE: from my understanding, you are saying that the mini-sale yard control would change itself based on the fact someone was viewing for parts & accessories <- is that correct? this change is outside the scope of the v1.1 mini-spec and therefore will need to wait 'til post launch for costing/implementation 6. [email from client to me...] Memo Louis, Following your v1.1 mini-spec and all your time paid in full for the work selected. We need to make the situation clear. There will be no further items held for post-launch. Do not expect us to pay for any further items other than those we have agreed upon. You have undertaken to complete the Parts and accessories feature as follows. Obviously, as part of this process the "mini search" will be effected, and will require "adaption to make sense". 7. [email from me to client...] hi [client], RE: "There will be no further items held for post-launch. Do not expect us to pay for any further items other than those we have agreed upon." a few points to consider: 1) the specification for the 'parts & accessories' feature was as follows: (i.e. [what] "...we have agreed upon.") 2) you have received the 'parts & accessories' feature free of charge (you have paid $0 for it). ive spent two days coding that feature as a gesture of good will i would request that you please consider these two facts carefully and sincerely 8. [email from client to me...] Memo Louis, I don't see how you are giving us anything for free. From your original fee proposal you have deleted more than 30 hours of included features. Your title "shelved features". Further you have charged us twice by adding back into the site, at an addition cost, some of those "shelved features" features. See v1.1 mini-spec. Did include in your original fee proposal a change request budget but then charge without discussion items included in v1.1 mini-spec. Included a further Features test plan for a regression test, a fee of 10 hours that would not have been required if the "shelved features" were not left out of the agreed fee proposal. I have made every attempt to satisfy your your uneven business sense by offering you everything your heart desired, in the v1.1 mini-spec, to be left once again with your attitude of "its too hard, lets leave it for post launch". I am no longer accepting anything less than what we have contracted you to do. That is clearly defined in v1.1 mini-spec, and you are paid in advance for delivering those items as an acceptable function. a few notes about the above email... i had to cull features from the original spec because it didnt fit into the budget. i explained this to the client at the start of the project (he wanted more features than he had budget hours to do them all) nothing has been charged for twice, i didnt charge the client for culled features. im charging him to now do those culled features the draft version of the project schedule included a change request budget of 10 hours, but i had to remove that to meet the budget (the client may not have been aware of this to be fair to them) what the client refers to as my attitude of 'too hard/leave it for post-launch', i called a change request protocol and a method for keeping scope creep under control 9. [email from me to client...] hi [client], RE: "...all your grievances..." i had originally written out a long email response; it was fantastic, it had all these great points of how 'you were wrong' and 'i was right', you would of loved it (and by 'loved it', i mean it would of just infuriated you more) so, i decided to deleted it start over, for two reasons: 1) a long email is being disrespectful of your time (youre a busy businessman with things to do) 2) whos wrong or right gets us no closer to fixing the problems we are experiencing what i propose is this... i prepare a bullet point list of your grievances and my grievances (yes, im unhappy too about how things are going - and it has little to do with money) i submit this list to you for you to add to as necessary we then both take a good hard look at this list, and we decide which areas we are willing to give ground on as an example, the list may look something like this: "louis, you keep taking away features you said you would do" [your grievance 2] [your grievance 3] [your grievance ...] "[client], i feel you dont properly read the specs i prepare for you..." [my grievance 2] [my grievance 3] [my grievance ...] if you are willing to give this a try, let me know will it work? who knows. but if it doesnt, we can always go back to arguing some more :) obviously, this will only work if you are willing to give it a genuine try, and you can accept that you may have to 'give some ground to get some ground' what do you think? 10. [email from client to me ...] Memo Louis, Instead of wasting your time listing grievances, I would prefer you complete the items in v1.1 mini-spec, to a satisfactory conclusion. We almost had the website ready for launch until you brought the v1.1 mini-spec into the frame. Obviously I expected you could complete the v1.1 mini-spec in a two-week time frame as you indicated and give the site a more profession presentation. Most of the problems have been caused by you not following our instructions, but deciding to do what you feel like at the time. And then arguing with us how the missing information is not necessary. For instance "Parts and Accessories". Why on earth would you leave out the parts heading, when it ties-in with the fields you have already developed. It replaces "model" and is just as important in the context of information that appears in the "Details" panel. We are at a stage where the the v1.1 mini-spec needs to be completed without further time wasting and the site is complete (subject to all features working). We are on standby at this end to do just that. Let me know when you are back, working on the site and we will process and complete each v1.1 mini-spec, item by item, until the job is complete. 11. [last email from me to client...] hi [client], based on this reply, and your demonstrated unwillingness to compromise/give any ground on issues at hand, i have decided to place your project on-hold for the moment i will be considering further options on how to over-come our challenges over the next few days i will contact you by monday 17/may to discuss any new options i have come up with, and if i believe it is appropriate to restart work on your project at that point or not told you it was long... what do you think?

Read the article

Help with Boost Grammar

- by Decmanc04

I have been using the following win32 console code to try to parse a B Machine Grammar embedded within C++ using Boost Spirit grammar template. I am a relatively new Boost user. The code compiles, but when I run the .exe file produced by VC++2008, the program partially parses the input file. I believe the problem is with my grammar definition or the functions attached as semantic atctions. The code is given below: // BIFAnalyser.cpp : Defines the entry point for the console application. // // /*============================================================================= Copyright (c) Temitope Jos Onunkun 2010 http://www.dcs.kcl.ac.uk/pg/onun/ Use, modification and distribution is subject to the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) =============================================================================*/ //////////////////////////////////////////////////////////////////////////// // // // B Machine parser using the Boost "Grammar" and "Semantic Actions". // // // //////////////////////////////////////////////////////////////////////////// #include <boost/spirit/core.hpp> #include <boost/tokenizer.hpp> #include <iostream> #include <string> #include <fstream> #include <vector> #include <utility> /////////////////////////////////////////////////////////////////////////////////////////// using namespace std; using namespace boost::spirit; /////////////////////////////////////////////////////////////////////////////////////////// // // Semantic actions // //////////////////////////////////////////////////////////////////////////// vector<string> strVect; namespace { //semantic action function on individual lexeme void do_noint(char const* str, char const* end) { string s(str, end); if(atoi(str)) { ; } else { strVect.push_back(s); cout << "PUSH(" << s << ')' << endl; } } //semantic action function on addition of lexemes void do_add(char const*, char const*) { cout << "ADD" << endl; for(vector<string>::iterator vi = strVect.begin(); vi < strVect.end(); ++vi) cout << *vi << " "; } //semantic action function on subtraction of lexemes void do_subt(char const*, char const*) { cout << "SUBTRACT" << endl; for(vector<string>::iterator vi = strVect.begin(); vi < strVect.end(); ++vi) cout << *vi << " "; } //semantic action function on multiplication of lexemes void do_mult(char const*, char const*) { cout << "\nMULTIPLY" << endl; for(vector<string>::iterator vi = strVect.begin(); vi < strVect.end(); ++vi) cout << *vi << " "; cout << "\n"; } //semantic action function on division of lexemes void do_div(char const*, char const*) { cout << "\nDIVIDE" << endl; for(vector<string>::iterator vi = strVect.begin(); vi < strVect.end(); ++vi) cout << *vi << " "; } //semantic action function on simple substitution void do_sSubst(char const* str, char const* end) { string s(str, end); //use boost tokenizer to break down tokens typedef boost::tokenizer<boost::char_separator<char> > Tokenizer; boost::char_separator<char> sep("-+/*:=()"); // default char separator Tokenizer tok(s, sep); Tokenizer::iterator tok_iter = tok.begin(); pair<string, string > dependency; //create a pair object for dependencies //save first variable token in simple substitution dependency.first = *tok.begin(); //create a vector object to store all tokens vector<string> dx; // for( ; tok_iter != tok.end(); ++tok_iter) //save all tokens in vector { dx.push_back(*tok_iter ); } vector<string> d_hat; //stores set of dependency pairs string dep; //pairs variables as string object for(int unsigned i=1; i < dx.size()-1; i++) { dependency.second = dx.at(i); dep = dependency.first + "|->" + dependency.second + " "; d_hat.push_back(dep); } cout << "PUSH(" << s << ')' << endl; for(int unsigned i=0; i < d_hat.size(); i++) cout <<"\n...\n" << d_hat.at(i) << " "; cout << "\nSIMPLE SUBSTITUTION\n"; } //semantic action function on multiple substitution void do_mSubst(char const* str, char const* end) { string s(str, end); //use boost tokenizer to break down tokens typedef boost::tokenizer<boost::char_separator<char> > Tok; boost::char_separator<char> sep("-+/*:=()"); // default char separator Tok tok(s, sep); Tok::iterator tok_iter = tok.begin(); // string start = *tok.begin(); vector<string> mx; for( ; tok_iter != tok.end(); ++tok_iter) //save all tokens in vector { mx.push_back(*tok_iter ); } mx.push_back("END\n"); //add a marker "end" for(unsigned int i=0; i<mx.size(); i++) { // if(mx.at(i) == "END" || mx.at(i) == "||" ) // break; // else if( mx.at(i) == "||") // do_sSubst(str, end); // else // { // do_sSubst(str, end); // } cout << "\nTokens ... " << mx.at(i) << " "; } cout << "PUSH(" << s << ')' << endl; cout << "MULTIPLE SUBSTITUTION\n"; } } //////////////////////////////////////////////////////////////////////////// // // Simple Substitution Grammar // //////////////////////////////////////////////////////////////////////////// // Simple substitution grammar parser with integer values removed struct Substitution : public grammar<Substitution> { template <typename ScannerT> struct definition { definition(Substitution const& ) { multi_subst = (simple_subst [&do_mSubst] >> +( str_p("||") >> simple_subst [&do_mSubst]) ) ; simple_subst = (Identifier >> str_p(":=") >> expression)[&do_sSubst] ; Identifier = alpha_p >> +alnum_p//[do_noint] ; expression = term >> *( ('+' >> term)[&do_add] | ('-' >> term)[&do_subt] ) ; term = factor >> *( ('*' >> factor)[&do_mult] | ('/' >> factor)[&do_div] ) ; factor = lexeme_d[( (alpha_p >> +alnum_p) | +digit_p)[&do_noint]] | '(' >> expression >> ')' | ('+' >> factor) ; } rule<ScannerT> expression, term, factor, Identifier, simple_subst, multi_subst ; rule<ScannerT> const& start() const { return multi_subst; } }; }; //////////////////////////////////////////////////////////////////////////// // // Main program // //////////////////////////////////////////////////////////////////////////// int main() { cout << "************************************************************\n\n"; cout << "\t\t...Machine Parser...\n\n"; cout << "************************************************************\n\n"; // cout << "Type an expression...or [q or Q] to quit\n\n"; //prompt for file name to be input cout << "Please enter a filename...or [q or Q] to quit:\n\n "; char strFilename[256]; //file name store as a string object cin >> strFilename; ifstream inFile(strFilename); // opens file object for reading //output file for truncated machine (operations only) Substitution elementary_subst; // Simple substitution parser object string str, next; // inFile.open(strFilename); while (inFile >> str) { getline(cin, next); str += next; if (str.empty() || str[0] == 'q' || str[0] == 'Q') break; parse_info<> info = parse(str.c_str(), elementary_subst, space_p); if (info.full) { cout << "\n-------------------------\n"; cout << "Parsing succeeded\n"; cout << "\n-------------------------\n"; } else { cout << "\n-------------------------\n"; cout << "Parsing failed\n"; cout << "stopped at: \": " << info.stop << "\"\n"; cout << "\n-------------------------\n"; } } cout << "Please enter a filename...or [q or Q] to quit\n"; cin >> strFilename; return 0; } The contents of the file I tried to parse, which I named "mf7.txt" is given below: debt:=(LoanRequest+outstandingLoan1)*20 || newDebt := loanammount-paidammount The output when I execute the program is: ************************************************************ ...Machine Parser... ************************************************************ Please enter a filename...or [q or Q] to quit: c:\tplat\mf7.txt PUSH(LoanRequest) PUSH(outstandingLoan1) ADD LoanRequest outstandingLoan1 MULTIPLY LoanRequest outstandingLoan1 PUSH(debt:=(LoanRequest+outstandingLoan1)*20) ... debt|->LoanRequest ... debt|->outstandingLoan1 SIMPLE SUBSTITUTION Tokens ... debt Tokens ... LoanRequest Tokens ... outstandingLoan1 Tokens ... 20 Tokens ... END PUSH(debt:=(LoanRequest+outstandingLoan1)*20) MULTIPLE SUBSTITUTION ------------------------- Parsing failedstopped at: ": " ------------------------- My intention is to capture only the variables in the file, which I managed to do up to the "||" string. Clearly, the program is not parsing beyond the "||" string in the input file. I will appreciate assistance to fix the grammar. SOS, please.

Read the article

In Email, Image (img) Source (src) Tags are rewritten as relative links. How to fix?

- by Noah Goodrich

I'm working on sending out an html based email, and every time it sends the image src tags and some of the anchor href tags are modified to be relative url's. Update 2: This is happening between when the body of the email is generated and sent and when it arrives in my inbox. Update: I am using Postfix on a LAMPP server. In addition, I am using Zend_Mail to send the emails out. For example, I have a link: src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/header.jpg" And it gets rewritten as: src="../../../../images/email/highpoint_2009_04/header.jpg" What can cause this to occur and how is it corrected? Email headers: Return-Path: <[email protected]> X-Original-To: [email protected] Delivered-To: [email protected] Received: by mail.example.com (Postfix, from userid 0) id 6BF012252; Tue, 14 Apr 2009 12:15:20 -0600 (MDT) To: Gabriel <[email protected]> Subject: Free Map to Sales Success From: Somebody <[email protected]> Date: Tue, 14 Apr 2009 12:15:20 -0600 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: multipart/related Content-Disposition: inline Message-Id: <[email protected]> Original content to be sent out: <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td valign="top"> <a href="http://www.furnituretrainingcompany.com"> <img moz-do-not-send="true" alt="The Furniture Training Company - Know More. Sell More." src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/header.jpg" border="0" height="123" width="600"> </a> </td> </tr> </tbody> </table> <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td valign="top"><img alt="Visit us at High Point to receive your free training poster" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/hero.jpg" moz-do-not-send="true" height="150" width="600"><br> </td> </tr> </tbody> </table> <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_content_left.jpg" moz-do-not-send="true" height="30" width="30"><br> </td> <td bgcolor="#ffffff" valign="top"><font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#000000" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><big><small><big><b>See you at Market</b></big><br> </small></big></big></big></big></font> <font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#000000" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><big><small><br> </small></big></big></big></big></font><small><font face="Helvetica, Arial, sans-serif">Visit our space to get your free Map to Sales Success poster! This unique 24 X 36 color poster is your guide to developing high volume salespeople with larger tickets. Find us in the new NHFA Retailer Resource Center located in the Plaza. <br> <br> Don’t miss Mark Lacy’s entertaining seminar "Help Wanted! My Sales Associates Can’t Sell Water to a Thirsty Camel." He’ll reveal powerful secrets for turning sales associates into furniture experts that will sell. See him Saturday, April 25th at 11:30 AM in the seminar room of the new NHFA Retail Resource Center in the Plaza. <br> <br> Stop by our space to learn how our ingenious internet-delivered training courses are easy to use, guaranteed to work, and cheaper than the daily donuts. Over 95% report increased sales. <br> <br> Plan to see us at High Point. </font></small> <font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#000000" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><big><small><small><br> <br> <br> <br> </small></small></big></big></big></big></font><small><font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#000000" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><small> </small></big></big></big></font></small> <a href="http://www.furnituretrainingcompany.com/map"><img alt="Find out more" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/image_content_left.jpg" moz-do-not-send="true" border="0" height="67" width="326"></a><br> <br> </td> <td bgcolor="#ffffff" valign="top"> <img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_content_middle.jpg" moz-do-not-send="true" height="28" width="28"><br> </td> <td bgcolor="#ffffff" valign="top"><img alt="Roadmap to Sales Success poster" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/image_content_right.jpg" moz-do-not-send="true" height="267" width="186"><br> <font face="Helvetica, Arial, sans-serif"><small><font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#000000" size="1"><big><big><big><small><b>Road Map to Sales Success<br> </b><br> </small></big></big></big></font>This beautiful poster is yours free for simply stopping by and visiting with us at High Point. <span class="moz-txt-slash">Our space is located inside the </span>new NHFA Retailer Resource Center in the Plaza Suites, 222 South Main St, 1st Floor. We will be at market from Sat April 25th until Thur April 30th. </small></font><br> </td> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_content_right.jpg" moz-do-not-send="true" height="30" width="30"><br> <br> </td> </tr> </tbody> </table> <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/disclaimer_divider.jpg" moz-do-not-send="true" height="25" width="600"><br> </td> </tr> </tbody> </table> <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_disclaimer_left.jpg" moz-do-not-send="true"></td> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_disclaimer_middle.jpg" moz-do-not-send="true"><br> <font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" color="#666666" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><big><small><small><small>If you are not attending the High Point market in April but would still like to receive a free Road Map to Sales Success poster visit us on the web at <u><a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="http://www.furnituretrainingcompany.com">www.furnituretrainingcompany.com</a></u>, or to speak with a Furniture Training Company representative, call toll free (866) 755-5996. We do not offer free shipping outside of the U.S. and Canada. Retailers outside of the U.S. and Canada may call for more information. Limit one free Road Map to Sales Success per company. Other copies of the poster may be purchased on our web site.<br> <br> </small></small></small></big></big></big></big></font> <font color="#666666"><small><font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><small><small>We hope you found this message to be useful. However, if you'd rather not receive future emails of this sort from The Furniture Training Company, please <a moz-do-not-send="true" href="http://www.furnituretraining.com/contact">click here to unsubscribe</a>.<br> <br> </small></small></big></big></big></font></small><small><font originaltag="yes" style="font-size: 9px; font-family: Verdana,Arial,Helvetica,sans-serif;" face="Verdana, Arial, Helvetica, sans-serif" size="1"><big><big><big><small><small>©Copyright 2009 The Furniture Training Company.<br> 1770 North Research Park Way, <br> North Logan, UT 84341. <br> All Rights Reserved.</small></small></big></big></big></font></small></font><br> </td> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer_disclaimer_right.jpg" moz-do-not-send="true"></td> </tr> </tbody> </table> <table align="center" border="0" cellpadding="0" cellspacing="0" width="600"> <tbody> <tr> <td bgcolor="#ffffff" valign="top"><img alt="" src="http://www.furnituretrainingcompany.com/images/email/highpoint_2009_04/footer.jpg" moz-do-not-send="true"> </td> </tr> </tbody> </table> <br> <br> Content that gets sent: <table border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"600" al= ign=3D"center">=0D=0A<tbody>=0D=0A<tr>=0D=0A<td valign=3D"top"><a href= =3D"http://www.furnituretrainingcompany.com"> <img src=3D"http://www.fur= nituretrainingcompany.com/images/email/highpoint_2009_04/header.jpg" bor= der=3D"0" alt=3D"The Furniture Training Company - Know More. Sell More."= width=3D"600" height=3D"123" /> </a></td>=0D=0A</tr>=0D=0A</tbody>=0D= =0A</table>=0D=0A<table border=3D"0" cellspacing=3D"0" cellpadding=3D"0"= width=3D"600" align=3D"center">=0D=0A<tbody>=0D=0A<tr>=0D=0A<td valign= =3D"top"><img src=3D"http://www.furnituretrainingcompany.com/images/emai= l/highpoint_2009_04/hero.jpg" alt=3D"Visit us at High Point to receive y= our free training poster" width=3D"600" height=3D"150" /><br /></td>=0D= =0A</tr>=0D=0A</tbody>=0D=0A</table>=0D=0A<table border=3D"0" cellspacin= g=3D"0" cellpadding=3D"0" width=3D"600" align=3D"center">=0D=0A<tbody>= =0D=0A<tr>=0D=0A<td valign=3D"top" bgcolor=3D"#ffffff"><img src=3D"http:= //www.furnituretrainingcompany.com/images/email/highpoint_2009_04/spacer= _content_left.jpg" alt=3D"" width=3D"30" height=3D"30" /><br /></td>=0D= =0A<td valign=3D"top" bgcolor=3D"#ffffff"><span style=3D"font-size: xx-s= mall; font-family: Verdana,Arial,Helvetica,sans-serif; color: #000000;">= <big><big><big><big><small><big><strong>See you at Market</strong></big>= <br /> </small></big></big></big></big></span> <span style=3D"font-size:= xx-small; font-family: Verdana,Arial,Helvetica,sans-serif; color: #0000= 00;"><big><big><big><big><small><br /> </small></big></big></big></big><= /span><small><span style=3D"font-family: Helvetica,Arial,sans-serif;">Vi= sit our space to get your free Map to Sales Success poster! This unique= 24 X 36 color poster is your guide to developing high volume salespeopl= e with larger tickets. Find us in the new NHFA Retailer Resource Center= located in the Plaza. <br /> <br /> Don’t miss Mark Lacy’s= entertaining seminar "Help Wanted! My Sales Associates Can’t Sell= Water to a Thirsty Camel." He’ll reveal powerful secrets for turn= ing sales associates into furniture experts that will sell. See him Satu= rday, April 25th at 11:30 AM in the seminar room of the new NHFA Retail= Resource Center in the Plaza. <br /> <br /> Stop by our space to learn= how our ingenious internet-delivered training courses are easy to use,= guaranteed to work, and cheaper than the daily donuts. Over 95% report= increased sales. <br /> <br /> Plan to see us at High Point. </span></s= mall> <span style=3D"font-size: xx-small; font-family: Verdana,Arial,Hel= vetica,sans-serif; color: #000000;"><big><big><big><big><small><small><b= r /> <br /> <br /> <br /> </small></small></big></big></big></big></span= ><small><span style=3D"font-size: xx-small; font-family: Verdana,Arial,H= elvetica,sans-serif; color: #000000;"><big><big><big><small> </small></b= ig></big></big></span></small> <a href=3D"http://www.furnituretrainingco= mpany.com/map"><img src=3D"http://www.furnituretrainingcompany.com/image= s/email/highpoint_2009_04/image_content_left.jpg" border=3D"0" alt=3D"Fi= nd out more" width=3D"326" height=3D"67" /></a><br /> <br /></td>=0D=0A<= td valign=3D"top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituretr= ainingcompany.com/images/email/highpoint_2009_04/spacer_content_middle.j= pg" alt=3D"" width=3D"28" height=3D"28" /><br /></td>=0D=0A<td valign=3D= "top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituretrainingcompan= y.com/images/email/highpoint_2009_04/image_content_right.jpg" alt=3D"Roa= dmap to Sales Success poster" width=3D"186" height=3D"267" /><br /> <spa= n style=3D"font-family: Helvetica,Arial,sans-serif;"><small><span style= =3D"font-size: xx-small; color: #000000;"><big><big><big><small><strong>= Road Map to Sales Success<br /> </strong><br /> </small></big></big></bi= g></span>This beautiful poster is yours free for simply stopping by and= visiting with us at High Point. <span class=3D"moz-txt-slash">Our space= is located inside the </span>new NHFA Retailer Resource Center in the P= laza Suites, 222 South Main St, 1st Floor. We will be at market from Sat= April 25th until Thur April 30th. </small></span><br /></td>=0D=0A<td v= align=3D"top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituretraini= ngcompany.com/images/email/highpoint_2009_04/spacer_content_right.jpg" a= lt=3D"" width=3D"30" height=3D"30" /><br /> <br /></td>=0D=0A</tr>=0D=0A= </tbody>=0D=0A</table>=0D=0A<table border=3D"0" cellspacing=3D"0" cellpa= dding=3D"0" width=3D"600" align=3D"center">=0D=0A<tbody>=0D=0A<tr>=0D=0A= <td valign=3D"top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituret= rainingcompany.com/images/email/highpoint_2009_04/disclaimer_divider.jpg= " alt=3D"" width=3D"600" height=3D"25" /><br /></td>=0D=0A</tr>=0D=0A</t= body>=0D=0A</table>=0D=0A<table border=3D"0" cellspacing=3D"0" cellpaddi= ng=3D"0" width=3D"600" align=3D"center">=0D=0A<tbody>=0D=0A<tr>=0D=0A<td= valign=3D"top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituretrai= ningcompany.com/images/email/highpoint_2009_04/spacer_disclaimer_left.jp= g" alt=3D"" /></td>=0D=0A<td valign=3D"top" bgcolor=3D"#ffffff"><img src= =3D"http://www.furnituretrainingcompany.com/images/email/highpoint_2009_= 04/spacer_disclaimer_middle.jpg" alt=3D"" /><br /> <span style=3D"font-s= ize: xx-small; font-family: Verdana,Arial,Helvetica,sans-serif; color: #= 666666;"><big><big><big><big><small><small><small>If you are not attendi= ng the High Point market in April but would still like to receive a free= Road Map to Sales Success poster visit us on the web at <span style=3D"= text-decoration: underline;"><a class=3D"moz-txt-link-abbreviated" href= =3D"http://www.furnituretrainingcompany.com">www.furnituretrainingcompan= y.com</a></span>, or to speak with a Furniture Training Company represen= tative, call toll free (866) 755-5996. We do not offer free shipping out= side of the U.S. and Canada. Retailers outside of the U.S. and Canada ma= y call for more information. Limit one free Road Map to Sales Success pe= r company. Other copies of the poster may be purchased on our web site.<= br /> <br /> </small></small></small></big></big></big></big></span> <sp= an style=3D"color: #666666;"><small><span style=3D"font-size: xx-small;= font-family: Verdana,Arial,Helvetica,sans-serif;"><big><big><big><small= ><small>We hope you found this message to be useful. However, if you'd r= ather not receive future emails of this sort from The Furniture Training= Company, please <a href=3D"http://www.furnituretraining.com/contact">cl= ick here to unsubscribe</a>.<br /> <br /> </small></small></big></big></= big></span></small><small><span style=3D"font-size: xx-small; font-famil= y: Verdana,Arial,Helvetica,sans-serif;"><big><big><big><small><small>&co= py;Copyright 2009 The Furniture Training Company.<br /> 1770 North Resea= rch Park Way, <br /> North Logan, UT 84341. <br /> All Rights Reserved.<= /small></small></big></big></big></span></small></span><br /></td>=0D=0A= <td valign=3D"top" bgcolor=3D"#ffffff"><img src=3D"http://www.furnituret= rainingcompany.com/images/email/highpoint_2009_04/spacer_disclaimer_righ= t.jpg" alt=3D"" /></td>=0D=0A</tr>=0D=0A</tbody>=0D=0A</table>=0D=0A<tab= le border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"600" align= =3D"center">=0D=0A<tbody>=0D=0A<tr>=0D=0A<td valign=3D"top" bgcolor=3D"#= ffffff"><img src=3D"http://www.furnituretrainingcompany.com/images/email= /highpoint_2009_04/footer.jpg" alt=3D"" /></td>=0D=0A</tr>=0D=0A</tbody>= =0D=0A</table>=0D=0A<p><br /></p><br><hr><a href=3D'http://localhost/ftc= /app/unsubscribe.php?action=3DoptOut&pid=3D6121&cid=3D19&email=3Dmarkl@f= urnituretrainingcompany.com'>Click to Unsubscribe</a>

Read the article

Delphi - TPerlRegEx / RegExBuddy Problem

- by Brad

I've got a problem with RegEx and Delphi 2k9 (Win32). I get the following Error: First chance exception at $7C812AFB. Exception class Exception with message 'TPerlRegEx.Compile() - Please specify a regular expression in RegEx first'. I've got the latest version of TPerlRegEx from the website. Using its defualt settings (Using DLL) I'm including demo source code. It's using the code generated by RegExBuddy, latest version. http://www.4shared.com/file/236428923/97478b61/googleresultstestdata.html http://www.4shared.com/file/236439483/e0acbe6d/Unit2.html Delphi FORM http://www.4shared.com/file/236439473/6734a2a2/Unit2.html Delphi PAS Thanks for any help -Brad Data is from Google External Keyword Tool RegEx could use some refinement... but works in RegExBuddy not in Delphi unit Unit2; interface uses Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms, Dialogs, StdCtrls, PerlRegEx; type TForm2 = class(TForm) Memo1: TMemo; Memo2: TMemo; Button1: TButton; procedure Button1Click(Sender: TObject); private { Private declarations } public { Public declarations } end; var Form2: TForm2; implementation {$R *.dfm} procedure TForm2.Button1Click(Sender: TObject); var Regex: TPerlRegEx; GroupIndex: Integer; begin Regex := TPerlRegEx.Create(nil); Regex.RegEx := 'criteria\.push\(new kpCriterion\('(?P<keyword>(.*?))', (?P<number1>(.*?)),'#13#10'''(?P<localsearch>(.*?))'', ''(?P<globalsearch>(.*?))'', (?P<localsearchnum>(.*?)), (?P<globalsearchnum>(.*?)), (.*+)'#13#10','#13#10''\$(?P<price>(.*?))', (?P<number2>(.*?)),'#13#10''(?P<range>(.*?))', (?P<number3>(.*+))'; Regex.Options := [preMultiLine]; Regex.Subject := memo1.text; if Regex.Match then begin memo2.Lines.Add('Matches Found'); repeat for GroupIndex := 0 to Regex.SubExpressionCount do begin memo2.lines.add( Regex.SubExpressions[GroupIndex]); //Add Results to memo // backreference text: Regex.SubExpressions[GroupIndex]; // backreference start: Regex.SubExpressionOffsets[GroupIndex]; // backreference length: Regex.SubExpressionLengths[GroupIndex]; end; until not Regex.MatchAgain; end else memo2.Lines.Add('No-Matches Found'); end; end. DFM object Form2: TForm2 Left = 0 Top = 0 Caption = 'Form2' ClientHeight = 247 ClientWidth = 480 Color = clBtnFace Font.Charset = DEFAULT_CHARSET Font.Color = clWindowText Font.Height = -11 Font.Name = 'Tahoma' Font.Style = [] OldCreateOrder = False PixelsPerInch = 96 TextHeight = 13 object Memo1: TMemo Left = 8 Top = 8 Width = 185 Height = 89 Lines.Strings = ( 'var showImpressions = false; var ' 'criteriaSuggestor = ' ''sensei_keyword'; var ' 'historicalTimePeriod = 'Mar ' '2009 - Feb 2010'; var ' 'historicalStartMonth = 2; var ' 'impressionTimePeriod = ' ''February'; var ' 'criteriaGroupsArray = new Array(); ' 'var captchaError = false; var ' 'quotaExceeded = false;' 'var criteria = new Array();' 'var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.4' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' '));' 'criteria.push(new ' 'kpCriterion('thunderstorm&#3' '9;, 1.9117305278778076,' #39'201,000'#39', '#39'550,000'#39', 201000, ' '550000, 0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.42' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.46' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.36' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' '));' 'criteria.push(new ' 'kpCriterion('[thunderstorm]&' '#39;, 1.9117305278778076,' #39'33,100'#39', '#39'90,500'#39', 33100, 90500, ' '0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '3' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.4' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' '));' 'criteria.push(new ' 'kpCriterion('\42thunderstorm\' '042', 1.9117305278778076,' #39'201,000'#39', '#39'450,000'#39', 201000, ' '450000, 0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.64' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.58' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' '));' 'criteria.push(new ' 'kpCriterion('thunderstorms&#' '39;, 1.8268921375274658,' #39'110,000'#39', '#39'201,000'#39', 110000, ' '201000, 0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.83' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.42' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.41' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.39' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.51' '));' 'criteria.push(new ' 'kpCriterion('[thunderstorms]&' '#39;, 1.8268921375274658,' #39'22,200'#39', '#39'40,500'#39', 22200, 40500, ' '0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.64' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.58' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' '));' 'criteria.push(new ' 'kpCriterion('\42thunderstorms' '\042', 1.8268921375274658,' #39'110,000'#39', '#39'165,000'#39', 110000, ' '165000, 0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' '));' 'criteria.push(new ' 'kpCriterion('lightning ' 'storm', 1.774579644203186,' #39'49,500'#39', '#39'90,500'#39', 49500, 90500, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.97' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.98' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.86' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' '));' 'criteria.push(new ' 'kpCriterion('[lightning ' 'storm]', 1.774579644203186,' #39'12,100'#39', '#39'22,200'#39', 12100, 22200, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.85' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.65' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' '));' 'criteria.push(new ' 'kpCriterion('\42lightning ' 'storm\042', ' '1.774579644203186,' #39'33,100'#39', '#39'60,500'#39', 33100, 60500, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.74' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' '));' 'criteria.push(new ' 'kpCriterion('rain storm', ' '1.7464053630828857,' #39'27,100'#39', '#39'49,500'#39', 27100, 49500, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '0' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.55' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.74' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.89' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' '));' 'criteria.push(new ' 'kpCriterion('[rain ' 'storm]', ' '1.7464053630828857,' #39'5,400'#39', '#39'8,100'#39', 5400, 8100, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '2' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.62' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.59' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' '));' 'criteria.push(new ' 'kpCriterion('\42rain ' 'storm\042', ' '1.7464053630828857,' #39'14,800'#39', '#39'27,100'#39', 14800, 27100, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '0' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.78' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' '));' 'criteria.push(new ' 'kpCriterion('lightning ' 'storms', ' '1.6842896938323975,' #39'14,800'#39', '#39'27,100'#39', 14800, 27100, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.9' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.9' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.88' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.63' '));' 'criteria.push(new ' 'kpCriterion('[lightning ' 'storms]', ' '1.6842896938323975,' #39'3,600'#39', '#39'8,100'#39', 3600, 8100, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.8' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.86' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.99' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.83' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.85' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.78' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.91' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' '));' 'criteria.push(new ' 'kpCriterion('\42lightning ' 'storms\042', ' '1.6842896938323975,' #39'12,100'#39', '#39'22,200'#39', 12100, 22200, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation =

Read the article

Delphi - TPerlRegEx / RegExBuddy Problem

- by Brad

I've got a problem with RegEx and Delphi 2k9 (Win32). I get the following Error: First chance exception at $7C812AFB. Exception class Exception with message 'TPerlRegEx.Compile() - Please specify a regular expression in RegEx first'. I've got the latest version of TPerlRegEx from the website. Using its defualt settings (Using DLL) I'm including demo source code. It's using the code generated by RegExBuddy, latest version. http://www.4shared.com/file/236428923/97478b61/googleresultstestdata.html http://www.4shared.com/file/236439483/e0acbe6d/Unit2.html Delphi FORM http://www.4shared.com/file/236439473/6734a2a2/Unit2.html Delphi PAS Thanks for any help -Brad Data is from Google External Keyword Tool RegEx could use some refinement... but works in RegExBuddy not in Delphi unit Unit2; interface uses Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms, Dialogs, StdCtrls, PerlRegEx; type TForm2 = class(TForm) Memo1: TMemo; Memo2: TMemo; Button1: TButton; procedure Button1Click(Sender: TObject); private { Private declarations } public { Public declarations } end; var Form2: TForm2; implementation {$R *.dfm} procedure TForm2.Button1Click(Sender: TObject); var Regex: TPerlRegEx; GroupIndex: Integer; begin Regex := TPerlRegEx.Create(nil); Regex.RegEx := 'criteria.push(new kpCriterion('(?P(.?))', (?P(.?)),'#13#10'''(?P(.?))'', ''(?P(.?))'', (?P(.?)), (?P(.?)), (.+)'#13#10','#13#10''\$(?P(.?))', (?P(.?)),'#13#10''(?P(.?))', (?P(.*+))'; Regex.Options := [preMultiLine]; Regex.Subject := memo1.text; if Regex.Match then begin memo2.Lines.Add('Matches Found'); repeat for GroupIndex := 0 to Regex.SubExpressionCount do begin memo2.lines.add( Regex.SubExpressions[GroupIndex]); //Add Results to memo // backreference text: Regex.SubExpressions[GroupIndex]; // backreference start: Regex.SubExpressionOffsets[GroupIndex]; // backreference length: Regex.SubExpressionLengths[GroupIndex]; end; until not Regex.MatchAgain; end else memo2.Lines.Add('No-Matches Found'); end; end. DFM object Form2: TForm2 Left = 0 Top = 0 Caption = 'Form2' ClientHeight = 247 ClientWidth = 480 Color = clBtnFace Font.Charset = DEFAULT_CHARSET Font.Color = clWindowText Font.Height = -11 Font.Name = 'Tahoma' Font.Style = [] OldCreateOrder = False PixelsPerInch = 96 TextHeight = 13 object Memo1: TMemo Left = 8 Top = 8 Width = 185 Height = 89 Lines.Strings = ( 'var showImpressions = false; var ' 'criteriaSuggestor = ' ''sensei_keyword'; var ' 'historicalTimePeriod = 'Mar ' '2009 - Feb 2010'; var ' 'historicalStartMonth = 2; var ' 'impressionTimePeriod = ' ''February'; var ' 'criteriaGroupsArray = new Array(); ' 'var captchaError = false; var ' 'quotaExceeded = false;' 'var criteria = new Array();' 'var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.4' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' '));' 'criteria.push(new ' 'kpCriterion('thunderstorm' '9;, 1.9117305278778076,' #39'201,000'#39', '#39'550,000'#39', 201000, ' '550000, 0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.42' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.46' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.36' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' '));' 'criteria.push(new ' 'kpCriterion('[thunderstorm]&' '#39;, 1.9117305278778076,' #39'33,100'#39', '#39'90,500'#39', 33100, 90500, ' '0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '3' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.43' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.4' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.45' '));' 'criteria.push(new ' 'kpCriterion('\42thunderstorm\' '042', 1.9117305278778076,' #39'201,000'#39', '#39'450,000'#39', 201000, ' '450000, 0.8666667' ',' ''$0.49', 493102,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.64' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.58' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' '));' 'criteria.push(new ' 'kpCriterion('thunderstorms&#' '39;, 1.8268921375274658,' #39'110,000'#39', '#39'201,000'#39', 110000, ' '201000, 0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.83' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.42' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.41' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.39' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.51' '));' 'criteria.push(new ' 'kpCriterion('[thunderstorms]&' '#39;, 1.8268921375274658,' #39'22,200'#39', '#39'40,500'#39', 22200, 40500, ' '0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.64' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.56' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.47' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.58' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' '));' 'criteria.push(new ' 'kpCriterion('\42thunderstorms' '\042', 1.8268921375274658,' #39'110,000'#39', '#39'165,000'#39', 110000, ' '165000, 0.8' ',' ''$0.56', 559074,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' '));' 'criteria.push(new ' 'kpCriterion('lightning ' 'storm', 1.774579644203186,' #39'49,500'#39', '#39'90,500'#39', 49500, 90500, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.97' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.98' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.86' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' '));' 'criteria.push(new ' 'kpCriterion('[lightning ' 'storm]', 1.774579644203186,' #39'12,100'#39', '#39'22,200'#39', 12100, 22200, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.85' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.65' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' '));' 'criteria.push(new ' 'kpCriterion('\42lightning ' 'storm\042', ' '1.774579644203186,' #39'33,100'#39', '#39'60,500'#39', 33100, 60500, ' '0.73333335' ',' ''$0.54', 535666,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '5' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.74' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' '));' 'criteria.push(new ' 'kpCriterion('rain storm', ' '1.7464053630828857,' #39'27,100'#39', '#39'49,500'#39', 27100, 49500, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '0' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.55' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.74' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.89' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' '));' 'criteria.push(new ' 'kpCriterion('[rain ' 'storm]', ' '1.7464053630828857,' #39'5,400'#39', '#39'8,100'#39', 5400, 8100, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '2' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.73' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.72' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.62' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.59' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' '));' 'criteria.push(new ' 'kpCriterion('\42rain ' 'storm\042', ' '1.7464053630828857,' #39'14,800'#39', '#39'27,100'#39', 14800, 27100, ' '0.6666667' ',' ''$0.53', 526334,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '0' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.87' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.78' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.79' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.61' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.92' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.82' '));' 'criteria.push(new ' 'kpCriterion('lightning ' 'storms', ' '1.6842896938323975,' #39'14,800'#39', '#39'27,100'#39', 14800, 27100, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.9' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.9' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.84' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.88' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.76' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.75' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.63' '));' 'criteria.push(new ' 'kpCriterion('[lightning ' 'storms]', ' '1.6842896938323975,' #39'3,600'#39', '#39'8,100'#39', 3600, 8100, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.8' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.86' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.99' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.83' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.85' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.78' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.91' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.81' '));' 'criteria.push(new ' 'kpCriterion('\42lightning ' 'storms\042', ' '1.6842896938323975,' #39'12,100'#39', '#39'22,200'#39', 12100, 22200, ' '0.73333335' ',' ''$0.42', 417108,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '4' ',' '''' ',' 'kpView.MATCH_PHRASE' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.54' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.5' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.66' '));' 'criteria.push(new ' 'kpCriterion('rain ' 'storms', ' '1.421982765197754,' #39'6,600'#39', '#39'9,900'#39', 6600, 9900, 0.6' ',' ''$0.32', 324834,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '0' ',' '''' ',' 'kpView.MATCH_BROAD' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '1.0' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.97' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.91' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.52' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.51' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.69' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.64' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.51' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.77' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.6' '));' 'criteria.push(new ' 'kpCriterion('[rain ' 'storms]', ' '1.421982765197754,' #39'1,300'#39', '#39'1,900'#39', 1300, 1900, 0.6' ',' ''$0.32', 324834,' ''1 - 3', 2' ',' '0' ',' '0' ',' 'monthlyVariation,' '2' ',' '''' ',' 'kpView.MATCH_EXACT' ',' '0' ')); var monthlyVariation = new ' 'Array();' 'monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.68' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.7' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.53' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.49' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.71' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.67' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.57' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity(' '0.48' ')); monthlyVariation.push(new ' 'kpMonthlyPopularity('

Read the article

Steganography Experiment - Trouble hiding message bits in DCT coefficients

- by JohnHankinson

I have an application requiring me to be able to embed loss-less data into an image. As such I've been experimenting with steganography, specifically via modification of DCT coefficients as the method I select, apart from being loss-less must also be relatively resilient against format conversion, scaling/DSP etc. From the research I've done thus far this method seems to be the best candidate. I've seen a number of papers on the subject which all seem to neglect specific details (some neglect to mention modification of 0 coefficients, or modification of AC coefficient etc). After combining the findings and making a few modifications of my own which include: 1) Using a more quantized version of the DCT matrix to ensure we only modify coefficients that would still be present should the image be JPEG'ed further or processed (I'm using this in place of simply following a zig-zag pattern). 2) I'm modifying bit 4 instead of the LSB and then based on what the original bit value was adjusting the lower bits to minimize the difference. 3) I'm only modifying the blue channel as it should be the least visible. This process must modify the actual image and not the DCT values stored in file (like jsteg) as there is no guarantee the file will be a JPEG, it may also be opened and re-saved at a later stage in a different format. For added robustness I've included the message multiple times and use the bits that occur most often, I had considered using a QR code as the message data or simply applying the reed-solomon error correction, but for this simple application and given that the "message" in question is usually going to be between 10-32 bytes I have plenty of room to repeat it which should provide sufficient redundancy to recover the true bits. No matter what I do I don't seem to be able to recover the bits at the decode stage. I've tried including / excluding various checks (even if it degrades image quality for the time being). I've tried using fixed point vs. double arithmetic, moving the bit to encode, I suspect that the message bits are being lost during the IDCT back to image. Any thoughts or suggestions on how to get this working would be hugely appreciated. (PS I am aware that the actual DCT/IDCT could be optimized from it's naive On4 operation using row column algorithm, or an FDCT like AAN, but for now it just needs to work :) ) Reference Papers: http://www.lokminglui.com/dct.pdf http://arxiv.org/ftp/arxiv/papers/1006/1006.1186.pdf Code for the Encode/Decode process in C# below: using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Drawing.Imaging; using System.Drawing; namespace ImageKey { public class Encoder { public const int HIDE_BIT_POS = 3; // use bit position 4 (1 << 3). public const int HIDE_COUNT = 16; // Number of times to repeat the message to avoid error. // JPEG Standard Quantization Matrix. // (to get higher quality multiply by (100-quality)/50 .. // for lower than 50 multiply by 50/quality. Then round to integers and clip to ensure only positive integers. public static double[] Q = {16,11,10,16,24,40,51,61, 12,12,14,19,26,58,60,55, 14,13,16,24,40,57,69,56, 14,17,22,29,51,87,80,62, 18,22,37,56,68,109,103,77, 24,35,55,64,81,104,113,92, 49,64,78,87,103,121,120,101, 72,92,95,98,112,100,103,99}; // Maximum qauality quantization matrix (if all 1's doesn't modify coefficients at all). public static double[] Q2 = {1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1, 1,1,1,1,1,1,1,1}; public static Bitmap Encode(Bitmap b, string key) { Bitmap response = new Bitmap(b.Width, b.Height, PixelFormat.Format32bppArgb); uint imgWidth = ((uint)b.Width) & ~((uint)7); // Maximum usable X resolution (divisible by 8). uint imgHeight = ((uint)b.Height) & ~((uint)7); // Maximum usable Y resolution (divisible by 8). // Start be transferring the unmodified image portions. // As we'll be using slightly less width/height for the encoding process we'll need the edges to be populated. for (int y = 0; y < b.Height; y++) for (int x = 0; x < b.Width; x++) { if( (x >= imgWidth && x < b.Width) || (y>=imgHeight && y < b.Height)) response.SetPixel(x, y, b.GetPixel(x, y)); } // Setup the counters and byte data for the message to encode. StringBuilder sb = new StringBuilder(); for(int i=0;i<HIDE_COUNT;i++) sb.Append(key); byte[] codeBytes = System.Text.Encoding.ASCII.GetBytes(sb.ToString()); int bitofs = 0; // Current bit position we've encoded too. int totalBits = (codeBytes.Length * 8); // Total number of bits to encode. for (int y = 0; y < imgHeight; y += 8) { for (int x = 0; x < imgWidth; x += 8) { int[] redData = GetRedChannelData(b, x, y); int[] greenData = GetGreenChannelData(b, x, y); int[] blueData = GetBlueChannelData(b, x, y); int[] newRedData; int[] newGreenData; int[] newBlueData; if (bitofs < totalBits) { double[] redDCT = DCT(ref redData); double[] greenDCT = DCT(ref greenData); double[] blueDCT = DCT(ref blueData); int[] redDCTI = Quantize(ref redDCT, ref Q2); int[] greenDCTI = Quantize(ref greenDCT, ref Q2); int[] blueDCTI = Quantize(ref blueDCT, ref Q2); int[] blueDCTC = Quantize(ref blueDCT, ref Q); HideBits(ref blueDCTI, ref blueDCTC, ref bitofs, ref totalBits, ref codeBytes); double[] redDCT2 = DeQuantize(ref redDCTI, ref Q2); double[] greenDCT2 = DeQuantize(ref greenDCTI, ref Q2); double[] blueDCT2 = DeQuantize(ref blueDCTI, ref Q2); newRedData = IDCT(ref redDCT2); newGreenData = IDCT(ref greenDCT2); newBlueData = IDCT(ref blueDCT2); } else { newRedData = redData; newGreenData = greenData; newBlueData = blueData; } MapToRGBRange(ref newRedData); MapToRGBRange(ref newGreenData); MapToRGBRange(ref newBlueData); for(int dy=0;dy<8;dy++) { for(int dx=0;dx<8;dx++) { int col = (0xff<<24) + (newRedData[dx+(dy*8)]<<16) + (newGreenData[dx+(dy*8)]<<8) + (newBlueData[dx+(dy*8)]); response.SetPixel(x+dx,y+dy,Color.FromArgb(col)); } } } } if (bitofs < totalBits) throw new Exception("Failed to encode data - insufficient cover image coefficients"); return (response); } public static void HideBits(ref int[] DCTMatrix, ref int[] CMatrix, ref int bitofs, ref int totalBits, ref byte[] codeBytes) { int tempValue = 0; for (int u = 0; u < 8; u++) { for (int v = 0; v < 8; v++) { if ( (u != 0 || v != 0) && CMatrix[v+(u*8)] != 0 && DCTMatrix[v+(u*8)] != 0) { if (bitofs < totalBits) { tempValue = DCTMatrix[v + (u * 8)]; int bytePos = (bitofs) >> 3; int bitPos = (bitofs) % 8; byte mask = (byte)(1 << bitPos); byte value = (byte)((codeBytes[bytePos] & mask) >> bitPos); // 0 or 1. if (value == 0) { int a = DCTMatrix[v + (u * 8)] & (1 << HIDE_BIT_POS); if (a != 0) DCTMatrix[v + (u * 8)] |= (1 << HIDE_BIT_POS) - 1; DCTMatrix[v + (u * 8)] &= ~(1 << HIDE_BIT_POS); } else if (value == 1) { int a = DCTMatrix[v + (u * 8)] & (1 << HIDE_BIT_POS); if (a == 0) DCTMatrix[v + (u * 8)] &= ~((1 << HIDE_BIT_POS) - 1); DCTMatrix[v + (u * 8)] |= (1 << HIDE_BIT_POS); } if (DCTMatrix[v + (u * 8)] != 0) bitofs++; else DCTMatrix[v + (u * 8)] = tempValue; } } } } } public static void MapToRGBRange(ref int[] data) { for(int i=0;i<data.Length;i++) { data[i] += 128; if(data[i] < 0) data[i] = 0; else if(data[i] > 255) data[i] = 255; } } public static int[] GetRedChannelData(Bitmap b, int sx, int sy) { int[] data = new int[8 * 8]; for (int y = sy; y < (sy + 8); y++) { for (int x = sx; x < (sx + 8); x++) { uint col = (uint)b.GetPixel(x,y).ToArgb(); data[(x - sx) + ((y - sy) * 8)] = (int)((col >> 16) & 0xff) - 128; } } return (data); } public static int[] GetGreenChannelData(Bitmap b, int sx, int sy) { int[] data = new int[8 * 8]; for (int y = sy; y < (sy + 8); y++) { for (int x = sx; x < (sx + 8); x++) { uint col = (uint)b.GetPixel(x, y).ToArgb(); data[(x - sx) + ((y - sy) * 8)] = (int)((col >> 8) & 0xff) - 128; } } return (data); } public static int[] GetBlueChannelData(Bitmap b, int sx, int sy) { int[] data = new int[8 * 8]; for (int y = sy; y < (sy + 8); y++) { for (int x = sx; x < (sx + 8); x++) { uint col = (uint)b.GetPixel(x, y).ToArgb(); data[(x - sx) + ((y - sy) * 8)] = (int)((col >> 0) & 0xff) - 128; } } return (data); } public static int[] Quantize(ref double[] DCTMatrix, ref double[] Q) { int[] DCTMatrixOut = new int[8*8]; for (int u = 0; u < 8; u++) { for (int v = 0; v < 8; v++) { DCTMatrixOut[v + (u * 8)] = (int)Math.Round(DCTMatrix[v + (u * 8)] / Q[v + (u * 8)]); } } return(DCTMatrixOut); } public static double[] DeQuantize(ref int[] DCTMatrix, ref double[] Q) { double[] DCTMatrixOut = new double[8*8]; for (int u = 0; u < 8; u++) { for (int v = 0; v < 8; v++) { DCTMatrixOut[v + (u * 8)] = (double)DCTMatrix[v + (u * 8)] * Q[v + (u * 8)]; } } return(DCTMatrixOut); } public static double[] DCT(ref int[] data) { double[] DCTMatrix = new double[8 * 8]; for (int v = 0; v < 8; v++) { for (int u = 0; u < 8; u++) { double cu = 1; if (u == 0) cu = (1.0 / Math.Sqrt(2.0)); double cv = 1; if (v == 0) cv = (1.0 / Math.Sqrt(2.0)); double sum = 0.0; for (int y = 0; y < 8; y++) { for (int x = 0; x < 8; x++) { double s = data[x + (y * 8)]; double dctVal = Math.Cos((2 * y + 1) * v * Math.PI / 16) * Math.Cos((2 * x + 1) * u * Math.PI / 16); sum += s * dctVal; } } DCTMatrix[u + (v * 8)] = (0.25 * cu * cv * sum); } } return (DCTMatrix); } public static int[] IDCT(ref double[] DCTMatrix) { int[] Matrix = new int[8 * 8]; for (int y = 0; y < 8; y++) { for (int x = 0; x < 8; x++) { double sum = 0; for (int v = 0; v < 8; v++) { for (int u = 0; u < 8; u++) { double cu = 1; if (u == 0) cu = (1.0 / Math.Sqrt(2.0)); double cv = 1; if (v == 0) cv = (1.0 / Math.Sqrt(2.0)); double idctVal = (cu * cv) / 4.0 * Math.Cos((2 * y + 1) * v * Math.PI / 16) * Math.Cos((2 * x + 1) * u * Math.PI / 16); sum += (DCTMatrix[u + (v * 8)] * idctVal); } } Matrix[x + (y * 8)] = (int)Math.Round(sum); } } return (Matrix); } } public class Decoder { public static string Decode(Bitmap b, int expectedLength) { expectedLength *= Encoder.HIDE_COUNT; uint imgWidth = ((uint)b.Width) & ~((uint)7); // Maximum usable X resolution (divisible by 8). uint imgHeight = ((uint)b.Height) & ~((uint)7); // Maximum usable Y resolution (divisible by 8). // Setup the counters and byte data for the message to decode. byte[] codeBytes = new byte[expectedLength]; byte[] outBytes = new byte[expectedLength / Encoder.HIDE_COUNT]; int bitofs = 0; // Current bit position we've decoded too. int totalBits = (codeBytes.Length * 8); // Total number of bits to decode. for (int y = 0; y < imgHeight; y += 8) { for (int x = 0; x < imgWidth; x += 8) { int[] blueData = ImageKey.Encoder.GetBlueChannelData(b, x, y); double[] blueDCT = ImageKey.Encoder.DCT(ref blueData); int[] blueDCTI = ImageKey.Encoder.Quantize(ref blueDCT, ref Encoder.Q2); int[] blueDCTC = ImageKey.Encoder.Quantize(ref blueDCT, ref Encoder.Q); if (bitofs < totalBits) GetBits(ref blueDCTI, ref blueDCTC, ref bitofs, ref totalBits, ref codeBytes); } } bitofs = 0; for (int i = 0; i < (expectedLength / Encoder.HIDE_COUNT) * 8; i++) { int bytePos = (bitofs) >> 3; int bitPos = (bitofs) % 8; byte mask = (byte)(1 << bitPos); List<int> values = new List<int>(); int zeroCount = 0; int oneCount = 0; for (int j = 0; j < Encoder.HIDE_COUNT; j++) { int val = (codeBytes[bytePos + ((expectedLength / Encoder.HIDE_COUNT) * j)] & mask) >> bitPos; values.Add(val); if (val == 0) zeroCount++; else oneCount++; } if (oneCount >= zeroCount) outBytes[bytePos] |= mask; bitofs++; values.Clear(); } return (System.Text.Encoding.ASCII.GetString(outBytes)); } public static void GetBits(ref int[] DCTMatrix, ref int[] CMatrix, ref int bitofs, ref int totalBits, ref byte[] codeBytes) { for (int u = 0; u < 8; u++) { for (int v = 0; v < 8; v++) { if ((u != 0 || v != 0) && CMatrix[v + (u * 8)] != 0 && DCTMatrix[v + (u * 8)] != 0) { if (bitofs < totalBits) { int bytePos = (bitofs) >> 3; int bitPos = (bitofs) % 8; byte mask = (byte)(1 << bitPos); int value = DCTMatrix[v + (u * 8)] & (1 << Encoder.HIDE_BIT_POS); if (value != 0) codeBytes[bytePos] |= mask; bitofs++; } } } } } } } UPDATE: By switching to using a QR Code as the source message and swapping a pair of coefficients in each block instead of bit manipulation I've been able to get the message to survive the transform. However to get the message to come through without corruption I have to adjust both coefficients as well as swap them. For example swapping (3,4) and (4,3) in the DCT matrix and then respectively adding 8 and subtracting 8 as an arbitrary constant seems to work. This survives a re-JPEG'ing of 96 but any form of scaling/cropping destroys the message again. I was hoping that by operating on mid to low frequency values that the message would be preserved even under some light image manipulation.

Read the article

Registration form validation not validating

- by jgray

I am a noob when it comes to web development. I am trying to validate a registration form and to me it looks right but it will not validate.. This is what i have so far and i am validating through a repository or database. Any help would be greatly appreciated. thanks <?php session_start(); $title = "User Registration"; $keywords = "Name, contact, phone, e-mail, registration"; $description = "user registration becoming a member."; require "partials/_html_header.php"; //require "partials/_header.php"; require "partials/_menu.php"; require "DataRepository.php"; // if all validation passed save user $db = new DataRepository(); // form validation goes here $first_nameErr = $emailErr = $passwordErr = $passwordConfirmErr = ""; $first_name = $last_name = $email = $password = $passwordConfirm = ""; if(isset($_POST['submit'])) { $valid = TRUE; // check if all fields are valid { if ($_SERVER["REQUEST_METHOD"] == "POST") { if (empty($_POST["first_name"])) {$first_nameErr = "Name is required";} else { // $first_name = test_input($_POST["first_name"]); // check if name only contains letters and whitespace if (!preg_match("/^[a-zA-Z ]*$/",$first_name)) { $first_nameErr = "Only letters and white space allowed"; } } if (empty($_POST["email"])) {$emailErr = "Email is required";} else { // $email = test_input($_POST["email"]); // check if e-mail address syntax is valid if (!preg_match("/([\w\-]+\@[\w\-]+\.[\w\-]+)/",$email)) { $emailErr = "Invalid email format"; } } if (!preg_match("/(......)/",$password)) { $passwordErr = "Subject must contain THREE or more characters!"; } if ($_POST['password']!= $_POST['passwordConfirm']) { echo("Oops! Password did not match! Try again. "); } function test_input($data) { $data = trim($data); $data = stripslashes($data); $data = htmlspecialchars($data); return $data; } } } if(!$db->isEmailUnique($_POST['email'])) { $valid = FALSE; //display errors in the correct places } // if still valid save the user if($valid) { $new_user = array( 'first_name' => $_POST['first_name'], 'last_name' => $_POST['last_name'], 'email' => $_POST['email'], 'password' => $_POST['password'] ); $results = $db->saveUser($new_user); if($results == TRUE) { header("Location: login.php"); } else { echo "WTF!"; exit; } } } ?> <head> <style> .error {color: #FF0000;} </style> </head> <h1 class="center"> World Wide Web Creations' User Registration </h1> <p><span class="error"></span><p> <form method="POST" action="<?php echo htmlspecialchars($_SERVER["PHP_SELF"]);?>" onsubmit="return validate_form()" > First Name: <input type="text" name="first_name" id="first_name" value="<?php echo $first_name;?>" /> <span class="error"> <?php echo $first_nameErr;?></span> <br /> <br /> Last Name(Optional): <input type="text" name="last_name" id="last_name" value="<?php echo $last_name;?>" /> <br /> <br /> E-mail: <input type="email" name="email" id="email" value="<?php echo $email;?>" /> <span class="error"> <?php echo $emailErr;?></span> <br /> <br /> Password: <input type="password" name="password" id="password" value="" /> <span class="error"> <?php echo $passwordErr;?></span> <br /> <br /> Confirmation Password: <input type="password" name="passwordConfirm" id="passwordConfirm" value="" /> <span class="error"> <?php echo $passwordConfirmErr;?></span> <br /> <br /> <br /> <br /> <input type="submit" name="submit" id="submit" value="Submit Data" /> <input type="reset" name="reset" id="reset" value="Reset Form" /> </form> </body> </html> <?php require "partials/_footer.php"; require "partials/_html_footer.php"; ?> class DataRepository { // version number private $version = "1.0.3"; // turn on and off debugging private static $debug = FALSE; // flag to (re)initialize db on each call private static $initialize_db = FALSE; // insert test data on initialization private static $load_default_data = TRUE; const DATAFILE = "203data.txt"; private $data = NULL; private $errors = array(); private $user_fields = array( 'id' => array('required' => 0), 'created_at' => array('required' => 0), 'updated_at' => array('required' => 0), 'first_name' => array('required' => 1), 'last_name' => array('required' => 0), 'email' => array('required' => 1), 'password' => array('required' => 1), 'level' => array('required' => 0, 'default' => 2), ); private $post_fields = array( 'id' => array('required' => 0), 'created_at' => array('required' => 0), 'updated_at' => array('required' => 0), 'user_id' => array('required' => 1), 'title' => array('required' => 1), 'message' => array('required' => 1), 'private' => array('required' => 0, 'default' => 0), ); private $default_user = array( 'id' => 1, 'created_at' => '2013-01-01 00:00:00', 'updated_at' => '2013-01-01 00:00:00', 'first_name' => 'Admin Joe', 'last_name' => 'Tester', 'email' => '[email protected]', 'password' => 'a94a8fe5ccb19ba61c4c0873d391e987982fbbd3', 'level' => 1, ); private $default_post = array( 'id' => 1, 'created_at' => '2013-01-01 00:00:00', 'updated_at' => '2013-01-01 00:00:00', 'user_id' => 1, 'title' => 'My First Post', 'message' => 'This is the message of the first post.', 'private' => 0, ); // constructor will load existing data into memory // if it does not exist it will create it and initialize if desired public function __construct() { // check if need to reset if(DataRepository::$initialize_db AND file_exists(DataRepository::DATAFILE)) { unlink(DataRepository::DATAFILE); } // if file doesn't exist, create the initial datafile if(!file_exists(DataRepository::DATAFILE)) { $this->log("Data file does not exist. Attempting to create it... (".__FUNCTION__.":".__LINE__.")"); // create initial file $this->data = array( 'users' => array( ), 'posts' => array() ); // load default data if needed if(DataRepository::$load_default_data) { $this->data['users'][1] = $this->default_user; $this->data['posts'][1] = $this->default_post; } $this->writeTheData(); } // load the data into memory for use $this->loadTheData(); } private function showErrors($break = TRUE, $type = NULL) { if(count($this->errors) > 0) { echo "<div style=\"color:red;font-weight: bold;font-size: 1.3em\":<h3>$type Errors</h3><ol>"; foreach($this->errors AS $error) { echo "<li>$error</li>"; } echo "</ol></div>"; if($break) { "</br></br></br>Exiting because of errors!"; exit; } } } private function writeTheData() { $this->log("Attempting to write the datafile: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); file_put_contents(DataRepository::DATAFILE, json_encode($this->data)); $this->log("Datafile written: ".DataRepository::DATAFILE." (line: ".__LINE__.")"); } private function loadTheData() { $this->log("Attempting to load the datafile: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); $this->data = json_decode(file_get_contents(DataRepository::DATAFILE), true); $this->log("Datafile loaded: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $this->data); } private function validateFields(&$info, $fields, $pre_errors = NULL) { // merge in any pre_errors if($pre_errors != NULL) { $this->errors = array_merge($this->errors, $pre_errors); } // check all required fields foreach($fields AS $field => $reqs) { if(isset($reqs['required']) AND $reqs['required'] == 1) { if(!isset($info[$field]) OR strlen($info[$field]) == 0) { $this->errors[] = "$field is a REQUIRED field"; } } // set any default values if not present if(isset($reqs['default']) AND (!isset($info[$field]) OR $info[$field] == "")) { $info[$field] = $reqs['default']; } } $this->showErrors(); if(count($this->errors) == 0) { return TRUE; } else { return FALSE; } } private function validateUser(&$user_info) { // check if the email is already in use $this->log("About to check pre_errors: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $user_info); $pre_errors = NULL; if(isset($user_info['email'])) { if(!$this->isEmailUnique($user_info['email'])) { $pre_errors = array('The email: '.$user_info['email'].' is already used in our system'); } } $this->log("After pre_error check: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $pre_errors); return $this->validateFields($user_info, $this->user_fields, $pre_errors); } private function validatePost(&$post_info) { // check if the user_id in the post actually exists $this->log("About to check pre_errors: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $post_info); $pre_errors = NULL; if(isset($post_info['user_id'])) { if(!isset($this->data['users'][$post_info['user_id']])) { $pre_errors = array('The posts must belong to a valid user. (User '.$post_info['user_id'].' does not exist in the data'); } } $this->log("After pre_error check: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $pre_errors); return $this->validateFields($post_info, $this->post_fields, $pre_errors); } private function log($message, $data = NULL) { $style = "background-color: #F8F8F8; border: 1px solid #DDDDDD; border-radius: 3px; font-size: 13px; line-height: 19px; overflow: auto; padding: 6px 10px;"; if(DataRepository::$debug) { if($data != NULL) { $dump = "<div style=\"$style\"><pre>".json_encode($data, JSON_PRETTY_PRINT)."</pre></div>"; } else { $dump = NULL; } echo "<code><b>Debug:</b> $message</code>$dump<br />"; } } public function saveUser($user_info) { $this->log("Entering saveUser: (".__FUNCTION__.":".__LINE__.")", $user_info); $mydata = array(); $update = FALSE; // check for existing data if(isset($user_info['id']) AND $this->data['users'][$user_info['id']]) { $mydata = $this->data['users'][$user_info['id']]; $this->log("Loaded prior user: ".print_r($mydata, TRUE)." (".__FUNCTION__.":".__LINE__.")"); } // copy over existing values $this->log("Before copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); foreach($user_info AS $k => $v) { $mydata[$k] = $user_info[$k]; } $this->log("After copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); // check required fields if($this->validateUser($mydata)) { // hash password if new if(isset($mydata['password'])) { $mydata['password'] = sha1($mydata['password']); } // if no id, add the next available one if(!isset($mydata['id']) OR (int)$mydata['id'] < 1) { $this->log("No id set: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); if(count($this->data['users']) == 0) { $mydata['id'] = 1; $this->log("Setting id to 1: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } else { $mydata['id'] = max(array_keys($this->data['users']))+1; $this->log("Found max id and added 1 [".$mydata['id']."]: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } } // set created date if null if(!isset($mydata['created_at'])) { $mydata['created_at'] = date ("Y-m-d H:i:s", time()); } // update modified time $mydata['modified_at'] = date ("Y-m-d H:i:s", time()); // copy into data and save $this->log("Before data save: (".__FUNCTION__.":".__LINE__.")", $this->data); $this->data['users'][$mydata['id']] = $mydata; $this->writeTheData(); } return TRUE; } public function getUserById($id) { if(isset($this->data['users'][$id])) { return $this->data['users'][$id]; } else { return array(); } } public function isEmailUnique($email) { // find the user that has the right username/password foreach($this->data['users'] AS $k => $v) { $this->log("Checking unique email: {$v['email']} == $email (".__FUNCTION__.":".__LINE__.")", NULL); if($v['email'] == $email) { $this->log("FOUND NOT unique email: {$v['email']} == $email (".__FUNCTION__.":".__LINE__.")", NULL); return FALSE; break; } } $this->log("Email IS unique: $email (".__FUNCTION__.":".__LINE__.")", NULL); return TRUE; } public function login($username, $password) { // hash password for validation $password = sha1($password); $this->log("Attempting to login with $username / $password: (".__FUNCTION__.":".__LINE__.")", NULL); $user = NULL; // find the user that has the right username/password foreach($this->data['users'] AS $k => $v) { if($v['email'] == $username AND $v['password'] == $password) { $user = $v; break; } } $this->log("Exiting login: (".__FUNCTION__.":".__LINE__.")", $user); return $user; } public function savePost($post_info) { $this->log("Entering savePost: (".__FUNCTION__.":".__LINE__.")", $post_info); $mydata = array(); // check for existing data if(isset($post_info['id']) AND $this->data['posts'][$post_info['id']]) { $mydata = $this->data['posts'][$post_info['id']]; $this->log("Loaded prior posts: ".print_r($mydata, TRUE)." (".__FUNCTION__.":".__LINE__.")"); } $this->log("Before copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); foreach($post_info AS $k => $v) { $mydata[$k] = $post_info[$k]; } $this->log("After copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); // check required fields if($this->validatePost($mydata)) { // if no id, add the next available one if(!isset($mydata['id']) OR (int)$mydata['id'] < 1) { $this->log("No id set: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); if(count($this->data['posts']) == 0) { $mydata['id'] = 1; $this->log("Setting id to 1: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } else { $mydata['id'] = max(array_keys($this->data['posts']))+1; $this->log("Found max id and added 1 [".$mydata['id']."]: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } } // set created date if null if(!isset($mydata['created_at'])) { $mydata['created_at'] = date ("Y-m-d H:i:s", time()); } // update modified time $mydata['modified_at'] = date ("Y-m-d H:i:s", time()); // copy into data and save $this->data['posts'][$mydata['id']] = $mydata; $this->log("Before data save: (".__FUNCTION__.":".__LINE__.")", $this->data); $this->writeTheData(); } return TRUE; } public function getAllPosts() { return $this->loadPostsUsers($this->data['posts']); } public function loadPostsUsers($posts) { foreach($posts AS $id => $post) { $posts[$id]['user'] = $this->getUserById($post['user_id']); } return $posts; } public function dump($line_number, $temp = 'NO') { // if(DataRepository::$debug) { if($temp == 'NO') { $temp = $this->data; } echo "<pre>Dumping from line: $line_number\n"; echo json_encode($temp, JSON_PRETTY_PRINT); echo "</pre>"; } } } /* * Change Log * * 1.0.0 * - first version * 1.0.1 * - Added isEmailUnique() function for form validation and precheck on user save * 1.0.2 * - Fixed getAllPosts() to include the post's user info * - Added loadPostsUsers() to load one or more posts with their user info * 1.0.3 * - Added autoload to always add admin Joe. */

Read the article

What are good design practices when working with Entity Framework

- by AD

This will apply mostly for an asp.net application where the data is not accessed via soa. Meaning that you get access to the objects loaded from the framework, not Transfer Objects, although some recommendation still apply. This is a community post, so please add to it as you see fit. Applies to: Entity Framework 1.0 shipped with Visual Studio 2008 sp1. Why pick EF in the first place? Considering it is a young technology with plenty of problems (see below), it may be a hard sell to get on the EF bandwagon for your project. However, it is the technology Microsoft is pushing (at the expense of Linq2Sql, which is a subset of EF). In addition, you may not be satisfied with NHibernate or other solutions out there. Whatever the reasons, there are people out there (including me) working with EF and life is not bad.make you think. EF and inheritance The first big subject is inheritance. EF does support mapping for inherited classes that are persisted in 2 ways: table per class and table the hierarchy. The modeling is easy and there are no programming issues with that part. (The following applies to table per class model as I don't have experience with table per hierarchy, which is, anyway, limited.) The real problem comes when you are trying to run queries that include one or many objects that are part of an inheritance tree: the generated sql is incredibly awful, takes a long time to get parsed by the EF and takes a long time to execute as well. This is a real show stopper. Enough that EF should probably not be used with inheritance or as little as possible. Here is an example of how bad it was. My EF model had ~30 classes, ~10 of which were part of an inheritance tree. On running a query to get one item from the Base class, something as simple as Base.Get(id), the generated SQL was over 50,000 characters. Then when you are trying to return some Associations, it degenerates even more, going as far as throwing SQL exceptions about not being able to query more than 256 tables at once. Ok, this is bad, EF concept is to allow you to create your object structure without (or with as little as possible) consideration on the actual database implementation of your table. It completely fails at this. So, recommendations? Avoid inheritance if you can, the performance will be so much better. Use it sparingly where you have to. In my opinion, this makes EF a glorified sql-generation tool for querying, but there are still advantages to using it. And ways to implement mechanism that are similar to inheritance. Bypassing inheritance with Interfaces First thing to know with trying to get some kind of inheritance going with EF is that you cannot assign a non-EF-modeled class a base class. Don't even try it, it will get overwritten by the modeler. So what to do? You can use interfaces to enforce that classes implement some functionality. For example here is a IEntity interface that allow you to define Associations between EF entities where you don't know at design time what the type of the entity would be. public enum EntityTypes{ Unknown = -1, Dog = 0, Cat } public interface IEntity { int EntityID { get; } string Name { get; } Type EntityType { get; } } public partial class Dog : IEntity { // implement EntityID and Name which could actually be fields // from your EF model Type EntityType{ get{ return EntityTypes.Dog; } } } Using this IEntity, you can then work with undefined associations in other classes // lets take a class that you defined in your model. // that class has a mapping to the columns: PetID, PetType public partial class Person { public IEntity GetPet() { return IEntityController.Get(PetID,PetType); } } which makes use of some extension functions: public class IEntityController { static public IEntity Get(int id, EntityTypes type) { switch (type) { case EntityTypes.Dog: return Dog.Get(id); case EntityTypes.Cat: return Cat.Get(id); default: throw new Exception("Invalid EntityType"); } } } Not as neat as having plain inheritance, particularly considering you have to store the PetType in an extra database field, but considering the performance gains, I would not look back. It also cannot model one-to-many, many-to-many relationship, but with creative uses of 'Union' it could be made to work. Finally, it creates the side effet of loading data in a property/function of the object, which you need to be careful about. Using a clear naming convention like GetXYZ() helps in that regards. Compiled Queries Entity Framework performance is not as good as direct database access with ADO (obviously) or Linq2SQL. There are ways to improve it however, one of which is compiling your queries. The performance of a compiled query is similar to Linq2Sql. What is a compiled query? It is simply a query for which you tell the framework to keep the parsed tree in memory so it doesn't need to be regenerated the next time you run it. So the next run, you will save the time it takes to parse the tree. Do not discount that as it is a very costly operation that gets even worse with more complex queries. There are 2 ways to compile a query: creating an ObjectQuery with EntitySQL and using CompiledQuery.Compile() function. (Note that by using an EntityDataSource in your page, you will in fact be using ObjectQuery with EntitySQL, so that gets compiled and cached). An aside here in case you don't know what EntitySQL is. It is a string-based way of writing queries against the EF. Here is an example: "select value dog from Entities.DogSet as dog where dog.ID = @ID". The syntax is pretty similar to SQL syntax. You can also do pretty complex object manipulation, which is well explained [here][1]. Ok, so here is how to do it using ObjectQuery< string query = "select value dog " + "from Entities.DogSet as dog " + "where dog.ID = @ID"; ObjectQuery<Dog> oQuery = new ObjectQuery<Dog>(query, EntityContext.Instance)); oQuery.Parameters.Add(new ObjectParameter("ID", id)); oQuery.EnablePlanCaching = true; return oQuery.FirstOrDefault(); The first time you run this query, the framework will generate the expression tree and keep it in memory. So the next time it gets executed, you will save on that costly step. In that example EnablePlanCaching = true, which is unnecessary since that is the default option. The other way to compile a query for later use is the CompiledQuery.Compile method. This uses a delegate: static readonly Func<Entities, int, Dog> query_GetDog = CompiledQuery.Compile<Entities, int, Dog>((ctx, id) => ctx.DogSet.FirstOrDefault(it => it.ID == id)); or using linq static readonly Func<Entities, int, Dog> query_GetDog = CompiledQuery.Compile<Entities, int, Dog>((ctx, id) => (from dog in ctx.DogSet where dog.ID == id select dog).FirstOrDefault()); to call the query: query_GetDog.Invoke( YourContext, id ); The advantage of CompiledQuery is that the syntax of your query is checked at compile time, where as EntitySQL is not. However, there are other consideration... Includes Lets say you want to have the data for the dog owner to be returned by the query to avoid making 2 calls to the database. Easy to do, right? EntitySQL string query = "select value dog " + "from Entities.DogSet as dog " + "where dog.ID = @ID"; ObjectQuery<Dog> oQuery = new ObjectQuery<Dog>(query, EntityContext.Instance)).Include("Owner"); oQuery.Parameters.Add(new ObjectParameter("ID", id)); oQuery.EnablePlanCaching = true; return oQuery.FirstOrDefault(); CompiledQuery static readonly Func<Entities, int, Dog> query_GetDog = CompiledQuery.Compile<Entities, int, Dog>((ctx, id) => (from dog in ctx.DogSet.Include("Owner") where dog.ID == id select dog).FirstOrDefault()); Now, what if you want to have the Include parametrized? What I mean is that you want to have a single Get() function that is called from different pages that care about different relationships for the dog. One cares about the Owner, another about his FavoriteFood, another about his FavotireToy and so on. Basicly, you want to tell the query which associations to load. It is easy to do with EntitySQL public Dog Get(int id, string include) { string query = "select value dog " + "from Entities.DogSet as dog " + "where dog.ID = @ID"; ObjectQuery<Dog> oQuery = new ObjectQuery<Dog>(query, EntityContext.Instance)) .IncludeMany(include); oQuery.Parameters.Add(new ObjectParameter("ID", id)); oQuery.EnablePlanCaching = true; return oQuery.FirstOrDefault(); } The include simply uses the passed string. Easy enough. Note that it is possible to improve on the Include(string) function (that accepts only a single path) with an IncludeMany(string) that will let you pass a string of comma-separated associations to load. Look further in the extension section for this function. If we try to do it with CompiledQuery however, we run into numerous problems: The obvious static readonly Func<Entities, int, string, Dog> query_GetDog = CompiledQuery.Compile<Entities, int, string, Dog>((ctx, id, include) => (from dog in ctx.DogSet.Include(include) where dog.ID == id select dog).FirstOrDefault()); will choke when called with: query_GetDog.Invoke( YourContext, id, "Owner,FavoriteFood" ); Because, as mentionned above, Include() only wants to see a single path in the string and here we are giving it 2: "Owner" and "FavoriteFood" (which is not to be confused with "Owner.FavoriteFood"!). Then, let's use IncludeMany(), which is an extension function static readonly Func<Entities, int, string, Dog> query_GetDog = CompiledQuery.Compile<Entities, int, string, Dog>((ctx, id, include) => (from dog in ctx.DogSet.IncludeMany(include) where dog.ID == id select dog).FirstOrDefault()); Wrong again, this time it is because the EF cannot parse IncludeMany because it is not part of the functions that is recognizes: it is an extension. Ok, so you want to pass an arbitrary number of paths to your function and Includes() only takes a single one. What to do? You could decide that you will never ever need more than, say 20 Includes, and pass each separated strings in a struct to CompiledQuery. But now the query looks like this: from dog in ctx.DogSet.Include(include1).Include(include2).Include(include3) .Include(include4).Include(include5).Include(include6) .[...].Include(include19).Include(include20) where dog.ID == id select dog which is awful as well. Ok, then, but wait a minute. Can't we return an ObjectQuery< with CompiledQuery? Then set the includes on that? Well, that what I would have thought so as well: static readonly Func<Entities, int, ObjectQuery<Dog>> query_GetDog = CompiledQuery.Compile<Entities, int, string, ObjectQuery<Dog>>((ctx, id) => (ObjectQuery<Dog>)(from dog in ctx.DogSet where dog.ID == id select dog)); public Dog GetDog( int id, string include ) { ObjectQuery<Dog> oQuery = query_GetDog(id); oQuery = oQuery.IncludeMany(include); return oQuery.FirstOrDefault; } That should have worked, except that when you call IncludeMany (or Include, Where, OrderBy...) you invalidate the cached compiled query because it is an entirely new one now! So, the expression tree needs to be reparsed and you get that performance hit again. So what is the solution? You simply cannot use CompiledQueries with parametrized Includes. Use EntitySQL instead. This doesn't mean that there aren't uses for CompiledQueries. It is great for localized queries that will always be called in the same context. Ideally CompiledQuery should always be used because the syntax is checked at compile time, but due to limitation, that's not possible. An example of use would be: you may want to have a page that queries which two dogs have the same favorite food, which is a bit narrow for a BusinessLayer function, so you put it in your page and know exactly what type of includes are required. Passing more than 3 parameters to a CompiledQuery Func is limited to 5 parameters, of which the last one is the return type and the first one is your Entities object from the model. So that leaves you with 3 parameters. A pitance, but it can be improved on very easily. public struct MyParams { public string param1; public int param2; public DateTime param3; } static readonly Func<Entities, MyParams, IEnumerable<Dog>> query_GetDog = CompiledQuery.Compile<Entities, MyParams, IEnumerable<Dog>>((ctx, myParams) => from dog in ctx.DogSet where dog.Age == myParams.param2 && dog.Name == myParams.param1 and dog.BirthDate > myParams.param3 select dog); public List<Dog> GetSomeDogs( int age, string Name, DateTime birthDate ) { MyParams myParams = new MyParams(); myParams.param1 = name; myParams.param2 = age; myParams.param3 = birthDate; return query_GetDog(YourContext,myParams).ToList(); } Return Types (this does not apply to EntitySQL queries as they aren't compiled at the same time during execution as the CompiledQuery method) Working with Linq, you usually don't force the execution of the query until the very last moment, in case some other functions downstream wants to change the query in some way: static readonly Func<Entities, int, string, IEnumerable<Dog>> query_GetDog = CompiledQuery.Compile<Entities, int, string, IEnumerable<Dog>>((ctx, age, name) => from dog in ctx.DogSet where dog.Age == age && dog.Name == name select dog); public IEnumerable<Dog> GetSomeDogs( int age, string name ) { return query_GetDog(YourContext,age,name); } public void DataBindStuff() { IEnumerable<Dog> dogs = GetSomeDogs(4,"Bud"); // but I want the dogs ordered by BirthDate gridView.DataSource = dogs.OrderBy( it => it.BirthDate ); } What is going to happen here? By still playing with the original ObjectQuery (that is the actual return type of the Linq statement, which implements IEnumerable), it will invalidate the compiled query and be force to re-parse. So, the rule of thumb is to return a List< of objects instead. static readonly Func<Entities, int, string, IEnumerable<Dog>> query_GetDog = CompiledQuery.Compile<Entities, int, string, IEnumerable<Dog>>((ctx, age, name) => from dog in ctx.DogSet where dog.Age == age && dog.Name == name select dog); public List<Dog> GetSomeDogs( int age, string name ) { return query_GetDog(YourContext,age,name).ToList(); //<== change here } public void DataBindStuff() { List<Dog> dogs = GetSomeDogs(4,"Bud"); // but I want the dogs ordered by BirthDate gridView.DataSource = dogs.OrderBy( it => it.BirthDate ); } When you call ToList(), the query gets executed as per the compiled query and then, later, the OrderBy is executed against the objects in memory. It may be a little bit slower, but I'm not even sure. One sure thing is that you have no worries about mis-handling the ObjectQuery and invalidating the compiled query plan. Once again, that is not a blanket statement. ToList() is a defensive programming trick, but if you have a valid reason not to use ToList(), go ahead. There are many cases in which you would want to refine the query before executing it. Performance What is the performance impact of compiling a query? It can actually be fairly large. A rule of thumb is that compiling and caching the query for reuse takes at least double the time of simply executing it without caching. For complex queries (read inherirante), I have seen upwards to 10 seconds. So, the first time a pre-compiled query gets called, you get a performance hit. After that first hit, performance is noticeably better than the same non-pre-compiled query. Practically the same as Linq2Sql When you load a page with pre-compiled queries the first time you will get a hit. It will load in maybe 5-15 seconds (obviously more than one pre-compiled queries will end up being called), while subsequent loads will take less than 300ms. Dramatic difference, and it is up to you to decide if it is ok for your first user to take a hit or you want a script to call your pages to force a compilation of the queries. Can this query be cached? { Dog dog = from dog in YourContext.DogSet where dog.ID == id select dog; } No, ad-hoc Linq queries are not cached and you will incur the cost of generating the tree every single time you call it. Parametrized Queries Most search capabilities involve heavily parametrized queries. There are even libraries available that will let you build a parametrized query out of lamba expressions. The problem is that you cannot use pre-compiled queries with those. One way around that is to map out all the possible criteria in the query and flag which one you want to use: public struct MyParams { public string name; public bool checkName; public int age; public bool checkAge; } static readonly Func<Entities, MyParams, IEnumerable<Dog>> query_GetDog = CompiledQuery.Compile<Entities, MyParams, IEnumerable<Dog>>((ctx, myParams) => from dog in ctx.DogSet where (myParams.checkAge == true && dog.Age == myParams.age) && (myParams.checkName == true && dog.Name == myParams.name ) select dog); protected List<Dog> GetSomeDogs() { MyParams myParams = new MyParams(); myParams.name = "Bud"; myParams.checkName = true; myParams.age = 0; myParams.checkAge = false; return query_GetDog(YourContext,myParams).ToList(); } The advantage here is that you get all the benifits of a pre-compiled quert. The disadvantages are that you most likely will end up with a where clause that is pretty difficult to maintain, that you will incur a bigger penalty for pre-compiling the query and that each query you run is not as efficient as it could be (particularly with joins thrown in). Another way is to build an EntitySQL query piece by piece, like we all did with SQL. protected List<Dod> GetSomeDogs( string name, int age) { string query = "select value dog from Entities.DogSet where 1 = 1 "; if( !String.IsNullOrEmpty(name) ) query = query + " and dog.Name == @Name "; if( age > 0 ) query = query + " and dog.Age == @Age "; ObjectQuery<Dog> oQuery = new ObjectQuery<Dog>( query, YourContext ); if( !String.IsNullOrEmpty(name) ) oQuery.Parameters.Add( new ObjectParameter( "Name", name ) ); if( age > 0 ) oQuery.Parameters.Add( new ObjectParameter( "Age", age ) ); return oQuery.ToList(); } Here the problems are: - there is no syntax checking during compilation - each different combination of parameters generate a different query which will need to be pre-compiled when it is first run. In this case, there are only 4 different possible queries (no params, age-only, name-only and both params), but you can see that there can be way more with a normal world search. - Noone likes to concatenate strings! Another option is to query a large subset of the data and then narrow it down in memory. This is particularly useful if you are working with a definite subset of the data, like all the dogs in a city. You know there are a lot but you also know there aren't that many... so your CityDog search page can load all the dogs for the city in memory, which is a single pre-compiled query and then refine the results protected List<Dod> GetSomeDogs( string name, int age, string city) { string query = "select value dog from Entities.DogSet where dog.Owner.Address.City == @City "; ObjectQuery<Dog> oQuery = new ObjectQuery<Dog>( query, YourContext ); oQuery.Parameters.Add( new ObjectParameter( "City", city ) ); List<Dog> dogs = oQuery.ToList(); if( !String.IsNullOrEmpty(name) ) dogs = dogs.Where( it => it.Name == name ); if( age > 0 ) dogs = dogs.Where( it => it.Age == age ); return dogs; } It is particularly useful when you start displaying all the data then allow for filtering. Problems: - Could lead to serious data transfer if you are not careful about your subset. - You can only filter on the data that you returned. It means that if you don't return the Dog.Owner association, you will not be able to filter on the Dog.Owner.Name So what is the best solution? There isn't any. You need to pick the solution that works best for you and your problem: - Use lambda-based query building when you don't care about pre-compiling your queries. - Use fully-defined pre-compiled Linq query when your object structure is not too complex. - Use EntitySQL/string concatenation when the structure could be complex and when the possible number of different resulting queries are small (which means fewer pre-compilation hits). - Use in-memory filtering when you are working with a smallish subset of the data or when you had to fetch all of the data on the data at first anyway (if the performance is fine with all the data, then filtering in memory will not cause any time to be spent in the db). Singleton access The best way to deal with your context and entities accross all your pages is to use the singleton pattern: public sealed class YourContext { private const string instanceKey = "On3GoModelKey"; YourContext(){} public static YourEntities Instance { get { HttpContext context = HttpContext.Current; if( context == null ) return Nested.instance; if (context.Items[instanceKey] == null) { On3GoEntities entity = new On3GoEntities(); context.Items[instanceKey] = entity; } return (YourEntities)context.Items[instanceKey]; } } class Nested { // Explicit static constructor to tell C# compiler // not to mark type as beforefieldinit static Nested() { } internal static readonly YourEntities instance = new YourEntities(); } } NoTracking, is it worth it? When executing a query, you can tell the framework to track the objects it will return or not. What does it mean? With tracking enabled (the default option), the framework will track what is going on with the object (has it been modified? Created? Deleted?) and will also link objects together, when further queries are made from the database, which is what is of interest here. For example, lets assume that Dog with ID == 2 has an owner which ID == 10. Dog dog = (from dog in YourContext.DogSet where dog.ID == 2 select dog).FirstOrDefault(); //dog.OwnerReference.IsLoaded == false; Person owner = (from o in YourContext.PersonSet where o.ID == 10 select dog).FirstOrDefault(); //dog.OwnerReference.IsLoaded == true; If we were to do the same with no tracking, the result would be different. ObjectQuery<Dog> oDogQuery = (ObjectQuery<Dog>) (from dog in YourContext.DogSet where dog.ID == 2 select dog); oDogQuery.MergeOption = MergeOption.NoTracking; Dog dog = oDogQuery.FirstOrDefault(); //dog.OwnerReference.IsLoaded == false; ObjectQuery<Person> oPersonQuery = (ObjectQuery<Person>) (from o in YourContext.PersonSet where o.ID == 10 select o); oPersonQuery.MergeOption = MergeOption.NoTracking; Owner owner = oPersonQuery.FirstOrDefault(); //dog.OwnerReference.IsLoaded == false; Tracking is very useful and in a perfect world without performance issue, it would always be on. But in this world, there is a price for it, in terms of performance. So, should you use NoTracking to speed things up? It depends on what you are planning to use the data for. Is there any chance that the data your query with NoTracking can be used to make update/insert/delete in the database? If so, don't use NoTracking because associations are not tracked and will causes exceptions to be thrown. In a page where there are absolutly no updates to the database, you can use NoTracking. Mixing tracking and NoTracking is possible, but it requires you to be extra careful with updates/inserts/deletes. The problem is that if you mix then you risk having the framework trying to Attach() a NoTracking object to the context where another copy of the same object exist with tracking on. Basicly, what I am saying is that Dog dog1 = (from dog in YourContext.DogSet where dog.ID == 2).FirstOrDefault(); ObjectQuery<Dog> oDogQuery = (ObjectQuery<Dog>) (from dog in YourContext.DogSet where dog.ID == 2 select dog); oDogQuery.MergeOption = MergeOption.NoTracking; Dog dog2 = oDogQuery.FirstOrDefault(); dog1 and dog2 are 2 different objects, one tracked and one not. Using the detached object in an update/insert will force an Attach() that will say "Wait a minute, I do already have an object here with the same database key. Fail". And when you Attach() one object, all of its hierarchy gets attached as well, causing problems everywhere. Be extra careful. How much faster is it with NoTracking It depends on the queries. Some are much more succeptible to tracking than other. I don't have a fast an easy rule for it, but it helps. So I should use NoTracking everywhere then? Not exactly. There are some advantages to tracking object. The first one is that the object is cached, so subsequent call for that object will not hit the database. That cache is only valid for the lifetime of the YourEntities object, which, if you use the singleton code above, is the same as the page lifetime. One page request == one YourEntity object. So for multiple calls for the same object, it will load only once per page request. (Other caching mechanism could extend that). What happens when you are using NoTracking and try to load the same object multiple times? The database will be queried each time, so there is an impact there. How often do/should you call for the same object during a single page request? As little as possible of course, but it does happens. Also remember the piece above about having the associations connected automatically for your? You don't have that with NoTracking, so if you load your data in multiple batches, you will not have a link to between them: ObjectQuery<Dog> oDogQuery = (ObjectQuery<Dog>)(from dog in YourContext.DogSet select dog); oDogQuery.MergeOption = MergeOption.NoTracking; List<Dog> dogs = oDogQuery.ToList(); ObjectQuery<Person> oPersonQuery = (ObjectQuery<Person>)(from o in YourContext.PersonSet select o); oPersonQuery.MergeOption = MergeOption.NoTracking; List<Person> owners = oPersonQuery.ToList(); In this case, no dog will have its .Owner property set. Some things to keep in mind when you are trying to optimize the performance. No lazy loading, what am I to do? This can be seen as a blessing in disguise. Of course it is annoying to load everything manually. However, it decreases the number of calls to the db and forces you to think about when you should load data. The more you can load in one database call the better. That was always true, but it is enforced now with this 'feature' of EF. Of course, you can call if( !ObjectReference.IsLoaded ) ObjectReference.Load(); if you want to, but a better practice is to force the framework to load the objects you know you will need in one shot. This is where the discussion about parametrized Includes begins to make sense. Lets say you have you Dog object public class Dog { public Dog Get(int id) { return YourContext.DogSet.FirstOrDefault(it => it.ID == id ); } } This is the type of function you work with all the time. It gets called from all over the place and once you have that Dog object, you will do very different things to it in different functions. First, it should be pre-compiled, because you will call that very often. Second, each different pages will want to have access to a different subset of the Dog data. Some will want the Owner, some the FavoriteToy, etc. Of course, you could call Load() for each reference you need anytime you need one. But that will generate a call to the database each time. Bad idea. So instead, each page will ask for the data it wants to see when it first request for the Dog object: static public Dog Get(int id) { return GetDog(entity,"");} static public Dog Get(int id, string includePath) { string query = "select value o " + " from YourEntities.DogSet as o " +

Search Results

Search found 3492 results on 140 pages for 'subject'.

Page 140/140 | < Previous Page | 136 137 138 139 140

- by ybbest

- by user12620111

- by Decmac04

- by fmz

- by Shaul

- by Sean Ochoa

- by lock

- by Graeme

- by Decmac04

- by louism

- by Decmanc04

- by Noah Goodrich

- by Brad

- by Brad

- by JohnHankinson

- by jgray

- by AD

< Previous Page | 136 137 138 139 140