Search Results

Search found 1848 results on 74 pages for 'printf'.

Page 74/74 | < Previous Page | 70 71 72 73 74 

• why am i getting error in this switch statement written in c

  - by mekasperasky

  I have a character array b which stores different identifiers in different iterations . I have to compare b with various identifiers of the programming language C and print it into a file . When i do it using the following switch statement it gives me errors b[i]='\0'; switch(b[i]) { case "if":fprintf(fp2,"if ----> IDENTIFIER \n"); case "then":fprintf(fp2,"then ----> IDENTIFIER \n"); case "else":fprintf(fp2,"else ----> IDENTIFIER \n"); case "switch":fprintf(fp2,"switch ----> IDENTIFIER \n"); case 'printf':fprintf(fp2,"prtintf ----> IDENTIFIER \n"); case 'scanf':fprintf(fp2,"else ----> IDENTIFIER \n"); case 'NULL':fprintf(fp2,"NULL ----> IDENTIFIER \n"); case 'int':fprintf(fp2,"INT ----> IDENTIFIER \n"); case 'char':fprintf(fp2,"char ----> IDENTIFIER \n"); case 'float':fprintf(fp2,"float ----> IDENTIFIER \n"); case 'long':fprintf(fp2,"long ----> IDENTIFIER \n"); case 'double':fprintf(fp2,"double ----> IDENTIFIER \n"); case 'char':fprintf(fp2,"char ----> IDENTIFIER \n"); case 'const':fprintf(fp2,"const ----> IDENTIFIER \n"); case 'continue':fprintf(fp2,"continue ----> IDENTIFIER \n"); case 'break':fprintf(fp2,"long ----> IDENTIFIER \n"); case 'for':fprintf(fp2,"long ----> IDENTIFIER \n"); case 'size of':fprintf(fp2,"size of ----> IDENTIFIER \n"); case 'register':fprintf(fp2,"register ----> IDENTIFIER \n"); case 'short':fprintf(fp2,"short ----> IDENTIFIER \n"); case 'auto':fprintf(fp2,"auto ----> IDENTIFIER \n"); case 'while':fprintf(fp2,"while ----> IDENTIFIER \n"); case 'do':fprintf(fp2,"do ----> IDENTIFIER \n"); case 'case':fprintf(fp2,"case ----> IDENTIFIER \n"); } the error being lex.c:94:13: warning: character constant too long for its type lex.c:95:13: warning: character constant too long for its type lex.c:96:13: warning: multi-character character constant lex.c:97:13: warning: multi-character character constant lex.c:98:13: warning: multi-character character constant lex.c:99:13: warning: character constant too long for its type lex.c:100:13: warning: multi-character character constant lex.c:101:13: warning: character constant too long for its type lex.c:102:13: warning: multi-character character constant lex.c:103:13: warning: character constant too long for its type lex.c:104:13: warning: character constant too long for its type lex.c:105:13: warning: character constant too long for its type lex.c:106:13: warning: multi-character character constant lex.c:107:13: warning: character constant too long for its type lex.c:108:13: warning: character constant too long for its type lex.c:109:13: warning: character constant too long for its type lex.c:110:12: warning: multi-character character constant lex.c:111:13: warning: character constant too long for its type lex.c:112:13: warning: multi-character character constant lex.c:113:13: warning: multi-character character constant lex.c: In function ‘int main()’: lex.c:90: error: case label does not reduce to an integer constant lex.c:91: error: case label does not reduce to an integer constant lex.c:92: error: case label does not reduce to an integer constant lex.c:93: error: case label does not reduce to an integer constant lex.c:94: warning: overflow in implicit constant conversion lex.c:95: warning: overflow in implicit constant conversion lex.c:95: error: duplicate case value lex.c:94: error: previously used here lex.c:96: warning: overflow in implicit constant conversion lex.c:97: warning: overflow in implicit constant conversion lex.c:98: warning: overflow in implicit constant conversion lex.c:99: warning: overflow in implicit constant conversion lex.c:99: error: duplicate case value lex.c:97: error: previously used here lex.c:100: warning: overflow in implicit constant conversion lex.c:101: warning: overflow in implicit constant conversion lex.c:102: warning: overflow in implicit constant conversion lex.c:102: error: duplicate case value lex.c:98: error: previously used here lex.c:103: warning: overflow in implicit constant conversion lex.c:103: error: duplicate case value lex.c:97: error: previously used here lex.c:104: warning: overflow in implicit constant conversion lex.c:104: error: duplicate case value lex.c:101: error: previously used here lex.c:105: warning: overflow in implicit constant conversion lex.c:106: warning: overflow in implicit constant conversion lex.c:106: error: duplicate case value lex.c:98: error: previously used here lex.c:107: warning: overflow in implicit constant conversion lex.c:107: error: duplicate case value lex.c:94: error: previously used here lex.c:108: warning: overflow in implicit constant conversion lex.c:108: error: duplicate case value lex.c:98: error: previously used here lex.c:109: warning: overflow in implicit constant conversion lex.c:109: error: duplicate case value lex.c:97: error: previously used here lex.c:110: warning: overflow in implicit constant conversion lex.c:111: warning: overflow in implicit constant conversion lex.c:111: error: duplicate case value lex.c:101: error: previously used here lex.c:112: warning: overflow in implicit constant conversion lex.c:112: error: duplicate case value lex.c:110: error: previously used here lex.c:113: warning: overflow in implicit constant conversion lex.c:113: error: duplicate case value lex.c:101: error: previously used here

  Read the article

• Reference a GNU C (POSIX) DLL built in GCC against Cygwin, from C#/NET

  - by Dale Halliwell

  Here is what I want: I have a huge legacy C/C++ codebase written for POSIX, including some very POSIX specific stuff like pthreads. This can be compiled on Cygwin/GCC and run as an executable under Windows with the Cygwin DLL. What I would like to do is build the codebase itself into a Windows DLL that I can then reference from C# and write a wrapper around it to access some parts of it programatically. I have tried this approach with the very simple "hello world" example at http://www.cygwin.com/cygwin-ug-net/dll.html and it doesn't seem to work. #include <stdio.h> extern "C" __declspec(dllexport) int hello(); int hello() { printf ("Hello World!\n"); return 42; } I believe I should be able to reference a DLL built with the above code in C# using something like: [DllImport("kernel32.dll")] public static extern IntPtr LoadLibrary(string dllToLoad); [DllImport("kernel32.dll")] public static extern IntPtr GetProcAddress(IntPtr hModule, string procedureName); [DllImport("kernel32.dll")] public static extern bool FreeLibrary(IntPtr hModule); [UnmanagedFunctionPointer(CallingConvention.Cdecl)] private delegate int hello(); static void Main(string[] args) { var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "helloworld.dll"); IntPtr pDll = LoadLibrary(path); IntPtr pAddressOfFunctionToCall = GetProcAddress(pDll, "hello"); hello hello = (hello)Marshal.GetDelegateForFunctionPointer( pAddressOfFunctionToCall, typeof(hello)); int theResult = hello(); Console.WriteLine(theResult.ToString()); bool result = FreeLibrary(pDll); Console.ReadKey(); } But this approach doesn't seem to work. LoadLibrary returns null. It can find the DLL (helloworld.dll), it is just like it can't load it or find the exported function. I am sure that if I get this basic case working I can reference the rest of my codebase in this way. Any suggestions or pointers, or does anyone know if what I want is even possible? Thanks. Edit: Examined my DLL with Dependency Walker (great tool, thanks) and it seems to export the function correctly. Question: should I be referencing it as the function name Dependency Walker seems to find (_Z5hellov)? Edit2: Just to show you I have tried it, linking directly to the dll at relative or absolute path (i.e. not using LoadLibrary): [DllImport(@"C:\.....\helloworld.dll")] public static extern int hello(); static void Main(string[] args) { int theResult = hello(); Console.WriteLine(theResult.ToString()); Console.ReadKey(); } This fails with: "Unable to load DLL 'C:.....\helloworld.dll': Invalid access to memory location. (Exception from HRESULT: 0x800703E6) *Edit 3: * Oleg has suggested running dumpbin.exe on my dll, this is the output: Dump of file helloworld.dll File Type: DLL Section contains the following exports for helloworld.dll 00000000 characteristics 4BD5037F time date stamp Mon Apr 26 15:07:43 2010 0.00 version 1 ordinal base 1 number of functions 1 number of names ordinal hint RVA name 1 0 000010F0 hello Summary 1000 .bss 1000 .data 1000 .debug_abbrev 1000 .debug_info 1000 .debug_line 1000 .debug_pubnames 1000 .edata 1000 .eh_frame 1000 .idata 1000 .reloc 1000 .text Edit 4 Thanks everyone for the help, I managed to get it working. Oleg's answer gave me the information I needed to find out what I was doing wrong. There are 2 ways to do this. One is to build with the gcc -mno-cygwin compiler flag, which builds the dll without the cygwin dll, basically as if you had built it in MingW. Building it this way got my hello world example working! However, MingW doesn't have all the libraries that cygwin has in the installer, so if your POSIX code has dependencies on these libraries (mine had heaps) you can't do this way. And if your POSIX code didn't have those dependencies, why not just build for Win32 from the beginning. So that's not much help unless you want to spend time setting up MingW properly. The other option is to build with the Cygwin DLL. The Cygwin DLL needs an initialization function init() to be called before it can be used. This is why my code wasn't working before. The code below loads and runs my hello world example. //[DllImport(@"hello.dll", EntryPoint = "#1",SetLastError = true)] //static extern int helloworld(); //don't do this! cygwin needs to be init first [DllImport("kernel32", CharSet = CharSet.Ansi, ExactSpelling = true, SetLastError = true)] static extern IntPtr GetProcAddress(IntPtr hModule, string procName); [DllImport("kernel32", SetLastError = true)] static extern IntPtr LoadLibrary(string lpFileName); public delegate int MyFunction(); static void Main(string[] args) { //load cygwin dll IntPtr pcygwin = LoadLibrary("cygwin1.dll"); IntPtr pcyginit = GetProcAddress(pcygwin, "cygwin_dll_init"); Action init = (Action)Marshal.GetDelegateForFunctionPointer(pcyginit, typeof(Action)); init(); IntPtr phello = LoadLibrary("hello.dll"); IntPtr pfn = GetProcAddress(phello, "helloworld"); MyFunction helloworld = (MyFunction)Marshal.GetDelegateForFunctionPointer(pfn, typeof(MyFunction)); Console.WriteLine(helloworld()); Console.ReadKey(); } Thanks to everyone that answered~~

  Read the article

• java - register problem

  - by Jake

  Hi! When i try to register a person with the name Eric for example, and then again registrating Eric it works. This should not happen with the code i have. Eric should not be registrated if theres already an Eric in the list. Here is my full code: import java.util.*; import se.lth.cs.pt.io.*; class Person { private String name; private String nbr; public Person (String name, String nbr) { this.name = name; this.nbr = nbr; } public String getName() { return name; } public String getNumber() { return nbr; } public String toString() { return name + " : " + nbr; } } class Register { private List<Person> personer; public Register() { personer = new ArrayList<Person>(); } // boolean remove(String name) { // } private Person findName(String name) { for (Person person : personer) { if (person.getName() == name) { return person; } } return null; } private boolean containsName(String name) { return findName(name) != null; } public boolean insert(String name, String nbr) { if (containsName(name)) { return false; } Person person = new Person(name, nbr); personer.add(person); Collections.sort(personer, new A()); return true; } //List<Person> findByPartOfName(String partOfName) { //} //List<Person> findByNumber(String nbr) { //} public List<Person> findAll() { List<Person> copy = new ArrayList<Person>(); for (Person person : personer) { copy.add(person); } return copy; } public void printList(List<Person> personer) { for (Person person : personer) { System.out.println(person.toString()); } } } class A implements Comparator < Person > { @Override public int compare(Person o1, Person o2) { if(o1.getName() != null && o2.getName() != null){ return o1.getName().compareTo(o2.getName()); } return 0; } } class TestScript { public static void main(String[] args) { new TestScript().run(); } void test(String msg, boolean status) { if (status) { System.out.println(msg + " -- ok"); } else { System.out.printf("==== FEL: %s ====\n", msg); } } void run() { Register register = new Register(); System.out.println("Vad vill du göra:"); System.out.println("1. Lägg in ny person."); System.out.println("2. Tag bort person."); System.out.println("3. Sök på del av namn."); System.out.println("4. Se vem som har givet nummer."); System.out.println("5. Skriv ut alla personer."); System.out.println("0. Avsluta."); int cmd = Keyboard.nextInt("Ange kommando (0-5): "); if (cmd == 0 ) { } else if (cmd == 1) { String name = Keyboard.nextLine("Namn: "); String nbr = Keyboard.nextLine("Nummer: "); System.out.println("\n"); String inlagd = "OK - " + name + " är nu inlagd."; String ejinlagd = name + " är redan inlagd."; test("Skapar nytt konto", register.insert(name, nbr) == true); System.out.println("\n"); } else if (cmd == 2) { } else if (cmd == 3) { } else if (cmd == 4) { } else if (cmd == 5) { System.out.println("\n"); register.printList(register.findAll()); System.out.println("\n"); } else { System.out.println("Inget giltigt kommando!"); System.out.println("\n"); } } }

  Read the article

• C question: Padding bits in unsigned integers and bitwise operations (C89)

  - by Anonymous Question Guy

  I have a lot of code that performs bitwise operations on unsigned integers. I wrote my code with the assumption that those operations were on integers of fixed width without any padding bits. For example an array of 32 bit unsigned integers of which all 32 bits available for each integer. I'm looking to make my code more portable and I'm focused on making sure I'm C89 compliant (in this case). One of the issues that I've come across is possible padded integers. Take this extreme example, taken from the GMP manual: However on Cray vector systems it may be noted that short and int are always stored in 8 bytes (and with sizeof indicating that) but use only 32 or 46 bits. The nails feature can account for this, by passing for instance 8*sizeof(int)-INT_BIT. I've also read about this type of padding in other places. I actually read of a post on SO last night (forgive me, I don't have the link and I'm going to cite something similar from memory) where if you have, say, a double with 60 usable bits the other 4 could be used for padding and those padding bits could serve some internal purpose so they cannot be modified. So let's say for example my code is compiled on a platform where an unsigned int type is sized at 4 bytes, each byte being 8 bits, however the most significant 2 bits are padding bits. Would UINT_MAX in that case be 0x3FFFFFFF (1073741823) ? #include <stdio.h> #include <stdlib.h> /* padding bits represented by underscores */ int main( int argc, char **argv ) { unsigned int a = 0x2AAAAAAA; /* __101010101010101010101010101010 */ unsigned int b = 0x15555555; /* __010101010101010101010101010101 */ unsigned int c = a ^ b; /* ?? __111111111111111111111111111111 */ unsigned int d = c << 5; /* ?? __111111111111111111111111100000 */ unsigned int e = d >> 5; /* ?? __000001111111111111111111111111 */ printf( "a: %X\nb: %X\nc: %X\nd: %X\ne: %X\n", a, b, c, d, e ); return 0; } is it safe to XOR two integers with padding bits? wouldn't I XOR whatever the padding bits are? I can't find this behavior covered in C89. furthermore is the c var guaranteed to be 0x3FFFFFFF or if for example the two padding bits were both on in a or b would c be 0xFFFFFFFF ? same question with d and e. am i manipulating the padding bits by shifting? I would expect to see this below, assuming 32 bits with the 2 most significant bits used for padding, but I want to know if something like this is guaranteed: a: 2AAAAAAA b: 15555555 c: 3FFFFFFF d: 3FFFFFE0 e: 01FFFFFF Also are padding bits always the most significant bits or could they be the least significant bits? Thanks guys EDIT 12/19/2010 5PM EST: Christoph has answered my question. Thanks! I had also asked (above) whether padding bits are always the most significant bits. This is cited in the rationale for the C99 standard, and the answer is no. I am playing it safe and assuming the same for C89. Here is specifically what the C99 rationale says for §6.2.6.2 (Representation of Integer Types): Padding bits are user-accessible in an unsigned integer type. For example, suppose a machine uses a pair of 16-bit shorts (each with its own sign bit) to make up a 32-bit int and the sign bit of the lower short is ignored when used in this 32-bit int. Then, as a 32-bit signed int, there is a padding bit (in the middle of the 32 bits) that is ignored in determining the value of the 32-bit signed int. But, if this 32-bit item is treated as a 32-bit unsigned int, then that padding bit is visible to the user’s program. The C committee was told that there is a machine that works this way, and that is one reason that padding bits were added to C99. Footnotes 44 and 45 mention that parity bits might be padding bits. The committee does not know of any machines with user-accessible parity bits within an integer. Therefore, the committee is not aware of any machines that treat parity bits as padding bits. EDIT 12/28/2010 3PM EST: I found an interesting discussion on comp.lang.c from a few months ago. Bitwise Operator Effects on Padding Bits (VelocityReviews reader) Bitwise Operator Effects on Padding Bits (Google Groups alternate link) One point made by Dietmar which I found interesting: Let's note that padding bits are not necessary for the existence of trap representations; combinations of value bits which do not represent a value of the object type would also do.

  Read the article

• flex and bison: wrong output

  - by user2972227

  I am doing a homework using flex and bison to make a complex number calculator. But my program cannot give a correct output. .lex file: %option noyywrap %{ #include<stdio.h> #include<stdlib.h> #include "complex_cal.h" #define YYSTYPE complex #include "complex_cal.tab.h" void RmWs(char* str); %} /* Add your Flex definitions here */ /* Some definitions are already provided to you*/ ws [ \t]+ digits [0-9] number (0|[1-9]+{digits}*)\.?{digits}* im [i] complexnum {ws}*[-]*{ws}*{number}{ws}*[+|-]{ws}*{number}{ws}*{im}{ws}* op [-+*/()] %% {complexnum} {RmWs(yytext); sscanf(yytext,"%lf %lf",&(yylval.real),&(yylval.img)); return CNUMBER;} {ws} /**/ {op} return *yytext; %% /* function provided to student to remove */ /* all the whitespaces from a string. */ void RmWs(char* str){ int i=0,j=0; char temp[strlen(str)+1]; strcpy(temp,str); while (temp[i]!='\0'){ while (temp[i]==' '){i++;} str[j]=temp[i]; i++; j++; } str[j]='\0'; } .y file: %{ #include <stdio.h> #include <stdlib.h> #include "complex_cal.h" /* prototypes of the provided functions */ complex complex_add (complex, complex); complex complex_sub (complex, complex); complex complex_mul (complex, complex); complex complex_div (complex, complex); /* prototypes of the provided functions */ int yylex(void); int yyerror(const char*); %} %token CNUMBER %left '+' '-' %left '*' '/' %nonassoc '(' ')' %% /* start: Add your grammar rules and actions here */ complexexp: complexexp '+' complexexpmultidiv {$$=complex_add($1, $3);} | complexexp '-' complexexpmultidiv {$$=complex_sub($1, $3);} | complexexpmultidiv {$$.real=$1.real;$$.img=$1.img;} ; complexexpmultidiv: complexexpmultidiv '*' complexsimple {$$=complex_mul($1, $3);} | complexexpmultidiv '/' complexsimple {$$=complex_div($1, $3);} | complexsimple {$$.real=$1.real;$$.img=$1.img;} ; complexsimple: '(' complexexp ')' {$$.real=$2.real;$$.img=$2.img;} | '(' CNUMBER ')' {$$.real=$2.real;$$.img=$2.img;} ; /* end: Add your grammar rules and actions here */ %% int main(){ return yyparse(); } int yyerror(const char* s){ printf("%s\n", s); return 0; } /* function provided to do complex addition */ /* input : complex numbers c1, c2 */ /* output: nothing */ /* side effect : none */ /* return value: result of addition in c3 */ complex complex_add (complex c1, complex c2){ /* c1 + c2 */ complex c3; c3.real = c1.real + c2.real; c3.img = c1.img + c2.img; return c3; } /* function provided to do complex subtraction */ /* input : complex numbers c1, c2 */ /* output: nothing */ /* side effect : none */ /* return value: result of subtraction in c3 */ complex complex_sub (complex c1, complex c2){ /* c1 - c2 */ complex c3; c3.real = c1.real - c2.real; c3.img = c1.img - c2.img; return c3; } /* function provided to do complex multiplication */ /* input : complex numbers c1, c2 */ /* output: nothing */ /* side effect : none */ /* return value: result of multiplication in c3 */ complex complex_mul (complex c1, complex c2){ /* c1 * c2 */ complex c3; c3.real = c1.real*c2.real - c1.img*c2.img; c3.img = c1.img*c2.real + c1.real*c2.img; return c3; } /* function provided to do complex division */ /* input : complex numbers c1, c2 */ /* output: nothing */ /* side effect : none */ /* return value: result of c1/c2 in c3 */ complex complex_div (complex c1, complex c2){ /* c1 / c2 (i.e. c1 divided by c2 ) */ complex c3; double d; /*divisor calculation using the conjugate of c2*/ d = c2.real*c2.real + c2.img*c2.img; c3.real = (c1.real*c2.real + c1.img*c2.img)/d; c3.img = (c1.img*c2.real - c1.real*c2.img)/d; return c3; } .h file: #include <string.h> /* struct for holding a complex number */ typedef struct { double real; double img; } complex; /* define the return type of FLEX */ #define YYSTYPE complex Script for compiling the file: bison -d -v complex_cal.y flex -ocomplex_cal.lex.yy.c complex_cal.lex gcc -o complex_cal complex_cal.lex.yy.c complex_cal.tab.c ./complex_cal Some correct sample run of the program: input:(5+6i)*(6+1i) output:24.000000+41.000000i input:(7+8i)/(-3-4i)*(5+7i) output:-11.720000-14.040000i input:(7+8i)/((-3-4i)*(5+7i)) output:-0.128108+0.211351i But when I run this program, the program only give an output which is identical to my input. For example, when I input (5+6i)(6+1i), it just gives (5+6i)(6+1i). Even if I input any other things, for example, input "abc" it just gives "abc" and is not syntax error. I don't know where the problem is and I hope to know how to solve it.

  Read the article

• [ebp + 6] instead of +8 in a JIT compiler

  - by David Titarenco

  I'm implementing a simplistic JIT compiler in a VM I'm writing for fun (mostly to learn more about language design) and I'm getting some weird behavior, maybe someone can tell me why. First I define a JIT "prototype" both for C and C++: #ifdef __cplusplus typedef void* (*_JIT_METHOD) (...); #else typedef (*_JIT_METHOD) (); #endif I have a compile() function that will compile stuff into ASM and stick it somewhere in memory: void* compile (void* something) { // grab some memory unsigned char* buffer = (unsigned char*) malloc (1024); // xor eax, eax // inc eax // inc eax // inc eax // ret -> eax should be 3 /* WORKS! buffer[0] = 0x67; buffer[1] = 0x31; buffer[2] = 0xC0; buffer[3] = 0x67; buffer[4] = 0x40; buffer[5] = 0x67; buffer[6] = 0x40; buffer[7] = 0x67; buffer[8] = 0x40; buffer[9] = 0xC3; */ // xor eax, eax // mov eax, 9 // ret 4 -> eax should be 9 /* WORKS! buffer[0] = 0x67; buffer[1] = 0x31; buffer[2] = 0xC0; buffer[3] = 0x67; buffer[4] = 0xB8; buffer[5] = 0x09; buffer[6] = 0x00; buffer[7] = 0x00; buffer[8] = 0x00; buffer[9] = 0xC3; */ // push ebp // mov ebp, esp // mov eax, [ebp + 6] ; wtf? shouldn't this be [ebp + 8]!? // mov esp, ebp // pop ebp // ret -> eax should be the first value sent to the function /* WORKS! */ buffer[0] = 0x66; buffer[1] = 0x55; buffer[2] = 0x66; buffer[3] = 0x89; buffer[4] = 0xE5; buffer[5] = 0x66; buffer[6] = 0x66; buffer[7] = 0x8B; buffer[8] = 0x45; buffer[9] = 0x06; buffer[10] = 0x66; buffer[11] = 0x89; buffer[12] = 0xEC; buffer[13] = 0x66; buffer[14] = 0x5D; buffer[15] = 0xC3; // mov eax, 5 // add eax, ecx // ret -> eax should be 50 /* WORKS! buffer[0] = 0x67; buffer[1] = 0xB8; buffer[2] = 0x05; buffer[3] = 0x00; buffer[4] = 0x00; buffer[5] = 0x00; buffer[6] = 0x66; buffer[7] = 0x01; buffer[8] = 0xC8; buffer[9] = 0xC3; */ return buffer; } And finally I have the main chunk of the program: void main (int argc, char **args) { DWORD oldProtect = (DWORD) NULL; int i = 667, j = 1, k = 5, l = 0; // generate some arbitrary function _JIT_METHOD someFunc = (_JIT_METHOD) compile(NULL); // windows only #if defined _WIN64 || defined _WIN32 // set memory permissions and flush CPU code cache VirtualProtect(someFunc,1024,PAGE_EXECUTE_READWRITE, &oldProtect); FlushInstructionCache(GetCurrentProcess(), someFunc, 1024); #endif // this asm just for some debugging/testing purposes __asm mov ecx, i // run compiled function (from wherever *someFunc is pointing to) l = (int)someFunc(i, k); // did it work? printf("result: %d", l); free (someFunc); _getch(); } As you can see, the compile() function has a couple of tests I ran to make sure I get expected results, and pretty much everything works but I have a question... On most tutorials or documentation resources, to get the first value of a function passed (in the case of ints) you do [ebp+8], the second [ebp+12] and so forth. For some reason, I have to do [ebp+6] then [ebp+10] and so forth. Could anyone tell me why?

  Read the article

• Pointers to Derived Class Objects Losing vfptr

  - by duckworthd

  To begin, I am trying to write a run-of-the-mill, simple Ray Tracer. In my Ray Tracer, I have multiple types of geometries in the world, all derived from a base class called "SceneObject". I've included the header for it here. /** Interface for all objects that will appear in a scene */ class SceneObject { public: mat4 M, M_inv; Color c; SceneObject(); ~SceneObject(); /** The transformation matrix to be applied to all points of this object. Identity leaves the object in world frame. */ void setMatrix(mat4 M); void setMatrix(MatrixStack mStack); void getMatrix(mat4& M); /** The color of the object */ void setColor(Color c); void getColor(Color& c); /** Alter one portion of the color, leaving the rest as they were. */ void setDiffuse(vec3 rgb); void setSpecular(vec3 rgb); void setEmission(vec3 rgb); void setAmbient(vec3 rgb); void setShininess(double s); /** Fills 'inter' with information regarding an intersection between this object and 'ray'. Ray should be in world frame. */ virtual void intersect(Intersection& inter, Ray ray) = 0; /** Returns a copy of this SceneObject */ virtual SceneObject* clone() = 0; /** Print information regarding this SceneObject for debugging */ virtual void print() = 0; }; As you can see, I've included a couple virtual functions to be implemented elsewhere. In this case, I have only two derived class -- Sphere and Triangle, both of which implement the missing member functions. Finally, I have a Parser class, which is full of static methods that do the actual "Ray Tracing" part. Here's a couple snippets for relevant portions void Parser::trace(Camera cam, Scene scene, string outputFile, int maxDepth) { int width = cam.getNumXPixels(); int height = cam.getNumYPixels(); vector<vector<vec3>> colors; colors.clear(); for (int i = 0; i< width; i++) { vector<vec3> ys; for (int j = 0; j<height; j++) { Intersection intrsct; Ray ray; cam.getRay(ray, i, j); vec3 color; printf("Obtaining color for Ray[%d,%d]\n", i,j); getColor(color, scene, ray, maxDepth); ys.push_back(color); } colors.push_back(ys); } printImage(colors, width, height, outputFile); } void Parser::getColor(vec3& color, Scene scene, Ray ray, int numBounces) { Intersection inter; scene.intersect(inter,ray); if(inter.isIntersecting()){ Color c; inter.getColor(c); c.getAmbient(color); } else { color = vec3(0,0,0); } } Right now, I've forgone the true Ray Tracing part and instead simply return the color of the first object hit, if any. As you have no doubt noticed, the only way the computer knows that a ray has intersected an object is through Scene.intersect(), which I also include. void Scene::intersect(Intersection& i, Ray r) { Intersection result; result.setDistance(numeric_limits<double>::infinity()); result.setIsIntersecting(false); double oldDist; result.getDistance(oldDist); /* Cycle through all objects, making result the closest one */ for(int ind=0; ind<objects.size(); ind++){ SceneObject* thisObj = objects[ind]; Intersection betterIntersect; thisObj->intersect(betterIntersect, r); double newDist; betterIntersect.getDistance(newDist); if (newDist < oldDist){ result = betterIntersect; oldDist = newDist; } } i = result; } Alright, now for the problem. I begin by creating a scene and filling it with objects outside of the Parser::trace() method. Now for some odd reason, I cast Ray for i=j=0 and everything works wonderfully. However, by the time the second ray is cast all of the objects stored in my Scene no longer recognize their vfptr's! I stepped through the code with a debugger and found that the information to all the vfptr's are lost somewhere between the end of getColor() and the continuation of the loop. However, if I change the arguments of getColor() to use a Scene& instead of a Scene, then no loss occurs. What crazy voodoo is this?

  Read the article

• Array subscript is not an integer

  - by Dimitri

  Hello folks, following this previous question Malloc Memory Corruption in C, now i have another problem. I have the same code. Now I am trying to multiply the values contained in the arrays A * vc and store in res. Then A is set to zero and i do a second multiplication with res and vc and i store the values in A. (A and Q are square matrices and mc and vc are N lines two columns matrices or arrays). Here is my code : int jacobi_gpu(double A[], double Q[], double tol, long int dim){ int nrot, p, q, k, tid; double c, s; double *mc, *vc, *res; int i,kc; double vc1, vc2; mc = (double *)malloc(2 * dim * sizeof(double)); vc = (double *)malloc(2 * dim * sizeof(double)); vc = (double *)malloc(dim * dim * sizeof(double)); if( mc == NULL || vc == NULL){ fprintf(stderr, "pb allocation matricre\n"); exit(1); } nrot = 0; for(k = 0; k < dim - 1; k++){ eye(mc, dim); eye(vc, dim); for(tid = 0; tid < floor(dim /2); tid++){ p = (tid + k)%(dim - 1); if(tid != 0) q = (dim - tid + k - 1)%(dim - 1); else q = dim - 1; printf("p = %d | q = %d\n", p, q); if(fabs(A[p + q*dim]) > tol){ nrot++; symschur2(A, dim, p, q, &c, &s); mc[2*tid] = p; vc[2 * tid] = c; mc[2*tid + 1] = q; vc[2*tid + 1] = -s; mc[2*tid + 2*(dim - 2*tid) - 2] = p; vc[2*tid + 2*(dim - 2*tid) - 2 ] = s; mc[2*tid + 2*(dim - 2*tid) - 1] = q; vc[2 * tid + 2*(dim - 2*tid) - 1 ] = c; } } for( i = 0; i< dim; i++){ for(kc=0; kc < dim; kc++){ if( kc < floor(dim/2)) { vc1 = vc[2*kc + i*dim]; vc2 = vc[2*kc + 2*(dim - 2*kc) - 2]; }else { vc1 = vc[2*kc+1 + i*dim]; vc2 = vc[2*kc - 2*(dim - 2*kc) - 1]; } res[kc + i*dim] = A[mc[2*kc] + i*dim]*vc1 + A[mc[2*kc + 1] + i*dim]*vc2; } } zero(A, dim); for( i = 0; i< dim; i++){ for(kc=0; kc < dim; k++){ if( k < floor(dim/2)){ vc1 = vc[2*kc + i*dim]; vc2 = vc[2*kc + 2*(dim - 2*kc) - 2]; }else { vc1 = vc[2*kc+1 + i*dim]; vc2 = vc[2*kc - 2*(dim - 2*kc) - 1]; } A[kc + i*dim] = res[mc[2*kc] + i*dim]*vc1 + res[mc[2*kc + 1] + i*dim]*vc2; } } affiche(mc,dim,2,"Matrice creuse"); affiche(vc,dim,2,"Valeur creuse"); } free(mc); free(vc); free(res); return nrot; } When i try to compile, i have this error : jacobi_gpu.c: In function ‘jacobi_gpu’: jacobi_gpu.c:103: error: array subscript is not an integer jacobi_gpu.c:103: error: array subscript is not an integer jacobi_gpu.c:118: error: array subscript is not an integer jacobi_gpu.c:118: error: array subscript is not an integer make: *** [jacobi_gpu.o] Erreur 1 The corresponding lines are where I store the results in res and A : res[kc + i*dim] = A[mc[2*kc] + i*dim]*vc1 + A[mc[2*kc + 1] + i*dim]*vc2; and A[kc + i*dim] = res[mc[2*kc] + i*dim]*vc1 + res[mc[2*kc + 1] + i*dim]*vc2; Can someone explain me what is this error and how can i correct it? Thanks for your help. ;)

  Read the article

• problem in adding image to the UIButton.

  - by monish

  Hi friends, I got an another problem in my application and I am wasting so much of time on that. Does pls anyone can help with this problem. Actually I had an Event and I should give rating for that event for that I wrote the code as: In CellForRowAtIndexPath......I had the code as: - (UITableViewCell *)tableView:(UITableView *)tv cellForRowAtIndexPath:(NSIndexPath *)indexPath { UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:@"MasterViewIdentifier"]; //UITableViewCell *cell = nil; if (cell == nil) { cell = [[[UITableViewCell alloc] initWithFrame:CGRectZero reuseIdentifier:@"MasterViewIdentifier"] autorelease]; cell.selectionStyle = UITableViewCellSelectionStyleNone; UIView* elementView = [[UIView alloc] initWithFrame:CGRectMake(20,170,320,280)]; elementView.tag = 0; elementView.backgroundColor=[UIColor clearColor]; [cell.contentView addSubview:elementView]; [elementView release]; } UIView* elementView = [cell.contentView viewWithTag:0]; elementView.backgroundColor=[UIColor clearColor]; for(UIView* subView in elementView.subviews) { [subView removeFromSuperview]; } if(indexPath.section == 8) { UIImage *whiteImg = [UIImage imageNamed:@"white_star.png"] ; UIImage *yellowImg = [UIImage imageNamed:@"yellow_Star.png"] ; UIButton *button1 = [[UIButton alloc]initWithFrame:CGRectMake(159, 15, 25, 20)]; [button1 addTarget:self action:@selector(buttonAction:) forControlEvents:UIControlEventTouchUpInside]; button1.tag = 1; UIButton *button2 = [[UIButton alloc]initWithFrame:CGRectMake(185, 15, 25, 20)]; [button2 addTarget:self action:@selector(buttonAction:) forControlEvents:UIControlEventTouchUpInside]; button2.tag = 2; UIButton *button3 = [[UIButton alloc]initWithFrame:CGRectMake(211, 15, 25, 20)]; [button3 addTarget:self action:@selector(buttonAction:) forControlEvents:UIControlEventTouchUpInside]; button3.tag = 3; UIButton *button4 = [[UIButton alloc]initWithFrame:CGRectMake(237, 15, 25, 20)]; [button4 addTarget:self action:@selector(buttonAction:) forControlEvents:UIControlEventTouchUpInside]; button4.tag = 4; UIButton *button5 = [[UIButton alloc]initWithFrame:CGRectMake(263, 15, 25, 20)]; [button5 addTarget:self action:@selector(buttonAction:) forControlEvents:UIControlEventTouchUpInside]; button5.tag = 5; if(event.eventRatings == 1) { [button1 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button2 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button3 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button4 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button5 setBackgroundImage:whiteImg forState:UIControlStateNormal]; } else if(event.eventRatings == 2) { [button1 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button2 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button3 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button4 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button5 setBackgroundImage:whiteImg forState:UIControlStateNormal]; } else if(event.eventRatings == 3) { [button1 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button2 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button3 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button4 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button5 setBackgroundImage:whiteImg forState:UIControlStateNormal]; } else if(event.eventRatings == 4) { [button1 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button2 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button3 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button4 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button5 setBackgroundImage:whiteImg forState:UIControlStateNormal]; } else if(event.eventRatings == 5) { [button1 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button2 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button3 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button4 setBackgroundImage:yellowImg forState:UIControlStateNormal]; [button5 setBackgroundImage:yellowImg forState:UIControlStateNormal]; } else { [button1 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button2 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button3 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button4 setBackgroundImage:whiteImg forState:UIControlStateNormal]; [button5 setBackgroundImage:whiteImg forState:UIControlStateNormal]; } [elementView addSubview:button1]; [button1 release]; [elementView addSubview:button2]; [button2 release]; [elementView addSubview:button3]; [button3 release]; [elementView addSubview:button4]; [button4 release]; [elementView addSubview:button5]; [button5 release]; if(isRightButton == YES) { button1.enabled = NO; button2.enabled = NO; button3.enabled = NO; button4.enabled = NO; button5.enabled = NO; } else if(isRightButton == NO) { button1.enabled = YES; button2.enabled = YES; button3.enabled = YES; button4.enabled = YES; button5.enabled = YES; } [elementView addSubview:ratingsTitleLabel]; cell.accessoryType = UITableViewCellAccessoryNone; } return cell; } And the action of the button is written as: -(void)buttonAction:(id)sender { rating = [sender tag]; printf("\n Ratig Value inside Button Action~~~~~~~~~~~~~~~~%d",rating); event.eventRatings = rating; [tableView reloadData]; } When I build the application in simlator of 3.1.2 O.S its working fine by displaying the star images. My porblem is when I build it in 3.1.2 O.S Device the images are not displaying.I checked the code for casesensitivity in file name and its gud but Im not gettig the images to display. Guys help me to solve this. Thank you, Monish Kumar.

  Read the article

• C programming: hashtable insertion/search

  - by Ricardo Campos

  Hello i have a problem with my hash table its implemented like this: #define HT_SIZE 10 typedef struct _list_t_ { char key[20]; char string[20]; char prevValue[20]; struct _list_t_ *next; } list_t; typedef struct _hash_table_t_ { int size; /* the size of the table */ list_t ***table; /* first */ sem_t lock; } hash_table_t; I have a Linked list with 3 pointers because i want a hash table with several partitions (shards), here is my initialization of my Hash table: hash_table_t *create_hash_table(int NUM_SERVER_THREADS, int num_shards){ hash_table_t *new_table; int j,i; if (HT_SIZE<1) return NULL; /* invalid size for table */ /* Attempt to allocate memory for the hashtable structure */ new_table = (hash_table_t*)malloc(sizeof(hash_table_t)*HT_SIZE); /* Attempt to allocate memory for the table itself */ new_table->table = (list_t ***)calloc(1,sizeof(list_t **)); /* Initialize the elements of the table */ for(j=0; j<num_shards; j++){ new_table->table[j] = (list_t **)calloc(1,sizeof(list_t *)); for(i=0; i<HT_SIZE; i++){ new_table->table[j][i] = (list_t *)calloc(1,sizeof(list_t )); } } /* Set the table's size */ new_table->size = HT_SIZE; sem_init(&new_table->lock, 0, 1); return new_table; } Here is my search function to search in the hash table list_t *lookup_string(hash_table_t *hashtable, char *key, int shardId){ list_t *list ; int hashval = hash(key); /* Go to the correct list based on the hash value and see if key is * in the list. If it is, return return a pointer to the list element. * If it isn't, the item isn't in the table, so return NULL. */ sem_wait(&hashtable->lock); for(list = hashtable->table[shardId][hashval]; list != NULL; list =list->next) { if (strcmp(key, list->key) == 0){ sem_post(&hashtable->lock); return list; } } sem_post(&hashtable->lock); return NULL; } And my insert function: char *add_string(hash_table_t *hashtable, char *str,char *key, int shardId){ list_t *new_list; list_t *current_list; unsigned int hashval = hash(key); /*printf("|%d|%d|%s|\n",hashval,shardId,key);*/ /* Lock for concurrency */ sem_wait(&hashtable->lock); /* Attempt to allocate memory for list */ new_list = (list_t*)malloc(sizeof(list_t)); /* Does item already exist? */ sem_post(&hashtable->lock); current_list = lookup_string(hashtable, key,shardId); sem_wait(&hashtable->lock); /* item already exists, don't insert it again. */ if (current_list != NULL){ strcpy(new_list->prevValue,current_list->string); strcpy(new_list->string,str); strcpy(new_list->key,key); new_list->next = hashtable->table[shardId][hashval]; hashtable->table[shardId][hashval] = new_list; sem_post(&hashtable->lock); return new_list->prevValue; } /* Insert into list */ strcpy(new_list->string,str); strcpy(new_list->key,key); new_list->next = hashtable->table[shardId][hashval]; hashtable->table[shardId][hashval] = new_list; /* Unlock */ sem_post(&hashtable->lock); return new_list->prevValue; } My main class runs some of tests by executing the insertion / reading / delete from the elements of the hash table the problem is when i have more than 4 partitions/shards the tests stop at the first reading element saying it returned the wrong value NULL on the search function, when its less than 4 it runs perfectly well and passes all the tests. You can see my main.c in here if you want to give a look: http://hostcode.sourceforge.net/view/1105 My complete Hash table code: http://hostcode.sourceforge.net/view/1103 And other functions where hash table code is executed: .c file http://hostcode.sourceforge.net/view/1104 .h file http://hostcode.sourceforge.net/view/1106 Thank for you time, i appreciate any help you can give to me this is a college important project that I'm trying to solve and I'm stuck here for 2 days.

  Read the article

• Mistake in display and insert methods (double-ended queue)

  - by MANAL

  1) My problem when i make remove from right or left program will be remove true but when i call diplay method the content wrong like this I insert 12 43 65 23 and when make remove from left program will remove 12 but when call display method show like this 12 43 65 and when make remove from right program will remove 23 but when call display method show like this 12 43 Why ?????? ); and when i try to make insert after remove write this Can not insert right because the queue is full . first remove right and then u can insert right where is the problem ?? Please Help me please 2) My code FIRST CLASS class dqueue { private int fullsize; //number of all cells private int item_num; // number of busy cells only private int front,rear; public int j; private double [] dqarr; //========================================== public dqueue(int s) //constructor { fullsize = s; front = 0; rear = -1; item_num = 0; dqarr = new double[fullsize]; } //========================================== public void insert(double data) { if (rear == fullsize-1) rear = -1; rear++; dqarr[rear] = data; item_num++; } public double removeLeft() // take item from front of queue { double temp = dqarr[front++]; // get value and incr front if(front == fullsize) front = 0; item_num --; // one less item return temp; } public double removeRight() // take item from rear of queue { double temp = dqarr[rear--]; // get value and decr rear if(rear == -1) // rear = item_num -1; item_num --; // one less item return temp; } //========================================= public void display () //display items { for (int j=0;j<item_num;j++) // for every element System.out.print(dqarr[j] +" " ); // display it System.out.println(""); } //========================================= public int size() //number of items in queue { return item_num; } //========================================== public boolean isEmpty() // true if queue is empty { return (item_num ==0); } } SECOND CLASS import java.util.Scanner; class dqueuetest { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println(" ***** Welcome here***** "); System.out.println(" ***** Mind Of Programming Group***** "); System.out.println(" _____________________________________________ "); System.out.println("enter size of your dqueue"); int size = input.nextInt(); dqueue mydq = new dqueue(size); System.out.println(""); System.out.println("enter your itemes"); //===================================== for(int i = 0;i<=size-1;i++) { System.out.printf("item %d:",i+1); double item = input.nextDouble(); mydq.insert(item); System.out.println(""); } //===================================== int queue =size ; int c = 0 ; while (c != 6) { System.out.println(""); System.out.println("************************************************"); System.out.println(" MAIN MENUE"); System.out.println("1- INSERT RIGHT "); System.out.println("2- REMOVE LEFT"); System.out.println("3- REMOVE RIGHT"); System.out.println("4- DISPLAY"); System.out.println("5- SIZE"); System.out.println("6- EXIT"); System.out.println("************************************************"); System.out.println("choose your operation by number(1-6)"); c = input.nextInt(); switch (c) { case 1: if (queue == size) System.out.print("Can not insert right because the queue is full . first remove right and then u can insert right "); else { System.out.print("enter your item: "); double item = input.nextDouble(); mydq.insert(item);} break; case 2: System.out.println("REMOVE FROM REAR :"); if( !mydq.isEmpty() ) { double item = mydq.removeLeft(); System.out.print(item + "\t"); } // end while System.out.println(""); mydq.display(); break; case 3: System.out.println("REMOVE FROM FRONT :"); if( !mydq.isEmpty() ) { double item = mydq.removeRight(); System.out.print(item + "\t"); } // end while System.out.println(""); mydq.display(); break; case 4: System.out.println("The items in Queue are :"); mydq.display(); break; case 5: System.out.println("The Size of the Queue is :"+mydq.size()); break; case 6: System.out.println("Good Bye"); break; default: System.out.println("wrong chiose enter again"); } //end switch } //end while } // end main }//end class

  Read the article

• Mistake in dispaly and insert method (double - ended queue)

  - by MANAL

  1) My problem when i make remove from right or left program will be remove true but when i call diplay method the content wrong like this I insert 12 43 65 23 and when make remove from left program will remove 12 but when call display method show like this 12 43 65 and when make remove from right program will remove 23 but when call display method show like this 12 43 Why ?????? ); and when i try to make insert after remove write this Can not insert right because the queue is full . first remove right and then u can insert right where is the problem ?? Please Help me please 2) My code FIRST CLASS class dqueue { private int fullsize; //number of all cells private int item_num; // number of busy cells only private int front,rear; public int j; private double [] dqarr; //========================================== public dqueue(int s) //constructor { fullsize = s; front = 0; rear = -1; item_num = 0; dqarr = new double[fullsize]; } //========================================== public void insert(double data) { if (rear == fullsize-1) rear = -1; rear++; dqarr[rear] = data; item_num++; } public double removeLeft() // take item from front of queue { double temp = dqarr[front++]; // get value and incr front if(front == fullsize) front = 0; item_num --; // one less item return temp; } public double removeRight() // take item from rear of queue { double temp = dqarr[rear--]; // get value and decr rear if(rear == -1) // rear = item_num -1; item_num --; // one less item return temp; } //========================================= public void display () //display items { for (int j=0;j //========================================= public int size() //number of items in queue { return item_num; } //========================================== public boolean isEmpty() // true if queue is empty { return (item_num ==0); } } SECOND CLASS import java.util.Scanner; class dqueuetest { public static void main(String[] args) { Scanner input = new Scanner(System.in); System.out.println(" Welcome here** "); System.out.println(" * Mind Of Programming Group*** "); System.out.println(" _________________________ "); System.out.println("enter size of your dqueue"); int size = input.nextInt(); dqueue mydq = new dqueue(size); System.out.println(""); System.out.println("enter your itemes"); //===================================== for(int i = 0;i<=size-1;i++) { System.out.printf("item %d:",i+1); double item = input.nextDouble(); mydq.insert(item); System.out.println(""); } //===================================== int queue =size ; int c = 0 ; while (c != 6) { System.out.println(""); System.out.println("**************************"); System.out.println(" MAIN MENUE"); System.out.println("1- INSERT RIGHT "); System.out.println("2- REMOVE LEFT"); System.out.println("3- REMOVE RIGHT"); System.out.println("4- DISPLAY"); System.out.println("5- SIZE"); System.out.println("6- EXIT"); System.out.println("**************************"); System.out.println("choose your operation by number(1-6)"); c = input.nextInt(); switch (c) { case 1: if (queue == size) System.out.print("Can not insert right because the queue is full . first remove right and then u can insert right "); else { System.out.print("enter your item: "); double item = input.nextDouble(); mydq.insert(item);} break; case 2: System.out.println("REMOVE FROM REAR :"); if( !mydq.isEmpty() ) { double item = mydq.removeLeft(); System.out.print(item + "\t"); } // end while System.out.println(""); mydq.display(); break; case 3: System.out.println("REMOVE FROM FRONT :"); if( !mydq.isEmpty() ) { double item = mydq.removeRight(); System.out.print(item + "\t"); } // end while System.out.println(""); mydq.display(); break; case 4: System.out.println("The items in Queue are :"); mydq.display(); break; case 5: System.out.println("The Size of the Queue is :"+mydq.size()); break; case 6: System.out.println("Good Bye"); break; default: System.out.println("wrong chiose enter again"); } //end switch } //end while } // end main }//end class

  Read the article

• files get uploaded just before they get cancelled

  - by user1763986

  Got a little situation here where I am trying to cancel a file's upload. What I have done is stated that if the user clicks on the "Cancel" button, then it will simply remove the iframe so that it does not go to the page where it uploads the files into the server and inserts data into the database. Now this works fine if the user clicks on the "Cancel" button in quickish time the problem I have realised though is that if the user clicks on the "Cancel" button very late, it sometimes doesn't remove the iframe in time meaning that the file has just been uploaded just before the user has clicked on the "Cancel" button. So my question is that is there a way that if the file does somehow get uploaded before the user clicks on the "Cancel" button, that it deletes the data in the database and removes the file from the server? Below is the image upload form: <form action="imageupload.php" method="post" enctype="multipart/form-data" target="upload_target_image" onsubmit="return imageClickHandler(this);" class="imageuploadform" > <p class="imagef1_upload_process" align="center"> Loading...<br/> <img src="Images/loader.gif" /> </p> <p class="imagef1_upload_form" align="center"> <br/> <span class="imagemsg"></span> <label>Image File: <input name="fileImage" type="file" class="fileImage" /></label><br/> <br/> <label class="imagelbl"><input type="submit" name="submitImageBtn" class="sbtnimage" value="Upload" /></label> </p> <p class="imagef1_cancel" align="center"> <input type="reset" name="imageCancel" class="imageCancel" value="Cancel" /> </p> <iframe class="upload_target_image" name="upload_target_image" src="#" style="width:0px;height:0px;border:0px;solid;#fff;"></iframe> </form> Below is the jquery function which controls the "Cancel" button: $(imageuploadform).find(".imageCancel").on("click", function(event) { $('.upload_target_image').get(0).contentwindow $("iframe[name='upload_target_image']").attr("src", "javascript:'<html></html>'"); return stopImageUpload(2); }); Below is the php code where it uploads the files and inserts the data into the database. The form above posts to this php page "imageupload.php": <body> <?php include('connect.php'); session_start(); $result = 0; //uploads file move_uploaded_file($_FILES["fileImage"]["tmp_name"], "ImageFiles/" . $_FILES["fileImage"]["name"]); $result = 1; //set up the INSERT SQL query command to insert the name of the image file into the "Image" Table $imagesql = "INSERT INTO Image (ImageFile) VALUES (?)"; //prepare the above SQL statement if (!$insert = $mysqli->prepare($imagesql)) { // Handle errors with prepare operation here } //bind the parameters (these are the values that will be inserted) $insert->bind_param("s",$img); //Assign the variable of the name of the file uploaded $img = 'ImageFiles/'.$_FILES['fileImage']['name']; //execute INSERT query $insert->execute(); if ($insert->errno) { // Handle query error here } //close INSERT query $insert->close(); //Retrieve the ImageId of the last uploded file $lastID = $mysqli->insert_id; //Insert into Image_Question Table (be using last retrieved Image id in order to do this) $imagequestionsql = "INSERT INTO Image_Question (ImageId, SessionId, QuestionId) VALUES (?, ?, ?)"; //prepare the above SQL statement if (!$insertimagequestion = $mysqli->prepare($imagequestionsql)) { // Handle errors with prepare operation here echo "Prepare statement err imagequestion"; } //Retrieve the question number $qnum = (int)$_POST['numimage']; //bind the parameters (these are the values that will be inserted) $insertimagequestion->bind_param("isi",$lastID, 'Exam', $qnum); //execute INSERT query $insertimagequestion->execute(); if ($insertimagequestion->errno) { // Handle query error here } //close INSERT query $insertimagequestion->close(); ?> <!--Javascript which will output the message depending on the status of the upload (successful, failed or cancelled)--> <script> window.top.stopImageUpload(<?php echo $result; ?>, '<?php echo $_FILES['fileImage']['name'] ?>'); </script> </body> UPDATE: Below is the php code "cancelimage.php" where I want to delete the cancelled file from the server and delete the record from the database. It is set up but not finished, can somebody finish it off so I can retrieve the name of the file and it's id using $_SESSION? <?php // connect to the database include('connect.php'); /* check connection */ if (mysqli_connect_errno()) { printf("Connect failed: %s\n", mysqli_connect_error()); die(); } //remove file from server unlink("ImageFiles/...."); //need to retrieve file name here where the ... line is //DELETE query statement where it will delete cancelled file from both Image and Image Question Table $imagedeletesql = " DELETE img, img_q FROM Image AS img LEFT JOIN Image_Question AS img_q ON img_q.ImageId = img.ImageId WHERE img.ImageFile = ?"; //prepare delete query if (!$delete = $mysqli->prepare($imagedeletesql)) { // Handle errors with prepare operation here } //Dont pass data directly to bind_param store it in a variable $delete->bind_param("s",$img); //execute DELETE query $delete->execute(); if ($delete->errno) { // Handle query error here } //close query $delete->close(); ?> Can you please provide an sample code in your answer to make it easier for me. Thank you

  Read the article

• Elfsign Object Signing on Solaris

  - by danx

  Elfsign Object Signing on Solaris Don't let this happen to you—use elfsign! Solaris elfsign(1) is a command that signs and verifies ELF format executables. That includes not just executable programs (such as ls or cp), but other ELF format files including libraries (such as libnvpair.so) and kernel modules (such as autofs). Elfsign has been available since Solaris 10 and ELF format files distributed with Solaris, since Solaris 10, are signed by either Sun Microsystems or its successor, Oracle Corporation. When an ELF file is signed, elfsign adds a new section the ELF file, .SUNW_signature, that contains a RSA public key signature and other information about the signer. That is, the algorithm used, algorithm OID, signer CN/OU, and time stamp. The signature section can later be verified by elfsign or other software by matching the signature in the file agains the ELF file contents (excluding the signature). ELF executable files may also be signed by a 3rd-party or by the customer. This is useful for verifying the origin and authenticity of executable files installed on a system. The 3rd-party or customer public key certificate should be installed in /etc/certs/ to allow verification by elfsign. For currently-released versions of Solaris, only cryptographic framework plugin libraries are verified by Solaris. However, all ELF files may be verified by the elfsign command at any time. Elfsign Algorithms Elfsign signatures are created by taking a digest of the ELF section contents, then signing the digest with RSA. To verify, one takes a digest of ELF file and compares with the expected digest that's computed from the signature and RSA public key. Originally elfsign took a MD5 digest of a SHA-1 digest of the ELF file sections, then signed the resulting digest with RSA. In Solaris 11.1 then Solaris 11.1 SRU 7 (5/2013), the elfsign crypto algorithms available have been expanded to keep up with evolving cryptography. The following table shows the available elfsign algorithms: Elfsign Algorithm Solaris Release Comments elfsign sign -F rsa_md5_sha1   S10, S11.0, S11.1 Default for S10. Not recommended* elfsign sign -F rsa_sha1 S11.1 Default for S11.1. Not recommended elfsign sign -F rsa_sha256 S11.1 patch SRU7+   Recommended ___ *Most or all CAs do not accept MD5 CSRs and do not issue MD5 certs due to MD5 hash collision problems. RSA Key Length. I recommend using RSA-2048 key length with elfsign is RSA-2048 as the best balance between a long expected "life time", interoperability, and performance. RSA-2048 keys have an expected lifetime through 2030 (and probably beyond). For details, see Recommendation for Key Management: Part 1: General, NIST Publication SP 800-57 part 1 (rev. 3, 7/2012, PDF), tables 2 and 4 (pp. 64, 67). Step 1: create or obtain a key and cert The first step in using elfsign is to obtain a key and cert from a public Certificate Authority (CA), or create your own self-signed key and cert. I'll briefly explain both methods. Obtaining a Certificate from a CA To obtain a cert from a CA, such as Verisign, Thawte, or Go Daddy (to name a few random examples), you create a private key and a Certificate Signing Request (CSR) file and send it to the CA, following the instructions of the CA on their website. They send back a signed public key certificate. The public key cert, along with the private key you created is used by elfsign to sign an ELF file. The public key cert is distributed with the software and is used by elfsign to verify elfsign signatures in ELF files. You need to request a RSA "Class 3 public key certificate", which is used for servers and software signing. Elfsign uses RSA and we recommend RSA-2048 keys. The private key and CSR can be generated with openssl(1) or pktool(1) on Solaris. Here's a simple example that uses pktool to generate a private RSA_2048 key and a CSR for sending to a CA: $ pktool gencsr keystore=file format=pem outcsr=MYCSR.p10 \ subject="CN=canineswworks.com,OU=Canine SW object signing" \ outkey=MYPRIVATEKEY.key $ openssl rsa -noout -text -in MYPRIVATEKEY.key Private-Key: (2048 bit) modulus: 00:d2:ef:42:f2:0b:8c:96:9f:45:32:fc:fe:54:94: . . . [omitted for brevity] . . . c9:c7 publicExponent: 65537 (0x10001) privateExponent: 26:14:fc:49:26:bc:a3:14:ee:31:5e:6b:ac:69:83: . . . [omitted for brevity] . . . 81 prime1: 00:f6:b7:52:73:bc:26:57:26:c8:11:eb:6c:dc:cb: . . . [omitted for brevity] . . . bc:91:d0:40:d6:9d:ac:b5:69 prime2: 00:da:df:3f:56:b2:18:46:e1:89:5b:6c:f1:1a:41: . . . [omitted for brevity] . . . f3:b7:48:de:c3:d9:ce:af:af exponent1: 00:b9:a2:00:11:02:ed:9a:3f:9c:e4:16:ce:c7:67: . . . [omitted for brevity] . . . 55:50:25:70:d3:ca:b9:ab:99 exponent2: 00:c8:fc:f5:57:11:98:85:8e:9a:ea:1f:f2:8f:df: . . . [omitted for brevity] . . . 23:57:0e:4d:b2:a0:12:d2:f5 coefficient: 2f:60:21:cd:dc:52:76:67:1a:d8:75:3e:7f:b0:64: . . . [omitted for brevity] . . . 06:94:56:d8:9d:5c:8e:9b $ openssl req -noout -text -in MYCSR.p10 Certificate Request: Data: Version: 2 (0x2) Subject: OU=Canine SW object signing, CN=canineswworks.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:d2:ef:42:f2:0b:8c:96:9f:45:32:fc:fe:54:94: . . . [omitted for brevity] . . . c9:c7 Exponent: 65537 (0x10001) Attributes: Signature Algorithm: sha1WithRSAEncryption b3:e8:30:5b:88:37:68:1c:26:6b:45:af:5e:de:ea:60:87:ea: . . . [omitted for brevity] . . . 06:f9:ed:b4 Secure storage of RSA private key. The private key needs to be protected if the key signing is used for production (as opposed to just testing). That is, protect the key to protect against unauthorized signatures by others. One method is to use a PIN-protected PKCS#11 keystore. The private key you generate should be stored in a secure manner, such as in a PKCS#11 keystore using pktool(1). Otherwise others can sign your signature. Other secure key storage mechanisms include a SCA-6000 crypto card, a USB thumb drive stored in a locked area, a dedicated server with restricted access, Oracle Key Manager (OKM), or some combination of these. I also recommend secure backup of the private key. Here's an example of generating a private key protected in the PKCS#11 keystore, and a CSR. $ pktool setpin # use if PIN not set yet Enter token passphrase: changeme Create new passphrase: Re-enter new passphrase: Passphrase changed. $ pktool gencsr keystore=pkcs11 label=MYPRIVATEKEY \ format=pem outcsr=MYCSR.p10 \ subject="CN=canineswworks.com,OU=Canine SW object signing" $ pktool list keystore=pkcs11 Enter PIN for Sun Software PKCS#11 softtoken: Found 1 asymmetric public keys. Key #1 - RSA public key: MYPRIVATEKEY Here's another example that uses openssl instead of pktool to generate a private key and CSR: $ openssl genrsa -out cert.key 2048 $ openssl req -new -key cert.key -out MYCSR.p10 Self-Signed Cert You can use openssl or pktool to create a private key and a self-signed public key certificate. A self-signed cert is useful for development, testing, and internal use. The private key created should be stored in a secure manner, as mentioned above. The following example creates a private key, MYSELFSIGNED.key, and a public key cert, MYSELFSIGNED.pem, using pktool and displays the contents with the openssl command. $ pktool gencert keystore=file format=pem serial=0xD06F00D lifetime=20-year \ keytype=rsa hash=sha256 outcert=MYSELFSIGNED.pem outkey=MYSELFSIGNED.key \ subject="O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com" $ pktool list keystore=file objtype=cert infile=MYSELFSIGNED.pem Found 1 certificates. 1. (X.509 certificate) Filename: MYSELFSIGNED.pem ID: c8:24:59:08:2b:ae:6e:5c:bc:26:bd:ef:0a:9c:54:de:dd:0f:60:46 Subject: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com Issuer: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com Not Before: Oct 17 23:18:00 2013 GMT Not After: Oct 12 23:18:00 2033 GMT Serial: 0xD06F00D0 Signature Algorithm: sha256WithRSAEncryption $ openssl x509 -noout -text -in MYSELFSIGNED.pem Certificate: Data: Version: 3 (0x2) Serial Number: 3496935632 (0xd06f00d0) Signature Algorithm: sha256WithRSAEncryption Issuer: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com Validity Not Before: Oct 17 23:18:00 2013 GMT Not After : Oct 12 23:18:00 2033 GMT Subject: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:bb:e8:11:21:d9:4b:88:53:8b:6c:5a:7a:38:8b: . . . [omitted for brevity] . . . bf:77 Exponent: 65537 (0x10001) Signature Algorithm: sha256WithRSAEncryption 9e:39:fe:c8:44:5c:87:2c:8f:f4:24:f6:0c:9a:2f:64:84:d1: . . . [omitted for brevity] . . . 5f:78:8e:e8 $ openssl rsa -noout -text -in MYSELFSIGNED.key Private-Key: (2048 bit) modulus: 00:bb:e8:11:21:d9:4b:88:53:8b:6c:5a:7a:38:8b: . . . [omitted for brevity] . . . bf:77 publicExponent: 65537 (0x10001) privateExponent: 0a:06:0f:23:e7:1b:88:62:2c:85:d3:2d:c1:e6:6e: . . . [omitted for brevity] . . . 9c:e1:e0:0a:52:77:29:4a:75:aa:02:d8:af:53:24: c1 prime1: 00:ea:12:02:bb:5a:0f:5a:d8:a9:95:b2:ba:30:15: . . . [omitted for brevity] . . . 5b:ca:9c:7c:19:48:77:1e:5d prime2: 00:cd:82:da:84:71:1d:18:52:cb:c6:4d:74:14:be: . . . [omitted for brevity] . . . 5f:db:d5:5e:47:89:a7:ef:e3 exponent1: 32:37:62:f6:a6:bf:9c:91:d6:f0:12:c3:f7:04:e9: . . . [omitted for brevity] . . . 97:3e:33:31:89:66:64:d1 exponent2: 00:88:a2:e8:90:47:f8:75:34:8f:41:50:3b:ce:93: . . . [omitted for brevity] . . . ff:74:d4:be:f3:47:45:bd:cb coefficient: 4d:7c:09:4c:34:73:c4:26:f0:58:f5:e1:45:3c:af: . . . [omitted for brevity] . . . af:01:5f:af:ad:6a:09:bf Step 2: Sign the ELF File object By now you should have your private key, and obtained, by hook or crook, a cert (either from a CA or use one you created (a self-signed cert). The next step is to sign one or more objects with your private key and cert. Here's a simple example that creates an object file, signs, verifies, and lists the contents of the ELF signature. $ echo '#include <stdio.h>\nint main(){printf("Hello\\n");}'>hello.c $ make hello cc -o hello hello.c $ elfsign verify -v -c MYSELFSIGNED.pem -e hello elfsign: no signature found in hello. $ elfsign sign -F rsa_sha256 -v -k MYSELFSIGNED.key -c MYSELFSIGNED.pem -e hello elfsign: hello signed successfully. format: rsa_sha256. signer: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com. signed on: October 17, 2013 04:22:49 PM PDT. $ elfsign list -f format -e hello rsa_sha256 $ elfsign list -f signer -e hello O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com $ elfsign list -f time -e hello October 17, 2013 04:22:49 PM PDT $ elfsign verify -v -c MYSELFSIGNED.key -e hello elfsign: verification of hello failed. format: rsa_sha256. signer: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com. signed on: October 17, 2013 04:22:49 PM PDT. Signing using the pkcs11 keystore To sign the ELF file using a private key in the secure pkcs11 keystore, replace "-K MYSELFSIGNED.key" in the "elfsign sign" command line with "-T MYPRIVATEKEY", where MYPRIVATKEY is the pkcs11 token label. Step 3: Install the cert and test on another system Just signing the object isn't enough. You need to copy or install the cert and the signed ELF file(s) on another system to test that the signature is OK. Your public key cert should be installed in /etc/certs. Use elfsign verify to verify the signature. Elfsign verify checks each cert in /etc/certs until it finds one that matches the elfsign signature in the file. If one isn't found, the verification fails. Here's an example: $ su Password: # rm /etc/certs/MYSELFSIGNED.key # cp MYSELFSIGNED.pem /etc/certs # exit $ elfsign verify -v hello elfsign: verification of hello passed. format: rsa_sha256. signer: O=Canine Software Works, OU=Self-signed CA, CN=canineswworks.com. signed on: October 17, 2013 04:24:20 PM PDT. After testing, package your cert along with your ELF object to allow elfsign verification after your cert and object are installed or copied. Under the Hood: elfsign verification Here's the steps taken to verify a ELF file signed with elfsign. The steps to sign the file are similar except the private key exponent is used instead of the public key exponent and the .SUNW_signature section is written to the ELF file instead of being read from the file. Generate a digest (SHA-256) of the ELF file sections. This digest uses all ELF sections loaded in memory, but excludes the ELF header, the .SUNW_signature section, and the symbol table Extract the RSA signature (RSA-2048) from the .SUNW_signature section Extract the RSA public key modulus and public key exponent (65537) from the public key cert Calculate the expected digest as follows:     signaturepublicKeyExponent % publicKeyModulus Strip the PKCS#1 padding (most significant bytes) from the above. The padding is 0x00, 0x01, 0xff, 0xff, . . ., 0xff, 0x00. If the actual digest == expected digest, the ELF file is verified (OK). Further Information elfsign(1), pktool(1), and openssl(1) man pages. "Signed Solaris 10 Binaries?" blog by Darren Moffat (2005) shows how to use elfsign. "Simple CLI based CA on Solaris" blog by Darren Moffat (2008) shows how to set up a simple CA for use with self-signed certificates. "How to Create a Certificate by Using the pktool gencert Command" System Administration Guide: Security Services (available at docs.oracle.com)

  Read the article

• Toorcon 15 (2013)

  - by danx

  The Toorcon gang (senior staff): h1kari (founder), nfiltr8, and Geo Introduction to Toorcon 15 (2013) A Tale of One Software Bypass of MS Windows 8 Secure Boot Breaching SSL, One Byte at a Time Running at 99%: Surviving an Application DoS Security Response in the Age of Mass Customized Attacks x86 Rewriting: Defeating RoP and other Shinanighans Clowntown Express: interesting bugs and running a bug bounty program Active Fingerprinting of Encrypted VPNs Making Attacks Go Backwards Mask Your Checksums—The Gorry Details Adventures with weird machines thirty years after "Reflections on Trusting Trust" Introduction to Toorcon 15 (2013) Toorcon 15 is the 15th annual security conference held in San Diego. I've attended about a third of them and blogged about previous conferences I attended here starting in 2003. As always, I've only summarized the talks I attended and interested me enough to write about them. Be aware that I may have misrepresented the speaker's remarks and that they are not my remarks or opinion, or those of my employer, so don't quote me or them. Those seeking further details may contact the speakers directly or use The Google. For some talks, I have a URL for further information. A Tale of One Software Bypass of MS Windows 8 Secure Boot Andrew Furtak and Oleksandr Bazhaniuk Yuri Bulygin, Oleksandr ("Alex") Bazhaniuk, and (not present) Andrew Furtak Yuri and Alex talked about UEFI and Bootkits and bypassing MS Windows 8 Secure Boot, with vendor recommendations. They previously gave this talk at the BlackHat 2013 conference. MS Windows 8 Secure Boot Overview UEFI (Unified Extensible Firmware Interface) is interface between hardware and OS. UEFI is processor and architecture independent. Malware can replace bootloader (bootx64.efi, bootmgfw.efi). Once replaced can modify kernel. Trivial to replace bootloader. Today many legacy bootkits—UEFI replaces them most of them. MS Windows 8 Secure Boot verifies everything you load, either through signatures or hashes. UEFI firmware relies on secure update (with signed update). You would think Secure Boot would rely on ROM (such as used for phones0, but you can't do that for PCs—PCs use writable memory with signatures DXE core verifies the UEFI boat loader(s) OS Loader (winload.efi, winresume.efi) verifies the OS kernel A chain of trust is established with a root key (Platform Key, PK), which is a cert belonging to the platform vendor. Key Exchange Keys (KEKs) verify an "authorized" database (db), and "forbidden" database (dbx). X.509 certs with SHA-1/SHA-256 hashes. Keys are stored in non-volatile (NV) flash-based NVRAM. Boot Services (BS) allow adding/deleting keys (can't be accessed once OS starts—which uses Run-Time (RT)). Root cert uses RSA-2048 public keys and PKCS#7 format signatures. SecureBoot — enable disable image signature checks SetupMode — update keys, self-signed keys, and secure boot variables CustomMode — allows updating keys Secure Boot policy settings are: always execute, never execute, allow execute on security violation, defer execute on security violation, deny execute on security violation, query user on security violation Attacking MS Windows 8 Secure Boot Secure Boot does NOT protect from physical access. Can disable from console. Each BIOS vendor implements Secure Boot differently. There are several platform and BIOS vendors. It becomes a "zoo" of implementations—which can be taken advantage of. Secure Boot is secure only when all vendors implement it correctly. Allow only UEFI firmware signed updates protect UEFI firmware from direct modification in flash memory protect FW update components program SPI controller securely protect secure boot policy settings in nvram protect runtime api disable compatibility support module which allows unsigned legacy Can corrupt the Platform Key (PK) EFI root certificate variable in SPI flash. If PK is not found, FW enters setup mode wich secure boot turned off. Can also exploit TPM in a similar manner. One is not supposed to be able to directly modify the PK in SPI flash from the OS though. But they found a bug that they can exploit from User Mode (undisclosed) and demoed the exploit. It loaded and ran their own bootkit. The exploit requires a reboot. Multiple vendors are vulnerable. They will disclose this exploit to vendors in the future. Recommendations: allow only signed updates protect UEFI fw in ROM protect EFI variable store in ROM Breaching SSL, One Byte at a Time Yoel Gluck and Angelo Prado Angelo Prado and Yoel Gluck, Salesforce.com CRIME is software that performs a "compression oracle attack." This is possible because the SSL protocol doesn't hide length, and because SSL compresses the header. CRIME requests with every possible character and measures the ciphertext length. Look for the plaintext which compresses the most and looks for the cookie one byte-at-a-time. SSL Compression uses LZ77 to reduce redundancy. Huffman coding replaces common byte sequences with shorter codes. US CERT thinks the SSL compression problem is fixed, but it isn't. They convinced CERT that it wasn't fixed and they issued a CVE. BREACH, breachattrack.com BREACH exploits the SSL response body (Accept-Encoding response, Content-Encoding). It takes advantage of the fact that the response is not compressed. BREACH uses gzip and needs fairly "stable" pages that are static for ~30 seconds. It needs attacker-supplied content (say from a web form or added to a URL parameter). BREACH listens to a session's requests and responses, then inserts extra requests and responses. Eventually, BREACH guesses a session's secret key. Can use compression to guess contents one byte at-a-time. For example, "Supersecret SupersecreX" (a wrong guess) compresses 10 bytes, and "Supersecret Supersecret" (a correct guess) compresses 11 bytes, so it can find each character by guessing every character. To start the guess, BREACH needs at least three known initial characters in the response sequence. Compression length then "leaks" information. Some roadblocks include no winners (all guesses wrong) or too many winners (multiple possibilities that compress the same). The solutions include: lookahead (guess 2 or 3 characters at-a-time instead of 1 character). Expensive rollback to last known conflict check compression ratio can brute-force first 3 "bootstrap" characters, if needed (expensive) block ciphers hide exact plain text length. Solution is to align response in advance to block size Mitigations length: use variable padding secrets: dynamic CSRF tokens per request secret: change over time separate secret to input-less servlets Future work eiter understand DEFLATE/GZIP HTTPS extensions Running at 99%: Surviving an Application DoS Ryan Huber Ryan Huber, Risk I/O Ryan first discussed various ways to do a denial of service (DoS) attack against web services. One usual method is to find a slow web page and do several wgets. Or download large files. Apache is not well suited at handling a large number of connections, but one can put something in front of it Can use Apache alternatives, such as nginx How to identify malicious hosts short, sudden web requests user-agent is obvious (curl, python) same url requested repeatedly no web page referer (not normal) hidden links. hide a link and see if a bot gets it restricted access if not your geo IP (unless the website is global) missing common headers in request regular timing first seen IP at beginning of attack count requests per hosts (usually a very large number) Use of captcha can mitigate attacks, but you'll lose a lot of genuine users. Bouncer, goo.gl/c2vyEc and www.github.com/rawdigits/Bouncer Bouncer is software written by Ryan in netflow. Bouncer has a small, unobtrusive footprint and detects DoS attempts. It closes blacklisted sockets immediately (not nice about it, no proper close connection). Aggregator collects requests and controls your web proxies. Need NTP on the front end web servers for clean data for use by bouncer. Bouncer is also useful for a popularity storm ("Slashdotting") and scraper storms. Future features: gzip collection data, documentation, consumer library, multitask, logging destroyed connections. Takeaways: DoS mitigation is easier with a complete picture Bouncer designed to make it easier to detect and defend DoS—not a complete cure Security Response in the Age of Mass Customized Attacks Peleus Uhley and Karthik Raman Peleus Uhley and Karthik Raman, Adobe ASSET, blogs.adobe.com/asset/ Peleus and Karthik talked about response to mass-customized exploits. Attackers behave much like a business. "Mass customization" refers to concept discussed in the book Future Perfect by Stan Davis of Harvard Business School. Mass customization is differentiating a product for an individual customer, but at a mass production price. For example, the same individual with a debit card receives basically the same customized ATM experience around the world. Or designing your own PC from commodity parts. Exploit kits are another example of mass customization. The kits support multiple browsers and plugins, allows new modules. Exploit kits are cheap and customizable. Organized gangs use exploit kits. A group at Berkeley looked at 77,000 malicious websites (Grier et al., "Manufacturing Compromise: The Emergence of Exploit-as-a-Service", 2012). They found 10,000 distinct binaries among them, but derived from only a dozen or so exploit kits. Characteristics of Mass Malware: potent, resilient, relatively low cost Technical characteristics: multiple OS, multipe payloads, multiple scenarios, multiple languages, obfuscation Response time for 0-day exploits has gone down from ~40 days 5 years ago to about ~10 days now. So the drive with malware is towards mass customized exploits, to avoid detection There's plenty of evicence that exploit development has Project Manager bureaucracy. They infer from the malware edicts to: support all versions of reader support all versions of windows support all versions of flash support all browsers write large complex, difficult to main code (8750 lines of JavaScript for example Exploits have "loose coupling" of multipe versions of software (adobe), OS, and browser. This allows specific attacks against specific versions of multiple pieces of software. Also allows exploits of more obscure software/OS/browsers and obscure versions. Gave examples of exploits that exploited 2, 3, 6, or 14 separate bugs. However, these complete exploits are more likely to be buggy or fragile in themselves and easier to defeat. Future research includes normalizing malware and Javascript. Conclusion: The coming trend is that mass-malware with mass zero-day attacks will result in mass customization of attacks. x86 Rewriting: Defeating RoP and other Shinanighans Richard Wartell Richard Wartell The attack vector we are addressing here is: First some malware causes a buffer overflow. The malware has no program access, but input access and buffer overflow code onto stack Later the stack became non-executable. The workaround malware used was to write a bogus return address to the stack jumping to malware Later came ASLR (Address Space Layout Randomization) to randomize memory layout and make addresses non-deterministic. The workaround malware used was to jump t existing code segments in the program that can be used in bad ways "RoP" is Return-oriented Programming attacks. RoP attacks use your own code and write return address on stack to (existing) expoitable code found in program ("gadgets"). Pinkie Pie was paid $60K last year for a RoP attack. One solution is using anti-RoP compilers that compile source code with NO return instructions. ASLR does not randomize address space, just "gadgets". IPR/ILR ("Instruction Location Randomization") randomizes each instruction with a virtual machine. Richard's goal was to randomize a binary with no source code access. He created "STIR" (Self-Transofrming Instruction Relocation). STIR disassembles binary and operates on "basic blocks" of code. The STIR disassembler is conservative in what to disassemble. Each basic block is moved to a random location in memory. Next, STIR writes new code sections with copies of "basic blocks" of code in randomized locations. The old code is copied and rewritten with jumps to new code. the original code sections in the file is marked non-executible. STIR has better entropy than ASLR in location of code. Makes brute force attacks much harder. STIR runs on MS Windows (PEM) and Linux (ELF). It eliminated 99.96% or more "gadgets" (i.e., moved the address). Overhead usually 5-10% on MS Windows, about 1.5-4% on Linux (but some code actually runs faster!). The unique thing about STIR is it requires no source access and the modified binary fully works! Current work is to rewrite code to enforce security policies. For example, don't create a *.{exe,msi,bat} file. Or don't connect to the network after reading from the disk. Clowntown Express: interesting bugs and running a bug bounty program Collin Greene Collin Greene, Facebook Collin talked about Facebook's bug bounty program. Background at FB: FB has good security frameworks, such as security teams, external audits, and cc'ing on diffs. But there's lots of "deep, dark, forgotten" parts of legacy FB code. Collin gave several examples of bountied bugs. Some bounty submissions were on software purchased from a third-party (but bounty claimers don't know and don't care). We use security questions, as does everyone else, but they are basically insecure (often easily discoverable). Collin didn't expect many bugs from the bounty program, but they ended getting 20+ good bugs in first 24 hours and good submissions continue to come in. Bug bounties bring people in with different perspectives, and are paid only for success. Bug bounty is a better use of a fixed amount of time and money versus just code review or static code analysis. The Bounty program started July 2011 and paid out $1.5 million to date. 14% of the submissions have been high priority problems that needed to be fixed immediately. The best bugs come from a small % of submitters (as with everything else)—the top paid submitters are paid 6 figures a year. Spammers like to backstab competitors. The youngest sumitter was 13. Some submitters have been hired. Bug bounties also allows to see bugs that were missed by tools or reviews, allowing improvement in the process. Bug bounties might not work for traditional software companies where the product has release cycle or is not on Internet. Active Fingerprinting of Encrypted VPNs Anna Shubina Anna Shubina, Dartmouth Institute for Security, Technology, and Society (I missed the start of her talk because another track went overtime. But I have the DVD of the talk, so I'll expand later) IPsec leaves fingerprints. Using netcat, one can easily visually distinguish various crypto chaining modes just from packet timing on a chart (example, DES-CBC versus AES-CBC) One can tell a lot about VPNs just from ping roundtrips (such as what router is used) Delayed packets are not informative about a network, especially if far away from the network More needed to explore about how TCP works in real life with respect to timing Making Attacks Go Backwards Fuzzynop FuzzyNop, Mandiant This talk is not about threat attribution (finding who), product solutions, politics, or sales pitches. But who are making these malware threats? It's not a single person or group—they have diverse skill levels. There's a lot of fat-fingered fumblers out there. Always look for low-hanging fruit first: "hiding" malware in the temp, recycle, or root directories creation of unnamed scheduled tasks obvious names of files and syscalls ("ClearEventLog") uncleared event logs. Clearing event log in itself, and time of clearing, is a red flag and good first clue to look for on a suspect system Reverse engineering is hard. Disassembler use takes practice and skill. A popular tool is IDA Pro, but it takes multiple interactive iterations to get a clean disassembly. Key loggers are used a lot in targeted attacks. They are typically custom code or built in a backdoor. A big tip-off is that non-printable characters need to be printed out (such as "[Ctrl]" "[RightShift]") or time stamp printf strings. Look for these in files. Presence is not proof they are used. Absence is not proof they are not used. Java exploits. Can parse jar file with idxparser.py and decomile Java file. Java typially used to target tech companies. Backdoors are the main persistence mechanism (provided externally) for malware. Also malware typically needs command and control. Application of Artificial Intelligence in Ad-Hoc Static Code Analysis John Ashaman John Ashaman, Security Innovation Initially John tried to analyze open source files with open source static analysis tools, but these showed thousands of false positives. Also tried using grep, but tis fails to find anything even mildly complex. So next John decided to write his own tool. His approach was to first generate a call graph then analyze the graph. However, the problem is that making a call graph is really hard. For example, one problem is "evil" coding techniques, such as passing function pointer. First the tool generated an Abstract Syntax Tree (AST) with the nodes created from method declarations and edges created from method use. Then the tool generated a control flow graph with the goal to find a path through the AST (a maze) from source to sink. The algorithm is to look at adjacent nodes to see if any are "scary" (a vulnerability), using heuristics for search order. The tool, called "Scat" (Static Code Analysis Tool), currently looks for C# vulnerabilities and some simple PHP. Later, he plans to add more PHP, then JSP and Java. For more information see his posts in Security Innovation blog and NRefactory on GitHub. Mask Your Checksums—The Gorry Details Eric (XlogicX) Davisson Eric (XlogicX) Davisson Sometimes in emailing or posting TCP/IP packets to analyze problems, you may want to mask the IP address. But to do this correctly, you need to mask the checksum too, or you'll leak information about the IP. Problem reports found in stackoverflow.com, sans.org, and pastebin.org are usually not masked, but a few companies do care. If only the IP is masked, the IP may be guessed from checksum (that is, it leaks data). Other parts of packet may leak more data about the IP. TCP and IP checksums both refer to the same data, so can get more bits of information out of using both checksums than just using one checksum. Also, one can usually determine the OS from the TTL field and ports in a packet header. If we get hundreds of possible results (16x each masked nibble that is unknown), one can do other things to narrow the results, such as look at packet contents for domain or geo information. With hundreds of results, can import as CSV format into a spreadsheet. Can corelate with geo data and see where each possibility is located. Eric then demoed a real email report with a masked IP packet attached. Was able to find the exact IP address, given the geo and university of the sender. Point is if you're going to mask a packet, do it right. Eric wouldn't usually bother, but do it correctly if at all, to not create a false impression of security. Adventures with weird machines thirty years after "Reflections on Trusting Trust" Sergey Bratus Sergey Bratus, Dartmouth College (and Julian Bangert and Rebecca Shapiro, not present) "Reflections on Trusting Trust" refers to Ken Thompson's classic 1984 paper. "You can't trust code that you did not totally create yourself." There's invisible links in the chain-of-trust, such as "well-installed microcode bugs" or in the compiler, and other planted bugs. Thompson showed how a compiler can introduce and propagate bugs in unmodified source. But suppose if there's no bugs and you trust the author, can you trust the code? Hell No! There's too many factors—it's Babylonian in nature. Why not? Well, Input is not well-defined/recognized (code's assumptions about "checked" input will be violated (bug/vunerabiliy). For example, HTML is recursive, but Regex checking is not recursive. Input well-formed but so complex there's no telling what it does For example, ELF file parsing is complex and has multiple ways of parsing. Input is seen differently by different pieces of program or toolchain Any Input is a program input executes on input handlers (drives state changes & transitions) only a well-defined execution model can be trusted (regex/DFA, PDA, CFG) Input handler either is a "recognizer" for the inputs as a well-defined language (see langsec.org) or it's a "virtual machine" for inputs to drive into pwn-age ELF ABI (UNIX/Linux executible file format) case study. Problems can arise from these steps (without planting bugs): compiler linker loader ld.so/rtld relocator DWARF (debugger info) exceptions The problem is you can't really automatically analyze code (it's the "halting problem" and undecidable). Only solution is to freeze code and sign it. But you can't freeze everything! Can't freeze ASLR or loading—must have tables and metadata. Any sufficiently complex input data is the same as VM byte code Example, ELF relocation entries + dynamic symbols == a Turing Complete Machine (TM). @bxsays created a Turing machine in Linux from relocation data (not code) in an ELF file. For more information, see Rebecca "bx" Shapiro's presentation from last year's Toorcon, "Programming Weird Machines with ELF Metadata" @bxsays did same thing with Mach-O bytecode Or a DWARF exception handling data .eh_frame + glibc == Turning Machine X86 MMU (IDT, GDT, TSS): used address translation to create a Turning Machine. Page handler reads and writes (on page fault) memory. Uses a page table, which can be used as Turning Machine byte code. Example on Github using this TM that will fly a glider across the screen Next Sergey talked about "Parser Differentials". That having one input format, but two parsers, will create confusion and opportunity for exploitation. For example, CSRs are parsed during creation by cert requestor and again by another parser at the CA. Another example is ELF—several parsers in OS tool chain, which are all different. Can have two different Program Headers (PHDRs) because ld.so parses multiple PHDRs. The second PHDR can completely transform the executable. This is described in paper in the first issue of International Journal of PoC. Conclusions trusting computers not only about bugs! Bugs are part of a problem, but no by far all of it complex data formats means bugs no "chain of trust" in Babylon! (that is, with parser differentials) we need to squeeze complexity out of data until data stops being "code equivalent" Further information See and langsec.org. USENIX WOOT 2013 (Workshop on Offensive Technologies) for "weird machines" papers and videos.

  Read the article

• Ancillary Objects: Separate Debug ELF Files For Solaris

  - by Ali Bahrami

  We introduced a new object ELF object type in Solaris 11 Update 1 called the Ancillary Object. This posting describes them, using material originally written during their development, the PSARC arc case, and the Solaris Linker and Libraries Manual. ELF objects contain allocable sections, which are mapped into memory at runtime, and non-allocable sections, which are present in the file for use by debuggers and observability tools, but which are not mapped or used at runtime. Typically, all of these sections exist within a single object file. Ancillary objects allow them to instead go into a separate file. There are different reasons given for wanting such a feature. One can debate whether the added complexity is worth the benefit, and in most cases it is not. However, one important case stands out — customers with very large 32-bit objects who are not ready or able to make the transition to 64-bits. We have customers who build extremely large 32-bit objects. Historically, the debug sections in these objects have used the stabs format, which is limited, but relatively compact. In recent years, the industry has transitioned to the powerful but verbose DWARF standard. In some cases, the size of these debug sections is large enough to push the total object file size past the fundamental 4GB limit for 32-bit ELF object files. The best, and ultimately only, solution to overly large objects is to transition to 64-bits. However, consider environments where: Hundreds of users may be executing the code on large shared systems. (32-bits use less memory and bus bandwidth, and on sparc runs just as fast as 64-bit code otherwise). Complex finely tuned code, where the original authors may no longer be available. Critical production code, that was expensive to qualify and bring online, and which is otherwise serving its intended purpose without issue. Users in these risk adverse and/or high scale categories have good reasons to push 32-bits objects to the limit before moving on. Ancillary objects offer these users a longer runway. Design The design of ancillary objects is intended to be simple, both to help human understanding when examining elfdump output, and to lower the bar for debuggers such as dbx to support them. The primary and ancillary objects have the same set of section headers, with the same names, in the same order (i.e. each section has the same index in both files). A single added section of type SHT_SUNW_ANCILLARY is added to both objects, containing information that allows a debugger to identify and validate both files relative to each other. Given one of these files, the ancillary section allows you to identify the other. Allocable sections go in the primary object, and non-allocable ones go into the ancillary object. A small set of non-allocable objects, notably the symbol table, are copied into both objects. As noted above, most sections are only written to one of the two objects, but both objects have the same section header array. The section header in the file that does not contain the section data is tagged with the SHF_SUNW_ABSENT section header flag to indicate its placeholder status. Compiler writers and others who produce objects can set the SUNW_SHF_PRIMARY section header flag to mark non-allocable sections that should go to the primary object rather than the ancillary. If you don't request an ancillary object, the Solaris ELF format is unchanged. Users who don't use ancillary objects do not pay for the feature. This is important, because they exist to serve a small subset of our users, and must not complicate the common case. If you do request an ancillary object, the runtime behavior of the primary object will be the same as that of a normal object. There is no added runtime cost. The primary and ancillary object together represent a logical single object. This is facilitated by the use of a single set of section headers. One can easily imagine a tool that can merge a primary and ancillary object into a single file, or the reverse. (Note that although this is an interesting intellectual exercise, we don't actually supply such a tool because there's little practical benefit above and beyond using ld to create the files). Among the benefits of this approach are: There is no need for per-file symbol tables to reflect the contents of each file. The same symbol table that would be produced for a standard object can be used. The section contents are identical in either case — there is no need to alter data to accommodate multiple files. It is very easy for a debugger to adapt to these new files, and the processing involved can be encapsulated in input/output routines. Most of the existing debugger implementation applies without modification. The limit of a 4GB 32-bit output object is now raised to 4GB of code, and 4GB of debug data. There is also the future possibility (not currently supported) to support multiple ancillary objects, each of which could contain up to 4GB of additional debug data. It must be noted however that the 32-bit DWARF debug format is itself inherently 32-bit limited, as it uses 32-bit offsets between debug sections, so the ability to employ multiple ancillary object files may not turn out to be useful. Using Ancillary Objects (From the Solaris Linker and Libraries Guide) By default, objects contain both allocable and non-allocable sections. Allocable sections are the sections that contain executable code and the data needed by that code at runtime. Non-allocable sections contain supplemental information that is not required to execute an object at runtime. These sections support the operation of debuggers and other observability tools. The non-allocable sections in an object are not loaded into memory at runtime by the operating system, and so, they have no impact on memory use or other aspects of runtime performance no matter their size. For convenience, both allocable and non-allocable sections are normally maintained in the same file. However, there are situations in which it can be useful to separate these sections. To reduce the size of objects in order to improve the speed at which they can be copied across wide area networks. To support fine grained debugging of highly optimized code requires considerable debug data. In modern systems, the debugging data can easily be larger than the code it describes. The size of a 32-bit object is limited to 4 Gbytes. In very large 32-bit objects, the debug data can cause this limit to be exceeded and prevent the creation of the object. To limit the exposure of internal implementation details. Traditionally, objects have been stripped of non-allocable sections in order to address these issues. Stripping is effective, but destroys data that might be needed later. The Solaris link-editor can instead write non-allocable sections to an ancillary object. This feature is enabled with the -z ancillary command line option. $ ld ... -z ancillary[=outfile] ...By default, the ancillary file is given the same name as the primary output object, with a .anc file extension. However, a different name can be provided by providing an outfile value to the -z ancillary option. When -z ancillary is specified, the link-editor performs the following actions. All allocable sections are written to the primary object. In addition, all non-allocable sections containing one or more input sections that have the SHF_SUNW_PRIMARY section header flag set are written to the primary object. All remaining non-allocable sections are written to the ancillary object. The following non-allocable sections are written to both the primary object and ancillary object. .shstrtab The section name string table. .symtab The full non-dynamic symbol table. .symtab_shndx The symbol table extended index section associated with .symtab. .strtab The non-dynamic string table associated with .symtab. .SUNW_ancillary Contains the information required to identify the primary and ancillary objects, and to identify the object being examined. The primary object and all ancillary objects contain the same array of sections headers. Each section has the same section index in every file. Although the primary and ancillary objects all define the same section headers, the data for most sections will be written to a single file as described above. If the data for a section is not present in a given file, the SHF_SUNW_ABSENT section header flag is set, and the sh_size field is 0. This organization makes it possible to acquire a full list of section headers, a complete symbol table, and a complete list of the primary and ancillary objects from either of the primary or ancillary objects. The following example illustrates the underlying implementation of ancillary objects. An ancillary object is created by adding the -z ancillary command line option to an otherwise normal compilation. The file utility shows that the result is an executable named a.out, and an associated ancillary object named a.out.anc. $ cat hello.c #include <stdio.h> int main(int argc, char **argv) { (void) printf("hello, world\n"); return (0); } $ cc -g -zancillary hello.c $ file a.out a.out.anc a.out: ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, not stripped, ancillary object a.out.anc a.out.anc: ELF 32-bit LSB ancillary 80386 Version 1, primary object a.out $ ./a.out hello worldThe resulting primary object is an ordinary executable that can be executed in the usual manner. It is no different at runtime than an executable built without the use of ancillary objects, and then stripped of non-allocable content using the strip or mcs commands. As previously described, the primary object and ancillary objects contain the same section headers. To see how this works, it is helpful to use the elfdump utility to display these section headers and compare them. The following table shows the section header information for a selection of headers from the previous link-edit example. Index Section Name Type Primary Flags Ancillary Flags Primary Size Ancillary Size 13 .text PROGBITS ALLOC EXECINSTR ALLOC EXECINSTR SUNW_ABSENT 0x131 0 20 .data PROGBITS WRITE ALLOC WRITE ALLOC SUNW_ABSENT 0x4c 0 21 .symtab SYMTAB 0 0 0x450 0x450 22 .strtab STRTAB STRINGS STRINGS 0x1ad 0x1ad 24 .debug_info PROGBITS SUNW_ABSENT 0 0 0x1a7 28 .shstrtab STRTAB STRINGS STRINGS 0x118 0x118 29 .SUNW_ancillary SUNW_ancillary 0 0 0x30 0x30 The data for most sections is only present in one of the two files, and absent from the other file. The SHF_SUNW_ABSENT section header flag is set when the data is absent. The data for allocable sections needed at runtime are found in the primary object. The data for non-allocable sections used for debugging but not needed at runtime are placed in the ancillary file. A small set of non-allocable sections are fully present in both files. These are the .SUNW_ancillary section used to relate the primary and ancillary objects together, the section name string table .shstrtab, as well as the symbol table.symtab, and its associated string table .strtab. It is possible to strip the symbol table from the primary object. A debugger that encounters an object without a symbol table can use the .SUNW_ancillary section to locate the ancillary object, and access the symbol contained within. The primary object, and all associated ancillary objects, contain a .SUNW_ancillary section that allows all the objects to be identified and related together. $ elfdump -T SUNW_ancillary a.out a.out.anc a.out: Ancillary Section: .SUNW_ancillary index tag value [0] ANC_SUNW_CHECKSUM 0x8724 [1] ANC_SUNW_MEMBER 0x1 a.out [2] ANC_SUNW_CHECKSUM 0x8724 [3] ANC_SUNW_MEMBER 0x1a3 a.out.anc [4] ANC_SUNW_CHECKSUM 0xfbe2 [5] ANC_SUNW_NULL 0 a.out.anc: Ancillary Section: .SUNW_ancillary index tag value [0] ANC_SUNW_CHECKSUM 0xfbe2 [1] ANC_SUNW_MEMBER 0x1 a.out [2] ANC_SUNW_CHECKSUM 0x8724 [3] ANC_SUNW_MEMBER 0x1a3 a.out.anc [4] ANC_SUNW_CHECKSUM 0xfbe2 [5] ANC_SUNW_NULL 0 The ancillary sections for both objects contain the same number of elements, and are identical except for the first element. Each object, starting with the primary object, is introduced with a MEMBER element that gives the file name, followed by a CHECKSUM that identifies the object. In this example, the primary object is a.out, and has a checksum of 0x8724. The ancillary object is a.out.anc, and has a checksum of 0xfbe2. The first element in a .SUNW_ancillary section, preceding the MEMBER element for the primary object, is always a CHECKSUM element, containing the checksum for the file being examined. The presence of a .SUNW_ancillary section in an object indicates that the object has associated ancillary objects. The names of the primary and all associated ancillary objects can be obtained from the ancillary section from any one of the files. It is possible to determine which file is being examined from the larger set of files by comparing the first checksum value to the checksum of each member that follows. Debugger Access and Use of Ancillary Objects Debuggers and other observability tools must merge the information found in the primary and ancillary object files in order to build a complete view of the object. This is equivalent to processing the information from a single file. This merging is simplified by the primary object and ancillary objects containing the same section headers, and a single symbol table. The following steps can be used by a debugger to assemble the information contained in these files. Starting with the primary object, or any of the ancillary objects, locate the .SUNW_ancillary section. The presence of this section identifies the object as part of an ancillary group, contains information that can be used to obtain a complete list of the files and determine which of those files is the one currently being examined. Create a section header array in memory, using the section header array from the object being examined as an initial template. Open and read each file identified by the .SUNW_ancillary section in turn. For each file, fill in the in-memory section header array with the information for each section that does not have the SHF_SUNW_ABSENT flag set. The result will be a complete in-memory copy of the section headers with pointers to the data for all sections. Once this information has been acquired, the debugger can proceed as it would in the single file case, to access and control the running program. Note - The ELF definition of ancillary objects provides for a single primary object, and an arbitrary number of ancillary objects. At this time, the Oracle Solaris link-editor only produces a single ancillary object containing all non-allocable sections. This may change in the future. Debuggers and other observability tools should be written to handle the general case of multiple ancillary objects. ELF Implementation Details (From the Solaris Linker and Libraries Guide) To implement ancillary objects, it was necessary to extend the ELF format to add a new object type (ET_SUNW_ANCILLARY), a new section type (SHT_SUNW_ANCILLARY), and 2 new section header flags (SHF_SUNW_ABSENT, SHF_SUNW_PRIMARY). In this section, I will detail these changes, in the form of diffs to the Solaris Linker and Libraries manual. Part IV ELF Application Binary Interface Chapter 13: Object File Format Object File Format Edit Note: This existing section at the beginning of the chapter describes the ELF header. There's a table of object file types, which now includes the new ET_SUNW_ANCILLARY type. e_type Identifies the object file type, as listed in the following table. NameValueMeaning ET_NONE0No file type ET_REL1Relocatable file ET_EXEC2Executable file ET_DYN3Shared object file ET_CORE4Core file ET_LOSUNW0xfefeStart operating system specific range ET_SUNW_ANCILLARY0xfefeAncillary object file ET_HISUNW0xfefdEnd operating system specific range ET_LOPROC0xff00Start processor-specific range ET_HIPROC0xffffEnd processor-specific range Sections Edit Note: This overview section defines the section header structure, and provides a high level description of known sections. It was updated to define the new SHF_SUNW_ABSENT and SHF_SUNW_PRIMARY flags and the new SHT_SUNW_ANCILLARY section. ... sh_type Categorizes the section's contents and semantics. Section types and their descriptions are listed in Table 13-5. sh_flags Sections support 1-bit flags that describe miscellaneous attributes. Flag definitions are listed in Table 13-8. ... Table 13-5 ELF Section Types, sh_type NameValue . . . SHT_LOSUNW0x6fffffee SHT_SUNW_ancillary0x6fffffee . . . ... SHT_LOSUNW - SHT_HISUNW Values in this inclusive range are reserved for Oracle Solaris OS semantics. SHT_SUNW_ANCILLARY Present when a given object is part of a group of ancillary objects. Contains information required to identify all the files that make up the group. See Ancillary Section. ... Table 13-8 ELF Section Attribute Flags NameValue . . . SHF_MASKOS0x0ff00000 SHF_SUNW_NODISCARD0x00100000 SHF_SUNW_ABSENT0x00200000 SHF_SUNW_PRIMARY0x00400000 SHF_MASKPROC0xf0000000 . . . ... SHF_SUNW_ABSENT Indicates that the data for this section is not present in this file. When ancillary objects are created, the primary object and any ancillary objects, will all have the same section header array, to facilitate merging them to form a complete view of the object, and to allow them to use the same symbol tables. Each file contains a subset of the section data. The data for allocable sections is written to the primary object while the data for non-allocable sections is written to an ancillary file. The SHF_SUNW_ABSENT flag is used to indicate that the data for the section is not present in the object being examined. When the SHF_SUNW_ABSENT flag is set, the sh_size field of the section header must be 0. An application encountering an SHF_SUNW_ABSENT section can choose to ignore the section, or to search for the section data within one of the related ancillary files. SHF_SUNW_PRIMARY The default behavior when ancillary objects are created is to write all allocable sections to the primary object and all non-allocable sections to the ancillary objects. The SHF_SUNW_PRIMARY flag overrides this behavior. Any output section containing one more input section with the SHF_SUNW_PRIMARY flag set is written to the primary object without regard for its allocable status. ... Two members in the section header, sh_link, and sh_info, hold special information, depending on section type. Table 13-9 ELF sh_link and sh_info Interpretation sh_typesh_linksh_info . . . SHT_SUNW_ANCILLARY The section header index of the associated string table. 0 . . . Special Sections Edit Note: This section describes the sections used in Solaris ELF objects, using the types defined in the previous description of section types. It was updated to define the new .SUNW_ancillary (SHT_SUNW_ANCILLARY) section. Various sections hold program and control information. Sections in the following table are used by the system and have the indicated types and attributes. Table 13-10 ELF Special Sections NameTypeAttribute . . . .SUNW_ancillarySHT_SUNW_ancillaryNone . . . ... .SUNW_ancillary Present when a given object is part of a group of ancillary objects. Contains information required to identify all the files that make up the group. See Ancillary Section for details. ... Ancillary Section Edit Note: This new section provides the format reference describing the layout of a .SUNW_ancillary section and the meaning of the various tags. Note that these sections use the same tag/value concept used for dynamic and capabilities sections, and will be familiar to anyone used to working with ELF. In addition to the primary output object, the Solaris link-editor can produce one or more ancillary objects. Ancillary objects contain non-allocable sections that would normally be written to the primary object. When ancillary objects are produced, the primary object and all of the associated ancillary objects contain a SHT_SUNW_ancillary section, containing information that identifies these related objects. Given any one object from such a group, the ancillary section provides the information needed to identify and interpret the others. This section contains an array of the following structures. See sys/elf.h. typedef struct { Elf32_Word a_tag; union { Elf32_Word a_val; Elf32_Addr a_ptr; } a_un; } Elf32_Ancillary; typedef struct { Elf64_Xword a_tag; union { Elf64_Xword a_val; Elf64_Addr a_ptr; } a_un; } Elf64_Ancillary; For each object with this type, a_tag controls the interpretation of a_un. a_val These objects represent integer values with various interpretations. a_ptr These objects represent file offsets or addresses. The following ancillary tags exist. Table 13-NEW1 ELF Ancillary Array Tags NameValuea_un ANC_SUNW_NULL0Ignored ANC_SUNW_CHECKSUM1a_val ANC_SUNW_MEMBER2a_ptr ANC_SUNW_NULL Marks the end of the ancillary section. ANC_SUNW_CHECKSUM Provides the checksum for a file in the c_val element. When ANC_SUNW_CHECKSUM precedes the first instance of ANC_SUNW_MEMBER, it provides the checksum for the object from which the ancillary section is being read. When it follows an ANC_SUNW_MEMBER tag, it provides the checksum for that member. ANC_SUNW_MEMBER Specifies an object name. The a_ptr element contains the string table offset of a null-terminated string, that provides the file name. An ancillary section must always contain an ANC_SUNW_CHECKSUM before the first instance of ANC_SUNW_MEMBER, identifying the current object. Following that, there should be an ANC_SUNW_MEMBER for each object that makes up the complete set of objects. Each ANC_SUNW_MEMBER should be followed by an ANC_SUNW_CHECKSUM for that object. A typical ancillary section will therefore be structured as: TagMeaning ANC_SUNW_CHECKSUMChecksum of this object ANC_SUNW_MEMBERName of object #1 ANC_SUNW_CHECKSUMChecksum for object #1 . . . ANC_SUNW_MEMBERName of object N ANC_SUNW_CHECKSUMChecksum for object N ANC_SUNW_NULL An object can therefore identify itself by comparing the initial ANC_SUNW_CHECKSUM to each of the ones that follow, until it finds a match. Related Other Work The GNU developers have also encountered the need/desire to support separate debug information files, and use the solution detailed at http://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html. At the current time, the separate debug file is constructed by building the standard object first, and then copying the debug data out of it in a separate post processing step, Hence, it is limited to a total of 4GB of code and debug data, just as a single object file would be. They are aware of this, and I have seen online comments indicating that they may add direct support for generating these separate files to their link-editor. It is worth noting that the GNU objcopy utility is available on Solaris, and that the Studio dbx debugger is able to use these GNU style separate debug files even on Solaris. Although this is interesting in terms giving Linux users a familiar environment on Solaris, the 4GB limit means it is not an answer to the problem of very large 32-bit objects. We have also encountered issues with objcopy not understanding Solaris-specific ELF sections, when using this approach. The GNU community also has a current effort to adapt their DWARF debug sections in order to move them to separate files before passing the relocatable objects to the linker. The details of Project Fission can be found at http://gcc.gnu.org/wiki/DebugFission. The goal of this project appears to be to reduce the amount of data seen by the link-editor. The primary effort revolves around moving DWARF data to separate .dwo files so that the link-editor never encounters them. The details of modifying the DWARF data to be usable in this form are involved — please see the above URL for details.

  Read the article

• wxWidgets in Code::Blocks

  - by Vlad

  Hello all, I'm trying to compile the minimal sample from the "Cross-Platform GUI Programming with wxWidgets" book but the following compile errors: ||=== minimal, Debug ===| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.text+0x918)||undefined reference to `__Unwind_Resume' | C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.text+0x931)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.text+0xa96)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.text+0xada)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.text+0xb1e)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_frame.o):frame.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_datacmn.o):datacmn.cpp:(.eh_frame+0x11)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.text+0x63a)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.text+0x696)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.text+0x6f2)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.text+0x74a)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.text+0x7a2)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdicmn.o):gdicmn.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.text+0x88f)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.text+0x927)||undefined reference to `__Unwind_Resume' | C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.text+0x9bf)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.text+0xb8b)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.text+0xc87)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menu.o):menu.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.text+0xbc0)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.text+0xc59)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.text+0xcf5)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.text+0xda6)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.text+0xdce)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_menucmn.o):menucmn.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.text+0x1ff)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.text+0x257)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.text+0x2af)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.text+0x2fc)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.text+0x36d)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_icon.o):icon.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.text+0x4a8)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.text+0x73a)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.text+0x813)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.text+0xc06)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.text+0xd3e)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_gdiimage.o):gdiimage.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.text+0x970)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.text+0xa80)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.text+0xb8c)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.text+0xc78)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.text+0xd4f)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_event.o):event.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.text+0x2ef)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.text+0x32b)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.text+0x43d)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.text+0x586)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.text+0x601)||undefined reference to `__Unwind_Resume'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_appcmn.o):appcmn.cpp:(.eh_frame+0x12)||undefined reference to `___gxx_personality_v0'| C:\SourceCode\Libraries\wxWidgets2.8\lib\gcc_lib\libwxmsw28u_core.a(corelib_app.o):app.cpp:(.text+0x1da)||undefined reference to `__Unwind_Resume'| ||More errors follow but not being shown.| ||Edit the max errors limit in compiler options...| ||=== Build finished: 50 errors, 0 warnings ===| Here's the code sample from the book: #include "wx/wx.h" #include "mondrian.xpm" // Declare the application class class MyApp : public wxApp { public: // Called on application startup virtual bool OnInit(); }; // Declare our main frame class class MyFrame : public wxFrame { public: // Constructor MyFrame(const wxString& title); // Event handlers void OnQuit(wxCommandEvent& event); void OnAbout(wxCommandEvent& event); private: // This class handles events DECLARE_EVENT_TABLE() }; // Implements MyApp& GetApp() DECLARE_APP(MyApp) // Give wxWidgets the means to create a MyApp object IMPLEMENT_APP(MyApp) // Initialize the application bool MyApp::OnInit() { // Create the main application window MyFrame *frame = new MyFrame(wxT("Minimal wxWidgets App")); // Show it frame->Show(true); // Start the event loop return true; } // Event table for MyFrame BEGIN_EVENT_TABLE(MyFrame, wxFrame) EVT_MENU(wxID_ABOUT, MyFrame::OnAbout) EVT_MENU(wxID_EXIT, MyFrame::OnQuit) END_EVENT_TABLE() void MyFrame::OnAbout(wxCommandEvent& event) { wxString msg; msg.Printf(wxT("Hello and welcome to %s"), wxVERSION_STRING); wxMessageBox(msg, wxT("About Minimal"), wxOK | wxICON_INFORMATION, this); } void MyFrame::OnQuit(wxCommandEvent& event) { // Destroy the frame Close(); } MyFrame::MyFrame(const wxString& title) : wxFrame(NULL, wxID_ANY, title) { // Set the frame icon SetIcon(wxIcon(mondrian_xpm)); // Create a menu bar wxMenu *fileMenu = new wxMenu; // The “About” item should be in the help menu wxMenu *helpMenu = new wxMenu; helpMenu->Append(wxID_ABOUT, wxT("&About...\tF1"), wxT("Show about dialog")); fileMenu->Append(wxID_EXIT, wxT("E&xit\tAlt-X"), wxT("Quit this program")); // Now append the freshly created menu to the menu bar... wxMenuBar *menuBar = new wxMenuBar(); menuBar->Append(fileMenu, wxT("&File")); menuBar->Append(helpMenu, wxT("&Help")); // ... and attach this menu bar to the frame SetMenuBar(menuBar); // Create a status bar just for fun CreateStatusBar(2); SetStatusText(wxT("Welcome to wxWidgets!")); } So what's happenning? Thanks! P.S.: I installed wxWidgets through wxPack wich afaik comes with everything precomplied and i also added the wxWidgets directory to Global variables-base in Code::Blocks so everything should be correctly set, right?

  Read the article

• How can I get the following compiled on UVA?

  - by Michael Tsang

  Note the comment below. It cannot compiled on UVA because of a bug in GCC. #include <cstdio> #include <cstring> #include <cctype> #include <map> #include <stdexcept> class Board { public: bool read(FILE *); enum Colour {none, white, black}; Colour check() const; private: struct Index { size_t x; size_t y; Index &operator+=(const Index &) throw(std::range_error); Index operator+(const Index &) const throw(std::range_error); }; const static std::size_t size = 8; char data[size][size]; // Cannot be compiled on GCC 4.1.2 due to GCC bug 29993 // http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29993 typedef bool CheckFunction(Colour, const Index &) const; CheckFunction pawn, knight, bishop, king, rook; bool queen(const Colour c, const Index &location) const { return rook(c, location) || bishop(c, location); } static char get_king(Colour c) { return c == white ? 'k' : 'K'; } template<std::size_t n> bool check_consecutive(Colour c, const Index &location, const Index (&offsets)[n]) const { for(const Index *p = offsets; p != (&offsets)[1]; ++p) { try { Index target = location + *p; for(; data[target.x][target.y] == '.'; target += *p) { } if(data[target.x][target.y] == get_king(c)) return true; } catch(std::range_error &) { } } return false; } template<std::size_t n> bool check_distinct(Colour c, const Index &location, const Index (&offsets)[n]) const { for(const Index *p = offsets; p != (&offsets)[1]; ++p) { try { Index target = location + *p; if(data[target.x][target.y] == get_king(c)) return true; } catch(std::range_error &) { } } return false; } }; int main() { Board board; for(int d = 1; board.read(stdin); ++d) { Board::Colour c = board.check(); const char *sp; switch(c) { case Board::black: sp = "white"; break; case Board::white: sp = "black"; break; case Board::none: sp = "no"; break; } std::printf("Game #%d: %s king is in check.\n", d, sp); std::getchar(); // discard empty line } } bool Board::read(FILE *f) { static const char empty[] = "........" "........" "........" "........" "........" "........" "........" "........"; // 64 dots for(char (*p)[size] = data; p != (&data)[1]; ++p) { std::fread(*p, size, 1, f); std::fgetc(f); // discard new-line } return std::memcmp(empty, data, sizeof data); } Board::Colour Board::check() const { std::map<char, CheckFunction Board::*> fp; fp['P'] = &Board::pawn; fp['N'] = &Board::knight; fp['B'] = &Board::bishop; fp['Q'] = &Board::queen; fp['K'] = &Board::king; fp['R'] = &Board::rook; for(std::size_t i = 0; i != size; ++i) { for(std::size_t j = 0; j != size; ++j) { CheckFunction Board::* p = fp[std::toupper(data[i][j])]; if(p) { Colour ret; if(std::isupper(data[i][j])) ret = white; else ret = black; if((this->*p)(ret, (Index){i, j}/* C99 extension */)) return ret; } } } return none; } bool Board::pawn(const Colour c, const Index &location) const { const std::ptrdiff_t sh = c == white ? -1 : 1; const Index offsets[] = { {sh, 1}, {sh, -1} }; return check_distinct(c, location, offsets); } bool Board::knight(const Colour c, const Index &location) const { static const Index offsets[] = { {1, 2}, {2, 1}, {2, -1}, {1, -2}, {-1, -2}, {-2, -1}, {-2, 1}, {-1, 2} }; return check_distinct(c, location, offsets); } bool Board::bishop(const Colour c, const Index &location) const { static const Index offsets[] = { {1, 1}, {1, -1}, {-1, -1}, {-1, 1} }; return check_consecutive(c, location, offsets); } bool Board::rook(const Colour c, const Index &location) const { static const Index offsets[] = { {1, 0}, {0, -1}, {0, 1}, {-1, 0} }; return check_consecutive(c, location, offsets); } bool Board::king(const Colour c, const Index &location) const { static const Index offsets[] = { {-1, -1}, {-1, 0}, {-1, 1}, {0, 1}, {1, 1}, {1, 0}, {1, -1}, {0, -1} }; return check_distinct(c, location, offsets); } Board::Index &Board::Index::operator+=(const Index &rhs) throw(std::range_error) { if(x + rhs.x >= size || y + rhs.y >= size) throw std::range_error("result is larger than size"); x += rhs.x; y += rhs.y; return *this; } Board::Index Board::Index::operator+(const Index &rhs) const throw(std::range_error) { Index ret = *this; return ret += rhs; }

  Read the article

• Cumulative +1/-1 Cointoss crashes on 1000 iterations. Please advise; c++ boost random libraries

  - by user1731972

  following some former advice Multithreaded application, am I doing it right? I think I have a threadsafe number generator using boost, but my program crashes when I input 1000 iterations. The output .csv file when graphed looks right, but I'm not sure why it's crashing. It's using _beginthread, and everyone is telling me I should use the more (convoluted) _beingthreadex, which I'm not familiar with. If someone could recommend an example, I would greatly appreciate it. Also... someone pointed out I should be applying a second parameter to my _beginthread for the array counting start positions, but I have no idea how to pass more than one parameter, other than attempting to use a structure, and I've read structure's and _beginthread don't get along (although, I could just use the boost threads...) #include <process.h> #include <windows.h> #include <iostream> #include <fstream> #include <time.h> #include <random> #include <boost/random.hpp> //for srand48_r(time(NULL), &randBuffer); which doesn't work #include <stdio.h> #include <stdlib.h> //#include <thread> using namespace std; using namespace boost; using namespace boost::random; void myThread0 (void *dummy ); void myThread1 (void *dummy ); void myThread2 (void *dummy ); void myThread3 (void *dummy ); //for random seeds void initialize(); //from http://stackoverflow.com/questions/7114043/random-number-generation-in-c11-how-to-generate-how-do-they-work uniform_int_distribution<> two(1,2); typedef std::mt19937 MyRNG; // the Mersenne Twister with a popular choice of parameters uint32_t seed_val; // populate somehow MyRNG rng1; // e.g. keep one global instance (per thread) MyRNG rng2; // e.g. keep one global instance (per thread) MyRNG rng3; // e.g. keep one global instance (per thread) MyRNG rng4; // e.g. keep one global instance (per thread) //only needed for shared variables //CRITICAL_SECTION cs1,cs2,cs3,cs4; // global int main() { ofstream myfile; myfile.open ("coinToss.csv"); int rNum; long numRuns; long count = 0; int divisor = 1; float fHolder = 0; long counter = 0; float percent = 0.0; //? //unsigned threadID; //HANDLE hThread; initialize(); HANDLE hThread[4]; const int size = 100000; int array[size]; printf ("Runs (uses multiple of 100,000) "); cin >> numRuns; for (int a = 0; a < numRuns; a++) { hThread[0] = (HANDLE)_beginthread( myThread0, 0, (void*)(array) ); hThread[1] = (HANDLE)_beginthread( myThread1, 0, (void*)(array) ); hThread[2] = (HANDLE)_beginthread( myThread2, 0, (void*)(array) ); hThread[3] = (HANDLE)_beginthread( myThread3, 0, (void*)(array) ); //waits for threads to finish before continuing WaitForMultipleObjects(4, hThread, TRUE, INFINITE); //closes handles I guess? CloseHandle( hThread[0] ); CloseHandle( hThread[1] ); CloseHandle( hThread[2] ); CloseHandle( hThread[3] ); //dump array into calculations //average array into fHolder //this could be split into threads as well for (int p = 0; p < size; p++) { counter += array[p] == 2 ? 1 : -1; //cout << array[p] << endl; //cout << counter << endl; } //this fHolder calculation didn't work //fHolder = counter / size; //so I had to use this cout << counter << endl; fHolder = counter; fHolder = fHolder / size; myfile << fHolder << endl; } } void initialize() { //seed value needs to be supplied //rng1.seed(seed_val*1); rng1.seed((unsigned int)time(NULL)); rng2.seed(((unsigned int)time(NULL))*2); rng3.seed(((unsigned int)time(NULL))*3); rng4.seed(((unsigned int)time(NULL))*4); }; void myThread0 (void *param) { //EnterCriticalSection(&cs1); //aquire the critical section object int *i = (int *)param; for (int x = 0; x < 25000; x++) { //doesn't work, part of merssene twister //i[x] = next(); i[x] = two(rng1); //original srand //i[x] = rand() % 2 + 1; //doesn't work for some reason. //uint_dist2(rng); //i[x] = qrand() % 2 + 1; //cout << i[x] << endl; } //LeaveCriticalSection(&cs1); // release the critical section object } void myThread1 (void *param) { //EnterCriticalSection(&cs2); //aquire the critical section object int *i = (int *)param; for (int x = 25000; x < 50000; x++) { //param[x] = rand() % 2 + 1; i[x] = two(rng2); //i[x] = rand() % 2 + 1; //cout << i[x] << endl; } //LeaveCriticalSection(&cs2); // release the critical section object } void myThread2 (void *param) { //EnterCriticalSection(&cs3); //aquire the critical section object int *i = (int *)param; for (int x = 50000; x < 75000; x++) { i[x] = two(rng3); //i[x] = rand() % 2 + 1; //cout << i[x] << endl; } //LeaveCriticalSection(&cs3); // release the critical section object } void myThread3 (void *param) { //EnterCriticalSection(&cs4); //aquire the critical section object int *i = (int *)param; for (int x = 75000; x < 100000; x++) { i[x] = two(rng4); //i[x] = rand() % 2 + 1; //cout << i[x] << endl; } //LeaveCriticalSection(&cs4); // release the critical section object }

  Read the article

• Little more help with writing a o buffer with libjpeg

  - by Richard Knop

  So I have managed to find another question discussing how to use the libjpeg to compress an image to jpeg. I have found this code which is supposed to work: Compressing IplImage to JPEG using libjpeg in OpenCV Here's the code (it compiles ok): /* This a custom destination manager for jpeglib that enables the use of memory to memory compression. See IJG documentation for details. */ typedef struct { struct jpeg_destination_mgr pub; /* base class */ JOCTET* buffer; /* buffer start address */ int bufsize; /* size of buffer */ size_t datasize; /* final size of compressed data */ int* outsize; /* user pointer to datasize */ int errcount; /* counts up write errors due to buffer overruns */ } memory_destination_mgr; typedef memory_destination_mgr* mem_dest_ptr; /* ------------------------------------------------------------- */ /* MEMORY DESTINATION INTERFACE METHODS */ /* ------------------------------------------------------------- */ /* This function is called by the library before any data gets written */ METHODDEF(void) init_destination (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->pub.next_output_byte = dest->buffer; /* set destination buffer */ dest->pub.free_in_buffer = dest->bufsize; /* input buffer size */ dest->datasize = 0; /* reset output size */ dest->errcount = 0; /* reset error count */ } /* This function is called by the library if the buffer fills up I just reset destination pointer and buffer size here. Note that this behavior, while preventing seg faults will lead to invalid output streams as data is over- written. */ METHODDEF(boolean) empty_output_buffer (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->pub.next_output_byte = dest->buffer; dest->pub.free_in_buffer = dest->bufsize; ++dest->errcount; /* need to increase error count */ return TRUE; } /* Usually the library wants to flush output here. I will calculate output buffer size here. Note that results become incorrect, once empty_output_buffer was called. This situation is notified by errcount. */ METHODDEF(void) term_destination (j_compress_ptr cinfo) { mem_dest_ptr dest = (mem_dest_ptr)cinfo->dest; dest->datasize = dest->bufsize - dest->pub.free_in_buffer; if (dest->outsize) *dest->outsize += (int)dest->datasize; } /* Override the default destination manager initialization provided by jpeglib. Since we want to use memory-to-memory compression, we need to use our own destination manager. */ GLOBAL(void) jpeg_memory_dest (j_compress_ptr cinfo, JOCTET* buffer, int bufsize, int* outsize) { mem_dest_ptr dest; /* first call for this instance - need to setup */ if (cinfo->dest == 0) { cinfo->dest = (struct jpeg_destination_mgr *) (*cinfo->mem->alloc_small) ((j_common_ptr) cinfo, JPOOL_PERMANENT, sizeof (memory_destination_mgr)); } dest = (mem_dest_ptr) cinfo->dest; dest->bufsize = bufsize; dest->buffer = buffer; dest->outsize = outsize; /* set method callbacks */ dest->pub.init_destination = init_destination; dest->pub.empty_output_buffer = empty_output_buffer; dest->pub.term_destination = term_destination; } /* ------------------------------------------------------------- */ /* MEMORY SOURCE INTERFACE METHODS */ /* ------------------------------------------------------------- */ /* Called before data is read */ METHODDEF(void) init_source (j_decompress_ptr dinfo) { /* nothing to do here, really. I mean. I'm not lazy or something, but... we're actually through here. */ } /* Called if the decoder wants some bytes that we cannot provide... */ METHODDEF(boolean) fill_input_buffer (j_decompress_ptr dinfo) { /* we can't do anything about this. This might happen if the provided buffer is either invalid with regards to its content or just a to small bufsize has been given. */ /* fail. */ return FALSE; } /* From IJG docs: "it's not clear that being smart is worth much trouble" So I save myself some trouble by ignoring this bit. */ METHODDEF(void) skip_input_data (j_decompress_ptr dinfo, INT32 num_bytes) { /* There might be more data to skip than available in buffer. This clearly is an error, so screw this mess. */ if ((size_t)num_bytes > dinfo->src->bytes_in_buffer) { dinfo->src->next_input_byte = 0; /* no buffer byte */ dinfo->src->bytes_in_buffer = 0; /* no input left */ } else { dinfo->src->next_input_byte += num_bytes; dinfo->src->bytes_in_buffer -= num_bytes; } } /* Finished with decompression */ METHODDEF(void) term_source (j_decompress_ptr dinfo) { /* Again. Absolute laziness. Nothing to do here. Boring. */ } GLOBAL(void) jpeg_memory_src (j_decompress_ptr dinfo, unsigned char* buffer, size_t size) { struct jpeg_source_mgr* src; /* first call for this instance - need to setup */ if (dinfo->src == 0) { dinfo->src = (struct jpeg_source_mgr *) (*dinfo->mem->alloc_small) ((j_common_ptr) dinfo, JPOOL_PERMANENT, sizeof (struct jpeg_source_mgr)); } src = dinfo->src; src->next_input_byte = buffer; src->bytes_in_buffer = size; src->init_source = init_source; src->fill_input_buffer = fill_input_buffer; src->skip_input_data = skip_input_data; src->term_source = term_source; /* IJG recommend to use their function - as I don't know **** about how to do better, I follow this recommendation */ src->resync_to_restart = jpeg_resync_to_restart; } All I need to do is replace the jpeg_stdio_dest in my program with this code: int numBytes = 0; //size of jpeg after compression char * storage = new char[150000]; //storage buffer JOCTET *jpgbuff = (JOCTET*)storage; //JOCTET pointer to buffer jpeg_memory_dest(&cinfo,jpgbuff,150000,&numBytes); So I need some help to incorporate the above four lines into this function which now works but writes to a file instead of a memory: int write_jpeg_file( char *filename ) { struct jpeg_compress_struct cinfo; struct jpeg_error_mgr jerr; /* this is a pointer to one row of image data */ JSAMPROW row_pointer[1]; FILE *outfile = fopen( filename, "wb" ); if ( !outfile ) { printf("Error opening output jpeg file %s\n!", filename ); return -1; } cinfo.err = jpeg_std_error( &jerr ); jpeg_create_compress(&cinfo); jpeg_stdio_dest(&cinfo, outfile); /* Setting the parameters of the output file here */ cinfo.image_width = width; cinfo.image_height = height; cinfo.input_components = bytes_per_pixel; cinfo.in_color_space = color_space; /* default compression parameters, we shouldn't be worried about these */ jpeg_set_defaults( &cinfo ); /* Now do the compression .. */ jpeg_start_compress( &cinfo, TRUE ); /* like reading a file, this time write one row at a time */ while( cinfo.next_scanline < cinfo.image_height ) { row_pointer[0] = &raw_image[ cinfo.next_scanline * cinfo.image_width * cinfo.input_components]; jpeg_write_scanlines( &cinfo, row_pointer, 1 ); } /* similar to read file, clean up after we're done compressing */ jpeg_finish_compress( &cinfo ); jpeg_destroy_compress( &cinfo ); fclose( outfile ); /* success code is 1! */ return 1; } Anybody could help me out a bit with it? I've tried meddling with it but I am not sure how to do it. I I just replace this line: jpeg_stdio_dest(&cinfo, outfile); It's not going to work. There is more stuff that needs to be changed a bit in that function and I am being a little lost from all those pointers and memory management.

  Read the article

• what "Debug Assertion Failed" mean and how to fix it, c++?

  - by nonon

  hi, why this program gives me a "Debug Assertion Failed" Error Message while running #include "stdafx.h" #include "iostream" #include "fstream" #include "string" using namespace std; int conv_ch(char b) { int f; f=b; b=b+0; switch(b) { case 48: f=0; break; case 49: f=1; break; case 50: f=2; break; case 51: f=3; break; case 52: f=4; break; case 53: f=5; break; case 54: f=6; break; case 55: f=7; break; case 56: f=8; break; case 57: f=9; break; default: f=0; } return f; } class Student { public: string id; size_t id_len; string first_name; size_t first_len; string last_name; size_t last_len; string phone; size_t phone_len; string grade; size_t grade_len; void print(); void clean(); }; void Student::clean() { id.erase (id.begin()+6, id.end()); first_name.erase (first_name.begin()+15, first_name.end()); last_name.erase (last_name.begin()+15, last_name.end()); phone.erase (phone.begin()+10, phone.end()); grade.erase (grade.begin()+2, grade.end()); } void Student::print() { int i; for(i=0;i<6;i++) { cout<<id[i]; } cout<<endl; for(i=0;i<15;i++) { cout<<first_name[i]; } cout<<endl; for(i=0;i<15;i++) { cout<<last_name[i]; } cout<<endl; for(i=0;i<10;i++) { cout<<phone[i]; } cout<<endl; for(i=0;i<2;i++) { cout<<grade[i]; } cout<<endl; } int main() { Student k[80]; char data[1200]; int length,i,recn=0; int rec_length; int counter = 0; fstream myfile; char x1,x2; char y1,y2; char zz; int ad=0; int ser,j; myfile.open ("example.txt",ios::in); int right; int left; int middle; string key; while(!myfile.eof()){ myfile.get(data,1200); char * pch; pch = strtok (data, "#"); printf ("%s\n", pch); j=0; for(i=0;i<6;i++) { k[recn].id[i]=data[j]; j++; } for(i=0;i<15;i++) { k[recn].first_name[i]=data[j]; j++; } for(i=0;i<15;i++) { k[recn].last_name[i]=data[j]; j++; } for(i=0;i<10;i++) { k[recn].phone[i]=data[j]; j++; } for(i=0;i<2;i++) { k[recn].grade[i]=data[j]; j++; } recn++; j=0; } //cout<<recn; string temp1; size_t temp2; int temp3; for(i=0;i<recn-1;i++) { for(j=0;j<recn-1;j++) { if(k[i].id.compare(k[j].id)<0) { temp1 = k[i].first_name; k[i].first_name = k[j].first_name; k[j].first_name = temp1; temp2 = k[i].first_len; k[i].first_len = k[j].first_len; k[j].first_len = temp2; temp1 = k[i].last_name; k[i].last_name = k[j].last_name; k[j].last_name = temp1; temp2 = k[i].last_len; k[i].last_len = k[j].last_len; k[j].last_len = temp2; temp1 = k[i].grade; k[i].grade = k[j].grade; k[j].grade = temp1; temp2 = k[i].grade_len; k[i].grade_len = k[j].grade_len; k[j].grade_len = temp2; temp1 = k[i].id; k[i].id = k[j].id; k[j].id = temp1; temp2 = k[i].id_len; k[i].id_len = k[j].id_len; k[j].id_len = temp2; temp1 = k[i].phone; k[i].phone = k[j].phone; k[j].phone = temp1; temp2 = k[i].phone_len; k[i].phone_len = k[j].phone_len; k[j].phone_len = temp2; } } } for(i=0;i<recn-1;i++) { k[i].clean(); } char z; string id_sear; cout<<"Enter 1 to display , 2 to search , 3 to exit:"; cin>>z; while(1){ switch(z) { case '1': for(i=0;i<recn-1;i++) { k[i].print(); } break; case '2': cin>>key; right=0; left=recn-2; while(right<=left) { middle=((right+left)/2); if(key.compare(k[middle].id)==0){ cout<<"Founded"<<endl; k[middle].print(); break; } else if(key.compare(k[middle].id)<0) { left=middle-1; } else { right=middle+1; } } break; case '3': exit(0); break; } cout<<"Enter 1 to display , 2 to search , 3 to exit:"; cin>>z; } return 0; } the program reads from a file example.txt 313121crewwe matt 0114323111A # 433444cristinaee john 0113344325A+# 324311matte richee 3040554032B # the idea is to read fixed size field structure with a text seprator record strucutre

  Read the article

• Using R to Analyze G1GC Log Files

  - by user12620111

  Using R to Analyze G1GC Log Files body, td { font-family: sans-serif; background-color: white; font-size: 12px; margin: 8px; } tt, code, pre { font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console', Consolas, Monaco, monospace; } h1 { font-size:2.2em; } h2 { font-size:1.8em; } h3 { font-size:1.4em; } h4 { font-size:1.0em; } h5 { font-size:0.9em; } h6 { font-size:0.8em; } a:visited { color: rgb(50%, 0%, 50%); } pre { margin-top: 0; max-width: 95%; border: 1px solid #ccc; white-space: pre-wrap; } pre code { display: block; padding: 0.5em; } code.r, code.cpp { background-color: #F8F8F8; } table, td, th { border: none; } blockquote { color:#666666; margin:0; padding-left: 1em; border-left: 0.5em #EEE solid; } hr { height: 0px; border-bottom: none; border-top-width: thin; border-top-style: dotted; border-top-color: #999999; } @media print { * { background: transparent !important; color: black !important; filter:none !important; -ms-filter: none !important; } body { font-size:12pt; max-width:100%; } a, a:visited { text-decoration: underline; } hr { visibility: hidden; page-break-before: always; } pre, blockquote { padding-right: 1em; page-break-inside: avoid; } tr, img { page-break-inside: avoid; } img { max-width: 100% !important; } @page :left { margin: 15mm 20mm 15mm 10mm; } @page :right { margin: 15mm 10mm 15mm 20mm; } p, h2, h3 { orphans: 3; widows: 3; } h2, h3 { page-break-after: avoid; } } pre .operator, pre .paren { color: rgb(104, 118, 135) } pre .literal { color: rgb(88, 72, 246) } pre .number { color: rgb(0, 0, 205); } pre .comment { color: rgb(76, 136, 107); } pre .keyword { color: rgb(0, 0, 255); } pre .identifier { color: rgb(0, 0, 0); } pre .string { color: rgb(3, 106, 7); } var hljs=new function(){function m(p){return p.replace(/&/gm,"&").replace(/"}while(y.length||w.length){var v=u().splice(0,1)[0];z+=m(x.substr(q,v.offset-q));q=v.offset;if(v.event=="start"){z+=t(v.node);s.push(v.node)}else{if(v.event=="stop"){var p,r=s.length;do{r--;p=s[r];z+=("")}while(p!=v.node);s.splice(r,1);while(r'+M[0]+""}else{r+=M[0]}O=P.lR.lastIndex;M=P.lR.exec(L)}return r+L.substr(O,L.length-O)}function J(L,M){if(M.sL&&e[M.sL]){var r=d(M.sL,L);x+=r.keyword_count;return r.value}else{return F(L,M)}}function I(M,r){var L=M.cN?'':"";if(M.rB){y+=L;M.buffer=""}else{if(M.eB){y+=m(r)+L;M.buffer=""}else{y+=L;M.buffer=r}}D.push(M);A+=M.r}function G(N,M,Q){var R=D[D.length-1];if(Q){y+=J(R.buffer+N,R);return false}var P=q(M,R);if(P){y+=J(R.buffer+N,R);I(P,M);return P.rB}var L=v(D.length-1,M);if(L){var O=R.cN?"":"";if(R.rE){y+=J(R.buffer+N,R)+O}else{if(R.eE){y+=J(R.buffer+N,R)+O+m(M)}else{y+=J(R.buffer+N+M,R)+O}}while(L1){O=D[D.length-2].cN?"":"";y+=O;L--;D.length--}var r=D[D.length-1];D.length--;D[D.length-1].buffer="";if(r.starts){I(r.starts,"")}return R.rE}if(w(M,R)){throw"Illegal"}}var E=e[B];var D=[E.dM];var A=0;var x=0;var y="";try{var s,u=0;E.dM.buffer="";do{s=p(C,u);var t=G(s[0],s[1],s[2]);u+=s[0].length;if(!t){u+=s[1].length}}while(!s[2]);if(D.length1){throw"Illegal"}return{r:A,keyword_count:x,value:y}}catch(H){if(H=="Illegal"){return{r:0,keyword_count:0,value:m(C)}}else{throw H}}}function g(t){var p={keyword_count:0,r:0,value:m(t)};var r=p;for(var q in e){if(!e.hasOwnProperty(q)){continue}var s=d(q,t);s.language=q;if(s.keyword_count+s.rr.keyword_count+r.r){r=s}if(s.keyword_count+s.rp.keyword_count+p.r){r=p;p=s}}if(r.language){p.second_best=r}return p}function i(r,q,p){if(q){r=r.replace(/^((]+|\t)+)/gm,function(t,w,v,u){return w.replace(/\t/g,q)})}if(p){r=r.replace(/\n/g,"")}return r}function n(t,w,r){var x=h(t,r);var v=a(t);var y,s;if(v){y=d(v,x)}else{return}var q=c(t);if(q.length){s=document.createElement("pre");s.innerHTML=y.value;y.value=k(q,c(s),x)}y.value=i(y.value,w,r);var u=t.className;if(!u.match("(\\s|^)(language-)?"+v+"(\\s|$)")){u=u?(u+" "+v):v}if(/MSIE [678]/.test(navigator.userAgent)&&t.tagName=="CODE"&&t.parentNode.tagName=="PRE"){s=t.parentNode;var p=document.createElement("div");p.innerHTML=""+y.value+"";t=p.firstChild.firstChild;p.firstChild.cN=s.cN;s.parentNode.replaceChild(p.firstChild,s)}else{t.innerHTML=y.value}t.className=u;t.result={language:v,kw:y.keyword_count,re:y.r};if(y.second_best){t.second_best={language:y.second_best.language,kw:y.second_best.keyword_count,re:y.second_best.r}}}function o(){if(o.called){return}o.called=true;var r=document.getElementsByTagName("pre");for(var p=0;p|=||=||=|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~";this.ER="(?![\\s\\S])";this.BE={b:"\\\\.",r:0};this.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[this.BE],r:0};this.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[this.BE],r:0};this.CLCM={cN:"comment",b:"//",e:"$"};this.CBLCLM={cN:"comment",b:"/\\*",e:"\\*/"};this.HCM={cN:"comment",b:"#",e:"$"};this.NM={cN:"number",b:this.NR,r:0};this.CNM={cN:"number",b:this.CNR,r:0};this.BNM={cN:"number",b:this.BNR,r:0};this.inherit=function(r,s){var p={};for(var q in r){p[q]=r[q]}if(s){for(var q in s){p[q]=s[q]}}return p}}();hljs.LANGUAGES.cpp=function(){var a={keyword:{"false":1,"int":1,"float":1,"while":1,"private":1,"char":1,"catch":1,"export":1,virtual:1,operator:2,sizeof:2,dynamic_cast:2,typedef:2,const_cast:2,"const":1,struct:1,"for":1,static_cast:2,union:1,namespace:1,unsigned:1,"long":1,"throw":1,"volatile":2,"static":1,"protected":1,bool:1,template:1,mutable:1,"if":1,"public":1,friend:2,"do":1,"return":1,"goto":1,auto:1,"void":2,"enum":1,"else":1,"break":1,"new":1,extern:1,using:1,"true":1,"class":1,asm:1,"case":1,typeid:1,"short":1,reinterpret_cast:2,"default":1,"double":1,register:1,explicit:1,signed:1,typename:1,"try":1,"this":1,"switch":1,"continue":1,wchar_t:1,inline:1,"delete":1,alignof:1,char16_t:1,char32_t:1,constexpr:1,decltype:1,noexcept:1,nullptr:1,static_assert:1,thread_local:1,restrict:1,_Bool:1,complex:1},built_in:{std:1,string:1,cin:1,cout:1,cerr:1,clog:1,stringstream:1,istringstream:1,ostringstream:1,auto_ptr:1,deque:1,list:1,queue:1,stack:1,vector:1,map:1,set:1,bitset:1,multiset:1,multimap:1,unordered_set:1,unordered_map:1,unordered_multiset:1,unordered_multimap:1,array:1,shared_ptr:1}};return{dM:{k:a,i:"",k:a,r:10,c:["self"]}]}}}();hljs.LANGUAGES.r={dM:{c:[hljs.HCM,{cN:"number",b:"\\b0[xX][0-9a-fA-F]+[Li]?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+(?:[eE][+\\-]?\\d*)?L\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+\\.(?!\\d)(?:i\\b)?",e:hljs.IMMEDIATE_RE,r:1},{cN:"number",b:"\\b\\d+(?:\\.\\d*)?(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\.\\d+(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"keyword",b:"(?:tryCatch|library|setGeneric|setGroupGeneric)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\.",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\d+(?![\\w.])",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\b(?:function)",e:hljs.IMMEDIATE_RE,r:2},{cN:"keyword",b:"(?:if|in|break|next|repeat|else|for|return|switch|while|try|stop|warning|require|attach|detach|source|setMethod|setClass)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"literal",b:"(?:NA|NA_integer_|NA_real_|NA_character_|NA_complex_)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"literal",b:"(?:NULL|TRUE|FALSE|T|F|Inf|NaN)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"identifier",b:"[a-zA-Z.][a-zA-Z0-9._]*\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"operator",b:"|=||   Using R to Analyze G1GC Log Files   Using R to Analyze G1GC Log Files Introduction Working in Oracle Platform Integration gives an engineer opportunities to work on a wide array of technologies. My team’s goal is to make Oracle applications run best on the Solaris/SPARC platform. When looking for bottlenecks in a modern applications, one needs to be aware of not only how the CPUs and operating system are executing, but also network, storage, and in some cases, the Java Virtual Machine. I was recently presented with about 1.5 GB of Java Garbage First Garbage Collector log file data. If you’re not familiar with the subject, you might want to review Garbage First Garbage Collector Tuning by Monica Beckwith. The customer had been running Java HotSpot 1.6.0_31 to host a web application server. I was told that the Solaris/SPARC server was running a Java process launched using a commmand line that included the following flags: -d64 -Xms9g -Xmx9g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=80 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCDateStamps -XX:+PrintFlagsFinal -XX:+DisableExplicitGC -XX:+UnlockExperimentalVMOptions -XX:ParallelGCThreads=8 Several sources on the internet indicate that if I were to print out the 1.5 GB of log files, it would require enough paper to fill the bed of a pick up truck. Of course, it would be fruitless to try to scan the log files by hand. Tools will be required to summarize the contents of the log files. Others have encountered large Java garbage collection log files. There are existing tools to analyze the log files: IBM’s GC toolkit The chewiebug GCViewer gchisto HPjmeter Instead of using one of the other tools listed, I decide to parse the log files with standard Unix tools, and analyze the data with R. Data Cleansing The log files arrived in two different formats. I guess that the difference is that one set of log files was generated using a more verbose option, maybe -XX:+PrintHeapAtGC, and the other set of log files was generated without that option. Format 1 In some of the log files, the log files with the less verbose format, a single trace, i.e. the report of a singe garbage collection event, looks like this: {Heap before GC invocations=12280 (full 61): garbage-first heap total 9437184K, used 7499918K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 1 young (4096K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. 2014-05-14T07:24:00.988-0700: 60586.353: [GC pause (young) 7324M->7320M(9216M), 0.1567265 secs] Heap after GC invocations=12281 (full 61): garbage-first heap total 9437184K, used 7496533K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 0 young (0K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. } A simple grep can be used to extract a summary: $ grep "\[ GC pause (young" g1gc.log 2014-05-13T13:24:35.091-0700: 3.109: [GC pause (young) 20M->5029K(9216M), 0.0146328 secs] 2014-05-13T13:24:35.440-0700: 3.459: [GC pause (young) 9125K->6077K(9216M), 0.0086723 secs] 2014-05-13T13:24:37.581-0700: 5.599: [GC pause (young) 25M->8470K(9216M), 0.0203820 secs] 2014-05-13T13:24:42.686-0700: 10.704: [GC pause (young) 44M->15M(9216M), 0.0288848 secs] 2014-05-13T13:24:48.941-0700: 16.958: [GC pause (young) 51M->20M(9216M), 0.0491244 secs] 2014-05-13T13:24:56.049-0700: 24.066: [GC pause (young) 92M->26M(9216M), 0.0525368 secs] 2014-05-13T13:25:34.368-0700: 62.383: [GC pause (young) 602M->68M(9216M), 0.1721173 secs] But that format wasn't easily read into R, so I needed to be a bit more tricky. I used the following Unix command to create a summary file that was easy for R to read. $ echo "SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime" $ grep "\[GC pause (young" g1gc.log | grep -v mark | sed -e 's/[A-SU-z\(\),]/ /g' -e 's/->/ /' -e 's/: / /g' | more SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime 2014-05-13T13:24:35.091-0700 3.109 20 5029 9216 0.0146328 2014-05-13T13:24:35.440-0700 3.459 9125 6077 9216 0.0086723 2014-05-13T13:24:37.581-0700 5.599 25 8470 9216 0.0203820 2014-05-13T13:24:42.686-0700 10.704 44 15 9216 0.0288848 2014-05-13T13:24:48.941-0700 16.958 51 20 9216 0.0491244 2014-05-13T13:24:56.049-0700 24.066 92 26 9216 0.0525368 2014-05-13T13:25:34.368-0700 62.383 602 68 9216 0.1721173 Format 2 In some of the log files, the log files with the more verbose format, a single trace, i.e. the report of a singe garbage collection event, was more complicated than Format 1. Here is a text file with an example of a single G1GC trace in the second format. As you can see, it is quite complicated. It is nice that there is so much information available, but the level of detail can be overwhelming. I wrote this awk script (download) to summarize each trace on a single line. #!/usr/bin/env awk -f BEGIN { printf("SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize\n") } ###################### # Save count data from lines that are at the start of each G1GC trace. # Each trace starts out like this: # {Heap before GC invocations=14 (full 0): # garbage-first heap total 9437184K, used 325496K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) ###################### /{Heap.*full/{ gsub ( "\\)" , "" ); nf=split($0,a,"="); split(a[2],b," "); getline; if ( match($0, "first") ) { G1GC=1; IncrementalCount=b[1]; FullCount=substr( b[3], 1, length(b[3])-1 ); } else { G1GC=0; } } ###################### # Pull out time stamps that are in lines with this format: # 2014-05-12T14:02:06.025-0700: 94.312: [GC pause (young), 0.08870154 secs] ###################### /GC pause/ { DateTime=$1; SecondsSinceLaunch=substr($2, 1, length($2)-1); } ###################### # Heap sizes are in lines that look like this: # [ 4842M->4838M(9216M)] ###################### /\[ .*]$/ { gsub ( "\\[" , "" ); gsub ( "\ \]" , "" ); gsub ( "->" , " " ); gsub ( "\\( " , " " ); gsub ( "\ \)" , " " ); split($0,a," "); if ( split(a[1],b,"M") > 1 ) {BeforeSize=b[1]*1024;} if ( split(a[1],b,"K") > 1 ) {BeforeSize=b[1];} if ( split(a[2],b,"M") > 1 ) {AfterSize=b[1]*1024;} if ( split(a[2],b,"K") > 1 ) {AfterSize=b[1];} if ( split(a[3],b,"M") > 1 ) {TotalSize=b[1]*1024;} if ( split(a[3],b,"K") > 1 ) {TotalSize=b[1];} } ###################### # Emit an output line when you find input that looks like this: # [Times: user=1.41 sys=0.08, real=0.24 secs] ###################### /\[Times/ { if (G1GC==1) { gsub ( "," , "" ); split($2,a,"="); UserTime=a[2]; split($3,a,"="); SysTime=a[2]; split($4,a,"="); RealTime=a[2]; print DateTime,SecondsSinceLaunch,IncrementalCount,FullCount,UserTime,SysTime,RealTime,BeforeSize,AfterSize,TotalSize; G1GC=0; } } The resulting summary is about 25X smaller that the original file, but still difficult for a human to digest. SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ... 2014-05-12T18:36:34.669-0700: 3985.744 561 0 0.57 0.06 0.16 1724416 1720320 9437184 2014-05-12T18:36:34.839-0700: 3985.914 562 0 0.51 0.06 0.19 1724416 1720320 9437184 2014-05-12T18:36:35.069-0700: 3986.144 563 0 0.60 0.04 0.27 1724416 1721344 9437184 2014-05-12T18:36:35.354-0700: 3986.429 564 0 0.33 0.04 0.09 1725440 1722368 9437184 2014-05-12T18:36:35.545-0700: 3986.620 565 0 0.58 0.04 0.17 1726464 1722368 9437184 2014-05-12T18:36:35.726-0700: 3986.801 566 0 0.43 0.05 0.12 1726464 1722368 9437184 2014-05-12T18:36:35.856-0700: 3986.930 567 0 0.30 0.04 0.07 1726464 1723392 9437184 2014-05-12T18:36:35.947-0700: 3987.023 568 0 0.61 0.04 0.26 1727488 1723392 9437184 2014-05-12T18:36:36.228-0700: 3987.302 569 0 0.46 0.04 0.16 1731584 1724416 9437184 Reading the Data into R Once the GC log data had been cleansed, either by processing the first format with the shell script, or by processing the second format with the awk script, it was easy to read the data into R. g1gc.df = read.csv("summary.txt", row.names = NULL, stringsAsFactors=FALSE,sep="") str(g1gc.df) ## 'data.frame': 8307 obs. of 10 variables: ## $ row.names : chr "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ... ## $ SecondsSinceLaunch: num 1.16 1.47 1.97 3.83 6.1 ... ## $ IncrementalCount : int 0 1 2 3 4 5 6 7 8 9 ... ## $ FullCount : int 0 0 0 0 0 0 0 0 0 0 ... ## $ UserTime : num 0.11 0.05 0.04 0.21 0.08 0.26 0.31 0.33 0.34 0.56 ... ## $ SysTime : num 0.04 0.01 0.01 0.05 0.01 0.06 0.07 0.06 0.07 0.09 ... ## $ RealTime : num 0.02 0.02 0.01 0.04 0.02 0.04 0.05 0.04 0.04 0.06 ... ## $ BeforeSize : int 8192 5496 5768 22528 24576 43008 34816 53248 55296 93184 ... ## $ AfterSize : int 1400 1672 2557 4907 7072 14336 16384 18432 19456 21504 ... ## $ TotalSize : int 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 ... head(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount ## 1 2014-05-12T14:00:32.868-0700: 1.161 0 ## 2 2014-05-12T14:00:33.179-0700: 1.472 1 ## 3 2014-05-12T14:00:33.677-0700: 1.969 2 ## 4 2014-05-12T14:00:35.538-0700: 3.830 3 ## 5 2014-05-12T14:00:37.811-0700: 6.103 4 ## 6 2014-05-12T14:00:41.428-0700: 9.720 5 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 1 0 0.11 0.04 0.02 8192 1400 9437184 ## 2 0 0.05 0.01 0.02 5496 1672 9437184 ## 3 0 0.04 0.01 0.01 5768 2557 9437184 ## 4 0 0.21 0.05 0.04 22528 4907 9437184 ## 5 0 0.08 0.01 0.02 24576 7072 9437184 ## 6 0 0.26 0.06 0.04 43008 14336 9437184 Basic Statistics Once the data has been read into R, simple statistics are very easy to generate. All of the numbers from high school statistics are available via simple commands. For example, generate a summary of every column: summary(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount FullCount ## Length:8307 Min. : 1 Min. : 0 Min. : 0.0 ## Class :character 1st Qu.: 9977 1st Qu.:2048 1st Qu.: 0.0 ## Mode :character Median :12855 Median :4136 Median : 12.0 ## Mean :12527 Mean :4156 Mean : 31.6 ## 3rd Qu.:15758 3rd Qu.:6262 3rd Qu.: 61.0 ## Max. :55484 Max. :8391 Max. :113.0 ## UserTime SysTime RealTime BeforeSize ## Min. :0.040 Min. :0.0000 Min. : 0.0 Min. : 5476 ## 1st Qu.:0.470 1st Qu.:0.0300 1st Qu.: 0.1 1st Qu.:5137920 ## Median :0.620 Median :0.0300 Median : 0.1 Median :6574080 ## Mean :0.751 Mean :0.0355 Mean : 0.3 Mean :5841855 ## 3rd Qu.:0.920 3rd Qu.:0.0400 3rd Qu.: 0.2 3rd Qu.:7084032 ## Max. :3.370 Max. :1.5600 Max. :488.1 Max. :8696832 ## AfterSize TotalSize ## Min. : 1380 Min. :9437184 ## 1st Qu.:5002752 1st Qu.:9437184 ## Median :6559744 Median :9437184 ## Mean :5785454 Mean :9437184 ## 3rd Qu.:7054336 3rd Qu.:9437184 ## Max. :8482816 Max. :9437184 Q: What is the total amount of User CPU time spent in garbage collection? sum(g1gc.df$UserTime) ## [1] 6236 As you can see, less than two hours of CPU time was spent in garbage collection. Is that too much? To find the percentage of time spent in garbage collection, divide the number above by total_elapsed_time*CPU_count. In this case, there are a lot of CPU’s and it turns out the the overall amount of CPU time spent in garbage collection isn’t a problem when viewed in isolation. When calculating rates, i.e. events per unit time, you need to ask yourself if the rate is homogenous across the time period in the log file. Does the log file include spikes of high activity that should be separately analyzed? Averaging in data from nights and weekends with data from business hours may alias problems. If you have a reason to suspect that the garbage collection rates include peaks and valleys that need independent analysis, see the “Time Series” section, below. Q: How much garbage is collected on each pass? The amount of heap space that is recovered per GC pass is surprisingly low: At least one collection didn’t recover any data. (“Min.=0”) 25% of the passes recovered 3MB or less. (“1st Qu.=3072”) Half of the GC passes recovered 4MB or less. (“Median=4096”) The average amount recovered was 56MB. (“Mean=56390”) 75% of the passes recovered 36MB or less. (“3rd Qu.=36860”) At least one pass recovered 2GB. (“Max.=2121000”) g1gc.df$Delta = g1gc.df$BeforeSize - g1gc.df$AfterSize summary(g1gc.df$Delta) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0 3070 4100 56400 36900 2120000 Q: What is the maximum User CPU time for a single collection? The worst garbage collection (“Max.”) is many standard deviations away from the mean. The data appears to be right skewed. summary(g1gc.df$UserTime) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.040 0.470 0.620 0.751 0.920 3.370 sd(g1gc.df$UserTime) ## [1] 0.3966 Basic Graphics Once the data is in R, it is trivial to plot the data with formats including dot plots, line charts, bar charts (simple, stacked, grouped), pie charts, boxplots, scatter plots histograms, and kernel density plots. Histogram of User CPU Time per Collection I don't think that this graph requires any explanation. hist(g1gc.df$UserTime, main="User CPU Time per Collection", xlab="Seconds", ylab="Frequency") Box plot to identify outliers When the initial data is viewed with a box plot, you can see the one crazy outlier in the real time per GC. Save this data point for future analysis and drop the outlier so that it’s not throwing off our statistics. Now the box plot shows many outliers, which will be examined later, using times series analysis. Notice that the scale of the x-axis changes drastically once the crazy outlier is removed. par(mfrow=c(2,1)) boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(dominated by a crazy outlier)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") crazy.outlier.df=g1gc.df[g1gc.df$RealTime > 400,] g1gc.df=g1gc.df[g1gc.df$RealTime < 400,] boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(crazy outlier excluded)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") box(which = "outer", lty = "solid") Here is the crazy outlier for future analysis: crazy.outlier.df ## row.names SecondsSinceLaunch IncrementalCount ## 8233 2014-05-12T23:15:43.903-0700: 20741 8316 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 8233 112 0.55 0.42 488.1 8381440 8235008 9437184 ## Delta ## 8233 146432 R Time Series Data To analyze the garbage collection as a time series, I’ll use Z’s Ordered Observations (zoo). “zoo is the creator for an S3 class of indexed totally ordered observations which includes irregular time series.” require(zoo) ## Loading required package: zoo ## ## Attaching package: 'zoo' ## ## The following objects are masked from 'package:base': ## ## as.Date, as.Date.numeric head(g1gc.df[,1]) ## [1] "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" ## [3] "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ## [5] "2014-05-12T14:00:37.811-0700:" "2014-05-12T14:00:41.428-0700:" options("digits.secs"=3) times=as.POSIXct( g1gc.df[,1], format="%Y-%m-%dT%H:%M:%OS%z:") g1gc.z = zoo(g1gc.df[,-c(1)], order.by=times) head(g1gc.z) ## SecondsSinceLaunch IncrementalCount FullCount ## 2014-05-12 17:00:32.868 1.161 0 0 ## 2014-05-12 17:00:33.178 1.472 1 0 ## 2014-05-12 17:00:33.677 1.969 2 0 ## 2014-05-12 17:00:35.538 3.830 3 0 ## 2014-05-12 17:00:37.811 6.103 4 0 ## 2014-05-12 17:00:41.427 9.720 5 0 ## UserTime SysTime RealTime BeforeSize AfterSize ## 2014-05-12 17:00:32.868 0.11 0.04 0.02 8192 1400 ## 2014-05-12 17:00:33.178 0.05 0.01 0.02 5496 1672 ## 2014-05-12 17:00:33.677 0.04 0.01 0.01 5768 2557 ## 2014-05-12 17:00:35.538 0.21 0.05 0.04 22528 4907 ## 2014-05-12 17:00:37.811 0.08 0.01 0.02 24576 7072 ## 2014-05-12 17:00:41.427 0.26 0.06 0.04 43008 14336 ## TotalSize Delta ## 2014-05-12 17:00:32.868 9437184 6792 ## 2014-05-12 17:00:33.178 9437184 3824 ## 2014-05-12 17:00:33.677 9437184 3211 ## 2014-05-12 17:00:35.538 9437184 17621 ## 2014-05-12 17:00:37.811 9437184 17504 ## 2014-05-12 17:00:41.427 9437184 28672 Example of Two Benchmark Runs in One Log File The data in the following graph is from a different log file, not the one of primary interest to this article. I’m including this image because it is an example of idle periods followed by busy periods. It would be uninteresting to average the rate of garbage collection over the entire log file period. More interesting would be the rate of garbage collect in the two busy periods. Are they the same or different? Your production data may be similar, for example, bursts when employees return from lunch and idle times on weekend evenings, etc. Once the data is in an R Time Series, you can analyze isolated time windows. Clipping the Time Series data Flashing back to our test case… Viewing the data as a time series is interesting. You can see that the work intensive time period is between 9:00 PM and 3:00 AM. Lets clip the data to the interesting period:     par(mfrow=c(2,1)) plot(g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Complete Log File", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") clipped.g1gc.z=window(g1gc.z, start=as.POSIXct("2014-05-12 21:00:00"), end=as.POSIXct("2014-05-13 03:00:00")) plot(clipped.g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Limited to Benchmark Execution", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") box(which = "outer", lty = "solid") Cumulative Incremental and Full GC count Here is the cumulative incremental and full GC count. When the line is very steep, it indicates that the GCs are repeating very quickly. Notice that the scale on the Y axis is different for full vs. incremental. plot(clipped.g1gc.z[,c(2:3)], main="Cumulative Incremental and Full GC count", xlab="Time of Day", col="#1b9e77") GC Analysis of Benchmark Execution using Time Series data In the following series of 3 graphs: The “After Size” show the amount of heap space in use after each garbage collection. Many Java objects are still referenced, i.e. alive, during each garbage collection. This may indicate that the application has a memory leak, or may indicate that the application has a very large memory footprint. Typically, an application's memory footprint plateau's in the early stage of execution. One would expect this graph to have a flat top. The steep decline in the heap space may indicate that the application crashed after 2:00. The second graph shows that the outliers in real execution time, discussed above, occur near 2:00. when the Java heap seems to be quite full. The third graph shows that Full GCs are infrequent during the first few hours of execution. The rate of Full GC's, (the slope of the cummulative Full GC line), changes near midnight.   plot(clipped.g1gc.z[,c("AfterSize","RealTime","FullCount")], xlab="Time of Day", col=c("#1b9e77","red","#1b9e77")) GC Analysis of heap recovered Each GC trace includes the amount of heap space in use before and after the individual GC event. During garbage coolection, unreferenced objects are identified, the space holding the unreferenced objects is freed, and thus, the difference in before and after usage indicates how much space has been freed. The following box plot and bar chart both demonstrate the same point - the amount of heap space freed per garbage colloection is surprisingly low. par(mfrow=c(2,1)) boxplot(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", horizontal = TRUE, col="red") hist(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", breaks=100, col="red") box(which = "outer", lty = "solid") This graph is the most interesting. The dark blue area shows how much heap is occupied by referenced Java objects. This represents memory that holds live data. The red fringe at the top shows how much data was recovered after each garbage collection. barplot(clipped.g1gc.z[,c("AfterSize","Delta")], col=c("#7570b3","#e7298a"), xlab="Time of Day", border=NA) legend("topleft", c("Live Objects","Heap Recovered on GC"), fill=c("#7570b3","#e7298a")) box(which = "outer", lty = "solid") When I discuss the data in the log files with the customer, I will ask for an explaination for the large amount of referenced data resident in the Java heap. There are two are posibilities: There is a memory leak and the amount of space required to hold referenced objects will continue to grow, limited only by the maximum heap size. After the maximum heap size is reached, the JVM will throw an “Out of Memory” exception every time that the application tries to allocate a new object. If this is the case, the aplication needs to be debugged to identify why old objects are referenced when they are no longer needed. The application has a legitimate requirement to keep a large amount of data in memory. The customer may want to further increase the maximum heap size. Another possible solution would be to partition the application across multiple cluster nodes, where each node has responsibility for managing a unique subset of the data. Conclusion In conclusion, R is a very powerful tool for the analysis of Java garbage collection log files. The primary difficulty is data cleansing so that information can be read into an R data frame. Once the data has been read into R, a rich set of tools may be used for thorough evaluation.

  Read the article

• Process is killed without a (obvious) reason and program stops working

  - by Krzysiek Gurniak

  Here's what my program is supposed to do: create 4 child processes: process 0 is reading 1 byte at a time from STDIN, then writing it into FIFO process 1 is reading this 1 byte from fifo and write its value as HEX into shared memory process 2 is reading HEX value from shared memory and writing it into pipe finally process 3 is reading from pipe and writing into STDOUT (in my case: terminal) I can't change communication channels. FIFO, then shared memory, then pipes are the only option. My problem: Program stops at random moments when some file is directed into stdin (for example:./program < /dev/urandom). Sometimes after writing 5 HEX values, sometimes after 100. Weird thing is that when it is working and in another terminal I write "pstree -c" there is 1 main process with 4 children processes (which is what I want), but when I write "pstree -c" after it stopped writing (but still runs) there are only 3 child processes. For some reason 1 is gone even though they all have while(1) in them.. I think I might have problem with synchronization here, but I am unable to spot it (I've tried for many hours). Here's the code: #include <unistd.h> #include <fcntl.h> #include <stdio.h> #include <string.h> #include <stdlib.h> #include <sys/shm.h> #include <sys/sem.h> #include <sys/types.h> #include <sys/wait.h> #include <sys/stat.h> #include <string.h> #include <signal.h> #define BUFSIZE 1 #define R 0 #define W 1 // processes ID pid_t p0, p1, p2, p3; // FIFO variables int fifo_fd; unsigned char bufor[BUFSIZE] = {}; unsigned char bufor1[BUFSIZE] = {}; // Shared memory variables key_t key; int shmid; char * tab; // zmienne do pipes int file_des[2]; char bufor_pipe[BUFSIZE*30] = {}; void proces0() { ssize_t n; while(1) { fifo_fd = open("/tmp/fifo",O_WRONLY); if(fifo_fd == -1) { perror("blad przy otwieraniu kolejki FIFO w p0\n"); exit(1); } n = read(STDIN_FILENO, bufor, BUFSIZE); if(n<0) { perror("read error w p0\n"); exit(1); } if(n > 0) { if(write(fifo_fd, bufor, n) != n) { perror("blad zapisu do kolejki fifo w p0\n"); exit(1); } memset(bufor, 0, n); // czyszczenie bufora } close(fifo_fd); } } void proces1() { ssize_t m, x; char wartosc_hex[30] = {}; while(1) { if(tab[0] == 0) { fifo_fd = open("/tmp/fifo", O_RDONLY); // otwiera plik typu fifo do odczytu if(fifo_fd == -1) { perror("blad przy otwieraniu kolejki FIFO w p1\n"); exit(1); } m = read(fifo_fd, bufor1, BUFSIZE); x = m; if(x < 0) { perror("read error p1\n"); exit(1); } if(x > 0) { // Konwersja na HEX if(bufor1[0] < 16) { if(bufor1[0] == 10) // gdy enter { sprintf(wartosc_hex, "0x0%X\n", bufor1[0]); } else { sprintf(wartosc_hex, "0x0%X ", bufor1[0]); } } else { sprintf(wartosc_hex, "0x%X ", bufor1[0]); } // poczekaj az pamiec bedzie pusta (gotowa do zapisu) strcpy(&tab[0], wartosc_hex); memset(bufor1, 0, sizeof(bufor1)); // czyszczenie bufora memset(wartosc_hex, 0, sizeof(wartosc_hex)); // przygotowanie tablicy na zapis wartosci hex x = 0; } close(fifo_fd); } } } void proces2() { close(file_des[0]); // zablokuj kanal do odczytu while(1) { if(tab[0] != 0) { if(write(file_des[1], tab, strlen(tab)) != strlen(tab)) { perror("blad write w p2"); exit(1); } // wyczysc pamiec dzielona by przyjac kolejny bajt memset(tab, 0, sizeof(tab)); } } } void proces3() { ssize_t n; close(file_des[1]); // zablokuj kanal do zapisu while(1) { if(tab[0] == 0) { if((n = read(file_des[0], bufor_pipe, sizeof(bufor_pipe))) > 0) { if(write(STDOUT_FILENO, bufor_pipe, n) != n) { perror("write error w proces3()"); exit(1); } memset(bufor_pipe, 0, sizeof(bufor_pipe)); } } } } int main(void) { key = 5678; int status; // Tworzenie plikow przechowujacych ID procesow int des_pid[2] = {}; char bufor_proces[50] = {}; mknod("pid0", S_IFREG | 0777, 0); mknod("pid1", S_IFREG | 0777, 0); mknod("pid2", S_IFREG | 0777, 0); mknod("pid3", S_IFREG | 0777, 0); // Tworzenie semaforow key_t klucz; klucz = ftok(".", 'a'); // na podstawie pliku i pojedynczego znaku id wyznacza klucz semafora if(klucz == -1) { perror("blad wyznaczania klucza semafora"); exit(1); } semafor = semget(klucz, 1, IPC_CREAT | 0777); // tworzy na podstawie klucza semafor. 1 - ilosc semaforow if(semafor == -1) { perror("blad przy tworzeniu semafora"); exit(1); } if(semctl(semafor, 0, SETVAL, 0) == -1) // ustawia poczatkowa wartosc semafora (klucz, numer w zbiorze od 0, polecenie, argument 0/1/2) { perror("blad przy ustawianiu wartosci poczatkowej semafora"); exit(1); } // Tworzenie lacza nazwanego FIFO if(access("/tmp/fifo", F_OK) == -1) // sprawdza czy plik istnieje, jesli nie - tworzy go { if(mkfifo("/tmp/fifo", 0777) != 0) { perror("blad tworzenia FIFO w main"); exit(1); } } // Tworzenie pamieci dzielonej // Lista pamieci wspoldzielonych, komenda "ipcs" // usuwanie pamieci wspoldzielonej, komenta "ipcrm -m ID_PAMIECI" shmid = shmget(key, (BUFSIZE*30), 0666 | IPC_CREAT); if(shmid == -1) { perror("shmget"); exit(1); } tab = (char *) shmat(shmid, NULL, 0); if(tab == (char *)(-1)) { perror("shmat"); exit(1); } memset(tab, 0, (BUFSIZE*30)); // Tworzenie lacza nienazwanego pipe if(pipe(file_des) == -1) { perror("pipe"); exit(1); } // Tworzenie procesow potomnych if(!(p0 = fork())) { des_pid[W] = open("pid0", O_WRONLY | O_TRUNC | O_CREAT); // 1 - zapis, 0 - odczyt sprintf(bufor_proces, "Proces0 ma ID: %d\n", getpid()); if(write(des_pid[W], bufor_proces, sizeof(bufor_proces)) != sizeof(bufor_proces)) { perror("blad przy zapisie pid do pliku w p0"); exit(1); } close(des_pid[W]); proces0(); } else if(p0 == -1) { perror("blad przy p0 fork w main"); exit(1); } else { if(!(p1 = fork())) { des_pid[W] = open("pid1", O_WRONLY | O_TRUNC | O_CREAT); // 1 - zapis, 0 - odczyt sprintf(bufor_proces, "Proces1 ma ID: %d\n", getpid()); if(write(des_pid[W], bufor_proces, sizeof(bufor_proces)) != sizeof(bufor_proces)) { perror("blad przy zapisie pid do pliku w p1"); exit(1); } close(des_pid[W]); proces1(); } else if(p1 == -1) { perror("blad przy p1 fork w main"); exit(1); } else { if(!(p2 = fork())) { des_pid[W] = open("pid2", O_WRONLY | O_TRUNC | O_CREAT); // 1 - zapis, 0 - odczyt sprintf(bufor_proces, "Proces2 ma ID: %d\n", getpid()); if(write(des_pid[W], bufor_proces, sizeof(bufor_proces)) != sizeof(bufor_proces)) { perror("blad przy zapisie pid do pliku w p2"); exit(1); } close(des_pid[W]); proces2(); } else if(p2 == -1) { perror("blad przy p2 fork w main"); exit(1); } else { if(!(p3 = fork())) { des_pid[W] = open("pid3", O_WRONLY | O_TRUNC | O_CREAT); // 1 - zapis, 0 - odczyt sprintf(bufor_proces, "Proces3 ma ID: %d\n", getpid()); if(write(des_pid[W], bufor_proces, sizeof(bufor_proces)) != sizeof(bufor_proces)) { perror("blad przy zapisie pid do pliku w p3"); exit(1); } close(des_pid[W]); proces3(); } else if(p3 == -1) { perror("blad przy p3 fork w main"); exit(1); } else { // proces macierzysty waitpid(p0, &status, 0); waitpid(p1, &status, 0); waitpid(p2, &status, 0); waitpid(p3, &status, 0); //wait(NULL); unlink("/tmp/fifo"); shmdt(tab); // odlaczenie pamieci dzielonej shmctl(shmid, IPC_RMID, NULL); // usuwanie pamieci wspoldzielonej printf("\nKONIEC PROGRAMU\n"); } } } } exit(0); }

  Read the article

< Previous Page | 70 71 72 73 74