Search Results

Search found 438 results on 18 pages for 'c (c2 0)'.

Page 5/18 | < Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >

• django-mptt: how to successfully move nodes around

  - by Parand

  django-mptt seems determined to drive me out of my mind. I'm trying to do something relatively simple: I'm going to delete a node, and need to do something reasonable with the node's children. Namely, I'd like to move them up one level so they're children of their current parent's parent. That is, if the tree looks like: Root | Grandpa | Father | | C1 C2 I'm going to delete Father, and would like C1 and C2 to be children of Grandpa. Here's the code I'm using: class Node(models.Model): first_name = models.CharField(max_length=80, blank=True) parent = models.ForeignKey('self', null=True, blank=True, related_name='children') def reparent_children(self, parent): print "Reparenting" for child in self.get_children(): print "Working on", child.first_name, "to parent", parent.email parent = Node.objects.get(id=parent.id) child.move_to(parent, 'last-child') child.save() So I'd call: father.reparent_children(grandpa) father.parent = None father.save() This works - almost. The children report their parents as Grandpa: c1.parent == grandpa # True Grandpa counts C1 and C2 among its children c1 in grandpa.children.all() # True However, Root disowns these kids. c1.get_root() == father # c1's root is father, instead of Root c1 in root.get_descendants() # False How do I get the children to move and their root not get corrupted?

  Read the article

• problem getting info from a cookie with javascript

  - by Jason

  I am having an issue with my cookies and I can't figure it out. Basically I have it set up so it checks for the cookie to see if the user is logged in, and then displays either a welcome message or a login link. It works - except that instead of returning the persons name in the welcome message it just is blank where the name should be. The cookie is there, with all the appropriate info.. not sure what I am doing wrong. var itm = new Array(); itm[0] = findCookie("ui"); if (itm[0] == null) { document.write("<h2><a href='logreg.html'>Log In or Sign Up</a></h2>"); } else { var c1 = itm[0].indexOf(","); var c2 = itm[0].indexOf(",",c1); var c3 = itm[0].indexOf(",",c2); var gname = itm[0].substring(c2,c3); document.write("<h2>Welcome "+gname+"!</h2>"); } The findCookie function is.. function findCookie(val){ var cookie = null; var findVal = val + "="; var dc = document.cookie; if (dc.length > 0) { var start = dc.indexOf(findVal); if (start >= 0) { start += findVal.length; lastVal = dc.indexOf(";", start); if (lastVal == -1) { lastVal = dc.length; } cookie = (dc.substring(start, lastVal)); } else { return cookie; } } return cookie; }

  Read the article

• "select * from table" vs "select colA,colB,etc from table" interesting behaviour in SqlServer2005

  - by kristof

  Apology for a lengthy post but I needed to post some code to illustrate the problem. Inspired by the question What is the reason not to use select * ? posted a few minutes ago, I decided to point out some observations of the select * behaviour that I noticed some time ago. So let's the code speak for itself: IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[starTest]') AND type in (N'U')) DROP TABLE [dbo].[starTest] CREATE TABLE [dbo].[starTest]( [id] [int] IDENTITY(1,1) NOT NULL, [A] [varchar](50) NULL, [B] [varchar](50) NULL, [C] [varchar](50) NULL ) ON [PRIMARY] GO insert into dbo.starTest(a,b,c) select 'a1','b1','c1' union all select 'a2','b2','c2' union all select 'a3','b3','c3' go IF EXISTS (SELECT * FROM sys.views WHERE object_id = OBJECT_ID(N'[dbo].[vStartest]')) DROP VIEW [dbo].[vStartest] go create view dbo.vStartest as select * from dbo.starTest go go IF EXISTS (SELECT * FROM sys.views WHERE object_id = OBJECT_ID(N'[dbo].[vExplicittest]')) DROP VIEW [dbo].[vExplicittest] go create view dbo.[vExplicittest] as select a,b,c from dbo.starTest go select a,b,c from dbo.vStartest select a,b,c from dbo.vExplicitTest IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[starTest]') AND type in (N'U')) DROP TABLE [dbo].[starTest] CREATE TABLE [dbo].[starTest]( [id] [int] IDENTITY(1,1) NOT NULL, [A] [varchar](50) NULL, [B] [varchar](50) NULL, [D] [varchar](50) NULL, [C] [varchar](50) NULL ) ON [PRIMARY] GO insert into dbo.starTest(a,b,d,c) select 'a1','b1','d1','c1' union all select 'a2','b2','d2','c2' union all select 'a3','b3','d3','c3' select a,b,c from dbo.vExplicittest select a,b,c from dbo.vStartest If you execute the following query and look at the results of last 2 select statements, the results that you will see will be as follows: select a,b,c from dbo.vExplicittest a1 b1 c1 a2 b2 c2 a3 b3 c3 select a,b,c from dbo.vStartest a1 b1 d1 a2 b2 d2 a3 b3 d3 As you can see in the results of select a,b,c from dbo.vStartest the data of column c has been replaced with the data from colum d. I believe that is related to the way the views are compiled, my understanding is that the columns are mapped by column indexes (1,2,3,4) as apposed to names. I though I would post it as a warning for people using select * in their sql and experiencing unexpected behaviour. Note: If you rebuild the view that uses select * each time after you modify the table it will work as expected

  Read the article

• Is the salt contained in a phpass hash or do you need to salt its input?

  - by Exception e

  phpass is a widely used hashing 'framework'. Is it good practice to salt the plain password before giving it to PasswordHash (v0.2), like so?: $dynamicSalt = $record['salt']; $staticSalt = 'i5ininsfj5lt4hbfduk54fjbhoxc80sdf'; $plainPassword = $_POST['password']; $password = $plainPassword . $dynamicSalt . $staticSalt; $passwordHash = new PasswordHash(8, false); $storedPassword = $passwordHash->HashPassword($password); For reference the phpsalt class: # Portable PHP password hashing framework. # # Version 0.2 / genuine. # # Written by Solar Designer <solar at openwall.com> in 2004-2006 and placed in # the public domain. # # # class PasswordHash { var $itoa64; var $iteration_count_log2; var $portable_hashes; var $random_state; function PasswordHash($iteration_count_log2, $portable_hashes) { $this->itoa64 = './0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'; if ($iteration_count_log2 < 4 || $iteration_count_log2 > 31) $iteration_count_log2 = 8; $this->iteration_count_log2 = $iteration_count_log2; $this->portable_hashes = $portable_hashes; $this->random_state = microtime() . getmypid(); } function get_random_bytes($count) { $output = ''; if (is_readable('/dev/urandom') && ($fh = @fopen('/dev/urandom', 'rb'))) { $output = fread($fh, $count); fclose($fh); } if (strlen($output) < $count) { $output = ''; for ($i = 0; $i < $count; $i += 16) { $this->random_state = md5(microtime() . $this->random_state); $output .= pack('H*', md5($this->random_state)); } $output = substr($output, 0, $count); } return $output; } function encode64($input, $count) { $output = ''; $i = 0; do { $value = ord($input[$i++]); $output .= $this->itoa64[$value & 0x3f]; if ($i < $count) $value |= ord($input[$i]) << 8; $output .= $this->itoa64[($value >> 6) & 0x3f]; if ($i++ >= $count) break; if ($i < $count) $value |= ord($input[$i]) << 16; $output .= $this->itoa64[($value >> 12) & 0x3f]; if ($i++ >= $count) break; $output .= $this->itoa64[($value >> 18) & 0x3f]; } while ($i < $count); return $output; } function gensalt_private($input) { $output = '$P$'; $output .= $this->itoa64[min($this->iteration_count_log2 + ((PHP_VERSION >= '5') ? 5 : 3), 30)]; $output .= $this->encode64($input, 6); return $output; } function crypt_private($password, $setting) { $output = '*0'; if (substr($setting, 0, 2) == $output) $output = '*1'; if (substr($setting, 0, 3) != '$P$') return $output; $count_log2 = strpos($this->itoa64, $setting[3]); if ($count_log2 < 7 || $count_log2 > 30) return $output; $count = 1 << $count_log2; $salt = substr($setting, 4, 8); if (strlen($salt) != 8) return $output; # We're kind of forced to use MD5 here since it's the only # cryptographic primitive available in all versions of PHP # currently in use. To implement our own low-level crypto # in PHP would result in much worse performance and # consequently in lower iteration counts and hashes that are # quicker to crack (by non-PHP code). if (PHP_VERSION >= '5') { $hash = md5($salt . $password, TRUE); do { $hash = md5($hash . $password, TRUE); } while (--$count); } else { $hash = pack('H*', md5($salt . $password)); do { $hash = pack('H*', md5($hash . $password)); } while (--$count); } $output = substr($setting, 0, 12); $output .= $this->encode64($hash, 16); return $output; } function gensalt_extended($input) { $count_log2 = min($this->iteration_count_log2 + 8, 24); # This should be odd to not reveal weak DES keys, and the # maximum valid value is (2**24 - 1) which is odd anyway. $count = (1 << $count_log2) - 1; $output = '_'; $output .= $this->itoa64[$count & 0x3f]; $output .= $this->itoa64[($count >> 6) & 0x3f]; $output .= $this->itoa64[($count >> 12) & 0x3f]; $output .= $this->itoa64[($count >> 18) & 0x3f]; $output .= $this->encode64($input, 3); return $output; } function gensalt_blowfish($input) { # This one needs to use a different order of characters and a # different encoding scheme from the one in encode64() above. # We care because the last character in our encoded string will # only represent 2 bits. While two known implementations of # bcrypt will happily accept and correct a salt string which # has the 4 unused bits set to non-zero, we do not want to take # chances and we also do not want to waste an additional byte # of entropy. $itoa64 = './ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'; $output = '$2a$'; $output .= chr(ord('0') + $this->iteration_count_log2 / 10); $output .= chr(ord('0') + $this->iteration_count_log2 % 10); $output .= '$'; $i = 0; do { $c1 = ord($input[$i++]); $output .= $itoa64[$c1 >> 2]; $c1 = ($c1 & 0x03) << 4; if ($i >= 16) { $output .= $itoa64[$c1]; break; } $c2 = ord($input[$i++]); $c1 |= $c2 >> 4; $output .= $itoa64[$c1]; $c1 = ($c2 & 0x0f) << 2; $c2 = ord($input[$i++]); $c1 |= $c2 >> 6; $output .= $itoa64[$c1]; $output .= $itoa64[$c2 & 0x3f]; } while (1); return $output; } function HashPassword($password) { $random = ''; if (CRYPT_BLOWFISH == 1 && !$this->portable_hashes) { $random = $this->get_random_bytes(16); $hash = crypt($password, $this->gensalt_blowfish($random)); if (strlen($hash) == 60) return $hash; } if (CRYPT_EXT_DES == 1 && !$this->portable_hashes) { if (strlen($random) < 3) $random = $this->get_random_bytes(3); $hash = crypt($password, $this->gensalt_extended($random)); if (strlen($hash) == 20) return $hash; } if (strlen($random) < 6) $random = $this->get_random_bytes(6); $hash = $this->crypt_private($password, $this->gensalt_private($random)); if (strlen($hash) == 34) return $hash; # Returning '*' on error is safe here, but would _not_ be safe # in a crypt(3)-like function used _both_ for generating new # hashes and for validating passwords against existing hashes. return '*'; } function CheckPassword($password, $stored_hash) { $hash = $this->crypt_private($password, $stored_hash); if ($hash[0] == '*') $hash = crypt($password, $stored_hash); return $hash == $stored_hash; } }

  Read the article

• Quick MEF + SL4 question

  - by Tom Allen

  I'm working on an app in the Silverlight 4 RC and i'm taking the oppertunity to learn MEF for handling plugin controls. I've got it working in a pretty basic manor, but it's not exactly tidy and I know there is a better way of importing multiple xap's. Essentially, in the App.xaml of my host app, I've got the following telling MEF to load my xap's: AggregateCatalog catalog = new AggregateCatalog(); DeploymentCatalog c1 = new DeploymentCatalog(new Uri("TestPlugInA.xap", UriKind.Relative)); DeploymentCatalog c2 = new DeploymentCatalog(new Uri("TestPlugInB.xap", UriKind.Relative)); catalog.Catalogs.Add(c1); catalog.Catalogs.Add(c2); CompositionHost.Initialize(catalog); c1.DownloadAsync(); c2.DownloadAsync(); I'm sure I'm not using the AggregateCatalog fully here and I need to be able to load any xap's that might be in the directory, not just hardcoding Uri's obviously.... Also, in the MainPage.xaml.cs in the host I have the following collection which MEF puts the plugin's into: [ImportMany(AllowRecomposition = true)] public ObservableCollection<IPlugInApp> PlugIns { get; set; } Again, this works, but I'm pretty sure I'm using ImportMany incorrectly.... Finally, the MainPage.xaml.cs file implements IPartImportsSatisfiedNotification and I have the following for handling the plugin's once loaded: public void OnImportsSatisfied() { sp.Children.Clear(); foreach (IPlugInApp plugIn in PlugIns) { if (plugIn != null) sp.Children.Add(plugIn.GetUserControl()); } } This works, but it seems filthy that it runs n times (n being the number of xap's to load). I'm having to call sp.Children.Clear() as if I don't, when loading the 2 plugin's, my stack panel is populated as follows: TestPlugIn A TestPlugIn A TestPlugIn B I'm clearly missing something here. Can anyone point out what I should be doing? Thanks!

  Read the article

• How can * be a safe hashed password?

  - by Exception e

  phpass is a widely used hashing 'framework'. While evaluating phpass' HashPassword I came across this odd method fragment. function HashPassword($password) { // <snip> trying to generate a hash… # Returning '*' on error is safe here, but would _not_ be safe # in a crypt(3)-like function used _both_ for generating new # hashes and for validating passwords against existing hashes. return '*'; } This is the complete phpsalt class: # Portable PHP password hashing framework. # # Version 0.2 / genuine. # # Written by Solar Designer <solar at openwall.com> in 2004-2006 and placed in # the public domain. # # # class PasswordHash { var $itoa64; var $iteration_count_log2; var $portable_hashes; var $random_state; function PasswordHash($iteration_count_log2, $portable_hashes) { $this->itoa64 = './0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'; if ($iteration_count_log2 < 4 || $iteration_count_log2 > 31) $iteration_count_log2 = 8; $this->iteration_count_log2 = $iteration_count_log2; $this->portable_hashes = $portable_hashes; $this->random_state = microtime() . getmypid(); } function get_random_bytes($count) { $output = ''; if (is_readable('/dev/urandom') && ($fh = @fopen('/dev/urandom', 'rb'))) { $output = fread($fh, $count); fclose($fh); } if (strlen($output) < $count) { $output = ''; for ($i = 0; $i < $count; $i += 16) { $this->random_state = md5(microtime() . $this->random_state); $output .= pack('H*', md5($this->random_state)); } $output = substr($output, 0, $count); } return $output; } function encode64($input, $count) { $output = ''; $i = 0; do { $value = ord($input[$i++]); $output .= $this->itoa64[$value & 0x3f]; if ($i < $count) $value |= ord($input[$i]) << 8; $output .= $this->itoa64[($value >> 6) & 0x3f]; if ($i++ >= $count) break; if ($i < $count) $value |= ord($input[$i]) << 16; $output .= $this->itoa64[($value >> 12) & 0x3f]; if ($i++ >= $count) break; $output .= $this->itoa64[($value >> 18) & 0x3f]; } while ($i < $count); return $output; } function gensalt_private($input) { $output = '$P$'; $output .= $this->itoa64[min($this->iteration_count_log2 + ((PHP_VERSION >= '5') ? 5 : 3), 30)]; $output .= $this->encode64($input, 6); return $output; } function crypt_private($password, $setting) { $output = '*0'; if (substr($setting, 0, 2) == $output) $output = '*1'; if (substr($setting, 0, 3) != '$P$') return $output; $count_log2 = strpos($this->itoa64, $setting[3]); if ($count_log2 < 7 || $count_log2 > 30) return $output; $count = 1 << $count_log2; $salt = substr($setting, 4, 8); if (strlen($salt) != 8) return $output; # We're kind of forced to use MD5 here since it's the only # cryptographic primitive available in all versions of PHP # currently in use. To implement our own low-level crypto # in PHP would result in much worse performance and # consequently in lower iteration counts and hashes that are # quicker to crack (by non-PHP code). if (PHP_VERSION >= '5') { $hash = md5($salt . $password, TRUE); do { $hash = md5($hash . $password, TRUE); } while (--$count); } else { $hash = pack('H*', md5($salt . $password)); do { $hash = pack('H*', md5($hash . $password)); } while (--$count); } $output = substr($setting, 0, 12); $output .= $this->encode64($hash, 16); return $output; } function gensalt_extended($input) { $count_log2 = min($this->iteration_count_log2 + 8, 24); # This should be odd to not reveal weak DES keys, and the # maximum valid value is (2**24 - 1) which is odd anyway. $count = (1 << $count_log2) - 1; $output = '_'; $output .= $this->itoa64[$count & 0x3f]; $output .= $this->itoa64[($count >> 6) & 0x3f]; $output .= $this->itoa64[($count >> 12) & 0x3f]; $output .= $this->itoa64[($count >> 18) & 0x3f]; $output .= $this->encode64($input, 3); return $output; } function gensalt_blowfish($input) { # This one needs to use a different order of characters and a # different encoding scheme from the one in encode64() above. # We care because the last character in our encoded string will # only represent 2 bits. While two known implementations of # bcrypt will happily accept and correct a salt string which # has the 4 unused bits set to non-zero, we do not want to take # chances and we also do not want to waste an additional byte # of entropy. $itoa64 = './ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789'; $output = '$2a$'; $output .= chr(ord('0') + $this->iteration_count_log2 / 10); $output .= chr(ord('0') + $this->iteration_count_log2 % 10); $output .= '$'; $i = 0; do { $c1 = ord($input[$i++]); $output .= $itoa64[$c1 >> 2]; $c1 = ($c1 & 0x03) << 4; if ($i >= 16) { $output .= $itoa64[$c1]; break; } $c2 = ord($input[$i++]); $c1 |= $c2 >> 4; $output .= $itoa64[$c1]; $c1 = ($c2 & 0x0f) << 2; $c2 = ord($input[$i++]); $c1 |= $c2 >> 6; $output .= $itoa64[$c1]; $output .= $itoa64[$c2 & 0x3f]; } while (1); return $output; } function HashPassword($password) { $random = ''; if (CRYPT_BLOWFISH == 1 && !$this->portable_hashes) { $random = $this->get_random_bytes(16); $hash = crypt($password, $this->gensalt_blowfish($random)); if (strlen($hash) == 60) return $hash; } if (CRYPT_EXT_DES == 1 && !$this->portable_hashes) { if (strlen($random) < 3) $random = $this->get_random_bytes(3); $hash = crypt($password, $this->gensalt_extended($random)); if (strlen($hash) == 20) return $hash; } if (strlen($random) < 6) $random = $this->get_random_bytes(6); $hash = $this->crypt_private($password, $this->gensalt_private($random)); if (strlen($hash) == 34) return $hash; # Returning '*' on error is safe here, but would _not_ be safe # in a crypt(3)-like function used _both_ for generating new # hashes and for validating passwords against existing hashes. return '*'; } function CheckPassword($password, $stored_hash) { $hash = $this->crypt_private($password, $stored_hash); if ($hash[0] == '*') $hash = crypt($password, $stored_hash); return $hash == $stored_hash; } }

  Read the article

• What is the best way to automatically transpose a LilyPond source file into multiple keys?

  - by Michael Steele

  problem I'm using LilyPond to typeset sheet music for a church choir to perform. Depending on who is available on any given week, songs will be played in various keys. We have an amazing pianist who can play anything we throw at her and the guitarists will typically pencil in alternate chords, but I want to make things easier by having beautifully typeset sheet music available in any key we want. So say we're going to sing our ABCs. First I'll take whatever source transcriptions available and enter it into a LilyPond script: melody = \relative c' { c c g g a a g2 f f e e d d c2 } I want the ability to transpose this automatically, so if I want the whole thing in 'G' I wrap the song in a \transpose call like so: melody = \transpose c g \relative c' { c c g g a a g2 f f e e d d c2 } What I really want is to substitute something for the 'g' and generate the output for melody multiple times. Simple LilyPond variables don't seem to work here, and so far I've been unsuccessful in defining a scheme function to do this. What I've resorted to for the moment is taking the above file, call it twinkle.ly and turning it into an M4 script called twinkle.ly.m4, the contents of which look like this: melody = \transpose c _key \relative c' { c c g g a a g2 f f e e d d c2 } I then compile the while thing by executing the following line: > m4 -D _key=g twinkle.ly.m4 > twinkle_g.ly && lilypond twinkle_g.ly I've written a Makefile to do this for me, defining rules for every song I have and every key I'm interested in. question There's got to be a better way of going about this. Given that Lilypond supports embedded scheme, I would prefer to not use a macro preprocessed on it. Has anybody else come up with a solution to this same problem?

  Read the article

• Java Reflection, java.lang.IllegalAccessException Error

  - by rubby

  Hi all, My goal is : Third class will read the name of the class as String from console.Upon reading the name of the class, it will automatically and dynamically (!) generate that class and call its writeout method. If that class is not read from input, it will not be initialized. And I am taking java.lang.IllegalAccessException: Class deneme.class3 can not access a member of class java.lang.Object with modifiers "protected" error. And i don't know how i can fix it.. Can anyone help me? import java.io.*; import java.lang.reflect.*; class class3 { public void run() { try { BufferedReader reader= new BufferedReader(new InputStreamReader(System.in)); String line=reader.readLine(); Class c1 = Class.forName("java.lang.String"); Object obj = new Object(); Class c2 = obj.getClass(); Method writeout = null; for( Method mth : c2.getDeclaredMethods() ) { writeout = mth; break; } Object o = c2.newInstance(); writeout.invoke( o ); } catch(Exception ee) { System.out.println("ERROR! "+ee); } } public void writeout3() { System.out.println("class3"); } } class class4 { public void writeout4() { System.out.println("class4"); } } class ornek { public static void main(String[] args) { System.out.println("Please Enter Name of Class : "); class3 example = new class3(); example.run(); } }

  Read the article

• Why execution of a portion of code loaded from external file is not halted by the OS?

  - by menjaraz

  I've harnessed a project released on internet a long time ago. Here comes the details, all irrelevant things being stripped off for sake of concision and clarity. A binary file whose content is descibed below HEX DUMP: 55 89 E5 83 EC 08 C7 45 FC 00 00 00 00 8B 45 FC 3B 45 10 72 02 EB 19 8B 45 FC 8B 55 0C 01 C2 8B 45 FC 03 45 08 8A 00 88 02 8D 45 FC FF 00 EB DD C6 45 FA 00 83 7D 10 01 76 6C 80 7D FA 00 74 02 EB 64 C6 45 FA 01 C7 45 FC 00 00 00 00 8B 45 10 48 39 45 FC 72 02 EB E2 8B 45 FC 8B 4D 0C 01 C1 8B 45 FC 03 45 0C 8D 50 01 8A 01 3A 02 73 30 8B 45 FC 03 45 0C 8A 00 88 45 FB 8B 45 FC 8B 55 0C 01 C2 8B 45 FC 03 45 0C 40 8A 00 88 02 8B 45 FC 03 45 0C 8D 50 01 8A 45 FB 88 02 C6 45 FA 00 8D 45 FC FF 00 EB A7 C9 C2 0C 00 90 90 90 90 90 90 is loaded into memory and executed using the following method snippet var MySrcArray, MyDestArray: array [1 .. 15] of Byte; // ... MyBuffer: Pointer; TheProc: procedure; SortIt: procedure(ASrc, ADest: Pointer; ASize: LongWord); stdcall; begin // Initialization of MySrcArray with random Bytes and display here ... // Instructions of loading of the binary file into MyBuffer using merely **GetMem** here ... @SortIt := MyBuffer; try SortIt(@MySrcArray, @MyDestArray, 15); // Display of MyDestArray (The outcome of the processing !) except // Invalid code error handling end; // Cleaning code here ... end; works like a charm on my box. My Question: How comes it works without using VirtualAlloc and/or VirtualProtect?

  Read the article

• activemessaging with stomp and activemq.prefetchSize=1

  - by Clint Miller

  I have a situation where I have a single activemq broker with 2 queues, Q1 and Q2. I have two ruby-based consumers using activemessaging. Let's call them C1 and C2. Both consumers subscribe to each queue. I'm setting activemq.prefetchSize=1 when subscribing to each queue. I'm also setting ack=client. Consider the following sequence of events: 1) A message that triggers a long-running job is published to queue Q1. Call this M1. 2) M1 is dispatched to consumer C1, kicking off a long operation. 3) Two messages that trigger short jobs are published to queue Q2. Call these M2 and M3. 4) M2 is dispatched to C2 which quickly runs the short job. 5) M3 is dispatched to C1, even though C1 is still running M1. It's able to dispatch to C1 because prefetchSize=1 is set on the queue subscription, not on the connection. So the fact that a Q1 message has already been dispatched doesn't stop one Q2 message from being dispatched. Since activemessaging consumers are single-threaded, the net result is that M3 sits and waits on C1 for a long time until C1 finishes processing M1. So, M3 is not processed for a long time, despite the fact that consumer C2 is sitting idle (since it quickly finishes with message M2). Essentially, whenever a long Q1 job is run and then a whole bunch of short Q2 jobs are created, exactly one of the short Q2 jobs gets stuck on a consumer waiting for the long Q1 job to finish. Is there a way to set prefetchSize at the connection level rather than at the subscription level? I really don't want any messages dispatched to C1 while it is processing M1. The other alternative is that I could create a consumer dedicated to processing Q1 and then have other consumers dedicated to processing Q2. But, I'd rather not do that since Q1 messages are infrequent--Q1's dedicated consumers would sit idle most of the day tying up memory.

  Read the article

• Generic callbacks

  - by bobobobo

  Extends So, I'm trying to learn template metaprogramming better and I figure this is a good exercise for it. I'm trying to write code that can callback a function with any number of arguments I like passed to it. // First function to call int add( int x, int y ) ; // Second function to call double square( double x ) ; // Third func to call void go() ; The callback creation code should look like: // Write a callback object that // will be executed after 42ms for "add" Callback<int, int, int> c1 ; c1.func = add ; c1.args.push_back( 2 ); // these are the 2 args c1.args.push_back( 5 ); // to pass to the "add" function // when it is called Callback<double, double> c2 ; c2.func = square ; c2.args.push_back( 52.2 ) ; What I'm thinking is, using template metaprogramming I want to be able to declare callbacks like, write a struct like this (please keep in mind this is VERY PSEUDOcode) <TEMPLATING ACTION <<ANY NUMBER OF TYPES GO HERE>> > struct Callback { double execTime ; // when to execute TYPE1 (*func)( TYPE2 a, TYPE3 b ) ; void* argList ; // a stored list of arguments // to plug in when it is time to call __func__ } ; So for when called with Callback<int, int, int> c1 ; You would automatically get constructed for you by < HARDCORE TEMPLATING ACTION > a struct like struct Callback { double execTime ; // when to execute int (*func)( int a, int b ) ; void* argList ; // this would still be void*, // but I somehow need to remember // the types of the args.. } ; Any pointers in the right direction to get started on writing this?

  Read the article

• My SQL query is only returning I need the parent aswell

  - by sico87

  My sql query is only returning the children of the parent I need it to return the parent as well, public function getNav($cat,$subcat){ //gets all sub categories for a specific category if(!$this->checkValue($cat)) return false; //checks data $query = false; if($cat=='NULL'){ $sql = "SELECT itemID, title, parent, url, description, image FROM p_cat WHERE deleted = 0 AND parent is NULL ORDER BY position;"; $query = $this->db->query($sql) or die($this->db->error); }else{ //die($cat); $sql = "SET @parent = (SELECT c.itemID FROM p_cat c WHERE url = '".$this->sql($cat)."' AND deleted = 0); SELECT c1.itemID, c1.title, c1.parent, c1.url, c1.description, c1.image, (SELECT c2.url FROM p_cat c2 WHERE c2.itemID = c1.parent LIMIT 1) as parentUrl FROM p_cat c1 WHERE c1.deleted = 0 AND c1.parent = @parent ORDER BY c1.position;"; $query = $this->db->multi_query($sql) or die($this->db->error); $this->db->store_result(); $this->db->next_result(); $query = $this->db->store_result(); } return $query; }

  Read the article

• Selecting date NOT NULL records between a specific range with Propel

  - by Jon Winstanley

  Using Propel I would like to find records which have a date field which is not null and also between a specific range. However, Propel seems to overwrite the criteria with the NOTNULL criteria. Is it possible to do this? //create the date ranges $start_date = mktime(0, 0, 0, date("m") , date("d")+$start, date("Y")); $end_date = mktime(0, 0, 0, date("m") , date("d")+$end, date("Y")); //add the start of the range $c1 = $c->getNewCriterion(TaskPeer::DUE_DATE, null); $c1->addAnd($c->getNewCriterion(TaskPeer::DUE_DATE, $end_date, Criteria::LESS_EQUAL)); $c->add($c1); //add the end of the range $c2 = $c->getNewCriterion(TaskPeer::DUE_DATE, null); $c2->addAnd($c->getNewCriterion(TaskPeer::DUE_DATE, $start_date, Criteria::GREATER_EQUAL)); $c->add($c2); //remove the null entries $c3 = $c->getNewCriterion(TaskPeer::DUE_DATE, null); $c3->addAnd($c->getNewCriterion(TaskPeer::DUE_DATE, null, Criteria::ISNULL)); $c->add($c3);

  Read the article

• problem with "select new Object ... join ... where"

  - by jacob

  Hi, I'm having a problem with an HQL query Three classes ClassOne is my BusinessObject public class ClassOne { private int id; private int status; private Set<ClassTwo> classTwos; + other fields/getters/setters/constructor etc } ClassTwo is referenced in a set of ClassOne and is kind of the history of an object of ClassOne public class ClassTwo { private int id; private int oldStatus; private int newStatus; private String message; //+ getters/setters/constructor etc } ClassThree is my DTO/VO with just one classTwo (not the whole history) public class ClassThree { private int id; private int status; private ClassTwo classTwo; public ClassThree(int pId, int pStatus, ClassTwo pClassTwo) { id=pId; status=pStatus; classTwo=pClassTwo; } //+ getters/setters etc } Now I'd like to create an HQL query like this: I'd like to get all objects of ClassThree with a certain status and if it exists the newest ClassTwo with a certain newStatus. For example: I'd like to get all the DTOs (ClassThree) of ClassOne whose status is now 1, but earlier in their history it has been 2 and I'd like to have the latest ClassTwo object which has 2 as newStatus. SELECT new ClassThree(c1.id, c1.status, c2) FROM ClassOne c1 LEFT JOIN c1.classtwos c2 (...) and (...) is where I don't know what to do, I'm not even sure if it's a join / join fetch Looked around and tried quite a lot already, but no clue. Especially with the join fetch I get some Hibernate errors like org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list. Fetching the BusinessObject like that is no problem SELECT distinct(c1) FROM ClassOne c1 LEFT OUTER JOIN FETCH c1.classtwos c2 and I get the ClassTwos as my field. Thanks in advance, Jacob P.S.: One thing might be important, ClassTwo has no reference to ClassOne!!

  Read the article

• Help needed for writing a Set Based query for finding the highest marks obtained by the students

  - by priyanka.sarkar_2

  I have the below table declare @t table (id int identity, name varchar(50),sub1 int,sub2 int,sub3 int,sub4 int) insert into @t select 'name1',20,30,40,50 union all select 'name2',10,30,40,50 union all select 'name3',40,60,100,50 union all select 'name4',80,30,40,80 union all select 'name5',80,70,40,50 union all select 'name6',10,30,40,80 The desired output should be Id Name Sub1 Sub2 Sub3 Sub4 3 Name3 100 4 Name4 80 80 5 Name5 80 70 6 Name6 80 What I have done so far is ;with cteSub1 as ( select rn1 = dense_rank() over(order by sub1 desc),t.id,t.name,t.sub1 from @t t ) ,cteSub2 as ( select rn2 = dense_rank() over(order by sub2 desc),t.id,t.name,t.sub2 from @t t ) ,cteSub3 as ( select rn3 = dense_rank() over(order by sub3 desc),t.id,t.name,t.sub3 from @t t ) ,cteSub4 as ( select rn4 = dense_rank() over(order by sub4 desc),t.id,t.name,t.sub4 from @t t ) select x1.id,x2.id,x3.id,x4.id ,x1.sub1,x2.sub2,x3.sub3,x4.sub4 from (select c1.id,c1.sub1 from cteSub1 c1 where rn1 =1) as x1 full join (select c2.id,c2.sub2 from cteSub2 c2 where rn2 =1)x2 on x1.id = x2.id full join (select c3.id,c3.sub3 from cteSub3 c3 where rn3 =1)x3 on x1.id = x3.id full join (select c4.id,c4.sub4 from cteSub4 c4 where rn4 =1)x4 on x1.id = x4.id which is giving me the output as id id id id sub1 sub2 sub3 sub4 5 5 NULL NULL 80 70 NULL NULL 4 NULL NULL 4 80 NULL NULL 80 NULL NULL 3 NULL NULL NULL 100 NULL NULL NULL NULL 6 NULL NULL NULL 80 Help needed. Also how can I reduce the number of CTE's?

  Read the article

• 3Ware 9650SE RAID-6, two degraded drives, one ECC, rebuild stuck

  - by cswingle

  This morning I came in the office to discover that two of the drives on a RAID-6, 3ware 9650SE controller were marked as degraded and it was rebuilding the array. After getting to about 4%, it got ECC errors on a third drive (this may have happened when I attempted to access the filesystem on this RAID and got I/O errors from the controller). Now I'm in this state: > /c2/u1 show Unit UnitType Status %RCmpl %V/I/M Port Stripe Size(GB) ------------------------------------------------------------------------ u1 RAID-6 REBUILDING 4%(A) - - 64K 7450.5 u1-0 DISK OK - - p5 - 931.312 u1-1 DISK OK - - p2 - 931.312 u1-2 DISK OK - - p1 - 931.312 u1-3 DISK OK - - p4 - 931.312 u1-4 DISK OK - - p11 - 931.312 u1-5 DISK DEGRADED - - p6 - 931.312 u1-6 DISK OK - - p7 - 931.312 u1-7 DISK DEGRADED - - p3 - 931.312 u1-8 DISK WARNING - - p9 - 931.312 u1-9 DISK OK - - p10 - 931.312 u1/v0 Volume - - - - - 7450.5 Examining the SMART data on the three drives in question, the two that are DEGRADED are in good shape (PASSED without any Current_Pending_Sector or Offline_Uncorrectable errors), but the drive listed as WARNING has 24 uncorrectable sectors. And, the "rebuild" has been stuck at 4% for ten hours now. So: How do I get it to start actually rebuilding? This particular controller doesn't appear to support /c2/u1 resume rebuild, and the only rebuild command that appears to be an option is one that wants to know what disk to add (/c2/u1 start rebuild disk=<p:-p...> [ignoreECC] according to the help). I have two hot spares in the server, and I'm happy to engage them, but I don't understand what it would do with that information in the current state it's in. Can I pull out the drive that is demonstrably failing (the WARNING drive), when I have two DEGRADED drives in a RAID-6? It seems to me that the best scenario would be for me to pull the WARNING drive and tell it to use one of my hot spares in the rebuild. But won't I kill the thing by pulling a "good" drive in a RAID-6 with two DEGRADED drives? Finally, I've seen reference in other posts to a bad bug in this controller that causes good drives to be marked as bad and that upgrading the firmware may help. Is flashing the firmware a risky operation given the situation? Is it likely to help or hurt wrt the rebuilding-but-stuck-at-4% RAID? Am I experiencing this bug in action? Advice outside the spiritual would be much appreciated. Thanks.

  Read the article

• Issuing Current Time Increments in StreamInsight (A Practical Example)

  The issuing of a Current Time Increment, Cti, in StreamInsight is very definitely one of the most important concepts to learn if you want your Streams to be responsive. A full discussion of how to issue Ctis is beyond the scope of this article but a very good explanation in addition to Books Online can be found in these three articles by a member of the StreamInsight team at Microsoft, Ciprian Gerea. Time in StreamInsight Series http://blogs.msdn.com/b/streaminsight/archive/2010/07/23/time-in-streaminsight-i.aspx http://blogs.msdn.com/b/streaminsight/archive/2010/07/30/time-in-streaminsight-ii.aspx http://blogs.msdn.com/b/streaminsight/archive/2010/08/03/time-in-streaminsight-iii.aspx A lot of the problems I see with unresponsive or stuck streams on the MSDN Forums are to do with how Ctis are enqueued or in a lot of cases not enqueued. If you enqueue events and never enqueue a Cti then StreamInsight will be perfectly happy. You, on the other hand, will never see data on the output as you have not told StreamInsight to flush the stream. This article deals with a specific implementation problem I had recently whilst working on a StreamInsight project. I look at some possible options and discuss why they would not work before showing the way I solved the problem. The stream of data I was dealing with on this project was very bursty that is to say when events were flowing they came through very quickly and in large numbers (1000 events/sec), but when the stream calmed down it could be a few seconds between each event. When enqueuing events into the StreamInsight engne it is best practice to do so with a StartTime that is given to you by the system producing the event . StreamInsight processes events and it doesn't matter whether those events are being pushed into the engine by a source system or the events are being read from something like a flat file in a directory somewhere. You can apply the same logic and temporal algebra to both situations. Reading from a file is an excellent example of where the time of the event on the source itself is very important. We could be reading that file a long time after it was written. Being able to read the StartTime from the events allows us to define windows that will hold the correct sets of events. I was able to do this with my stream but this is where my problems started. Below is a very simple script to create a SQL Server table and populate it with sample data that will show exactly the problem I had. CREATE TABLE [dbo].[t] ( [c1] [int] PRIMARY KEY, [c2] [datetime] NULL ) INSERT t VALUES (1,'20100810'),(2,'20100810'),(3,'20100810') Column c2 defines the StartTime of the event on the source and as you can see the values in all 3 rows of data is the same. If we read Ciprian’s articles we know that we can define how Ctis get injected into the stream in 3 different places The Stream Definition The Input Factory The Input Adapter I personally have always been a fan of enqueing Ctis through the factory. Below is code typical of what I would use to do this On the class itself I do some inheriting public class SimpleInputFactory : ITypedInputAdapterFactory<SimpleInputConfig>, ITypedDeclareAdvanceTimeProperties<SimpleInputConfig> And then I implement the following function public AdapterAdvanceTimeSettings DeclareAdvanceTimeProperties<TPayload>(SimpleInputConfig configInfo, EventShape eventShape) { return new AdapterAdvanceTimeSettings( new AdvanceTimeGenerationSettings(configInfo.CtiFrequency, TimeSpan.FromTicks(-1)), AdvanceTimePolicy.Adjust); } The configInfo .CtiFrequency property is a value I pass through to define after how many events I want a Cti to be injected and this in turn will flush through the stream of data. I usually pass a value of 1 for this setting. The second parameter determines the CTI timestamp in terms of a delay relative to the events. -1 ticks in the past results in 1 tick in the future, i.e., ahead of the event. The problem with this method though is that if consecutive events have the same StartTime then only one of those events will be enqueued. In this example I use the following to define how I assign the StartTime of my events currEvent.StartTime = (DateTimeOffset)dt.c2; If I go ahead and run my StreamInsight process with this configuration i can see on the output adapter that two events have been removed To see this in a little more depth I can use the StreamInsight Debugger and see what happens internally. What is happening here is that the first event arrives and a Cti is injected with a time of 1 tick after the StartTime of that event (Also the EndTime of the event). The second event arrives and it has a StartTime of before the Cti and even though we specified AdvanceTimePolicy.Adjust on the factory we know that a point event can never be adjusted like this and the event is dropped. The same happens for the third event as well (The second and third events get trumped by the Cti). For a more detailed discussion of why this happens look here http://www.sqlis.com/sqlis/post/AdvanceTimePolicy-and-Point-Event-Streams-In-StreamInsight.aspx We end up with a single event being pushed into the output adapter and our result now makes sense. The next way I tried to solve this problem by changing the value of the second parameter to TimeSpan.Zero Here is how my factory code now looks public AdapterAdvanceTimeSettings DeclareAdvanceTimeProperties<TPayload>(SimpleInputConfig configInfo, EventShape eventShape) { return new AdapterAdvanceTimeSettings( new AdvanceTimeGenerationSettings(configInfo.CtiFrequency, TimeSpan.Zero), AdvanceTimePolicy.Adjust); } What I am doing here is declaring a policy that says inject a Cti together with every event and stamp it with a StartTime that is equal to the start time of the event itself (TimeSpan.Zero). This method has plus points as well as a downside. The upside is that no events will be lost by having the same StartTime as previous events. The Downside is that because the Cti is declared with the StartTime of the event itself then it does not actually flush that particular event because in the StreamInsight algebra, a Cti commits only those events that occurred strictly before them. To flush the events we need a Cti to be enqueued with a greater StartTime than the events themselves. Here is what happened when I ran this configuration As you can see all we got through was the Cti and none of the events. The debugger output shows the stamps on the Cti and the events themselves. Because the Cti issued has the same timestamp (StartTime) as the events then none of the events get flushed. I was nearly there but not quite. Because my stream was bursty it was possible that the next event would not come along for a few seconds and this was far too long for an event to be enqueued and not be flushed to the output adapter. I needed another solution. Two possible solutions crossed my mind although only one of them made sense when I explored it some more. Where multiple events have the same StartTime I could add 1 tick to the first event, two to the second, three to third etc thereby giving them unique StartTime values. Add a timer to manually inject Ctis The problem with the first implementation is that I would be giving the events a new StartTime. This would cause me the following problems If I want to define windows over the stream then some events may not be captured in the right windows and therefore any calculations on those windows I did would be wrong What would happen if we had 10,000 events with the same StartTime? I would enqueue them with StartTime + n ticks. Along comes a genuine event with a StartTime of the very first event + 1 tick. It is now too far in the past as far as my stream is concerned and it would be dropped. Not what I would want to do at all. I decided then to look at the Timer based solution I created a timer on my input adapter that elapsed every 200ms. private Timer tmr; public SimpleInputAdapter(SimpleInputConfig configInfo) { ctx = new SimpleTimeExtractDataContext(configInfo.ConnectionString); this.configInfo = configInfo; tmr = new Timer(200); tmr.Elapsed += new ElapsedEventHandler(t_Elapsed); tmr.Enabled = true; } void t_Elapsed(object sender, ElapsedEventArgs e) { ts = DateTime.Now - dtCtiIssued; if (ts.TotalMilliseconds >= 200 && TimerIssuedCti == false) { EnqueueCtiEvent(System.DateTime.Now.AddTicks(-100)); TimerIssuedCti = true; } }   In the t_Elapsed event handler I find out the difference in time between now and when the last event was processed (dtCtiIssued). I then check to see if that is greater than or equal to 200ms and if the last issuing of a Cti was done by the timer or by a genuine event (TimerIssuedCti). If I didn’t do this check then I would enqueue a Cti every time the timer elapsed which is not something I wanted. If the difference between the two times is greater than or equal to 500ms and the last event enqueued was by a real event then I issue a Cti through the timer to flush the event Queue, otherwise I do nothing. When I enqueue the Ctis into my stream in my ProduceEvents method I also set the values of dtCtiIssued and TimerIssuedCti   currEvent = CreateInsertEvent(); currEvent.StartTime = (DateTimeOffset)dt.c2; TimerIssuedCti = false; dtCtiIssued = currEvent.StartTime; If I go ahead and run this configuration I see the following in my output. As we can see the first Cti gets enqueued as before but then another is enqueued by the timer and because this has a later timestamp it flushes the enqueued events through the engine. Conclusion Hopefully this has shown how the enqueuing of Ctis can have a dramatic effect on the responsiveness of your output in StreamInsight. Understanding the temporal nature of the product is for me one of the most important things you can learn. I have attached my solution for the demos. It is all in one project and testing each variation is a simple matter of commenting and un-commenting the parts in the code we have been dealing with here.

  Read the article

• Running a simple integration scenario using the Oracle Big Data Connectors on Hadoop/HDFS cluster

  - by hamsun

  Between the elephant ( the tradional image of the Hadoop framework) and the Oracle Iron Man (Big Data..) an english setter could be seen as the link to the right data Data, Data, Data, we are living in a world where data technology based on popular applications , search engines, Webservers, rich sms messages, email clients, weather forecasts and so on, have a predominant role in our life. More and more technologies are used to analyze/track our behavior, try to detect patterns, to propose us "the best/right user experience" from the Google Ad services, to Telco companies or large consumer sites (like Amazon:) ). The more we use all these technologies, the more we generate data, and thus there is a need of huge data marts and specific hardware/software servers (as the Exadata servers) in order to treat/analyze/understand the trends and offer new services to the users. Some of these "data feeds" are raw, unstructured data, and cannot be processed effectively by normal SQL queries. Large scale distributed processing was an emerging infrastructure need and the solution seemed to be the "collocation of compute nodes with the data", which in turn leaded to MapReduce parallel patterns and the development of the Hadoop framework, which is based on MapReduce and a distributed file system (HDFS) that runs on larger clusters of rather inexpensive servers. Several Oracle products are using the distributed / aggregation pattern for data calculation ( Coherence, NoSql, times ten ) so once that you are familiar with one of these technologies, lets says with coherence aggregators, you will find the whole Hadoop, MapReduce concept very similar. Oracle Big Data Appliance is based on the Cloudera Distribution (CDH), and the Oracle Big Data Connectors can be plugged on a Hadoop cluster running the CDH distribution or equivalent Hadoop clusters. In this paper, a "lab like" implementation of this concept is done on a single Linux X64 server, running an Oracle Database 11g Enterprise Edition Release 11.2.0.4.0, and a single node Apache hadoop-1.2.1 HDFS cluster, using the SQL connector for HDFS. The whole setup is fairly simple: Install on a Linux x64 server ( or virtual box appliance) an Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 server Get the Apache Hadoop distribution from: http://mir2.ovh.net/ftp.apache.org/dist/hadoop/common/hadoop-1.2.1. Get the Oracle Big Data Connectors from: http://www.oracle.com/technetwork/bdc/big-data-connectors/downloads/index.html?ssSourceSiteId=ocomen. Check the java version of your Linux server with the command: java -version java version "1.7.0_40" Java(TM) SE Runtime Environment (build 1.7.0_40-b43) Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode) Decompress the hadoop hadoop-1.2.1.tar.gz file to /u01/hadoop-1.2.1 Modify your .bash_profile export HADOOP_HOME=/u01/hadoop-1.2.1 export PATH=$PATH:$HADOOP_HOME/bin export HIVE_HOME=/u01/hive-0.11.0 export PATH=$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin (also see my sample .bash_profile) Set up ssh trust for Hadoop process, this is a mandatory step, in our case we have to establish a "local trust" as will are using a single node configuration copy the new public keys to the list of authorized keys connect and test the ssh setup to your localhost: We will run a "pseudo-Hadoop cluster", in what is called "local standalone mode", all the Hadoop java components are running in one Java process, this is enough for our demo purposes. We need to "fine tune" some Hadoop configuration files, we have to go at our $HADOOP_HOME/conf, and modify the files: core-site.xml hdfs-site.xml mapred-site.xml check that the hadoop binaries are referenced correctly from the command line by executing: hadoop -version As Hadoop is managing our "clustered HDFS" file system we have to create "the mount point" and format it , the mount point will be declared to core-site.xml as: The layout under the /u01/hadoop-1.2.1/data will be created and used by other hadoop components (MapReduce = /mapred/...) HDFS is using the /dfs/... layout structure format the HDFS hadoop file system: Start the java components for the HDFS system As an additional check, you can use the GUI Hadoop browsers to check the content of your HDFS configurations: Once our HDFS Hadoop setup is done you can use the HDFS file system to store data ( big data : )), and plug them back and forth to Oracle Databases by the means of the Big Data Connectors ( which is the next configuration step). You can create / use a Hive db, but in our case we will make a simple integration of "raw data" , through the creation of an External Table to a local Oracle instance ( on the same Linux box, we run the Hadoop HDFS one node cluster and one Oracle DB). Download some public "big data", I use the site: http://france.meteofrance.com/france/observations, from where I can get *.csv files for my big data simulations :). Here is the data layout of my example file: Download the Big Data Connector from the OTN (oraosch-2.2.0.zip), unzip it to your local file system (see picture below) Modify your environment in order to access the connector libraries , and make the following test: [oracle@dg1 bin]$./hdfs_stream Usage: hdfs_stream locationFile [oracle@dg1 bin]$ Load the data to the Hadoop hdfs file system: hadoop fs -mkdir bgtest_data hadoop fs -put obsFrance.txt bgtest_data/obsFrance.txt hadoop fs -ls /user/oracle/bgtest_data/obsFrance.txt [oracle@dg1 bg-data-raw]$ hadoop fs -ls /user/oracle/bgtest_data/obsFrance.txt Found 1 items -rw-r--r-- 1 oracle supergroup 54103 2013-10-22 06:10 /user/oracle/bgtest_data/obsFrance.txt [oracle@dg1 bg-data-raw]$hadoop fs -ls hdfs:///user/oracle/bgtest_data/obsFrance.txt Found 1 items -rw-r--r-- 1 oracle supergroup 54103 2013-10-22 06:10 /user/oracle/bgtest_data/obsFrance.txt Check the content of the HDFS with the browser UI: Start the Oracle database, and run the following script in order to create the Oracle database user, the Oracle directories for the Oracle Big Data Connector (dg1 it’s my own db id replace accordingly yours): #!/bin/bash export ORAENV_ASK=NO export ORACLE_SID=dg1 . oraenv sqlplus /nolog <<EOF CONNECT / AS sysdba; CREATE OR REPLACE DIRECTORY osch_bin_path AS '/u01/orahdfs-2.2.0/bin'; CREATE USER BGUSER IDENTIFIED BY oracle; GRANT CREATE SESSION, CREATE TABLE TO BGUSER; GRANT EXECUTE ON sys.utl_file TO BGUSER; GRANT READ, EXECUTE ON DIRECTORY osch_bin_path TO BGUSER; CREATE OR REPLACE DIRECTORY BGT_LOG_DIR as '/u01/BG_TEST/logs'; GRANT READ, WRITE ON DIRECTORY BGT_LOG_DIR to BGUSER; CREATE OR REPLACE DIRECTORY BGT_DATA_DIR as '/u01/BG_TEST/data'; GRANT READ, WRITE ON DIRECTORY BGT_DATA_DIR to BGUSER; EOF Put the following in a file named t3.sh and make it executable, hadoop jar $OSCH_HOME/jlib/orahdfs.jar \ oracle.hadoop.exttab.ExternalTable \ -D oracle.hadoop.exttab.tableName=BGTEST_DP_XTAB \ -D oracle.hadoop.exttab.defaultDirectory=BGT_DATA_DIR \ -D oracle.hadoop.exttab.dataPaths="hdfs:///user/oracle/bgtest_data/obsFrance.txt" \ -D oracle.hadoop.exttab.columnCount=7 \ -D oracle.hadoop.connection.url=jdbc:oracle:thin:@//localhost:1521/dg1 \ -D oracle.hadoop.connection.user=BGUSER \ -D oracle.hadoop.exttab.printStackTrace=true \ -createTable --noexecute then test the creation fo the external table with it: [oracle@dg1 samples]$ ./t3.sh ./t3.sh: line 2: /u01/orahdfs-2.2.0: Is a directory Oracle SQL Connector for HDFS Release 2.2.0 - Production Copyright (c) 2011, 2013, Oracle and/or its affiliates. All rights reserved. Enter Database Password:] The create table command was not executed. The following table would be created. CREATE TABLE "BGUSER"."BGTEST_DP_XTAB" ( "C1" VARCHAR2(4000), "C2" VARCHAR2(4000), "C3" VARCHAR2(4000), "C4" VARCHAR2(4000), "C5" VARCHAR2(4000), "C6" VARCHAR2(4000), "C7" VARCHAR2(4000) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY "BGT_DATA_DIR" ACCESS PARAMETERS ( RECORDS DELIMITED BY 0X'0A' CHARACTERSET AL32UTF8 STRING SIZES ARE IN CHARACTERS PREPROCESSOR "OSCH_BIN_PATH":'hdfs_stream' FIELDS TERMINATED BY 0X'2C' MISSING FIELD VALUES ARE NULL ( "C1" CHAR(4000), "C2" CHAR(4000), "C3" CHAR(4000), "C4" CHAR(4000), "C5" CHAR(4000), "C6" CHAR(4000), "C7" CHAR(4000) ) ) LOCATION ( 'osch-20131022081035-74-1' ) ) PARALLEL REJECT LIMIT UNLIMITED; The following location files would be created. osch-20131022081035-74-1 contains 1 URI, 54103 bytes 54103 hdfs://localhost:19000/user/oracle/bgtest_data/obsFrance.txt Then remove the --noexecute flag and create the external Oracle table for the Hadoop data. Check the results: The create table command succeeded. CREATE TABLE "BGUSER"."BGTEST_DP_XTAB" ( "C1" VARCHAR2(4000), "C2" VARCHAR2(4000), "C3" VARCHAR2(4000), "C4" VARCHAR2(4000), "C5" VARCHAR2(4000), "C6" VARCHAR2(4000), "C7" VARCHAR2(4000) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY "BGT_DATA_DIR" ACCESS PARAMETERS ( RECORDS DELIMITED BY 0X'0A' CHARACTERSET AL32UTF8 STRING SIZES ARE IN CHARACTERS PREPROCESSOR "OSCH_BIN_PATH":'hdfs_stream' FIELDS TERMINATED BY 0X'2C' MISSING FIELD VALUES ARE NULL ( "C1" CHAR(4000), "C2" CHAR(4000), "C3" CHAR(4000), "C4" CHAR(4000), "C5" CHAR(4000), "C6" CHAR(4000), "C7" CHAR(4000) ) ) LOCATION ( 'osch-20131022081719-3239-1' ) ) PARALLEL REJECT LIMIT UNLIMITED; The following location files were created. osch-20131022081719-3239-1 contains 1 URI, 54103 bytes 54103 hdfs://localhost:19000/user/oracle/bgtest_data/obsFrance.txt This is the view from the SQL Developer: and finally the number of lines in the oracle table, imported from our Hadoop HDFS cluster SQL select count(*) from "BGUSER"."BGTEST_DP_XTAB"; COUNT(*) ---------- 1151 In a next post we will integrate data from a Hive database, and try some ODI integrations with the ODI Big Data connector. Our simplistic approach is just a step to show you how these unstructured data world can be integrated to Oracle infrastructure. Hadoop, BigData, NoSql are great technologies, they are widely used and Oracle is offering a large integration infrastructure based on these services. Oracle University presents a complete curriculum on all the Oracle related technologies: NoSQL: Introduction to Oracle NoSQL Database Using Oracle NoSQL Database Big Data: Introduction to Big Data Oracle Big Data Essentials Oracle Big Data Overview Oracle Data Integrator: Oracle Data Integrator 12c: New Features Oracle Data Integrator 11g: Integration and Administration Oracle Data Integrator: Administration and Development Oracle Data Integrator 11g: Advanced Integration and Development Oracle Coherence 12c: Oracle Coherence 12c: New Features Oracle Coherence 12c: Share and Manage Data in Clusters Oracle Coherence 12c: Oracle GoldenGate 11g: Fundamentals for Oracle Oracle GoldenGate 11g: Fundamentals for SQL Server Oracle GoldenGate 11g Fundamentals for Oracle Oracle GoldenGate 11g Fundamentals for DB2 Oracle GoldenGate 11g Fundamentals for Teradata Oracle GoldenGate 11g Fundamentals for HP NonStop Oracle GoldenGate 11g Management Pack: Overview Oracle GoldenGate 11g Troubleshooting and Tuning Oracle GoldenGate 11g: Advanced Configuration for Oracle Other Resources: Apache Hadoop : http://hadoop.apache.org/ is the homepage for these technologies. "Hadoop Definitive Guide 3rdEdition" by Tom White is a classical lecture for people who want to know more about Hadoop , and some active "googling " will also give you some more references. About the author: Eugene Simos is based in France and joined Oracle through the BEA-Weblogic Acquisition, where he worked for the Professional Service, Support, end Education for major accounts across the EMEA Region. He worked in the banking sector, ATT, Telco companies giving him extensive experience on production environments. Eugen currently specializes in Oracle Fusion Middleware teaching an array of courses on Weblogic/Webcenter, Content,BPM /SOA/Identity-Security/GoldenGate/Virtualisation/Unified Comm Suite) throughout the EMEA region.

  Read the article

• How to solve package issues/dependencies

  - by Wolfgang Kuehne

  Background info I am trying to install Veins simulation environment by following the tutorial provided by the author. In step 1 it is required to install some packages in Linux, the tutorial suggest this commands to be executed on Terminal: sudo apt-get install build-essential gcc g++ bison flex perl tcl-dev tk-dev blt libxml2-dev zlib1g-dev default-jre doxygen graphviz libwebkitgtk-1.0-0 openmpi-bin libopenmpi-dev libpcap-dev autoconf automake libtool libxerces-c2-dev proj libgdal1-dev libfox-1.6-dev When I execute this command, I immediately get: E: Package 'proj' has no installation candidate Then I remove the proj from the command and execute it again without proj in it, next I get: The following packages have unmet dependencies: libgdal1-dev : Depends: libgdal-dev but it is not going to be installed E: Unable to correct problems, you have held broken packages. So, I remove libgdal1-dev from the command as well. And it executes file, by downloading the remaining packages. To troubleshoot the problem with proj and libdgal1-dev I go to the Synaptic Package Manager. libgdal1-dev I search for libgdal1-dev in Synaptic Package Manager and I get an entry. I Mark for Installation and then Synaptic Package Manager suggests removing libxerces-c2-dev which is actually added via the initial command. Should I trust Synaptic Package Manager with this suggestion, and proceed further? proj What should I do about proj. There are some packages in Synaptic Package Manager such as proj-bin or libproj-dev. Should I install them? I think proj has to do with this and this What should I do to make sure that this simulation tool works fine?

  Read the article

• Diamond-square terrain generation problem

  - by kafka

  I've implemented a diamond-square algorithm according to this article: http://www.lighthouse3d.com/opengl/terrain/index.php?mpd2 The problem is that I get these steep cliffs all over the map. It happens on the edges, when the terrain is recursively subdivided: Here is the source: void DiamondSquare(unsigned x1,unsigned y1,unsigned x2,unsigned y2,float range) { int c1 = (int)x2 - (int)x1; int c2 = (int)y2 - (int)y1; unsigned hx = (x2 - x1)/2; unsigned hy = (y2 - y1)/2; if((c1 <= 1) || (c2 <= 1)) return; // Diamond stage float a = m_heightmap[x1][y1]; float b = m_heightmap[x2][y1]; float c = m_heightmap[x1][y2]; float d = m_heightmap[x2][y2]; float e = (a+b+c+d) / 4 + GetRnd() * range; m_heightmap[x1 + hx][y1 + hy] = e; // Square stage float f = (a + c + e + e) / 4 + GetRnd() * range; m_heightmap[x1][y1+hy] = f; float g = (a + b + e + e) / 4 + GetRnd() * range; m_heightmap[x1+hx][y1] = g; float h = (b + d + e + e) / 4 + GetRnd() * range; m_heightmap[x2][y1+hy] = h; float i = (c + d + e + e) / 4 + GetRnd() * range; m_heightmap[x1+hx][y2] = i; DiamondSquare(x1, y1, x1+hx, y1+hy, range / 2.0); // Upper left DiamondSquare(x1+hx, y1, x2, y1+hy, range / 2.0); // Upper right DiamondSquare(x1, y1+hy, x1+hx, y2, range / 2.0); // Lower left DiamondSquare(x1+hx, y1+hy, x2, y2, range / 2.0); // Lower right } Parameters: (x1,y1),(x2,y2) - coordinates that define a region on a heightmap (default (0,0)(128,128)). range - basically max. height. (default 32) Help would be greatly appreciated.

  Read the article

• Is a disk/ata timeout exception dangerous?

  - by j-g-faustus

  I have a few hard drives in mdadm RAID 5 configured to go to standby after a few minutes of inactivity. (Using hdparm.conf spindown_time.) At irregular intervals I get messages like these in dmesg: [ 1840.251661] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen [ 1840.251722] ata4.00: failed command: SMART [ 1840.251758] ata4.00: cmd b0/d5:01:06:4f:c2/00:00:00:00:00/00 tag 0 pio 512 in [ 1840.251759] res 40/00:14:50:2e:04/00:00:02:00:00/40 Emask 0x4 (timeout) [ 1840.251858] ata4.00: status: { DRDY } [ 1840.251888] ata4: hard resetting link [ 1840.600742] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [ 1840.601521] ata4.00: configured for UDMA/133 [ 1840.601547] ata4: EH complete [337877.713988] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen [337877.714019] ata4.00: failed command: SMART [337877.714038] ata4.00: cmd b0/d5:01:06:4f:c2/00:00:00:00:00/00 tag 0 pio 512 in [337877.714039] res 40/00:04:90:10:81/00:00:00:00:00/40 Emask 0x4 (timeout) [337877.714089] ata4.00: status: { DRDY } [337877.714107] ata4: hard resetting link [337878.063085] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [337878.063743] ata4.00: configured for UDMA/133 [337878.063764] ata4: EH complete I think the exception is caused by smartd when a drive does not wake up quickly enough. There are no issues (that I can tell) in accessing the drives normally through the file system - it takes a few seconds longer than normal when they are asleep, but there are no exceptions. Is this something I should worry about, as a potential symptom on something that could corrupt a drive over time? Or can I safely ignore it as part of normal operation? Edit: By request: smartctl -a for sdaand sde, both disks are members of the array. If ata4is the same as scsi-4 then sde is the one that gave the error above, according to /dev/disk/by-path.

  Read the article

• Simple (I hope) Excel question about

  - by Princess

  I am doing a directory for my neighborhood. We had most of the information from a previous directory. The information was entered: A1 name, B1 address and C1 phone number; B1 name, B2 address, C2 phone number etc. The publisher wants the information in a different format A1 name, A2 address, A3 phone number, A4 blank; A5 name, A6 address, A7 phone number, A8 blank etc... Is there an easy (or heck - a not so easy) way to have Excel change the format of the information without me having to hand type 1300 households information? I will also need to reformat the information a second time into a crisscross. The format for that one is: A1 Street name, A2 Address Number, B2 Resident Name and C2 Phone number.

  Read the article

• SQL DB design to support user feeds (in application like facebook)

  - by Yoav

  I have a social network server with a MySql DB. I want to show the users feeds like done in Facebook. Example - UserX now Friend with userY, userX did like on postX etc. Currently I have table: C1 : UserId C2 : LogType (now friend, did like etc) C3 : ObjectId (Can be userId or postId) - set depending on the LogType. Currently to get all related logs to show to the user I do the following queries: 1. Get All user Friends userIds 2. Query all rows which C1 is in userIds (I query completed) 3. Scan the DB and see - if LogType equals DidLike, check if post's OwnerId is the userId - if yes add it to logs. And so on. Obvious this is not efficient at all. I am looking for a better way. I thought I had in mind: Create a new table (in addition to the Log table) C1 : UserId C2 : LogId (from Log table) C3 : UserID of the one who did the action When querying logs - look in the table and get related Logs (by LogId) from LogTable. Updating the table: Whenever user doing action that should be in the log: 1. Add the Log entry to LogTable. 2. Scan the DB and see which users are interested with the Log (Who my friends are, Who is the owner of the post) and add related entries to the new table. (must be done in BG). 3. If user UNFRIEND another user - then look in the logs for all rows where C3 == UNFRIENDED user id and delete them. Any opinions? Other suggestions?

  Read the article

• Did bittorrent sync take advantage of multiple peers

  - by NachoChip

  I am currently testing BT sync, and really like it. However, I cannot find much of the technical detail regarding to how it works. I wonder did anyone know that is the program take advantage of multiple peers connection? For example, if I have a File A(FA), which exist in Computer 1 (C1) and Computer 2 (C2). If I set up a sync from C1 and C2 to a folder in Computer 3 (C3), will I get a better sync up time because I can get FA from 2 sources at the same time (of course, assume I have enough downstream)?

  Read the article

• Visual Studio 2010 tool to order members inside #region

  - by Michael Swan

  There exist a tool for Visual studio which order alphabetically the members grouped by #region ? means #region A1() ... A2() ... A0() ... B1() ... Z1() ... B5() ... #endregion #region C1() ... C2() ... C0() ... D1() ... X1() ... Y5() ... #endregion With that tool, I want like: #region A0() ... A1() ... A2() ... B1() ... B5() ... Z1() ... #endregion #region C0() ... C1() ... C2() ... D1() ... X1() ... Y5() ... #endregion Thank you

  Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10 11 12  | Next Page >