Search Results

Search found 6694 results on 268 pages for 'wait states'.

Page 114/268 | < Previous Page | 110 111 112 113 114 115 116 117 118 119 120 121  | Next Page >

• Replication Services as ETL extraction tool

  - by jorg

  In my last blog post I explained the principles of Replication Services and the possibilities it offers in a BI environment. One of the possibilities I described was the use of snapshot replication as an ETL extraction tool: “Snapshot Replication can also be useful in BI environments, if you don’t need a near real-time copy of the database, you can choose to use this form of replication. Next to an alternative for Transactional Replication it can be used to stage data so it can be transformed and moved into the data warehousing environment afterwards. In many solutions I have seen developers create multiple SSIS packages that simply copies data from one or more source systems to a staging database that figures as source for the ETL process. The creation of these packages takes a lot of (boring) time, while Replication Services can do the same in minutes. It is possible to filter out columns and/or records and it can even apply schema changes automatically so I think it offers enough features here. I don’t know how the performance will be and if it really works as good for this purpose as I expect, but I want to try this out soon!” Well I have tried it out and I must say it worked well. I was able to let replication services do work in a fraction of the time it would cost me to do the same in SSIS. What I did was the following: Configure snapshot replication for some Adventure Works tables, this was quite simple and straightforward. Create an SSIS package that executes the snapshot replication on demand and waits for its completion. This is something that you can’t do with out of the box functionality. While configuring the snapshot replication two SQL Agent Jobs are created, one for the creation of the snapshot and one for the distribution of the snapshot. Unfortunately these jobs are  asynchronous which means that if you execute them they immediately report back if the job started successfully or not, they do not wait for completion and report its result afterwards. So I had to create an SSIS package that executes the jobs and waits for their completion before the rest of the ETL process continues. Fortunately I was able to create the SSIS package with the desired functionality. I have made a step-by-step guide that will help you configure the snapshot replication and I have uploaded the SSIS package you need to execute it. Configure snapshot replication   The first step is to create a publication on the database you want to replicate. Connect to SQL Server Management Studio and right-click Replication, choose for New.. Publication…   The New Publication Wizard appears, click Next Choose your “source” database and click Next Choose Snapshot publication and click Next   You can now select tables and other objects that you want to publish Expand Tables and select the tables that are needed in your ETL process In the next screen you can add filters on the selected tables which can be very useful. Think about selecting only the last x days of data for example. Its possible to filter out rows and/or columns. In this example I did not apply any filters. Schedule the Snapshot Agent to run at a desired time, by doing this a SQL Agent Job is created which we need to execute from a SSIS package later on. Next you need to set the Security Settings for the Snapshot Agent. Click on the Security Settings button.   In this example I ran the Agent under the SQL Server Agent service account. This is not recommended as a security best practice. Fortunately there is an excellent article on TechNet which tells you exactly how to set up the security for replication services. Read it here and make sure you follow the guidelines!   On the next screen choose to create the publication at the end of the wizard Give the publication a name (SnapshotTest) and complete the wizard   The publication is created and the articles (tables in this case) are added Now the publication is created successfully its time to create a new subscription for this publication.   Expand the Replication folder in SSMS and right click Local Subscriptions, choose New Subscriptions   The New Subscription Wizard appears   Select the publisher on which you just created your publication and select the database and publication (SnapshotTest)   You can now choose where the Distribution Agent should run. If it runs at the distributor (push subscriptions) it causes extra processing overhead. If you use a separate server for your ETL process and databases choose to run each agent at its subscriber (pull subscriptions) to reduce the processing overhead at the distributor. Of course we need a database for the subscription and fortunately the Wizard can create it for you. Choose for New database   Give the database the desired name, set the desired options and click OK You can now add multiple SQL Server Subscribers which is not necessary in this case but can be very useful.   You now need to set the security settings for the Distribution Agent. Click on the …. button Again, in this example I ran the Agent under the SQL Server Agent service account. Read the security best practices here   Click Next   Make sure you create a synchronization job schedule again. This job is also necessary in the SSIS package later on. Initialize the subscription at first synchronization Select the first box to create the subscription when finishing this wizard Complete the wizard by clicking Finish The subscription will be created In SSMS you see a new database is created, the subscriber. There are no tables or other objects in the database available yet because the replication jobs did not ran yet. Now expand the SQL Server Agent, go to Jobs and search for the job that creates the snapshot:   Rename this job to “CreateSnapshot” Now search for the job that distributes the snapshot:   Rename this job to “DistributeSnapshot” Create an SSIS package that executes the snapshot replication We now need an SSIS package that will take care of the execution of both jobs. The CreateSnapshot job needs to execute and finish before the DistributeSnapshot job runs. After the DistributeSnapshot job has started the package needs to wait until its finished before the package execution finishes. The Execute SQL Server Agent Job Task is designed to execute SQL Agent Jobs from SSIS. Unfortunately this SSIS task only executes the job and reports back if the job started succesfully or not, it does not report if the job actually completed with success or failure. This is because these jobs are asynchronous. The SSIS package I’ve created does the following: It runs the CreateSnapshot job It checks every 5 seconds if the job is completed with a for loop When the CreateSnapshot job is completed it starts the DistributeSnapshot job And again it waits until the snapshot is delivered before the package will finish successfully Quite simple and the package is ready to use as standalone extract mechanism. After executing the package the replicated tables are added to the subscriber database and are filled with data:   Download the SSIS package here (SSIS 2008) Conclusion In this example I only replicated 5 tables, I could create a SSIS package that does the same in approximately the same amount of time. But if I replicated all the 70+ AdventureWorks tables I would save a lot of time and boring work! With replication services you also benefit from the feature that schema changes are applied automatically which means your entire extract phase wont break. Because a snapshot is created using the bcp utility (bulk copy) it’s also quite fast, so the performance will be quite good. Disadvantages of using snapshot replication as extraction tool is the limitation on source systems. You can only choose SQL Server or Oracle databases to act as a publisher. So if you plan to build an extract phase for your ETL process that will invoke a lot of tables think about replication services, it would save you a lot of time and thanks to the Extract SSIS package I’ve created you can perfectly fit it in your usual SSIS ETL process.

  Read the article

• Try a sample: Using the counter predicate for event sampling

  - by extended_events

  Extended Events offers a rich filtering mechanism, called predicates, that allows you to reduce the number of events you collect by specifying criteria that will be applied during event collection. (You can find more information about predicates in Using SQL Server 2008 Extended Events (by Jonathan Kehayias)) By evaluating predicates early in the event firing sequence we can reduce the performance impact of collecting events by stopping event collection when the criteria are not met. You can specify predicates on both event fields and on a special object called a predicate source. Predicate sources are similar to action in that they typically are related to some type of global information available from the server. You will find that many of the actions available in Extended Events have equivalent predicate sources, but actions and predicates sources are not the same thing. Applying predicates, whether on a field or predicate source, is very similar to what you are used to in T-SQL in terms of how they work; you pick some field/source and compare it to a value, for example, session_id = 52. There is one predicate source that merits special attention though, not just for its special use, but for how the order of predicate evaluation impacts the behavior you see. I’m referring to the counter predicate source. The counter predicate source gives you a way to sample a subset of events that otherwise meet the criteria of the predicate; for example you could collect every other event, or only every tenth event. Simple CountingThe counter predicate source works by creating an in memory counter that increments every time the predicate statement is evaluated. Here is a simple example with my favorite event, sql_statement_completed, that only collects the second statement that is run. (OK, that’s not much of a sample, but this is for demonstration purposes. Here is the session definition: CREATE EVENT SESSION counter_test ON SERVERADD EVENT sqlserver.sql_statement_completed    (ACTION (sqlserver.sql_text)    WHERE package0.counter = 2)ADD TARGET package0.ring_bufferWITH (MAX_DISPATCH_LATENCY = 1 SECONDS) You can find general information about the session DDL syntax in BOL and from Pedro’s post Introduction to Extended Events. The important part here is the WHERE statement that defines that I only what the event where package0.count = 2; in other words, only the second instance of the event. Notice that I need to provide the package name along with the predicate source. You don’t need to provide the package name if you’re using event fields, only for predicate sources. Let’s say I run the following test queries: -- Run three statements to test the sessionSELECT 'This is the first statement'GOSELECT 'This is the second statement'GOSELECT 'This is the third statement';GO Once you return the event data from the ring buffer and parse the XML (see my earlier post on reading event data) you should see something like this: event_name sql_text sql_statement_completed SELECT ‘This is the second statement’ You can see that only the second statement from the test was actually collected. (Feel free to try this yourself. Check out what happens if you remove the WHERE statement from your session. Go ahead, I’ll wait.) Percentage Sampling OK, so that wasn’t particularly interesting, but you can probably see that this could be interesting, for example, lets say I need a 25% sample of the statements executed on my server for some type of QA analysis, that might be more interesting than just the second statement. All comparisons of predicates are handled using an object called a predicate comparator; the simple comparisons such as equals, greater than, etc. are mapped to the common mathematical symbols you know and love (eg. = and >), but to do the less common comparisons you will need to use the predicate comparators directly. You would probably look to the MOD operation to do this type sampling; we would too, but we don’t call it MOD, we call it divides_by_uint64. This comparator evaluates whether one number is divisible by another with no remainder. The general syntax for using a predicate comparator is pred_comp(field, value), field is always first and value is always second. So lets take a look at how the session changes to answer our new question of 25% sampling: CREATE EVENT SESSION counter_test_25 ON SERVERADD EVENT sqlserver.sql_statement_completed    (ACTION (sqlserver.sql_text)    WHERE package0.divides_by_uint64(package0.counter,4))ADD TARGET package0.ring_bufferWITH (MAX_DISPATCH_LATENCY = 1 SECONDS)GO Here I’ve replaced the simple equivalency check with the divides_by_uint64 comparator to check if the counter is evenly divisible by 4, which gives us back every fourth record. I’ll leave it as an exercise for the reader to test this session. Why order matters I indicated at the start of this post that order matters when it comes to the counter predicate – it does. Like most other predicate systems, Extended Events evaluates the predicate statement from left to right; as soon as the predicate statement is proven false we abandon evaluation of the remainder of the statement. The counter predicate source is only incremented when it is evaluated so whether or not the counter is incremented will depend on where it is in the predicate statement and whether a previous criteria made the predicate false or not. Here is a generic example: Pred1: (WHERE statement_1 AND package0.counter = 2)Pred2: (WHERE package0.counter = 2 AND statement_1) Let’s say I cause a number of events as follows and examine what happens to the counter predicate source. Iteration Statement Pred1 Counter Pred2 Counter A Not statement_1 0 1 B statement_1 1 2 C Not statement_1 1 3 D statement_1 2 4 As you can see, in the case of Pred1, statement_1 is evaluated first, when it fails (A & C) predicate evaluation is stopped and the counter is not incremented. With Pred2 the counter is evaluated first, so it is incremented on every iteration of the event and the remaining parts of the predicate are then evaluated. In this example, Pred1 would return an event for D while Pred2 would return an event for B. But wait, there is an interesting side-effect here; consider Pred2 if I had run my statements in the following order: Not statement_1 Not statement_1 statement_1 statement_1 In this case I would never get an event back from the system because the point at which counter=2, the rest of the predicate evaluates as false so the event is not returned. If you’re using the counter target for sampling and you’re not getting the expected events, or any events, check the order of the predicate criteria. As a general rule I’d suggest that the counter criteria should be the last element of your predicate statement since that will assure that your sampling rate will apply to the set of event records defined by the rest of your predicate. Aside: I’m interested in hearing about uses for putting the counter predicate criteria earlier in the predicate statement. If you have one, post it in a comment to share with the class. - Mike Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

  Read the article

• SQL SERVER – Weekly Series – Memory Lane – #052

  - by Pinal Dave

  Let us continue with the final episode of the Memory Lane Series. Here is the list of selected articles of SQLAuthority.com across all these years. Instead of just listing all the articles I have selected a few of my most favorite articles and have listed them here with additional notes below it. Let me know which one of the following is your favorite article from memory lane. 2007 Set Server Level FILLFACTOR Using T-SQL Script Specifies a percentage that indicates how full the Database Engine should make the leaf level of each index page during index creation or alteration. fillfactor must be an integer value from 1 to 100. The default is 0. Limitation of Online Index Rebuld Operation Online operation means when online operations are happening in the database are in normal operational condition, the processes which are participating in online operations does not require exclusive access to the database. Get Permissions of My Username / Userlogin on Server / Database A few days ago, I was invited to one of the largest database company. I was asked to review database schema and propose changes to it. There was special username or user logic was created for me, so I can review their database. I was very much interested to know what kind of permissions I was assigned per server level and database level. I did not feel like asking Sr. DBA the question about permissions. Simple Example of WHILE Loop With CONTINUE and BREAK Keywords This question is one of those questions which is very simple and most of the users get it correct, however few users find it confusing for the first time. I have tried to explain the usage of simple WHILE loop in the first example. BREAK keyword will exit the stop the while loop and control is moved to the next statement after the while loop. CONTINUE keyword skips all the statement after its execution and control is sent to the first statement of while loop. Forced Parameterization and Simple Parameterization – T-SQL and SSMS When the PARAMETERIZATION option is set to FORCED, any literal value that appears in a SELECT, INSERT, UPDATE or DELETE statement is converted to a parameter during query compilation. When the PARAMETERIZATION database option is SET to SIMPLE, the SQL Server query optimizer may choose to parameterize the queries. 2008 Transaction and Local Variables – Swap Variables – Update All At Once Concept Summary : Transaction have no effect over memory variables. When UPDATE statement is applied over any table (physical or memory) all the updates are applied at one time together when the statement is committed. First of all I suggest that you read the article listed above about the effect of transaction on local variant. As seen there local variables are independent of any transaction effect. Simulate INNER JOIN using LEFT JOIN statement – Performance Analysis Just a day ago, while I was working with JOINs I find one interesting observation, which has prompted me to create following example. Before we continue further let me make very clear that INNER JOIN should be used where it cannot be used and simulating INNER JOIN using any other JOINs will degrade the performance. If there are scopes to convert any OUTER JOIN to INNER JOIN it should be done with priority. 2009 Introduction to Business Intelligence – Important Terms & Definitions Business intelligence (BI) is a broad category of application programs and technologies for gathering, storing, analyzing, and providing access to data from various data sources, thus providing enterprise users with reliable and timely information and analysis for improved decision making. Difference Between Candidate Keys and Primary Key Candidate Key – A Candidate Key can be any column or a combination of columns that can qualify as unique key in database. There can be multiple Candidate Keys in one table. Each Candidate Key can qualify as Primary Key. Primary Key – A Primary Key is a column or a combination of columns that uniquely identify a record. Only one Candidate Key can be Primary Key. 2010 Taking Multiple Backup of Database in Single Command – Mirrored Database Backup I recently had a very interesting experience. In one of my recent consultancy works, I was told by our client that they are going to take the backup of the database and will also a copy of it at the same time. I expressed that it was surely possible if they were going to use a mirror command. In addition, they told me that whenever they take two copies of the database, the size of the database, is always reduced. Now this was something not clear to me, I said it was not possible and so I asked them to show me the script. Corrupted Backup File and Unsuccessful Restore The CTO, who was also present at the location, got very upset with this situation. He then asked when the last successful restore test was done. As expected, the answer was NEVER.There were no successful restore tests done before. During that time, I was present and I could clearly see the stress, confusion, carelessness and anger around me. I did not appreciate the feeling and I was pretty sure that no one in there wanted the atmosphere like me. 2011 TRACEWRITE – Wait Type – Wait Related to Buffer and Resolution SQL Trace is a SQL Server database engine technology which monitors specific events generated when various actions occur in the database engine. When any event is fired it goes through various stages as well various routes. One of the routes is Trace I/O Provider, which sends data to its final destination either as a file or rowset. DATEDIFF – Accuracy of Various Dateparts If you want to have accuracy in seconds, you need to use a different approach. In the first example, the accurate method is to find the number of seconds first and then divide it by 60 to convert it in minutes. Dedicated Access Control for SQL Server Express Edition http://www.youtube.com/watch?v=1k00z82u4OI Book Signing at SQLPASS 2012 Who I Am And How I Got Here – True Story as Blog Post If there was a shortcut to success – I want to know. I learnt SQL Server hard way and I am still learning. There are so many things, I have to learn. There is not enough time to learn everything which we want to learn. I am constantly working on it every day. I welcome you to join my journey as well. Please join me in my journey to learn SQL Server – more the merrier. Vacation, Travel and Study – A New Concept Even those who have advanced degrees and went to college for years, or even decades, find studying hard.  There is a difference between studying for a career and studying for a certification.  At least to get a degree there is a variety of subjects, with labs, exams, and practice problems to make things more interesting. Order By Numeric Values Formatted as String We have a table which has a column containing alphanumeric data. The data always has first as an integer and later part as a string. The business need is to order the data based on the first part of the alphanumeric data which is an integer. Now the problem is that no matter how we use ORDER BY the result is not produced as expected. Let us understand this with an example. Resolving SQL Server Connection Errors – SQL in Sixty Seconds #030 – Video One of the most famous errors related to SQL Server is about connecting to SQL Server itself. Here is how it goes, most of the time developers have worked with SQL Server and knows pretty much every error which they face during development language. However, hardly they install fresh SQL Server. As the installation of the SQL Server is a rare occasion unless you are a DBA who is responsible for such an instance – the error faced during installations are pretty rare as well. http://www.youtube.com/watch?v=1k00z82u4OI Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Memory Lane, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology

  Read the article

• No GLX on Intel card with multiseat with additional nVidia card

  - by MeanEYE

  I have multiseat configured and my Xorg has 2 server layouts. One is for nVidia card and other is for Intel card. They both work, but display server assigned to Intel card doesn't have hardware acceleration since DRI and GLX module being used is from nVidia driver. So my question is, can I configure layouts somehow to use right DRI and GLX with each card? My Xorg.conf: Section "ServerLayout" Identifier "Default" Screen 0 "Screen0" 0 0 Option "Xinerama" "0" EndSection Section "ServerLayout" Identifier "TV" Screen 0 "Screen1" 0 0 Option "Xinerama" "0" EndSection Section "Monitor" # HorizSync source: edid, VertRefresh source: edid Identifier "Monitor0" VendorName "Unknown" ModelName "DELL E198WFP" HorizSync 30.0 - 83.0 VertRefresh 56.0 - 75.0 Option "DPMS" EndSection Section "Monitor" Identifier "Monitor1" VendorName "Unknown" Option "DPMS" EndSection Section "Device" Identifier "Device0" Driver "nvidia" VendorName "NVIDIA Corporation" BoardName "GeForce GT 610" EndSection Section "Device" Identifier "Device1" Driver "intel" BusID "PCI:0:2:0" Option "AccelMethod" "uxa" EndSection Section "Screen" Identifier "Screen0" Device "Device0" Monitor "Monitor0" DefaultDepth 24 Option "Stereo" "0" Option "nvidiaXineramaInfoOrder" "DFP-1" Option "metamodes" "DFP-0: nvidia-auto-select +1440+0, DFP-1: nvidia-auto-select +0+0" SubSection "Display" Depth 24 EndSubSection EndSection Section "Screen" Identifier "Screen1" Device "Device1" Monitor "Monitor1" DefaultDepth 24 SubSection "Display" Depth 24 EndSubSection EndSection Log file for Intel: [ 18.239] X.Org X Server 1.13.0 Release Date: 2012-09-05 [ 18.239] X Protocol Version 11, Revision 0 [ 18.239] Build Operating System: Linux 2.6.24-32-xen x86_64 Ubuntu [ 18.239] Current Operating System: Linux bytewiper 3.5.0-18-generic #29-Ubuntu SMP Fri Oct 19 10:26:51 UTC 2012 x86_64 [ 18.239] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.5.0-18-generic root=UUID=fc0616fd-f212-4846-9241-ba4a492f0513 ro quiet splash [ 18.239] Build Date: 20 September 2012 11:55:20AM [ 18.239] xorg-server 2:1.13.0+git20120920.70e57668-0ubuntu0ricotz (For technical support please see http://www.ubuntu.com/support) [ 18.239] Current version of pixman: 0.26.0 [ 18.239] Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. [ 18.239] Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. [ 18.239] (==) Log file: "/var/log/Xorg.1.log", Time: Wed Nov 21 18:32:14 2012 [ 18.239] (==) Using config file: "/etc/X11/xorg.conf" [ 18.239] (==) Using system config directory "/usr/share/X11/xorg.conf.d" [ 18.239] (++) ServerLayout "TV" [ 18.239] (**) |-->Screen "Screen1" (0) [ 18.239] (**) | |-->Monitor "Monitor1" [ 18.240] (**) | |-->Device "Device1" [ 18.240] (**) Option "Xinerama" "0" [ 18.240] (==) Automatically adding devices [ 18.240] (==) Automatically enabling devices [ 18.240] (==) Automatically adding GPU devices [ 18.240] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (WW) The directory "/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType" does not exist. [ 18.240] Entry deleted from font path. [ 18.240] (==) FontPath set to: /usr/share/fonts/X11/misc, /usr/share/fonts/X11/Type1, built-ins [ 18.240] (==) ModulePath set to "/usr/lib/x86_64-linux-gnu/xorg/extra-modules,/usr/lib/xorg/extra-modules,/usr/lib/xorg/modules" [ 18.240] (II) The server relies on udev to provide the list of input devices. If no devices become available, reconfigure udev or disable AutoAddDevices. [ 18.240] (II) Loader magic: 0x7f6917944c40 [ 18.240] (II) Module ABI versions: [ 18.240] X.Org ANSI C Emulation: 0.4 [ 18.240] X.Org Video Driver: 13.0 [ 18.240] X.Org XInput driver : 18.0 [ 18.240] X.Org Server Extension : 7.0 [ 18.240] (II) config/udev: Adding drm device (/dev/dri/card0) [ 18.241] (--) PCI: (0:0:2:0) 8086:0152:1043:84ca rev 9, Mem @ 0xf7400000/4194304, 0xd0000000/268435456, I/O @ 0x0000f000/64 [ 18.241] (--) PCI:*(0:1:0:0) 10de:104a:1458:3546 rev 161, Mem @ 0xf6000000/16777216, 0xe0000000/134217728, 0xe8000000/33554432, I/O @ 0x0000e000/128, BIOS @ 0x????????/524288 [ 18.241] (II) Open ACPI successful (/var/run/acpid.socket) [ 18.241] Initializing built-in extension Generic Event Extension [ 18.241] Initializing built-in extension SHAPE [ 18.241] Initializing built-in extension MIT-SHM [ 18.241] Initializing built-in extension XInputExtension [ 18.241] Initializing built-in extension XTEST [ 18.241] Initializing built-in extension BIG-REQUESTS [ 18.241] Initializing built-in extension SYNC [ 18.241] Initializing built-in extension XKEYBOARD [ 18.241] Initializing built-in extension XC-MISC [ 18.241] Initializing built-in extension SECURITY [ 18.241] Initializing built-in extension XINERAMA [ 18.241] Initializing built-in extension XFIXES [ 18.241] Initializing built-in extension RENDER [ 18.241] Initializing built-in extension RANDR [ 18.241] Initializing built-in extension COMPOSITE [ 18.241] Initializing built-in extension DAMAGE [ 18.241] Initializing built-in extension MIT-SCREEN-SAVER [ 18.241] Initializing built-in extension DOUBLE-BUFFER [ 18.241] Initializing built-in extension RECORD [ 18.241] Initializing built-in extension DPMS [ 18.241] Initializing built-in extension X-Resource [ 18.241] Initializing built-in extension XVideo [ 18.241] Initializing built-in extension XVideo-MotionCompensation [ 18.241] Initializing built-in extension XFree86-VidModeExtension [ 18.241] Initializing built-in extension XFree86-DGA [ 18.241] Initializing built-in extension XFree86-DRI [ 18.241] Initializing built-in extension DRI2 [ 18.241] (II) LoadModule: "glx" [ 18.241] (II) Loading /usr/lib/x86_64-linux-gnu/xorg/extra-modules/libglx.so [ 18.247] (II) Module glx: vendor="NVIDIA Corporation" [ 18.247] compiled for 4.0.2, module version = 1.0.0 [ 18.247] Module class: X.Org Server Extension [ 18.247] (II) NVIDIA GLX Module 310.19 Thu Nov 8 01:12:43 PST 2012 [ 18.247] Loading extension GLX [ 18.247] (II) LoadModule: "intel" [ 18.248] (II) Loading /usr/lib/xorg/modules/drivers/intel_drv.so [ 18.248] (II) Module intel: vendor="X.Org Foundation" [ 18.248] compiled for 1.13.0, module version = 2.20.13 [ 18.248] Module class: X.Org Video Driver [ 18.248] ABI class: X.Org Video Driver, version 13.0 [ 18.248] (II) intel: Driver for Intel Integrated Graphics Chipsets: i810, i810-dc100, i810e, i815, i830M, 845G, 854, 852GM/855GM, 865G, 915G, E7221 (i915), 915GM, 945G, 945GM, 945GME, Pineview GM, Pineview G, 965G, G35, 965Q, 946GZ, 965GM, 965GME/GLE, G33, Q35, Q33, GM45, 4 Series, G45/G43, Q45/Q43, G41, B43, B43, Clarkdale, Arrandale, Sandybridge Desktop (GT1), Sandybridge Desktop (GT2), Sandybridge Desktop (GT2+), Sandybridge Mobile (GT1), Sandybridge Mobile (GT2), Sandybridge Mobile (GT2+), Sandybridge Server, Ivybridge Mobile (GT1), Ivybridge Mobile (GT2), Ivybridge Desktop (GT1), Ivybridge Desktop (GT2), Ivybridge Server, Ivybridge Server (GT2), Haswell Desktop (GT1), Haswell Desktop (GT2), Haswell Desktop (GT2+), Haswell Mobile (GT1), Haswell Mobile (GT2), Haswell Mobile (GT2+), Haswell Server (GT1), Haswell Server (GT2), Haswell Server (GT2+), Haswell SDV Desktop (GT1), Haswell SDV Desktop (GT2), Haswell SDV Desktop (GT2+), Haswell SDV Mobile (GT1), Haswell SDV Mobile (GT2), Haswell SDV Mobile (GT2+), Haswell SDV Server (GT1), Haswell SDV Server (GT2), Haswell SDV Server (GT2+), Haswell ULT Desktop (GT1), Haswell ULT Desktop (GT2), Haswell ULT Desktop (GT2+), Haswell ULT Mobile (GT1), Haswell ULT Mobile (GT2), Haswell ULT Mobile (GT2+), Haswell ULT Server (GT1), Haswell ULT Server (GT2), Haswell ULT Server (GT2+), Haswell CRW Desktop (GT1), Haswell CRW Desktop (GT2), Haswell CRW Desktop (GT2+), Haswell CRW Mobile (GT1), Haswell CRW Mobile (GT2), Haswell CRW Mobile (GT2+), Haswell CRW Server (GT1), Haswell CRW Server (GT2), Haswell CRW Server (GT2+), ValleyView PO board [ 18.248] (++) using VT number 8 [ 18.593] (II) intel(0): using device path '/dev/dri/card0' [ 18.593] (**) intel(0): Depth 24, (--) framebuffer bpp 32 [ 18.593] (==) intel(0): RGB weight 888 [ 18.593] (==) intel(0): Default visual is TrueColor [ 18.593] (**) intel(0): Option "AccelMethod" "uxa" [ 18.593] (--) intel(0): Integrated Graphics Chipset: Intel(R) Ivybridge Desktop (GT1) [ 18.593] (**) intel(0): Relaxed fencing enabled [ 18.593] (**) intel(0): Wait on SwapBuffers? enabled [ 18.593] (**) intel(0): Triple buffering? enabled [ 18.593] (**) intel(0): Framebuffer tiled [ 18.593] (**) intel(0): Pixmaps tiled [ 18.593] (**) intel(0): 3D buffers tiled [ 18.593] (**) intel(0): SwapBuffers wait enabled ... [ 20.312] (II) Module fb: vendor="X.Org Foundation" [ 20.312] compiled for 1.13.0, module version = 1.0.0 [ 20.312] ABI class: X.Org ANSI C Emulation, version 0.4 [ 20.312] (II) Loading sub module "dri2" [ 20.312] (II) LoadModule: "dri2" [ 20.312] (II) Module "dri2" already built-in [ 20.312] (==) Depth 24 pixmap format is 32 bpp [ 20.312] (II) intel(0): [DRI2] Setup complete [ 20.312] (II) intel(0): [DRI2] DRI driver: i965 [ 20.312] (II) intel(0): Allocated new frame buffer 1920x1080 stride 7680, tiled [ 20.312] (II) UXA(0): Driver registered support for the following operations: [ 20.312] (II) solid [ 20.312] (II) copy [ 20.312] (II) composite (RENDER acceleration) [ 20.312] (II) put_image [ 20.312] (II) get_image [ 20.312] (==) intel(0): Backing store disabled [ 20.312] (==) intel(0): Silken mouse enabled [ 20.312] (II) intel(0): Initializing HW Cursor [ 20.312] (II) intel(0): RandR 1.2 enabled, ignore the following RandR disabled message. [ 20.313] (**) intel(0): DPMS enabled [ 20.313] (==) intel(0): Intel XvMC decoder enabled [ 20.313] (II) intel(0): Set up textured video [ 20.313] (II) intel(0): [XvMC] xvmc_vld driver initialized. [ 20.313] (II) intel(0): direct rendering: DRI2 Enabled [ 20.313] (==) intel(0): hotplug detection: "enabled" [ 20.332] (--) RandR disabled [ 20.335] (EE) Failed to initialize GLX extension (Compatible NVIDIA X driver not found) [ 20.335] (II) intel(0): Setting screen physical size to 508 x 285 [ 20.338] (II) XKB: reuse xkmfile /var/lib/xkb/server-B20D7FC79C7F597315E3E501AEF10E0D866E8E92.xkm [ 20.340] (II) config/udev: Adding input device Power Button (/dev/input/event1) [ 20.340] (**) Power Button: Applying InputClass "evdev keyboard catchall" [ 20.340] (II) LoadModule: "evdev" [ 20.340] (II) Loading /usr/lib/xorg/modules/input/evdev_drv.so

  Read the article

• DBCC CHECKDB on VVLDB and latches (Or: My Pain is Your Gain)

  - by Argenis

    Does your CHECKDB hurt, Argenis? There is a classic blog series by Paul Randal [blog|twitter] called “CHECKDB From Every Angle” which is pretty much mandatory reading for anybody who’s even remotely considering going for the MCM certification, or its replacement (the Microsoft Certified Solutions Master: Data Platform – makes my fingers hurt just from typing it). Of particular interest is the post “Consistency Options for a VLDB” – on it, Paul provides solid, timeless advice (I use the word “timeless” because it was written in 2007, and it all applies today!) on how to perform checks on very large databases. Well, here I was trying to figure out how to make CHECKDB run faster on a restored copy of one of our databases, which happens to exceed 7TB in size. The whole thing was taking several days on multiple systems, regardless of the storage used – SAS, SATA or even SSD…and I actually didn’t pay much attention to how long it was taking, or even bothered to look at the reasons why - as long as it was finishing okay and found no consistency errors. Yes – I know. That was a huge mistake, as corruption found in a database several days after taking place could only allow for further spread of the corruption – and potentially large data loss. In the last two weeks I increased my attention towards this problem, as we noticed that CHECKDB was taking EVEN LONGER on brand new all-flash storage in the SAN! I couldn’t really explain it, and were almost ready to blame the storage vendor. The vendor told us that they could initially see the server driving decent I/O – around 450Mb/sec, and then it would settle at a very slow rate of 10Mb/sec or so. “Hum”, I thought – “CHECKDB is just not pushing the I/O subsystem hard enough”. Perfmon confirmed the vendor’s observations. Dreaded @BlobEater What was CHECKDB doing all the time while doing so little I/O? Eating Blobs. It turns out that CHECKDB was taking an extremely long time on one of our frankentables, which happens to be have 35 billion rows (yup, with a b) and sucks up several terabytes of space in the database. We do have a project ongoing to purge/split/partition this table, so it’s just a matter of time before we deal with it. But the reality today is that CHECKDB is coming to a screeching halt in performance when dealing with this particular table. Checking sys.dm_os_waiting_tasks and sys.dm_os_latch_stats showed that LATCH_EX (DBCC_OBJECT_METADATA) was by far the top wait type. I remembered hearing recently about that wait from another post that Paul Randal made, but that was related to computed-column indexes, and in fact, Paul himself reminded me of his article via twitter. But alas, our pathologic table had no non-clustered indexes on computed columns. I knew that latches are used by the database engine to do internal synchronization – but how could I help speed this up? After all, this is stuff that doesn’t have a lot of knobs to tweak. (There’s a fantastic level 500 talk by Bob Ward from Microsoft CSS [blog|twitter] called “Inside SQL Server Latches” given at PASS 2010 – and you can check it out here. DISCLAIMER: I assume no responsibility for any brain melting that might ensue from watching Bob’s talk!) Failed Hypotheses Earlier on this week I flew down to Palo Alto, CA, to visit our Headquarters – and after having a great time with my Monkey peers, I was relaxing on the plane back to Seattle watching a great talk by SQL Server MVP and fellow MCM Maciej Pilecki [twitter] called “Masterclass: A Day in the Life of a Database Transaction” where he discusses many different topics related to transaction management inside SQL Server. Very good stuff, and when I got home it was a little late – that slow DBCC CHECKDB that I had been dealing with was way in the back of my head. As I was looking at the problem at hand earlier on this week, I thought “How about I set the database to read-only?” I remembered one of the things Maciej had (jokingly) said in his talk: “if you don’t want locking and blocking, set the database to read-only” (or something to that effect, pardon my loose memory). I immediately killed the CHECKDB which had been running painfully for days, and set the database to read-only mode. Then I ran DBCC CHECKDB against it. It started going really fast (even a bit faster than before), and then throttled down again to around 10Mb/sec. All sorts of expletives went through my head at the time. Sure enough, the same latching scenario was present. Oh well. I even spent some time trying to figure out if NUMA was hurting performance. Folks on Twitter made suggestions in this regard (thanks, Lonny! [twitter]) …Eureka? This past Friday I was still scratching my head about the whole thing; I was ready to start profiling with XPERF to see if I could figure out which part of the engine was to blame and then get Microsoft to look at the evidence. After getting a bunch of good news I’ll blog about separately, I sat down for a figurative smack down with CHECKDB before the weekend. And then the light bulb went on. A sparse column. I thought that I couldn’t possibly be experiencing the same scenario that Paul blogged about back in March showing extreme latching with non-clustered indexes on computed columns. Did I even have a non-clustered index on my sparse column? As it turns out, I did. I had one filtered non-clustered index – with the sparse column as the index key (and only column). To prove that this was the problem, I went and setup a test. Yup, that'll do it The repro is very simple for this issue: I tested it on the latest public builds of SQL Server 2008 R2 SP2 (CU6) and SQL Server 2012 SP1 (CU4). First, create a test database and a test table, which only needs to contain a sparse column: CREATE DATABASE SparseColTest; GO USE SparseColTest; GO CREATE TABLE testTable (testCol smalldatetime SPARSE NULL); GO INSERT INTO testTable (testCol) VALUES (NULL); GO 1000000 That’s 1 million rows, and even though you’re inserting NULLs, that’s going to take a while. In my laptop, it took 3 minutes and 31 seconds. Next, we run DBCC CHECKDB against the database: DBCC CHECKDB('SparseColTest') WITH NO_INFOMSGS, ALL_ERRORMSGS; This runs extremely fast, as least on my test rig – 198 milliseconds. Now let’s create a filtered non-clustered index on the sparse column: CREATE NONCLUSTERED INDEX [badBadIndex] ON testTable (testCol) WHERE testCol IS NOT NULL; With the index in place now, let’s run DBCC CHECKDB one more time: DBCC CHECKDB('SparseColTest') WITH NO_INFOMSGS, ALL_ERRORMSGS; In my test system this statement completed in 11433 milliseconds. 11.43 full seconds. Quite the jump from 198 milliseconds. I went ahead and dropped the filtered non-clustered indexes on the restored copy of our production database, and ran CHECKDB against that. We went down from 7+ days to 19 hours and 20 minutes. Cue the “Argenis is not impressed” meme, please, Mr. LaRock. My pain is your gain, folks. Go check to see if you have any of such indexes – they’re likely causing your consistency checks to run very, very slow. Happy CHECKDBing, -Argenis ps: I plan to file a Connect item for this issue – I consider it a pretty serious bug in the engine. After all, filtered indexes were invented BECAUSE of the sparse column feature – and it makes a lot of sense to use them together. Watch this space and my twitter timeline for a link.

  Read the article

• Profiling Startup Of VS2012 &ndash; SpeedTrace Profiler

  - by Alois Kraus

  SpeedTrace is a relatively unknown profiler made a company called Ipcas. A single professional license does cost 449€+VAT. For the test I did use SpeedTrace 4.5 which is currently Beta. Although it is cheaper than dotTrace it has by far the most options to influence how profiling does work. First you need to create a tracing project which does configure tracing for one process type. You can start the application directly from the profiler or (much more interesting) it does attach to a specific process when it is started. For this you need to check “Trace the specified …” radio button and enter the process name in the “Process Name of the Trace” edit box. You can even selectively enable tracing for processes with a specific command line. Then you need to activate the trace project by pressing the Activate Project button and you are ready to start VS as usual. If you want to profile the next 10 VS instances that you start you can set the Number of Processes counter to e.g. 10. This is immensely helpful if you are trying to profile only the next 5 started processes. As you can see there are many more tabs which do allow to influence tracing in a much more sophisticated way. SpeedTrace is the only profiler which does not rely entirely on the profiling Api of .NET. Instead it does modify the IL code (instrumentation on the fly) to write tracing information to disc which can later be analyzed. This approach is not only very fast but it does give you unprecedented analysis capabilities. Once the traces are collected they do show up in your workspace where you can open the trace viewer. I do skip the other windows because this view is by far the most useful one. You can sort the methods not only by Wall Clock time but also by CPU consumption and wait time which none of the other products support in their views at the same time. If you want to optimize for CPU consumption sort by CPU time. If you want to find out where most time is spent you need Clock Total time and Clock Waiting. There you can directly see if the method did take long because it did wait on something or it did really execute stuff that did take so long. Once you have found a method you want to drill deeper you can double click on a method to get to the Caller/Callee view which is similar to the JetBrains Method Grid view. But this time you do see much more. In the middle is the clicked method. Above are the methods that call you and below are the methods that you do directly call. Normally you would then start digging deeper to find the end of the chain where the slow method worth optimizing is located. But there is a shortcut. You can press the magic   button to calculate the aggregation of all called methods. This is displayed in the lower left window where you can see each method call and how long it did take. There you can also sort to see if this call stack does only contain methods (e.g. WCF connect calls which you cannot make faster) not worth optimizing. YourKit has a similar feature where it is called Callees List. In the Functions tab you have in the context menu also many other useful analysis options One really outstanding feature is the View Call History Drilldown. When you select this one you get not a sum of all method invocations but a list with the duration of each method call. This is not surprising since SpeedTrace does use tracing to get its timings. There you can get many useful graphs how this method did behave over time. Did it become slower at some point in time or was only the first call slow? The diagrams and the list will tell you that. That is all fine but what should I do when one method call was slow? I want to see from where it was coming from. No problem select the method in the list hit F10 and you get the call stack. This is a life saver if you e.g. search for serialization problems. Today Serializers are used everywhere. You want to find out from where the 5s XmlSerializer.Deserialize call did come from? Hit F10 and you get the call stack which did invoke the 5s Deserialize call. The CPU timeline tab is also useful to find out where long pauses or excessive CPU consumption did happen. Click in the graph to get the Thread Stacks window where you can get a quick overview what all threads were doing at this time. This does look like the Stack Traces feature in YourKit. Only this time you get the last called method first which helps to quickly see what all threads were executing at this moment. YourKit does generate a rather long list which can be hard to go through when you have many threads. The thread list in the middle does not give you call stacks or anything like that but you see which methods were found most often executing code by the profiler which is a good indication for methods consuming most CPU time. This does sound too good to be true? I have not told you the best part yet. The best thing about this profiler is the staff behind it. When I do see a crash or some other odd behavior I send a mail to Ipcas and I do get usually the next day a mail that the problem has been fixed and a download link to the new version. The guys at Ipcas are even so helpful to log in to your machine via a Citrix Client to help you to get started profiling your actual application you want to profile. After a 2h telco I was converted from a hater to a believer of this tool. The fast response time might also have something to do with the fact that they are actively working on 4.5 to get out of the door. But still the support is by far the best I have encountered so far. The only downside is that you should instrument your assemblies including the .NET Framework to get most accurate numbers. You can profile without doing it but then you will see very high JIT times in your process which can severely affect the correctness of the measured timings. If you do not care about exact numbers you can also enable in the main UI in the Data Trace tab logging of method arguments of primitive types. If you need to know what files at which times were opened by your application you can find it out without a debugger. Since SpeedTrace does read huge trace files in its reader you should perhaps use a 64 bit machine to be able to analyze bigger traces as well. The memory consumption of the trace reader is too high for my taste. But they did promise for the next version to come up with something much improved.

  Read the article

• 5 Ways Android Still Disappoints (Me)

  - by TStewartDev

  Let me make this clear: I'm annoyed with Apple. I don't like their current policies and I don't like where Steve Jobs is taking the company. In general, I don't like it when any one company gets too much control in a market. When that happens, the leading company dictates the game and as consumers, our options all but disappear. That said, I'm still going to buy a new iPhone next week. My Apple-hating friends seem to desperately want me to go Android instead, but frankly, it's not good enough for me, and here are the reasons why. The Modern WinMo One of the reasons that Microsoft has identified for Windows Mobile's rapid decline is the breadth of hardware. They exercised little control over manufacturer's implementations. In theory, that sounds great. We as consumers have lots of choice. In practice, though, it meant among other things that updates to the devices were left up to the manufacturers. As a result, that rarely happened. (I'm still bitter at Toshiba for leaving me hanging back in 2002.) And now, Google is doing the same thing with Android. Case in point: my wife has a Motorola Backflip that we bought in April. It was released in March. Motorola says it will get Android 2.1 "sometime in Q3". Great. Meanwhile, I pull down the latest version of iPhone OS (now iOS) and install it the same day it's released. You may say that I can't judge Android by one lazy manufacturer. Yup, I sure can. With Apple, my original iPhone has been supported perfectly for 3 years. With Android, I will have to wait for upgrades after Google releases them, possibly indefinitely. Not cool. AT&T We signed a new contract with AT&T in April to get my wife's phone. I've had a reasonable experience with them. I don't imagine my experience with Verizon would be any better, and I'm relatively confident that Sprint doesn't have the coverage it takes to work well for us. The fact is, AT&T, for whatever reason, doesn't have jack for Android phones. May not be Android's fault, but it's still a shortcoming that prevents me from having it just like the iPhone's exclusivity keeps some folks on other networks from having it. Innovation? What Innovation? Android has a nice dashboard and a great notification system and… nothing else original. I keep reading about how disappointing the iPhone is nowadays. "It has no innovation," people say. Who does? Android has modeled its behavior after the iPhone. That's fine, but if all you've got is a similar product and I'm invested both skill-wise and app-wise in my current platform, why should I change? Microsoft's new Windows Phone 7 looks somewhat innovative, and I'm pretty excited to see what they'll bring to the table, but that's another six months away, at least. I've got a 3 year old phone that has some annoying issues now (thanks to recent encounters with water). I need a new phone now. Is This Going to Work? There's no shortage of criticism of Apple over its App Store policies, and I've vented my own anger about it. However, I will give them credit: their screening of apps has done a great job of weeding out the crap and gives an excellent indication that the app will work on my device. How about Android? Nope. It might work on your phone. Maybe. You'll have to try it to see. Get burned by it? Well, write a nasty review to try to keep others from making the mistake you did. If you don't mind doing that stuff, then Android is the platform for you. Personally, I'd rather have a receptionist screening out the telemarketing and survey calls than hang up on them myself, but that's your call. Slow, Slowing, Slower All this yapping about multitasking. This is an area I've been on Apple's side from the beginning. Sorry folks, but this is the number one reason I hated Windows Mobile: the longer you use it, the slower it gets because it doesn't kill apps. I'm with Steve Jobs on this one: if you see a task manager, we're doing it wrong. I don't want to have to manually kill apps. I hate doing that on Windows let alone on a mobile device. To me, priority one should be keeping the device speedy. Waiting for your device to respond is unacceptable. Bonus! Taken from iPhone Letdown? 8 Things Apple Didn't Announce, here are my responses: 4G Yeah, let me know if your area actually has it. I live in Lincoln, Nebraska. No carrier is going to have 4G here for at least 3 years. Meanwhile, you still get to pay for it. Yay! Cloud iTunes/OTA Sync You got me here. Of course, whether or not your Android device will be able to do it is always a good question. 3G Video Chat You got me here, too. I'm sure you spent countless hours in front of your phone with video chat. Also, I can't wait for the "No Video Chat While Driving" laws. Mobile Hotspot This is a neat feature, but as the author points out, it's left up to the carrier whether to implement it or not. Pretty sure any Android phones that come to AT&T won't have this enabled in the foreseeable future. Is Verizon even allowing this? I just figured Sprint was because they're failing so hard at keeping customers. Free MobileMe I use Google's services with my iPhone. The only people I know who use MobileMe are Apple fanboys and fangirls. If you choose to pay for a service that you can get for free, that's your decision, not Apple's. Voice Input Voice input has been available on phones (even "dumb" phones) for years now. iPhone does have the ability, though limited. Why don't I hear people telling their phones what to do? Maybe because it's still easier to use your fingers than talk to it. Get back to me when this becomes an important feature. Free Navigation Maybe this will be a bigger deal to me now that I'm getting a phone with GPS, but when using my buddy's 3gs, Google maps has worked just fine. Maybe I just don't trust turn-by-turn navigation enough to want it. Dashboard The only legitimate complaint on this list, to me. iPhone's home screen is pathetic, doubly so for the iPad. What a waste of perfectly usable space. I also want to add notifications to this list. Android's notification panel is far superior to the iPhone's. I don't want to hunt all over my screen to find little red dots. Put 'em in one place, Apple.

  Read the article

• Conducting Effective Web Meetings

  - by BuckWoody

  There are several forms of corporate communication. From immediate, rich communications like phones and IM messaging to historical transactions like e-mail, there are a lot of ways to get information to one or more people. From time to time, it's even useful to have a meeting. (This is where a witty picture of a guy sleeping in a meeting goes. I won't bother actually putting one here; you're already envisioning it in your mind) Most meetings are pointless, and a complete waste of time. This is the fault, completely and solely, of the organizer. It's because he or she hasn't thought things through enough to think about alternate forms of information passing. Here's the criteria for a good meeting - whether in-person or over the web: 100% of the content of a meeting should require the participation of 100% of the attendees for 100% of the time It doesn't get any simpler than that. If it doesn't meet that criteria, then don't invite that person to that meeting. If you're just conveying information and no one has the need for immediate interaction with that information (like telling you something that modifies the message), then send an e-mail. If you're a manager, and you need to get status from lots of people, pick up the phone.If you need a quick answer, use IM. I once had a high-level manager that called frequent meetings. His real need was status updates on various processes, so 50 of us would sit in a room while he asked each one of us questions. He believed this larger meeting helped us "cross pollinate ideas". In fact, it was a complete waste of time for most everyone, except in the one or two moments that they interacted with him. So I wrote some code for a Palm Pilot (which was a kind of SmartPhone but with no phone and no real graphics, but this was in the days when we had just discovered fire and the wheel, although the order of those things is still in debate) that took an average of the salaries of the people in the room (I guessed at it) and ran a timer which multiplied the number of people against the salaries. I left that running in plain sight for him, and when he asked about it, I explained how much the meetings were really costing the company. We had far fewer meetings after. Meetings are now web-enabled. I believe that's largely a good thing, since it saves on travel time and allows more people to participate, but I think the rule above still holds. And in fact, there are some other rules that you should follow to have a great meeting - and fewer of them. Be Clear About the Goal This is important in any meeting, but all of us have probably gotten an invite with a web link and an ambiguous title. Then you get to the meeting, and it's a 500-level deep-dive on something everyone expects you to know. This is unfair to the "expert" and to the participants. I always tell people that invite me to a meeting that I will be as detailed as I can - but the more detail they can tell me about the questions, the more detailed I can be in my responses. Granted, there are times when you don't know what you don't know, but the more you can say about the topic the better. There's another point here - and it's that you should have a clearly defined "win" for the meeting. When the meeting is over, and everyone goes back to work, what were you expecting them to do with the information? Have that clearly defined in your head, and in the meeting invite. Understand the Technology There are several web-meeting clients out there. I use them all, since I meet with clients all over the world. They all work differently - so I take a few moments and read up on the different clients and find out how I can use the tools properly. I do this with the technology I use for everything else, and it's important to understand it if the meeting is to be a success. If you're running the meeting, know the tools. I don't care if you like the tools or not, learn them anyway. Don't waste everyone else's time just because you're too bitter/snarky/lazy to spend a few minutes reading. Check your phone or mic. Check your video size. Install (and learn to use)  ZoomIT (http://technet.microsoft.com/en-us/sysinternals/bb897434.aspx). Format your slides or screen or output correctly. Learn to use the voting features of the meeting software, and especially it's whiteboard features. Figure out how multiple monitors work. Try a quick meeting with someone to test all this. Do this *before* you invite lots of other people to your meeting.   Use a WebCam I'm not a pretty man. I have a face fit for radio. But after attending a meeting with clients where one Microsoft person used a webcam and another did not, I'm convinced that people pay more attention when a face is involved. There are tons of studies around this, or you can take my word for it, but toss a shirt on over those pajamas and turn the webcam on. Set Up Early Whether you're attending or leading the meeting, don't wait to sign on to the meeting at the time when it starts. I can almost plan that a 10:00 meeting will actually start at 10:10 because the participants/leader is just now installing the web client for the meeting at 10:00. Sign on early, go on mute, and then wait for everyone to arrive. Mute When Not Talking No one wants to hear your screaming offspring / yappy dog / other cubicle conversations / car wind noise (are you driving in a desert storm or something?) while the person leading the meeting is trying to talk. I use the Lync software from Microsoft for my meetings, and I mute everyone by default, and then tell them to un-mute to talk to the group. Share Collateral If you have a PowerPoint deck, mail it out in case you have a tech failure. If you have a document, share it as an attachment to the meeting. Don't make people ask you for the information - that's why you're there to begin with. Even better, send it out early. "But", you say, "then no one will come to the meeting if they have the deck first!" Uhm, then don't have a meeting. Send out the deck and a quick e-mail and let everyone get on with their productive day. Set Actions At the Meeting A meeting should have some sort of outcome (see point one). That means there are actions to take, a follow up, or some deliverable. Otherwise, it's an e-mail. At the meeting, decide who will do what, when things are needed, and so on. And avoid, if at all possible, setting up another meeting, unless absolutely necessary. So there you have it. Whether it's on-premises or on the web, meetings are a necessary evil, and should be treated that way. Like politicians, you should have as few of them as are necessary to keep the roads paved and public libraries open.

  Read the article

• Notifications for Expiring DBSNMP Passwords

  - by Courtney Llamas

  Most user accounts these days have a password profile on them that automatically expires the password after a set number of days.   Depending on your company’s security requirements, this may be as little as 30 days or as long as 365 days, although typically it falls between 60-90 days. For a normal user, this can cause a small interruption in your day as you have to go get your password reset by an admin. When this happens to privileged accounts, such as the DBSNMP account that is responsible for monitoring database availability, it can cause bigger problems. In Oracle Enterprise Manager 12c you may notice the error message “ORA-28002: the password will expire within 5 days” when you connect to a target, or worse you may get “ORA-28001: the password has expired". If you wait too long, your monitoring will fail because the password is locked out. Wouldn’t it be nice if we could get an alert 10 days before our DBSNMP password expired? Thanks to Oracle Enterprise Manager 12c Metric Extensions (ME), you can! See the Oracle Enterprise Manager Cloud Control Administrator’s Guide for more information on Metric Extensions. To create a metric extension, select Enterprise / Monitoring / Metric Extensions, and then click on Create. On the General Properties screen select either Cluster Database or Database Instance, depending on which target you need to monitor.  If you have both RAC and Single instance you may need to create one for each. In this example we will create a Cluster Database metric.  Enter a Name for the ME and a Display Name. Then select SQL for the Adapter.  Adjust the Collection Schedule as desired, for this example we will collect this metric every 1 day. Notice for metric collected every day, we can determine the exact time we want to collect. On the Adapter page, enter the query that you wish to execute.  In this example we will use the query below that specifically checks for the DBSNMP user that is expiring within 10 days. Of course, you can adjust this query to alert for any user that can cause an outage such as an application account or service account such as RMAN. select username, account_status, trunc(expiry_date-sysdate) days_to_expirefrom dba_userswhere username = 'DBSNMP'and expiry_date is not null; The next step is to create the columns to store the data returned from the query.  Click Add and add a column for each of the fields in the same order that data is returned.  The table below will help you complete the column additions. Name Display Name Column Type Value Type Metric Category Unit Username User Name Key String Security AccountStatus Account Status Data String Security DaysToExpire Days Until Expiration Data Number Security Days When creating the DaysToExpire column, you can add a default threshold here for Warning and Critical (say < 10 and 5).  When all columns have been added, click Next. On the Credentials page, you can choose to use the default monitoring credentials or specify new credentials.  We will use the default credentials established for our target (dbsnmp). The next step is to test your Metric Extension.  Click on Add to select a target for testing, then click Select. Now click the button Run Test to execute the test against the selected target(s). We can see in the example below that the Metric Extension has executed and returned a value of 68 days to expire. Click Next to proceed. Review the metric extension in the final screen and click Finish. The metric will be created in Editable status.  Select the metric, click Actions and select Deployable Draft. You can do this once more to move to Published. Finally, we want to apply this metric to a target. When managing many targets, it’s best to add your metric to a template, for details on adding a Metric Extension to a template see the Administrator’s Guide. For this example, we will deploy this to a target directly. Select Actions / Deploy to Targets. Click Add and select the target you wish to deploy to and click Submit.  Once deployment is complete, we can go to the target and view the Metric & Collection Settings to see the new metric and its thresholds.   After some time, you will find the metric has collected and the days to expiration for DBSNMP user can be seen in the All Metrics view.   For metrics collected once per day, you may have to wait up to 24 hours to see the metric and current severity. In the example below, the current severity is Clear (green check) as it is not scheduled to expire within 10 days. To test the notification, we can edit the thresholds for the new metric so they trigger an alert.  Our password expires in 139 days, so we’ll change our Warning to 140 and leave Critical at 5, in our example we also changed the collection time to every 5 minutes.  At the next collection, you’ll find that the current severity changes to a Warning and any related Incident Rules would be triggered to create an Incident or Notification as desired. Now that you get a notification that your DBSNMP passwords is about to expire, you can use OEM Command Line Interface (EM CLI) verb update_db_password to change it at both the database target and the OEM target in one step.  The caveat is you must know the existing password to use the update_db_password command.  To learn more about EM CLI, see the Oracle Enterprise Manager Command Line Interface Guide.  Below is an example of changing the password with the update_db_password verb.  $ ./emcli update_db_password -target_name=emrep -target_type=oracle_database -user_name=dbsnmp -change_at_target=yes -change_all_references=yes Enter value for old_password :Enter value for new_password :Enter value for retype_new_password :Successfully submitted a job to change the password in Enterprise Manager and on the target database: "emrep"Execute "emcli get_jobs -job_id=FA66C1C4D663297FE0437656F20ACC84" to check the status of the job.Search for job name "CHANGE_PWD_JOB_FA66C1C4D662297FE0437656F20ACC84" on the Jobs home page to check job execution details. The subsequent job created will typically run quickly enough that a blackout is not needed, however if you submit a script with many targets to change, your job may run slower so adding a blackout to the script is recommended. $ ./emcli get_jobs -job_id=FA66C1C4D663297FE0437656F20ACC84 Name Type Job ID Execution ID Scheduled Completed TZ Offset Status Status ID Owner Target Type Target Name CHANGE_PWD_JOB_FA66C1C4D662297FE0437656F20ACC84 ChangePassword FA66C1C4D663297FE0437656F20ACC84 FA66C1C4D665297FE0437656F20ACC84 2014-05-28 09:39:12 2014-05-28 09:39:18 GMT-07:00 Succeeded 5 SYSMAN oracle_database emrep After implementing the above Metric Extension and using the EM CLI update_db_password verb, you will be able to stay on top of your DBSNMP password changes without experiencing an unplanned monitoring outage.  

  Read the article

• Problem with Android emulator

  - by benasio

  Projects do not run, on screen emulator only "ANDROID" WinXP pro SP3/Eclipse Galileo java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing) My actions: 1.Start the emulator(Platform:2.1 API Level:7), wait until the window DDMS status will change to ONLINE 2.Launches helloandroid from examples - Run as Android Application Console: Android Launch! [2010-05-03 21:44:34 - HelloAndroid] adb is running normally. [2010-05-03 21:44:34 - HelloAndroid] Performing com.example.helloandroid.HelloAndroid activity launch [2010-05-03 21:44:34 - HelloAndroid] Automatic Target Mode: using existing emulator 'emulator-5554' running compatible AVD 'my_vm' [2010-05-03 21:44:34 - HelloAndroid] WARNING: Application does not specify an API level requirement! [2010-05-03 21:44:34 - HelloAndroid] Device API version is 7 (Android 2.1) [2010-05-03 21:44:34 - HelloAndroid] Uploading HelloAndroid.apk onto device 'emulator-5554' [2010-05-03 21:44:35 - HelloAndroid] Installing HelloAndroid.apk... [2010-05-03 21:45:07 - HelloAndroid] Success! [2010-05-03 21:45:08 - HelloAndroid] Starting activity com.example.helloandroid.HelloAndroid on device [2010-05-03 21:45:28 - HelloAndroid] ActivityManager: DDM dispatch reg wait timeout [2010-05-03 21:45:28 - HelloAndroid] ActivityManager: Can't dispatch DDM chunk 52454151: no handler defined [2010-05-03 21:45:28 - HelloAndroid] ActivityManager: Can't dispatch DDM chunk 48454c4f: no handler defined [2010-05-03 21:45:28 - HelloAndroid] ActivityManager: Can't dispatch DDM chunk 46454154: no handler defined [2010-05-03 21:45:28 - HelloAndroid] ActivityManager: Can't dispatch DDM chunk 4d505251: no handler defined [2010-05-03 21:45:52 - HelloAndroid] Device not ready. Waiting 3 seconds before next attempt. [2010-05-03 21:45:52 - HelloAndroid] ActivityManager: android.util.AndroidException: Can't connect to activity manager; is the system running? [2010-05-03 21:45:55 - HelloAndroid] Starting activity com.example.helloandroid.HelloAndroid on device [2010-05-03 21:46:11 - HelloAndroid] ActivityManager: DDM dispatch reg wait timeout ...... DDMS console (only errors and warnings) 05-03 17:43:52.437: ERROR/vold(26): Error opening switch name path '/sys/class/switch/test2' (No such file or directory) 05-03 17:43:52.437: ERROR/vold(26): Error bootstrapping switch '/sys/class/switch/test2' (No such file or directory) 05-03 17:43:52.437: ERROR/vold(26): Error opening switch name path '/sys/class/switch/test' (No such file or directory) 05-03 17:43:52.437: ERROR/vold(26): Error bootstrapping switch '/sys/class/switch/test' (No such file or directory) 05-03 17:48:34.036: WARN/Zygote(29): Preloaded drawable resource #0x1080093 (res/drawable-mdpi/sym_def_app_icon.png) that varies with configuration!! 05-03 17:48:34.406: WARN/Zygote(29): Preloaded drawable resource #0x1080002 (res/drawable-mdpi/arrow_down_float.png) that varies with configuration!! 05-03 17:48:35.836: WARN/Zygote(29): Preloaded drawable resource #0x10800b4 (res/drawable/btn_check.xml) that varies with configuration!! 05-03 17:48:36.076: WARN/Zygote(29): Preloaded drawable resource #0x10800b7 (res/drawable-mdpi/btn_check_label_background.9.png) that varies with configuration!! 05-03 17:48:36.106: WARN/Zygote(29): Preloaded drawable resource #0x10800b8 (res/drawable-mdpi/btn_check_off.png) that varies with configuration!! 05-03 17:48:36.147: WARN/Zygote(29): Preloaded drawable resource #0x10800bd (res/drawable-mdpi/btn_check_on.png) that varies with configuration!! 05-03 17:48:36.437: WARN/Zygote(29): Preloaded drawable resource #0x1080004 (res/drawable/btn_default.xml) that varies with configuration!! 05-03 17:48:36.716: WARN/Zygote(29): Preloaded drawable resource #0x1080005 (res/drawable/btn_default_small.xml) that varies with configuration!! 05-03 17:48:36.966: WARN/Zygote(29): Preloaded drawable resource #0x1080006 (res/drawable/btn_dropdown.xml) that varies with configuration!! 05-03 17:48:37.326: WARN/Zygote(29): Preloaded drawable resource #0x1080008 (res/drawable/btn_plus.xml) that varies with configuration!! 05-03 17:48:37.707: WARN/Zygote(29): Preloaded drawable resource #0x1080007 (res/drawable/btn_minus.xml) that varies with configuration!! 05-03 17:48:38.057: WARN/Zygote(29): Preloaded drawable resource #0x1080009 (res/drawable/btn_radio.xml) that varies with configuration!! 05-03 17:48:38.776: WARN/Zygote(29): Preloaded drawable resource #0x108000a (res/drawable/btn_star.xml) that varies with configuration!! 05-03 17:48:39.327: WARN/Zygote(29): Preloaded drawable resource #0x1080125 (res/drawable/btn_toggle.xml) that varies with configuration!! 05-03 17:48:39.416: WARN/Zygote(29): Preloaded drawable resource #0x1080187 (res/drawable-mdpi/ic_emergency.png) that varies with configuration!! 05-03 17:48:39.506: WARN/Zygote(29): Preloaded drawable resource #0x1080012 (res/drawable-mdpi/divider_horizontal_bright.9.png) that varies with configuration!! 05-03 17:48:39.576: WARN/Zygote(29): Preloaded drawable resource #0x1080014 (res/drawable-mdpi/divider_horizontal_dark.9.png) that varies with configuration!! 05-03 17:48:40.126: WARN/Zygote(29): Preloaded drawable resource #0x1080016 (res/drawable/edit_text.xml) that varies with configuration!! 05-03 17:48:40.507: WARN/Zygote(29): Preloaded drawable resource #0x1080161 (res/drawable/expander_group.xml) that varies with configuration!! 05-03 17:48:41.036: WARN/Zygote(29): Preloaded drawable resource #0x1080062 (res/drawable/list_selector_background.xml) that varies with configuration!! 05-03 17:48:41.177: WARN/Zygote(29): Preloaded drawable resource #0x1080217 (res/drawable-mdpi/menu_background.9.png) that varies with configuration!! 05-03 17:48:41.256: WARN/Zygote(29): Preloaded drawable resource #0x1080218 (res/drawable-mdpi/menu_background_fill_parent_width.9.png) that varies with configuration!! 05-03 17:48:41.567: WARN/Zygote(29): Preloaded drawable resource #0x1080219 (res/drawable/menu_selector.xml) that varies with configuration!! 05-03 17:48:41.706: WARN/Zygote(29): Preloaded drawable resource #0x1080224 (res/drawable-mdpi/panel_background.9.png) that varies with configuration!! 05-03 17:48:41.849: WARN/Zygote(29): Preloaded drawable resource #0x108022e (res/drawable-mdpi/popup_bottom_bright.9.png) that varies with configuration!! 05-03 17:48:42.026: WARN/Zygote(29): Preloaded drawable resource #0x108022f (res/drawable-mdpi/popup_bottom_dark.9.png) that varies with configuration!! 05-03 17:48:42.156: WARN/Zygote(29): Preloaded drawable resource #0x1080230 (res/drawable-mdpi/popup_bottom_medium.9.png) that varies with configuration!! 05-03 17:48:42.276: WARN/Zygote(29): Preloaded drawable resource #0x1080231 (res/drawable-mdpi/popup_center_bright.9.png) that varies with configuration!! 05-03 17:48:42.376: WARN/Zygote(29): Preloaded drawable resource #0x1080232 (res/drawable-mdpi/popup_center_dark.9.png) that varies with configuration!! 05-03 17:48:42.507: WARN/Zygote(29): Preloaded drawable resource #0x1080235 (res/drawable-mdpi/popup_full_dark.9.png) that varies with configuration!! 05-03 17:48:42.606: WARN/Zygote(29): Preloaded drawable resource #0x1080238 (res/drawable-mdpi/popup_top_bright.9.png) that varies with configuration!! 05-03 17:48:42.696: WARN/Zygote(29): Preloaded drawable resource #0x1080239 (res/drawable-mdpi/popup_top_dark.9.png) that varies with configuration!! 05-03 17:48:42.946: WARN/Zygote(29): Preloaded drawable resource #0x108006d (res/drawable/progress_indeterminate_horizontal.xml) that varies with configuration!! 05-03 17:48:43.076: WARN/Zygote(29): Preloaded drawable resource #0x108023f (res/drawable/progress_small.xml) that varies with configuration!! 05-03 17:48:43.456: WARN/Zygote(29): Preloaded drawable resource #0x1080240 (res/drawable/progress_small_titlebar.xml) that varies with configuration!! 05-03 17:48:43.957: WARN/Zygote(29): Preloaded drawable resource #0x1080262 (res/drawable-mdpi/scrollbar_handle_horizontal.9.png) that varies with configuration!! 05-03 17:48:44.036: WARN/Zygote(29): Preloaded drawable resource #0x1080263 (res/drawable-mdpi/scrollbar_handle_vertical.9.png) that varies with configuration!! 05-03 17:48:44.176: WARN/Zygote(29): Preloaded drawable resource #0x1080071 (res/drawable/spinner_dropdown_background.xml) that varies with configuration!! 05-03 17:48:44.317: WARN/Zygote(29): Preloaded drawable resource #0x1080326 (res/drawable-mdpi/title_bar_shadow.9.png) that varies with configuration!! 05-03 17:48:44.496: WARN/Zygote(29): Preloaded drawable resource #0x10801c6 (res/drawable-mdpi/indicator_code_lock_drag_direction_green_up.png) that varies with configuration!! 05-03 17:48:44.607: WARN/Zygote(29): Preloaded drawable resource #0x10801c7 (res/drawable-mdpi/indicator_code_lock_drag_direction_red_up.png) that varies with configuration!! 05-03 17:48:45.956: WARN/Zygote(29): Preloaded drawable resource #0x10801c8 (res/drawable-mdpi/indicator_code_lock_point_area_default.png) that varies with configuration!! 05-03 17:48:46.407: WARN/Zygote(29): Preloaded drawable resource #0x10801c9 (res/drawable-mdpi/indicator_code_lock_point_area_green.png) that varies with configuration!! 05-03 17:48:46.696: WARN/Zygote(29): Preloaded drawable resource #0x10801ca (res/drawable-mdpi/indicator_code_lock_point_area_red.png) that varies with configuration!! 05-03 17:48:56.307: ERROR/BatteryService(170): usbOnlinePath not found 05-03 17:48:56.336: ERROR/BatteryService(170): batteryVoltagePath not found 05-03 17:48:56.350: ERROR/BatteryService(170): batteryTemperaturePath not found 05-03 17:48:56.696: ERROR/SurfaceFlinger(170): Couldn't open /sys/power/wait_for_fb_sleep or /sys/power/wait_for_fb_wake 05-03 17:48:57.847: WARN/SurfaceFlinger(170): ro.sf.lcd_density not defined, using 160 dpi by default. 05-03 17:49:02.116: WARN/UsageStats(170): Usage stats version changed; dropping 05-03 17:49:05.036: WARN/zipro(182): Unable to open zip '/data/local/bootanimation.zip': No such file or directory 05-03 17:49:06.297: WARN/zipro(182): Unable to open zip '/system/media/bootanimation.zip': No such file or directory 05-03 17:49:50.637: WARN/PackageManager(170): Running ENG build: no pre-dexopt! 05-03 17:53:59.196: WARN/PackageManager(170): Unknown permission com.google.android.providers.gmail.permission.WRITE_GMAIL in package com.android.settings 05-03 17:53:59.238: WARN/PackageManager(170): Unknown permission com.google.android.providers.gmail.permission.READ_GMAIL in package com.android.settings 05-03 17:53:59.286: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH in package com.android.settings 05-03 17:53:59.517: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH in package com.android.providers.contacts 05-03 17:53:59.656: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH.cp in package com.android.providers.contacts 05-03 17:53:59.717: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH.mail in package com.android.contacts 05-03 17:53:59.796: WARN/PackageManager(170): Unknown permission android.permission.ADD_SYSTEM_SERVICE in package com.android.phone 05-03 17:54:00.126: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH in package com.android.development 05-03 17:54:00.206: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH.ALL_SERVICES in package com.android.development 05-03 17:54:00.206: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH.YouTubeUser in package com.android.development 05-03 17:54:00.237: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.ACCESS_GOOGLE_PASSWORD in package com.android.development 05-03 17:54:00.258: WARN/PackageManager(170): Unknown permission com.google.android.googleapps.permission.GOOGLE_AUTH in package com.android.browser 05-03 17:54:25.456: WARN/ResourceType(170): Resources don't contain package for resource number 0x7f0700e5 05-03 17:54:25.486: WARN/ResourceType(170): Resources don't contain package for resource number 0x7f020031 05-03 17:54:25.536: WARN/ResourceType(170): Resources don't contain package for resource number 0x7f020030 05-03 17:54:25.576: WARN/ResourceType(170): Resources don't contain package for resource number 0x7f050000 05-03 17:54:38.708: WARN/SharedBufferStack(182): waitForCondition(LockCondition) timed out (identity=0, status=0). CPU may be pegged. trying again.

  Read the article

• Crash: iPhone Threading with Blocks

  - by jtbandes

  I have some convenience methods set up for threading with blocks (using PLBlocks). Then in the main portion of my code, I call the method -fetchArrivalsForLocationIDs:callback:errback:, which runs some web API calls in the background. Here's the problem: when I comment out the 2 NSAutoreleasePool-related lines in JTBlockThreading.m, of course I get lots of Object 0x6b31280 of class __NSArrayM autoreleased with no pool in place - just leaking errors. However, if I uncomment them, the app frequently crashes on the [pool release]; line, sometimes saying malloc: *** error for object 0x6e3ae10: pointer being freed was not allocated" *** set a breakpoint in malloc_error_break to debug. I assume I've made a horrible mistake/assumption in threading somewhere, but can anyone figure out what exactly the problem is? // JTBlockThreading.h #import <Foundation/Foundation.h> #import <PLBlocks/Block.h> #define JT_BLOCKTHREAD_BACKGROUND [self invokeBlockInBackground:^{ #define JT_BLOCKTHREAD_MAIN [self invokeBlockOnMainThread:^{ #define JT_BLOCKTHREAD_END }]; #define JT_BLOCKTHREAD_BACKGROUND_END_WAIT } waitUntilDone:YES]; @interface NSObject (JTBlockThreading) - (void)invokeBlockInBackground:(void (^)())block; - (void)invokeBlockOnMainThread:(void (^)())block; - (void)invokeBlockOnMainThread:(void (^)())block waitUntilDone:(BOOL)wait; - (void)invokeBlock:(void (^)())block; @end // JTBlockThreading.m #import "JTBlockThreading.h" @implementation NSObject (JTBlockThreading) - (void)invokeBlockInBackground:(void (^)())block { [self performSelectorInBackground:@selector(invokeBlock:) withObject:[block copy]]; } - (void)invokeBlockOnMainThread:(void (^)())block { [self invokeBlockOnMainThread:block waitUntilDone:NO]; } - (void)invokeBlockOnMainThread:(void (^)())block waitUntilDone:(BOOL)wait { [self performSelectorOnMainThread:@selector(invokeBlock:) withObject:[block copy] waitUntilDone:wait]; } - (void)invokeBlock:(void (^)())block { //NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; block(); [block release]; //[pool release]; } @end - (void)fetchArrivalsForLocationIDs:(NSString *)locIDs callback:(JTWSCallback)callback errback:(JTWSErrback)errback { JT_PUSH_NETWORK(); JT_BLOCKTHREAD_BACKGROUND NSError *error = nil; // Create API call URL NSURL *url = [NSURL URLWithString: [NSString stringWithFormat:@"%@/arrivals/appID/%@/locIDs/%@", TRIMET_BASE_URL, appID, locIDs]]; if (!url) { JT_BLOCKTHREAD_MAIN errback(@"That’s not a valid Stop ID!"); JT_POP_NETWORK(); JT_BLOCKTHREAD_END return; } // Call API NSData *data = [NSData dataWithContentsOfURL:url options:0 error:&error]; if (!data) { JT_BLOCKTHREAD_MAIN errback([NSString stringWithFormat: @"Had trouble downloading the arrival data! %@", [error localizedDescription]]); JT_POP_NETWORK(); JT_BLOCKTHREAD_END return; } CXMLDocument *doc = [[CXMLDocument alloc] initWithData:data options:0 error:&error]; if (!doc) { JT_BLOCKTHREAD_MAIN // TODO: further error description // (TouchXML doesn't provide information with the error) errback(@"Had trouble reading the arrival data!"); JT_POP_NETWORK(); JT_BLOCKTHREAD_END return; } NSArray *nodes = nil; CXMLElement *resultSet = [doc rootElement]; // Begin building the response model JTWSResponseArrivalData *response = [[[JTWSResponseArrivalData alloc] init] autorelease]; response.queryTime = [NSDate JT_dateWithTriMetWSTimestamp: [[resultSet attributeValueForName:@"queryTime"] longLongValue]]; if (!response.queryTime) { // TODO: further error check? NSLog(@"Hm, query time is nil in %s... response %@, resultSet %@", __PRETTY_FUNCTION__, response, resultSet); } nodes = [resultSet nodesForXPath:@"//arrivals:errorMessage" namespaceMappings:namespaceMappings error:&error]; if ([nodes count] > 0) { NSString *message = [[nodes objectAtIndex:0] stringValue]; response.errorMessage = message; // TODO: this probably won't be used... JT_BLOCKTHREAD_MAIN errback([NSString stringWithFormat: @"TriMet error: “%@”", message]); JT_POP_NETWORK(); JT_BLOCKTHREAD_END return; } // Build location models nodes = [resultSet nodesForXPath:@"/arrivals:location" namespaceMappings:namespaceMappings error:&error]; if ([nodes count] <= 0) { NSLog(@"Hm, no locations returned in %s... xpath error %@, response %@, resultSet %@", __PRETTY_FUNCTION__, error, response, resultSet); } NSMutableArray *locations = [NSMutableArray arrayWithCapacity:[nodes count]]; for (CXMLElement *loc in nodes) { JTWSLocation *location = [[[JTWSLocation alloc] init] autorelease]; location.desc = [loc attributeValueForName:@"desc"]; location.dir = [loc attributeValueForName:@"dir"]; location.position = [[[CLLocation alloc] initWithLatitude:[[loc attributeValueForName:@"lat"] doubleValue] longitude:[[loc attributeValueForName:@"lng"] doubleValue]] autorelease]; location.locID = [[loc attributeValueForName:@"locid"] integerValue]; } // Build arrival models nodes = [resultSet nodesForXPath:@"/arrivals:arrival" namespaceMappings:namespaceMappings error:&error]; if ([nodes count] <= 0) { NSLog(@"Hm, no arrivals returned in %s... xpath error %@, response %@, resultSet %@", __PRETTY_FUNCTION__, error, response, resultSet); } NSMutableArray *arrivals = [[NSMutableArray alloc] initWithCapacity:[nodes count]]; for (CXMLElement *arv in nodes) { JTWSArrival *arrival = [[JTWSArrival alloc] init]; arrival.block = [[arv attributeValueForName:@"block"] integerValue]; arrival.piece = [[arv attributeValueForName:@"piece"] integerValue]; arrival.locID = [[arv attributeValueForName:@"locid"] integerValue]; arrival.departed = [[arv attributeValueForName:@"departed"] boolValue]; // TODO: verify arrival.detour = [[arv attributeValueForName:@"detour"] boolValue]; // TODO: verify arrival.direction = (JTWSRouteDirection)[[arv attributeValueForName:@"dir"] integerValue]; arrival.estimated = [NSDate JT_dateWithTriMetWSTimestamp: [[arv attributeValueForName:@"estimated"] longLongValue]]; arrival.scheduled = [NSDate JT_dateWithTriMetWSTimestamp: [[arv attributeValueForName:@"scheduled"] longLongValue]]; arrival.fullSign = [arv attributeValueForName:@"fullSign"]; arrival.shortSign = [arv attributeValueForName:@"shortSign"]; NSString *status = [arv attributeValueForName:@"status"]; if ([status isEqualToString:@"estimated"]) { arrival.status = JTWSArrivalStatusEstimated; } else if ([status isEqualToString:@"scheduled"]) { arrival.status = JTWSArrivalStatusScheduled; } else if ([status isEqualToString:@"delayed"]) { arrival.status = JTWSArrivalStatusDelayed; } else if ([status isEqualToString:@"canceled"]) { arrival.status = JTWSArrivalStatusCanceled; } else { NSLog(@"Unknown arrival status %s in %@... response %@, arrival %@", status, __PRETTY_FUNCTION__, response, arv); } NSArray *blockPositions = [arv nodesForXPath:@"/arrivals:blockPosition" namespaceMappings:namespaceMappings error:&error]; if ([blockPositions count] > 1) { // The schema allows for any number of blockPosition elements, // but I'm really not sure why... NSLog(@"Hm, more than one blockPosition in %s... response %@, arrival %@", __PRETTY_FUNCTION__, response, arv); } if ([blockPositions count] > 0) { CXMLElement *bpos = [blockPositions objectAtIndex:0]; JTWSBlockPosition *blockPosition = [[JTWSBlockPosition alloc] init]; blockPosition.reported = [NSDate JT_dateWithTriMetWSTimestamp: [[bpos attributeValueForName:@"at"] longLongValue]]; blockPosition.feet = [[bpos attributeValueForName:@"feet"] integerValue]; blockPosition.position = [[[CLLocation alloc] initWithLatitude:[[bpos attributeValueForName:@"lat"] doubleValue] longitude:[[bpos attributeValueForName:@"lng"] doubleValue]] autorelease]; NSString *headingStr = [bpos attributeValueForName:@"heading"]; if (headingStr) { // Valid CLLocationDirections are > 0 CLLocationDirection heading = [headingStr integerValue]; while (heading < 0) heading += 360.0; blockPosition.heading = heading; } else { blockPosition.heading = -1; // indicates invalid heading } NSArray *tripData = [bpos nodesForXPath:@"/arrivals:trip" namespaceMappings:namespaceMappings error:&error]; NSMutableArray *trips = [[NSMutableArray alloc] initWithCapacity:[tripData count]]; for (CXMLElement *tripDatum in tripData) { JTWSTrip *trip = [[JTWSTrip alloc] init]; trip.desc = [tripDatum attributeValueForName:@"desc"]; trip.destDist = [[tripDatum attributeValueForName:@"destDist"] integerValue]; trip.direction = (JTWSRouteDirection)[[tripDatum attributeValueForName:@"dir"] integerValue]; trip.pattern = [[tripDatum attributeValueForName:@"pattern"] integerValue]; trip.progress = [[tripDatum attributeValueForName:@"progress"] integerValue]; trip.route = [[tripDatum attributeValueForName:@"route"] integerValue]; [trips addObject:trip]; [trip release]; } blockPosition.trips = trips; [trips release]; NSArray *layoverData = [bpos nodesForXPath:@"/arrivals:layover" namespaceMappings:namespaceMappings error:&error]; NSMutableArray *layovers = [[NSMutableArray alloc] initWithCapacity:[layoverData count]]; for (CXMLElement *layoverDatum in layoverData) { JTWSLayover *layover = [[JTWSLayover alloc] init]; layover.start = [NSDate JT_dateWithTriMetWSTimestamp: [[layoverDatum attributeValueForName:@"start"] longLongValue]]; layover.end = [NSDate JT_dateWithTriMetWSTimestamp: [[layoverDatum attributeValueForName:@"end"] longLongValue]]; // TODO: it seems the API can send a <location> inside a layover (undocumented)... support? [layovers addObject:layover]; [layover release]; } blockPosition.layovers = layovers; [layovers release]; arrival.blockPosition = blockPosition; [blockPosition release]; } [arrivals addObject:arrival]; [arrival release]; } // Add arrivals to corresponding locations for (JTWSLocation *loc in locations) { loc.arrivals = [arrivals selectWithBlock:^BOOL (id arv) { return loc.locID == ((JTWSArrival *)arv).locID; }]; } [arrivals release]; response.locations = locations; [locations release]; // Build route status models (used in inclement weather) nodes = [resultSet nodesForXPath:@"/arrivals:routeStatus" namespaceMappings:namespaceMappings error:&error]; NSMutableArray *routeStatuses = [NSMutableArray arrayWithCapacity:[nodes count]]; for (CXMLElement *stat in nodes) { JTWSRouteStatus *status = [[JTWSRouteStatus alloc] init]; status.route = [[stat attributeValueForName:@"route"] integerValue]; NSString *statusStr = [stat attributeValueForName:@"status"]; if ([statusStr isEqualToString:@"estimatedOnly"]) { status.status = JTWSRouteStatusTypeEstimatedOnly; } else if ([statusStr isEqualToString:@"off"]) { status.status = JTWSRouteStatusTypeOff; } else { NSLog(@"Unknown route status type %s in %@... response %@, routeStatus %@", status, __PRETTY_FUNCTION__, response, stat); } [routeStatuses addObject:status]; [status release]; } response.routeStatuses = routeStatuses; [routeStatuses release]; JT_BLOCKTHREAD_MAIN callback(response); JT_POP_NETWORK(); JT_BLOCKTHREAD_END JT_BLOCKTHREAD_END }

  Read the article

• Reading Data from DDFS ValueError: No JSON object could be decoded

  - by secumind

  I'm running dozens of map reduce jobs for a number of different purposes using disco. My data has grown enormous and I thought I would try using DDFS for a change rather than standard txt files. I've followed the DISCO map/reduce example Counting Words as a map/reduce job, without to much difficulty and with the help of others, Reading JSON specific data into DISCO I've gotten past one of my latest problems. I'm trying to read data in/out of ddfs to better chunk and distribute it but am having a bit of trouble. Here's an example file: file.txt {"favorited": false, "in_reply_to_user_id": null, "contributors": null, "truncated": false, "text": "I'll call him back tomorrow I guess", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016843603968", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/305726905/FASHION-3.png", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1818996723/image_normal.jpg", "profile_sidebar_fill_color": "292727", "is_translator": false, "id": 113532729, "profile_text_color": "000000", "followers_count": 78, "protected": false, "location": "With My Niggas In Paris!", "default_profile_image": false, "listed_count": 0, "utc_offset": -21600, "statuses_count": 6733, "description": "Made in CHINA., Educated && Making My Own $$. Fear GOD && Put Him 1st. #TeamFollowBack #TeamiPhone\n", "friends_count": 74, "profile_link_color": "b03f3f", "profile_image_url": "http://a2.twimg.com/profile_images/1818996723/image_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "1f9199", "id_str": "113532729", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/305726905/FASHION-3.png", "name": "Bee'Jay", "lang": "en", "profile_background_tile": true, "favourites_count": 19, "screen_name": "OohMyBEEsNice", "url": "http://www.bitchimpaid.org", "created_at": "Fri Feb 12 03:32:54 +0000 2010", "contributors_enabled": false, "time_zone": "Central Time (US & Canada)", "profile_sidebar_border_color": "000000", "default_profile": false, "following": null}, "in_reply_to_screen_name": null, "retweet_count": 0, "geo": null, "id": 168931016843603968, "source": "<a href=\"http://twitter.com/#!/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"} {"favorited": false, "in_reply_to_user_id": 50940453, "contributors": null, "truncated": false, "text": "@LegaMrvica @MimozaBand makasi om artis :D kadoo kadoo", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": "168653037894770688", "coordinates": null, "in_reply_to_user_id_str": "50940453", "entities": {"user_mentions": [{"indices": [0, 11], "screen_name": "LegaMrvica", "id": 50940453, "name": "Lega_thePianis", "id_str": "50940453"}, {"indices": [12, 23], "screen_name": "MimozaBand", "id": 375128905, "name": "Mimoza", "id_str": "375128905"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": 168653037894770688, "id_str": "168931016868761600", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "profile_sidebar_fill_color": "DDFFCC", "is_translator": false, "id": 48293450, "profile_text_color": "333333", "followers_count": 182, "protected": false, "location": "\u00dcT: -6.906799,107.622383", "default_profile_image": false, "listed_count": 0, "utc_offset": -28800, "statuses_count": 3052, "description": "Fashion design maranatha '11 // traditional dancer (bali) at sanggar tampak siring & Natya Nataraja", "friends_count": 206, "profile_link_color": "0084B4", "profile_image_url": "http://a3.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "9AE4E8", "id_str": "48293450", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "name": "nana afiff", "lang": "en", "profile_background_tile": true, "favourites_count": 2, "screen_name": "hasnfebria", "url": null, "created_at": "Thu Jun 18 08:50:29 +0000 2009", "contributors_enabled": false, "time_zone": "Pacific Time (US & Canada)", "profile_sidebar_border_color": "BDDCAD", "default_profile": false, "following": null}, "in_reply_to_screen_name": "LegaMrvica", "retweet_count": 0, "geo": null, "id": 168931016868761600, "source": "<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry\u00ae</a>"} {"favorited": false, "in_reply_to_user_id": 27260086, "contributors": null, "truncated": false, "text": "@justinbieber u were born to be somebody, and u're super important in beliebers' life. thanks for all biebs. I love u. follow me? 84", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": "27260086", "entities": {"user_mentions": [{"indices": [0, 13], "screen_name": "justinbieber", "id": 27260086, "name": "Justin Bieber", "id_str": "27260086"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016856178688", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/416005864/Captura.JPG", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "profile_sidebar_fill_color": "f5e7f3", "is_translator": false, "id": 406750700, "profile_text_color": "333333", "followers_count": 1122, "protected": false, "location": "Adentro de una supra.", "default_profile_image": false, "listed_count": 0, "utc_offset": -14400, "statuses_count": 20966, "description": "Mi \u00eddolo es @justinbieber , si te gusta \u00a1genial!, si no, solo respetalo. El cambi\u00f3 mi vida completamente y mi sue\u00f1o es conocerlo #TrueBelieber . ", "friends_count": 1015, "profile_link_color": "9404b8", "profile_image_url": "http://a1.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "notifications": null, "show_all_inline_media": false, "geo_enabled": false, "profile_background_color": "f9fcfa", "id_str": "406750700", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/416005864/Captura.JPG", "name": "neversaynever,right?", "lang": "es", "profile_background_tile": false, "favourites_count": 22, "screen_name": "True_Belieebers", "url": "http://www.wehavebieber-fever.tumblr.com", "created_at": "Mon Nov 07 04:17:40 +0000 2011", "contributors_enabled": false, "time_zone": "Santiago", "profile_sidebar_border_color": "C0DEED", "default_profile": false, "following": null}, "in_reply_to_screen_name": "justinbieber", "retweet_count": 0, "geo": null, "id": 168931016856178688, "source": "<a href=\"http://yfrog.com\" rel=\"nofollow\">Yfrog</a>"} I load it into DDFS with: # ddfs chunk data:test1 ./file.txt created: disco://localhost/ddfs/vol0/blob/44/file_txt-0$549-db27b-125e1 I test that the file is indeed loaded into ddfs with: # ddfs xcat data:test1 {"favorited": false, "in_reply_to_user_id": null, "contributors": null, "truncated": false, "text": "I'll call him back tomorrow I guess", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": null, "entities": {"user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016843603968", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/305726905/FASHION-3.png", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1818996723/image_normal.jpg", "profile_sidebar_fill_color": "292727", "is_translator": false, "id": 113532729, "profile_text_color": "000000", "followers_count": 78, "protected": false, "location": "With My Niggas In Paris!", "default_profile_image": false, "listed_count": 0, "utc_offset": -21600, "statuses_count": 6733, "description": "Made in CHINA., Educated && Making My Own $$. Fear GOD && Put Him 1st. #TeamFollowBack #TeamiPhone\n", "friends_count": 74, "profile_link_color": "b03f3f", "profile_image_url": "http://a2.twimg.com/profile_images/1818996723/image_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "1f9199", "id_str": "113532729", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/305726905/FASHION-3.png", "name": "Bee'Jay", "lang": "en", "profile_background_tile": true, "favourites_count": 19, "screen_name": "OohMyBEEsNice", "url": "http://www.bitchimpaid.org", "created_at": "Fri Feb 12 03:32:54 +0000 2010", "contributors_enabled": false, "time_zone": "Central Time (US & Canada)", "profile_sidebar_border_color": "000000", "default_profile": false, "following": null}, "in_reply_to_screen_name": null, "retweet_count": 0, "geo": null, "id": 168931016843603968, "source": "<a href=\"http://twitter.com/#!/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>"} {"favorited": false, "in_reply_to_user_id": 50940453, "contributors": null, "truncated": false, "text": "@LegaMrvica @MimozaBand makasi om artis :D kadoo kadoo", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": "168653037894770688", "coordinates": null, "in_reply_to_user_id_str": "50940453", "entities": {"user_mentions": [{"indices": [0, 11], "screen_name": "LegaMrvica", "id": 50940453, "name": "Lega_thePianis", "id_str": "50940453"}, {"indices": [12, 23], "screen_name": "MimozaBand", "id": 375128905, "name": "Mimoza", "id_str": "375128905"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": 168653037894770688, "id_str": "168931016868761600", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "profile_sidebar_fill_color": "DDFFCC", "is_translator": false, "id": 48293450, "profile_text_color": "333333", "followers_count": 182, "protected": false, "location": "\u00dcT: -6.906799,107.622383", "default_profile_image": false, "listed_count": 0, "utc_offset": -28800, "statuses_count": 3052, "description": "Fashion design maranatha '11 // traditional dancer (bali) at sanggar tampak siring & Natya Nataraja", "friends_count": 206, "profile_link_color": "0084B4", "profile_image_url": "http://a3.twimg.com/profile_images/1803845596/Picture_20124_normal.jpg", "notifications": null, "show_all_inline_media": false, "geo_enabled": true, "profile_background_color": "9AE4E8", "id_str": "48293450", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/347686061/Galungan_dan_Kuningan.jpg", "name": "nana afiff", "lang": "en", "profile_background_tile": true, "favourites_count": 2, "screen_name": "hasnfebria", "url": null, "created_at": "Thu Jun 18 08:50:29 +0000 2009", "contributors_enabled": false, "time_zone": "Pacific Time (US & Canada)", "profile_sidebar_border_color": "BDDCAD", "default_profile": false, "following": null}, "in_reply_to_screen_name": "LegaMrvica", "retweet_count": 0, "geo": null, "id": 168931016868761600, "source": "<a href=\"http://blackberry.com/twitter\" rel=\"nofollow\">Twitter for BlackBerry\u00ae</a>"} {"favorited": false, "in_reply_to_user_id": 27260086, "contributors": null, "truncated": false, "text": "@justinbieber u were born to be somebody, and u're super important in beliebers' life. thanks for all biebs. I love u. follow me? 84", "created_at": "Mon Feb 13 05:34:27 +0000 2012", "retweeted": false, "in_reply_to_status_id_str": null, "coordinates": null, "in_reply_to_user_id_str": "27260086", "entities": {"user_mentions": [{"indices": [0, 13], "screen_name": "justinbieber", "id": 27260086, "name": "Justin Bieber", "id_str": "27260086"}], "hashtags": [], "urls": []}, "in_reply_to_status_id": null, "id_str": "168931016856178688", "place": null, "user": {"follow_request_sent": null, "profile_use_background_image": true, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/416005864/Captura.JPG", "verified": false, "profile_image_url_https": "https://si0.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "profile_sidebar_fill_color": "f5e7f3", "is_translator": false, "id": 406750700, "profile_text_color": "333333", "followers_count": 1122, "protected": false, "location": "Adentro de una supra.", "default_profile_image": false, "listed_count": 0, "utc_offset": -14400, "statuses_count": 20966, "description": "Mi \u00eddolo es @justinbieber , si te gusta \u00a1genial!, si no, solo respetalo. El cambi\u00f3 mi vida completamente y mi sue\u00f1o es conocerlo #TrueBelieber . ", "friends_count": 1015, "profile_link_color": "9404b8", "profile_image_url": "http://a1.twimg.com/profile_images/1808883280/Captura6_normal.JPG", "notifications": null, "show_all_inline_media": false, "geo_enabled": false, "profile_background_color": "f9fcfa", "id_str": "406750700", "profile_background_image_url": "http://a3.twimg.com/profile_background_images/416005864/Captura.JPG", "name": "neversaynever,right?", "lang": "es", "profile_background_tile": false, "favourites_count": 22, "screen_name": "True_Belieebers", "url": "http://www.wehavebieber-fever.tumblr.com", "created_at": "Mon Nov 07 04:17:40 +0000 2011", "contributors_enabled": false, "time_zone": "Santiago", "profile_sidebar_border_color": "C0DEED", "default_profile": false, "following": null}, "in_reply_to_screen_name": "justinbieber", "retweet_count": 0, "geo": null, "id": 168931016856178688, "source": "<a href=\"http://yfrog.com\" rel=\"nofollow\">Yfrog</a> At this point everything is great, I load up the script that resulted from a previous Stack Post: from disco.core import Job, result_iterator import gzip def map(line, params): import unicodedata import json r = json.loads(line).get('text') s = unicodedata.normalize('NFD', r).encode('ascii', 'ignore') for word in s.split(): yield word, 1 def reduce(iter, params): from disco.util import kvgroup for word, counts in kvgroup(sorted(iter)): yield word, sum(counts) if __name__ == '__main__': job = Job().run(input=["tag://data:test1"], map=map, reduce=reduce) for word, count in result_iterator(job.wait(show=True)): print word, count NOTE: That this script runs file if the input=["file.txt"], however when I run it with "tag://data:test1" I get the following error: # DISCO_EVENTS=1 python count_normal_words.py Job@549:db30e:25bd8: Status: [map] 0 waiting, 1 running, 0 done, 0 failed 2012/11/25 21:43:26 master New job initialized! 2012/11/25 21:43:26 master Starting job 2012/11/25 21:43:26 master Starting map phase 2012/11/25 21:43:26 master map:0 assigned to solice 2012/11/25 21:43:26 master ERROR: Job failed: Worker at 'solice' died: Traceback (most recent call last): File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/__init__.py", line 329, in main job.worker.start(task, job, **jobargs) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/__init__.py", line 290, in start self.run(task, job, **jobargs) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/classic/worker.py", line 286, in run getattr(self, task.mode)(task, params) File "/home/DISCO/data/solice/01/Job@549:db30e:25bd8/usr/local/lib/python2.7/site-packages/disco/worker/classic/worker.py", line 299, in map for key, val in self['map'](entry, params): File "count_normal_words.py", line 12, in map File "/usr/lib64/python2.7/json/__init__.py", line 326, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 384, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded 2012/11/25 21:43:26 master WARN: Job killed Status: [map] 1 waiting, 0 running, 0 done, 1 failed Traceback (most recent call last): File "count_normal_words.py", line 28, in <module> for word, count in result_iterator(job.wait(show=True)): File "/usr/local/lib/python2.7/site-packages/disco/core.py", line 348, in wait timeout, poll_interval * 1000) File "/usr/local/lib/python2.7/site-packages/disco/core.py", line 309, in check_results raise JobError(Job(name=jobname, master=self), "Status %s" % status) disco.error.JobError: Job Job@549:db30e:25bd8 failed: Status dead The Error states: ValueError: No JSON object could be decoded. Again, this works fine using the text file as input but now DDFS. Any ideas, I'm open to suggestions?

  Read the article

• Enterprise Process Maps: A Process Picture worth a Million Words

  - by raul.goycoolea

  p { margin-bottom: 0.08in; }h1 { margin-top: 0.33in; margin-bottom: 0in; color: rgb(54, 95, 145); page-break-inside: avoid; }h1.western { font-family: "Cambria",serif; font-size: 14pt; }h1.cjk { font-family: "DejaVu Sans"; font-size: 14pt; }h1.ctl { font-size: 14pt; } Getting Started with Business Transformations A well-known proverb states that "A picture is worth a thousand words." In relation to Business Process Management (BPM), a credible analyst might have a few questions. What if the picture was taken from some particular angle, like directly overhead? What if it was taken from only an inch away or a mile away? What if the photographer did not focus the camera correctly? Does the value of the picture depend on who is looking at it? Enterprise Process Maps are analogous in this sense of relative value. Every BPM project (holistic BPM kick-off, enterprise system implementation, Service-oriented Architecture, business process transformation, corporate performance management, etc.) should be begin with a clear understanding of the business environment, from the biggest picture representations down to the lowest level required or desired for the particular project type, scope and objectives. The Enterprise Process Map serves as an entry point for the process architecture and is defined: the single highest level of process mapping for an organization. It is constructed and evaluated during the Strategy Phase of the Business Process Management Lifecycle. (see Figure 1) Fig. 1: Business Process Management Lifecycle Many organizations view such maps as visual abstractions, constructed for the single purpose of process categorization. This, in turn, results in a lesser focus on the inherent intricacies of the Enterprise Process view, which are explored in the course of this paper. With the main focus of a large scale process documentation effort usually underlying an ERP or other system implementation, it is common for the work to be driven by the desire to "get to the details," and to the type of modeling that will derive near-term tangible results. For instance, a project in American Pharmaceutical Company X is driven by the Director of IT. With 120+ systems in place, and a lack of standardized processes across the United States, he and the VP of IT have decided to embark on a long-term ERP implementation. At the forethought of both are questions, such as: How does my application architecture map to the business? What are each application's functionalities, and where do the business processes utilize them? Where can we retire legacy systems? Well-developed BPM methodologies prescribe numerous model types to capture such information and allow for thorough analysis in these areas. Process to application maps, Event Driven Process Chains, etc. provide this level of detail and facilitate the completion of such project-specific questions. These models and such analysis are appropriately carried out at a relatively low level of process detail. (see figure 2) Fig. 2: The Level Concept, Generic Process HierarchySome of the questions remaining are ones of documentation longevity, the continuation of BPM practice in the organization, process governance and ownership, process transparency and clarity in business process objectives and strategy. The Level Concept in Brief Figure 2 shows a generic, four-level process hierarchy depicting the breakdown of a "Process Area" into progressively more detailed process classifications. The number of levels and the names of these levels are flexible, and can be fit to the standards of the organization's chosen terminology or any other chosen reference model that makes logical sense for both short and long term process description. It is at Level 1 (in this case the Process Area level), that the Enterprise Process Map is created. This map and its contained objects become the foundation for a top-down approach to subsequent mapping, object relationship development, and analysis of the organization's processes and its supporting infrastructure. Additionally, this picture serves as a communication device, at an executive level, describing the design of the business in its service to a customer. It seems, then, imperative that the process development effort, and this map, start off on the right foot. Figuring out just what that right foot is, however, is critical and trend-setting in an evolving organization. Key Considerations Enterprise Process Maps are usually not as living and breathing as other process maps. Just as it would be an extremely difficult task to change the foundation of the Sears Tower or a city plan for the entire city of Chicago, the Enterprise Process view of an organization usually remains unchanged once developed (unless, of course, an organization is at a stage where it is capable of true, high-level process innovation). Regardless, the Enterprise Process map is a key first step, and one that must be taken in a precise way. What makes this groundwork solid depends on not only the materials used to construct it (process areas), but also the layout plan and knowledge base of what will be built (the entire process architecture). It seems reasonable that care and consideration are required to create this critical high level map... but what are the important factors? Does the process modeler need to worry about how many process areas there are? About who is looking at it? Should he only use the color pink because it's his boss' favorite color? Interestingly, and perhaps surprisingly, these are all valid considerations that may just require a bit of structure. Below are Three Key Factors to consider when building an Enterprise Process Map: Company Strategic Focus Process Categorization: Customer is Core End-to-end versus Functional Processes Company Strategic Focus As mentioned above, the Enterprise Process Map is created during the Strategy Phase of the Business Process Management Lifecycle. From Oracle Business Process Management methodology for business transformation, it is apparent that business processes exist for the purpose of achieving the strategic objectives of an organization. In a prescribed, top-down approach to process development, it must be ensured that each process fulfills its objectives, and in an aggregated manner, drives fulfillment of the strategic objectives of the company, whether for particular business segments or in a broader sense. This is a crucial point, as the strategic messages of the company must therefore resound in its process maps, in particular one that spans the processes of the complete business: the Enterprise Process Map. One simple example from Company X is shown below (see figure 3). Fig. 3: Company X Enterprise Process Map In reviewing Company X's Enterprise Process Map, one can immediately begin to understand the general strategic mindset of the organization. It shows that Company X is focused on its customers, defining 10 of its process areas belonging to customer-focused categories. Additionally, the organization views these end-customer-oriented process areas as part of customer-fulfilling value chains, while support process areas do not provide as much contiguous value. However, by including both support and strategic process categorizations, it becomes apparent that all processes are considered vital to the success of the customer-oriented focus processes. Below is an example from Company Y (see figure 4). Fig. 4: Company Y Enterprise Process Map Company Y, although also a customer-oriented company, sends a differently focused message with its depiction of the Enterprise Process Map. Along the top of the map is the company's product tree, overarching the process areas, which when executed deliver the products themselves. This indicates one strategic objective of excellence in product quality. Additionally, the view represents a less linear value chain, with strong overlaps of the various process areas. Marketing and quality management are seen as a key support processes, as they span the process lifecycle. Often, companies may incorporate graphics, logos and symbols representing customers and suppliers, and other objects to truly send the strategic message to the business. Other times, Enterprise Process Maps may show high level of responsibility to organizational units, or the application types that support the process areas. It is possible that hundreds of formats and focuses can be applied to an Enterprise Process Map. What is of vital importance, however, is which formats and focuses are chosen to truly represent the direction of the company, and serve as a driver for focusing the business on the strategic objectives set forth in that right. Process Categorization: Customer is Core In the previous two examples, processes were grouped using differing categories and techniques. Company X showed one support and three customer process categorizations using encompassing chevron objects; Customer Y achieved a less distinct categorization using a gradual color scheme. Either way, and in general, modeling of the process areas becomes even more valuable and easily understood within the context of business categorization, be it strategic or otherwise. But how one categorizes their processes is typically more complex than simply choosing object shapes and colors. Previously, it was stated that the ideal is a prescribed top-down approach to developing processes, to make certain linkages all the way back up to corporate strategy. But what about external influences? What forces push and pull corporate strategy? Industry maturity, product lifecycle, market profitability, competition, etc. can all drive the critical success factors of a particular business segment, or the company as a whole, in addition to previous corporate strategy. This may seem to be turning into a discussion of theory, but that is far from the case. In fact, in years of recent study and evolution of the way businesses operate, cross-industry and across the globe, one invariable has surfaced with such strength to make it undeniable in the game plan of any strategy fit for survival. That constant is the customer. Many of a company's critical success factors, in any business segment, relate to the customer: customer retention, satisfaction, loyalty, etc. Businesses serve customers, and so do a business's processes, mapped or unmapped. The most effective way to categorize processes is in a manner that visualizes convergence to what is core for a company. It is the value chain, beginning with the customer in mind, and ending with the fulfillment of that customer, that becomes the core or the centerpiece of the Enterprise Process Map. (See figure 5) Fig. 5: Company Z Enterprise Process Map Company Z has what may be viewed as several different perspectives or "cuts" baked into their Enterprise Process Map. It has divided its processes into three main categories (top, middle, and bottom) of Management Processes, the Core Value Chain and Supporting Processes. The Core category begins with Corporate Marketing (which contains the activities of beginning to engage customers) and ends with Customer Service Management. Within the value chain, this company has divided into the focus areas of their two primary business lines, Foods and Beverages. Does this mean that areas, such as Strategy, Information Management or Project Management are not as important as those in the Core category? No! In some cases, though, depending on the organization's understanding of high-level BPM concepts, use of category names, such as "Core," "Management" or "Support," can be a touchy subject. What is important to understand, is that no matter the nomenclature chosen, the Core processes are those that drive directly to customer value, Support processes are those which make the Core processes possible to execute, and Management Processes are those which steer and influence the Core. Some common terms for these three basic categorizations are Core, Customer Fulfillment, Customer Relationship Management, Governing, Controlling, Enabling, Support, etc. End-to-end versus Functional Processes Every high and low level of process: function, task, activity, process/work step (whatever an organization calls it), should add value to the flow of business in an organization. Suppose that within the process "Deliver package," there is a documented task titled "Stop for ice cream." It doesn't take a process expert to deduce the room for improvement. Though stopping for ice cream may create gain for the one person performing it, it likely benefits neither the organization nor, more importantly, the customer. In most cases, "Stop for ice cream" wouldn't make it past the first pass of To-Be process development. What would make the cut, however, would be a flow of tasks that, each having their own value add, build up to greater and greater levels of process objective. In this case, those tasks would combine to achieve a status of "package delivered." Figure 3 shows a simple example: Just as the package can only be delivered (outcome of the process) without first being retrieved, loaded, and the travel destination reached (outcomes of the process steps), some higher level of process "Play Practical Joke" (e.g., main process or process area) cannot be completed until a package is delivered. It seems that isolated or functionally separated processes, such as "Deliver Package" (shown in Figure 6), are necessary, but are always part of a bigger value chain. Each of these individual processes must be analyzed within the context of that value chain in order to ensure successful end-to-end process performance. For example, this company's "Create Joke Package" process could be operating flawlessly and efficiently, but if a joke is never developed, it cannot be created, so the end-to-end process breaks. Fig. 6: End to End Process Construction That being recognized, it is clear that processes must be viewed as end-to-end, customer-to-customer, and in the context of company strategy. But as can also be seen from the previous example, these vital end-to-end processes cannot be built without the functionally oriented building blocks. Without one, the other cannot be had, or at least not in a complete and organized fashion. As it turns out, but not discussed in depth here, the process modeling effort, BPM organizational development, and comprehensive coverage cannot be fully realized without a semi-functional, process-oriented approach. Then, an Enterprise Process Map should be concerned with both views, the building blocks, and access points to the business-critical end-to-end processes, which they construct. Without the functional building blocks, all streams of work needed for any business transformation would be lost mess of process disorganization. End-to-end views are essential for utilization in optimization in context, understanding customer impacts, base-lining all project phases and aligning objectives. Including both views on an Enterprise Process Map allows management to understand the functional orientation of the company's processes, while still providing access to end-to-end processes, which are most valuable to them. (See figures 7 and 8). Fig. 7: Simplified Enterprise Process Map with end-to-end Access Point The above examples show two unique ways to achieve a successful Enterprise Process Map. The first example is a simple map that shows a high level set of process areas and a separate section with the end-to-end processes of concern for the organization. This particular map is filtered to show just one vital end-to-end process for a project-specific focus. Fig. 8: Detailed Enterprise Process Map showing connected Functional Processes The second example shows a more complex arrangement and categorization of functional processes (the names of each process area has been removed). The end-to-end perspective is achieved at this level through the connections (interfaces at lower levels) between these functional process areas. An important point to note is that the organization of these two views of the Enterprise Process Map is dependent, in large part, on the orientation of its audience, and the complexity of the landscape at the highest level. If both are not apparent, the Enterprise Process Map is missing an opportunity to serve as a holistic, high-level view. Conclusion In the world of BPM, and specifically regarding Enterprise Process Maps, a picture can be worth as many words as the thought and effort that is put into it. Enterprise Process Maps alone cannot change an organization, but they serve more purposes than initially meet the eye, and therefore must be designed in a way that enables a BPM mindset, business process understanding and business transformation efforts. Every Enterprise Process Map will and should be different when looking across organizations. Its design will be driven by company strategy, a level of customer focus, and functional versus end-to-end orientations. This high-level description of the considerations of the Enterprise Process Maps is not a prescriptive "how to" guide. However, a company attempting to create one may not have the practical BPM experience to truly explore its options or impacts to the coming work of business process transformation. The biggest takeaway is that process modeling, at all levels, is a science and an art, and art is open to interpretation. It is critical that the modeler of the highest level of process mapping be a cognoscente of the message he is delivering and the factors at hand. Without sufficient focus on the design of the Enterprise Process Map, an entire BPM effort may suffer. For additional information please check: Oracle Business Process Management.

  Read the article

• CodePlex Daily Summary for Friday, May 28, 2010

  CodePlex Daily Summary for Friday, May 28, 2010New ProjectsBang: BangBox Office: Event Management for Community Theater Groups: Box Office is an event management web application to help theater groups manage & promote their shows. Manage performance schedules, sell tickets, ...CellsOnWeb: El espacio de las células del Programa Académico Microsoft en Argentina. CRM 4.0 Plugin Queue Item Counter: This is a crm 4.0 plugin to count queue items in each folder and display the number at the end of the name. For example, if the queue name is "Tes...Date Calculator: Date Calculator is a small desktop utility developed using Windows Forms .NET technology. This utility is analogous to the "Date calculation" modul...Enterprise Library Investigate: Enterprise Library Investigate ProjecteProject Management: Ứng dụng nền tảng web hỗ trợ quản lí và giám sát tiến độ dự án của tổ chức doanh nghiệp.Fiddler TreeView Panel Extension: Extension for Fiddler, to display the session information in a TreeView panel instead of the default ListBox, so it groups the information logicall...Git Source Control Provider: Git Source Control Provider is a Visual Studio Plug-in that integrates Git with Visual Studio.InspurProjects: Project on Inspur Co.Kryptonite: The Kryptonite project aims to improve development of websites based on the Kentico CMS. MLang .NET Wrapper: Detect the encoding of a text without BOM (Byte Order Mask) and choose the best Encoding for persistence or network transport of textMondaze: Proof of concept using Windows Azure.MultipointControls: A collection of controls that applied Windows Multipoint Mouse SDK. Windows Multipoint Mouse SDK enable app to have multiple mice interact simultan...Mundo De Bloques: "Mundo de bloques" makes it easier for analists to find the shortest way between two states in a problem using an heuristic function for Artificial...MyRPGtests: Just some tests :)OffInvoice Add-in for MS Office 2010: Project Description: The project it's based in the ability to extend funtionality in the Microsoft Office 2010 suite.OpenGraph .NET: A C# client for the Facebook Graph API. Supports desktop, web, ASP.NET MVC, and Silverlight connections and real-time updates. PLEASE NOTE: I dis...Portable Extensible Metadata (PEM) Data Annotation Generator: This project intends to help developers who uses PEM - Portable Extensible Metadata for Entity Framework generating Data Annotation information fro...Production and sale of plastic window systems: Automation company produces window design, production and sale of plastic window systems, management of sales contracts and their execution, print ...Renjian Storm (Renjian Image Viewer Uploader): Renjian Image Viewer UploaderShark Web Intelligence CMS: Shark Web Intelligence Inc. Content Management System.Shuffleboard Game for Windows Phone 7: This is a sample Shuffleboard game written in Silverlight for Windows Phone 7. It demonstrates physics, procedural animation, perspective transform...Silverlight Property Grid: Visual Studio Style PropertyGrid for Silverlight.SvnToTfs: Simple tool that migrates every Subversion revision toward Team Foundation Server 2010. It is developed in C# witn a WPF front-end.Tamias: Basic Cms Mvc Contrib Portable Area: The goal of this project is to have a easy-to-integrate basic cms for ASP.NET MVC applications based on MVC Contrib Portable Areas.TwitBy: TwitBy is a Twitter client for anyone who uses Twitter. It's easy to use and all of the major features are there. More features to come. H...Under Construction: A simple site that can be used as a splash for sites being upgraded or developed. UO Editor: The Owner & Organisation Editor makes it easy to view and edit the names of the registered owner and registered organization for your Windows OS. N...webform2010: this is the test projectWireless Network: ssWiX Toolset: The Windows Installer XML (WiX) is a toolset that builds Windows installation packages from XML source code. The toolset supports a command line en...Xna.Extend: A collection of easy to use Xna components for aiding a game programmer in developing thee next big thing. I plan on using the components from this...New ReleasesA Guide to Parallel Programming: Drop 4 - Guide Preface, Chapters 1 - 5, and code: This is Drop 4 with Guide Preface, Chapters 1 - 5, and References, and the accompanying code samples. This drop requires Visual Studio 2010 Beta 2 ...Ajax Toolkit for ASP.NET MVC: MAT 1.1: MAT 1.1Community Forums NNTP bridge: Community Forums NNTP Bridge V09: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release solves ...Community Forums NNTP bridge: Community Forums NNTP Bridge V10: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release has add...Community Forums NNTP bridge: Community Forums NNTP Bridge V11: Release of the Community Forums NNTP Bridge to access the social and anwsers MS forums with a single, open source NNTP bridge. This release has add...CSS 360 Planetary Calendar: Beta Release: =============================================================================== Beta Release Version: 0.2 Description: This is the beta release de...Date Calculator: DateCalculator v1.0: This is the first release and as far as I know this is a stable version.eComic: eComic 2010.0.0.4: Version 2010.0.0.4 Change LogFixed issues in the "Full Screen Control Panel" causing it to lack translucence Added loupe magnification control ...Expression Encoder Batch Processor: Runtime Application v0.2: New in this version: Added more error handling if files not exist. Added button/feature to quit after current encoding job. Added code to handl...Fiddler TreeView Panel Extension: FiddlerTreeViewPanel 0.7: Initial compiled version of the assembly, ready to use. Please refer to http://fiddlertreeviewpanel.codeplex.com/ for instructions and installation.Gardens Point LEX: Gardens Point LEX v1.1.4: The main distribution is a zip file. This contains the binary executable, documentation, source code and the examples. ChangesVersion 1.1.4 corre...Gardens Point Parser Generator: Gardens Point Parser Generator v1.4.1: Version 1.4.1 differs from version 1.4.0 only in containing a corrected version of a previously undocumented feature which allows the generation of...IsWiX: IsWiX 1.0.264.0: Build 1.0.264.0 - built against Fireworks 1.0.264.0. Adds support for autogenerating the SourceDir prepreprocessor variable and gives user choice t...Matrix: Matrix 0.5.2: Updated licenseMesopotamia Experiment: Mesopotamia 1.2.90: Release Notes - Ugraded to Microsoft Robotics Developer Studio 2008 R3 Bug Fixes - Fix to keep any sole organisms that penetrate to the next fitne...Microsoft Crm 4.0 Filtered Lookup: Microsoft Crm 4.0 Filtered Lookup: How to use: Allow passing custom querystring values: Create a DWORD registry key named [DisableParameterFilter] under [HKEY_LOCAL_MACHINE\SOFTWAR...MSBuild Extension Pack: May 2010: The MSBuild Extension Pack May 2010 release provides a collection of over 340 MSBuild tasks. A high level summary of what the tasks currently cover...MultiPoint Vote: MultiPointVote v.1: This accepts user inputs: number of participants, poll/survey title and the list of options A text file containing the items listed line per line...Mundo De Bloques: Mundo de Bloques, Release 1: "Mundo de bloques" makes it easier for analists to find the shortest way between two states in a problem using an heuristic function for Artificial...OffInvoice Add-in for MS Office 2010: OffInvoice for Office 2010 V1.0 Installer: Add-in for MS Word 2010 or MS Excel 2010 to allow the management (issuing, visualization and reception) of electronic invoices, based in the XML fo...OpenGraph .NET: 0.9.1 Beta: This is the first public release of OpenGraph .NET.patterns & practices: Composite WPF and Silverlight: Prism v2.2 - May 2010 Release: Composite Application Guidance for WPF and Silverlight - May 2010 Release (Prism V2.2) The Composite Application Guidance for WPF and Silverlight ...Portable Extensible Metadata (PEM) Data Annotation Generator: Release 49376: First release.Production and sale of plastic window systems: Yanuary 2009: NOTEBefore loading program, make sure you have installed MySQL and created DataBase that store in Source Code (look at below) Where Is The Source?...PROGRAMMABLE SOFTWARE DEVELOPMENT ENVIRONMENT: PROGRAMMABLE SOFTWARE DEVELOPMENT ENVIRONMENT--3.2: The current version of the Programmable Software Development Environment has the capability of reading an optional text file in each source develop...Rapidshare Episode Downloader: RED 0.8.6: - Fixed Edit form to actually save the data - Added Bypass Validation to enable future episodes - Added Search parameter to Edit form - Added refr...Renjian Storm (Renjian Image Viewer Uploader): Renjian Storm 0.6: 人间风暴 v0.6 稳定版sELedit: sELedit v1.1b: + Fixed: when export and import items to text files, there was a bug with "NULL" bytes in the unicode stringShake - C# Make: Shake v0.1.21: Changes: FileTask CopyDir method modified, see documentationSharePoint Labs: SPLab7001A-ENU-Level100: SPLab7001A-ENU-Level100 This SharePoint Lab will teach how to analyze and audit WSP files. WSP files are somewhere in a no man's land between ITPro...SharePoint Rsync List: 1.0.0.3: Fix spcontext dispose bug in menu try and run jobs only on central admin server mark a single file failure if file not copied don't delete destinat...Shuffleboard Game for Windows Phone 7: Shuffleboard 1.0.0.1: Source code, solution files, and assets.Software Is Hardwork: Sw. Is Hw. Lib. 3.0.0.x+04: Sw. Is Hw. Lib. 3.0.0.x+04SoulHackers Demon Unite(Chinese version): WPFClient pre alpha: can unite 2, 3 or more demons. can un-unite 1 demon to 2 demon (no triple un-unite yet).Team Deploy: Team Deploy 2010 R1: This is the initial release for Team Deploy 2010 for TFS Team Build 2010. All features from Team Build 2.x are functional in this version. Comple...Under Construction: Under Construction: All Files required to show under construction page. The Page will pull through the Domain name that the site is being run on this allows you to use...Unit Driven: Version 0.0.5: - Tests nested by namespace parts. - Run buttons properly disabled based on currently running tests. - Timeouts for async tests enabled.UO Editor: UO Editor v1.0: Initial ReleaseVCC: Latest build, v2.1.30527.0: Automatic drop of latest buildWeb Service Software Factory Contrib: Import WSDL 2010: Generate Service Contract models from existing WSDL documents for Web Service Software Factory 2010. Usage: Install the vsix and right click on a S...Most Popular ProjectsRawrWBFS ManagerAJAX Control ToolkitMicrosoft SQL Server Product Samples: DatabaseSilverlight ToolkitWindows Presentation Foundation (WPF)patterns & practices – Enterprise LibraryMicrosoft SQL Server Community & SamplesPHPExcelASP.NETMost Active ProjectsAStar.netpatterns & practices – Enterprise LibraryGMap.NET - Great Maps for Windows Forms & PresentationSqlServerExtensionsBlogEngine.NETRawrpatterns & practices: Windows Azure Security GuidanceCodeReviewCustomer Portal Accelerator for Microsoft Dynamics CRMIonics Isapi Rewrite Filter

  Read the article

• Metro: Understanding CSS Media Queries

  - by Stephen.Walther

  If you are building a Metro style application then your application needs to look great when used on a wide variety of devices. Your application needs to work on tiny little phones, slates, desktop monitors, and the super high resolution displays of the future. Your application also must support portable devices used with different orientations. If someone tilts their phone from portrait to landscape mode then your application must still be usable. Finally, your Metro style application must look great in different states. For example, your Metro application can be in a “snapped state” when it is shrunk so it can share screen real estate with another application. In this blog post, you learn how to use Cascading Style Sheet media queries to support different devices, different device orientations, and different application states. First, you are provided with an overview of the W3C Media Query recommendation and you learn how to detect standard media features. Next, you learn about the Microsoft extensions to media queries which are supported in Metro style applications. For example, you learn how to use the –ms-view-state feature to detect whether an application is in a “snapped state” or “fill state”. Finally, you learn how to programmatically detect the features of a device and the state of an application. You learn how to use the msMatchMedia() method to execute a media query with JavaScript. Using CSS Media Queries Media queries enable you to apply different styles depending on the features of a device. Media queries are not only supported by Metro style applications, most modern web browsers now support media queries including Google Chrome 4+, Mozilla Firefox 3.5+, Apple Safari 4+, and Microsoft Internet Explorer 9+. Loading Different Style Sheets with Media Queries Imagine, for example, that you want to display different content depending on the horizontal resolution of a device. In that case, you can load different style sheets optimized for different sized devices. Consider the following HTML page: <!DOCTYPE html> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <title>U.S. Robotics and Mechanical Men</title> <link href="main.css" rel="stylesheet" type="text/css" /> <!-- Less than 1100px --> <link href="medium.css" rel="stylesheet" type="text/css" media="(max-width:1100px)" /> <!-- Less than 800px --> <link href="small.css" rel="stylesheet" type="text/css" media="(max-width:800px)" /> </head> <body> <div id="header"> <h1>U.S. Robotics and Mechanical Men</h1> </div> <!-- Advertisement Column --> <div id="leftColumn"> <img src="advertisement1.gif" alt="advertisement" /> <img src="advertisement2.jpg" alt="advertisement" /> </div> <!-- Product Search Form --> <div id="mainContentColumn"> <label>Search Products</label> <input id="search" /><button>Search</button> </div> <!-- Deal of the Day Column --> <div id="rightColumn"> <h1>Deal of the Day!</h1> <p> Buy two cameras and get a third camera for free! Offer is good for today only. </p> </div> </body> </html> The HTML page above contains three columns: a leftColumn, mainContentColumn, and rightColumn. When the page is displayed on a low resolution device, such as a phone, only the mainContentColumn appears: When the page is displayed in a medium resolution device, such as a slate, both the leftColumn and the mainContentColumns are displayed: Finally, when the page is displayed in a high-resolution device, such as a computer monitor, all three columns are displayed: Different content is displayed with the help of media queries. The page above contains three style sheet links. Two of the style links include a media attribute: <link href="main.css" rel="stylesheet" type="text/css" /> <!-- Less than 1100px --> <link href="medium.css" rel="stylesheet" type="text/css" media="(max-width:1100px)" /> <!-- Less than 800px --> <link href="small.css" rel="stylesheet" type="text/css" media="(max-width:800px)" /> The main.css style sheet contains default styles for the elements in the page. The medium.css style sheet is applied when the page width is less than 1100px. This style sheet hides the rightColumn and changes the page background color to lime: html { background-color: lime; } #rightColumn { display:none; } Finally, the small.css style sheet is loaded when the page width is less than 800px. This style sheet hides the leftColumn and changes the page background color to red: html { background-color: red; } #leftColumn { display:none; } The different style sheets are applied as you stretch and contract your browser window. You don’t need to refresh the page after changing the size of the page for a media query to be applied: Using the @media Rule You don’t need to divide your styles into separate files to take advantage of media queries. You can group styles by using the @media rule. For example, the following HTML page contains one set of styles which are applied when a device’s orientation is portrait and another set of styles when a device’s orientation is landscape: <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Application1</title> <style type="text/css"> html { font-family:'Segoe UI Semilight'; font-size: xx-large; } @media screen and (orientation:landscape) { html { background-color: lime; } p.content { width: 50%; margin: auto; } } @media screen and (orientation:portrait) { html { background-color: red; } p.content { width: 90%; margin: auto; } } </style> </head> <body> <p class="content"> Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. </p> </body> </html> When a device has a landscape orientation then the background color is set to the color lime and the text only takes up 50% of the available horizontal space: When the device has a portrait orientation then the background color is red and the text takes up 90% of the available horizontal space: Using Standard CSS Media Features The official list of standard media features is contained in the W3C CSS Media Query recommendation located here: http://www.w3.org/TR/css3-mediaqueries/ Here is the official list of the 13 media features described in the standard: · width – The current width of the viewport · height – The current height of the viewport · device-width – The width of the device · device-height – The height of the device · orientation – The value portrait or landscape · aspect-ratio – The ratio of width to height · device-aspect-ratio – The ratio of device width to device height · color – The number of bits per color supported by the device · color-index – The number of colors in the color lookup table of the device · monochrome – The number of bits in the monochrome frame buffer · resolution – The density of the pixels supported by the device · scan – The values progressive or interlace (used for TVs) · grid – The values 0 or 1 which indicate whether the device supports a grid or a bitmap Many of the media features in the list above support the min- and max- prefix. For example, you can test for the min-width using a query like this: (min-width:800px) You can use the logical and operator with media queries when you need to check whether a device supports more than one feature. For example, the following query returns true only when the width of the device is between 800 and 1,200 pixels: (min-width:800px) and (max-width:1200px) Finally, you can use the different media types – all, braille, embossed, handheld, print, projection, screen, speech, tty, tv — with a media query. For example, the following media query only applies to a page when a page is being printed in color: print and (color) If you don’t specify a media type then media type all is assumed. Using Metro Style Media Features Microsoft has extended the standard list of media features which you can include in a media query with two custom media features: · -ms-high-contrast – The values any, black-white, white-black · -ms-view-state – The values full-screen, fill, snapped, device-portrait You can take advantage of the –ms-high-contrast media feature to make your web application more accessible to individuals with disabilities. In high contrast mode, you should make your application easier to use for individuals with vision disabilities. The –ms-view-state media feature enables you to detect the state of an application. For example, when an application is snapped, the application only occupies part of the available screen real estate. The snapped application appears on the left or right side of the screen and the rest of the screen real estate is dominated by the fill application (Metro style applications can only be snapped on devices with a horizontal resolution of greater than 1,366 pixels). Here is a page which contains style rules for an application in both a snap and fill application state: <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>MyWinWebApp</title> <style type="text/css"> html { font-family:'Segoe UI Semilight'; font-size: xx-large; } @media screen and (-ms-view-state:snapped) { html { background-color: lime; } } @media screen and (-ms-view-state:fill) { html { background-color: red; } } </style> </head> <body> <p class="content"> Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Maecenas porttitor congue massa. Fusce posuere, magna sed pulvinar ultricies, purus lectus malesuada libero, sit amet commodo magna eros quis urna. </p> </body> </html> When the application is snapped, the application appears with a lime background color: When the application state is fill then the background color changes to red: When the application takes up the entire screen real estate – it is not in snapped or fill state – then no special style rules apply and the application appears with a white background color. Querying Media Features with JavaScript You can perform media queries using JavaScript by taking advantage of the window.msMatchMedia() method. This method returns a MSMediaQueryList which has a matches method that represents success or failure. For example, the following code checks whether the current device is in portrait mode: if (window.msMatchMedia("(orientation:portrait)").matches) { console.log("portrait"); } else { console.log("landscape"); } If the matches property returns true, then the device is in portrait mode and the message “portrait” is written to the Visual Studio JavaScript Console window. Otherwise, the message “landscape” is written to the JavaScript Console window. You can create an event listener which triggers code whenever the results of a media query changes. For example, the following code writes a message to the JavaScript Console whenever the current device is switched into or out of Portrait mode: window.msMatchMedia("(orientation:portrait)").addListener(function (mql) { if (mql.matches) { console.log("Switched to portrait"); } }); Be aware that the event listener is triggered whenever the result of the media query changes. So the event listener is triggered both when you switch from landscape to portrait and when you switch from portrait to landscape. For this reason, you need to verify that the matches property has the value true before writing the message. Summary The goal of this blog entry was to explain how CSS media queries work in the context of a Metro style application written with JavaScript. First, you were provided with an overview of the W3C CSS Media Query recommendation. You learned about the standard media features which you can query such as width and orientation. Next, we focused on the Microsoft extensions to media queries. You learned how to use –ms-view-state to detect whether a Metro style application is in “snapped” or “fill” state. You also learned how to use the msMatchMedia() method to perform a media query from JavaScript.

  Read the article

• The Incremental Architect&acute;s Napkin - #2 - Balancing the forces

  - by Ralf Westphal

  Originally posted on: http://geekswithblogs.net/theArchitectsNapkin/archive/2014/06/02/the-incremental-architectacutes-napkin---2---balancing-the-forces.aspxCategorizing requirements is the prerequisite for ecconomic architectural decisions. Not all requirements are created equal. However, to truely understand and describe the requirement forces pulling on software development, I think further examination of the requirements aspects is varranted. Aspects of Functionality There are two sides to Functionality requirements. It´s about what a software should do. I call that the Operations it implements. Operations are defined by expressions and control structures or calls to frameworks of some sort, i.e. (business) logic statements. Operations calculate, transform, aggregate, validate, send, receive, load, store etc. Operations are about behavior; they take input and produce output by considering state. I´m not using the term “function” here, because functions - or methods or sub-programs - are not necessary to implement Operations. Functions belong to a different sub-aspect of requirements (see below). Operations alone are not enough, though, to make a customer happy with regard to his/her Functionality requirements. Only correctly implemented Operations provide full value. This should make clear, why testing is so important. And not just manual tests during development of some operational feature, but automated tests. Because only automated tests scale when over time the number of operations increases. Without automated tests there is no guarantee formerly correct operations are still correct after more got added. To retest all previous operations manually is infeasible. So whoever relies just on manual tests is not really balancing the two forces Operations and Correctness. With manual tests more weight is put on the side of the scale of Operations. That might be ok for a short period of time - but in the long run it will bite you. You need to plan for Correctness in the long run from the first day of your project on. Aspects of Quality As important as Functionality is, it´s not the driver for software development. No software has ever been written to just implement some operation in code. We don´t need computers just to do something. All computers can do with software we can do without them. Well, at least given enough time and resources. We could calculate the most complex formulas without computers. We could do auctions with millions of people without computers. The only reason we want computers to help us with this and a million other Operations is… We don´t want to wait for the results very long. Or we want less errors. Or we want easier accessability to complicated solutions. So the main reason for customers to buy/order software is some Quality. They want some Functionality with a higher Quality (e.g. performance, scalability, usability, security…) than without the software. But Qualities come in at least two flavors: Most important are Primary Qualities. That´s the Qualities software truely is written for. Take an online auction website for example. Its Primary Qualities are performance, scalability, and usability, I´d say. Auctions should come within reach of millions of people; setting up an auction should be very easy; finding a suitable auction and bidding on it should be as fast as possible. Only if those Qualities have been implemented does security become relevant. A secure auction website is important - but not as important as a fast auction website. Nobody would want to use the most secure auction website if it was unbearably slow. But there would be people willing to use the fastest auction website even it was lacking security. That´s why security - with regard to online auction software - is not a Primary Quality, but just a Secondary Quality. It´s a supporting quality, so to speak. It does not deliver value by itself. With a password manager software this might be different. There security might be a Primary Quality. Please get me right: I don´t want to denigrate any Quality. There´s a long list of non-functional requirements at Wikipedia. They are all created equal - but that does not mean they are equally important for all software projects. When confronted with Quality requirements check with the customer which are primary and which are secondary. That will help to make good economical decisions when in a crunch. Resources are always limited - but requirements are a bottomless ocean. Aspects of Security of Investment Functionality and Quality are traditionally the requirement aspects cared for most - by customers and developers alike. Even today, when pressure rises in a project, tunnel vision will focus on them. Any measures to create and hold up Security of Investment (SoI) will be out of the window pretty quickly. Resistance to customers and/or management is futile. As long as SoI is not placed on equal footing with Functionality and Quality it´s bound to suffer under pressure. To look closer at what SoI means will help to become more conscious about it and make customers and management aware of the risks of neglecting it. SoI to me has two facets: Production Efficiency (PE) is about speed of delivering value. Customers like short response times. Short response times mean less money spent. So whatever makes software development faster supports this requirement. This must not lead to duct tape programming and banging out features by the dozen, though. Because customers don´t just want Operations and Quality, but also Correctness. So if Correctness gets compromised by focussing too much on Production Efficiency it will fire back. Customers want PE not just today, but over the whole course of a software´s lifecycle. That means, it´s not just about coding speed, but equally about code quality. If code quality leads to rework the PE is on an unsatisfactory level. Also if code production leads to waste it´s unsatisfactory. Because the effort which went into waste could have been used to produce value. Rework and waste cost money. Rework and waste abound, however, as long as PE is not addressed explicitly with management and customers. Thanks to the Agile and Lean movements that´s increasingly the case. Nevertheless more could and should be done in many teams. Each and every developer should keep in mind that Production Efficiency is as important to the customer as Functionality and Quality - whether he/she states it or not. Making software development more efficient is important - but still sooner or later even agile projects are going to hit a glas ceiling. At least as long as they neglect the second SoI facet: Evolvability. Delivering correct high quality functionality in short cycles today is good. But not just any software structure will allow this to happen for an indefinite amount of time.[1] The less explicitly software was designed the sooner it´s going to get stuck. Big ball of mud, monolith, brownfield, legacy code, technical debt… there are many names for software structures that have lost the ability to evolve, to be easily changed to accomodate new requirements. An evolvable code base is the opposite of a brownfield. It´s code which can be easily understood (by developers with sufficient domain expertise) and then easily changed to accomodate new requirements. Ideally the costs of adding feature X to an evolvable code base is independent of when it is requested - or at least the costs should only increase linearly, not exponentially.[2] Clean Code, Agile Architecture, and even traditional Software Engineering are concerned with Evolvability. However, it seems no systematic way of achieving it has been layed out yet. TDD + SOLID help - but still… When I look at the design ability reality in teams I see much room for improvement. As stated previously, SoI - or to be more precise: Evolvability - can hardly be measured. Plus the customer rarely states an explicit expectation with regard to it. That´s why I think, special care must be taken to not neglect it. Postponing it to some large refactorings should not be an option. Rather Evolvability needs to be a core concern for every single developer day. This should not mean Evolvability is more important than any of the other requirement aspects. But neither is it less important. That´s why more effort needs to be invested into it, to bring it on par with the other aspects, which usually are much more in focus. In closing As you see, requirements are of quite different kinds. To not take that into account will make it harder to understand the customer, and to make economic decisions. Those sub-aspects of requirements are forces pulling in different directions. To improve performance might have an impact on Evolvability. To increase Production Efficiency might have an impact on security etc. No requirement aspect should go unchecked when deciding how to allocate resources. Balancing should be explicit. And it should be possible to trace back each decision to a requirement. Why is there a null-check on parameters at the start of the method? Why are there 5000 LOC in this method? Why are there interfaces on those classes? Why is this functionality running on the threadpool? Why is this function defined on that class? Why is this class depending on three other classes? These and a thousand more questions are not to mean anything should be different in a code base. But it´s important to know the reason behind all of these decisions. Because not knowing the reason possibly means waste and having decided suboptimally. And how do we ensure to balance all requirement aspects? That needs practices and transparency. Practices means doing things a certain way and not another, even though that might be possible. We´re dealing with dangerous tools here. Like a knife is a dangerous tool. Harm can be done if we use our tools in just any way at the whim of the moment. Over the centuries rules and practices have been established how to use knifes. You don´t put them in peoples´ legs just because you´re feeling like it. You hand over a knife with the handle towards the receiver. You might not even be allowed to cut round food like potatos or eggs with it. The same should be the case for dangerous tools like object-orientation, remote communication, threads etc. We need practices to use them in a way so requirements are balanced almost automatically. In addition, to be able to work on software as a team we need transparency. We need means to share our thoughts, to work jointly on mental models. So far our tools are focused on working with code. Testing frameworks, build servers, DI containers, intellisense, refactoring support… That´s all nice and well. I don´t want to miss any of that. But I think it´s not enough. We´re missing mental tools, tools for making thinking and talking about software (independently of code) easier. You might think, enough of such tools already exist like all those UML diagram types or Flow Charts. But then, isn´t it strange, hardly any team is using them to design software? Or is that just due to a lack of education? I don´t think so. It´s a matter value/weight ratio: the current mental tools are too heavy weight compared to the value they deliver. So my conclusion is, we need lightweight tools to really be able to balance requirements. Software development is complex. We need guidance not to forget important aspects. That´s like with flying an airplane. Pilots don´t just jump in and take off for their destination. Yes, there are times when they are “flying by the seats of their pants”, when they are just experts doing thing intuitively. But most of the time they are going through honed practices called checklist. See “The Checklist Manifesto” for very enlightening details on this. Maybe then I should say it like this: We need more checklists for the complex businss of software development.[3] But that´s what software development mostly is about: changing software over an unknown period of time. It needs to be corrected in order to finally provide promised operations. It needs to be enhanced to provide ever more operations and qualities. All this without knowing when it´s going to stop. Probably never - until “maintainability” hits a wall when the technical debt is too large, the brownfield too deep. Software development is not a sprint, is not a marathon, not even an ultra marathon. Because to all this there is a foreseeable end. Software development is like continuously and foreever running… ? And sometimes I dare to think that costs could even decrease over time. Think of it: With each feature a software becomes richer in functionality. So with each additional feature the chance of there being already functionality helping its implementation increases. That should lead to less costs of feature X if it´s requested later than sooner. X requested later could stand on the shoulders of previous features. Alas, reality seems to be far from this despite 20+ years of admonishing developers to think in terms of reusability.[1] ? Please don´t get me wrong: I don´t want to bog down the “art” of software development with heavyweight practices and heaps of rules to follow. The framework we need should be lightweight. It should not stand in the way of delivering value to the customer. It´s purpose is even to make that easier by helping us to focus and decreasing waste and rework. ?

  Read the article

• ASPNET WebAPI REST Guidance

  - by JoshReuben

  ASP.NET Web API is an ideal platform for building RESTful applications on the .NET Framework. While I may be more partial to NodeJS these days, there is no denying that WebAPI is a well engineered framework. What follows is my investigation of how to leverage WebAPI to construct a RESTful frontend API.   The Advantages of REST Methodology over SOAP Simpler API for CRUD ops Standardize Development methodology - consistent and intuitive Standards based à client interop Wide industry adoption, Ease of use à easy to add new devs Avoid service method signature blowout Smaller payloads than SOAP Stateless à no session data means multi-tenant scalability Cache-ability Testability   General RESTful API Design Overview · utilize HTTP Protocol - Usage of HTTP methods for CRUD, standard HTTP response codes, common HTTP headers and Mime Types · Resources are mapped to URLs, actions are mapped to verbs and the rest goes in the headers. · keep the API semantic, resource-centric – A RESTful, resource-oriented service exposes a URI for every piece of data the client might want to operate on. A REST-RPC Hybrid exposes a URI for every operation the client might perform: one URI to fetch a piece of data, a different URI to delete that same data. utilize Uri to specify CRUD op, version, language, output format: http://api.MyApp.com/{ver}/{lang}/{resource_type}/{resource_id}.{output_format}?{key&filters} · entity CRUD operations are matched to HTTP methods: · Create - POST / PUT · Read – GET - cacheable · Update – PUT · Delete - DELETE · Use Uris to represent a hierarchies - Resources in RESTful URLs are often chained · Statelessness allows for idempotency – apply an op multiple times without changing the result. POST is non-idempotent, the rest are idempotent (if DELETE flags records instead of deleting them). · Cache indication - Leverage HTTP headers to label cacheable content and indicate the permitted duration of cache · PUT vs POST - The client uses PUT when it determines which URI (Id key) the new resource should have. The client uses POST when the server determines they key. PUT takes a second param – the id. POST creates a new resource. The server assigns the URI for the new object and returns this URI as part of the response message. Note: The PUT method replaces the entire entity. That is, the client is expected to send a complete representation of the updated product. If you want to support partial updates, the PATCH method is preferred DELETE deletes a resource at a specified URI – typically takes an id param · Leverage Common HTTP Response Codes in response headers 200 OK: Success 201 Created - Used on POST request when creating a new resource. 304 Not Modified: no new data to return. 400 Bad Request: Invalid Request. 401 Unauthorized: Authentication. 403 Forbidden: Authorization 404 Not Found – entity does not exist. 406 Not Acceptable – bad params. 409 Conflict - For POST / PUT requests if the resource already exists. 500 Internal Server Error 503 Service Unavailable · Leverage uncommon HTTP Verbs to reduce payload sizes HEAD - retrieves just the resource meta-information. OPTIONS returns the actions supported for the specified resource. PATCH - partial modification of a resource. · When using PUT, POST or PATCH, send the data as a document in the body of the request. Don't use query parameters to alter state. · Utilize Headers for content negotiation, caching, authorization, throttling o Content Negotiation – choose representation (e.g. JSON or XML and version), language & compression. Signal via RequestHeader.Accept & ResponseHeader.Content-Type Accept: application/json;version=1.0 Accept-Language: en-US Accept-Charset: UTF-8 Accept-Encoding: gzip o Caching - ResponseHeader: Expires (absolute expiry time) or Cache-Control (relative expiry time) o Authorization - basic HTTP authentication uses the RequestHeader.Authorization to specify a base64 encoded string "username:password". can be used in combination with SSL/TLS (HTTPS) and leverage OAuth2 3rd party token-claims authorization. Authorization: Basic sQJlaTp5ZWFslylnaNZ= o Rate Limiting - Not currently part of HTTP so specify non-standard headers prefixed with X- in the ResponseHeader. X-RateLimit-Limit: 10000 X-RateLimit-Remaining: 9990 · HATEOAS Methodology - Hypermedia As The Engine Of Application State – leverage API as a state machine where resources are states and the transitions between states are links between resources and are included in their representation (hypermedia) – get API metadata signatures from the response Link header - in a truly REST based architecture any URL, except the initial URL, can be changed, even to other servers, without worrying about the client. · error responses - Do not just send back a 200 OK with every response. Response should consist of HTTP error status code (JQuery has automated support for this), A human readable message , A Link to a meaningful state transition , & the original data payload that was problematic. · the URIs will typically map to a server-side controller and a method name specified by the type of request method. Stuff all your calls into just four methods is not as crazy as it sounds. · Scoping - Path variables look like you’re traversing a hierarchy, and query variables look like you’re passing arguments into an algorithm · Mapping URIs to Controllers - have one controller for each resource is not a rule – can consolidate - route requests to the appropriate controller and action method · Keep URls Consistent - Sometimes it’s tempting to just shorten our URIs. not recommend this as this can cause confusion · Join Naming – for m-m entity relations there may be multiple hierarchy traversal paths · Routing – useful level of indirection for versioning, server backend mocking in development ASPNET WebAPI Considerations ASPNET WebAPI implements a lot (but not all) RESTful API design considerations as part of its infrastructure and via its coding convention. Overview When developing an API there are basically three main steps: 1. Plan out your URIs 2. Setup return values and response codes for your URIs 3. Implement a framework for your API.   Design · Leverage Models MVC folder · Repositories – support IoC for tests, abstraction · Create DTO classes – a level of indirection decouples & allows swap out · Self links can be generated using the UrlHelper · Use IQueryable to support projections across the wire · Models can support restful navigation properties – ICollection<T> · async mechanism for long running ops - return a response with a ticket – the client can then poll or be pushed the final result later. · Design for testability - Test using HttpClient , JQuery ( $.getJSON , $.each) , fiddler, browser debug. Leverage IDependencyResolver – IoC wrapper for mocking · Easy debugging - IE F12 developer tools: Network tab, Request Headers tab     Routing · HTTP request method is matched to the method name. (This rule applies only to GET, POST, PUT, and DELETE requests.) · {id}, if present, is matched to a method parameter named id. · Query parameters are matched to parameter names when possible · Done in config via Routes.MapHttpRoute – similar to MVC routing · Can alternatively: o decorate controller action methods with HttpDelete, HttpGet, HttpHead,HttpOptions, HttpPatch, HttpPost, or HttpPut., + the ActionAttribute o use AcceptVerbsAttribute to support other HTTP verbs: e.g. PATCH, HEAD o use NonActionAttribute to prevent a method from getting invoked as an action · route table Uris can support placeholders (via curly braces{}) – these can support default values and constraints, and optional values · The framework selects the first route in the route table that matches the URI. Response customization · Response code: By default, the Web API framework sets the response status code to 200 (OK). But according to the HTTP/1.1 protocol, when a POST request results in the creation of a resource, the server should reply with status 201 (Created). Non Get methods should return HttpResponseMessage · Location: When the server creates a resource, it should include the URI of the new resource in the Location header of the response. public HttpResponseMessage PostProduct(Product item) {     item = repository.Add(item);     var response = Request.CreateResponse<Product>(HttpStatusCode.Created, item);     string uri = Url.Link("DefaultApi", new { id = item.Id });     response.Headers.Location = new Uri(uri);     return response; } Validation · Decorate Models / DTOs with System.ComponentModel.DataAnnotations properties RequiredAttribute, RangeAttribute. · Check payloads using ModelState.IsValid · Under posting – leave out values in JSON payload à JSON formatter assigns a default value. Use with RequiredAttribute · Over-posting - if model has RO properties à use DTO instead of model · Can hook into pipeline by deriving from ActionFilterAttribute & overriding OnActionExecuting Config · Done in App_Start folder > WebApiConfig.cs – static Register method: HttpConfiguration param: The HttpConfiguration object contains the following members. Member Description DependencyResolver Enables dependency injection for controllers. Filters Action filters – e.g. exception filters. Formatters Media-type formatters. by default contains JsonFormatter, XmlFormatter IncludeErrorDetailPolicy Specifies whether the server should include error details, such as exception messages and stack traces, in HTTP response messages. Initializer A function that performs final initialization of the HttpConfiguration. MessageHandlers HTTP message handlers - plug into pipeline ParameterBindingRules A collection of rules for binding parameters on controller actions. Properties A generic property bag. Routes The collection of routes. Services The collection of services. · Configure JsonFormatter for circular references to support links: PreserveReferencesHandling.Objects Documentation generation · create a help page for a web API, by using the ApiExplorer class. · The ApiExplorer class provides descriptive information about the APIs exposed by a web API as an ApiDescription collection · create the help page as an MVC view public ILookup<string, ApiDescription> GetApis()         {             return _explorer.ApiDescriptions.ToLookup(                 api => api.ActionDescriptor.ControllerDescriptor.ControllerName); · provide documentation for your APIs by implementing the IDocumentationProvider interface. Documentation strings can come from any source that you like – e.g. extract XML comments or define custom attributes to apply to the controller [ApiDoc("Gets a product by ID.")] [ApiParameterDoc("id", "The ID of the product.")] public HttpResponseMessage Get(int id) · GlobalConfiguration.Configuration.Services – add the documentation Provider · To hide an API from the ApiExplorer, add the ApiExplorerSettingsAttribute Plugging into the Message Handler pipeline · Plug into request / response pipeline – derive from DelegatingHandler and override theSendAsync method – e.g. for logging error codes, adding a custom response header · Can be applied globally or to a specific route Exception Handling · Throw HttpResponseException on method failures – specify HttpStatusCode enum value – examine this enum, as its values map well to typical op problems · Exception filters – derive from ExceptionFilterAttribute & override OnException. Apply on Controller or action methods, or add to global HttpConfiguration.Filters collection · HttpError object provides a consistent way to return error information in the HttpResponseException response body. · For model validation, you can pass the model state to CreateErrorResponse, to include the validation errors in the response public HttpResponseMessage PostProduct(Product item) {     if (!ModelState.IsValid)     {         return Request.CreateErrorResponse(HttpStatusCode.BadRequest, ModelState); Cookie Management · Cookie header in request and Set-Cookie headers in a response - Collection of CookieState objects · Specify Expiry, max-age resp.Headers.AddCookies(new CookieHeaderValue[] { cookie }); Internet Media Types, formatters and serialization · Defaults to application/json · Request Accept header and response Content-Type header · determines how Web API serializes and deserializes the HTTP message body. There is built-in support for XML, JSON, and form-urlencoded data · customizable formatters can be inserted into the pipeline · POCO serialization is opt out via JsonIgnoreAttribute, or use DataMemberAttribute for optin · JSON serializer leverages NewtonSoft Json.NET · loosely structured JSON objects are serialzed as JObject which derives from Dynamic · to handle circular references in json: json.SerializerSettings.PreserveReferencesHandling =    PreserveReferencesHandling.All à {"$ref":"1"}. · To preserve object references in XML [DataContract(IsReference=true)] · Content negotiation Accept: Which media types are acceptable for the response, such as “application/json,” “application/xml,” or a custom media type such as "application/vnd.example+xml" Accept-Charset: Which character sets are acceptable, such as UTF-8 or ISO 8859-1. Accept-Encoding: Which content encodings are acceptable, such as gzip. Accept-Language: The preferred natural language, such as “en-us”. o Web API uses the Accept and Accept-Charset headers. (At this time, there is no built-in support for Accept-Encoding or Accept-Language.) · Controller methods can take JSON representations of DTOs as params – auto-deserialization · Typical JQuery GET request: function find() {     var id = $('#prodId').val();     $.getJSON("api/products/" + id,         function (data) {             var str = data.Name + ': $' + data.Price;             $('#product').text(str);         })     .fail(         function (jqXHR, textStatus, err) {             $('#product').text('Error: ' + err);         }); }            · Typical GET response: HTTP/1.1 200 OK Server: ASP.NET Development Server/10.0.0.0 Date: Mon, 18 Jun 2012 04:30:33 GMT X-AspNet-Version: 4.0.30319 Cache-Control: no-cache Pragma: no-cache Expires: -1 Content-Type: application/json; charset=utf-8 Content-Length: 175 Connection: Close [{"Id":1,"Name":"TomatoSoup","Price":1.39,"ActualCost":0.99},{"Id":2,"Name":"Hammer", "Price":16.99,"ActualCost":10.00},{"Id":3,"Name":"Yo yo","Price":6.99,"ActualCost": 2.05}] True OData support · Leverage Query Options $filter, $orderby, $top and $skip to shape the results of controller actions annotated with the [Queryable]attribute. [Queryable]  public IQueryable<Supplier> GetSuppliers()  · Query: ~/Suppliers?$filter=Name eq ‘Microsoft’ · Applies the following selection filter on the server: GetSuppliers().Where(s => s.Name == “Microsoft”)  · Will pass the result to the formatter. · true support for the OData format is still limited - no support for creates, updates, deletes, $metadata and code generation etc · vnext: ability to configure how EditLinks, SelfLinks and Ids are generated Self Hosting no dependency on ASPNET or IIS: using (var server = new HttpSelfHostServer(config)) {     server.OpenAsync().Wait(); Tracing · tracability tools, metrics – e.g. send to nagios · use your choice of tracing/logging library, whether that is ETW,NLog, log4net, or simply System.Diagnostics.Trace. · To collect traces, implement the ITraceWriter interface public class SimpleTracer : ITraceWriter {     public void Trace(HttpRequestMessage request, string category, TraceLevel level,         Action<TraceRecord> traceAction)     {         TraceRecord rec = new TraceRecord(request, category, level);         traceAction(rec);         WriteTrace(rec); · register the service with config · programmatically trace – has helper extension methods: Configuration.Services.GetTraceWriter().Info( · Performance tracing - pipeline writes traces at the beginning and end of an operation - TraceRecord class includes aTimeStamp property, Kind property set to TraceKind.Begin / End Security · Roles class methods: RoleExists, AddUserToRole · WebSecurity class methods: UserExists, .CreateUserAndAccount · Request.IsAuthenticated · Leverage HTTP 401 (Unauthorized) response · [AuthorizeAttribute(Roles="Administrator")] – can be applied to Controller or its action methods · See section in WebApi document on "Claim-based-security for ASP.NET Web APIs using DotNetOpenAuth" – adapt this to STS.--> Web API Host exposes secured Web APIs which can only be accessed by presenting a valid token issued by the trusted issuer. http://zamd.net/2012/05/04/claim-based-security-for-asp-net-web-apis-using-dotnetopenauth/ · Use MVC membership provider infrastructure and add a DelegatingHandler child class to the WebAPI pipeline - http://stackoverflow.com/questions/11535075/asp-net-mvc-4-web-api-authentication-with-membership-provider - this will perform the login actions · Then use AuthorizeAttribute on controllers and methods for role mapping- http://sixgun.wordpress.com/2012/02/29/asp-net-web-api-basic-authentication/ · Alternate option here is to rely on MVC App : http://forums.asp.net/t/1831767.aspx/1

  Read the article

• PASS: Bylaw Change 2013

  - by Bill Graziano

  PASS launched a Global Growth Initiative in the Summer of 2011 with the appointment of three international Board advisors.  Since then we’ve thought and talked extensively about how we make PASS more relevant to our members outside the US and Canada.  We’ve collected much of that discussion in our Global Growth site.  You can find vision documents, plans, governance proposals, feedback sites, and transcripts of Twitter chats and town hall meetings.  We also address these plans at the Board Q&A during the 2012 Summit. One of the biggest changes coming out of this process is around how we elect Board members.  And that requires a change to the bylaws.  We published the proposed bylaw changes as a red-lined document so you can clearly see the changes.  Our goal in these bylaw changes was to address the changes required by the global growth initiatives, conduct a legal review of the document and address other minor issues in the document.  There are numerous small wording changes throughout the document.  For example, we replaced every reference of “The Corporation” with the word “PASS” so it now reads “PASS is organized…”. Board Composition The biggest change in these bylaw changes is how the Board is composed and elected.  This discussion starts in section VI.2.  This section now says that some elected directors will come from geographic regions.  I think this is the best way to make sure we give all of our members a voice in the leadership of the organization.  The key parts of this section are: The remaining Directors (i.e. the non-Officer Directors and non-Vendor Appointed Directors) shall be elected by the voting membership (“Elected Directors”). Elected Directors shall include representatives of defined PASS regions (“Regions”) as set forth below (“Regional Directors”) and at minimum one (1) additional Director-at-Large whose selection is not limited by region. Regional Directors shall include, but are not limited to, two (2) seats for the Region covering Canada and the United States of America. Additional Regions for the purpose of electing additional Regional Directors and additional Director-at-Large seats for the purpose of expanding the Board shall be defined by a majority vote of the current Board of Directors and must be established prior to the public call for nominations in the general election. Previously defined Regions and seats approved by the Board of Directors shall remain in effect and can only be modified by a 2/3 majority vote by the then current Board of Directors. Currently PASS has six At-Large Directors elected by the members.  These changes allow for a Regional Director position that is elected by the members but must come from a particular region.  It also stipulates that there must always be at least one Director-at-Large who can come from any region. We also understand that PASS is currently a very US-centric organization.  Our Summit is held in America, roughly half our chapters are in the US and Canada and most of the Board members over the last ten years have come from America.  We wanted to reflect that by making sure that our US and Canadian volunteers would continue to play a significant role by ensuring that two Regional seats are reserved specifically for Canada and the US. Other than that, the bylaws don’t create any specific regional seats.  These rules allow us to create Regional Director seats but don’t require it.  We haven’t fully discussed what the criteria will be in order for a region to have a seat designated for it or how many regions there will be.  In our discussions we’ve broadly discussed regions for United States and Canada Europe, Middle East, and Africa (EMEA) Australia, New Zealand and Asia (also known as Asia Pacific or APAC) Mexico, South America, and Central America (LATAM) As you can see, our thinking is that there will be a few large regions.  I’ve also considered a non-North America region that we can gradually split into the regions above as our membership grows in those areas.  The regions will be defined by a policy document that will be published prior to the elections. I’m hoping that over the next year we can begin to publish more of what we do as Board-approved policy documents. While the bylaws only require a single non-region specific At-large Director, I would expect we would always have two.  That way we can have one in each election.  I think it’s important that we always have one seat open that anyone who is eligible to run for the Board can contest.  The Board is required to have any regions defined prior to the start of the election process. Board Elections – Regional Seats We spent a lot of time discussing how the elections would work for these Regional Director seats.  Ultimately we decided that the simplest solution is that every PASS member should vote for every open seat.  Section VIII.3 reads: Candidates who are eligible (i.e. eligible to serve in such capacity subject to the criteria set forth herein or adopted by the Board of Directors) shall be designated to fill open Board seats in the following order of priority on the basis of total votes received: (i) full term Regional Director seats, (ii) full term Director-at-Large seats, (iii) not full term (vacated) Regional Director seats, (iv) not full term (vacated) Director-at-Large seats. For the purposes of clarity, because of eligibility requirements, it is contemplated that the candidates designated to the open Board seats may not receive more votes than certain other candidates who are not selected to the Board. We debated whether to have multiple ballots or one single ballot.  Multiple ballot elections get complicated quickly.  Let’s say we have a ballot for US/Canada and one for Region 2.  After that we’d need a mechanism to merge those two together and come up with the winner of the at-large seat or have another election for the at-large position.  We think the best way to do this is a single ballot and putting the highest vote getters into the most restrictive seats.  Let’s look at an example: There are seats open for Region 1, Region 2 and at-large.  The election results are as follows: Candidate A (eligible for Region 1) – 550 votes Candidate B (eligible for Region 1) – 525 votes Candidate C (eligible for Region 1) – 475 votes Candidate D (eligible for Region 2) – 125 votes Candidate E (eligible for Region 2) – 75 votes In this case, Candidate A is the winner for Region 1 and is assigned that seat.  Candidate D is the winner for Region 2 and is assigned that seat.  The at-large seat is filled by the high remaining vote getter which is Candidate B. The key point to understand is that we may have a situation where a person with a lower vote total is elected to a regional seat and a person with a higher vote total is excluded.  This will be true whether we had multiple ballots or a single ballot.  Board Elections – Vacant Seats The other change to the election process is for vacant Board seats.  The actual changes are sprinkled throughout the document. Previously we didn’t have a mechanism that allowed for an election of a Board seat that we knew would be vacant in the future.  The most common case is when a Board members moves to an Officer role in the middle of their term.  One of the key changes is to allow the number of votes members have to match the number of open seats.  This allows each voter to express their preference on all open seats.  This only applies when we know about the opening prior to the call for nominations.  This all means that if there’s a seat will be open at the start of the next Board term, and we know about it prior to the call for nominations, we can include that seat in the elections.  Ultimately, the aim is to have PASS members decide who sits on the Board in as many situations as possible. We discussed the option of changing the bylaws to just take next highest vote-getter in all other cases.  I think that’s wrong for the following reasons: All voters aren’t able to express an opinion on all candidates.  If there are five people running for three seats, you can only vote for three.  You have no way to express your preference between #4 and #5. Different candidates may have different information about the number of seats available.  A person may learn that a Board member plans to resign at the end of the year prior to that information being made public. They may understand that the top four vote getters will end up on the Board while the rest of the members believe there are only three openings.  This may affect someone’s decision to run.  I don’t think this creates a transparent, fair election. Board members may use their knowledge of the election results to decide whether to remain on the Board or not.  Admittedly this one is unlikely but I don’t want to create a situation where this accusation can be leveled. I think the majority of vacancies in the future will be handled through elections.  The bylaw section quoted above also indicates that partial term vacancies will be filled after the full term seats are filled. Removing Directors Section VI.7 on removing directors has always had a clause that allowed members to remove an elected director.  We also had a clause that allowed appointed directors to be removed.  We added a clause that allows the Board to remove for cause any director with a 2/3 majority vote.  The updated text reads: Any Director may be removed for cause by a 2/3 majority vote of the Board of Directors whenever in its judgment the best interests of PASS would be served thereby. Notwithstanding the foregoing, the authority of any Director to act as in an official capacity as a Director or Officer of PASS may be suspended by the Board of Directors for cause. Cause for suspension or removal of a Director shall include but not be limited to failure to meet any Board-approved performance expectations or the presence of a reason for suspension or dismissal as listed in Addendum B of these Bylaws. The first paragraph is updated and the second and third are unchanged (except cleaning up language).  If you scroll down and look at Addendum B of these bylaws you find the following: Cause for suspension or dismissal of a member of the Board of Directors may include: Inability to attend Board meetings on a regular basis. Inability or unwillingness to act in a capacity designated by the Board of Directors. Failure to fulfill the responsibilities of the office. Inability to represent the Region elected to represent Failure to act in a manner consistent with PASS's Bylaws and/or policies. Misrepresentation of responsibility and/or authority. Misrepresentation of PASS. Unresolved conflict of interests with Board responsibilities. Breach of confidentiality. The bold line about your inability to represent your region is what we added to the bylaws in this revision.  We also added a clause to section VII.3 allowing the Board to remove an officer.  That clause is much less restrictive.  It doesn’t require cause and only requires a simple majority. The Board of Directors may remove any Officer whenever in their judgment the best interests of PASS shall be served by such removal. Other There are numerous other small changes throughout the document. Proxy voting.  The laws around how members and Board members proxy votes are specific in Illinois law.  PASS is an Illinois corporation and is subject to Illinois laws.  We changed section IV.5 to come into compliance with those laws.  Specifically this says you can only vote through a proxy if you have a written proxy through your authorized attorney.  English language proficiency.  As we increase our global footprint we come across more members that aren’t native English speakers.  The business of PASS is conducted in English and it’s important that our Board members speak English.  If we get big enough to afford translators, we may be able to relax this but right now we need English language skills for effective Board members. Committees.  The language around committees in section IX is old and dated.  Our lawyers advised us to clean it up.  This section specifically applies to any committees that the Board may form outside of portfolios.  We removed the term limits, quorum and vacancies clause.  We don’t currently have any committees that this would apply to.  The Nominating Committee is covered elsewhere in the bylaws. Electronic Votes.  The change allows the Board to vote via email but the results must be unanimous.  This is to conform with Illinois state law. Immediate Past President.  There was no mechanism to fill the IPP role if an outgoing President chose not to participate.  We changed section VII.8 to allow the Board to invite any previous President to fill the role by majority vote. Nominations Committee.  We’ve opened the language to allow for the transparent election of the Nominations Committee as outlined by the 2011 Election Review Committee. Revocation of Charters. The language surrounding the revocation of charters for local groups was flagged by the lawyers. We have allowed for the local user group to make all necessary payment before considering returning of items to PASS if required. Bylaw notification. We’ve spent countless meetings working on these bylaws with the intent to not open them again any time in the near future. Should the bylaws be opened again, we have included a clause ensuring that the PASS membership is involved. I’m proud that the Board has remained committed to transparency and accountability to members. This clause will require that same level of commitment in the future even when all the current Board members have rolled off. I think that covers everything.  I’d encourage you to look through the red-line document and see the changes.  It’s helpful to look at the language that’s being removed and the language that’s being added.  I’m happy to answer any questions here or you can email them to [email protected].

  Read the article

• 3D Graphics with XNA Game Studio 4.0 bug in light map?

  - by Eibis

  i'm following the tutorials on 3D Graphics with XNA Game Studio 4.0 and I came up with an horrible effect when I tried to implement the Light Map http://i.stack.imgur.com/BUWvU.jpg this effect shows up when I look towards the center of the house (and it moves with me). it has this shape because I'm using a sphere to represent light; using other light shapes gives different results. I'm using a class PreLightingRenderer: using System; using System.Collections.Generic; using System.Linq; using System.Text; using Microsoft.Xna.Framework; using Microsoft.Xna.Framework.Graphics; using Dhpoware; using Microsoft.Xna.Framework.Content; namespace XNAFirstPersonCamera { public class PrelightingRenderer { // Normal, depth, and light map render targets RenderTarget2D depthTarg; RenderTarget2D normalTarg; RenderTarget2D lightTarg; // Depth/normal effect and light mapping effect Effect depthNormalEffect; Effect lightingEffect; // Point light (sphere) mesh Model lightMesh; // List of models, lights, and the camera public List<CModel> Models { get; set; } public List<PPPointLight> Lights { get; set; } public FirstPersonCamera Camera { get; set; } GraphicsDevice graphicsDevice; int viewWidth = 0, viewHeight = 0; public PrelightingRenderer(GraphicsDevice GraphicsDevice, ContentManager Content) { viewWidth = GraphicsDevice.Viewport.Width; viewHeight = GraphicsDevice.Viewport.Height; // Create the three render targets depthTarg = new RenderTarget2D(GraphicsDevice, viewWidth, viewHeight, false, SurfaceFormat.Single, DepthFormat.Depth24); normalTarg = new RenderTarget2D(GraphicsDevice, viewWidth, viewHeight, false, SurfaceFormat.Color, DepthFormat.Depth24); lightTarg = new RenderTarget2D(GraphicsDevice, viewWidth, viewHeight, false, SurfaceFormat.Color, DepthFormat.Depth24); // Load effects depthNormalEffect = Content.Load<Effect>(@"Effects\PPDepthNormal"); lightingEffect = Content.Load<Effect>(@"Effects\PPLight"); // Set effect parameters to light mapping effect lightingEffect.Parameters["viewportWidth"].SetValue(viewWidth); lightingEffect.Parameters["viewportHeight"].SetValue(viewHeight); // Load point light mesh and set light mapping effect to it lightMesh = Content.Load<Model>(@"Models\PPLightMesh"); lightMesh.Meshes[0].MeshParts[0].Effect = lightingEffect; this.graphicsDevice = GraphicsDevice; } public void Draw() { drawDepthNormalMap(); drawLightMap(); prepareMainPass(); } void drawDepthNormalMap() { // Set the render targets to 'slots' 1 and 2 graphicsDevice.SetRenderTargets(normalTarg, depthTarg); // Clear the render target to 1 (infinite depth) graphicsDevice.Clear(Color.White); // Draw each model with the PPDepthNormal effect foreach (CModel model in Models) { model.CacheEffects(); model.SetModelEffect(depthNormalEffect, false); model.Draw(Camera.ViewMatrix, Camera.ProjectionMatrix, Camera.Position); model.RestoreEffects(); } // Un-set the render targets graphicsDevice.SetRenderTargets(null); } void drawLightMap() { // Set the depth and normal map info to the effect lightingEffect.Parameters["DepthTexture"].SetValue(depthTarg); lightingEffect.Parameters["NormalTexture"].SetValue(normalTarg); // Calculate the view * projection matrix Matrix viewProjection = Camera.ViewMatrix * Camera.ProjectionMatrix; // Set the inverse of the view * projection matrix to the effect Matrix invViewProjection = Matrix.Invert(viewProjection); lightingEffect.Parameters["InvViewProjection"].SetValue(invViewProjection); // Set the render target to the graphics device graphicsDevice.SetRenderTarget(lightTarg); // Clear the render target to black (no light) graphicsDevice.Clear(Color.Black); // Set render states to additive (lights will add their influences) graphicsDevice.BlendState = BlendState.Additive; graphicsDevice.DepthStencilState = DepthStencilState.None; foreach (PPPointLight light in Lights) { // Set the light's parameters to the effect light.SetEffectParameters(lightingEffect); // Calculate the world * view * projection matrix and set it to // the effect Matrix wvp = (Matrix.CreateScale(light.Attenuation) * Matrix.CreateTranslation(light.Position)) * viewProjection; lightingEffect.Parameters["WorldViewProjection"].SetValue(wvp); // Determine the distance between the light and camera float dist = Vector3.Distance(Camera.Position, light.Position); // If the camera is inside the light-sphere, invert the cull mode // to draw the inside of the sphere instead of the outside if (dist < light.Attenuation) graphicsDevice.RasterizerState = RasterizerState.CullClockwise; // Draw the point-light-sphere lightMesh.Meshes[0].Draw(); // Revert the cull mode graphicsDevice.RasterizerState = RasterizerState.CullCounterClockwise; } // Revert the blending and depth render states graphicsDevice.BlendState = BlendState.Opaque; graphicsDevice.DepthStencilState = DepthStencilState.Default; // Un-set the render target graphicsDevice.SetRenderTarget(null); } void prepareMainPass() { foreach (CModel model in Models) foreach (ModelMesh mesh in model.Model.Meshes) foreach (ModelMeshPart part in mesh.MeshParts) { // Set the light map and viewport parameters to each model's effect if (part.Effect.Parameters["LightTexture"] != null) part.Effect.Parameters["LightTexture"].SetValue(lightTarg); if (part.Effect.Parameters["viewportWidth"] != null) part.Effect.Parameters["viewportWidth"].SetValue(viewWidth); if (part.Effect.Parameters["viewportHeight"] != null) part.Effect.Parameters["viewportHeight"].SetValue(viewHeight); } } } } that uses three effect: PPDepthNormal.fx float4x4 World; float4x4 View; float4x4 Projection; struct VertexShaderInput { float4 Position : POSITION0; float3 Normal : NORMAL0; }; struct VertexShaderOutput { float4 Position : POSITION0; float2 Depth : TEXCOORD0; float3 Normal : TEXCOORD1; }; VertexShaderOutput VertexShaderFunction(VertexShaderInput input) { VertexShaderOutput output; float4x4 viewProjection = mul(View, Projection); float4x4 worldViewProjection = mul(World, viewProjection); output.Position = mul(input.Position, worldViewProjection); output.Normal = mul(input.Normal, World); // Position's z and w components correspond to the distance // from camera and distance of the far plane respectively output.Depth.xy = output.Position.zw; return output; } // We render to two targets simultaneously, so we can't // simply return a float4 from the pixel shader struct PixelShaderOutput { float4 Normal : COLOR0; float4 Depth : COLOR1; }; PixelShaderOutput PixelShaderFunction(VertexShaderOutput input) { PixelShaderOutput output; // Depth is stored as distance from camera / far plane distance // to get value between 0 and 1 output.Depth = input.Depth.x / input.Depth.y; // Normal map simply stores X, Y and Z components of normal // shifted from (-1 to 1) range to (0 to 1) range output.Normal.xyz = (normalize(input.Normal).xyz / 2) + .5; // Other components must be initialized to compile output.Depth.a = 1; output.Normal.a = 1; return output; } technique Technique1 { pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_2_0 PixelShaderFunction(); } } PPLight.fx float4x4 WorldViewProjection; float4x4 InvViewProjection; texture2D DepthTexture; texture2D NormalTexture; sampler2D depthSampler = sampler_state { texture = ; minfilter = point; magfilter = point; mipfilter = point; }; sampler2D normalSampler = sampler_state { texture = ; minfilter = point; magfilter = point; mipfilter = point; }; float3 LightColor; float3 LightPosition; float LightAttenuation; // Include shared functions #include "PPShared.vsi" struct VertexShaderInput { float4 Position : POSITION0; }; struct VertexShaderOutput { float4 Position : POSITION0; float4 LightPosition : TEXCOORD0; }; VertexShaderOutput VertexShaderFunction(VertexShaderInput input) { VertexShaderOutput output; output.Position = mul(input.Position, WorldViewProjection); output.LightPosition = output.Position; return output; } float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0 { // Find the pixel coordinates of the input position in the depth // and normal textures float2 texCoord = postProjToScreen(input.LightPosition) + halfPixel(); // Extract the depth for this pixel from the depth map float4 depth = tex2D(depthSampler, texCoord); // Recreate the position with the UV coordinates and depth value float4 position; position.x = texCoord.x * 2 - 1; position.y = (1 - texCoord.y) * 2 - 1; position.z = depth.r; position.w = 1.0f; // Transform position from screen space to world space position = mul(position, InvViewProjection); position.xyz /= position.w; // Extract the normal from the normal map and move from // 0 to 1 range to -1 to 1 range float4 normal = (tex2D(normalSampler, texCoord) - .5) * 2; // Perform the lighting calculations for a point light float3 lightDirection = normalize(LightPosition - position); float lighting = clamp(dot(normal, lightDirection), 0, 1); // Attenuate the light to simulate a point light float d = distance(LightPosition, position); float att = 1 - pow(d / LightAttenuation, 6); return float4(LightColor * lighting * att, 1); } technique Technique1 { pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_2_0 PixelShaderFunction(); } } PPShared.vsi has some common functions: float viewportWidth; float viewportHeight; // Calculate the 2D screen position of a 3D position float2 postProjToScreen(float4 position) { float2 screenPos = position.xy / position.w; return 0.5f * (float2(screenPos.x, -screenPos.y) + 1); } // Calculate the size of one half of a pixel, to convert // between texels and pixels float2 halfPixel() { return 0.5f / float2(viewportWidth, viewportHeight); } and finally from the Game class I set up in LoadContent with: effect = Content.Load(@"Effects\PPModel"); models[0] = new CModel(Content.Load(@"Models\teapot"), new Vector3(-50, 80, 0), new Vector3(0, 0, 0), 1f, Content.Load(@"Textures\prova_texture_autocad"), GraphicsDevice); house = new CModel(Content.Load(@"Models\house"), new Vector3(0, 0, 0), new Vector3((float)-Math.PI / 2, 0, 0), 35.0f, Content.Load(@"Textures\prova_texture_autocad"), GraphicsDevice); models[0].SetModelEffect(effect, true); house.SetModelEffect(effect, true); renderer = new PrelightingRenderer(GraphicsDevice, Content); renderer.Models = new List(); renderer.Models.Add(house); renderer.Models.Add(models[0]); renderer.Lights = new List() { new PPPointLight(new Vector3(0, 120, 0), Color.White * .85f, 2000) }; where PPModel.fx is: float4x4 World; float4x4 View; float4x4 Projection; texture2D BasicTexture; sampler2D basicTextureSampler = sampler_state { texture = ; addressU = wrap; addressV = wrap; minfilter = anisotropic; magfilter = anisotropic; mipfilter = linear; }; bool TextureEnabled = true; texture2D LightTexture; sampler2D lightSampler = sampler_state { texture = ; minfilter = point; magfilter = point; mipfilter = point; }; float3 AmbientColor = float3(0.15, 0.15, 0.15); float3 DiffuseColor; #include "PPShared.vsi" struct VertexShaderInput { float4 Position : POSITION0; float2 UV : TEXCOORD0; }; struct VertexShaderOutput { float4 Position : POSITION0; float2 UV : TEXCOORD0; float4 PositionCopy : TEXCOORD1; }; VertexShaderOutput VertexShaderFunction(VertexShaderInput input) { VertexShaderOutput output; float4x4 worldViewProjection = mul(World, mul(View, Projection)); output.Position = mul(input.Position, worldViewProjection); output.PositionCopy = output.Position; output.UV = input.UV; return output; } float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0 { // Sample model's texture float3 basicTexture = tex2D(basicTextureSampler, input.UV); if (!TextureEnabled) basicTexture = float4(1, 1, 1, 1); // Extract lighting value from light map float2 texCoord = postProjToScreen(input.PositionCopy) + halfPixel(); float3 light = tex2D(lightSampler, texCoord); light += AmbientColor; return float4(basicTexture * DiffuseColor * light, 1); } technique Technique1 { pass Pass1 { VertexShader = compile vs_1_1 VertexShaderFunction(); PixelShader = compile ps_2_0 PixelShaderFunction(); } } I don't have any idea on what's wrong... googling the web I found that this tutorial may have some bug but I don't know if it's the LightModel fault (the sphere) or in a shader or in the class PrelightingRenderer. Any help is very appreciated, thank you for reading!

  Read the article

• The builds tools for WindowsKernelModeDriver8.1 (Platform Toolset = 'WindowsKernelModeDriver8.1') cannot be found

  - by Rajeshwar

  I am attempting to build this example. However I am getting the error error MSB8020: The builds tools for WindowsApplicationForDrivers8.1 (Platform Toolset = 'WindowsApplicationForDrivers8.1') cannot be found. To build using the WindowsApplicationForDrivers8.1 build tools, either click the Project menu or right-click the solution, and then select "Update VC++ Projects...". Install WindowsApplicationForDrivers8.1 to build using the WindowsApplicationForDrivers8.1 build tools. I have already installed WDK 8.1 .exe andwhen I right click on the solution I cant find any thing that states "Update VC++ Projects" Any suggestions on how I may resolve this issue and build it. ? I am running Windows 7 and VS2012 Pro.

  Read the article

• JSON.NET "System.OutOfMemoryException" when deserializing into a DataSet

  - by Tiago

  Very new to C# and JSON. I have been struggling with this problem for about a day and can't figure it out. JSONLint states both JSON strings are valid. Trying to deserialize the following {"items":[{"id":"1122267","quantity":"1","bundle":"1"}],"seconds_ago":"1"} throws the exception An unhandled exception of type 'System.OutOfMemoryException' occurred in Newtonsoft.Json.dll If I try {"items":[{"id":"1122267","quantity":"1","bundle":"1"}]} it works. I'm reading the JSON string from a textbox and then deserializing using the following string json = textBox1.Text; DataSet dataSet = JsonConvert.DeserializeObject<DataSet>(json);

  Read the article

• Multiselect combobox with ExtJs

  - by Justin Johnson

  How do you implement a multiselect combobox as part of a Ext.FormPanel using ExtJs? I've been looking, but can't seem to find a solution that is compatible with the latest version of ExtJs (this question is similar, but doesn't have a working/current solution). This is what I have so far, but it's a single select: new Ext.FormPanel({ labelAlign: 'top', frame: true, width: 800, items: [{ layout: 'column', items:[{ columnWidth: 1, layout: 'form', items: [{ xtype: 'combo', fieldLabel: 'Countries', name: 'c[]', anchor: '95%', allowBlank: false, typeAhead: true, triggerAction: 'all', lazyRender: true, mode: 'local', store: new Ext.data.ArrayStore({ id: 0, fields: ['myId', 'displayText'], data: [ ["CA", 'Canada'], ["US", 'United States'], ["JP", 'Japan'], ] }), valueField: 'myId', displayField: 'displayText' }] }] }] }).render(document.body); I didn't see any parameters in the documentation that suggests that this is supported. I also found this and this but I could only get them working with Ext 2.

  Read the article

• Decoding AAC using MediaCodec API on Android

  - by Benjamin Reed

  I'm trying to used the MediaCodec API on Android to decode an AAC stream. (It's raw AAC.) I tried using the MediaFormat.createAudioFormat() to create the format object to pass to MediaCodec.configure(), but I kept getting errors when using AAC (audio/mp4a-latm). (It works with MP3 (audio/mpeg) though...) Finally I created a MediaExtractor for an AAC file and looked at the format object it was producing. I saw that it included the key "csd-0" for a ByteBuffer composed of two bytes both with the value 0x12. If I include that key and value in the format object that I used to configure the AAC codec, everything works. Does anyone have an idea what is going on? The documentation states that I shouldn't configure that key. Does anyone have a pointer to MediaCodec examples to decode AAC files without using MediaExtractor to generate the format object?

  Read the article

• view .doc, .docx, .rtf, .ppt file in iphone using Webview

  - by Ekra

  Hi friends I want to view .doc, .docx, .rtf, .ppt file in iphone. But I guess something is going wrong at my side and its not working for the above formats but my code is working fine for .txt and .pdf files. I have the read the document regarding Webview it states it supports viewing of the above document. below is my snippet for .doc [webView loadData:requestData MIMEType:@"application/msword" textEncodingName:@"UTF-8" baseURL:nil]; for .ppt I am using MIME type as "application/vnd.ms-powerpoint" Note: If I am making MIME type as "text/html" for .doc then it displays some garbage data So I think there is something missing in MIME type from my side. Any help is highly appreciated. Waiting for your reply.

  Read the article

• WPF databind Image.Source in MVVM

  - by BrettRobi

  I'm using MVVM and am trying to databind the Source property of Image to my ViewModel in such a way that I can change the icon on the fly. What is the best pattern to follow for this? I still have the flexibility to change my ViewModel to suit, but I don't know where to start in either the xaml or ViewModel. To be clear, I don't want my ViewModel to know about the specific images (that's for the View to know), just the state that triggers different images. For now I have just two states, lets say Red and Green. Should I create an Enum property or a bool? And then how do I databind to switch the image source?

  Read the article

< Previous Page | 110 111 112 113 114 115 116 117 118 119 120 121  | Next Page >