Search Results

Search found 20275 results on 811 pages for 'general performance'.

Page 305/811 | < Previous Page | 301 302 303 304 305 306 307 308 309 310 311 312  | Next Page >

  • Stairway to PowerPivot and DAX - Level 3: The DAX DISTINCT() Function and Basic Distinct Counts

    Bill Pearson, Business Intelligence architect and author, exposes the DAX DISTINCT() function, and then provides some hands-on exposure to its use in generating distinct counts. Moreover, he further explores working with measures in the PivotTable in this, the third Level of our new Stairway to PowerPivot and DAX series. Optimize SQL Server performance“With SQL Monitor, we can be proactive in our optimization process, instead of waiting until a customer reports a problem,” John Trumbul, Sr. Software Engineer. Optimize your servers with a free trial.

    Read the article

  • Oracle E-Business Suite Release 12 GM Speaks

    Murali Subramanian, Group Vice President and General Manager of Oracle's E-Business Suite Applications, discusses with Cliff the latest updates to Oracle's E-Business Suite Release 12 including benefits to all customers and why customers should consider upgrading.

    Read the article

  • On-Demand Webcast: Managing Oracle Exadata with Oracle Enterprise Manager 11g

    - by Scott McNeil
    Watch this on-demand webcast and discover how Oracle Enterprise Manager 11g's unique management capabilities allow you to efficiently manage all stages of Oracle Exadata's lifecycle, from testing applications on Exadata to deployment. You'll learn how to: Maximize and predict database performance Drive down IT operational costs through automation Ensure service quality with proactive management Register today and unlock the potential of Oracle Exadata for your enterprise. Register Now!

    Read the article

  • What Would a CyberWar Do To Your Business?

    - by [email protected]
    In mid-February the Bipartisan Policy Center in the United States hosted Cyber ShockWave, a simulation of how the country might respond to a catastrophic cyber event. An attack takes place, they can't isolate where it came from or who did it, simulated press reports and market impacts...and the participants in the exercise have to brief the President and advise him/her on what to do. Last week, Former Department of Homeland Security Secretary Michael Chertoff who participated in the exercise summarized his findings in Federal Computer Weekly. The article, given FCW's readership and the topic is obviously focused on the public sector and US Federal policies. However, it touches on some broader issues that impact the private sector as well--which are applicable to any government and country/region-- such as: · How would the US (or any) government collaborate to identify and defeat such an attack? Chertoff calls this out as a current gap. How do the public and private sector collaborate today? How would the massive and disparate collection of agencies and companies act together in a crunch? · What would the impact on industries and global economies be? Chertoff, and a companion article in Government Computer News, only touch briefly on the subject--focusing on the impact on capital markets. "There's no question this has a disastrous impact on the economy," said Stephen Friedman, former director of the National Economic Council under President George W. Bush who played the role of treasury secretary. "You have financial markets shut down at this point, ordinary transactions are dramatically depleted, there's no question that this has a major impact on consumer confidence." That Got Me Thinking · How would it impact Oracle's customers? I know they have business continuity plans--is this one of their scenarios? What if it's not? How would it impact manufacturing lines, ATM networks, customer call centers... · How would it impact me and the companies I rely on? The supermarket down the street, my Internet Service Provider, the service station where I bought gas last night. I sure don't have any answers, and neither do Chertoff or the participants in the exercise. "I have to tell you that ... we are operating in a bit of unchartered territory." said Jamie Gorelick, a former deputy attorney general who played the role of attorney general in the exercise. But it is a good thing that governments and businesses are considering this scenario and doing what they can to prevent it from happening.

    Read the article

  • Can anyone recommend online .Net training courses?

    - by james
    I am looking for peoples experiences with paid for online .Net training courses. In your experience, are these an able replacement for in-person training? Are they better than the many free ones provided on MSDN and the like? Are there any specific paid for ones you'd recommend? I usually prefer general book/web research myself, I have one specific provider in mind that looks really good, but I'll omit this for fear of advertising :)

    Read the article

  • A System Monitoring Tool Primer

    <b>CertCities:</b> "Linux comes with a number of utilities that can be used to monitor one or more of these performance parameters. The following sections introduce a few of these utilities and show how to understand the information presented by them"

    Read the article

  • VirtualBox 4.2.14 is now available

    - by user12611829
    The VirtualBox development team has just released version 4.2.14, and it is now available for download. This is a maintenance release for version 4.2 and contains quite a few fixes. Here is the list from the official Changelog. VMM: another TLB invalidation fix for non-present pages VMM: fixed a performance regression (4.2.8 regression; bug #11674) GUI: fixed a crash on shutdown GUI: prevent stuck keys under certain conditions on Windows hosts (bugs #2613, #6171) VRDP: fixed a rare crash on the guest screen resize VRDP: allow to change VRDP parameters (including enabling/disabling the server) if the VM is paused USB: fixed passing through devices on Mac OS X host to a VM with 2 or more virtual CPUs (bug #7462) USB: fixed hang during isochronous transfer with certain devices (4.1 regression; Windows hosts only; bug #11839) USB: properly handle orphaned URBs (bug #11207) BIOS: fixed function for returning the PCI interrupt routing table (fixes NetWare 6.x guests) BIOS: don't use the ENTER / LEAVE instructions in the BIOS as these don't work in the real mode as set up by certain guests (e.g. Plan 9 and QNX 4) DMI: allow to configure DmiChassisType (bug #11832) Storage: fixed lost writes if iSCSI is used with snapshots and asynchronous I/O (bug #11479) Storage: fixed accessing certain VHDX images created by Windows 8 (bug #11502) Storage: fixed hang when creating a snapshot using Parallels disk images (bug #9617) 3D: seamless + 3D fixes (bug #11723) 3D: version 4.2.12 was not able to read saved states of older versions under certain conditions (bug #11718) Main/Properties: don't create a guest property for non-running VMs if the property does not exist and is about to be removed (bug #11765) Main/Properties: don't forget to make new guest properties persistent after the VM was terminated (bug #11719) Main/Display: don't lose seamless regions during screen resize Main/OVF: don't crash during import if the client forgot to call Appliance::interpret() (bug #10845) Main/OVF: don't create invalid appliances by stripping the file name if the VM name is very long (bug #11814) Main/OVF: don't fail if the appliance contains multiple file references (bug #10689) Main/Metrics: fixed Solaris file descriptor leak Settings: limit depth of snapshot tree to 250 levels, as more will lead to decreased performance and may trigger crashes VBoxManage: fixed setting the parent UUID on diff images using sethdparentuuid Linux hosts: work around for not crashing as a result of automatic NUMA balancing which was introduced in Linux 3.8 (bug #11610) Windows installer: force the installation of the public certificate in background (i.e. completely prevent user interaction) if the --silent command line option is specified Windows Additions: fixed problems with partial install in the unattended case Windows Additions: fixed display glitch with the Start button in seamless mode for some themes Windows Additions: Seamless mode and auto-resize fixes Windows Additions: fixed trying to to retrieve new auto-logon credentials if current ones were not processed yet Windows Additions installer: added the /with_wddm switch to select the experimental WDDM driver by default Linux Additions: fixed setting own timed out and aborted texts in information label of the lightdm greeter Linux Additions: fixed compilation against Linux 3.2.0 Ubuntu kernels (4.2.12 regression as a side effect of the Debian kernel build fix; bug #11709) X11 Additions: reduced the CPU load of VBoxClient in drag'and'drop mode OS/2 Additions: made the mouse wheel work (bug #6793) Guest Additions: fixed problems copying and pasting between two guests on an X11 host (bug #11792) The full changelog can be found here. You can download binaries for Solaris, Linux, Windows and MacOS hosts at http://www.virtualbox.org/wiki/Downloads Technocrati Tags: Oracle Virtualization VirtualBox

    Read the article

  • Problem with user generated content

    - by grasshopper
    In general, what do you think is better in regards to adding content to a site, to allow users to add content to the site and put a flag button to report it if it doesn't fit with the site, or should only I add the content and remove that option? It will be a small site but I don't know if I'll manage to scan the site constantly or deal with the flags and on the other hand I'm worried that the site wont move forward because there will be lot less content, thoughts?

    Read the article

  • Using an AGPL 3.0-licensed library for extra functionality in an iOS app

    - by Arnold Sakhnov
    I am building an iOS application, and I am planning to incorporate an AGPL 3.0-licensed library to it to provide some extra (non-essential) functionality. I’ve got a couple of questions regarding this: Do I understand it right that this still obliges me to publish the source of my app open? If yes, does it have to be open to the general public, or only to those who have downloaded the app? Does this allow me to distribute the app for a fee?

    Read the article

  • StreamInsight is in all editions (except express)

    - by simonsabin
    Contrary to many posts and even press releases from Microsoft StreamInsight is not just for Data Center edition. It is available in all paid for editions. If you read the license terms http://go.microsoft.com/fwlink/?LinkID=186261&clcid=0x409 you will see you get StreamInsight in all paid editions. Whats confusing is the performance/limitations in each edition. The only reference I could find of these limitations is here http://blogs.msdn.com/b/streaminsight/archive/2010/02/10/streaminsight-versions...(read more)

    Read the article

  • Differences between TypeScript and Dart

    - by margabit
    Microsoft recently unveiled Typescript, a new JavaScript-like programming language. Some time ago, I heard about Dart, a new programming language created by Google to solve problems related to Javascript like performance, scalability, etc.. The purpose of both new languages seem the same to me.. What do you think? Are the purposes of the languages the same? What are the real differences about them?

    Read the article

  • How would you rank these programming skills in order of learning them? [closed]

    - by mumtaz
    As a general purpose programmer, what should you learn first and what should you learn later on? Here are some skills I wonder about... SQL Regular Expressions Multi-threading / Concurrency Functional Programming Graphics The mastery of your mother programming language's syntax/semantics/featureset The mastery of your base class framework libraries Version Control System Unit Testing XML Do you know other important ones? Please specify them... On which skills should I focus first?

    Read the article

  • SQL SERVER – Get 2 of My Books FREE at Koenig Tech Day – Where Technologies Converge!

    - by pinaldave
    As a regular reader of my blog – you must be aware of that I love to write books and talk about various subjects of my book. The founders of Koenig Solutions are my very old friends, I know them for many years. They have been my biggest supporter of my books. Coming weekend they have a technology event at their Bangalore Location. Every attendee of the technology event will get a set of two books worth Rs. 450 – ‘SQL Server Interview Questions And Answers‘ and ‘SQL Wait Stats Joes 2 Pros‘. I am going to cover a couple of topics of the books and present  as well. I am very confident that every attendee will be having a great time. I will be covering following subjects: SQL Server Tricks and Tips for Blazing Fast Performance Slow Running Queries (SQL) are the most common problem that developers face while working with SQL Server. While it is easy to blame the SQL Server for unsatisfactory performance, however the issue often persists with the way queries have been written, and how SQL Server has been set up. The session will focus on the ways of identifying problems that slow down SQL Servers, and tricks to fix them. Developers will walk out with scripts and knowledge that can be applied to their servers, immediately post the session. After the session is over – I will point to what exact location in the book where you can continue for the further learning. I am pretty excited, this is more like book reading but in entire different format. The one day event will cover four technologies in four separate interactive sessions on: Microsoft SQL Server Security VMware/Virtualization ASP.NET MVC Date of the event: Dec 15, 2012 9 AM to 6PM. Location of the event:  Koenig Solutions Ltd. # 47, 4th Block, 100 feet Road, 3rd Floor, Opp to Shanthi Sagar, Koramangala, Bangalore- 560034 Mobile : 09008096122 Office : 080- 41127140 Organizers have informed me that there are very limited seats for this event and technical session based on my book will start at Sharp 9 AM. If you show up late there are chances that you will not get any seats. Registration for the event is a MUST. Please visit this link for further information. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, SQLAuthority Author Visit, SQLAuthority News, T SQL, Technology

    Read the article

  • SQLAuthority News Three Posts on Reporting T-SQL Tuesday #005

    If you are following my blog, you already know that I am more of T-SQL and Performance Tuning type of person. I do have a good understanding of Business Intelligence suit and I also do certain training sessions on the same subject. When I was writing the blog post for T-SQL Tuesday #005 Reporting, [...]...Did you know that DotNetSlackers also publishes .net articles written by top known .net Authors? We already have over 80 articles in several categories including Silverlight. Take a look: here.

    Read the article

  • Business Strategy - Google Case Study

    Business strategy defined by SMBTN.com is a term used in business planning that implies a careful selection and application of resources to obtain a competitive advantage in anticipation of future events or trends. In more general terms business strategy is positioning a company so that it has the greatest competitive advantage over others in the markets and industries that they participate in. This process involves making corporate decisions regarding which markets to provide goods and services, pricing, acceptable quality levels, and how to interact with others in the marketplace. The primary objective of business strategy is to create and increase value for all of its shareholders and stakeholders through the creation of customer value. According to InformationWeek.com, Google has a distinctive technology advantage over its competitors like Microsoft, eBay, Amazon, Yahoo. Google utilizes custom high-performance systems which are cost efficient because they can scale to extreme workloads. This hardware allows for a huge cost advantage over its competitors. In addition, InformationWeek.com interviewed Stephen Arnold who stated that Google’s programmers are 50%-100% more productive compared to programmers working for their competitors.  He based this theory on Google’s competitors having to spend up to four times as much just to keep up. In addition to Google’s technological advantage, they also have developed a decentralized management schema where employees report directly to multiple managers and team project leaders. This allows for the responsibility of the technology department to be shared amongst multiple senior level engineers and removes the need for a singular department head to oversee the activities of the department.  This is a unique approach from the standard management style. Typically a department head like a CIO or CTO would oversee the department’s global initiatives and business functionality.  This would then be passed down and administered through middle management and implemented by programmers, business analyst, network administrators and Database administrators. It goes without saying that an IT professional’s responsibilities would be directed by Google’s technological advantage and management strategy.  Simply because they work within the department, and would have to design, develop, and support the high-performance systems and would have to report multiple managers and project leaders on a regular basis. Since Google was established and driven by new and immerging technology, all other departments would be directly impacted by the technology department.  In fact, they would have to cater to the technology department since it is a huge driving for in the success of Google. Reference: http://www.smbtn.com/smallbusinessdictionary/#b http://www.informationweek.com/news/software/linux/showArticle.jhtml?articleID=192300292&pgno=1&queryText=&isPrev=

    Read the article

  • Inventory Management concepts in XNA game

    - by user1332755
    I am trying to code the inventory system in my first real game so I have very little experience in both c# and game engine development. Basically, I need some general guidance and tips with how to structure and organize these sorts of systems. Please tell me if I am on the right track or not before I get too deep into making some badly structured system. It's fine if you don't feel like looking through my code, suggestions about general structure would also be appreciated. What I am aiming to end up with is some sort of system like Minecraft or Terraria. It must include: main inventory GUI (items can be dragged and placed in whatever slot desired Itembar outside of the main inventory which can be assigned to certain items the ability to use items from either location So far, I have 4 main classes: Inventory holds the general info and methods, inventoryslot holds info for individual slots, Itembar holds all info and methods for itself, and finally, ItemManager to manage interactions between the two and hold a master list of items. So far, my itembar works perfectly and interacts well with mousedragging items into and out of it as well as activating the item effect. Here is the code I have so far: (there is a lot but I will try to keep it relevant) This is the code for the itembar on the main screen: class Itembar { public Texture2D itembarfull, iSelected; public static Rectangle itembar = new Rectangle(5, 218, 40, 391); public Rectangle box1 = new Rectangle(itembar.X, 218, 40, 40); //up to 10 Rectangles for each slot public int Selected = 0; private ItemManager manager; public Itembar(Texture2D texture, Texture2D texture3, ItemManager mann) { itembarfull = texture; iSelected = texture3; manager = mann; } public void Update(GameTime gametime) { } public void Draw(SpriteBatch spriteBatch) { spriteBatch.Draw( itembarfull, new Vector2 (itembar.X, itembar.Y), null, Color.White, 0.0f, Vector2.Zero, 1.0f, SpriteEffects.None, 1.0f); if (Selected == 1) spriteBatch.Draw(iSelected, new Rectangle(box1.X-3, box1.Y-3, box1.Width+6, box1.Height+6), Color.White); //goes up to 10 slots } public int Box1Query() { foreach (Item item in manager.items) { if(box1.Contains(item.BoundingBox)) return manager.items.IndexOf(item); } return 999; } //10 different box queries It is working fine right now. I just put an Item in there and the box will query things like the item's effects, stack number, consumable or not etc...This one is basically almost complete. Here is the main inventory class: class Inventory { public bool isActive; public List<Rectangle> mainSlots = new List<Rectangle>(24); public List<InventorySlot> mainSlotscheck = new List<InventorySlot>(24); public static Rectangle inv = new Rectangle(841, 469, 156, 231); public Rectangle invfull = new Rectangle(inv.X, inv.Y, inv.Width, inv.Height); public Rectangle inv1 = new Rectangle(inv.X + 4, inv.Y +3, 32, 32); //goes up to inv24 resulting in a 6x4 grid of Rectangles public Inventory() { mainSlots.Add(inv1); mainSlots.Add(inv2); mainSlots.Add(inv3); mainSlots.Add(inv4); //goes up to 24 foreach (Rectangle slot in mainSlots) mainSlotscheck.Add(new InventorySlot(slot)); } //update and draw methods are empty because im not too sure what to put there public int LookforfreeSlot() { int slotnumber = 999; for (int x = 0; x < mainSlots.Count; x++) { if (mainSlotscheck[x].isFree) { slotnumber = x; break; } } return slotnumber; } } } LookforFreeSlot() method is meant to be called when I do AddtoInventory(). I'm kinda stumped about what other things I need to put in this class. Here is the inventorySlot class: (its main purpose is to check the bool "isFree" to see whether or not something already occupies the slot. But i guess it can also do other stuff like get item info.) class InventorySlot { public int X, Y; public int Width = 32, Height = 32; public Vector2 Position; public int slotnumber; public bool free = true; public int? content = null; public bool isFree { get { return free; } set { free = value; } } public InventorySlot(Rectangle slot) { slot = new Rectangle(X, Y, Width, Height); } } } Finally, here is the ItemManager (I am omitting the master list because it is too long) class ItemManager { public List<Item> items = new List<Item>(20); public List<Item> inventory1 = new List<Item>(24); public List<Item> inventory2 = new List<Item>(24); public List<Item> inventory3 = new List<Item>(24); public List<Item> inventory4 = new List<Item>(24); public Texture2D icon, filta; private Rectangle msRect; MouseState mouseState; public int ISelectedIndex; Inventory inventory; SpriteFont font; public void GenerateItems() { items.Add(new Item(new Rectangle(0, 0, 32, 32), icon, font)); items[0].name = "Grass Chip"; items[0].itemID = 0; items[0].consumable = true; items[0].stackable = true; items[0].maxStack = 99; items.Add(new Item(new Rectangle(32, 0, 32, 32), icon, font)); //master list continues. it will generate all items in the game; } public ItemManager(Inventory inv, Texture2D itemsheet, Rectangle mouseRectt, MouseState ms, Texture2D fil, SpriteFont f) { icon = itemsheet; msRect = mouseRectt; filta = fil; mouseState = ms; inventory = inv; font = f; } //once again, no update or draw public void mousedrag() { items[0].DestinationRect = new Rectangle (msRect.X, msRect.Y, 32, 32); items[0].dragging = true; } public void AddtoInventory(Item item) { int index = inventory.LookforfreeSlot(); if (index == 999) return; item.DestinationRect = inventory.mainSlots[index]; inventory.mainSlotscheck[index].content = item.itemID; inventory.mainSlotscheck[index].isFree = false; item.IsActive = true; } } } The mousedrag works pretty well. AddtoInventory doesn't work because LookforfreeSlot doesn't work. Relevant code from the main program: When I want to add something to the main inventory, I do something like this: foreach (Particle ether in ether1.ethers) { if (ether.isCollected) itemmanager.AddtoInventory(itemmanager.items[14]); } This turned out to be much longer than I had expected :( But I hope someone is interested enough to comment.

    Read the article

  • Outstanding SQL Saturday

    - by merrillaldrich
    I had the privilege to attend the SQL Saturday held in Redmond today, and it was really outstanding. Among the many sessions, I especially enjoyed and took a lot of useful information away from Greg Larsen’s Dynamic Management Views session, Kalen Delaney’s Compression Session – I am planning to implement 2008 Enterprise compression on my company’s data warehouse later this year – Remus Rusanu’s session on Service Broker to process NAP data, and Matt Masson’s presentation on high performance SSIS...(read more)

    Read the article

  • T-SQL User-Defined Functions: the good, the bad, and the ugly (part 4)

    - by Hugo Kornelis
    Scalar user-defined functions are bad for performance. I already showed that for T-SQL scalar user-defined functions without and with data access, and for most CLR scalar user-defined functions without data access , and in this blog post I will show that CLR scalar user-defined functions with data access fit into that picture. First attempt Sticking to my simplistic example of finding the triple of an integer value by reading it from a pre-populated lookup table and following the standard recommendations...(read more)

    Read the article

  • PASS Data Architecture VC presents Neil Hambly on Improve Data Quality & Integrity using Constraints

    On Tuesday June 19th 12PM noon Central, Neil Hambly will discuss "Leveraging the power of constraints to improve both data quality and performance of your databases." What are your servers really trying to tell you? Find out with new SQL Monitor 3.0, an easy-to-use tool built for no-nonsense database professionals.For effortless insights into SQL Server, download a free trial today.

    Read the article

  • What is new in Oracle SOA Suite 11g R1 PS6? by Shanny Anoep

    - by JuergenKress
    Oracle has released a new version 11.1.1.7.0 for their Oracle Fusion Middleware product line. This version includes Patch Set #6 (PS6) for Oracle SOA Suite 11g R1, with a big list of improvements and fixes for each component in that suite. In this post we will highlight some of the interesting updates with regards to troubleshooting, performance, reliability and scalability. Infrastructure/Purging scripts Database growth is a common problem for large-scale Oracle SOA Suite deployments. Oracle already provides multiple purging strategies for the SOA Suite runtime database. This patch set includes two new scripts for purging most of the runtime data: Table Recreation Script (TRS): This script can be used to reclaim as much database space as possible, while still retaining the open instances. It can be used as a corrective action for databases that grew excessively, for example when purging was not performed at all. This should be used as a single corrective action only; the script does not replace the normal purging scripts. Truncate script: Remove all records from the SOA Suite runtime tables without dropping the tables. This script can be used for cloning SOA Suite environments without copying the instance data, or for recreating test scenarios by cleaning all the runtime data. The Oracle SOA Suite Administrator's guide contains a table with the available purging strategies. Diagnostic dumps Using WLST you could already dump diagnostic information about various components of the SOA Suite. This version adds support to retrieve more information on BPEL and Adapters from the command-line. Diagnostic dumps for BPEL New diagnostic dumps are available for BPEL to get information on thread pools, average processing time for BPEL components, and average waiting times for asynchronous instances. This information can be very useful for performance analysis or troubleshooting. With WLST this information can be retrieved from the command-line and included for monitoring or reporting. Read the full article here. SOA & BPM Partner Community For regular information on Oracle SOA Suite become a member in the SOA & BPM Partner Community for registration please visit www.oracle.com/goto/emea/soa (OPN account required) If you need support with your account please contact the Oracle Partner Business Center. Blog Twitter LinkedIn Facebook Wiki Mix Forum Technorati Tags: SOA Suite PS6,SOA Community,Oracle SOA,Oracle BPM,Community,OPN,Jürgen Kress

    Read the article

  • Google I/O 2010 - Google Analytics APIs: End to end

    Google I/O 2010 - Google Analytics APIs: End to end Google I/O 2010 - Google Analytics APIs: End to end Google APIs 201 Nick Mihailovski Google Analytics measures performance of your website. Learn advanced techniques on how to use our tracking, processing and data export APIs as we walk you through an example of creating a most visited pages web element for your website. For all I/O 2010 sessions, please go to code.google.com From: GoogleDevelopers Views: 6 0 ratings Time: 55:42 More in Science & Technology

    Read the article

  • External File Upload Optimizations for Windows Azure

    - by rgillen
    [Cross posted from here: http://rob.gillenfamily.net/post/External-File-Upload-Optimizations-for-Windows-Azure.aspx] I’m wrapping up a bit of the work we’ve been doing on data movement optimizations for cloud computing and the latest set of data yielded some interesting points I thought I’d share. The work done here is not really rocket science but may, in some ways, be slightly counter-intuitive and therefore seemed worthy of posting. Summary: for those who don’t like to read detailed posts or don’t have time, the synopsis is that if you are uploading data to Azure, block your data (even down to 1MB) and upload in parallel. Set your block size based on your source file size, but if you must choose a fixed value, use 1MB. Following the above will result in significant performance gains… upwards of 10x-24x and a reduction in overall file transfer time of upwards of 90% (eg, uploading a 1GB file averaged 46.37 minutes prior to optimizations and averaged 1.86 minutes afterwards). Detail: For those of you who want more detail, or think that the claims at the end of the preceding paragraph are over-reaching, what follows is information and code supporting these claims. As the title would indicate, these tests were run from our research facility pointing to the Azure cloud (specifically US North Central as it is physically closest to us) and do not represent intra-cloud results… we have performed intra-cloud tests and the overall results are similar in notion but the data rates are significantly different as well as the tipping points for the various block sizes… this will be detailed separately). We started by building a very simple console application that would loop through a directory and upload each file to Azure storage. This application used the shipping storage client library from the 1.1 version of the azure tools. The only real variation from the client library is that we added code to collect and record the duration (in ms) and size (in bytes) for each file transferred. The code is available here. We then created a directory that had a collection of files for the following sizes: 2KB, 32KB, 64KB, 128KB, 512KB, 1MB, 5MB, 10MB, 25MB, 50MB, 100MB, 250MB, 500MB, 750MB, and 1GB (50 files for each size listed). These files contained randomly-generated binary data and do not benefit from compression (a separate discussion topic). Our file generation tool is available here. The baseline was established by running the application described above against the directory containing all of the data files. This application uploads the files in a random order so as to avoid transferring all of the files of a given size sequentially and thereby spreading the affects of periodic Internet delays across the collection of results.  We then ran some scripts to split the resulting data and generate some reports. The raw data collected for our non-optimized tests is available via the links in the Related Resources section at the bottom of this post. For each file size, we calculated the average upload time (and standard deviation) and the average transfer rate (and standard deviation). As you likely are aware, transferring data across the Internet is susceptible to many transient delays which can cause anomalies in the resulting data. It is for this reason that we randomized the order of source file processing as well as executed the tests 50x for each file size. We expect that these steps will yield a sufficiently balanced set of results. Once the baseline was collected and analyzed, we updated the test harness application with some methods to split the source file into user-defined block sizes and then to upload those blocks in parallel (using the PutBlock() method of Azure storage). The parallelization was handled by simply relying on the Parallel Extensions to .NET to provide a Parallel.For loop (see linked source for specific implementation details in Program.cs, line 173 and following… less than 100 lines total). Once all of the blocks were uploaded, we called PutBlockList() to assemble/commit the file in Azure storage. For each block transferred, the MD5 was calculated and sent ensuring that the bits that arrived matched was was intended. The timer for the blocked/parallelized transfer method wraps the entire process (source file splitting, block transfer, MD5 validation, file committal). A diagram of the process is as follows: We then tested the affects of blocking & parallelizing the transfers by running the updated application against the same source set and did a parameter sweep on the block size including 256KB, 512KB, 1MB, 2MB, and 4MB (our assumption was that anything lower than 256KB wasn’t worth the trouble and 4MB is the maximum size of a block supported by Azure). The raw data for the parallel tests is available via the links in the Related Resources section at the bottom of this post. This data was processed and then compared against the single-threaded / non-optimized transfer numbers and the results were encouraging. The Excel version of the results is available here. Two semi-obvious points need to be made prior to reviewing the data. The first is that if the block size is larger than the source file size you will end up with a “negative optimization” due to the overhead of attempting to block and parallelize. The second is that as the files get smaller, the clock-time cost of blocking and parallelizing (overhead) is more apparent and can tend towards negative optimizations. For this reason (and is supported in the raw data provided in the linked worksheet) the charts and dialog below ignore source file sizes less than 1MB. (click chart for full size image) The chart above illustrates some interesting points about the results: When the block size is smaller than the source file, performance increases but as the block size approaches and then passes the source file size, you see decreasing benefit to the point of negative gains (see the values for the 1MB file size) For some of the moderately-sized source files, small blocks (256KB) are best As the size of the source file gets larger (see values for 50MB and up), the smallest block size is not the most efficient (presumably due, at least in part, to the increased number of blocks, increased number of individual transfer requests, and reassembly/committal costs). Once you pass the 250MB source file size, the difference in rate for 1MB to 4MB blocks is more-or-less constant The 1MB block size gives the best average improvement (~16x) but the optimal approach would be to vary the block size based on the size of the source file.    (click chart for full size image) The above is another view of the same data as the prior chart just with the axis changed (x-axis represents file size and plotted data shows improvement by block size). It again highlights the fact that the 1MB block size is probably the best overall size but highlights the benefits of some of the other block sizes at different source file sizes. This last chart shows the change in total duration of the file uploads based on different block sizes for the source file sizes. Nothing really new here other than this view of the data highlights the negative affects of poorly choosing a block size for smaller files.   Summary What we have found so far is that blocking your file uploads and uploading them in parallel results in significant performance improvements. Further, utilizing extension methods and the Task Parallel Library (.NET 4.0) make short work of altering the shipping client library to provide this functionality while minimizing the amount of change to existing applications that might be using the client library for other interactions.   Related Resources Source code for upload test application Source code for random file generator ODatas feed of raw data from non-optimized transfer tests Experiment Metadata Experiment Datasets 2KB Uploads 32KB Uploads 64KB Uploads 128KB Uploads 256KB Uploads 512KB Uploads 1MB Uploads 5MB Uploads 10MB Uploads 25MB Uploads 50MB Uploads 100MB Uploads 250MB Uploads 500MB Uploads 750MB Uploads 1GB Uploads Raw Data OData feeds of raw data from blocked/parallelized transfer tests Experiment Metadata Experiment Datasets Raw Data 256KB Blocks 512KB Blocks 1MB Blocks 2MB Blocks 4MB Blocks Excel worksheet showing summarizations and comparisons

    Read the article

  • Is Google a reliable document search engine?

    - by Miriam Schwab
    I have a site with PDFs and Word documents that I know have been indexed by Google because they appear in search results with filetype:pdf (or doc), and if I search for some very specific terms with quotation marks, they appear as well. But they don't appear for general search terms that do exist in the documents. Is Google a reliable document search engine? If not, are there other options for managing many documents and making them searchable to users?

    Read the article

< Previous Page | 301 302 303 304 305 306 307 308 309 310 311 312  | Next Page >