Search Results

Search found 44 results on 2 pages for 'chunking'.

Page 1/2 | 1 2  | Next Page >

• WCF chunking/streaming - make it transparent for client

  - by bybor

  While developing WCF service i've faced problem of transferring large data as method params ( 4 Mb of raw size, not considering transfer/message overhead). The solution for this problem is to use chunking or streaming, but all the samples i've seen assume client is aware of used method and uses available block size for sending/receiving portions of data, and the problem (for me) is that it's not possible to call just one method, like SaveData(DataInformation info) but write wrapper method which will instead iterate smth like SaveDataChunk(byte[] buffer) Could it be somehow made transparent for client, just calling 'SaveData'?

  Read the article

• Chunking a List - .NET vs Python

  - by Abhijeet Patel

  Chunking a List As I mentioned last time, I'm knee deep in python these days. I come from a statically typed background so it's definitely a mental adjustment. List comprehensions is BIG in Python and having worked with a few of them I can see why. Let's say we need to chunk a list into sublists of a specified size. Here is how we'd do it in C#  static class Extensions   {       public static IEnumerable<List<T>> Chunk<T>(this List<T> l, int chunkSize)       {           if (chunkSize <0)           {               throw new ArgumentException("chunkSize cannot be negative", "chunkSize");           }           for (int i = 0; i < l.Count; i += chunkSize)           {               yield return new List<T>(l.Skip(i).Take(chunkSize));           }       }    }    static void Main(string[] args)  {           var l = new List<string> { "a", "b", "c", "d", "e", "f","g" };             foreach (var list in l.Chunk(7))           {               string str = list.Aggregate((s1, s2) => s1 + "," + s2);               Console.WriteLine(str);           }   }   A little wordy but still pretty concise thanks to LINQ.We skip the iteration number plus chunkSize elements and yield out a new List of chunkSize elements on each iteration. The python implementation is a bit more terse. def chunkIterable(iter, chunkSize):      '''Chunks an iterable         object into a list of the specified chunkSize     '''        assert hasattr(iter, "__iter__"), "iter is not an iterable"      for i in xrange(0, len(iter), chunkSize):          yield iter[i:i + chunkSize]    if __name__ == '__main__':      l = ['a', 'b', 'c', 'd', 'e', 'f']      generator = chunkIterable(l,2)      try:          while(1):              print generator.next()      except StopIteration:          pass   xrange generates elements in the specified range taking in a seed and returning a generator. which can be used in a for loop(much like using a C# iterator in a foreach loop) Since chunkIterable has a yield statement, it turns this method into a generator as well. iter[i:i + chunkSize] essentially slices the list based on the current iteration index and chunksize and creates a new list that we yield out to the caller one at a time. A generator much like an iterator is a state machine and each subsequent call to it remembers the state at which the last call left off and resumes execution from that point. The caveat to keep in mind is that since variables are not explicitly typed we need to ensure that the object passed in is iterable using hasattr(iter, "__iter__").This way we can perform chunking on any object which is an "iterable", very similar to accepting an IEnumerable in the .NET land

  Read the article

• JSONP & jQuery Chunking

  - by Tom

  Hi Guys, I am wanting to utilize JSONP for a project with x-domain scripting but little concerned with the 2048 character limit in IE. Does JSONP support "chunking" automatically if character size exceeds 2048 ? If yes, does anyone have any examples they can share ? Thx

  Read the article

• Prevent Apache from chunking gzipped content

  - by Bear

  When using mod_deflate in Apache2, Apache will chunk gzipped content, setting the Transfer-encoding: chunked header. While this results in a faster download time, I cannot display a progress bar. If I handle the compression myself in PHP, I can gzip it completely first and set the Content-length header, so that I can display a progress bar to the user. Is there any way to change Apache's behavior, and have Apache set a Content-length header instead of chunking the response, so that I don't have to handle the compression myself?

  Read the article

• Chunking large rsync transfers?

  - by Gabe Martin-Dempesy

  We use rsync to update a mirror of our primary file server to an off-site colocated backup server. One of the issues we currently have is that our file server has 1TB of mostly smaller files (in the 10-100kb range), and when we're transferring this much data, we often end up with the connection being dropped several hours into the transfer. Rsync doesn't have a resume/retry feature that simply reconnects to the server to pickup where it left off -- you need to go through the file comparison process, which ends up being very length with the amount of files we have. The solution that's recommended to get around is to split up your large rsync transfer into a series of smaller transfers. I've figured the best way to do this is by first letter of the top-level directory names, which doesn't give us a perfectly even distribution, but is good enough. I'd like to confirm if my methodology for doing this is sane, or if there's a more simple way to accomplish the goal. To do this, I iterate through A-Z, a-z, 0-9 to pick a one character $prefix. Initially I was thinking of just running rsync -av --delete --delete-excluded --exclude "*.mp3" "src/$prefix*" dest/ (--exclude "*.mp3" is just an example, as we have a more lengthy exclude list for removing things like temporary files) The problem with this is that any top-level directories in dest/ that are no longer present present on src will not get picked up by --delete. To get around this, I'm instead trying the following: rsync \ --filter 'S /$prefix*' \ --filter 'R /$prefix*' \ --filter 'H /*' \ --filter 'P /*' \ -av --delete --delete-excluded --exclude "*.mp3" src/ dest/ I'm using the show and hide over include and exclude, because otherwise the --delete-excluded will delete anything that doesn't match $prefix. Is this the most effective way of splitting the rsync into smaller chunks? Is there a more effective tool, or a flag that I've missed, that might make this more simple?

  Read the article

• How to best transfer large payloads of data using wsHttp with WCF with message security

  - by jpierson

  I have a case where I need to transfer large amounts of serialized object graphs (via NetDataContractSerializer) using WCF using wsHttp. I'm using message security and would like to continue to do so. Using this setup I would like to transfer serialized object graph which can sometimes approach around 300MB or so but when I try to do so I've started seeing a exception of type System.InsufficientMemoryException appear. After a little research it appears that by default in WCF that a result to a service call is contained within a single message by default which contains the serialized data and this data is buffered by default on the server until the whole message is completely written. Thus the memory exception is being caused by the fact that the server is running out of memory resources that it is allowed to allocate because that buffer is full. The two main recommendations that I've come across are to use streaming or chunking to solve this problem however it is not clear to me what that involves and whether either solution is possible with my current setup (wsHttp/NetDataContractSerializer/Message Security). So far I understand that to use streaming message security would not work because message encryption and decryption need to work on the whole set of data and not a partial message. Chunking however sounds like it might be possible however it is not clear to me how it would be done with the other constraints that I've listed. If anybody could offer some guidance on what solutions are available and how to go about implementing it I would greatly appreciate it. Related resources: Chunking Channel How to: Enable Streaming Large attachments over WCF Custom Message Encoder Another spotting of InsufficientMemoryException I'm also interested in any type of compression that could be done on this data but it looks like I would probably be best off doing this at the transport level once I can transition into .NET 4.0 so that the client will automatically support the gzip headers if I understand this properly.

  Read the article

• String chunking algorithm with natural language context

  - by Chris Ballance

  I have a arbitrarily large string of text from the user that needs to be split into 10k chunks (potentially adjustable value) and sent off to another system for processing. Chunks cannot be longer than 10k (or other arbitrary value) Text should be broken with natural language context in mind split on punctuation when possible split on spaces if no punction exists break a word as a last resort I'm trying not to re-invent the wheel with this, any suggestions before I roll this from scratch? Using C#.

  Read the article

• Upload 1GB files using chunking in PHP

  - by rjha94

  I have a web application that accepts file uploads of up to 4 MB. The server side script is PHP and web server is NGINX. Many users have requested to increase this limit drastically to allow upload of video etc. However there seems to be no easy solution for this problem with PHP. First, on the client side I am looking for something that would allow me to chunk files during transfer. SWFUpload does not seem to do that. I guess I can stream uploads using Java FX (http://blogs.sun.com/rakeshmenonp/entry/javafx_upload_file ) but I can not find any equivalent of request.getInputStream in PHP. Increasing browser client_post limits or php.ini upload or max_execution times is not really a solution for really large files (~ 1GB) because maybe the browser will time out and think of all those blobs stored in memory. is there any way to solve this problem using PHP on server side? I would appreciate your replies.

  Read the article

• Does the chunk of the System.Collections.Concurrent.Partitioner need to be thread safe?

  - by Scott Chamberlain

  I am working with the Parallel libraries in .net 4 and I am creating a Partitioner and the example shown in the MSDN only has a chunk size of 1 (every time a new result is retrieved it hits the data source instead of the local cache. The version I am writing will pull 10000 SQL rows at a time then feed the rows from the cache until it is empty then pull another batch. Each partition in the Partitioner has its own chunk. I know every time I call to the IEnumerator in from the SQL data-source that needs to be thread safe but for use in a Parallel.ForEach do I need to make every call to the cache for the chunking thread safe?

  Read the article

• cancel stream request from WCF server to client

  - by ArsenMkrt

  Hi, I posted about stream request here [wcf-chunk-data-with-stream]:http://stackoverflow.com/questions/853448/wcf-chunk-data-with-stream I solved that task but now when i close request in client part server continue to send data. is it possible to cancel stream request from WCF server to client?

  Read the article

• How to send EML data in chuck to Google Apps Mail using Google API ver 2 ?

  - by Preeti

  Hi, I am migrating EML mails to Google Apps. When i try to Migrate a EML file with two attachment 2.1 MB and 1.96 MB. It is throwing exception: "The request was aborted: The request was canceled." I am using below code: MailItemEntry[] entries = new MailItemEntry[1]; String msg = File.ReadAllText(EmlPath); entries[0] = new MailItemEntry(); entries[0].Rfc822Msg = new Rfc822MsgElement(msg); ........ MailItemFeed feed = mailItemService.Batch(domain, UserName, entries); I think sending data can resolve this issue.So,how can send this EML data in chunk to Google Apps? Thanx

  Read the article

• Is there a max recommended size on bundling js/css files due to chunking or packet loss?

  - by George Mauer

  So we all have heard that its good to bundle your javascript. Of course it is, but it seems to me that the story is too simple. See if my logic makes sense here. Obviously fewer HTTP requests is fewer round trips and hence better. However - and I don't know much about bare http - aren't http responses sent in chunks? And if a file is larger than one of those chunks doesn't it have to be downloaded as multiple (possibly synchronous?) round trips? As opposed to this, several requests for files just under the chunking size would arrive much quicker since modern web browsers download resources like javascripts in parallel. Even if chunking is not an issue, it seems like there would be some max recommended size just due to likelyhood of packet loss alone since a bundled file must wait till it is entirely downloaded to execute, versus the more lenient native rule that scripts must execute in order. Obviously there's also matters of browser caching and code volatility to consider but can someone confirm this or explain why I'm off base? Does anyone have any numbers to put to it?

  Read the article

• Sending Big Files with WCF

  - by Sean Feldman

  I had to look into a project that submits large files to WCF service. Implementation is based on data chunking. This is a good approach when your client and server are not both based on WCF, bud different technologies. The problem with something like this is that chunking (either you wish it or not) complicates the overall solution. Alternative would be streaming. In WCF to WCF scenario, this is a piece of cake. When client is Java, it becomes a bit more challenging (has anyone implemented Java client streaming data to WCF service?). What I really liked about .NET implementation with WCF, is that sending header info along with stream was dead simple, and from the developer point of view looked like it’s all a part of the DTO passed into the service. [ServiceContract] public interface IFileUpload { [OperationContract] void UploadFile(SendFileMessage message); } Where SendFileMessage is [MessageContract] public class SendFileMessage { [MessageBodyMember(Order = 1)] public Stream FileData; [MessageHeader(MustUnderstand = true)] public FileTransferInfo FileTransferInfo; }

  Read the article

• Exchange 2007 issue internet receive connector

  - by user223779

  I have issue with yahoo.co.uk if I send a mail from within the yahoo webconsole the mail arrives in my inbox on the exchange server If I send mail from Iphone configure to send via mail box configure with yahoo setting mail is dropped. It is not the phone I can send perfectly fine to other exchange 2007 servers same service pack etc. if you look at the smtprec log below. this message sent from the phone you can see stops after 354 Start mail input; end with . ,<,EHLO nm26-vm7.bullet.mail.ir2.yahoo.com, ,,250-mail.marcocm.com Hello [212.82.97.49], ,,250-SIZE 10485760, ,,250-PIPELINING, ,,250-DSN, ,,250-ENHANCEDSTATUSCODES, ,,250-AUTH, ,,250-8BITMIME, ,,250-BINARYMIME, ,,250 CHUNKING, ,<,MAIL FROM:, ,*,08D13F3CADECA060;2014-06-04T11:26:50.898Z;1,receiving message ,,250 2.1.0 Sender OK, ,<,RCPT TO:, ,,250 2.1.5 Recipient OK, ,<,DATA, ,,354 Start mail input; end with ., ,+,, This is the message hitting the same server sent from yahoo webmail. ,"220 mail.marcocm.com Microsoft ESMTP MAIL Service ready at Wed, 4 Jun 2014 12:29:26 +0100", ,<,EHLO nm4-vm6.bullet.mail.ir2.yahoo.com, ,,250-mail.xxx.com Hello [212.82.96.104], ,,250-SIZE 10485760, ,,250-PIPELINING, ,,250-DSN, ,,250-ENHANCEDSTATUSCODES, ,,250-AUTH, ,,250-8BITMIME, ,,250-BINARYMIME, ,,250 CHUNKING, ,<,MAIL FROM:, ,*,08D13F3CADECA06B;2014-06-04T11:29:26.237Z;1,receiving message ,,250 2.1.0 Sender OK, ,<,RCPT TO:, ,,250 2.1.5 Recipient OK, ,<,DATA, ,,354 Start mail input; end with ., 2,,250 2.6.0 <[email protected] Queued mail for delivery, <,QUIT, ,,221 2.0.0 Service closing transmission channel, ,-,,Local ,+,, Any Thoughts how to fix this issue much appreciated.

  Read the article

• What can Go chan do that a list cannot?

  - by alpav

  I want to know in which situation Go chan makes code much simpler than using list or queue or array that is usually available in all languages. As it was stated by Rob Pike in one of his speeches about Go lexer, Go channels help to organize data flow between structures that are not homomorphic. I am interested in a simple Go code sample with chan that becomes MUCH more complicated in another language (for example C#) where chan is not available. I am not interested in samples that use chan just to increase performance by avoiding waiting of data between generating list and consuming the list (which can be solved by chunking) or as a way to organize thread safe queue or thread-safe communication (which can be easily solved by locking primitives). I am interested in a sample that makes code simpler structurally disregarding size of data. If such sample does not exist then sample where size of data matters. I guess desired sample would contain bi-directional communication between generator and consumer. Also if someone could add tag [channel] to the list of available tags, that would be great.

  Read the article

• Which useful alternative control structures do you know?

  - by bigown

  Similar question was closed on SO. Sometimes when we're programming, we find that some particular control structure would be very useful to us, but is not directly available in our programming language. What alternative control structures do you think are a useful way of organizing computation? The goal here is to get new ways of thinking about structuring code, in order to improve chunking and reasoning. You can create a wishful syntax/semantic not available now or cite a less known control structure on an existent programming language. Answers should give ideas for a new programming language or enhancing an actual language. Think of this as brainstorming, so post something you think is a crazy idea but it can be viable in some scenario. It's about imperative programming.

  Read the article

• Introduction to JBatch

  - by reza_rahman

  It seems batch processing is moving more and more into the realm of the Java developer. In recognition of this fact, JBatch (aka Java Batch, JSR 352, Batch Applications for the Java Platform) was added to Java EE 7. In a recent article JBatch specification lead Chris Vignola of IBM provides a high level overview of the API. He discusses the core concepts/motivation, the Job Specification Language, the reader-processor-writer pattern, job operator, job repository, chunking, packaging, partitions, split/flow and the like. You can also check out the official specification yourself or try things out with the newly released Java EE 7 SDK.

  Read the article

• WCF ReliableSession and Timeouts

  - by user80108

  I have a WCF service used mainly for managing documents in a repository. I used the chunking channel sample from MS so that I could upload/download huge files. Now I implemented reliable session with the service and I am seeing some strange behaviors. Here are the timeout values I am using. this.SendTimeout = new TimeSpan(0,10,0); this.OpenTimeout = new TimeSpan(0, 1, 0); this.CloseTimeout = new TimeSpan(0, 1, 0); this.ReceiveTimeout = new TimeSpan(0,10, 0); reliableBe.InactivityTimeout = new TimeSpan(0,2,0); I have the following issues. 1. If the Service is not up & running, the clients are not get disconnected after OpenTimeout. I tried it with my test client. Scenario 1: Without Reliable Session: I get the following exception: Could not connect to net.tcp://localhost:8788/MediaManagementService/ep1. The connection attempt lasted for a time span of 00:00:00.9848790. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:8788 This is the correct behavior as I have given the OpenTimeout as 1 sec. Scenario 2: With ReliableSession: I get the same exception: Could not connect to net.tcp://localhost:8788/MediaManagementService/ep1. The connection attempt lasted for a time span of 00:00:00.9692460. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:8788. But this message comes after around 10 mintes . (I believe after SendTimeout) So here I just have enabled the reliable session and now it looks like the OpenTimeout = SendTimeout for the client. Is this desired behavior? 2: Issue while uploading huge files with ReliableSession: The general rule is that you have to set a huge value for the maxReceivedMessageSize, SendTimeout and ReceiveTimeout. But in the case of Chunking channel, the max received message size doesn't matter as the data is sent in chunks. So I set a huge value for Send and ReceiveTimeout : say 10 hours. Now the upload is going fine, but it has a side effect that, even if the Service is not up, it takes 10 hours to timeout the client connection due to the behavior mentioned in (1). Please let me know your thoughts on this behavior.

  Read the article

• Silverlight 4 RC File Upload with Upload Progress: how to?

  - by steve

  It's been stated that one of the new features of Silverlight 4 RC is that it now supports upload progress. I'm assuming this means it's possible to make an upload file progress bar without "chunking" but I can't figure out how to do this, so how do we do this? Source code examples would be great. Thanks!

  Read the article

• What's a clean way to break up a DataTable into chunks of a fixed size with Linq?

  - by Michael Haren

  Update: Here's a similar question Suppose I have a DataTable with a few thousand DataRows in it. I'd like to break up the table into chunks of smaller rows for processing. I thought C#3's improved ability to work with data might help. This is the skeleton I have so far: DataTable Table = GetTonsOfData(); // Chunks should be any IEnumerable<Chunk> type var Chunks = ChunkifyTableIntoSmallerChunksSomehow; // ** help here! ** foreach(var Chunk in Chunks) { // Chunk should be any IEnumerable<DataRow> type ProcessChunk(Chunk); } Any suggestions on what should replace ChunkifyTableIntoSmallerChunksSomehow? I'm really interested in how someone would do this with access C#3 tools. If attempting to apply these tools is inappropriate, please explain! Update 3 (revised chunking as I really want tables, not ienumerables; going with an extension method--thanks Jacob): Final implementation: Extension method to handle the chunking: public static class HarenExtensions { public static IEnumerable<DataTable> Chunkify(this DataTable table, int chunkSize) { for (int i = 0; i < table.Rows.Count; i += chunkSize) { DataTable Chunk = table.Clone(); foreach (DataRow Row in table.Select().Skip(i).Take(chunkSize)) { Chunk.ImportRow(Row); } yield return Chunk; } } } Example consumer of that extension method, with sample output from an ad hoc test: class Program { static void Main(string[] args) { DataTable Table = GetTonsOfData(); foreach (DataTable Chunk in Table.Chunkify(100)) { Console.WriteLine("{0} - {1}", Chunk.Rows[0][0], Chunk.Rows[Chunk.Rows.Count - 1][0]); } Console.ReadLine(); } static DataTable GetTonsOfData() { DataTable Table = new DataTable(); Table.Columns.Add(new DataColumn()); for (int i = 0; i < 1000; i++) { DataRow Row = Table.NewRow(); Row[0] = i; Table.Rows.Add(Row); } return Table; } }

  Read the article

• Robust Large File Transfer with WCF

  - by Sharov

  I want to transfer big files (1GB) over unreliable transport channels. When connection is interrupted, I don't want start file transfering from the begining. I can partially store it in a temp table and store last readed position, so when connection is reestablished I can request continue uploading of file from this position. Is there any best-practice for such kind of things. I'm currently use chunking channel.

  Read the article

• Email is stuck in the queue with 421 4.2.2 Connection dropped due to SocketError

  - by e0594cn

  We recently installed an Exchange 2010 Server and we are having some problems sending emails to certain domains. Email is stuck in the queue with 421 4.2.2 Connection dropped due to SocketError. Any Suggestion? The below is the message when using telnet command: EHLO etla.com.cn 250-aa6061.com Hello [58.215.221.50] 250-TURN 250-SIZE 15360000 250-ETRN 250-PIPELINING 250-DSN 250-ENHANCEDSTATUSCODES 250-8bitmime 250-BINARYMIME 250-CHUNKING 250-VRFY 250-X-EXPS GSSAPI NTLM LOGIN 250-X-EXPS=LOGIN 250-AUTH GSSAPI NTLM LOGIN 250-AUTH=LOGIN 250-X-LINK2STATE 250-XEXCH50 250 OK MAIL FROM:[email protected] 250 2.1.0 [email protected] OK RCPT TO:[email protected] NOTIFY=success,failure **550 5.7.1 Your email messages have been blocked by the recipient OR by Trend Mic ro Email Reputation Service. Contact the recipient or his/her administrator usin g alternate means to resolve the issue.**

  Read the article

• April 2013 Release of the Ajax Control Toolkit

  - by Stephen.Walther

  I’m excited to announce the April 2013 release of the Ajax Control Toolkit. For this release, we focused on improving two controls: the AjaxFileUpload and the MaskedEdit controls. You can download the latest release from CodePlex at http://AjaxControlToolkit.CodePlex.com or, better yet, you can execute the following NuGet command within Visual Studio 2010/2012: There are three builds of the Ajax Control Toolkit: .NET 3.5, .NET 4.0, and .NET 4.5. A Better AjaxFileUpload Control We completely rewrote the AjaxFileUpload control for this release. We had two primary goals. First, we wanted to support uploading really large files. In particular, we wanted to support uploading multi-gigabyte files such as video files or application files. Second, we wanted to support showing upload progress on as many browsers as possible. The previous version of the AjaxFileUpload could show upload progress when used with Google Chrome or Mozilla Firefox but not when used with Apple Safari or Microsoft Internet Explorer. The new version of the AjaxFileUpload control shows upload progress when used with any browser. Using the AjaxFileUpload Control Let me walk-through using the AjaxFileUpload in the most basic scenario. And then, in following sections, I can explain some of its more advanced features. Here’s how you can declare the AjaxFileUpload control in a page: <ajaxToolkit:ToolkitScriptManager runat="server" /> <ajaxToolkit:AjaxFileUpload ID="AjaxFileUpload1" AllowedFileTypes="mp4" OnUploadComplete="AjaxFileUpload1_UploadComplete" runat="server" /> The exact appearance of the AjaxFileUpload control depends on the features that a browser supports. In the case of Google Chrome, which supports drag-and-drop upload, here’s what the AjaxFileUpload looks like: Notice that the page above includes two Ajax Control Toolkit controls: the AjaxFileUpload and the ToolkitScriptManager control. You always need to include the ToolkitScriptManager with any page which uses Ajax Control Toolkit controls. The AjaxFileUpload control declared in the page above includes an event handler for its UploadComplete event. This event handler is declared in the code-behind page like this: protected void AjaxFileUpload1_UploadComplete(object sender, AjaxControlToolkit.AjaxFileUploadEventArgs e) { // Save uploaded file to App_Data folder AjaxFileUpload1.SaveAs(MapPath("~/App_Data/" + e.FileName)); } This method saves the uploaded file to your website’s App_Data folder. I’m assuming that you have an App_Data folder in your project – if you don’t have one then you need to create one or you will get an error. There is one more thing that you must do in order to get the AjaxFileUpload control to work. The AjaxFileUpload control relies on an HTTP Handler named AjaxFileUploadHandler.axd. You need to declare this handler in your application’s root web.config file like this: <configuration> <system.web> <compilation debug="true" targetFramework="4.5" /> <httpRuntime targetFramework="4.5" maxRequestLength="42949672" /> <httpHandlers> <add verb="*" path="AjaxFileUploadHandler.axd" type="AjaxControlToolkit.AjaxFileUploadHandler, AjaxControlToolkit"/> </httpHandlers> </system.web> <system.webServer> <validation validateIntegratedModeConfiguration="false"/> <handlers> <add name="AjaxFileUploadHandler" verb="*" path="AjaxFileUploadHandler.axd" type="AjaxControlToolkit.AjaxFileUploadHandler, AjaxControlToolkit"/> </handlers> <security> <requestFiltering> <requestLimits maxAllowedContentLength="4294967295"/> </requestFiltering> </security> </system.webServer> </configuration> Notice that the web.config file above also contains configuration settings for the maxRequestLength and maxAllowedContentLength. You need to assign large values to these configuration settings — as I did in the web.config file above — in order to accept large file uploads. Supporting Chunked File Uploads Because one of our primary goals with this release was support for large file uploads, we added support for client-side chunking. When you upload a file using a browser which fully supports the HTML5 File API — such as Google Chrome or Mozilla Firefox — then the file is uploaded in multiple chunks. You can see chunking in action by opening F12 Developer Tools in your browser and observing the Network tab: Notice that there is a crazy number of distinct post requests made (about 360 distinct requests for a 1 gigabyte file). Each post request looks like this: http://localhost:24338/AjaxFileUploadHandler.axd?contextKey={DA8BEDC8-B952-4d5d-8CC2-59FE922E2923}&fileId=B7CCE31C-6AB1-BB28-2940-49E0C9B81C64 &fileName=Sita_Sings_the_Blues_480p_2150kbps.mp4&chunked=true&firstChunk=false Each request posts another chunk of the file being uploaded. Notice that the request URL includes a chunked=true parameter which indicates that the browser is breaking the file being uploaded into multiple chunks. Showing Upload Progress on All Browsers The previous version of the AjaxFileUpload control could display upload progress only in the case of browsers which fully support the HTML5 File API. The new version of the AjaxFileUpload control can display upload progress in the case of all browsers. If a browser does not fully support the HTML5 File API then the browser polls the server every few seconds with an Ajax request to determine the percentage of the file that has been uploaded. This technique of displaying progress works with any browser which supports making Ajax requests. There is one catch. Be warned that this new feature only works with the .NET 4.0 and .NET 4.5 versions of the AjaxControlToolkit. To show upload progress, we are taking advantage of the new ASP.NET HttpRequest.GetBufferedInputStream() and HttpRequest.GetBufferlessInputStream() methods which are not supported by .NET 3.5. For example, here is what the Network tab looks like when you use the AjaxFileUpload with Microsoft Internet Explorer: Here’s what the requests in the Network tab look like: GET /WebForm1.aspx?contextKey={DA8BEDC8-B952-4d5d-8CC2-59FE922E2923}&poll=1&guid=9206FF94-76F9-B197-D1BC-EA9AD282806B HTTP/1.1 Notice that each request includes a poll=1 parameter. This parameter indicates that this is a polling request to get the size of the file buffered on the server. Here’s what the response body of a request looks like when about 20% of a file has been uploaded: Buffering to a Temporary File When you upload a file using the AjaxFileUpload control, the file upload is buffered to a temporary file located at Path.GetTempPath(). When you call the SaveAs() method, as we did in the sample page above, the temporary file is copied to a new file and then the temporary file is deleted. If you don’t call the SaveAs() method, then you must ensure that the temporary file gets deleted yourself. For example, if you want to save the file to a database then you will never call the SaveAs() method and you are responsible for deleting the file. The easiest way to delete the temporary file is to call the AjaxFileUploadEventArgs.DeleteTemporaryData() method in the UploadComplete handler: protected void AjaxFileUpload1_UploadComplete(object sender, AjaxControlToolkit.AjaxFileUploadEventArgs e) { // Save uploaded file to a database table e.DeleteTemporaryData(); } You also can call the static AjaxFileUpload.CleanAllTemporaryData() method to delete all temporary data and not only the temporary data related to the current file upload. For example, you might want to call this method on application start to ensure that all temporary data is removed whenever your application restarts. A Better MaskedEdit Extender This release of the Ajax Control Toolkit contains bug fixes for the top-voted issues related to the MaskedEdit control. We closed over 25 MaskedEdit issues. Here is a complete list of the issues addressed with this release: · 17302 MaskedEditExtender MaskType=Date, Mask=99/99/99 Undefined JS Error · 11758 MaskedEdit causes error in JScript when working with 2-digits year · 18810 Maskededitextender/validator Date validation issue · 23236 MaskEditValidator does not work with date input using format dd/mm/yyyy · 23042 Webkit based browsers (Safari, Chrome) and MaskedEditExtender · 26685 MaskedEditExtender@(ClearMaskOnLostFocus=false) adds a zero character when you each focused to target textbox · 16109 MaskedEditExtender: Negative amount, followed by decimal, sets value to positive · 11522 MaskEditExtender of AjaxtoolKit-1.0.10618.0 does not work properly for Hungarian Culture · 25988 MaskedEditExtender – CultureName (HU-hu) > DateSeparator · 23221 MaskedEditExtender date separator problem · 15233 Day and month swap in Dynamic user control · 15492 MaskedEditExtender with ClearMaskOnLostFocus and with MaskedEditValidator with ClientValidationFunction · 9389 MaskedEditValidator – when on no entry · 11392 MaskedEdit Number format messed up · 11819 MaskedEditExtender erases all values beyond first comma separtor · 13423 MaskedEdit(Extender/Validator) combo problem · 16111 MaskedEditValidator cannot validate date with DayMonthYear in UserDateFormat of MaskedEditExtender · 10901 MaskedEdit: The months and date fields swap values when you hit submit if UserDateFormat is set. · 15190 MaskedEditValidator can’t make use of MaskedEditExtender’s UserDateFormat property · 13898 MaskedEdit Extender with custom date type mask gives javascript error · 14692 MaskedEdit error in “yy/MM/dd” format. · 16186 MaskedEditExtender does not handle century properly in a date mask · 26456 MaskedEditBehavior. ConvFmtTime : function(input,loadFirst) fails if this._CultureAMPMPlaceholder == “” · 21474 Error on MaskedEditExtender working with number format · 23023 MaskedEditExtender’s ClearMaskOnLostFocus property causes problems for MaskedEditValidator when set to false · 13656 MaskedEditValidator Min/Max Date value issue Conclusion This latest release of the Ajax Control Toolkit required many hours of work by a team of talented developers. I want to thank the members of the Superexpert team for the long hours which they put into this release.

  Read the article

• Backing up SQL Azure

  - by Herve Roggero

  That's it!!! After many days and nights... and an amazing set of challenges, I just released the Enzo Backup for SQL Azure BETA product (http://www.bluesyntax.net). Clearly, that was one of the most challenging projects I have done so far. Why??? Because to create a highly redundant system, expecting failures at all times for an operation that could take anywhere from a couple of minutes to a couple of hours, and still making sure that the operation completes at some point was remarkably challenging. Some routines have more error trapping that actual code... Here are a few things I had to take into account: Exponential Backoff (explained in another post) Dual dynamic determination of number of rows to backup  Dynamic reduction of batch rows used to restore the data Implementation of a flexible BULK Insert API that the tool could use Implementation of a custom Storage REST API to handle automatic retries Automatic data chunking based on blob sizes Compression of data Implementation of the Task Parallel Library at multiple levels including deserialization of Azure Table rows and backup/restore operations Full or Partial Restore operations Implementation of a Ghost class to serialize/deserialize data tables And that's just a partial list... I will explain what some of those mean in future blob posts. A lot of the complexities had to do with implementing a form of retry logic, depending on the resource and the operation.

  Read the article

• very large string in memory

  - by bushman

  Hi, I am writing a program for formatting 100s of MB String data (nearing a gig) into xml == And I am required to return it as a response to an HTTP (GET) request . I am using a StringWriter/XmlWriter to build an XML of the records in a loop and returning the stringWriter.ToString() during testing I saw a few --out of memory exceptions-- and quite clueless on how to find a solution? do you guys have any suggestions for a memory optimized delivery of the response? is there a memory efficient way of encoding the data? or maybe chunking the data -- I just can not think of how to return it without building the whole thing into one HUGE string object thanks

  Read the article

1 2  | Next Page >