Search Results

Search found 87971 results on 3519 pages for 'gzip open file'.

Page 65/3519 | < Previous Page | 61 62 63 64 65 66 67 68 69 70 71 72  | Next Page >

• PHP File Serving Script: Unreliable Downloads?

  - by JGB146

  This post started as a question on ServerFault ( http://serverfault.com/questions/131156/user-receiving-partial-downloads ) but I determined that our php script was the culprit. So I'm issuing an updated question here about what I believe is the actual issue. I am using a php script to verify permissions and then serve up a file for users of my website to download. Most of the time, this works, but recently one user has been seeing problems with larger downloads. He is only getting ~80% of downloads for files that are 100MB in size. Also, all downloads from this script fail to report a filesize. Further, tests revealed that the same user COULD reliably download each of the failed files if given a direct link (at which point the filesize is reported). Here's the relevant snippet of code that we are using to serve the file: header("Content-type:$contenttype"); $len = filesize($filename); header("Content-Length: $len"); header("Content-Disposition: attachment; filename=".$title.".".$ext); readfile($filename); Note that $contenttype, $filename, $title, and $ext are all set correctly before we get here. These have been triple-checked. None of them are the problem. Also, $len does provide the correct filesize. While researching this issue, I came across this post: http://stackoverflow.com/questions/1334471/content-length-header-always-zero It seems that I am encountering the same issue. When I use the script, I get chunked encoding on the file and no size is set for content-length. I'm hypothesizing that something is going wrong on the large downloads, leading him to get a zero-length chunk before the end of the file. Here's what the headers look like for a direct request: http://www.grinderschool.com/videos/zfff5061b65ae00e8b21/KillsAids021.wmv GET /videos/zfff5061b65ae00e8b21/KillsAids021.wmv HTTP/1.1 Host: www.grinderschool.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Referer: http://www.grinderschool.com/phpBB3/viewtopic.php?f=14&p=29468 Cookie: style_cookie=printonly; phpbb3_7c544_u=2; phpbb3_7c544_k=44b832912e5f887d; phpbb3_7c544_sid=e8852df42e08cc1b2250300c2897f78f; __utma=174624884.2719561324781918700.1251850714.1270986325.1270989003.575; __utmz=174624884.1264524375.411.12.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=low%20stakes%20poker%20videos; phpbb3_cmviy_k=; phpbb3_cmviy_u=2; phpbb3_cmviy_sid=d8df5c0943863004ca40ef9c392d371d; __utmb=174624884.4.10.1270989003; __utmc=174624884 Pragma: no-cache Cache-Control: no-cache HTTP/1.1 200 OK Date: Sun, 11 Apr 2010 12:57:41 GMT Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8l DAV/2 mod_auth_passthrough/2.1 FrontPage/5.0.2.2635 Last-Modified: Sun, 04 Apr 2010 12:51:06 GMT Etag: "eb42d6-7d9b843-48368aa6dc280" Accept-Ranges: bytes Content-Length: 131708995 Keep-Alive: timeout=10, max=30 Connection: Keep-Alive Content-Type: video/x-ms-wmv And here's what they look like for the request answered by my script: http://www.grinderschool.com/download_video_test.php?t=KillsAids021&format=wmv GET /download_video_test.php?t=KillsAids021&format=wmv HTTP/1.1 Host: www.grinderschool.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Cookie: style_cookie=printonly; phpbb3_7c544_u=2; phpbb3_7c544_k=44b832912e5f887d; phpbb3_7c544_sid=e8852df42e08cc1b2250300c2897f78f; __utma=174624884.2719561324781918700.1251850714.1270986325.1270989003.575; __utmz=174624884.1264524375.411.12.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=low%20stakes%20poker%20videos; phpbb3_cmviy_k=; phpbb3_cmviy_u=2; phpbb3_cmviy_sid=d8df5c0943863004ca40ef9c392d371d; __utmb=174624884.4.10.1270989003; __utmc=174624884 HTTP/1.1 200 OK Date: Sun, 11 Apr 2010 12:58:02 GMT Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8l DAV/2 mod_auth_passthrough/2.1 FrontPage/5.0.2.2635 X-Powered-By: PHP/5.2.11 Content-Disposition: attachment; filename=KillsAids021.wmv Vary: Accept-Encoding Content-Encoding: gzip Keep-Alive: timeout=10, max=30 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: video/x-ms-wmv So the question is...what can I do to make downloads from the script work properly? Again, for 99% of users, it works as is (though I find it annoying now that no filesize is reported and thus that no time estimate can be computed about the download).

  Read the article

• How to code a batch file to copy and rename the most recently dated file?

  - by david.murtagh.keltie.com

  I'm trying to code a batch file to copy only the most recently dated file in a given folder to another directory on the local machine, and simultaneously rename it as it does. I've found a very similar question here http://stackoverflow.com/questions/97371/batch-script-to-copy-newest-file and have managed to cobble together the below code from other forums too, but have hit a brick wall as it only results in the batch file itself being copied to the destination folder. It doesn't matter to me where the batch file itself sits in order for this to run. The source folder is C:! BATCH and the destination folder is C:\DROP The code is below, apologies if this is a glaringly obvious answer but it's literally the first foray into coding batch files for me... Thanks! @echo off setLocal EnableDelayedExpansion pushd C:\! BATCH for /f "tokens=* delims= " %%G in ('dir/b/od') do (set newest=%%G) copy "!newest!" C:\DROP\ PAUSE

  Read the article

• Best way to choose a random file from a directory in a shell script

  - by jhs

  What is the best way to choose a random file from a directory in a shell script? Here is my solution in Bash but I would be very interested for a more portable (non-GNU) version for use on Unix proper. dir='some/directory' file=`/bin/ls -1 "$dir" | sort --random-sort | head -1` path=`readlink --canonicalize "$dir/$file"` # Converts to full path echo "The randomly-selected file is: $path" Anybody have any other ideas? Edit: lhunath makes a good point about parsing ls. I guess it comes down to whether you want to be portable or not. If you have the GNU findutils and coreutils then you can do: find "$dir" -maxdepth 1 -mindepth 1 -type f -print0 \ | sort --zero-terminated --random-sort \ | sed 's/\d000.*//g/' Whew, that was fun! Also it matches my question better since I said "random file". Honsetly though, these days it's hard to imagine a Unix system deployed out there having GNU installed but not Perl 5.

  Read the article

• Export GridView to TXT, then upload file to server

  Basically what I want to do is export an array (or GridView) to a file called "getpathin.dat". That is easy, but the method I am using downloads the file to my computer, which is what I don't want. I want to write an array to either a PRE-EXISTING file that is on the server OR create a new file on the server in a folder, and this new file will contain either the array or the gridview, which I have stored in the array. i'll be doing this in... Visual Studio 2008/SQL Server using C#

  Read the article

• How to use application config file in C#?

  - by badpanda

  I am trying to use a config file in my C# console application. I created the file within the project by going New -- Application Configuration File, and naming it myProjectName.config. My config file looks like this: <?xml version="1.0" encoding="utf-8" ?> <configuration> <appSettings> <add key="SSDirectory" value="D:\Documents and Settings\****\MyDocuments\****" /> </appSettings> </configuration> The code to access it looks like this: private FileValidateUtil() { sSDirFilePath = ConfigurationSettings.AppSettings["SSDirectory"]; if (sSDirFilePath == null) Console.WriteLine("config file not reading in."); } Can anyone lend a hint as to why this is not working? (I am getting the error message.) Thanks!! badPanda

  Read the article

• Need to get the uploaded file to my local PC

  - by Suhail

  Hi, I have created a test form which will ask users to enter a name and upload the image file: <html lang="en"> <head> <title>Testing image upload</title> </head> <body> <form action="/services/upload" method="POST" enctype="multipart/form-data"> File Description: <input name='fdesc' type='text'><br> File name: <input type="file" name="fname"><br> <div><input type="submit"></div> </form> </body> </html> i need to get the file uploaded by the user and store it on my local PC. can this be done in python ? please let me know.

  Read the article

• Java how to copy part of a file

  - by user3479074

  I have to read a file and depending of the content of the last lines, I have to copy most of its content into a new file. Unfortunately I didn't found a way to copy first n lines or chars of a file in java. The only way I found, is copying the file using nio FileChannels where I can specifiy the length in bytes. However, therefore I would need to know how many bytes the stuff I read needed in the source-file. Does anyone know a solution for one of these problems?

  Read the article

• Unknown symbols when I read file

  - by Sergey Gavruk

  I read file, but in the end of file i get unknown symbols: int main() { char *buffer, ch; int i = 0, size; FILE *fp = fopen("file.txt", "r"); if(!fp){ printf("File not found!\n"); exit(1); } fseek(fp, 0, SEEK_END); size = ftell(fp); printf("%d\n", size); fseek(fp, 0, SEEK_SET); buffer = malloc(size * sizeof(*buffer)); while(((ch = fgetc(fp)) != NULL) && (i <= size)){ buffer[i++] = ch; } printf(buffer); fclose(fp); free(buffer); getch(); return 0; }

  Read the article

• Invoking a function (main()) from a binary file in C

  - by Dhara Darji

  I have simple c program like, my_bin.c: #include <stdio.h> int main() { printf("Success!\n"); return 0; } I compile it with gcc and got executable: my_bin. Now I want to invoke main (or run this my_bin) using another C program. That I did with mmap and function pointer like this: #include <stdio.h> #include <fcntl.h> #include <sys/mman.h> int main() { void (*fun)(); int fd; int *map; fd = open("./my_bin", O_RDONLY); map = mmap(0, 8378, PROT_READ, MAP_SHARED, fd, 0); fun = map; fun(); return 0; } PS: I went through some tutorial, for how to read binary file and execute. But this gives Seg fault, any help appreciated! Thanks!

  Read the article

• saving file name from html form input

  - by user343934

  I am working on python and biopython right now. I have a file upload form and whatever file is uploaded suppose(abc.fasta) then i want to pass same name in execute (abc.fasta) function parameter and display function parameter (abc.aln). Workflow goes like this. ----If submit is not true then display only header and form part --- if submit is true then call execute() and get file name from form input --- Then display the save file result in the same page. File name is same as input. My raw code is here -- http://pastebin.com/FPUgZSSe Any suggestions, changes and algorithm is appreciated Thanks

  Read the article

• Reading file data during form's clean method

  - by Dominic Rodger

  So, I'm working on implementing the answer to my previous question. Here's my model: class Talk(models.Model): title = models.CharField(max_length=200) mp3 = models.FileField(upload_to = u'talks/', max_length=200) Here's my form: class TalkForm(forms.ModelForm): def clean(self): super(TalkForm, self).clean() cleaned_data = self.cleaned_data if u'mp3' in self.files: from mutagen.mp3 import MP3 if hasattr(self.files['mp3'], 'temporary_file_path'): audio = MP3(self.files['mp3'].temporary_file_path()) else: # What goes here? audio = None # setting to None for now ... return cleaned_data class Meta: model = Talk Mutagen needs file-like objects - the first case (where the uploaded file is larger than the size of file handled in memory) works fine, but I don't know how to handle InMemoryUploadedFile that I get otherwise. I've tried: # TypeError (coercing to Unicode: need string or buffer, InMemoryUploadedFile found) audio = MP3(self.files['mp3']) # TypeError (coercing to Unicode: need string or buffer, cStringIO.StringO found) audio = MP3(self.files['mp3'].file) # Hangs seemingly indefinitely audio = MP3(self.files['mp3'].file.read()) Is there something wrong with mutagen, or am I doing it wrong?

  Read the article

• Instant file sharing between users in PHP

  - by Skyfe

  Hi there, Working on a rather complex system in which users can directly exchange files with eachother from the website. However is any of these things possible: EITHER * Have another user download a file which is still being uploaded by another user ( in progress ) OR * Make a user automaticly ( instant ) download a file from another users PC through our website OR * Make a user automaticly (instant) download a file from our server ( so it's directly downloaded to the users pc and the progress shown on our website of the download progress, without the normal internet explorer dialog downloading the file or firefox ). Thank you very much in advanced, Best Regards, Webcodez.net. UPDATE: an example would be MSN's file sharing but then through a website instead of application.

  Read the article

• Read a file with 2048 bytes

  - by Suresh S

  Guys i have a file which has only one line. The file has no encoding it is a simple text file with single line. For every 2048 byte in a line , there is new record of 151 byte (totally 13*151 byte = 1945 records + 85 byte empty space). similarly for the next 2048 bytes. What is the best file i/o to use? i am thinking of reading 2048 bytes from file and storing it in an array . while (offset < fileLength &&(numRead=in.read(recordChunks, offset,alength)) >= 0) { } how can i get from the read statement only 2048 bytes at a time . i am getting IndexOutofBoundException.

  Read the article

• Delete file after sharing via intent.

  - by Matt

  I'm trying to delete a temporary file after sharing it via android's Intent.ACTION_SEND feature. Right now I am starting the activity for a result and in OnActivityResult, I am deleting the file. Unfortunately this only works if I am debugging it with a breakpoint, but when I let it run freely and say, email the file, the email has no attachment. I think what is happening is my activity is deleting the file before it had been emailed. What I don't get is why, shouldn't onActivityResult only be called AFTER the other activity is finished? I have also tried deleting the file in onResume, but no luck. Is there a better way to do this?

  Read the article

• How can I convert a specific .cs file into either a .bat or a .exe file?

  - by user2974969

  I am trying to convert around 1,900 .txp files (a proprietary Sony image format) to .png. However, the .exe program I am using to convert these files only allows me to select one file at a time. I am running Windows 7. There's a TXP.CS file in the 'src' folder of the program, so I figured that if I was able to turn that into an executable file, I'd be able to use it to mass convert these files to PNG. However, whenever I try to convert the file using csc.exe, I get the CS0246 error (the type or namespace name 'Tools' could not be found. Are you missing a using directive or an assembly reference?). I can't use Microsoft Visual Studio right now, so I'm hoping someone can walk me through this, or maybe convert the file to either a .bat or an .exe for me. Thank you. TXP.CS

  Read the article

• C: Reading file with a starting point

  - by Shinka

  A simple question but I can't find the answer in my book. I want to read a binary file to seed a random number generator, but I don't want to seed my generator with the same seed each time I call the function, so I will need to keep a variable for my position in the file (not a problem) and I would need to know how to read a file starting a specific point in the file (no idea how). The code: void rng_init(RNG* rng) { // ... FILE *input = fopen("random.bin", "rb"); unsigned int seed[32]; fread(seed, sizeof(unsigned int), 32, input); // seed 'rng'... fclose(input); }

  Read the article

• Accessing a file (for writing) from a JBoss Web Service

  - by Andreas Grech

  Let's say I have this structure of my Java Web Application: TheProject -- [Web Pages] -- -- abc.txt -- -- index.jsp -- [Source Packages] -- -- [wservices] -- -- -- WS.java WS.java is my Web Service, which is situated in a wservices package. Now from this service, I need to access the abc.txt file and write to it. These are my urls: http://127.0.0.1:8080/TheProject/WS <- the webservice http://127.0.0.1:8080/TheProject/abc.txt <- the file I want to access To read the file, I tried with getResourceAsStream and I was successful in reading from it. But now I also want to write to this file, and I tried such a method but failed. Is there a way I can get access to the abc.txt file from WS.java and be able to successfully read from and write to it?

  Read the article

• Problem with access to file

  - by phenevo

  Hi, I have winforms application and it has reference to library MyLibrary. MyLibrary has method: string[] GiveMeNamesOfAirports() { string[] lines= File.ReadLines("airports.txt"); foreach(string line in lines) ... } And when I run my Winforms application: I get error: file couldn't be find. I was trying other function: string[] lines = File.ReadAllLines(Path.Combine(System.Environment.CurrentDirectory, "airports.txt")); string[] lines = File.ReadAllLines(Path.Combine(Assembly.GetExecutingAssembly().Location, "airports.txt")); string[] lines = File.ReadAllLines(Path.Combine(Assembly.GetAssembly(typeof(Airport)).Location, "airports.txt"));

  Read the article

• Removing words from a file

  - by user1765792

  I'm trying to take a regular text file and remove words identified in a separate file (stopwords) containing the words to be removed separated by carriage returns ("\n"). Right now I'm converting both files into lists so that the elements of each list can be compared. I got this function to work, but it doesn't remove all of the words I have specified in the stopwords file. Any help is greatly appreciated. def elimstops(file_str): #takes as input a string for the stopwords file location stop_f = open(file_str, 'r') stopw = stop_f.read() stopw = stopw.split('\n') text_file = open('sample.txt') #Opens the file whose stop words will be eliminated prime = text_file.read() prime = prime.split(' ') #Splits the string into a list separated by a space tot_str = "" #total string i = 0 while i < (len(stopw)): if stopw[i] in prime: prime.remove(stopw[i]) #removes the stopword from the text else: pass i += 1 # Creates a new string from the compilation of list elements # with the stop words removed for v in prime: tot_str = tot_str + str(v) + " " return tot_str

  Read the article

• Multiple file upload with asp.net 4.5 and Visual Studio 2012

  - by Jalpesh P. Vadgama

  This post will be part of Visual Studio 2012 feature series. In earlier version of ASP.NET there is no way to upload multiple files at same time. We need to use third party control or we need to create custom control for that. But with asp.net 4.5 now its possible to upload multiple file with file upload control. With ASP.NET 4.5 version Microsoft has enhanced file upload control to support HTML5 multiple attribute. There is a property called ‘AllowedMultiple’ to support that attribute and with that you can easily upload the file. So what we are waiting for!! It’s time to create one example. On the default.aspx file I have written following. <asp:FileUpload ID="multipleFile" runat="server" AllowMultiple="true" /> <asp:Button ID="uploadFile" runat="server" Text="Upload files" onclick="uploadFile_Click"/> Here you can see that I have given file upload control id as multipleFile and I have set AllowMultiple file to true. I have also taken one button for uploading file.For this example I am going to upload file in images folder. As you can see I have also attached event handler for button’s click event. So it’s time to write server side code for this. Following code is for the server side. protected void uploadFile_Click(object sender, EventArgs e) { if (multipleFile.HasFiles) { foreach(HttpPostedFile uploadedFile in multipleFile.PostedFiles) { uploadedFile.SaveAs(System.IO.Path.Combine(Server.MapPath("~/Images/"),uploadedFile.FileName)); Response.Write("File uploaded successfully"); } } } Here in the above code you can see that I have checked whether multiple file upload control has multiple files or not and then I have save that in Images folder of web application. Once you run the application in browser it will look like following. I have selected two files. Once I have selected and clicked on upload file button it will give message like following. As you can see now it has successfully upload file and you can see in windows explorer like following. As you can see it’s very easy to upload multiple file in ASP.NET 4.5. Stay tuned for more. Till then happy programming. P.S.: This feature is only supported in browser who support HTML5 multiple file upload. For other browsers it will work like normal file upload control in asp.net.

  Read the article

• make-like build tools for data?

  - by miku

  Make is a standard tools for building software. But make decides whether a target needs to be regenerated by comparing file modification times. Are there any proven, preferably small tools that handle builds not for software but for data? Something that regenerates targets not only on mod times but on certain other properties (e.g. completeness). (Or alternatively some paper that describes such a tool.) As illustration: I'd like to automate the following process: get data (e.g. a tarball) from some regularly updated source copy somewhere if it's not there (based e.g. on some filename-scheme) convert the files to different format (but only if there aren't successfully converted ones there - e.g. from a previous attempt - custom comparison routine) for each file find a certain data element and fetch some additional file from say an URL, but only if that hasn't been downloaded yet (decide on existence of file and file "freshness") finally compute something (e.g. word count for something identifiable and store it in the database, but only if the DB does not have an entry for that exact ID yet) Observations: there are different stages each stage is usually simple to compute or implement in isolation each stage may be simple, but the data volume may be large each stage may produce a few errors each stage may have different signals, on when (re)processing is needed Requirements: builds should be interruptable and idempotent (== robust) when interrupted, already processed objects should be reused to speedup the next run data paths should be easy to adjust (simple syntax, nothing new to learn, internal dsl would be ok) some form of dependency graph, that describes the process would be nice for later visualizations should leverage existing programs, if possible I've done some research on make alternatives like rake and have worked a lot with ant and maven in the past. All these tools naturally focus on code and software build, not on data builds. A system we have in place now for a task similar to the above is pretty much just shell scripts, which are compact (and are a ok glue for a variety of other programs written in other languages), so I wonder if worse is better?

  Read the article

• Oracle Data Integrator 11.1.1.5 Complex Files as Sources and Targets

  - by Alex Kotopoulis

  Overview ODI 11.1.1.5 adds the new Complex File technology for use with file sources and targets. The goal is to read or write file structures that are too complex to be parsed using the existing ODI File technology. This includes: Different record types in one list that use different parsing rules Hierarchical lists, for example customers with nested orders Parsing instructions in the file data, such as delimiter types, field lengths, type identifiers Complex headers such as multiple header lines or parseable information in header Skipping of lines  Conditional or choice fields Similar to the ODI File and XML File technologies, the complex file parsing is done through a JDBC driver that exposes the flat file as relational table structures. Complex files are mapped to one or more table structures, as opposed to the (simple) file technology, which always has a one-to-one relationship between file and table. The resulting set of tables follows the same concept as the ODI XML driver, table rows have additional PK-FK relationships to express hierarchy as well as order values to maintain the file order in the resulting table.   The parsing instruction format used for complex files is the nXSD (native XSD) format that is already in use with Oracle BPEL. This format extends the XML Schema standard by adding additional parsing instructions to each element. Using nXSD parsing technology, the native file is converted into an internal XML format. It is important to understand that the XML is streamed to improve performance; there is no size limitation of the native file based on memory size, the XML data is never fully materialized.  The internal XML is then converted to relational schema using the same mapping rules as the ODI XML driver. How to Create an nXSD file Complex file models depend on the nXSD schema for the given file. This nXSD file has to be created using a text editor or the Native Format Builder Wizard that is part of Oracle BPEL. BPEL is included in the ODI Suite, but not in standalone ODI Enterprise Edition. The nXSD format extends the standard XSD format through nxsd attributes. NXSD is a valid XML Schema, since the XSD standard allows extra attributes with their own namespaces. The following is a sample NXSD schema: <?xml version="1.0"?> <xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd" elementFormDefault="qualified" xmlns:tns="http://xmlns.oracle.com/pcbpel/demoSchema/csv" targetNamespace="http://xmlns.oracle.com/pcbpel/demoSchema/csv" attributeFormDefault="unqualified" nxsd:encoding="US-ASCII" nxsd:stream="chars" nxsd:version="NXSD"> <xsd:element name="Root">         <xsd:complexType><xsd:sequence>       <xsd:element name="Header">                 <xsd:complexType><xsd:sequence>                         <xsd:element name="Branch" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy=","/>                         <xsd:element name="ListDate" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}"/>                         </xsd:sequence></xsd:complexType>                         </xsd:element>                 </xsd:sequence></xsd:complexType>         <xsd:element name="Customer" maxOccurs="unbounded">                 <xsd:complexType><xsd:sequence>                 <xsd:element name="Name" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy=","/>                         <xsd:element name="Street" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="," />                         <xsd:element name="City" type="xsd:string" nxsd:style="terminated" nxsd:terminatedBy="${eol}" />                         </xsd:sequence></xsd:complexType>                         </xsd:element>                 </xsd:sequence></xsd:complexType> </xsd:element> </xsd:schema> The nXSD schema annotates elements to describe their position and delimiters within the flat text file. The schema above uses almost exclusively the nxsd:terminatedBy instruction to look for the next terminator chars. There are various constructs in nXSD to parse fixed length fields, look ahead in the document for string occurences, perform conditional logic, use variables to remember state, and many more. nXSD files can either be written manually using an XML Schema Editor or created using the Native Format Builder Wizard. Both Native Format Builder Wizard as well as the nXSD language are described in the Application Server Adapter Users Guide. The way to start the Native Format Builder in BPEL is to create a new File Adapter; in step 8 of the Adapter Configuration Wizard a new Schema for Native Format can be created:   The Native Format Builder guides through a number of steps to generate the nXSD based on a sample native file. If the format is complex, it is often a good idea to “approximate” it with a similar simple format and then add the complex components manually.  The resulting *.xsd file can be copied and used as the format for ODI, other BPEL constructs such as the file adapter definition are not relevant for ODI. Using this technique it is also possible to parse the same file format in SOA Suite and ODI, for example using SOA for small real-time messages, and ODI for large batches. This nXSD schema in this example describes a file with a header row containing data and 3 string fields per row delimited by commas, for example: Redwood City Downtown Branch, 06/01/2011 Ebeneezer Scrooge, Sandy Lane, Atherton Tiny Tim, Winton Terrace, Menlo Park The ODI Complex File JDBC driver exposes the file structure through a set of relational tables with PK-FK relationships. The tables for this example are: Table ROOT (1 row): ROOTPK Primary Key for root element SNPSFILENAME Name of the file SNPSFILEPATH Path of the file SNPSLOADDATE Date of load Table HEADER (1 row): ROOTFK Foreign Key to ROOT record ROWORDER Order of row in native document BRANCH Data BRANCHORDER Order of Branch within row LISTDATE Data LISTDATEORDER Order of ListDate within row Table ADDRESS (2 rows): ROOTFK Foreign Key to ROOT record ROWORDER Order of row in native document NAME Data NAMEORDER Oder of Name within row STREET Data STREETORDER Order of Street within row CITY Data CITYORDER Order of City within row Every table has PK and/or FK fields to reflect the document hierarchy through relationships. In this example this is trivial since the HEADER and all CUSTOMER records point back to the PK of ROOT. Deeper nested documents require this to identify parent elements. All tables also have a ROWORDER field to define the order of rows, as well as order fields for each column, in case the order of columns varies in the original document and needs to be maintained. If order is not relevant, these fields can be ignored. How to Create an Complex File Data Server in ODI After creating the nXSD file and a test data file, and storing it on the local file system accessible to ODI, you can go to the ODI Topology Navigator to create a Data Server and Physical Schema under the Complex File technology. This technology follows the conventions of other ODI technologies and is very similar to the XML technology. The parsing settings such as the source native file, the nXSD schema file, the root element, as well as the external database can be set in the JDBC URL: The use of an external database defined by dbprops is optional, but is strongly recommended for production use. Ideally, the staging database should be used for this. Also, when using a complex file exclusively for read purposes, it is recommended to use the ro=true property to ensure the file is not unnecessarily synchronized back from the database when the connection is closed. A data file is always required to be present  at the filename path during design-time. Without this file, operations like testing the connection, reading the model data, or reverse engineering the model will fail.  All properties of the Complex File JDBC Driver are documented in the Oracle Fusion Middleware Connectivity and Knowledge Modules Guide for Oracle Data Integrator in Appendix C: Oracle Data Integrator Driver for Complex Files Reference. David Allan has created a great viewlet Complex File Processing - 0 to 60 which shows the creation of a Complex File data server as well as a model based on this server. How to Create Models based on an Complex File Schema Once physical schema and logical schema have been created, the Complex File can be used to create a Model as if it were based on a database. When reverse-engineering the Model, data stores(tables) for each XSD element of complex type will be created. Use of complex files as sources is straightforward; when using them as targets it has to be made sure that all dependent tables have matching PK-FK pairs; the same applies to the XML driver as well. Debugging and Error Handling There are different ways to test an nXSD file. The Native Format Builder Wizard can be used even if the nXSD wasn’t created in it; it will show issues related to the schema and/or test data. In ODI, the nXSD  will be parsed and run against the existing test XML file when testing a connection in the Dataserver. If either the nXSD has an error or the data is non-compliant to the schema, an error will be displayed. Sample error message: Error while reading native data. [Line=1, Col=5] Not enough data available in the input, when trying to read data of length "19" for "element with name D1" from the specified position, using "style" as "fixedLength" and "length" as "". Ensure that there is enough data from the specified position in the input. Complex File FAQ Is the size of the native file limited by available memory? No, since the native data is streamed through the driver, only the available space in the staging database limits the size of the data. There are limits on individual field sizes, though; a single large object field needs to fit in memory. Should I always use the complex file driver instead of the file driver in ODI now? No, use the file technology for all simple file parsing tasks, for example any fixed-length or delimited files that just have one row format and can be mapped into a simple table. Because of its narrow assumptions the ODI file driver is easy to configure within ODI and can stream file data without writing it into a database. The complex file driver should be used whenever the use case cannot be handled through the file driver. Are we generating XML out of flat files before we write it into a database? We don’t materialize any XML as part of parsing a flat file, either in memory or on disk. The data produced by the XML parser is streamed in Java objects that just use XSD-derived nXSD schema as its type system. We use the nXSD schema because is the standard for describing complex flat file metadata in Oracle Fusion Middleware, and enables users to share schemas across products. Is the nXSD file interchangeable with SOA Suite? Yes, ODI can use the same nXSD files as SOA Suite, allowing mixed use cases with the same data format. Can I start the Native Format Builder from the ODI Studio? No, the Native Format Builder has to be started from a JDeveloper with BPEL instance. You can get BPEL as part of the SOA Suite bundle. Users without SOA Suite can manually develop nXSD files using XSD editors. When is the database data written back to the native file? Data is synchronized using the SYNCHRONIZE and CREATE FILE commands, and when the JDBC connection is closed. It is recommended to set the ro or read_only property to true when a file is exclusively used for reading so that no unnecessary write-backs occur. Is the nXSD metadata part of the ODI Master or Work Repository? No, the data server definition in the master repository only contains the JDBC URL with file paths; the nXSD files have to be accessible on the file systems where the JDBC driver is executed during production, either by copying or by using a network file system. Where can I find sample nXSD files? The Application Server Adapter Users Guide contains nXSD samples for various different use cases.

  Read the article

• Microsoft and the open source community

  - by Charles Young

  For the last decade, I have repeatedly, in my imitable Microsoft fan boy style, offered an alternative view to commonly held beliefs about Microsoft's stance on open source licensing.  In earlier times, leading figures in Microsoft were very vocal in resisting the idea that commercial licensing is outmoded or morally reprehensible.  Many people interpreted this as all-out corporate opposition to open source licensing.  I never read it that way. It is true that I've met individual employees of Microsoft who are antagonistic towards FOSS (free and open source software), but I've met more who are supportive or at least neutral on the subject.  In any case, individual attitudes of employees don't necessarily reflect a corporate stance.  The strongest opposition I've encountered has actually come from outside the company.  It's not a charitable thought, but I sometimes wonder if there are people in the .NET community who are opposed to FOSS simply because they believe, erroneously, that Microsoft is opposed. Here, for what it is worth, are the points I've repeated endlessly over the years and which have often been received with quizzical scepticism. a)  A decade ago, Microsoft's big problem was not FOSS per se, or even with copyleft.  The thing which really kept them awake at night was the fear that one day, someone might find, deep in the heart of the Windows code base, some code that should not be there and which was published under GPL.  The likelihood of this ever happening has long since faded away, but there was a time when MS was running scared.  I suspect this is why they held out for a while from making Windows source code open to inspection.  Nowadays, as an MVP, I am positively encouraged to ask to see Windows source. b)  Microsoft has never opposed the open source community.  They have had problems with specific people and organisations in the FOSS community.  Back in the 1990s, Richard Stallman gave time and energy to a successful campaign to launch antitrust proceedings against Microsoft.  In more recent times, the negative attitude of certain people to Microsoft's submission of two FOSS licences to the OSI (both of which have long since been accepted), and the mad scramble to try to find any argument, however tenuous, to block their submission was not, let us say, edifying. c) Microsoft has never, to my knowledge, written off the FOSS model.  They certainly don't agree that more traditional forms of licensing are inappropriate or immoral, and they've always been prepared to say so.  One reason why it was so hard to convince people that Microsoft is not rabidly antagonistic towards FOSS licensing is that so many people think they have no involvement in open source.  A decade ago, there was virtually no evidence of any such involvement.  However, that was a long time ago.  Quietly over the years, Microsoft has got on with the job of working out how to make use of FOSS licensing and how to support the FOSS community.  For example, as well as making increasingly extensive use of Github, they run an important FOSS forge (CodePlex) on which they, themselves, host many hundreds of distinct projects.  The total count may even be in the thousands now.  I suspect there is a limit of about 500 records on CodePlex searches because, for the past few years, whenever I search for Microsoft-specific projects on CodePlex, I always get approx. 500 hits.  Admittedly, a large volume of the stuff they publish under FOSS licences amounts to code samples, but many of those 'samples' have grown into useful and fully featured frameworks, libraries and tools. All this is leading up to the observation that yesterday's announcement by Scott Guthrie marks a significant milestone and should not go unnoticed.  If you missed it, let me summarise.   From the first release of .NET, Microsoft has offered a web development framework called ASP.NET.  The core libraries are included in the .NET framework which is released free of charge, but which is not open source.   However, in recent years, the number of libraries that constitute ASP.NET have grown considerably.  Today, most professional ASP.NET web development exploits the ASP.NET MVC framework.  This, together with several other important parts of the ASP.NET technology stack, is released on CodePlex under the Apache 2.0 licence.   Hence, today, a huge swathe of web development on the .NET/Azure platform relies four-square on the use of FOSS frameworks and libraries. Yesterday, Scott Guthrie announced the next stage of ASP.NET's journey towards FOSS nirvana.  This involves extending ASP.NET's FOSS stack to include Web API and the MVC Razor view engine which is rapidly becoming the de facto 'standard' for building web pages in ASP.NET.  However, perhaps the more important announcement is that the ASP.NET team will now accept and review contributions from the community.  Scott points out that this model is already in place elsewhere in Microsoft, and specifically draws attention to development of the Windows Azure SDKs.  These SDKs are central to Azure development.   The .NET and Java SDKs are published under Apache 2.0 on Github and Microsoft is open to community contributions.  Accepting contributions is a more profound move than simply releasing code under FOSS licensing.  It means that Microsoft is wholeheartedly moving towards a full-blooded open source approach for future evolution of some of their central and most widely used .NET and Azure frameworks and libraries.  In conjunction with Scott's announcement, Microsoft has also released Git support for CodePlex (at long last!) and, perhaps more importantly, announced significant new investment in their own FOSS forge. Here at Solidsoft we have several reasons to be very interested in Scott's announcement. I'll draw attention to one of them.  Earlier this year we wrote the initial version of a new UK Government web application called CloudStore.  CloudStore provides a way for local and central government to discover and purchase applications and services. We wrote the web site using ASP.NET MVC which is FOSS.  However, this point has been lost on the ladies and gentlemen of the press and, I suspect, on some of the decision makers on the government side.  They announced a few weeks ago that future versions of CloudStore will move to a FOSS framework, clearly oblivious of the fact that it is already built on a FOSS framework.  We are, it is fair to say, mildly irked by the uninformed and badly out-of-date assumption that “if it is Microsoft, it can't be FOSS”.  Old prejudices live on.

  Read the article

• Microsoft sera présent à Solutions Linux : l'interopérabilité au menu du salon de l'Open Source

  Microsoft sera présent à Solutions Linux : l'interopérabilité au menu du salon de l'Open Source

  Read the article

• Open XML SDK 2 Released

  - by Tim Murphy

  Note: Cross posted from Coding The Document. Permalink This post is a little late since the SDK was released about a week ago.  At PSC we have been using the Open XML SDK 2 since its earliest beta.  It is a very powerful tool for generating documents without using the Office DLLs.  It is also the main technology that I have been working with for the last six months.  I would suggest giving it a try.  Stay tuned here.  In the near future I will be presenting at different locations on this and other document generation technologies. Download the Open XML SDK here. del.icio.us Tags: Office Open XML,Open XML SDK 2

  Read the article

< Previous Page | 61 62 63 64 65 66 67 68 69 70 71 72  | Next Page >