Search Results

Search found 227 results on 10 pages for 'l1'.

Page 5/10 | < Previous Page | 1 2 3 4 5 6 7 8 9 10  | Next Page >

  • How to visualize a list of lists of lists of ... in R?

    - by Martin
    Hi there, I have a very deep list of lists in R. Now I want to print this list to the standard output to get a better overview of the elements. It should look like the way the StatET plugin for eclipse shows a list. Example list: l6 = list() l6[["h"]] = "one entry" l6[["g"]] = "nice" l5 = list() l5[["e"]] = l6 l4 = list() l4[["f"]] = "test" l4[["d"]] = l5 l3 = list() l3[["c"]] = l4 l2 = list() l2[["b"]] = l3 l1 = list() l1[["a"]] = l2 This should print like: List of 1 $ a:List of 1 ..$ b:List of 1 .. ..$ c:List of 2 .. .. ..$ f: chr "test" .. .. ..$ d:List of 1 .. .. .. ..$ e:List of 2 .. .. .. .. ..$ h: chr "one entry" .. .. .. .. ..$ g: chr "nice" I know this is possible with recursion and the deepness. But is there a way to do this with the help of rapply or something like that? Thanx in advance, Martin

    Read the article

  • take values from table cells and turn into array

    - by liz
    using jquery I need to retrieve an array from table cells, format the data and pass it into a js function. the code i am using is this: var l1 = new Array(); $('table#datatable tbody td:first-child').each(function() { l1.push($(this).text()); }); this is the table fragment <tr> <th scope="row">Age: 0-4</th> <td>0</td> <td>9.7</td> </tr> <tr> <th scope="row">5-17</th> <td>23.6</td> <td>18.0</td> </tr> <tr> <th scope="row">Total 0-17</th> <td>20.6</td> <td>16.1</td> </tr> the table's id is "datatable". i want to return an array of the contents of each first td and then format it like this: 0,23.6,20.6 i am very new to using arrays...

    Read the article

  • How do I pass a LuaTable between two Lua states using LuaInterface?

    - by user316675
    I've been trying to pass a LuaTable class between two Lua states, like so: LuaManager L1 = new Lua(); LuaManager L2 = new Lua(); LuaTable table = L1.DoString("return {apple = 25}")[0]; L2["tbl"] = table; double results = L2.DoString("return tbl[\"apple\"]")[0]; Assert.AreEqual(25.0, results); The above test fails; I receive a return value of nil. Using the Immediate Window confirms that "table" is a non-null object, and that table["apple"] returns 25; it's something that's being lost in translation to L2. Interestingly, when the object is loaded back into the same state, the test works, like so: //Succeeds LuaManager lua = new Lua(); LuaTable table = lua.DoString("return {apple = 25}")[0]; lua["tbl"] = table; double results = lua.DoString("return tbl[\"apple\"]")[0]; Assert.AreEqual(25.0, results); How can I safely pass the LuaTables without hassles? Thanks in advance!

    Read the article

  • Spider a Website and Return URLs Only

    - by Rob Wilkerson
    I'm not quite sure how best to define/articulate this, but I'm looking for a way to pseudo-spider a website. The key is that I don't actually want the content, but rather a simple list of URIs. I can get reasonably close to this idea with Wget using the --spider option, but when piping that output through a grep, I can't seem to find the right magic to make it work: wget --spider --force-html -r -l1 http://somesite.com | grep 'Saving to:' The grep filter seems to have absolutely no affect on the wget output. Have I got something wrong or is there another tool I should try that's more geared towards providing this kind of limited result set? Thanks. UPDATE So I just found out offline that, by default, wget writes to stderr. I missed that in the man pages (in fact, I still haven't found it if it's in there). Once I piped the return to stdout, I got closer to what I need: wget --spider --force-html -r -l1 http://somesite.com 2>&1 | grep 'Saving to:' I'd still be interested in other/better means for doing this kind of thing, if any exist.

    Read the article

  • Will Windows 7 work at all on my old toshiba [closed]

    - by andrew
    Windows 7 requires the following specifications: 1 gigahertz (GHz) or faster 32-bit (x86) or 64-bit (x64) processor 1 gigabyte (GB) RAM (32-bit) or 2 GB RAM (64-bit) 16 GB available hard disk space (32-bit) or 20 GB (64-bit) DirectX 9 graphics device with WDDM 1.0 or higher driver Will it work at all on my old toshiba Satellite A100 PSAA8C-SK400E Intel® Core™ Solo processor T1350 (1.86GHz, 533MHz FSB, L1 Cache 32KB/32KB, L2 Cache 2MB) Standard Memory: 2x512 MB DDR2 Intel® Graphics Media Accelerator 950 with 8MB-128MB. The main problem I can see is that the graphics is not up to it.

    Read the article

  • Best Processor for MediaSmart Server?

    - by Kent Boogaart
    I'm trying to figure out what the best possible processor is that I can stick in my HP MediaSmart server. I'm clueless when it comes to correlating CPUs to motherboards. I suspect it's the socket type I care about, but I worry that there's more to it. CPU-Z gives me (excerpt): Processors Information ------------------------------------------------------------------------- Processor 1 ID = 0 Number of cores 1 (max 1) Number of threads 1 (max 1) Name AMD Sempron LE-1150 Codename Sparta Specification AMD Sempron(tm) Processor LE-1150 Package Socket AM2 (940) CPUID F.F.1 Extended CPUID F.7F Brand ID 1 Core Stepping DH-G1 Technology 65 nm Core Speed 1000.0 MHz Multiplier x FSB 5.0 x 200.0 MHz HT Link speed 800.0 MHz Stock frequency 2000 MHz Instructions sets MMX (+), 3DNow! (+), SSE, SSE2, SSE3, x86-64 L1 Data cache 64 KBytes, 2-way set associative, 64-byte line size L1 Instruction cache 64 KBytes, 2-way set associative, 64-byte line size L2 cache 256 KBytes, 16-way set associative, 64-byte line size FID/VID Control yes Max FID 10.0x Max VID 1.350 V P-State FID 0x2 - VID 0x12 (5.0x - 1.100 V) P-State FID 0xA - VID 0x0C (9.0x - 1.250 V) P-State FID 0xC - VID 0x0A (10.0x - 1.300 V) K8 Thermal sensor yes K8 Revision ID 6.0 Attached device PCI device at bus 0, device 24, function 0 Attached device PCI device at bus 0, device 24, function 1 Attached device PCI device at bus 0, device 24, function 2 Attached device PCI device at bus 0, device 24, function 3 Chipset ------------------------------------------------------------------------- Northbridge SiS 761GX rev. 02 Southbridge SiS 966 rev. 59 Graphic Interface AGP AGP Revision 3.0 AGP Transfer Rate 8x AGP SBA supported, enabled Memory Type DDR2 Memory Size 2048 MBytes Channels Single Memory Frequency 200.0 MHz (CPU/5) CAS# latency (CL) 5.0 RAS# to CAS# delay (tRCD) 5 RAS# Precharge (tRP) 5 Cycle Time (tRAS) 15 Bank Cycle Time (tRC) 21 Command Rate (CR) 1T DMI ------------------------------------------------------------------------- DMI BIOS vendor Phoenix Technologies, LTD version R03 date 05/08/2008 DMI System Information manufacturer HP product MediaSmart Server version unknown serial CN68330DGH UUID A482007B-B0CC7593-DD11736A-407B7067 DMI Baseboard vendor Wistron model SJD4 revision A.0 serial unknown DMI System Enclosure manufacturer HP chassis type Desktop chassis serial unknown DMI Processor manufacturer AMD model AMD Sempron(tm) Processor LE-1150 clock speed 2000.0 MHz FSB speed 200.0 MHz multiplier 10.0x DMI Memory Controller correction 64-bit ECC Max module size 4096 MBytes DMI Memory Module designation A0 size 2048 MBytes (double bank) DMI Memory Module designation A1 DMI Memory Module designation A2 DMI Memory Module designation A3 DMI Port Connector designation PS/2 Mouse (internal) port type Mouse Port connector PS/2 connector PS/2 DMI Port Connector designation USB0 (external) port type USB DMI Physical Memory Array location Motherboard usage System Memory correction None max capacity 16384 MBytes max# of devices 4 DMI Memory Device designation A0 format DIMM type unknown total width 64 bits data width 64 bits size 2048 MBytes DMI Memory Device designation A1 format DIMM type unknown total width 64 bits data width 64 bits DMI Memory Device designation A2 format DIMM type unknown total width 64 bits data width 64 bits DMI Memory Device designation A3 format DIMM type unknown total width 64 bits data width 64 bits How do I figure out what options I have for an upgrade?

    Read the article

  • [Ubuntu]df Total size is not correct compared with the size of the disk

    - by John John
    I'm running Ubuntu Squeeze and on one of the partitions df is showing the Total size as 335G: Filesystem Size Used Avail Use% Mounted on /dev/sdb 335G 225G 94G 71% /mnt However in the past it was showing as 360GB (which is the actual size): fdisk -l /dev/sdb Disk /dev/sdb: 365.0 GB, 365041287168 bytes lsof +L1 does not return anything (and anyway if this would be the case the Total space should not be affected.) On this partition I'm writing (and deleting) a lot of files and this happened before in the past, but problem solved by itself.

    Read the article

  • lshw tells me my processor is a 64 bits but my motherboard has a 32 bits width

    - by bpetit
    Recently I noticed lshw tells me a strange thing. Here is the first part of my lshw output: bpetit-1025c description: Notebook product: 1025C (1025C) vendor: ASUSTeK COMPUTER INC. version: x.x serial: C3OAAS000774 width: 32 bits capabilities: smbios-2.7 dmi-2.7 smp-1.4 smp configuration: boot=normal chassis=notebook cpus=2 family=Eee PC... *-core description: Motherboard product: 1025C vendor: ASUSTeK COMPUTER INC. physical id: 0 version: x.xx serial: EeePC-0123456789 slot: To be filled by O.E.M. *-firmware description: BIOS vendor: American Megatrends Inc. physical id: 0 version: 1025C.0701 date: 01/06/2012 size: 64KiB capacity: 1984KiB capabilities: pci upgrade shadowing cdboot bootselect socketedrom edd... *-cpu:0 description: CPU product: Intel(R) Atom(TM) CPU N2800 @ 1.86GHz vendor: Intel Corp. physical id: 4 bus info: cpu@0 version: 6.6.1 serial: 0003-0661-0000-0000-0000-0000 slot: CPU 1 size: 798MHz capacity: 1865MHz width: 64 bits clock: 533MHz capabilities: x86-64 boot fpu fpu_exception wp vme de pse tsc ... configuration: cores=2 enabledcores=1 id=2 threads=2 *-cache:0 description: L1 cache physical id: 5 slot: L1-Cache size: 24KiB capacity: 24KiB capabilities: internal write-back unified *-cache:1 description: L2 cache physical id: 6 slot: L2-Cache size: 512KiB capacity: 512KiB capabilities: internal varies unified *-logicalcpu:0 description: Logical CPU physical id: 2.1 width: 64 bits capabilities: logical *-logicalcpu:1 description: Logical CPU physical id: 2.2 width: 64 bits capabilities: logical *-logicalcpu:2 description: Logical CPU physical id: 2.3 width: 64 bits capabilities: logical *-logicalcpu:3 description: Logical CPU physical id: 2.4 width: 64 bits capabilities: logical *-memory description: System Memory physical id: 13 slot: System board or motherboard size: 2GiB *-bank:0 description: SODIMM [empty] product: [Empty] vendor: [Empty] physical id: 0 serial: [Empty] slot: DIMM0 *-bank:1 description: SODIMM DDR3 Synchronous 1066 MHz (0.9 ns) product: SSZ3128M8-EAEEF vendor: Xicor physical id: 1 serial: 00000004 slot: DIMM1 size: 2GiB width: 64 bits clock: 1066MHz (0.9ns) *-cpu:1 physical id: 1 bus info: cpu@1 version: 6.6.1 serial: 0003-0661-0000-0000-0000-0000 size: 798MHz capacity: 798MHz capabilities: ht cpufreq configuration: id=2 *-logicalcpu:0 description: Logical CPU physical id: 2.1 capabilities: logical *-logicalcpu:1 description: Logical CPU physical id: 2.2 capabilities: logical *-logicalcpu:2 description: Logical CPU physical id: 2.3 capabilities: logical *-logicalcpu:3 description: Logical CPU physical id: 2.4 capabilities: logical So here I see my processor is effectively a 64 bits one. However, I'm wondering how my motherboard can have a "32 bits width". I've browsed the web to find an answer, without success. I imagine it's just a technical fact that I don't know about. Thanks.

    Read the article

  • removeFirst and addLast methods of LinkedList Class are Unknown

    - by user318068
    I have a problem with my code in C# . if i click in compiler button , I get the following errors 'System.Collections.Generic.LinkedList<int?>' does not contain a definition for 'removeFirst' and no extension method 'removeFirst' accepting a first argument of type 'System.Collections.Generic.LinkedList<int?>' could be found (are you missing a using directive or an assembly reference?). and 'System.Collections.Generic.LinkedList<Hanoi_tower.Sol>' does not contain a definition for 'addLast' and no extension method 'addLast' accepting a first argument of type 'System.Collections.Generic.LinkedList<Hanoi_tower.Sol>' could be found (are you missing a using directive or an assembly reference?) This is my program using System.; using System.Collections.Generic; using System.Linq; using System.Text; namespace Hanoi_tower { public class Sol { public LinkedList<int?> tower1 = new LinkedList<int?>(); public LinkedList<int?> tower2 =new LinkedList<int?>(); public LinkedList<int?> tower3 =new LinkedList<int?>(); public int depth; public LinkedList<Sol> neighbors; public Sol(LinkedList<int?> tower1, LinkedList<int?> tower2, LinkedList<int?> tower3) { this.tower1 = tower1; this.tower2 = tower2; this.tower3 = tower3; neighbors = new LinkedList<Sol>(); } public virtual void getneighbors() { Sol temp = this.copy(); Sol neighbor1 = this.copy(); Sol neighbor2 = this.copy(); Sol neighbor3 = this.copy(); Sol neighbor4 = this.copy(); Sol neighbor5 = this.copy(); Sol neighbor6 = this.copy(); if (temp.tower1.Count != 0) { if (neighbor1.tower2.Count != 0) { if (neighbor1.tower1.First.Value < neighbor1.tower2.First.Value) { neighbor1.tower2.AddFirst(neighbor1.tower1.RemoveFirst); neighbors.AddLast(neighbor1); } } else { neighbor1.tower2.AddFirst(neighbor1.tower1.RemoveFirst()); neighbors.AddLast(neighbor1); } if (neighbor2.tower3.Count != 0) { if (neighbor2.tower1.First.Value < neighbor2.tower3.First.Value) { neighbor2.tower3.AddFirst(neighbor2.tower1.RemoveFirst()); neighbors.AddLast(neighbor2); } } else { neighbor2.tower3.AddFirst(neighbor2.tower1.RemoveFirst()); neighbors.AddLast(neighbor2); } } //------------- if (temp.tower2.Count != 0) { if (neighbor3.tower1.Count != 0) { if (neighbor3.tower2.First.Value < neighbor3.tower1.First.Value) { neighbor3.tower1.AddFirst(neighbor3.tower2.RemoveFirst()); neighbors.AddLast(neighbor3); } } else { neighbor3.tower1.AddFirst(neighbor3.tower2.RemoveFirst()); neighbors.AddLast(neighbor3); } if (neighbor4.tower3.Count != 0) { if (neighbor4.tower2.First.Value < neighbor4.tower3.First.Value) { neighbor4.tower3.AddFirst(neighbor4.tower2.RemoveFirst()); neighbors.AddLast(neighbor4); } } else { neighbor4.tower3.AddFirst(neighbor4.tower2.RemoveFirst()); neighbors.AddLast(neighbor4); } } //------------------------ if (temp.tower3.Count() != 0) { if (neighbor5.tower1.Count() != 0) { if(neighbor5.tower3.ElementAtOrDefault() < neighbor5.tower1.ElementAtOrDefault()) { neighbor5.tower1.AddFirst(neighbor5.tower3.RemoveFirst()); neighbors.AddLast(neighbor5); } } else { neighbor5.tower1.AddFirst(neighbor5.tower3.RemoveFirst()); neighbors.AddLast(neighbor5); } if (neighbor6.tower2.Count() != 0) { if(neighbor6.tower3.element() < neighbor6.tower2.element()) { neighbor6.tower2.addFirst(neighbor6.tower3.removeFirst()); neighbors.addLast(neighbor6); } } else { neighbor6.tower2.addFirst(neighbor6.tower3.removeFirst()); neighbors.addLast(neighbor6); } } } public override string ToString() { string str; str="tower1"+ tower1.ToString() + " tower2" + tower2.ToString() + " tower3" + tower3.ToString(); return str; } public Sol copy() { Sol So; LinkedList<int> l1= new LinkedList<int>(); LinkedList<int> l2=new LinkedList<int>(); LinkedList<int> l3 = new LinkedList<int>(); for(int i=0;i<=this.tower1.Count() -1;i++) { l1.AddLast(tower1.get(i)); } for(int i=0;i<=this.tower2.size()-1;i++) { l2.addLast(tower2.get(i)); } for(int i=0;i<=this.tower3.size()-1;i++) { l3.addLast(tower3.get(i)); } So = new Sol(l1, l2, l3); return So; } public bool Equals(Sol sol) { if (this.tower1.Equals(sol.tower1) & this.tower2.Equals(sol.tower2) & this.tower3.Equals(sol.tower3)) return true; return false; } public virtual bool containedin(Stack<Sol> vec) { bool found =false; for(int i=0;i<= vec.Count-1;i++) { if(vec.get(i).tower1.Equals(this.tower1) && vec.get(i).tower2.Equals(this.tower2) && vec.get(i).tower3.Equals(this.tower3)) { found=true; break; } } return found; } } }

    Read the article

  • Improving Partitioned Table Join Performance

    - by Paul White
    The query optimizer does not always choose an optimal strategy when joining partitioned tables. This post looks at an example, showing how a manual rewrite of the query can almost double performance, while reducing the memory grant to almost nothing. Test Data The two tables in this example use a common partitioning partition scheme. The partition function uses 41 equal-size partitions: CREATE PARTITION FUNCTION PFT (integer) AS RANGE RIGHT FOR VALUES ( 125000, 250000, 375000, 500000, 625000, 750000, 875000, 1000000, 1125000, 1250000, 1375000, 1500000, 1625000, 1750000, 1875000, 2000000, 2125000, 2250000, 2375000, 2500000, 2625000, 2750000, 2875000, 3000000, 3125000, 3250000, 3375000, 3500000, 3625000, 3750000, 3875000, 4000000, 4125000, 4250000, 4375000, 4500000, 4625000, 4750000, 4875000, 5000000 ); GO CREATE PARTITION SCHEME PST AS PARTITION PFT ALL TO ([PRIMARY]); There two tables are: CREATE TABLE dbo.T1 ( TID integer NOT NULL IDENTITY(0,1), Column1 integer NOT NULL, Padding binary(100) NOT NULL DEFAULT 0x,   CONSTRAINT PK_T1 PRIMARY KEY CLUSTERED (TID) ON PST (TID) );   CREATE TABLE dbo.T2 ( TID integer NOT NULL, Column1 integer NOT NULL, Padding binary(100) NOT NULL DEFAULT 0x,   CONSTRAINT PK_T2 PRIMARY KEY CLUSTERED (TID, Column1) ON PST (TID) ); The next script loads 5 million rows into T1 with a pseudo-random value between 1 and 5 for Column1. The table is partitioned on the IDENTITY column TID: INSERT dbo.T1 WITH (TABLOCKX) (Column1) SELECT (ABS(CHECKSUM(NEWID())) % 5) + 1 FROM dbo.Numbers AS N WHERE n BETWEEN 1 AND 5000000; In case you don’t already have an auxiliary table of numbers lying around, here’s a script to create one with 10 million rows: CREATE TABLE dbo.Numbers (n bigint PRIMARY KEY);   WITH L0 AS(SELECT 1 AS c UNION ALL SELECT 1), L1 AS(SELECT 1 AS c FROM L0 AS A CROSS JOIN L0 AS B), L2 AS(SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 AS B), L3 AS(SELECT 1 AS c FROM L2 AS A CROSS JOIN L2 AS B), L4 AS(SELECT 1 AS c FROM L3 AS A CROSS JOIN L3 AS B), L5 AS(SELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B), Nums AS(SELECT ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS n FROM L5) INSERT dbo.Numbers WITH (TABLOCKX) SELECT TOP (10000000) n FROM Nums ORDER BY n OPTION (MAXDOP 1); Table T1 contains data like this: Next we load data into table T2. The relationship between the two tables is that table 2 contains ‘n’ rows for each row in table 1, where ‘n’ is determined by the value in Column1 of table T1. There is nothing particularly special about the data or distribution, by the way. INSERT dbo.T2 WITH (TABLOCKX) (TID, Column1) SELECT T.TID, N.n FROM dbo.T1 AS T JOIN dbo.Numbers AS N ON N.n >= 1 AND N.n <= T.Column1; Table T2 ends up containing about 15 million rows: The primary key for table T2 is a combination of TID and Column1. The data is partitioned according to the value in column TID alone. Partition Distribution The following query shows the number of rows in each partition of table T1: SELECT PartitionID = CA1.P, NumRows = COUNT_BIG(*) FROM dbo.T1 AS T CROSS APPLY (VALUES ($PARTITION.PFT(TID))) AS CA1 (P) GROUP BY CA1.P ORDER BY CA1.P; There are 40 partitions containing 125,000 rows (40 * 125k = 5m rows). The rightmost partition remains empty. The next query shows the distribution for table 2: SELECT PartitionID = CA1.P, NumRows = COUNT_BIG(*) FROM dbo.T2 AS T CROSS APPLY (VALUES ($PARTITION.PFT(TID))) AS CA1 (P) GROUP BY CA1.P ORDER BY CA1.P; There are roughly 375,000 rows in each partition (the rightmost partition is also empty): Ok, that’s the test data done. Test Query and Execution Plan The task is to count the rows resulting from joining tables 1 and 2 on the TID column: SET STATISTICS IO ON; DECLARE @s datetime2 = SYSUTCDATETIME();   SELECT COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID;   SELECT DATEDIFF(Millisecond, @s, SYSUTCDATETIME()); SET STATISTICS IO OFF; The optimizer chooses a plan using parallel hash join, and partial aggregation: The Plan Explorer plan tree view shows accurate cardinality estimates and an even distribution of rows across threads (click to enlarge the image): With a warm data cache, the STATISTICS IO output shows that no physical I/O was needed, and all 41 partitions were touched: Running the query without actual execution plan or STATISTICS IO information for maximum performance, the query returns in around 2600ms. Execution Plan Analysis The first step toward improving on the execution plan produced by the query optimizer is to understand how it works, at least in outline. The two parallel Clustered Index Scans use multiple threads to read rows from tables T1 and T2. Parallel scan uses a demand-based scheme where threads are given page(s) to scan from the table as needed. This arrangement has certain important advantages, but does result in an unpredictable distribution of rows amongst threads. The point is that multiple threads cooperate to scan the whole table, but it is impossible to predict which rows end up on which threads. For correct results from the parallel hash join, the execution plan has to ensure that rows from T1 and T2 that might join are processed on the same thread. For example, if a row from T1 with join key value ‘1234’ is placed in thread 5’s hash table, the execution plan must guarantee that any rows from T2 that also have join key value ‘1234’ probe thread 5’s hash table for matches. The way this guarantee is enforced in this parallel hash join plan is by repartitioning rows to threads after each parallel scan. The two repartitioning exchanges route rows to threads using a hash function over the hash join keys. The two repartitioning exchanges use the same hash function so rows from T1 and T2 with the same join key must end up on the same hash join thread. Expensive Exchanges This business of repartitioning rows between threads can be very expensive, especially if a large number of rows is involved. The execution plan selected by the optimizer moves 5 million rows through one repartitioning exchange and around 15 million across the other. As a first step toward removing these exchanges, consider the execution plan selected by the optimizer if we join just one partition from each table, disallowing parallelism: SELECT COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID WHERE $PARTITION.PFT(T1.TID) = 1 AND $PARTITION.PFT(T2.TID) = 1 OPTION (MAXDOP 1); The optimizer has chosen a (one-to-many) merge join instead of a hash join. The single-partition query completes in around 100ms. If everything scaled linearly, we would expect that extending this strategy to all 40 populated partitions would result in an execution time around 4000ms. Using parallelism could reduce that further, perhaps to be competitive with the parallel hash join chosen by the optimizer. This raises a question. If the most efficient way to join one partition from each of the tables is to use a merge join, why does the optimizer not choose a merge join for the full query? Forcing a Merge Join Let’s force the optimizer to use a merge join on the test query using a hint: SELECT COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID OPTION (MERGE JOIN); This is the execution plan selected by the optimizer: This plan results in the same number of logical reads reported previously, but instead of 2600ms the query takes 5000ms. The natural explanation for this drop in performance is that the merge join plan is only using a single thread, whereas the parallel hash join plan could use multiple threads. Parallel Merge Join We can get a parallel merge join plan using the same query hint as before, and adding trace flag 8649: SELECT COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID OPTION (MERGE JOIN, QUERYTRACEON 8649); The execution plan is: This looks promising. It uses a similar strategy to distribute work across threads as seen for the parallel hash join. In practice though, performance is disappointing. On a typical run, the parallel merge plan runs for around 8400ms; slower than the single-threaded merge join plan (5000ms) and much worse than the 2600ms for the parallel hash join. We seem to be going backwards! The logical reads for the parallel merge are still exactly the same as before, with no physical IOs. The cardinality estimates and thread distribution are also still very good (click to enlarge): A big clue to the reason for the poor performance is shown in the wait statistics (captured by Plan Explorer Pro): CXPACKET waits require careful interpretation, and are most often benign, but in this case excessive waiting occurs at the repartitioning exchanges. Unlike the parallel hash join, the repartitioning exchanges in this plan are order-preserving ‘merging’ exchanges (because merge join requires ordered inputs): Parallelism works best when threads can just grab any available unit of work and get on with processing it. Preserving order introduces inter-thread dependencies that can easily lead to significant waits occurring. In extreme cases, these dependencies can result in an intra-query deadlock, though the details of that will have to wait for another time to explore in detail. The potential for waits and deadlocks leads the query optimizer to cost parallel merge join relatively highly, especially as the degree of parallelism (DOP) increases. This high costing resulted in the optimizer choosing a serial merge join rather than parallel in this case. The test results certainly confirm its reasoning. Collocated Joins In SQL Server 2008 and later, the optimizer has another available strategy when joining tables that share a common partition scheme. This strategy is a collocated join, also known as as a per-partition join. It can be applied in both serial and parallel execution plans, though it is limited to 2-way joins in the current optimizer. Whether the optimizer chooses a collocated join or not depends on cost estimation. The primary benefits of a collocated join are that it eliminates an exchange and requires less memory, as we will see next. Costing and Plan Selection The query optimizer did consider a collocated join for our original query, but it was rejected on cost grounds. The parallel hash join with repartitioning exchanges appeared to be a cheaper option. There is no query hint to force a collocated join, so we have to mess with the costing framework to produce one for our test query. Pretending that IOs cost 50 times more than usual is enough to convince the optimizer to use collocated join with our test query: -- Pretend IOs are 50x cost temporarily DBCC SETIOWEIGHT(50);   -- Co-located hash join SELECT COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID OPTION (RECOMPILE);   -- Reset IO costing DBCC SETIOWEIGHT(1); Collocated Join Plan The estimated execution plan for the collocated join is: The Constant Scan contains one row for each partition of the shared partitioning scheme, from 1 to 41. The hash repartitioning exchanges seen previously are replaced by a single Distribute Streams exchange using Demand partitioning. Demand partitioning means that the next partition id is given to the next parallel thread that asks for one. My test machine has eight logical processors, and all are available for SQL Server to use. As a result, there are eight threads in the single parallel branch in this plan, each processing one partition from each table at a time. Once a thread finishes processing a partition, it grabs a new partition number from the Distribute Streams exchange…and so on until all partitions have been processed. It is important to understand that the parallel scans in this plan are different from the parallel hash join plan. Although the scans have the same parallelism icon, tables T1 and T2 are not being co-operatively scanned by multiple threads in the same way. Each thread reads a single partition of T1 and performs a hash match join with the same partition from table T2. The properties of the two Clustered Index Scans show a Seek Predicate (unusual for a scan!) limiting the rows to a single partition: The crucial point is that the join between T1 and T2 is on TID, and TID is the partitioning column for both tables. A thread that processes partition ‘n’ is guaranteed to see all rows that can possibly join on TID for that partition. In addition, no other thread will see rows from that partition, so this removes the need for repartitioning exchanges. CPU and Memory Efficiency Improvements The collocated join has removed two expensive repartitioning exchanges and added a single exchange processing 41 rows (one for each partition id). Remember, the parallel hash join plan exchanges had to process 5 million and 15 million rows. The amount of processor time spent on exchanges will be much lower in the collocated join plan. In addition, the collocated join plan has a maximum of 8 threads processing single partitions at any one time. The 41 partitions will all be processed eventually, but a new partition is not started until a thread asks for it. Threads can reuse hash table memory for the new partition. The parallel hash join plan also had 8 hash tables, but with all 5,000,000 build rows loaded at the same time. The collocated plan needs memory for only 8 * 125,000 = 1,000,000 rows at any one time. Collocated Hash Join Performance The collated join plan has disappointing performance in this case. The query runs for around 25,300ms despite the same IO statistics as usual. This is much the worst result so far, so what went wrong? It turns out that cardinality estimation for the single partition scans of table T1 is slightly low. The properties of the Clustered Index Scan of T1 (graphic immediately above) show the estimation was for 121,951 rows. This is a small shortfall compared with the 125,000 rows actually encountered, but it was enough to cause the hash join to spill to physical tempdb: A level 1 spill doesn’t sound too bad, until you realize that the spill to tempdb probably occurs for each of the 41 partitions. As a side note, the cardinality estimation error is a little surprising because the system tables accurately show there are 125,000 rows in every partition of T1. Unfortunately, the optimizer uses regular column and index statistics to derive cardinality estimates here rather than system table information (e.g. sys.partitions). Collocated Merge Join We will never know how well the collocated parallel hash join plan might have worked without the cardinality estimation error (and the resulting 41 spills to tempdb) but we do know: Merge join does not require a memory grant; and Merge join was the optimizer’s preferred join option for a single partition join Putting this all together, what we would really like to see is the same collocated join strategy, but using merge join instead of hash join. Unfortunately, the current query optimizer cannot produce a collocated merge join; it only knows how to do collocated hash join. So where does this leave us? CROSS APPLY sys.partitions We can try to write our own collocated join query. We can use sys.partitions to find the partition numbers, and CROSS APPLY to get a count per partition, with a final step to sum the partial counts. The following query implements this idea: SELECT row_count = SUM(Subtotals.cnt) FROM ( -- Partition numbers SELECT p.partition_number FROM sys.partitions AS p WHERE p.[object_id] = OBJECT_ID(N'T1', N'U') AND p.index_id = 1 ) AS P CROSS APPLY ( -- Count per collocated join SELECT cnt = COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID WHERE $PARTITION.PFT(T1.TID) = p.partition_number AND $PARTITION.PFT(T2.TID) = p.partition_number ) AS SubTotals; The estimated plan is: The cardinality estimates aren’t all that good here, especially the estimate for the scan of the system table underlying the sys.partitions view. Nevertheless, the plan shape is heading toward where we would like to be. Each partition number from the system table results in a per-partition scan of T1 and T2, a one-to-many Merge Join, and a Stream Aggregate to compute the partial counts. The final Stream Aggregate just sums the partial counts. Execution time for this query is around 3,500ms, with the same IO statistics as always. This compares favourably with 5,000ms for the serial plan produced by the optimizer with the OPTION (MERGE JOIN) hint. This is another case of the sum of the parts being less than the whole – summing 41 partial counts from 41 single-partition merge joins is faster than a single merge join and count over all partitions. Even so, this single-threaded collocated merge join is not as quick as the original parallel hash join plan, which executed in 2,600ms. On the positive side, our collocated merge join uses only one logical processor and requires no memory grant. The parallel hash join plan used 16 threads and reserved 569 MB of memory:   Using a Temporary Table Our collocated merge join plan should benefit from parallelism. The reason parallelism is not being used is that the query references a system table. We can work around that by writing the partition numbers to a temporary table (or table variable): SET STATISTICS IO ON; DECLARE @s datetime2 = SYSUTCDATETIME();   CREATE TABLE #P ( partition_number integer PRIMARY KEY);   INSERT #P (partition_number) SELECT p.partition_number FROM sys.partitions AS p WHERE p.[object_id] = OBJECT_ID(N'T1', N'U') AND p.index_id = 1;   SELECT row_count = SUM(Subtotals.cnt) FROM #P AS p CROSS APPLY ( SELECT cnt = COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID WHERE $PARTITION.PFT(T1.TID) = p.partition_number AND $PARTITION.PFT(T2.TID) = p.partition_number ) AS SubTotals;   DROP TABLE #P;   SELECT DATEDIFF(Millisecond, @s, SYSUTCDATETIME()); SET STATISTICS IO OFF; Using the temporary table adds a few logical reads, but the overall execution time is still around 3500ms, indistinguishable from the same query without the temporary table. The problem is that the query optimizer still doesn’t choose a parallel plan for this query, though the removal of the system table reference means that it could if it chose to: In fact the optimizer did enter the parallel plan phase of query optimization (running search 1 for a second time): Unfortunately, the parallel plan found seemed to be more expensive than the serial plan. This is a crazy result, caused by the optimizer’s cost model not reducing operator CPU costs on the inner side of a nested loops join. Don’t get me started on that, we’ll be here all night. In this plan, everything expensive happens on the inner side of a nested loops join. Without a CPU cost reduction to compensate for the added cost of exchange operators, candidate parallel plans always look more expensive to the optimizer than the equivalent serial plan. Parallel Collocated Merge Join We can produce the desired parallel plan using trace flag 8649 again: SELECT row_count = SUM(Subtotals.cnt) FROM #P AS p CROSS APPLY ( SELECT cnt = COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID WHERE $PARTITION.PFT(T1.TID) = p.partition_number AND $PARTITION.PFT(T2.TID) = p.partition_number ) AS SubTotals OPTION (QUERYTRACEON 8649); The actual execution plan is: One difference between this plan and the collocated hash join plan is that a Repartition Streams exchange operator is used instead of Distribute Streams. The effect is similar, though not quite identical. The Repartition uses round-robin partitioning, meaning the next partition id is pushed to the next thread in sequence. The Distribute Streams exchange seen earlier used Demand partitioning, meaning the next partition id is pulled across the exchange by the next thread that is ready for more work. There are subtle performance implications for each partitioning option, but going into that would again take us too far off the main point of this post. Performance The important thing is the performance of this parallel collocated merge join – just 1350ms on a typical run. The list below shows all the alternatives from this post (all timings include creation, population, and deletion of the temporary table where appropriate) from quickest to slowest: Collocated parallel merge join: 1350ms Parallel hash join: 2600ms Collocated serial merge join: 3500ms Serial merge join: 5000ms Parallel merge join: 8400ms Collated parallel hash join: 25,300ms (hash spill per partition) The parallel collocated merge join requires no memory grant (aside from a paltry 1.2MB used for exchange buffers). This plan uses 16 threads at DOP 8; but 8 of those are (rather pointlessly) allocated to the parallel scan of the temporary table. These are minor concerns, but it turns out there is a way to address them if it bothers you. Parallel Collocated Merge Join with Demand Partitioning This final tweak replaces the temporary table with a hard-coded list of partition ids (dynamic SQL could be used to generate this query from sys.partitions): SELECT row_count = SUM(Subtotals.cnt) FROM ( VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10), (11),(12),(13),(14),(15),(16),(17),(18),(19),(20), (21),(22),(23),(24),(25),(26),(27),(28),(29),(30), (31),(32),(33),(34),(35),(36),(37),(38),(39),(40),(41) ) AS P (partition_number) CROSS APPLY ( SELECT cnt = COUNT_BIG(*) FROM dbo.T1 AS T1 JOIN dbo.T2 AS T2 ON T2.TID = T1.TID WHERE $PARTITION.PFT(T1.TID) = p.partition_number AND $PARTITION.PFT(T2.TID) = p.partition_number ) AS SubTotals OPTION (QUERYTRACEON 8649); The actual execution plan is: The parallel collocated hash join plan is reproduced below for comparison: The manual rewrite has another advantage that has not been mentioned so far: the partial counts (per partition) can be computed earlier than the partial counts (per thread) in the optimizer’s collocated join plan. The earlier aggregation is performed by the extra Stream Aggregate under the nested loops join. The performance of the parallel collocated merge join is unchanged at around 1350ms. Final Words It is a shame that the current query optimizer does not consider a collocated merge join (Connect item closed as Won’t Fix). The example used in this post showed an improvement in execution time from 2600ms to 1350ms using a modestly-sized data set and limited parallelism. In addition, the memory requirement for the query was almost completely eliminated  – down from 569MB to 1.2MB. The problem with the parallel hash join selected by the optimizer is that it attempts to process the full data set all at once (albeit using eight threads). It requires a large memory grant to hold all 5 million rows from table T1 across the eight hash tables, and does not take advantage of the divide-and-conquer opportunity offered by the common partitioning. The great thing about the collocated join strategies is that each parallel thread works on a single partition from both tables, reading rows, performing the join, and computing a per-partition subtotal, before moving on to a new partition. From a thread’s point of view… If you have trouble visualizing what is happening from just looking at the parallel collocated merge join execution plan, let’s look at it again, but from the point of view of just one thread operating between the two Parallelism (exchange) operators. Our thread picks up a single partition id from the Distribute Streams exchange, and starts a merge join using ordered rows from partition 1 of table T1 and partition 1 of table T2. By definition, this is all happening on a single thread. As rows join, they are added to a (per-partition) count in the Stream Aggregate immediately above the Merge Join. Eventually, either T1 (partition 1) or T2 (partition 1) runs out of rows and the merge join stops. The per-partition count from the aggregate passes on through the Nested Loops join to another Stream Aggregate, which is maintaining a per-thread subtotal. Our same thread now picks up a new partition id from the exchange (say it gets id 9 this time). The count in the per-partition aggregate is reset to zero, and the processing of partition 9 of both tables proceeds just as it did for partition 1, and on the same thread. Each thread picks up a single partition id and processes all the data for that partition, completely independently from other threads working on other partitions. One thread might eventually process partitions (1, 9, 17, 25, 33, 41) while another is concurrently processing partitions (2, 10, 18, 26, 34) and so on for the other six threads at DOP 8. The point is that all 8 threads can execute independently and concurrently, continuing to process new partitions until the wider job (of which the thread has no knowledge!) is done. This divide-and-conquer technique can be much more efficient than simply splitting the entire workload across eight threads all at once. Related Reading Understanding and Using Parallelism in SQL Server Parallel Execution Plans Suck © 2013 Paul White – All Rights Reserved Twitter: @SQL_Kiwi

    Read the article

  • MPD to play music to single channel of my multi-channel card?

    - by hany tawfik
    I installed an Ubuntu 12-04 LTS server for a special background music application of mine, where I am using the server with an Asus Xonar DS sound card. The installation is successful, the MPD is working, the sound card is working with Alsa and its libraries installed accept for Alsa-oss. Alsamixer is working fine with left/right sides of each channel volume control through Q/Z letters shortcut when alsamixer is open in terminal . using the command " speaker-test -Dplug:surround71 -c8 -l1 -twav " I can hear every voice message coming out from the card at the right connector, so "front right/ front left" voice message are coming from first output in the card back, while the other outputs are silent..so on. The problem is that MPD is playing on all outputs simultaneously the same audio. I have been trying various configurations for the last 12 days with out any success, including trying to put mappings in the /etc/asound.conf Can any body help me achieve the above, or direct me to the right configuration of MPD or asound.conf

    Read the article

  • Is there any difference between processor and core?

    - by Salvador
    The following two command seems to give me different information about the same hardware srs@ubuntu:~$ cat /proc/cpuinfo | grep -e processor -e cores processor : 0 cpu cores : 4 processor : 1 cpu cores : 4 processor : 2 cpu cores : 4 processor : 3 cpu cores : 4 srs@ubuntu:~$ sudo dmidecode -t processor # dmidecode 2.9 SMBIOS 2.6 present. Handle 0x0004, DMI type 4, 42 bytes Processor Information Socket Designation: LGA1155 Type: Central Processor Family: <OUT OF SPEC> Manufacturer: Intel ID: A7 06 02 00 FF FB EB BF Version: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz Voltage: 1.0 V External Clock: 100 MHz Max Speed: 3800 MHz Current Speed: 3300 MHz Status: Populated, Enabled Upgrade: Other L1 Cache Handle: 0x0005 L2 Cache Handle: 0x0006 L3 Cache Handle: 0x0007 Serial Number: To Be Filled By O.E.M. Asset Tag: To Be Filled By O.E.M. Part Number: To Be Filled By O.E.M. Core Count: 4 Core Enabled: 1 Characteristics: 64-bit capable Until today I thought I had a single processor with 4 independent cores. I also thought that within each core can be used different threads.

    Read the article

  • ffmpeg - How to determine if -movflags faststart is enabled? PHP

    - by IIIOXIII
    While I am able to encode an mp4 file which I can plan on my local windows machine, I am having trouble encoding files to mp4 which are readable when streaming by safari, etc. After a bit of reading, I believe my issue is that I must move the metadata from the end of the file to the beginning in order for the converted mp4 files to be streamable. To that end, I am trying to find out if the build of ffmpeg that I am currently using is able to use the -movflags faststart option through php - as my current outputted mp4 files are not working when streamed online. This is the way I am now echoing the -help, -formats, -codecs, but I am not seeing anything about -movflags faststart in any of the lists: exec($ffmpegPath." -help", $codecArr); for($ii=0;$ii<count($codecArr);$ii++){ echo $codecArr[$ii].'</br>'; } Is there a similar method of determining if -movflags fastart is available to my ffmpeg build? Any other way? Should it be listed with any of the previously suggested commands? -help/-formats? Can someone that knows it is enabled in their version of ffmpeg check to see if it is listed under -help or -formats, etc.? TIA. EDIT: COMPLETE CONSOLE OUTPUT FOR BOTH THE CONVERSION COMMAND AND -MOVFLAGS COMMAND BELOW: COMMAND: ffmpeg_new -i C:\vidtests\Wildlife.wmv -s 640x480 C:\vidtests\Wildlife.mp4 OUTPUT: ffmpeg version N-54207-ge59fb3f Copyright (c) 2000-2013 the FFmpeg developers built on Jun 25 2013 21:55:00 with gcc 4.7.3 (GCC) configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp eex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo- amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs -- enable-libxvid --enable-zlib libavutil 52. 37.101 / 52. 37.101 libavcodec 55. 17.100 / 55. 17.100 libavformat 55. 10.100 / 55. 10.100 libavdevice 55. 2.100 / 55. 2.100 libavfilter 3. 77.101 / 3. 77.101 libswscale 2. 3.100 / 2. 3.100 libswresample 0. 17.102 / 0. 17.102 libpostproc 52. 3.100 / 52. 3.100 [asf @ 00000000002ed760] Stream #0: not enough frames to estimate rate; consider increasing probesize Guessed Channel Layout for Input Stream #0.0 : stereo Input #0, asf, from 'C:\vidtests\Wildlife.wmv' : Metadata: SfOriginalFPS : 299700 WMFSDKVersion : 11.0.6001.7000 WMFSDKNeeded : 0.0.0.0000 comment : Footage: Small World Productions, Inc; Tourism New Zealand | Producer: Gary F. Spradling | Music: Steve Ball title : Wildlife in HD copyright : -¬ 2008 Microsoft Corporation IsVBR : 0 DeviceConformanceTemplate: AP@L3 Duration: 00:00:30.09, start: 0.000000, bitrate: 6977 kb/s Stream #0:0(eng): Audio: wmav2 (a[1][0][0] / 0x0161), 44100 Hz, stereo, fltp , 192 kb/s Stream #0:1(eng): Video: vc1 (Advanced) (WVC1 / 0x31435657), yuv420p, 1280x7 20, 5942 kb/s, 29.97 tbr, 1k tbn, 1k tbc [libx264 @ 00000000002e6980] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 [libx264 @ 00000000002e6980] profile High, level 3.0 [libx264 @ 00000000002e6980] 264 - core 133 r2334 a3ac64b - H.264/MPEG-4 AVC cod ec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 r ef=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed _ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pski p=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 deci mate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_ adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=2 5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.6 0 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'C:\vidtests\Wildlife.mp4': Metadata: SfOriginalFPS : 299700 WMFSDKVersion : 11.0.6001.7000 WMFSDKNeeded : 0.0.0.0000 comment : Footage: Small World Productions, Inc; Tourism New Zealand | Producer: Gary F. Spradling | Music: Steve Ball title : Wildlife in HD copyright : -¬ 2008 Microsoft Corporation IsVBR : 0 DeviceConformanceTemplate: AP@L3 encoder : Lavf55.10.100 Stream #0:0(eng): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 6 40x480, q=-1--1, 30k tbn, 29.97 tbc Stream #0:1(eng): Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16, 128 kb/s Stream mapping: Stream #0:1 -> #0:0 (vc1 -> libx264) Stream #0:0 -> #0:1 (wmav2 -> libvo_aacenc) Press [q] to stop, [?] for help frame= 53 fps= 49 q=29.0 size= 0kB time=00:00:00.13 bitrate= 2.9kbits/ frame= 63 fps= 40 q=29.0 size= 0kB time=00:00:00.46 bitrate= 0.8kbits/ frame= 74 fps= 35 q=29.0 size= 0kB time=00:00:00.83 bitrate= 0.5kbits/ frame= 85 fps= 32 q=29.0 size= 0kB time=00:00:01.20 bitrate= 0.3kbits/ frame= 95 fps= 30 q=29.0 size= 0kB time=00:00:01.53 bitrate= 0.3kbits/ frame= 107 fps= 28 q=29.0 size= 0kB time=00:00:01.93 bitrate= 0.2kbits/ Queue input is backward in time [mp4 @ 00000000003ef800] Non-monotonous DTS in output stream 0:1; previous: 7616 , current: 7063; changing to 7617. This may result in incorrect timestamps in th e output file. frame= 118 fps= 28 q=29.0 size= 113kB time=00:00:02.30 bitrate= 402.6kbits/ frame= 129 fps= 26 q=29.0 size= 219kB time=00:00:02.66 bitrate= 670.7kbits/ frame= 141 fps= 26 q=29.0 size= 264kB time=00:00:03.06 bitrate= 704.2kbits/ frame= 152 fps= 25 q=29.0 size= 328kB time=00:00:03.43 bitrate= 782.2kbits/ frame= 163 fps= 25 q=29.0 size= 431kB time=00:00:03.80 bitrate= 928.1kbits/ frame= 174 fps= 24 q=29.0 size= 568kB time=00:00:04.17 bitrate=1116.3kbits/ frame= 190 fps= 25 q=29.0 size= 781kB time=00:00:04.70 bitrate=1359.9kbits/ frame= 204 fps= 25 q=29.0 size= 1006kB time=00:00:05.17 bitrate=1593.1kbits/ frame= 218 fps= 25 q=29.0 size= 1058kB time=00:00:05.63 bitrate=1536.8kbits/ frame= 229 fps= 25 q=29.0 size= 1093kB time=00:00:06.00 bitrate=1490.9kbits/ frame= 239 fps= 24 q=29.0 size= 1118kB time=00:00:06.33 bitrate=1444.4kbits/ frame= 251 fps= 24 q=29.0 size= 1150kB time=00:00:06.74 bitrate=1397.9kbits/ frame= 265 fps= 24 q=29.0 size= 1234kB time=00:00:07.20 bitrate=1402.3kbits/ frame= 278 fps= 25 q=29.0 size= 1332kB time=00:00:07.64 bitrate=1428.3kbits/ frame= 294 fps= 25 q=29.0 size= 1403kB time=00:00:08.17 bitrate=1405.7kbits/ frame= 308 fps= 25 q=29.0 size= 1547kB time=00:00:08.64 bitrate=1466.4kbits/ frame= 323 fps= 25 q=29.0 size= 1595kB time=00:00:09.14 bitrate=1429.5kbits/ frame= 337 fps= 25 q=29.0 size= 1702kB time=00:00:09.60 bitrate=1450.7kbits/ frame= 351 fps= 25 q=29.0 size= 1755kB time=00:00:10.07 bitrate=1427.1kbits/ frame= 365 fps= 25 q=29.0 size= 1820kB time=00:00:10.54 bitrate=1414.1kbits/ frame= 381 fps= 25 q=29.0 size= 1852kB time=00:00:11.07 bitrate=1369.6kbits/ frame= 396 fps= 26 q=29.0 size= 1893kB time=00:00:11.57 bitrate=1339.5kbits/ frame= 409 fps= 26 q=29.0 size= 1923kB time=00:00:12.01 bitrate=1311.8kbits/ frame= 421 fps= 25 q=29.0 size= 1967kB time=00:00:12.41 bitrate=1298.3kbits/ frame= 434 fps= 25 q=29.0 size= 1998kB time=00:00:12.84 bitrate=1274.0kbits/ frame= 445 fps= 25 q=29.0 size= 2018kB time=00:00:13.21 bitrate=1251.3kbits/ frame= 458 fps= 25 q=29.0 size= 2048kB time=00:00:13.64 bitrate=1229.5kbits/ frame= 471 fps= 25 q=29.0 size= 2067kB time=00:00:14.08 bitrate=1202.3kbits/ frame= 484 fps= 25 q=29.0 size= 2189kB time=00:00:14.51 bitrate=1235.5kbits/ frame= 497 fps= 25 q=29.0 size= 2260kB time=00:00:14.94 bitrate=1238.3kbits/ frame= 509 fps= 25 q=29.0 size= 2311kB time=00:00:15.34 bitrate=1233.3kbits/ frame= 523 fps= 25 q=29.0 size= 2429kB time=00:00:15.81 bitrate=1258.1kbits/ frame= 535 fps= 25 q=29.0 size= 2541kB time=00:00:16.21 bitrate=1283.5kbits/ frame= 548 fps= 25 q=29.0 size= 2718kB time=00:00:16.64 bitrate=1337.5kbits/ frame= 560 fps= 25 q=29.0 size= 2845kB time=00:00:17.05 bitrate=1367.1kbits/ frame= 571 fps= 25 q=29.0 size= 2965kB time=00:00:17.41 bitrate=1394.6kbits/ frame= 580 fps= 25 q=29.0 size= 3025kB time=00:00:17.71 bitrate=1398.7kbits/ frame= 588 fps= 25 q=29.0 size= 3098kB time=00:00:17.98 bitrate=1411.1kbits/ frame= 597 fps= 25 q=29.0 size= 3183kB time=00:00:18.28 bitrate=1426.1kbits/ frame= 606 fps= 24 q=29.0 size= 3279kB time=00:00:18.58 bitrate=1445.2kbits/ frame= 616 fps= 24 q=29.0 size= 3441kB time=00:00:18.91 bitrate=1489.9kbits/ frame= 626 fps= 24 q=29.0 size= 3650kB time=00:00:19.25 bitrate=1553.0kbits/ frame= 638 fps= 24 q=29.0 size= 3826kB time=00:00:19.65 bitrate=1594.7kbits/ frame= 649 fps= 24 q=29.0 size= 3950kB time=00:00:20.02 bitrate=1616.3kbits/ frame= 660 fps= 24 q=29.0 size= 4067kB time=00:00:20.38 bitrate=1634.1kbits/ frame= 669 fps= 24 q=29.0 size= 4121kB time=00:00:20.68 bitrate=1631.8kbits/ frame= 682 fps= 24 q=29.0 size= 4274kB time=00:00:21.12 bitrate=1657.9kbits/ frame= 696 fps= 24 q=29.0 size= 4446kB time=00:00:21.58 bitrate=1687.1kbits/ frame= 709 fps= 24 q=29.0 size= 4590kB time=00:00:22.02 bitrate=1707.3kbits/ frame= 719 fps= 24 q=29.0 size= 4772kB time=00:00:22.35 bitrate=1748.5kbits/ frame= 732 fps= 24 q=29.0 size= 4852kB time=00:00:22.78 bitrate=1744.3kbits/ frame= 744 fps= 24 q=29.0 size= 4973kB time=00:00:23.18 bitrate=1756.9kbits/ frame= 756 fps= 24 q=29.0 size= 5099kB time=00:00:23.59 bitrate=1770.8kbits/ frame= 768 fps= 24 q=29.0 size= 5149kB time=00:00:23.99 bitrate=1758.4kbits/ frame= 780 fps= 24 q=29.0 size= 5227kB time=00:00:24.39 bitrate=1755.7kbits/ frame= 797 fps= 24 q=29.0 size= 5377kB time=00:00:24.95 bitrate=1765.0kbits/ frame= 813 fps= 24 q=29.0 size= 5507kB time=00:00:25.49 bitrate=1769.5kbits/ frame= 828 fps= 24 q=29.0 size= 5634kB time=00:00:25.99 bitrate=1775.5kbits/ frame= 843 fps= 24 q=29.0 size= 5701kB time=00:00:26.49 bitrate=1762.9kbits/ frame= 859 fps= 24 q=29.0 size= 5830kB time=00:00:27.02 bitrate=1767.0kbits/ frame= 872 fps= 24 q=29.0 size= 5926kB time=00:00:27.46 bitrate=1767.7kbits/ frame= 888 fps= 24 q=29.0 size= 6014kB time=00:00:27.99 bitrate=1759.7kbits/ frame= 900 fps= 24 q=29.0 size= 6332kB time=00:00:28.39 bitrate=1826.9kbits/ frame= 901 fps= 24 q=-1.0 Lsize= 6717kB time=00:00:30.10 bitrate=1828.0kbits /s video:6211kB audio:472kB subtitle:0 global headers:0kB muxing overhead 0.513217% [libx264 @ 00000000002e6980] frame I:8 Avg QP:21.77 size: 39744 [libx264 @ 00000000002e6980] frame P:433 Avg QP:25.69 size: 11490 [libx264 @ 00000000002e6980] frame B:460 Avg QP:29.25 size: 2319 [libx264 @ 00000000002e6980] consecutive B-frames: 5.4% 78.6% 2.7% 13.3% [libx264 @ 00000000002e6980] mb I I16..4: 21.8% 48.8% 29.5% [libx264 @ 00000000002e6980] mb P I16..4: 0.7% 4.0% 1.3% P16..4: 37.1% 22.2 % 15.5% 0.0% 0.0% skip:19.2% [libx264 @ 00000000002e6980] mb B I16..4: 0.1% 0.5% 0.2% B16..8: 43.5% 7.0 % 2.1% direct: 2.2% skip:44.5% L0:36.4% L1:52.7% BI:10.9% [libx264 @ 00000000002e6980] 8x8 transform intra:62.8% inter:56.2% [libx264 @ 00000000002e6980] coded y,uvDC,uvAC intra: 74.2% 78.8% 44.0% inter: 2 3.6% 14.5% 1.0% [libx264 @ 00000000002e6980] i16 v,h,dc,p: 48% 24% 9% 20% [libx264 @ 00000000002e6980] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 17% 15% 7% 8% 11% 8% 10% 8% [libx264 @ 00000000002e6980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 17% 15% 7% 10% 11% 8% 7% 7% [libx264 @ 00000000002e6980] i8c dc,h,v,p: 53% 21% 18% 7% [libx264 @ 00000000002e6980] Weighted P-Frames: Y:0.7% UV:0.0% [libx264 @ 00000000002e6980] ref P L0: 62.4% 19.0% 12.0% 6.6% 0.0% [libx264 @ 00000000002e6980] ref B L0: 90.5% 8.9% 0.7% [libx264 @ 00000000002e6980] ref B L1: 97.9% 2.1% [libx264 @ 00000000002e6980] kb/s:1692.37 AND THE –MOVFLAGS COMMAND: C:\XSITE\SITE>ffmpeg_new -i C:\vidtests\Wildlife.mp4 -movflags faststart C:\vidtests\Wildlife_fs.mp4 AND THE –MOVFLAGS OUTPUT ffmpeg version N-54207-ge59fb3f Copyright (c) 2000-2013 the FFmpeg developers built on Jun 25 2013 21:55:00 with gcc 4.7.3 (GCC) configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab le-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetyp e --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --ena ble-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-l ibopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libsp eex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo- amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs -- enable-libxvid --enable-zlib libavutil 52. 37.101 / 52. 37.101 libavcodec 55. 17.100 / 55. 17.100 libavformat 55. 10.100 / 55. 10.100 libavdevice 55. 2.100 / 55. 2.100 libavfilter 3. 77.101 / 3. 77.101 libswscale 2. 3.100 / 2. 3.100 libswresample 0. 17.102 / 0. 17.102 libpostproc 52. 3.100 / 52. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\vidtests\Wildlife.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 title : Wildlife in HD encoder : Lavf55.10.100 comment : Footage: Small World Productions, Inc; Tourism New Zealand | Producer: Gary F. Spradling | Music: Steve Ball copyright : -¬ 2008 Microsoft Corporation Duration: 00:00:30.13, start: 0.036281, bitrate: 1826 kb/s Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480, 1692 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc Metadata: handler_name : VideoHandler Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 12 8 kb/s Metadata: handler_name : SoundHandler [libx264 @ 0000000004360620] using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64 [libx264 @ 0000000004360620] profile High, level 3.0 [libx264 @ 0000000004360620] 264 - core 133 r2334 a3ac64b - H.264/MPEG-4 AVC cod ec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: cabac=1 r ef=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed _ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pski p=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 deci mate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_ adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=2 5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.6 0 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to 'C:\vidtests\Wildlife_fs.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 title : Wildlife in HD copyright : -¬ 2008 Microsoft Corporation comment : Footage: Small World Productions, Inc; Tourism New Zealand | Producer: Gary F. Spradling | Music: Steve Ball encoder : Lavf55.10.100 Stream #0:0(eng): Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 6 40x480, q=-1--1, 30k tbn, 29.97 tbc Metadata: handler_name : VideoHandler Stream #0:1(eng): Audio: aac (libvo_aacenc) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, s16, 128 kb/s Metadata: handler_name : SoundHandler Stream mapping: Stream #0:0 -> #0:0 (h264 -> libx264) Stream #0:1 -> #0:1 (aac -> libvo_aacenc) Press [q] to stop, [?] for help frame= 52 fps=0.0 q=29.0 size= 29kB time=00:00:01.76 bitrate= 133.9kbits/ frame= 63 fps= 60 q=29.0 size= 104kB time=00:00:02.14 bitrate= 397.2kbits/ frame= 74 fps= 47 q=29.0 size= 176kB time=00:00:02.51 bitrate= 573.2kbits/ frame= 87 fps= 41 q=29.0 size= 265kB time=00:00:02.93 bitrate= 741.2kbits/ frame= 101 fps= 37 q=29.0 size= 358kB time=00:00:03.39 bitrate= 862.8kbits/ frame= 113 fps= 34 q=29.0 size= 437kB time=00:00:03.79 bitrate= 943.7kbits/ frame= 125 fps= 33 q=29.0 size= 520kB time=00:00:04.20 bitrate=1012.2kbits/ frame= 138 fps= 32 q=29.0 size= 606kB time=00:00:04.64 bitrate=1069.8kbits/ frame= 151 fps= 31 q=29.0 size= 696kB time=00:00:05.06 bitrate=1124.3kbits/ frame= 163 fps= 30 q=29.0 size= 780kB time=00:00:05.47 bitrate=1166.4kbits/ frame= 176 fps= 30 q=29.0 size= 919kB time=00:00:05.90 bitrate=1273.9kbits/ frame= 196 fps= 31 q=29.0 size= 994kB time=00:00:06.57 bitrate=1237.4kbits/ frame= 213 fps= 31 q=29.0 size= 1097kB time=00:00:07.13 bitrate=1258.8kbits/ frame= 225 fps= 30 q=29.0 size= 1204kB time=00:00:07.53 bitrate=1309.8kbits/ frame= 236 fps= 30 q=29.0 size= 1323kB time=00:00:07.91 bitrate=1369.4kbits/ frame= 249 fps= 29 q=29.0 size= 1451kB time=00:00:08.34 bitrate=1424.6kbits/ frame= 263 fps= 29 q=29.0 size= 1574kB time=00:00:08.82 bitrate=1461.3kbits/ frame= 278 fps= 29 q=29.0 size= 1610kB time=00:00:09.30 bitrate=1416.9kbits/ frame= 296 fps= 30 q=29.0 size= 1655kB time=00:00:09.91 bitrate=1368.0kbits/ frame= 313 fps= 30 q=29.0 size= 1697kB time=00:00:10.48 bitrate=1326.4kbits/ frame= 330 fps= 30 q=29.0 size= 1737kB time=00:00:11.05 bitrate=1286.5kbits/ frame= 345 fps= 30 q=29.0 size= 1776kB time=00:00:11.54 bitrate=1260.4kbits/ frame= 361 fps= 30 q=29.0 size= 1813kB time=00:00:12.07 bitrate=1230.3kbits/ frame= 377 fps= 30 q=29.0 size= 1847kB time=00:00:12.59 bitrate=1201.4kbits/ frame= 395 fps= 30 q=29.0 size= 1880kB time=00:00:13.22 bitrate=1165.0kbits/ frame= 410 fps= 30 q=29.0 size= 1993kB time=00:00:13.72 bitrate=1190.2kbits/ frame= 424 fps= 30 q=29.0 size= 2080kB time=00:00:14.18 bitrate=1201.4kbits/ frame= 439 fps= 30 q=29.0 size= 2166kB time=00:00:14.67 bitrate=1209.4kbits/ frame= 455 fps= 30 q=29.0 size= 2262kB time=00:00:15.21 bitrate=1217.5kbits/ frame= 469 fps= 30 q=29.0 size= 2341kB time=00:00:15.68 bitrate=1223.0kbits/ frame= 484 fps= 30 q=29.0 size= 2430kB time=00:00:16.19 bitrate=1229.1kbits/ frame= 500 fps= 30 q=29.0 size= 2523kB time=00:00:16.71 bitrate=1236.3kbits/ frame= 515 fps= 30 q=29.0 size= 2607kB time=00:00:17.21 bitrate=1240.4kbits/ frame= 531 fps= 30 q=29.0 size= 2681kB time=00:00:17.73 bitrate=1238.2kbits/ frame= 546 fps= 30 q=29.0 size= 2758kB time=00:00:18.24 bitrate=1238.2kbits/ frame= 561 fps= 30 q=29.0 size= 2824kB time=00:00:18.75 bitrate=1233.4kbits/ frame= 576 fps= 30 q=29.0 size= 2955kB time=00:00:19.25 bitrate=1256.8kbits/ frame= 586 fps= 29 q=29.0 size= 3061kB time=00:00:19.59 bitrate=1279.6kbits/ frame= 598 fps= 29 q=29.0 size= 3217kB time=00:00:19.99 bitrate=1318.4kbits/ frame= 610 fps= 29 q=29.0 size= 3354kB time=00:00:20.39 bitrate=1347.2kbits/ frame= 622 fps= 29 q=29.0 size= 3483kB time=00:00:20.78 bitrate=1372.6kbits/ frame= 634 fps= 29 q=29.0 size= 3593kB time=00:00:21.19 bitrate=1388.6kbits/ frame= 648 fps= 29 q=29.0 size= 3708kB time=00:00:21.66 bitrate=1402.3kbits/ frame= 661 fps= 29 q=29.0 size= 3811kB time=00:00:22.08 bitrate=1413.5kbits/ frame= 674 fps= 29 q=29.0 size= 3978kB time=00:00:22.53 bitrate=1446.3kbits/ frame= 690 fps= 29 q=29.0 size= 4133kB time=00:00:23.05 bitrate=1468.4kbits/ frame= 706 fps= 29 q=29.0 size= 4263kB time=00:00:23.58 bitrate=1480.4kbits/ frame= 721 fps= 29 q=29.0 size= 4391kB time=00:00:24.08 bitrate=1493.8kbits/ frame= 735 fps= 29 q=29.0 size= 4524kB time=00:00:24.55 bitrate=1509.4kbits/ frame= 748 fps= 29 q=29.0 size= 4661kB time=00:00:24.98 bitrate=1528.2kbits/ frame= 763 fps= 29 q=29.0 size= 4835kB time=00:00:25.50 bitrate=1553.1kbits/ frame= 778 fps= 29 q=29.0 size= 4993kB time=00:00:25.99 bitrate=1573.6kbits/ frame= 795 fps= 29 q=29.0 size= 5149kB time=00:00:26.56 bitrate=1588.1kbits/ frame= 814 fps= 29 q=29.0 size= 5258kB time=00:00:27.18 bitrate=1584.4kbits/ frame= 833 fps= 29 q=29.0 size= 5368kB time=00:00:27.82 bitrate=1580.2kbits/ frame= 851 fps= 29 q=29.0 size= 5469kB time=00:00:28.43 bitrate=1575.9kbits/ frame= 870 fps= 29 q=29.0 size= 5567kB time=00:00:29.05 bitrate=1569.5kbits/ frame= 889 fps= 29 q=29.0 size= 5688kB time=00:00:29.70 bitrate=1568.4kbits/ Starting second pass: moving header on top of the file frame= 902 fps= 28 q=-1.0 Lsize= 6109kB time=00:00:30.14 bitrate=1659.8kbits /s dup=1 drop=0 video:5602kB audio:472kB subtitle:0 global headers:0kB muxing overhead 0.566600% [libx264 @ 0000000004360620] frame I:8 Avg QP:20.52 size: 39667 [libx264 @ 0000000004360620] frame P:419 Avg QP:25.06 size: 10524 [libx264 @ 0000000004360620] frame B:475 Avg QP:29.03 size: 2123 [libx264 @ 0000000004360620] consecutive B-frames: 3.2% 79.6% 0.3% 16.9% [libx264 @ 0000000004360620] mb I I16..4: 20.7% 52.3% 26.9% [libx264 @ 0000000004360620] mb P I16..4: 0.7% 4.2% 1.1% P16..4: 39.4% 21.4 % 13.8% 0.0% 0.0% skip:19.3% [libx264 @ 0000000004360620] mb B I16..4: 0.1% 0.9% 0.3% B16..8: 41.8% 6.4 % 1.7% direct: 1.7% skip:47.1% L0:36.4% L1:53.3% BI:10.3% [libx264 @ 0000000004360620] 8x8 transform intra:65.7% inter:58.8% [libx264 @ 0000000004360620] coded y,uvDC,uvAC intra: 71.2% 76.6% 35.7% inter: 2 0.7% 13.0% 0.5% [libx264 @ 0000000004360620] i16 v,h,dc,p: 48% 24% 8% 20% [libx264 @ 0000000004360620] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 18% 15% 6% 8% 11% 8% 10% 8% [libx264 @ 0000000004360620] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 16% 15% 7% 10% 11% 8% 8% 7% [libx264 @ 0000000004360620] i8c dc,h,v,p: 51% 22% 19% 9% [libx264 @ 0000000004360620] Weighted P-Frames: Y:0.7% UV:0.0% [libx264 @ 0000000004360620] ref P L0: 63.4% 19.7% 11.0% 5.9% 0.0% [libx264 @ 0000000004360620] ref B L0: 90.7% 8.7% 0.7% [libx264 @ 0000000004360620] ref B L1: 98.4% 1.6% [libx264 @ 0000000004360620] kb/s:1524.54

    Read the article

  • OSX, G/AWK, Bash - "illegal statement"

    - by S1syphus
    I have a script that somebody from SO kindly provided to solve an issue I was having, However, I'm having some issues getting it to work on OSX. gawk --version GNU Awk 3.1.6 awk --version awk version 20100208 The original source is: awk -F, -vOFS=, -vc=1 ' NR == 1 { for (i=1; i<NF; i++) { if ($i != "") { g[c]=i; f[c++]=$i } } } NR>2 { for (i=1; i < c; i++) { print $1,$2, $g[i] > "output_"f[i]".csv } }' data.csv When I run the script it gives the following error: awk: syntax error at source line 12 context is print $1,$2, $g[i] > >>> "output_"f <<< [i]".csv awk: illegal statement at source line 13 From the look of it the variable of [i] isn't been amended to the output file, but I don't know why. If I change AWK to GAWK and run the original script here is the output: gawk: cmd. line:11: print $1,$2, $g[i] > "output_"f[i]".csv gawk: cmd. line:11: ^ unterminated string So I edit the relevant line to fix the unterminated string print $1,$2, $g[i] > "output_"f[i]".csv" Then it runs through fine produces no errors, but there is no output files. Any ideas? I spent the majority of last night and this morning pouring over this. A sample input file: ,,L1,,,L2,,,L3,,,L4,,,L5,,,L6,,,L7,,,L8,,,L9,,,L10,,,L11, Title,r/t,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,neede d,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst EXAMPLEfoo,60,6,6,6,0,0,0,0,0,0,6,6,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLEbar,30,6,6,12,6,7,14,6,6,12,6,6,12,6,8,16,6,7,14,6,7.5,15,6,6,12,6,8,16,6,0,0,6,7,14 EXAMPLE1,60,3,3,3,3,5,5,3,4,4,3,3,3,3,6,6,3,4,4,3,3,3,3,4,4,3,8,8,3,0,0,3,4,4 EXAMPLE2,120,6,6,3,0,0,0,6,8,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLE3,60,6,6,6,6,8,8,6,6,6,6,6,6,0,0,0,0,0,0,6,8,8,6,6,6,0,0,0,0,0,0,0,10,10 EXAMPLE4,30,6,6,12,6,7,14,6,6,12,6,6,12,3,5.5,11,6,7.5,15,6,6,12,6,0,0,6,9,18,6,0,0,6,6.5,13 And the example out put should be So for L1 an example out put would look like: EXAMPLEfoo,60,6 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,6 EXAMPLE3,60,6 EXAMPLE4,30,6 And for L2: EXAMPLEfoo,60,0 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,0 EXAMPLE3,60,6 EXAMPLE4,30,6

    Read the article

  • OSX, G/AWK, Bash - "illegal statement, unterminated string" and no file output.

    - by S1syphus
    I have a script that somebody from SO kindly provided to solve an issue I was having, However, I'm having some issues getting it to work on OSX. gawk --version GNU Awk 3.1.6 awk --version awk version 20100208 The original source is: awk -F, -vOFS=, -vc=1 ' NR == 1 { for (i=1; i<NF; i++) { if ($i != "") { g[c]=i; f[c++]=$i } } } NR>2 { for (i=1; i < c; i++) { print $1,$2, $g[i] > "output_"f[i]".csv } }' data.csv When I run the script it gives the following error: awk: syntax error at source line 12 context is print $1,$2, $g[i] > >>> "output_"f <<< [i]".csv awk: illegal statement at source line 13 From the look of it the variable of [i] isn't been amended to the output file, but I don't know why. If I change AWK to GAWK and run the original script here is the output: gawk: cmd. line:11: print $1,$2, $g[i] > "output_"f[i]".csv gawk: cmd. line:11: ^ unterminated string So I edit the relevant line to fix the unterminated string print $1,$2, $g[i] > "output_"f[i]".csv" Then it runs through fine produces no errors, but there is no output files. Any ideas? I spent the majority of last night and this morning pouring over this. A sample input file: ,,L1,,,L2,,,L3,,,L4,,,L5,,,L6,,,L7,,,L8,,,L9,,,L10,,,L11, Title,r/t,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,neede d,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst,needed,actual,Inst EXAMPLEfoo,60,6,6,6,0,0,0,0,0,0,6,6,6,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLEbar,30,6,6,12,6,7,14,6,6,12,6,6,12,6,8,16,6,7,14,6,7.5,15,6,6,12,6,8,16,6,0,0,6,7,14 EXAMPLE1,60,3,3,3,3,5,5,3,4,4,3,3,3,3,6,6,3,4,4,3,3,3,3,4,4,3,8,8,3,0,0,3,4,4 EXAMPLE2,120,6,6,3,0,0,0,6,8,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 EXAMPLE3,60,6,6,6,6,8,8,6,6,6,6,6,6,0,0,0,0,0,0,6,8,8,6,6,6,0,0,0,0,0,0,0,10,10 EXAMPLE4,30,6,6,12,6,7,14,6,6,12,6,6,12,3,5.5,11,6,7.5,15,6,6,12,6,0,0,6,9,18,6,0,0,6,6.5,13 And the example out put should be So for L1 an example out put would look like: EXAMPLEfoo,60,6 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,6 EXAMPLE3,60,6 EXAMPLE4,30,6 And for L2: EXAMPLEfoo,60,0 EXAMPLEbar,30,6 EXAMPLE1,60,3 EXAMPLE2,120,0 EXAMPLE3,60,6 EXAMPLE4,30,6

    Read the article

  • Create matplotlib legend out of the figure

    - by Werner
    I added the legend this way: leg = fig.legend((l0,l1,l2,l3,l4,l5,l6), ('0 Cl : r2, slope, origin', '1 Cl :'+str(r1b)+' , '+str(m1)+' , '+str(b1), '2 Cl :'+str(r2b)+' , '+str(m2)+' , '+str(b2), '3 Cl :'+str(r3b)+' , '+str(m3)+' , '+str(b3), '4 Cl :'+str(r4b)+' , '+str(m4)+' , '+str(b4), '5 Cl :'+str(r5b)+' , '+str(m5)+' , '+str(b5), '6 Cl :'+str(r6b)+' , '+str(m6)+' , '+str(b6), ), 'upper right') but the legend appears inside the plot. How can I tell matplotlib to put it to the right of the plot and at the right?

    Read the article

  • Can processor cores thrash each other's caches?

    - by Jørgen Fogh
    If more than one core on a processor is accessing the same memory address, will they thrash each other's caches or will some snooping protocol allow each to keep the data in L1-cache? I am interested in a general answer as well as answers for specific processors. How many layers of cache are invalidated? Will accessing another address within the same cache-line invalidate the entire line? What can you do to alleviate these problems?

    Read the article

  • Look for match in a nested list in Python

    - by elfuego1
    Hello everybody, I have two nested lists of different sizes: A = [[1, 7, 3, 5], [5, 5, 14, 10]] B = [[1, 17, 3, 5], [1487, 34, 14, 74], [1487, 34, 3, 87], [141, 25, 14, 10]] I'd like to gather all nested lists from list B if A[2:4] == B[2:4] and put it into list L: L = [[1, 17, 3, 5], [141, 25, 14, 10]] Additionally if the match occurs then I want to change last element of sublist B into first element of sublist A so the final solution would look like this: L1 = [[1, 17, 3, 1], [141, 25, 14, 5]]

    Read the article

  • in Python find number of same elements in 2 lists

    - by John
    Hi, In Python if I have 2 lists say: l1 = ['a', 'b', 'c', 'd'] l2 = ['c', 'd', 'e'] is there a way to find out how many elements they have the same. In the case about it would be 2 (c and d) I know I could just do a nested loop but is there not a built in function like in php with the array_intersect function Thanks

    Read the article

  • Approximate timings for various operations on a "typical desktop PC" anno 2010

    - by knorv
    In the article "Teach Yourself Programming in Ten Years" Peter Norvig (Director of Research, Google) gives the following approximate timings for various operations on a typical 1GHz PC back in 2001: execute single instruction = 1 nanosec = (1/1,000,000,000) sec fetch word from L1 cache memory = 2 nanosec fetch word from main memory = 10 nanosec fetch word from consecutive disk location = 200 nanosec fetch word from new disk location (seek) = 8,000,000 nanosec = 8 millisec What would the corresponding timings be for your definition of a typical PC desktop anno 2010?

    Read the article

  • passing back answers in prolog

    - by AhmadAssaf
    i have this code than runs perfectly .. returns a true .. when tracing the values are ok .. but its not returning back the answer .. it acts strangely when it ends and always return empty list .. uninstantiated variable .. test :- extend(4,12,[4,3,1,2],[[1,5],[3,4],[6]],_ExtendedBins). %printing basic information about the extend(NumBins,Capacity,RemainingNumbers,BinsSoFar,_ExtendedBins) :- getNumberofBins(BinsSoFar,NumberOfBins), msort(RemainingNumbers,SortedRemaining),nl, format("Current Number of Bins is :~w\n",[NumberOfBins]), format("Allowed Capacity is :~w\n",[Capacity]), format("maximum limit in bin is :~w\n",[NumBins]), format("Trying to fit :~w\n\n",[SortedRemaining]), format("Possible Solutions :\n\n"), fitElements(NumBins,NumberOfBins, Capacity,SortedRemaining,BinsSoFar,[]). %this is were the creation for possibilities will start %will check first if the number of bins allowed is less than then %we create a new list with all the possible combinations %after that we start matching to other bins with capacity constraint fitElements(NumBins,NumberOfBins, Capacity,RemainingNumbers,Bins,ExtendedBins) :- ( NumberOfBins < NumBins -> print('Creating new set: '); print('Sorry, Cannot create New Sets')), createNewList(Capacity,RemainingNumbers,Bins,ExtendedBins). createNewList(Capacity,RemainingNumbers,Bins,ExtendedBins) :- createNewList(Capacity,RemainingNumbers,Bins,[],ExtendedBins), print(ExtendedBins). createNewList(0,Bins,Bins,ExtendedBins,ExtendedBins). createNewList(_,[],_,ExtendedBins,ExtendedBins). createNewList(Capacity,[Element|Rest],Bins,Temp,ExtendedBins) :- conjunct_to_list(Element,ListedElement), append(ListedElement,Temp,NewList), sumlist(NewList,Sum), (Sum =< Capacity, append(ListedElement,ExtendedBins,Result); Capacity = 0), createNewList(Capacity,Rest,Bins,NewList,Result). fit(0,[],ExtendedBins,ExtendedBins). fit(Capacity,[Element|Rest],Bin,ExtendedBins) :- conjunct_to_list(Element,Listed), append(Listed,Bin,NewBin), sumlist(NewBin,Sum), (Sum =< Capacity -> fit(Capacity,Rest,NewBin,ExtendedBins); Capacity = 0, append(NewBin,ExtendedBins,NewExtendedBins), print(NewExtendedBins), fit(0,[],NewBin,ExtendedBins)). %get the number of bins provided getNumberofBins(List,NumberOfBins) :- getNumberofBins(List,0,NumberOfBins). getNumberofBins([],NumberOfBins,NumberOfBins). getNumberofBins([_List|Rest],TempCount,NumberOfBins) :- NewCount is TempCount + 1, %calculate the count getNumberofBins(Rest,NewCount,NumberOfBins). %recursive call %Convert set of terms into a list - used when needed to append conjunct_to_list((A,B), L) :- !, conjunct_to_list(A, L0), conjunct_to_list(B, L1), append(L0, L1, L). conjunct_to_list(A, [A]). Greatly appreciate the help

    Read the article

< Previous Page | 1 2 3 4 5 6 7 8 9 10  | Next Page >