xsltproc killed, out of memory
- by David Parks
I'm trying to split up a 13GB xml file into small ~50MB xml files with this XSLT style sheet.
But this process kills xsltproc after I see it taking up over 1.7GB of memory (that's the total on the system).
Is there any way to deal with huge XML files with xsltproc? Can I change my style sheet? Or should I use a different processor? Or am I just S.O.L.?
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
xmlns:exsl="http://exslt.org/common"
extension-element-prefixes="exsl"
xmlns:fn="http://www.w3.org/2005/xpath-functions">
<xsl:output method="xml" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:param name="block-size" select="75000"/>
<xsl:template match="/">
<xsl:copy>
<xsl:apply-templates select="mysqldump/database/table_data/row[position() mod $block-size = 1]" />
</xsl:copy>
</xsl:template>
<xsl:template match="row">
<exsl:document href="chunk-{position()}.xml">
<add>
<xsl:for-each select=". | following-sibling::row[position() < $block-size]" >
<doc>
<xsl:for-each select="field">
<field>
<xsl:attribute name="name"><xsl:value-of select="./@name"/></xsl:attribute>
<xsl:value-of select="."/>
</field>
<xsl:text>
</xsl:text>
</xsl:for-each>
</doc>
</xsl:for-each>
</add>
</exsl:document>
</xsl:template>
