xsltproc killed, out of memory
Posted
by
David Parks
on Ask Ubuntu
See other posts from Ask Ubuntu
or by David Parks
Published on 2012-10-20T02:43:48Z
Indexed on
2012/10/20
5:22 UTC
Read the original article
Hit count: 406
I'm trying to split up a 13GB xml file into small ~50MB xml files with this XSLT style sheet.
But this process kills xsltproc after I see it taking up over 1.7GB of memory (that's the total on the system).
Is there any way to deal with huge XML files with xsltproc? Can I change my style sheet? Or should I use a different processor? Or am I just S.O.L.?
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"
xmlns:exsl="http://exslt.org/common"
extension-element-prefixes="exsl"
xmlns:fn="http://www.w3.org/2005/xpath-functions">
<xsl:output method="xml" indent="yes"/>
<xsl:strip-space elements="*"/>
<xsl:param name="block-size" select="75000"/>
<xsl:template match="/">
<xsl:copy>
<xsl:apply-templates select="mysqldump/database/table_data/row[position() mod $block-size = 1]" />
</xsl:copy>
</xsl:template>
<xsl:template match="row">
<exsl:document href="chunk-{position()}.xml">
<add>
<xsl:for-each select=". | following-sibling::row[position() < $block-size]" >
<doc>
<xsl:for-each select="field">
<field>
<xsl:attribute name="name"><xsl:value-of select="./@name"/></xsl:attribute>
<xsl:value-of select="."/>
</field>
<xsl:text>
</xsl:text>
</xsl:for-each>
</doc>
</xsl:for-each>
</add>
</exsl:document>
</xsl:template>
© Ask Ubuntu or respective owner