Home » RADICORE development » XML+XSL matters » xsl transformation slow
 xsl transformation slow [message #281] |
Wed, 20 September 2006 16:49  |
semcycle
Messages: 8
Registered: June 2006
Location: Michigan, USA
|
Junior Member |
|
|
a list screen of 100 records times out on a P3 server unless you allow the script to run longer than 30 seconds. then the page comes up with (XSLT = 34+ seconds)
to demonstrate this i have added records to the person database of the radicore demo
http://www.radicore.org/demo/
if you go to the person_list screen and hit the list 100 link you end up with a lost network connection because the script times out.
is there something in the xsl transformation that can be optimized ?
the transformation should not be so hard on the processor, should it
Sem Abrahams
www.semcycle.com
|
|
|
|
|
|
|
|
| Re: xsl transformation slow [message #692 is a reply to message #281] |
Mon, 19 March 2007 01:13  |
braingeyser
Messages: 17
Registered: November 2006
Location: Melbourne Australia
|
Junior Member |
|
|
After some further testing I've found the following:
a)The speed DECREASE due to record size is not linear, seems to be more like a squared increase in time. i.e. double the records = quadruple the time.
b) The speed INCREASE is consistent regardless of recordset size for a given table. It's as big as 400% on some tables in my test db. 15 seconds down from from over 60.
c)The size of the individual records also seems to affect it as much as the size of the recordset. This is probably a bigger worry than the large recordsets, since a user can't change the size of a record.
This would suggest one further optimisation from a developers perspective may be to limit the fields returned from a query if they're not all displayed on the list screen. i.e. hidden fields.
For example if you have 6 fields in a table, 3 of which are hidden(as many are in the radicore example system, mostly dates), then instead of the default ...
which selects all fields in the table including hidden ones, you could perhaps have something like...
$sql_select = field1, field2, field2;
where fields 4, 5 & 6 are hidden and not displayed anyway. Naturally this requires more maintenance, so you'd need to weigh up the benefits of quicker display of larger recordsets versus the extra maintenance.
One thing I noticed during my testing was that the number of pages that benefited from changing 2 lines of code was huge. Every single page that shows multiple records in a horizontal format was affected. Further testament to Radicore's very cool design.
â
[Updated on: Mon, 19 March 2007 02:03] Report message to a moderator |
|
|
|
| Re: xsl transformation slow [message #693 is a reply to message #281] |
Mon, 19 March 2007 01:59 |
braingeyser
Messages: 17
Registered: November 2006
Location: Melbourne Australia
|
Junior Member |
|
|
BTW, the server performing these transformations is a remote host and is running php4.
When I asked if there were any plans to upgrade to php5, well... their answer was polite, but you could tell they were laughing. Seems Tony's not the only who doesn't like php5's "improvements". 
â
[Updated on: Mon, 19 March 2007 02:06] Report message to a moderator |
|
|
|
| Re: xsl transformation slow [message #694 is a reply to message #281] |
Mon, 19 March 2007 03:25 |
braingeyser
Messages: 17
Registered: November 2006
Location: Melbourne Australia
|
Junior Member |
|
|
OK, I've found one further significant increase in speed, but it's entirely dependant on the answer to this question...
Is the relative position of the <structure> tag in any xml document that uses the "display_horizontal" template in it's xsl transformation ever going to be anything other than /root/structure ? If so then never mind.
If not then read on...
Again, the use of "//" has crippled the performance of the transformation. The xsl:for-each loop I referred to in my original post has the following select criteria...
<xsl:for-each select="//structure/*[name()=$zone]/row/cell[@field]">
what this selection does is to search the entire xml document,including the table data, for a <structure> tag at any level however deeply buried, before applying the remaining selection criteria of...
/*[name()=$zone]/row/cell[@field]
Now for smaller documents (i.e. recordsets) this is not an issue, but for very large documents the overhead is enormous and totally unneccesary since the table data, which is the vast majority of the information in the document, will never contain the <structure> tag.
If the relative position of the <structure> tag never changes, or at least is known at runtime, then the for_each criteria can be changed to this...
<xsl:for-each select="/root/structure/*[name()=$zone]/row/cell[@field]">
This allows the transformation engine to go directly to the to the <structure> tag without searching the entire document. It's like the difference between giving someone specific street directions to your house or simply telling them the suburb you live in and letting them work it out from there.
The effect of this is to give a further 200% increase in speed for a total of almost a whopping 600%. Yes, that's not a typo, nearly a 6-fold improvement in xsl transformation time. The transformation times of my largest tables for 100 records are now about 12 seconds down from over 60. Needless to say, smaller recordsets are now like greased lightning 
If this logic is also applied to the table_tmp variable created in my original post
<xsl:variable name="table_tmp" select="/root/*[name()=$table]" />
instead of
<xsl:variable name="table_tmp" select="//*[name()=$table]" />
then the transformation time drops again to 8 seconds. But I suspect the data structure for this might not be so fixed, so this one is of dubious validity.
I have no idea if any of these changes have ramifications for the rest of the system, only Tony would know that, but isn't it worth investigating a 6-fold increase in speed for bugger-all effort?
â
[Updated on: Mon, 19 March 2007 04:29] Report message to a moderator |
|
|
|
|
|
|
|
| Re: xsl transformation slow [message #698 is a reply to message #694] |
Mon, 19 March 2007 10:28 |
braingeyser
Messages: 17
Registered: November 2006
Location: Melbourne Australia
|
Junior Member |
|
|
| braingeyser wrote on Mon, 19 March 2007 03:25 |
If this logic is also applied to the table_tmp variable created in my original post
<xsl:variable name="table_tmp" select="/root/*[name()=$table]" />
instead of
<xsl:variable name="table_tmp" select="//*[name()=$table]" />
then the transformation time drops again to 8 seconds. But I suspect the data structure for this might not be so fixed, so this one is of dubious validity.
|
Ok scratch this one. As I suspected the data structure is not fixed sufficiently to make this work for all the places it's used. For example a link1, which has the link table data nested within the outer table data, making the above XPath incorrect for the nested data. The existing XPath is the better for keeping things generic.
If you wanted to make this work, you would need to pass in the context node from the external calling code. I tried this and it works fine, but it means adding an extra parameter in all of the places this template is called. Not a trivial amount of work I'd guess. Doing this makes the whole template a lot clearer though since you do away with most of the complicated xpaths.
If you add this...
<xsl:param name="context_node" />
then you don't need this...
<xsl:variable name="table_x" select="//*[name()=$table]" />
and you can replace this...
<xsl:variable name="field" select="$table_x[position()=$position]/*[name()=$fieldname]" />
with this...
<xsl:variable name="field" select="$context_node/*[name()=$fieldname]" />
The context node parameter (in the case of list1 for eg) comes from std.list1.xsl here...
<xsl:for-each select="//*[name()=$main][count(*)>0]">
!-- display all the fields in the current row -->
<xsl:call-template name="display_horizontal">
<xsl:with-param name="zone" select="'main'"/>
<xsl:with-param name="context_node" select="." />
</xsl:call-template>
</xsl:for-each>
This is the code that would need to be changed in every template call in order to make it work. It speeds things up because your'e no longer getting the same table data node twice, once in the outer calling loop and once more in the display_horizontal template.
The extra time gains seem to be in the order of double, which is not a lot more than you would already get by implmenting the previous changes. In my case, I gained an extra 4 or 5 seconds on my originally 60+ sec page load, now down to between 7 & 8.
â
[Updated on: Mon, 19 March 2007 10:57] Report message to a moderator |
|
|
|
|
|
|
|
| Re: xsl transformation slow [message #701 is a reply to message #281] |
Mon, 19 March 2007 11:18 |
braingeyser
Messages: 17
Registered: November 2006
Location: Melbourne Australia
|
Junior Member |
|
|
Just in case my ramblings have confused you a little, here is my final version of "display_horizontal"
<!-- display details horizontally -->
<xsl:template name="display_horizontal">
<xsl:param name="zone"/> <!-- could be 'main', 'inner', 'outer', etc -->
<xsl:param name="table_record" />
<xsl:param name="multiple"/> <!-- set this for more than one occurrence -->
<xsl:variable name="table" select="name()"/> <!-- current table name -->
<xsl:variable name="position" select="position()"/> <!-- current row within table -->
<tr>
<!-- set the row class to 'odd' or 'even' to determine the colour -->
<xsl:attribute name="class">
<xsl:choose>
<xsl:when test="position()mod 2">odd</xsl:when>
<xsl:otherwise>even</xsl:otherwise>
</xsl:choose>
</xsl:attribute>
<!-- step through the fields defined in the STRUCTURE element -->
<xsl:for-each select="/root/structure/*[name()=$zone]/row/cell[@field]">
<!-- get fieldname from the FIELD attribute -->
<xsl:variable name="fieldname" select="@field" />
<!-- select the field (identified in STRUCTURE) from the current row of the specified table -->
<xsl:variable name="field" select="$table_record/*[name()=$fieldname]" />
<td>
<!-- process the named field from the current row -->
<xsl:call-template name="datafield">
<xsl:with-param name="item" select="$field"/>
<xsl:with-param name="itemname" select="$fieldname"/>
<xsl:with-param name="path" select="$table"/>
<xsl:with-param name="multiple" select="$multiple"/>
<xsl:with-param name="position" select="$position"/>
</xsl:call-template>
</td>
</xsl:for-each>
</tr>
</xsl:template> <!-- display_horizontal -->
and this is the change needed in the calling scripts to make it work...
std.link1.xsl
<!-- process each non-empty row in the MAIN table of the XML file -->
<xsl:for-each select="//*[name()=$main][count(*)>0]">
<!-- display all the fields in the current row -->
<xsl:call-template name="display_horizontal">
<xsl:with-param name="zone" select="'main'"/>
<xsl:with-param name="table_record" select="." /> <!-- ADD THIS LINE TO ALL CALLING SCRIPTS -->
</xsl:call-template>
</xsl:for_each>
similar change in std.link1.xsl
<!-- process each non-empty row in the INNER/CHILD table of the XML file -->
<xsl:for-each select="//*[name()=$link][count(*)>0]">
<!-- display all the fields in the current row -->
<xsl:call-template name="display_horizontal">
<xsl:with-param name="zone" select="'link'"/>
<xsl:with-param name="table_record" select="." />
<xsl:with-param name="multiple" select="'y'"/>
</xsl:call-template>
</xsl:for-each>
etc etc...
Seems to work correctly. And it's lightning fast. Give it a whirl, what's the worst that could happen? 
â
[Updated on: Mon, 19 March 2007 11:30] Report message to a moderator |
|
|
|
|
|
|
|
|
|
|
|
Goto Forum:
Current Time: Wed Jun 18 11:03:48 EDT 2025
Total time taken to generate the page: 0.01609 seconds
|
Radicore Forum
Search
Help
Members
Register
Login
Home
