Skip to main content

DynamicReports and Cocoon integration

This is one more tutorial on how to exploit DynamicReports reporting library in an existing web application. This time the application is based on Cocoon 2.2 framework. This post is a continuation to the previous posts: Choosing Java reporting tool - part 2 and DynamicReports and Spring MVC integration. The complete code won't be provided here but only the essential code snippets together with usage remarks.

First of all I'm writing the plan of this how to:
  1. Description of existing application data flow.
  2. Adding project dependencies.
  3. Implementing export servlet.
  4. Implementing XSLT extension for the export data source.
  5. Implementing XSLT for postprocessing.
  6. Modifying JX template.
  7. Modifying Cocoon sitemap.
Description of existing application data flow
Cocoon framework organizes data flow using the concept of pipelines. For details please refer to Cocoon documentation. My application uses the following component sequence in a pipeline:
  • Standard JX template generator produces xquery code.
  • Custom XQuery generator executes searching xquery over Sedna XML DB and returns resulting xml data as an html table.
  • Standard XSLT transformer performs required postprocessing of data (formatting numbers and fetching data from sources other than XML DB) and returns ready to use html table with data.
  • Finally the generated html table is being asynchronously inserted into the web page (this happens outside of Cocoon pipeline and is performed by AJAX means of YUI - it won't be covered here being outside of this post's scope).
I'm going to provide some sources below for better understanding.

sitemap.xmap:
<map:pipelines>
    <map:pipeline id="services">
        <map:match pattern="xquery-macro/**">
            <map:generate src="xquery/{1}.jx" type="jx"/>
            <map:serialize type="text"/>
        </map:match>
        
        <map:match pattern="xquery/**">
            <map:generate src="cocoon:/xquery-macro/{1}" type="queryStringXquery">
                <map:parameter name="contextPath" value="{request:contextPath}"/>
            </map:generate>
            <map:transform src="xslt/postprocessXqueryResults.xslt" type="saxon"/>
            <map:serialize type="xml"/>
        </map:match>
    </map:pipeline>
</map:pipelines>

<map:components>
    <map:serializers>
        <map:serializer logger="sitemap.serializer.xml" mime-type="text/xml" name="xml" src="org.apache.cocoon.serialization.XMLSerializer">
            <encoding>UTF-8</encoding>
        </map:serializer>
    </map:serializers>
    <map:transformers>
        <map:transformer name="saxon" src="org.apache.cocoon.transformation.TraxTransformer">
            <xslt-processor-role>saxon</xslt-processor-role>
        </map:transformer>
    </map:transformers>
    <map:generators>
        <map:generator name="queryStringXquery" src="org.lagivan.prototype.generator.QueryStringXQueryGenerator">
            <map:parameter name="db-url" value="${xmldb.url}"/>
            <map:parameter name="db-user" value="${xmldb.user}"/>
            <map:parameter name="db-password" value="${xmldb.password}"/>
            <map:parameter name="cache-validity" value="${xmldb.cache-validity}"/>
        </map:generator>
    </map:generators>
</map:components>
In the sitemap you can see the first three steps described above and several components being configured inside map:components tag. I omitted mentioning text and xml serializers just for simplicity. Also I won't provide javascript that launchs this pipeline as it's out of this post's scope.

Adding project dependencies
We do use Maven to manage project dependencies. Actually in this case dependencies (that are related to DynamicReports) are exactly the same as in the previous post.

Implementing export servlet
This is the main part of export functionality that is implemented according to the official example. Servlet implementation requires writing a Java class extending HttpServlet below and also a piece of configuration that is Cocoon specific.

ExportServlet.java:
package org.lagivan.prototype.export;

import net.sf.dynamicreports.jasper.builder.JasperReportBuilder;
import net.sf.dynamicreports.report.exception.DRException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.Map;

/**
 * @author Ivan Lagunov
 */
public class ExportServlet extends HttpServlet {

    private static final Logger log = LoggerFactory.getLogger(ExportServlet.class);

    private static final String PARAMETER_TYPE = "type";
    private static final String VALUE_TYPE_PDF = "pdf";
    private static final String VALUE_TYPE_XLS = "xls";

    private static final Map<String, String> FILE_TYPE_2_CONTENT_TYPE = new HashMap<String, String>();
    static {
        FILE_TYPE_2_CONTENT_TYPE.put(VALUE_TYPE_PDF, "application/pdf");
        FILE_TYPE_2_CONTENT_TYPE.put(VALUE_TYPE_XLS, "application/vnd.ms-excel");
    }

    private IExportDataSource dataSource;

    public void setDataSource(IExportDataSource dataSource) {
        this.dataSource = dataSource;
    }

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
        String fileType = request.getParameter(PARAMETER_TYPE);
        log.info("Exporting {} report", fileType);

        response.setContentType(FILE_TYPE_2_CONTENT_TYPE.get(fileType));
        OutputStream out = response.getOutputStream();
        try {
            JasperReportBuilder jrb = createJasperReport();

            if (VALUE_TYPE_PDF.equals(fileType)) {
                jrb.toPdf(out);
            } else if (VALUE_TYPE_XLS.equals(fileType)) {
                jrb.toExcelApiXls(out);
            }
        } catch (DRException e) {
            throw new ServletException(e);
        }
        out.close();
    }

    private JasperReportBuilder createJasperReport() {
        // Here I used DynamicReports API to build a report 
        // and to fill it with the datasource.
    }
}
The servlet must be defined properly in the Spring servlet configuration file.

myblock-servlet-service.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:servlet="http://cocoon.apache.org/schema/servlet"
       xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://cocoon.apache.org/schema/servlet http://cocoon.apache.org/schema/servlet/cocoon-servlet-1.0.xsd">

    <bean class="org.apache.cocoon.sitemap.SitemapServlet" name="org.lagivan.prototype.myblock.service">
        <servlet:context context-path="blockcontext:/myblock/" mount-path="/myblock">
            <servlet:connections>
                <entry key="ajax" value-ref="org.apache.cocoon.ajax.impl.servlet"/>
                <entry key="forms" value-ref="org.apache.cocoon.forms.impl.servlet"/>
                <entry key="export" value-ref="org.lagivan.prototype.export.service"/>
            </servlet:connections>
        </servlet:context>
    </bean>

    <bean class="org.lagivan.prototype.export.ExportServlet" name="org.lagivan.prototype.export.service">
        <servlet:context context-path="blockcontext:/myblock/" mount-path="/export"/>
        <property name="dataSource" ref="exportDataSource"/>
    </bean>
</beans>
You might have got a question where exportDataSource bean is defined. Well, it's defined in the Spring application context configuration file.

myblock-application-context.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">

    <bean class="org.lagivan.prototype.export.ExportDataSource" id="exportDataSource" scope="session">
        <aop:scoped-proxy proxy-target-class="false"/>
    </bean>

    <bean class="org.lagivan.prototype.xslt.ExportDataSourceExtension" id="exportDataSourceExtension">
        <property name="dataSource" ref="exportDataSource"/>
    </bean>
</beans>
Here you can see two beans: exportDataSource and exportDataSourceExtension. The first is a storage for the data source that implements an interface IExportDataSource to access stored data; the second is a Java XSLT extension that simply delegates calls to the data source object (sources for these files will be given in the next section below). Also it's worth mentioning that exportDataSource needs to be proxied with aop:scoped-proxy due to its session scope (refer the Spring reference for the details).

Implementing XSLT extension for the export data source
Thus, Java XSLT extension is meant for saving xquery search results data into the session-scoped data source property. The same data source object is used then by the export servlet to generate the final report document. In this section I'll provide sources for Java XSLT extension and the Java data source.

IExportDataSource.java:
package org.lagivan.prototype.export;

import net.sf.jasperreports.engine.JRRewindableDataSource;

/**
 * @author Ivan Lagunov
 */
public interface IExportDataSource extends JRRewindableDataSource {

    // To be used inside ExportServlet.createJasperReport() to set columns. 
    String[] getColumns();

    // To be called from Java XSLT extension to set columns and their values.
    void setColumns(String... columns);
    void setColumnValue(int columnIndex, String value);

    // To be called from Java XSLT extension after all column values for a single table row are already set.
    void onColumnValueCompleted();
}
ExportDataSource.java:
/**
 * DynamicReports - Free Java reporting library for creating reports dynamically
 *
 * Copyright (C) 2010 - 2011 Ricardo Mariaca
 * http://dynamicreports.sourceforge.net
 *
 * This file is part of DynamicReports.
 *
 * DynamicReports is free software: you can redistribute it and/or modify
 * it under the terms of the GNU Lesser General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * DynamicReports is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with DynamicReports. If not, see <http://www.gnu.org/licenses/>.
 */
package org.lagivan.prototype.export;

import net.sf.jasperreports.engine.JRException;
import net.sf.jasperreports.engine.JRField;

import java.util.*;

/**
 * @author Ricardo Mariaca ([email protected])
 * @author Ivan Lagunov
 */
public class ExportDataSource implements IExportDataSource {
    private String[] columns;
    private String[] columnValues;
    private List<Map<String, Object>> values;
    private Iterator<Map<String, Object>> iterator;
    private Map<String, Object> currentRecord;

    public ExportDataSource() {
    }

    @Override
    public String[] getColumns() {
        return columns;
    }

    @Override
    public void setColumns(String... columns) {
        this.columns = columns;
        this.values = new ArrayList<Map<String, Object>>();
        this.columnValues = new String[columns.length];
        Arrays.fill(columnValues, "");
    }

    @Override
    public void setColumnValue(int columnIndex, String value) {
        columnValues[columnIndex] = value;
    }

    @Override
    public void onColumnValueCompleted() {
        add(columnValues);
        Arrays.fill(columnValues, "");
    }

    @Override
    public Object getFieldValue(JRField field) throws JRException {
        return currentRecord.get(field.getName());
    }

    @Override
    public boolean next() throws JRException {
        if (iterator == null) {
            this.iterator = values.iterator();
        }
        boolean hasNext = iterator.hasNext();
        if (hasNext) {
            currentRecord = iterator.next();
        }
        return hasNext;
    }

    @Override
    public void moveFirst() throws JRException {
        this.iterator = null;
    }

    private void add(Object... values) {
        Map<String, Object> row = new HashMap<String, Object>();
        for (int i = 0; i < values.length; i++) {
            row.put(columns[i], values[i]);
        }
        this.values.add(row);
    }
}
The data source code is based on net.sf.dynamicreports.examples.DataSource taken from DynamicReports zip archive. It can be easily set for a report using JasperReportBuilder.setDataSource(dataSource) later on (that's how I did in ExportServlet). As you can see, the ExportDataSource simply saves String values for each column of a row and stores them at once when onColumnValueCompleted happens.

ExportDataSourceExtension.java:
package org.lagivan.prototype.xslt;

import org.lagivan.prototype.export.IExportDataSource;

/**
 * @author Ivan Lagunov
 */
public class ExportDataSourceExtension {

    private static IExportDataSource dataSource;

    public void setDataSource(IExportDataSource dataSource) {
        ExportDataSourceExtension.dataSource = dataSource;
    }

    public static void setColumns(String... columns) {
        dataSource.setColumns(columns);
    }

    public static void setColumnValue(int columnIndex, String value) {
        dataSource.setColumnValue(columnIndex - 1, value);
    }

    public static void onColumnValueCompleted() {
        dataSource.onColumnValueCompleted();
    }
}
You can refer to this tutorial about extending XSLT with Java for more details. The main thing to understand that it's recommended to make a method static to be able to call it simply from XSLT. In spite of being a static field, the dataSource will still be session-scoped as soon as you mark the Spring bean with aop:scoped-proxy.

Implementing XSLT for postprocessing
Now let's turn to the XSLT code that saves data in the ExportDataSource using the ExportDataSourceExtension.

postprocessExport.xslt:
<?xml version="1.0" encoding="UTF-8"?>
<!--
  Author: Ivan Lagunov
  This stylesheet saves XQuery search results in ExportDataSource
-->
<xsl:stylesheet version="2.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:exp="java:org.lagivan.prototype.xslt.ExportDataSourceExtension">

  <xsl:output method="html" omit-xml-declaration="yes"/>

  <!-- set columns -->
  <xsl:template match="thead">
    <xsl:copy>
      <xsl:value-of select="exp:setColumns(data(tr/th[not(@exportIgnore)]/text()))"/>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!-- set column values -->
  <xsl:template match="tbody">
    <xsl:copy>
      <xsl:for-each select="tr">
        <xsl:for-each select="td[not(@exportIgnore)]">
          <xsl:choose>
            <xsl:when test="a/text()">
              <xsl:value-of select="exp:setColumnValue(position(), data(a/text()))"/>
            </xsl:when>
            <xsl:when test="normalize-space(.)">
              <xsl:value-of select="exp:setColumnValue(position(), normalize-space(.))"/>
            </xsl:when>
          </xsl:choose>
        </xsl:for-each>
        <xsl:value-of select="exp:onColumnValueCompleted()"/>
      </xsl:for-each>
      <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
  </xsl:template>

  <!-- remove exportIgnore attributes -->
  <xsl:template match="@exportIgnore"/>

  <!-- copy all nodes and attributes which are not processed by one of available templates -->
  <xsl:template match="@*|node()">
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:apply-templates/>
    </xsl:copy>
  </xsl:template>

</xsl:stylesheet>

Modifying JX template
Here is a sample of JX template that the postprocessExport.xslt can be applied to.

xquery_search.jx:
<?xml version="1.0" encoding="UTF-8"?>


  <jx:macro name="xquery-macro">
    <set-results/>
    return
    <return-results/>
  </jx:macro>

  
    <![CDATA[
    let $results := collection("products")/ProductItem
    ]]>
  

  
    <![CDATA[
    <table style="width: 100%" id="search-results-table">
      <caption class="header">
        <span style="float:left">{count($results)} result(s) found</span>
        <span style="float:right">
          <a title="Export to Excel" href="{$contextPath}/export?type=xls">
            <img style="border: none;" src="resource/external/icons/mimetypes/xls_16.png"/>
          </a>
          <a title="Export to PDF" href="{$contextPath}/export?type=pdf">
            <img style="border: none;" src="resource/external/icons/mimetypes/pdf_16.png"/>
          </a>
        </span>
      </caption>
      <thead>
        <tr>
          <th class="header">ID</th>
          <th class="header">Product name</th>
          <th class="header" exportIgnore="">Download</th>
        </tr>
      </thead>
      <tbody>
      {
        for $result in $results
        let $id := data($result/@id)
        order by $id
        return 
        <tr id="{$id}">
          <td><a href="{$contextPath}/myblock/product/preview.html?id={$id}" target="_blank">{$id}</a></td>
          <td>{$result/Name/text()}</td>
          <td exportIgnore="">
            <a href="{$contextPath}/myblock/product/download/{$id}.xml" target="_blank">
              <img src="resource/external/icons/mimetypes/xml_16.png"/>
            </a>
          </td>
        </tr>
      }    
      </tbody>
    </table>
    ]]>
  

As you can see, this template produces a valid xquery code to output products data in the html table format. It worked flawlessly even before adding the export feature. As soon as export feature was implemented, I added exportIgnore attribute to filter out unnecessary column in export results and also links in the caption to trigger the ExportServlet.

Modifying Cocoon sitemap
Finally it's only the Cocoon sitemap left to be updated. We need to process XQuery results with postprocessExport.xslt and to enable ExportServlet to process "{$contextPath}/export" URL. I provided the initial sitemap.xmap contents above and here are the changes.

sitemap.xmap:
<map:pipelines>
    <map:pipeline id="services">
        <map:match pattern="xquery/**">
            <map:generate src="cocoon:/xquery-macro/{1}" type="queryStringXquery">
                <map:parameter name="contextPath" value="{request:contextPath}"/>
            </map:generate>
            <map:transform src="xslt/postprocessXqueryResults.xslt" type="saxon"/>
            <map:transform src="xslt/postprocessExport.xslt" type="saxon"/>
            <map:serialize type="xml"/>
        </map:match>

        <map:match pattern="export">
            <map:generate src="servlet:export:/"/>
        </map:match>
    </map:pipeline>
</map:pipelines>

Comments

  1. One small note about using jx:templates to generate the XQuery. I chose that approach because I was not sure wheter it made sense to store application specific (but nonetheless reusable) xquery-parts as modules in Sedna XMLDB itself. But after having done both, I feel that this approach was the wrong one and storing xquery modules makes much more sense. Even if they are application specific, one could separate app-specific functions in a separate module. The benefit from using modules is that you can more easily import functions at any time.

    ReplyDelete
  2. Thanks for the remark!

    I've just imagined how much it will influence the development process.

    The advantage is an opportunity to deploy XMLDB modules without any significant delay, thus no interruption during the application work. But it's not that much valuable in our case.

    However, in this post XSLT processing logic is closely related to the syntax generated by XQuery. I feel like it's not a nice design solution to divide application logic and move application specific code to XML DB. Moreover, if we're going to set up any Continuous Integration tool finally (which is worth doing btw - to be discussed), we may face some related issues here.

    ReplyDelete
  3. You make a valid statement. I don't even care if they are stored in the XMLDB itself. The problem is that you can only have a single source where to import xquery modules from. At least this is the case for Sedna. If we would be able to store actual modules inside our Cocoon app as well, and somehow configure we would import them from our own app, that would be the best solution.

    ReplyDelete
  4. I got your idea. I agree it would be really nice to have such an opportunity. However, it's indeed not available in Sedna XML DB for now. I've created a feature request for them: http://sourceforge.net/tracker/?func=detail&atid=713733&aid=3446873&group_id=129076

    ReplyDelete
  5. Nice. I actually did investigate this possibility in the past and stumbled upon this article explaining that vendors could implement such a feature: http://www.stylusstudio.com/xquery/xquery_functions.html

    E.g.:
    import module namespace hr = "http://hr.example.com/" at "hr-module.xquery";

    You might as well have found this mail ;-)
    http://article.gmane.org/gmane.text.xml.sedna/2318

    ReplyDelete

Post a Comment

Popular posts from this blog

DynamicReports and Spring MVC integration

This is a tutorial on how to exploit DynamicReports reporting library in an existing Spring MVC based web application. It's a continuation to the previous post where DynamicReports has been chosen as the most appropriate solution to implement an export feature in a web application (for my specific use case). The complete code won't be provided here but only the essential code snippets together with usage remarks. Also I've widely used this tutorial that describes a similar problem for an alternative reporting library.
So let's turn to the implementation description and start with a short plan of this how-to:
Adding project dependencies.Implementing the Controller part of the MVC pattern.Modifying the View part of the MVC pattern.Modifying web.xml.Adding project dependencies
I used to apply Maven Project Builder throughout my Java applications, thus the dependencies will be provided in the Maven format.

Maven project pom.xml file:
net.sourceforge.dynamicreportsdynamicrepo…

Choosing Java reporting tool - part 2

I've provided a general overview of possible solutions to get a reporting/exporting functionality in the previous post. This is the second overview of alternatives based on JasperReports reporting engine.

Since the previous part I've done the following:
Implemented a simple report using both DynamicJasper and DynamicReports to compare them from technical side.Investigated JasperServer features and tried to implement a simple report for JasperServer instance (it appeared we already have a ready licensed installation of JasperServer that makes it unreasonable to install a fresh one).
First, the comparison results of Java libraries (DynamicJasper and DynamicReports):
Both libraries suffer from poor-quality or missing Java docs but they look a bit better in DynamicJasper.Taking into account the point 1, a developer has to use online documentation and to review the code. Here the code looks definitely nicer and more readable for DynamicReports. With respect t…

Do It Yourself Java Profiling

This article is a free translation of the Russian one that is a transcript of the Russian video lecture done by Roman Elizarov at the Application Developer Days 2011 conference.
The lecturer talked about profiling of Java applications without any standalone tools. Instead, it's suggested to use internal JVM features (i.e. threaddumps, java agents, bytecode manipulation) to implement profiling quickly and efficiently. Moreover, it can be applied on Production environments with minimal overhead. This concept is called DIY or "Do It Yourself". Below the lecture's text and slides begin.
Today I'm giving a lecture "Do It Yourself Java Profiling". It's based on the real life experience that was gained during more than 10 years of developing high-loaded finance applications that work with huge amounts of data, millions currency rate changes per second and thousands of online users. As a result, we have to deal with profiling. Application profiling is an i…