WebMaker provides a filtering select control out of the box which filters a set of available options as the user types in the field. Although this control works well, it does require all the available options to be available in advance as the page loads. This is fine for many scenarios, but if you have thousands of records this is not practical.
This FAQ aims to illustrate an approach that can be used to dynamically query the server for the matching options as the user enters a value. Please note that this is intended for more experienced users.
The approach illustrated here continues to use the Filtering Select control (based on the dojo Filtering Select widget) along with a dojo data QueryReadStore to handle the server calls. This means that visually there will not be any change, but it does require a few more configuration steps on the server.
(The flexibility that WebMaker provides means that you can of course use a different type of control/approach if you desire)
This example is based around querying a SQL table that contains +?+?++id+?+?+? and +?+?++name+?+?+? columns. The names will be displayed in the drop down list and searched on as the user types into the field, with the id value being sent back for the selected value.
Implementation Steps - 1) Setup the Field Control
First add a new controller to the Application Map for your project which will be used to find the relevant matches (e.g. by querying a SQL database in this example.)
Now add a link to this controller from the page that will contain the filtering select control and one back the other way. Ensure the link to the controller has an appropriate action name (e.g. +?+?++getMatches+?+?+?)
Use the Page Design tab (Page Structure in WebMaker v3) to add a Filtering Select control to your page.
Using the Properties tab (Field Details in v3), remove all the default static options under Data Constraints for this control, as instead these values will be retrieved from the server. (We don+?+?+?t want to use the dynamic options setting as this requires the values to be present on page load.)
The next step is to adjust the custom attributes for this control so that it knows how to get the options to display, by linking it to a dojo data store.
For WebMaker v4, add a new custom attribute called data-dojo-props with the following content: store:filteringSelectStore, searchAttr: "name", pageSize:50
For earlier WebMaker versions you need to add three separate custom attributes, store, searchAttr, pageSize with the same values as above.
These details can be adjusted if required. The store name (filteringSelectStore) will be created in the next step, the pageSize sets how many records will be returned at one time, and the searchAtt indicates which attribute we are searching on. In this example, it is the name column of the SQL Table.
To create the definition of this dojo data store add an onbeforeload page event with a Custom Script action. For WebMaker v4 (dojo 1.7.2) use this content:
The name of the store created must match that used in the custom attribute value against the Filtering Select, and the URL must match the name given for the action to the controller on the Application Map screen. Please note that the .do needs to be added to the name of the action shown on the Application Map screen, as this is the physical name of the action to be processed at runtime.
For WebMaker 3.1.2 and earlier the dojo release included with the studio does not include the dojox package. Therefore you will need to manually download this and add the dojox directory to your project in the repository. This new folder needs to be placed in the +?+?++webapp/js/dojo+?+?+? directory for your project.
You can get the dojo 1.6.1 release (as used with WebMaker 3.1.2) from http://download.dojotoolkit.org/release-1.6.1/
This completes the changes needed to the page.
Implementation Steps - 2) Setup the Controller Rules
The remaining work is all to do with getting the new controller to return the correct data matches in a format that the dojo data store will understand.
When the new controller is called it will be passed three parameters that will be visible in the control section of the eForm message, 'name', 'start', and 'count'. These are used by the dojo data store to ensure the correct data is returned.
The most common use will probably be to query a SQL database to find the matching records to return.
To do this you should first create the default database select rule in the new controller. You can do this manually, or generate it by temporarily dragging on a new field from the database table onto your page, generating the application, and then removing the field. This will have generated the initial rule for you. (Remember that when you add a database field to the page, it will create a SQL 'select' statement in all the controllers that call your page. Therefore you may need to remove the generated rule in any other controllers you have linked to the page.)
Once you have the rule, you can adjust the SQL as needed to perform the correct query. Note that the 'name' parameter is submitted with a '*' as a wildcard character by dojo, e.g. 'abc*'. Therefore you will likely want to translate this to a '%' for use in a SQL LIKE clause.
Note: The format of this SQL will depend on the database you are using, but for SQL Server 2012 for example you could use something like:
Once you have retrieved the data from the SQL database, it will need to be formatted correctly for the Dojo data store. The Dojo data store needs to receive a JSON structured response, so we need to convert the standard XML information into a JSON string. Assuming your data is in the standard SQL result format that WebMaker generates, and contains an 'id' and a 'name' column, you could use the following XSL transform to achieve this:
This XSL (attached as db2json.xsl) makes use of xml2json.xslt from http://code.google.com/p/xml2json-xslt/ which is also attached for convenience. It will output a single 'json_output' element that will contain the JSON string.
Note: If you need to use different column names, then the label and identifier values near the start of the XSLT above will need to be edited accordingly.
This should be used in a Transform Action at the end of the SQL processing rule (usually having generated Actions of SQL Statement, Rename and Move Actions). The Action should look like the example below:
Note: The xpath with .../account will likely need to be changed to the name of the actual SQL Table used.
You will need to attach the xml2json.xslt file as a resource of the controller in addition to the db2json.xsl file. The easiest way to do this is to first select the xml2json.xslt as the stylesheet file, and then select the correct db2json.xsl file.
You should also disable the generated +?+?++write_targetPage+?+?-?+?+?+? rule as we will not be returning an HTML page in this case.
This setup will ensure that the factbase will now contain a single element with the JSON string, so the final step is to get the platform to return this string as JSON data rather than trying to return the XML.
Implementation Steps - 3) Setup the XGate plugin
To do this we need to use a custom XGate plugin. This can be done with a very simple plugin that just detects the presence of the json_output element, and if found simply outputs the JSON string to the browser. The java code for this plugin is provided below, and it is attached in class form that can be included into the webapp directly.
You will need to ensure this java code is available to the platform at runtime. This can be done by setting up the necessary directory and files in the design project repository webapp directory. Place the class file in the WEB-INF/classes/com/hyfinity directory within the webapp. If the directories are not present, then you will need to add each nested directory. When a full application deployment is performed in FormMaker, then the files are copied across to the runtime environment.
The final step is to get the platform to use this XGate plugin. To do this, look in the design project repository for an existing xgate.xml configuration file for your project, or add one if there is not one present. This should be located in a directory called doc within the webapp structure for your project. This file needs to include the custom plugin definition for this processing. You may already have other custom plugin definitions that need to be retained. For more detail on the xgate file, refer to the documentation at http://www.hyfinity.net/mvcdocumentation/Morphyc/Morphyc%20Installation%20and%20Administration%20Guide/XGate%20Configuration%20Files.html
An example configuration file is shown below. It is important to turn off the 'ensure_html_response' option if currently enabled, by setting it to false. This is because we will now not always be returning valid HTML through xgate.
Once you have deployed your project and made sure the custom plugin code is available, you should find that the filtering select control now correctly requests information from the server and displays the results as the user enters characters into the box.
In order for the control to correctly handle scenarios where the number of matches is greater than the page size in use (set to 50 above by the custom attribute), then the response JSON data needs to also indicate how many records in total match the criteria. To do this you would need to use another SQL query to get the total count of matching records, and then include this in the JSON string. The db2json.xsl will look for a 'total' element within the XML structure, and if found will return this value.
Therefore, you should add additional actions like shown below to create this total element before the existing transform action is processed.
For more information on the dojo filtering select control and how to connect it up to a server data store you can look through the dojo documentation, e.g. http://dojotoolkit.org/reference-guide/1.8/dijit/form/FilteringSelect.html and http://dojotoolkit.org/reference-guide/1.8/dojox/data/QueryReadStore.html
This FAQ aims to illustrate an approach that can be used to dynamically query the server for the matching options as the user enters a value. Please note that this is intended for more experienced users.
The approach illustrated here continues to use the Filtering Select control (based on the dojo Filtering Select widget) along with a dojo data QueryReadStore to handle the server calls. This means that visually there will not be any change, but it does require a few more configuration steps on the server.
(The flexibility that WebMaker provides means that you can of course use a different type of control/approach if you desire)
This example is based around querying a SQL table that contains +?+?++id+?+?+? and +?+?++name+?+?+? columns. The names will be displayed in the drop down list and searched on as the user types into the field, with the id value being sent back for the selected value.
Implementation Steps - 1) Setup the Field Control
First add a new controller to the Application Map for your project which will be used to find the relevant matches (e.g. by querying a SQL database in this example.)
Now add a link to this controller from the page that will contain the filtering select control and one back the other way. Ensure the link to the controller has an appropriate action name (e.g. +?+?++getMatches+?+?+?)
Use the Page Design tab (Page Structure in WebMaker v3) to add a Filtering Select control to your page.
Using the Properties tab (Field Details in v3), remove all the default static options under Data Constraints for this control, as instead these values will be retrieved from the server. (We don+?+?+?t want to use the dynamic options setting as this requires the values to be present on page load.)
The next step is to adjust the custom attributes for this control so that it knows how to get the options to display, by linking it to a dojo data store.
For WebMaker v4, add a new custom attribute called data-dojo-props with the following content: store:filteringSelectStore, searchAttr: "name", pageSize:50
For earlier WebMaker versions you need to add three separate custom attributes, store, searchAttr, pageSize with the same values as above.
These details can be adjusted if required. The store name (filteringSelectStore) will be created in the next step, the pageSize sets how many records will be returned at one time, and the searchAtt indicates which attribute we are searching on. In this example, it is the name column of the SQL Table.
To create the definition of this dojo data store add an onbeforeload page event with a Custom Script action. For WebMaker v4 (dojo 1.7.2) use this content:
require(['dojox/data/QueryReadStore'], function(qrs){
filteringSelectStore = new qrs({url: 'getMatches.do'});
});
For WebMaker 3.1.2 (dojo 1.6.1) use:
dojo.require('dojox.data.QueryReadStore');
filteringSelectStore = new dojox.data.QueryReadStore({url: 'getMatches.do'});
The name of the store created must match that used in the custom attribute value against the Filtering Select, and the URL must match the name given for the action to the controller on the Application Map screen. Please note that the .do needs to be added to the name of the action shown on the Application Map screen, as this is the physical name of the action to be processed at runtime.
For WebMaker 3.1.2 and earlier the dojo release included with the studio does not include the dojox package. Therefore you will need to manually download this and add the dojox directory to your project in the repository. This new folder needs to be placed in the +?+?++webapp/js/dojo+?+?+? directory for your project.
You can get the dojo 1.6.1 release (as used with WebMaker 3.1.2) from http://download.dojotoolkit.org/release-1.6.1/
This completes the changes needed to the page.
Implementation Steps - 2) Setup the Controller Rules
The remaining work is all to do with getting the new controller to return the correct data matches in a format that the dojo data store will understand.
When the new controller is called it will be passed three parameters that will be visible in the control section of the eForm message, 'name', 'start', and 'count'. These are used by the dojo data store to ensure the correct data is returned.
The most common use will probably be to query a SQL database to find the matching records to return.
To do this you should first create the default database select rule in the new controller. You can do this manually, or generate it by temporarily dragging on a new field from the database table onto your page, generating the application, and then removing the field. This will have generated the initial rule for you. (Remember that when you add a database field to the page, it will create a SQL 'select' statement in all the controllers that call your page. Therefore you may need to remove the generated rule in any other controllers you have linked to the page.)
Once you have the rule, you can adjust the SQL as needed to perform the correct query. Note that the 'name' parameter is submitted with a '*' as a wildcard character by dojo, e.g. 'abc*'. Therefore you will likely want to translate this to a '%' for use in a SQL LIKE clause.
Note: The format of this SQL will depend on the database you are using, but for SQL Server 2012 for example you could use something like:
Once you have retrieved the data from the SQL database, it will need to be formatted correctly for the Dojo data store. The Dojo data store needs to receive a JSON structured response, so we need to convert the standard XML information into a JSON string. Assuming your data is in the standard SQL result format that WebMaker generates, and contains an 'id' and a 'name' column, you could use the following XSL transform to achieve this:
<xsl :s tylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:import href="xml2json.xslt" />
<xsl:template match="/">
<json_output xmlns="">
<xsl:text>{label: 'name', identifier: 'id', numRows: </xsl:text>
<xsl:choose>
<xsl:when test="/*/total">
<xsl:value-of select="/*/total" />
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="count(//record)"/>
</xsl:otherwise>
</xsl:choose>
<xsl:if test="//record">
<xsl:text>, items:[</xsl:text>
<xsl:for-each select="//record">
<xsl:apply-templates select="node()" />
<xsl:if test="following-sibling::*">,</xsl:if>
</xsl:for-each>
<xsl:text>]</xsl:text>
</xsl:if>
<xsl:text>}</xsl:text>
</json_output>
</xsl:template>
<xsl:template match="*[count(child::node())=0]">
<xsl:if test="not(preceding-sibling::*)">{</xsl:if>
<xsl:apply-imports/>
<xsl:if test="not(following-sibling::*)">}</xsl:if>
</xsl:template>
</xsl :s tylesheet>
This XSL (attached as db2json.xsl) makes use of xml2json.xslt from http://code.google.com/p/xml2json-xslt/ which is also attached for convenience. It will output a single 'json_output' element that will contain the JSON string.
Note: If you need to use different column names, then the label and identifier values near the start of the XSLT above will need to be edited accordingly.
This should be used in a Transform Action at the end of the SQL processing rule (usually having generated Actions of SQL Statement, Rename and Move Actions). The Action should look like the example below:
Note: The xpath with .../account will likely need to be changed to the name of the actual SQL Table used.
You will need to attach the xml2json.xslt file as a resource of the controller in addition to the db2json.xsl file. The easiest way to do this is to first select the xml2json.xslt as the stylesheet file, and then select the correct db2json.xsl file.
You should also disable the generated +?+?++write_targetPage+?+?-?+?+?+? rule as we will not be returning an HTML page in this case.
This setup will ensure that the factbase will now contain a single element with the JSON string, so the final step is to get the platform to return this string as JSON data rather than trying to return the XML.
Implementation Steps - 3) Setup the XGate plugin
To do this we need to use a custom XGate plugin. This can be done with a very simple plugin that just detects the presence of the json_output element, and if found simply outputs the JSON string to the browser. The java code for this plugin is provided below, and it is attached in class form that can be included into the webapp directly.
package com.hyfinity;
import com.hyfinity.Namespaces;
import com.hyfinity.utils.xml.XDocument;
import com.hyfinity.xgate.XGatePlugin;
import com.hyfinity.utils.xml.DOMUtils;
import com.hyfinity.xplatform.XPlatformException;
import java.io.IOException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.w3c.dom.Node;
/**
* XGate plugin for returning a JSON string to the browser
* This looks for the presence of a 'json_output' root element in the
* response message, and if present returns it's text content.
*/
public class JSONOutputPlugin implements XGatePlugin
{
public JSONOutputPlugin()
{
}
/**
* Process the request before it is passed to the platform.
* @param input An XDocument object containing the current state of the XML message.
* The document this contains should be updated as required.
* @param request The HttpServletRequest object that defines the request being processed.
* This provides access to any request or seesion parameters etc.
* @param response The HttpServletResponse object that will eventually be written to.
*/
public void processInput(XDocument input, HttpServletRequest request, HttpServletResponse response)
{
}
/**
* Process the response recieved from the platform, before it is returned to the browser.
* @param output An XDocument object containing the current state of the XML message (this will often be HTML).
* The document this contains should be updated as required.
* @param request The HttpServletRequest object that defines the request being processed.
* This provides access to any request or seesion parameters etc.
* @param response The HttpServletResponse object that will eventually be written to.
*/
public void processOutput(XDocument output, HttpServletRequest request, HttpServletResponse response)
{
Node json = output.selectSingleNode("/json_output");
if (json != null)
{
//stop the document being returned
output.setDocument(null);
//get the JSON string content
String jsonString = DOMUtils.getText(json);
try
{
//return it to the browser
response.setContentType("application/json; charset=utf-8");
response.getWriter().print(jsonString);
}
catch (IOException e)
{
e.printStackTrace();
throw new XPlatformException("Error outputting JSON string", e);
}
}
}
} You will need to ensure this java code is available to the platform at runtime. This can be done by setting up the necessary directory and files in the design project repository webapp directory. Place the class file in the WEB-INF/classes/com/hyfinity directory within the webapp. If the directories are not present, then you will need to add each nested directory. When a full application deployment is performed in FormMaker, then the files are copied across to the runtime environment.
The final step is to get the platform to use this XGate plugin. To do this, look in the design project repository for an existing xgate.xml configuration file for your project, or add one if there is not one present. This should be located in a directory called doc within the webapp structure for your project. This file needs to include the custom plugin definition for this processing. You may already have other custom plugin definitions that need to be retained. For more detail on the xgate file, refer to the documentation at http://www.hyfinity.net/mvcdocumentation/Morphyc/Morphyc%20Installation%20and%20Administration%20Guide/XGate%20Configuration%20Files.html
An example configuration file is shown below. It is important to turn off the 'ensure_html_response' option if currently enabled, by setting it to false. This is because we will now not always be returning valid HTML through xgate.
<xgate>
<product>mvc</product>
<plugins>
<sxforms delete_bound="true" mark_unbound="true">true</sxforms>
<ensure_html_response>false</ensure_html_response>
<output_doctype doctype_public="-//W3C//DTD XHTML 1.0 Transitional//EN" doctype_system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">true</output_doctype>
<set_locale>false</set_locale>
<custom_plugins>
<custom_plugin name="JSON output" priority="60" runtime_instance="com.hyfinity.JSONOutputPlugin"/>
</custom_plugins>
</plugins>
</xgate>
Once you have deployed your project and made sure the custom plugin code is available, you should find that the filtering select control now correctly requests information from the server and displays the results as the user enters characters into the box.
In order for the control to correctly handle scenarios where the number of matches is greater than the page size in use (set to 50 above by the custom attribute), then the response JSON data needs to also indicate how many records in total match the criteria. To do this you would need to use another SQL query to get the total count of matching records, and then include this in the JSON string. The db2json.xsl will look for a 'total' element within the XML structure, and if found will return this value.
Therefore, you should add additional actions like shown below to create this total element before the existing transform action is processed.
For more information on the dojo filtering select control and how to connect it up to a server data store you can look through the dojo documentation, e.g. http://dojotoolkit.org/reference-guide/1.8/dijit/form/FilteringSelect.html and http://dojotoolkit.org/reference-guide/1.8/dojox/data/QueryReadStore.html
