Sunday, May 29, 2011

Java Annotation Patterns Engine (JAPE)

In this article, we will introduce the Java Annotation Patterns Engine (JAPE), a component of the open-source General Architecture for Text Engineering (GATE) platform.

Example of JAPE Grammar

See below for an example of JAPE grammar (by convention, file type of jape). And see the detailed descriptions of its components in later sections. The context under which this grammar will be executed includes:
  • doc —Implements gate.Document interface. JAPE rules always work on an input annotation set associated with a specific document.
  • inputAS —Implements gate.AnnotationSet interface and represents the input annotation set.
  • outputAS - Implements gate.AnnotationSet interface and represents the output annotation set.
  • ontology - Implements gate.creole.ontology.Ontology interface, which can be used in language analyzing.
// Valentin Tablan, 29/06/2001
// $id$

Input: Token SpaceToken
Options: control = appelt debug = true

// CR+LF | CR |LF+CR -> One single SpaceToken
Rule: NewLine
({SpaceToken.string=="\n"}) |
({SpaceToken.string=="\r"}) |
({SpaceToken.string=="\n"}{SpaceToken.string=="\r"}) |
gate.AnnotationSet toRemove = (gate.AnnotationSet)bindings.get("left");
//get the tokens
java.util.ArrayList tokens = new java.util.ArrayList(toRemove);
//define a comparator for annotations by start offset
Collections.sort(tokens, new gate.util.OffsetComparator());
String text = "";
Iterator tokIter = tokens.iterator();
text += (String)((Annotation)"string");

gate.FeatureMap features = Factory.newFeatureMap();
features.put("kind", "control");
features.put("string", text);
features.put("length", Integer.toString(text.length()));
outputAS.add(toRemove.firstNode(), toRemove.lastNode(), "SpaceToken", features);

What's JAPE?

JAPE is a finite state transducer that operates over annotations based on regular expressions. Thus it is useful for pattern-matching, semantic extraction, and many other operations over syntactic trees such as those produced by natural language parsers.

A JAPE grammar consists of a set of phases, each of which consists of a set of pattern/action rules. The phases run sequentially and constitute a cascade of finite state transducers over annotations. One of the main reasons for using a sequence of phases is that a pattern can only be used once in each phase, but it can be reused in a later phase.

Input annotations could be specified at the beginning of the grammar. This specifies what types of annotations will be processed by the rules of grammar. Other not-mentioned types will be ignored. By default the transducer will include Token, SpaceToken and Lookup.

Options specification defines the method of rule matching (i.e., control) and debug flag for the rules (i.e., debug) in the grammar. There are five control styles:
  • brill — When more than one rule matches the same region of the document, they all are fired.
  • all — Similar to brill, in that it will also execute all matching rules, but the matching will continue from the next offset to the current one.
  • first — With the first style, a rule fires for the first match that‘s found.
  • once — Once a rule has fired, the whole JAPE phase exits after the first match.
  • appelt — Only one rule can be fired for the same region of text, according to a set of priority rules. The appelt control style is the most appropriate for named entity recognition as under appelt only one rule can fire for the same pattern.
If debug is set to true, any rule-firing conflicts will be displayed in the messages window if the grammar is running in appelt mode and there is more than one possible match.

Following the declaration portions of grammar, a list of rules are specified. Each rule consists of a left-hand-side (LHS) and a right-hand-side (RHS). The LHS of the rules consists of an annotation pattern description. The RHS consists of annotation manipulation statements. Annotations matched on the LHS of a rule may be referred to on the RHS by means of labels that are attached to pattern elements.


LHS contains annotation pattern that may contain regular expression operators e.g. ("+", "?" , "*"). However, you should avoid the use of “*” based regular expressions for better performance.

There are 3 main ways in which the pattern can be specified:
  • As a string of text
    • e.g., {Token.string == "Oracle"}
    • This pattern matches a string of text with the value of "Oracle".
  • As the attributes (and values) of an annotation
    • e.g., ({Token.kind == word, Token.category == NNP, Token.orth == upperInitial})?
    • The above pattern uses the Part of Speech (POS) annotation where kind=word, category=NNP and orth=upperInitial.
  • As an annotation previously assigned from a gazetteer, tokeniser, or other module
    • ({Company})?:c1 ({Positives}):v ({Company})?:c2 ({Split}|{CC})?
    • The above pattern matches annotations of Company type, followed by annotations of Positives type, etc. The first-matched pattern element is labeled as c1, the second-matched pattern element is labeled as v, etc.
There are different kind of operators supported:
  • Equality operators (“==” and “!=”)
    • {Token.kind == "number"}, {Token.length != 4}
  • Comparison operators (“<”, “<=”, “>=” and “>”)
    • {Token.string > "aardvark"}, {Token.length < 10}
  • Regular expression operators (“=~”, “==~”, “!~” and “!=~”)
    • {Token.string =~ "[Dd]ogs"}, {Token.string !~ "(?i)hello"}
    • ==~ and !=~ are also provided, for whole-string matching
  • {X contains Y} and {X within Y} for checking annotations within the context of other annotations
You can even define custom operators by implementing gate.jape.constraint.ConstraintPredicate.


The right-hand-side (RHS) consists of annotation manipulation statements. For example, you can add/remove/update annotations associated with a document. Alternatively, RHS can contain Java code to create or manipulate annotations. In this article, we will focus only on RHS' implemented in Java code.

On the RHS, Java code can reference the following variables (which are passed as parameters to the RHS action):
  • doc
  • bindings
  • annotations
  • inputAS
  • outputAS
  • ontology
annotations is provided for backward compatibility and should not be used for new implementations. inputAS and outputAS represent the input and output annotation set. Normally, these would be the same (by default when using ANNIE, these will be the “Default” annotation set) . However, the user is at liberty to change the input and output annotation sets in the parameters of the JAPE transducer at runtime. Therefore, it cannot be guaranteed that the input and output annotation sets will be the same, and we should specify the annotation set we are referring to.

Annotations matched on the LHS of a rule may be referred to on the RHS by means of labels that are attached to pattern elements. They can be retrieved by using bindings as follows:
gate.AnnotationSet toRemove = (gate.AnnotationSet)bindings.get("c1");
This returns a temporary annotation set which holds all the annotations matched on the LHS that have been labeled as "c1."

In the following discussions, we assume you use ANNIE and both inputAS and outputAS points to the same annotation set.

On the RHS, you can do any of the following:
  • Remove annotations from document's annotation set(s)
  • Update annotations in document's annotation set(s)
  • Add new annotations to document's annotation set(s)
However, if you try to remove annotations while using the same iterator for other tasks at the same time, you may see:
  • java.util.ConcurrentModificationException
The solution is to collect all to-be-removed annotations on a list and process them at the end.


To understand the RHS action, you need to know how a JAPE rule is translated to its executable binary in Java. For example, rule ConjunctionIdentifier2

Rule: ConjunctionIdentifier2

gate.AnnotationSet matchedAnns= (gate.AnnotationSet) bindings.get("conj2");
gate.FeatureMap newFeatures= Factory.newFeatureMap();
outputAS.add(matchedAnns.firstNode(),matchedAnns.lastNode(),"CC", newFeatures);

will be translated into:
1  // ConjunctionIdentifierConjunctionIdentifier2ActionClass14
2 package japeactionclasses;
3 import*;
4 import java.util.*;
5 import gate.*;
6 import gate.jape.*;
7 import gate.creole.ontology.*;
8 import gate.annotation.*;
9 import gate.util.*;
11 public class ConjunctionIdentifierConjunctionIdentifier2ActionClass14
12 implements, RhsAction {
13 public void doit(gate.Document doc,
14 java.util.Map bindings,
15 gate.AnnotationSet annotations,
16 gate.AnnotationSet inputAS, gate.AnnotationSet outputAS,
17 gate.creole.ontology.Ontology ontology) throws gate.jape.JapeException {
18 gate.AnnotationSet conj2Annots = bindings.get("conj2");
19 if(conj2Annots != null && conj2Annots.size() != 0) {
22 gate.AnnotationSet matchedAnns= (gate.AnnotationSet) bindings.get("conj2");
23 gate.FeatureMap newFeatures= Factory.newFeatureMap();
24 newFeatures.put.("rule","ConjunctionIdentifierr21");
25 outputAS.add(matchedAnns.firstNode(),matchedAnns.lastNode(),"CC", newFeatures);
27 }
28 }
29 }

Notice that RHS of the rule is wrapped into doit method which has the following signature:
public void doit(gate.Document doc,
java.util.Map bindings,
gate.AnnotationSet annotations,
gate.AnnotationSet inputAS, gate.AnnotationSet outputAS,
gate.creole.ontology.Ontology ontology)

That's why you can reference:
  • doc
  • bindings
  • annotations
  • inputAS
  • outputAS
  • ontology
without declaring them.

Also notice that the return type of doit method is void. It means that you can exit from the middle of action execution by issuing a return statement:
if (annotation.getFeatures().get("tagged") != null)

  1. JAPE (linguistics)
  2. GATE JAPE Grammar Tutorial
  3. JAPE: Regular Expressions over Annotations

Wednesday, May 25, 2011

Cannot get GATE Home. Pease set it manually!

When you run your application using GATE Embedded, you often run into an error:
  • Cannot get GATE Home. Pease set it manually!
This means that you need to set gate.home property before calling Gate.init(). You can do that in two ways:
  1. In your Java code
    • Gate.setGateHome(File)
  2. In the Java command that launches your program
    • -Dgate.home=path/to/gate/home
GATE also needs to initialize the paths to local files of interest like:
  • Installed plugins home
  • Site configuration file
  • User configuration file
if these are not at their default locations. To help configure these paths, you can use the following system properties:
sets the location of the GATE install directory. This should point to the top level directory of your GATE installation. This is the only property that is required. If this is not set, the system will display an error message and them it will attempt to guess the correct value.
points to the location of the directory containing installed plugins (a.k.a. CREOLE directories). If this is not set then the default value of {gate.home}/plugins is used.
points to the location of the configuration file containing the site-wide options. If not set this will default to {gate.home}/gate.xml. The site configuration file must exist!
points to the file containing the user’s options. If not specified, or if the specified file does not exist at startup time, the default value of gate.xml (.gate.xml on Unix platforms) in the user’s home directory is used.
is a path-like structure, i.e. a list of URLs separated by ‘;’. All directories listed here will be loaded as CREOLE plugins during initialisation. This has similar functionality with the the -d command line option.
is a URL pointing to the location of GATE’s built-in CREOLE directory. This is the location of the creole.xml file that defines the fundamental GATE resource types, such as documents, document format handlers, controllers and the basic visual resources that make up GATE. The default points to a location inside gate.jar and should not generally need to be overridden.
As described above, the only property that is required is gate.home if you lay out other resources at their default locations.

In this article, we will show you one way to run your GATE application in Oracle WebLogic Server (WLS). This allows you to test your deployed application quickly.

Classloading in Java Platform and Oracle WebLogic Server

If the application you are creating has dependencies on some third-party code (for example, gate.jar), what is the proper way to package these libraries so that they can be used by a portable J2EE application?

In the J2EE platform, there are mechanisms[4] available for including libraries in a portable application:
  1. The WEB-INF/lib Directory
  2. Bundled Optional Classes
  3. Installed Packages (or installed optional packages mechanism)
Since these mechanisms are well-documented, they will not be repeated here.

To use these third-party libraries along with your application code, you face the decision of which packaging mechanism to choose. The decision you make can have major effects on the following:
  • The portability of your application
  • The size of your WAR and EAR files
  • The maintenance of the application
  • Version control as libraries and application servers are updated
Some solutions for packaging library JAR files are specific to a particular application server: for example, placing a library JAR file in an application server's classpath so that applications can use the APIs in that JAR file. Some application servers have container-specific locations where you can place JAR files to be shared by applications and modules. But these mechanisms are not portable, unlike the mechanisms provided by the J2EE platform.

In this article, we will introduce one WLS-specific mechanism to use for the GATE installation. This will allow you to quick-test your GATE application.

In WLS, you can place JAR files to be shared by applications and modules at the following location:
  • $DOMAIN_DIR/lib
This is the domain library directory. The domain library directory is one mechanism that can be used for adding application libraries to the server classpath. The jars located in this directory will be picked up and added dynamically to the end of the server classpath at server startup. The jars will be ordered lexically in the classpath.

It is possible to override the $DOMAIN_DIR/lib directory using the -Dweblogic.ext.dirs system property during startup. This property specifies a list of directories to pick up jars from and dynamically append to the end of the server classpath using as the delimiter between path entries.

Default GATE Installation Layout

The GATE architecture is based on components. Each component (i.e., a Java Beans), is a reusable chunks of software with well-defined interfaces that may be deployed in a variety of contexts.

You can define applications with processing pipelines using these reusable components. In GATE, these resources are officially named CREOLE (i.e., Collection of REusable Objects for Language Engineering). You can read this article to understand how GATE plugins and CREOLE resources are configured.

In the following, we show how GATE's resources are laid out in the WLS' domain library directory:
/wls_domain/lib/gatehome (i.e., GATE's home directory)
+-- lib/
+-- Bib2HTML.jar
+-- GnuGetOpt.jar
+-- ...
+-- plugins/
+-- ANNIE/
+-- ANNIE_with_defaults.gapp
+-- build.xml
+-- creole.xml
+-- resources/
+-- Tools/
+-- build.xml
+-- creole.xml
+-- doc/
+-- resources/
+-- src/
+-- tools.jar
+-- gate .xml

After you've installed GATE's libraries and resources in the domain library directory. The next step you need to do is setting gate.home property in wls_domain/bin/


Final Words

As mentioned before, this is not the best way to configure GATE's installation in a WLS. However, this approach will allow you to test your deployed GATE application quickly on it.

The domain library directory in WLS is intended for JAR files that change infrequently and are required by all or most applications deployed in the server, or by WebLogic Server itself. For example, you might use the lib directory to store third-party utility classes that are required by all deployments in a domain. You can also use it to apply patches to WebLogic Server.

The domain library directory is not recommended as a general-purpose method for sharing a JARs between one or two applications deployed in a domain, or for sharing JARs that need to be updated periodically. If you update a JAR in the lib directory, you must reboot all servers in the domain in order for applications to realize the change. If you need to share a JAR file or Java EE modules among several applications, use the Java EE libraries feature here. Alternatively, you can write custom class loaders to better fit your application's needs.

  1. Packaging Utility Classes or Library JAR Files in a Portable J2EE Application
  2. Understanding WebLogic Server Application Classloading
  3. Overview of WebLogic Server Application Classloading
  4. Mechanisms for Using Libraries in J2EE Applications
  5. Class Gate
  6. GATE Embedded
  7. Using System Properties with GATE
  8. GATE Plugins and CREOLE Resources

Tuesday, May 24, 2011

Configuring JDBC Data Sources in JDeveloper and Oracle WebLogic Server

In a previous article, we have introduced different ways of creating and managing database connections in JDeveloper. Those connections (i.e. using JDBC URL) are good for the testing of JDeveloper Design-time, Business Components Tester, and Integrated WebLogic Server.

In the production environment, the web application would typically reference a JDBC datasource, since it simplifies maintenance of the connection credentials by allowing them to be centrally managed in the Java EE container. In this article, we will show you how to:
  • Switch from JDBC URL to JDBC DataSource in JDeveloper.
  • Setup global JDBC DataSource for database connection in the WebLogic Server
  • Deploy a Web Application to the WebLogic Server
JDBC URL vs. JDBC DataSource

If you use JDBC URL connection type, you provide settings as shown below:
However, if you use JDBC DataSource connection type, its settings are done on the server. To avoid passwords being present in plain text in deployed files, JDeveloper uses password indirection, which means that passwords for the data sources must be set on the server before the application will run correctly.

A data source object enables a Java Database Connectivity (JDBC) client to obtain a DBMS connection from a JDBC pool. A data source is a vendor-independent encapsulation of a database server connection. The data source offers advantages over a JDBC URL connection because the data source can be tuned, reconfigured, or remapped without changing the deployed application.

In WebLogic Server, you configure database connectivity by adding data sources to your WebLogic domain. WebLogic JDBC data sources provide database access and database connection management. Each data source contains a pool of database connections that are created when the data source is created and at server startup.

Choosing JDBC DataSource

After you finish testing your application using Integrated WebLogic Server and decide to deploy it to a standalone WebLogic Server, then it's time to configure JDBC DataSource as your application's connection type.

This configuration step is done on the Application Module in the Business Layer.
Edit Business Components Configuration dialog is used to create a new application module configuration or edit an existing one. To open it, do:

Application Navigator context menu for application modules > Configurations > Edit > Application Module tab

You choose to connect to a JDBC DataSource from the dropdown list. The JDeveloper helps create the data source name based on available resource connections. This data source naming convention follows the Java EE standard which specifies that resource connections appear in the application-specific name space java:comp/env/jdbc/. For example, for a connection named 'SmmApp', the JDBC Datasource name would be 'java:comp/env/jdbc/SmmAppDS'.

When you deploy the application and run it in standalone Oracle WebLogic Server, the specified data source connection in the application module configuration will be used.

Setting up Global JDBC DataSource

You can create a global data source on Oracle WebLogic Server Administration Console:

To set up a global data source:
  1. Login to the Oracle WebLogic Server Administration Console. Click on the Data Sources link under JDBC. On the Summary of JDBC Data Sources page, click New. In the Create a New JDBC Data Source page, enter details of the data source.
  2. The name can be anything.
  3. The JNDI name must be of the form jdbc/connection-nameDS. For example, if the application has a connection name SmmApp, the JNDI name is jdbc/SmmAppDS. Ensure that the database type is Oracle and that the driver is Oracle’s Driver (Thin) for Service Connections;Version 9.0.1,9.2.0,10,11. Click Next twice to navigate to the Create a New JDBC Data Source page, where you enter the connection details.
  4. The database name is the Oracle SID.
  5. The host name is the the name of the machine the database is on.
  6. The default port is 1521.
  7. Enter the user name and password, for example hr/hr. Click Next and click Test Configuration. Click Next to navigate to the Select Targets page, where you select a target for this data source. If you fail to select a target, the data source is created but not deployed. Click Finish.

Steps to Deploy to WebLogic Server

Application Properties Dialog is used to to define the deployment profiles that will be available to an application. To open Deployment page, do:

Application menu > Application Properties > Deployment page

Makre sure "Auto Generate and Synchronize weblogic-jdbc.xml Descriptors During Deployment" is unchecked. When this field is not selected, this means that your deployed application will use global data sources on the server.
You can also edit the default deployment profile (i.e. SmmTest_application1--EAR File) or create a new one if needed.

There are two options for the deployment:
  1. Deployed to Ear
  2. Deployed to Application Server
If you want to deploy your application directly to the server, configure Resource Palette with a new Application Server Connection (i.e., xbox) as follows:
You then deploy your application using context menu. Right select your application and select Deploy. Choose the destination to be your application server (i.e., xbox).

  1. Database Connections in JDeveloper 11
  2. JDBC Datasources Work in the Business Components Browser in 11g
  3. Configuring JDBC Data Sources
  4. Configuring JDBC Data Sources (WebLogic)
  5. What You May Need to Know About JDBC DataSource
  6. Oracle WebLogic Server Downloads 
  7. Oracle® Fusion Middleware Configuring and Managing JDBC Data Sources for Oracle WebLogic Server 11g Release 1 (10.3.6)

Tuesday, May 17, 2011

Exposing List Model Instance as Table Component in Oracle ADF

Oracle ADF is based on the Model-View-Controller (MVC) design pattern. Separating applications into these layers simplifies maintenance and reuse of components across applications.
The Task

In this article, we will show how to create a table component (i.e., af:table) using a custom method in the Application Module. This custom method returns a List instance whose items will be displayed as rows in the table.

Here af:table exists in the View layer. Our custom method which provides table contents exists in the Business layer. This custom method is exposed on the client interface of an Application Module and exported to the Model layer as data control and binding object.

In this article, we will show how to achieve this. Before you start, read this companion article first.

The Background

The Table component uses a model to access the data in the underlying list. The specific model class is You may also use other model instances, e.g., java.util.List, array, and javax.faces.model.DataModel. The Table will automatically convert the instance into a CollectionModel.

Normally, you would create table using ADF View Object which is exported to the Model layer. You create an ADF bound ADF Faces table by dragging a collection from the ADF Data Controls panel to a page or page fragment. Using this approach, you can get access to its CollectionModel easily. However, for some cases, creating a View Object can be difficult. For example, our query using Oracle Semantic Technology doesn't allow us to use bind variable.
select  postab.post_id, postab.content_value
from table(sem_match('(?s <>   <>)',
  sem_rulebases('owlprime'), null, null))  graph,
  smm.smm_vpd_rdf_data_1 app,
  smm.smm_posts postab
where app.triple.rdf_p_id=oracle_orardf_res2vid('')
  and app.triple.rdf_c_id=oracle_orardf_res2vid('')
  and graph.s$RDFVID=app.triple.rdf_s_id
  and postab.post_id=app.post_id
Ideally, we would like to make object in the RDF triple a bind variable. Since sem_match is an operator and doesn't allow that. The workaround will be creating a custom method which returns a List instance. Within the method, we create a custom query statement based on the name of an object (i.e., "oracle"). The signature of the custom method will be:
public List<postobj> positivePostsByBrand(String brandName) {
The Steps

Because of dependencies, we will work from Business Layer back to View Layer. All Fusion Web Applications are created with two projects:
  • Model
  • ViewController
For step 1 below, we will work in the Model project. For other steps, we will work in the ViewController project.

Step 1: Add Custom Method to the AM

Add our custom method (i.e., positivePostsByBrand) to the Application Module:
Shuttle custom method from the available list to the selected list:
In the Data Controls panel, you should find the new custom method listed under your application module.

Step 2: Create MethodAction Binding Object

Select the JSF page to which you want to add the new table. In its Bindings view, create a new binding object using the new data control object (i.e. our custom method positivePostsByBrand). By default, the binding objects are named after the data control object that was used to create it.
Our new binding object will be methodAction (under Generic Bindings) item type:

On Create Action Binding, select your AM (i.e., SemAppModuleDataControl). No, don't expand its contents. Look underneath for an Operation drop-down list. Select your custom method. Take a note of the name of its parameter (i.e., brandName) which you need to use later.
LinkStep 3: Create a New Getter Method on a Managed Bean

We need a getter method on a managed bean, which will fetch table contents on behave of View layer. For this exercise, we have created a managed bean named ActionBean which it has been registered on the ADF's unbounded task flow (i.e., adfc-conf.xml):

This managed bean has a session scope. Our getter method in looks like this:
public List<PostObj> getPositivesByBrand() {
  if (getBrandName() == null || getBrandName().length() == 0)
    return new ArrayList<PostObj>();
  else {
    OperationBinding oper = getOperBindings("positivePostsByBrand")  ;
    Map params = oper.getParamsMap();
    params.put("brandName", brandName);
    return (List<PostObj>)oper.getResult();
Within it, it uses a binding object named positivePostsByBrand to fetch table contents from the business layer (i.e., via our custom method defined in the Application Module implementation file). To execute our custom method, we also need a parameter named brandName. We will discuss how to provide this parameter at runtime later.

Step 4: Create New Table on JSF Page

Select the component (i.e., af:panelHeader) that you want to insert the new table to and insert a new ADF Faces component (i.e., af:table):
On Insert ADF Faces Item, select Table:
On Create ADF Faces Table, check Bind Data Now and click Browse...:
You will find our variable named positivesByBrand (i.e., exposed by the getter method) under ADF Managed Bean > actionBean. Select it and an EL will be created automatically for you.
Next we will create new columns mapped to the properties of the List instance object (i.e., PostObj). Click New:
Add new columns as shown. Their values will be mapped to the property's on our List instance object (i.e.,PostObj).
The Missing Link

As noted above, to execute the custom method we need a parameter (i.e., brandName). To provide that, we need to dynamically select that name from the UI. For example, in a master table, you may display rows with different brand name on each row. The brand name (or the count in our case) can then be implemented as an af:commandLink:

<af:commandLink  id="cl1" actionListener="#{actionsBean.positivePostsByBrand}">
  <af:outputText value="#{row.count}" id="ot5"/>
  <f:attribute name="posBrandName" value="#{row.brand}"/>

Upon clicking, it will dispatch an ActionEvent which will be consumed by a listener (i.e., positivePostsByBrand method on our managed bean). Within the method, it will retrieve an attribute named posBrandName from the event's source object as such:
public void positivePostsByBrand( ActionEvent actionEvent){
  RichCommandLink _table = (RichCommandLink) actionEvent.getSource();
  String brandName = (String) _table.getAttributes().get("posBrandName");
  // set brand name to be used in getPositivesByBrand method

Note that attribute posBrandName will be set to be:
as defined in the af:commandLink.

  1. ADF Faces Rich Client
  2. Implement Contextual Event
  3. How-to Build a Generic Selection Listener for ADF bound Tables
  4. Invoking Application Module Custom Methods from Oracle ADF Frontend—Why and How?
  5. Action Bindings and Method Bindings in Oracle ADF 11g

Sunday, May 15, 2011

Diagnosing Java.lang.OutOfMemoryError

The heap is one of the foremost components that should be monitored to trace performance issues[18, 19,22]. Heap pressure is created when the heap usage approaches the maximum heap size permitted. This leads to frequent full garbage collection events. This steals CPU cycles available for processing and the overall response times degrade. Extreme cases can lead to OutOfMemory conditions, which are not recoverable without a JVM restart.


When I ran my application, the following exceptions have been thrown in sequence:
  • java.lang.OutOfMemoryError: GC overhead limit exceeded[7,9,15]
  • java.lang.OutOfMemoryError: Java heap space
The first message means that, for some reason, the garbage collector is taking an excessive amount of time and recovers very little memory in each run. After I removed the following statement:
  • System.gc();
The 1st message was gone. However, the system threw the 2nd message. So, obviously my heap space issue remains. Here are the steps that I took to investigate it:
  1. Add the following Java Options
    • -Xloggc:gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
      • System will generate a gc.log file
    • -XX:+HeapDumpOnOutOfMemoryError
      • System will generate a heap dump file
      • There is an additional HotSpot VM command line option that allows a user to specify a path where the heap dump will be placed
        • -XX:HeapDumpPath=
  2. Analyze the log files:
    • Use regular text editor to examine gc.log file
    • Use Eclipse Memory Analyzer to examine heap dump file (i.e., java_xxx.hprof)
    Note that all command options discussed in this article applied to Hotspot VM.

    JVM Options

    The command line options specify:
    • -XX:+PrintGCDetails
      • prints more details at garbage collection.
    • -XX:+PrintGCTimeStamps
      • prints a time stamp representing the number of seconds since the HotSpot VM was launched until the garbage collection occurred.
    • -Xloggc:gc.log
      • causes information about the heap and garbage collection to be printed at each collection.

    To set Java Options in JDeveloper, do the following:
    1. Right select your project (i.e., ViewController) and bring up the context menu
    2. Select Project Properties...
    3. Select Run/Debug/Profile
    4. Select your Run Configuration (i.e., Default)
    5. Click Edit button
    6. Specify -Xloggc:gc.log -XX:-PrintGCDetails in the Java Options field
    Run your application and reproduce the out-of-memory exception. A log file named gc.log will be generated. I've found mine in the following default location:
    • .../system11.
    because my web application was deployed to the Integrated WLS[4] and run from DefaultDomain. To understand the format of gc.log, read [5,15] for details.

    However, gc.log file was not really helpful because it simply pointed out there was a heap issue. But, it didn't say where.

    The next step I have taken is running my server with the following flag:


    it generated a java_pid30835.hprof file when my server encountered a heap error.

    Eclipse Memory Analyzer

    The heap dump file (i.e., java_pid30835.hprof) is generated by HPROF—a heap and cpu profiling tool. My heap dump file was generated in binary format. Therefore I need to use Eclpse Memory Analyzer to examine it.

    You can install Eclipse MAT via the Eclipse Update manager . Select "General Purpose Tools " and install "Memory Analyser (Incubation)" and "Memory Analyser (Charts)".
    After installation, double-click your heap dump file and select "Leak Suspects Report".
    Eclipse MAT will show a diagram:
    and problem suspects:
    You can click on the "Details" link to investigate.

    Heap Size Adjustment

    If you observe many full GCs, try to determine if your old generation is sized too small to hold all the live objects collected from the Survivor and Eden spaces. Alternatively, there may be too many live objects that do not fit into the configured heap size. If it is the latter, increase the overall heap size.

    Based on whether the old generation space or the permanent generation space is running out of memory, you may adjust the sizes of heap and meta spaces in this way[10]:
    • For old generation space OutOfMemoryErrors
      • Increase -Xms and -Xmx
    • For permanent generation OutOfMemoryErrors
      • Increase -XX:PermSize and -XX:MaxPermSize

    1. Eclipse Update Manager
    2. Eclipse Memory Analyzer
    3. Java Hotspot VM Options
    4. Integrated WebLogic Server (WLS)
    5. Diagnosing a Garbage Collection problem
    6. Frequently Asked Questions about Garbage Collection
    7. GC Overhead Limit Exceeded
    8. HPROF: A Heap/CPU Profiling Tool in J2SE 5.0
    9. Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning
    10. Java Performance by Charlie Hunt and Binu John
    11. Understanding Garbage Collection
    12. Java HotSpot VM Options
    13. GCViewer (a free open source tool)
    14. Understanding Garbage Collector Output of Hotspot VM
    15. A Case Study of java.lang.OutOfMemoryError: GC overhead limit exceeded
    16. Memory Analyzer Downloads
      • The stand-alone Memory Analyzer is based on Eclipse RCP.
      • Can find the update site here too.
    17. Shallow vs. Retained Heap (MAT)
    18. Diagnosing Heap Stress in HotSpot (XML and More)
    19. Diagnosing OutOfMemoryError or Memory Leaks in JRockit (XML and More)
    20. MAT Documentation
    21. Out of Memory Error while Running the Memory Analyzer
      • I need to modify -Xmx512m to -Xmx4g in the file eclipse.ini to analyze a 2GB hprof file.
    22. Eclipse MAT: Querying Heap Objects Using OQL (Xml and More)
    23. Eclipse MAT: Understand Incoming and Outgoing References  (Xml and More)
    24. Memory Mapped File and IO in Java
      • Memory used to load Memory mapped file is outside of Java heap Space.  If OOM is caused by memory-mapped files, you may want to reduce your Java heap allocation (i.e., reducing -Xmx).
        • Map failed

    Sunday, May 1, 2011

    Invoking Application Module Custom Methods from Oracle ADF Frontend—Why and How?

    In an Oracle Fusion web application, you should use managed beans[4] to store logic that is related to the UI rendering only. All application data and processing should be handled by logic in the business layer of the application.

    In this article, we'll show how to refresh a calculated attribute[6] named Total Comments after user adds a new comment using Create Comment Dialog. As noted above, refreshing Total Comments should be handled in ADF backend (i.e., ADF Business Components), not ADF frontend (i.e., ADF Faces). Note that ADF backend refers to Model project and ADF frontend refers to ViewController project in a Fusion web application.

    Background Information

    In the described application, we have two view objects (i.e., TestVO and TestCommentVO) with master-detail relationship. "TotalComments" on the TestVO is a calculated attribute which computes total number of chronological comments (represented by TestCommentVO) added to a test.

    On the Test View page, it displays Total Comments column which is also a link. By clicking on the link, user can navigate to the Test Comment page. On the Test Comment page, user is allowed to add new comments. After adding new comments, the Total Comments column on the original Test View page need to reflect the new total.

    The Task

    To achieve this task, we need to invoke a custom method implemented in ADF Business Components from the Test Comment component implemented in ADF Faces. Method invokation is triggered when user clicks the Save and Close button to save the newly created comment.
    The bridge provided between ADF frontend and ADF backend is the Data Binding Layer in Fusion web application[3]. ADF data-binding layer implements the Data Controls and Bindings from JSR-227. The data controls abstract the back-end business service implementations and binding objects link front-end user interface controls in a declarative way to back-end data.

    The steps of the task ordered from backend (in blue) to frontend (in black) are summarized here:
    1. Creating a custom method named refreshTotalComments in the application module Class (i.e.,
    2. Exporting refreshTotalComments by including the method on the application module's UI client interface.
    3. Adding a new action binding[5] named refreshTotalComments on TestCommentPageDef.xml.
    4. Creating a managed bean named with a method action named saveAndClose. In the method, it will retrieve and execute refreshTotalComments operation binding.
    5. Registering new managed bean in adc-config.xml with the scope of request.
    6. Linking method action saveAndClose on the managed bean to the actionlistener of Save and Close button.
    In the following sections, we'll describe those steps in details.

    Step 1 — Adding Custom Method to the AppModuleImpl File

    To add a custom service method (i.e., refreshTotalComments) to your application module, you must generate the application module class files first.

    To generate Java files for your application module class:
    1. In the Application Navigator, double-click the application module.
    2. In the overview editor, click the Java navigation tab and click the Edit java options button.
    3. In the Select Java Options dialog, select Generate Application Module Class.
    4. Click OK.

    In file, you add your custom service method — refreshTotalComments. Here we have left out the details of refreshTotalComments. For ADF developers need to write a custom method such as refreshTotalComments, you can read this document. It provides a high-level description of the key ADF Business Components classes in the Oracle Application Development Framework and it summarizes the methods that ADF developers write, call, and override most frequently.

    Step 2 — Publishing Custom Method on the Application Module’s Client Interface

    To include a public method from your application module's custom Java class on the client interface, use the Java Classes page of the overview editor for the application module, and then click the Edit icon in the Client Interface section of the page to display the Edit Client Interface dialog. Select method refreshTotalComments from the Available list and click the Add button to shuttle them into the Selected list. Then click OK to close the editor.

    You can include any custom method in the client interface that obeys these implementation rules:
    • If the method has a non-void return type, the type must be serializable (i.e., implements the interface).
    • If the method accepts any parameters, all their types must be serializable.
    • If the method signature includes a throws clause, the exception must be an instance of JboException in the oracle.jbo package.

    Step 3 — Adding a New Action Binding

    In the ViewController project, select Test Comment page and right click it. From the context menu, select "Go to Page Definition." This will bring you to the TestCommentPageDef.xml.

    In the Overview editor for the Test Comment page definition’s Bindings and Executables tab, click the Add icon in the Bindings section. Then, follow these steps:
    1. In the Insert Item dialog, select methodAction and click OK.
    2. In the Create Action Binding dialog:
      • Select the data collection (i.e., SelTestPortalAMDataControl) where you have created your handler.
      • From the Operation dropdown list, select the handler (i.e., refreshTotalComments).
      • Click OK.

    Step 4 — Creating a New Managed Bean

    In the ViewController project, create a managed bean named with a method action named saveAndClose as shown below.
    public void saveAndClose(ActionEvent actionEvent) {
    // Commit the transaction
    OperationBinding  oper = getOperBindings("Commit");
    // refresh total comments
    oper = getOperBindings("refreshTotalComments");
    }catch(Exception e){
    * This methos returns the operation bindings of the passed method name
    * @param operation name
    * @return
    private OperationBinding getOperBindings(String operation) throws Exception
    OperationBinding oper=null;
    FacesContext facesContext = FacesContext.getCurrentInstance();
    ExpressionFactory exp = facesContext.getApplication().getExpressionFactory();
    DCBindingContainer bindingContainer =
    (DCBindingContainer) exp.createValueExpression(facesContext.getELContext(),
    // A ControlBinding that binds an datacontrol action, including navigational
    // actions such as "Next" and "Prev", or other actions such as "Commit" and
    // "Rollback", to a view component.
    oper = bindingContainer.getOperationBinding(operation);
    catch(Exception e)
    return oper;
    } //end getOperBindings

    As shown in the code, we need to retrieve two operation bindings (i.e., Commit and refreshTotalComments) from the binding container[7]. Operation binding refreshTotalComments is used at runtime to invoke our Application Module custom method.

    Step 5 — Registering New Managed Bean in adc-config.xml

    As a general rule for Fusion web applications, a bean that may be used in more than one page or task flow, or one that is used by pages within the main unbounded task flow (adfc-config), should be registered in the adfc-config.xml configuration file.

    To register TotalCommentBean in adc-config.xml, do the following:
    1. In the Application Navigator, double-click the adfc-config.xml file in theWEB-INF folder.
    2. In the editor window for the adfc-config.xml file, click the Overview tab.
    3. In the Managed Beans page, in the Managed Beans section click the Add icon and enter TotalCommentBean as the name of the bean, enter the fully qualified class name, and select request scope[8].

    Step 6 — Linking Method Action to the ActionListener of Button

    The final step is to link method action saveAndClose on the managed bean to the actionListener of Save and Close button as shown below:
    <af:commandbutton id="FAsc1"
    Note that you can achieve this linking declaratively using Property Inspector.

    The Explanation

    An application module can expose its data model of view objects to clients without requiring any custom Java code. This allows client code to use the ApplicationModule, ViewObject, RowSet, and Row interfaces in the oracle.jbo package to work directly with any view object in the data model.

    Whenever there is a need to provide custom codes as shown in our sample application, you should encapsulate the details by writing a custom method in your application module's Java class.

    When working with Fusion web applications using the ADF Model layer for data binding, JDeveloper configures a servlet filter in your user interface project (i.e., ViewController) called the ADFBindingFilter. It orchestrates the automatic acquisition and release of an appropriate application module instance based on declarative binding metadata, and ensures that the service is available to be looked up as a data control using a known action binding or iterator binding, specified by any page definition file in the user interface project.

    At runtime, you can access the application module's client interface from the DCBindingContainer by naming an ADF action binding or an ADF iterator binding (see step 4). You can also reference the binding context[7] and call methods on the custom client interface in any JSF managed bean.


    1. Most Commonly Used Methods in ADF Business Components
    2. A detailed look at Binding Model Parameter Options (NDOption)
    3. Oracle ADF Model In Depth
    4. Managed Beans in Oracle Fusion Web Applications
    5. Action Bindings and Method Bindings in Oracle ADF 11g
    6. Examining View Object Attributes in Oracle ADF 11g
    7. Oracle ADF BindingContext and BindingContainer
    8. Types of Memory Scopes in Fusion Web Applications
    9. Oracle Fusion Developer Guide: Building Rich Internet Applications with Oracle ADF Business Components and Oracle ADF Faces
    10. Open Source sample applications for Oracle Fusion Middleware technology
    11. Using Bind Variable to Implement Range Selection Declaratively (Xml and More)
    12. Book Review: Developing Web Applications with Oracle ADF Essentials (Xml and More)