Productive Enterprise Web Development with ExtJS and Clear Data Builder

It’s not likely that you’ll start developing an enterprise HTML5 applications without using one of the JavaScript frameworks. One of the most feature-complete frameworks is ExtJS from Sencha, but its learning curve is a bit steep. Our company, Farata Systems, has developed an open source software Clear Toolkit for ExtJS. Clear Toolkit includes an Eclipse plugin called Clear Data Builder (CDB). This is a productivity tool – a code generator – that can create a CRUD application for you in no time. This application will have HTML/JavaScript/ExtJS client and Java-based server. In this article you will learn how jump start development of such Web applications.

Part One: ExtJS MVC Application Scaffolding

In part one I’ll cover the following topics:

  • What is Clear Toolkit for ExtJS
  • How to create ExtJS MVC application for Java-based project
  • How deploy and run your first ExtJS+Java application on Apache Tomcat server

Clear Toolkit for ExtJS contains of following parts:

  • Clear Data Builder – Eclipse plugin that supports code generation ExtJS MVC artifacts based on Java code. CDB comes with wizards to start new project with plain Java or with popular frameworks like Hibernate, Spring, MyBatis.
  • Clear JS – set of JavaScript components that extends ExtJS standard components. E.g. Clear JS contains ChangeObject – a universal way to trace the state changes between old and new versions of the same item in a store.
  • Clear Runtime – Java components that implements server side part of ChangeObject, DirectOptions and etc.

The phrase “to be more productive” means to write less code and produce the results faster. This is what CDB is for. In this article you’ll see how Clear Data Builder helps you to integrate the client side with the back end using the RPC style and how to implements data pagination for your application.

CDB distribution available as plug-in for Eclipse IDE. The current update site of CDB is located here. The current version is 4.1.4 (don’t be surprised – this is a five-year old code generator, and its previous versions were made for generating the UI for Adobe Flex framework). You can install this plug-in via Install New Software menu in Eclipse IDE. The [FIG-1] shows how you can validate plug-in installation. If you see “Clear Data Builder for Ext JS feature” in the list of Installed Software in your Eclipse IDE, you’re good to go.

Important You have to have “Eclipse for Java EE Developers” installed, which includes the plugins for automation of the Web applications.
image
Figure 1. Verifying CDB installation

Clear Data Builder comes with a set of prepared examples that demonstrate the integration with popular Java frameworks – MyBatis, Hibernate, and Spring. Also, a plain Java project example that doesn’t use any of the frameworks is available as well. Let’s start with the creation of the new project by selecting the menu File → New → Other → Clear, and then press Next.

image
Figure 2. New CDB Project Wizard

First of all let’s call the new project episode_1_intro. CDB supports different ways of linking the ExtJS framework to the application. In my case, I already have installed ExtJS libraries under my Web server (Apache Tomcat). We’re going to use this local ExtJS URL, but you can just specify any folder in your machine and CDB will copy the ExtJS file inside your project. Lastly, you can use ExtJS from the Sencha’s CDN, if you don’t want to store these libraries inside your project. Besides, using a common CDN will allow Web browser to reuse the cached version of ExtJS.

For this project we are not going to use any server-side (like MyBatis or Hibernate). Just click the button Finish. First of all, CDB will print some initial messages on the Eclipse console. When CDB runs for the first time it initializes directory structure in the WebContent folder. Inside the WebContent directory CDB creates directory structure which is recommended by Sencha for MVC applications. Also you’ll get the HTML wrapper – index.html – for this application, which contains the link to the entry point of our application.

CDB generates an empty project with one sample controller and one view – Viewport.js. To run this application, you need to add the newly generated Dynamic Web Project to Tomcat and start the server (right-click on the Tomcat in the Servers view of Eclipse).

image
Figure 3. Adding web project to Tomcat

Let me switch to web browser to open this application on http://localhost:8080/episode_1_intro . Voila! Just in couple minutes we’ve setup new Dynamic Web Project with the ExtJS framework support and one fancy button on UI.

image
Figure 4. Running scaffolded application

The next step is to make something useful out of this basic application.

Part Two: Generating a CRUD application

CRUD stands for Create-Retrieve-Update-Delete. It’s a well known term for describing the applications that support data manipulation. They can retrieve the data from some data source and update them too. The Part Two agenda is covering exactly this:

  • Create a simple CRUD ExtJS+Java application
    • Create a POJO and the corresponding Ext.data.Model
    • Create a Java service and populate Ext.data.Store with data from service
    • Use the auto-generated ExtJS application
    • Extend the auto-generated CRUD methods
    • Use ChangeObject

Now I would like to show how to use Clear Data Builder to create a CRUD application. I’ll show you how you can turn your Java POJO class into the ExtJS model. I’ll explain the following:

  • how you can populate the ExtJS store from a remote service
  • how you can use automatically generated UI for that application
  • how you can extend it
  • what the ChangeObject class is for

I will extend the application from Part 1. For my CRUD application I need a Java POJO. First, I’ve created the class Person.java in the package dto. Then I’ve added the fields firstName, lastName, address, ssn and phone and id. Also I need getters and setters for these fields. It’s good to have a constructor for the that uses these fields, and a DTO class should have a toString() method. [LISTING_1]
Now I need the same corresponding ExtJS model for my Person. Just annotate this class with the CDB annotation called @JSClass to ask CDB to generate the ExtJS model.

Person data transfer object

package dto;

import com.farata.dto2extjs.annotations.JSClass;
import com.farata.dto2extjs.annotations.JSGeneratedId;

@JSClass
public class Person {

  @JSGeneratedId
  private Integer id;
  private String firstName;
  private String lastName;
  private String phone;
  private String ssn;

  public Person(Integer id, String firstName, String lastName, String phone,
      String ssn) {
    super();
    this.id = id;
    this.firstName = firstName;
    this.lastName = lastName;
    this.phone = phone;
    this.ssn = ssn;
  }

  // Getters and Setter are omitted

}

Now I need to annotate the id field with the CDB annotation @JSGeneratedId. With this annotation I’ll instruct CDB to threat this field as auto generated id. Let’s examine the directory of ExtJS MVC application and lets take a look inside the model folder. Inside the model folder (the JavaScript section) we have the folder dto which corresponds to the Java dto package where the class PersonModel.java resides.

image
Figure 5. Generated from Java class ExtJS model

As you can see Clear Data Builder generated two files as recommended by the Generation Gap pattern, which is about keeping the generated and handwritten parts separate by putting them in different classes linked by inheritance. Let’s open the person model. In our case the PersonModel.js is extended from the generated _PersonModel.js. Should we need to customize this class, we’ll do it inside the PersonModel.js, but this underscore-prefixed file will be regenerated each and every time when we change something in our model. CDB follows this pattern for all generated artifacts – Java services, ExtJS models and stores. This model contains all the fields from our Person DTO.

Now we need to create a Java service to populate the ExtJS store with the data. Let’s create an interface PersonService in the package service. This service will to return the list of persons. This interface contains one method -List<Person> getPersons().

I need to ask CDB to expose this service as a remote object, which is done by the annotation called @JSService. Another annotation @JSGenetareStore will instruct CDB to generate the store. In this case CDB will create the destination-aware store. This means that store will know from where to populate its content. All configurations of the store’s proxies will be handled by the code generator. With @JSFillMethod annotation we will identify our main read method (remember the “R” from CRUD).

Also it would be nice to have some sort of UI to test the service – the annotation @JSGenerateSample will help here. CDB will examine the interface PersonService, and based on these annotations will generate all ExtJS MVC artifacts (models, views, controller) with the sample application.

PersonService interface annotated with CDB annotations

@JSService
public interface PersonService {
    @JSGenerateStore
    @JSFillMethod
    @JSGenerateSample
    List getPersons();
}

When the code generation is complete, you’ll get the implementation for the service – PersonServiceImpl. The store folder inside the application folder (WebContent\app) has the store, which is bound to the PersonModel. And my person model was generated previously. In this case, Clear Data Builder generated store that binds to the remote service.

image
Figure 6. Structure of store and model folders

All this intermediate translation from the JavaScript to Java and from Java to JavaScript is done by DirectJNgine, which is a server side implementation of the Ext Direct Protocol. You can read about this protocol in Ext JS documentation.

There is one more thing not to be missed – Clear Data Builder generated a UI for us! Check out the samples directory shown on [FIG-7].

image
Figure 7. Folder with generated samples

You can see the SampleController and SampleGridPanel inside the samples folder. CDB also generates the JavaScript application entry point – sampleApp.js. To test this application just copy the file SampleController.js into the controller folder, SampleGridPanel.js into the view folder, and the sample application in the root of our WebContent folder. We need to change the application entry point with sampleApp in index.html.

This is how the generated UI of the sample application looks like [FIG-7]

image
Figure 8. Scaffolded CRUD application template

Let’s return to the server side code. For services CDB also follows the Generation Gap Pattern and it generated stubs for the service methods. Override these methods when you’re ready to implement the CRUD functionality [LISTING_3].

Implementation of PersonService interface
package service;
import java.util.ArrayList;
import java.util.List;

import clear.data.ChangeObject;
import dto.Person;
import service.generated.*;

public class PersonServiceImpl extends _PersonServiceImpl { // 

  @Override
  public List getPersons() {        // 
    List result = new ArrayList&lt;&gt;();  
    Integer id= 0;
    result.add(new Person(++id, "Joe", "Doe", "555-55-55", "1111-11-1111"));
    result.add(new Person(++id, "Joe", "Doe", "555-55-55", "1111-11-1111"));
    result.add(new Person(++id, "Joe", "Doe", "555-55-55", "1111-11-1111"));
    result.add(new Person(++id, "Joe", "Doe", "555-55-55", "1111-11-1111"));
    return result;    //
  }

  @Override
  public void getPersons_doCreate(ChangeObject changeObject) {  // 
    Person dto = (Person) deserializeObject(
        (Map) changeObject.getNewVersion(),
        Person.class);

    System.out.println(dto.toString());
  }

  @Override
  public void getPersons_doUpdate(ChangeObject changeObject) {  // 
    // TODO Auto-generated method stub
    super.getPersons_doUpdate(changeObject);
  }

  @Override
  public void getPersons_doDelete(ChangeObject changeObject) {  // 
    // TODO Auto-generated method stub
    super.getPersons_doDelete(changeObject);
  }

}
1. We need to extend the generated class and provide the actual implementation.
2. getPerson is our retrieve method (the R in CRUD) .
3. For this sample application we can use java.util.ArrayList class as in-memory server side storage of the Person objects. In real world applications you’d use a database or other persistent storage.
4. fillmethod+doCreate() is our create method (the C in CRUD).
5. fillmethod+doUpdate is our update method (the U in CRUD).
6. fillmethod+doDelete is our delete method (the D in CRUD).

Click on the Load menu and you’ll get 4 persons from our server.

To demonstrate the rest of the CRUD methods we’ll ask the user to insert one new row, modify three existing ones and remove two rows using the generated Web client. The Clear.data.DirectStore object will automatically create a collection of six `ChangeObject`s – one to represent a new row, three to represent the modified ones, and two for the removed rows.

When the user clicks on the Sync menu the changes will be sent to the corresponding do… remote method. When you sync() a standard Ext.data.DirectStore ExtJS is POST-ing new, modified and deleted items to the server. When the request is complete the server’s reply data is applied to the store, expecting that some items can be modified by the server. In case of Clear.data.DirectStore instead of passing around items, we pass the delta, wrapped in the ChangeObject.

Each instance of the ChangeOject contains the following:

  • newVersion – it’s an instance of the newly inserted or modified item. On the Java side it’s available via getNewVersion().
  • prevVersion – it’s an instance of the deleted of old version of modified item. On the Java side it’s available via getPrevVersion().
  • array of changepropertyNames if this ChangeObject represents an update operation.

The rest of ChangeObject details described on wiki on github (see the link in Useful Links section).

The corresponding Java implementation of ChangeObject is available on the server side and Clear Toolkit passes ChangeObject instances to the appropriate do* method of the service class. Take a look at the getPersons_doCreate() method from [LISTING_3]. When the server needs to read the data arrived from the client your Java class has to invoke the method changeObject.getNewVersion(). This method will return the JSON object that you need to deserialize into the object Person. This is done in [LISTING_3] and looks like this.

 Person dto = (Person) deserializeObject(
                (Map) changeObject.getNewVersion(),Person.class);

When the new version of the `Person` object is extracted from the `ChangeObject` you can do with it whatever has to be done to persist it in the appropriate storage. In our example we just print the new person information on the server-side Java console. This is why we said earlier, that it may be a good idea to provide a pretty printing feature on the class `Person` by overriding method `toString()`. Similarly, when you need to do a delete, the `changeObject.getPrevVersion()` would give you a person to be deleted.

Part Three: Data Pagination

The pagination feature is needed in almost every enterprise web application. Often you don’t want to bring all the data to the client at once – a page by page feeds is a lot more responsive. The user can navigate back and forth between the pages using pagination UI components. To do that, we need to split our data on the server side into chunks so we can display it only once specific page is required. Implementing pagination is the agenda for the Part Three of this article. Here’s the plan:

  • Add the data pagination to our sample CRUD ExtJS+Java application:
    • Add the Ext.toolbar.Paging component
    • Bind both grid and pagingtoolbar to the same store
    • Use DirectOptions class to read the pagination parameters

We are going to extend our CRUD application by adding the paging toolbar component bound to the same store as the grid. The class DirectOptions will handle pagination parameters on the server side.

In Part 1 and 2 you’ve learned how to generate the UI from the Java back end service and how to generate the ExtJS store and ExtJS model. In this part you’ll how to add the pagination functionality to our CRUD application by asking the server to send only portions of data. I’ll need to refactor the service code from previous example to generate a little bit more data than only five records. [LISTING_4]

Refactored implementation of PersonService Interface


public class PersonServiceImpl extends _PersonServiceImpl {
  @Override
    public List<Person> getPersons() {
        List<Person> result = new ArrayList<>();
        for (int i=0; i<1000; i++){
            result.add(new Person(i, "Joe", "Doe", "555-55-55", "1111-11-1111"));
        }
        return result;
    }   
}

The Google Chrome Console shows that PersonStore was populated with one thousand records. Let’s add the pagination component using the Ext toolbarpaging component. Let’s add it to the file sampleApp.js [LISTING_5].

Sample Application Entry
Ext.Loader.setConfig({
  disableCaching : false,
  enabled : true,
  paths : {
    episode_3_pagination : 'app',
    Clear : 'clear'
  }
});

Ext.syncRequire('episode_3_pagination.init.InitDirect');
// Define GridPanel
var myStore = Ext.create('episode_3_pagination.store.dto.PersonStore',{});    //
Ext.define('episode_3_pagination.view.SampleGridPanel', {
  extend : 'Ext.grid.Panel',
  store : myStore,
  alias : 'widget.samplegridpanel',
  autoscroll : true,
  plugins : [{
    ptype : 'cellediting'
  }],
  dockedItems: [
    {
      xtype: 'pagingtoolbar',    //
      displayInfo: true,
      dock: 'top',
      store: myStore      //
    }
  ],
  columns : [
    {header : 'firstName', dataIndex : 'firstName', editor : {xtype : 'textfield'}, flex : 1 },
    {header : 'id', dataIndex : 'id', flex : 1 },
    {header : 'lastName', dataIndex : 'lastName', editor : {xtype : 'textfield'}, flex : 1 },
    {header : 'phone', dataIndex : 'phone', editor : {xtype : 'textfield'}, flex : 1 },
    {header : 'ssn', dataIndex : 'ssn', editor : {xtype : 'textfield'}, flex : 1 }],
  tbar : [
    {text : 'Load', action : 'load'},
    {text : 'Add', action : 'add'},
    {text : 'Remove', action : 'remove'},
    {text : 'Sync', action : 'sync'}
    ]
  });
// Launch the application
Ext.application({
  name : 'episode_3_pagination',
  requires : ['Clear.override.ExtJSOverrider'],
  controllers : ['SampleController'],
  launch : function() {
    Ext.create('Ext.container.Viewport', {
      items : [{
        xtype : 'samplegridpanel'
      }]
    });
  }
});
1. Let’s manually instantiate this store – create a separate variable myStore for this store and with empty config object.
2. Adding the xtype pagingtoolbar to this component docked items property – I’d like to display the information and dock this element at the top.
3. Now the paging toolbar is also connected to same store.

Now we need to fix automatically generated controller to control loading of data on
click of Load button [LISTING_6].

Controller for sample application

Ext.define('episode_3_pagination.controller.SampleController', {
  extend: 'Ext.app.Controller',
  stores: ['episode_3_pagination.store.dto.PersonStore'],
  refs: [{                //
    ref: 'ThePanel',
    selector: 'samplegridpanel'
  }],

  init: function() {
    this.control({
      'samplegridpanel button[action=load]': {
        click: this.onLoad
      }
    });
  },

  onLoad: function() {
    // returns instance of PersonStore
    var store = this.getThePanel().getStore();    //
    store.load();
  }
});

1. We can bind the store instance to our grid panel. In controller’s refs property I’m referencing our simplegrid panel with ThePanel alias.
2. In this case I don’t need to explicitly retrieve the store instance by name. Instead, we can use getters getPanel() and getStore() automatically generated by the ExtJS framework.

When the user clicks the button next or previous the method loadPage of the underlying store is called. Let’s examine the directprovider URL – server side router of remoting calls – and see how the direct request looks like. Open Google Chrome Developer Tools from the menu View → Developer, refresh the Web page and go to the Network tab. You’ll that each time the user clicks on the next or previous buttons on the pagination toolbar the component sends directOptions as a part of the request.

image
Figure 9. Request payload details

Default Ext Direct request doesn’t carry any information about page size. Clear JS, the client side extension of the ExtJS framework, adds some extra functionality to Ext.data.DirectStore component to pass the page start and limit values to the server side. At this point, the directOptions request property shown in [FIG-9] can be extracted on the server side to get the information about the page’s boundaries. Let’s return to the code of PersonServiceImpl and add some extra code there. Right now the pagination doesn’t work. The server sends the entire thousand records, because the server is not aware that the data has to be paginated. We’ll fix it in [LISTING_7].

Implementation of PersonService with pagination

package service;
import java.util.ArrayList;
import java.util.List;

import clear.djn.DirectOptions;      //

import dto.Person;
import service.generated.*;

public class PersonServiceImpl extends _PersonServiceImpl {
  @Override
  public List getPersons() {
    List result = new ArrayList&lt;&gt;();
    for (int i=0; i
    int start = ((Double)DirectOptions.getOption("start")).intValue();
    int limit = ((Double)DirectOptions.getOption("limit")).intValue();

    limit = Math.min(start+limit, result.size() );    //
    DirectOptions.setOption("total", result.size());  //
    result = result.subList(start, limit);      //

    return result;
  }
}

1. On the server side there is a special object called DirectOptions, which comes with Clear Toolkit.
2. We’re interested in start and in limit options (see [FIG-9]).
3. Calculate the actual limit. Assign the size of the data collection to the limit variable if it’s less than the page size (start+limit).
4. Notify the component about the total number of elements on the server side by using DirectOptions.setOption() method with total option.
5. Before returning the result, create a subset, an actual page of data. In this case I need to use method of java.util.List.sublist() which produces the view of the portion of this list between indexes specified by the start and the limit parameters.

As you can see from the Network tab in [FIG-8], we limited the data load to 25 elements per page. Clicking on next or previous buttons will get you only a page worth of data. The Google Chrome Developers Tools Network tab shows that that we are sending start and limit every time and our response contains object with 25 elements.

If you’d like to repeat all of the above steps on you own, watch the screencasts where I perform all the actions described in this article.

Conclusion

The development of enterprise web application involves many steps that need to be done by developer. But with the right set of tools the repetitive steps can be automated. Remember the DRY principle – don’t repeat yourself. Try to do more with less efforts. I hope this article will help your to get started start with Clear Data Builder, which should make any ExtJS enterprise software developer more productive.

Subscribe to our youtube channel to get access to the latest videos, screencasts and learning materials. You are invited to read our upcoming book “Enterprise Web Development” (work in progress) that includes coverage of ExtJS.

Viktor Gamov