Taming BlazeDS for Android with LongCalling

This post is a sequence to our announcement that we taught Adobe AIR to talk to native Android API by placing BlazeDS inside Android. Initially, we embedded BlazeDS into AIR-Android APK (watch this video) to use Google voice recognition for the data entry. The plan was to be able to invoke the following Java code from BlazeDS :
[quickcode:noclick] Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
intent.putExtra(RecognizerIntent.EXTRA_PROMPT, prompt); // We wanted to use Text-to-Speech as well
startActivityForResult(intent, 12345);
[/quickcode]

The first challenge was that features like voice recognition, Toast (Android popups) and other dialog functions, are supposed to be run in UI thread vs. a worker thread of the BlazeDS. In particular, startActivityForResult() is a method of Activity class. This was an easy to solve problem, because we could place this code in our own activity and start this activity via Intent.

The real challenge was that the voice recognition software does not return anything until the computer-human interaction is complete. In other words, the Java piece to remote is asynchronous –  you start the Recognizer activity by invoking the function startActivityForResult() and, sometime later,  get notification via an async callback when the results are ready:

public void startVoiceRecognition(int requestCode, String prompt) {
    Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
    intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
    intent.putExtra(RecognizerIntent.EXTRA_PROMPT, prompt); // Customized prompt
    startActivityForResult(intent, REQUEST_CODE_RECOGNIZE_SPEECH);
    return;
}

The low-hanging solution to this asynchronicity  was to push the results from BlazeDS to a messaging destination that would be listened to by an AIR application:

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    switch (requestCode) {
        case REQUEST_CODE_RECOGNIZE_SPEECH:
            if (resultCode == RESULT_OK) {
                ArrayList recognizedPhrases = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                // Now we are ready to pass results back to Air code.
                MessageBroker msgBroker = MessageBroker.getMessageBroker(null);
                AsyncMessage msg = new AsyncMessage();
                msg.setDestination("voiceRecognitionResult");
                msg.setClientId(UUIDUtils.createUUID(true));
                msg.setMessageId(UUIDUtils.createUUID(true));
                msg.setTimestamp(System.currentTimeMillis());
                msg.setBody(recognizedPhrases);
                msgBroker.routeMessageToService(msg, null);
            }
            break;
        }
    super.onActivityResult(requestCode, resultCode, data);
}

This worked, but made the AIR code ugly: we had a RemoteObject with a non-functional result handler plus the messaging destination and a Consumer, which clearly would be replicated for each call of that sort.

LongCalling to the Rescue

And then we realized that this was a déjà vu. We had been there when a stored procedure in one financial application would not return for 3 minutes. We helped that customer without breaking the remoting model. We used the “long calling” instead.

Long calling is Farata’s term to label the customization of a Java adapter and RemoteObject that allows us to quickly receive a dummy return of the original long remote call and reincarnate the mx.rpc.ResultEvent when the real data is ready.  By using a customized RemoteObject the application code becomes agnostic to the fact that remoting operation has “two legs”.  The only requirement that we add is that the name of such remoting method has to end with “AndWait” as in  recognizeVoiceAndWait(). This signals to the ActionScript side that this invocation is a “long calling” and this knowledge is carried forward to the custom Java adapter.

On the Java side, we implemented a PlatformServices object with several methods, most importantly,  startActivityAndWait() and complete(). Here is how we start our own SpeechRecognitionActivity. Notice that the first parameter of the runActivityAndWait() is an intent to start an activity, while the second one is a class of the expected result:

public List recognizeVoiceAndWait(final String prompt) {
    final Intent intent = new Intent(PlatformServices.context(), SpeechRecognitionActivity.class);
    intent.putExtra(RecognizerIntent.EXTRA_PROMPT, prompt);
    return PlatformServices.runActivityAndWait(intent, STRING_LIST);
}
final private static Class> STRING_LIST = null;

The SpeechRecognitionActivity() will start the Recognizer activity precisely like we did in the earlier code snippets. Meanwhile the worker thread that started it will be blocked. It will remain in the blocked state until the onActivityResult() callback from inside the SpeechRecognitionActivity issues the complete() passing exactly the same intent as was used during runActivityAndWait() – this intent is preserved in the onStart() of the SpeechRecognitionAcitivity:
@Override
public void onActivityResult(final int requestCode, final int resultCode, final Intent request) {
    if (requestCode == REQUEST_CODE_RECOGNIZE_SPEECH) {
        final List result;
        if (resultCode == RESULT_OK) {
            result = request.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
        } else {
            result = null;
        }
        complete(originalIntent, result); // We preserved originalIntent during onStart() of the activity
        finish();
    }
    super.onActivityResult(requestCode, resultCode, request);
}

That’s it. The only limitation is that due to the blocking mechanism of runActivityAndWait() only one at a time call can be executed. Accordingly, the AIR application should avoid sending several “AndWait” requests in one AMF batch.

LongCalling in Action

And, of course, the AIR’s code won’t care about runActivityAndWait and complete() pairs at all. For all it knows, there will be regular remoting calls, albeit ending with “AndWait”:

<?xml version="1.0" encoding="utf-8"?>
<s:View xmlns:fx="http://ns.adobe.com/mxml/2009" 
    xmlns:s="library://ns.adobe.com/flex/spark" 
    xmlns:c="library://ns.cleartoolkit.com/flex/clear"
    title="Voice Recognition"
>
  <fx:Declarations>
    <c:RemoteObject id="service" destination="AndroidJavaDestination"/>
  </fx:Declarations>
  <fx:Script><![CDATA[
 
  import mx.collections.IList;
  import mx.rpc.AsyncToken;
  import mx.rpc.Responder;
  import mx.rpc.events.FaultEvent;
  import mx.rpc.events.ResultEvent;
 
  [Bindable] private var recognizedPhrases:IList;
 
  protected function onTextInputFocusIn(event:FocusEvent):void {
    var target:TextInput = event.currentTarget as TextInput;
    promptAndListen(target.toolTip || target.id, target);
  }
 
  private function promptAndListen(prompt:String, target:Object):AsyncToken {
     recognizedPhrazes = null;
     var token:AsyncToken = service.recognizeVoiceAndWait(prompt);
     token.addResponder(
       new mx.rpc.Responder(
    onRecognizeVoiceResult, onRecognizeVoiceFault
       )
     );
     token.target = target;
     return token;
  }
 
  private function onRecognizeVoiceResult(event:ResultEvent):void {
     recognizedPhrases = event.result as IList;
     var textInput:TextInput = event.token.target as TextInput;
           var bestMatch:String ;
 
           .... Find the best match from recognizedPhrases  
 
     textInput.text = bestMatch;
  }        
  ]]></fx:Script>
  .  .  .
  <s:Form width="100%">
    <s:FormItem label="Name:">
      <s:TextInput focusIn="onTextInputFocusIn(event)" 
        toolTip="Employee name"/>
    </s:FormItem>
    <s:FormItem label="Phone:">
      <s:TextInput focusIn="onTextInputFocusIn(event)" 
        toolTip="Phone number"/>
    </s:FormItem>
          .  .  .
  </s:Form>
</s:View>

If you are interested to see this solution in action, I’ll be showing it in August during our fourth annual symposium on enterprise software.

Victor

We taught Adobe AIR to talk to Native Android API

Adobe AIR is the most productive tool for developing the UI for Android.  But as of today, AIR can’t access native Android API. By the end of this year Adobe plans to offer some integration/bridge to the native Android applications, but it’s not clear how it’s going to be implemented.
Traditionally, Farata Systems is trying to get into emerging and promising technologies as soon as possible and the first results are already achieved.  We taught AIR to talk to the native Android API. I mean it. You’ll see a demo, where a user talks to an AIR application, which communicate with the native Android voice API, which recognizes his commands and fills out the AIR UI form.
Without going into much details, we are using a different from Adobe approach – we put their BlaseDS server right inside the Android device. This opens endless opportunities, and we are trying to find the best use for this solution that goes under the working name “Server in your pocket”.
My colleague Victor works full time on integrating AIR and Android. He has recorded a short video that features him talking to the AIR application on the Xoom tablet, which communicates to the native Android voice recognition API and fills out the AIR form. Everything is happening inside the Xoom tablet. This addition to our Clear Toolkit has a working name Clear APK. See it for yourself.
We’ll present this demo live in August during our fourth annual symposium on enterprise software.

Yakov Fain

Applications Everywhere: Introduction

We are wearing multiple hats at Farata Systems all the time. For our corporate clients, we are the Rich Internet Applications shop that does project management and software development within constrains of the client vision and the product timeline.

But we also work on the projects for startups in which we are major stakeholders. There we apply our sense of what world becomes and what RIA are meant to be.

I will be talking about one of those experiences during Flex360 in Denver this April. In particular, I will be talking about application for the financial/insurance industry we have been working on for the last 18 months.

It started as a typical AIR desktop application with PDF integration and some Java back-end workflow. As application grew, the clients (200K-1M of prospective paid clients) demanded availability of the functionality on the desktop, web and mobile devices. In this series of the articles I will talk about solutions we had to build in order to provide them with consistent easy to use platform that essentially brings us to the next frontier of the application development.

Enter “virtual solution provider”. A typical insurance agent is 50+ years old (technology challenged) high school graduate with GPA of 3.0 (as your financial adviser) and a single handed to manage both client expectations (CRM) and insurance companies/state bureaucracy (workflow and compliance). We consider it a huge opportunity for an integrated solution (most of the insurance agencies still howl trunks full of paper insurance forms with them) that has to be simple enough so any agent can do it. The insurance agents are not the same users as the corporate ones – they work for a distributed MLM with extremely high turnover rate.

The client base is almost consumer grade. They can have a laptop now with the Internet connection that works 70% of the time. Tomorrow they would rather have a tablet and touch-screen devices – they need simple devices. The phone integration is essential as they spend 50%+ developing leads. Phishing and real-time advertisement by current media/internet companies are the ways they get most of the new clients.

That’s why we build this insurance application as an example of what the platform for the future consumer applications should be. The Internet moved publishing/media closer to consumers and made it more personal. The most successful companies (Google, Facebook, etc) turned hundreds of millions users into their implicit workforce. The next generation of applications needs to do the same. Rather then come with a fixed workflow it has to capitalize on the everyday experiences of the human race and become an expert system in any application domain. That experience should guide your workflow in transparent way, when and where you need it.

The articles in this series will talk about changes we need to make to the features we are embedding in the applications so they become a ubiquitous part of our life.

Anatole Tartakovsky

Master Class on Adobe Flex in Brussels and more

After teaching our popular advanced Flex master class in New York, Boston, Toronto, London, and Moscow we are hitting the beer capital of the world: Brussels, Belgium. This class is scheduled for the first days of March, and we hope that Flex 4 will be officially released by this date. Our book Enterprise Development with Flex should hit the book stores by then too.

To take advantage of the early bird prices register early at http://bit.ly/59DdIU.

If you can’t make it to Brussells, I’ll be glad to meet with you at 360 Flex in San Jose, CA on March 7-10, where I’ll be presenting on one of the topic from this master class.

If you can’t make it to San Jose, I’ll be very happy to meet you at Flash and the City conference in May in New York City. Most likely I’ll show you a really cool way of embedding a Flex application into a PDF file to be played by Acrobat Reader.

If none of these dates/locations work for you, you can always invite us to teach this class privately on site in your organization almost anywhere on the planet Earth.

Yours
Yakov Fain

One approach to Enterprise Flex development

During the last several years our company, Farata Systems have architected and developed a number of Flex projects ranging from creating widgets for startups to large applications having two hundred views. Some projects were created from scratch, and we had a luxury to do it our way. In some cases we had to pick up the code left by other developers.

Three years ago Adobe Flex has been completely revamped (I’m referring to Flex 2) and nobody knew about it. But today, the word Flex doesn’t make Java developers angry any longer. Java/Flex projects are considered business as usual. Many enterprise Java developers went through Flex training and are ready to roll up their sleeves…

After spending 10+ years with Java I can easily put myself in the state of mind of Java developers – they often start with selecting the right framework. As a matter of fact, our perspective customers often request to include an item “Flex framework analysis and recommendations” in the agenda of our very first joint meeting. Here’s a typical conversation that takes place during such meetings:

“Yakov, we have a team of experienced Java developers and some of them have exposure to Flex. Which Flex MVC framework do you recommend? By the way, does the ABC framework support XYZ functionality?”
“No, MVC-based frameworks don’t support any additional functionality that doesn’t exist in Flex. Their goal is to rearrange your code and change the way components of your RIA communicate with each other. But why do you need an MVC framework for your project?”
“Well, everyone’s using them. Having a framework makes the project development more structured and organized.”

Is it so?

Most of the Flex MVC frameworks are intrusive and require developers to write more code just to support the life cycle of the framework itself. Usually, they are built on singletons, and this complicates modularization of the RIA. I’ve been writing and speaking against Flex MVC frameworks for years, and if you’d like to read a detailed analysis and comparison of some of them, read the first chapter of the book “Enterprise Development with Flex”.
Last year at MAX ’08 conference, I participated in the panel “The Flex architecture faceoff”. Two out of four panelists (Chafic Kazoun and yours truly) were advocating component-based approach without squeezing a RIA inside any MVC framework. It was a lively discussion, and you can listen to its recording at http://tv.adobe.com/#vi+f15384v1055
At this point you may say, “OK, it’s easy to critique frameworks, but how do you approach development of a new project especially while having a distributed team of developers with minimal exposure to Flex?”
And I’m glad you’ve asked this question.
In this article I’ll try to identify common artifacts that exist in literally every Flex enterprise project. I’ll also highlight major principles and introduce you to a component-based approach that our developers use in almost every project.

Flex to Java Communication Protocols

The role of Java side of RIA is to deal with data: get the data from a DBMS, a Web Service or any other data source and deliver it to the Flex client as quickly as possible. The fastest way of arranging Java-Flex data exchange is AMF or RTMP protocols. The open source AMF protocol is built on top of HTTP and it offers efficient data serialization between Flex and Java. RTMP offers faster socket-based full duplex communication.

If you install an open source server-side component BlazeDS under a servlet container of your choice, or will use Granite Data Services – AMF will be your only option for data transfer. Commercial LiveCycle Data Services ES (LCDS) will give you both RTMP and AMF implementations. LCDS will also offer you an automated way of data synchronization between Flex and Java.
There is a small number of high performing enterprise applications that would really benefit from using RTMP protocol, and beside using LCDS, an open source server Red5 may become the only alternative.

Note. RTMP protocol is also widely used for streaming video, but these applications are out of the scope of this article.

Challenges of Flex/Java enterprise projects

Let’s dissect a typical enterprise application that’s built with Flex on the client and Java on the server. If you are leading such a project, it’ll present the following tasks, needs, haves, and nice to haves:

1. Keep track of changes that the user makes via UI controls and send modified data to the server
2. It’ll have a number of views that will display the data in the form of grids
3. It’ll have a number of views that will contain forms
4. Some views will represent master-detail relations, i.e. selection of a row in a grid has to display the detail information in a form container.
5. The data grid columns will need custom item renderers ranging from centered checkboxes to dropdowns populated with some reference data by making a separate remote call to the server.
6. It would be nice if a form component could have its own data provider similar to the one that List-based controls use.
7. Data forms must be validated and it would be nice to have
a) reusable validators (i.e. two date fields should be able to reuse the same instance or the Validator object)
b) embedded validators, encapsulated in component such as DataGrid.
8. The main view of your application must appear on the user’s screen ASAP even if they have slow Internet connections.
9. The data changes made by one user may need to be automatically synchronized with the server and notification of the changes should be sent to other users potentially looking working with the same data set at the moment. This functionality is supported by Data Management Services in LCDS, but you may want to have the same functionality in open source BlazeDS.
10. If you are building an AIR/BlazeDS application, you need to come up with a custom solution for automatic data synchronization for occasionally connected applications.
11. Printing is an Achilles heel of Flex. You’d like to be able to generate PDF on the client for your Flex/BlazeDS or disconnected AIR applications.
12. Since your modularized application may consist of five-ten-fifteen Flash Builder projects, writing build scripts becomes a project in its own. Can writing script be automated?
13. Your Flex project may need to consume a dozen or more of Java data transfer objects hence you need to develop the same number of ActionScript classes for efficient data serialization. During the project development cycle the structure of these DTOs may change if not daily, then weekly. An automation tool for generation of DTOs can save you some project development time.
14. Some of the views in your application represent CRUD functionality and using code generators or wizards that lower the amount of manual coding is highly desirable.
15. How do I link all these libraries – RSL/Merge-in/External? Should I use Flex SDK as RSL?

Over the years, we at Farata Systems came up with our solutions to all of the items from the above laundry list. At this point every other reader would exclaim, “Finally, here comes the selling part. I knew it!”
Wrong! All of the components implementing the above functionality not only are available for free, but we also open sourced them as Clear Toolkit framework and (see SourceForge at https://sourceforge.net/projects/cleartoolkit/ ).
Now I will write yet another list of solutions that you might want to research further. By the way, even though we call our components a framework, you can use most of them ala cart.

How we deal with challenges of Flex/Java enterprise projects

We’ve created and are happy to share with the community a number of handy classes, tools, and techniques. You can get the code, which has been open sourced, and you can read our books and articles describing how to use them.

I’ll give you quick references that help in finding solutions to the above challenges (I’ll keep the same order and numbering).

1. Clear Toolkit has a library clear.swc, which has a component DataCollection that supports automatic tracking of the user changes. Look at the ChangeObject class – we’ve implemented it similarly to the one in LCDS, but ours can be used with BlazeDS too. If you need to make your updates transactional, use the BatchService class.
2. We use DataCollection (a subclass of ArrayCollection) as a data provider for data grids. Besides keeping tracks of changes, it encapsulates Flex remoting and reduces the amount of manual coding.
3. Research the DataForm component. Read about it in the sample chapter mentioned at the end of this article.
4. DataCollection supports proper updates in Master-Detail scenarios (see http://flexblog.faratasystems.com/?p=407).
5. Read about resources in the sample chapter mentioned at the end of this article.
6. Read about DataForm component in the sample chapter mentioned at the end of this article.
7. Read about Validator component in the sample chapter mentioned at the end of this article.
8. Read about custom preloaders and how to build every application as a mini-portal with a light-weight starting application in Chapter 8 of the upcoming O’Reilly book.
9. Read about DataCollection in general and hierarchical data collections with deep data synchronization in particular (http://flexblog.faratasystems.com/?p=407)
10. Read about the OfflineDataCollection (Chapter 9 of the upcoming book) and see a pre-recorded demo of a sample application that’s using it (http://flexblog.faratasystems.com/?p=394 ).
11. Explore the Clear Toolkit’s package com.farata.printing – it includes extended Flex components that can expose themselves in a format suitable for PDF generation. Read chapter 11 of the upcoming book for more details and samples.
12. An approach and techniques of minimizing the download time is described in Chapter 8 of the upcoming book.
13. Explore our DTO2Fx code generator that comes with Clear Toolikt: https://sourceforge.net/projects/cleartoolkit/files/
14. Read about Clear Data Builder 3.1 that can generate the entire CRUD application based on either SQL or arbitrary Java DTO’s at https://sourceforge.net/projects/cleartoolkit/files/
15. Read chapters 7 and 8 about linking libraries and loading modules in Flex RIA.

Where can I learn more about these solutions?

First, you can read the book Rich Internet Applications (http://riabook.com/) with Adobe Flex and Java that I co-authored with Anatole and Victor, my colleagues at Farata.
Second, you should read an upcoming O’Reilly book Enterprise development with Flex, written by the same authors.
Third, you may explore the source code of Clear Toolkit components and use the tools published at SourceForge.
Fourth, we invite you to enroll in Advanced Flex trainings, seminars and symposiums that we run on a regular basis. During these events we demonstrate most of the techniques mentioned above as well as our latest findings. Find the up-to-date information about such events under the Training section at http://faratasystems.com/
Fifth, read our blog at http://flexblog.faratasystems.com
To get a better feeling about the functionality of some of the extended Flex components, please read the sample chapter of the upcoming book Enterprise development with Flex. O’Reilly decided to publish this chapter as a three-part blog under my name, but I was only one of three co-authors, and all of us are accountable for these texts. Read the sample chapter here:
Part 1. http://www.insideria.com/2009/05/chapter-preview-building-an-en.html
Part 2. http://www.insideria.com/2009/05/building-an-enterprise-framewo.html
Part 3. http://www.insideria.com/2009/05/building-an-enterprise-framewo-1.html

Summary

I just want to give a full credit to my colleagues, excellent software engineers at Farata that work day in and day out on enhancing the functionality of Flex components and decreasing the amount of manual coding required by application programmers. Open sourcing our component have allowed us to bring more people to testing (our big thanks to people who use Clear Toolkit and report issues and make suggestions at Source Forge forums).

We invite Fle and Java developers to become active contributors and submit their version of enhanced components to make Clear Toolkit, a real platform for all who value open source solutions. There are no geniuses that can beat the collective intelligence!

Yakov Fain

How to use Clear Data Builder with AIR applications

I’ve added a new How To entry to the Wiki section of  our open source Clear Toolkit framework. It explains the steps required to turn a CRUD Flex application generated by Clear Data Builder into an AIR application.

The Wiki page of Clear Toolkit is http://cleartoolkit.wiki.sourceforge.net/

The lates build, documentation and the source code of all Clear Toolkit components you can find at https://sourceforge.net/projects/cleartoolkit/

Victor

Flex framework Clear Toolkit goes open source

We started the process of open sourcing of our Clear Toolkit framework for developing enterprise Rich Internet Applications with Adobe Flex and Java.  The source code is located in the Sourceforge CVS repository at  http://sourceforge.net/projects/cleartoolkit/ . The current documentation, demos, user forums, and bug trackers are also there.

Clear Toolkit 3.1 includes the following Eclipse plugins and components:

Clear Data Builder 3.1 is an Eclipse plugin that allows to generate CRUD applications for BlazeDS or LCDS based on either an SQL statement or a Java data transfer object.

DTO2Fx
is a utility that automatically generates proper ActionScript classes based on their Java peers.

Log4Fx is an Eclipse plugin built on top of Flex logging API, but it automates and makes the logging process more flexible and user friendly.

Fx2Ant is a generator of optimized ANT build scripts for your Flex Builder projects.

clear.swc is an enhanced Flex component library. The source code of all components is located in the package com.farata.components.

Clear Toolkit roadmap includes the following enhancements this year:

1. Offer data synchronization solution for AIR/BlaseDS applications – February 2009
2. Enhance Flex UI component to support PDF generation on the client – April 2009
3. Offer a solution for Flex-based portals – May 2009
4 Document classes from our component library (clear.swc) – June 2009.

All these components were (and still are) offered for free at http://www.myflex.org but now we are inviting developers from around the world to join us in making Clear Toolkit the open source platform of choice for all Flex developers

Yakov Fain

Data Synchronization with AIR and BlazeDS

One of the main selling points of Adobe AIR is its ability to support occasionally connected applications. Imagine a salesman on the road working with an application without having a network connection.  As an example, this can be  a salesman of a pharmaceutical company visits medical offices offering their new pills called Xyzin.

While in doctor’s office, she uses an AIR application to take notes about this visit. The notes are saved in the local SQLite database that comes with AIR.  But as soon as the Internet connection becomes available (at Starbucks or at home), the local data get synchronized with a central database.

Adobe offers a data synchronization solution based on Data Management Services for those who own the licenses of LiveCycle Data Service.  But those application developers who use an open source BlazeDS don’t have any generic way of setting such data synchronization.

We came up with a such solution that will be included in the upcoming version 3.2 of our framework Clear Toolkit.

Watch the following screencast to see a demo application which shows such data synchronization in action: http://www.myflex.org/demos/PharmaAir/PharmaAir.html.

Yakov Fain