We taught Adobe AIR to talk to Native Android API

Adobe AIR is the most productive tool for developing the UI for Android.  But as of today, AIR can’t access native Android API. By the end of this year Adobe plans to offer some integration/bridge to the native Android applications, but it’s not clear how it’s going to be implemented.
Traditionally, Farata Systems is trying to get into emerging and promising technologies as soon as possible and the first results are already achieved.  We taught AIR to talk to the native Android API. I mean it. You’ll see a demo, where a user talks to an AIR application, which communicate with the native Android voice API, which recognizes his commands and fills out the AIR UI form.
Without going into much details, we are using a different from Adobe approach – we put their BlaseDS server right inside the Android device. This opens endless opportunities, and we are trying to find the best use for this solution that goes under the working name “Server in your pocket”.
My colleague Victor works full time on integrating AIR and Android. He has recorded a short video that features him talking to the AIR application on the Xoom tablet, which communicates to the native Android voice recognition API and fills out the AIR form. Everything is happening inside the Xoom tablet. This addition to our Clear Toolkit has a working name Clear APK. See it for yourself.
We’ll present this demo live in August during our fourth annual symposium on enterprise software.

Yakov Fain

4 thoughts on “We taught Adobe AIR to talk to Native Android API

  1. Hi,

    My name is Phiphou and I’m interested in everything’s new in the Adobe AIR context.

    While surfing I found your demo about calling native android voice recognition api from AIR, and I also found your code, as you published it on http://code.google.com/p/clear-android/ .

    So, instead of taking your demo “as is”, I preferred imagine my own demo, and thought about adding compass support in AIR.

    To do that, I thought about listening the sensor manager and send a message using Clear-Android every time a sensor event occurred.

    So I implemented the sensor manager like this :

    @Override
    public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    Intent clearDataServiceIntent = new Intent(this, ClearDataService.class);
    startService(clearDataServiceIntent);
    // Real sensor manager
    sensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);
    sensorManager.registerListener(this,
    sensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION),
    SensorManager.SENSOR_DELAY_NORMAL);
    }

    public void onAccuracyChanged(Sensor sensor, int accuracy) {
    // TODO Auto-generated method stub
    }

    public void onSensorChanged(SensorEvent event) {
    // TODO Auto-generated method stub
    myAzimuth = Math.round(event.values[0]);

    MessageBroker msgBroker = MessageBroker.getMessageBroker(null);
    AsyncMessage msg = new AsyncMessage();
    msg.setDestination(“voiceRecognition”);
    msg.setClientId(UUIDUtils.createUUID(true));
    msg.setMessageId(UUIDUtils.createUUID(true));
    msg.setTimestamp(System.currentTimeMillis());
    msg.setBody(myAzimuth);
    msgBroker.routeMessageToService(msg, null);
    }

    And on AIR side, declared a regular consumer, with no max-frequency setting (so it would be the default 0…), and start receive messages from clear-android.

    And here is my logs:


    server: 18:45:41:789 client: 18:45:42:306 time diff: 59 azimuth: 167°
    server: 18:45:41:553 client: 18:45:42:259 time diff: 1317 azimuth: 168°
    server: 18:45:41:53 client: 18:45:42:200 time diff: 90 azimuth: 168°
    server: 18:45:40:576 client: 18:45:40:883 time diff: 36 azimuth: 170°
    server: 18:45:40:171 client: 18:45:40:793 time diff: 48 azimuth: 171°
    server: 18:45:39:613 client: 18:45:40:757 time diff: 49 azimuth: 171°
    server: 18:45:39:441 client: 18:45:40:709 time diff: 1335 azimuth: 173°
    server: 18:45:39:439 client: 18:45:40:660 time diff: 40 azimuth: 175°
    server: 18:45:38:791 client: 18:45:39:325 time diff: 39 azimuth: 177°
    server: 18:45:38:635 client: 18:45:39:285 time diff: 40 azimuth: 176°
    server: 18:45:38:367 client: 18:45:39:246 time diff: 52 azimuth: 171°
    server: 18:45:38:177 client: 18:45:39:206 time diff: 39 azimuth: 166°
    server: 18:45:37:992 client: 18:45:39:154 time diff: 45 azimuth: 162°
    server: 18:45:37:876 client: 18:45:39:115 time diff: 1298 azimuth: 158°
    server: 18:45:37:874 client: 18:45:39:70 time diff: 38 azimuth: 154°
    server: 18:45:37:353 client: 18:45:37:772 time diff: 32 azimuth: 148°
    server: 18:45:37:171 client: 18:45:37:734 time diff: 38 azimuth: 144°
    server: 18:45:36:927 client: 18:45:37:702 time diff: 35 azimuth: 147°
    server: 18:45:36:749 client: 18:45:37:664 time diff: 42 azimuth: 153°
    server: 18:45:36:555 client: 18:45:37:629 time diff: 46 azimuth: 158°
    server: 18:45:36:323 client: 18:45:37:587 time diff: 1320 azimuth: 162°
    server: 18:45:36:318 client: 18:45:37:541 time diff: 36 azimuth: 162°
    server: 18:45:35:851 client: 18:45:36:221 time diff: 32 azimuth: 163°
    server: 18:45:35:638 client: 18:45:36:185 time diff: 42 azimuth: 163°
    server: 18:45:35:436 client: 18:45:36:153 time diff: 41 azimuth: 162°
    server: 18:45:35:179 client: 18:45:36:111 time diff: 1251 azimuth: 161°
    server: 18:45:34:992 client: 18:45:36:70 time diff: 34 azimuth: 162°

    As you can see, there is a weird issue, don’t you think ? “Diff” is client time minus server time, and you’ll notice some big delays on some messages…

    So, currently, my demo has a “refresh time” of about 1 second. Sure we can do better regarding the small delays on some other messages…

    My first tests used an “amf-streaming” channel, I tried with “polling-amf”, with a small polling delay but the result are the same…

    I also tried another way : doing regular remote calls with a timer and your remote object, but I’m now facing a “duplicate session” issue, and it’s not a good idea, because I just need to “update” my AIR app when sensor values changes…

    Do you have an idea ?

    Maybe you can try it by yourself and tell me if you got the same…

    I’m using my Samsung Galaxy S, 2.2.1, with latest AIR release.

    Regards,

    Phiphou

    PS : I also noticed that the “subscription step” takes about 2 seconds, what seems to me to be very long, don’t you think ?

  2. The code that you’re referring to was reflecting our first experiments with this technology. It was offered on as is basis.

  3. Phiphou,

    Here at Farata we tested application with HTC Desire (Android 2.2), Motorola Xoom (Android 3.1) and Samsung Galaxy Tab (2.2) — neither device was able to run application over streaming AMF connection. I’m wondering how it works at your side; probably it falls back to polling. Did you check Android log for streaming errors — error message should start with ‘Broken pipe’ ?

    Next, it would be helpful to see relevant parts of Flex code, in particular the consumer message handler that calculates time difference. What grab my attention is:
    1. Short delays correlate very well with default Flash frame rate (25 fps, or 40 ms delay)
    2. Sum of short delays + long delay is very close to polling interval of 2 seconds, the same delay you noticed as delay for “subscription step”

    So probably your calculations are just misinterpreted results — your server queues all messages from sensor at server-side, next polling loop takes them all and transfers to client side, than your code starts to print per-frame delay rather than server-client diff (wrong calculations?), finally application logs polling delay minus sum of all previous “frame time” for recent poll. At least it looks like this.

    Valery

  4. Hi Phiphou,

    First of all, the code is now in Farata repository: you are looking at the very early snapshot. Streaming will not work at all, so both your experiments are, in fact, amf-polling. By the nature of amf-polling, the “pick-up” of the outgoing server messages happens once in a while, so the diff between client time and server time should reflect that. That said, I would expect the diff sequence looking more like 6-5-4-3-2-1-0 (given your 7 ticks between two spikes) rather then 1-1-1-1-1-1-7. Perhaps the issue is in the client/application code (Flex)?
    That would be my guess.

    WBR,
    Victor

Comments are closed.