Sharing output streams through a JNI interface

Maybe it’s me, but there seems to be a shortage of expert information on the net these days. I dunno about you but I constantly find myself running out of Google links and StackOverflow replies when looking for an answer to some rather valid technical issues. Am I the only one out there trying some of these crazy ideas or is the internet playing a savage game on me? Hi, I’m Cliff, and you’re here because you were waiting for me to post something crazy and far reaching. Maybe you were waiting on the, “How to tether your 8830 to your 1st Gen iPad” article. Maybe you were looking for info on how to install a Linux based firmware on your brand new Samsung 3D television. Whatever it is that brought you here, welcome to my little corner of the net where solutions to obscure technical issues are the norm.

I recently went on a journey into JNI land. I had visited many years ago plugging native MS Excel reporting into legacy RPG/400 programs via ILE wrappers around the POI project. Things were so much smoother back then. Flash forward to recent weeks as I wade through my old C manuals and skip through google links for Makefile tutorials. My project was to take a 3rd party native API, with C source code examples into the modern age. We need to use this API in a serious multi-user environment. To put it in simple yet technical terms, I needed to stream binary data from this API over the internet to a mobile device. Without being too specific and revealing let me give a generic example that colors the picture of what I’ve accomplished.

Say you have this blob of binary data. Go ahead, say it. Now say it once more but this time wait for a co-worker to walk by. Say the binary blob is generated from a native API that costs, oh about $2000 a seat. (The cost is not nearly as important as the seat.) Now say the API is not documented with any examples in the programming language of your choice. (Say that last sentence aloud with a girlie giggle between the 6th and 7th words.) Let’s pretend the only example given uses an old fashioned main function C reading cmd line arguments. Let’s pretend you have clients that need this blob on the other side of the internet. What are your options?

You could call the API using the command line example which writes to the file system. You could then read the generated file back across the internet using your favorite Ruby/Python/PHP programming toy. Then you would have to deal with file write collisions, the overhead of disk access, cleaning up the file after its written, file locks if you’re on Windows, and many other headaches.

You could modify the example to write directly to the internet. Then you have to deal with either implementing HTTP or talking to yet another 3rd party API written in the same language. You’d also have to deal with the language which you only use on the rarest of occasions… like never.

You could opt for the Language compatibility API that comes in the language of your liking with the additional purchase of two other vendor products. (API not sold separately.) You could do a number of different things each with their own drawbacks or you could do what I did… design a native interface.

I used JNI to write binary data from C back to a Java object. My first reaction was, “Gosh! How do I write a C++ output stream like thing to a Java output stream?” My second reaction was, “Don’t I need an input stream instead of an outputstream? Which end of which stream plugs in where?” (I always have that reaction when I deal with I/O streams.) the short answer, which applies equally to pure Java, C++, and Java/C++ is that you need threads to connect two streams… ALWAYS. There’s no safe way to deal with threads so thought a little harder. Because the output was delivered incrementally in my case, I opted to use a callback strategy. In other words, the API used callbacks to hand chunks of binary in array form to a calling application. I simply plugged in my JNI C code to accept these arrays and relay them to the original invoking Java object.

You have to be really careful with JNI native array conversion. There’s a proper way to copy bytes from C to Java and then there’s the whole, “is this a copy or a direct reference?” issue. Also you might get confused with array pinning in later VMs, but that really shouldn’t matter if you only need to materialize the data briefly in C.

In all, the general strategy for sharing binary data (A/V files, images, etc.) from C with Java requires byte arrays. You create a Java byte array in C like this:

const char[] rawData = {0,1,2,3,4,5,6,7,8,9}; //Or get some raw data from somewhere
int dataSize = sizeof(rawData);
printf("Building raw data array copy\n");
jbyteArray rawDataCopy = env->NewByteArray(dataSize);
env->SetByteArrayRegion(rawDataCopy, 0, dataSize, rawData);

And pass it to Java like this:

printf("Finding callback method\n");
//Assumes obj is the Java instance that will receive the raw data via callback
jmethodID aMethodId = env->GetMethodID(env->GetObjectClass(obj),"handleData","([B)V"); 
if(0==aMethodId) throw MyRuntimeException("Method not found error");
printf("Invoking the callback\n");
env->CallVoidMethod(obj,aMethodId, &rawDataCopy);

you would have a Java object that looked something like this:

public class MyDataHandler {
  OutputStream dataStream;
  public MyDataHandler(OutputStream writeTo) { dataStream = writeTo;}
  public void handleData(byte[] incomingData) { dataStream.write(incomingData); }
}

That handler would be passed to C via native method like so:

public class NativeIntegration {
  public native void generateBinaryWithHandler(MyDataHandler handler);

  //Here we assume response is something like a network stream
  public void doCallNativeFunction(ResponseStream response) {
    MyDataHandler handler = new MyDataHandler(response);
    generateBinaryWithHandler(handler);
  }
}

That’s a rather niave example without the necessary consideration of error handling/reporting but it gets the point across.