ActiveExcel

This will be the future house of ActiveExcel documentation. No time today though.

My new favorite Sentence in "Black Swan"

"If you hear a 'prominent' economist using the word equilibrium or normal distribution don't argue with him; just ignore him or try to put a rat down his shirt."

webMethods IData => java.util.Map => IData

I am not a fan of webMethods IData construct. Working with IData is needlessly obtuse, in my opinion. For example, I have an IData object nested inside another IData object, and I want to access values in the child IData object:

IData top = someObj.callServiceWithResultsPutInIData(params);
IDataCursor t_curs = top.getCursor();
IData child = IDataUtil.getIData(t_curs,"ChildKey");
IDataCursor c_curs = child.getCursor(child);
String val = IDataUtil.get(c_curs,"Key");

Thats alot of code to access a value in a nested IData object. I know, however, what the cursor is. Its really just an external iterator for IData objects. IData is really just a collection, but it doesn't implement Java's collection interface. Unlike most collections (think lists, arrays, trees, vectors, etc.) IData forces you to use the cursor in order to access its elements, this leads to ugly, inefficient code like what you see above.

Besides ugly code, however, there is something much more sinister at play here. When interacting with WM services from within java code, developers are tempted to sprinkle references to IData throughout their code. From a strategic point of view, this gives webMethods (actually, Software AG, the company that owns webMethods) lock in by raising our switching costs. The further up your application stack you allow the IData tentacles to stretch, the higher these switching costs become, and more and more "bargaining power of suppliers" is given to Software AG. In my opinion, this is bad.

Again, I hate working with IData. But, I have a solution and it is exceptionally easy: convert IData to java.util.Map. Mathematically, the two structures are identical - anything you can represent in IData can be represented in Map and vice versa. A nested IData object is really just a Tree, and you can make a Tree out of a Map. Within my method to do the conversion I use the concrete class HashMap, but this will work with anything that satisfies the Map interface. Here is how to convert IData to Map:

public Map convertIDataDocToHash(IData input) 
           throws DataFormatException{
  HashMap output = new HashMap();
  IDataCursor i_curs = input.getCursor();
  String key = "";
  Object o = null;
  while(i_curs.next() != false){
   key = i_curs.getKey();
   o = i_curs.getValue();
   if(o instanceof IData){
    output.put(key, convertIDataDocToHash((IData)o));
   }else{
    if(!(o instanceof String)){
     throw new DataFormatException("Scalar value in Hash is not a 
                   String, all scalar values need to be Strings.");
    }
    output.put(key,o);
   }
   
  }
  
  i_curs.destroy();
  return output;
 }

Using a pretty simple recursion, I can convert any IData document of arbitray depth and width into a HashMap of the same dimension. Since the two structures are identical, the same algorithm works to convert HashMap (or any Map object) to IData:

public IData convertHashToIDataDocument(Map m_data) 
             throws DataFormatException{
  //create a new toplevel IData object
  Set key_set = m_data.keySet();
  IData inputs = IDataFactory.create();
  IDataCursor i_cur = inputs.getCursor();
  String key = "";
  for(Iterator it = key_set.iterator(); it.hasNext();){
   key = (String)it.next();
   Object o = m_data.get(key);
   if(o instanceof Map){
    IDataUtil.put(i_cur,key , convertHashToIDataDocument((Map)o));
   }else{
    //this needs to be a string since we are working with a document
    if(!(o instanceof String)){
     throw new DataFormatException("Scalar value in Hash is not a 
                   String, all scalar values need to be Strings.");
    }
    IDataUtil.put(i_cur, key, (String)o);
    
   }
  }
  i_cur.destroy();
  return inputs;
  
 }

So, now, I can call a service and use Java collections in my business critical code, instead of IData. Also, when I invoke the service, I can pass a HashMap to a wrapper method that will call the service for me, converting the HashMap to IData on the fly:

public HashMap runService(String url, String user, String pass, 
                                  String serv, HashMap inputs){
  //Establish the context
  Context cntxt = new Context();
  System.out.println("Context Established....");
  try {
   //make the connection
   cntxt.connect(url, user, pass);
   System.out.println("Connection Established with Integration Server....");
  } catch (Exception e) {
   System.exit(0);
  }
  IData id_input = null;
  try {
   id_input = this.convertHashToIDataDocument(inputs);
  } catch (DataFormatException e1) {
   e1.printStackTrace();
  }
  IData output = null;
  try {
   output = cntxt.invoke(serv.split("\\:")[0],
              serv.split("\\:")[1], id_input);
  } catch (Exception e) {
   e.printStackTrace();
  }
  
  //convert the IData doc back to a hash
  try {
   return (HashMap)this.convertIDataDocToHash(output);
  } catch (DataFormatException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
   return null;
  }
 }

Now that I have everything wrapped in Map, strategically I am back on solid ground and my code to access the data returned by the service is much cleaner:

HashMap inputs = this.buildInputs();
HashMap m = client.runService(url,user,pass,service,inputs)
//Get a top level scalar value:
String val1 = m.get("Top_Key_1");
//get a child hash, and access a value
HashMap child = (HashMap)m.get("child_1");
String val2 = child.get("Child_Key_1");

Thats much nicer. So, if you are going to be calling WM services with client code outside of the IntegrationServer, do yourself a big favor and use a pattern like this to push the WM specific code as far down into the application stack as possible.

Taking the iPhone 3.0 Plunge

I am making good use of my Apple developer membership and I am downloading the iPhone 3.0 Beta 3 SDK and OS beta for my iPhone 3G. All of the documentation says that you shouldn't put this beta release onto your daily use phone, so the first thing I am going to do when the download finishes is put it on my daily use phone. (I don't have a development phone yet, and dammit, I just can't wait any longer for cut/copy/paste.)

The main reason I am taking the plunge into the deep end of the iPhone pool is this video here:

iPhone 3.0 Demonstration

Its an hour long, but so chock full of gee-whiz stuff, that I just have to get it on my phone today! Hopefully I can convince my wife to be one of my "testers" so I can load it on her phone as well.

In the new SDK there is a CocoaTouch Map control, which I am going to see if I can write a "find your buddy" application leveraging this new control, and also see if it fits into the App I am currently working on.

I am going to document the trials and tribulations that I go through with this new endeavor, because that is what an exocortex is for.

Update 1, 28 April 2009 13:04

The downloads are finished - one full hour of downloading. The SDK is over 2 GB. the journey begins....*melodrama*

Update 2, 28 April 2009 14:34

Doing two things at once. I started the process of setting up my phone as a development device. I have generated a development certificate using the Keychain Access application on my machine, and I have uploaded this cert to the Apple Developer Portal. In order to provision your iPhone to be used as a development device, you need a 40 digit Hex number known as a Unique Device Identifier (UDID). This identifier is found by starting up Xcode, plugging in your iPhone, then opening the "Organizer" window (Window -> Organizer). This looks to be pretty close to a step-by-step set of instructions for device ID discovery:

Devices How To

I stopped here, though, because I am still running XCode with the 2.x SDK, so now I am installing the 3.0 SDK and then I will continue on with the provisioning. I have found no information that says I need to have the 3.0 SDK installed before I can provision and install OS 3.0, but I am a little paranoid so I am going this route anyways.

Update 3, 3 May 2009 11:37

After much reading and googling and pondering and waffling, I am now actually loading 3.0b4 onto my iPhone. It is, actually, alot easier than I thought. On the iPhone developer site is a link for iTunes 8.2, download and install that. It is slightly confusing because everything about the package (the readme, etc.) makes it seem like you are just installing 8.1.1, but doing iTunes > About iTunes confirms that it is in fact 8.2 beta. With iTunes open, plug in the iPhone and hold the Option (alt) key while clicking "Check for updates". This will bring up a finder dialog, find the OS 3.0 beta 4 package, click it and press Open on the dialog. Thats when the magic happens. Hopefully all goes well, it is still in the process of installing.

Update 4, 3 May 2009 12:16

So far so good, I have iPhone OS 3.0 beta 4 on the phone and it appears that everything except my contacts were preserved. Bit of a scare however. Once iTunes loaded 3.0 onto the phone, it gave me the dialog letting me know it needed to reboot the phone. The reboot hung and I really thought I had bricked the phone. After letting it sit for about 10 minutes, I disconnected the phone from iTunes, then plugged it back in. iTunes recognized my phone, so I was very relieved. Next, I Option-clicked Restore to bring up a finder dialog, and in the dialog selected the iPhone OS 3.0b4 package to "restore". I installed the OS and my phone came back to life, phew!! I am in the process of syncing my contacts back onto the phone, so I am not out of the woods yet.

I can post a blog from my iPhone. Screw twitter!

Ruby RJB: Confusion about Rjb::bind

Currently, I am working on an automated testing project that requires consuming webMethods services from an non-webMethods client. I like the productivity gain I get when working with Ruby so I want to build the "business" and presentation layers (the things that make up test cases) in Ruby, but I don't want to have to spend alot of time figuring out how to call WM services using SOAP with Ruby. The reason for this is because I already have a pretty good Java client that calls WM services just fine.

So my shortcut is to use RJB (or, Ruby Java Bridge) to be able to use the java WM client inside of my Ruby code. RJB uses JNI to to talk to the JVM (which is loaded by calling Rjb::load(classpath=.,jvmargs=[]) ) instead of taking the JRuby approach of compiling ruby code into the java bytecode that can be executed in the JVM.

The documentation on the RJB page ( rjb.rubyforge.org), while good enough to get you started, is rather sparse. One of the documented methods is Rjb::bind(obj,int), where obj is a ruby object that satisfies a Java interface, and int is the fully qualified name of a java interface, eg java.util.Map.

This allows you to bind a ruby object to an interface inside the JVM. The first impression I gleaned from this is that if you are calling a method on a Java object from inside Ruby, you need to have Ruby objects that satisfy the interfaces of the arguments to the method.

For instance, The Java object I am using in my Ruby code is com.gxs.wmclient.ServiceClient and specifically, the method I need to execute is: public HashMap runService(String url, String user, String pass, String serv, HashMap inputs). RJB handles translations of strings, so a Ruby string is automatically translated to a Java String. The HashMap at the end of the arg list is what I was scratching my head over. And it seemed to me, the way to do it was to implement a HashMap in Ruby and provide method definitions for all the Map interface methods. Then, in my ruby script that calls the WM Service, instantate a Ruby HashMap, bind it to java.util.Map with the Rjb::bind method, then pass it into the call to runService(). All of that looks something like this (I am leaving out the ruby HashMap):

  def setup
     #use the setup method to load the JAR files required for ServiceClient
     @path = File.dirname(__FILE__) +"/../lib/wmjsonclient.jar;"
     @path = @path + File.dirname(__FILE__) + "/../lib/client.jar;"
     @path = @path + File.dirname(__FILE__) + "/../lib/enttoolkit.jar"
     Rjb::load(@path)
  end

  def test_service_call_with_hashmap

    #import the java class I want to use
   wm_serv = Rjb::import("com.gxs.wmclient.ServiceClient")
   #create a new instance of ServiceClient
   client = wm_serv.new

   #Create a new Ruby HashMap
   input = HashMap.new()

   #bind the HashMap to java.util.Map and add two objects to it
   input = Rjb::bind(input,"java.util.Map")
   input.put("num1","3")
   input.put("num2","4")
   assert_equal 2, input.size()

   #create a hashmap to catch the outputs, bind this to
   #java.util.Map as well
   outputs = HashMap.new()
   outputs = Rjb::bind(outputs,"java.util.Map")

   #execute the runService method
   outputs = client.runService("WMServer.inside.gxs.com","AUser","APassWrd",inputs);

   s = outputs.get("value")
   puts "s type: #{s._classname}"
   assert_equal 3, outputs.size()
   assert_equal "7", s.toString()
   
  end

Unexpectedly (for me at least) this fails miserably:

NoMethodError: undefined method `put' for #<#<Class:0x2d48b48>:0x2cf1884>
C:/Documents and Settings/WhitenerR/RubymineProjects/JavaBridgeTests/tests/hash_map_test.rb:67:in `test_service_call_with_hashmap'

No 'put' method? The Map interface most definitely defines a put() method, and I have implmented it in my Ruby HashMap:

   class HashMap < Hash
    ...
    #stores an object o at key k in this hash map
     def put(k,o)
       self[k] = o
     end
    ...
   end

I looked and looked on the web, did alot of Google searching, read alot of Ruby forum posts, thumbed through my "Enterprise Integration with Ruby" book, and nothing. I had a hard time figuring out why the put() method isn't visible after I bind my Ruby object to the Map interface. The only explanation I can come up with is this:

NoMethodError: undefined method `put' for #<#<Class:0x2d48b48>:0x2cf1884>

The bind method appears to have wrapped a class of some kind around my HashMap class. I am guessing that this wrapper is something from the JVM, probably the Map interface. I am now trying to use this "interface" in my Ruby code as if it were a concrete object - but it isn't, and since it isn't, when it was created in the Ruby VM, it probably didn't come with any methods. If this is true, what is the point of having the bind operation?

Anyways, I am not getting paid to figure out the intricacies of Rjb::bind, I am getting paid to provide a solution. Here is how I got around this: instantiate java.util.HashMap objects to hold inputs and catch outputs and use those instead of trying to bind Ruby objects to interfaces inside the JVM. Here is what it looks like:

  #this one no longer uses the Ruby hashmap.  It is apparent
  #that I don't need to pass ruby objects into the JVM, since I can
  #instantiate Java objects in Ruby and then pass those in.
  def test_service_call_with_hashmap

   #import the java classes I want to use
   wm_serv = Rjb::import("com.gxs.wmclient.ServiceClient")
   j_hashmap = Rjb::import("java.util.HashMap")

   #create a new instance of ServiceClient
   client = wm_serv.new

   #create a new java HashMap, and add some objects to it.
   j_inputs = j_hashmap.new
   j_inputs.put("num1","3")
   j_inputs.put("num2", "4")
   assert_equal 2, j_inputs.size() 

   #execute the runService method, notice I can just catch the HashMap output,
   #I can use outputs in my ruby code just like its a ruby object!
   outputs = client.runService("WMServer.inside.gxs.com","AUser","APassWrd","pub.math:addInts",j_inputs);

   #get the output I am looking for from the service call
   s = outputs.get("value")
   puts "s type: #{s._classname}"
   assert_equal 3, outputs.size()
   assert_equal "7", s.toString()
   
  end
  

And the results:

  4 tests, 7 assertions, 0 failures, 0 errors
  Test suite finished: 27.126 seconds

  key1 => value1
  key2 => value2
  Context Established....
  Connection Established with Integration Server....
  JSON::
  {
    "num1": "3",
    "num2": "4"
  }
  Executing Service> pub.math:addInts
  Service invocation complete....
  s type: java.lang.String
  #

  Process finished with exit code 0

Maybe one day I will figure out a use for binding Ruby objects to interfaces in the JVM, but it won't be today.

The Exocortex

There is a fabulous book that is part of the Pragmatic Programmer bookshelf:  "Pragmatic Thinking & Learning:  How to refactor your wetware".  From this book, I have lifted the term "Exocortex"  (Please don't sue me Mr. Hunt, I have bought several of your company's books! (and I don't have any money)).

The exocortex is your external memory.  All the "open loops" (in GTD parlance) that are clogging up your cogs of cognition are thoughts, ideas, to-do's and worries, which frequently pop up in your consciousness from seemingly nowhere, belong on your exocortex.  And, once these open loops have left you riddled with anxiety,  they sink back down into the abyss to be forgotten until the next most inopportune time to arise.  Encoding this information somewhere outside of your brain helps to unclog yourself, become more productive and also make sure you are working on  things that are valuable to you instead of always chasing your tail trying to put out fires.  Thats the idea anyway, hopefully it works.

This blog is going to serve as part of my exocortex, so there is going to be a gaggle of seemingly random posts on here.  Its mainly going to consist of ideas that pop into my head, and also technical hurdles that I overcome when working with the tools I enjoy working with.  I am guessing there is going to be of Java, Ruby, iPhone, and automated testing related posts here.