As a follow-up to the VoldemortSpace post and after one month of development and testing we are not proud (see caveats below) to announce the vdmspace module and its supporting voldemort module. I've added two modules to jposee/opt, 'voldemort' and 'vdmspace' (r153). I splitted them in two different modules because I think there might be situations where accessing a Voldemort store directly could be desirable, without the Space implementation layer. During the last month playing with this exciting technology I faced many problems and tried different implementations. I tried not to use a coordinator, but Voldemort provides no easy locking mechanism in its StoreClient interface other than optimistic locking, making some space operations difficult to implement. So here are the caveats:

• Using the vdmspace for high concurrency input / output (in space terms, in/out) over the same keys will work, but could eventually fail.
• An out with timeout works as expected, but requires an extra write in order to register the entry for later review by the garbage collector.
• You want to make sure that you run just one garbage collector in the cluster (this is a temporary limitation that can be easily solved).

Having said the cons, I believe the vdmspace is good for:

• Storing massive number of transactions, provided each transaction is stored under its own unique key (i.e. could be used to implement a SOR).
• Implementing a second level cache for configuration data that doesn't change too often.

In order to test this, you need to enable opt/commons, opt/voldemort and opt/vdmspace, then you can use simple code like this:

import org.jpos.space.*; Space sp = SpaceFactory.getSpace ("vdm:space"); ... ...

This needs a lot of test, so feedback is Welcome!

jPOS business partners from Jethro LTD UK will be visiting several jPOS sites in Naerobi, Kenya next week. If you are interested to meet their crew, please contact us and we'll make it happen. NOTE: the blond pretty lady from  their front page is not coming, she needs to stay in the front page :)

When we call:

   Object obj = sp.in ("MyQueue");
   ...
   ....
   // do some processing using 'obj'
   ...
   ...

I always feel worried about what could happen between the point in time you take the object off the Space and you place it in a safe storage for further processing. There's always some single point of failure toward the outer services, for instance, while you are reading a message from a socket, if that process dies, your message is lost, but in the case of a Space, specially a reliable one (such as the VoldemortSpace), it's just too sad to loose a message that was previously carefully stored in a highly available fail-safe way, just because a client worker crashes. The initial solutions that came to mind would involve passing around some kind of Transaction Object (like Jini's Transaction or JTA) so we could make the Space participate in the transaction, but that's something easy to say, but not that easy to implement. Then sometime ago, I've read about how Amazon's SQS nicely solves a problem like this with temporary reads, when you read from SQS, you get the object for a couple of minutes and you need to confirm that you actually got it, otherwise, the object would show up again in the queue. I think an easy way to implement that using very simple space operations, would be to implement a temporary 'move' operation. I think that a proper syntax would be something like this:

   K move (K from, long timeout);

You get the idea, you'd use something like:

  Object tmpKey = sp.move ("MyQueue", 60000L);
  try {
      Object value = sp.rdp (tmpKey);
      ...
      ... process the entry
  } finally {
      sp.inp (tmpKey);
  }

If for some reason, you fail to 'inp' from tmpKey in your 60 seconds time-window, the object would show up again in the 'MyQueue' queue. I'm not implementing this now, but I may add the method as a place holder in the Space interface.

When I start writing a new Space implementation, a Friend's episode where Phoebe's brother tells Phoebe he "melts stuff" comes to mind (hilarious). He melts stuff; I write Space implementations [nerdy laugh here]. Back in the late 90s I started reading about the Linda Coordination Language and loved it,  I deployed a 25 node Jini/JavaSpaces (Outrigger) based application (small supermarket chain, each node was a member of the Jini federation). Due to the slow links, some early problems with OutRigger and a low memory budget on the nodes, we had to replace that implementation with a minimalistic one, and that's why I wrote the first jPOS Space implementation. But I always wanted to have a reliable, self-healing, highly scalable space, and guess what, I believe Project Voldemort is a God send for that, a perfect fit. We still need a couple of weeks in order to be able to provide a first release, we still have some locking problems and need to work on the garbage collector in order to efficiently reclaim space used by expired entries, but I thought it could be a good idea to share with you the fact that we are working with this new and exciting technology just in case you want to get yourself up to speed with it. Meanwhile, you may want to read and play with Project Voldemort, Amazon Dynamo and the miniSpaces tutorial.

People sometimes use jPOS as a library and do the wiring using Java code, something that Q2 already does in a clean and easy to maintain way. In r2735 we've introduced a couple of changes to Q2 in order to simplify the way you can embed it in your application in order to get the best of both worlds. You basically need to instantiate a Q2 object and call its start() method. That's it. At shutdown time, you call stop() on that instance. There's no reason not to use Q2 now, you just need to:

import org.jpos.q2.Q2;
...
...
  Q2 q2 = new Q2("path/to/your/deploy/directory");
  q2.start();
...
...

At stop time, you simply call:

  q2.stop();

Ditto, jPOS 1.6.4 has been released. Here is the ChangeLog. We are moving development to 1.6.5 now. jPOS-EE modules/jpos/lib has been updated with a pre-compiled version of 1.6.4. Please report problems to jpos-dev mailing list (see Resources).

/by apr/ This has nothing to do with jPOS, but since I don't have a personal blog I thought it would be a good idea to share this funny finding. I think I've "invented" a very easy way to add a some level of security to my regular credit cards, I'm testing it and works great on my MasterCard. It's very simple. I've memorized the CVC on the back of the card and then erased it with my little swiss army knife (you've got one, right? otherwise you wouldn't be reading this blog). This won't protect you against a problem like this but it's good if your card gets robbed or you lost your wallet. The procedure to use the "protected" card is very simple, you need to stay close to the operator and when you see a weird expression in her/his face when [s]he turns around the card to read the CVC, you just say "it's kind of erased, the number is NNN". So far it has worked all the time, I just need to refine the method for restaurants with no wireless POS where my wife bugs me to use an "unprotected" card in order to avoid all the fuzz (she ends up using hers most of the time, which is not bad either)

Hey, check this out, here is jPOS long time contributor Bharavi!

/by apr/ I though that the 900 TPS jPOS system developed by The Gladiators was a record, but I'm in conversations with a jPOS PEP developer and I was impressed by some of his comments: I got a permission to quote some parts of our chat:

• We made performance tests using jPOS and it scaled very well. We are inserting jPOS in an ATM system for [customer name removed]

• We hope in a few months we gonna have a large number of ATMs using it.

• We made tests and for a complete transaction with data base access, log generation, and state machine control we got 1.050 tps in a central system.

• The goal was to achieve 700 tps with a maximum time of 1,5 seconds. We got 1050 with 0,5 response time for the complete transaction. Including the authorization. We made the tests in three platforms:

  • zOS + Websphere + DB2;
  • Dell x86 + Linux  + JBoss + Postgres;
  • Sparc + Solaris + SjSAS + Oracle;

  The configurations were big, but the best performance we reached in zOs and with Dell+Linux(Debian). Both were very close in the final result.

I'm happily impressed, and you?

/by apr/ If you recall my r1000 post two years ago, it's nice to see that the system has processed over half a billion transactions by this week. It's really nice to see our little jPOS and in particular the transaction manager (which was the key addition to jPOS when we worked on this system) working so well under real world load (around a million transactions a day). Congratulations to the OLS Team!! (make sure you listen to the screencast).