Parallel Processing

/by apr/ A nice side effect of going up the food chain (in addition to getting tastier food) is that you get to interact with really smart and knowledgeable people looking for elegant solutions to complex and challenging problems.

We have production systems now doing 600k txn/day, but we are now shooting for 2M/day.

We have systems that can do 300 sustained TPS for 30 minutes, but we face a requirement to do 2000 TPS sustained for five hours now.

In a recent meeting while Andy was doing a walk-through of their OLS.Switch TransactionManager's configuration, we where asked if the participants could run in parallel.

Although we tried to explain that the role of the TransactionManager was to assemble the flow of the transaction in an "execution chain" where the order had to be preserved, one the engineers there actually had a very good point. There are many situations where you want to run things in parallel, for example, you can do some database validations while firing a pin translation message to the HSM pool.

On a multi-database environment, you may want to simultaneously fire different queries to different databases. I liked the idea and started to work as soon as the meeting was over. With the new org.jpos.transaction.participant.Join participant, you can convert something like this:

<participant class="com.your.company.DoPinEncryption">
   ...
</participant>
<participant class="com.your.company.QueryDatabase">
  ...
</participant>

into:

<participant class="org.jpos.transaction.participant.Join">
 <participant class="com.your.company.DoPinEncryption">
   ...
 </participant>
 <participant class="com.your.company.QueryDatabase">
   ...
 </participant>
</participant>

just by wrapping them inside the Join participant and run both in parallel (you are not limited to just two inner participants, you can place there as many as you want). Join honors the AbortParticipant contract so you can have your inner participants called even if the transaction is meant to abort.

I had to do some minor changes to the TransactionManager implementation as we needed a reference to the QFactory in order to reate the inner participants. In order not to break existing participants and to avoid adding a new interface, the TM uses Reflection in order to attempt to push a reference to the TransactionManager to the participant at init time. If your participant has a method:

   public void setTransactionManager (TransactionManager mgr);

then it would get called. You can use that reference to pull a reference to the QFactory which in turns uses the the MBeanServer to instantiate the participants using an appropriate dynamic classloader. If you are still not using the TransactionManager in your jPOS application, you should consider giving it a try, we believe it's the way to go.