Quick note to let you know that as of subversion r3019, new jPOS development will take place in github.com/jpos. The jPOS Project Guide has been updated.

/by apr/ I've spent a great time listening to great talks at RubyConf Uruguay organized by Cubox. Now I totally understand when they say "Old School Teamwork Fun People, Serious pros: The mix you’re looking for" in their website. It is exactly that. I was delighted to listen to Charles Nutter (JRuby's author) talk but was sad with this tweet from him a few hours ago: So I tried latest JRuby on latest jPOS, it took just a few minutes. I've added an optional 'jruby' module to jPOS-EE (just depends on the 'commons' module) and you can now deploy a QBean that looks like this:

puts "Hello jRuby!"

And that's all there is to it :) If you want to try this in a new small project, you can follow the HOWTO in jPOS Project Guide around page 35, just add the 'commons' and 'jruby' module as svn:externals, and you'd be ready to go. And the output, you guess it: Hello jRuby! Mr. Nutter, Welcome to this southern Montevideo and beware with the dogs here :)

Inspired by this talk, I got reminded to play with EAVs in jPOS-EE. I've created a jPOS-EE modules called "things" that I believe it's a good thing to have in most applications. I don't think this is a one-size-fits-all solution, you probably don't want to store your core tables (such as the transaction log, miniGL stuff, whatever) using this open schema approach, but for everything else, having this open schema can save your day (in the same way a 'flags' field in most tables can help at some point). Using it is very simple, you create a "thing" using the ThingsManager:

DB db = new DB();
ThingsManager mgr = new ThingsManager(db); Thing t = mgr.create("MyThing");

Then you can put Strings, Longs, Dates, Timestamps, long Strings and BigDecimals to your thing, i.e:

t.put ("AString", "This is a string"); t.put ("ALong", Long.MAX_VALUE); t.put ("MyInteger", i); t.put ("Date", d); t.put ("Timestamp", new Timestamp(now)); t.put ("BigDecimal", ONE_THOUSAND); t.putText("Text", "The quick brown fox is brown and the dog is lazy, and jumps.");

The ThingsManager has some handy methods already, and we'll add more as the need arise.

public List getAll (String type) ; public Thing getLast (String type); public List listByStringName (String type, String name); public List listByStringValue (String type, String value); public List listByStringNameValue (String type, String name,String value); public List listByTextName (String type, String name); public List listByTextValue (String type, String value); ... ...

Some immediate use comes to mind, we could use this to provide a more type-safe, versionable version of our existing SysConfig table (i.e. we could have a thing called 'SysConfig' and keep previous versions as 'SysConfig;1', 'SysConfig;2', etc. (or another numbering scheme if you find this one too VMS-eske). It's also good to handle extremely proprietary stuff such as fee configuration, currency exchange, etc. found in many jPOS applications. The things module is available in jPOS-EE as of r307

In jCard and jPTS we use the concept of Stations, we have Source Stations (SS), Destination Stations (DS), Monitoring Stations (MS), Control Stations (CS) and Cryptographic Stations (HSMS), etc. Every station usually requires a handful Q2 services (QBeans), such as a MUX or MUXPool, one or more ChannelAdaptors with their filters, LogonManager, KeyExchangeManagers, eventually an independent logger, etc. You can configure those manually for a small set of stations, you can also use some scripts, but in order to support a large number of stations, and to easily manage them from the UI, we use a database to keep their basic configuration (station type, host, port, timeouts, etc.) and then use these new transient services provided by Q2. When Q2 starts, it's create an unique transient UUID, that FYI is displayed by the SystemMonitor task:

jPOS 1.6.9 r2950 e424833b-c2c1-4f8b-b743-8a69271912a2 00:00:00.170 ... ...

When you deploy a QBean using Q2's deployElement method (which is now public), you can flag that qbean as 'transient' (there's a boolean parameter). In that case, Q2 will remove the file on exit, but just in case the system crashes, it also adds a 'transient' attribute to the QBean, i.e:

... ...

The next time it runs, Q2 will generate a new instance ID, so in the rare situation where an old deployment descriptor is present in the deploy directory, it will be ignored and deleted (as it should have been removed at the previous exit). Q2.deployElement is a handy method that requires a JDom Element. If you are not a member of the JDom church, you can always create your QBean manually, you can get to know Q2's instance ID by calling its getInstanceId() method that gives you an UUID,. Note: when you deploy a bundle using Q2 --config=/path/to/your/bundle, Q2 now flags the exploded descriptors as transient. This is available as of jPOS 1.6.9 r2950

Just a quick note to let you know that jPOS 1.6.8 has been released (ChangeLog). This is a maintenance release that fixes several bugs and adds some reasonable defaults that can avoid some problems (i.e.TransactionManager's paused-timeout default issue). jPOS-EE users just 'svn update' and you'll get it from modules/jpos/lib/jpos.jar (r290).

/by apr/ There are several recurring questions that jPOS new users keep asking:

• I want to build jPOS in Netbeans/Eclipse/IDEA
• I want to log to log4j
• I want to run it in Tomcat/Spring/Websphere/Weblogic/JBoss (note the mix of technologies here)

Here is how I read them:

• I want to build jPOS in my IDE because I don't have a clue about compiling a java program using ant and I'm lazy to learn something new. I don't even know how to 'cd' to the jPOS directory.
• I know how to configure log4j and don't want to learn how to configure other stuff. jPOS comes pre-configured to use a very nice PCI-compliant log, but I'm so lazy that I don't know that, so I ask how to use it with log4j.
• I don't know how to start a Java application as a daemon/service, but my Tomcat/Spring/Websphere somehow is already running in my server (the previous admin left it running before quitting), so I want to run jPOS inside one of those.

So people don't care about the business logic, about why they need jPOS, they just want to have something running in their environment, even if the environment is not the best choice to run their jPOS applications. jPOS is a first class application that will process transactions worth millions of dollars for you. You should consider asking us what's the best environment, the best DB, the OS to run jPOS instead of forcing it to run in weird ways. Amen.

/by apr/

As a follow-up to the initial post about jCard and jPTS, I'd like to explain what is jCard after all. jCard is an interface between the ISO-8583 world and a double-entry accounting system. -- or better yet -- jCard is an interface between the ISO-8583 v2003 based jPOS-CMF world and a multi-layer, multi-currency general purpose double-entry accounting system (miniGL). Here is an the initial ER diagram of the core components that we used at design time, although somehow changed to support customer's requirements, it still gives you the idea of how the pieces fit together: You can see here that an Issuer has CardProducts which in turn has Cards. A Card is our handle to the CardHolder which in turn can have multiple GL Accounts (think Checking, Savings, Stored Value accounts). Imagine this little chart of accounts for a pre-paid card: As seen from jCard's perspective, a deposit transaction will debit our 'Received money' account (an asset) and credit the Cardholder's SV account (it's a liability now for us). So a deposit transaction will look like this: USD 100 goes to the 'Received Money' account and 100 to the Cardholder's account. If you enlarge the picture, you'll see a little '840' number to the right of the account code, i.e. 11.001.00840, that's our layer (in this case the accounting USD layer, 840 is the ISO-4217 currency number). Here is a purchase transaction that involve some fees, followed by a reversal: You can see we just change who we owe to, instead of owing this money to the cardholder, we owe it now to the acquirer (or merchant/branch, depending on the situation). The ISO-8583 side honors the jPOS CMF, so this purchase transaction looks like this:

This particular CardProduct has a flat fee of USD 3.50 plus a 3.25% for CashAdvance transactions, this is configured like this:

mysql> select * from cardproduct_fees where id = 1;
+----+--------+----------------------+
| id | fee    | type                 |
+----+--------+----------------------+
|  1 | 3.2500 | CashAdvance.%.840    |
|  1 | 3.5000 | CashAdvance.flat.840 |
+----+--------+----------------------+

that's why we ended-up charging USD 24.65. For those of you familiar with jPOS, looking at its TransactionManager main configuration may give you an idea of what we are talking about here:

So the previous transaction was a 200.01 (MTI=200, Processing code 013000), so we processed the following groups:

• financial
• prepareresponse
• logit
• sendresponse

The financial group looks like this: First, we perform some sanity checks, we verify that the mandatory fields are present, we allow some optional fields too.

Then we create a TranLog record (that's our master transaction log file)

We check that the card exists, it's valid, belongs to a CardHolder, etc. We do this in a PCI compliant way, that 'KID' configuration there is the DUKPT BDK Key ID.

The terminal has to be valid too:

And so is the acquirer (we'll have to pay them at some point)

We know the Card, so we know the CardHolder. We know the CardHolder has accounts, and based on the processing code (data element 3), we choose which account to use (checking, saving, credit, stored value, loyalty, whatever)

It can happen under certain scenarios (mostly due to small network outages) that we could receive a reversal for a given transaction before or almost at the same time as the transaction itself, so we check if this transaction was previously reversed:

Then, based on the CardProduct, we can perform multiple velocity checks:

And now we are ready to generate the GL transaction and compute the balances after that.

Here is the source code for org.jpos.jcard.Financial

public class Financial extends Authorization {
    public int prepare (long id, Serializable context) {
        return super.prepare (id, context);
    }
    protected short getLayerOffset() {
        // financial transactions goes to the main layer for the
        // given currency
        return 0;  
    }
    protected String getTransactionName() {
        return "Financial";
    }
}

Pretty simple, huh? Please pay attention to that 'getLayerOffset()' method that returns 0. The Authorization participant is slightly more complex:

public class Authorization extends JCardTxnSupport {
    public static final String FEE_PREFIX = "CashAdvance";

    public Authorization() {
        super();
    }
    public int prepare (long id, Serializable context) {
        Context ctx = (Context) context;
        try {
            ctx.checkPoint ("authorization-start");
            TranLog tl  = (TranLog) ctx.get (TRANLOG);
            GLSession gls = getGLSession(ctx);
            ISOMsg m = (ISOMsg) ctx.get (REQUEST);
            Card card = (Card) ctx.get (CARD);
            CardHolder cardHolder = (CardHolder) ctx.get (CARDHOLDER);
            Issuer issuer = (Issuer) ctx.get (ISSUER);
    
            String accountType = ctx.getString (PCODE\_ACCOUNT\_TYPE);
            BigDecimal amount = (BigDecimal) ctx.get (AMOUNT);
            BigDecimal acquirerFee = getAcquirerFee (m);
            BigDecimal issuerFee = ZERO;
    
            assertNotNull (issuer, INVALID_REQUEST, "Invalid Issuer");
            assertNotNull (card, INVALID_REQUEST, "Invalid Card");
            assertNotNull (card.getCardProduct(), INVALID_REQUEST, "Invalid CardProduct");
            assertNotNull (
                 card.getCardProduct().getIssuedAccount(), INVALID_REQUEST, 
                "Invalid CardProduct Issued Account"
            ); 
            assertNotNull (cardHolder, INVALID_REQUEST, "Invalid CardHolder");
            assertNotNull (amount, INVALID_AMOUNT);
            assertFalse (ZERO.equals (amount), INVALID_AMOUNT, "Zero amount not valid");
            assertNotNull (accountType,
                INVALID_REQUEST, "Invalid processing code"
            );
            assertFalse(REFUND\_ACCOUNT\_TYPE.equals (accountType),
                INVALID_REQUEST, "Refund account not allowed"
            );
            String acctid = accountType+"."+ctx.getString(CURRENCY);
            FinalAccount acct = (FinalAccount) 
                cardHolder.getAccounts().get (acctid);
    
            assertNotNull (acct, 
                ACCOUNT\_NOT\_FOUND, 
                "Account type '"+acctid+"' is not defined");
    
            Journal journal = issuer.getJournal();
            assertNotNull (
                journal, SYSERR_DB, 
                "Journal not found for issuer " + issuer.getId() + " ("
                \+ issuer.getName() + ")"
            );
    
            ctx.checkPoint ("authorization-pre-lock-journal");
            gls.lock (journal, acct);
            ctx.checkPoint ("authorization-post-lock-journal");
    
            short currency = getCurrency (ctx.getString(CURRENCY));
            short\[\] realAndPending = new short\[\] {
                currency, (short) (currency + PENDING_OFFSET)
            };
            BigDecimal balance = gls.getBalance (journal, acct, realAndPending);
            ctx.checkPoint ("authorization-compute-balance");
            BigDecimal amountPlusFees = amount.add(acquirerFee);
    
            ctx.put (ACCOUNT, acct);
            String surchargeDescription = null;
            if (amountPlusFees.compareTo (balance) > 0) {
                BigDecimal creditLine = gls.getBalance (
                    journal, acct, 
                    new short\[\] { (short)(currency+CREDIT_OFFSET) }
                );
                ctx.checkPoint ("authorization-get-credit-line");
                if (acct.isDebit())
                    creditLine = creditLine.negate();
    
                StringBuilder sb = new StringBuilder();
                issuerFee = issuerFee.add(calcSurcharge (ctx, card, amount, FEE_PREFIX, sb));
                if (sb.length() > 0)
                    surchargeDescription = sb.toString();
                amountPlusFees = amountPlusFees.add (issuerFee);
                if (!isForcePost() && amountPlusFees.compareTo
                    (balance.add(creditLine)) >= 0) 
                {
                    throw new BLException (NOT\_SUFFICIENT\_FUNDS, 
                        "Credit line is " + creditLine 
                       +", issuer fee=" + issuerFee); 
                }
            }
            Acquirer acquirer = (Acquirer) ctx.get (ACQUIRER);
    
            GLTransaction txn = new GLTransaction (
                getTransactionName() + " " + Long.toString(id)
            );
            txn.setPostDate ((Date) ctx.get (CAPTURE_DATE));
            txn.createDebit (
                acct, amountPlusFees, null, 
                    (short) (currency + getLayerOffset())
            );
            txn.createCredit (
                acquirer.getTransactionAccount(), amount, 
                null, (short) (currency + getLayerOffset())
            );
            if (!ZERO.equals(issuerFee)) {
                txn.createCredit (
                    card.getCardProduct().getFeeAccount(), issuerFee, 
                    surchargeDescription, (short) (currency + getLayerOffset())
                );
                ctx.put (ISSUER_FEE, issuerFee);
            }
            if (!ZERO.equals(acquirerFee)) {
                txn.createCredit (
                    acquirer.getFeeAccount(), acquirerFee, 
                    null, (short) (currency + getLayerOffset())
                );
                ctx.put (ACQUIRER_FEE, acquirerFee);
            }
            gls.post (journal, txn);
            ctx.checkPoint ("authorization-post-transaction");
            tl.setGlTransaction (txn);
            ctx.put (RC, TRAN_APPROVED);
            ctx.put (APPROVAL_NUMBER, getRandomAuthNumber());
            return PREPARED | NO_JOIN;
        } catch (ObjectNotFoundException e) {
            ctx.put (RC, CARD\_NOT\_FOUND);
        } catch (GLException e) {
            ctx.put (RC, NOT\_SUFFICIENT\_FUNDS);
            ctx.put (EXTRC, e.getMessage());
        } catch (BLException e) {
            ctx.put (RC, e.getMessage ());
            if (e.getDetail() != null)
                ctx.put (EXTRC, e.getDetail());
        } catch (Throwable t) {
            ctx.log (t);
        } finally {
            checkPoint (ctx);
        }
        return ABORTED;
    }
    public void commit (long id, Serializable context) { }
    public void abort  (long id, Serializable context) { }
    
    protected short getLayerOffset() {
        return PENDING_OFFSET;
    }
    protected String getTransactionName() {
        return "Authorization";
    }
    protected BigDecimal calcSurcharge 
        (Context ctx, Card card, BigDecimal amount, StringBuilder detail) 
    {
        BigDecimal issuerFee = ZERO;
        Map fees = card.getCardProduct().getFees();
        BigDecimal flatFee = fees.get(
            FEE\_PREFIX + FEE\_FLAT + ctx.getString(CURRENCY)
        );
        BigDecimal percentageFee = fees.get(
            FEE\_PREFIX + FEE\_PERCENTAGE + ctx.getString(CURRENCY)
        );
        if (flatFee != null) {
            issuerFee = issuerFee.add (
                flatFee.setScale(2, ROUNDING_MODE)
            );
            detail.append ("Surcharge: $");
            detail.append (flatFee);
        }
        if (percentageFee != null) {
            issuerFee = issuerFee.add (
                amount.multiply(
                    percentageFee.movePointLeft(2)
                ).setScale(2, ROUNDING_MODE)
            );
            if (flatFee != null)
                detail.append ('+');
            else
                detail.append ("Surcharge: ");
            detail.append (percentageFee);
            detail.append ('%');
        }
        return issuerFee;
    }
    protected boolean isForcePost () {
        return false;
    }
} 

In this case getLayerOffset() returns PENDING_LAYER (a constant set to 1000). So what happens here is that when we perform an authorization, we impact the PENDING layer (i.e. 1840 if the transaction was in USD) instead of the ACCOUNTING layer (840). When we compute the USD ACCOUNTING BALANCE, we check for transactions on the 840 layer, but when we check for the available balance, we take into account layer 840 merged with layer 1840 (this is why miniGL has layers). So to recap, this is a financial transaction (MTI 200): And here is an interesting sequence:

• Pre-Auth USD 100
• Void of the preauth
• Reversal of the previous void (authorization becomes active again, if available funds permit)
• Partial completion for USD 90

If you pay attention to the layers involved, you can see that the pre-auth works on layer 1840 while the completion works on 840, the accounting layer (offset=0). We use that layers scheme to handle overdrafts and credit account (offset is 2000), so for a credit account, we pick the balances from 840,1840,2840 (provided the transaction is performed in USD). jCard was developed in paralell with the jPOS CMF and the jCard selftest facility, based on jPOS-EE's clientsimulator. Whenever we touch a single line of it, we automatically run an extensive set of transactions that gives us some confidence that we are not introducing any major issue, i.e:

    77: Card 6009330000000020 - savings balance is 102     \[OK\] 50ms.
    78: Card 6009330000000020 - $1 from checking to savings with $0.50 fee \[OK\] 126ms.
    79: Card 6009330000000020 - savings balance is 104     \[OK\] 48ms.
    80: $95.51 from checking to savings with no fee (NSF)  \[OK\] 78ms.
    81: $95.01 from checking to savings with $0.50 fee (NSF) \[OK\] 75ms.
    82: $95.00 from checking to savings with $0.50 fee - GOOD \[OK\] 70ms.
    83: Card 6009330000000020 - savings balance is now 199 \[OK\] 111ms.
    84: Reverse previous transfer                          \[OK\] 57ms.
    85: Reverse previous transfer (repeat)                 \[OK\] 70ms.
    86: savings balance after reverse is 104               \[OK\] 58ms.
    87: Withdrawal $20 from credit account with 0.50 fee   \[OK\] 85ms.
    88: credit balance check                               \[OK\] 59ms.
    89: Reverse withdrawal                                 \[OK\] 57ms.
    90: credit balance check - should be back to 1000      \[OK\] 50ms.
    91: $100.00 from credit to checking with $0.75 fee - GOOD \[OK\] 138ms.
    92: Reverse transfer                                   \[OK\] 52ms.
    93: credit balance check - should be back to 1000      \[OK\] 57ms.
    94: POS Purchase $20 from credit account with 0.50 fee to be voided \[OK\] 107ms.
    95: Void previous transaction                          \[OK\] 51ms.
    96: Void repeat                                        \[OK\] 24ms.
    97: Auth for $100 from savings account, no fees        \[OK\] 82ms.
    98: Void completed auth                                \[OK\] 51ms.
    99: Void repeat                                        \[OK\] 29ms.
    100: Invalid completion for USD 90.00 (previously voided) \[OK\] 34ms.
    101: Reverse void                                       \[OK\] 56ms.
    102: Reverse void retransmission                        \[OK\] 32ms.
    103: completion for USD 90.00 (void has been reversed)  \[OK\] 48ms.
    104: completion for USD 90.00 retransmission            \[OK\] 38ms.
    105: check savings balance - should be $14.00           \[OK\] 110ms.
    106: Refund for $16                                     \[OK\] 70ms.
    107: Reverse previous refund                            \[OK\] 47ms.
    108: refund reversed, balance back to 0.00              \[OK\] 67ms.

I hope that this post can give you an idea of what jCard is and why we are sometimes quiet on the blog, we are having fun developing all this.

We've migrated our issue tracking system to YouTrack. I have reviewed old issues and migrated outstanding ones to the new system. The old system will remain online for a while, but will be decommissioned at some point during 2010. Some of the issues there are kind of trivial to fix, I hope some of you --specially new jPOS developers-- could contribute some time to provide patches. Implementing features and bug fixes is the best way to learn jPOS (or any other OpenSource project), so PLEASE HELP!

jPOS 1.6.6 is out, the new development version is 1.6.7. See ChangeLog for details. jPOS-EE has been updated to the latest version.

As of jPOS 1.6.5 r2845, we added a new input-space property to the TransactionManager. With this minor addition, you can easily distribute the load among multiple Q2 instances running in the same or different machines. I'm a true believer in long-lived lightweight Q2 based components. Now thanks to our very simple Space interface and Bela Ban's awesome JGroups that allowed us to implement the ReplicatedSpace, creating this kind of jPOS clustered setup is now feasible.