Skip to main content

148 posts tagged with "jpos"

View All Tags

jPOS Maven Archetype

· One min read
Alejandro Revilla
jPOS project founder

If you use Maven, there's an extremely easy way to create a jPOS application now:

mvn archetype:generate
-DarchetypeGroupId=org.jpos
-DarchetypeArtifactId=jpos-archetype
-DarchetypeVersion=1.8.2
-DarchetypeRepository=http://jpos.org/maven

If you don't want to use the remote archetype, you can always 'mvn install' jPOS source code and then:

mvn archetype:generate -DarchetypeCatalog=local

UPDATED to use 1.8.2 instead of 1.8.1-SNAPSHOT UPDATED to use 'archetypeRepository' instead of 'DremoteRepositories'

jPOS Performance

· 3 min read
Alejandro Revilla
jPOS project founder

First Data NOC When companies -- specially start-ups -- ask us regarding jPOS performance, they usually tell us they need to support a massive load in the 1000+ TPS range. In order to give some perspective to these requirements, we always exchange some industry information with jPOS power developer Andy whose company manage some jPOS systems reaching 200M transactions per month, and today Andy sent me a very nice link (via OLS' Ron Barker) about one of the largest acquirers in the world, during a really busy day, requiring 988 TPS, the important part: "The record day on Nov. 26 translated to about 988 payment authorizations per second." So I thought it would be nice to start having some stable performance tests here and there in jPOS, so we can have something to measure where we are now, and even if it's really fast for most applications, attempt to improve it. Because many developers use jPOS just to pack and unpack ISO-8583 messages, I thought it would be nice to start by measuring where we are now in the pack/unpack department. I believe there's plenty of room for optimization, but I hope you'll agree that we are in pretty good shape. I've just added a performance test to PackagerTestCase.java to try:

  • 100000 pack operations
  • 100000 unpack operations
  • 100000 pack/unpack operations
  • 100000 update, pack, and unpack operations

and here are the results:

PACK tps=31531, peak=31531, avg=25156.00 [3779.2/3779.2] UNPACK tps=32371, peak=37026, avg=29081.00 [3047.9/6827.2] PACK/UNPACK tps=16605, peak=16634, avg=16144.25 [6254.2/13081.5] UPDATE/PACK/UNPACK tps=10862, peak=10862, avg=10815.16 [10430.9/23512.5] end [23512.6/23512.6]

We are operating with a pretty heavy message, way bigger than a real-life ISO-8583 message:

The last test is the more realistic one, because we tweak the message by changing some fields (i.e. 7,11,12,13), then pack and unpack. In most jPOS applications, we receive a message, change a few fields, and send it back, for instance, a typical server will reply by just changing a few fields (i.e. 39, 38) and eventually resetting a few others. I think there's tremendous room for optimization by caching the binary image of the packed message at the field level, avoiding a re-pack unless the field has changed. Now that we know where we are, I hope to start testing some optimizations soon. You're encourage to do so too. BTW, these figures were taken on an MBP, it would be nice to have some feedback from runs on larger servers.

jPOS 1.8 series

· One min read
Alejandro Revilla
jPOS project founder

Starting in jPOS 1.8, we are going to manage jPOS builds using Maven. There's a mvn branch soon to be merged with master. It seems 1.7.2 will be the last version of the old series managed by 'ant', will jump to 1.8.0 soon. You can try 1.8.2 by adding to your pom.xml:

jpos jPOS Central Repository http://jpos.org/maven default

org.jpos jpos 1.8.2

1.7.2 has already been committed to jPOS EE. We'll continue with the old module-based build system on jPOS-EE for a while. UPDATE: changed 1.8-SNAPSHOT to 1.8.0 UPDATE: changed 1.8-SNAPSHOT to 1.8.2

JRuby support

· One min read
Alejandro Revilla
jPOS project founder

/by apr/ I've spent a great time listening to great talks at RubyConf Uruguay organized by Cubox. Now I totally understand when they say "Old School Teamwork Fun People, Serious pros: The mix you’re looking for" in their website. It is exactly that. I was delighted to listen to Charles Nutter (JRuby's author) talk but was sad with this tweet from him a few hours ago: Headius Sad So I tried latest JRuby on latest jPOS, it took just a few minutes. I've added an optional 'jruby' module to jPOS-EE (just depends on the 'commons' module) and you can now deploy a QBean that looks like this:

puts "Hello jRuby!"

And that's all there is to it :) If you want to try this in a new small project, you can follow the HOWTO in jPOS Project Guide around page 35, just add the 'commons' and 'jruby' module as svn:externals, and you'd be ready to go. And the output, you guess it: Hello jRuby! Mr. Nutter, Welcome to this southern Montevideo and beware with the dogs here :)

jPOS-EE Things

· 2 min read
Alejandro Revilla
jPOS project founder

Inspired by this talk, I got reminded to play with EAVs in jPOS-EE. I've created a jPOS-EE modules called "things" that I believe it's a good thing to have in most applications. I don't think this is a one-size-fits-all solution, you probably don't want to store your core tables (such as the transaction log, miniGL stuff, whatever) using this open schema approach, but for everything else, having this open schema can save your day (in the same way a 'flags' field in most tables can help at some point). Using it is very simple, you create a "thing" using the ThingsManager:

DB db = new DB();
ThingsManager mgr = new ThingsManager(db); Thing t = mgr.create("MyThing");

Then you can put Strings, Longs, Dates, Timestamps, long Strings and BigDecimals to your thing, i.e:

t.put ("AString", "This is a string"); t.put ("ALong", Long.MAX_VALUE); t.put ("MyInteger", i); t.put ("Date", d); t.put ("Timestamp", new Timestamp(now)); t.put ("BigDecimal", ONE_THOUSAND); t.putText("Text", "The quick brown fox is brown and the dog is lazy, and jumps.");

The ThingsManager has some handy methods already, and we'll add more as the need arise.

public List getAll (String type) ; public Thing getLast (String type); public List listByStringName (String type, String name); public List listByStringValue (String type, String value); public List listByStringNameValue (String type, String name,String value); public List listByTextName (String type, String name); public List listByTextValue (String type, String value); ... ...

Some immediate use comes to mind, we could use this to provide a more type-safe, versionable version of our existing SysConfig table (i.e. we could have a thing called 'SysConfig' and keep previous versions as 'SysConfig;1', 'SysConfig;2', etc. (or another numbering scheme if you find this one too VMS-eske). It's also good to handle extremely proprietary stuff such as fee configuration, currency exchange, etc. found in many jPOS applications. The things module is available in jPOS-EE as of r307

Q2 transient services

· 2 min read
Alejandro Revilla
jPOS project founder

In jCard and jPTS we use the concept of Stations, we have Source Stations (SS), Destination Stations (DS), Monitoring Stations (MS), Control Stations (CS) and Cryptographic Stations (HSMS), etc. Every station usually requires a handful Q2 services (QBeans), such as a MUX or MUXPool, one or more ChannelAdaptors with their filters, LogonManager, KeyExchangeManagers, eventually an independent logger, etc. You can configure those manually for a small set of stations, you can also use some scripts, but in order to support a large number of stations, and to easily manage them from the UI, we use a database to keep their basic configuration (station type, host, port, timeouts, etc.) and then use these new transient services provided by Q2. When Q2 starts, it's create an unique transient UUID, that FYI is displayed by the SystemMonitor task:

jPOS 1.6.9 r2950 e424833b-c2c1-4f8b-b743-8a69271912a2 00:00:00.170 ... ...

When you deploy a QBean using Q2's deployElement method (which is now public), you can flag that qbean as 'transient' (there's a boolean parameter). In that case, Q2 will remove the file on exit, but just in case the system crashes, it also adds a 'transient' attribute to the QBean, i.e:

... ...

The next time it runs, Q2 will generate a new instance ID, so in the rare situation where an old deployment descriptor is present in the deploy directory, it will be ignored and deleted (as it should have been removed at the previous exit). Q2.deployElement is a handy method that requires a JDom Element. If you are not a member of the JDom church, you can always create your QBean manually, you can get to know Q2's instance ID by calling its getInstanceId() method that gives you an UUID,. Note: when you deploy a bundle using Q2 --config=/path/to/your/bundle, Q2 now flags the exploded descriptors as transient. This is available as of jPOS 1.6.9 r2950

jPOS 1.6.8 has been released

· One min read
Alejandro Revilla
jPOS project founder

Just a quick note to let you know that jPOS 1.6.8 has been released (ChangeLog). This is a maintenance release that fixes several bugs and adds some reasonable defaults that can avoid some problems (i.e.TransactionManager's paused-timeout default issue). jPOS-EE users just 'svn update' and you'll get it from modules/jpos/lib/jpos.jar (r290).

Technology XYZ

· 2 min read
Alejandro Revilla
jPOS project founder

/by apr/ There are several recurring questions that jPOS new users keep asking:

  • I want to build jPOS in Netbeans/Eclipse/IDEA
  • I want to log to log4j
  • I want to run it in Tomcat/Spring/Websphere/Weblogic/JBoss (note the mix of technologies here)

Here is how I read them:

  • I want to build jPOS in my IDE because I don't have a clue about compiling a java program using ant and I'm lazy to learn something new. I don't even know how to 'cd' to the jPOS directory.
  • I know how to configure log4j and don't want to learn how to configure other stuff. jPOS comes pre-configured to use a very nice PCI-compliant log, but I'm so lazy that I don't know that, so I ask how to use it with log4j.
  • I don't know how to start a Java application as a daemon/service, but my Tomcat/Spring/Websphere somehow is already running in my server (the previous admin left it running before quitting), so I want to run jPOS inside one of those.

So people don't care about the business logic, about why they need jPOS, they just want to have something running in their environment, even if the environment is not the best choice to run their jPOS applications. jPOS is a first class application that will process transactions worth millions of dollars for you. You should consider asking us what's the best environment, the best DB, the OS to run jPOS instead of forcing it to run in weird ways. Amen.

What is jCard?

· 10 min read
Alejandro Revilla
jPOS project founder

/by apr/

As a follow-up to the initial post about jCard and jPTS, I'd like to explain what is jCard after all. jCard is an interface between the ISO-8583 world and a double-entry accounting system. -- or better yet -- jCard is an interface between the ISO-8583 v2003 based jPOS-CMF world and a multi-layer, multi-currency general purpose double-entry accounting system (miniGL). Here is an the initial ER diagram of the core components that we used at design time, although somehow changed to support customer's requirements, it still gives you the idea of how the pieces fit together: You can see here that an Issuer has CardProducts which in turn has Cards. A Card is our handle to the CardHolder which in turn can have multiple GL Accounts (think Checking, Savings, Stored Value accounts). Imagine this little chart of accounts for a pre-paid card: As seen from jCard's perspective, a deposit transaction will debit our 'Received money' account (an asset) and credit the Cardholder's SV account (it's a liability now for us). So a deposit transaction will look like this: USD 100 goes to the 'Received Money' account and 100 to the Cardholder's account. If you enlarge the picture, you'll see a little '840' number to the right of the account code, i.e. 11.001.00840, that's our layer (in this case the accounting USD layer, 840 is the ISO-4217 currency number). Here is a purchase transaction that involve some fees, followed by a reversal: You can see we just change who we owe to, instead of owing this money to the cardholder, we owe it now to the acquirer (or merchant/branch, depending on the situation). The ISO-8583 side honors the jPOS CMF, so this purchase transaction looks like this:

This particular CardProduct has a flat fee of USD 3.50 plus a 3.25% for CashAdvance transactions, this is configured like this:

mysql> select * from cardproduct_fees where id = 1;
+----+--------+----------------------+
| id | fee | type |
+----+--------+----------------------+
| 1 | 3.2500 | CashAdvance.%.840 |
| 1 | 3.5000 | CashAdvance.flat.840 |
+----+--------+----------------------+

that's why we ended-up charging USD 24.65. For those of you familiar with jPOS, looking at its TransactionManager main configuration may give you an idea of what we are talking about here:

So the previous transaction was a 200.01 (MTI=200, Processing code 013000), so we processed the following groups:

  • financial
  • prepareresponse
  • logit
  • sendresponse

The financial group looks like this: First, we perform some sanity checks, we verify that the mandatory fields are present, we allow some optional fields too.

Then we create a TranLog record (that's our master transaction log file)

We check that the card exists, it's valid, belongs to a CardHolder, etc. We do this in a PCI compliant way, that 'KID' configuration there is the DUKPT BDK Key ID.

The terminal has to be valid too:

And so is the acquirer (we'll have to pay them at some point)

We know the Card, so we know the CardHolder. We know the CardHolder has accounts, and based on the processing code (data element 3), we choose which account to use (checking, saving, credit, stored value, loyalty, whatever)

It can happen under certain scenarios (mostly due to small network outages) that we could receive a reversal for a given transaction before or almost at the same time as the transaction itself, so we check if this transaction was previously reversed:

Then, based on the CardProduct, we can perform multiple velocity checks:

And now we are ready to generate the GL transaction and compute the balances after that.

Here is the source code for org.jpos.jcard.Financial

public class Financial extends Authorization {
public int prepare (long id, Serializable context) {
return super.prepare (id, context);
}
protected short getLayerOffset() {
// financial transactions goes to the main layer for the
// given currency
return 0;
}
protected String getTransactionName() {
return "Financial";
}
}

Pretty simple, huh? Please pay attention to that 'getLayerOffset()' method that returns 0. The Authorization participant is slightly more complex:

public class Authorization extends JCardTxnSupport {
public static final String FEE_PREFIX = "CashAdvance";

public Authorization() {
super();
}
public int prepare (long id, Serializable context) {
Context ctx = (Context) context;
try {
ctx.checkPoint ("authorization-start");
TranLog tl = (TranLog) ctx.get (TRANLOG);
GLSession gls = getGLSession(ctx);
ISOMsg m = (ISOMsg) ctx.get (REQUEST);
Card card = (Card) ctx.get (CARD);
CardHolder cardHolder = (CardHolder) ctx.get (CARDHOLDER);
Issuer issuer = (Issuer) ctx.get (ISSUER);

String accountType = ctx.getString (PCODE\_ACCOUNT\_TYPE);
BigDecimal amount = (BigDecimal) ctx.get (AMOUNT);
BigDecimal acquirerFee = getAcquirerFee (m);
BigDecimal issuerFee = ZERO;

assertNotNull (issuer, INVALID_REQUEST, "Invalid Issuer");
assertNotNull (card, INVALID_REQUEST, "Invalid Card");
assertNotNull (card.getCardProduct(), INVALID_REQUEST, "Invalid CardProduct");
assertNotNull (
card.getCardProduct().getIssuedAccount(), INVALID_REQUEST,
"Invalid CardProduct Issued Account"
);
assertNotNull (cardHolder, INVALID_REQUEST, "Invalid CardHolder");
assertNotNull (amount, INVALID_AMOUNT);
assertFalse (ZERO.equals (amount), INVALID_AMOUNT, "Zero amount not valid");
assertNotNull (accountType,
INVALID_REQUEST, "Invalid processing code"
);
assertFalse(REFUND\_ACCOUNT\_TYPE.equals (accountType),
INVALID_REQUEST, "Refund account not allowed"
);
String acctid = accountType+"."+ctx.getString(CURRENCY);
FinalAccount acct = (FinalAccount)
cardHolder.getAccounts().get (acctid);

assertNotNull (acct,
ACCOUNT\_NOT\_FOUND,
"Account type '"+acctid+"' is not defined");

Journal journal = issuer.getJournal();
assertNotNull (
journal, SYSERR_DB,
"Journal not found for issuer " + issuer.getId() + " ("
\+ issuer.getName() + ")"
);

ctx.checkPoint ("authorization-pre-lock-journal");
gls.lock (journal, acct);
ctx.checkPoint ("authorization-post-lock-journal");

short currency = getCurrency (ctx.getString(CURRENCY));
short\[\] realAndPending = new short\[\] {
currency, (short) (currency + PENDING_OFFSET)
};
BigDecimal balance = gls.getBalance (journal, acct, realAndPending);
ctx.checkPoint ("authorization-compute-balance");
BigDecimal amountPlusFees = amount.add(acquirerFee);

ctx.put (ACCOUNT, acct);
String surchargeDescription = null;
if (amountPlusFees.compareTo (balance) > 0) {
BigDecimal creditLine = gls.getBalance (
journal, acct,
new short\[\] { (short)(currency+CREDIT_OFFSET) }
);
ctx.checkPoint ("authorization-get-credit-line");
if (acct.isDebit())
creditLine = creditLine.negate();

StringBuilder sb = new StringBuilder();
issuerFee = issuerFee.add(calcSurcharge (ctx, card, amount, FEE_PREFIX, sb));
if (sb.length() > 0)
surchargeDescription = sb.toString();
amountPlusFees = amountPlusFees.add (issuerFee);
if (!isForcePost() && amountPlusFees.compareTo
(balance.add(creditLine)) >= 0)
{
throw new BLException (NOT\_SUFFICIENT\_FUNDS,
"Credit line is " + creditLine
+", issuer fee=" + issuerFee);
}
}
Acquirer acquirer = (Acquirer) ctx.get (ACQUIRER);

GLTransaction txn = new GLTransaction (
getTransactionName() + " " + Long.toString(id)
);
txn.setPostDate ((Date) ctx.get (CAPTURE_DATE));
txn.createDebit (
acct, amountPlusFees, null,
(short) (currency + getLayerOffset())
);
txn.createCredit (
acquirer.getTransactionAccount(), amount,
null, (short) (currency + getLayerOffset())
);
if (!ZERO.equals(issuerFee)) {
txn.createCredit (
card.getCardProduct().getFeeAccount(), issuerFee,
surchargeDescription, (short) (currency + getLayerOffset())
);
ctx.put (ISSUER_FEE, issuerFee);
}
if (!ZERO.equals(acquirerFee)) {
txn.createCredit (
acquirer.getFeeAccount(), acquirerFee,
null, (short) (currency + getLayerOffset())
);
ctx.put (ACQUIRER_FEE, acquirerFee);
}
gls.post (journal, txn);
ctx.checkPoint ("authorization-post-transaction");
tl.setGlTransaction (txn);
ctx.put (RC, TRAN_APPROVED);
ctx.put (APPROVAL_NUMBER, getRandomAuthNumber());
return PREPARED | NO_JOIN;
} catch (ObjectNotFoundException e) {
ctx.put (RC, CARD\_NOT\_FOUND);
} catch (GLException e) {
ctx.put (RC, NOT\_SUFFICIENT\_FUNDS);
ctx.put (EXTRC, e.getMessage());
} catch (BLException e) {
ctx.put (RC, e.getMessage ());
if (e.getDetail() != null)
ctx.put (EXTRC, e.getDetail());
} catch (Throwable t) {
ctx.log (t);
} finally {
checkPoint (ctx);
}
return ABORTED;
}
public void commit (long id, Serializable context) { }
public void abort (long id, Serializable context) { }

protected short getLayerOffset() {
return PENDING_OFFSET;
}
protected String getTransactionName() {
return "Authorization";
}
protected BigDecimal calcSurcharge
(Context ctx, Card card, BigDecimal amount, StringBuilder detail)
{
BigDecimal issuerFee = ZERO;
Map fees = card.getCardProduct().getFees();
BigDecimal flatFee = fees.get(
FEE\_PREFIX + FEE\_FLAT + ctx.getString(CURRENCY)
);
BigDecimal percentageFee = fees.get(
FEE\_PREFIX + FEE\_PERCENTAGE + ctx.getString(CURRENCY)
);
if (flatFee != null) {
issuerFee = issuerFee.add (
flatFee.setScale(2, ROUNDING_MODE)
);
detail.append ("Surcharge: $");
detail.append (flatFee);
}
if (percentageFee != null) {
issuerFee = issuerFee.add (
amount.multiply(
percentageFee.movePointLeft(2)
).setScale(2, ROUNDING_MODE)
);
if (flatFee != null)
detail.append ('+');
else
detail.append ("Surcharge: ");
detail.append (percentageFee);
detail.append ('%');
}
return issuerFee;
}
protected boolean isForcePost () {
return false;
}
}

In this case getLayerOffset() returns PENDING_LAYER (a constant set to 1000). So what happens here is that when we perform an authorization, we impact the PENDING layer (i.e. 1840 if the transaction was in USD) instead of the ACCOUNTING layer (840). When we compute the USD ACCOUNTING BALANCE, we check for transactions on the 840 layer, but when we check for the available balance, we take into account layer 840 merged with layer 1840 (this is why miniGL has layers). So to recap, this is a financial transaction (MTI 200): And here is an interesting sequence:

  • Pre-Auth USD 100
  • Void of the preauth
  • Reversal of the previous void (authorization becomes active again, if available funds permit)
  • Partial completion for USD 90

If you pay attention to the layers involved, you can see that the pre-auth works on layer 1840 while the completion works on 840, the accounting layer (offset=0). We use that layers scheme to handle overdrafts and credit account (offset is 2000), so for a credit account, we pick the balances from 840,1840,2840 (provided the transaction is performed in USD). jCard was developed in paralell with the jPOS CMF and the jCard selftest facility, based on jPOS-EE's clientsimulator. Whenever we touch a single line of it, we automatically run an extensive set of transactions that gives us some confidence that we are not introducing any major issue, i.e:

    77: Card 6009330000000020 - savings balance is 102     \[OK\] 50ms.
78: Card 6009330000000020 - $1 from checking to savings with $0.50 fee \[OK\] 126ms.
79: Card 6009330000000020 - savings balance is 104 \[OK\] 48ms.
80: $95.51 from checking to savings with no fee (NSF) \[OK\] 78ms.
81: $95.01 from checking to savings with $0.50 fee (NSF) \[OK\] 75ms.
82: $95.00 from checking to savings with $0.50 fee - GOOD \[OK\] 70ms.
83: Card 6009330000000020 - savings balance is now 199 \[OK\] 111ms.
84: Reverse previous transfer \[OK\] 57ms.
85: Reverse previous transfer (repeat) \[OK\] 70ms.
86: savings balance after reverse is 104 \[OK\] 58ms.
87: Withdrawal $20 from credit account with 0.50 fee \[OK\] 85ms.
88: credit balance check \[OK\] 59ms.
89: Reverse withdrawal \[OK\] 57ms.
90: credit balance check - should be back to 1000 \[OK\] 50ms.
91: $100.00 from credit to checking with $0.75 fee - GOOD \[OK\] 138ms.
92: Reverse transfer \[OK\] 52ms.
93: credit balance check - should be back to 1000 \[OK\] 57ms.
94: POS Purchase $20 from credit account with 0.50 fee to be voided \[OK\] 107ms.
95: Void previous transaction \[OK\] 51ms.
96: Void repeat \[OK\] 24ms.
97: Auth for $100 from savings account, no fees \[OK\] 82ms.
98: Void completed auth \[OK\] 51ms.
99: Void repeat \[OK\] 29ms.
100: Invalid completion for USD 90.00 (previously voided) \[OK\] 34ms.
101: Reverse void \[OK\] 56ms.
102: Reverse void retransmission \[OK\] 32ms.
103: completion for USD 90.00 (void has been reversed) \[OK\] 48ms.
104: completion for USD 90.00 retransmission \[OK\] 38ms.
105: check savings balance - should be $14.00 \[OK\] 110ms.
106: Refund for $16 \[OK\] 70ms.
107: Reverse previous refund \[OK\] 47ms.
108: refund reversed, balance back to 0.00 \[OK\] 67ms.

I hope that this post can give you an idea of what jCard is and why we are sometimes quiet on the blog, we are having fun developing all this.