jPOS 2.0.0 has been released, the new development version is now 2.0.1-SNAPSHOT - see ChangeLog

jPOS-EE 2.0.7-SNAPSHOT has now upgraded dependencies and uses jPOS 2.0.0

jPOS-template has been upgraded to use jPOS 2.0.0 as well.

MavenCentral has been updated (2.0.0) and jPOS Maven Repository already has 2.0.1-SNAPSHOT available.

From time to time, we may ask for a message hex dump, and here is how a dump looks like:

0000 08 00 A0 20 00 00 00 80 00 00 04 00 00 00 00 00 ... ............ 0010 00 00 00 00 00 00 00 01 32 39 31 31 30 30 30 31 ........29110001 0020 00 01 ..

See? It's ASCII text that one can easily copy and paste in order to load it in a simulator or test program (please move your mouse over the hex digits, you can see they are selectable).

It is extremely simple, you can capture a message using NetCat and then you can use the hd Unix command, or just send us the binary image in raw format, but please PLEASE P L E A S E, don't send us a screen capture in graphic format inside a Word document or inside a cell of an spreadsheet.

When you send a graphic capture of a dump, like this:

We need to re-type the dump in a text editor, which is error-prone and a waste of time. This is a very small example, but with large messages, the chances of introducing an error are high.

Another very common problem when we ask for a dump is that we get a high level representation of the message, some times in ASCII, copy and pasted into a message, but the mailers usually wrap around lines, not preserve blanks, and don't show unprintable characters, so it's extremely naive to think that we can do something useful with those kind of "dumps".

When we ask for a message dump, we don't want a full wall of IP-packets either, we are not debugging the TCP/IP protocol, we believe it works, so we are not interested in the session establishing, ICMP packets and IP-level headers.

We just ask, actually we cry for, a simple hex dump. Is that difficult?

• jPOS 1.9.8 has been released, the new development version is 1.9.9-SNAPSHOT
• jPOS-EE 2.0.6-SNAPSHOT has now upgraded dependencies, including support for Jetty 9
• jPOS-template has a new genDocker task that installs a jpostemplate image

See ChangeLog for details.

TransactionParticipants get called by the TransactionManager using their prepare, prepareForAbort, commit and abort callbacks, but for situations where one needs access to the Context in a deeper class, called by the participant (i.e. Managers), we now have a couple of static ThreadLocal based methods:

• Serializable getSerializable()
• Context getContext() (in case your Serializable is actually an instance of org.jpos.transaction.Context)
• Long getId ()

Please note returned values may be null when run outside the TM life-cycle.

Also note that the TM takes care of PAUSED transactions, setting these values on the resumed thread.

See ChangeLog - 6da5f3 for details.

This is a personal story not related to jPOS, but it's somewhat related to payment networks and security, so I hope you enjoy it.

Back in the 80s here in Uruguay, when I was in my early 20s, credit cards started to become popular and merchants started to use CATs (credit authorization terminals) that used some mysterious protocol to talk to some servers in order to get authorizations.

I didn't have a card, but my partner in crime since age 7 -- my friend @dflc -- got one, I think it was a VISA.

We analyzed the card and of course, we were very interested to figure out what was written in that magnetic stripe, but we didn't have a reader. We probably tried with some tape recorder heads in order to get some audio, I don't remember, but I'm sure we had to try that.

One day, we called a store in the new mall in the city, Montevideo Shopping Center, for personal reasons (probably wanted to buy a present or something like that). Not 100% sure I was the one that placed the call, but I think I was. I'm very anxious, so I never asked our secretary to place the calls for me, if a number was busy I would dial 100 times in a minute until the call completes (and this was rotary dialing). If I recall correctly, the store was Pascualini, still popular these days. After busy, busy, busy, I got to hear some 'click click click click', followed by silence...

When you are into modems, and BBSs, there's not doubt what you do in a situation like that, you whistle! A simple short whistle starting at around 900Hz and going up to 1200~1300Hz is easy to whistle and you get V.21 and Bell 103 modems to start their connection establishing dance.

So I whistled (or my friend), and heard the modem, we knew it wasn't a FAX (no birps), we knew exactly what was that, that new tiny CAT thing, an Omron CAT 90 that we've started to see at some stores.

We knew exactly what was happening, that thing wasn't detecting that the line was free before blindly start dialing. Our eyes opened, we simultaneously smiled, it took us probably a few milliseconds to know what was next: Man in the middle!

We also saw a business opportunity (we were hungry): we knew we could build a little hardware to sense the DC voltage of a free versus busy line and sell it to the local acquirers (free line tone were not standard those rotary-dialing days).

As a first step, we planned for a proof of concept. We wanted to monitor a transaction, record it on tape, and let the real acquirer process the transaction. We were into BBSs and ran our own BBS those days. My friend had his home land line, plus 8 BBS lines in his bedroom (along side with a MicroVAX with two SCSI 500MB mirrored noisy disks spinning day and night), so we could use one line to dial the merchant, and another one to dial the acquirer. We had to do some war dialing and small social engineering to get the acquirer's listed phone numbers, lucky for us, numbers were in the phone book.

We played with phones since we were 8 or 10 years old, I remember I used to short-circuit the phone to break my mother's long calls when I needed it. We did phone patches for the ham radio stations, my friend @dflc used to develop his own telephone answering machine using discrete IC components (4011s and 4001s here and there) and a pair of cassette recorders, so the required hardware was ready in a couple days.

We needed a way to know when to initiate a call to the store exactly when the transaction was going to be initiated. There were no cell phones those days, but of course, we had VHF handhelds, actually a pair of Icom IC-02ATs. I used handhelds since high school, I thought they were the coolest thing to have and I still don't understand why ladies were not impressed by a guy with that kind of technology hanging in his belt, unbelievable...

[]

The distance between my friends' and the mall was small, just 600 meters

[]

The plan was easy: My friend (who owned the card) would go to the mall, buy something, send some signals (without talking, just a few push to talk pushes - for those in the know, that would be an A1 encoding) at the right time when the lady at the store was about to process the transaction. I'd be in our NOC (his bedroom) calling the merchant, the acquirer, hitting 'REC' on the recorder, and patching both lines with our little hardware (also monitoring with headphones).

We did a test VHF connection and although those handheld transceivers could be used to cover 80+ kilometers with good conditions, the mall was a Faraday cage, I didn't hear him. So we needed a plan B. My friend's brother got a mobile VHF in his car, plenty of power (50W). So we called him (via radio) and luckily he was close to the area. We explained the mission, although no questions were asked, he would take us seriously. He parked the car close to the mall (so he could listen the short transmission from inside the mall) and QSP to me. (QSP, Q2 aka QSP version 2, rings a bell?). FTW, QSP is the Q-signal code for "relay message".

So we did the transaction, everything worked on the first try, @dflc bought himself a leather wallet or a belt, can't remember, knowing him, I'm sure he still have it as a trophy and a way to remember that fun hacking day. The transaction was properly approved by the Visa acquirer (who BTW, now runs jPOS), we were just men in the middle.

We've got the transaction and I remember we analyzed it in several ways, replayed it against different modems, etc.

On the business side, we had meetings with the local card acquirers where we explained their vulnerability and offered a solution. Of course, they didn't like the fact that we, young suspicious "hackers/crackers" were telling them what to do so they did nothing.

We kept our grin for a good while, it was a nice, albeit pretty simple, hack.

Every single week, for the last 14 years I have discussions with developers, CTOs and CIOs about channel timeouts.

The discussions usually start by a customer requirement asking us to keep established socket connections forever.

They say "We want the socket to stay always connected, forever. We don't want to see disconnects. Our systems are very reliable, our remote endpoint partners are very reliable, we don't want a timeout".

So I usually start with the Fallacies of distributed computing but I'm never lucky. I try to explain that I don't want to die, but it just so happens that I will certainly die, sooner or later. It's life.

Disconnections happen, networking problems happen all the time, router and firewall reboots, and the most evil situation, a paranoid firewall administrator configuring very tight timeouts.

When jPOS is the client, and the channel is idle for a long period of time, having no timeout is actually not a big deal. Imagine a situation where the channel is connected for say 5 minutes, but our paranoid FW administrator had set a timeout of 3 minutes to disconnect the session. While jPOS believes we are connected, we are actually not connected, so when a real transaction arrives, and we try to send it, we find out we are no longer connected. That will raise an exception, we'll reconnect, and we'll send the message (a few seconds later). So the problem is just a delay that may put us out of the SLA for this particular transaction, but it's still not a big deal, the system will recover nicely.

But when jPOS is the server, and we don't have a timeout, the client will establish a new connection, but the old one will remain connected forever. A few hours/days later, these connection will accumulate and we'll hit the maxSessions of the QServer configuration (see the Programmer's Guide section 8.4). Only way to recover is to restart that particular QServer, something that needs to be done manually.

You can set SO_KEEPALIVE at the channel level in order to detect these broken connections, and in order to prevent some firewalls from disconnecting your session, but the KEEPALIVE time is OS dependent.

Our recommendation is to send network management messages from time to time (i.e. every 5 minutes) and have a reasonable timeout of say 6 minutes.

There's another situation where you want a timeout. Imagine an ideal network (I call it 'Disney LAN') where the connection remains ESTABLISHED from a TCP/IP standpoint, but the remote host's application is dead and is not answering to your replies. You can of course detect that at the application level (i.e. MUX) and proactively initiate a reconnection, but if that logic fails (or you never implemented it), a reasonable timeout will recover automatically from the situation. The remote host doesn't reply, the call to channel receive time us out out, we reconnect, and with a little bit of luck, we get to connect to a new session that actually works.

First a disclaimer: I know excellent auditors, starting with my friend Dave from the Payments Systems Blog, but I also know really retarded ones, and here is a little story of a system I built some 8 or 10 years ago that would have been resilient to the HeartBleed bug, but of course, the auditor couldn't understand it, and it had the word MD5 which puts them to cry like tiny little girls, so we have to "improve" it to make it less secure.

I've been into amateur packet radio and BBS systems in the 80s where monitoring the air, or a serial line was easy, so things like one time passwords and two factor authentication have been always within my area of interest. When it came to provide an internal user interface to jPOS, and I had to design a login form, I wanted to protect the user's passwords against an operator with access to the system; I wanted people to be able to use a password like TheBossIsAnIdiot if they wanted, making it difficult for the programmer/operator on the server side to see it.

So the solution was easy. The server would generate a nonce and send it to the client, the client would use that nonce, some other data (like the session id) and the password, and send an MD5 of all that to the server.

I wasn't and I'm not a JS expert, and we didn't have things like jQuery or Angular those days, but I wrote this little piece of code that implemented the login form:

    function doHash(frm) {
        var username = frm.username.value;
        var password = frm.password.value;
    
        if (username.length < 3 || password.length < 3) {
            alert ("Invalid Username and/or Password.");
            return false;
        }
        var hash     = frm.hash.value;
        var seed     = readCookie ("JSESSIONID") + hash;
        var pass     = hex_md5 (username + password);
    
        frm.password.readOnly = true;
        frm.password.value = hex_md5 (seed + pass);
    }

The server would do the same computation in order to verify the login.

I wasn't comfortable with the solution, because somehow the initial password was either entered by the operator, or sent via email, so I forced a password change in the first login, and the password change would just send an XOR of the existing password hash so that the server could apply the same XOR and upgrade the password to the latest version.

But here comes the auditor, with a bucket in his head, and reports the process as insecure (with strong copy/pasted wording to scare management) for the following reasons:

1. On password change, the complexity of the password (you know, password length, use lower and upper and all that crap) is validated in client side, not in the server side. And there goes a rant that says "Passwords should be at least XXX characters, and have lower case and upper case letters in order to be secure, yada yada yada", so the manager would look at me like "We trusted in you... look what you did to us, our passwords can be less than XXX characters if the user hacks the client side code!").

2. It has the word MD5 and we just heard that MD5 is broken (remember this was 2005~2006), and there goes the rant about how MD5 was recently cracked, and again, the manager would give you that look, like saying, you're a lost case, I've been always scared of open source and the freetards around it.

I think the tradeoff between having one user hacking the JS to force himself a weak password, compared to protecting all users from easy eavesdropping is a good one. I also think that sending an MD5 over the wire is better than sending a clear password (although there's of course SSL involved, I'm talking about 'clear' from an application perspective). It has the side benefit of staying secure while in memory on the server side.

Auditors are obsessed with the things they were told to look after, SQL injection, XSS, or things they test with automated tools. That's fine and welcome (you don't need an auditor for that, BTW, but it's good to have more eyes on the problem). But I've never seen an auditor testing to SQL inject say a field 35/45 in an ISO8583 message, something anyone can do forging the track2 of a card and going to a shop around the corner. Take what they say with a grain of salt, and remember not all, but most of them, are just talkers.

I'm playing with the idea of making the client perform a really large number of iterations on the hash, to slow it down (kind of a client side bcrypt) without requiring too much CPU on the server, and then have the server do some more rounds, perhaps with a bcrypt final pass. I'm planning to send some timing information to the server too, in order to alert on client hardware/software changes (how long did it take you to run 100K hashes?). We'll have to figure out how to explain this in our next audit..

In jPOS 1.9.7 cce6a27 we've added a new transient trace flag to the Context that can be very useful during development.

Those of you using the TransactionManager with a large number of participants know that sometimes it becomes difficult to know who placed what in the Context.

You get to see a Context with many entries (REQUEST, RESPONSE, IRC, SOURCE, TRANLOG, TIMESTAMP, AMOUNT, PAN, ADDITIONAL_AMOUNT, etc.) but pin pointing where a given value is place gets difficult.

If the Context new trace boolean is set to true (something you can do via a configuration property in one of the initial participants such as PrepareContext or even closer to the incoming message, in the ISORequestListener when you create the Context), the Debug output would look like this:

Although it may look verbose, this could be very useful while coding, it helps you spot problems and assist on debugging.

I just found one issue in the jCard system while writing this blog post, look at this, we set the ACCOUNT in SelectAccount

ACCOUNT='org.jpos.gl.FinalAccount@7f7a1bec[id=28,code=22.0000000002]' [org.jpos.jcard.SelectAccount.prepare(SelectAccount.java:49)] [1.0/75.6]

then we set it again in Authorization.

ACCOUNT='org.jpos.gl.FinalAccount@7f7a1bec[id=28,code=22.0000000002]' [org.jpos.jcard.Authorization.prepare(Authorization.java:110)] [0.1/108.2]

Not a big deal, it's the same account, but worth checking why we are doing that.

The jPOS QMUX service uses the Space (usually the default global space) in order to communicate with other components such as the ChannelAdaptor or QServer using its in and out queues. But in addition, it implements the MUX interface by storing selected parts of a request message (known as the QMUX key), as shown in the picture below:

In high traffic systems, with many QMUXes, every thread waiting for a response would wake up, albeit for a tiny little while, when something happens in the space. This small patch done in 1.9.7 keep using the global space for the QMUX in and out queues, but uses an internal Space (currently a TSpace) to perform the key-matching dance.

The change should be transparent for most users, but we've seen some implementations out there that dangerously peek and poke our entries in the Space, and actually this patch, in addition to improve performance, intend to discourage such use in the future (by not exposing the internal space to other components). But anyway, for backward compatibility, we honor a new property reuse-space that if set to true, would revert to the old implementation, using the global space.

You might have heard a thousand times, push is good, IoC is good. pull is bad, and I have to agree.

jPOS components get their configurations pushed by the Q2 container when they implement the Configurable interface.

But if you're used to jPOS configurations, which can be filtered at build time by the Gradle build based on the desired target profile, or can be decorated by means of @vsalaman's contributed decorator, you may find yourself reinventing the wheel and figuring out how to get some Configuration object into a non jPOS component (such as a servlet or any other non jPOSsy code).

To solve that in a standard way, we've created QConfig. QConfig is a minimalistic QBean that just register its own Configuration object into the NameRegistrar (with a "config." prefix). So for example, you can deploy something like this:

The word config has been registered in QFactory.properties so the <config> element above is equivalent to:

So non jPOS running inside Q2 can get a reference to 'config' configuration by calling:

Configuration cfg = QConfig.getConfiguration("myconfigname");

While we were at it, we added the ability to merge configuration objects in other QBeans; There many ways to achieve the same without using this technique, for example, you can use <property file="xxx" /> in different QBeans to pull the same config, or you can use XML entities for that, but, because we can, we just offer this additional way to do it, which is quite simple.

Any QBean descriptor now accepts an optional attribute called merge-configuration that accepts a list of QConfig configurations and merges them on-the-fly at QBean configuration time. Here is a simple example:

deploy/00_config.xml

deploy/01_config.xml

deploy/90_script.xml

Because this merge-configuration handling is honored by QFactory, used by other components such as the TransactionManager to instantiate its participants, you can use it in TM participants as well (i.e. to pull reused configuration, such as result codes and the like).