In 2.0.4 we’ve added a SpaceLogListener that can be used to write log output in an asynchronous way.

It works with a companion LoggerService that pull entries from the Space and actually log them.

The configuration is very simple, here is an example:


<logger name="Q2" class="org.jpos.q2.qbean.LoggerAdaptor">
  <log-listener class="org.jpos.util.SpaceLogListener">
    <property name="queue" value="logger.q2" />
    <property name="space" value="tspace:default" />
    <property name="frozen" value="true" />


(this is basically your old 00_logger.xml, synchronously writing to disk)

<logger name="DAEMON" class="org.jpos.q2.qbean.LoggerAdaptor">
    <log-listener class="org.jpos.util.DailyLogListener">
        <property name="window" value="86400" />
        <property name="prefix" value="log/q2" />
        <property name="suffix" value=".log"/>
        <property name="date-format" value="-yyyy-MM-dd-HH"/>
        <property name="compression-format" value="gzip"/> 


<logger-service class="org.jpos.q2.qbean.LoggerService" logger="DAEMON">
    <property name="queue" value="logger.q2" />
    <property name="space" value="tspace:default" />

The logger service pulls entries from logger.q2 space queue and uses the second logger as its output (we arbitrarily called it DAEMON).

The SpaceLogListener can send to the Space the original “Live” (so to speak) LogEvent, or create a Frozen one (see the frozen property). When set to true, log filters (i.e. the ProtectedLogFilter) has to be applied before the SpaceLogListener, otherwise, it won’t work on the frozen LogEvent.

A FrozenLogEvent is Serializable, so we are not limited to TSpace, one can use the ReplicatedSpace in order to implement a very simple central logging.

ReplicatedSpace – 10 years later

/by apr/

10 years ago I add an experimental ReplicatedSpace implementation to the jPOS-EE project. While it was fun to develop and test, it didn’t get too much traction (or so I thought).

It turned out that I’ve started to hear about several high-end production systems using it, so I migrated it from the old GoogleCode repo to the new Github one.

Want to give it a quick try? Here are the very simple instructions:

Clone de jPOS-Template into a test directory

git clone rspace && cd rspace

Edit build.gradle

Add the dependency compile "", your dependencies will look like this:

dependencies {
  compile ('org.jpos:jpos:2.0.+') {
    exclude(module: 'junit')
    exclude(module: 'hamcrest-core')
  testCompile 'junit:junit:4.8.2'
  compile ""

Install resources

Call gradle installResources

This will extract the ReplicatedSpace configuration from its jar and will place it in your src/dist directory

Remove the file src/dist/deploy/01_multi_instance_prevention.xml so that you can run multiple instances on the same machine

Build a distribution

gradle dist

This will create a build/distributions/rspace-2.0.0.tar.gz that you can extract in several boxes. For a quick local run, you can gradle installApp and cd build/install/rspace to have the same effect.

Run the system

Call bin/q2 and you’ll see something like this:

<log realm="" at="Thu Jun 04 15:18:17.406 UYT 2015" lifespan="1ms">
   [apr-19136|0] (1) [apr-19136]

If you run it again, or in a different box in the same LAN, you’ll see several view-accepted messages.

At this point, you can create a little script to perform some space operations, i.e:

Space sp = SpaceFactory.getSpace("rspace:rspace");

A nice way to run a script is to deploy something like this:

Add a file called say 90_script.xml to your deploy directory:

<script> server(6666); </script>

Then telnet localhost 6667 (please note the server says ‘6666’ but you connect to 6666+1, this is BeanShell stuff.

At the bsh% prompt, you can type your space operations.

Message HEX dump

From time to time, we may ask for a message hex dump, and here is how a dump looks like:

0000  08 00 A0 20 00 00 00 80  00 00 04 00 00 00 00 00  ... ............
0010  00 00 00 00 00 00 00 01  32 39 31 31 30 30 30 31  ........29110001
0020  00 01                                             ..

See? It’s ASCII text that one can easily copy and paste in order to load it in a simulator or test program (please move your mouse over the hex digits, you can see they are selectable).

It is extremely simple, you can capture a message using NetCat and then you can use the hd Unix command, or just send us the binary image in raw format, but please PLEASE P L E A S E, don’t send us a screen capture in graphic format inside a Word document or inside a cell of an spreadsheet.

When you send a graphic capture of a dump, like this:


We need to re-type the dump in a text editor, which is error-prone and a waste of time. This is a very small example, but with large messages, the chances of introducing an error are high.

Another very common problem when we ask for a dump is that we get a high level representation of the message, some times in ASCII, copy and pasted into a message, but the mailers usually wrap around lines, not preserve blanks, and don’t show unprintable characters, so it’s extremely naive to think that we can do something useful with those kind of “dumps”.

When we ask for a message dump, we don’t want a full wall of IP-packets either, we are not debugging the TCP/IP protocol, we believe it works, so we are not interested in the session establishing, ICMP packets and IP-level headers.

We just ask, actually we cry for, a simple hex dump. Is that difficult?

jPOS 1.9.8 released

  • jPOS 1.9.8 has been released, the new development version is 1.9.9-SNAPSHOT
  • jPOS-EE 2.0.6-SNAPSHOT has now upgraded dependencies, including support for Jetty 9
  • jPOS-template has a new genDocker task that installs a jpostemplate image

See ChangeLog for details.

TransactionManager getId and getContext

TransactionParticipants get called by the TransactionManager using
their prepare, prepareForAbort, commit and abort callbacks, but for
situations where one needs access to the Context in a deeper class,
called by the participant (i.e. Managers), we now have a couple of
static ThreadLocal based methods:

  • Serializable getSerializable()
  • Context getContext() (in case your Serializable is actually an instance of
  • Long getId ()

Please note returned values may be null when run outside the TM life-cycle.

Also note that the TM takes care of PAUSED transactions, setting
these values on the resumed thread.

See ChangeLog – 6da5f3 for details.

Poor man in the middle

This is a personal story not related to jPOS, but it’s somewhat related to payment networks and security, so I hope you enjoy it.

Back in the 80s here in Uruguay, when I was in my early 20s, credit cards started to become popular and merchants started to use CATs (credit authorization terminals) that used some mysterious protocol to talk to some servers in order to get authorizations.

I didn’t have a card, but my partner in crime since age 7 — my friend @dflc — got one, I think it was a VISA.

We analyzed the card and of course, we were very interested to figure out what was written in that magnetic stripe, but we didn’t have a reader. We probably tried with some tape recorder heads in order to get some audio, I don’t remember, but I’m sure we had to try that.

One day, we called a store in the new mall in the city, Montevideo Shopping Center, for personal reasons (probably wanted to buy a present or something like that). Not 100% sure I was the one that placed the call, but I think I was. I’m very anxious, so I never asked our secretary to place the calls for me, if a number was busy I would dial 100 times in a minute until the call completes (and this was rotary dialing). If I recall correctly, the store was Pascualini, still popular these days. After busy, busy, busy, I got to hear some ‘click click click click’, followed by silence…

When you are into modems, and BBSs, there’s not doubt what you do in a situation like that, you whistle! A simple short whistle starting at around 900Hz and going up to 1200~1300Hz is easy to whistle and you get V.21 and Bell 103 modems to start their connection establishing dance.

So I whistled (or my friend), and heard the modem, we knew it wasn’t a FAX (no birps), we knew exactly what was that, that new tiny CAT thing, an Omron CAT 90 that we’ve started to see at some stores.


We knew exactly what was happening, that thing wasn’t detecting that the line was free before blindly start dialing. Our eyes opened, we simultaneously smiled, it took us probably a few milliseconds to know what was next: Man in the middle!

We also saw a business opportunity (we were hungry): we knew we could build a little hardware to sense the DC voltage of a free versus busy line and sell it to the local acquirers (free line tone were not standard those rotary-dialing days).

As a first step, we planned for a proof of concept. We wanted to monitor a transaction, record it on tape, and let the real acquirer process the transaction. We were into BBSs and ran our own BBS those days. My friend had his home land line, plus 8 BBS lines in his bedroom (along side with a MicroVAX with two SCSI 500MB mirrored noisy disks spinning day and night), so we could use one line to dial the merchant, and another one to dial the acquirer. We had to do some war dialing and small social engineering to get the acquirer’s listed phone numbers, lucky for us, numbers were in the phone book.

We played with phones since we were 8 or 10 years old, I remember I used to short-circuit the phone to break my mother’s long calls when I needed it. We did phone patches for the ham radio stations, my friend @dflc used to develop his own telephone answering machine using discrete IC components (4011s and 4001s here and there) and a pair of cassette recorders, so the required hardware was ready in a couple days.

We needed a way to know when to initiate a call to the store exactly when the transaction was going to be initiated. There were no cell phones those days, but of course, we had VHF handhelds, actually a pair of Icom IC-02ATs. I used handhelds since high school, I thought they were the coolest thing to have and I still don’t understand why ladies were not impressed by a guy with that kind of technology hanging in his belt, unbelievable…


The distance between my friends’ and the mall was small, just 600 meters


The plan was easy: My friend (who owned the card) would go to the mall, buy something, send some signals (without talking, just a few push to talk pushes – for those in the know, that would be an A1 encoding) at the right time when the lady at the store was about to process the transaction. I’d be in our NOC (his bedroom) calling the merchant, the acquirer, hitting ‘REC’ on the recorder, and patching both lines with our little hardware (also monitoring with headphones).

We did a test VHF connection and although those handheld transceivers could be used to cover 80+ kilometers with good conditions, the mall was a Faraday cage, I didn’t hear him. So we needed a plan B. My friend’s brother got a mobile VHF in his car, plenty of power (50W). So we called him (via radio) and luckily he was close to the area. We explained the mission, although no questions were asked, he would take us seriously. He parked the car close to the mall (so he could listen the short transmission from inside the mall) and QSP to me. (QSP, Q2 aka QSP version 2, rings a bell?). FTW, QSP is the Q-signal code for “relay message”.

So we did the transaction, everything worked on the first try, @dflc bought himself a leather wallet or a belt, can’t remember, knowing him, I’m sure he still have it as a trophy and a way to remember that fun hacking day. The transaction was properly approved by the Visa acquirer (who BTW, now runs jPOS), we were just men in the middle.

We’ve got the transaction and I remember we analyzed it in several ways, replayed it against different modems, etc.

On the business side, we had meetings with the local card acquirers where we explained their vulnerability and offered a solution. Of course, they didn’t like the fact that we, young suspicious “hackers/crackers” were telling them what to do so they did nothing.

We kept our grin for a good while, it was a nice, albeit pretty simple, hack.

You want a timeout

Every single week, for the last 14 years I have discussions with developers, CTOs and CIOs about channel timeouts.

The discussions usually start by a customer requirement asking us to keep established socket connections forever.

They say “We want the socket to stay always connected, forever. We don’t want to see disconnects. Our systems are very reliable, our remote endpoint partners are very reliable, we don’t want a timeout”.

So I usually start with the Fallacies of distributed computing but I’m never lucky. I try to explain that I don’t want to die, but it just so happens that I will certainly die, sooner or later. It’s life.

Disconnections happen, networking problems happen all the time, router and firewall reboots, and the most evil situation, a paranoid firewall administrator configuring very tight timeouts.

When jPOS is the client, and the channel is idle for a long period of time, having no timeout is actually not a big deal. Imagine a situation where the channel is connected for say 5 minutes, but our paranoid FW administrator had set a timeout of 3 minutes to disconnect the session. While jPOS believes we are connected, we are actually not connected, so when a real transaction arrives, and we try to send it, we find out we are no longer connected. That will raise an exception, we’ll reconnect, and we’ll send the message (a few seconds later). So the problem is just a delay that may put us out of the SLA for this particular transaction, but it’s still not a big deal, the system will recover nicely.

But when jPOS is the server, and we don’t have a timeout, the client will establish a new connection, but the old one will remain connected forever. A few hours/days later, these connection will accumulate and we’ll hit the maxSessions of the QServer configuration (see the Programmer’s Guide section 8.4). Only way to recover is to restart that particular QServer, something that needs to be done manually.

You can set SO_KEEPALIVE at the channel level in order to detect these broken connections, and in order to prevent some firewalls from disconnecting your session, but the KEEPALIVE time is OS dependent.

Our recommendation is to send network management messages from time to time (i.e. every 5 minutes) and have a reasonable timeout of say 6 minutes.

There’s another situation where you want a timeout. Imagine an ideal network (I call it ‘Disney LAN’) where the connection remains ESTABLISHED from a TCP/IP standpoint, but the remote host’s application is dead and is not answering to your replies. You can of course detect that at the application level (i.e. MUX) and proactively initiate a reconnection, but if that logic fails (or you never implemented it), a reasonable timeout will recover automatically from the situation. The remote host doesn’t reply, the call to channel receive time us out out, we reconnect, and with a little bit of luck, we get to connect to a new session that actually works.