Skip to main content

174 posts tagged with "jpos"

View All Tags

jPOS Next

· 3 min read
Alejandro Revilla
Alejandro Revilla
jPOS project founder

jPOS has evolved based on simple principles:

  • Latest version is always production ready
  • We don't rock the boat. All changes are backward compatible.

When Java started its new release cadence, we kept the baseline source compatibility to Java 8, while making sure jPOS was able to run on newer JDKs, but that prevented us to take advantage of new language features. There were features—like a long time waiting for multi-line Strings—that were nice to have, but one could still live without, but then there was one very interesting one for us, Project Loom (formerly fibers) that almost made it to Java 17, that we want to test at some large sites along with the TransactionManager.

Sealed classes, Records, Switch pattern matching, being prepared for JEP411 that will deprecate the Security Manager are all things we want to play with.

In addition, there are some changes we want to try in jPOS that would certainly rock the boat, related to audit logging (yes, we want to be agreegator/JSON/ELK friendly), using Strings instead of Integers in ISOMsg keys (so one doesn't have to create an arbitrary mapping on EMV related messages), Fluent APIs to create components for those that hate XML (funny, people find XML configs archaic but still love pom.xml), in-memory deployments, per-participant profiling on the TransationManager, etc.

In addition, we need to get rid of some baggage. The compat_1_5_2 module will remain available in the repo, as a reference, but it doesn't have to exist in the main tree. Some AWT/Swing based classes may go back to jPOS-EE as a module, and we may have a sibling module using JavaFX in order to see if it takes traction (JavaFX looks promising).

The CLI is up for a refactor. It's super useful to implement jPOS commands, but the API, initially very tied to JLine has proven to be kind of problematic once we added SSH support. We can do better on that side.

So... here is what we are planning to do:

  • jPOS 2.x.x series will become an LTS version with just security patches.
  • All the fun will happen in jPOS 3.x.x series, that currently has a baseline on Java 17 (but that may change if our Loom tests go well and we need it as a baseline).

For the time being, there's a next branch (already available in Github) that builds jPOS 3.0.0-SNAPSHOT.

The master branch currently has jPOS 2.1.8-SNAPSHOT but once we release 2.1.8, we'll switch next to become master (or take the opportunity to rename it to main) and use 2.1.8 as the forking point for an LTS v2 (or similar name) branch.

jpos-branch-plan

jPOS 2.1.7 has been released

· One min read
Alejandro Revilla
Alejandro Revilla
jPOS project founder

jPOS 2.1.7 has been released. New development version is 2.1.8-SNAPSHOT.

See ChangeLog for details.

Work has been started toward jPOS 3.0.0 targeting Java 17 as a baseline.

We will continue to maintain the 2.x.x series for a long while, so you can consider the 2.x.x series an LTS version, but all the fun and new features will happen in 3.x.x.

BadSharedMemories

· 4 min read
Barzilai Spinak
Barzilai Spinak
The Facilitator

/by Barzilai Spinak a.k.a. barspi/

Recently we spent two or three days hunting down a bunch of weird behaviors on a client's project.

It was a typical jPTS setup, and we were testing a Source Station (SS), talking to the central jPTS, talking to a destination station (DS). The different subsystems were chained together with the standard set of QueryHost + QMUX + ChannelAdaptor pointing to the QServer down the line.

The client wanted to do some stress load testing, and was using JMeter ^*^ with the jPOS-based ISO-8583 plugin (sending a very high load of messages to the SS).

Despite being a "standard setup", there were several custom features that were giving us all kinds of trouble (it's never a standard setup, is it?). The initial implementation was performing very badly with respect to transactions per second (TPS), and also losing many transactions due to timeouts. We had to isolate the different sources of the problems (extra queries to a very slow parallel database, a DB connection pool too small for the expected concurrency, an HSM that would choke when being hit with some ridiculously high number of TPS...). Things got much better after some refactoring, but we still got many responses with a timeout error code, or just dropped transactions, for which we never got a response. When analyzing the logs further, we saw some warnings from the multiplexers about duplicate keys, and we detected that it was better to use a different key set than QMUX's default 11, 41 (the terminal id, field 41, would often be constant, so we decided to add field 37, the Retrieval Reference Number, to the key set).

But still, many transactions were timing out... until we detected something even stranger. For example, RRN's, or other fields, that would change in the response, returning a value that belonged to a different request. Our ISOMSg's were having some weird genetic recombination!

Now, we should clarify that this system was developed mostly by the client, so we weren't familiar with all the details of their code and logic hidden in some of their custom classes. We were told that the DS was supposed to talk to a Mastercard remote node, but that at the moment it only had some "autoresponder simulator" in order to perform these tests. We imagined a simple RequestListener that just changed and added a couple of fields, with a standard success response in field 39.

In reality, the "autoresponder simulator", was a full transaction manager with several participants, one of which was a not-so-simple equivalent to the following (here, extremely simplified, for clarity and brevity)

public class SomeParticipant implements TransactionParticipant, Configurable {
    private String requestKey;              // GOOD
    private ISOMsg theMsg;             	    // BAD; DON'T DO!

    private void readRequestFromContext(Context ctx) {
(1)     theMsg= ctx.get(requestKey);  		// BAD; DON'T DO!
        // ...manipulate theMsg...
    }
    
    protected int prepare(long id, Serializable context) throws Exception {
        Context ctx = (Context)context;

(2)     readRequestFromContext(ctx);  		// OOPS!
        
(3)     theMsg.set(38, getApprovalCode());	// is theMsg still the same object
(4)     theMsg.set(39, getResponseCode());  // ...and now?

(5)     ctx.put(Constants.DS_RESPONSE, theMsg);	// what about here?

        return PREPARED | NO_JOIN;
    }

    public void setConfiguration(Configuration cfg) throws ConfigurationException {
        requestKey = cfg.get("request-key", "REQUEST");  // GOOD
    }
}

Can you spot the problem? The participant is using an instance variable, called theMsg, to store a reference to the request message in line (1). And what's wrong with that? As explained in Chapter 9 of the jPOS Programmer’s Guide, the TransactionManager creates only one instance of each participant, but there may be multiple concurrent transactions (i.e. TransactioManager sessions) running. So, by using an instance member variable to store the value retrieved in line (1), each concurrent transaction (thread) being processed may be overwriting that value with a different ISOMsg. In fact, at each step where we use theMsg, such as in lines (2), (3), and (4), we could be working with a different ISOMsg!

Also, notice how it's perfectly OK to use instance variables to store configuration options for this participant. These are usually read at the time of participant instantiation and initialization, and set in the setConfiguration() method. Those values shouldn't normally change during the life of the participant.

So, what should we do? Always use either local variables, or pass the values as method arguments, or store them in the Context object and retrieve them from there.

Never use the instance variables of a participant to store transaction state, or you may incur in hard-to-debug race conditions

(*) We may have a blog post about using JMeter with jPOS in the near future

Asking a smart question

· 3 min read
Alejandro Revilla
Alejandro Revilla
jPOS project founder

/by apr/

Among the many helpful jPOS developers in our community, @marklsalter stands out for his professional, accurate, and detailed answers, but he has his standards when it comes to how you ask questions. You need to ask a smart question. jPOS has Mark (and Victor, and Andy, and Dave, and Matias, and Chhil, and Barzi, and, and, and), but if you go to any other open source community asking for free advice, you'll find another Mark, or worst than that, you'll find no Mark and your question will just get ignored and you won't even know why.

This is what you should expect as a response if you don't do your homework and ask a smart question (from a recent reply in jpos-users, in this case, related to a vague question about to the Transaction Manager - could have been anything else).

Please always start by asking a smart question.

Please read this now :-

http://www.catb.org/esr/faqs/smart-questions.html

Yes, the whole things, go on, treat yourself, it will take 5-10 minutes and save us both hours going forward.

Preparing to ask a smart question should cause you to read the available documentation, to understand it and enable you to make sure you include all the relevant details needed for another remote person to help you, but should also make sure you have understood the documentation and how it applies to your need.

I can honestly say that on this opening post that I could see that it was going to be another thread that would drag on - without you ...

  1. Apparently making any effort to understand what you have done incorrectly or "misunderstood".
  2. Trying first to understand how and why your set-up is broken
  3. Referring to the documentation on the life cycle of the TransactionManager and it's given participants - which perhaps surprisingly, works perfectly when users follow the few simple rules and grasp what it does for you.

By the way, I understand that a TM configuration might not be obvious straight away, but often the best things are worth the effort. I still refer to the documentation again and again (and again).

I will include some comment below to, in the hope that you read it and take the time next time to ask a smart question.

Remember as you read through that I am not taking the piss out of you, but trying to highlight why this is a terrible opening question and how you can (hopefully) help yourself next time.