Skip to main content

148 posts tagged with "jpos"

View All Tags

jPOS 1.9.4 released

· One min read
Alejandro Revilla
jPOS project founder

jPOS 1.9.4 has been released and it includes the following changes, most notably OSGi support.

  • Added len,description constructor to IF_NOP
  • Added IFB_LLHEX (can be used to deal with encrypted track2s)
  • Added HexNibblesPrefixer (required by IFB_LLHEX)
  • Added OSGi support
  • Added 'qnode' (OSGi testbed)
  • DirPoll now supports file compression
  • Profiler can be reenabled
  • TransactionManager PAUSED transactions reuse profiler to provide combined elapsed times
  • Added org.jpos.iso.GenericSSLSocketFactory
  • jPOS-105 QServer should unregister as a space listener
  • jPOS-106 ChannelAdaptor reconnect flag uses non serializable Object
  • jPOS-108 FSDMsg consuming input stream
  • DirPoll.scan is now protected
  • MUX interface now extends ISOSource
  • QMUX.send now checks isConnected()
  • DirPoll now accepts a priority.regex boolean property (73c2f84)
  • jPOS-110 QMUX major start-up issue (1526dab)
  • DirPoll Retry when archive.timestamp is set to true (pull/33)
  • Generate optional app specific version info 02f739a

See full ChangeLog.

1.9.4 is available in Maven Central.

W55Y

· 4 min read
Alejandro Revilla
jPOS project founder

What API designers could learn from the payments industry


We all know the Fallacies of Distributed Computing:

  1. The network is reliable.
  2. Latency is zero.
  3. Bandwidth is infinite.
  4. The network is secure.
  5. Topology doesn't change.
  6. There is one administrator.
  7. Transport cost is zero.
  8. The network is homogeneous.

I think there's a 9th: REST API designers know the meaning of the word Fallacy.

So to clarify and as a public service, we need to start talking about distributed computing facts instead:

  1. The network IS NOT reliable.
  2. Latency IS NOT zero.
  3. Bandwidth is limited.
  4. The network is not secure (you should know that already)
  5. Topology does change, and at the worst possible time.
  6. The administrator is dead.
  7. Moving bits around the net has a cost.
  8. The network heterogeneous.

Interesting enough, a very old protocol, ISO-8583, designed in the 80s to support slow 300 and 1200 bps dialup links is extremely aware of these facts and work around these problems with a very simple and asynchronous message flow.

Take a look any popular REST payments API, you usually call a method to authorize a transaction and you pass parameters like this:

  • Card and Expiration (or a token)
  • Amount, perhaps currency
  • A description

Lovely, simple, and wrong!

Using that popular design, you POST the transaction and pray to get a response from the server. If you get a response, either a 200, 201, or even a 500 from the server, everything is alright, but if the request times out, or you go down, or the servers goes down, or the ISP is reconfiguring a router, you can't really tell what happened. If you're lucky and the server didn't receive your request, then that's fine, you can retry it, but if the server did receive your request, and authorized against its upper level acquirer, then you'll have an angry cardholder with a hold in its account, and perhaps even a debit (because I don't see many payment gateways accepting reconciliation messages or settlement files).

In ISO-8583, when you send an authorization request or a financial request and you don't get a response, you queue a reversal in a persistent store and forward (SAF) queue. So the next time you contact the server, before sending a new transaction, you send that reversal. If you receive a response from the server, but the response comes late and you have already timed out, you also send a 'late-response' reversal to the server.

In the same way, when you post a transaction that already took place (i.e. adjusting a previously approved transaction for a different amount, something that happens all the time at restaurants that support tips, or gas pump transactions where you approve for $100 but then complete for $20), and you don't get a response, you send a retransmission of the same transaction, as many times as necessary in order to deal with transient network problems.

In order to support reversals and retransmissions, you need a transaction unique identifier. Different networks use different transaction identifiers, either the Terminal ID + a Serial Trace Audit Number, or a Retrieval Reference Number, or in ISO-8583 v2003 the new Transaction life cycle identification data, the client generates a unique identifier so that when you send a follow-up transaction, you can send the server a reference to the original one.

I believe all payment APIs out there (including those super very cool ones) should consider adding three things:

A new parameter, RRN

Client code could generate a UUID and use it as the RRN (Retrieval Reference Number)

Support reversals (DELETE)

Could be as simple as adding a new verb, DELETE, where the only parameter can be -- along with authentication data -- the RRN

Support for retransmissions (PUT)

If your initial transaction was a POST, I propose that you also accept a PUT, with the same parameters. On the server side the difference between the POST and the PUT operations would be just an extra step to make sure that you didn't process the original POST, and return the original auth code if that was the case.

Of course, if you're designing a new super cool minimalistic REST API you probably don't listen to people with grey hair, but just in case, my 2c :)

@apr

jPOS Programmer's Guide

· One min read
Alejandro Revilla
jPOS project founder

For those not regularly checking the [jPOS main site], please note there's a new Learn tab in the main site with a direct link to download the new and free jPOS Programmer's guide draft.

While it's still work in progress, it provides useful information related to jPOS 1.9.x series that complement the standard for-sale guide.

Feedback is of course very Welcome!

Eating our own dogfood

· 5 min read
Alejandro Revilla
jPOS project founder

I really like the TransactionManager, it allows me to clearly define a transaction as if it was processed in an assembly line, with each small reusable participant doing its own little part, and the TransactionManager giving useful profiling information.

But now we live in a RESTful world, we need to implement RESTful based services here and there.

A typical REST call implementation looks like this:

    @Path("/customers/{customer_id}/wallets/{wallet_id}/credit")
public class WalletCredit extends WalletCreditDebitSupport {
@POST
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
public Response credit (
@Context UriInfo uriInfo,
@PathParam("customer_id") String customerId,
@PathParam("wallet_id") String walletId,
@FormParam("rrn") String rrn,
@FormParam("detail") String detail,
@FormParam("amount") BigDecimal amount,
@FormParam("currency") String currency)
throws IOException, URISyntaxException, BLException, ISOException
{




}
}

so the first reaction is to just start coding the business logic as part of the body of that method. You need to:

  • sanity check the parameters
  • open a database connection
  • create a TranLog record
  • Validate the Customer, Account, etc.
  • create a GL Transaction
  • do some audit logging
  • commit the JDBC transaction

This of course work fine, but hey, we already have code that does that:

  • We have a standard transaction Open participant
  • We usually have a CreateTranLog participant
  • We have a CheckCustomer participant and CheckAccount
  • We have participants that generate GLTransactions
  • And of course, the Close and Debug participants

Problem with a traditional implementation inside that body is that we need to reinvent the wheel, repeat code, add custom profiling or otherwise we wouldn't know which operations are fast and which ones are slow.

The solution was to eat our own dog food and use the TransactionManager.

So in this jCard case I've created a rest_txnmgr.xml that looks like this:

    <!DOCTYPE txnmgr [
<!ENTITY PAN_PATTERN "^[\d]{10}$">
<!ENTITY AMOUNT_PATTERN "[+-]?(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?">
<!ENTITY CURRENCY_PATTERN "^\d{1,4}">
<!ENTITY RRN_PATTERN "^\d{1,12}">
<!ENTITY DETAIL_PATTERN "^.{0,255}$">
<!ENTITY WALLET_PATTERN "^[\d]{1,64}$">
<!ENTITY TEXT50_PATTERN "^[\w\s.\-\']{0,50}$">
...
...
]>

<txnmgr name='rest-txnmgr' class="org.jpos.transaction.TransactionManager" logger="Q2" realm='rest-txnmgr'>
<property name="queue" value="JCARD.RESTAPI.TXN" />
<property name="sessions" value="2" />
<property name="max-sessions" value="64" />
<property name="debug" value="true" />

<participant class="org.jpos.jcard.PrepareContext" logger="Q2" realm="prepare-context" />
<participant class="org.jpos.jcard.Switch" logger="Q2" realm="Switch">
<property name="WalletCreate" value="walletcreate close trigger-response" />
<property name="WalletCredit" value="walletcredit close trigger-response" />
<property name="WalletDebit" value="walletdebit close trigger-response" />
<property name="BalanceInquiry" value="balance-inquiry close trigger-response" />
<property name="MiniStatement" value="mini-statement close trigger-response" />

..

</participant>


<group name="walletcreate">
<participant class="org.jpos.jcard.rest.ValidateParams" logger="Q2">
<mandatory>
<param name="PAN">&PAN_PATTERN;</param>
<param name="WALLET_NUMBER">&WALLET_PATTERN;</param>
</mandatory>
<optional>
<!-- no optional fields -->
</optional>
</participant>
<participant class="org.jpos.transaction.Open" logger="Q2" realm="open">
<property name="checkpoint" value="open" />
<property name="timeout" value="300" />
</participant>
<participant class="org.jpos.jcard.CreateTranLog" logger="Q2"
realm="create-tranlog">
<property name="capture-date" value="capture-date" />
<property name="node" value="01" />
</participant>
<participant class="org.jpos.jcard.CheckCustomer"
logger="Q2" realm="checkcustomer">
</participant>
<participant class="org.jpos.jcard.rest.WalletCreateParticipant" logger="Q2" realm="create-wallet">
<property name="chart" value="jcard" />
<property name="customers-account" value="21" />
<property name="kid" value="&KID;" />
</participant>
</group>
...
...

</txnmgr>

Now the code looks like this:

    @Path("/customers/{customer_id}/wallets/{wallet_id}/credit")
public class WalletCredit extends WalletCreditDebitSupport {
@POST
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
public Response credit (
@Context UriInfo uriInfo,
@PathParam("customer_id") String customerId,
@PathParam("wallet_id") String walletId,
@FormParam("rrn") String rrn,
@FormParam("detail") String detail,
@FormParam("amount") BigDecimal amount,
@FormParam("currency") String currency)
throws IOException, URISyntaxException, BLException, ISOException
{
return process(uriInfo, customerId, walletId, rrn, detail, amount, currency);
}
}
public abstract class WalletCreditDebitSupport extends RestSupport {
@SuppressWarnings("unchecked")
public Response process (
UriInfo uriInfo,
String customerId,
String walletId,
String rrn,
String detail,
BigDecimal amount,
String currency)
throws IOException, URISyntaxException, BLException, ISOException
{
org.jpos.transaction.Context ctx = new org.jpos.transaction.Context();
ctx.put(TXNNAME, getClass().getSimpleName());
ctx.put(PAN, customerId);
ctx.put(WALLET_NUMBER, walletId);
ctx.put(CURRENCY, currency);
ctx.put(RRN, rrn);
ctx.put(AMOUNT, amount);
ctx.put(DETAIL, detail);

int result = queryTxnMgr(ctx);
Map<String,Object> resp = createResponseMap();
if (result == TransactionManager.PREPARED) {
String irc = (String) ctx.get (IRC);
resp.put("success", TRAN_APPROVED.equals(irc));
resp.put("balance", ctx.get (LEDGER_BALANCE));
return Response.ok(toJson(resp), MediaType.APPLICATION_JSON)
.status(Response.Status.CREATED)
.location(URI.create(uriInfo.getAbsolutePath().toString())).build();
} else {
resp.put("success", false);
resp.put("message", ctx.getString(TxnConstants.RC));
return Response.status(Response.Status.BAD_REQUEST).entity(toJson(resp)).build();
}
}
}

And the best part of this, is that we get our familiar 'Debug' and 'Trace' events:

    <log realm="rest-txnmgr" at="Wed Oct 09 20:07:01 UYST 2013.489" lifespan="925ms">
<debug>
rest-txnmgr-0:idle:1
prepare: org.jpos.jcard.PrepareContext NO_JOIN
prepare: org.jpos.jcard.Switch READONLY NO_JOIN
selector: walletcreate close trigger-response
prepare: org.jpos.jcard.rest.ValidateParams READONLY NO_JOIN
prepare: org.jpos.transaction.Open READONLY NO_JOIN
prepare: org.jpos.jcard.CreateTranLog NO_JOIN
prepare: org.jpos.jcard.CheckCustomer NO_JOIN
prepare: org.jpos.jcard.rest.WalletCreateParticipant READONLY NO_JOIN
prepare: org.jpos.transaction.Close READONLY
prepare: org.jpos.jcard.rest.TriggerResponse READONLY
prepare: org.jpos.transaction.Debug READONLY
commit: org.jpos.transaction.Close
commit: org.jpos.jcard.rest.TriggerResponse
commit: org.jpos.transaction.Debug
head=2, tail=2, outstanding=0, active-sessions=2/64, tps=0, peak=0, avg=0.00, elapsed=925ms
<profiler>
prepare: org.jpos.jcard.PrepareContext [0.1/0.1]
prepare: org.jpos.jcard.Switch [0.0/0.2]
prepare: org.jpos.jcard.rest.ValidateParams [0.2/0.5]
prepare: org.jpos.transaction.Open [0.7/1.2]
prepare: org.jpos.jcard.CreateTranLog [2.3/3.6]
prepare: org.jpos.jcard.CheckCustomer [8.5/12.2]
prepare: org.jpos.jcard.rest.WalletCreateParticipant [556.5/568.7]
prepare: org.jpos.transaction.Close [0.1/568.9]
prepare: org.jpos.jcard.rest.TriggerResponse [0.1/569.1]
prepare: org.jpos.transaction.Debug [0.1/569.2]
commit: org.jpos.transaction.Close [344.9/914.2]
commit: org.jpos.jcard.rest.TriggerResponse [0.6/914.8]
commit: org.jpos.transaction.Debug [9.8/924.7]
end [0.9/925.7]
</profiler>
</debug>
</log>
<log realm="debug" at="Wed Oct 09 20:07:03 UYST 2013.845">
<commit>
<id>2</id>
<context>
<entry key='PAN'>0000000005</entry>
<entry key='RRN'>000000000001</entry>
<entry key='LOGEVT'><log realm="" at="Wed Oct 09 20:07:03 UYST 2013.846" lifespan="38ms">
<log>

<![CDATA[
<transaction id="97" date="20131009200703" post-date="20131009" journal="jcard">
<detail>WalletCredit 2</detail>
<entry account="11.001.00" type="debit" layer="840">
<amount>100.00</amount>
</entry>
<entry account="21.0000000005.1" type="credit" layer="840">
<amount>100.00</amount>
</entry>
</transaction>
]]>
</log>
</log>
</entry>
<entry key='WALLET_NUMBER'>0000000005</entry>
<entry key='CARDHOLDER'>org.jpos.ee.CardHolder@64a5efa3[id=5]</entry>
<entry key='DB'>org.jpos.ee.DB@63130f0d</entry>
<entry key='LEDGER_BALANCE'>100.00</entry>
<entry key='TRANLOG'>org.jpos.ee.TranLog@76250246[id=172]</entry>
<entry key='AMOUNT'>100.00</entry>
<entry key='SWITCH'>WalletCredit (walletcredit close trigger-response)</entry>
<entry key='ACCOUNT'>org.jpos.gl.FinalAccount@20c0fe7c[id=49,code=21.0000000005.1]</entry>
<entry key='TXNRESULT'>1</entry>
<entry key='TXNMGR'>rest-txnmgr</entry>
<entry key='IRC'>0000</entry>
<entry key='CURRENCY'>840</entry>
<entry key='DETAIL'>Test wallet credit</entry>
<entry key='WALLET'>org.jpos.ee.Wallet@71df1c8a[id=1,number=0000000005]</entry>
<entry key='CAPTURE_DATE'>Wed Oct 09 00:00:00 UYST 2013</entry>
<entry key='TXNNAME'>WalletCredit</entry>
<entry key='TIMESTAMP'>Wed Oct 09 20:07:03 UYST 2013</entry>
<entry key='PROFILER'>
<profiler>
prepare-context [0.0/0.0]
open [1.1/1.2]
close [533.2/534.4]
end [10.9/545.4]
</profiler>
</entry>
<entry key='ISSUER'>org.jpos.ee.Issuer@6f59db4f[id=1,name=jcard]</entry>
</context>
</commit>
</log>
<log realm="rest-txnmgr" at="Wed Oct 09 20:07:03 UYST 2013.864" lifespan="553ms">
<debug>
rest-txnmgr-0:idle:2
prepare: org.jpos.jcard.PrepareContext NO_JOIN
prepare: org.jpos.jcard.Switch READONLY NO_JOIN
selector: walletcredit close trigger-response
prepare: org.jpos.jcard.rest.ValidateParams READONLY NO_JOIN
prepare: org.jpos.transaction.Open READONLY NO_JOIN
prepare: org.jpos.jcard.CreateTranLog NO_JOIN
prepare: org.jpos.jcard.CheckCustomer NO_JOIN
prepare: org.jpos.jcard.CheckWallet READONLY NO_JOIN
prepare: org.jpos.jcard.rest.WalletTransactionParticipant READONLY NO_JOIN
prepare: org.jpos.transaction.Close READONLY
prepare: org.jpos.jcard.rest.TriggerResponse READONLY
prepare: org.jpos.transaction.Debug READONLY
commit: org.jpos.transaction.Close
commit: org.jpos.jcard.rest.TriggerResponse
commit: org.jpos.transaction.Debug
head=3, tail=3, outstanding=0, active-sessions=2/64, tps=0, peak=1, avg=0.50, elapsed=553ms
<profiler>
prepare: org.jpos.jcard.PrepareContext [0.1/0.1]
prepare: org.jpos.jcard.Switch [0.1/0.2]
prepare: org.jpos.jcard.rest.ValidateParams [0.2/0.5]
prepare: org.jpos.transaction.Open [0.8/1.3]
prepare: org.jpos.jcard.CreateTranLog [2.4/3.8]
prepare: org.jpos.jcard.CheckCustomer [8.7/12.5]
prepare: org.jpos.jcard.CheckWallet [7.3/19.9]
prepare: org.jpos.jcard.rest.WalletTransactionParticipant [487.2/507.2]
prepare: org.jpos.transaction.Close [0.1/507.3]
prepare: org.jpos.jcard.rest.TriggerResponse [0.0/507.4]
prepare: org.jpos.transaction.Debug [0.0/507.5]
commit: org.jpos.transaction.Close [27.0/534.5]
commit: org.jpos.jcard.rest.TriggerResponse [0.1/534.7]
commit: org.jpos.transaction.Debug [18.3/553.0]
end [1.0/554.1]
</profiler>
</debug>
</log>

Hope you consider using the TransactionManager the next time you need to write a REST API call.

Preliminary OSGi support

· One min read
Alejandro Revilla
jPOS project founder

As of f19a445d we added support for OSGi.

The jar task produces the jPOS OSGi bundle (in build/libs directory)

The bundleFull task creates a bigger bundle that include jPOS dependencies in its lib directory under the name jpos-1.9.3-SNAPSHOT-bundle-full.jar (precompiled versions can be downloaded from jpos-1.9.3-SNAPSHOT.jar and jpos-1.9.3-SNAPSHOT-bundle-full.jar in the jPOS Bundles repository.

There's a new qnode module that you can use to test the jPOS bundle (if you don't have an OSGi container already installed). You can go to qnode directory (sibling with the jpos one), run gradle installApp and then copy your bundle to the 'bundles' directory.

Here is a full session:

cd jpos gradle bundleFull

cd ../qnode gradle installApp

cp ../jpos/build/libs/jpos-1.9.3-SNAPSHOT-bundle-full.jar build/install/qnode/bundle build/install/qnode/bin/qnode

This should launch Q2 installed as a Bundle in an OSGi container (in this case, Apache Felix).

@apr

Setting up the Client Simulator

· 2 min read
Alejandro Revilla
jPOS project founder

There has been some interest in jPOS-Users mailing list regarding how to setup the Client Simulator module from jPOS-EE.

I'll show you here how to use it using the jPOS-template.

Step 1: Install a fresh copy of the jPOS template

git clone https://github.com/jpos/jPOS-template.git or download a copy of it.

Step 2: rename the cloned or downloaded directory

Let's call it 'clientsimulator'. After renaming it, cd clientsimulator.

Step 3: edit build.gradle

Add a line inside the dependencies block

compile group:'org.jpos.ee', name:'jposee-client-simulator', version:'2.2.1-SNAPSHOT'

Step 4: call gradle installResources

Please note when I say gradle, you can either use your locally installed gradle, or the gradlew wrapper available in the jPOS template that you just downloaded.

This will copy some sample client simulator configuration from the client-simulator.jar to your local src/dist directory.

After running that, you'll see a few new files in src/dist/deploy and src/dist/cfg, i.e:

  • 10_clientsimulator_channel.xml
  • 20_clientsimulator_mux.xml
  • 25_clientsimulator_ui.xml # remove it if you're running headless
  • 30_clientsimulator.xml
  • echo_s and echo_r in the src/dist/cfg directory

Step 5: call gradle run

As an alternative, you can navigate to your build/install/clientsimulator directory and call bin/q2 (or bin\q2.bat if you're on Windows).

As next step, you can edit your src/dist/deploy/10_clientsimulator.xml file and change it to use your selected packager.

PADChannel delay on send

· 3 min read
Alejandro Revilla
jPOS project founder

According to WikiPedia "TCP provides reliable, ordered, error-checked delivery of a stream of octets between programs running on computers connected to an intranet or the public Internet.".

It provides a reliable delivery of a "stream" of octets, not a sequence of "packets".

That's the reason most ISO-8583 wire protocol implementations use some kind of message boundary delimiters, typically using a header with a length indicator or some markers such as ETX character.

So the PADChannel was a tough packager to write, and that's the reason why ISOPackager has a stream based unpack method and all ISOFieldPackager implementations have stream based unpack operations, to read the message on-the-fly.

public void unpack (ISOComponent m, InputStream in);

When you send message over TCP/IP, you can't guarantee that the message will be transmitted in a single IP packet and will be available entirely in a single socket 'read' operation on the receiving end. You just can't. Here are some reasons why:

  • The TCP/IP stack on your machine may join together a couple messages in a single packet if it fits its MTU
  • Intermediate routers/firewalls may split your messages in smaller packets if their MTU is smaller (we found some very small MTUs in dedicated satellite links and GPRS networks).
  • You could be lucky and your message may fit a single TCP/IP packet all over the way, but for some reason, the receiving application could be busy or had a very short performance glitch, so when it arrives to the socket read operation, perhaps there are more than one message waiting. That's the reason why you see the Recv-Q column in a 'netstat' command going up and down.
  • There could be some packet loss so the sender TCP/IP stack needs to resend a message and could assemble multiple packets in a single bigger one

The list goes on, I recommend you read Uyless Black's TCP/IP books.

But with all that said, we regularly find people imagining that an ISO-8583 message transmitted on one end will arrive on a single socket read at the other end. Probably legacy implementations migrated from X.25 (where that was true) to TCP/IP without designing any message boundary delimiter strategy.

There's no solution to that, but we found this issue impossible to explain to some network admins, so we provided a very limited work around that just works on highly reliable networks with low traffic and good performance on the receiving end, adding a small 'delay' after we send a message so that we can pray that the TCP/IP stack and intermediate network could hopefully deliver the message in a single packet.

So PADChannel now has a 'delay' property (expressed in millis). If you're facing this problem, I'd use 100ms as a starting value.

jPOS JVM options

· 2 min read
Alejandro Revilla
jPOS project founder

/by @apr/

jPOS applications have a very low memory footprint, so we usually are fine running a simple

java -server -jar jpos.jar

But for applications more memory intensive (due to caching), such as jCard, the stop-the-world Full GC becomes a problem, freezing the JVM for as much as 5 seconds, or more.

If your response time is usually good but from time to time you have an inexplicable spike, I suggest you add:

-Xloggc:log/gc.log

that will create a nice log file that you can tail -f to get some insight into the JVM GC whereabouts.

If you see too many Full GCs, it's time to dig deeper. First thing to check is your -Xmx. If you set a -Xmx parameter which is high (say 1GB) but you don't set -Xms with the same value, the JVM will try to bring down the memory in use, even if you are not reaching the max value, causing frequent Full GCs. You may want to try:

Xmx1G -Xms1G

and check how the gc.log goes.

If that solves the problem, leave the JVM default options alone, you're done for now. If that doesn't solve your problem I suggest you change your GC implementation. My choice is the Concurrent Mark Sweep GC implementation that you can enable using:

-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode

This will probably solve your problem, but while you are at it, I suggest some of these:

jPOS' Q2 forces a full GC (calling System.gc()) when re-deploying jars in the deploy/lib directory. You could be using RMI that also initiates a distributed GC calling System.gc() too, or an operator could be fascinated by the GC button in his JMX console, so you want to prevent a Full GC in those situations, you do that using:

-XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses

If you care about processing transactions fast after a restar, you may set:

-XX:+TieredCompilation

and to get some latest JVM optizations:

-XX:+AggressiveOpts

The full list of my jCard JAVA_OPTS p0rn is:

java -server
-Dappname=jCard
-Dcom.sun.management.jmxremote
-Xloggc:log/gc.log
-Xmx1G -Xms1G
-XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses
-XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+CMSClassUnloadingEnabled
-XX:+CMSScavengeBeforeRemark
-XX:+AggressiveOpts
-XX:+ParallelRefProcEnabled
-XX:+TieredCompilation
-jar jcardinternal-2.0.0-SNAPSHOT.jar "$@"

UPDATE: Just came across this tweet by @gclaramunt with a pointer to this very interesting presentation about GC.

References: - Java Performance - Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning

SystemMonitor scripts

· 2 min read
Alejandro Revilla
jPOS project founder

We recently added a new small but useful feature to the SystemMonitor, the ability to run external scripts.

Here is an example:

3600000true

output looks like this:

OS: Mac OS X host: alejandro-revillas-macbook-pro.local/192.168.1.110 version: 1.9.1-SNAPSHOT (8a4a517) instance: bac8e399-ccf2-4778-97df-d37d704bb011 uptime: 00:00:08.877 processors: 8 drift : 0 memory(t/u/f): 989/177/812 threads: 56 Thread[Reference Handler,10,system] Thread[Finalizer,8,system] Thread[Signal Dispatcher,9,system] Thread[RMI TCP Accept-0,5,system] Thread[Keep-Alive-Timer,8,system] Thread[Q2-bac8e399-ccf2-4778-97df-d37d704bb011,5,main] Thread[DestroyJavaVM,5,main] Thread[Timer-0,5,main] Thread[pool-1-thread-1,5,main] Thread[DefaultQuartzScheduler_Worker-1,5,main] Thread[DefaultQuartzScheduler_Worker-2,5,main] Thread[DefaultQuartzScheduler_Worker-3,5,main] ... ... ... Thread[PooledThread-0,5,ThreadPool-0-1] Thread[PooledThread-1,5,ThreadPool-2-3] name-registrar: txnmgr: org.jpos.transaction.TransactionManager logger.: org.jpos.util.Logger tspace:default: org.jpos.space.TSpace $TAILLOCK.1878750663 0 0,0

logger.Q2.buffered: org.jpos.util.BufferedLogListener server.jcard-server: org.jpos.iso.ISOServer connected=0, rx=0, tx=0, last=0 ssm: org.jpos.security.jceadapter.SSM jcard-xml-server: org.jpos.q2.iso.QServer tspace:org.jpos.transaction.TransactionManager@6ffb75c7: org.jpos.space.TSpace $HEAD $TAIL 1 0,0

capture-date: org.jpos.ee.CaptureDate server.jcard-xml-server: org.jpos.iso.ISOServer connected=0, rx=0, tx=0, last=0 ks: org.jpos.security.SimpleKeyFile logger.Q2: org.jpos.util.Logger jcard-server: org.jpos.q2.iso.QServer uname -a: Darwin alejandro-revillas-macbook-pro.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64 id: uid=502(apr) gid=20(staff) groups=20(staff),401(com.apple.access_screensharing),12(everyone),33(_appstore),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),100(_lpoperator),204(_developer) pwd: /Users/apr/git/jcardinternal/build/install/jcardinternal uptime: 10:04 up 8 days, 14:45, 5 users, load averages: 1.26 1.17 1.35 vm_stat: Mach Virtual Memory Statistics: (page size of 4096 bytes) Pages free: 4494. Pages active: 552609. Pages inactive: 271878. Pages speculative: 245. Pages wired down: 216855. "Translation faults": 116355062. Pages copy-on-write: 5347593. Pages zero filled: 52951322. Pages reactivated: 2098799. Pageins: 3732245. Pageouts: 467375. Object cache: 73 hits of 543605 lookups (0% hit rate) jps -l: 14371 sun.tools.jps.Jps 12512 org.gradle.launcher.daemon.bootstrap.GradleDaemon 14365 jcardinternal-2.0.0-SNAPSHOT.jar

You can have handy vmstat 1 30 as a default so that you can get an idea of the overall system performance when you get a high 'drift' (drift is the new name for the old 'elapsed' info).

Six years under the AGPL

· 2 min read
Alejandro Revilla
jPOS project founder

/by @apr thinking out loud/

We moved jPOS to the AGPL license about six years ago, in hindsight I'd like to share my thoughts about the move.

  • The AGPL is a good license, perfect for our project, if people were to read it.
  • It is based on the honor system, but nobody cares about honor these days unless honor is enforced someway or another.
  • My perception is that for a large number of developers, OpenSource is the same as Free Software, Apache license is the same as GPL, LGPL, AGPL, MPL, Potatoes, Potatos, same thing.

We used to sell commercial licenses under the jPOS PEP which is a combination of license+coaching/hand-holding. Participants get it for the hand-holding part, they rarely care to get a signed agreement, we need to push to get them signed. [UPDATE: we are not accepting new PEP members as of August/2010] We have some true license purchases, those come either from large companies with large legal teams reviewing every license or from companies being purchased/merged doing due-diligence procedures

The downside of a license like this is IMHO:

  • People not willing to release their code under a compatible license (that's probably 100% of our users) and not willing to purchase a commercial license (that's about 99.96% according to our guestimates) feel guilty and go dark, never participate, never contribute code, and limit themselves to ask questions under public e-mail addresses with lots of numbers in it. They know they are free riders and nobody is proud of that, so they hide.
  • Some open source power users and projects know the license is somehow restrictive, so if they can, they avoid it.

So my belief now is that the AGPL just slows down a project like ours. It's probably perfect for a larger organization with the ability to go to the street and enforce it, but this is not our case, we don't have the resources nor the willing to do so. That said, it's still the best fit for us, so we'll stick to it for the time being.