Java EE CTS goes open source!

Java EE CTS stands for Java EE Compatibility Test Suite. It's the proprietary testsuite developed initially by Sun and later by Oracle that can be used to verify that a Java EE implementation is indeed compatible. Or else, it's a huge testsuite that has been enhanced over the years to ensure compliance with the latest Java EE specifications. It tests both individual APIs, as well as the platform and the provided configuration profiles (Full and Web) as a whole. For Java EE 8 the CTS contains more than 44k tests and that doesn't include some individual Test Compatibility Kits (TCKs) for a couple of specifications that were open source from the start, like CDI and Bean Validation.

Up until now getting access to the CTS involved negotiating a license with Sun/Oracle. I remember the early JBoss days and how we were (I believe) the first open source Java EE implementation that got access to the CTS in exchange of a seven figure dollar amount, which was a lot of money back then and it only became possible because JBoss had just received funding from a Venture Capital. As an open source project and without proper funding we wouldn't have any chance of getting to it.

But that was the past. As of Sep 14, 2018 you may just as well access the CTS here:

https://github.com/eclipse-ee4j/jakartaee-tck

The Java EE CTS is on it's way to be transformed into the Jakarta EE 8 CTS. The initial code drop has been done and there is some IP checking to be completed while some further massaging of the code is necessary before it can be called Jakarta EE 8 CTS. But we are not far from that milestone and that would get us a step closer into the re-incarnation of Java EE as Jakarta EE. (BTW, you can check the overall project status here).

The CTS holds a special place in my heart because that was the first (surprise!) project I was asked to work on when I was hired by JBoss back in 2004. I was a volunteer committer on JBoss when I've got the call by Mark Fleury to join the company. (For more details, check out this blog entry.) I was told I would work on "Core Development" but no one told me on what exactly. As soon as I started, Sacha Labourey (now CEO at CloudBees) broke the news to me: my first major assignment would be to help complete J2EE 1.4 certification for JBoss 4.0.

The core development team had already managed to bring the TCK up to around 80% pass rate but, as with most things, the hardest parts were left for the end. Interoperability testing was one of the toughest areas of the testsuite that involved calls between JBoss and the Reference Implementation (RI). Different types of EJBs deployed on both servers were calling each other and transaction and security context had to propagate from server to server over RMI/IIOP - which really meant CORBA underneath.

JBoss had an amazing implementation of RMI/IIOP put in place by a bunch of developers with Francisco Reverbel as component lead and committer on the JacORB project. There was even a published paper that explained the technological innovations entitled: "Dynamic Deployment of IIOP-Enabled Components in the JBoss Server". In short, while every other application server out there required you to pre-compile the RMI/IIOP Stubs, JBoss could generate them on the fly upon deployment. Not only that, but the dynamic stubs could be transported over the wire to clients accessing the server. On the flip side, the implementation was quite complex because there was a lot of classloader magic involved to pull this off.

At that point I've realized that one of the reasons I was hired was that I was the resident CORBA expert in my previous company. Which came really handy after I've found out that the interop testsuites were failing not due to some nasty bug, but simply because a whole lot of functionality was  missing  - the dreaded CSIv2 support. (Common Secure Interoperability Protocol Version 2 - a protocol implementing security features for inter-ORB communication.)

Which meant that I've had to go and read the OMG specifications and implement the missing protocols from scratch, but also spend an enormous amount of time analyzing and debugging the low level message exchanges between the different servers. Apparently not everything was described sufficiently by the spec, you've had to figure out how the different implementations actually behaved. And that was before the RI was opensourced, so I didn't even have access to the code I was testing with.

The CTS itself seemed like a monster. It was the largest testsuite I've ever worked with, consisting of something like twenty to thirty thousands tests. Not only it was challenging to setup and implement the necessary test drivers to link your implementation to the testsuite harness, but running the tests themselves took a lot of time, in the order of hours for the different parts of the testsuite and days if you wanted to run everything. We did have periodic CI-like testing for the main JBoss testsuite back then but not for the TCK.  We run the TCK on our local machines.

To make it even worse, interoperability testing was the most resource hungry part of the testsuite. A Swing GUI (jvm1) was invoking a test client (jvm2) that was accessing the JBoss server (jvm3) that was calling the Reference Implementation (jvm4) or the reverse, with both of the servers using a back end database (jvm5) to store data. Those JVMs plus your IDE of choice to step through the code could easily bring a beefed up laptop down to its knees.

Passing the TCK became pretty much an obsession. I didn't have holidays that magical summer of 2004 in which Greece won the Euro, Athens was hosting the Summer Olympic Games and I've had to earn my stripes as a Core Developer working around the clock on the coolest company on the planet next to some legendary JBoss Developers of the likes of Adrian Brock, Scott Stark, Bill Burke, Thomas Diesler and others.

As it came to be, passing the interop CSIv2 tests was the last remaining piece of the puzzle that sealed our J2EE 1.4 certification and lead to the release of JBoss 4.0 on Sep 20th, 2004, exactly 14 years ago. You can still read in the relevant What's New in JBoss 4 release notes the announcement of JBoss 4 as the industry's first officially certified J2EE 1.4 application server.

Passing the CTS was a huge boost for JBoss. It meant that we could really be in the same league with the big guys and we could start chasing competitive migrations from the other eighteen J2EE 1.4 compliant implementations. That was the power of standards in general and Java EE in particular, offering choice to the developer and this is still the reason why standards are important: so that portability is possible in this brave new Cloud Native era.

An open CTS/TCK levels the playing field, reduces the barrier to entry and allows new implementation to compete on features. It also facilitates collaboration between creators and implementers of new APIs and the broader community. It took some time for Oracle to let Java EE free but they finally did it and they should be applauded for this. It is now up to all of us to make Jakarta EE a success.

/Dimitris