Wednesday, November 28, 2012

Arquillian and TestNG

The RHQ Project has historically run on JBoss AS 4.2.3.  That is changing as we leap forward to JBoss AS 7.1.1.Final for our next release.  There are a lot of advantages to this move but it also presented a lot of challenges, one of which was our integration test approach. 

When deployed on AS4 RHQ used the JBoss Embedded Container to perform unit/integration tests on domain (JPA persistence) and server jar (SLSB) classes. Now that we deploy on AS7, the embedded container was no longer an option. The de facto alternative is Arquillian. Arquillian is a tool that integrates multiple test frameworks, namely JUnit and TestNG, against several containers, namely Glassfish, OpenEJB and, most importantly for us, AS7.

For a more detailed account of this change in RHQ, see this wiki page about the testing impact.  Here I will just summarize some lessons learned and tips for using Arquillian with TestNG.
This is written with reference to Arquillian version 1.0.2.

Arquillian runs the TestNG lifecycle for every test
This is critical to understand. It means that BeforeXXX/AfterXXX methods are all called for every test. Effectively this means that the only methods you can effectively use are BeforeMethod and AfterMethod.
Avoid BeforeClass/AfterClass or other flavors of these methods.
If you absolutely must you may be able to use "stand-ins" for these by using @Test methods instead. By using dependencies, or priority, you can sort of simulate BeforeClass and AfterClass. But note that test priority is evaluated only in the scope of the group its test is in. Also realize that BeforeClass and AfterClass often cause unexpected issues due to test class interactions, and stand-ins can present the same problems. Any startup/teardown actions must be impervious to random test orderings amongst all classes.
Do not share data with instance variables.
The test class is effectively new'd for each test. This can make life difficult. Even though dependencies in test order execution are honored, it does not mean that you can set variable 'foo' in testA and use the value of 'foo' in testB. You have a few alternatives:
  • Store and fetch data in the DB.
    This only works if the DB data somehow survives the local Before/After methods, and those of any other test class.
  • Store and fetch data to a file by serializing data.
    A simple util is available in AbstractEJB3Test. (This is the class that all of the relevant RHQ integration tests extend. Feel free to take a look.)
  • Just write a single, big test, that spans all of the tests you were thinking of splitting up and sharing data between.
    This is probably the easiest route. In this case I suggest writing the tests separately, as private methods, as if they were real tests. And then creating a single consolidatedTest that calls them each in order.
Know that BeforeMethod and AfterMethod are actually invoked twice per test.
Arquillian has two modes of testing, client and in-container. Client testing is such that the tests are basically remote clients and hit the deployed server in that fashion. We use the other mode, which is to run the tests in the container, meaning we can inject InitialContext, EntityManager, etc.. into the tests. Due to architectural integration issues, Arquillian calls BeforeMethod twice, once for each mode. We need to execute our code only for the in-container invocation.
The preferred mechanism for a BeforeMethod or AfterMethod is now to extend the base class methods offered in AbstractEJB3Test. So:

// NOT recommended
public void myBeforeMethodName();

// Recommended
// By overriding the method you'll get built in in-container invocation only, as well as standard error handling.  Note that there are a few flavors
// of the method, some taking parameters.
// IMPORTANT: use protected, not public.  Public methods are considered test methods by testng if the class is annotated with @Test.
protected void beforeMethod();

// If you must use the annotation ensure you protect the invocation
@BeforeMethod( some attributes you just have to have )
public void myBeforeMethodName() {
  if ( !inContainer()) {

Know that false positives result if exceptions are thrown in BeforeMethod

Warning! Currently if an exception is thrown from your BeforeMethod it will be swallowed and the test will appear to pass. I'm not sure if there is a way to get the test to fail, currently I am wrapping the logic in a try/catch and printing blatant warnings in the output.

Know that there is an extension for running several test classes against a single deployment
By default Arquillian with testng will for each class in the test suite:
  1. Start container
  2. Deploy test deployment
  3. Undeploy test deployment
  4. Shutdown container
Although good for test exclusivity, it is prohibitively slow for a project with many test classes that doesn't need that level of test isolation. Working with the Arquillian developers we've put in place an Arquillian extension that allows us a single container start and deploy. Our extension has only nominal changes applied to the extension provided at
It should be noted that this extension uses Arquillian's (not testng) BeforeClass/AfterClass hooks, so those become unavailable for other use, but that is not an issue for us as we don't have any other extensions and observing those (or any other) events.
Hope this helps.

Tuesday, September 18, 2012

Marking target directories derived for Eclipse Maven projects

Just a quick one.  It drives me crazy when I do a search in our maven-based Eclipse project and it includes all the class files etc found in target directories.  This is one way around it.

There may be better ways to do this because for one, Monkey scripting for Eclipse seems to be archived.  And two, I can see that Eclipse has a variety of change requests for supporting this natively.  But although I can see the support for derived files/folders in Eclipse, I couldn't see a built-in way for marking Maven target directories as derived in some automated way.  And Resource filtering was too aggressive as it affected the project building.

On the RHQ Project we had an older script to do this, written in Javascript and using an old Monkey script plugin.  This is an adapted script for a newer (although maybe still old) plugin called GroovyMonkey.  You can install it into Eclipse from here: 

Use the plugin to create a new script.  It will have given you a new Groovy Monkey entry on the main menu and will add a new GroovyMonkyScripts project for you.  Make the name (you'll need to check the 'override' box):

Here is the script:

 * Menu: Make Targets Derived > Python
 * Script-Path: /GroovyMonkeyScripts/monkey/
 * Kudos: jshaughn
 * License: EPL 1.0
 * LANG: Python
 * Job: UIJob

# Install GroovyMonkey from:

files = resources.filesMatching(".*/pom\\.xml")
for file in files:
  targetFolder = file.eclipseObject.parent.findMember("target")
  if targetFolder != None:

files = resources.filesMatching(".*/dev-container/jbossas/bin/run\\.sh")
for file in files:

window.getActivePage().showView( 'org.eclipse.ui.views.TaskList' )


Only the first for loop is working on the target folders, the second pass is just for our own benefit but shows that you can add as many different file pattern sections as you like.

Hope this helps..

Friday, July 20, 2012

Availability Duration Alerting in RHQ/JBossON

Red Hat has recently released version 3.1 of JBoss ON (JON), built on the upstream RHQ Project. RHQ version 4.4 made significant improvements to Availability handling.  Availability is a critical management component, basically monitoring and reporting on whether your resources are running (UP) or not (DOWN).  For an overview of the availability changes in RHQ 4.4, both performance enhancements and new features, see the RHQ wiki.   You can also check out the blog by Mazz that talks in some more detail about some of the changes.

Here I just want to bring to your attention the cool new feature of Availability Duration alerting.  Prior to this release RHQ had the ability to alert only on availability state changes.  Specifically,  "Goes DOWN" and "Goes UP".  This was very rigid because it means you'll generate alerts on brief periods of down time, even if that downtime was anticipated.

In many real-world use cases you'd prefer to be alerted only if an resource goes down and "Stays DOWN" for a certain length of time.  This is what Availability Duration alerting is all about.

The following demo, about 5 minutes long, walks you through the new feature with a simple example:

If you were watching closely you may have noticed a few other things, like the quick availability reporting after lifecycle events.  That is just one of many improvements for making availability reporting more timely and therefore more useful.

We're always interested in feedback, let us know what you think by commenting here, on the rhq-users mailing list, or #rhq on freenode!  Thanks!

Saturday, January 28, 2012

Jboss ON and Reactive Alert Handling

Red Hat has recently released version 3.0 of JBoss ON, built on the upstream RHQ Project. There are several cool features in 3.0, but more than that, the feature set integrates more than ever before. This example shows how several subsystems can come together to handle some high level tasks.

The following demo features reactive alerting, meaning, the ability to automatically take corrective action in response to an alert fired in your environment. It combines the new Drift Management feature, for detecting configuration drift, Alerting to notify that drift has been detected, JON's CLI Scripting for reacting to the alert, and Bundle provisioning for correcting the situation. Also, the new JBoss ON GUI, written in SmartGWT, is on display.

You can watch the demo here, and then take a look at some of the details below.

RHQ Demo: File remediation using alert CLI notification, drift detection and bundles

The demo showed how you can monitor your system for file drift, and alert on that drift when detected. And then how that alert can in turn invoke RHQ CLI scripts to take action. The RHQ CLI is powerful, it offers a robust API and many built-in features, in a Rhino Javascript environment. In the example we use the following script, which calls into the RHQ server and leverages the remote API for bundle provisioning. Here is the script for reference:
// This script performs some mock remediation on Drift detection by performing
// a clean bundle deploy of the corrupted file.

// note, the 'alert' variable is seeded by the alert sender

// Log what we're doing to a file tied to the fired alert id
var e = exporter
e.setTarget( 'raw', 'c:/temp/alert-cli-demo/logs/alert-' + + '.log' )

// Dump the alert
e.write( alert )

// get a proxy for the alerted-on Resource
var alertResource = ProxyFactory.getResource(

// Dump the resource
e.write( " " )
e.write( alertResource )

// Remediate file

// Find the Bundle Destination
var destCrit = new BundleDestinationCriteria()
destCrit.addFilterName('Alert CLI Demo')
var result = BundleManager.findBundleDestinationsByCriteria( destCrit )
var dest = result.get( 0 )

// Find the Bundle Version
var versionCrit = new BundleVersionCriteria()
result = BundleManager.findBundleVersionsByCriteria( versionCrit )
var ver = result.get( 0 )

// Create a new Deployment for the bundle version and the destination
var deployment = BundleManager.createBundleDeployment(ver.getId(), dest.getId(), 'remediate drift', new Configuration())

// Schedule a clean deploy of the deployment. This will wipe out the edited file and lay down a clean copy
BundleManager.scheduleBundleDeployment(deployment.getId(), true)

e.write( " " )

The demo neglects to show the resulting log file. The log file gives you an idea of what the alert variable data looks like. The alert data, which is provided to the script automatically, allows the script writer to access lots of other information he may need. In this example, the resource info. Here is the log file generated by the demo:
acknowledgeTime: -1
alertDefinition: org.rhq.core.domain.alert.AlertDefinition[ id=10001, name=Drift!, conditionExpression=ANY, priority=Medium, resourceId=10001 ]
alertNotificationLogs: []
conditionLogs: [org.rhq.core.domain.alert.AlertConditionLog[ id=10191, value=Drift Detected, extraInfo=[[Fri Jan 27 17:19:53 EST 2012] Drift detected in [1] files using drift definition [Alert CLI Demo]], org.rhq.core.domain.alert.AlertCondition[ id=10041, category=Drift Detected, name=Alert CLI Demo, comparator='null', threshold=null, option=null ] ]]
ctime: 1327702795100
id: 10181
recoveryId: 0
willRecover: false

OSName: Win32
OSVersion: 6.0
architecture: x86
contentTypes: {InstalledSoftware=Installed Software}
createdDate: Mon Jan 23 14:11:38 EST 2012
description: Microsoft Windows Operating System
freeMemory: 805.0MB
freeSwapSpace: 3.9GB
hostname: jshaughnessy-PC
id: 10001
idle: 0.0%
measurements: [Wait Load, Used Memory, System Load, Total Memory, OS Name, Free Memory, Hostname, Architecture, Idle, Total Swap Space, Used Swap Space, User Load, OS Version, Free Swap Space]
modifiedDate: Mon Jan 23 14:11:38 EST 2012
name: jshaughn
operations: [viewProcessList, manualAutodiscovery]
pluginConfigurationDefinition: ConfigurationDefinition[id=10136, name=Windows]
resourceType: Windows
systemLoad: 0.0%
totalMemory: 3.0GB
totalSwapSpace: 7.4GB
usedMemory: 2.2GB
usedSwapSpace: 3.5GB
userLoad: --no data available--
version: Win32 6.0
waitLoad: 0.0%


The above script begins to help you build your own, but JBoss ON (and RHQ) include a more powerful base script for putting together a CLI-based provisioning solution. It can be downloaded in the GUI, via Administration->Downloads.