We’ll want to JNDI-lookup the queue, too…

In our system we have proved out looking up the JMS connection factory from JNDI, but up to this point we have declared the Queue bean using an <amq:queue> element in the application bean files.  This ties our application code to the ActiveMQ broker, which we don’t want.  We want to move toward looking up the connection factory and the queue from JNDI.  That way the application bean file only needs to know the JNDI name, and we can configure the queue in the connection factory datasource configuration file on the app server (e.g., activemq-jms-ds.xml), probably as an mbean element in there:

   <mbean code="org.jboss.resource.deployment.AdminObject" name="activemq.queue:name=TheMBeanNameYouSeeInTheJMXConsole">
      <attribute name="JNDIName">this/is/the/jndi/name/the/bean/files/will/use</attribute>
      <depends optional-attribute-name="RARName">jboss.jca:service=RARDeployment,name='activemq-rar-5.1.0.rar'</depends>
      <attribute name="Type">javax.jms.Queue</attribute>
      <attribute name="Properties">PhysicalName=theQueueName.YouWillSee.In.ActiveMQ</attribute>

But Who Will Care for the Babies?

This will be fine for a production environment — but what about our integration tests?  Right now we use Bitronix Transaction Manager 1.3’s JNDI server facilities to provide JNDI lookup of the JDBC datasource for our integration tests.  I thought we’d also use BTM to look up the JMS connection factory… but BTM does not support JNDI-looking-up queues as well.  Ludovic says:

If you really need a complete JNDI server, you can use Tomcat’s which is quite easy to reuse. This is what I recommended using before BTM 1.3, there is an example here: http://docs.codehaus.org/display/BTM/Hibernate

That’s probably what we’ll need to look into when the time comes…

It got overwhelmed and stopped listening

We’re trying to set up to test the relative performance of XA transactions vs. local transactions on our technology stack.  We have a producer and a consumer .war running on separate servers; an Oracle server; and an ActiveMQ server.  We’re running a LoadTest from soapUI, barraging the producer with web service calls telling it to send a message to the queue — and then the consumer gets the message and writes a record to the database.

The problem was, after only a few minutes, the producer and consumer would just seem to hang — the producer would not accept any more messages from soapUI, and the consumer would not receive any more messages from the queue (even though there were plenty there for it to receive!)  From that point on, the producer would place about one message in the queue per minute, but the consumer would never pick up any more messages.  Huh!

A search turned up a suggestion to check the producer flow control setting.

We changed our conf/activemq.xml to add producerFlowControl=”false”:

<policyEntry queue=">" producerFlowControl="false" ...

This time both producer and consumer kept going.

How to locally add a repository in Maven

I’m experiencing an issue where if I have my producer send 100 JMS messages to my ActiveMQ queue while a consumer is listening, the listener will pick up maybe 90 of them and then no more.  This is not good!  I need to find out why this is.

I’m running ActiveMQ 5.1.0 at the moment, but one of the suggestions I found was to upgrade to the 5.2.0 Release Candidate.  I updated my pom to depend on that version, but since this artifact is not in the main Maven repository yet, Maven Eclipse couldn’t find it and I couldn’t build my project.

I had run into this type of thing before when I was trying out a Bitronix Transaction Manager 1.3 release candidate.  What I needed was to add this to my pom:



That worked — after running what we call “Maven Eclipse” (mvn eclipse:clean eclipse:eclipse), now activemq-core-5.2.0.jar shows up in my Referenced Libraries!

The mysteriously necessary sessionTransacted=true

I’ve encountered this before, but just ran into it again: Despite suggestions to the contrary, it is necessary (at least with Spring 2.5.5, ActiveMQ 5.1.0, and JBoss 4.2.2.GA) to set sessionTransacted=true in the DefaultMessageListenerContainer even when it’s set up to connect to an external JTA transaction manager. If I set sessionTransacted=true, then when my messageListener throws a RuntimeException, the message is rolled back onto the queue; if I don’t, it doesn’t.

This has been documented elsewhere regarding Atomikos, but not ActiveMQ. I’ve posted a question to the Spring Remoting and JMS forum. We’ll see what comes of it!

The jms:listener element: hardcoded queue name

I thought I’d try using Spring’s jms: namespace to define my DefaultMessageListener container.  So I changed this:

<bean id=”jmsContainer” class=”org.springframework.jms.listener.DefaultMessageListenerContainer”>
<property name=”connectionFactory” ref=”queueConnectionFactory” />
<property name=”destination” ref=”queue” />
<property name=”messageListener” ref=”messageListener” />
<property name=”transactionManager” ref=”transactionManager” />

to this:

<jms:listener-container connection-factory=”queueConnectionFactory” transaction-manager=”transactionManager”>

<jms:listener destination=”queue” ref=”messageListener” />


(the queue bean was defined this way:

<amq:queue id=”queue” physicalName=”gloriousTest.Queue” />

But now when I deployed my .war project, ActiveMQ had a queue named “queue”, and no consumers were listening to the gloriousTest.Queue queue.

As it turns out, the jms:listener element’s destination attribute refers to a physical queue name — it’s not a bean reference.  So I guess if I were going to use the jms:listener element I would need to hardcode gloriousTest.Queue there.  I’m not ready to do that at this point, so I think I’m going to revert back to the plain old <bean> element usage for my DefaultMessageListenerContainer.

Something works!

After days of untested theories, head-scratching, dead ends, and rabbit trails, I’m happy to report that something now works.

The Beginning of Unstuckitude

First thing was, Keith got back, took a look at the class cast errors in the stack trace and said something along the lines of, “That looks like a dependency issue.”

And that is what it was: my .war project was including javax.jms.ConnectionFactory more than once. (There is also a Spring forum question and answer for this problem.)

Step 1: activemq-all to activemq-core

We browsed through the .jar files in the activemq-all .jar file (manually, by opening each enclosing .jar in WinZip — anyone have a better way of doing this?) and indeed it did have javax.jms.* classes in it.  Keith knew that having the same classes multiple times in your .war can cause problems as the app server may load the wrong ones, so in my project’s pom.xml we replaced the dependency on activemq-all with a dependency on activemq-core instead:


When we tried to do a Maven Install of this, though, we got:

1) org.apache.activemq:activeio-core:jar:3.1-SNAPSHOT
2) org.apache.activemq:activeio-core:test-jar:tests:3.1-SNAPSHOT
2 required artifacts are missing.

Step 2: Exclude activeio-core dependency

We don’t need the activeio-core tests anyway, so we put in an exclusion for this:


This time Maven Install ran cleanly.

Step 3: Scope = Provided

When we deployed, though, we got errors like this:

10:12:59,417 WARN  [JBossManagedConnectionPool] Throwable while attempting to get a new connection: null
javax.resource.ResourceException: Could not create connection.
Caused by: javax.jms.JMSException: Could not create Transport. Reason: java.io.IOException: Transport scheme NOT recognized: [tcp]
08:35:36,453 INFO  [DefaultMessageListenerContainer] Could not refresh JMS Connection - retrying in 5000 ms
javax.jms.JMSException: Could not create Transport. Reason: java.io.IOException: Transport scheme NOT recognized: [tcp]

The DefaultMessageListenerContainer, finding itself unable to make a JMS connection, kept trying every five seconds —  boom, bang, bang, boom, zowie!  (I’m still getting acclimated to the spectacular errors you can get when deploying to an app server. :)

At first I didn’t notice one of the Caused Bys farther down, which said:

Caused by: java.lang.ClassCastException: org.apache.activemq.transport.tcp.TcpTransportFactory

But sure enough, it turns out that this error was caused by double-deployed classes.  This time, it was the activemq-core jar that is already contained inside the activemq-ra.rar file. Our project already depends on that .rar file being there, and we don’t want two copies of the jar, so we updated our pom file’s activemq-core dependency definition by setting scope to provided (line 5), as follows:



This time, our .war file deploys without stack traces (woohoo!),  and when I point my browser to the ActiveMQ Console I see that my “gloriousTest.Queue” queue shows up.  I use the ActiveMQ Console to send a message to the queue and… my MessageListener gets the message and prints its debug output to the JBoss console!

The Files That Worked

applicationContext.xml, the Spring bean file

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context" xmlns:amq="http://activemq.org/config/1.0"
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
        http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd
        http://activemq.org/config/1.0 http://activemq.apache.org/schema/core/activemq-core-5.0.0.xsd
        http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-2.5.xsd">

    <jee:jndi-lookup id="queueConnectionFactory" jndi-name="java:activemq/QueueConnectionFactory" />

    <amq:queue id="queue" physicalName="gloriousTest.Queue" />

    <bean id="messageListener" class="com.something.dmn.MessageListener" />

    <bean id="transactionManager" class="org.springframework.transaction.jta.JtaTransactionManager" />

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="queueConnectionFactory" />
<property name="destination" ref="queue" />
<property name="messageListener" ref="messageListener" />
<property name="transactionManager" ref="transactionManager" />

pom.xml (relevant portions)




This file I was able to use without modification from the example on the ActiveMQ site.  I noticed, though, that the <ServerUrl> elements under connection-factories -> tx-connection-factory seem to be ignored.  (Maybe that is because in our case the ServerUrl is already being pre-specified in the activemq-ra.rar’s ra.xml file.)

ra.xml (in activemq-ra.rar)

I think the only change I had to make to this resource adapter archive was to the connector -> resourceadapter -> config-property-value element for the ServerUrl config-property-name, to change it from vm://localhost to tcp://localhost:61616.  (Or maybe it was already tcp… I don’t remember now.)


Now that we have the JCA/JMS/JNDI/JBoss/Spring/ActiveMQ stuff playing nicely together, we can try to actually do some XA stuff!

The mysterious mirrored message queue

I have a local instance of the ActiveMQ message service running on good ol’ localhost:61616, to try out queue stuff locally.  A few weeks ago, when I had been working on a small queue usage proof-of-concept project which would be committed to the research area of the source repository, one of the domain guys asked about an “OurQueue” queue that had shown up on the prototype’s ActiveMQ server.  He asked if I knew anything about it or if it was needed.  That was the name of the queue I was using on my local test project — I was surprised to see it show up there on the prototype’s server, but I figured I had been late in reconfiguring one of the tests to point to my local server instead of the one on the network.  I told him it was fine for him to get rid of the queue, and went on.

But then today I was starting up ActiveMQ locally and saw these log messages:

INFO  DiscoveryNetworkConnector      – Establishing network connection between from vm://localhost to tcp://OTHERSERVER:61616
INFO  TransportConnector             – Connector vm://localhost Started
INFO  DemandForwardingBridge         – Network connection between vm://localhost#0 and tcp://OTHERSERVER/ has been established.

(server name and IP address not the actual ones)

Hey, why is my local ActiveMQ talking to the remote ActiveMQ server?

To the Bridge!

In short, under broker -> networkConnectors, there is a networkConnector element to this effect:

<!-- by default just auto discover the other brokers -->
<amq:networkConnector name="default-nc" uri="multicast://default"/>

If I comment out the networkConnector element, I no longer get the “Establishing network connection” message, nor the DemandForwardingBridge message.  Now I think I should be able to test locally without my silly-named queues showing up on the remote server.  :)

Trying out XA, part 5: a resource adapter rabbit trail

In Part 4, we ended up stuck and wanting to post a question somewhere.  As often happens, as part of pulling together data to ask a question, I do more research…

Part of a thread from the ActiveMQ Users list says:

> Are the connection factories/queues/topics just exposed by the RAR for use
> within the J2EE server only.

Yes – as they are not using the regular JMS client API – they are
using J2EE RA stuff

You’ll need to setup a regular JMS connection and any destination
objects you want in JNDI somewhere. Using JBoss’s JNDI provider could
well work fine though – just make sure its actually normal JMS client
objects (ConnectionFactory & destinations) – and not RA stuff.  Also
make sure the J2EE server and your stand alone client are using the
exact same ActiveMQ jars since its gonna be using Java serialization
to transmit the Java objects in JNDI

(Emphasis mine.)  Ok, so looks like the classes I’m JNDI-lookuping aren’t intended for outside-the-app-server use.  But there is a way… the rest of the thread discusses an alternative, but I’m not sure that it would work with XA transactions.

But……… ActiveMQ’s JNDI Support page shows how to use a jndi.properties file to specify, among other things, a connection factory.  Let’s try that tomorrow!

Update 7/23/2008: See the Something works! post for what was actually the cause of the ClassCastExceptions…

Too many rollbacks

In my testing of JMS messaging using regular JMS transactions, I noticed that if the same message was received but then rolled back seven times in a row, it would disappear out of the message queue. “Odd,” I thought. So I looked in the ActiveMQ API docs for the Session interface (and its implementors)’ rollback method and in the activemq.xml configuration file, but did not see a setting there for this.

Next I searched the activemq forums for postings about rollbacks and found this message, which points you to an activemq.dtd file. The link is dead, and it appears that the .dtd file has been replaced with an .xsd, activemq.xsd, which I found in the activemq-all-5.1.0.jar file. I pored through this schema until I noticed a “redeliveryPolicy” element with a “maximumRedeliveries” attribute. I bet that’s how you configure this.

Somewhere along in here, I thought of trying to go to the link http://activemq.apache.org/schema/core. (Probably because I saw this string in the activemq.xml configuration file.) I found the redeliveryPolicy element in the HTML view of the xsd file.

So, it appears that the setting is configurable and that the hierarchy to get to the redeliveryPolicy element is broker -> regionBroker -> commandAgent -> connectionFactory -> redeliveryPolicy… but I can’t figure out how to get the regionBroker element in the XML. There’s already a <broker> element in the activemq.xml configuration file, and I tried adding a <regionBroker> under that, but ActiveMQ choked when I tried to start it up. (Also Eclipse’s auto-complete (Ctrl+Space) doesn’t show regionBroker as one of the valid element types I can add there… but come to think of it, Eclipse’s auto-complete in this case is only showing as available the elements that already exist in activemq.xml, so maybe I can’t trust it to show me everything that’s available in this case.)

The schema doc says that the the type of regionBroker is:

loggingBrokerPlugin | multicastTraceBrokerPlugin | timeStampingBrokerPlugin | udpTraceBrokerPlugin


Update: Huh – they’re not being discarded at all… there’s an ActiveMQ.DLQ (Dead Letter Queue, get it? :) queue that has 16 messages in it.  When I thought the messages were being discarded, they were being put in there.

Update July 8, 2008: I found the ActiveMQ documentation for the Redelivery Policy.