The SEVERE but ignorable error

There’s a certain error-and-stack-trace we get every time we bring up an instance of JBoss (or is it every time we deploy one of our war projects?), to wit:

09:15:37,945 ERROR [STDERR] Oct 16, 2008 9:15:37 AM oracle.jdbc.driver.OracleDriver registerMBeans
SEVERE: Error while registering Oracle JDBC Diagnosability MBean.
javax.management.MalformedObjectNameException: Invalid character '
' in value part of property
        at javax.management.ObjectName.construct(ObjectName.java:529)
        at javax.management.ObjectName.(ObjectName.java:1314)
        at oracle.jdbc.driver.OracleDriver.registerMBeans(OracleDriver.java:303)
        at oracle.jdbc.driver.OracleDriver$1.run(OracleDriver.java:213)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.jdbc.driver.OracleDriver.(OracleDriver.java:209)
        at oracle.jdbc.pool.OracleDataSource.(OracleDataSource.java:94)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
        at java.lang.Class.newInstance0(Class.java:350)
        at java.lang.Class.newInstance(Class.java:303)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.getXADataSource(XAManagedConnectionFactory.java:219)
        at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:136)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:577)
        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:262)
        at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:500)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:341)
        at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:315)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:396)
        at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:842)
        at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:88)
        at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:82)
        at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:84)
        at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2073)
        at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1298)
        at org.hibernate.cfg.AnnotationConfiguration.buildSessionFactory(AnnotationConfiguration.java:859)
        at org.springframework.orm.hibernate3.LocalSessionFactoryBean.newSessionFactory(LocalSessionFactoryBean.java:814)
        at org.springframework.orm.hibernate3.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:732)
        at org.springframework.orm.hibernate3.AbstractSessionFactoryBean.afterPropertiesSet(AbstractSessionFactoryBean.java:211)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1368)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1334)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:473)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:409)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:380)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:264)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:221)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:261)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:185)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:423)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:729)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:381)
        at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139)
        at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:105)
        [...]

We’re used to this, and it doesn’t seem to cause any harm, so we ignore it, but wish it weren’t there.

A post to the SQLJ/JDBC forum at Oracle Technology Network Discussion Forums brings up this same issue, and the reply says that a patch to ojdbc5.jar has been released, dealing with this issue.  My results show we’re not using the patched version:

C:\path\to\jboss-4.2.2.GA\server\default\lib>java -jar ojdbc5-11.1.0.6.0.jar
Oracle 11.1.0.6.0-Production JDBC 3.0 compiled with JDK5

(If we had the patch, the output would be

Oracle 11.1.0.6.0-Production+ JDBC 3.0 compiled with JDK5

).

The drivers are available from Oracle.  Perhaps soon we’ll upgrade to them…

Oracle Treajure hunt

I wanted to find the Javadoc for Oracle’s JDBC datasource classes, to see what XA interfaces they implement (…to form a more focused question to ask on the Bitronix Transaction Manager forum).  I’d looked for the Javadoc before there and I thought I’d found some, though it was hard to find.  I decided to search again and this time, bookmark it.

(Usually one would just post the solution, but I’m cataloging what my process was too.  Maybe writing down what I did will allow us to analyze my process and improve on it next time.)

Buried

After some ask.com, Google, and dogpile.com searches didn’t seem to turn up anything (well, I did find something on someone else’s personal site, but I wanted my bookmark to point to Oracle) I decided to see if I could find what I was looking for by browsing Oracle’s site.  From the main page there were Products and Services, Industries, Support, Partners, Communities, and About across the top; Database, Middleware, Applications, and Industries in the middle, and Featured Downloads, Events and News, and Resources down the right hand side.

I picked Support, but didn’t see Java stuff there, so I went back and picked Communities.  The Oracle Technology Network looked promising (it turns out I could have gotten to this directly from the main page using a link under Resources there), and from there I went to Java under Technologies on the left hand sidebar, and then to JDBC under Focus Areas.

At this point, though, I couldn’t see a Javadoc link.  Dead end.

XA Marks the Spot

I paused and thought about how I would know what I was looking for.  I remembered that Javadoc tends to have a standard look to it, with standard links.  I had the idea to go to some other Javadoc I had up and use the link text from those standard links to use as part of my search:

OracleXADataSource Overview Package Class Use Tree Deprecated Index Help site:oracle.com

This time, dogpile.com came through for me, finding the Oracle JDBC Javadoc!

XA, JNDI and Bitronix, part 2: JDBC

In our last episode, we worked to get the JMS connection factory able to be looked up via JNDI using the Bitronix Transaction Manager… and after fixing several things, God helping us, we succeeded.  Today we want to re-enable the rest of the Spring beans in our project (Hibernate beans, default message listener container, etc.) and see them all working with Bitronix.

Updating to BTM 1.3

Over the weekend, Bitronix Transaction Manager version 1.3 final was released.  I wanted to update to that from version 1.3-RC2, so I updated the <version> tag of the btm artifact in my project’s pom file and removed the <repository> section that had been pointing to the codehaus-snapshots repository (the full 1.3 release is now on the regular Maven 2 central repository, so I no longer need to reference the early releases repository).

I re-ran my integration test and it still does the JNDI lookup fine.  That was easy!

Bringing In the Beans

Our goal is to not have to modify production Spring bean files for integration testing… but this project will help us determine how to set up the production bean files to make this possible.  So this time around, I’m going to have to modify my mock-production bean files to work in the non-app-server environment

First, let’s get rid of the JTA transaction manager bean ( <tx:jta-transaction-manager /> ) from our main bean file.  The definition of the transactionManager bean needs to be different in a production environment than with Bitronix, so that bean doesn’t belong in a production beans file.  Along with this, we need to rename our transaction manager bean from JtaTransactionManager to transactionManager, since that’s what the jta-transaction-manager bean provided and the Hibernate sessionFactory is expecting.

We also need to remove the “java:” from the jndi-name our queueConnectionFactory expects to look up: <jee:jndi-lookup id="queueConnectionFactory" jndi-name="java:activemq/QueueConnectionFactory" />

…and similar with the Oracle datasource in spring-beans-hibernate-SimplePersonDAO.xml:     <jee:jndi-lookup id="myDataSource" jndi-name="java:XAOracleDS" />

Can’t Find the Database Datasource

Now when we run our integration test, Spring complains that nothing named XAOracleDS is available from JNDI.  Ah, yes — we need to give Bitronix the details of what’s needed so it can make a datasource available under that name.

So, combining step 2 of the Spring procedure with some hints from the JDBC pools setup documentation, I cobbled together this bean:

    <bean id="oracleDataSource" class="bitronix.tm.resource.jdbc.PoolingDataSource" init-method="init" destroy-method="close">
<property name="className" value="oracle.jdbc.xa.client.OracleXADataSource" />
<property name="uniqueName" value="XAOracleDS" />
<property name="maxPoolSize" value="5" />
<property name="driverProperties">
<props>
<prop key="user">myUser</prop>
<prop key="password">myPassword</prop>
<prop key="URL">jdbc:oracle:thin:@localhost:1521:XE</prop>
            </props>
        </property>
    </bean>

I don’t expect this to work right off, because I don’t think I have the Oracle XE library listed in my pom file.  But — vaguely TDD-style — let’s try it first and verify that we do get an error.  Sure enough:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘oracleDataSource’ defined in class path resource [spring-beans-bitronix.xml]: Invocation of init method failed; nested exception is bitronix.tm.resource.ResourceConfigurationException: cannot create JDBC datasource named XAOracleDS

And way down toward the end of the stack trace is this:

Caused by: java.lang.ClassNotFoundException: oracle.jdbc.xa.client.OracleXADataSource

Let’s add the Oracle jdbc driver dependency to our pom file:

        <dependency>
            <groupId>com.oracle</groupId>
            <artifactId>ojdbc5</artifactId>
            <version>11.1.0.6.0</version>
        </dependency>

Bitronix Transaction Manager Refuses to Start?

After Maven Eclipsing to update my project with the new dependency, I now get this:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘BitronixTransactionManager’ defined in class path resource [spring-beans-bitronix.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public static synchronized bitronix.tm.BitronixTransactionManager bitronix.tm.TransactionManagerServices.getTransactionManager()] threw exception; nested exception is bitronix.tm.utils.InitializationException: recovery failed, cannot safely start the transaction manager

Why can BTM not safely start?  Is it trying to force me to come along with its view of how much risk is acceptable in a system’s configuration?  …But wait, let’s look at some of those Caused-bys farther down in the stack trace:

Caused by: bitronix.tm.recovery.RecoveryException: error running recovery on resource XAOracleDS

And farther down:

Caused by: java.sql.SQLException: Io exception: The Network Adapter could not establish the connection

Aha — I forgot to fire up my local Oracle XE instance.  Oops!

Thank You for Watching the Caused-By Show

(Remember The Cosby Show? :)

After starting up Oracle and re-running my test… I get the same error again, but — now I’m getting some XA verbiage showing up in the Caused-Bys:

Caused by: bitronix.tm.recovery.RecoveryException: error running recovery on resource XAOracleDS (XAER_RMERR)

and

Caused by: javax.transaction.xa.XAException

I wonder if that means we’re getting close?

After messing around a while, I searched the BTM forum and found that the Oracle user under which I’m performing the integration test needs to have certain additional database privileges:

grant select on sys.dba_pending_transactions to myUser;
grant select on sys.pending_trans$ to myUser;
grant select on sys.dba_2pc_pending to myUser;
grant execute on sys.dbms_system to myUser;

[First, drop to Oracle SQL command line and do connect / as sysdba]

Hey, now the integration test passes!  (Granted, it’s only instantiating all the beans so far, but that means the Oracle JDBC JNDI lookup must have succeeded…!)

Guess allowLocalTransactions needs to stay

Following one of the examples, I had put the following two settings in my Oracle data source bean definition:

<property name="allowLocalTransactions" value="true" />
<property name="testQuery" value="SELECT 1 FROM DUAL" />

Do we really need these settings?  Let’s try removing them.

We get an error:

java.sql.SQLException: error enlisting a JdbcConnectionHandle of a JdbcPooledConnection from datasource XAOracleDS in state ACCESSIBLE wrapping oracle.jdbc.driver.T4CXAConnection@948069 on oracle.jdbc.driver.LogicalConnection@17a7706

…with this Caused-By:

Caused by: bitronix.tm.internal.BitronixSystemException: resource ‘XAOracleDS’ cannot be used outside XA transaction scope. Set allowLocalTransactions to true if you want to allow this and you know your resource supports this.

Removing the testQuery property setting doesn’t seem to have any similarly bad effect, but maybe I’ll just leave both of these settings in for the moment.

Where We Are

We now have all our beans instantiated, including the big three (JDBC datasource, JMS connection factory, and XA transaction manager), and the only changes we had to make to the “production” bean files were to remove the java: namespace from the jndi-names and get rid of the jta-transaction-manager bean declaration.

In our next installment, we’ll try sending a message and see what happens.

Getting the Oracle XA datasource in place

Last time, we seemed to see a successful rollback of one resource (the JMS message receive).  Today we want to try to get the JDBC datasource to participate in the transaction, to set up for demonstrating a rollback during a full two-phase commit.

---

I had an Oracle datasource configuration file sitting around waiting for when I was ready to test the Hibernate portion of my XA transaction, but yesterday I took a look at it and it looks like a non-XA datasource.  For one thing, the bulk of the content is within a  <local-tx-datasource> element.  For another, there is this comment near the top:

<!--| This datasource connects to oracle.  It is a LOCAL TRANSACTION data source (as opposed to
| an XA transaction datasource
|-->

I’ll need an XA datasource, though, not a local one…

An example to work from

I found Section 8.3.2. Installing the JDBC Driver and Deploying the DataSource in JBoss’s Getting Started with JBoss 4.0 guide, which has an example Oracle XA datasource.  I started with that example and made the following changes:

• For the URL, I used jdbc:oracle:thin:@localhost:1521:XE
• Different username/password — I followed the instructions (also in Section 8.3.2.) for creating a user via the Oracle SQL command line
• Commented out the mbean element

(When I first deployed this datasource, I got a stacktrace containing the following error:

Caused by: java.lang.ClassNotFoundException: No ClassLoaders found for: org.jboss.resource.adapter.jdbc.xa.oracle.OracleXAExceptionFormatter

)

[Update 8/21/2008: I found instructions for fixing the missing ExceptionFormatter class so that this mbean no longer needs to be commented out: the class name in the code attribute needed to be corrected from org.jboss.resource.adapter.jdbc.xa.oracle.OracleXAExceptionFormatter to org.jboss.resource.adapter.jdbc.vendor.OracleXAExceptionFormatter.]

Now when I deploy I see:

09:29:09,112 INFO  [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=XAOracleDS' to JNDI name 'java:
XAOracleDS'

Added JNDI lookup to Spring bean file

I went to the Spring beans file for the Hibernate stuff and put this JNDI lookup there:

<jee:jndi-lookup id="myDataSource" jndi-name="java:XAOracleDS" />

So, now that will be there when I’m ready to try it out.

Update: Oracle JDBC driver

I was looking through some notes I had and another part of the setup, which I had done previously (besides installing Oracle Database 10g Express Edition), was to go in our Maven repository to the following location:

• group id:  com.oracle
• artifact id: ojdbc5
• version: 11.1.0.6.0

and put the ojdbc5-11.1.0.6.0.jar from there in JBoss’s server\default\lib directory.

Next time we’ll configure the Hibernate session factory…