Automation-assisted manual transformations

I had been unit testing an SQL generator and had a bunch of tests that gave various input to the generator and tested its output against expected SQL.   The SQL in my tests looked like this:

Now I was ready to feed the SQL to a database engine to verify that it was validly formed.  I would generally use grep for this type of task; but here my SQL statements were formatted multiline for easier reading, and grep operates in a line-by-line mode.  There were over 100 test cases, so it was worth figuring out an automated solution.  I also wanted to avoid writing a single-purpose text-processing utility if possible.

I ended up writing down the following steps for myself:

  1. Turn the tests into a single line of text for ease of working with tools
    On command line: cat Test*.cpp | tr -d "\n\r" > all-one-line.out
  2. Discard everything but the queries, inserting a newline after each
    In editor (SlickEdit for me), open all-one-line.out and Replace All (using Perl-style regexes):
    .*?("(?:\( )*SELECT[^;]+?;)
  3. Clean up what the regex didn’t
    Delete the last line
  4. Get rid of quotes
    Replace \" with nothing
  5. Get rid of semicolons
    Replace ;$ with nothing
  6. Get rid of extra spaces
    Replace <space>+ with <space>
  7. Save the file in the editor
  8. Get rid of Oracle-specific tests
    grep --invert-match TO_DATE < all-one-line.out > all-one-line.sql
  9. Let cool for 5 minutes before serving
    Paste all-one-line.sql into MS SQL Server Management Studio and execute (with Results to Text)

This may look like a large number of steps, but I got to where I could run through them in about 30 seconds and test all 130  queries on the server.  Nice!

Future improvements

Once I had the ability to test my test output against the database server, I wanted to do that each time the tests’ expected results changed.  So where I had originally envisioned a single smoke test run, I ended up going through these automation-assisted manual steps ten or twenty times.  In retrospect, the single-purpose utility script would clearly have been the better approach after all.  I need to get more comfortable whipping up such scripts to lower the barrier to writing them when these occasions arise.

Twelve years in, I would think I would be at the top of my craft by now, but there are still things that seem pretty basic that I’m learning.  Hmmm…I wonder if life really does begin at 40?

Automated refactoring for C++

During the year that I  was on a Java project, I found Eclipse’s automated refactoring tools such a productivity boost.  I remember doing probably my first automated Extract Method and seeing it automatically deduce which parameters would need to be passed, and saying “No way.  No way!” And then you could rename methods (and it would update the dependencies); you could extract a local variable; you could do a quick fix… I would find myself giggling and sometimes laughing out loud – there were all these mundane, tedious and error-prone manual manipulations of source code that I was used to doing, being careful and manually checking everywhere to try not to miss something – and now that I could use the Refactor menu my mind was freed to think about the problem I needed to solve.  Wow!

Automated refactoring was such a productivity and energy boost.  I don’t want to go back to the way it was before.  So, now that I’m in C++,  I’m interested in finding an automated refactoring tool for C++ – preferably one that integrates with Visual Studio.

What’s Out There?

For C++ development, a quick Google search turned up devexpress‘s Refactor! for C++ and Whole Tomato Software‘s Visual Assist product, both of which integrate with Microsoft Visual Studio.  Refactor! for C++ is a free download, whereas Visual Assist costs $249 with $49 maintenance renewals (or $99 for a license that is not eligible for the renewals).

This post on refers to SlickEdit as an alternative.  Ah yes, apparently SlickEdit can do refactorings in C++ .  And it appears that some people around here already use it. Hmm!

For Tonight’s Program, the Screenshots Part Will Be Played By…

Here is the part where I would take a bunch of screen shots demonstrating the use of Extract Method… but Whole Tomato has a nice post about that on codeproject already.  Helpful to see the tool in action.

Out With a Whimper

And to add to the letdown of me not having my own screenshots… I haven’t come to a conclusion yet either.  I have SlickEdit, the trial version of Visual Assist X, and Refactor! for C++ installed.  Guess we’ll try ’em and see!

(Const) References

Refactoring Tools Shoot Out by Legalize Adulthood

All developers are equal, but some are more equal… (Maciek Talaska)

A couple of Stack Overflow exchanges pointing to Refactor and VAX

SlickEdit – Programmer’s Editor (

Are We There Yet? SlickEdit’s C++ Refactoring