# Posts Tagged ‘utilities’

## Deleting all that Subversion metadata

Posted by danielmeyer on January 8, 2009

Someone just sent me a Java project, which I need to import into Eclipse and take a look at.  (The changes aren’t ready to commit to Subversion yet.)

The project came to me with a target directory, which is easy to delete:

    rm -r target

but it also came with the Subversion metadata — an .svn folder under each regular folder.  The line I used to get rid of all these:

    find . -name *.svn* | xargs rm -r

Yeah xargs!

Posted in Technical Stuff | Tagged: , | 1 Comment »

## dsmbackup2007

Posted by danielmeyer on December 31, 2008

My Dad has an external hard drive he uses to back up his My Documents folder to.  He was interested in an automated way to do this and then also to delete files off the backup drive that have been deleted off the main drive; so sometime last year (I’m guessing based on the the name I gave the main batch file), I created a couple of batch files for him to do that.

The main batch file is dsmbackup2007.cmd, and its helper is the snazzily-named-but-obscurely-written MaybeDelete.cmd.

Dad was having trouble with the backup program — it was giving errors — and it turned out to be that the MaybeDelete.cmd script had gotten misplaced.  I added better error messages to the dsmbackup2007.cmd script for this case and thought I’d post both scripts here (so I don’t lose ‘em again):

## Scripts

### dsmbackup2007.cmd


@echo off

REM You can change this drive letter as needed.
set BACKUPDRIVE=K
set MAYBEDELETECMD=c:\bin\MaybeDelete.cmd

set COPYFROM=%USERPROFILE%\My Documents\

echo Ready to copy %USER%'s My Documents from %COPYFROM% to %COPYTO% .
pause

REM Copy from the current user's My Documents folder to a folder on the backup drive
REM /S - Copy subdirectories also
REM /E - Copy even empty subdirectories
REM /M - Copy only modified files, and reset the "modified" flag once they're copied
REM /Y - Don't prompt to overwrite files on the backup drive (we want to automatically overwrite them with the newer version).
xcopy "%COPYFROM%*" %COPYTO%  /S /E /M /Y

REM See if there was an error.  If so, display an error message.

:Good
echo Backup appears to have been successful!
goto RemoveStale

echo *****BACKUP FAILED*****
pause

:RemoveStale
REM Delete files that are now gone from the copy-from folder.
echo.
echo Ready to delete files from backup drive that no longer exist on source drive...
pause
if not exist %MAYBEDELETECMD% echo. & echo ERROR: Could not find MaybeDelete.cmd file & echo (expected it to be at %MAYBEDELETECMD%) & echo Please put the MaybeDelete.cmd file in the expected directory. & echo. & goto End
echo (This may take a little while, we're pedaling as fast as we can...)
for /R %COPYTO% %%F in (*) do call %MAYBEDELETECMD% "%%F" "%COPYFROM%"
echo Done.

:End
pause


### MaybeDelete.cmd


REM @echo off
set FullPathToFile=%1
set MainDir=%2

REM Strip quotes off the maindir path.
REM for /F %%S in (%2) do set MainDir=%%~fnxS

REM Takes off the first few parts of the directory path that
REM only apply to the backup drive.
for /F "tokens=4* delims=\" %%c in (%FullPathToFile%) do set RelPathToFile=%%d

if not exist %MainDir%"%RelPathToFile%" (echo Deleting %FullPathToFile% & del %FullPathToFile%)


## Potential Future Enhancements

More could be done:

1. It looks like the MaybeDelete.cmd script fails to delete a file if the file to delete has spaces anywhere in its name;
2. dsmbackup2007.cmd inflexibly expects its companion, MaybeDelete.cmd, to be in the c:\bin directory

## Sometimes there’s an easier way

Posted by danielmeyer on December 2, 2008

I was working to get snapshots of the Hibernate AnnotationConfiguration properties in effect when Hibernate local transactions are in use versus when a JTA transaction manager is in use, and halfway through I found there was a much easier way to capture my data.

## What I did First

1. While a debugging session was stopped at a breakpoint at line 732 of org.springframework.orm.hibernate3.LocalSessionFactoryBean (In buildSessionFactory() where the AnnotationConfiguration is done being built), I went to the Expressions view and selected the AnnotationConfiguration named “config”;
2. Opened config’s properties field and manually expanded all 116 nodes, exposing their key/value pairs;
3. Selected the 116 nodes-with-key/value-pairs and chose Copy Expressions from the context menu;
4. Pasted into Notepad and manually removed the “[0...99]” and “[100...116]” lines;
5. Saved the modified file as hibernate-annotation-config-jta.txt
6. Ran the following commands:
egrep --invert-match "\[" hibernate-annotation-config-jta.txt > keys-values.txt
egrep key= keys-values.txt | sed 's/[ \t]*key= \"$$.*$$\".*$/\1/' > keys.txt egrep value= keys-values.txt | sed 's/[ \t]*value= \"$$.*$$\".*$/\1/' > values.txt
paste --delimiter== keys.txt values.txt > keys-values-jta.txt


This worked — it got all 116 entries into key=value format, one key/value pair per line — and I prepared to do it again while running my test in Hibernate mode.

## Then I noticed…

Then I saw another pane in the Expressions view I hadn’t really noticed before, on the right hand side.  When I clicked on one of the AnnotationConfig’s properties in the Expressions window (even without expanding that entry to show the key and value), I saw the entry in key=value format over in that right hand pane.  And… if I selected all 116 nodes in the left pane of the Expressions view, all 116 key=value pairs showed up in the right pane, ready for me to select and copy!

My new workflow, then, is:

1. While a debugging session is stopped at a breakpoint at line 732 of org.springframework.orm.hibernate3.LocalSessionFactoryBean (In buildSessionFactory() where the AnnotationConfiguration is done being built), go to the Expressions view, select the AnnotationConfiguration named “config”, and within that select the properties field;
2. Make sure the “Show Logical Structure” button is pressed;
3. Expand the properties field so that you can see the individual [0], [1] [2]… entries
4. Select all these entries:
5. In the Expressions view’s right hand pane, Select All and copy.

Those are nicer steps than what I did first!  Eclipse comes through again!

Posted in Technical Stuff | Tagged: , , , | 1 Comment »

## Defaulting to the bash shell

Posted by danielmeyer on December 2, 2008

A comment on a previous post got me thinking — why do I always drop to a Windows command prompt?  Why don’t I make more use of the Cygwin bash shell?  The shell script language is more structured than batch files (more flow control options, etc.)…

I decided to change over to using a Cygwin bash shell as the command prompt my keyboard shortcut is set up to drop me to.  At some point, though, all the output in the shell was colored blue.  Not liking this, I searched around and ended up with a better solution than I expected to get.

## rxvt

I used Daniel Ambrósio’s customization of Eric Dobbs’ Cygwin + Rxvt post, to use rxvt instead of the Windows command prompt as the virtual terminal for Cygwin.  I followed these steps:

1. Used the Cygwin setup program to install the rxvt package
2. Created .Xdefaults as Ambrósio suggests:
! ~/.Xdefaults - X default resource settings
Rxvt*geometry: 120x60
Rxvt*background: #000020
Rxvt*foreground: #ffffbf
!Rxvt*borderColor: Blue
!Rxvt*scrollColor: Blue
!Rxvt*troughColor: Gray
Rxvt*scrollBar: True
Rxvt*scrollBar_right: True
Rxvt*font: Lucida Console-12
Rxvt*saveLines: 10000
! VIM-like colors
Rxvt*color0: #000000
Rxvt*color1: #FFFFFF
Rxvt*color2: #00A800
Rxvt*color3: #FFFF00
Rxvt*color4: #0000A8
Rxvt*color5: #A800A8
Rxvt*color6: #00A8A8
Rxvt*color7: #D8D8D8
Rxvt*color8: #000000
Rxvt*color9: #FFFFFF
Rxvt*color10: #00A800
Rxvt*color11: #FFFF00
Rxvt*color12: #0000A8
Rxvt*color13: #A800A8
Rxvt*color14: #00A8A8
Rxvt*color15: #D8D8D8
! eof
3. Created my cygwin.bat replacement, calling it cygwin-rxvt.cmd
4. Placed a shortcut to cygwin-rxvt.cmd on my desktop and modified gave it the keyboard shortcut that had formerly launched a Windows command prompt (to get me in the habit of using this)

The rxvt terminal is resizable in the horizontal and vertical direction without having to go to the system menu, properties, and looks pretty nice!  Maybe now I’ll more easily notice when a problem would lend itself to being solved by a shell script.

Finally, Ben sends a link to the Advanced Bash-Scripting Guide — thanks!  Looks like a good resource I’ve been missing.

## Just general enough

Posted by danielmeyer on December 2, 2008

When I write a utility — a shell script, batch file, Python script… — I’m always trying to write it so that a co-worker could receive a copy of the script and not have to do rework in order to use it.

Yesterday, though, I followed this procedure for the Nth time:

1. Shut down JBoss
2. Open up folder F; Select sample-project(T-transaction-manager).war; Copy
3. Open up JBoss server/default/deploy folder;Delete sample-project.war
4. Paste sample-project(T-transaction-manager).war; rename to sample-project.war
5. Restart JBoss

(T can be hibernate or jta)

Having the two different .wars pre-built and able to just be copied to the deploy directory was already an optimization of my procedure; before that I had been toggling the transaction manager and rebuilding the project each time I needed to redeploy!

As I went, though, I found myself wanting to get a little information from the one kind of deployment, then quickly switch to the other and try the same thing to compare results.  It was time for something more streamlined, something that didn’t break my flow so much.  Here’s what I came up with.

## txdeploy.cmd

I renamed the two .wars to sample-project(hib).war and sample-project(jta).war and wrote a supporting batch file named txdeploy.cmd:

@echo off
if not %1==jta if not %1==hib if not %1==which echo Usage: txdeploy jta (or) txdeploy hib (or) txdeploy which && goto End

if not %1==which goto deploy

:which
for %%f in (jta hib) do fc "C:\path\to\sample-project(%%f).war" C:\path\to\jboss-4.2.2.GA\server\default\deploy\sample-project.war > NUL && echo %%f version is deployed
goto End

:deploy
copy /y "C:\path\to\sample-project(%1).war" C:\path\to\jboss-4.2.2.GA\server\default\deploy\sample-project.war
goto which

:End


Now I can run txdeploy hib from the Windows command prompt to deploy the version that uses a Hibernate transaction manager, and txdeploy jta to deploy the JTA one instead.  I can also run txdeploy which, which displays hib version is deployed or jta version is deployed, as appropriate.

Now my workflow is:

1. Shut down JBoss
2. type txdeploy hib (for example) at the Windows command prompt
3. Restart JBoss

Much nicer!

Posted in Technical Stuff | Tagged: | 5 Comments »

## Wake on LAN

Posted by danielmeyer on August 5, 2008

This technology has been around for a while, but I still think it’s cool.

If you have a computer you sometimes want to start up from a different computer, then you could send a Wake On LAN packet to the computer if it’s enabled for that.

Here’s a Python script I wrote that to wake up my computer at work if it was shut down, so I wouldn’t have to drive in just to start it up — the wake-on-LAN packet would be sent from another server on the internal network.

# Program: wakeup.py
# Purpose: Sends a magic wakeup packet to the MAC address specified in the current user's .wolrc file
# Programmer: Daniel Meyer
#
# Notes: The magic packet is actually built by wolpython.py, written by Marc Balmer.
# See http://gsd.di.uminho.pt/jpo/software/wakeonlan/mini-howto/wolpython.txt

import os,sys,re,string
import wolpython

def mac_well_formed(mac):
return re.match('[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]', mac)

# Main
walrus_filename = os.environ['HOME'] + '/.wolrc'
if not os.path.exists(walrus_filename):
print 'wakeup: please place the MAC address of the computer you want to wake in ' + walrus_filename
sys.exit()

walrus = open(os.environ['HOME'] + '/.wolrc')
while mac != '':
if mac_well_formed(mac):
print 'Sending wakeup message to ' + mac + '...',
wolpython.WakeOnLan(mac)
print 'done.'
else:
print 'MAC address "' + mac + '" skipped (not in xx:xx:xx:xx:xx:xx format).'


If the computer-you-want-to-wake is running Linux, you can find its network address by running /sbin/ifconfig and looking for the HWaddr; on Windows, run ipconfig /all from the command prompt and look for the Physical Address.

This script is a little silly, because it has this idea of a .wolrc (“walrus”) file that contains the computer you always want to wake up, and it is only designed to work on Unix/Linux (it expects a HOME environment variable) — not very full-featured.  I was proud of the walrus filename though. :)

You’d probably be better off using java-wakeonlan!

(Is it even really worth having my home computer’s network card always be awake listening for a wake-on-LAN packet that it might get once every couple of weeks?  Hmm…)

## Multi-file search and replace: replacefromfile.py

Posted by danielmeyer on March 12, 2008

The code editor we use lacks support for multi-file search and replace, and this makes renaming a method or class a lot more work-intensive than if such IDE support were there. Recently I needed to do several such renames. My work started out as modifications and additions to one large multipurpose class, but I noticed that I was adding still more responsibilities to the class. The class’s methods already showed that it had multiple natures: CreateA, ModifyA, DeleteA, CreateB, ModifyB, etc. I was adding methods to deal with C’s and D’s as well.

I didn’t refactor first off, though, because the class was not under test yet. I was afraid that during the process of renaming I might break something and not know it. So the first step I took was to write a unit test for the class. Next, I added in all my new methods for dealing with C’s and D’s into the multipurpose class (looking back, I think I wasted time by doing this, as it increased the amount of renaming I would need to do later, but I did not see this at the time), and wrote unit tests for them.

Now, with everything under basic test, I felt more confident to move methods out to their new classes. I created an A class and moved the CreateA, ModifyA, and DeleteA methods in there. Then I was able to rename the methods to just Create, Modify, and Delete, since it was now clear what kind of thing they operated on.

The remaining step was to replace all references to the old class name and method names with the new class names and method names. To do this, I wrote a Python script that would take a file containing search strings and replacement strings and do the replacements. I then created my replacements file containing lines like this (though actually there were about twenty replacements):

CreateA Create
ModifyA Modify
DeleteA Delete

Then I exported my source code project as XML (a trick I learned from Aaron Alexander – thanks, Aaron!) and ran my replace utility like this:

python replacefromfile.py –replacements-file=replacements.txt < myproject.xml > changed.xml

Then I re-ran my tests. If I recall correctly, they still passed.

replacefromfile.py.txt

## Match all lines that aren’t in a list

Posted by danielmeyer on March 11, 2008

I wanted to take the output of a grep search and use that to build another search, such that any line of the input file for the second search that matched any of the patterns built from the first search would be excluded from the resulting output. I’ve wanted this kind of thing before but I think I have always resorted to several single-purpose searches.

Could I have accomplished the same thing with a single cunningly-devised regular expression? I suspect so — I need to get better at using regular expressions. Anyway, here’s the script (I used the bash shell version 3.2.33 on Cygwin…)

#!/bin/sh
# Program: find-not-in-list.sh
# Author:  Daniel Meyer
# Date:    3/11/2008
# Purpose: Given an input file name and one or more awk-style patterns to match,
#          this program outputs all lines that do NOT match any of the specified patterns.
#
# Example 0:
#   Suppose you have a file inputfile.txt, and you want to output the lines in this file minus the ones that
#   start with 422 AND contain a couple of values (2838 and 01523). You could do this using a series of
#   "grep -v" (invert match) statements:
#
#   < inputfile.txt grep -v "^422.*2838" | grep -v "^422.*01523"
#
# That's fine if the number of patterns to match is small and/or the input file size is small.
# But it's a lot of hand typing if you need to invert-match many patterns... and if the input file is large,
# reading the file many times could take a while.
#
# Example 1:
# This program solves the second problem: By hand-typing patterns for lines you *don't* want from the inputfile,
# you can
#
#   echo /^422.*2838/ /^422.*01523/ | xargs find-not-in-list.sh inputfile.txt > shavedfile.txt
#
# Example 2:
# This example solves the first problem.  Instead of typing in each pattern by hand as in example 1, the patterns
# could be built from the output of a search. For example, in a tab-delimited file if you want to take the second
# field of each line that starts with 143 and contains the string INTERALIA, and then show the input file excluding
# lines that start with 422 and contain the contents of that column, you could execute the following command:
#
#   for x in grep "^143.*INTERALIA" < inputfile.txt | cut --fields=2; do echo /^422.*$x/; done | xargs find-not-in-list.sh inputfile.txt # # Notice how we build our pattern by adding on to what was in the second column in the file using an echo command # to standard output and then piping that to xargs. (The xargs command passes the contents of its standard input # as parameters, so in this example if the grep command found 2838 and 01523 in inputfile.txt, find-not-in-list.sh # would be called with three parameters, as follows: # # find-not-in-list.sh inputfile.txt /^422.*2838/ /^422.*01523/ if [$# -lt 2 ]; then
echo "Usage: sh find-not-in-list.sh inputfile pattern [pattern...]"
echo "       Each pattern should be an AWK-style regular expression beginning and ending with '/'"
exit 1
fi
inputfile=$1 shifttempfile=mktemp # Build an awk command for not matching the first pattern echo -n "\$0 !~ $1" >>$tempfile

# Add in all the rest of the patterns to not match
while [ $# -gt 1 ]; do shift echo -n " && \$0 !~ $1" >>$tempfile
done

# Run the awk command
awk -f $tempfile <$inputfile

rm \$tempfile
exit 0


find-not-in-list.sh.txt