Someone just sent me a Java project, which I need to import into Eclipse and take a look at. (The changes aren’t ready to commit to Subversion yet.)
The project came to me with a target directory, which is easy to delete:
rm -r target
but it also came with the Subversion metadata — an .svn folder under each regular folder. The line I used to get rid of all these:
find . -name *.svn* | xargs rm -r
Yeah xargs!
My Dad has an external hard drive he uses to back up his My Documents folder to. He was interested in an automated way to do this and then also to delete files off the backup drive that have been deleted off the main drive; so sometime last year (I’m guessing based on the the name I gave the main batch file), I created a couple of batch files for him to do that.
The main batch file is dsmbackup2007.cmd, and its helper is the snazzily-named-but-obscurely-written MaybeDelete.cmd.
Dad was having trouble with the backup program — it was giving errors — and it turned out to be that the MaybeDelete.cmd script had gotten misplaced. I added better error messages to the dsmbackup2007.cmd script for this case and thought I’d post both scripts here (so I don’t lose ’em again):
@echo off REM You can change this drive letter as needed. set BACKUPDRIVE=K set MAYBEDELETECMD=c:\bin\MaybeDelete.cmd set USER=%USERNAME% set COPYFROM=%USERPROFILE%\My Documents\ set COPYTO=%BACKUPDRIVE%:\DSMBackup2007\%COMPUTERNAME%\%USERNAME%\ echo Ready to copy %USER%'s My Documents from %COPYFROM% to %COPYTO% . pause REM Copy from the current user's My Documents folder to a folder on the backup drive REM /S - Copy subdirectories also REM /E - Copy even empty subdirectories REM /M - Copy only modified files, and reset the "modified" flag once they're copied REM /Y - Don't prompt to overwrite files on the backup drive (we want to automatically overwrite them with the newer version). xcopy "%COPYFROM%*" %COPYTO% /S /E /M /Y REM See if there was an error. If so, display an error message. REM If not, jump to the end. if ERRORLEVEL 1 goto Bad :Good echo Backup appears to have been successful! goto RemoveStale :Bad echo *****BACKUP FAILED***** pause :RemoveStale REM Delete files that are now gone from the copy-from folder. echo. echo Ready to delete files from backup drive that no longer exist on source drive... pause if not exist %MAYBEDELETECMD% echo. & echo ERROR: Could not find MaybeDelete.cmd file & echo (expected it to be at %MAYBEDELETECMD%) & echo Please put the MaybeDelete.cmd file in the expected directory. & echo. & goto End echo (This may take a little while, we're pedaling as fast as we can...) for /R %COPYTO% %%F in (*) do call %MAYBEDELETECMD% "%%F" "%COPYFROM%" echo Done. :End pause
REM @echo off set FullPathToFile=%1 set MainDir=%2 REM Strip quotes off the maindir path. REM for /F %%S in (%2) do set MainDir=%%~fnxS REM Takes off the first few parts of the directory path that REM only apply to the backup drive. for /F "tokens=4* delims=\" %%c in (%FullPathToFile%) do set RelPathToFile=%%d if not exist %MainDir%"%RelPathToFile%" (echo Deleting %FullPathToFile% & del %FullPathToFile%)
More could be done:
I was working to get snapshots of the Hibernate AnnotationConfiguration properties in effect when Hibernate local transactions are in use versus when a JTA transaction manager is in use, and halfway through I found there was a much easier way to capture my data.
egrep --invert-match "\[" hibernate-annotation-config-jta.txt > keys-values.txt egrep key= keys-values.txt | sed 's/[ \t]*key= \"\(.*\)\".*$/\1/' > keys.txt egrep value= keys-values.txt | sed 's/[ \t]*value= \"\(.*\)\".*$/\1/' > values.txt paste --delimiter== keys.txt values.txt > keys-values-jta.txt
This worked — it got all 116 entries into key=value format, one key/value pair per line — and I prepared to do it again while running my test in Hibernate mode.
Then I saw another pane in the Expressions view I hadn’t really noticed before, on the right hand side. When I clicked on one of the AnnotationConfig’s properties in the Expressions window (even without expanding that entry to show the key and value), I saw the entry in key=value format over in that right hand pane. And… if I selected all 116 nodes in the left pane of the Expressions view, all 116 key=value pairs showed up in the right pane, ready for me to select and copy!
My new workflow, then, is:
Those are nicer steps than what I did first! Eclipse comes through again!
A comment on a previous post got me thinking — why do I always drop to a Windows command prompt? Why don’t I make more use of the Cygwin bash shell? The shell script language is more structured than batch files (more flow control options, etc.)…
I decided to change over to using a Cygwin bash shell as the command prompt my keyboard shortcut is set up to drop me to. At some point, though, all the output in the shell was colored blue. Not liking this, I searched around and ended up with a better solution than I expected to get.
I used Daniel Ambrósio’s customization of Eric Dobbs’ Cygwin + Rxvt post, to use rxvt instead of the Windows command prompt as the virtual terminal for Cygwin. I followed these steps:
! ~/.Xdefaults - X default resource settings Rxvt*geometry: 120x60 Rxvt*background: #000020 Rxvt*foreground: #ffffbf !Rxvt*borderColor: Blue !Rxvt*scrollColor: Blue !Rxvt*troughColor: Gray Rxvt*scrollBar: True Rxvt*scrollBar_right: True Rxvt*font: Lucida Console-12 Rxvt*saveLines: 10000 Rxvt*loginShell: True ! VIM-like colors Rxvt*color0: #000000 Rxvt*color1: #FFFFFF Rxvt*color2: #00A800 Rxvt*color3: #FFFF00 Rxvt*color4: #0000A8 Rxvt*color5: #A800A8 Rxvt*color6: #00A8A8 Rxvt*color7: #D8D8D8 Rxvt*color8: #000000 Rxvt*color9: #FFFFFF Rxvt*color10: #00A800 Rxvt*color11: #FFFF00 Rxvt*color12: #0000A8 Rxvt*color13: #A800A8 Rxvt*color14: #00A8A8 Rxvt*color15: #D8D8D8 ! eof
The rxvt terminal is resizable in the horizontal and vertical direction without having to go to the system menu, properties, and looks pretty nice! Maybe now I’ll more easily notice when a problem would lend itself to being solved by a shell script.
Finally, Ben sends a link to the Advanced Bash-Scripting Guide — thanks! Looks like a good resource I’ve been missing.
When I write a utility — a shell script, batch file, Python script… — I’m always trying to write it so that a co-worker could receive a copy of the script and not have to do rework in order to use it.
Yesterday, though, I followed this procedure for the Nth time:
(T can be hibernate or jta)
Having the two different .wars pre-built and able to just be copied to the deploy directory was already an optimization of my procedure; before that I had been toggling the transaction manager and rebuilding the project each time I needed to redeploy!
As I went, though, I found myself wanting to get a little information from the one kind of deployment, then quickly switch to the other and try the same thing to compare results. It was time for something more streamlined, something that didn’t break my flow so much. Here’s what I came up with.
I renamed the two .wars to sample-project(hib).war and sample-project(jta).war and wrote a supporting batch file named txdeploy.cmd:
@echo off if not %1==jta if not %1==hib if not %1==which echo Usage: txdeploy jta (or) txdeploy hib (or) txdeploy which && goto End if not %1==which goto deploy :which for %%f in (jta hib) do fc "C:\path\to\sample-project(%%f).war" C:\path\to\jboss-4.2.2.GA\server\default\deploy\sample-project.war > NUL && echo %%f version is deployed goto End :deploy copy /y "C:\path\to\sample-project(%1).war" C:\path\to\jboss-4.2.2.GA\server\default\deploy\sample-project.war goto which :End
Now I can run txdeploy hib from the Windows command prompt to deploy the version that uses a Hibernate transaction manager, and txdeploy jta to deploy the JTA one instead. I can also run txdeploy which, which displays hib version is deployed or jta version is deployed, as appropriate.
Now my workflow is:
Much nicer!
This technology has been around for a while, but I still think it’s cool.
If you have a computer you sometimes want to start up from a different computer, then you could send a Wake On LAN packet to the computer if it’s enabled for that.
Here’s a Python script I wrote that to wake up my computer at work if it was shut down, so I wouldn’t have to drive in just to start it up — the wake-on-LAN packet would be sent from another server on the internal network.
# Program: wakeup.py
# Purpose: Sends a magic wakeup packet to the MAC address specified in the current user's .wolrc file
# Programmer: Daniel Meyer
#
# Notes: The magic packet is actually built by wolpython.py, written by Marc Balmer.
# See http://gsd.di.uminho.pt/jpo/software/wakeonlan/mini-howto/wolpython.txt
import os,sys,re,string
import wolpython
def mac_well_formed(mac):
return re.match('[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]:[0-9a-fA-F][0-9a-fA-F]', mac)
# Main
walrus_filename = os.environ['HOME'] + '/.wolrc'
if not os.path.exists(walrus_filename):
print 'wakeup: please place the MAC address of the computer you want to wake in ' + walrus_filename
sys.exit()
walrus = open(os.environ['HOME'] + '/.wolrc')
mac = string.strip(walrus.readline())
while mac != '':
if mac_well_formed(mac):
print 'Sending wakeup message to ' + mac + '...',
wolpython.WakeOnLan(mac)
print 'done.'
else:
print 'MAC address "' + mac + '" skipped (not in xx:xx:xx:xx:xx:xx format).'
mac = string.strip(walrus.readline())
If the computer-you-want-to-wake is running Linux, you can find its network address by running /sbin/ifconfig and looking for the HWaddr; on Windows, run ipconfig /all from the command prompt and look for the Physical Address.
This script is a little silly, because it has this idea of a .wolrc (“walrus”) file that contains the computer you always want to wake up, and it is only designed to work on Unix/Linux (it expects a HOME environment variable) — not very full-featured. I was proud of the walrus filename though. :)
You’d probably be better off using java-wakeonlan!
(Is it even really worth having my home computer’s network card always be awake listening for a wake-on-LAN packet that it might get once every couple of weeks? Hmm…)
The code editor we use lacks support for multi-file search and replace, and this makes renaming a method or class a lot more work-intensive than if such IDE support were there. Recently I needed to do several such renames. My work started out as modifications and additions to one large multipurpose class, but I noticed that I was adding still more responsibilities to the class. The class’s methods already showed that it had multiple natures: CreateA, ModifyA, DeleteA, CreateB, ModifyB, etc. I was adding methods to deal with C’s and D’s as well.
I didn’t refactor first off, though, because the class was not under test yet. I was afraid that during the process of renaming I might break something and not know it. So the first step I took was to write a unit test for the class. Next, I added in all my new methods for dealing with C’s and D’s into the multipurpose class (looking back, I think I wasted time by doing this, as it increased the amount of renaming I would need to do later, but I did not see this at the time), and wrote unit tests for them.
Now, with everything under basic test, I felt more confident to move methods out to their new classes. I created an A class and moved the CreateA, ModifyA, and DeleteA methods in there. Then I was able to rename the methods to just Create, Modify, and Delete, since it was now clear what kind of thing they operated on.
The remaining step was to replace all references to the old class name and method names with the new class names and method names. To do this, I wrote a Python script that would take a file containing search strings and replacement strings and do the replacements. I then created my replacements file containing lines like this (though actually there were about twenty replacements):
CreateA Create
ModifyA Modify
DeleteA Delete
Then I exported my source code project as XML (a trick I learned from Aaron Alexander – thanks, Aaron!) and ran my replace utility like this:
python replacefromfile.py –replacements-file=replacements.txt < myproject.xml > changed.xml
Then I re-ran my tests. If I recall correctly, they still passed.
I wanted to take the output of a grep search and use that to build another search, such that any line of the input file for the second search that matched any of the patterns built from the first search would be excluded from the resulting output. I’ve wanted this kind of thing before but I think I have always resorted to several single-purpose searches.
Could I have accomplished the same thing with a single cunningly-devised regular expression? I suspect so — I need to get better at using regular expressions. Anyway, here’s the script (I used the bash shell version 3.2.33 on Cygwin…)
#!/bin/sh
# Program: find-not-in-list.sh
# Author: Daniel Meyer
# Date: 3/11/2008
# Purpose: Given an input file name and one or more awk-style patterns to match,
# this program outputs all lines that do NOT match any of the specified patterns.
#
# Example 0:
# Suppose you have a file inputfile.txt, and you want to output the lines in this file minus the ones that
# start with 422 AND contain a couple of values (2838 and 01523). You could do this using a series of
# “grep -v” (invert match) statements:
#
# < inputfile.txt grep -v "^422.*2838" | grep -v "^422.*01523"
#
# That's fine if the number of patterns to match is small and/or the input file size is small.
# But it's a lot of hand typing if you need to invert-match many patterns... and if the input file is large,
# reading the file many times could take a while.
#
# Example 1:
# This program solves the second problem: By hand-typing patterns for lines you *don't* want from the inputfile,
# you can
#
# echo /^422.*2838/ /^422.*01523/ | xargs find-not-in-list.sh inputfile.txt > shavedfile.txt
#
# Example 2:
# This example solves the first problem. Instead of typing in each pattern by hand as in example 1, the patterns
# could be built from the output of a search. For example, in a tab-delimited file if you want to take the second
# field of each line that starts with 143 and contains the string INTERALIA, and then show the input file excluding
# lines that start with 422 and contain the contents of that column, you could execute the following command:
#
# for x in `grep “^143.*INTERALIA” < inputfile.txt | cut --fields=2`; do echo /^422.*$x/; done | xargs find-not-in-list.sh inputfile.txt
#
# Notice how we build our pattern by adding on to what was in the second column in the file using an echo command
# to standard output and then piping that to xargs. (The xargs command passes the contents of its standard input
# as parameters, so in this example if the grep command found 2838 and 01523 in inputfile.txt, find-not-in-list.sh
# would be called with three parameters, as follows:
#
# find-not-in-list.sh inputfile.txt /^422.*2838/ /^422.*01523/
if [ $# -lt 2 ]; then
echo "Usage: sh find-not-in-list.sh inputfile pattern [pattern...]"
echo " Each pattern should be an AWK-style regular expression beginning and ending with '/'"
exit 1
fi
inputfile=$1 shifttempfile=`mktemp`
# Build an awk command for not matching the first pattern
echo -n "\$0 !~ $1" >> $tempfile
# Add in all the rest of the patterns to not match
while [ $# -gt 1 ]; do
shift
echo -n ” && \$0 !~ $1″ >> $tempfile
done
# Run the awk command
awk -f $tempfile < $inputfile
rm $tempfile
exit 0
[/sourcecode]
find-not-in-list.sh.txt
Our Craft 