Home

Recursive Search and Replace Bash Script

Steve FranciaAug 05,2008
  • Share/Save

I had a need to perform a search and replace operation on all files in a directory of files. Effectively I needed a recursive sed. I couldn’t find one and wrote this script. This is version .1, but worked for my needs, hopefully others find it useful. It is probably a smart practice to backup your set of files before running any batch operation on them. The script is released under the GPL2 License. 

  1. #!/bin/bash
  2.  
  3. #   ––––––––––––––––––––––
  4. #
  5. #   Shell program to run a search and replace operation on files recursively.
  6. #
  7. #   Copyright 2008, Steve Francia <steve.francia+search_and_replace@gmail.com>.
  8. #
  9. #   This program is free software; you can redistribute it and/or
  10. #   modify it under the terms of the <span class="caps">GNU</span> General Public License as
  11. #   published by the Free Software Foundation; either version 2 of the
  12. #   License, or (at your option) any later version.
  13. #
  14. #   This program is distributed in the hope that it will be useful,
  15. #   but <span class="caps">WITHOUT</span> <span class="caps">ANY</span> <span class="caps">WARRANTY</span>; without even the implied warranty of
  16. #   <span class="caps">MERCHANTABILITY</span> or <span class="caps">FITNESS</span> <span class="caps">FOR</span> A <span class="caps">PARTICULAR</span> <span class="caps">PURPOSE</span>. See the <span class="caps">GNU</span>
  17. #   General Public License for more details.
  18. #
  19. #   Description:
  20. #
  21. #
  22. #   Usage:
  23. #
  24. #       search_and_replace.sh [ -h | —help ] [<span class="caps">OPTIONS</span>] [<span class="caps">ARGUMENT1</span>] [<span class="caps">ARGUMENT2</span>]
  25. #
  26. #   Options:
  27. #
  28. #       -h, —help   Display this help message and exit.
  29. #       —version    Display program version.
  30. #
  31. #   Arguments:
  32. #
  33. #       <span class="caps">ARGUMENT1</span>   The token to search for.
  34. #       <span class="caps">ARGUMENT2</span>   The token to replace it with.
  35. #
  36. #   External programs:
  37. #
  38. #       getopt   - parse command options (enhanced)
  39. #       mktemp   - make temporary filename (unique)
  40. #       sed      - stream editor
  41. #       basename - strip directory and suffix from filenames
  42. #
  43. #   Revision History:
  44. #
  45. #   2008-07-25      Initial Script Created
  46. #                   See ChangeLog for additional changes
  47. #
  48. #
  49. #   ––––––––––––––––––––––
  50.  
  51. ##### Preamble #####
  52.  
  53. #   ––––––––––––––––––––––
  54. #   Constants
  55. #   ––––––––––––––––––––––
  56.  
  57.     declare -r <span class="caps">PROGNAME</span>=$(basename $0)
  58.     declare -r <span class="caps">VERSION</span>="0.1"
  59.     declare -r <span class="caps">DESCRIPTION</span>="a Shell program to run a search and replace operation on files recursively."
  60.  
  61.  
  62. ##### Functions #####
  63.  
  64. #   ––––––––––––––––––––––
  65. #   Functions
  66. #   ––––––––––––––––––––––
  67.  
  68.  
  69. function clean_up() {
  70.  
  71. #   ––––––––––––––––––––––
  72. #   Function to remove temporary files and other housekeeping
  73. #       No arguments
  74. #   ––––––––––––––––––––––
  75.  
  76.     #rm -f ${TEMP_FILE1}
  77.     return
  78.  
  79. }   # end of clean_up
  80.  
  81.  
  82. function error_exit() {
  83.  
  84. #   ––––––––––––––––––––––
  85. #   Function for exit due to fatal program error
  86. #       Arguments:
  87. #           1 (optional) string containing descriptive error message
  88. #   ––––––––––––––––––––––
  89.  
  90.  
  91.     echo "${<span class="caps">PROGNAME</span>}: ${1:-"Unknown Error"}" &gt;&amp;2
  92.     clean_up
  93.     exit 1
  94.  
  95. }   # end of error_exit
  96.  
  97.  
  98. function graceful_exit() {
  99.  
  100. #   ––––––––––––––––––––––
  101. #   Function called for a graceful exit
  102. #       No arguments
  103. #   ––––––––––––––––––––––
  104.  
  105.     clean_up
  106.     exit
  107.  
  108. }   # end of graceful_exit
  109.  
  110.  
  111. function signal_exit() {
  112.  
  113. #   ––––––––––––––––––––––
  114. #   Function to handle termination signals
  115. #       Arguments:
  116. #           1 (optional) signal_spec
  117. #   ––––––––––––––––––––––
  118.  
  119.     case $1 in
  120.         <span class="caps">INT</span>) 
  121.             echo "$<span class="caps">PROGNAME</span>: Program aborted by user" &gt;&amp;2
  122.             clean_up
  123.             exit
  124.             ;;
  125.         <span class="caps">TERM</span>)
  126.             echo "$<span class="caps">PROGNAME</span>: Program terminated" &gt;&amp;2
  127.             clean_up
  128.             exit
  129.             ;;
  130.         *)
  131.             error_exit "$<span class="caps">PROGNAME</span>: Terminating on unknown signal"
  132.             ;;
  133.  
  134.     esac
  135.  
  136. }   # end of signal_exit
  137.  
  138.  
  139. function make_temp_files() {
  140.  
  141. #   ––––––––––––––––––––––
  142. #   Function to create temporary files
  143. #       No arguments
  144. #   ––––––––––––––––––––––
  145.  
  146.     # Use user's local tmp directory if it exists
  147.  
  148.     if [ -d ~/tmp ]; then
  149.         TEMP_DIR=~/tmp
  150.     else
  151.         TEMP_DIR=/tmp
  152.     fi
  153.  
  154.     # Temp file for this script, using paranoid method of creation to
  155.     # insure that file name is not predictable.  This is for security
  156.     # to avoid "tmp race" attacks.  If more files are needed, create
  157.     # using the same form.
  158.  
  159.     TEMP_FILE1=$(mktemp -q "${TEMP_DIR}/${<span class="caps">PROGNAME</span>}.$$.<span class="caps">XXXXXX</span>")
  160.     if [ "$TEMP_FILE1" = "" ]; then
  161.         error_exit "cannot create temp file!"
  162.     fi
  163.  
  164. }   # end of make_temp_files
  165.  
  166.  
  167. function usage() {
  168.  
  169. #   ––––––––––––––––––––––
  170. #   Function to display usage message (does not exit)
  171. #       No arguments
  172. #   ––––––––––––––––––––––
  173.  
  174.     echo "Usage: ${<span class="caps">PROGNAME</span>} [-h | —help] [<span class="caps">OPTIONS</span>] [<span class="caps">SEARCH</span>] [<span class="caps">REPLACE</span>]"
  175.  
  176. }   # end of usage
  177.  
  178.  
  179. function helptext() {
  180.  
  181. #   ––––––––––––––––––––––
  182. #   Function to display help message for program
  183. #       No arguments
  184. #   ––––––––––––––––––––––
  185.  
  186.     cat &lt;<eof ${progname}="" ${version}="" this="" is="" a="" program="" to="" ${description}.="" $(usage)="" options:="" -h,="" —help="" display="" help="" message="" and="" exit.="" -t,="" —test="" what="" actualy="" being="" passed="" sed.="" -e="" [ext],="" —extension="" [ext]="" only="" replace="" text="" found="" in="" files="" with="" extension="" —version="" version.="" arguments:="" search="" the="" token="" for.="" it="" with.="" hints:="" please="" escape="" your="" strings.="" if="" you="" are="" unsure,="" test="" -t="" first.="" eof="" }="" #="" end="" of="" helptext="" function="" search_and_replace()="" {="" ––––––––––––––––––––––="" [insert="" description]="" 1="" (required)="" arg1="" 2="" arg2="" [="" $testmode="" -ne="" 0="" ]="" ;="" then="" echo="" passing="" s="" $1="" $2="" g'="" into="" sed"="" graceful_exit;="" fi="" fatal="" error="" required="" arguments="" missing="" $1"="" ];="" error_exit="" search_and_replace:="" argument="" 1"="" $2"="" 2"="" sed="" $1"'="" $2"'="" $file"=""> $TEMP_FILE1
  187. 	mv $TEMP_FILE1 "$file"
  188.  
  189.     return
  190.  
  191. }   # end of search_and_replace
  192.  
  193. # Recursive Directory Traverser
  194. # Author: Kaz Kylheku
  195. # Date: Feb 27, 1999
  196. # Copyright 1999
  197.  
  198. function recurse() {
  199. # Function parameter usage:
  200. # $1 directory to search
  201. # $2 pattern to search for
  202. # $3 command to execute
  203. # $4 secret argument for passing down path
  204.  
  205.     if [ "$1" = "" ]; then 
  206.         error_exit "recurse: missing argument 1"
  207.     fi
  208.     if [ "$2" = "" ]; then 
  209.         error_exit "recurse: missing argument 2"
  210.     fi
  211.     if [ "$3" = "" ]; then 
  212.         error_exit "recurse: missing argument 3"
  213.     fi
  214.  
  215.     local file
  216.     local path
  217.  
  218.     if [ "$4" = "" ] ; then
  219.         path="${1%/}/"
  220.     else
  221.         path="$4$1/"
  222.     fi
  223.  
  224.     if cd "$1" ; then
  225.         for file in $2; do
  226.             if [ -f "$file" -o -d "$file" ]; then
  227.                 eval "$3"
  228.             fi
  229.         done
  230.         for file in .* * ; do
  231.             if [ "$file" = "." -o "$file" = ".." ] ; then
  232.                 continue
  233.             fi
  234.             if [ -d "$file" -a ! -L "$file" ]; then
  235.                 recurse "$file" "$2" "$3" "$path"
  236.             fi
  237.         done
  238.         cd ..
  239.     fi
  240. }
  241.  
  242.  
  243. #   ––––––––––––––––––––––
  244. #   Program starts here
  245. #   ––––––––––––––––––––––
  246.  
  247. ##### Initialization And Setup #####
  248.  
  249. ## Set file creation mask so that all files are created with 600
  250. ## permissions.
  251. ##
  252. #umask 066
  253.  
  254. # Trap <span class="caps">TERM</span>, <span class="caps">HUP</span>, and <span class="caps">INT</span> signals and properly exit
  255.  
  256. trap "signal_exit <span class="caps">TERM</span>" <span class="caps">TERM</span> <span class="caps">HUP</span>
  257. trap "signal_exit <span class="caps">INT</span>"  <span class="caps">INT</span>
  258.  
  259. ## Create temporary file(s)
  260. #
  261. make_temp_files
  262.  
  263.  
  264. ##### Command Line Processing #####
  265.  
  266.  # if at least one argument is required…
  267.  if [ $# -eq 0 ]; then
  268.      echo "This is ${<span class="caps">PROGNAME</span>} ${<span class="caps">VERSION</span>}"
  269.      usage
  270.      clean_up
  271.      exit 1
  272.  fi
  273.  
  274. # Note that we use `"$@"' to let each command-line parameter expand to
  275. # a separate word. The quotes around `$@' are essential! We need
  276. # GETOPT_TEMP as the `eval set —' would nuke the return value of
  277. # getopt.
  278. GETOPT_TEMP=$(getopt -o +te:h —long test,extension:,help,version -n "$<span class="caps">PROGNAME</span>" — "$@")
  279.  
  280. if [ $? != 0 ] ; then
  281.     error_exit "Error parsing command line. Terminating…"
  282. fi
  283.  
  284. # Note the quotes around `$GETOPT_TEMP': they are essential!
  285. eval set — "$GETOPT_TEMP"
  286.  
  287. # no error checking necessary; sanity of command line and required
  288. # arguments has been checked by getopt program
  289.  
  290. ext="*";
  291. testmode=0;
  292.  
  293. while true ; do
  294.     case $1 in
  295.         -t|—test)
  296.             echo "<span class="caps">TEST</span> <span class="caps">MODE</span>" ;
  297. 			testmode=1;
  298.             shift
  299.             ;;
  300.         -e|—extension)
  301.             echo "search for extension: $2"
  302. 			ext=$2 
  303.             shift 2
  304.             ;;
  305.         -h|—help)
  306.             helptext ;
  307.             graceful_exit
  308.             ;;
  309.         —version)
  310.             echo "${<span class="caps">PROGNAME</span>} ${<span class="caps">VERSION</span>}" ;
  311.             shift
  312.             ;;
  313.         —)
  314.             shift ;
  315.             break
  316.             ;;
  317.         *)
  318.             # should be impossible to reach: getopt should have caught
  319.             # an error
  320.             error_exit "This should not have happened; unknown option '$1'. Terminating…"
  321.             ;;
  322.     esac
  323. done
  324. unset GETOPT_TEMP
  325.  
  326. # processing remaining arguments for the client
  327. if [ $# -ne 0  ]; then
  328.     echo "Searching for: $@"
  329. fi
  330.  
  331.  
  332. ##### Main Logic #####
  333.  
  334. recurse `/bin/pwd` "*.$ext" "search_and_replace $1 $2"
  335.  
  336. graceful_exit
  337.  
  338. # end of search_and_replace.sh
  339.  
  340. </eof></steve.francia+search_and_replace@gmail.com>

Bookmark/Search this post with:
AnonymousDec 11,2008

great. I have to try your

great. I have to try your code. Thanks. // Jadu, unstableme.blogspot.com

Warlei Alves (not verified)Sep 30,2009

A little detail

I like to thanks and congratulate you for your so well written bash script.

It worked fine to me!

But when I using it I typed some expression wrong and this caused a error message in sed. But program didn't stop and the original files was replaced by the temporary blank files.

To avoid this replacement in situations like that I put a conditional expression considering the value of "$?" before do the replacement.

So:

...

sed -e 's/'"$1"'/'"$2"'/g' "$file" > $TEMP_FILE1

if [ "$?" = "0" ]; then
mv $TEMP_FILE1 "$file"
fi

...

Thanks one more time!

Post new comment

The content of this field is kept private and will not be shown publicly.