Replaceutil.py

Find & Replace in text files using Python

For a number of reasons it may be useful to have a script automatically find and replace certain strings in text files (namelists, source code files, etc.). Below we look at how to use python to do just that.

The first thing we'll need is the Python file replaceutil.py (source: http://stackoverflow.com/questions/39086/search-and-replace-a-line-in-a-file-in-python), the code is as follows:

from tempfile import mkstemp
from shutil import move
from os import remove, close

def replace(file, pattern, subst):
    #Create temp file
    fh, abs_path = mkstemp()
    new_file = open(abs_path,'w')
    old_file = open(file)
    for line in old_file:
        new_file.write(line.replace(pattern, subst))
    #close temp file
    new_file.close()
    close(fh)
    old_file.close()
    #Remove original file
    remove(file)
    #Move new file
    move(abs_path, file)

We have now created a custom python module called 'replaceutil' that we can import into our own scripts. replaceutil.py makes use of three other pre-existing python modules 'tempfile', 'shutil' (shell utilities), and 'os' (for command-line tasks).

A toy example usage

To test out replaceutil, let's use the UNIX command-line to quickly create a short text file (temp.txt) we would like to have replaceutil edit:

$ echo "I like amath." > temp.txt

We can check that this was done correctly by:

$ cat temp.txt
I like amath.

Now (in the same directory) let's create our own python script that will import replaceutil.py and modify the file temp.txt. Let's call it testreplace.py, and it's code will be as follows:

import replaceutil as ru

myfile = "temp.txt"
pattern = "amath"
subst = "cabbage"
    
print "Replacing instances of", pattern, "with", subst, "in", myfile, "..."
ru.replace(myfile,pattern,subst)
print "Done."

We can run testreplace.py from the command-line via

$ python testreplace.py

Replacing instances of amath with cabbage in temp.txt ...
Done.


Alternatively, we could have run it as an executable ./testreplace.py by inserting the bang #!/usr/bin/python at the top of testreplace.py and switching on the executable bit using chmod a+x testreplace.py.

To check that testreplace.py did what we wanted it do, let's check the contents of temp.txt:

$ cat temp.txt

I like cabbage.

And we see that the word "amath" has been replaced with "cabbage", as desired.

(TODO: Show a more practical application of this for e.g., automated parameter-sweeps in model runs)