Keep.sh: Difference between revisions
(Created page with '== Cleaning up model output with keep.sh == After doing a large model run and the analysis of all the output files is complete, it is often a good idea to delete output files to…') |
|||
Line 1: | Line 1: | ||
== Cleaning up model output with keep.sh == | == Cleaning up model output with keep.sh == | ||
After doing a large model run and the analysis of all the output files is complete, it is often a good idea to delete output files to free up hard drive space on the machine. However, deleting all the data files is undesirable since the model would need to be re- | After doing a large model run and the analysis of all the output files is complete, it is often a good idea to delete output files to free up hard drive space on the machine. However, deleting all the data files is undesirable since the model would need to be re-run to reproduce all of the results leading up to the final output time. As a compromise, you may consider keeping every Nth output file, while deleting the rest. This task can be automated, as explained below. | ||
You can use the shell script '''keep.sh''' which should be placed in your '''$HOME/bin'''. You also need to make the script executable, use the command: '''chmod 755 keep.sh'''. The code for keep.sh follows: | You can use the shell script '''keep.sh''' which should be placed in your '''$HOME/bin'''. You also need to make the script executable, use the command: '''chmod 755 keep.sh'''. The code for keep.sh follows: | ||
Line 38: | Line 38: | ||
echo done. | echo done. | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== Example Usage == | == Example Usage == |
Revision as of 12:48, 13 October 2012
Cleaning up model output with keep.sh
After doing a large model run and the analysis of all the output files is complete, it is often a good idea to delete output files to free up hard drive space on the machine. However, deleting all the data files is undesirable since the model would need to be re-run to reproduce all of the results leading up to the final output time. As a compromise, you may consider keeping every Nth output file, while deleting the rest. This task can be automated, as explained below.
You can use the shell script keep.sh which should be placed in your $HOME/bin. You also need to make the script executable, use the command: chmod 755 keep.sh. The code for keep.sh follows:
#!/bin/bash
#Usage: keep <pattern> <N>
#Purpose: keep every Nth file with name matching <pattern>
pattern=$1
N=$2
FILES=$pattern
count=0
for f in $FILES
do
if [ $count -lt $N ]
then
if [ $count -ne 0 ]
then
rm $f
fi
fi
if [ $count -eq $N ]
then
count=0
fi
count=$(($count+1))
done
echo done.
Example Usage
The command keep.sh 'out*.mat' 10 would keep the files, "out010.mat, out020.mat", etc. if the original list was all files numbered from 001, 002, ... ,100.
Notes:
- We have to enclose the pattern in single-quotes (') so that bash will properly parse the wild-card string (*).
- Having zero-padded output files is recommended to avoid sequence loss.