Bash Parameter Expansion

Part of the book: Bash: The Linux Command Line

We've seen environment variables, these are really just an arbitrary group of variables that have special meaning to shells or commands. We can use variables for our own personal use too. Typically variables are used in shell scripts or in complex shell commands and can be substituted into a command line by prefixing the variable name with a dollar sign ($.) This is also called Parameter Expansion.

There are interesting ways to assign variables as well as substituting them. You've seen the typical assignment and substitution:

filename="firecracker coleslaw recipe.txt"
echo "$filename"

Beyond simple substitution we can use syntax that will modify the variables value for the substitution and optionally modify the variable at the same time. Additionally it's possible to cause a shell command or script to fail if a variable is not set.

For my first example I'll be using the for command we will rename files en mass by changing the file extension from .TXT to .txt.

First I'll show you a simple loop that uses a file glob to select files and uses the for command to assign each file name to a variable one at a time, looping for each file name, then we'll print it just to show it works:

for filename in *.TXT
do
    echo "$filename"
done

If all goes well, and you have .TXT files in the current directory, you'll see a list of file names. Now we will want to use the mv command to rename the files, but do to so we need to give mv the destination name, a filename with a .txt extension. This is Linux so there are several ways to do this. We could use sed to do a text replace, but it's far easier to use a special Bash substitution syntax, that does the replace for us. We will use the ${varname/%.TXT$/.txt}. This command does a pattern match on the contents of varname and, because of the /% it matches trailing text so if the filename ends with .TXT it is replaced by .txt.

Here is the combined result:

for filename in *.TXT
do
    mv "$filename" "${filename/%.TXT/.txt}"
done

The above substitution command changes the substitution, not the variable contents so if we were to look at $filename after the mv command it would still show the .TXT extension.



There are many types of pattern substitution, some allowing replacement text and others simple removing matched text. Check the man page for a full listing of substitution commands. 

The pattern is the same as shell glob pattern matching and you can use variables as part of the pattern or the replacement text. Here is a more complex example:

#!/bin/bash

old="$1"
new="$2"
# Now do the replace
for filename in *$old
do
    mv "$filename" "${filename/%$old/$new}"
done

For the above commands to work it should be copied into a file (let's say rename.ext,) then the file should be run using bash rename.ext .from .to. The variables $1 and $2 are set to command line arguments 1 and 2. So to change .TXT to .txt use bash rename.ext .TXT .txt. If you omit the first line (#!/bin/bash) you can make a shell function from this too.
There are many types of special substitution commands that change how the variable is substituted and others that change the variable itself. Consider these two similar substitutions, they substitute the same value, but they differ in that one sets var and the other doesn't. These particular substitutions take place only when the variable is empty.
# Set var to nothing, this expands to a value, but doesn't set var.
var=""
echo ${var:-not set}
echo "the value of var=$var"

# try again using different expansion, this sets the value of var too.
var=""
echo ${var:=not set}
echo "the value of var=$var"

The above examples are very useful in setting variables to default values inside scripts if they are not set var="${var:-default}".
Or perhaps you want the length of the variables contents
var="hello world"
echo ${#var}

Or perhaps a sub string of a variable
# Print out the from letter 6 the following 5 chars.
var="hello world"
echo ${var:6:5}

# Print from letter 0 the next 5 chars
echo ${var:0:5}

There are many more variable substitution syntaxes to use. Some care if the variable is set, others just that it is empty (or unset)
Of the pattern matching syntaxes, some work on the beginning of a string, some on the end. Some match the most text, others the least. Some will substitute all occurrences of a pattern while others only substitute the first occurrence. I won't list them all here, check the man page (man bash and use the / command to search for Parameter Expansion.