process a text file line by line 

  
while read -r line; do
    echo $line
done <"filename.txt"

crop image files on the command line 

  
convert -crop +<x>+<y> <sourcefile> <outfile> # crop x pixels from left, y pixels from top
convert -crop -<x>-<y> <sourcefile> <outfile> # crop x pixels from right, y pixels from bottom
convert <sourcefile> -crop <w>x<h>+<x>+<y> <outfile> # copy out a w*h rectangle (using the letter 'x' instead of '*') starting with top left corner (x, y)

rotate image files on the command line 

  
convert input.jpg -rotate 90 output.jpg
find . -maxdepth 1 -type f -name "*.jpg" -exec convert {} -rotate 90 {} \; # bonus - rotate all jpgs in current folder

extract PDF pages as image files 

pdftoppm -png -f <first page to extract> -l <last page to extract> <source file>.pdf <output files common prefix>

extract images from PDF 

pdfimages <sourcefile>.pdf <basename for all exported images> -png -p

View the label of an ext4 partition 

sudo e2label /dev/sdb1/

Relabel an ext4 partition 

sudo e2label /dev/sdb1/ backup-a

Initial copy to a backup partition (non-system backup, just my data) / Poor man's rsync
(Don't use rsync for the initial placement of data, it slows down as it goes) 

 
nohup cp -upr /archive/* /media/sean/backup-a/archive
u = update: only copy when destination is older or does not exists
p = preserve metadata
r = recurse

"if... and ..." 

  
if [[ condition 1 ]] && [[ condition 2 ]]; then

"if... or ..." 

  
if [[ condition 1 ]] || [[ condition 2 ]]; then

"declare" - declare a variable as a type, and/or get var info

declare -p myvar # get all kinds of info about the variable declare -a myvar # indexed array declare -A myvar # hash table

Read a file into an array. "-t" means remove trailing delimiter (newline) 

  
mapfile -t myArray < sourcefile.txt

Print out all array values 

  
echo "${myArray[@]}"

Loop through an array. 

  
for i in "${array[@]}"; do
    echo $i
done

Test that an element exists in an array. Just loop through the array. Apparently, there is no built-in array member function for this 

  
for i in "${array[@]}"; do
    if [[ "$i" == "$yourValue" ]]; then
        echo "Found"
        break 
    fi
done

Hash tables 

  
declare -A animals
animals['cow']='moo'
animals["dog"]="arf"
echo "${animals["cow"]}"

Hash tables, populate from file 

  
TODO

Get the length of a string 

  
echo ${#mystring}

Increment an int in bash 

  
n="1"
((n++))

Validate correct number of arguments 

if [ $# -ne "1" ]
then
    echo "usage myscript <arg1>"
    exit
fi

Validate allowable values for argument 

  
case $1 in
"value1")    
	SCRIPTVAR="value1 or some conversion to it"
	;;
"value2")    
	SCRIPTVAR="value2 or some conversion to it"
	;;
*)
    echo "usage: myscript value1|value2"
	exit
esac

Prevent script from running as root 

  
if [ "$(id -u)" -eq 0  ]; then
    echo "Do not run this script as super-user. Allow the sudos in this script to determine which of the commands should be run as su, and which should be run as sean."
    exit 1
fi

Strip lines from a file which match a string. Notice that the wildcards go outside the quotes 

  
file="xyz.html"
while read -r line; do
    if [[ $line != *"dont want this line"* ]]; then
        echo $line > "newfile.html"
    fi
done <$file

Rename files whose names contain a string to filter by 

  
if [ $# -ne 1 ]; then
    echo "Usage: rename-xyz.sh <filter>"
    exit
fi

# Do not quote *$1* when constructing the "for" loop.
# This works for file names with spaces as it is.
# If you quote, then $F is one long string consisting of
# all matching file names concatenated.

for f in *$1*; do
    # newname=$( echo $f | sed 's/[A-Z]/\L&/' ) # rename first char to lower
    # newname=$( echo $f | sed 's/ \(.\)/\U\1/g' ) # camel-ize a space-separated string (the dot in the escaped parans is a capture group, so we can upper-case it as \1)
    # newname=$( echo $f | sed 's/[A-Z]/\L&/g' ) # make the whole name lower case
    if [ "$newname" != "$f" ]; then
        mv "$f" "$newname"
    fi
done

"convert" image formats, part of the ImageMagick suite of tools, example (I'm not sure what a good value for the quality arg is) 

  
for sourcefile in *.tif*; do
    basename="${sourcefile%.*}"
    extension="${sourcefile##*.}"
    convert "$sourcefile" -compress JPEG -quality 50 "${basename}.jpg"
done

get all files whose base name contains only letters 

  
find . -type f -regex '\.\/[a-zA-Z]*\..*'

Split a master flac file into separate tracks using a cue file
Although the flac and cue file are always(?) provided with the same base name, I don't think it's nesessary. What's necessary is for the contents of the cue file to reference the name of the flac file correctly. 

  
cuebreakpoints file.cue | shnsplit -o flac file.flac

wav to flac. The easy and intuitive "flac" command failed on some wav files from Gamazda - didn't like the flac version or something. This is almost as simple and worked for those wav files. 

  
for i in *.wav; do 
    ffmpeg -i "$i" -c:a flac "${i%.*}.flac"
done

Take action based on the first character of a string 

  
c1=${name:0:1}
if [[ $c1 < "D"  ]]; then
    echo "update hubris.pdfs set link = 'http://www.swordsnwizardry.com/.../all-monsters/monster-details-a-through-c#TOC-$name' where code = 'sw' and creature = '$critter';"
elif [[ $c1 = "D" ]]; then
    echo "update hubris.pdfs set link = 'http://www.swordsnwizardry.com/.../all-monsters/monster-details-d#TOC-$name' where code = 'sw' and creature = '$critter';"
...
    ...   
else
    echo "update hubris.pdfs set link = 'http://www.swordsnwizardry.com/.../all-monsters/monster-details-t-through-z#TOC-$name' where code = 'sw' and creature = '$critter';"
fi

Expand critter code (camel format) to probable critter name, capitalized 

  
for f in *"_mycode.pdf"; do
    # remove _mycode.pdf 
    name=$(echo $f | sed 's/_.*//')
    # place a space in front of all upper-case chars
    name=$(echo $name | sed 's/[A-Z]/ &/g')
    # make the first char upper-case
    name=$(echo $name | sed 's/[a-z]/\U&/')
done

Use perl to replace all links in a string with just the link text -OR- see the next block for easy removel of all tags

Can't use sed because it is always greedy. Adding a "?" after every wildcard group, meaning ".*", keeps perl from making greedy matches.

In the case I used this for, there were sometimes a class name between "<a" and "href=", which is why there is a wildcard group between them

I believe the "-pe" options mean "print" and "execute". The order matters. "-ep" does not work. 

  
strippedhref=$(echo $line | perl -pe 's|<a.*?href.*?>(.*?)</a>|\1|g')

... OR remove all html tags

I believe the "-pe" options mean "print" and "execute". The order matters. "-ep" does not work. 

  
line=$(echo $line | perl -pe 's|<.*?>||g')

Prep a string to be used as an sql statement by escaping single quotes/apostrophes 

  
sql=$(echo $sql | sed "s/'/\\\\'/g")

Given a long string, but not too long, iterate/loop through it based on a delimiter. This cut out after 212 array elements were read. The weird spacing of the less-thans is due to reading in from a file. If reading in from the string directly,
I think you would use the "here" string construct, which is triple less-thans 

  
IFS='@' read -r -a myArray < <(cat fileConsistingOfOneLongString.txt)

Given one very long string, break into/process substrings with specified delimiter '@'
If the string orignally has a multi-char delimiter, use vim or something to replace it with a single-char delimiter 

  
IFS='@'
html=$(cat $f)
array=( $html )
for line in "${array[@]}"; do
    echo $line
done

send contents of an sql file to the mysql server 

  
mysql -u sean -pbatman Valor < hb.sql

fetch and process from mysql 

  
export MYSQL_PWD=batman
mysql -sN -u sean --execute="SELECT issue FROM Comics.Spiderman;" | while read myissue; do
    echo "$myissue"
done

copy all files in subfolders of a given extension to a destination folder 

  
find . -name \*.html -exec cp {} destinationfolder \;

Extract pages from a pdf file. 2025 version. I don't recall needing the '.' before, but there it is.
In on-line examples, some repeat the full name of the infile in place of the dot. 

  
qpdf infile.pdf --pages . 3-7 -- outfile.pdf