Quick Tip: Run Bash Commands in Parallel

I have a useful script that demonstrates a few different bash techniques.

This script:

Executes multiple commands on multiple SSH client hosts
Connects to each host in parallel, and waits for all host processes to terminate
As a freebie, demonstrates running apt in a non-interactive process

Additionally, it shows some bash features:

Job control
Array variables
and a few more

Here’s the script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39


#!/usr/bin/env bash

# All command line arguments are treated as host names. If no arguments
# are given, use localhost
remote_hosts="${*-localhost}"

# Set remote_command to the heredoc; quote the marker 'EOF' to prevent
# bash from evaluating the contents while we read it into the variable
read -r -d '' remote_command <<'EOF'
sudo bash -c '
  export DEBIAN_FRONTEND=noninteractive
  apt-get -qqy update
  apt-get -qqy dist-upgrade
  apt-get -qqy autoremove --purge
  apt-get -qqy autoclean
';
[ -f "/run/reboot-required" ] && echo "REBOOT required on $HOSTNAME";
EOF

echo "Updating: ${remote_hosts}"

# Make job control messages less verbose
set +m

# Declare pids as a bash array
pids=()
for rhost in ${remote_hosts}; do
  {
    ssh -q -t "${rhost}" "${remote_command}" 
    echo "Done ${rhost}"
  } &
  # append the pid of the pipeline job above to the list of pids
  pids+=($!)
done

# wait for all pids representing the jobs to finish
for pid in ${pids[*]}; do
  wait "$pid"
done

Quick Tip: Run Bash Commands in Parallel

- Wednesday, May 18, 2022