Cron Jobs with Bash: Scheduled Automation for DevOps

Cron jobs in Bash for DevOps

In the previous article we managed variables, .env files, CLI arguments, and special Bash variables. Once a script works reliably when you run it manually, the next question is: how do you make it run at the right time automatically?

cron is the classic Linux/Unix tool for running commands or scripts on a schedule. In DevOps, cron is often used for periodic backups, log cleanup, health checks, file sync, reports, SSL certificate checks, or lightweight maintenance endpoints.

The important detail is that cron runs with a much smaller environment than your interactive terminal. A script that works manually may still fail in cron if it depends on PATH, the current directory, environment variables, or clear logging.

What are cron and crontab?

cron is a background daemon that reads schedule tables and runs commands when their time matches. Those schedule tables are commonly called crontab files.

Basic commands:

1
2
3
crontab -l   # list the current user's crontab
crontab -e   # edit the current user's crontab
crontab -r   # remove the current user's crontab

You usually edit your own schedule with crontab -e. Be careful with crontab -r on production servers because it removes all jobs for that user.

A simple cron entry:

1
*/5 * * * * /opt/scripts/check-health.sh >> /var/log/check-health.log 2>&1

This runs the script every five minutes and appends both stdout and stderr to a log file.

The five-field schedule syntax

A common cron job has five time fields followed by the command:

1
2
3
4
5
6
7
* * * * * command-to-run
│ │ │ │ │
│ │ │ │ └── day of week: 0-7 (0 or 7 usually means Sunday)
│ │ │ └──── month: 1-12
│ │ └────── day of month: 1-31
│ └──────── hour: 0-23
└────────── minute: 0-59

Common examples:

1
2
3
4
5
6
* * * * *      # every minute
*/5 * * * *    # every 5 minutes
0 * * * *      # at the start of every hour
30 2 * * *     # every day at 02:30
0 3 * * 0      # every Sunday at 03:00
0 1 1 * *      # at 01:00 on the first day of every month

Cron also supports lists, ranges, and step values:

1
2
0 9,17 * * 1-5     # 09:00 and 17:00 from Monday to Friday
*/10 8-18 * * *    # every 10 minutes from 08:00 through 18:59

For production schedules, add a comment above each job so the next operator understands why it exists.

Shortcuts: `@daily`, `@hourly`, `@reboot`

Many cron implementations support shortcuts that make crontab easier to read:

1
2
3
4
5
@hourly   /opt/scripts/hourly-report.sh
@daily    /opt/scripts/backup-db.sh
@weekly   /opt/scripts/cleanup-old-logs.sh
@monthly  /opt/scripts/monthly-report.sh
@reboot   /opt/scripts/start-worker.sh

@daily usually means once per day at midnight. @reboot runs when the cron daemon starts and is useful for small boot-time tasks.

Do not overuse @reboot for important long-running services. A systemd service is usually better for daemons because it provides restart policies, dependencies, and clear logs through journalctl.

Write cron-friendly Bash scripts

A script executed by cron should prepare its own environment instead of depending on an interactive shell.

A good starter structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#!/usr/bin/env bash
set -euo pipefail

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
LOG_DIR="/var/log/example-app"

mkdir -p "${LOG_DIR}"

log_info() {
  printf '%s [INFO] %s\n' "$(date -Is)" "$*"
}

log_error() {
  printf '%s [ERROR] %s\n' "$(date -Is)" "$*" >&2
}

cd "${SCRIPT_DIR}"
log_info "Script started"

Key points:

Use a clear shebang.
Enable set -euo pipefail so failures are not silently ignored.
Define PATH in the script or in crontab.
Use absolute paths for important files.
Log with timestamps.
Do not assume cron runs from your project directory.

Redirect logs in cron

Cron can send output through local mail if the system is configured for it. In real DevOps work, writing to log files is often easier to control.

Redirect stdout and stderr:

1
*/5 * * * * /opt/scripts/check-health.sh >> /var/log/check-health.log 2>&1

Meaning:

>> /var/log/check-health.log: append stdout to the file.
2>&1: send stderr to the same destination as stdout.

If you want separate error logs:

1
*/5 * * * * /opt/scripts/check-health.sh >> /var/log/check-health.out 2>> /var/log/check-health.err

For frequent jobs, make sure you rotate or limit logs. Otherwise a log file can eventually fill the disk.

Check cron logs

Cron log locations depend on the distribution and logging setup.

On many Ubuntu/Debian systems using syslog:

1
grep CRON /var/log/syslog

On systems using the systemd journal:

1
2
journalctl -u cron
journalctl -u crond

Some distributions name the service crond instead of cron. If you are not sure, check both:

1
2
systemctl status cron
systemctl status crond

Remember that cron logs usually tell you whether cron started the command. They do not prove that the command completed successfully. For that, redirect output or write logs inside the script.

`PATH` and environment problems in cron

Cron does not load your full .bashrc, .profile, or terminal environment. This is a common reason why a script works manually but fails in cron with command not found.

Define PATH at the top of crontab:

1
2
3
4
SHELL=/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

*/5 * * * * /opt/scripts/check-health.sh >> /var/log/check-health.log 2>&1

Or use absolute paths in the script:

1
/usr/bin/curl --fail --silent --show-error https://example.com/health

For app-specific variables, prefer loading a config file you control:

1
2
3
4
5
set -a
source /etc/example-app/backup.env
set +a

: "${DATABASE_URL:?DATABASE_URL is required}"

Avoid putting secrets directly in crontab if multiple people can read crontabs or system backups. For important secrets, use a secret manager, a tightly permissioned file, or the secret mechanism of your platform.

DevOps practice: nightly database backup

The example below shows a nightly PostgreSQL backup script. It loads variables from a config file, creates a backup directory, compresses the dump, and deletes old backups.

File /opt/scripts/backup-db.sh:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
#!/usr/bin/env bash
set -euo pipefail

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
ENV_FILE="/etc/example-app/backup.env"
BACKUP_DIR="/var/backups/example-app"
RETENTION_DAYS="${RETENTION_DAYS:-7}"

log() {
  printf '%s [%s] %s\n' "$(date -Is)" "$1" "$2"
}

if [[ ! -r "${ENV_FILE}" ]]; then
  log "ERROR" "Cannot read env file: ${ENV_FILE}" >&2
  exit 1
fi

set -a
source "${ENV_FILE}"
set +a

: "${DATABASE_NAME:?DATABASE_NAME is required}"
: "${DATABASE_USER:?DATABASE_USER is required}"
: "${DATABASE_HOST:?DATABASE_HOST is required}"

mkdir -p "${BACKUP_DIR}"

backup_file="${BACKUP_DIR}/${DATABASE_NAME}-$(date +%Y%m%d%H%M%S).sql.gz"

log "INFO" "Starting backup to ${backup_file}"

pg_dump \
  --host "${DATABASE_HOST}" \
  --username "${DATABASE_USER}" \
  --dbname "${DATABASE_NAME}" \
  --format plain \
  --no-owner \
  | gzip > "${backup_file}"

find "${BACKUP_DIR}" -type f -name "${DATABASE_NAME}-*.sql.gz" -mtime +"${RETENTION_DAYS}" -delete

log "INFO" "Backup completed"

File /etc/example-app/backup.env:

1
2
3
4
5
DATABASE_NAME=blog_app
DATABASE_USER=backup_user
DATABASE_HOST=db-01.example.com
PGPASSWORD=<database-password>
RETENTION_DAYS=14

Set strict permissions:

1
2
3
sudo chown root:root /etc/example-app/backup.env
sudo chmod 600 /etc/example-app/backup.env
sudo chmod 700 /opt/scripts/backup-db.sh

Add a cron job that runs every day at 02:30:

1
30 2 * * * /opt/scripts/backup-db.sh >> /var/log/backup-db.log 2>&1

In a real environment, test restores regularly. A backup is only useful when you know it can be restored.

DevOps practice: healthcheck every five minutes

This script checks a /health endpoint, retries a few times, and writes logs. If the endpoint still fails, the script exits with a non-zero code so the failure appears in logs.

File /opt/scripts/check-health.sh:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/usr/bin/env bash
set -euo pipefail

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
HEALTH_URL="${HEALTH_URL:-https://example.com/health}"
MAX_ATTEMPTS="${MAX_ATTEMPTS:-3}"
DELAY_SECONDS="${DELAY_SECONDS:-5}"

log() {
  printf '%s [%s] %s\n' "$(date -Is)" "$1" "$2"
}

attempt=1

while (( attempt <= MAX_ATTEMPTS )); do
  if curl --fail --silent --show-error --max-time 10 "${HEALTH_URL}" > /dev/null; then
    log "INFO" "Healthcheck OK: ${HEALTH_URL}"
    exit 0
  fi

  log "WARN" "Healthcheck attempt ${attempt}/${MAX_ATTEMPTS} failed"

  if (( attempt == MAX_ATTEMPTS )); then
    log "ERROR" "Healthcheck failed after ${MAX_ATTEMPTS} attempts" >&2
    exit 1
  fi

  sleep "${DELAY_SECONDS}"
  ((attempt++))
done

Crontab:

1
*/5 * * * * HEALTH_URL=https://example.com/health /opt/scripts/check-health.sh >> /var/log/check-health.log 2>&1

An inline environment variable is fine for non-sensitive values. If you need a token or secret header, load it from a tightly permissioned file or a secret manager instead of putting it directly in crontab.

Avoid overlapping jobs

If a job runs longer than its schedule interval, the next run can start before the previous run finishes. This is risky for backups, cleanup tasks, and deployments.

A common solution is flock:

1
*/5 * * * * flock -n /tmp/check-health.lock /opt/scripts/check-health.sh >> /var/log/check-health.log 2>&1

flock -n does not wait for the lock. If the previous job is still running, the new one exits immediately. For backup:

1
30 2 * * * flock -n /tmp/backup-db.lock /opt/scripts/backup-db.sh >> /var/log/backup-db.log 2>&1

Not every system has flock installed, but on many Linux distributions it comes from util-linux. If your environment does not have it, use another locking mechanism and handle stale locks carefully.

Common mistakes

Forgetting log redirection: The job fails but you have no output to debug.
Depending on the working directory: Cron does not guarantee your project directory.
Missing PATH: Commands such as docker, kubectl, node, or python may not be found.
Using relative paths: Config, log, and backup files should use absolute paths.
Allowing overlapping runs: Backup or cleanup jobs may run concurrently and damage data.
Putting secrets directly in crontab: This is hard to manage, audit, and protect.
Not testing the command before saving crontab: Run the script as the same user that cron will use.

Implementation notes

When applying this to your project, standardize each cron job with a small checklist:
- The script has a shebang, set -euo pipefail, and absolute paths.
- The crontab includes a comment explaining the job.
- Output is redirected to a log file or centralized logging system.
- Long-running jobs use a lock to avoid overlap.
- Secrets live in an appropriate place, not hardcoded in public scripts.
Best practices:
- Test the script manually as the correct user: sudo -u <app-user> /opt/scripts/job.sh.
- Include timestamps in logs for incident analysis.
- Define PATH clearly in the script or crontab.
- Use flock for jobs that may run longer than expected.
- Watch log and backup size with logrotate or a retention policy.
Troubleshooting:
- Job not running? → Check crontab -l, schedule syntax, timezone, and cron logs.
- Job runs but the command fails? → Check the script’s own log file.
- Works manually but fails in cron? → Check PATH, working directory, permissions, and environment variables.

🎯 Conclusion

Cron turns Bash scripts into scheduled operational automation: nightly backups, periodic health checks, log cleanup, and reports. To make cron jobs reliable, write scripts that control their own environment, use absolute paths, log clearly, and prevent overlapping runs when jobs may take longer than expected.

In the next article, we will debug Bash scripts with bash -x, bash -n, set -x, set -e, set -u, pipefail, trap ERR, PS4, and shellcheck so problems become easier to find. 🚀

References

man7.org — crontab(5) — crontab syntax, the five time fields, and cron environment variables.
man7.org — cron(8) — cron daemon behavior and schedule processing.
man7.org — flock(1) — using flock to avoid overlapping jobs.
PostgreSQL Documentation — pg_dump — official documentation for pg_dump used in the backup example.
curl Documentation — reference for --fail, --silent, --show-error, and --max-time used in the healthcheck example.