Upstart notes

Linux-related notes Linux user notes Shell, admin, and both: Shell - command line and bash notes · shell login - profiles and scripts ·· find and xargs and parallel · screen and tmux ·· Shell and process nitty gritty ·· Isolating shell environments ·· Shell flow control notes Linux admin - disk and filesystem · Linux networking · Init systems and service management (upstart notes, systemd notes) · users and permissions · Debugging · security enhanced linux · PAM notes · health and statistics · Machine Check Events · kernel modules · YP notes · unsorted and muck · Automounters and permissions Logging and graphing - Logging · RRDtool and munin notes Network admin - Firewalling and other packet stuff · Remote desktops

Intro

Upstart was developed for ubuntu, as a more efficient and flexible alternative to (mostly) the above.

It adds things like on demand starting and restarting if the process dies,

Short story:

• /etc/init/name.conf will get picked up

used automatically as their 'start on' stanzas control

and/or manually by you

will put a log in /var/log/upstart/name.log

(errors may go to syslog, though(verify))

See also:

• http://upstart.ubuntu.com/cookbook/#design-history - for some reasons behind upstart
• http://upstart.ubuntu.com/cookbook/

introduction by example

Two existing examples:

description     "deferred execution scheduler"

start on runlevel [2345]
stop on runlevel [!2345]

expect fork
respawn

exec atd

description "SMB/CIFS File Server"

start on (local-filesystems and net-device-up)
stop on runlevel [!2345]

respawn

pre-start script
        RUN_MODE="daemons"
        [ -r /etc/default/samba ] && . /etc/default/samba
        [ "$RUN_MODE" = inetd ] && { stop; exit 0; }
        install -o root -g root -m 755 -d /var/run/samba
end script

exec smbd -F

Notes:

• expect fork (and expect daemon) is about single-forking and double-forking processes
• Instead of exec, you can specify a few lines of script
• you can specify things to do before and afterwards (also often script-style)

I had at one point written:

description "Temporary autossh-like tunnel for remote access"
author "Me"

start on (local-filesystems and net-device-up IFACE=eth0)
stop on runlevel [016]

setuid worker
setgid worker

# respawn when it disconnects - indefinitely, but not fast, so it doesn't bother things too much
# (note: may still trigger denyhosts/fail2ban)
respawn
respawn limit 0 60

exec ssh -t -R 2222:localhost:22 -t -o "BatchMode=yes" -o "ExitOnForwardFailure=yes" tunneler@sshproxy.example.com

(Note: It sometimes failed to reconnect, I never checked out why)

More technical conf notes

✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

What to run

Must have either

• exec line or
• a script section (run via /bin/sh)

May also have pre-start, post-start, pre-stop, post-stop (exec or script)

Telling when upstart should start and stop

Examples:

start on startup
start on runlevel [23]
start on runlevel [2345]
start on stopped rcS
start on started tty1
start on net-device-up IFACE=eth0
start on net-device-up IFACE!=lo
start on starting mountall
start on filesystem
start on local-filesystems
start on virtual-filesystems or static-network-up
start on (starting network-interface
          or starting network-manager
          or starting networking)
stop on runlevel [!2345]
stop on runlevel [!23]
stop on runlevel [06]
stop on (stopped network-interface JOB=$JOB INTERFACE=$INTERFACE
         or stopped network-manager JOB=$JOB
         or stopped networking JOB=$JOB)

Notes:

• static-network-up
  • emitted by the /etc/network/if-up.d/upstart
  • when every interface configured as 'auto' in /etc/network/interfaces is up

(which in some cases is not what you want)

• see also man upstart-events

Unsorted

See also:

• http://upstart.ubuntu.com/getting-started.html
• http://upstart.ubuntu.com/wiki/Stanzas

Where output goes

• console log (default(verify)) - stdin to /dev/null, stdout and stderr to /var/log/upstart/name.log
• console output - stdin, stdout, and stderr connected to a console
• console owner - like console output, but the process becomes owner of the console
• console none - everything to /dev/null

logging

loglevels are:

debug

info

message (default)

warn

error

fatal

So to see more you may want:

initctl log-priority info   # or maybe debug

These are basically the active filter on what gets stored in the logs(verify) so have no effect on earlier logging (also, to get this at boot, add --verbose or --debug to the kernel options[1])

See also: http://upstart.ubuntu.com/cookbook/#initctl-log-priority

debugging upstart jobs

✎ This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

Check whether it's loaded (meaning it'll have a status)

initctl list | grep myprogname

It'll generally say

myprogname stop/waiting

or

myprogname start/running, process 179723

If it's not loaded, you could do

init-checkconfig /etc/init/myprogname.conf

If you don't have init-checkconfig, you probably have an ancient upstart (e.g. RHEL6)

If you want the

exec 2>>/var/log/upstart.log
set -x   # also adds the commands being run, generally useful debugging bash scripts

If you need that in boot before rootfs is present, you may need to do something like:

exec 2>>/dev/.initramfs/myjob.log

See also:

• https://blog.thesparktree.com/debugging-upstart-jobs-on-ubuntu-1404

• http://upstart.ubuntu.com/wiki/Debugging