Screen and tmux

From Helpful
Jump to navigation Jump to search
Linux-related notes
Linux user notes

Shell, admin, and both:

Shell - command line and bash notes · shell login - profiles and scripts ·· find and xargs and parallel · screen and tmux ·· Shell and process nitty gritty ·· Isolating shell environments ·· Shell flow control notes


Linux admin - disk and filesystem · Linux networking · Init systems and service management (upstart notes, systemd notes) · users and permissions · Debugging · security enhanced linux · PAM notes · health and statistics · Machine Check Events · kernel modules · YP notes · unsorted and muck


Logging and graphing - Logging · RRDtool and munin notes
Network admin - Firewalling and other packet stuff ·


Remote desktops



Note: For something similar for X11, see xpra

Intro

Why it's useful

When you open a shell (or more) inside tmux or screen session, you can disconnect from that screen/tmux session, the shell it will stick around inside that session, and you can reconnect to that session later.


Learn by doing:

  • Log into a remote host (ssh, putty)
  • Run tmux, or screen
  • You are now again at a terminal.
Do stuff with it.
  • Disconnect
there's a nice way of disconnecting
but you can also simulate a broken network connection e.g. by closing the terminal window you're currently using
  • Connect to the remote host again and run tmux att or screen -rd (whichever you were using)
Notice you are now looking at the same interactive terminal you had before the disconnect


Of these two programs,

screen is the better known and more likely to be installed. It's a fairly stagnated project now, though.
tmux is more actively developed, so has a few more features.


💤 More detailed info

Screen and tmux distinguish themselves from a regular shell in a few significant ways.


One of the most complex cases in too much detail.
Much of the point is that the things you are really running are only tied to that background session hosting process -- so when anything else breaks (e.g. the viewing terminal, which might be remote access as diagrammed above) it has no effect on them


Terminal multiplexing means they can host/view many process at a time.

It means one hosted session can contain many interactive shells side by side,

  • ...analogous to a tabbed web browser, though shown a little differently


They run processes separately from the shell you logged in with

You can think of screen/tmux as a background service that hosts interactive processes.

  • when you start starting screen/tmux, you creates that host process, plus an interactive viewer into it
  • Each 'window' you start within that service defaults to start its own shell (you get one, to get you started)


The viewer can switch between these windows

Doing so mainly amounts to connecting your the keyboard and output to different ones at different times.
When said viewer stops, it doesn't affect what it was looking at.



This is useful for anyone who who

  • runs long-term jobs,
  • wants to leave a text editor or other work open until tomorrow/later,
  • wants to work remotely but has a internet/WiFi connection that sometimes drops,

or whatnot.


💤 For contrast, the usual case without screen, tmux, or similar:

A program is a child of a shell, that shell is the child of either the graphical terminal, or sshd or such.

When that parent stops for any reason, all proceses under it gets cleaned up.

Additionally, because the other issue is that your process's stdin, stdout, and stderr (keyboard input, terminal output) are typically tied to the shell you started it in. When one of those streams gets disconnected, the program tends to not deal with that and crashes.


You can get some relief from nohup.

Instead of said process cleanup, the process is moved to be a child of init instead. This usually means you can no longer interact, though.

nohup can also handle stdin/stderr/stout by redirecting them elsehwere. Tying its input/output to a terminal again may even be possible, but a pain at best, so you don't really want this for interactive stuff.


tmux

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

For newcomers, you are probably most interested in:

  • starting a new session tmux (or tmux new)
  • creating new windows: Prefix-c
  • disconnecting from a session: Prefix-d
  • reconnecting to a session: tmux att

Closing/removing a window is often done by exiting the shell. (If something is hung in it, you can kill the window explicitly)


Prefix is Ctrlb by default, but you may want to change it to Ctrla to avoid confusion when you're on a host that only has screen. Because that will happen. (For the same reason, I like to rebind keys so that overlapping features have the same keys. See the example config below.)


Tmux when compared to screen

From those who know screen, the difference is mainly:

  • both has different key shortcuts
but both are configurable
my switch from screen to tmux went surprisingly smoothly once I rebound keys to imitate screen :) (see below)
  • tmux is a more active project than screen, so more likely to have/get features you want
  • tmux can be less bother when you use X forwarding
in that new windows will fetch DISPLAY according to the outside environment (of the most recent attach (verify))
  • tmux can be more flexible layout-wise
and the split+layout is remembered better
  • tmux can be more easily made to automatically rename windows according to what they are running
  • is a bit more modular and scriptable

Terminology

You, as a client, connect to a session (more than one client can connect to a session).

Multiple sessions can be useful, say, to split work and home use.

Within a client/session you look at one window at a time (named, fixed numbering, often switched to by index).

Windows can be split into panes if you think that useful (numbered by current position, can be cycled through).

Maybe you'll prefer putting distinct tasks into distinct windows (e.g. IRC, different coding projects), and put related bits of the one task in panes (e.g. editing code, running it, viewing the log).


Interaction I use most

There are plenty of cheat sheets out therem.
The way I use it most of the time:


tmux's Prefix is Ctrl-b by default, but most people (apparently including the tmux devs) seem to remap it to Ctrl-a so that your screen-related muscle memory works. (For the remapping mentioned here, see the config file example somewhere below)


starting, attaching, detaching

  • tmux, or tmux new starts a new session
  • tmux attach attach - if you have just one, you don't have to worry about the details
By default it will leave other connected clients attached (and use the smallest window size of the set, padding the rest). If you want it to detach other clients after attaching, the easiest way is probably tmux att -d
  • tmux ls - list sessions
  • Prefix-d Detach the current client (same as in screen)


scrollback

  • Prefix-[ - start navigating in tmux scrollback for the pane
You can then also do Ctrl-S and Ctrl-R (forward and backward search)


window

  • Prefix-c Create a new window (same as in screen)
  • Prefix-0 through Prefix-9 Move to window by number (same as in screen. I like to configure tmux so that 1 is the first window, not 0)
  • Prefix-w List windows (I like to remap this to Prefix-")
  • Prefix-l Move to the previously selected window (same as in screen. I like to add Prefix-Prefix to do the same)
  • Prefix-& Kill the current window (I like to remap this to Prefix-K)
  • Prefix-, Rename the current window (I like to remap this to Prefix-A)


Pane

  • Prefix-% Split into two panes, vertically. (I like to remap this to Prefix-| )
  • Prefix-" Split into two panes, horizontally (I like to remap this to Prefix-- )
  • Prefix-x Kill current pane (...but I typically just log out of its shell)
  • Navigate
    • I don't like the standard way (that I've seen)
    • I like to map Prefix-arrowkeys to do this (and only ever use that)
  • Relayout:
    • Prefix-space use next layout (easier), OR
    • Prefix-Alt-1 through Prefix-Alt-5 to use specific layouts. When I actually care to relayout, I seem to use 5 most.
  • Resize
    • I like to map Prefix-Alt-arrowkeys to do this


Session

  • Prefix-s - list sessions
  • Prefix-( - switch to user's previous session
  • Prefix-) - switch to user's next session


Misc

  • Prefix-? List all key bindings (same as in screen)


on tmux's copy mode

More on panes and windows

Example config (~/.tmux.conf)

Note that tmux is under active development. And some distros are a bunch of versions behind that,

so there are plenty of things that lead to heavy complains in other versions, and trial-and-erroring and "works for me". This can be messy.
(non-old versions report their version using tmux -V. Alternatively, look at your package management to see which version you have installed)


Config notes

  • for key binding commands
bind is the same as bind-key
-r means 'repeatable without repeating the prefix'
e.g. useful when using arrow keys for pane resizes
-n means 'does not need prefix'
probably rare because that makes the key generally unusable. I e.g. have F12 as a 'go to next pane': bind -n f12 select-pane -t :.+
Pressing Prefix ? shows a list of current keybinds


Config examples

# be screen-like by using Ctrl-a, not Ctrl-b, as Prefix
unbind C-b
set -g prefix C-a
bind C-a send-prefix

# announce in TERM that we are good with colors
set -g default-terminal "screen-256color"

# execute things from the config file, avoids having to restart to get a new config
# (note: not quite the same as a reload, because it leaves what it doesn't overwrite)
unbind r
bind r source-file ~/.tmux.conf

# more history than the default 2000. Can be unlimited, but you may not want that.
set -g history-limit 8192

# lock session after inactivity
set -g lock-command vlock
set -g lock-after-time 300   # Seconds; 0 = never
# I learned this habit imitaging windows/gnome's Win-L.  Overrides last-session binding
bind L lock-session    

# mouse used to have more separate options, was generalized around 2.1.
#   I used this mainly for navigation (there is more) 
#   but do not use it enough to merit extra behaviour
set -g mouse off

#set -g set-clipboard off



### WINDOW STUFF  ###########################################################################################

# things like running a shell command will rename a window  (unless/until we set one manually)
set -g set-titles on
set -g set-titles-string '#W,    #T'
bind A command-prompt "rename-window '%%'"
set -g automatic-rename on
set -g allow-rename off



### PANE STUFF  ###########################################################################################
# I can remember - and | as horizontal and vertical splits better than % and "
unbind %
bind | splitw -h
unbind '"'
bind - splitw -v
bind _ splitw -v

# Shift-arrows resizes panes in that direction
bind -r S-Left resize-pane -L 2
bind -r S-Right resize-pane -R 2
bind -r S-Down resize-pane -D 2
bind -r S-Up resize-pane -U 2


# highlight current pane
# (tmux 3ish style, earlier was different)
set -g pane-active-border fg=green
set -ag pane-active-border bg=magenta



### NAVIGATION ###########################################################################################
# start numbering at 1, eases window switch by number (C-a 1, C-a 2)
set -g base-index 1


# toggle between last two windows
unbind ^A
bind ^A last-window

# choose a window visually
bind '"' choose-window


# Prefix arrows  moves through panes in that direction  (arrows repeatable)
bind -r Left select-pane -L
bind -r Right select-pane -R
bind -r Down select-pane -D
bind -r Up select-pane -U


## Experiments
# cycle through panes
bind -n f12 select-pane   -t :.+
# cycle through windows
bind -n f10 select-window -n
# cycle through sessions   -   note there's always ( and )
bind -n f8  switch-client -n


Special values in titles include:

#(shell-command)  First line of the command's output
#H                local hostname
#F                Current window flag                     
#I                Current window index                    
#P                Current pane index                      
#S                Session name
#T                Current window title (a shell thing? typically user@host:path ?)    
#W                Current window name  (tmux's own, the one shown in the bottom bar)
##                A literal ‘#’
#[attributes]     Colour or attribute change

You can also use strftime variables. Keep in mind those start with a percent sign instead.

The color/attribute changes look like:

#[nobright fg=green]

When doing complex things, it may help to know that #[] is interpreted after all other # sequences.



See also:


Color based on host

That is, have the bottom bar colored according to a hash of the hostname.

When administering many hosts, this can help lessen confusion of which host you're on.

See e.g. the solutions at http://unix.stackexchange.com/questions/122508/set-tmux-status-line-color-based-on-hostname

On status lines and titles

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


For tmux

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

What is shown on-screen is configurable (and not exposed anywhere - it's not a hardstatus at all)

comes from multiple places, including window titles and configured things
  • status-interval - how often to update
  • status-left (default [#S])
  • status-right (default is current window title, date, time)
  • ...and a whole bunch more, mainly controlling what to put in there, and its formatting


What it exposed in its own status line is controlled via: (verify)

  • set-titles on (off by default)
(checks via termcap; uses tsl, fsl)
(does this set tmux window title, or its terminal hard)
  • set-titles-string

The interesting part is the difference between contained-term status and window names (and pane titles).


Primarily that each window (though you'ld usually control this globally) has settings:

  • automatic-rename (added in 1.6, default seems to be on(verify))
whether to rename a window automatically (using automatic-rename-format)
disabled after an explicit name is set (...at start via new-window or new-session, or later with rename-window)
  • automatic-rename-format (added in 1.9, default #{pane_current_cmd})
(in 1.6..1.8, current pane command was the only automatic-rename behaviour there was)
  • allow-rename (default was on in 1.6..2.6(verify), off in 2.7)
don't listen to escape sequence in contained terminal(verify)
(does it also disable automatic-rename? or imply it?)

(versions matter because most things are on older versions still, e.g. AOTW RHEL seems to be on 1.8, Ubuntu on 2.1)



If your tmux windows won't stop renaming

...even after you've manually set a name, it's not automatic-rename but your contained shell doing so.

In my case it was RHEL's /etc/bashrc trying to help screen (and the issue is arguably tmux pretending too to be a screen-type TERM) by setting:

PROMPT_COMMAND='printf "\033k%s@%s:%s\033\\" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/~}"'

The fix would be removing that.


oh-my-zsh can be doing this (can be disabled in it)

For screen

Semi-sorted (tmux)

More than ~eight colors

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

It looks like

  • Tmux plays conservative about assuming its environment supports more than the basic eight-or-so colors.
  • when it sees 256-color and true-color escapes it seems to actively rewrite them into more basic colors
  • you can get more colors by starting tmux with -2



Scripting tmux

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

via startup commands

command line arguments

You can manage existing sessions via command line arguments (list-sessions, list-windows, new-window, etc.)

It's a little verbose to do directly, perhaps, so you may prefers some code wrappers e.g. this python library.


Control mode

Control mode is tmux itself exposing a basic text-mode protocol, separate from the sessions it manages.

It is meant to let programs automate use of tmux, as well as be able to give you some (asynchronous) notifications.

It can also be used for better UI integration - see e.g. what iTerm2 does.


See also:

Tmux doesn't change COLUMNS / shell and programs draw incorrectly

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)


Environment

Screen

The manual calls screen "a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells)."


Usage basics

Windows

The distinct things within an instantiation of screen are called windows.

By default, each window starts with your default shell, so each window is useful for a distinct task, say, running and editor, running a program, viewing logs.

Window-related commands you may well use:

  • Creating a new one: Ctrl-a c
  • You can switch to a window by its number Ctrl-a number
  • You can toggle to/from the last used window with Ctrl-a Ctrl-a
  • You can get a list of windows with Ctrl-a "


You may like to have an on-screen list of windows. See configuration notes below (or look to byobu).


Instantiations of screen

starting, stopping

To start a screen, type screen. This starts a new screen process and creates a single window inside it, with a shell according to your user settings.

Most commands are Ctrl-a followed by something. For example, help is Ctrl-a, h

The usual way to stop a screen by stopping the processes from its last window (or, if you can't easily stop the contained process, kill that window - Ctrl-a, k).


Detach, reattach

You detach (disconnect) from a running screen instantiation with Ctrl-a, d.


Reattaching comes in quite a few flavours, though out of habit you'll probably remember just one or two convenient ones.

  • screen -list shows all the running screens for the current user. Makes sense when you run more than one.
  • reconnect commands can often be used with or without argument.
    • without tends to mean:
      • "If there is exactly one screen process, reconnect to it"
      • "If there are more, show a list"
    • with tends to mean "reconnect when there is one unique string-starting match"
      • For example, if you have 18720.pts-0.hostnm and 23972.pts-0.hostnm, then screen -r 23972 but also screen -r 23 will connect to the second.

If you don't want names like 5730.pts-0.hostnm, you can specify your own with -S


As to the different reconnect commands: (verify) (this needs clarification / double-checking)

  • -r for 'resume'. Won't connect to an already-attached screen (and won't create a screen)
  • -R - largely like like -r, but if no screen exists (or it is attached and there is no request to detach it), it will create one
  • -RR - basically "connect to the first (detached) screen you can find; if one isn't running, start a new screen."
  • -d - when connecting to a currently attached screen, this will disconnect the others first
  • -D - like -d, plus it causes logout from the shell the other connection came from.
    • Usually means your other remote login (terminal/emulator) will be closed or logged out, which is nice for peace of mind.
  • -x, which means 'join an attached screen without detaching others'
    • This means sharing both input and output, without any choice. You may want multiuser mode instead
    • (the smallest window size between the logins will be used)


These can be combined. There's more than a dozen combinations

  • -rd roughly means "connect to the last one and detach others"
  • -RD roughly means "-rd, detach+logout if it was attached, create it if necessary" - possibly the easiest do-what-I-want

Some more key commands

The key shortcuts I consider useful are:

  • Window stuff:
    • Ctrl-a 1 switches to window by number, 1 though 9.
    • Ctrl-a Ctrl-a switches between last activated windows
    • Ctrl-a " gives list of windows, also allows switching with arrows and enter
    • Ctrl-a c creates new window (removing one can be done with a logout, or:)
    • Ctrl-a K removes (kills) current window (useful for things you can't Ctrl-C)
  • Split screen:
    • Ctrl-a S (capital S) splits screen top-bottom
    • Ctrl-a | (pipe) splits screen left-right
    • Ctrl-a Ctrl-i switches to next part of split screen
    • Ctrl-a - makes current part of split screen smaller
    • Ctrl-a + (on US keyboards meaning Ctrl-a Shift-=) makes current part of split screen larger
  • Other:
    • Ctrl-a ? displays various keyboard commands
    • Ctrl-a d disconnects


Scrollback

Copy mode (which doubles as a scrollback viewer):

  • Use Ctrl-a [ to enter copy mode, Esc to exit.

In scrollback/copy mode, you can:

  • Navigate
    • vi-style line/column movement: h, j, k, l
    • page-based movement:
      • PgUp and PgDn often work
      • C-u, C-d: half page up, down
      • C-b, C-f: full page up, down
    • For more, see e.g. [1]
  • Search
    • using / (forward) and ? (backward)

Configuration

Line drawing and/or Unicode

Screen picks up on locale settings and should just work. If it doesn't, it usually means the locale settings do not handle what you want, or your terminal program is using some different interpretation.


Assuming you want to use UTF8 (and unless you specifically must have something else, you want UTF8), you should generally check two things:

  • that (your account at) the server has an UTF-8 based locale (check whether LANG and/or LC_ALL(verify)) are set this way.
  • that the terminal program interprets the server correctly. For example, PuTTY defaults to 8859-1 and not UTF8.


If LANG nor LC_ALL does not mention utf8, you probably have to set up your system first; see Localization,_internationalization#Linux_locale_setting

Alternatively, you can also force UTF8 treatment throughout screen by starting screen with the -U option, but this isn't always ideal (it ignores the context of the system it runs in).


A bunch of configuration possibles

You can create ~/.screenrc to have some settings, for example:

# hide the 'this is screen' message that appears when you start 
startup_message off

# allow more scrollback than 100
defscrolback 2000


# If you are primarily using remote text logins, you probably don't want 
# X programs assuming they can use DISPLAY, so clear it:
unsetenv DISPLAY


# remove some Ctrl-A <key> bindings that you only really use by mistake, 
#  e.g. lock screen, hardcopy, write termcap, perhaps others
bind ^x
bind h
bind .


Caption and hardstatus

This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)
# always show caption, not only when using split screen 
# (I like seeing that I'm using screen, and what host I'm on)
caption always
caption always "%H | %>%{kw}%?%-Lw%?%{wb}%n*%f %t%?(%u)%?%{kw}%?%+Lw%?%<"

The funky caption line is to show the list of all windows, with the current one highlighted -- because by default it will show only the current window number and name. Example result:

You can do much funkier things here - see for example byobu (formerly screen-profiles).



This article/section is a stub — probably a pile of half-sorted notes and is probably a first version, is not well-checked, so may have incorrect bits. (Feel free to ignore, or tell me)

Copied examples that show the window list:

# white bar, current window blue
caption always "%H | %>%{kw}%?%-Lw%?%{wb}%n*%f %t%?(%u)%?%{kw}%?%+Lw%?%<"

# transparent bar, current window inverse (subtler)
caption always "%>%{wk}%?%-Lw%?%{bw}%n*%f %t%?(%u)%?%{wk}%?%+Lw%?%<"



hardstatus

The hardstatus is a slightly more complex subject. It's roughly a separate string that can be updated via terminal control codes. Your terminal/emulator has to support it, as well as allow setting it.


You can tell various terminals and emulators (particularly graphical ones, e.g. PuTTY, the regular ones in *nices,) to show it. For example, you can alter your termcap with something like:

# advertize hardstatus support to terminal 
#   and use escapes to put it in the window title
termcapinfo xterm 'hs:ts=\E]2;:fs=\007:ds=\E]2;\007'
termcapinfo rxvt  'hs:ts=\E]2;:fs=\007:ds=\E]2;\007'

This can be handy when screen opts to set the title to just 'screen'


You can tell what screen should do when hardstatus support is not advertised (via termcap), one of:

hardstatus ignore       # if not supported by terminal, don't show
hardstatus lastline     # reserve last line in terminal
hardstatus message      # use whatever screen's message mechanism currently is

Prepending always to any of these (alwaysignore, alwayslastline, alwaysmessage) will force the option even if the terminal supports hardstatus. Just always seems to be shorthand for the always variant of the default (lastline?).


You can tell screen to set particular things in the hardstatus. For example:

# basic message to the hardstatus
hardstatus on            # if off, hardstatus messages are printed on screen (verify)
hardstatus string "%H [%n%?: %t%?] %h"  


The hardstatus string line tells screen how to use the hardsatus.

%h refers to the stored hardstatus (settable via "ESC]0;<string>^G" or "ESC_<string>ESC\")

Titles for screen's own windows

While you can set these manually (Ctrl-A Shift-A).

You can also set them via escapes, but the most convenient way for that to work is automatically.

If you want to also show things like the current directory, the place you probably want to send those escapes from is the prompt string.

See e.g.


Warnings

"Attaching from inside of screen?"

Note that if you see "Attaching from inside of screen?", you probably did a screen -rd (or so) from within screen.

It says this in the screen you tried to do it from - so if you did a -rd or such, you'll only see it once you return to the screen. (which may be hard to see unless you use a caption-like thing to help you see when you're in a screen session)


In the case where you (implicitly or explicitly) referred to the screen you were in, in which case the net effect is that you will be disconnected (you'll see [remote detached]), and you will see the "Attaching from inside of screen?" message once you (re)connect to that screen.



Semi-sorted (screen)

Flow control

Particularly emacs users might accidentally trigger flow control, since Ctrl-S is stop (Ctrl-Q is continue).

Can be enabled/disabled with Ctrl-a f. If that solves your problem, you can disable it for all new sessions via screen config: flow off and/or defflow off


screen and X tunneling

Screen will mess with SSH tunneling of X.

Once you reconnect to screen it will contain the DISPLAY inherited from the shell you started screen from, which will likely result in a:

cannot open display


The SSH client did set the new X tunnel in the connecting shell's DISPLAY, of course, so you'll have to update it manually. Basically:

# after connecting to the host, before reconnecting to screen, find the new SSH connection's DISPLAY value:
user@host:~ $ echo $DISPLAY
localhost:12.0

# connect to the screen and in the relevant shell(s) do something like:
export DISPLAY="localhost:12.0"


TODO:

  • figure out whether there is any more automated way to do this.
  • tmux will try to do this automatically

"Cannot open your terminal ‘/dev/pts/x’ – please check."

Probably means you used su to change users.


The problem is that the permissions on that pseudoterminal do not include you (even if you're root(verify)). An ls -l /dev/pts will probably show something like:

crw--w---- 1 myusername tty 136, 6 Apr 12 13:16 /dev/pts/6


To work around this, you can do:

chmod a+rw /dev/pts/6

It's insecure, but useful enough for, say, an admin checking up on screens before rebooting.



If you su'd to a user you want to actively use for a while, you may like workarounds:

See also

  • the man page (an online version here)
  • the user manual (like here).



On byobu

tl;dr: Byobu

  • is a wrapper to tmux and screen (defaults to tmux when available)
  • adds system information in the status bar (package, battery, cpu, memory, network, and more)


To check

https://github.com/spolu/warp