Friday September 25 2020

Effective Work Environments with Tmux and (optionally) Vim.

A number of years ago I read Tom Ryder’s series on Unix as an IDE. Which is a great read and starting point for anyone interested in this topic. It will be most useful to those who already spend much of their time in the terminal.

If you’ve spent any amount of time working with modern software frameworks like Django, Laravel, Rails, React, VueJS, Creating an HTTP application in Go or most any other modern software you know that it often involves spinning up a local server as well as a database–or worse–several.

If you’re in a corporate environment, often times the database(s) have been pieced together over years of last minute changes and poorly designed additions so there’s no easy story to spin up a local development environment.

So what’s the goal here? Well the goal should be able to set up everything you need to get started working straight away with a single command.

Tmux

Similar to the screen program that’s been around for years tmux is a more of a modern take on the terminal multiplexer with a bit more of a permissive liscense and IMO a more usable interface.

The long story short is that it lets you monitor a long running program detach, and reattach from a new SSH session, have multiple people attached at once viewing the same terminal and such.

Using tmux only for those features is leaving a lot on the table, the scripting interface is excellent and will likely be a huge improvement to your workflow.

Tmux crash course

If you’re not already familiar with tmux this is going to be a quick introduction that will have you creating a couple of sessions, windows, panes, switching between them as well as detaching and re-attaching.

Feel free to skip this if you’re already familiar.

Follow the commands one by one right after the other, read the comments as you go. Lines starting with $ or : are meant to be typed out without those chars, anything in <> is to be key presses or combination thereof. If there are multiple <> in one line enter them in quick succession, e.g. press <control-b>, let up then quickly <c>

Note that some of the commands listed below are context dependent in order to make sense.

$ tmux # Creates a new session by default, and attaches you to it
$ echo $TMUX # This displays info used by tmux to figure out your session
<control-b> <c> # *Creates* a new window
$ ls -la # list all of the files in the current directory.
<control-b> <p> # Go to *previous* window
<control-b> <n> # Go to *next* window
<control-b> <"> # Split window horizontally
<control-b> <o> # Cycle to *other* windows
<control-b> <%> # Split window horizontally
<control-b> <Alt-1> # Select even horizontal layout
<control-b> <Alt-2> # Select even vertical layout
<control-b> <Alt-3> # Select main horizontal layout
<control-b> <Alt-4> # Select main vertical layout
<control-b> <Alt-5> # Select tiled layout
<control-b> <z> # *Zoom* in on pane
$ hexdump -C < /dev/urandom | grep -i --color=force 'ca fe' | sed 1000q
<control-b> <[> # Enter copy mode
<alt-v> # Page up
<control-v> # Page down
<control-b> <:> # Enter command mode
:set mode-keys vi
<control-b> # Page up in vi mode
<control-f> # Page down in vi mode
<control-u> # Half Page up in vi mode
<control-d> # Half down in vi mode
:set mode-keys emacs # goes back to emacs mode. Choose whatever you prefer.
<control-b> <z> # *Zoom* out on pane
<control-b> <d> # Detach from session
$ tmux ls # List tmux sessions
$ tmux new -s WEE # Creates a new session named "WEE"
<control-b> <d> # Detach again
$ tmux att -t 0 # Attach now to the session we created earlier
<control-b> <w> # Opens the "chooser-tree"
<k> # Go up one line
<j> # Go down one line
<control-p> # Go up to previous line
<control-n> # Go to next line
# Now take note of this view, it's showing you the sessions you have open
# as well as all of the windows and panes. From here you can quickly
# select any of them. You can also search using `</>` or `<control-s>` just
# like you would in either Vim or Emacs respectively. This is possible
# in the copy-mode as well.
<control-b> <?> # Shows all of the bound hot keys in the form of tmux commands.

Now I want you to go through the help menu you just pulled up, and look for all of the hotkeys and commands you just entered. They’re doing nothing more than sending commands to tmux and all of them can be re-bound utilizing a configuration file.

Once you’re done with that take a moment and search for some of the commands in the man page and read up on their options.

Putting the pieces together

Now that you have a grasp on tmux, and the idea that everything can be sent as a command let’s go ahead and create a session with three windows. If you’re still attached to a tmux session, you should detach now.

$ tmux new-session -s "Three windows!" -n "Window #0" \;\
    new-window -n "Window #1!" \; new-window -n "Window #2!"

This should create a new session, called “Three windows!” ( which if you’re using the default configuration will have the session name on the lower left hand corner of the window cut off ) and three windows across the bottom with the names we specified as values to the -n flags above.

That’s all well and good, but the real magic comes in when we start specifying working directories and entering commands.

$ cd $HOME
$ tmux new-session -s "workdir" -c /tmp -n "In /tmp" \;\
   send-keys -t 0 "pwd" \; send-keys Enter

-c sets the working directory for the new-session command and send-keys will quite literally, send keys to a window.

We can then build upon what we’ve learned and create an even more complex, but useful example:

#!/bin/sh
tmux new-session -s "dispatch" -c "~/code/dispatch-tracker/app" -n "editor" \;\  
    send-keys "$EDITOR" \;\ send-keys Enter \;\
    new-window -n "shell" \; send-keys ". env/bin/activate" \;\ send-keys Enter \;\
    new-window -n "server" \; send-keys ". env/bin/activate" \;\ send-keys Enter \;\
      send-keys "python manage.py runserver 127.0.0.1:8090" \;\ send-keys Enter \;\

Breaking this apart, we have a new-session called dispatch, in the working directory of ~/code/dispatch-tracker/app and window name of editor.

From there we send $EDITOR and enter it in the first window.

We then create a second window called shell which will inherit the session’s working directory. We’ll then activate the python virtualenv by entering a command.

The third “server” window then does the same thing as the previous shell window but runs python manage.py runserver afterwords, so we don’t have to handle that by hand.

Expanding further

We can create a more useful script that checks out the repository, sets up the virtual environment, creates a Postgres server and starts the command line client too:

#!/bin/sh
set -e
workdir="$HOME/code/dispatch-tracker"
python="python3"
EDITOR="${EDITOR:-vi}"

if ! [ -d "$workdir" ] ; then
		git clone -b demo https://git.riedstra.dev/mitch/dispatch-tracker "$workdir"
fi

cd "$workdir"

if ! [ -d "app/env" ] ; then
		cd app
		$python -m venv env
		. env/bin/activate
		pip install -r requirements.txt
		deactivate
		cd ..
fi

! [ -d db ] && pg_ctl -D db initdb

tmux new-session -s "dispatch" -c "$workdir/app" -n "editor" \;\
	send-keys "$EDITOR" \;\ send-keys Enter \;\
	new-window -n "shell" \;\
		send-keys ". env/bin/activate" \; send-keys Enter \;\
	new-window -n "server" \;\
		send-keys ". env/bin/activate" \; send-keys Enter \;\
		send-keys "python manage.py runserver 127.0.0.1:8090" \; send-keys Enter \;\
	new-window -n "database" \;\
		send-keys "cd .. && postgres -D db" \; send-keys Enter \;\
	new-window -n "psql" \;\
		send-keys "sleep 3; psql postgres" \; send-keys Enter \;\
	select-window -t 0 \;\

From here it will take you next to no time to setup the config.yml and run ./manage.py migrate making future development on the application close to painless since the script does the work for you.

If you’re on Linux, you can even get clever and replace the postgres commands to run it from a docker image.

Organizing the scripts

I deal with a number of clients and projects, each with their own unique setup and requirements. Having the scripts easily accessible and in a layout that gives us tab completion is ideal.

For this I simply use the ~/dev directory in my home folder. Creating a layout like so:

$ tree dev
dev
├── <client_0>
│   ├── <project_0>
│   ├── <project_1>
│   ├── <project_2>
│   └── <project_3>
├── <client_1>
│   ├── <project_0>
│   ├── <project_1>
│   ├── <project_2>
│   └── <project_3>
├── <client_2>
│   ├── <project_0>
│   ├── <project_1>
│   ├── <project_2>
│   └── <project_3>
└── <client_3>
    ├── <project_0>
    ├── <project_1>
    ├── <project_2>
    └── <project_3>

Where <client> is a directory and <project> is a shell script that sets up the environment and everything I need to get to work on it.

I can easily see what projects I have for a client by simply doing:

$ dev/client/<tab><tab>

Or the clients with

$ dev/<tab><tab>

If necessary for complicated projects with sub-projects it’s also possible to just create a project directory with multiple shell scripts inside instead of just a script.

Removing repetition from the scripts

In nearly every development environment I setup I’m going to have three windows, one for my editor, one for a shell and another shell window specifically for git. In my shell configuration I have some functions that make the setup a little bit easier.

_tmux_session() {
session=""
window_name=""
working_directory="$(pwd)"
while [ $# -gt 0 ] ; do case $1 in
--) shift ; break;;
-s) session="$2" ; shift ; shift ;;
-n) window_name="$2" ; shift ; shift ;;
-w) working_directory="$2" ; shift ; shift ;;
*)echo "Invalid option $1" ; return ;;
esac ; done

cd "$working_directory" || { echo "Cannot change to: $working_directory" ; return; }

tmux new-session -s "$session" -n "$window_name" -c "$working_directory" \;\
  "$@"
}


_tmux_dev() {
session=""
working_directory="$(pwd)"
window_name="editor"
while [ $# -gt 0 ] ; do case $1 in
--) shift ; break;;
-s) session="$2" ; shift ; shift ;;
-n) window_name="$2" ; shift ; shift ;;
-w) working_directory="$2" ; shift ; shift ;;
*)echo "Invalid option $1" ; return ;;
esac ; done

cd "$working_directory" || { echo "Cannot change to: $working_directory" ; return; }

_tmux_session -s "$session" -w "$working_directory" -n "$window_name" \
  -- \
  send-keys -t 0 "$EDITOR" \; send-keys -t 0 Enter  \;\
  new-window -n "shell" \;\
  new-window -n "git" \;\
  send-keys -t 0 "git status" \; send-keys -t 0 Enter  \;\
  "$@"

}

With those defined in my ~/.kshrc ( or ~/.profile, or ~/.bashrc, adjust to suit your needs ) I can adjust my scripts a little bit further to look something like this:

#!/bin/sh
. ~/.kshrc

EDITOR='${EDITOR:-vi}'
export website_dir="/v/me/riedstra.dev"
go_website_dir="/v/me/go-website"
listen="127.0.0.1:8080"

_tmux_dev -s "website" -w "$website_dir" -- \
	new-window -c "$go_website_dir" -n "server" -t 9 \;\
	send-keys -t 9 "go run ./cmd/server -l \"$listen\" -d \"$website_dir\"" \;\
	send-keys -t 9 Enter \;\
	select-window -t 0 \;\

Which will create a session with four windows. My editor, shell and git windows as mentioned above and an additional one to run the go server that is used for development against my website.

This allows me to not only keep my scripts a bit shorter, but also gives me a central location in which to alter the way I setup the environments in the future if I so choose.

Logging into multiple servers at once

Another great feature of tmux is the ability to synchronize the panes and run the same commands interactively on multiple servers at once.

The following snippet will ssh into four servers, server0 through server3 tile the panes, and turn on synchronization:

#!/bin/sh

tmux new-session -s server-example \;\
	 send-keys 'ssh server0' \;\
	 split-window \;\
	 send-keys 'ssh server1' \;\
	 split-window \;\
	 send-keys 'ssh server2' \;\
	 split-window \;\
	 send-keys 'ssh server3' \;\
	 set synchronize-panes on \;\
	 select-layout tiled \;\
	 send-keys Enter

It works, though copying send-keys and split-window a bunch of times which is less than ideal.

With a little bit of effort into a shell function:

_tmux_servers_split_commands() {
n=0
server_cmds=""
while [ $# -gt 0 ] ; do
  if [ $n -eq 0 ] ; then
    server_cmds='send-keys "ssh \"'"$1"'\"" \; send-keys Enter \; '
  else
    server_cmds="${server_cmds} split-window \; "'send-keys "ssh \"'"$1"'\"" \; send-keys Enter \; '
  fi
  n="$(echo "$n+1" | bc)"
  shift
done
echo "$server_cmds"
}

_tmux_servers() {
session=""
working_directory="$HOME"
servers=""
layout="even-vertical"
while [ $# -gt 0 ] ; do case $1 in
--) shift ; break;;
-s) session="$2" ; shift ; shift ;;
-w) working_directory="$2" ; shift ; shift ;;
-servers) servers="$2"; shift ; shift ;;
-l) layout="$2"; shift ; shift ;;
*)echo "Invalid option $1" ; return ;;
esac ; done
cd "$working_directory" || { echo "Cannot change to: $working_directory" ; return; }

layout="select-layout $layout ';' set-window-option synchronize-panes on ';'"

eval _tmux_session -s "\$session" -w "\$working_directory" -n "main" \
  -- \
  $(_tmux_servers_split_commands $servers) \
  $layout \
  "\$@" \

}

We can reduce the above down to just:

#!/bin/sh
. ~/.kshrc
_tmux_servers -s server-example -l tiled -servers "server0 server1 server2 server3"

Which yields the same result.

Sorting these scripts out on disk is also easy enough, I populate ~/servers with a similar format I did above for ~/dev which allows me to quickly access a particular setup for a client through tab completion. For instance:

$ tree servers
servers
├── <client0>
│   ├── <service_0>
│   ├── <service_1>
│   └── <service_2>
├── <client1>
│   ├── <service_0>
│   ├── <service_1>
│   └── <service_2>
└── <client2>
    ├── <service_0>
    ├── <service_1>
    └── <service_2>

Vim file management, sessions, and windowing

Many will use tmux’s windowing instead of leveraging vim’s built in windowing, buffers and session management. While tmux does allow for copy, paste and such I think this is a mistake considering vim’s window management works well and the advantage that utilizing multiple buffers gives you.

Tabs aren’t your friend–at first. Get familiar with :ls and :b <filename> to switch files quickly.

Then, branch out, :vs [<filename>] can be used to vertically split, :sp [<filename>] to split horizontally. c-w [h,j,k,l] can be used to navigate whatever arbitrary setup you create.

If you resize a window, c-w = can be used to resize the windows so that they’re all split somewhat evenly. Additionally there are more commands that can be prefixed with a number to fine tune the size of the windows. :help windows can be of use here.

Next are tabs. They essentially hold a layout of windows, distinct from buffers. :tabnew, :tabn ( next ) and :tabp ( previous ) are of use here.

Lastly, if you have several tabs open, and a window layout that you like for a specific project you can save where you’re at currently with :mkses ses.vim Later on you can fire up vim with vim -S ses.vim and it will automatically restore everything just as you had it, tabs, windows and such.

Final thoughts

Hopefully this has been helpful, giving you a bit of insight as to how to organize and quickly bring up all of the tools you need to work on a given project. Additionally the shell functions provided above are available in my shell configuration. Which is also worth reading over as it includes a myriad of other useful utilities.