In a previous post I explained how to set up Emacs Org-mode for working on a remote cluster. If your cluster uses Slurm to manage jobs, you will need to specify a set of options for submission. This isn’t difficult, but it’s tedious. We can automate away most of the tedium with Emacs, specifically the YASnippet package. (YASnippet stands for “yet another snippet package”, by the way). The animation above shows what we’re aiming for.
Install YASnippet
The homepage includes instructions on how to install it. The easiest uses the Emacs package manager.
If you want to use it by default, you should add the following to your config:
(yas-global-mode 1)
Alternatively, you can set it for each mode you want it applied to. For
example, the following turns it on for org-mode:
(add-hook ‘org-mode-hook #‘yas-minor-mode)
You may also need the following line to ensure all your previously defined templates are loaded:
(yas-reload-all)
Writing Snippets
A snippet is a bit of text that you can automatically insert into a file. It can include:
-
text, inserted ‘as-is’;
-
tab stop fields, which you can navigate through and enter text (with or without default values) when the template is inserted;
-
more complex variations combining fields, transformations, mirrors and elisp code
To get started, call the command M-x yas-new-snippet. This will open a
new buffer with the following text:
# -*- mode: snippet -*-
# name:
# key:
# --
The cursor will be on the name row. This value is for our benefit, so
something descriptive is appropriate. I’ll use the phrase ‘Slurm header’.
This buffer is actually a snippet itself, so we can use the TAB key to
jump to the next tab stop, which is on the next line.
The key is a combination of one or more letters that we’ll use to insert
the template. It can’t contain a space. It should be memorable, but not
something we’ll type in other contexts. In this case, slrm will do. Hit
TAB again, and point will move down into the body of the snippet.
This is where we put the text for our submission script.
For starters, we can include any text we want to appear ‘as-is’:
# -*- mode: snippet -*-
# name: Slurm header
# key: slrm
# --
#+BEGIN_SRC bash :results output
date
sbatch <<SUBMITSCRIPT
#!/bin/bash
#SBATCH --job-name=MYJOB
#SBATCH --output=MYJOB.log
#SBATCH --open-mode=truncate
#SBATCH --partition=standard
#SBATCH --time=24:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
$0
SUBMITSCRIPT
#+END_SRC
I start and end with the org-mode block header/footer, with the language
set to bash and the :results output option. These never change.
I start the code chunk with the date command, so it will capture the time and
date I submitted the script, which will be recorded in the org file.
<<SUBMITSCRIPT
...
SUBMITSCRIPT
Defines the script that is passed to Slurm, as discussed in my previous.
After that we find the actual Slurm directives with default values for each line.
There is one special field here, $0. This is where the cursor will be
after the template is inserted.
That’s fine, but we still need to go back and fill in the actual values for
the directives. We can improve this with by adding tab stops, along with
default values and mirrors:
# -*- mode: snippet -*-
# name: Slurm header
# expand-env: ((yas-indent-line 'fixed))
# key: slrm
# --
#+BEGIN_SRC bash :results output
date
sbatch <<SUBMITSCRIPT
#!/bin/bash
#SBATCH --job-name=${1:NAME}
#SBATCH --output=$1.log
#SBATCH --open-mode=truncate
#SBATCH --partition=standard
#SBATCH --time=${2:24:00:00}
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=${3:1}
$0
SUBMITSCRIPT
#+END_SRC
When we insert this template, the cursor starts at the --job-name
directive where the ${1:NAME} field is. The 1 indicates this is the
first tab stop. The value NAME is the default for this field. Whatever
value we enter here will be mirrored to the line below with the suffix
log, i.e., --output=NAME.log.
After we’ve added the name, pressing <tab> takes us to the second tab
stop, for the time directive. My default is 24:00:00, but I can change
that to whatever I need. The third tab stop is for cpus-per-task. After
that we tab to stop zero, where we can enter the body of our script.
One final tweak: I want yasnippet to leave the indentation of this template
exactly as I’ve written it (i.e., ‘fixed’). To accomplish this, I’ve set
the expand-env options in the header to use fixed indentation.
# expand-env: ((yas-indent-line 'fixed))
You can test out your snippet by calling M-x yas-try-snippet, which is
bound to C-c C-t. This will open a temporary buffer with the body of your
snippet inserted. You can then enter text and tab through the stops to see
how it works. C-x k will kill the buffer when you’re done.
When you’re done, you can install the snippet with C-c C-c, (or M-x yas-load-snippet-buffer-and-close). Emacs will ask you what mode you want
to use the snippet in. In this case, it’s org-mode.
Using Snippets
With that out of the way, you can insert your snippet in an org file with
the key sequence slurm<tab>. Yasnippet provides a lot of additional
features. If you’re interested, see the online
manual.
Interacting with Slurm
Once you’ve composed your Slurm script, you’ll want to submit it to the
cluster. If you’ve set up your org file to point to the cluster, as I
described in my previous post, all you need to
do is type C-c C-c, while the cursor (point) is in your Slurm code block.
It may take a few moments to connect, depending on your network. Once it does, you should see something like this appear in your org file:
#+RESULTS:
: Tue 27 Aug 2024 12:17:15 PM EDT
: Submitted batch job 2931015
We could log into the server to check on the status of our job. But with another snippet, we can have Emacs do that for us.
I use the following snippet for this purpose:
# name: sacct
# key: sss
# expand-env: ((yas-indent-line 'fixed))
# --
#+BEGIN_SRC bash :results output
date
sacct --jobs=`(save-excursion
(re-search-backward "^: Submitted batch job \\([[:digit:]]+\\)")
(buffer-substring-no-properties
(match-beginning 1) (match-end 1)))`$0
#+END_SRC
I’ve used a bit of elisp code to do some work here. Yasnippet will process any text between back ticks ("`") as elisp code. Let’s step through that:
`(save-excursion
(re-search-backward "^: Submitted batch job \\([[:digit:]]+\\)")
(buffer-substring-no-properties (match-beginning 1)
(match-end 1)))`
We start with save-excursion. That tells Emacs that we want to come back
to the spot we started when the code is finished.
Next we use re-search-backward to find a line that starts with (^) the
string “: Submitted batch job”, followed by a number. The number is wrapped
in \\( and \\): these symbols tell Emacs to record the number as a
‘match group’.
Finally, we return the number with the buffer-substring-no-properties
call. This returns the substring (text) in the current buffer, starting
with (match-beginning 1) (which is the first digit in our number), and
ending with (match-end 2), which is the last digit in our number.
This results in the following text being inserted in the buffer:
#+BEGIN_SRC bash :results output
date
sacct --jobs=3472303
#+END_SRC
As written, this bit of code assumes the job we want to see the status of is somewhere above the point that we call this snippet. It’s not very robust if this isn’t true, but in my limited use-case it works well enough.
The snippet leaves the cursor inside the code block. If you type C-c C-c
at that point, it will submit the command to the cluster, and after a
moment or two you should see something like this:
#+RESULTS:
: Fri 10 Jan 2025 05:28:53 PM EST
: JobID JobName Partition Account AllocCPUS State ExitCode
: ------------ ---------- ---------- ---------- ---------- ---------- --------
: 3472303 my_job standard grdi_gena+ 1 COMPLETED 0:0
: 3472303.bat+ batch grdi_gena+ 1 COMPLETED 0:0
: 3472303.ext+ extern grdi_gena+ 1 COMPLETED 0:0
Viewing Job Output
The preceding covers most of my needs for interacting with my cluster. One last trick that can be handy is linking to log files or program output.
You can insert a link to a file in org mode with C-c C-l. You will be
prompted for the file location. This can include remote files with the
syntax: /ssh:user@host:path/to/file; this can be shortened to
/ssh:host:path/to/file if you’ve set the appropriate options in
.ssh/config as I describe in a previous
post.
Next you’re prompted for the name of the link, which can be anything you
like. Once the link is complete, Emacs will colour it to let you know it
has a special power: you can view the file by pressing enter on the link.
I don’t do this often enough to have made a snippet for it.