Simplest Possible Process (SPP) for Publishing Documentation

by the Open Source Security Foundation (OpenSSF), 2023-06-28

This document describes the “Simplest Possible Process” (SPP), a process for publishing nice-looking static results on a website using GitHub-hosted repositories.

In the SPP, the materials are maintained using GitHub and deployed as simple static pages:

1. Each GitHub repository uses GitHub pages to regenerate and serve its static web pages results whenever its main branch is changed (typically using files in markdown format in its docs subdirectory).
2. Each repository’s results are accessed using a separate DNS subdomain (e.g., https://best.openssf.org).
3. If documents are being published in this format, consider applying the tips below. In particular, please include a (publication) date in YYYY-MM-DD format in their contents, and update that during content changes.

This document is written primarily for use by technical contributors to the Open Source Security Foundation (OpenSSF), but others are free to use it.

Background

The Open Source Security Foundation (OpenSSF) produces many results, some of which should be published in a nice-looking easily-consumed format. However, it has been unclear how to publish final results. David A. Wheeler considered the options and proposed using this SPP (see the rationale) unless there was some reason the process wouldn’t work. The Best Practices Working Group experimented with the SPP and found it helpful to them.

As a result, the OpenSSF TAC declared on 2023-06-27 that any OpenSSF working group, SIG, or project is welcome to use this SPP if they determined that it met their needs (minutes, TAC issue 176). The OpenSSF staff will implement this for an OpenSSF repository, but only on request (to avoid unnecessary work).

Tips

Here are a few tips for using a repository that is using the SPP. In general, try to be consistent; special configurations are possible but it’s easier if all pages use the same approach.

Organize your repository’s subdirectories. Choose upper and lower case carefully, since paths are case-sensitive. Directory and file names with spaces are supported, but not recommended, because the resulting URLs look odd (space becomes %20).

All markdown (.md) files will be converted into HTML (.html) files; you can omit the trailing .html when accessing them using a web browser.

When using markdown:

• We recommend including markdownlint in your CI/CD, possibly configured specially, to detect and prevent problems ahead-of-time.
• Try to limit yourself to de facto CommonMark where reasonable.
• However, feel free to use HTML directly within markdown where it’s necessary or less confusing. Also, you can use kramdown extensions when CommonMark isn’t enough; see markdown kramdown tips and tricks for a summary.
• Begin with a single H1 heading (#) with its title. This makes it easy to generate a correct document title in the metadata. In the following paragraph (after a blank line) include a line with AUTHOR, YYYY-MM-DD, like this: *by the [Open Source Security Foundation (OpenSSF)](https://openssf.org) [Best Practices Working Group](https://best.openssf.org/), 2023-06-14*
• Update the publication date when you change the material in the main branch. We suggest making date updates a separate pull request (to reduce the number of merge conflicts that need to be resolved).
• Use ## headings for logical Heading1 entries, ### for logical Heading2 entries, and so on. Each heading name should be unique (so that the generated ids are unique). You can reference any heading via its ID, e.g., [text](#ID). Here is the algorithm for determining header IDs.
• If you want a table-of-contents, use :toc surrounded by curly braces on its own line.
• By default GitHub pages uses the Jekyll engine for generating web pages. By default, Jekyll uses kramdown for processing markdown and the Minima Jekyll template. Thus the kramdown cheatsheet may be helpful.

When you creating a new markdown document, you should begin with a YAML front matter block surrounded by triple-dashed lines, at least to provide a description (this will be included in metadata sent to users). E.g.:

---
description: "This document ..."
---

If you don’t want to include a YAML front matter block (e.g., because people are directly looking at the markdown file), you can instead include this information in the repository file _config.yml.

Once changes are checked into the main branch they are eligible for regeneration, which generally doesn’t take long. You can see its status from a repository by clicking on Actions and looking at its workflow run.

One-time setup

The SPP requires a one-time setup for each repository. Here’s how to do that:

1. Create a docs subdirectory. While optional, we recommend creating a docs subdirectory in the repository root directory. Move all documents that might eventually be published into this docs subdirectory (e.g., git mv). This way only materials intended to be eventually published will be included. Do not wait until the documents are final before putting them in docs; it’s important to be able to review and fix formatting bugs ahead-of-time. Instead, don’t create hyperlinks to those materials claiming they are final releases.
2. Set up GitHub pages for the repository. At the main repository page select Settings. Select Code and automation/Pages. Under Build and deployment select Deploy from a branch using branch main and folder /docs, then save. Under custom domain enter your custom domain (e.g., best.openssf.org) and select Save; this will create a file name CNAME in the root directory.
3. Configure DNS. Change the DNS setting for the (new) domain name (e.g., best.openssf.org) so its CNAME points to the organization GitHub site (e.g., ossf.github.io). Note that every repository of an organization uses the same CNAME (GitHub uses the CNAME file in your repository to figure out which site will be used). OpenSSF members can just email operations at openssf.org; Linux Foundation employees can send the request to LF support or more specifically Domains & DNS. After a little while TLS (https:) will be ready!
4. Configure the repository. Copy some configuration files to make the results look good, mainly the file ./_config.yml and directories _includes/ and assets/. These must start at the docs directory if you’re using a docs directory. The directories (e.g., _includes/ can override a template’s file simply by creating another file with the same name in the same place. We intend to add more about doing this in the future. For now, you can use the OpenSSF best practices WG as a demonstration.

Rationale for the SPP

Problem

The initial approach we (the OpenSSF) have been been using for distributing most of our results has many problems. The initial approach shows readers lots of irrelevant text (the GitHub source repo interface), we cannot control its formatting, the URLs aren’t related to our (openssf.org) domain, most metadata (such as the description) is wrong, and search engines are likely to give it low scores (because it’s “just a random page on GitHub”). For an example of this “ugly” view, see the Guide to implementing a coordinated vulnerability disclosure process for open source projects

There are a practically infinite number of ways to publish results on a website that are better. This abundance of choices was part of the problem of picking any particular way to publish results. This is especially a challenge for us, because many participants in the OpenSSF are fully capable of understanding and implementing complex workflows if the situation called for it.

How the SPP was identified

On 2023-06-06 the OpenSSF best practices WG re-raised the lack of a good “publishing” process to a website was a problem. This was recorded as Best Practices WG issue 158, Revamp publishing of guides such as concise guides & scm guide.

OpenSSF staff quickly developed and deployed an experimental solution. The overall approach to selecting the SPP has been to find the “easiest way to do it” (given our current state) and to prefer system defaults (e.g., using default GitHub pages with its default site generation process (Jekyll), markdown processors (kramdown) and default template (Minima). In short, the “simplest possible process” (SPP) focuses on trying to work with existing tools & minimize what needs to be done:

1. The easiest way to generate web pages suitable for publication, if you already have markdown on GitHub in a repo like we do, is to enable GitHub pages on that repo. This means that any change to the repo’s main branch will immediately trigger recreating those pages and only the pages in that repo. The results rapidly deploy, since the trigger is known and a relatively small number of pages are regenerated. Making the results “pretty” requires adding a few files in a way that’s trivial to do and can be done incrementally.
2. The easiest way to make those GitHub pages viewable is to assign a subdomain to every repo, e.g,. best.openssf.org for the best practices WG. This involves easy one-time actions (add a DNS record and configure GitHub pages to use that DNS entry). GitHub will automatically get TLS certificates, so we quickly serve web pages securely (using https).

This is not only easy to set up, but it’s also easy to use. Contributors can simply use git and GitHub as usual, and the updates are quickly published once they merged into the main branch.

You can see the results in the Concise Guide for Developing More Secure Software. Notice that its URL is «https://best.openssf.org/Concise-Guide-for-Developing-More-Secure-Software». We think the new results look much better than the old view of the Concise Guide for Developing More Secure Software, whose URL was the much uglier «https://github.com/ossf/wg-best-practices-os-developers/blob/main/docs/Concise-Guide-for-Developing-More-Secure-Software.md» You can see similar positive results in the Concise Guide for Evaluating Open Source Software.

Of course, we didn’t create this process as a new concept. Many other organizations already use this kind of process. GitHub implemented these mechanisms and this is how they’re intended to be used. However, we had an overwhelming number of alternatives, so we needed to identify and name a specific approach to use.

Experimentation in the Best Practices Working Group showed this process, the SPP, was enough for most of our simple use cases.

Alternatives

There are so many other ways to generate pages, but they all add complexity that seems unnecessary for most of our use cases:

• We can write workflows that push data to other places, but then we need to write that code and manage the keys.
• We can write workflows that gather the data (but then we need to write that code, and these typically have significant deployment delays because they’re often periodic and would handle much more data).
• There are additional operational complications if we try to put everything in the domain openssf.org, as that is currently a WordPress instance. We would then have to synchronize these systems.

The SPP cannot meet all needs, in particular, it’s not enough if you need a dynamic site. In those cases the relevant group will need to use something else.

Advantages of the SPP

Here are advantages of the SPP compared to other approaches for generating a static site:

• The results look nice and are relatively easy to reconfigure.
• It has good search engine results (it’s not just “another page on GitHub” and we control can serve correct metadata values).
• OpenSSF participants don’t need to change how they work (just keep using GitHub for version control).
• This approach works with the tools instead of against the tools.
• Page regeneration is trivial to trigger (it’s automatically handled by GitHub), and the regeneration is fast (because there are relatively few pages to regenerate).
• They all under *.openssf.org, making it clear they the work from our group (the OpenSSF).

Once set up, these web pages are quickly and automatically updated once the repo’s main branch is updated. The source materials are version-controlled, and the contributors can simply “work like they’ve always worked”.

Non-problems

Some potential issues have been noted, but we believe we have adequate solutions:

1. Common CSS. If many repos use this approach, it would be a pain to have to individually update each one to update the CSS if that happens. The “obvious” solution is to have a repo for just the CSS, and then have all other OpenSSF repos include that CSS. Then there would be one place to update CSS. If we have that many repos, we can implement this, and have the one-time cost of changing each publishing repo to use it.
2. Abandoned domain names. If a repo is deleted, we also need to delete the repo entry. Repo deletion is incredibly rare (we’ve never done it in 3 years), so we think this could be handled as part of a manual process. In short, if someone is going to delete a repo that is serving GitHub pages and has a CNAME entry, we need to delete its DNS name first. We could also automate checking for dangling DNS entries if we wanted to (that would be good anyway).
3. Changing later. If the SPP later turns out to be not enough, the group can switch to another process and automatically redirect requesters to their new locations (where appropriate). We couldn’t automatically redirect requesters when they directly used the pages on the GitHub repo site, so when people stop using the SPP, using the SPP will still be a win.
4. Many subdomains. A side-effect of SPP is that every repo ends up with its own subdomain, e.g., best.openssf.org for the Best Practices WG. This may create a large number of subdomains (a WG might have several repos that publish, not just one). David A. Wheeler doesn’t view that as a drawback, as such subdomain names are obviously part of OpenSSF. When certificates cost money those extra domains were a real problem, but with Let’s Encrypt this isn’t an issue. The OpenSSF staff only plans to take the one-time SPP steps by request, so they will only need to take those steps if someone plans to use it.
5. Forge changes. We have no plans and no reason to stop using GitHub. If we did move, however, the key components that implement this process (the Jekyll engine, the kramdown markdown processor, and the Minima Jekyll template) are all OSS and we could reconstitute them elsewhere. Even if they weren’t, there are other similar tools we could use instead, as it’s basically just converting markdown to HTML and posting the results.
6. Publication dates. As currently implemented the SPP does not automatically add publication dates. There are Jekyll plug-ins we could use to create publication dates, but installing them seems complex and probably not worth it. For now, we ask that groups just include the date in the document being published. While groups could occasionally forget to update them, this also means that the date will be robustly included in the file itself once they are updated, and we expect people will easily understand when they need to update a simple date.

Limitations

The SPP only generates static websites. You can serve JavaScript, but that doesn’t work when users will not execute client-side JavaScript. If you want a truly dynamic website running server-side code, the SPP is inadequate, and groups should look for a different solution.

However, many of our groups are serving simple static documents; a simple process like the SPP should serve them well.

To be done

The OpenSSF Best Practices WG has various configuration files set up for its use. We should “clean up” those files to make it easier for other groups to reuse them, where that makes sense.

This site is open source. Improve this page.