cf-docs-rfc-proposal

Code-block survey

Status: draft, incomplete — covers what is known as of 2026-10-11.

This survey counts every code block in the 15 cloudfoundry/docs-* repositories and in the book’s own repository, and records how each one is written, where it sits, what it holds, and what the book renders from it. It was done on 2026-10-11 to choose the test pages of the code blocks round (R-06 typed code blocks).

Terms used below:

File What it is
blocks.csv One row per code block, 2,957 rows: its form, info string, place, how it closes, what it holds, what the book and a CommonMark parser make of it, with a GitHub link to the block at the surveyed commit
rendered.csv One row per code block the book’s renderer finds (1,170), with the Rouge lexer it gets: the output of book-render.rb
commonmark.csv One row per code block a CommonMark parser finds in the same pages (1,180): the output of commonmark-render.mjs
shas.txt The 15 repositories and the commit surveyed in each: the partials survey’s list
survey.py Finds the code blocks in a set of clones and writes blocks.csv
book-render.rb A model of the book’s renderer: each Markdown page through Redcarpet and Rouge as the book’s build runs them
commonmark-render.mjs, package.json The same pages through commonmark.js 0.31.2, the CommonMark reference parser
compare-published.py, published-pairs.txt, model-check.txt The model’s <pre> elements compared with 20 published pages: all the same
summary.py Prints the counts below from the three CSV files
# from concepts/code-blocks/survey, with the clones in REPOS
GEM_HOME=GEMS ruby book-render.rb REPOS shas.txt rendered.csv WORK      # needs redcarpet 3.2.3, rouge 1.11.1
npm ci && node commonmark-render.mjs WORK commonmark.csv
python3 -I survey.py REPOS shas.txt rendered.csv commonmark.csv blocks.csv
python3 -I summary.py blocks.csv rendered.csv commonmark.csv
python3 -I compare-published.py WORK published-pairs.txt

Scope

How the book renders a code block

The book pins bookbindery 9.12.1, which runs Middleman 3.4 (partials survey). Lines are cited at Bookbinder’s v9.12.0 tag (f608a44), as the partials survey does.

  1. Markdown engine and options. Bookbinder’s Middleman config loads middleman-syntax, sets Redcarpet with tables, autolink, smartypants and fenced_code_blocks, and activates the syntax extension (config.rb L2, L11-L16, L32). Bookbinder requires middleman-syntax ~> 2.0, rouge != 1.9.1 and redcarpet ~> 3.2.3 (bookbinder.gemspec L22-L24), and middleman-syntax 2.1.0 requires rouge ~> 1.0 (gemspec L19). The newest releases those allow are middleman-syntax 2.1.0, Rouge 1.11.1 and Redcarpet 3.2.3 (RubyGems: middleman-syntax, rouge, redcarpet). The book has no Gemfile.lock, so which releases its build installs is not recorded; the published pages match these (the model, below).
  2. Every Markdown code block goes to Rouge. middleman-syntax replaces Redcarpet’s block_code, for fenced and indented blocks alike (redcarpet_code_renderer.rb L6-L8). It finds a lexer with Rouge::Lexer.find_fancy(language, code), falls back to plain text, and writes the class highlight plus the lexer’s tag (highlighter.rb L7-L16). Rouge’s formatter writes <pre class="highlight TAG"><code>…</code></pre> (html.rb L48-L52, Rouge 1.11.1).
  3. A block with no info string gets a guessed lexer. find_fancy with no language asks Rouge to guess from the code (lexer.rb L48-L66), and an unknown name finds no lexer, so plain text. Names are matched as written: SQL and JSON find no lexer.
  4. console is Rouge 1’s shell_session lexer, which also answers to terminal (shell_session.rb L5-L25). It colours a line that starts with $ or # : the prompt, the command, its arguments. Every other line is plain text.
  5. <pre> blocks are HTML and reach the page as written, after ERB has put in any variables. SmartyPants, the step that turns straight quotes into typographic ones, leaves the text inside <pre>, <code> and four other tags alone (html_smartypants.c L327-L329). Bookbinder’s stylesheet colours .terminal blocks (base.scss L333-L337) and holds the highlighting rules, among them pre .gp for a prompt (_syntax-highlight.scss L19).
  6. No copy button. The layout loads one script, all.js (layout.erb L21), made of Bookbinder’s bookbinder.js and the book’s book.js (all.js L1-L2). Neither touches code blocks (book.js, bookbinder.js); book.js also loads Mermaid for diagrams. The all.js that docs.cloudfoundry.org served on 2026-10-11 holds no call to the clipboard, and the four pages saved in published/ hold no <button>.

A model of the book’s renderer. book-render.rb runs each Markdown page through Redcarpet 3.2.3 and Rouge 1.11.1 the way points 1–3 describe. ERB is replaced, not run: a <%= vars.name %> becomes its value, every other tag nothing. The model adds one option no source line sets, Redcarpet’s no_intra_emphasis: without it the model made cell_id … exit_description emphasis inside a <pre> in a list item where the published pages do not. Which layer of the real build sets it was not traced, as the headings round could not trace heading ids (headings results).

With that option, the model’s <pre> elements are the same, character for character, as the published ones on all 20 pages compared, 186 <pre> elements (model-check.txt), once e-mail addresses are decoded (below). Eight more pages fetched from docs.cloudfoundry.org on 2026-10-11 and not saved gave the same result, among them web-config-transform-buildpack, java-tips, application-binding and opentelemetry (as served that day; not pinned). The published classes confirm Rouge 1: <pre class="highlight shell_session">, with the prompt in a w span (saved routes-domains, line 846). From Rouge 2.1.0 the lexer is named console (console.rb L6-L7), and the class would read highlight console.

The published HTML hides e-mail addresses. Inside code blocks too, user@example.com reaches the page as a link reading “[email protected]”, which a Cloudflare script turns back into the address in the browser (saved gsg-grails, the script tag email-decode.min.js). Some sources write username<span>@</span>example.com to prevent it (routes-domains L150). The code-block check decodes such links as the script does.

Where the book’s Markdown engine differs from CommonMark. Redcarpet 3.2.3 reads fences more loosely than CommonMark:

Method

survey.py reads every file and finds three forms:

Form What is found
fence, fence-tilde a line of three or more backticks or tildes, any indentation, and an optional info string; it ends as Redcarpet ends it (above), and the row says whether CommonMark ends it at the same line
pre, pre-code an HTML <pre> element, without or with <code> inside
indented, fence-as-text a block that only the two parsers’ output shows: an indented code block, or one whose first line is a fence (a fence read as text)

For each block it records the info string or the <pre> attributes, the indentation, how it closes, and where it sits: at the top level, in a Markdown list item, in an HTML list item, in a note, in a table cell, in a blockquote. It records what the block holds: variables (<%= vars.x %>) and other ERB, character references (&lt;), HTML elements inside a <pre>, blank lines, braces, {{, {% and {# (template delimiters in Zensical, Markdoc and Jinja), Markdown characters inside a <pre> (*, _, backticks), tabs, and whether its first line starts with a prompt ($ , # , > ).

Then it joins each block with the two parsers’ rows by its first line, in document order, and records the kind each parser makes of it (fenced, indented, HTML, or nothing) and, for the book, the Rouge lexer. Lines in front matter and HTML comments are skipped. The join is approximate where the two parsers pair fences differently; the file-level comparison in summary.py is exact.

Totals

  Code blocks Files
The book (14 repositories and the book repository) 1,777 205
bosh.io (docs-bosh) 1,180 —

The book’s 1,777: fences 1,132 (1,110 with backticks, 22 with tildes), <pre> blocks 592 (573 <pre>, 19 <pre><code>), indented blocks 47, and 6 fences that the book shows as text. By repository: docs-dev-guide 524, docs-buildpacks 404, docs-cf-admin 227, docs-running-cf 89, docs-deploying-cf 83, docs-uaa 81, docs-loggregator 77, docs-credhub 61, docs-cloudfoundry-concepts 58, docs-bbr 57, docs-cf-cli 54, docs-services 34, docs-dotnet-core-tutorial 21, docs-book-cloudfoundry 7. The book’s model finds 1,170 blocks in Markdown (fenced and indented); commonmark.js finds 1,180.

Findings

Most fences name no type. 807 of the 1,132 fences (71%) have no info string. Rouge guessed plain text for 788 of them, JSON for 11 (among them {{module "…"}} lines in an NGINX file), and shell for 2; 6 are not code blocks in the book at all (below). The 325 with an info string: console 196, yaml 51, bash 17, json 10, xml 8, shell 7, java 6, ruby 6, text 4, csharp 4, script 4, path 2, go 2, and one each of ssh, toml, c#, SQL, javascript, python, jq, yml. Rouge found no lexer for script, path, ssh, jq, SQL and JSON.

console usually holds commands without prompts. 164 of the 196 console fences start without $ , for example cf marketplace. Rouge 1’s shell_session shows such a line as plain text, so on the published pages these blocks are not coloured. Terminal sessions are mostly <pre class="terminal">: 413 of its 554 uses start with $ , and 188 hold a prompt and output lines.

<pre> blocks are a third of the book’s code. 592 blocks: class="terminal" 546, class='terminal' 5, no attribute 19, <pre><code> 19, and 3 with typographic quotes, class=‘terminal’, which browsers read as the class ‘terminal’ (audit-events L42, usage-events L31, L102). What their text holds matters to tools that read the text of HTML as Markdown (results: Docusaurus and Starlight):

Where blocks sit. In a Markdown list item: 619 fences (556 indented four spaces, 47 eight, 10 six, 9 two) and 242 <pre> blocks; at the top level 513 fences and 334 <pre> blocks. In an HTML list item 8, in a table cell 6 (four in _c2c_oss_enable, two in oms-nozzle), and in a note 2, the notes round’s open item: gsg-grails L158-L162 and gsg-spring L130-L134.

Variables inside code. 64 blocks hold 136 variable uses: 43 fences, 15 <pre>, 6 <pre><code>; docs-dev-guide 42 blocks, docs-cf-admin 10, docs-loggregator 8. No block holds other ERB, and none writes <%% (ERB’s way to show a literal <%). Six blocks hold {{, five of them NGINX buildpack templates (nginx/index L52-L56), one a uaac command whose JSON holds Go template fields, {{.Subject}} (notifications L109-L112).

Character references inside fences are shown as written. In a fence, &lt;file&gt; is text, so the published page shows &lt;file&gt; (nginx/index L147-L149, L167; and two YAML blocks in manifest-attributes L97, L847).

Tabs: 73 blocks hold a tab character (44 fences, 16 <pre>, 13 indented).

Fences only the book’s engine closes, and other disagreements

The headings survey found two fences in backup that only the book closes (headings survey). Comparing the book’s model with commonmark.js over every page finds 14 files where the two find different code blocks, and 2 more where they differ only in a block’s trailing blank line (docs-dotnet-core-tutorial). By cause:

A conversion has to repair these before or while it converts, reading each block the way the published page shows it or the way the author meant it (R-24 markup cleanup before conversion). The test pages hold four of them (backup ×3, multiple-processes).

bosh.io

docs-bosh writes 1,180 fences, almost all typed: shell 504, yaml 430, json 96, text 94, bash 18, ruby 15, powershell 6, and a few others; 2 have no info string. It writes no <pre> block. Its blocks were counted, not converted.

Not covered