ci: publish client builds to a downloads page under player-facing names

Each run now stages what it built as DorkNet-<release>-PC-Full.zip,
-PC-Patch.zip, -Quest.xapk and -Quest-Installer.zip, writes their sizes and
SHA-256 to a manifest, and uploads them with a static page to an S3 bucket
served read-only (ci/downloads). The build scripts keep their own names,
which the launcher depends on; only the published copies are renamed.

PUBLISH_DOWNLOADS (on by default) turns the upload off per run. Jenkins
archives the same renamed files instead of the raw build output.
This commit is contained in:
Alexa
2026-10-25 10:40:46 +02:00
parent 2e09042344
commit 6e5b011059
5 changed files with 648 additions and 9 deletions
Vendored
+36 -1
View File
@@ -11,6 +11,10 @@
//
// On a push, only the platforms whose code changed are rebuilt. A manual run
// picks the platforms with PLATFORMS. See ci/jenkins/README.md.
//
// Finished builds are renamed for players and published to the downloads page
// (ci/downloads) unless PUBLISH_DOWNLOADS is off. The build scripts keep their
// own file names, which the launcher depends on.
// The builds this branch targets. Any other archived build can be picked per
// run with the *_SNAPSHOT_TAGS parameters.
@@ -18,6 +22,11 @@ def PC_VERSION_KEY = 'march_2023_03_21'
def PC_DEFAULT_TAGS = 'manifest=7490748483298966814' // Steam 2023-03-21 (appVersion 20230317)
def QUEST_DEFAULT_TAGS = 'version-code=2128210' // Quest 20230317
// How this branch's builds are listed on the downloads page.
def RELEASE_ID = '2023.03'
def RELEASE_TITLE = 'March 2023'
def RELEASE_GAME_VERSION = '2023-03-21'
def buildPc = false
def buildQuest = false
@@ -49,6 +58,8 @@ pipeline {
description: 'Quest restic repository folder on the FTP. Empty: ARCHIVE_QUEST_REPO set on the agent.')
booleanParam(name: 'LIST_SNAPSHOTS', defaultValue: false,
description: 'Only list the snapshots matching the tags in both repositories (to find a build), then stop.')
booleanParam(name: 'PUBLISH_DOWNLOADS', defaultValue: true,
description: 'Publish what this run built to the downloads page. Push builds publish too.')
}
stages {
@@ -178,11 +189,35 @@ pipeline {
}
}
}
stage('Downloads') {
when { expression { buildPc || buildQuest } }
steps {
sh """
python3 ci/downloads/publish.py stage --out out --release ${RELEASE_ID} \\
--title '${RELEASE_TITLE}' --game-version ${RELEASE_GAME_VERSION} --version-key ${PC_VERSION_KEY}
"""
script {
if (params.PUBLISH_DOWNLOADS) {
// The bucket's S3 endpoint and name come from the agent's
// environment (DOWNLOADS_S3_ENDPOINT / DOWNLOADS_BUCKET).
withCredentials([usernamePassword(credentialsId: 'downloads-s3',
usernameVariable: 'DOWNLOADS_S3_KEY_ID', passwordVariable: 'DOWNLOADS_S3_SECRET')]) {
sh 'python3 ci/downloads/publish.py upload --staged out/downloads'
}
} else {
echo 'PUBLISH_DOWNLOADS is off: the files are only archived.'
}
}
}
}
}
post {
success {
archiveArtifacts artifacts: 'out/**', allowEmptyArchive: true, fingerprint: true
// The renamed, player-facing files (and release.json): the same set the
// downloads page offers.
archiveArtifacts artifacts: 'out/downloads/**', allowEmptyArchive: true, fingerprint: true
}
cleanup {
// Runs after archiving. Keeps the restic cache and restored stock
+243
View File
@@ -0,0 +1,243 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>DorkNet Downloads</title>
<link rel="icon" href="dorknet.svg" type="image/svg+xml">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500&display=swap" rel="stylesheet">
<!--
The DorkNet downloads page. Published by ci/downloads/publish.py next to
manifest.json, which lists the files; this page only renders it. Colours
follow DorkNet.Server/site (tailwind.config.js).
-->
<style>
:root {
--bg: #0a0b0e; --surface: #131418; --surface-2: #1b1d22; --border: #23252b; --border-2: #34363d;
--text: #ececef; --muted: #a9abb4; --faint: #7e818d;
--brand: #0b95c7; --brand-hi: #22cfee; --brand-soft: rgba(11, 149, 199, .14); --on-brand: #fff;
--ok: #3ecf8e; --danger: #ef5b6b;
color-scheme: dark;
}
@media (prefers-color-scheme: light) {
:root:not([data-theme="dark"]) {
--bg: #f7f7f8; --surface: #ffffff; --surface-2: #f7f7f8; --border: #ececef; --border-2: #d3d4d9;
--text: #131418; --muted: #43464f; --faint: #5d606b;
--brand: #087aa8; --brand-hi: #0b95c7; --brand-soft: rgba(11, 149, 199, .10);
--ok: #1f9d68; --danger: #c93a4b;
color-scheme: light;
}
}
:root[data-theme="light"] {
--bg: #f7f7f8; --surface: #ffffff; --surface-2: #f7f7f8; --border: #ececef; --border-2: #d3d4d9;
--text: #131418; --muted: #43464f; --faint: #5d606b;
--brand: #087aa8; --brand-hi: #0b95c7; --brand-soft: rgba(11, 149, 199, .10);
--ok: #1f9d68; --danger: #c93a4b;
color-scheme: light;
}
* { box-sizing: border-box; }
body {
margin: 0; background: var(--bg); color: var(--text);
font: 15px/1.55 Inter, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
-webkit-font-smoothing: antialiased;
}
a { color: var(--brand-hi); }
code, .mono { font-family: "JetBrains Mono", SFMono-Regular, Menlo, monospace; font-size: .86em; }
.wrap { max-width: 980px; margin: 0 auto; padding: 0 16px; }
header { border-bottom: 1px solid var(--border); background: var(--surface); }
header .wrap { display: flex; align-items: center; gap: 12px; height: 60px; }
header img { width: 30px; height: 30px; }
header .name { font-weight: 700; letter-spacing: -.01em; }
header .name span { color: var(--faint); font-weight: 500; }
header .spacer { flex: 1; }
header a.signout { color: var(--muted); font-size: 13px; text-decoration: none; }
header a.signout:hover { color: var(--text); }
.intro { padding: 36px 0 8px; }
h1 { font-size: 28px; line-height: 1.2; letter-spacing: -.02em; margin: 0 0 6px; }
.lede { color: var(--muted); margin: 0; max-width: 60ch; }
.release { padding: 20px 0 8px; }
.release-head { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 12px; margin-bottom: 14px; }
.release-head h2 { font-size: 20px; margin: 0; letter-spacing: -.01em; }
.release-head .sub { color: var(--faint); font-size: 13px; }
.platform { margin: 0 0 22px; }
.platform h3 {
display: flex; align-items: center; gap: 8px; font-size: 13px; text-transform: uppercase;
letter-spacing: .06em; color: var(--muted); margin: 0 0 10px; font-weight: 600;
}
.platform h3 svg { width: 16px; height: 16px; }
.platform .hint { color: var(--faint); font-size: 13px; margin: -4px 0 10px; }
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(min(280px, 100%), 1fr)); gap: 12px; }
.card {
display: flex; flex-direction: column; gap: 10px; padding: 18px;
background: var(--surface); border: 1px solid var(--border); border-radius: 12px;
}
.card.primary { border-color: color-mix(in srgb, var(--brand) 55%, var(--border)); box-shadow: 0 0 0 3px var(--brand-soft); }
.card .top { display: flex; align-items: center; gap: 8px; }
.card h4 { margin: 0; font-size: 16px; }
.tag { font-size: 11px; font-weight: 600; padding: 2px 8px; border-radius: 999px; background: var(--brand-soft); color: var(--brand-hi); }
.card p { margin: 0; color: var(--muted); font-size: 14px; }
.card .file { color: var(--faint); word-break: break-all; }
.meta { display: flex; flex-wrap: wrap; gap: 4px 14px; color: var(--faint); font-size: 12.5px; }
.actions { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; margin-top: auto; padding-top: 4px; }
.btn {
display: inline-flex; align-items: center; gap: 8px; height: 38px; padding: 0 14px; border-radius: 8px;
font: 600 14px/1 inherit; text-decoration: none; cursor: pointer; border: 1px solid transparent;
}
.btn svg { width: 16px; height: 16px; }
.btn-primary { background: var(--brand); color: var(--on-brand); }
.btn-primary:hover { background: var(--brand-hi); }
.btn-secondary { background: var(--surface-2); color: var(--text); border-color: var(--border-2); }
.btn-secondary:hover { border-color: var(--faint); }
.btn:focus-visible { outline: 2px solid var(--brand-hi); outline-offset: 2px; }
.hash {
display: inline-flex; align-items: center; gap: 6px; height: 30px; padding: 0 8px; border-radius: 6px;
background: transparent; border: 1px dashed var(--border-2); color: var(--faint); cursor: pointer; font: inherit; font-size: 12px;
}
.hash:hover { color: var(--text); }
.hash.copied { color: var(--ok); border-color: var(--ok); }
.note { color: var(--faint); font-size: 13px; padding: 8px 0 40px; border-top: 1px solid var(--border); margin-top: 12px; }
.note p { margin: 12px 0 0; }
.state { padding: 48px 0; color: var(--muted); }
.state.error { color: var(--danger); }
</style>
</head>
<body>
<header>
<div class="wrap">
<img src="dorknet.svg" alt="">
<div class="name">DorkNet <span>Downloads</span></div>
<div class="spacer"></div>
<a class="signout" href="/cdn-cgi/access/logout">Sign out</a>
</div>
</header>
<main class="wrap">
<section class="intro">
<h1>Get Rec Room for DorkNet</h1>
<p class="lede">Pick your platform. Every file is built from the DorkNet source by our build server; the
SHA-256 next to each one lets you check your download.</p>
</section>
<div id="releases"><p class="state">Loading builds…</p></div>
<section class="note">
<p>These builds contain Rec Room's own game files. They're for the DorkNet community only: please don't
re-share them.</p>
</section>
</main>
<script>
(() => {
const ICONS = {
pc: '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="2" y="3" width="20" height="14" rx="2"/><path d="M8 21h8M12 17v4"/></svg>',
quest: '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M3 9a3 3 0 0 1 3-3h12a3 3 0 0 1 3 3v5a3 3 0 0 1-3 3h-3l-2-2.5h-2L9 17H6a3 3 0 0 1-3-3z"/></svg>',
download: '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M12 3v12M7 10l5 5 5-5M5 21h14"/></svg>',
copy: '<svg viewBox="0 0 24 24" width="13" height="13" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="12" height="12" rx="2"/><path d="M5 15V5a2 2 0 0 1 2-2h10"/></svg>',
};
const PLATFORMS = [
{ id: 'pc', title: 'PC', hint: 'Windows, with or without a VR headset.' },
{ id: 'quest', title: 'Meta Quest', hint: 'Turn on developer mode for your headset and allow USB debugging when asked.' },
];
const esc = (s) => String(s ?? '').replace(/[&<>"']/g, (c) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' })[c]);
const size = (b) => b >= 2 ** 30 ? `${(b / 2 ** 30).toFixed(2)} GB`
: b >= 2 ** 20 ? `${(b / 2 ** 20).toFixed(1)} MB` : `${Math.max(1, Math.round(b / 2 ** 10))} KB`;
const date = (iso) => iso ? new Date(iso).toLocaleDateString(undefined, { year: 'numeric', month: 'short', day: 'numeric' }) : '';
// What each file is for, in players' terms. Keyed by the file ids publish.py writes.
function describe(f, rel, site) {
const questPage = site ? `<a href="${esc(site)}/quest" target="_blank" rel="noopener">Quest install page</a>` : 'Quest install page of the DorkNet website';
switch (f.id) {
case 'pc-full': return {
title: 'Full game', tag: 'Start here', primary: true,
body: `Rec Room ${esc(rel.title)} with DorkNet already set up. Unzip it anywhere and run
${rel.pc_exe ? `<code>${esc(rel.pc_exe)}</code>` : 'the game'}.`,
};
case 'pc-patch': return {
title: 'Patch only',
body: `Just the DorkNet part (MelonLoader and the DorkNet mod), for a copy of the game you already have.
The DorkNet launcher installs this for you. Skip it if you got the full game.`,
};
case 'quest': return {
title: 'Quest app', tag: 'Start here', primary: true,
body: `Plug in your headset and install from the ${questPage} (Chrome or Edge), or drop the file onto SideQuest.`,
};
case 'quest-installer': return {
title: 'Install scripts',
body: `Prefer the command line? Unzip these next to the Quest app and run <code>install.cmd</code> (Windows)
or <code>install.sh</code> (macOS / Linux). Needs adb.`,
};
default: return { title: f.name, body: '' };
}
}
function card(f, rel, site) {
const d = describe(f, rel, site);
const build = f.build ? `<span>Build ${esc(f.build)}</span>` : '';
return `
<article class="card${d.primary ? ' primary' : ''}">
<div class="top"><h4>${esc(d.title)}</h4>${d.tag ? `<span class="tag">${esc(d.tag)}</span>` : ''}</div>
<p>${d.body}</p>
<div class="meta"><span class="mono file">${esc(f.name)}</span></div>
<div class="meta"><span>${size(f.size)}</span><span>Updated ${esc(date(f.built_at))}</span>${build}</div>
<div class="actions">
<a class="btn ${d.primary ? 'btn-primary' : 'btn-secondary'}" href="${esc(f.path)}" download>${ICONS.download}Download</a>
<button class="hash mono" type="button" data-sha="${esc(f.sha256)}" title="Copy SHA-256: ${esc(f.sha256)}">${ICONS.copy}SHA-256 ${esc(f.sha256.slice(0, 12))}…</button>
</div>
</article>`;
}
function render(m) {
const root = document.getElementById('releases');
const releases = (m.releases || []).filter((r) => (r.files || []).length);
if (!releases.length) { root.innerHTML = '<p class="state">No builds have been published yet.</p>'; return; }
root.innerHTML = releases.map((rel) => {
const sections = PLATFORMS.map((p) => {
const files = rel.files.filter((f) => f.platform === p.id);
if (!files.length) return '';
return `
<div class="platform">
<h3>${ICONS[p.id]}${esc(p.title)}</h3>
<p class="hint">${esc(p.hint)}</p>
<div class="grid">${files.map((f) => card(f, rel, m.site_url)).join('')}</div>
</div>`;
}).join('');
const game = rel.game_version ? `Game build of ${esc(date(rel.game_version + 'T12:00:00Z'))}` : '';
return `
<section class="release">
<div class="release-head"><h2>Rec Room ${esc(rel.title)}</h2><span class="sub">${game}</span></div>
${sections}
</section>`;
}).join('');
}
document.addEventListener('click', async (e) => {
const b = e.target.closest('.hash');
if (!b) return;
try {
await navigator.clipboard.writeText(b.dataset.sha);
b.classList.add('copied');
setTimeout(() => b.classList.remove('copied'), 1500);
} catch { /* clipboard blocked: the full hash is in the tooltip */ }
});
fetch('manifest.json', { cache: 'no-store' })
.then((r) => { if (r.status === 404) return {}; if (!r.ok) throw new Error(`HTTP ${r.status}`); return r.json(); })
.then(render)
.catch((err) => {
document.getElementById('releases').innerHTML =
`<p class="state error">Couldn't load the list of builds (${esc(err.message)}). Try reloading the page.</p>`;
});
})();
</script>
</body>
</html>
+294
View File
@@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""Publish DorkNet client builds to the downloads page.
stage the build output in out/ -> out/downloads/<release>/ under the
names players see, plus release.json describing those files
upload that folder -> the downloads bucket, then merge release.json into
the bucket's manifest.json and upload the page (index.html)
The build scripts keep their own file names on purpose: the launcher fetches
dorknet-clientpatch-<VersionKey>.zip from GitHub Releases by that exact name.
Only the published copies are renamed:
dorknet-client-<key>.zip DorkNet-<release>-PC-Full.zip
dorknet-clientpatch-<key>.zip DorkNet-<release>-PC-Patch.zip
DorkNet-Quest-<ver>/DorkNet-<ver>.xapk DorkNet-<release>-Quest.xapk
DorkNet-Quest-<ver>/install.*, README DorkNet-<release>-Quest-Installer.zip
The bucket is served read-only as a static website (Garage's web endpoint).
upload talks to its S3 API with rclone; the endpoint, bucket and key come
from the environment:
DOWNLOADS_S3_ENDPOINT S3 API URL, e.g. http://garage-<uuid>:3900
DOWNLOADS_BUCKET bucket name, e.g. downloads.example.com
DOWNLOADS_S3_REGION optional, default garage
DOWNLOADS_S3_KEY_ID / DOWNLOADS_S3_SECRET
DORKNET_SITE_URL optional: the DorkNet server's website, linked
from the page for the browser Quest installer
python3 ci/downloads/publish.py stage --out out --release 2023.03 \\
--title "March 2023" --game-version 2023-03-21 --version-key march_2023_03_21
python3 ci/downloads/publish.py upload --staged out/downloads
"""
from __future__ import annotations
import argparse
import datetime
import hashlib
import json
import os
import shutil
import subprocess
import sys
import zipfile
from pathlib import Path
from typing import NoReturn
HERE = Path(__file__).resolve().parent
REPO = HERE.parent.parent
PAGE_FILES = { # bucket path -> source
"index.html": HERE / "index.html",
"dorknet.svg": REPO / "DorkNet.Server" / "site" / "public" / "brand" / "dorknet.svg",
}
NO_CACHE = "Cache-Control: no-cache"
INSTALLER_NOTE = """Unzip these files into the same folder as {xapk}, then
run install.cmd (Windows) or ./install.sh (macOS / Linux).
"""
def fail(msg: str) -> NoReturn:
print(f"error: {msg}", file=sys.stderr)
sys.exit(1)
def step(msg: str) -> None:
print(f"==> {msg}", flush=True)
def human(n: int) -> str:
return f"{n / 2**20:,.1f} MB" if n >= 2**20 else f"{max(1, round(n / 2**10))} KB"
def sha256(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as f:
while chunk := f.read(1 << 20):
h.update(chunk)
return h.hexdigest()
def link_or_copy(src: Path, dst: Path) -> None:
"""Hard-link where possible: the full client is several GB."""
dst.unlink(missing_ok=True)
try:
os.link(src, dst)
except OSError:
shutil.copy2(src, dst)
def game_exe(full_zip: Path) -> str | None:
"""The game's exe: the one with a matching <name>_Data folder at the root."""
with zipfile.ZipFile(full_zip) as z:
names = z.namelist()
tops = {n.split("/", 1)[0] for n in names}
exes = sorted(t for t in tops if t.lower().endswith(".exe") and f"{t[:-4]}_Data" in tops)
return exes[0] if exes else None
# ---- stage ----------------------------------------------------------------
def cmd_stage(a: argparse.Namespace) -> None:
out = Path(a.out)
dest = Path(a.dest) if a.dest else out / "downloads"
rel_dir = dest / a.release
if rel_dir.exists():
shutil.rmtree(rel_dir)
rel_dir.mkdir(parents=True)
prefix = f"DorkNet-{a.release}"
staged: list[tuple[str, str, Path]] = [] # (id, platform, staged file)
extra: dict[str, str] = {}
full = out / "pc" / f"dorknet-client-{a.version_key}.zip"
patch = out / "pc" / f"dorknet-clientpatch-{a.version_key}.zip"
if full.is_file():
dst = rel_dir / f"{prefix}-PC-Full.zip"
link_or_copy(full, dst)
staged.append(("pc-full", "pc", dst))
exe = game_exe(full)
if exe:
extra["pc_exe"] = exe
if patch.is_file():
dst = rel_dir / f"{prefix}-PC-Patch.zip"
link_or_copy(patch, dst)
staged.append(("pc-patch", "pc", dst))
packages = sorted((out / "quest").glob("DorkNet-Quest-*/"))
if len(packages) > 1:
fail(f"more than one Quest package folder in {out / 'quest'}: {', '.join(p.name for p in packages)}")
if packages:
pkg = packages[0]
xapks = sorted(pkg.glob("*.xapk"))
if len(xapks) != 1:
fail(f"expected one .xapk in {pkg}, found {len(xapks)}")
xapk = rel_dir / f"{prefix}-Quest.xapk"
link_or_copy(xapks[0], xapk)
staged.append(("quest", "quest", xapk))
installer = rel_dir / f"{prefix}-Quest-Installer.zip"
with zipfile.ZipFile(installer, "w", zipfile.ZIP_DEFLATED) as z:
for name in ("install.cmd", "install.ps1", "install.sh"):
src = pkg / name
if not src.is_file():
fail(f"{src} missing")
info = zipfile.ZipInfo(name, date_time=(1980, 1, 1, 0, 0, 0))
info.external_attr = (0o755 if name.endswith(".sh") else 0o644) << 16
info.compress_type = zipfile.ZIP_DEFLATED
z.writestr(info, src.read_bytes())
readme = pkg / "README.txt"
text = INSTALLER_NOTE.format(xapk=xapk.name) + (readme.read_text(encoding="utf-8") if readme.is_file() else "")
info = zipfile.ZipInfo("README.txt", date_time=(1980, 1, 1, 0, 0, 0))
info.compress_type = zipfile.ZIP_DEFLATED
z.writestr(info, text)
staged.append(("quest-installer", "quest", installer))
if not staged:
fail(f"nothing to publish in {out}")
built_at = datetime.datetime.now(datetime.timezone.utc).replace(microsecond=0).isoformat()
files = []
for fid, platform, path in staged:
step(f"{path.name}: hashing")
files.append({
"id": fid,
"platform": platform,
"name": path.name,
"path": f"{a.release}/{path.name}",
"size": path.stat().st_size,
"sha256": sha256(path),
"build": a.build,
"commit": a.commit,
"built_at": built_at,
})
release = {"id": a.release, "title": a.title, "game_version": a.game_version, **extra, "files": files}
(dest / "release.json").write_text(json.dumps(release, indent=2) + "\n", encoding="utf-8")
print()
for f in files:
print(f" {f['name']} {human(f['size'])} {f['sha256'][:16]}…")
print(f"Staged {len(files)} file(s) in {rel_dir}")
# ---- upload ---------------------------------------------------------------
def rclone_env() -> dict[str, str]:
env = dict(os.environ)
for k in ("DOWNLOADS_S3_ENDPOINT", "DOWNLOADS_BUCKET", "DOWNLOADS_S3_KEY_ID", "DOWNLOADS_S3_SECRET"):
if not env.get(k):
fail(f"{k} is not set")
env.update({
"RCLONE_CONFIG_DL_TYPE": "s3",
"RCLONE_CONFIG_DL_PROVIDER": "Other",
"RCLONE_CONFIG_DL_ENDPOINT": env["DOWNLOADS_S3_ENDPOINT"],
"RCLONE_CONFIG_DL_REGION": env.get("DOWNLOADS_S3_REGION") or "garage",
"RCLONE_CONFIG_DL_ACCESS_KEY_ID": env["DOWNLOADS_S3_KEY_ID"],
"RCLONE_CONFIG_DL_SECRET_ACCESS_KEY": env["DOWNLOADS_S3_SECRET"],
"RCLONE_CONFIG_DL_FORCE_PATH_STYLE": "true",
"RCLONE_CONFIG_DL_NO_CHECK_BUCKET": "true",
})
return env
def rclone(env: dict[str, str], *args: str, stdin: str | None = None, check: bool = True) -> subprocess.CompletedProcess:
r = subprocess.run(["rclone", *args], env=env, input=stdin, capture_output=True, text=True)
if check and r.returncode != 0:
fail(f"rclone {args[0]} failed ({r.returncode}): {r.stderr.strip()}")
return r
def merge(manifest: dict, release: dict) -> dict:
"""Replace this release's files that were just built; keep the others
(a push build may rebuild only one platform)."""
releases = [r for r in manifest.get("releases", []) if r.get("id") != release["id"]]
old = next((r for r in manifest.get("releases", []) if r.get("id") == release["id"]), {})
new_ids = {f["id"] for f in release["files"]}
kept = [f for f in old.get("files", []) if f.get("id") not in new_ids]
merged = {**old, **release, "files": kept + release["files"]}
if "pc_exe" not in release and "pc_exe" in old:
merged["pc_exe"] = old["pc_exe"]
order = ["pc-full", "pc-patch", "quest", "quest-installer"]
merged["files"].sort(key=lambda f: order.index(f["id"]) if f["id"] in order else len(order))
releases.append(merged)
releases.sort(key=lambda r: r["id"], reverse=True)
out = {**manifest, "releases": releases,
"updated": datetime.datetime.now(datetime.timezone.utc).replace(microsecond=0).isoformat()}
site = os.environ.get("DORKNET_SITE_URL", "").rstrip("/")
if site:
out["site_url"] = site
return out
def cmd_upload(a: argparse.Namespace) -> None:
staged = Path(a.staged)
release = json.loads((staged / "release.json").read_text(encoding="utf-8"))
env = rclone_env()
bucket = f"dl:{env['DOWNLOADS_BUCKET']}"
rel = release["id"]
# 1. the files, before the manifest that points at them
for f in release["files"]:
step(f"uploading {f['name']} ({human(f['size'])})")
rclone(env, "copyto", str(staged / f["path"]), f"{bucket}/{f['path']}",
"--header-upload", NO_CACHE, "--s3-upload-concurrency", "8", "--s3-chunk-size", "64M")
# 2. merge into the live manifest
r = rclone(env, "cat", f"{bucket}/manifest.json", check=False)
manifest = json.loads(r.stdout) if r.returncode == 0 and r.stdout.strip() else {}
manifest = merge(manifest, release)
# 3. the page, then the manifest
for path, src in PAGE_FILES.items():
step(f"uploading {path}")
rclone(env, "copyto", str(src), f"{bucket}/{path}", "--header-upload", NO_CACHE)
step("uploading manifest.json")
rclone(env, "rcat", f"{bucket}/manifest.json", "--header-upload", NO_CACHE,
"--header-upload", "Content-Type: application/json", stdin=json.dumps(manifest, indent=2) + "\n")
# 4. drop files of this release the manifest no longer lists (renames)
listed = {f["path"] for r in manifest["releases"] if r["id"] == rel for f in r["files"]}
r = rclone(env, "lsf", f"{bucket}/{rel}/", "--files-only", check=False)
for name in r.stdout.split():
if f"{rel}/{name}" not in listed:
step(f"removing stale {rel}/{name}")
rclone(env, "deletefile", f"{bucket}/{rel}/{name}")
print(f"Published {len(release['files'])} file(s) for {release['title']}.")
# ---- cli ------------------------------------------------------------------
def main() -> None:
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = p.add_subparsers(dest="cmd", required=True)
s = sub.add_parser("stage", help="out/ -> out/downloads/<release>/ + release.json")
s.add_argument("--out", default="out", help="the build output folder (default: out)")
s.add_argument("--dest", help="where to stage (default: <out>/downloads)")
s.add_argument("--release", required=True, help="release id used in file names and paths, e.g. 2023.03")
s.add_argument("--title", required=True, help='shown on the page, e.g. "March 2023"')
s.add_argument("--game-version", required=True, help="the stock build's date, e.g. 2023-03-21")
s.add_argument("--version-key", required=True, help="the PC build's version key, e.g. march_2023_03_21")
s.add_argument("--build", default=os.environ.get("BUILD_NUMBER", ""), help="CI build number (default: $BUILD_NUMBER)")
s.add_argument("--commit", default=os.environ.get("GIT_COMMIT", ""), help="source commit (default: $GIT_COMMIT)")
s.set_defaults(fn=cmd_stage)
u = sub.add_parser("upload", help="staged folder -> downloads bucket + manifest.json + page")
u.add_argument("--staged", default="out/downloads", help="the folder `stage` wrote (default: out/downloads)")
u.set_defaults(fn=cmd_upload)
a = p.parse_args()
a.fn(a)
if __name__ == "__main__":
main()
+70 -8
View File
@@ -11,14 +11,24 @@ The builds run on a **build agent**, not on Jenkins itself:
job runs on it through the `dorknet-builder` label.
- **Jenkins itself:** stays on the stock image and runs no builds.
| Platform | Output (Jenkins artifacts) | How |
|---|---|---|
| PC | `out/pc/dorknet-clientpatch-march_2023_03_21.zip`: MelonLoader, mod, config template, Steam stub, `manifest.json`; no game files | `tools/build_pc.py patcher` |
| PC | `out/pc/dorknet-client-march_2023_03_21.zip`: the stock Steam build with the patcher applied and your config; ready to run | `tools/build_pc.py client` |
| Quest | `out/quest/dorknet-recroom-<code>.apk` and `out/quest/DorkNet-Quest-<ver>/` (`.xapk` + installers) | `DorkNet.Quest/scripts/build.py build` |
Each build is published to the **downloads page** under a name players can
read, and the same files are kept as Jenkins artifacts
(`out/downloads/2023.03/`):
The full PC client and the Quest packages contain game files, so keep them on
Jenkins. Don't publish them anywhere public.
| Download | What it is | Built from |
|---|---|---|
| `DorkNet-2023.03-PC-Full.zip` | the stock Steam build with the patcher applied and your config; ready to run | `tools/build_pc.py client` → `dorknet-client-march_2023_03_21.zip` |
| `DorkNet-2023.03-PC-Patch.zip` | MelonLoader, mod, config template, Steam stub, `manifest.json`; no game files | `tools/build_pc.py patcher` → `dorknet-clientpatch-march_2023_03_21.zip` |
| `DorkNet-2023.03-Quest.xapk` | patched APK + OBB, for the browser installer or SideQuest | `DorkNet.Quest/scripts/build.py build` → `DorkNet-Quest-<ver>/DorkNet-<ver>.xapk` |
| `DorkNet-2023.03-Quest-Installer.zip` | `install.cmd` / `install.ps1` / `install.sh` + README, unzipped next to the `.xapk` | the same package folder |
The build scripts keep their own names: the launcher downloads
`dorknet-clientpatch-<VersionKey>.zip` from GitHub Releases by that exact
name. `ci/downloads/publish.py` renames only the published copies.
The full PC client and the Quest packages contain game files. The downloads
page is behind Cloudflare Access for that reason; don't publish them
anywhere public.
## Where the stock builds come from
@@ -54,7 +64,11 @@ Resolve a table for another build locally with `build.py build --dump-cs`.
- `Jenkinsfile` and `ci/jenkins/` rebuild both.
- **Build with Parameters** runs a manual build:
- `PLATFORMS` (both / pc / quest), and `PC_FULL_CLIENT` for the full PC client;
- snapshot tags, and per-run repository overrides.
- snapshot tags, and per-run repository overrides;
- `PUBLISH_DOWNLOADS` (on by default) to publish to the downloads page.
Push builds publish too, but only the platform they rebuilt: the page keeps
the other platform's last files.
## Setup
@@ -124,6 +138,9 @@ Save, open the node, and copy its **secret**.
| `ARCHIVE_PC_REPO` | Steam repository folder on the FTP, e.g. `Rec Room Versions/steam-v2` (plain text, not URL-encoded) |
| `ARCHIVE_QUEST_REPO` | Quest repository folder, e.g. `Rec Room Versions/quest-v2` |
| `ARCHIVE_FTP_TLS` | `explicit` (default), `implicit` or `none` |
| `DOWNLOADS_S3_ENDPOINT` | the downloads bucket's S3 API, internal address, e.g. `http://garage-<service uuid>:3900` |
| `DOWNLOADS_BUCKET` | its name, e.g. `downloads.example.com` |
| `DORKNET_SITE_URL` | optional: the DorkNet server's website (e.g. `https://dorknet.example.com`); the page links its Quest installer |
3. **Deploy.** The first image build downloads the Android NDK (~1.5 GB) and
takes a while. The node shows as connected in Jenkins once the agent is up.
@@ -159,6 +176,7 @@ Add these under **Manage Jenkins → Credentials**, with exactly these IDs:
| `quest-keystore-password` | Secret text | its store/key password |
| `clientmod-config` | Secret file | your `dorknet-clientmod.json` |
| `quest-config` | Secret file | your `dorknet-quest.cfg` |
| `downloads-s3` | Username with password | S3 key for the downloads bucket: access key ID / secret. Give it read and write on that bucket only. |
**Use the keystore your players' installs were signed with**, the
`dorknet-debug.keystore` from your earlier local builds (password `dorknet`). A
@@ -184,6 +202,50 @@ In the repository's **Settings → Webhooks**:
- Content type: `application/json`
- Events: just the push event
### 8. The downloads page
The page (`ci/downloads/index.html`) is a static file in an S3 bucket. It
renders the bucket's `manifest.json`, which `publish.py upload` rewrites after
each build. With Garage (the S3 store on the Coolify server):
1. **Bucket and key.** Create the bucket, serve it as a website, and give
Jenkins a key for that bucket only:
```sh
garage bucket create downloads.example.com
garage bucket website --allow downloads.example.com
garage key create jenkins-downloads
garage bucket allow --read --write --key <key id> downloads.example.com
```
Store the key as the `downloads-s3` credential. Garage's web endpoint (port
`3902`) serves a bucket whose name is the request's host name, read-only.
2. **Route.** Point the hostname at the web endpoint with a Coolify dynamic
proxy configuration (**Servers → Proxy → Dynamic Configurations**):
```yaml
http:
routers:
downloads:
entryPoints: [http]
service: downloads
rule: Host(`downloads.example.com`)
services:
downloads:
loadBalancer:
passHostHeader: true
servers:
- url: "http://garage-<service uuid>:3902"
```
3. **Access.** Put the hostname behind a Cloudflare Access application
**before** the first publish, with the policies of the people who may
download. That group needs no other application.
Files are uploaded with `Cache-Control: no-cache`, so Cloudflare never serves
an old build after a new one replaces it. Each release keeps one copy of each
file; a new build overwrites it.
### Disk
The agent's `dorknet_cache` volume holds the restored stock builds, several GB
+5
View File
@@ -28,6 +28,11 @@ services:
ARCHIVE_PC_REPO: ${ARCHIVE_PC_REPO:-}
ARCHIVE_QUEST_REPO: ${ARCHIVE_QUEST_REPO:-}
ARCHIVE_FTP_TLS: ${ARCHIVE_FTP_TLS:-explicit}
# The downloads page's bucket (S3 API, internal address) and the DorkNet
# website the page links to. See ci/downloads/publish.py.
DOWNLOADS_S3_ENDPOINT: ${DOWNLOADS_S3_ENDPOINT:-}
DOWNLOADS_BUCKET: ${DOWNLOADS_BUCKET:-}
DORKNET_SITE_URL: ${DORKNET_SITE_URL:-}
volumes:
- dorknet_cache:/home/jenkins/dorknet-cache
- agent_work:/home/jenkins/agent