← All posts

A self-updating open-data site with GitHub Actions, Pages, and data branches

Village Finder uses GitHub Pages, Actions, releases, and isolated branches to publish changing open data without an application server.

A self-updating open-data site with GitHub Actions, Pages, and data branches
Contents

Briefly

Village Finder serves maps and information for more than 50,000 Indian villages, refreshes geographic datasets daily, and publishes market prices without an application backend or database server. Its core infrastructure bill is effectively zero.

The project combines GitHub Pages, GitHub Actions, releases, and dedicated data branches. Its useful lesson is not that every system should avoid servers: it is that reviewed source records and high-churn generated files need different delivery paths.

What happened

The author split incoming information into two pipelines. Slowly changing administrative registries and village boundaries go through a conventional pull request. More than 90 tests verify their structure, and a changelog explains which records moved or changed classification. The resulting commit history becomes an auditable public record.

Daily agricultural-market quotes, weekly welfare updates, and regenerated map tiles follow another lifecycle. Reviewing a 10,000-row JSON diff every day is rarely a useful human task, so automation writes these reproducible artifacts directly to isolated branches. Each publication starts from a fresh, history-free commit and force-pushes the current snapshot.

That keeps generated history from expanding the main repository. Files in the public branch are also available through raw.githubusercontent.com, so the browser can fetch the latest JSON with cross-origin access enabled.

Why it matters

A static site often implies that every data change requires a full rebuild and deployment. Here, the Pages frontend is decoupled from volatile datasets: the page requests the current file from a data branch, allowing users to see updated market figures without publishing the interface again.

This is not a universal replacement for a backend. GitHub has limits around file size, rate, and storage patterns. For gigabyte-scale cadastral maps, the project retains Cloudflare R2 because it needs economical distribution and HTTP range requests for geospatial formats.

The split also makes ownership explicit. Authoritative administrative data receives validation and review, while derived snapshots can be replaced automatically. Keeping that boundary is more important than the branch trick itself; otherwise source facts, temporary outputs, and application code quickly become indistinguishable.

In practice

Start with the data lifecycle, not with another scheduled job. This pattern is most suitable when public data can be rebuilt from an upstream source and a published snapshot is acceptable.

  1. Keep pull requests, structural tests, and a readable changelog for infrequent, consequential source changes.
  2. Publish frequent reproducible files to an isolated branch, assembled in a temporary directory away from the main checkout.
  3. Fetch small JSON assets directly from the public branch, after checking GitHub limits and access expectations.
  4. Use purpose-built storage for large files requiring HTTP range requests, or overlay a branch snapshot into the static deployment workspace.
  5. Distinguish temporary upstream API failures from code failures. The project exits with EX_TEMPFAIL (75), preserves yesterday’s snapshot, and emits a concise notice instead of turning planned upstream downtime into a false alarm.

There are sharp edges. git diff --quiet ignores untracked files during initial setup, and Git pathspec rules are not shell globs. Test empty repositories and inspect git status --porcelain so a green workflow cannot silently omit a first data upload.

Takeaway

Village Finder is a practical example of using GitHub tools for more than source code. Pages delivers the interface, Actions compiles data, releases distribute versioned downloads, and branches offer a simple channel for frequently replaced files.

The architecture fits open catalogs, statistics, maps, and similar datasets where snapshots are acceptable and no custom request-handling server is required. Its value is not the slogan of “zero servers,” but the disciplined separation of reviewed records from disposable generated artifacts.