Initial commit: WebGPU portfolio — generative-art site with DOM overlay
This commit is contained in:
328
AGENTS.md
Normal file
328
AGENTS.md
Normal file
@@ -0,0 +1,328 @@
|
||||
# AGENTS.md
|
||||
|
||||
Conventions and operational notes for AI agents (and humans) working in this repo.
|
||||
Read this before editing. Keep it updated when conventions change.
|
||||
|
||||
## Project
|
||||
|
||||
A portfolio website using a **hybrid architecture**: a WebGPU `<canvas>` renders
|
||||
the generative-art background, and a DOM overlay on top of it owns every UI
|
||||
element (nav, titles, body text, button). The canvas is the only GPU surface;
|
||||
all interactive UI is standard HTML/CSS. Pointer position is still captured at
|
||||
the window level and fed to the shader as a uniform for background reactivity.
|
||||
|
||||
### North-star goals (from the kickoff decisions)
|
||||
- Tech: **Vite + TypeScript**, `@webgpu/types` for ambient types.
|
||||
- Rendering: a **single fullscreen-triangle fragment shader** draws the procedural
|
||||
generative-art background only. All UI (text, nav, button) is **DOM overlay**.
|
||||
- Aesthetic: **abstract generative art** (domain-warped fbm flow fields, noise).
|
||||
- Sections: **Hero / About / Projects / Contact**, driven by `scene` state in the
|
||||
app loop. Scene changes update the DOM overlay content and shift the shader's
|
||||
`palette`.
|
||||
- Deliver style: **solid foundation first, extend later.**
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev # vite dev server, opens browser
|
||||
npm run build # tsc --noEmit (typecheck) + vite build
|
||||
npm run typecheck # tsc --noEmit — run before considering work done
|
||||
npm run preview # serve the built dist/
|
||||
```
|
||||
|
||||
**Every task that changes code MUST end with `npm run typecheck` (and `npm run build`
|
||||
if you want a full check).** Do not commit code that fails typecheck.
|
||||
|
||||
## Stack / tooling notes
|
||||
|
||||
- TypeScript `strict`, plus `noUnusedLocals` / `noUnusedParameters`. Remove unused
|
||||
params/vars or prefix with `_` rather than leaving them.
|
||||
- WGSL shaders live in `src/engine/shaders/*.wgsl` and are imported with Vite's
|
||||
`?raw` suffix (`import src from './x.wgsl?raw'`). Keep all GPU code in `.wgsl`
|
||||
files — do not inline large shader strings in `.ts`.
|
||||
- `@webgpu/types` provides global `GPUDevice`, etc. Do not import them; they are
|
||||
ambient via `types` in `tsconfig.json`.
|
||||
- Target a modern WebGPU browser (recent Chrome/Edge/ChromeOS/Safari preview).
|
||||
`index.html` ships a `#fallback` notice shown when `navigator.gpu` is missing.
|
||||
- `opentype.js` is a runtime dep (font parsing for the Slug text generator);
|
||||
`@types/opentype.js` is a dev dep. **The Slug text engine is dormant** — see
|
||||
the "Text rendering" section. The deps are kept so the dormant files still
|
||||
typecheck; they are not wired into the render loop.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.ts # boot: create App, fall back on no WebGPU
|
||||
├── app.ts # rAF loop, input→uniforms, scene state, DOM overlay wiring
|
||||
├── engine/
|
||||
│ ├── renderer.ts # device/context/pipeline, single procedural pass
|
||||
│ ├── shaders/
|
||||
│ │ └── fullscreen.wgsl # procedural background only, one fragment shader
|
||||
│ └── text/ # DORMANT Slug text engine (kept, not wired in)
|
||||
│ ├── slug-generator.ts # CPU: opentype.js → curve/band texture data (Slug port)
|
||||
│ ├── text-layout.ts # CPU: text string → instance buffer data per glyph
|
||||
│ ├── text-renderer.ts # GPU: instanced quad pipeline + textures + bind groups
|
||||
│ └── text.wgsl # WGSL Slug shader (ray-traces quadratic Béziers per frag)
|
||||
├── ui/
|
||||
│ ├── input.ts # window-level pointer position capture (mouse uniform)
|
||||
│ └── overlay.ts # DOM overlay: scene toggling, resume/project HTML builders
|
||||
└── data/
|
||||
├── resume.ts # ResumeData type + RESUME object (name, contact, work, edu, skills)
|
||||
└── projects/
|
||||
├── index.ts # ProjectPage type + PROJECTS registry (add new projects here)
|
||||
├── radiance-caching.ts
|
||||
├── point-cloud.ts
|
||||
├── evol-engine.ts
|
||||
├── auv-simulation.ts
|
||||
└── improv-gfx.ts
|
||||
```
|
||||
|
||||
### Layout: DOM-owned
|
||||
|
||||
UI layout is owned by **CSS** in `index.html` (flexbox + viewport units). There is
|
||||
no shared pixel-space layout contract between the DOM and the shader anymore: the
|
||||
shader draws only the fullscreen background, so it needs only `resolution`, `time`,
|
||||
`scene`, and `mouse` — no per-element bounds.
|
||||
|
||||
When you add a new UI element: add it to the DOM overlay in `index.html`, style it
|
||||
with CSS, and wire any click/hover in `overlay.ts`. You do not need to touch
|
||||
`renderer.ts`, `fullscreen.wgsl`, or uniforms unless the background itself must
|
||||
react to the new element.
|
||||
|
||||
### Hybrid DOM-overlay principle
|
||||
|
||||
The **DOM overlay** (`#overlay` in `index.html`, `z-index: 1` over the canvas)
|
||||
owns all visible UI: the nav bar, scene title/subtitle/body, and the cycle button.
|
||||
The canvas underneath draws the generative background only. The overlay is
|
||||
`pointer-events: none` with interactive children set to `pointer-events: auto`, so
|
||||
pointermove still reaches the window for the shader's mouse uniform while clicks
|
||||
land on the right element. Do not add UI shapes inside the fragment shader — if
|
||||
you need a new visual element, prefer a DOM element with CSS.
|
||||
|
||||
### Data layer
|
||||
|
||||
Content lives in `src/data/` as typed TypeScript objects. No runtime parsing, no
|
||||
fetches — data is imported directly and rendered into HTML at first visit.
|
||||
|
||||
**Resume** (`src/data/resume.ts`):
|
||||
- Exports `ResumeData` interface and `RESUME` object.
|
||||
- Fields: `name`, `contact[]`, `location`, `summary`, `work[]`, `education[]`, `skills[]`.
|
||||
- `overlay.ts` has `buildResumeHTML(data)` that generates the full resume DOM from
|
||||
the typed object. To update resume content, edit `resume.ts`.
|
||||
|
||||
**Projects** (`src/data/projects/`):
|
||||
- Each project is its own `.ts` file exporting a `ProjectPage`:
|
||||
`{ id, name, tagline, date, tech[], content }` where `content` is free-form HTML.
|
||||
- `index.ts` exports the `ProjectPage` type and the `PROJECTS` registry array.
|
||||
- **To add a project:** create `src/data/projects/my-project.ts`, import it in
|
||||
`index.ts`, add to the `PROJECTS` array. It auto-appears on the Projects page.
|
||||
- **To remove a project:** remove its import and array entry from `index.ts`.
|
||||
- Clicking a project card opens a detail view; a back button returns to the list.
|
||||
Navigation is handled by event delegation on `#projects-section` in `overlay.ts`.
|
||||
- The list resets on every nav entry (if you're viewing a detail and switch tabs,
|
||||
coming back shows the list again).
|
||||
|
||||
**To add a new data-driven section**, follow the Projects/Resume pattern:
|
||||
1. Create `src/data/whatever.ts` with a typed interface + data export.
|
||||
2. Add a `#whatever-section` div to `index.html` with the same CSS pattern
|
||||
(opacity transition, `.visible` class with fade-in animation).
|
||||
3. In `overlay.ts`: query the div, add a `whateverBuilt` flag, build the HTML
|
||||
from data on first visit, toggle `.visible`/`.hidden` classes in `setScene()`.
|
||||
4. Extend `colorBase`/`speedBase` arrays in `app.ts` and update cycle modulo.
|
||||
|
||||
### Coordinate conventions
|
||||
|
||||
- Layout space = **drawing-buffer pixels** (CSS pixels × `dpr`, capped at 2).
|
||||
- Shader convention: origin **bottom-left**, **y up**. `app.ts` flips input y:
|
||||
`mouse.y = res.y - clientY * dpr`.
|
||||
- `Renderer` keeps `resolution` in actual device pixels.
|
||||
|
||||
## WGSL / shader conventions
|
||||
|
||||
- One oversized triangle, 3 hardcoded vertices in the vertex shader; no vertex buffers.
|
||||
- Uniforms are a single `var<uniform> u: Uniforms` at group 0, binding 0.
|
||||
- Update `UNIFORM_FLOATS` in `renderer.ts` whenever you add a uniform field, and keep
|
||||
the field order in TS `render(state)` identical to the WGSL struct declaration.
|
||||
- Keep the shader readable: section it with `// ---------- name ----------` comments.
|
||||
|
||||
## SDF reference helpers already in the shader
|
||||
|
||||
- `sdRoundBox(p, halfExtent, radius)` — rounded rect (kept for future background motifs)
|
||||
- `hash`, `noise`, `fbm` — value-noise fbm for the generative background
|
||||
- `palette(s)` — hue shift keyed off `scene`
|
||||
|
||||
To add more shapes, add `sd*` helpers next to the existing ones.
|
||||
|
||||
## Currently rendered
|
||||
|
||||
- **Background (shader):** domain-warped fbm flow field, palette indexed by `scene`,
|
||||
vignette + grain. `scene` cycles 0→1→2→3→4 on each button click or nav click; each
|
||||
scene shifts `palette`. `colorBase` and `speedBase` arrays in `app.ts` are indexed
|
||||
by scene and must be extended when adding a new scene.
|
||||
- **Nav bar (DOM):** top-center row of 5 `<button>` labels (HERO / ABOUT / PROJECTS /
|
||||
CONTACT / RESUME). Clicking a label switches to that scene. The active scene is
|
||||
highlighted via the `.active` class (set by `overlay.setScene`); hover handled by CSS.
|
||||
- **Scene content (DOM):** per-scene title, subtitle, and body text from the `SCENES`
|
||||
array in `app.ts`. Used for Hero (0), About (1), and Contact (3). Hidden via
|
||||
`opacity: 0` transition for Projects (2) and Resume (4).
|
||||
- **Projects section (DOM):** `#projects-section` — shown for scene 2. Renders
|
||||
project cards from `src/data/projects/`. Clicking a card opens a detail view
|
||||
with a back button (event delegation in `overlay.ts`). Returns to list on every
|
||||
nav entry. Hides the cycle button.
|
||||
- **Resume section (DOM):** `#resume-section` — shown for scene 4. Rendered from
|
||||
`RESUME` data in `src/data/resume.ts` via `buildResumeHTML()`. Hides the cycle button.
|
||||
- **Cycle button (DOM):** centered `<#cycle-button>` cycles `(scene + 1) % 5`.
|
||||
Hidden on Projects (2) and Resume (4) — set via `style.display = 'none'` in
|
||||
`setScene()`.
|
||||
|
||||
### Scene → section mapping
|
||||
|
||||
| Scene | Nav label | Content source | Cycle btn |
|
||||
|-------|-----------|---------------------------|-----------|
|
||||
| 0 | HERO | `SCENES[0]` → #scene-content | visible |
|
||||
| 1 | ABOUT | `SCENES[1]` → #scene-content | visible |
|
||||
| 2 | PROJECTS | `PROJECTS[]` → #projects-section | hidden |
|
||||
| 3 | CONTACT | `SCENES[3]` → #scene-content | visible |
|
||||
| 4 | RESUME | `RESUME` → #resume-section | hidden |
|
||||
|
||||
### Section transitions
|
||||
|
||||
Special sections (`#projects-section`, `#resume-section`) use **opacity crossfades**
|
||||
instead of `display: none/block`:
|
||||
- Default state: `opacity: 0; pointer-events: none; transition: opacity 0.35s`
|
||||
- Visible state (`.visible` class): `opacity: 1; pointer-events: auto; animation: fadeIn 0.45s`
|
||||
- The fade-in animation lives on the `.visible` class so it **re-triggers every visit**,
|
||||
not just the first render.
|
||||
- `#scene-content` uses the same pattern with a `.hidden` class (`opacity: 0`).
|
||||
|
||||
## Roadmap (in suggested order)
|
||||
|
||||
1. **~~SDF text~~** ✅ — Slug algorithm ported; renders crisp in-canvas text labels
|
||||
(HERO / ABOUT / PROJECTS / CONTACT) that change on button click. **Now dormant**
|
||||
in favor of the DOM overlay; kept for reference.
|
||||
2. **~~Nav + multiple buttons~~** — extend `layout.ts` with a list of element bounds;
|
||||
wire hover/click per element; animate section transitions in the shader.
|
||||
✅ Nav bar with 4 clickable labels, active/hover underlines, per-scene text content.
|
||||
**Now DOM-owned** (no `layout.ts`); underlines are CSS.
|
||||
3. **Scroll input** — wheel/touch drag → uniform `scroll`; drive the single-scrolling
|
||||
scene in the shader.
|
||||
4. **Interactive 3D space** — a second pipeline (raymarched or rasterized 3D)
|
||||
composited behind the UI; WASD/click navigation; project nodes placed in space.
|
||||
5. **Content authoring** — projects/about/contact data keyed to sections.
|
||||
|
||||
## Do / don't
|
||||
|
||||
- **Do** run `npm run typecheck` after every change.
|
||||
- **Do** add UI to the DOM overlay in `index.html` and wire it in `overlay.ts`.
|
||||
- **Do** keep `y up / bottom-left` in mind when wiring input.
|
||||
- **Do** prefix intentionally-unused params with `_` (strict unused checks are on).
|
||||
- **Do** add an `sd*` helper in the shader rather than inlining math.
|
||||
- **Don't** add UI shapes inside the fragment shader. Prefer a DOM element + CSS.
|
||||
- **Don't** inline shader strings in `.ts`; use `*.wgsl` + `?raw`.
|
||||
- **Don't** commit changes unless explicitly asked. (opencode default.)
|
||||
- **Don't** add code comments unless requested.
|
||||
|
||||
## WebGPU gotchas
|
||||
|
||||
- Always configure the canvas context after device acquisition and on resize; the
|
||||
texture size comes from `canvas.width/height` (device pixels), not CSS pixels.
|
||||
- Cap DPR at 2 for perf on high-density displays (see `DPR_CAP` in `app.ts`).
|
||||
- Guard `device.lost` and set `configured = false` so a re-init path is possible.
|
||||
- Bind group layouts come from `'auto'` pipeline layout; when adding a second pass,
|
||||
create explicit bind group layouts to share resources (atlas texture, sampler).
|
||||
|
||||
## Text rendering (Slug algorithm) — DORMANT
|
||||
|
||||
Text is rendered entirely inside WebGPU using a port of the **Slug** algorithm
|
||||
(Eric Lengyel's GPU text method, originally C++/commercial). We ported it from
|
||||
[manthrax/JSlug](https://github.com/manthrax/JSlug) (a Three.js/WebGL JS port) to
|
||||
plain TS + WGSL. The key difference from atlas-based SDF text: **no glyph atlas** —
|
||||
each glyph's TrueType quadratic Bézier curves are ray-traced directly in the
|
||||
fragment shader for resolution-independent, perfectly sharp text.
|
||||
|
||||
> **Status:** Dormant. The files under `src/engine/text/` and the `opentype.js`
|
||||
> deps are retained so they still typecheck, but they are **not wired into the
|
||||
> render loop**. All visible text is now handled by the DOM overlay. The notes
|
||||
> below are kept for reference in case the Slug engine is reactivated later.
|
||||
|
||||
### Pipeline: two passes, one command encoder
|
||||
|
||||
1. **Pass 1 — procedural** (`fullscreen.wgsl`): the fullscreen-triangle fragment
|
||||
shader draws the background + SDF UI into the canvas texture (`storeOp: 'store'`).
|
||||
2. **Pass 2 — text** (`text.wgsl`): loads the canvas texture (`loadOp: 'load'`),
|
||||
draws instanced glyph quads with alpha-blended Slug coverage over the
|
||||
procedural background. Skipped entirely if no text or glyph count is 0.
|
||||
|
||||
The text pass uses an explicit bind group layout (not `'auto'`) because it needs
|
||||
two textures (curvesTexture, bandsTexture) in addition to its uniform buffer.
|
||||
|
||||
### CPU side: three files, three responsibilities
|
||||
|
||||
```
|
||||
slug-generator.ts opentype.js Font → SlugData (curve/band arrays + metrics)
|
||||
text-layout.ts text string + SlugData → TextLayoutResult (instance buffer data)
|
||||
text-renderer.ts creates GPU textures + pipeline + buffers, renders the text pass
|
||||
```
|
||||
|
||||
1. **`SlugGenerator.generate(font)`** — iterates every glyph, extracts quadratic
|
||||
Bézier curves from the path (converting `L`/`Z` commands to degenerate
|
||||
quadratics with midpoint control points), packs curves into a `Float32Array`
|
||||
(RGBA32F texture data), spatially bins curves into horizontal and vertical
|
||||
bands stored in a `Uint32Array` (RG32UI texture data), and collects per-codepoint
|
||||
metadata (width, height, advanceWidth, bearings, band dimensions, texture coords).
|
||||
Default: ASCII 32–126 only; pass `fullRange: true` or `whitelist` for more.
|
||||
|
||||
2. **`layoutText(text, slugData, options)`** — iterates codepoints, measures line
|
||||
width, applies justification, and emits per-glyph instance buffer data:
|
||||
- `scaleBias` (vec4f): half-extent + center of the glyph quad in pixel space
|
||||
- `glyphBandScale` (vec4f): glyph width/height in font units + band scale factors
|
||||
- `bandMaxTexCoords` (vec4u): band count - 1 + band texture base coordinates
|
||||
|
||||
3. **`TextRenderer`** — on `init(device, format, slugData)`: creates the curves
|
||||
texture (`rgba32float`), bands texture (`rg32uint`), the render pipeline (6-vert
|
||||
quad, instanced, alpha-blended), and 4 vertex buffers (quad + 3 instance).
|
||||
`updateText(layout)` uploads instance data; `renderPass(...)` adds the text
|
||||
pass to an existing command encoder.
|
||||
|
||||
### WGSL shader notes (`text.wgsl`)
|
||||
|
||||
- Uses `textureLoad` (not `texelFetch`), `@interpolate(flat)` for per-instance data.
|
||||
- WGSL has no fragment derivatives (`dFdx`/`dFdy`), so `pixelsPerEm` is passed as a
|
||||
flat-interpolated `vec2f` computed in the vertex shader from the quad's pixel
|
||||
half-extent (`2.0 * aScaleBias.xy`).
|
||||
- Text coordinates are in **pixel space** (origin bottom-left, y up), same as the
|
||||
procedural pass. The vertex shader maps pixel coords → clip coords via
|
||||
`px / resolution * 2.0 - 1.0`.
|
||||
- The `trace_ray_curve_h` function is the heart of Slug: solves the quadratic for
|
||||
the ray-curve intersection, then computes coverage from the intersection's x
|
||||
coordinate scaled by `pixelsPerEm` for anti-aliasing.
|
||||
|
||||
### Adding text labels (Slug — dormant)
|
||||
|
||||
The Slug engine is currently **not wired in**. To reactivate GPU text, you would
|
||||
also need to restore the text pass in `renderer.ts` (the `initText` / `updateText`
|
||||
methods and the second render pass were removed when text moved to the DOM overlay).
|
||||
|
||||
```ts
|
||||
// 1. Generate slug data (once, on font load)
|
||||
const gen = new SlugGenerator()
|
||||
const slugData = await gen.generateFromUrl('/SomeFont.ttf')
|
||||
|
||||
// 2. Init the text renderer (once, after device init)
|
||||
renderer.initText(slugData)
|
||||
|
||||
// 3. Layout text (re-call when text or position changes)
|
||||
const layout = layoutText('Hello', slugData, {
|
||||
fontScale: 0.3 * dpr, // ~0.3 of font units per pixel
|
||||
startX: resX * 0.5, // pixel-space, origin bottom-left
|
||||
startY: resY * 0.5,
|
||||
justify: 'center'
|
||||
})
|
||||
renderer.updateText(layout)
|
||||
// 4. The render loop picks it up automatically each frame.
|
||||
```
|
||||
|
||||
Font files go in `public/` and are fetched at runtime. Uses `opentype.js` (runtime
|
||||
dep) + `@types/opentype.js` (dev dep).
|
||||
Reference in New Issue
Block a user