Skip to content

feat: leave example queries and test out with skipExamples (LA-2388) - #734

Closed
atilafassina wants to merge 4 commits into
mainfrom
work/16-drop-placeholder-queries-and-t
Closed

atilafassina wants to merge 4 commits into
mainfrom
work/16-drop-placeholder-queries-and-t

Conversation

@atilafassina

@atilafassina atilafassina commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

What

Lets the Builder scaffold without the starter files it deletes on nearly every run (LA-2388):

  • config/queries/hello_world.sql
  • config/queries/mocked_sales.sql
  • server/example.test.ts

In the NFL-Scout evals (51 trajectories), both queries were deleted in virtually every run. example.test.ts was deleted in 22 runs and caused lint errors in 8.

Approach: an opt-in template variable, default unchanged

The files are gated on a new .skipExamples template variable, which databricks apps init --skip-examples sets (CLI change below). I picked an opt-in flag over removing the files for everyone:

  • People running apps init --features analytics by hand still get a working Analytics page with charts. That demo needs both queries.
  • The template checks not .skipExamples. A CLI without the flag leaves the key out, which Go reads as false, so older CLIs and every other user keep exactly what they get today.
  • Removing the files for everyone would have meant removing the Analytics demo too, and that's a separate product call.

With skipExamples:

  • Both queries and the example test render empty. The CLI already skips files that render empty, and it removes config/queries/ once that folder is empty.
  • AnalyticsPage.tsx becomes a page that reads no queries and says to add one under config/queries/. The app builds and runs with no queries dir. Analytics and typegen both already handle a missing folder.
  • The guidance the starter files gave is now in the template README: a SQL Queries section (file naming, .obo.sql, -- @param, useAnalyticsQuery/charts, typegen) and a Testing section (createTestApp, createTestPluginContext, expectStream). These sections render in both modes.

Other changes

  • The example test now passes the template lint config. It had 3 typescript/require-await errors: async setup(), async *greet and the route handler. I couldn't reproduce no-empty-object-type on HEAD.
  • Smoke test renderer. tools/template-render.ts is a small renderer for Go-template if/else if/else chains on .plugins, .appEnv and .skipExamples, with trim markers. Every other action is left as-is. It replaces the smoke test's regex stripper, which couldn't handle and/not, else or nesting. The smoke test gets a --skip-examples flag.
  • CI: the raw template's own npm test step now renders example.test.ts first, since the file now starts with {{if not .skipExamples -}}. The step moved to after the zip (the artifact has to keep the conditional) and before the upload (a broken example still fails the job).
  • Analytics scaffolding rule: the should rule used to tell agents to make sure config/queries/ has a .sql file before typegen. It now says to add queries only for what the app runs, because typegen and build work without the folder (packages/appkit/src/plugins/analytics/manifest.json, synced to template/appkit.plugins.json).
  • Docs: .skipExamples added to the template-variables table in docs/development/templates.md.

Outside this repo

Databricks CLI (cmd/apps/init.go)

Add a --skip-examples bool flag and pass it to the template as skipExamples. This is the whole change. I built it and tested it against this template:

diff --git a/cmd/apps/init.go b/cmd/apps/init.go
index 7ba36f799..d6b672eed 100644
--- a/cmd/apps/init.go
+++ b/cmd/apps/init.go
@@ -97,6 +97,7 @@ func newInitCmd() *cobra.Command {
 		setValues      []string
 		autoApprove    bool
 		skipInstall    bool
+		skipExamples   bool
 		packageManager string
 		authMode       string
 	)
@@ -191,6 +192,7 @@ Environment variables:
 				setValues:             setValues,
 				autoApprove:           autoApprove,
 				skipInstall:           skipInstall,
+				skipExamples:          skipExamples,
 				packageManager:        packageManager,
 				packageManagerChanged: cmd.Flags().Changed("package-manager"),
 				authMode:              authMode,
@@ -215,6 +217,7 @@ Environment variables:
 	cmd.Flags().BoolVar(&autoApprove, "auto-approve", false, "Skip confirmation prompts for optional resources. Optional resources are only configured when their values are provided via --set.")
 	cmd.Flags().StringVar(&authMode, "auth-mode", "", "Default resource auth mode: obo (on behalf of the user) or sp (service principal). With obo, resources that cannot be accessed on behalf of the user use sp. Prompts if not provided in interactive mode, otherwise sp.")
 	cmd.Flags().BoolVar(&skipInstall, "skip-install", false, "Skip installing project dependencies (e.g. npm install / uv sync). Cannot be combined with --run.")
+	cmd.Flags().BoolVar(&skipExamples, "skip-examples", false, "Leave out the template's example files (sample queries and tests) that a real app replaces")
 	cmd.Flags().StringVar(&packageManager, "package-manager", "", "Node.js package manager (pnpm, npm; default: inferred from template)")
 
 	return cmd
@@ -243,6 +246,7 @@ type createOptions struct {
 	setValues             []string // --set plugin.resourceKey.field=value pairs
 	autoApprove           bool
 	skipInstall           bool
+	skipExamples          bool
 	packageManager        string
 	packageManagerChanged bool
 	authMode              string // --auth-mode default for all resources
@@ -527,6 +531,9 @@ type templateVars struct {
 	DotEnv         dotEnvVars
 	AppEnv         string
 	PackageManager string
+	// SkipExamples is set by --skip-examples. The template leaves its example
+	// files out when it is true; a missing key (older CLIs) keeps them.
+	SkipExamples bool
 	// Plugins maps plugin name to its metadata
 	// Missing keys return nil, enabling {{if .plugins.analytics}} conditionals.
 	Plugins map[string]*pluginVar
@@ -1807,6 +1814,7 @@ func runCreate(ctx context.Context, opts createOptions) (runErr error) {
 		},
 		AppEnv:         generator.GenerateAppEnv(selectedPluginList, genConfig),
 		PackageManager: selectedManager.Name,
+		SkipExamples:   opts.skipExamples,
 		Plugins:        plugins,
 	}
 
@@ -2271,6 +2279,7 @@ func templateData(vars templateVars) map[string]any {
 		"appDescription": vars.AppDescription,
 		"workspaceHost":  vars.WorkspaceHost,
 		"packageManager": vars.PackageManager,
+		"skipExamples":   vars.SkipExamples,
 		"bundle": map[string]any{
 			"variables":       vars.Bundle.Variables,
 			"resources":       vars.Bundle.Resources,

Suggested CLI test: an executeTemplate case showing that {{if not .skipExamples}}x{{end}} renders x without the flag and nothing with it.

Builder (universe)

  • Once a CLI with the flag ships, add --skip-examples to the Builder's databricks apps init call. On a CLI without the flag, init fails with "unknown flag", so put this behind the CLI version floor.
  • Later, maybe: add --skip-examples to scaffolding.flags in the template manifest so scaffolding agents find it. I left it out on purpose, because agents on older CLIs would pass a flag those CLIs reject. That can wait until cli-compat.json pins this template to CLIs that have the flag.

Testing

  • tools/template-render.test.ts has 11 tests: renderer semantics (trim markers, not/and/or, else-if chains, unsupported actions kept as-is, unbalanced blocks), plus template-level checks. Every template file is rendered with all plugins, with and without skipExamples. The checks: the gated files are present or absent as expected, no rendered file mentions hello_world/mocked_sales in builder mode, and the gated files have no leftover template syntax.
  • The smoke test passes in default mode and with --skip-examples: install, build, lint fixtures, HTTP start. With no plugins, the renderer's output matches the old regex stripper except for trim-marker whitespace, and the analytics-gated .sql files are now dropped, as the CLI already did.
  • End to end with the patched CLI: I ran databricks apps init --template ./template --features analytics,server with and without --skip-examples:
    • Builder mode has no config/queries/ and no server/example.test.ts.
    • oxlint, tsc -b, build:server and build:client all pass.
    • The production server returns 200 for /, /analytics and /health against a fake workspace. An unknown query gets a clean 404.
    • For the gated files and App.tsx, tools/template-render.ts produces byte-identical output to the real CLI in both modes.
  • Default mode: the CLI-rendered example test passes (3/3) and lints clean.
  • pnpm check, the tools test project, pnpm build and pnpm docs:build all pass.

Also noticed (not changed here)

  • In default mode, AnalyticsPage.tsx only typechecks and lints once typegen has run against a warehouse. Without generated types, data[0].value is unknown. This was already the case before this PR. Builder mode avoids it.

This pull request and its description were written by Isaac.

atilafassina and others added 2 commits November 12, 2026 21:40
The smoke test stripped `{{if .plugins.X}}` blocks with a regex, which
cannot follow `and`/`not`, `else` branches or nesting, ignored trim
markers and skipped `.sql` files. Add a small renderer for `if` chains
on `.plugins`, `.appEnv` and `.skipExamples` that leaves every other
action verbatim, and use it in the smoke test, with a `--skip-examples`
flag to scaffold the way the Builder will.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Atila Fassina <atila@fassina.eu>
The Builder deletes `config/queries/hello_world.sql`,
`config/queries/mocked_sales.sql` and `server/example.test.ts` in nearly
every run, and the test failed the template's own lint config first
(LA-2388).

Gate them on a new `.skipExamples` template variable, set by
`databricks apps init --skip-examples` (CLI change pending). The check is
`not .skipExamples`, so CLIs without the flag and every other user keep
the files. With the flag, the Analytics page renders a query-free
placeholder instead of reading the example queries.

- Fix the example test's `require-await` errors so it passes the
  template lint config
- Document queries and the testing kit in the template README
- Render the example test before CI runs the raw template's tests
- Add template tests for both modes

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Atila Fassina <atila@fassina.eu>
@atilafassina
atilafassina requested a review from a team as a code owner November 12, 2026 21:44
@atilafassina
atilafassina requested review from MarioCadenas and a balanced review from Copilot November 12, 2026 21:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 43318825916 -R databricks/appkit -n appkit-template-0.94.1-pr.45f1c6d-work-16-drop-placeholder-queries-and-t-734 -D appkit-pr-734 \
  && unzip -o "appkit-pr-734/appkit-template-0.94.1-pr.45f1c6d-work-16-drop-placeholder-queries-and-t-734.zip" -d "appkit-pr-734" \
  && databricks apps init --template "appkit-pr-734"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

…pegen

The analytics `should` rule asked agents to make sure config/queries/ has
a .sql file before typegen. With `apps init --skip-examples` the folder
is gone on purpose, and typegen, build and the server all work without
it, so the rule would send the Builder back to create a placeholder.
Say to add queries only for what the app runs.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Atila Fassina <atila@fassina.eu>
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 1.3 MB (-49 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 1.3 MB (-168 B) 479 KB (-1 B)
Type declarations 470 KB 169 KB
Source maps 2.6 MB (-47 B) 889 KB (-29 B)
Other 11 KB 3.7 KB
Total 4.4 MB (-215 B) 1.5 MB (-30 B)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 108 KB (+15 B) 2.5 KB (+1 B) 110 KB (+16 B) external 348 KB (+14 B)
./beta 113 KB 486 B 114 KB external 339 KB
./testing 46 KB 33 KB (-3 B) 79 KB (-3 B) external 229 KB
./tsdown 520 B 0 B 520 B external 813 B
./type-generator 26 KB 0 B 26 KB external 74 KB

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 103 KB
. utils.js initial 4.6 KB
. remote-tunnel-manager.js lazy 2.5 KB
./beta beta.js initial 93 KB
./beta modular.js initial 6.0 KB
./beta stream-manager.js initial 5.9 KB
./beta service-context.js initial 4.3 KB
./beta databricks.js initial 3.3 KB
./beta client.js initial 648 B
./beta index.js initial 20 B
./beta supervisor-api.js lazy 192 B
./beta databricks.js lazy 179 B
./beta index.js lazy 115 B
./testing manifest.js initial 33 KB
./testing index.js initial 10 KB
./testing wide-event-emitter.js initial 2.9 KB
./testing index.js lazy 29 KB
./testing remote-tunnel-manager.js lazy 2.5 KB
./testing utils.js lazy 1.8 KB
./tsdown index.js initial 520 B
./type-generator index.js initial 26 KB

@databricks/appkit-ui

npm tarball (packed): 406 KB — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 436 KB 150 KB
Type declarations 271 KB 100 KB
Source maps 862 KB 291 KB
CSS 16 KB 3.2 KB
Total 1.5 MB 544 KB
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 2.1 KB 0 B 2.1 KB 0 B 5.0 KB
./react 440 KB 49 KB 489 KB 1.4 MB 178 KB
./react/beta 14 KB 0 B 14 KB 25 KB 16 KB

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 2.1 KB
./react index.js initial 438 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 14 KB

No released Databricks CLI has `apps init --skip-examples` yet. Describe
`.skipExamples` by what it does, note that the CLI flag is planned and
that the variable is unset until then, and drop the flag from the comment
in the scaffolded example test and from internal comments.

Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Atila Fassina <atila@fassina.eu>

@calvarjorge calvarjorge left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. A missing skipExamples is falsy in the CLI (missingkey=zero), so older CLIs keep the examples. template-render.ts output matches Go's text/template byte for byte on every template file, in default, skip and no-plugins mode.

} as PluginManifest<'greeter'>;

async setup() {
setup() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#733 makes the same require-await fix here. Keep it in one of the two PRs, or whichever lands second conflicts on this file.

export function AnalyticsPage() {
return (
<div className="w-full max-w-7xl mx-auto">
<Card className="shadow-lg">

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#653 removes shadow-lg from the other Analytics cards. Whichever of the two lands second should drop it here too, so the placeholder matches.

@atilafassina

Copy link
Copy Markdown
Contributor Author

Review 6 · Pass

Does the change do what the task asked? Yes. Gate: major and worse must be fixed.

The AppKit template change meets the requested opt-in behavior: Builder mode omits the placeholder queries and test, while the default mode keeps the demo. The rendered empty Analytics page, template tests, lint, typecheck, and a Builder-style client build passed; the PR documents the required CLI and Builder follow-up.

No findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants