CAN ASTRA DO X?
Questions become experiments when the method is frozen.
A database of tests to run, not a gallery of guesses.
Each experiment has a question, exact prompt slot, environment, success criteria, stop conditions, comparison plan, and last-tested field. The answer remains blocked until the run exists.
EXPERIMENT STATUS
Eight definitions ready. Zero Astra results invented.
Definitions are useful preparation; they are not evidence that Astra can perform the task.
EXPERIMENT DATABASE
The first questions worth answering.
Expand a record to see what must be frozen before a result can be published. User submissions can use the same shape after moderation is added.
Needs update Can Astra build a small app from a written brief? build-an-app · definition ready · not tested
- Exact prompt
- To be filled with the frozen benchmark prompt before the first run.
- Success criteria
-
- Build runs
- Core user journey works
- Tests and limitations are documented
- Stop conditions
-
- No progress after the time budget
- Unsafe external action requested
- Required access unavailable
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra analyze a 10-K with traceable evidence? analyze-10k · definition ready · not tested
- Exact prompt
- To be frozen with the document fixture, extraction rules, and citation format.
- Success criteria
-
- Material claims cite page evidence
- Arithmetic reproduces
- Unknowns are disclosed
- Stop conditions
-
- Document cannot be accessed
- Evidence cannot be traced
- Output exceeds review budget
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra debug a repository without scope drift? debug-repository · definition ready · not tested
- Exact prompt
- To be frozen with the repository commit, issue, and allowed tool policy.
- Success criteria
-
- Root cause is demonstrated
- Patch is minimal
- Regression test passes
- Stop conditions
-
- Patch would alter unrelated behavior
- Credentials or network access are required
- Evidence is inconclusive
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra solve a graduate-level mathematics task and verify it? graduate-math · definition ready · not tested
- Exact prompt
- To be frozen with the problem source, allowed references, and independent rubric.
- Success criteria
-
- Solution is correct
- Assumptions are explicit
- Independent checker agrees
- Stop conditions
-
- No proof or counterexample is established
- Verification is unavailable
- Time budget is exhausted
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra review scientific literature without inventing evidence? literature-review · definition ready · not tested
- Exact prompt
- To be frozen with a bounded paper set and a citation verification protocol.
- Success criteria
-
- Claims map to papers
- Contradictions are visible
- Missing evidence is labeled
- Stop conditions
-
- Source access fails
- Citation identity cannot be verified
- Scope expands beyond the paper set
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra analyze a large dataset with reproducible outputs? large-dataset · definition ready · not tested
- Exact prompt
- To be frozen with a synthetic dataset, task, and artifact validation rules.
- Success criteria
-
- Output reproduces
- Memory/time behavior is recorded
- Data handling is safe
- Stop conditions
-
- Data cannot be processed safely
- Artifact cannot be reproduced
- Tool budget is exhausted
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra use a browser reliably while respecting boundaries? browser-reliability · definition ready · not tested
- Exact prompt
- To be frozen with a sandboxed site, allowed actions, and no-account test data.
- Success criteria
-
- Actions stay within the allowlist
- State is verified after each action
- No sensitive data is requested
- Stop conditions
-
- Site state is ambiguous
- Action would be irreversible
- Browser access is not officially supported
- Last tested
- Not tested — public access and execution protocol are not available.
Needs update Can Astra generate or repair a Lean proof with a clean build? lean-proof · definition ready · not tested
- Exact prompt
- To be frozen with a pinned Lean environment and a proof task not present in training artifacts.
- Success criteria
-
- Lean build passes
- No hidden axioms are added
- Proof is reviewed
- Stop conditions
-
- Environment cannot be reproduced
- Proof relies on unapproved axioms
- Time budget is exhausted
- Last tested
- Not tested — public access and execution protocol are not available.
COMMUNITY LAYER
Submission comes after the method.
A future gallery can accept prompts, demos, repositories, screenshots, results, cost, runtime, and comparison models—but only after review, spam controls, and a clear publication status exist.
Submit
Creator supplies the task, environment, exact prompt, artifacts, and provenance.
Review
Human moderation checks safety, attribution, reproducibility, and unsupported claims.
Publish
Approved records receive a shareable permalink and a visible “last tested” date.