Sample tasks

All task families across twelve open-source projects. Each task ships with a sandboxed environment, an expert-authored validation spec, and a reference solution. Tasteful Solve is the share of frontier-agent attempts that pass both the functional verifier and the taste review.
better-auth-fix-api-key-run
Investigate
performance
Solve rate: 0%
 By better-auth
better-auth-fix-api-return-response
Investigate
bug
Solve rate: 0%
 By better-auth
better-auth-fix-oauth-provider-return
Investigate
bug
Solve rate: 0%
 By better-auth
better-auth-fix-resolve-dynamic-baseurl
Investigate
bug
Solve rate: 0%
 By better-auth
electric-feat-add-variadic-function
Design
feature
Solve rate: 0%
 By electric
electric-feat-sync-service-start
Design
feature
Solve rate: 33%
 By electric
electric-fix-classify-admission-control
Investigate
bug
Solve rate: 11%
 By electric
electric-fix-elixir-client-cache
Investigate
bug
Solve rate: 22%
 By electric
electric-fix-resolve-pending-shapes
Investigate
bug
Solve rate: 0%
 By electric
electric-perf-array-filter-eval
Investigate
performance
Solve rate: 0%
 By electric
firezone-feat-portal-add-recent
Design
feature
Solve rate: 0%
 By firezone
firezone-fix-connlib-align-device
Investigate
bug
Solve rate: 0%
 By firezone
gitea-add-project-column-picker
Design
feature
Solve rate: 0%
 By gitea
gitea-feat-fast-forward-only
Design
feature
Solve rate: 0%
 By gitea
gitea-fix-codeql-code-scanning
Investigate
bug
Solve rate: 0%
 By gitea
gitea-fix-diff-highlight-overlap
Investigate
bug
Solve rate: 0%
 By gitea
gitea-fix-force-push-timeline
Investigate
bug
Solve rate: 0%
 By gitea
gitea-fix-incorrect-viewed-files
Investigate
bug
Solve rate: 11%
 By gitea
gitea-refactor-auth-middleware
Design
migration
Solve rate: 78%
 By gitea
harbor-add-agent-file-retention
Design
feature
Solve rate: 11%
 By harbor
harbor-add-multi-step-tasks
Design
feature
Solve rate: 0%
 By harbor
harbor-add-windows-tasks-support
Design
feature
Solve rate: 11%
 By harbor
harbor-refactor-optional-sandbox-deps
Design
feature
Solve rate: 0%
 By harbor
immich-feat-recently-added-assets
Design
feature
Solve rate: 13%
 By immich
immich-feat-release-candidate-detection
Design
feature
Solve rate: 0%
 By immich
immich-fix-server-live-photo
Investigate
bug
Solve rate: 0%
 By immich
paperless-ngx-feat-saved-view-sharing
Design
feature
Solve rate: 29%
 By paperless-ngx
paperless-ngx-perf-document-counts
Investigate
performance
Solve rate: 11%
 By paperless-ngx
paperless-ngx-perf-workflow-queries
Investigate
performance
Solve rate: 89%
 By paperless-ngx
paperless-ngx-refactor-task-system
Design
feature
Solve rate: 0%
 By paperless-ngx
plausible-feat-shared-dashboard-deeplink
Design
feature
Solve rate: 0%
 By plausible
plausible-fix-cross-site-resource-attach
Investigate
bug
Solve rate: 0%
 By plausible
plausible-fix-top-pages-comparison
Investigate
bug
Solve rate: 33%
 By plausible
posthog-feat-approval-gating
Design
feature
Solve rate: 22%
 By posthog
posthog-feat-llma-enable-tagger
Design
feature
Solve rate: 67%
 By posthog
posthog-feat-personhog-writer-add
Design
feature
Solve rate: 0%
 By posthog
posthog-feat-prompt-versioning
Design
feature
Solve rate: 56%
 By posthog
posthog-feat-schema-ingestion-block
Design
feature
Solve rate: 0%
 By posthog
posthog-fix-llm-gateway-add
Investigate
bug
Solve rate: 0%
 By posthog
posthog-fix-replay-buffering
Investigate
bug
Solve rate: 0%
 By posthog
prefect-add-isolated-workspace-resolver
Design
feature
Solve rate: 0%
 By prefect
prefect-feat-dbt-per-node-hardening
Design
feature
Solve rate: 0%
 By prefect
prefect-fix-resolve-race-condition
Investigate
bug
Solve rate: 22%
 By prefect
prefect-fix-subflow-cancellation
Design
feature
Solve rate: 0%
 By prefect
teleport-add-traits-matching-logic
Design
feature
Solve rate: 44%
 By teleport
teleport-fix-scoped-kube-clusters
Investigate
bug
Solve rate: 0%
 By teleport
turborepo-feat-add-circular-package
Design
feature
Solve rate: 11%
 By turborepo
turborepo-fix-preserve-package-json
Investigate
bug
Solve rate: 0%
 By turborepo
turborepo-fix-prune-missing-sources
Investigate
bug
Solve rate: 0%
 By turborepo
turborepo-perf-reuse-input-hashes
Investigate
performance
Solve rate: 0%
 By turborepo
Show all problems