GPT-5.6 (Sol / Terra / Luna) is now evaluated on TrustVector — with day-1 independent verification, incl. METR's benchmark-cheating findings.

Read the evaluation
Evaluation record · google-jules

Google Jules

vGA (2025-08-06)

Google

Agentcoding-agentautonomousasynchronousgoogle
71
Adequate
About This Agent

Asynchronous autonomous coding agent from Google. Jules clones a repository into an isolated Google Cloud VM, plans and writes code in the background, runs tests, and opens pull requests for human review. Powered by Gemini 2.5 Pro on the free tier and Gemini 3 Pro on paid Google AI Pro/Ultra tiers.

Last Evaluated: July 9, 2026
Official Website

Trust Vector Analysis

Dimension Breakdown

🚀Performance & Reliability
+
task completion accuracy

Assessment of reported task outcomes and hands-on PR quality review

Evidence
Google Blog - Jules now available — GA after public beta with hundreds of thousands of tasks completed; handles bug fixes, version bumps, tests, and feature work
mediumVerified: 2026-07-09
tool use reliability

VM tooling and environment setup reliability assessment

Evidence
Jules Documentation — Full VM environment with shell access, dependency installation, environment setup scripts, and test execution
mediumVerified: 2026-07-09
multi step planning

Plan generation and decomposition quality review

Evidence
Jules Product Page — Presents an explicit plan with reasoning before executing; users can review and steer the plan before and during execution
highVerified: 2026-07-09
memory persistence

Cross-task context persistence evaluation

Evidence
Jules Documentation — Per-repo environment configuration and snapshots persist across tasks; memory feature retains repo-specific preferences and corrections
mediumVerified: 2026-07-09
error recovery

Failure handling and iteration behavior testing

Evidence
Jules Changelog — Runs tests in the VM and iterates on failures; users can intervene mid-task via chat to correct course
mediumVerified: 2026-07-09
agent collaboration

Concurrency and orchestration capability assessment

Evidence
Jules Product Page — Single autonomous agent per task; supports concurrent independent tasks (3-60 depending on plan) but no multi-agent orchestration
mediumVerified: 2026-07-09
🛡️Security
+
tool sandboxing

Execution isolation architecture review

Evidence
Google Blog - Jules now available — Each task runs in an isolated, ephemeral Google Cloud VM, separating agent execution from user machines and other tasks
highVerified: 2026-07-09
access control

Repository permission model assessment

Evidence
Jules Documentation — GitHub App installation with per-repository access selection; changes land on branches as pull requests
mediumVerified: 2026-07-09
prompt injection defense

Injection surface review focusing on autonomy boundaries and human gates

Evidence
Jules Product Page — Human PR review gate plus plan approval limit blast radius; repository content itself remains an injection vector
mediumVerified: 2026-07-09
data isolation

Tenant and task isolation review

Evidence
Jules Documentation — Per-task VM isolation keeps repository data separated between tasks and tenants
mediumVerified: 2026-07-09
open source transparency

Source availability review

Evidence
Jules Product Page — Proprietary closed-source service; only client tooling (Jules Tools CLI) and docs are public
highVerified: 2026-07-09
🔒Privacy & Compliance
+
data retention

Privacy terms and data handling review

Evidence
Jules Documentation - Privacy — Google states private repository code is not used to train models; data handling governed by Google's privacy terms
mediumVerified: 2026-07-09
gdpr compliance

Compliance posture assessment

Evidence
Google Privacy Policy — Covered by Google's general privacy framework; lacks dedicated enterprise compliance attestations of Google Cloud products
mediumVerified: 2026-07-09
third party data sharing

Data flow analysis

Evidence
Jules Documentation — Full repository contents are cloned to Google-managed cloud VMs and processed by Gemini models
mediumVerified: 2026-07-09
local deployment option

Deployment options assessment

Evidence
Jules Product Page — Cloud-only service; no self-hosted or on-premises deployment option
highVerified: 2026-07-09
👁️Trust & Transparency
+
documentation quality

Documentation completeness review

Evidence
Jules Documentation — Clear docs covering setup, environment configuration, API, Jules Tools CLI, and changelog
highVerified: 2026-07-09
execution traceability

Execution visibility assessment

Evidence
Jules Product Page — Shows its plan, reasoning, file diffs, and activity feed during execution; all changes arrive as reviewable PRs
highVerified: 2026-07-09
decision explainability

Explainability features assessment

Evidence
Google Blog - Jules now available — Upfront plan with per-step reasoning that users approve or modify before code changes are made
mediumVerified: 2026-07-09
open source code

Open source assessment

Evidence
Jules Product Page — Proprietary service built on closed Gemini models; core agent is not open source
highVerified: 2026-07-09
community activity

Release cadence and community engagement analysis

Evidence
Jules Changelog — Frequent feature updates since GA (API, CLI, memory, GitHub issues integration) and active user community
mediumVerified: 2026-07-09
⚙️Operational Excellence
+
ease of integration

Onboarding and integration assessment

Evidence
Jules Product Page — Connect GitHub, pick a repo, write a prompt; also triggerable from GitHub issues by assigning a label, plus API and CLI
highVerified: 2026-07-09
scalability

Concurrency and quota analysis

Evidence
Jules Pricing — Concurrent task limits by plan: 3 concurrent on free, 15 on Pro, 60 on Ultra; cloud VMs scale per task
mediumVerified: 2026-07-09
cost predictability

Pricing model analysis

Evidence
Jules usage limits documentation — Flat subscription tiers confirmed: Free (15 tasks/day, 3 concurrent, Gemini 2.5 Pro), Google AI Pro $19.99/mo (100 tasks/day, 15 concurrent, Gemini 3 Pro), AI Ultra (300 tasks/day, 60 concurrent); Ultra entry price cut to $99.99/mo (5x Pro) at Google I/O 2026, with a $200/mo top tier (20x)
highVerified: 2026-07-09
monitoring capabilities

Monitoring features assessment

Evidence
Jules Documentation - API — Activity feed, task history, and API for programmatic task tracking; lacks enterprise audit log integrations
mediumVerified: 2026-07-09
production readiness

Product maturity assessment

Evidence
Google Blog - Jules now available — Exited Google Labs beta to general availability on 2025-08-06 with paid plans and continued feature investment
highVerified: 2026-07-09
Strengths
  • +Strong isolation: every task runs in an ephemeral Google Cloud VM
  • +Transparent plans with reasoning that users approve before execution
  • +Human PR review gate on all changes limits autonomous blast radius
  • +Predictable flat-rate pricing with a usable free tier (15 tasks/day)
  • +Asynchronous and parallel: fire-and-forget tasks with mid-task steering
  • +API, CLI (Jules Tools), and GitHub issue-label triggers for automation
Limitations
  • !Proprietary, cloud-only service with no self-hosted option
  • !Full repository contents are uploaded to Google-managed VMs
  • !Daily task limits even on paid tiers (~100/day Pro, ~300/day Ultra)
  • !Locked to Gemini models with no model choice
  • !GitHub-focused; weaker support for other source forges
  • !Lacks enterprise compliance attestations and audit tooling of Google Cloud products
  • !Paid Jules tiers are currently restricted to individual Google Accounts (@gmail.com); no workspace/organizational upgrade path yet
Metadata
license: Proprietary
supported models
0: Gemini 2.5 Pro (free tier)
1: Gemini 3 Pro (Google AI Pro/Ultra tiers)
programming languages
0: Most major languages (Python, JavaScript/TypeScript, Java, Go, Rust, and more)
deployment type: Cloud-only SaaS (isolated Google Cloud VMs per task)
tool support
0: Cloud VM shell
1: Environment setup scripts
2: Test execution
3: GitHub PR creation
4: Jules API and Jules Tools CLI
first release: Public beta May 2025 (Google I/O); GA 2025-08-06
pricing: Free: 15 tasks/day, 3 concurrent; Google AI Pro $19.99/mo (100 tasks/day, 15 concurrent); Google AI Ultra $99.99/mo entry or $200/mo top tier (300 tasks/day, 60 concurrent); paid tiers limited to individual @gmail accounts

Use Case Ratings

code generation

Purpose-built for asynchronous coding tasks: bug fixes, tests, dependency bumps, and features delivered as PRs

data analysis

Can write and run analysis scripts in its VM, but the workflow is optimized for repository changes, not analytics

education

Visible plans and reasoning make it a useful learning aid; free tier suits students

research assistant

Limited to codebase-scoped investigation; not designed for general research