What The Diff Review (2026)

What The Diff Overview

What The Diff is an AI-powered application that generates human-readable descriptions of code changes in pull requests. Rather than attempting to review code for bugs, security vulnerabilities, or quality issues, it focuses exclusively on one task: translating code diffs into plain-language summaries that developers, product managers, and other stakeholders can understand. The tool analyzes the git diff of each pull request and produces structured descriptions that explain what changed, why it matters, and what reviewers should pay attention to.

The tool was created to address a common friction point in software development: most pull requests have inadequate descriptions. Developers often write terse or empty PR descriptions because writing a thorough summary of code changes is time-consuming and feels redundant to someone who just wrote the code. What The Diff automates this documentation burden, ensuring every PR has a clear, consistent description regardless of whether the developer takes the time to write one. It also generates changelog entries and weekly progress reports, extending its utility beyond individual PRs into team-wide communication.

What The Diff occupies a unique position in the developer tools market because it is complementary to, rather than competitive with, AI code review tools like CodeRabbit, Diffray, or Ellipsis. Those tools analyze code for bugs and quality issues; What The Diff generates documentation about what the code does. Many teams use both: an AI reviewer to catch issues and What The Diff to handle the communication and documentation side of the pull request workflow. The tool supports both GitHub and GitLab, works with all programming languages, and does not store any user code.

Feature Deep Dive

AI-Generated PR Descriptions. The core feature of What The Diff is automatic generation of structured, human-readable pull request descriptions. When a PR is opened or updated, the tool analyzes the diff and produces a description that covers what files were changed, what the changes accomplish, and what areas reviewers should focus on. These descriptions are designed to be understandable by both technical and non-technical readers, making them valuable for cross-functional teams.

Automated Changelog Generation. What The Diff compiles PR summaries into a beautiful public changelog page that teams can share with users, customers, or stakeholders. The changelog is also available through a JSON API, allowing teams to integrate change data into their own dashboards, release notes, or documentation systems. For teams that maintain public changelogs, this feature alone can justify the cost of the tool by eliminating hours of manual changelog compilation at release time.

Weekly Progress Reports. The tool generates automated weekly summaries that aggregate all changes across the team into a single report. These reports are sent directly to team members, providing a high-level view of development activity without anyone having to manually compile updates. This is particularly valuable for engineering managers, product managers, and executives who need visibility into what the engineering team is delivering.

Inline AI Refactoring Suggestions. Beyond summarization, What The Diff can suggest minor code refactoring improvements directly within the pull request. While this does not replace a full AI code review tool, it adds practical value by catching straightforward improvement opportunities alongside the summary generation. These suggestions focus on readability and minor optimizations rather than deep bug detection.

Multi-Language Summary Output. What The Diff can generate summaries in languages beyond English, making it useful for international development teams where not all stakeholders are fluent in English. The AI model handles translation naturally as part of the summary generation process.

Granular Token Management. The platform provides controls for managing token consumption, including the ability to skip CI-generated pull requests, delay processing of draft PRs, and set token limits per repository. These controls help teams stay within their token allocation and avoid unexpected overages.

Privacy-First Architecture. What The Diff does not store user code. The diff is processed through the AI model and discarded. The service explicitly states it does not train AI models on user code, which is an important consideration for organizations with sensitive codebases.

Notification Integrations. Beyond GitHub and GitLab, What The Diff integrates with Slack, webhooks, and email to distribute PR summaries to stakeholders who do not live in the version control platform. This makes it easy to keep product managers, designers, and QA engineers informed about code changes without requiring them to monitor GitHub or GitLab directly.

Pricing and Plans

What The Diff uses a token-based pricing model, where each pull request consumes a number of tokens proportional to the size of the diff.

Free Plan. Includes 25,000 tokens per month, which is sufficient for approximately 10 average-sized pull requests (the average PR consumes about 2,300 tokens). No credit card is required to start. This is a reasonable evaluation tier but quite limited for any team doing more than a handful of PRs per week.

Pro 200k Plan ($19/month). Provides 200,000 tokens per month, covering approximately 40 average PRs. Includes priority email support, inline refactoring suggestions, Slack and webhook notifications, and custom summary templates. For a small team of 3 to 5 developers submitting 8 to 10 PRs per week, this plan should be sufficient.

Higher Token Tiers. What The Diff offers additional plans with larger token allocations for teams with higher volume. Specific pricing for these tiers is available on their website and scales with token count.

An important consideration is that unused tokens do not roll over to the next month, and the application does not automatically upgrade plans when tokens are exhausted. If a team runs out of tokens mid-month, they must manually upgrade or wait for the next billing cycle. Large PRs with extensive diffs consume significantly more tokens than average, which can make costs somewhat unpredictable for teams with variable PR sizes.

Compared to other tools in the PR description space, What The Diff’s pricing is reasonable. It is significantly cheaper than full AI code review tools like CodeRabbit (24 dollars per user per month) or Sourcery (29 dollars per user per month), but those tools provide actual code review in addition to summaries. The question for teams is whether automated PR descriptions alone provide enough value to justify a separate subscription, or whether a more comprehensive tool would be a better investment.

How What The Diff Works

Installation. Getting started requires installing the What The Diff GitHub App or GitLab integration. The setup process takes under five minutes and requires authorizing access to the repositories you want covered. Despite requiring full repository access (a limitation of the GitHub and GitLab APIs), What The Diff only reads the diff of each pull request and does not access or store the full codebase.

Summary Generation. When a developer opens or updates a pull request, What The Diff receives a webhook notification, fetches the diff through the platform API, and sends it to its AI model for processing. The model generates a structured summary describing the changes in plain language. This summary is posted as a comment on the pull request, typically within one to two minutes of the PR being opened.

Changelog Compilation. As PRs are merged, their summaries are automatically compiled into a chronological changelog. Teams can configure a public changelog page with a custom URL, or consume the changelog data through a JSON API for integration into their own documentation or release note systems.

Weekly Reports. At the end of each week, What The Diff generates a summary of all PRs processed during the week and distributes it through the configured notification channels (Slack, email, or webhooks). These reports provide a bird’s-eye view of development activity without any manual effort.

Token Consumption. Each PR summary consumes tokens based on the size of the diff. Average PRs use approximately 2,300 tokens. Large PRs with hundreds of changed lines will consume proportionally more tokens. Teams can monitor their token usage through the dashboard and configure limits to prevent unexpected overages.

Who Should Use What The Diff

Teams with compliance or documentation requirements are the strongest fit for What The Diff. Organizations that need consistent, detailed PR descriptions for audit trails, regulatory compliance, or internal documentation standards get immediate value from automated summary generation. The tool ensures every PR has a thorough description, regardless of individual developer discipline.

Cross-functional teams where product managers, designers, QA engineers, or other non-technical stakeholders need to understand what is being shipped benefit significantly. The plain-language summaries and weekly reports keep everyone informed without requiring them to read code diffs.

Teams maintaining public changelogs save significant time with the automated changelog feature. Instead of manually compiling release notes from git logs and PR descriptions, the changelog builds itself incrementally as PRs are merged.

Small teams with limited PR volume (under 40 per month) can get substantial value from the Pro plan at 19 dollars per month. At this scale, the per-PR cost is well under 50 cents and the time savings on writing descriptions and changelogs easily justify the expense.

Teams NOT well served by What The Diff include those looking for actual code review, bug detection, or security scanning (use CodeRabbit, Diffray, or Semgrep instead), teams with very high PR volumes where token costs become significant, and organizations using Bitbucket or Azure DevOps (which are not supported).

What The Diff vs Alternatives

What The Diff vs CodeRabbit. CodeRabbit is a comprehensive AI code review tool that also generates PR summaries. At 24 dollars per user per month for Pro (with a free tier), it costs more but provides actual code review with bug detection, security scanning, 40-plus built-in linters, and auto-fix suggestions in addition to summaries. For teams that need both review and summarization, CodeRabbit is the better all-in-one choice. What The Diff is the better choice for teams that already have a code review workflow and just need automated documentation.

What The Diff vs Graphite. Graphite is a developer workflow tool that includes PR description generation alongside stacked PR management and merge queue capabilities. If your team is already using or considering Graphite for its workflow features, its built-in description generation may make What The Diff redundant. What The Diff offers deeper summarization capabilities and changelog generation that Graphite does not match.

What The Diff vs GitHub Copilot PR Descriptions. GitHub Copilot (available in Copilot for Business and Enterprise) can generate PR descriptions as part of its broader AI coding assistant capabilities. Copilot’s advantage is native GitHub integration with no additional app installation. What The Diff’s advantages include GitLab support, automated changelog generation, weekly reports, and a focused feature set that does summarization better than a generalist tool.

What The Diff vs Manual PR Templates. Many teams use PR templates to standardize descriptions. Templates ensure structural consistency but still require developers to fill in the content. What The Diff automates the content generation itself, which is more effective at ensuring descriptions are actually informative. The two approaches can be combined: use a template for structure and What The Diff for content.

Pros and Cons Deep Dive

Strengths:

The quality of generated summaries is genuinely good. What The Diff produces descriptions that are clear, well-structured, and informative enough to orient a reviewer before they dive into the code. The summaries are language-agnostic, working equally well for a Python backend change as for a React component update. Multiple users report that the summaries are often better than what developers would write manually.

The changelog feature is an underappreciated differentiator. Many teams spend hours at the end of each release cycle compiling changelogs from PR descriptions and git logs. What The Diff automates this entirely, with a public page and JSON API that integrate into existing documentation workflows. For teams that ship frequently, the cumulative time savings are substantial.

GitHub and GitLab support gives it broader platform coverage than many competitor tools. While some AI review tools are GitHub-only, What The Diff works with both major platforms, making it accessible to a wider range of teams.

The privacy model is strong. No code storage, no AI training on user data, and minimal data retention. For organizations that are cautious about third-party access to their codebase, What The Diff’s approach provides meaningful assurance.

Weaknesses:

The biggest limitation is what the tool is not: it is not a code review tool. Teams that adopt What The Diff hoping for bug detection, security scanning, or code quality analysis will be disappointed. It is purely a summarization and documentation tool, and it should be evaluated only on those terms.

The token-based pricing model creates unpredictability. Teams with variable PR sizes may find themselves running out of tokens mid-month, especially when large refactoring PRs consume disproportionate token counts. The lack of token rollover compounds this issue — tokens not used in a slow month cannot offset a busy month.

The free tier at 25,000 tokens per month covers only about 10 PRs, which is barely enough for a single developer’s weekly output. This makes the free plan useful only for evaluation rather than sustained use.

The tool does not provide a mechanism for learning team preferences or adapting its summary style over time. Each summary is generated independently based on the diff, without considering how previous summaries were received or what style the team prefers. Custom templates help, but they are a static solution to what would benefit from dynamic adaptation.