Crosshatch Reviewer: Easier code reviews with AI

Code reviews are a critical part of the software development process, often requiring significant time and attention from developers. Crosshatch Reviewer is a tool that we built for ourselves to streamline this process by automatically summarizing pull requests on GitHub repositories. We thought it was very useful, so we made it available for the public. The code is open source and available in our Github repo.

Simplifying Code Reviews

Crosshatch Reviewer generates concise summaries of code changes in pull requests, helping developers quickly understand the scope and impact of modifications.

Setup is straightforward:

1. Visit the website
2. Click "Install on GitHub"
3. Select the repository for integration
4. Voila! Whenever a pull request is opened or updated, Crosshatch Reviewer will automatically review it and add a comment with its analysis.

Powered by Crosshatch Mixes

The tool uses the LMSys Coding mix, an endpoint that consistently directs requests to the top-performing Language Model (LLM) for code-related tasks. This approach ensures that Crosshatch Reviewer always uses the best models available. More information about this mix, which you can use today in your development workflow or LLM-workflows involving code, is available at https://crosshatch.app/mixes/lmsys-coding.

Open Source and Community-Driven

Crosshatch PR Reviewer's code is open source, available at https://github.com/catena-labs/crosshatch-reviewer/. We encourage you to deploy it on your repos, customize it, and contribute to its development. We hope this open approach will inspire the creation of other AI-assisted development tools.

Key Features

1. Automated PR Summaries: Crosshatch Reviewer automatically generates concise summaries of pull requests, highlighting the most significant changes and potential areas of concern.
2. GitHub Integration: The tool integrates with GitHub, activating automatically when new pull requests are opened or existing ones are updated.
3. AI-Powered Analysis: Utilizing the LMSys Coding index mix, Crosshatch PR Reviewer employs top-performing language models to understand and summarize code changes effectively.
4. Diff Analysis: The tool performs a detailed analysis of the diff between branches, extracting meaningful changes while filtering out less relevant modifications like whitespace changes.
5. Comment Management: Crosshatch Reviewer creates or updates comments directly on the pull request, ensuring that the summary is always readily available to reviewers.

How It Works

1. Event Triggering: When a pull request is opened or updated on a GitHub repository where Crosshatch PR Reviewer is installed, it triggers a webhook event.
2. Diff Retrieval: The tool uses the GitHub API to fetch the diff of the pull request, capturing all the changes made.
3. Diff Analysis: The Reviewer processes the raw diff, extracting meaningful changes and filtering out less significant modifications.
4. AI Summarization: The extracted changes are sent to the AI model (via the LMSys Coding index mix) for analysis. The AI model generates a comprehensive summary focusing on notable changes, potential concerns, and suggested improvements.
5. Comment Creation/Update: The tool then interacts with the GitHub pull request, either creating a new comment with the summary or updating an existing one.
6. Continuous Monitoring: Crosshatch Reviewer continues to monitor the pull request for any updates, repeating the process as necessary to keep the summary current.

By automating these steps, Crosshatch Reviewer provides a constant, reliable assistant in the code review process, helping teams maintain high-quality standards while potentially improving the efficiency of their development workflow.

Conclusion

Crosshatch Reviewer represents just an example of how we might intelligently AI assistance into the code review process. While it currently offers automated summaries of pull requests, there's significant potential for expanding its capabilities to address a broader range of development needs.

Potential areas for extension include:

1. Automated Testing Integration: Crosshatch Reviewer could be enhanced to automatically run and report on unit tests or integration tests associated with the changed code. This could provide immediate feedback on whether the proposed changes break existing functionality.
2. Enhanced Security Analysis: By incorporating specialized security-focused models or tools, the reviewer could offer more in-depth analysis of potential security vulnerabilities introduced by code changes.
3. Performance Impact Assessment: An interesting extension could involve estimating the performance impact of code changes, potentially flagging modifications that might significantly affect application speed or resource usage.
4. Code Style and Best Practices: The tool could be expanded to check for adherence to coding standards and best practices specific to the project or language being used.
5. Dependency Analysis: Integrating functionality to analyze and report on changes to project dependencies could help teams stay on top of potential conflicts or security issues related to third-party libraries.
6. Documentation Suggestions: The reviewer could be extended to suggest updates to documentation based on code changes, helping to keep project documentation in sync with the codebase.

We’re excited for your feedback and contributions!