Process
How to Do a Security Review
Last updated:
April 28, 2022

This document outlines a simple approach for conducting a security review. We don't use the related term "security audit" because the word "audit" would traditionally imply a certified professional completing this work to provide a level of verification.

IMPORTANT. We cannot provide a guarantee that this process will lead to discovering all vulnerabilities, but it's a good starting point to refining your own process whether you are doing this internally or on behalf of clients. There may be a temptation to consider this document as an exhaustive "checklist". Unfortunately, security is more an art than a science and each set of contracts require their own distinct treatment. We hope that this document is a helpful reference if you forgot to consider one or the other attack vector, but it should not constitute the entirety of your security process.

Scope

Before starting, identify and agree on the scope of the review. Reviews can encompass a whole patch or focus on as little as one function. Other ways scope can manifest are the following:

  • Effort level. This can be measured in time, attention, resources or all the above. The term "light review" is used to refer to a lower effort review than a "full review"
  • Reporting. Some reviews require bugs to be simply raised as GitHub issues, others require a finalized and dated audit report with the findings. Reporting levels are usually correlated with effort levels
  • Classes of issues. Reviews can focus exclusively on flow of value (loss of funds), ownership, economic issues, etc.
  • Tool usage. Reviews involving static analysis, formal verification and fuzzing tools are usually more time intensive than pure human review, especially if they require defining custom specifications or security properties

Agree on a specific commit.

Prioritize

Since security reviews often happen in a limited time frame, you should focus your attention on areas that are more likely to reveal vulnerabilities:

  • Focus on the current patch of changes rather than the whole code base
  • Focus on areas where value and/or ownership can be lost as well as interactions with external contracts
  • Review previous audit reports or related audit reports to similarly structured systems and check issues that could reoccur
  • Focus on types of vulnerabilities that are more common (e.g., error handling, incorrect assembly code, incorrect use of time and block information)
  • Focus on code boundaries
  • Focus on suspicious code (e.g., code covered by less tests, code that's been hastily written, code that's unaudited, code that's been part of a new requirement, code that's written in bad style, code that is really complex or difficult to understand, code that is not well documented in specification/architecture docs)
  • Focus on areas that are difficult to test and may have been avoided by the developers
  • Focus on problem areas flagged by tooling (e.g., bad security patterns)

A good way to explicitly capture these priorities is to create a list of questions to answer during the review, e.g., "establish that user funds cannot be lost".

Setup an auditing environment

As you look for and discover vulnerabilities, you will need a way to test them. Learn how to use the tooling in the given code repository in order to create your own tests or set up a dedicated environment for experimentation. A test net deployment of the code base could also help with this purpose but is not always feasible.

Read the code

Follow How to Quickly Understand a Protocol to speed this process up.

Make sure you get as much context as possible. Unless you are specifically required to do a black box code review, you should use any and all available documentation to get up to speed with what the system is trying to do. If some areas are not well documented or confusing, ask questions to solidify your understanding. Chances are the ambiguity may have led to an actual issue.

Study the most significant dependencies (e.g., libraries, external code, etc.). Read the audit reports for each dependency as they could suggest potential issues in usage or incorrect assumptions.

Generate diagrams

Use a Visual Inspection Tool like Surya to generate overview diagrams for the code base. These will come in handy in a wide variety of situations, for example:

  • verifying access control and state transition behavior
  • identifying the most important dependencies or areas of complexity
  • finding dead or unused code.

Use appropriate review techniques as demanded by your questions

From your list of questions, you can use the references below to find the most appropriate review techniques and tools to answer these questions. The use of each technique should be informed by the specific nature of the smart contracts being reviewed.

Review techniques

Review tools

Overview of techniques & tools

Technique Visual Inspection Debugger
Static Analysis Fuzzing Formal verification
EARLY SCANS
How to Quickly Understand a Protocol
x x
Line-by-line code review x
Static Analysis x
Vulnerability Scanning x
Code Quality Review x x
CORE REVIEWS
Storage Analysis x x
State Transition Review x x
Access Control Review x x
Flow of Value Analysis x x x x
External Interactions Review x x
How to profile and optimize gas consumption x x
Test Quality Review
DEEP DIVES
Upgradeability Review x x
ERC Standards Review x
Numerical Review x x x x
Assembly Code Review x
Oracle Review
Potential Integrations Review
Risk Communication

Prepare a summary report

Even for internal reviews, it's recommended to prepare a review output for the following purposes:

  • Make it clear to engineers the complete list of issues that need to be addressed and allow the resolution of those issues to be tracked in an issue tracker
  • Eliminate any ambiguity of whether a security issue was raised or not and how it was communicated
  • Enable a review retrospective to occur later on that could improve the review process
  • Use the trust models and other associated documentation as an on-boarding reference for future engineers joining the team

Reflect

After the security review is complete, you may discover that some issues were not discovered at the time. This is an opportunity to come back to the process and analyze if you missed the vulnerability simply due to human error or because of a missing step in the process. Ask yourself, what kind of auditing technique would have helped discover this issue? Implement it for the next review. Please get in touch with us on Twitter if you discover a new auditing technique.

Disclaimer. We are not a professional security auditing firm. We have merely listed techniques that we have found useful from time to time, use them carefully and at your own judgment.

See Also: