Having a rich, yet standardized intermediate representation (IR) has enabled a great deal of compiler work to be done by allowing new transformations (i.e. optimization passes) to be easily implemented. In this talk, I will describe our work to develop a similar framework for binary analysis. The key building block for this system is a simple annotation framework that allows various cooperating tools to annotate representations of instructions and other binary analysis data structures. Using this framework, new analysis (such as slicing, live register analysis, etc.) can be created that both use annotations created in previous passes and add their own annotations.