The Architecture of Open Source Applications: The Blog

  • Michael DiBernardo - Tue 24 November 2015 -

    Today we've published the eleventh chapter in our early access release for 500 Lines. The chapter was written by Dessy Daskalov.

    As computers continue to become more fully integrated into human life, more of us are writing programs that are forced to interact with the messiness of the real, physical world. Dessy demonstrates how we can build and refine effective models of reality (much like many other scientists do every day!) in order to accomplish the practical goal of building a working pedometer.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Thu 12 November 2015 -

    Today we've published the tenth chapter in our early access release for 500 Lines. The chapter was written by Allison Kaptur.

    In this chapter, Allison demystifies the process of writing an interpreter by building one up from very modest beginnings. In the process, we learn about stack machines and how they can be used as a foundation for simply expressing what appear at first glance to be very complicated computations.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Fri 06 November 2015 -

    Today we've published the ninth chapter in our early access release for 500 Lines. The chapter was written by Erick Dransch.

    Any student of 3D graphics programming will quickly encounter the pattern of the 'scene graph', which is used to organize and orient the components of a scene to be rendered. Erick's chapter teaches us about this pattern, but additionally shows us how this core data structure can be integrated into an interactive system.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Wed 28 October 2015 -

    Today we've published the eighth chapter in our early access release for 500 Lines. The chapter was written by Taavi Burns.

    Continuing with our persistent datastore theme from last week, Taavi's chapter focuses on how we characterize and implement guaranteed behaviour in our systems. This chapter is also different than many of the others, in that we learn about this system the way that most programmers do; by exploring it in its completed form, rather than building it up in increments.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Tue 20 October 2015 -

    Today we've published the seventh chapter in our early access release for 500 Lines. The chapter was written by Yoav Rubin.

    Programmers often encounter some sort of persistent data store very early in our careers. There is nothing quite as opaque and mysterious as a full-featured database system, regardless of the underlying paradigm used for data organization.

    In this chapter, Yoav shows how using immutability as a guiding principle allows us to build an astonishingly capable database system in fewer than 500 lines of Clojure. This includes a transaction system, a miniature declarative query language, and a host of other features.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Wed 07 October 2015 -

    Today we've published the sixth chapter in our early access release for 500 Lines. The chapter was written by Carl Friedrich Bolz.

    In this chapter, Carl teaches us that we don't need to know how to write a compiler just to experiment with producing different semantics. He also teaches us a bit about test-driven development along the way.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Tue 29 September 2015 -

    Today we've published the fifth chapter in our early access release for 500 Lines. The chapter was written by Dustin J. Mitchell.

    Donald Knuth once said that the best theory is motivated by practice, and the best practice by theory. In this chapter, Dustin does something that many programmers have do at least once in their career; he implements a rather opaque algorithm from primary sources (PAXOS) that solves a very practical problem (distributed consensus.) He then demonstrates how sound engineering practices allow us to implement something that at first seems slightly beyond our capability, and to convince ourselves that it actually works.

    As usual, if you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Tue 22 September 2015 -

    Today we've published the fourth chapter in our early access release for 500 Lines. The chapter was written by Leah Hanson.

    Many programmers treat their programming language implementations as black boxes. Static analysis tools or language analysis tools can seem indistinguishable from magic, and good treatments of these topics have been buried deep in academic curricula that few of us make time for.

    In our opinion, Leah has done a terrific job of demystifying the nature of the techniques used in static analysis and introspection in her chapter. While her work is specific to Julia, the lessons learned are applicable in other languages. Enjoy!

    If you find errors you think are worth reporting, please open an issue on our GitHub tracker.

    We'd also like to take this opportunity to remind our readers that this book would not have been possible without the financial support of PagerDuty. Please take this opportunity to let them know if you've found this work useful.


  • Michael DiBernardo - Tue 15 September 2015 -

    Today we've published the third chapter in our early access release for 500 Lines. The chapter was written by A. Jesse Jiryu Davis and Guido van Rossum.

    We hope that this chapter provides Python programmers with a deep introduction to how coroutines work, and why and when we should use them. Coroutines first became a hot topic in Python with the release of the asyncio framework; now, in Python 3.5, they are also built directly into the language itself.

    Beyond Python, there has been a longstanding tension between thread-based and event-based systems in computer science. It can be helpful to a practising programmer to understand the nuances in this debate, as anyone building things atop frameworks that use these constructs will eventually have to understand them.

    We will have chapters on building thread-based and event-based systems in "500 Lines or Less", and we think that this chapter will serve as an interesting contrast to both of these contributions in the final draft of the book.

    If you find errors you think are worth reporting, please open an issue on our GitHub tracker.


  • Michael DiBernardo - Tue 08 September 2015 -

    Today we've published the second chapter in our early access release for 500 Lines. Audrey Tang has contributed chapters to two previous volumes of AOSA, and we're very happy to have her back for "500 Lines or Less."

    When Audrey's chapter was first presented to readers, many protested at how dependent it was on external frameworks and libraries. This is only half of the truth; Instead of just leveraging these components, Audrey provides a miniature ontology for the kinds of patterns that web programmers can expect to use now and in the future. (For the curious, there is also a 99-line version that uses only DOM APIs.)

    Most engineers spend a lot of time figuring out how to build software based on the work of others. We feel that reading about how an experienced engineer identifies the patterns in software she is reusing is a valuable lesson for us all.

    This chapter was written way back in 2014 when Traceur was the leading ES6 compiler implementation. Audrey has informed us that she probably would have used Babel if she was writing this chapter now. If you are interested in updating the source and the chapter to reflect this, we're open to pull requests!

    If you find errors you think are worth reporting, please open an issue on our GitHub tracker.

    Audrey also helpfully translated her chapter into Traditional Chinese; you can find the source for this version here.


  • Michael DiBernardo - Tue 01 September 2015 -

    It has been a long time coming, but I'm happy to announce that the first artifact from 500 Lines or Less will be published on today.

    Writing a chapter for 500 Lines is a rigorous process. The code for each chapter is first reviewed by technical reviewers; then, the chapter drafts are reviewed by different readers for content and accessibility. Finally, our heroic copy editor Amy Brown examines each line of text for grammatical errors or tricky phrasings. And yet, we're sure there are still bugs in this book waiting to be found.

    Just as it is very expensive to fix a bug once it hits production, it is similarly painful to find an error in a book once it has already gone to print. To this end, we're going to be doing an early-access release of each chapter on the 500Lines website at the rate of one per week for the next 20 weeks. We hope that getting these chapters out early will give all of you the chance to find the snags that we missed throughout this process. We expect to go to print very shortly after this process winds down.

    These early-access chapters will only be published from this blog, and will be linked from the news section on and announced on Twitter from @aosabook. The production version of the website will be published around the same time as the final draft goes to print.

    With all of that out of the way, I'm happy to share Ned Batchelder's template engine chapter with you. Ned's chapter is a great example of what we were hoping to accomplish with 500Lines: it is compact, well-explained, and there are several discussions of interesting decisions and trade-offs he had to make while writing it.

    If you find errors you think are worth reporting, please open an issue on our GitHub tracker. We'll try our best to collate issues sent to us on Twitter and via other channels, but we have a lot of book to work on still, and we're probably going to miss things unless they're persisted in that tracker somehow.

    Many thanks again to PagerDuty for their generous sponsorship of this project. Please give them a public thank-you or two throughout this early-access release for their contribution.

  • Michael DiBernardo - Mon 02 March 2015 -

    We're very happy to announce that PagerDuty will be sponsoring the production of "500 Lines or Less", the fourth book in the Architecture of Open Source Applications series.

    The AOSA books are published and distributed on a not-for-profit basis, with all of our proceeds going to Amnesty International. However, there are always labor costs associated with the production of each book. With PagerDuty's support, we will be able to continue to donate all of the book's proceeds while still covering these costs.

    PagerDuty is well-known for its strong engineering culture, and we feel that they are the perfect partner to support the kind of work we're doing with AOSA.

    We are very grateful for their support. If you are too, please let them know @PagerDuty, or learn more about them.

  • Michael DiBernardo - Thu 03 April 2014 -

    Are you coming to PyCon 2014? If so, we invite you to join us at the 500 Lines or Less Open Space!

    The purpose of the Open Space is to discuss some sample programs and chapters that have been written so far, and to brainstorm possible directions that this work could take from here.

    The open space is at 1PM on Sunday April 14th in room 522A.

    There's no requirement to sign up beforehand -- all you have to do is show up.

    Hope to see you there!

  • Michael DiBernardo - Fri 03 January 2014 -

    While the Performance of Open Source Applications was released just a few months ago, work on the fourth AOSA book is already well underway!

    This volume is called 500 Lines or Less, and is focused on short but complete implementations of canonical programs and architectural patterns in computer science.

    The motivation is described succinctly in the project statement:

    Every architect studies family homes, apartments, schools, and other common types of buildings during her training. Equally, every programmer ought to know how a compiler turns text into instructions, how a spreadsheet updates cells, and how a browser decides what to put where when it renders a page. This book's goal is to give readers that broad-ranging overview, and while doing so, to help them understand how software designers think.

    Each chapter will consist of a (max) 500 line, self-contained program, and a narrative surrounding that program that describes how it works, and (more importantly) why it is designed the way it is.

    Contributors are already working on their 500 line implementations. Once each contributor completes a "first draft" of their code, we assign them a technical reviewers to provide constructive commentary.

    That's where we need your help.

    There are a lot of contributors to this project, and we want them to get as much feedback as possible before they begin writing up their chapters!

    We're looking for reviewers at all levels of experience. If you're still early in your programming career, we'd especially love to hear from you, as your opinions as to which parts are easily accessible and which are confusing will be incredibly helpful.

    In terms of scheduling, our first-draft code submissions are due by end of February, but many contributors are ahead of schedule and could use reviewers right now!

    We're also hoping that our code reviewers will stick around to provide commentary on the drafts of completed chapters once those start to roll in from March onwards.

    All technical reviewers will of course receive credit for their work in the book, and will be supporting a good cause -- all royalties from paid-for versions of the book will go to Amnesty International.

    If this sounds fun to you, please sign up here.

    Questions? Please ask at We look forward to having you on board!

  • Tavish Armstrong - Sun 03 November 2013 -

    The Performance of Open Source Applications is now available for the Kindle through Amazon. As with the other electronic versions, this one costs USD$5.99, with royalties going to Amnesty International.

    If you don't have a Kindle, see the Buy page for more information on available formats.


  • Tavish Armstrong - Fri 01 November 2013 -

    The Performance of Open Source Applications is now available in ePub and PDF formats on Both cost USD$5.99 and as with the print edition, the royalties go to Amnesty International.


  • Tavish Armstrong - Mon 07 October 2013 -

    I'm happy to announce that The Performance of Open Source Applications is finally out. You can read it on our main site or buy a paper copy from Lulu for US$25.00. As with the previous two books, all royalties go to Amnesty International.

    Thank you to everyone who contributed, and everyone who cheered us on. I hope you learn as much from the authors as I did.

    The cover for The Performance of Open Source Applications

  • Tavish Armstrong - Thu 08 August 2013 -

    The Performance of Open Source Applications isn't here yet, but it will be soon. In the mean time, I'll post the chapter list.

    1. High Performance Networking in Chromium by Ilya Grigorik
    2. From SocialCalc to EtherCalc by Audrey Tang
    3. Ninja by Evan Martin
    4. Parsing XML at the speed of light by Arseny Kapoulkine
    5. Memshrink by Kyle Huey
    6. Applying Optimization Principle Patterns to Component Deployment and Configuration Tools by Doug C. Schmidt, William R. Otte, and Aniruddha Gokhale
    7. Infinispan by Manik Surtani
    8. Talos by Clint Talbert and Joel Maher
    9. Zotonic by Michiel KLønhammer, Arjan Scherpenisse, and Marc Worrell
    10. Secrets of Mobile Network Performance by Bryce Howard
    11. Warp by Kazu Yamamoto, Michael Snoyman and Andreas Voellmy
    12. Working with Big Data in Bioinformatics by Eric McDonald and C. Titus Brown

    I'd like to take this moment also to thank everyone who responded to this tweet asking for help with proofreading. You know who you are, and you are great.

  • Greg Wilson - Wed 30 January 2013 -

    Releng 2013 is a one day workshop (co-located with ICSE) to bring together release engineers and researchers to discuss the challenges in release engineering and develop areas for further research. Areas of discussion include research and practice of all activities in between regular development and actual usage of a software product by the end user, i.e., integration, build, test execution, packaging and delivery of software. The conference is May 20 in San Francisco and deadline for submission of talks in February 7th. For more information see

  • Greg Wilson - Mon 12 November 2012 -

    One of the reasons we started this project is that almost every other book on software architecture doesn't actually show readers the architectures of any actual systems. In particular, most university students can get through a four-year degree without ever seeing how large applications are put together. We're therefore very excited to learn that Dr. Neil Ernst is using AOSA in an undergrad course he's teaching at the University of British Columbia. If you'd like to see what his students think of the systems we've described, their presentations are online. And if you know of anyone else doing this, please send us a pointer.

  • Tony Arkles - Tue 30 October 2012 -

    Yesterday we received our first first draft for POSA! My guess is that things are going to start getting busy soon...

  • Greg Wilson - Thu 04 October 2012 -


    Now, if only we could tell how much of each page each visitor is reading... :-)

  • Tony Arkles - Tue 25 September 2012 -

    Well, it's been a busy two months since Tavish or I have posted anything on here. Things have been progressing nicely though! We've got 21 authors lined up, and have already gone through 13 proposed outlines. I'm extremely excited with the proposed chapters so far! We've got a great mixture of low-level nitty-gritty projects and higher level applications.

    Here's a few more examples:

    • Ilya Grigorik is going to be discussing the insane number of hoops that the Chrome team has jumped through to improve Chrome's network performance
    • Clint Talbert and Joel Maher are going to be recounting the journey of improving Talos, the performance testing framework at Mozilla.
    • Michael Snoyman and Kazu Yamamoto are writing about Warp, their high-performance Haskell-based HTTP engine.

    The first drafts of the chapters will (hopefully) start rolling in, and we should have a preliminary table of contents ready to go in the next month or so.

    Thanks again to everyone who has responded! Reading through your outlines, I'm super excited about the book and the fantastic material that's going into it!



  • Greg Wilson - Sun 23 September 2012 -

    Dear GitHub,

    You provide a wonderful workspace where millions of people can share what they make, but there's something you could do to make yourself even better. Could you please provide a way for us to create plug applications directly into your site the way Facebook does? Thousands of really cool software tools are just waiting to be discovered, from code analyzers to project history visualizers, and while your APIs already let them use your data on their own web sites, people would be much more likely to find them and use them if they were nested directly in projects' GitHub pages. It wouldn't be much of a technological challenge—given the way Facebook's stock has been doing, you could probably even persuade one or two of their engineers to come and help build it—and it would kickstart the same healthy explosion of third-party tools we saw when Eclipse first came on the scene.

    Thanks very much,


    p.s. If you like this idea, please tweet the following:

    Dear @github, please let us build plugins #githubplugins

  • Tavish Armstrong - Fri 21 September 2012 -

    We're happy to announce that Volume II of The Architecture of Open Source Applications is now available in ePub format. As always, royalties go to Amnesty International. Happy reading!

  • Greg Wilson - Thu 30 August 2012 -

    John Hunter, the creator of matplotlib and a contributor to Volume 2 of this series, passed away on August 28 as a result of complications arising from treatment for cancer. A memorial fund has been set up to help with his three children's education; please give generously.

  • Tavish Armstrong - Sat 21 July 2012 -

    A few weeks ago Greg posted about the next book we're doing: The Performance of Open Source Applications. Well, we don't have a book yet, but we've made some progress. Earlier this week we had our 15th "yes" from an author, which puts us close to the chapter counts of AOSA. We're excited about that and we hope you are too. Here are a couple of the chapters we're planning:

    • Jessica McKellar on Twisted again, getting into the details of asynchronous I/O.
    • Audrey Tang on profiling and optimizing Ethercalc, a Google Docs-esque spreadsheet application written in node.js.
    • Kyle Huey on Mozilla's Memshrink project, which aims to make Firefox faster and "slimmer".
    • Rosangela Canino-Koning and Eric McDonald on processing terabytes of data in the Khmer project, which is a "library and toolkit for doing k-mer-based dataset analysis and transformations".

    Remember, if you want to see your favourite Open Source Application discussed in the new book, we're accepting contributions. If you're interested, please get in touch at If you don't want to write a chapter yourself, why not send someone our way?

  • Greg Wilson - Wed 27 June 2012 -

    We are pleased to announce that we are starting work on a third book in this series, which will be titled The Performance of Open Source Applications. Each chapter will discuss a performance issue in a real open source system—it could be an over-the-shoulder view of how a performance problem was fixed, a discussion of how design decisions affected performance in a particular application, or something else along those lines. Each entry will be 12-15 pages long, and we hope to have first drafts by the end of October so that we can publish the book in Spring 2013. As with AOSA, royalties will go to Amnesty International and the book will be available for free online under a Creative Commons license. If you are interested in participating, please contact us at

    Why performance rather than architecture? Because it's something that every programmer has to deal with eventually, but which is usually left out of their education. The last general book on making programs fast that we know of was Jon Louis Bentley's Writing Efficient Programs, which was published thirty years ago. There have been lots of more specialized books since (we're particularly fond of Steve Souders' High Performance Web Sites and Even Faster Web Sites, and of John Lakos's Large-Scale C++ Software Design), but we think the time is right for something that touches on everything from squeezing the last few cycles out of every precious milliwat in an embedded sensor to maximizing throughput of large-scale e-commerce applications. We hope you'll think so too, and we look forward to hearing from you.

  • Greg Wilson - Fri 08 June 2012 -

    Amy Brown, our indefatigable editor, has posted a series of articles on her blog about what's involved in going from LaTeX to a published book on Lulu. We think this is a great alternative to traditional (read: expensive and restricted) academic publishing, and would welcome other tips and tricks.

    1. Headers and Footers
    2. Fonts and Captions
    3. Table of Contents and Chapter Title Pages
    4. Custom Commands and Environments
    5. Other Useful Packages and Settings
    6. Pulling It All Together
    7. Making AOSA: Tools

Content © The AOSA Editors.
Proudly powered by bootstrap, pelican, python and Alex!