Win-Vector Blog » Reverse Accumulation http://www.win-vector.com/blog The Applied Theorist's Point of View Sat, 04 Feb 2012 17:42:12 +0000 en hourly 1 http://wordpress.org/?v=3.3.1 Gradients via Reverse Accumulation http://www.win-vector.com/blog/2010/07/gradients-via-reverse-accumulation/?utm_source=rss&utm_medium=rss&utm_campaign=gradients-via-reverse-accumulation http://www.win-vector.com/blog/2010/07/gradients-via-reverse-accumulation/#comments Thu, 15 Jul 2010 00:00:04 +0000 John Mount http://www.win-vector.com/blog/?p=1493
  • Automatic Differentiation with Scala
  • Automatic Detection of Potential Deadlock
  • Must Have Software
    • ]]>     We extend the ideas of from Automatic Differentiation with Scala to include the reverse accumulation. Reverse accumulation is a non-obvious improvement to automatic differentiation that can in many cases vastly speed up calculations of gradients.
    As the tables, diagrams and equations do not translate well into HTML, our full article is available here in PDF: http://www.win-vector.com/dfiles/ReverseAccumulation.pdf.

    The purpose of our article is to explain reverse accumulation automatic differentiation clearly (and to release some sample code and timing results). A side effect of the article is to make sense of the following two diagrams:

    If the following is picture of standard or forward differentiation:

    cutFwd.png

    then the following is a picture of reverse accumulation:

    cutRev.png

    Example code now distributed from: github.com/WinVector/AutoDiff.

    Related posts:

    1. Automatic Differentiation with Scala
    2. Automatic Detection of Potential Deadlock
    3. Must Have Software

    ]]>     http://www.win-vector.com/blog/2010/07/gradients-via-reverse-accumulation/feed/ 0