A paradigm shift is underway in reinforcement learning (RL) research. Yhe dominant approach to RL over the past 20 years has been based on dynamic programming, whereby RL agents klearn asn abstract value function based upon expected discounted future rewards.

An alternative approach, Direct Reinforcement (DR), has recently been revisited, wherein DR agents learn strastegies to solve problems directly. DR can enable a simpler problem representation, avoid Bellman's curse of dimensionality, and offer compelling advantages in efficiency. Moreover, by considering not just expected rewards, but the distribution of rewards and associated risks, more robust policies can be learned. I call this approach Risk Averse Reinforcement.

I review and contrast the major approaches to RL, present Direct Reinforcement, describe an approach to Risk Averse Reinforcement and apply these methods to asset management with transaction costs. In this very challenging domain, DR agents seeks to discover strategies that maximize profit, economic utility or risk adjusted returns. The potentiasl powers of DR are illustrated through an asset allocator and an intradaily foreign exchange trader.

Other promising applications of DR and Risk Averse Reinforcement may be found in robotics, autoomous vehicles, industrial control, and decision support. As an example, I present preliminary work on an autonomous gent that plays a simple competitive game., John Moody is a Professor of Computer Science at the Oregon Health & Science University. His research interests include machine learning, neural and statistical computing, time series analysis and computational finance. He previously held positions in Computer Science and Neuroscience at Yale University and at the Institute for Theoretical Physics in Santa Barbara. Moody received his B.A. in Physics from the University of Chicago, and earned his Ph.D. in Theoretical Physics at Princeton.