Month: September 2016

Why does deep and cheap learning work so well?

We show how the success of deep learning depends not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can be approximated through “cheap learning” with exponentially fewer parameters than generic ones, because they have simplifying properties tracing back to the laws of physics. The exceptional simplicity of physics-based functions hinges on properties such as symmetry, locality, compositionality and polynomial log-probability, and we explore how these properties translate into exceptionally simple neural networks approximating both natural phenomena such as images and abstract representations thereof such as drawings. We further argue that when the statistical process generating the data is of a certain hierarchical form prevalent in physics and machine-learning, a deep neural network can be more efficient than a shallow one. We formalize these claims using information theory and discuss the relation to renormalization group procedures. Various “no-flattening theorems” show when these efficient deep networks cannot be accurately approximated by shallow ones without efficiency loss – even for linear networks.


Why does deep and cheap learning work so well?
Henry W. Lin, Max Tegmark


Collective motion of groups of self-propelled particles following interacting leaders

In order to keep their cohesiveness during locomotion gregarious animals must make collective decisions. Many species boast complex societies with multiple levels of communities. A common case is when two dominant levels exist, one corresponding to leaders and the other consisting of followers. In this paper we study the collective motion of such two-level assemblies of self-propelled particles. We present a model adapted from one originally proposed to describe the movement of cells resulting in a smoothly varying coherent motion. We shall use the terminology corresponding to large groups of some mammals where leaders and followers form a group called a harem. We study the emergence (self-organization) of sub-groups within a herd during locomotion by computer simulations. The resulting processes are compared with our prior observations of a Przewalski horse herd (Hortob\’agy, Hungary) which we use as results from a published case study. We find that the model reproduces key features of a herd composed of harems moving on open ground, including fights for followers between leaders and bachelor groups (group of leaders without followers). One of our findings, however, does not agree with the observations. While in our model the emerging group size distribution is normal, the group size distribution of the observed herd based on historical data have been found to follow lognormal distribution. We argue that this indicates that the formation (and the size) of the harems must involve a more complex social topology than simple spatial-distance based interactions.


Collective motion of groups of self-propelled particles following interacting leaders
Bence Ferdinandy, Katalin Ozogány, Tamás Vicsek


Computational Personality Analysis | Yair Neuman

The emergence of intelligent technologies, sophisticated natural language processing methodologies and huge textual repositories, invites a new approach for the challenge of automatically identifying personality dimensions through the analysis of textual data. This short book aims to (1) introduce the challenge of computational personality analysis, (2) present a unique approach to personality analysis and (3) illustrate this approach through case studies and worked-out examples.
This book is of special relevance to psychologists, especially those interested in the new insights offered by new computational and data-intensive tools, and to computational social scientists interested in human personality and language processing.


Computational Personality Analysis: Introduction, Practical Applications and Novel Directions
Neuman, Yair


The price of complexity in financial networks

Estimating systemic risk in networks of financial institutions represents, today, a major challenge in both science and financial policy making. This work shows how the increasing complexity of the network of contracts among institutions comes with the price of increasing inaccuracy in the estimation of systemic risk. The paper offers a quantitative method to estimate systemic risk and its accuracy.


The price of complexity in financial networks
Stefano Battiston, Guido Caldarelli, Robert M. May, Tarik Roukny, and Joseph E. Stiglitz

PNAS vol. 113 no. 36