The Illusion of Managing Big Data and Big Computation

Classical problems in climate, genomics, energy, health care and indeed all research areas have always involved data and computation.  Half a millennium ago, Kepler translated the remarkable data of Tycho Brahe into his three beautiful laws of planetary motion, and Newton gave a closed form solution getting around a seemingly insurmountable calculation using calculus.  During the last three decades, we have seen extraordinary advances in computing, networking, and storage, each following a Moore’s law exponential growth with roughly equal bases.  But data and computational complexity have likewise grown at least exponentially with bigger bases.  In what we hope will be an interactive session, we explore the challenge of managing data and computational problems growing much faster than the technologies to move, manipulate and store, and talk about some cases, one local to Stony Brook, where a closed form solution is found for an otherwise massive problem.  Finally, we will talk about the transport component, and the NYSERNet network and exchange point specifically, and its relationship with the other components of a tightly coupled technology ecosystem necessary for data driven research.