Seventy years ago cybernetics was a hot field; thirty years ago, it was catastrophe theory. Those Greek-inspired words for disciplines that once brought hope of explaining human behavior now evoke a quaint nostalgia, like Polaroids of long-haired young people in bell-bottomed jeans and tie-dyed T-shirts. The new buzzword nowadays is Big Data, the fashionable term for capturing and analyzing the vast collections of information that people reveal about themselves when shopping on Amazon, Travelocity, and Netflix, or when writing about themselves on Facebook and Twitter. Big Data utilizes a mix of computer science, information technology, mathematics, and applied statistics. It is increasingly used to sell you products or persuade you to vote for politicians by tailoring the product’s or politician’s image to your particular data-generated persona. Some talking heads like to say that computer-aided analysis of patterns will soon replace our traditional methods of discovering the truth, in medicine and the social sciences as well as in physics.

What are the classic ways of knowing? Recall the great triumph at the dawn of modern science, the understanding of gravitation and motion. How did that come about?

For millenia after the Greeks, scientists’ prejudices led them describe all planetary movements in terms of circles about a stationary earth. But the motion of a planet, as seen from the orbiting earth itself, is too complicated for a single circle -- sometimes it seems to move backwards relative to the earth --  and so it needs circles moving on circles moving on circles, i.e. epicycles. Eventually, Galileo pointed out that the earth wasn’t stationary, that the earth and planets orbited the sun, and that the planets’ weird apparently retrograde motions were not intrinsically theirs but rather a consequence of their being observed from the moving earth.

Intuition, followed by checking the data

In the early 1600s Kepler examined the data on planetary positions and formulated his three astonishing laws of planetary motion: planets move in ellipses (not circles) about the sun, the line between the Sun and a planet sweeps out equal areas in equal times, and the square of the orbital period is proportional to the cube of the distance from the sun.

If you want to glimpse the miracle of  discovery, think about Kepler’s second law: the line between the Sun and a planet sweeps out equal areas in equal times. This deep symmetry of planetary motion implies that the closer the planet to the sun, the more rapidly it moves, as shown below.

The astonishing thing is that there is no line between a planet and the sun for Kepler to observe. His data consisted of planetary positions in the night sky. How then did he decide to describe the motion of the planets in terms of an invisible imaginary line?  No one knows exactly, but it involved long immersion, struggle, and strange associative thinking that arose from somewhere inside him, and then - Aha! - intuition, followed by checking the data.

How to discover theories

Intuition is the first means of knowing. The observer becomes so close to the object (or person) observed that he begins to experience their existence from both outside and inside them. Intuition is a merging of the observer with the observed. It’s almost quantum-like, the ability to be in two places at the same time.