Size Matters

One of the facts I vividly remember of studying physics in university (this was in the mid 90’s in Geneva, Switzerland) was a certain disconnection between Relativity and Quantum Mechanics. The former is a theory used to describe phenomena at macro level, like galaxies, planets, stellar systems, while the latter describes the interactions at micro level, the atoms, light, particles, energy at microscopic levels.

When you apply some relativity equations to atoms, you get results that are not supported by experimentation; and the same happens when you apply some quantum theory equations to objects like planets. It is not that all the relativistic facts do not apply at micro level, or that all quantum facts do not apply in macro level; it is that there is no unifying theory that explains everything, and this quest is the graal of modern physics.

Fast-forward 5 years.

One of the facts I vividly remember of studying Economics in university (this time in Buenos Aires) was a certain disconnection between Microeconomy and Macroeconomy. The models that describe the behavior of the consumer (which is the heart of the study of Microeconomy) yield wrong conclusions when applied to issues like unemployment, foreign trade or other matters that are usually better explained by Macroeconomy; and similarly, well, you get the picture.

As a matter of fact, my Macroeconomics teacher would say that whatever we learnt in Microeconomics class was wrong, and that he had the right answers; of course, the Micro teacher told us the same the year before.

Fast-forward 5 years.

One of the facts that I vividly remember of studying Computer Science during my master degree program was a certain disconnection between small and big software projects. What works in small, simple applications and systems, including human and technical factors, does not usually work in bigger, more complex projects.

It is not the same to work on a startup project with some friends in a garage to create the next social networking site, where coordination is easy, most of the tools required are available for free, where the projects rarely have any dedicated quality assurance team, than working in, say, a bigger organization like Microsoft, together with other 1500 engineers and testers, all dedicated full time to writing and testing the next version of Windows.

The hardware requirements are not the same, either; in small projects you could use a couple of Mac Minis and a cloud hosting service and you are done; at Google they have MapReduce and a couple hundred thousand computers in a datacenter with air conditioning and security 24/7, and they still require more infrastructure every day.

However, and this is my main point, there is no proven recipe that can help a company grow from 10 to 10'000 people and from 10 to 10 million customers in a snap; there are some good techniques and principles, here and there, to make your software grow; but nobody actually knows of a generic recipe for every software company.

There are so many factors in macro problems, that the interaction of those factors has to be taken into account; not only the factors themselves, but also their interdependencies. I guess you see where I am going with this. This problem is usually called scaling in computer circles, and I think that the word can be applied to economy and physics.

As humans, we have trouble scaling. Scale is important in our eyes, because we tend to think that bigger is better. Bigger is more money, in general, but not necessarily better; we have trouble going from small to big and vice versa. Not only in facts; also in concepts. We cannot foresee the implications of scaling. At least, not completely, and not so far.

This disconnection creates lots of problems in our society. Politicians forget the human being altogether, buried beneath tons of numbers and statistics. Voters do not understand that managing a country is not like managing your household economy. Schools do not teach how to solve scalability problems; heck, they do not even properly teach kids how to work in teams to solve small, micro problems.

Small companies do not understand that scaling is neither automatic nor a required process, and that not all companies should grow; some companies work better when small than when they grow up, and that’s why they sometimes fail. Venture capitalists that are not familiar with technology cannot understand this fact, and will sometimes sacrifice good working teams for just making more money or for getting into the stock market.

The knowledge we have about the problem of scaling is limited; I actually sometimes ask myself whether there is a solution to it, that would justify the search of a global theory in physics, a unified theory in economics, or a generic scaling procedure for companies and software systems.

I do not have the answer; the fact is that size matters, and that this pattern has to do with the world we are living in; it does not matter whether you are a physicist, an economist or a programmer; this is how the world works.