It does not matter how deep your learning is!
João Pires da Cruz, Closer Consulting
Have you ever thought about why do you have a brain? Let me be honest with you: technologically speaking, your brain is a piece of crap. Or, in the British way of saying these things, “it has some challenges”.
It is too heavy, too big, too sensitive to mechanical impacts, it has a very narrow temperature band of working that leads to the need for a massive power plant to supply the energy needed and considerable water consumption. In fact, if your brain had to design itself, it could not make it worse than it is.
But it was not designed by you. It was crafted by means of natural selection, which is a very efficient way of putting crappy things aside in the evolution of species: either it justifies the costs, or it is eliminated because specimens with lesser fitness get eaten by predators. That means that the vast cost carried by having a brain is compensated by a considerable surplus. What is that surplus?
Even though your brain can do amazing things, including making computer programs that emulate itself, we must think that primary forms of humanoids could not do those amazing things. Even basic forms of mammals, like dogs or rats, are unable to perform more complicated “intelligent” tasks. But they do have a brain! Why?
Let us think about the image that you have in front of you now. Instead of thinking of 2048x1024 pixels, let us make it simpler and let us consider that you have a flat screen with 10x10 rectangles and that in each rectangle, you can put between 0 and 5 different objects because your universe only has 5 objects. Very modest numbers, right? Well, with these very modest numbers, we are speaking about 5^100 different possible images. If you want to learn this simplistic scenario and learn 1 billion possibilities each day, you will need much, much more than the entire age of the universe to understand it. If you were a primary humanoid in the wild, you would be eaten before you could even know where to start. And we are just setting numbers very, very low!
What you need is a piece of machinery that can cut the numbers. How? Well, let us think about an elementary business problem. You have 100 people in your building and 10 different merchants in your street. One might think that you will have 1000 different possibilities to build a database with the customer per merchant figures. Then you realize that your mother doesn’t go to the pub. In fact, from the 10 merchants, your mother goes to 2. And you only go to one. So, in reality, we are not speaking about 1000 possibilities; we are talking about a much lower number because this “underlying physics” throws only a percentage of the customers to a percentage of the merchants. This “underlying physics” is a way of cutting the numbers.
Back to the scenario, to cut numbers of the primary humanoid, you need a similar thing, except for the fact that there is no “underlying physics” to connect objects as you did with customers and merchants. You do it by using neurons. Neurons are a way to interconnect objects that appear in the scenario you have in front of you. You do what is called “distributed representation” in AI metalanguage, which interconnects things by common features, like color and form. Doing that, you project that infinite-dimensional universe that you would take gazillions of years to process into a very low dimensional space that you can quickly process. That is the entire purpose of a neural network, to cut the numbers of an infinite-dimensional scenario, projecting it into a very low dimensional space of common features of the original space. If we think that this is all designed by nature, we must agree that it is quite brilliant and cool. And I do not know if this is why we have a brain, but it sure looks like a very good reason. People from AI say that we are unveiling the hidden patterns of data while, indeed, we are putting it there.
If we think more analytically, we can project things like we start doing. When I said it is a 10x10 set, I built an X-Y reference frame that generates a flat Euclidean space. Moreover, the way how I described the problem in the first place is too much for a primary brain because it holds 2500 years of mathematics in it. Also, it carries two significant assumptions that are not that obvious. The first is that each of the rectangles is independent of the others, and so are the objects we put in them. The second is that all the rectangles are equal. These two assumptions hold this kind of geometry, which is the geometry that takes us to the big number of 5^100. And that is the geometry of the world around us and the geometry that supports Statistics, Calculus, Thermodynamics, Mechanics, among others, and all the Science and Engineering that we have built for the last centuries. It makes sense that it is so because we have built Science and Engineering to transform the world around us.
But what if the universe that surrounds us changes? Now you think that this guy is a moron... I mean, imagine a world where things, instead of being independent, were already interconnected. Instead of having a flat Euclidean geometry where we could divide the set into equal rectangles, would we have something completely different? We know that our brain was “built” to compress reality from infinite-dimensional space into a very low dimension. Still, if the world was made of objects already interconnected, that infinite-dimensional space does not exist! Like your mother and the pub, they do not interconnect because an “underlying physics” governs that universe. Our brain was made to cut the numbers of a geometrical flat space but, if the “underlying physics” interconnects the objects, the numbers are already cut! And they are cut because of what that set of mechanisms impose and not on the way it looks feasible to the set of neural networks that form our brain. And things get truly messy… Well, you can argue that everybody told you that building a neural network was a form of unveiling hidden patterns. But now the patterns are not “hidden”, are they?
Now you are thinking, “yeah, but the universe didn’t change, did it?!”. No, the only reason you see a flat geometry is because the world you see, and the primary humanoids saw, is in the same length scale as you are because, remember, you are part of that universe. You are a member of a club where everything looks flat and more or less of the same size. So, at the beginning of things, you wouldn’t need anything better. But your world is not like that in reality. The economic system is entirely interconnected; the social relations are completely interconnected; subatomic particles are completely interconnected; planets, stars, galaxies are all interconnected (“gravity”). Everything, except for those simple things primary humanoids saw, is interconnected, and as such, everything besides those simple things becomes a challenge when we speak about artificial or even natural intelligence. Because we will try to cut numbers that are already cut.
The reverse is also true. If we impose things to be in the same scale, what you come up with is a Euclidean well-behaved flat geometry where all the good math and physics work perfectly, all the random variables are Gaussian distributed, and correlations are measured as Pearson said. But you will be drowning in a sea of mental boredom that baths the shores of an illusional laboratory like universe and none of your problems will be solved.
So, our problem is a fundamental one. It does not matter how deep your learning is; how big your data is. Copying whatever the human brain is doing does not lead you anywhere. It is straightforward to show that whatever inflationary system we are dealing with - doesn’t matter if it is the universe of galaxies, subatomic particles, economic agents – that system will be interconnected and highly heterogeneous. Those systems cannot be learned, even if we try to assume them flat Euclidean geometries. And it is not only artificial intelligence that is on hold due to this problem. Many problems in science are on hold because they depend on the interconnection of objects, on a different geometry of the one over which we have built 25 centuries of math and physics.
What we can say for now is that it is not an IT problem. It is an analytics problem that is solved with paper, pencil and, especially, rubber.
Do you want to know more? Schedule a meeting with us here.
We will be glad to share our success stories with you.