Security, Humans and Innovation Limitations

Security, Humans and Innovation Limitations

January 19, 2019 DATAcated Challenge 0

There are a few things worth noting, at a first sight:

  1. The security breaches
  2. Humans are involved
  3. Innovation is kinda off limits

The security:

The security of deep learning is clumsy and in a way it is its own paradox: how do you learn everything and at the same time protect yourself from learning.

There has been plenty of work done in the field of black-box model hacking, and I will not go in details here. I will, however, give a simple example: many public parking garages at malls around the world rely on license plate image recognition for timekeeping and charging fees. It is incredibly easy to create a havoc in this environment – all you need is a bit of time, some dirt or a marker pen and place some innocent dots around the license plates of some cars. All of a sudden, to the poor machine 5s will look like 6s, 3s will look like 8s and so on, even though to a human eye there won’t be too much wrong with the picture.

Furthermore, if you have a model which distinguishes between cats and dogs, it is just matter of time and strategy before an attacker convinces the model that cats are dogs and dogs are cats.

The human factor:

If humans are involved in the process, this means that the process is inevitably susceptible to the same cognitive biases. If a model is created by humans for humans, there is always a risk that humans can be targeting humans to believe what they want to believe. This is a big challenge right now, especially in decision-making, politics, economy and so on.


In short, you may have a reinforcement learning model doing all kinds of amazing work and finding outliers. However, it will almost never spot the malicious outlier, which is a super winner if put in a different context. Think of antibiotics – you can have a model detect all kinds of mould , but it will never think of using the mould for antibiotics.

All of the above are great challenges and they have no answer. And these are just some of the simplest limitations. Think of quantum layer, think of mathematics and think of optimization problems… But this is a topic for a book, I guess. Anyone interested?!

By: Feodor Georgiev


Leave a Reply

Your email address will not be published. Required fields are marked *