One thing that I have learned in my life is that everyone lies – to an extent. That even means people can make up little white lies to protect peoples' feelings or hide the absolute truth. Some of the lies can be acts of commission – a deliberate statement of untruth – whereas others are lies of omission.
In the latter case, some are telling a true fact, but they are leaving out very important points, which can also result in a different understanding than what they are wanting to tell people.
A few years ago when I was developing a web application that focuses on security first – basically security above everything else because we could never trust people to not mess anything up. Unlike many of the clients who I worked with this one goes to claim that their project is unhackable.
One way that they proved this was to submit for a security review to an independent – pretty well known and respected – company. The project received a clean bill of health at the end of the day. There was no programming errors nor any critical vulnerabilities that could compromise the project. One member of the management team gloated with joy he told several group of people that this was a fact and not just a half truth.
I asked if he would share the detailed review report with me. He willingly said yes. As I knew what was going to happen, he just sent me the one-page summary page – which did say that the project was reviewed and found to practically be bug free.
Most people would take these results to mean that the project was found to be bug free and completely unhackable. This could not be more farther than the truth than anyone could imagine who works in programming and cybersecurity industries. The management member of this company didn't know it but I was pretty good friends with a few of the people who worked for the firm that reviewed the source code for him. I'd learn an ugly secret from this experience.
The company who owns the project lies. It's more of an act of omission than commission in this case.
What cybersecurity firms don't tell a lot of people is that when you submit a project for bug and security auditing is that the project will go under two reviews.
The first review will find all of the bugs, exploits, vulnerabilities, and anything that could go wrong with the source code of the project. Then the source code is submitted again for review – where the reviewing firm would reevaluate and retest the same items again – declaring the project to be bug, exploit, and vulnerability free.
At the end of this process, the company which has the source code can tell potential customers that the project is unhackable. The companies know that they will almost always end up with a clean bill of health from a independent cybersecurity firm that audits peoples' source codes.
Here's what I need people to understand and always remember from this. No software is one-hundred percent bug, exploit, or vulnerability free – no matter what any report or anyone may ever tell you it could be for numerous reasons.
First, the intent of most security reviews of this type is to end up with a public letter saying the reviewed product is flawless. If that’s the intent before the contract is signed, how can there be a different outcome? Needless to say, it changes how intently security review companies look for bugs.
Second, no single code reviewer or hacker team ever finds every bug. They find every bug they've been trained to find by their tools and experience in the amount of time they've been given. Add more teams (for experience, skills, and tools) and in time, you'll find more bugs. That's 100 percent guaranteed.
Third, when you're on a security review team, you normally find hundreds to thousands of bugs -- often the same bug repeated over and over. But in your review, once you've found enough bugs to fill up hundreds of pages of a report and "earned your money,” where's incentive to find more bugs? At some point, you feel like you've done your job.
Fourth, and most important, the real test of any product occurs when it goes mainstream. Your product can have hundreds of thousands, even a million users, but its past security record doesn't mean a thing until it’s installed on many millions of computers.
When a product goes mainstream, hundreds or thousands of unwanted code reviewers and product testers start pounding on it. They’ll find the bugs that others did not find -- and if the vendor is unlucky, they’ll use that evidence to scare away customers.
Tell me another one
When a vendor tells me its product is unhackable, I immediately think: Are you clueless or lying to me? My respect goes way down. I starting wondering what else they’re lying about.
To impress me, a vendor needs to present its product to a reputable, experienced security review company -- once -- and let me see the detailed report. The "all clear" second review is merely a lie of omission.
This is not to say that security reviews conducted by trusted, experienced companies are worthless. On the contrary, they discover security bugs and give you a chance to fix them. Your product is more secure than it was before. But that reality is a far cry from saying a product can't be hacked.
If a vendor really wants to impress me, it should do as many major software vendors do and run "bug bounty" contests where anyone can participate, with scheduled professional reviews from a respected company. That's the best of both worlds.