Note: This article was originally published on Red Hat’s Enable Sysadmin blog, https://www.redhat.com/sysadmin
In parts one and two of this series, I walked you through hardening your system by identifying unneeded services, then segmenting and firewalling. Now that you’re all locked down, let’s talk about how we might validate that all of that hard work is actually providing the desired results. In this final installment, we’re going to talk about how to scan your network for open ports, and we’ll even touch on vulnerability scanning.
Attackers are constantly scanning your network. In one of my other articles, about locking down SSH, I stood up a system on Digital Ocean, with SSH open to the world, and waited for someone to attack. The goal there was to see how long it would take for the brute force attempts to start, and that was about an hour, but the first scan happened about 20 minutes in! 20 MINUTES! It reminds me of the old days when the internet was stricken by worms attacking Windows servers. If you connected an unpatched Windows machine to the internet, it would be infected in minutes. If the bad guys are scanning your network to see what you’ve got open for them to attack, shouldn’t you do the same thing? The answer is yes, but there are two different avenues. We’ll look at both approaches today.
Scanning for open ports
Your network is like a castle, a wall around the outside, and if you’ve segmented your network, doors and walls within that allow or deny access to different areas. There’s the outer wall, that the world sees, and then the inner doorways and passages that your users and servers see. The simplest wall would likely be the outer wall, so I’ll start there. If you have a public service, like your company’s website, it doesn’t do you much good if the internet can’t reach it, but of course you don’t want wide open access from the internet. That’s where firewalling comes in. The perimeter firewall, like the wall around a castle, should only allow the ports which are required for your internet-accessible services to work, it’s like opening the draw-bridge for things you’re expecting. We covered that in part two of this series. So you’ve configured your firewall, you’re good right? Maybe, but how do you know? You scan. Scanning your network is like having an inspector come in, and look at your outer wall, just to see if the draw-bridge opens for the right people, and not for others.
First let me point you at Shodan. Shodan is a service with both free and paid access, which is always scanning the entire internet, and maintains a database of all of the open ports it sees. It’s not exactly up to the minute, but if you put a service online, you can bet that sooner rather than later, Shodan will know about it. A good first stop would be to go to Shodan, and ask it what it knows about your network. I looked up the IP of my own website, here’s just a tiny snippet of the information it returned.
And it’s MUCH more thorough than that. It also gives you a decent amount of possible vulnerability information, but we’ll talk more about that later.
Shodan is great, and maybe it’s enough to do your external scan. You could also fire up a cloud instance, and scan your network using something like nmap. Nmap is a great free tool, included in many Linux distributions. You can even script it, and have it output in a friendly, easy to parse, format. A friend of mine is in the process of writing a cloud scanner, with nmap as its back-end, that he calls “DABS”. It is designed to run on cloud providers, and scan whatever your point it at, and log its results into a database. The beauty of this sort of system is, you can put agents at different vantage points in your network.
This allows you to also validate your network segmentation and firewalling. This is sort of like, having an inspector inside your castle, verifying that they can’t get into your treasury unless he has the right key. Do this by placing a scanner inside your various segments, and scanning the other segments from there. Place a scanning agent in, say, your web server network, and have it scan your database network. Check those results against your expected firewall rules, and make sure they match! You’d also do this from your various user networks. Can your desktops reach the proper services? Can they reach things they shouldn’t? Just because they’re your employees, doesn’t mean you can implicitly trust their systems. It’s like making sure the average citizens inside of your castle walls can’t walk into the king’s bedroom, or the armory, but can walk up to the fish merchant and buy dinner!
Once you’ve scanned from all of the various places you’d like to scan from, check the results against your expectations, fix anything that looks out of place, and scan again. Once you’re happy with the results, call that your baseline. A baseline is a known good configuration that you can compare against in the future. If you’re scanning regularly, you can compare future scans against your baseline, and flag any changes as something you should either look into, or add to your baseline. Did some new port you didn’t expect show up in your scan? That might be something to worry about.
Vulnerability scans
If you’d like to take it a step further, you should consider not just scanning for open ports, but using a vulnerability assessment tool. Vulnerability assessment tools actually look at the services they find on these open ports, and report out what sort of nasty things they might be susceptible to. To stick with our castle analogy, it’s like your inspector not just checking if he can reach the fish merchant, but actually verifying that the fish merchant sells fish! Tenable produces a for-pay vulnerability assessment tool called Nessus. A free tool that’s comparable is OpenVAS. OpenVAS, Nessus, and other vulnerability assessment tools will scan for open ports, and then actually look at the services running on those ports, try to figure out what they are, and report back on what they find. They’ll even do some low impact probing to see if these services are vulnerable to any known issues. Like your inspector trying to steal a fish from the fish merchant, to see if the guards stop him. This is usually just based on what version of a piece of software they detect, but they can get more advanced and actually try some automated exploit code. If you’d like, you can pay a professional to actually attack your services to see how far they can get, but that’s a little further than I’m going with this article.
Vulnerability assessment tools can also be used to get different viewpoints across your network, as I described earlier. Many of them are designed to be run in an automated fashion, so you can have a near real-time look at your network, and the state of your services. I wouldn’t call this a monitoring tool, but a piece of your infosec strategy.
Managing the complexity
You might be reading all of this, and thinking, “This is great, but when the heck would I have time to do all of this?”. We’re all busy as Sysadmins, and this is just one more thing on the plate right? I have just one response. Automation is key. Nmap can be automated, DABS is DESIGNED to be automated, OpenVAS and Nessus can be automated. As for remediation of the issues that these scans uncover? Just imagine an automated scan, finding some new problem, and firing off an ansible playbook to remediate the issue immediately. I can’t say that I know of a tool that does this just yet, but it sure would be a great next step.
Thanks for reading this series! I hope I’ve shared with you a decent path to improving your security posture, and protecting what’s important. Here are a few links to tools I’ve talked about in this article, and a great talk on baselining your network from DerbyCon 1.0.
Nmap – https://nmap.org
DABS – https://gitlab.godshell.com/Godshell/dabs
OpenVAS – https://www.openvas.org
Nessus – https://tennable.com
Shodan – https://shodan.io
And for a great talk on baselining your network from DerbyCon 1.0 check out: http://www.irongeek.com/i.php?page=videos/derbycon1/mick-douglas-blue-team-is-sexy-refocusing-on-defense-part-ii-all-you-baseline-are-belong-to-us