The Case of the Brute Force attack – and how NetScaler helped!

During a recent visit to one of our clients we received complaints about random account lockouts throughout the organization. People were frequently locked for unknown reason. I was asked to help investigate this issue.

We started by enabling audit logging in their Default Domain Controller policy. We’ve enabled these settings:

• Audit Policy : Audit account logon events (succes/failure)
• Advanced Audit Configuration-Logon/Logoff : Audit Account Lockout

Next we applied a filter to the Domain Controllers security logfiles. EventID 4740 will show when a lockout has happened. It didn’t take long before we actually saw the machine responsible. By examining the event you can determine the Caller Computer Name. That will be the machine on which you will have to focus.

Now we knew it was one of the Microsoft AD FS servers locking out the users. That raised the obvious questions like: why? and how? Fortunately AD FS can provide you with detailed logging which first needs to be enabled. In the Local Group Policy Editor enable Audit Application Generated (Succes and Failures). This will return a world of information.

And sure enough, after waiting a while we noticed lockouts again. We also noticed something we didn’t expect. In the eventlog we first noticed the URL that was accessed during the lockout. But we also noticed the location it was accessed from. We noticed IP’s from countries all over the world. Brute Force attack in progress!

Now we knew what was happening we decided to leverage Citrix ADC (NetScaler) to help mitigate. Denying traffic on Geo IP Location would certainly stop this attack. I started by creating 2 String Maps with the following purpose:

• SM_GEOIP_IP_WHITELIST – Purpose : Whitelist IP’s that will not be blocked
• SM_GEOIP_URL_WHITELIST – Purpose : List URLs that will be excluded from Geo IP blocking

Next we imported the builtin GeoIP Location database already present on NetScaler.

After importing the database we need to set the locationParameter with the matchWilcardto any parameter. Do so with the following command:

With show locationparameter the functionality of the GeoIP Database can be tested, as with nsmap -d -t and entering an IP address. The last command will actually show the notation we need further ahead.

Then an Advanced Policy Expression was created to actually make the IP whitelist function:

And another one to define the countries that we would like to allow:

Finally we need a Responder policy that will return some useful information about why a user is blocked. If you prefer you can ofcourse just drop traffic. It would also be nice if the Responder Policy did some logging in the ns.log when it actually kicks in. You can then easily search your NetScaler logfiles if the Geo IP block did it’s work. To make it work first you need to enable User Configurable Log Messages in both the Syslog as nslog parameters.

After that a Audit Message Action can be created that can later be bound in the Responder Policy:

Now it’s time to create the Responder Action and Policy

And to make it all work bind the Responder Policy to the desired Load Balancing vServer. In my case I bound it to the Content Switching vServer so all resources would be protected.

After a while the Brute Force Attack stopped and we haven’t seen it again.