Click here to get back home

Hacked 2003 SBS Server - temp fix required

 HomeNewsGroups | Search | About
 microsoft.public.windows.server.security    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Hacked 2003 SBS Server - temp fix required Paul Hadfield 04-13-2008
Posted by Paul Hadfield on April 13, 2008, 2:35 pm
Please log in for more thread options
All,

I'm trying to get thoughts and ideas on how to clean an infected/hacked
server as a temp measure to delay actually formatting and rebuilding it:

A customer has had their 2003 SBS Prem SP2 server hacked into (Exch 2003
SP2, SQL 2000 SP4) - it seems someone decided it was a good idea to open up
TCP:1433 on the NAT firewall to allow incoming traffic from anywhere, so
that they could access the DB from home - and the sa password was blank!!
Less than a day later and the server was on it's knees.

Symptoms included many new services installed taking up much CPU time,
'interesting' traffic going out onto the Internet and quite a few new
accounts being created in AD with random names each ending with a dollar
symbol - all with Administrator rights!!!

I've strongly suggested to the customer that they immediately back up all
data and format the server to start again (it's their only server) - but
they want to wait until the end of next month as this is their peak trading
time and don't want to risk any downtime of the server. My arguments that
the server probably won't even last till the end of tomorrow fail to
persuade them.

So I'm on a mission of damage limitation to get as much reliability from the
server as possible for a month until they will let me reformat - and would
appreciate any advice. So far I have taken the following steps:

1) Closed TCP:1433 on the NAT firewall - only local LAN PCs have access to
SQL
2) Set a secure sa password
3) Stopped the SQL service (and agent service) from running under system -
they now run as a user with run as service rights.
4) Used MSconfig to stop all non MS services and all start-up programs,
rebooted and run full AV scan (Symantec Corporate 10.1), and also run the
latest MS malicious software removal tool
5) Manually removed all non-MS services that look like they should not be
there - only genuine Symantec, Adaptec, Dell and a few other trusted
services remain.
6) Booted from SBS 2003 CD and used fixmbr command to write clean boot
information (just in case)
7) Booted up normally with all services back to normal, and run another Av
scan and MS malicious software removal tool (all full scans)
8) Ran MBSA against the server
9) Performed similar scans/checks on all desktop client PCs on the domain
(about 13 Win XP pro machines)

The MS malicious software removal tool found quite a few back-door programs
installed on the server and claimed to have cleaned them all. Nothing
discovered on the client PCs


After all this, the server ran fine for about a week, then a few more random
user accounts were created with Administrator rights, and tools like eBay
turbo lister were installed and running from a disconnected TS session using
one of the new admin accounts!!!


Does anyone have any suggestions as to what steps I can take next to prolong
the life of this pillaged, tortured and humiliated little server? I'm
considering the following:

1) Remove the xp_cmdshell from SQL server - I don't think the app they have
makes use of this.
2) Install 3rd party addware/spyware removal tools like AdWare from Lavasoft
and scan the server with those. Not sure I trust desktop/home user software
like these on a server - but I'm running out of ideas
3) Remove and re-install SP2?


As a side note, there are two strange bits of behavior I have noticed from
the server:

a) Automatic updates no longer work. The service is configured to
auto-download and prompt for manual installation - but it seems not to
download anything. When you visit the website, IE6 either displays an empty
white page, or crashes.
b) I'm suspecting that some of the files in system32 have been replaced with
infected files. If I open the folder and display all files sorted by last
modified date, the net.exe command has been modified very recently, and when
you click 'properties' over the file, the description tells you that it's
actually the MS Pint command. Indeed, if you open a command prompt and type
'net' you get the ping command. If I try to delete the net.exe command to
replace it from another clean server, the OS seems to think I've tampered
with the file, and automatically un-does what I have done, and put the ping
command back again! Very strange. I know windows server 2003 protects
certain key files in system32 to protect against deletion etc - could it be
that where ever it gets its source files from is corrupt/hacked??


Thanks very much in advabce for any comments - though I'm quite sure you
think we're mad even going down this route on a live production server - the
sooner I get to format it the better!!

Cheers,
Paul.



Posted by Al Dunbar on April 13, 2008, 5:50 pm
Please log in for more thread options
Your strong suggestion seems likely to have been the best option. I hope
that you have a letter from your client to the effect that you are not to be
held accountable because your damage limitation mission was insufficient to
keep them from going out of business because some hacker somewhere owns
their server. In fact, you have already proven the d.l.m. was insufficient,
otherwise new admin accounts would not have been created. If they are going
to allow you to continue applying bandaids, they need to be advised that it
is in the nature of "best effort". As it is, it seems to me you will succeed
only if you reverse engineer all of the possible threats to figure out how
to neutralize them, and that is a job that a number of anti-malware
companies are spending lots of resources on. Good luck.

/Al

> All,
>
> I'm trying to get thoughts and ideas on how to clean an infected/hacked
> server as a temp measure to delay actually formatting and rebuilding it:
>
> A customer has had their 2003 SBS Prem SP2 server hacked into (Exch 2003
> SP2, SQL 2000 SP4) - it seems someone decided it was a good idea to open
> up TCP:1433 on the NAT firewall to allow incoming traffic from anywhere,
> so that they could access the DB from home - and the sa password was
> blank!! Less than a day later and the server was on it's knees.
>
> Symptoms included many new services installed taking up much CPU time,
> 'interesting' traffic going out onto the Internet and quite a few new
> accounts being created in AD with random names each ending with a dollar
> symbol - all with Administrator rights!!!
>
> I've strongly suggested to the customer that they immediately back up all
> data and format the server to start again (it's their only server) - but
> they want to wait until the end of next month as this is their peak
> trading time and don't want to risk any downtime of the server. My
> arguments that the server probably won't even last till the end of
> tomorrow fail to persuade them.
>
> So I'm on a mission of damage limitation to get as much reliability from
> the server as possible for a month until they will let me reformat - and
> would appreciate any advice. So far I have taken the following steps:
>
> 1) Closed TCP:1433 on the NAT firewall - only local LAN PCs have access to
> SQL
> 2) Set a secure sa password
> 3) Stopped the SQL service (and agent service) from running under system -
> they now run as a user with run as service rights.
> 4) Used MSconfig to stop all non MS services and all start-up programs,
> rebooted and run full AV scan (Symantec Corporate 10.1), and also run the
> latest MS malicious software removal tool
> 5) Manually removed all non-MS services that look like they should not be
> there - only genuine Symantec, Adaptec, Dell and a few other trusted
> services remain.
> 6) Booted from SBS 2003 CD and used fixmbr command to write clean boot
> information (just in case)
> 7) Booted up normally with all services back to normal, and run another Av
> scan and MS malicious software removal tool (all full scans)
> 8) Ran MBSA against the server
> 9) Performed similar scans/checks on all desktop client PCs on the domain
> (about 13 Win XP pro machines)
>
> The MS malicious software removal tool found quite a few back-door
> programs installed on the server and claimed to have cleaned them all.
> Nothing discovered on the client PCs
>
>
> After all this, the server ran fine for about a week, then a few more
> random user accounts were created with Administrator rights, and tools
> like eBay turbo lister were installed and running from a disconnected TS
> session using one of the new admin accounts!!!
>
>
> Does anyone have any suggestions as to what steps I can take next to
> prolong the life of this pillaged, tortured and humiliated little server?
> I'm considering the following:
>
> 1) Remove the xp_cmdshell from SQL server - I don't think the app they
> have makes use of this.
> 2) Install 3rd party addware/spyware removal tools like AdWare from
> Lavasoft and scan the server with those. Not sure I trust desktop/home
> user software like these on a server - but I'm running out of ideas
> 3) Remove and re-install SP2?
>
>
> As a side note, there are two strange bits of behavior I have noticed from
> the server:
>
> a) Automatic updates no longer work. The service is configured to
> auto-download and prompt for manual installation - but it seems not to
> download anything. When you visit the website, IE6 either displays an
> empty white page, or crashes.
> b) I'm suspecting that some of the files in system32 have been replaced
> with infected files. If I open the folder and display all files sorted by
> last modified date, the net.exe command has been modified very recently,
> and when you click 'properties' over the file, the description tells you
> that it's actually the MS Pint command. Indeed, if you open a command
> prompt and type 'net' you get the ping command. If I try to delete the
> net.exe command to replace it from another clean server, the OS seems to
> think I've tampered with the file, and automatically un-does what I have
> done, and put the ping command back again! Very strange. I know windows
> server 2003 protects certain key files in system32 to protect against
> deletion etc - could it be that where ever it gets its source files from
> is corrupt/hacked??
>
>
> Thanks very much in advabce for any comments - though I'm quite sure you
> think we're mad even going down this route on a live production server -
> the sooner I get to format it the better!!
>
> Cheers,
> Paul.
>
>



Posted by Leythos on April 14, 2008, 6:40 am
Please log in for more thread options
> All,
>
> I'm trying to get thoughts and ideas on how to clean an infected/hacked
> server as a temp measure to delay actually formatting and rebuilding it:
>
> A customer has had their 2003 SBS Prem SP2 server hacked into (Exch 2003
> SP2, SQL 2000 SP4) - it seems someone decided it was a good idea to open up
> TCP:1433 on the NAT firewall to allow incoming traffic from anywhere, so
> that they could access the DB from home - and the sa password was blank!!
> Less than a day later and the server was on it's knees.

Stop, don't even consider trying to remove the malware and others.

The first thing you need to do is block ALL INBOUND AND OUTBOUND.

The Second thing you need to do is BACKUP DATA.

The Third thing you need to do is WIPE the system, rebuild from scratch,
meaning format and reinstall from CD.

Install a quality Server based AV solution - Symantec Corp Ed 10.1.7 or
End Point Protection would be my preference.

Backup OS/Base config

Load apps from CD/Media, not tape/backup or anywhere that was saved from
compromised computer/network.

Restore data, not apps, from backup, do not restore EXE/COM/BAT file
types.

Scan for malware using AV, Spy Bot Search and Destroy and AdAware

Enable on the the following ports inbound:

SMTP (TCP 25)
RWW (TCP 4125)
HTTPS (TCP 443)

Block outbound ports tcp 135-139, 455, 1433, 1434

Shoot person that put hold for DB into NAT device.

Teach them how to use RWW to access their computers remotely.

There is nothing you can run, nothing you can do, to ensure that the
server is 100% clean. Yes, people with large EGO's will tell you that
you can clean it, that you don't need to wipe it, but anyone that would
certify, with liability included, will tell you that they won't clean
it, that they would wipe/reinstall in a CLEAN environment.

With SA access they had complete access to the OS, to browse their
network, to do anything they wanted as a domain administrator.

--
- Igitur qui desiderat pacem, praeparet bellum.
- Calling an illegal alien an "undocumented worker" is like calling a
drug dealer an "unlicensed pharmacist"
spam999free@rrohio.com (remove 999 for proper email address)

Posted by kj [MVP SBS] on April 18, 2008, 4:34 pm
Please log in for more thread options

>> All,
>>
>> I'm trying to get thoughts and ideas on how to clean an infected/hacked
>> server as a temp measure to delay actually formatting and rebuilding it:
>>
>> A customer has had their 2003 SBS Prem SP2 server hacked into (Exch 2003
>> SP2, SQL 2000 SP4) - it seems someone decided it was a good idea to open
>> up
>> TCP:1433 on the NAT firewall to allow incoming traffic from anywhere, so
>> that they could access the DB from home - and the sa password was blank!!
>> Less than a day later and the server was on it's knees.
>

Agreed to all of Leythos recommendations, plus since this is Premium, use
the included edition of ISA 2004 installation if a proper firewall doesn't
exist otherwise.



> Stop, don't even consider trying to remove the malware and others.
>
> The first thing you need to do is block ALL INBOUND AND OUTBOUND.
>
> The Second thing you need to do is BACKUP DATA.
>
> The Third thing you need to do is WIPE the system, rebuild from scratch,
> meaning format and reinstall from CD.
>
> Install a quality Server based AV solution - Symantec Corp Ed 10.1.7 or
> End Point Protection would be my preference.
>
> Backup OS/Base config
>
> Load apps from CD/Media, not tape/backup or anywhere that was saved from
> compromised computer/network.
>
> Restore data, not apps, from backup, do not restore EXE/COM/BAT file
> types.
>
> Scan for malware using AV, Spy Bot Search and Destroy and AdAware
>
> Enable on the the following ports inbound:
>
> SMTP (TCP 25)
> RWW (TCP 4125)
> HTTPS (TCP 443)
>
> Block outbound ports tcp 135-139, 455, 1433, 1434
>
> Shoot person that put hold for DB into NAT device.
>
> Teach them how to use RWW to access their computers remotely.
>
> There is nothing you can run, nothing you can do, to ensure that the
> server is 100% clean. Yes, people with large EGO's will tell you that
> you can clean it, that you don't need to wipe it, but anyone that would
> certify, with liability included, will tell you that they won't clean
> it, that they would wipe/reinstall in a CLEAN environment.
>
> With SA access they had complete access to the OS, to browse their
> network, to do anything they wanted as a domain administrator.
>
> --
> - Igitur qui desiderat pacem, praeparet bellum.
> - Calling an illegal alien an "undocumented worker" is like calling a
> drug dealer an "unlicensed pharmacist"
> spam999free@rrohio.com (remove 999 for proper email address)



Similar ThreadsPosted
Server has been hacked, need to delete hidden user account May 25, 2007, 5:44 am
MS05-046 required? October 17, 2005, 4:32 am
Domain name required? April 12, 2007, 6:56 pm
Rights required to allow changing directory ownership? August 30, 2005, 3:37 pm
Permissions required for the Cluster service account? July 7, 2006, 6:51 am
Is third-party middleware required when deploying smartcards? October 1, 2007, 12:02 pm
Authentication required when mapping a drive from a Terminal Serve October 31, 2006, 10:34 am
Determine if hacked May 1, 2006, 1:15 pm
Been hacked about 4 times now. Wanna be the 5th? June 2, 2006, 8:59 pm
solution required; Someone accidentally or intentionally deleted some files and folders from network resource. June 19, 2007, 4:47 am

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap