Tracking Software Installation and Removal Using Event IDs 11707, 11724, and 592

In these days of malware, spyware, and compliance regulations, a lot of admins are looking to track the installation of unauthorized programs, and/or the removal of required programs from client desktops. There are actually several events you can look for in both the Application Event Log and Security Event Log that will help you do this.

In the Application log, setup packages that use the Windows Installer to install themselves will create numerous events, all with an event source of MsiInstaller.

Event ID 11707 tells you when a install completes successfully, and also the user who executed the install package.

Event Type: Information
Event Source: MsiInstaller
Event Category: None
Event ID: 11707
Date: 11/9/2006
Time: 3:21:45 PM
User: DOMAIN\USER
Computer: COMPUTERNAME
Description:
Product: Event Archiver Enterprise -- Installation operation completed successfully.

Event ID 11724 tells you when a software package is removed successfully, again logging the user behind the operation.

Event Type: Information
Event Source: MsiInstaller
Event Category: None
Event ID: 11724
Date: 11/12/2007
Time: 7:50:13 PM
User: DOMAIN\USER
Computer: COMPUTERNAME
Description:
Product: Event Archiver Enterprise -- Removal completed successfully.

You can track both of these events in our Event Analyst software by setting up appropriate filters and building a custom report.

Also, if you want to correlate the name of the executable setup package that was executed to install a piece of software, turn on Process Tracking auditing on the relevant Group Policy Object for one or more computers (e.g. Domain Security Policy, Local Security Policy), and look for events with Event ID 592 in the Security log that occur around the time of the 11707 event in the Application log, e.g.

Event Type: Success Audit
Event Source: Security
Event Category: Detailed Tracking
Event ID: 592
Date: 11/9/2006
Time: 3:20:30 PM
User: DOMAIN\USER
Computer: COMPUTERNAME
Description:
A new process has been created:
New Process ID: 2816
Image File Name: \EvntArch.exe
Creator Process ID: 516
User Name: USER
Domain: DOMAIN
Logon ID: (0x0,0x3E7)

Event Analyst also has a built-in Process Usage report that is very useful for viewing all of the executable files that were loaded and unloaded on one or more systems for a given time frame. It automatically determines the executable files that are run the most frequently for any given user.

Free Software Offer For Early Vista/EVTX Log Format Adopters

As promised in our previous post on Event Analyst's full support for working with EVT and EVTX log files natively on Windows Vista™, we are making a special free software offer available to admins that wish to use Vista as their OS platform for log management.

Here are the details of the offer, directly from our sales division:

Do you already have some Windows Vista machines generating EVTX logs? Great. We'd like to give you some software. That's right. At no charge. We're offering 5 server license packs of Event Archiver™ and Event Analyst™ bundled together. Basic email-based support is included with all licenses. If you wish to pick up an upgrade service or another of our more advanced support options, we can arrange for the purchase. Interested? Simply request more details at
http://www.doriansoft.com/evtxsoftwareoffer.

As you can gather, this is a fantastic promotion, as it allows you to gather event log data from both your non-Vista and Vista systems and report on that data by running Event Archiver and Event Analyst on a Microsoft Vista workstation. We're convinced that once you see the power of Dorian's LogRefiner™ technology in action, you'll be much more comfortable in putting forth a plan for log management for your larger migration to Microsoft Windows Vista and Windows Server 2008™. As we've stated numerous times before, our exclusive LogRefiner technology is here and ready for you whenever that migration begins.

Event Analyst Works With EVT and EVTX Files, Side-By-Side!

Greetings, gentle readers. It's been a while since our last blog post, but that's because we've been slaving on the Version 6 release of Event Analyst. And what a fantastic release it is!

Just like our Event Archiver release of a few months ago, this version of Event Analyst is completely Microsoft Vista™ compatible, and features our revolutionary LogRefiner™ technology. You can download it here: http://www.doriansoft.com/download.

We've already mentioned in a bunch of posts that trying to read saved, legacy EVT files on Windows Vista is quite a chore, with missing fields and information being quite common. In fact, a recent blog posting from the Performance Team at Microsoft shows you how to perform a whole bunch of contortions in an attempt to convert an EVT file to an EVTX file, with of course there being no guarantee that the converted log will parse properly when you attempt to read it.

Well here's the good news. Thanks to our pioneering LogRefiner™ technology, you can work with EVT and EVTX files natively and side-by-side when Event Analyst is installed to a Microsoft Vista computer. No weird conversions or intermediate steps are necessary, and you get all the data parsed correctly from both log formats the first time. For those admins who are attempting to run Windows Vista on their workstations, this is a big plus because now you can use Event Analyst as your preferred log reader/analysis tool/reporting tool for all of your systems and your saved EVT log files. You no longer need to convert EVT files or juggle both the Microsoft Classic Event Viewer and the new Vista Event Viewer when switching back and forth between EVT and EVTX files.

Here's a screenshot of both an EVT and EVTX log being viewed within Event Analyst 6 at the same time:




Again, bear in mind that this technology lets you work with active AND saved EVT files from your older operating systems all natively inside Vista. It's very cool stuff.

We'll have more information for you on this technology soon, including a very nice licensing promotion, so please stay tuned.

New EVTX Log Format Whitepaper Released

Earlier this week, we released a new whitepaper that outlines many of the key changes in the new EVTX event log format found in both Microsoft Vista and Windows Server 2008. We've included some choice pull quotes below, but you can download the full version by registering here:

http://www.doriansoft.com/evtx

Beginning with Microsoft® Windows Vista™ and Windows Server® 2008, Microsoft has completely redesigned its event log format. This new EVTX file format stores event log records as a stream of binary XML records. Accessing data in the new EVTX files requires the use of a new application programming interface that is not available in older Windows operating systems. In addition, the number of, structure of, and data within the fields in the EVTX log records has changed significantly.

Because the new Windows Event Log API functions are only available inside Windows Vista and later operating systems, legacy Windows operating systems like XP and 2003 cannot read previously saved EVTX files at all - there is simply no forward compatibility for consuming saved EVTX files. And while the legacy Event Log API can be used to read some of the events from an "active" EVTX file (that is to say an EVTX file currently being maintained by the EventLog service on a Vista machine), it cannot properly read and parse some events recorded by the new API.

In summary, both forward compatibility to EVTX files from legacy Windows operating systems and backward compatibility to EVT files are severely hampered, if available at all. As a result, organizations that rely on their own scripts and automation techniques may be tempted to develop two different systems for log management - one supporting legacy EVT files on legacy operating systems, and another supporting EVTX files on Windows Vista and Windows Server 2008. Such a strategy has the potential to decentralize log collection and reporting, as well as substantially increase costs over time.

Again, to read the full version, please register here:

http://www.doriansoft.com/evtx

The AUXSOURCE Switch

In a previous post, we discussed how EVT files contain references to other message files, which are parsed together with the EVT data to produce readable events. Typically, when you relocate EVT files away from the network where they were generated, you cannot completely parse all of the data fields properly. However, there is a very useful yet undocumented/underdocumented command-line switch you can use with the Microsoft Event Viewer to force it to look elsewhere for critical message files. That command line switch is /AUXSOURCE

For example, if you had a security log that originated from a Windows® 2003 server, but you were not currently connected to the network where that log came from, you could use the /AUXSOURCE switch to load message data from a Windows 2003 server that was on your local network instead. The command-line syntax would look like this:

mmc /a c:\windows\system32\eventvwr.msc /auxsource=REFERENCECOMPUTER

where REFERENCECOMPUTER is the network name or IP address of the computer that will act as the lookup computer for message file resolution.

Once you load the Event Viewer with the AUXSOURCE flag, you can then open up your saved EVT file, and the Event Viewer will always use the REFERENCECOMPUTER for message file data when it attempts to parse events from the saved log.

There are some caveats with this approach that are listed below:

1.) The AUXSOURCE switch is only available for use on Windows XP and Windows 2003 versions of the Event Viewer, not Windows 2000 versions.
2.) AUXSOURCE will not help you properly view saved DNS Server, Directory Service, or File Replication Service logs from a Windows XP workstation or Windows 2003 member server, even if you point the REFERENCECOMPUTER to a domain controller. Instead, you have to be logged on to a Domain Controller to view these saved files.
3.) If you use AUXSOURCE with Application or System logs, you may still get incomplete Description fields, because chances are the REFERENCECOMPUTER will not have all the same software and hardware installed as the machine where the EVT file came from.

Fortunately, we have decided to provide functionality that exceeds what the /AUXSOURCE switch can do in the upcoming release of Event Analyst. The new version of Event Analyst will allow you to use any Windows machine available on the network (e.g. Windows NT, Windows 2000, Windows XP, Windows 2003) as a reference computer for message files for saved EVT files. No minimum OS platform is required for this functionality - Event Analyst can be installed on Windows NT 4.0, Windows 2000, Windows XP, Windows 2003, etc.

In Theory And In Reality

I will build a car for the great multitude. It will be large enough for the family, but small enough for the individual to run and care for. It will be constructed of the best materials, by the best men to be hired, after the simplest designs that modern engineering can devise. But it will be low in price that no man making a good salary will be unable to own one-and enjoy with his family the blessing of hours of pleasure in God's great open spaces."

-- Henry Ford

"The greatest improvement in the productive powers of labour, and the greater part of the skill, dexterity and judgement with which it is any where directed, or applied, seem to have been the effects of the division of labour."

-- Adam Smith, The Wealth Of Nations

Not surprisingly, our last post on the perils of "One Size Fits All" log management got a heated response from a blogger whose company tilts at the windmills of "mega-SEM" log management. We were called "profoundly stupid," "naive," "incompetent," and "idiotic." We were happy to receive such high praise for our company, which has been producing software in the log management niche since 1997, over twice as long as many of the johnny-come-latelies into the market. Obviously, we're doing something terribly wrong over here :)

Interestingly, the meat of our post, namely that you can put together a good log management system by combining best-of-breed packages that target different types of logs, was not rebutted. Arguably, it is pretty easy to pull some quotes from a blog posting without actually debating the core philosophy or issue. However, we're not really interested in debating this issue, because it would probably devolve into some sort of academic exercise with plenty of jargon and buzzwords that probably don't mean a hill of beans to you, our gentle readers.

One of the most interesting things in the software industry is the disconnect between the "wouldn't it be awesome if?..." theory and the ugly reality of the marketplace. Nowhere is this more painfully obvious than in the area of SEM and log management. In that spirit, and in the spirit of when academia meets reality, we're going to flesh out our previous blog posting into a little thing we call "In Theory and In Reality."

In theory, every possible device, operating system, or program that generates a log would adhere to a common schema or format when doing so. It seems that every year some new working group releases a paper or proposal detailing that very thing.

In reality, only some devices, operating systems, and programs that generate logs adhere to a common format. Cynically or not, vendors of said devices, operating systems, and programs have discovered that there is money to be made selling consulting services and reporting packages for logs written in their proprietary formats. Some of the most popular logging formats, such as Windows EVT files, syslog, and the W3 logging format have gotten that way due to widespread industry adoption and market penetration, not the other way around. On top of that, even if a log is written in a common format, the devil is in the details of the event!

In theory, every organization looking to automate log management has a budget for that project in excess of $50K, or maybe even $100K. On top of that, they obviously would want a log management package that claims to manage hundreds of devices, even though they only have 5 Windows servers, 100 Windows workstations, a UNIX mail server, and a router/firewall on their network.

In reality, many of the admins we work with daily are lucky if their management has blessed them with $5K to spend on log management, never mind $50K.

In theory, most organizations want a large, macro view of logging activity and trends happening across their network. Highly detailed information and reporting would be nice to have, but the big picture is fine for right now.

In reality, if organizations cannot produce detailed, OS/device-specific levels of information for their auditors, they fail audits.

In theory, IT departments are well-staffed with highly-compensated admins who have plenty of free time to spend on extensive consulting and training for the log management packages they adopt. Really!

In reality, IT departments are often poorly-staffed with admins forced into reactive, as opposed to proactive, positions. They need easy-to-configure software that can produce detailed levels of information quickly and without much fuss.

In theory, only expensive, over-engineered SEM packages can produce any useful level of correlation between different devices and operating systems.

In reality, many device and platform-specific SEM packages for SMBs can output aggregated log data into mineable formats such as database tables, or pass that data over the fence to another logging platform (e.g. syslog concentrator, etc), where data can be routinely grepped and mined as needed for key IP addresses, ports, etc.

In theory, all large Fortune 500 companies and huge government entities would naturally want to adopt a mega-SEM package, because it's the only thing that can even come close to dealing with their diverse, heterogenous logging environment.

In reality, many large Fortune 500 companies purchase specific device and platform-targeted log management packages to get a detailed handle on logging data within a certain department. Often, this is after they've been sold a bill of goods by the mega-SEM vendor and they're facing the crunch time of an audit.

We now conclude this chapter of "In Theory and In Reality." We'll soon take you back to your regularly scheduled programming.

The Perils of "One Size Fits All" SEM and Log Management Packages

"Smokey my friend, you're entering a world of pain."
-Walter Sobchak, The Big Lebowski

"A Jack of all trades is a master of none"

Today's post is going be a little outside the technical realm of log management, but is an important post nonetheless.

Often, we receive RFPs (requests for proposal) from companies wanting us to run through a "supports/does not support" checklist of log generating devices. It seems that upper management loves to approach enterprise log management as a quest for the one holy grail product that can manage logs from hundreds of different devices and operating systems, in addition to folding the laundry and making coffee.

This approach to procuring log management technology is fatally flawed from the outset.

The thousands of log generating devices and operating systems in today's marketplace truly and completely prevents any vendor from being a polymath at all of them. Some vendors may try to lay claim to supporting tens, or even a hundred of said devices, but often the reality is empty marketing rhetoric without the robust technology present to deliver on the claims.

For example, the level of nuance and detail in the Microsoft Windows ® event log alone is enough to keep a substantially sized development team busy all the time. We can testify to this, as the Microsoft Windows event log is our area of expertise. Multiply this level of nuance and detail by a factor of hundred, or even a thousand, and you have an untenable goal for even the largest of software corporations.

Moreover, value gets diluted very quickly when you start looking at the price tag of "one size fits all" log management packages, especially when compared to picking up a handful of best-of-breed tools that specialize in log management for specific operating systems or devices. Take a hard look at the reporting in one of those mega-SEM packages and see if that "value dilution" is not readily apparent. 10 to 20 log generating devices may be "supported", but reporting will often be limited to a handful of reports per device.

To play devil's advocate for a minute, let's assume that one of these mega-SEM vendors has a very diligent, hard working development team that cranks out new reports as often as possible. What happens when the way an event gets logged on a particular OS changes or a new service pack is applied? Whoops! Back to the drawing board. Patch, patch, patch and fix all of those previously "finished" reports. As the number of reports increases, each new logging change that happens after an OS upgrade or device firmware patch increases that mega-SEM vendor's work by an order of magnitude. Eventually, entropy will take over, making quality suffer while updates are issued in a less timely fashion. It's a battle that cannot be won, even with the best development efforts and the most earnest intentions.

It's tempting for CIOs and CTOs to buy into the mega-SEM hype - the fantasy of having the logs of hundreds of different devices and computers all neatly aggregated with hundreds of ready-to-be-summoned reports at their fingertips. In fact, one can argue that many of these mega-SEM vendors aren't selling software - they're selling the CxO's dreams right back to them. Unfortunately, these dreams are never fully realized. And the results are tragic:

• Hundreds of thousands, if not millions of dollars, spent on the actual software or appliance


• More hundreds of thousands spent on service contracts and consulting


• Lost employee hours attempting to get the behemoth package to work


• Significant opportunity costs to the business during this process


• Additional software costs when new vendor packages are purchased to produce the sort of information the mega-SEM package was supposed to be delivering in the first place.
  

Enough doom and gloom. Here's a novel philosophy that CxOs can use to reduce the pain and maximize the gain of procuring log management technology:

Step 1: Delegate the work of procuring SEM and log management packages to the department heads that manage the different assets of your network (e.g. the Windows Platform team lead, the *Nix Platform team lead, the Infrastructure/Router/Switch/Firewall team lead).

Step 2: Instruct your various department heads to research and test the best-of-breed log management offerings that are directly relevant to the devices and computers they manage. These department heads are in a unique position to understand the subtle details that can sink or swim a particular SEM package in your environment. They can also tell you the role and quantity of the devices they manage, so you can make a more more targeted distribution of resources (e.g. 80% of all managed devices are Windows servers, 15% are *Nix, and 5% are Other).

Step 3: Empower your department heads to procure the log management package that best suits their realm of your network, and make them responsible for managing, operating, and documenting the software, producing reports on a recurring basis that can be directed to you as needed.

It is our contention that if you adopt this approach, your log management project and procured technology will be:

• Under budget


• Less prone to failure


• Less vulnerable to obsolescense or downtime caused by critical changes in event logging


• More likely to produce higher ROI

Thus we conclude our public service announcement on this topic.

That Infernal Road, Paved With Good Intentions...

Eric, the head auditing guru at Microsoft, posted today on his blog that he is receiving an ever-increasing number of complaints on the lack of documentation regarding the new Event IDs in the Windows Vista™ security log. Specifically, he says that our earlier post "complains" about how sometimes the "add 4096" rule works in Vista's security log, but not in all cases.

With that background, let me take some time here to clarify our original comments and attempt to speak to the source of the frustration Eric is hearing from log management vendors, log scripting enthusiasts, and security admins.

First off, our earlier post on the 4096 offset trick in Vista was not a complaint in so much as it was an attempt to draw attention to a very significant change in the Windows Vista security log. Keep in mind, while Microsoft has made subtle changes to security events ever since Windows NT, the changes in auditing from Windows® NT to Windows 2000 to Windows XP to Windows 2003 are nowhere near as complex as the changes from Windows 2003 to Windows Vista and the forthcoming Windows Server 2008™.

Expanding on this, the complete renumbering of security events in Vista is just the tip of the iceberg. Compounding this trauma of sorts is:

A.) A completely new logging file format, the EVTX file
B.) A completely new API that is used to manage these EVTX files
C.) New, different auditing categories (Tasks) in the Vista security log
D.) Shifting of user account information out of the User field altogether in security events
E.) Other changes to the "traditional" log fields that were present in the legacy EVT files (e.g. the Level/Type field)
F.) Other issues related to forward and reverse compatibility as it relates to log management on pre-Vista and Vista.
... etc

That being said, we know that Eric is not responsible for all of these changes. He did not create the new EVTX log format or the API used to access it, for instance.

Collectively, though, all of these challenges together are most likely frustrating third-party log management vendors, as well as the admins who have developed scripts to automate security event management. Unfortunately, it would appear that Eric is getting the brunt of that frustration. Perhaps he should post contact information for the team at Microsoft that developed the Crimson logging format and accessory APIs so that constructive criticism and questions can be more properly distributed.

At Dorian, our approach is to adapt and innovate around the changes to Microsoft Vista's new logging format and auditing system, and we are proud of our efforts to date. Still, we hear every day the issues that small and medium sized businesses face regarding log management, often directly due to compliance regulations. Not every organization has the budget or resources needed to procure a commercial log management package, and for those facing a complete rearchitecture of their log automation scripts in Windows Vista and Windows Server 2008, those limited resources just got stretched even tighter.

Highlights From the Event Archiver 7 Press Release

Initial feedback on Version 7 of Event Archiver® from customers has been very good. Version 7, in case you didn't read the earlier blog posting, has direct support for Microsoft Windows Vista™ EVTX logs throughout the program. We also added a bunch of cool new features to help overcome some shortcomings in Vista eventing which we are calling LogRefiner™ technology.

This week, we sent out a press release regarding our launch of Event Archiver 7. Here are some highlights, with some of the most interesting sections highlighted in bold:

Dorian Software Creations, Inc. www.doriansoftware.com today announced the release of Event Archiver 7 (www.eventarchiver.com), the latest version of its automated log file collection and consolidation tool.

Having announced earlier in the year a U.S. patent for its Total Event Log Management Solution ™, the globally recognized leader in log management is again charting new territory within the SEM and SIEM markets. This time, Dorian is striking early at the looming onslaught of EVTX files – logs generated by the new Windows Vista and upcoming Windows Server ® 2008 operating systems – that compliance and security specialists face.

Dorian’s development team has been warning for some time in its blog at http://eventlogs.blogspot.com/ that the change in log formats from the existing EVT format to the new EVTX is rife with pitfalls - for admins and particularly, compliance and security specialists seeking consistency and reliability for log audits. The warnings have not articulated a preference between the log types but have instead stressed the importance of understanding the pitfalls before moving forward with Windows Vista and Windows Server 2008 migrations.

Many network administrators and those attempting to audit existing log data have just gotten the hang of the EVT format. Now, within the Windows ®platform alone, these security professionals face the specter of disparate formats and all the problems those differences bring: new event IDs; different formatting of data; and last but not least, changes in the way logs are handled for collection, monitoring, and reporting. Microsoft's shift to the EVTX format in Windows Vista and Windows Server 2008 is truly the elephant in the room for those tasked with ensuring compliance and log retention.

The differences in the log formats and the methodologies behind them are far greater than many in the industry are willing to admit. We are responding to these changes not by forcing upgrades to our software or encouraging adoption of the new format, but by focusing instead on the management of these log types side-by-side. After all, the adoption of the new log format within the private and public sectors is just beginning, and many requirements force organizations to store years-worth of log data. That means, in many cases, auditors and forensic investigators will be looking at the “old” EVT logs for another 5-10 years at least.

...

As a result, Dorian Software Creations, Inc. is introducing its exclusive LogRefiner ™ technology. The focus of this new technology is the careful management of both log formats side-by-side, streamlining the management of both formats via consistent logic and methodology. Therefore, early adopters of Windows Vista and Windows Server 2008 - the operating systems that generate the new EVTX format - can take advantage of log management capability in Event Archiver today. This again sets Dorian Software apart from other log management vendors - almost all of which have been notably mute or at least guarded in their response to the major changes facing SEM and SIEM efforts.

...

Because the management of both log file formats will be necessary for yearsto come, Dorian Software stresses that any releases including the LogRefiner technology will not abandon those who continue to work with the EVT format.

...

Windows Vista EVTX File Support

Event Archiver has the capability to collect and convert EVTX log files. This is the new logging format first introduced in Windows Vista and planned for use in Microsoft Windows Server 2008. Simply install Event Archiver to a Windows
Vista workstation to start collecting EVTX files from other Vista workstations.

LogRefiner ™ Technology Makes Downlevel EVT File Processing in Windows Vista Possible

Dorian's exclusive LogRefiner technology can archive and convert EVT files from downlevel systems directly alongside the EVTX files from Windows Vista and newer operating systems - the converting and reading of EVT files being the very thing that the Microsoft Event Viewer on Windows Vista has difficulty doing correctly. With Event Archiver's special new technology, no information goes missing when converting downlevel EVT files into new formats – all event log fields are processed properly the first time.

Streamlines Fields Between EVT and EVTX Logs With LogRefiner Technology

Did you know that Windows Vista’s EVTX logs have even more fields? Event Archiver 7 can be instructed to automatically consolidate these fields - the Keyword and Opcode fields specifically - into the Task (Category) field so that you can have a uniform data structure for EVT and EVTX exported log files.

LogRefiner Technology Maintains Field Consistency Across
Logs

In the Windows Vista Security Log, no information about the user performing the action or affected by the action is recorded in the User field when an event is logged. Instead, all user information is placed in the Description of the event. Event Archiver 7, however, has the ability to place the most relevant user information back into the User field as it converts EVTX files into new formats. By helping maintain the consistency of log data and its formatting, this feature greatly aids the administrator or compliance officer in charge of reviewing the consolidated data.

Defines Success Audits Versus Failure Audits Using LogRefiner
Technology

Another major change in the Windows Vista security log is that all events are recorded as “Informational.” To discern whether or not the event represents a failed or successful action, the administrator must refer to the Keyword of the event.

But, Event Archiver 7 - when converting security EVTX Files - has the ability to properly record whether or not the event was a Success Audit or Failure Audit, greatly aiding the reviewer of log data generated from both EVT and EVTX log files.

To sum up, our LogRefiner™ technology in Event Archiver 7 means that:

1.) You can migrate to Windows Vista and Windows Server 2008 when you are good and ready, knowing that,
2.) Our software will process the downlevel EVT files for you right alongside the newer EVTX files, and
3.) Event Archiver has advanced technology that standardizes the collected data for reporting and other compliance purposes.

From Windows NT to Windows Server 2008, Event Archiver 7 has you covered. If you'd like to take it for a test drive, you can download your free 30-day evaluation copy at http://www.doriansoft.com/download. Happy archiving!

Storage Requirements for the Windows Vista™ Security Log

Recently, we've created a few blog postings that talk about some of the new events present in the Microsoft Windows Vista™ security log. From a security standpoint, Vista's increased number of auditable events is excellent, as administrators and compliance officers can get a much deeper picture of the actions taking place on a computer prior to and during a security incident.

However, if you are required to retain those security events, either by law (e.g. HIPAA, SOX, GLB, PCI, etc) or by policy, you need to start budgeting for more storage before you start your Vista and Windows Server 2008™ migrations.

Here are a few examples of how Vista security logs tend to grow much more quickly than their predecessors:

1.) Looking at some of our internal Vista security logs, there are tons of events relating to the blocking or accepting of network data via the Windows Filtering Platform. Some organizations may find this data valuable, especially if the machine is exposed to the public, however others may not.

2.) Some events log extra information at the end of the Description field that serves no other purpose than to further explain the parameters in the Description field. For instance, every 4608 event (Windows is starting up) also tells you that:

"This event is logged when the LSASS.exe starts and the auditing subsystem is initialized."

Similarly, every 4634 event (An account was logged off) feels the need to mention that:

"This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer."

These are just two brief examples, but note well: your Vista logs will use up more space than your XP and Windows 2000 workstation logs. If you are reassuring yourself now by thinking that you only need to retain server logs, bear in mind that Windows Server 2008 will share Vista's new events and logging tendencies!

Fortunately, the current release (and several prior releases) of our Event Archiver™ software offers you techniques to help you manage your storage of log data. Event Archiver allows you to automatically prune your database tables by date, selectively import only key events or exclude non-key events into database tables with global import filters, and keep your data in multiple compressed formats for storage efficiency. As the number of auditable events increase and expand in size, these features become increasingly important.

Vista-Compatible Release of Event Archiver is Here!

As promised, we released version 7.0 of Event Archiver yesterday. Event Archiver 7.0 is the first of our log management titles to support Microsoft Windows Vista™ and the new EVTX log format. However, Event Archiver 7.0 is more than just "compatible" with Windows Vista, as it also introduces some very cool abilities our marketing department refers to as LogRefiner™ technology.

Specifically, the biggest LogRefiner™ technology accomplishment is that downlevel EVT files from previous Microsoft Windows® versions get processed correctly when Event Archiver is running on Windows Vista, which the built-in Event Viewer on Vista cannot do properly. Beyond that, it encompasses numerous other features, such as consolidating fields in EVTX files, appropriately categorizing security events as Success Audits and Failure Audits, and placing user information from a Security EVTX file back in the User field. You can read all of the features here:

http://www.doriansoft.com/ourcompany/announcements/6-07.htm

As far as we know, we're the first log management ISV to offer this level of dual EVT/EVTX file processing technology. But, we've also been in the market since 1997, so pioneering new log management techniques is nothing new to us! .

On top of the Windows Vista features, we also added MD5 cryptographic hashing of archived log files and a Working Directory feature for local processing of remote log files.

Needless to say, this is a huge accomplishment that we're very proud of. Now, it's back to the skunkworks to get our other log management titles working with Vista.

Vista-Compatible Release of Event Archiver is Near

With all the posting we've been doing about Vista events recently, you may have assumed we've been doing a lot of work to get our software titles Vista-ready. You'd be correct.

Stay tuned to the blog, as next week we're going to reveal Version 7.0 of Event Archiver, with tons of really cool new Vista-specific features. We're introducing some pretty radical technology, and we think you'll be quite impressed! :)

Auditing Changes To Permissions (Event ID 4670)

Last week, I mentioned that Vista had a neat new event (Event ID 4907) that told you when the SACL (e.g. the list of users/groups who generate security events *when they access* a file/folder/securable object) was changed. Well, there is another new event that you could say is the twin brother to Event ID 4907.

Event ID 4670 gets logged when anyone changes the DACL (Discretionary Access Control List) on a file, folder, or securable object. For more information on DACLs and SACLs, you can refer to this post below, but as a reminder, the DACL of a file/folder/object is the list of users/groups that *can access* or are *denied access* a file/folder. In other words, that file or folder's permissions.

Prior to Vista, you had to root around in the description field of Event ID 560 or 566/567 and check the Accesses granted to a user that touched a file to see if they could have (or actually did) change the permissions on a file. Now in Vista, Event ID 4670 will tell you immediately if the permissions get changed, who changed them, what they used to look like, and what they look like now. Here's a sample of how the event looks:

Permissions on an object were changed.

Subject:
Security ID: DOMAIN\Admin
Account Name: Admin
Account Domain: DOMAIN
Logon ID: 0x11b8ffd

Object:
Object Server: Security
Object Type: File
Object Name: C:\financials.txt
Handle ID: 0xf50

Process:
Process ID: 0x50c
Process Name: C:\Windows\explorer.exe

Permissions Change:
Original Security Descriptor: D:AI(A;ID;FA;;;BA)(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)(A;ID;0x1301bf;;;AU)
New Security Descriptor: D:ARAI(A;;0x1e01bf;;;WD)(A;ID;FA;;;BA)(A;ID;FA;;;SY)(A;ID;0x1200a9;;;BU)(A;ID;0x1301bf;;;AU)


So, you can see it looks a lot like its brother, Event ID 4907, even down to using the same SDDL strings to indicate the changes to user/groups who have permissions on the file. Very cool stuff.

Auditing Changes To Your Auditing (Event ID 4907)

Here's another nice new security event that has been added to Vista - Event ID 4907. It has been tasked with recording every time an administrator or program changes the SACL on an object, typically a file or folder. What's a SACL, you ask? Let's have a brief primer on that subject.

Every securable object (e.g. file, folder, registry key, etc) in Windows has a Security Descriptor assigned to it. The security descriptor, among other things, specifies:

1.) the user owner of the object
2.) the group of the object (used by Unix apps that run under POSIX)
3.) the DACL (Discretionary Access Control List), and
4.) the SACL (System Access Control List)

When you use Windows Explorer or Group Policy to change who can access a file or folder, you are changing the DACL. Similarly, when you click the "Advanced" button in Windows Explorer on a file or folders property page, and visit the Auditing tab, you are changing the SACL.

The SACL is what the operating system uses to determine which users, groups, and identities cause auditing events to be generated in the Security log when said users perform various actions on files, folders, registry keys, etc.

So to summarize: When you change the users/groups that *can access* a file/folder, you are changing the DACL. When you change the users/groups who generate auditing events *when they access* a file/folder, you are changing the SACL.

Anyway, back to Event ID 4907. In Vista, this event gets logged any time an administrator changes how a file/folder is audited. Here's a sample of the event description:

Auditing settings on object were changed.

Subject:
Security ID: DOMAIN\Admin
Account Name: Admin
Account Domain: DOMAIN
Logon ID: 0x1f472

Object:
Object Server: Security
Object Type: File
Object Name: C:\Folder
Handle ID: 0x28c

Process Information:
Process ID: 0x690
Process Name: C:\Windows\explorer.exe

Auditing Settings:
Original Security Descriptor:
New Security Descriptor: S:ARAI(AU;OICISAFA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)


Reviewing the description of this event, we see that we can determine:

1.) Who changed the SACL (DOMAIN\Admin)
2.) What program they used to change the SACL (explorer.exe)
3.) The name and type of the object changed (c:\folder, file)
4.) A SDDL representation of the old Security Descriptor and new Security Descriptor.

So now, in Vista, you can track anyone who changes how a critical file/folder is audited, including how it was audited BEFORE the change, and how it will be audited AFTER the change. Again, this is great from an accountability standpoint in organizations governed by compliance regulations.

Oh, and if you're curious about how to translate the SDDL string into something meaningful, please read this article.

Auditing and Storage Requirements

One thing that admins tend to overlook when setting up a SEM or event log management package on their network is the amount of storage required to house all of the event log data.

Regardless of which vendor you choose (or even if you decide to attempt to do it in house with scripts), you need to keep in mind that the data output from native Windows event log files (e.g. EVT/EVTX files converted into database tables) will be greater in size than the native event log files themselves.

As mentioned briefly in this earlier post, EVT files contain references to other information not present in the log file itself. The resolution of those references into meaningful data is one reason behind the increase in data size after conversion.

Another factor that causes data size expansion is the field structure of the database itself. The number of bytes allocated for certain fields, as well as the use of UNICODE strings (e.g. two bytes per character) can both further contribute to this phenomenon.

A while ago, we wrote a freeware utility that does a nice job estimating the storage required to maintain your log data in various formats over time. You can download our Auditing Volume Analyzer tool here.

One final caveat: some SEM vendors in the marketplace attempt to take the log storage issue out of the equation by providing their own "black box" or appliance for accumulated log data. In general, we frown upon that approach here as it has the potential to hold your data captive. Should an audit or other incident arise - the last thing you want is data held captive in a proprietary storage system. For more on this topic and our general philosophy on event log management, please read this article.

Who's that user changin' that key? It's me! It's me!

The gang at Microsoft did quite a number on auditing in Microsoft Vista and Longhorn Server. In addition to making legacy auditing events more granular in the Vista security log (e.g. containing more information), they've also added completely new events which will be well-received by administrators and compliance officers.

An example of one such new event is 4657 (Registry Value Changed). In Vista, if you set your audit policy correctly, you can tell Windows to log an event every time one or more values underneath a specific registry key are changed. Here's a sample of what the event looks like when it is logged:

A registry value was modified.

Subject:
Security ID: DOMAIN\SomeUser
Account Name: SomeUser
Account Domain: DOMAIN
Logon ID: 0x11b8ffd

Object:
Object Name: \REGISTRY\MACHINE\SOFTWARE\AppVendor\ProgramName
Object Value Name: AdminEmail
Handle ID: 0x2e8
Operation Type: Existing registry value modified

Process Information:
Process ID: 0xb40
Process Name: C:\Windows\regedit.exe

Change Information:
Old Value Type: REG_SZ
Old Value: adminold@domainXYZ.com
New Value Type: REG_SZ
New Value: adminnew@domainABC.com

Taking a look at the meat of the event, we can ascertain 1.) who changed the value, 2.) with what program, 3.) the name of the value, 4.) the old value data, and 5.) the new value data.

Pretty impressive. However, this does raise an interesting paradox. If certain registry data is so valuable that you want audit access to it, do you want that same data splashed into the event log? Yes, you can control access to the log, but having the data in the log to begin with raises some issues.

It might be cool if Microsoft had a tweak for this event that allowed it to be audited with everything BUT the value data included. Just a thought.

Backing Up Your EVT Files

Occasionally, we have admins send us EVT files that they have restored from live server backups (or recovered from the hard disk of a server that crashed) that cannot be opened in the Microsoft Event Viewer. This often generates a substantial amount of concern on their part, and so it merits discussion here.

When a Microsoft Windows NT, Windows 2000, Windows XP, or Windows 2003 Server is running, the EventLog service maintains an open handle to each of the active event logs on the system. From what we understand, each active event log is treated like a memory-mapped file. Simply performing a live backup of the event log files in the \system32\config folder, even if the backup software can work with open file handles, is ineffective. The linked data structures in the active event log file may not be "finalized" so that they can be read by the OpenBackupEventLog function, and so attempts to read these logs as if they were properly saved may fail.

Here's a quick way to test this sort of behavior. On your workstation, navigate to the \Windows (or Winnt)\System32\Config folder using Windows Explorer. Copy the active security event log file (e.g. SecEvent.evt) to your desktop. Then, open the Microsoft Event Viewer, and try and open the SecEvent.evt file you copied onto your desktop. The Event Viewer will tell you that the file is corrupt.

So that being said, how is it that you can still read active event log files via the Microsoft Event Viewer when the computer is online? Simple - the request to read the file is made directly to the EventLog service, as opposed to trying to read the data directly out of the active file itself. The EventLog service, in addition to logging new records to active event logs, also acts as a proxy "log record fetcher" for the benefit of authorized users that need to read the contents of the active logs.

A very interesting phenomenon can be observed, however, if you attempt to read an "active" event log file from the \system32\config directory on a hard disk partition when the operating system is not loaded. If the operating system located on that partition was shutdown properly (e.g. it didn't crash), the EVT files in this directory should be able to be read by the Microsoft Event Viewer as saved event log files. It would appear that the EventLog service, when shut down normally, makes sure that the linked data structures in the file are organized properly before the file is closed.

What can we learn from this?

1.) Normal backup software, even software that can backup open files, is not a reliable way to archive your EVT files should you need to access them in the future.

2.) Likewise, trying to read EVT files in the \system32\config folder on a hard disk partition where the last operating system session crashed, or where the computer was shut down dirty, may fail.

3.) The EVT files in the \system32\config folder on a hard disk partition where the last operating system session was shut down properly can most likely be read as saved EVT files in the Microsoft Event Viewer. So, provided the machine was shut down normally before the hard disk was removed (e.g. in a forensic examination for instance), chances are good the log data will be accessible.

To combat these sorts of issues, we introduced our Event Archiver(tm) software many years ago. Event Archiver can archive EVT files correctly via the EventLog service on multiple computers, so that they will be accessible for review many years down the road. Of course, this is one small aspect of its feature set, but is a very important feature nonetheless.

Tip of the hat to Eric at Microsoft

Many kudos to Eric Fitzgerald, the head of Microsoft's Windows Auditing Team, for his very candid explanation on the transposing of security event ids in Vista. He writes:

Doriansoft noticed that there's a relationship between our pre-Vista security event IDs and our Vista-era security event IDs.

For most security events:VistaEventId = PreVistaEventId + 4096

Why is this?

We needed to differentiate the Vista events from the pre-Vista events, because we were significantly changing the event content and didn't want to break automation. However we wanted to preserve the knowledge that security professionals already had in their heads about security events, so we wanted to make sure that there was a relationship between old and new event IDs.

We decided to offset the old IDs by some constant to get the new IDs. I wanted to offset them by a decimal number (say 6000, so 528 would become 6528, etc.). However event IDs are declared in hex in the source code and are all 3 digits long (528 = 0x210), and Raghu, my developer, wanted to conserve effort, and he won that battle so we added 0x1000 (4096) to the existing event IDs.

For what it's worth, I think Eric's initial approach would have been best, as I think most non-developers can deal with Base 10 offsets in their head much more easily than Base 16. Still, his candor in addressing the issue is refreshing, and is much appreciated.

Crash ... Into Me

Many of our high-security clients must enable the CrashOnAuditFail setting on their servers, as per government policy. In case you're not familiar with this setting, open up your Registry Editor, and visit the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa

One of the values under this registry key is "CrashOnAuditFail" ... By default, this value is set to 0. If you set it to 1 (and I don't recommend you do, unless you have a test machine you're tinkering with), your system will Blue Screen as soon as the security log fills up, provided you have also prevented your workstation or server from overwriting events automatically in the security log. At that point, only an Administrator can log back on to the machine after a restart to clear the security log and reset the CrashOnAuditFail flag.

The purpose of this special setting is to prevent a computer from being used (e.g. logged into) by anyone other than administrators unless auditable events can be recorded in the security log. Thus, it is a very important setting in high-security networks.

Interestingly enough, Vista adds a new event related to this special registry value. In Vista (and presumably Longhorn server as well), Event ID 4621 gets logged when an administrator successfully recovers the system from a crash related to the Security log filling up. From a documentation and accountability standpoint, this is a nice new event that Vista brings to the table.

References Available On Request...

Today we're going to tackle one of the biggest misconceptions among admins about the Windows Event Log. In particular, the notion that it is a.) a self-contained file and/or b.) a text file. It is in fact neither one.

If you drop a saved EVT file into a text editor like Notepad, you will see some text strings, but you're also going to see a lot a gibberish. Together, the readable text and the gibberish make up a bunch of linked data structures. Microsoft calls this structure an EVENTLOGRECORD, and a saved EVT file has a ton of them, one for each log entry in the file. An EVENTLOGRECORD contains a whole bunch of information, including the date/time the log entry was written to the file, the security identifier (SID) of the user performing or logging the action, the category of the event, the source, event identifier, etc.

Earlier, it was mentioned that event log files are not self-contained. For example, the log file doesn't contain the name of the user performing the action - instead, it contains the SID number (security identifier). In other words, a log entry generated by NT AUTHORITY\SYSTEM wouldn't have NT AUTHORITY\SYSTEM as a text string in the EVT file - it would contain the SID "S-1-5-18" represented in binary. In some respects this seems tedious, but since usernames get renamed often in a Windows domain, it makes sense. If you decide to review an EVT file again in the Microsoft Event Viewer many months after it was initially saved, you want to see the usernames the SIDs resolve to at that instant, not what they resolved to months ago. So, any application that consumes or reads event log files needs to be able to resolve that information, typically via the domain controller or the member server/workstation where the log file originated from.

Another wrinkle in the Windows Event Log format is how the Description field - which often contains very valuable information - gets put together. EVT files don't contain all the data that you see in the Description field of events you view in the Microsoft Event Viewer. Instead, they contain the key nuggets of data that get combined together with message frameworks from resource files to form a complete description.

For instance, your antivirus software may elect to write Event ID 20 to the Application Event Log. Event ID 20 corresponds to the following message template in the Antivirus software's message files: Virus %1 was detected in file %2

However, the Antivirus software will only send the two parameters needed by the message framework when it logs the event. In this example, MyDoom for the virus name and c:\dropper.exe for the file.

In other words, your EVT file will only contain MyDoom and c:\dropper.exe ... and it's up to the program displaying the event to you to put together the complete description, which in this case is: Virus MyDoom was detected in file c:\dropper.exe.

Given that event log files contain so many references to information that in many cases can only be obtained from the machine where the files were generated, here are some tips that will serve you well:

1.) If you routinely save your event log files, consider saving them in both the EVT/EVTX format and another self-contained format like comma-delimited text or a database table. The EVT file will contain valuable information like the original SIDs, and the text file/database table will contain a snapshot of how that data was translated at a certain instant in time.

2.) If you uninstall software, you may find entries in your event logs where the Description field cannot be correctly formatted. This is because message files often get uninstalled alongside the software. If you reinstall the software, even temporarily, you should be able to see the full event descriptions again.

3.) Transplanting Windows Event Log files to a network outside the one they originated from is a losing proposition. When the Microsoft Event Viewer or other log viewing application attempts to resolve the many references in the file, it will more often than not fail and produce incomplete data. As a result, you will only see limited amounts of information, such as Well-Known SIDs and events from software/hardware that match the software/hardware installed on the originating machine.

4096 Security Events Lane

What's a big difference in the Vista security log? Here's your clue ...

2^12
1000 in Base 16/Hex
1000000000000 in Base 2/Binary
4096 in Base 10

If you scan through your security log in Vista, you're going to see some very unfamiliar Event IDs.... 4616 (System Time Changed), 4624 (Successful Logon), etc.

Let's do some quick math:

4616 - 4096 = Our old friend Event ID 520
4624 - 4096 = Our old friend Event ID 528

For fun (I'm sure they had a more legitimate reason, right?), Microsoft decided to add 4096 to quite a few of the old well-known Security Event IDs in Vista. Now bear in mind this "subtract 4096" trick doesn't work for every event, and also understand that some of your favorite Event IDs have gone missing.

Missing Event IDs? Sure.

Like 540 (Successful Network Logon) ... he's been forced to reside with his first cousin 528 (Successful Logon) at 4624 No Caps Lock Drive.

Don't feel bad for 540 though. Just ask those naughty logon failure IDs of yesteryear, like 530 (Account Logon Time Restriction Violation) and 535 (The account password has expired). They - and several of their siblings - now have to live at 4625 Fat Fingers Boulevard.

For all those folks out there using scripts for security log management ... you have some updating to do.

Seeing Crimson...

Sorry, couldn't resist that little play on words for this inaugural blog post. We're wrapping up development right now on new libraries to support the management of logs on Microsoft Vista. In case you haven't heard, Microsoft has introduced a new logging format in Vista, EVTX files. Back in Vista's beta days, this logging format was code-named Crimson.

So far, the development experience has been quite an eye opener. While the ReportEvent function, which is the cornerstone function for writing to the event log in the legacy EventLog API, works great in Vista, other Legacy API techniques do not. For instance, if you try calling the OpenBackupEventLog function on Vista to open a saved legacy EVT file, the function will fail. Interestingly enough, Microsoft has still not updated its documentation at MSDN to reflect this problem as of this writing: http://msdn2.microsoft.com/en-us/library/aa363671.aspx

Going in the other direction, legacy Windows clients (e.g. NT/XP/2000/2003) can open a handle to a "live" Crimson/EVTX log on Vista remotely, but the traditional techniques used to parse through and render the data on such a log will most likely fail due to a variety of other reasons that relate to the hardening of Vista's networking and new message provider data stores. On top of that, legacy Windows clients simply have no mechanism for reading saved Vista EVTX log files whatsoever.

While the Crimson/EVTX format does confer advantages over its predecessor, such as XPath queries, we're still not sure why Microsoft elected to cripple the OpenBackupEventLog function on Vista while supporting other legacy EventLog API calls. Certainly, from a programmatic standpoint, it would appear that the function does little more then read linked data structures out of a saved binary file. One perhaps controversial theory is that Microsoft wanted to make transitioning to Vista much easier for publishers of Windows events (e.g. software developers whose programs write to the log), than for the consumers of the those events (e.g. the utility software vendors whose programs manage and analyze log files). Considering Microsoft's efforts to increase market share in the server management market, that could be the case.

Regardless, we've found cool new ways to work around these potential trouble spots and look forward to introducing Vista/Longhorn compatible versions of our software very shortly.