1. Introduction

The core TMG behavior (which is the same on ISA) for handling spoofed packets is well known. Basically, if the IP address belongs to the internal network and it is received on the external interface, it is expected that the packet will be dropped. By definition, the intrusion detection system on TMG will do the following:

Note: if you to the logs for this traffic, you will see that it drops the packet for the following reason: FWX_E_FWE_SPOOFING_PACKET_DROPPED.

2. Troubleshooting

When you receive spoofing error, the first action that you should take is to make sure that your network definition is correct. In other words, make sure that the source address (in this case 172.16.0.2) does not belong to your internal network. For this particular example that we are using on this post, TMG didn’t have this range as part of the internal network as shown below:

Notice that we have internal, external and also the 10.80.80.0/24 network, which is the VPN Client network, but 172.16.0.0/16 is not part of the internal network. So, why TMG is blocking the traffic as spoof?

The reason why this happens is because TMG analyzes the source IP of the packet and verify if this network is reachable via external interface (where it was received), if it is not then it interprets this traffic as spoofed. A snip from the lower level TMG tracing shows the following:

This problem was fixed by adding a default gateway on the external network of TMG. If you can’t add (for some network infrastructure reason) a default gateway on the external adapter of TMG, you will need to manually add a route to the network that it is getting dropped as spoofed so TMG knows how to get there.

3. Conclusion

This post explains a scenario where a missing element on the configuration (default gateway on the external interface) caused legitimate traffic to be denied as spoofed. Keep in mind those rules of thumb while configuring TMG networks so that you can avoid issues of this nature.

Author
Yuri Diogenes
Sr. Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Technical Reviewers
Mohit Kumar
Sr. Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Thomas Detzner
Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Forefront TMG 2010 provides an enhanced array management feature known as array join / disjoin.  This article describes a problem that may be encountered as a result of a failure during this process, the solution you can apply as well as a discussion of how and why this problem occurs.

You attempt to join a TMG server to an empty EMS-managed array, but this action fails after some time because Intern Jamie unplugged the network cable from the EMS server by mistake (or so you choose to believe) during this process. 

You have reconnected the EMS server to the network (and Jamie to his basic networking books) and attempt to join the server to the array once more, but now it fails with “The operation failed” message as shown below:

Although you have verified that Jamie is nowhere near the EMS server or any connecting devices between the EMS server and prospective array member, the array join failures continue with the same message; even with a completely different TMG server.

This problem does not occur when joining any array that contains functioning array members (which is why it can’t happen for Standalone arrays).

Note: when you encounter an array join failure, you must close and reopen the TMG manglement console before the array join wizard link is available.

If you encounter this error during an array join process, perform the following steps on any computer that can establish connectivity to the EMS server on TCP port 2171.

1. Log on to the computer using an account with administrative rights to the TMG Enterprise
2. Download http://www.isatools.org/tools/cleartmgrptsrvrs.wsf to c:\tmgtools
3. Open a cmd window
   1. Hold down the Windows key and press the “R” key
   2. In the Run dialog, type cmd and hit <Enter>
4. In the cmd window, type
   1. cd c:\tmgtools
   2. cscript cleartmgrptsvrs.wsf /css:NameOfEmsServer

Note: the /css option may be omitted if you are running the script on the EMS server itself

The script performs the following actions:

1. connect to the specified EMS server (or “localhost” if none is provided)
2. check all arrays in the TMG Enterprise configuration
3. verify that the defined reporting server exists in the array servers set
4. if the server is not found in the array servers set, remove the server reference from the reporting configuration
5. log all actions to a text file

If the defined reporting server is not found in the array servers set, the reporting server definition is cleared and the following messages are displayed (and logged):

If the array is empty and has never experienced an array join failure of this type, the following messages are displayed (and logged):

If the array is properly configured, the following messages are displayed (and logged):

The script also creates an execution log file in the same folder from which it was run.  If you followed the instructions, it will be found as c:\tmgtools\cleartmgrptsrvrs.log.

Here’s what happens in this scenario…

When a server joins an array; whether this is the first server or the last, the array join wizard process performs several actions that make the new server known to the EMS and any array members that may already exist.  One of these actions is to review the target array’s reporting configuration.  If no server reference is found there, the array join wizard creates one, adds the new server’s UUID to it and moves along with the remainder of the array join actions.

If the array reporting configuration does include a server reference, the array join wizard tries to locate that server in the array’s server collection.  If the defined reporting server cannot be found in the array servers collection, the “0x80070002” error is returned and the wizards halts; displaying the error results to the user.

When the array join wizard encounters a recoverable failure, it is designed to roll back (remove) all changes made in the target array up to the point of failure so as to leave the array in a clean state. 

The core problem is caused because the array join wizard does not clean up the reporting server configuration during the rollback process.

This article describes only one cause and resolution for this sort of problem.  It may actually be caused by any number of array management actions that (for any number of reasons) fail to complete and roll back successfully.  In cases where the script fails to resolve array join failures to empty arrays that produce the same error message, it may actually be simpler to remove the empty array and replace it with a new array.

Author
Jim Harrison, Program Manager, AAG

Reviewers
Dima Datsenko, SDET, TUBT
Bala Natarajan. Sr Support Escalation Engineer

Introduction

E-Mail Protection feature on TMG is one of the features that allows TMG to leverage other resources available in the environment. In this case, TMG uses Exchange Edge technology as well as Forefront Protection for Exchange. Before start reading this post, make sure to get familiarized with E-Mail Protection feature on TMG by reading the following articles (in this order):

• Understanding E-Mail Protection on Forefront TMG
• Installing prerequisites for e-mail protection
• Using Mail Protection with Exchange EdgeSync on Forefront TMG

This post is about an issue where after following those steps to configure E-Mail Protection on TMG, e-mails coming from the Internet were getting stuck on Exchange Edge Queue with the error code 4.7.0.

Troubleshooting

The SMTP error 4.7.0 translates to “temporary authentication failure”, which means that Exchange Edge is having problems to authenticate to the internal email server (in this case an Exchange Hub). Notice that this is pure an Exchange issue, however since TMG is the one responsible for configuring the parameters for Exchange Edge, the initial troubleshooting starts by reviewing the configuration on TMG side.

As of now (TMG 2010 RTM or with SP1) requires that all configuration must be done via TMG UI, if changes are made directly to Exchange Edge or Forefront Protection for Exchange, you will notice that the following alert will appear on TMG and the configuration that was changed on those products will be replaced by the one present on Forefront TMG.

 
Figure 1

Since we are dealing with authentication issue (remember SMTP code 4.7.0) the first thing to review is the authentication method that was configured to allow Exchange Edge to talk to the internal mail server. To do that on TMG, follow the steps below:

1. Open TMG Console, go to E-Mail Policy, right click on the Internal SMTP Route and choose Properties.

 
Figure 2

2. On the Internal_Mail_Server Properties click Routing tab and click Advanced button.

3. The Advanced window appears as shown below. Make sure to take note of the authentication method used in this option, you will need to match it up with the authentication used by the internal mail server.

 
Figure 3

4. Click OK twice.

Now that you know which authentication method is used on Exchange Edge (configured by TMG), you need to verify which authentication method is in use by the internal mail server, which in this case is an Exchange Hub. To verify that follow the steps below on Exchange Hub:

1. Open Exchange Management Console.

2. Click Server Configuration and click Hub Transport.

3. Right click on the Receive Connector, choose Properties and the general information for this connector will appear as shown below:

 
Figure 4

4. Click Authentication tab and make sure that the authentication matches with the one specified on Exchange Edge, which in this case is matching as you can see below:

 
Figure 5

For this particular scenario the authentication was matching, so what could be going wrong to receive the 4.7.0? When troubleshooting issues of this nature one simple test that you can do is to try to telnet the IP of the internal mail server from TMG command prompt on port 25 and the name that appears on the SMTP header as shown below:

 
Figure 6

Notice that the name in there is relay.contoso.com, which doesn’t match with the receive connector that we previously verified on Exchange Hub (Figure 4), which is excsrv.contoso.com.

Conclusion

The reason why this issue was happening was because when Exchange Edge was trying to communicate with the internal Exchange Hub, the receive connector that was processing the message was not the default one; it was another one that had a different authentication method. The reason why the other connector was processing the message instead of the default connector was because this custom connector had an IP range more specific than the default connector as shown below:

 
Figure 7

When you have multiple connectors on Exchange, the one that will process the message is the one that has the IP range definition closer to the source IP (in this case TMG was 10.20.20.1) that is trying to connect to the connector. To fix this issue we changed the authentication method of this connector to match with what we have on Exchange Edge. Once this was done all the message that were accumulated on Exchange Edge Queue started to correctly flow.

Author
Yuri Diogenes
Sr Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Technical Reviewer
Ed Bringas
Sr Support Escalation Engineer
Microsoft Exchange CSI Team

Today, the Hyper-V team released a few important Hyper-V updates in a single package via Windows Update. These updates have been publicly available for some time now, but they're now including these as a single roll-up for the convenience of customers. There are three issues resolved with this update:

·         Compatibility with Intel Nehalem Processors

·         Compatibility with Intel Westmere Processors

·         A network issue when, under extreme network load, the VM network connection is lost.

For specifics on the issues resolved with this update, please see the following KB article:

http://support.microsoft.com/kb/2264080

Given the traffic load that a Forefront Edge deployment can place on the virtual network, this seems like a good patch to apply to your Hyper-V parents.

Introduction

This post is about two scenarios which involved ISA server and third party content application. Let’s go over each one of those and the troubleshooting steps involved in order to fix it.

Scenario 1

ISA was doing a Proxy Chain with a third party product, when the client (using Windows XP SP2 with Internet Explorer 6) was browsing a HTTPS web site behind the ISA Server the Internet Explorer crashed with an access violation. If the clients bypassed the ISA Server and use only the upstream third party proxy the issue did not happen.

Scenario 2

Clients using Windows XP SP2 were unable to update via Windows update. When we took one of the laptops and bypassed ISA which was the proxy in the corporate LAN, they were able to update without any problem. Moreover, the automatic update service (which runs in the context of SVCHOST) was crashing when we tried from LAN. ISA was forwarding web traffic to a third party product.

Troubleshooting

The data that was collected to troubleshoot these included

1) Network Capture on ISA on all adapters concerned.
2) ADPLUS crash dump for the respective processes on the client.
3) Application logs from the client machine s.

Data Analysis

For the first scenario the ISA Logging did not show up any suspicious error and the network traces pretty much shows the normal traffic:

1. The client requesting the SSL object from a Web server.
2. The “CONNECT Server_name:443 HTTP/1.1” being sent to port 8080 on the ISA Server computer.
3. The ISA Server connecting to the destination Web server on port 443.
4. The TCP connection established and the ISA Server returning the HTTP/1.1 200 connection established.

Fortunately Internet Explorer was raising an exception (c0000005 – Access Violation) and it was possible to get the dump during the crash. The exception was happening on the module wininet.dll, handling the function HTTP_REQUEST_HANDLE_OBJECT OpenProxyTunnel_Fsm on the client side

For the second scenario the error in the Application log of the client machine when automatic update failed was

Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date: 7/12/2010
Time: 2:18:22 PM
User: N/A
Computer: XXXX
Description:
Faulting application svchost.exe, version 5.2.3790.3959, faulting module winhttp.dll, version 5.2.3790.4427, fault address 0x000000000003e521.

Upon analysis of the crash dump from client machine we found:

The stored exception information can be accessed via .ecxr.
(97c.ba8): Access violation - code c0000005 (first/second chance not available)

We crashed in:

4d51b7d3 c7402400000100  mov     dword ptr [eax+24h],10000h ds:0023:00000024=????????

The module and function in concern was

015ef4d8 4d51b9bc winhttp!HTTP_REQUEST_HANDLE_OBJECT::OpenProxyTunnel_Fsm+0x44b

Loaded symbol image file: winhttp.dll

    Image path: C:\WINDOWS\system32\winhttp.dll
    Image name: winhttp.dll
    Timestamp:        Wed Aug 04 13:27:07 2004 (411096D3)
    CheckSum:         00055D26
    ImageSize:        00058000
    File version:     5.1.2600.2180
    Product version:  5.1.2600.2180
    File flags:       0 (Mask 3F)
    File OS:          40004 NT Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
    Translations:     0409.04b0
    CompanyName:      Microsoft Corporation
    ProductName:      Microsoft® Windows® Operating System
    InternalName:     winhttp.dll
    OriginalFilename: winhttp.dll
    ProductVersion:   5.1.2600.2180
    FileVersion:      5.1.2600.2180 (xpsp_sp2_rtm.040803-2158)
    FileDescription:  Windows HTTP Services
    LegalCopyright:   © Microsoft Corporation. All rights reserved.

The reason it was crashing was because the TCP connection was getting closed. Reviewing the network capture we could see the following:

Client to ISA

CONNECT www.update.microsoft.com:443 HTTP/1.0
User-Agent: Windows-Update-Agent
Host: www.update.microsoft.com
Content-Length: 0
Proxy-Connection: Keep-Alive

ISA to Client

HTTP/1.1 200 Connection Established
Content-Length: 0
Proxy-Agent: ThirdPartyProduct/3.0
Via: 1.1 ISA
Connection: Keep-Alive
Proxy-Connection: Keep-Alive

The client was resetting the packet as soon as it got this packet.

Conclusion

Both the scenarios revolved around the same problem:

• First Scenario - During this scenario ISA Server was not the root cause for the problem; the wininet.dll had a bug that caused the access violation handling SSL tunneling. A hotfix was shipped to fix this issue and can be downloaded on the article 923535 - Internet Explorer 6 crashes when you try to access an HTTPS URL from a computer that is running Windows XP Service Pack 2. Such behaviour is described in this post by Yuri Diogenes.
• Second Scenario - We checked and we found that this issue have been taken care in windows 2008 and windows 7. Never the less bypassing the third party proxy fixed the issue.

Proxy Connect Method is used by the client to tunnel the TCP based traffic from the proxy server. After the connection is established the proxy server simply tunnels the traffic between the client and the internet web server. To tunnel SSL traffic though proxy, browsers or any other application can make a CONNECT request to the proxy server.

More info see http://tools.ietf.org/html/draft-luotonen-web-proxy-tunneling-01#section-3.2.1

Understanding the CONNECT Method

The SSL session is established between the client generating the request, and the destination (secure) Web server; the proxy server in between is merely tunneling the encrypted data, and does not take any other part in the secure transaction.

Request

The client connects to the proxy, and uses the CONNECT method to specify the hostname and the port number to connect to. The hostname and port number are separated by a colon, and both of them must be specified.

Example:

CONNECT mail.microsoft.com:443 HTTP/1.0
User-Agent Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; MS-AOC 4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; MS-RTC EA 2; MS-RTC LM 8; .NET4.0C; .NET4.0E)

Proxy Response

The client will wait for a response from the proxy. The proxy will evaluate the request, make sure that it is valid, and that the user is authorized to request such a connection. If everything is in order, the proxy will make a connection to the destination server, and, if successful, send a "200 Connection established" response to the client

An example of a tunneling request/response in an interleaved:

Although there is no specification details about Content length header in Proxy Connect Request and Response headers, never the less having a content length in the CONNECT header is not of much use. Connect is used only make an initial connection to the proxy server for requesting to tunnel traffic

Author
Saurav Datta
Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Technical Reviewer
Yuri Diogenes
Sr Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

TMG can be configured in a Mail protection role. In such configurations Forefront Protection for Exchange and Exchange Server (edge transport role) are installed on the same machine as TMG.

We have identified problems when installing Microsoft Exchange Server 2010 Service Pack 1 (SP1) that was released last week on such deployments.

Root cause

SP1 made some changes to the SDK including removing some of the existing cmdlets (see more information here).

When Email protection is configured on TMG and Spam Filtering functionality is enabled, TMG uses one of the cmdlets that has been removed (get-antispamupdates) in SP1. As a result, Microsoft Forefront TMG Managed Control service fails to start and the event viewer will contain a message that the service terminated with the following error : %%-2146233088 :

What we are doing to address this problem

The TMG team is fully committed to addressing this problem and is working on a fix which will be publically available soon. We recommend refraining from installing Exchange 2010 SP1 on TMG machines until the fix is available. We will publish another blog post when the fix becomes available.

If you are already affected by this problem and need urgent assistance, please contact Microsoft support (http://support.microsoft.com).

Thank you for your patience,

The TMG team

Have you ever wondered what is the latest public hotfix or update available for ISA, IAG, TMG or UAG?

Good news!!!!!!!!!!!!!! We have launched the Update Center for Microsoft Forefront and Related Technologies where you can now go and check for the latest updates related to any forefront product. The site is located at http://technet.microsoft.com/en-us/forefront/ff899332.aspx and will give you information such as Product Version, Latest Service Pack, Latest Cumulative Update and General Guidance for all forefront technologies.

Author
Mohit Saxena

Reviewers
Yuri Diogenes
Ophir Polotsky

Customers who have an existing Websense® installation and want to move to Forefront Threat Management Gateway (TMG) as a Secure Web Gateway can migrate without creating rules from scratch.  SecureGUARD (http://www.secureguard.at/Shop/ ) provides the Web Configuration Migration Wizard, a migration tool that converts an existing Websense configuration to a Forefront TMG 2010 installation, with reduced setup time and optimized security.

The wizard enables you to export an existing WebSense configuration and import the configuration into Microsoft Forefront TMG. Based on the information retrieved, for each WebSense Policy an equivalent TMG rule set will be created.

The following components of the WebSense configuration can be migrated to TMG 2010:

• Users and Groups Information
• Computer and Network objects
• Time Schedules
• URL Category Mappings

Based on the information retrieved for each WebSense Policy rule, corresponding TMG policy rules will be created. These rules will be configured with the appropriate User and Group Information, existing time schedules, computer and network objects and also URL categories. This mapping of the WebSense URL category to the matching Microsoft MSR (Microsoft Reputation Service) category will be done by the SecureGUARD Migration Wizard.

Recently I came across a scenario where we had a PPTP Site-to-Site VPN between two TMG servers. We were able to access the shares of one TMG server from the other but we were unable to access the shares in the opposite direction as shown in the figure below:

Live logging was enabled on TMG from Site B and there the following error was appearing:

The IP address 10.10.11.1 showing above in the above Log as the ‘Source’ is the IP address of Internal Network Card of the TMG server. We checked the Live Logging on the Remote TMG server from where we could access the shares on this Local TMG server and we noticed that in the ‘Source’ it was showing the IP address of the RAS interface of the server.

During the investigation process to understand why TMG (site B) was showing us the Internal IP of the TMG server in the ‘Source’ instead of the RAS interface IP we checked the Binding Order of the Network Cards and we came to know that the Network Card which was on the top of the list was a third Network Card installed on the server (as shown in the image below) where Local Area Connection 5 is the third Network Card:

We moved the Internal NIC (Local Area Connection 3) to the top of the Binding Order. After making this change, when we tried to access the shares on the remote site it worked just fine. At this time the logging was showing the successful access:

The take away from this post is: remember to always use the internal NIC on the top. Here are some other cases where bind order was the culprit:

Unable to install Forefront TMG 2010 – Error 0x80074e46
Another performance caveat when troubleshooting TMG or ISA slow browsing behavior
Troubleshooting Tips for VPN Client Access on ISA Server 2006

Author
Nitin Singh
Support Engineer
Microsoft CSS Forefront (ISA/TMG) Team

Technical Reviewer
Yuri Diogenes
Sr Support Escalation Engineer
Microsoft Forefront (ISA/TMG) Team

On this TMG quick tip the troubleshooting scenario is very straight forward, consider the following: TMG administrator is trying to create a TMG 2010 Standalone Array in Workgroup (or Domain) and he gets the error below while trying to join Server to an array:

After reviewing the TMG log file “ISA_JoinArray” log in %temp% folder he was able to see the following:

This particular scenario was caused by the fact that TMG servers were not on same build. One TMG server had SP1 installed and other one was without TMG SP1. After installing SP1 on other TMG server then we were able to join server to an Array successfully.

Notice that not only the TMG build version is the same, but the OS version also needs to be the same as showed in the unsupported configurations article, as shown below:

Author
Junaid Ahmad Jan
Security Support Engineer
Microsoft CSS Forefront Security Edge Team

Technical Reviewer
Yuri Diogenes
Sr Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

The Microsoft TMG/UAG Server support team in Stockholm is looking to hire a Full Time Employee. If you’d be interested to come and work with us and learn more about TMG/UAG internals than you probably would anywhere else in the world (apart from Redmond maybe), then this might be the job for you.

Although you’ll get some training and access to innumerable amounts of internal only documentation, we’re looking for someone very experienced who can hit the ground running and could be expected to work the majority of our easier support cases without too much assistance. Therefore you’d need a minimum of 5 years’ experience (preferably more) working with ISA/TMG/UAG Server day to day in an administrative capacity (probably as a production administrator or developer ). Specialties of particular interest to us would be:

Setup / configuration / migration

Administration of ISA/TMG and ideally IAG/UAG

Good Network skills like reading network traces and such

Crashes, Hangs and Unexpected behavior

Beyond that if you have experience of other Microsoft server products (Windows, SharePoint, Exchange), windbg, c# or c++, it won’t do you any harm.

If you think you’d be interested, please send a brief summary of your experience and a CV in an email to the manager in Stockholm, Rolf Henningson. His email is rolfhe(at)microsoft.com
Note:  You need to be resident in Sweden (Swedish language or other Nordic languages would be of use, but are not essential) and available to work full time in Kista / Akalla.

Recently we’ve seen some cases where users were unable to download files larger than 4GB when they were using ISA 200x as a forward proxy server. When the user tried to download a large file, e.g. an ISO image, they would see the download stop after the client had downloaded only a small chunk of data.

When we analyzed the problem we found that the issue was related to the Content-Length HTTP header. Due to a design limitation, ISA 200x is unable to correctly parse the large Content-Length header and convert it to an Unsigned Integer 32-Bit value. For those of you, not too familiar with coding, the maximum value for UInt32 is 4.294.967.295. If the parsed Content-Length header exceeds 4.294.967.295, then the download will stop after the client has downloaded the remainder of the overflowed integer. E.g. If the file size is less than 8GB the amount of data downloaded would be Filesize-~4GB.

These are the HTTP response headers you might see in a repro. As you can see we sent the client the full content length.

HTTP/1.1 200 OK
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Content-Length: 4313199008
Via: 1.1 ISA2006EE1
Date: Tue, 24 Mar 2009 17:11:05 GMT
Content-Type: *
ETag: "f5f31f26a0acc91:688"
Server: Microsoft-IIS/6.0
Cache-Control: max-age=2592000
Last-Modified: Tue, 24 Mar 2009 16:46:59 GMT
Accept-Ranges: bytes
X-Powered-By: ASP.NET

In this case the download will stop after getting ~18,223,284 bytes (4,313,199,008 bytes - 4,294,967,296 bytes = ~18.xxx.xxx bytes)

Microsoft has accepted that this is a limitation in the following products running on 32-bit operating systems:

• Microsoft Internet Security & Acceleration Server 2000
• Microsoft Internet Security & Acceleration Server 2004
• Microsoft Internet Security & Acceleration Server 2006

To avoid facing this problem, please consider using one of the following workarounds:

1. Upgrade to Forefront Threat Management Gateway 2010 – As TMG will only run on 64-bit Windows operating systems, the limitation of a 32-bit variable is no longer an issue.

2. Use FTPoverHTTP to download big files in ISA 200x – As FTPoverHTTP uses different mechanisms to download big files, the issue doesn’t occur when you’re using FTPoverHTTP.

3. Use Download Managers – Most download managers will split downloads into multiple streams using byte-ranges, avoiding the big content length headers.

Also, please be aware that Internet Explorer 7 only supports downloading of files up to 4GB.

Author

Philipp Sand

Microsoft CSS Forefront Security Edge Team

Technical Reviewer

Lars Bentzen

Escalation Engineer - Microsoft CSS Forefront Security Edge Team

The Forefront TMG SDK documentation is live on MSDN. This is the most up-to-date release of the documentation, and includes a linked list of New COM Elements in Forefront TMG.

You can download the SDK here.

Nathan Bigman, Content Publishing Manager

After you configure TMG to log into a SQL database, you may have noticed, that some logging fields like Malware Inspection Reason, IPS Scan result or URL Category, do not contain the string that can see in the live logging or in the integrated TMG Reports (see screenshot below), but just a number.

TMG only logs those numbers, because it’s way more faster to query and index numbers, and numbers take significantly less space in the data base.

The problem you run into, when you want to create any reports based on the data TMG did log, is quite obvious… where do I get the information which number represents which value?

Good news is, that most of this information is very well documented in the TMG SDK:

URL Categorization Reason

For detailed information: http://msdn.microsoft.com/en-us/library/ff796883.aspx

Malware Inspection Threat Level

For detailed information: http://msdn.microsoft.com/en-us/library/dd447138.aspx

Malware Inspection Action Reason

For detailed information: http://msdn.microsoft.com/en-us/library/dd447105.aspx

Malware Inspection Action

For detailed information: http://msdn.microsoft.com/en-us/library/dd447104.aspx

IPS Scan Result

For detailed information: http://msdn.microsoft.com/en-us/library/dd436151.aspx

Unfortunately Microsoft didn’t provide any documentation about the URLCategories … until now ;-)

In the following you can see the list of all mappings for the URL Categories used in TMG.

It’s possible, that Microsoft extends these categories in the future, however if Microsoft adds categories, these will be added at the end of the list, in order to prevent that the current mappings become invalid.

I hope this information will help configuring your reporting!

Author

Philipp Sand

Microsoft CSS Forefront Security Edge Team

Technical Reviewer

Thomas Detzner

Escalation Engineer - Microsoft CSS Forefront Security Edge Team

We are happy to announce the availability of Software Update 1 for Microsoft Forefront TMG SP1.

This update contains the following enhancements:

• SafeSearch Enforcement. Forefront TMG can enforce blocking adult text, images and videos from search results by popular search engines. SafeSearch can be enforced on specific groups or to the entire organization.
• Including non-primary URL filtering categorizations. Forefront TMG uses an algorithm (described in this post) to select a URL’s “primary” category from among up to four categorizations provided by Microsoft Reputation Services (MRS). In Update 1 you can control access to sites that match any of the non-primary categorizations provided by MRS. For example, a URL with a primary categorization of News can now match a rule by any of its non-primary categorizations (such as Web Mail).
• Support for Exchange 2010 SP1 – This update provides support for Exchange 2010 SP1, including a fix to the problem reported in this post.
• Bug fixes and various other improvements

In the coming days we will publish additional posts going into more details about these enhancements.

The update is available on the Microsoft download center. Please note it must be installed on top of TMG 2010 SP1. If SP1 is not already installed, please make sure to install if first.

The update is currently available in English, but additional languages will be available on the download center soon.

We hope you will find the update useful.

The Forefront TMG team

When Forefront TMG queries MRS for a URL categorization, the result may include multiple categories (for example, a site can be categorized as both a Portal and Web Mail site). By default Forefront TMG uses the most significant category (otherwise known as the primary category) according to the category precedence list. In Forefront TMG Update 1, we have added the ability to configure Forefront TMG to use the non-primary categories as well. This ability applies to the following elements that utilize URL categories:

· Destination list (the To: tab) in access rules (per rule)

· Destination Exceptions in malware inspection (per array)

· Destination Exceptions in HTTPS outbound inspection (per array)

Configuring multiple categories

There is a checkbox in the designated property page, called "For URL filtering, apply to non-primary categorizations". When checked, all categories returned for the queried URL will be considered by the appropriate policy element during policy evaluation.

Note: The default configuration for these elements is to apply the primary category only (as in previous versions).

Examples

For example, assume that the URL www.contoso.com is categorized by MRS as Alcohol, Streaming Media and News:

The exclamation mark icon indicates the category that Forefront TMG has determined as the primary category, and the check mark icons indicate the non-primary categorizations.

Assume also that we have a rule denying access to Alcohol, News and Streaming media URL categories. Now let's see how the rule engine outcome changes according to the new functionality and different categories in the list:

Each Forefront TMG element dealing with multiple categories may match a different category. For example, if destination exceptions in malware inspection contain News category and exceptions in HTTPS outbound inspection contain Streaming Media category, a request to www.contoso.com would both be excluded from malware inspection and HTTPS outbound inspection.

Logging and Reporting

When you include non-primary categorizations, there is an effect on logging and reporting. Instead of logging the primary category, Forefront TMG logs the category that appears in both matched rule and MRS result that has the highest place in the precedence list. For example, if the matched rule contains a list of categories including News, Sports and Travel categories and the requested URL was categorized as News, Travel and Chat, the logged category would be the most significant of News and Travel (which is Travel). Reporting utilizes logged categories, so according to multiple categories configuration state, reported categories may vary for the same traffic logged with or without non-primary categorizations.

URL categories applied to malware inspection and HTTPS outbound inspection aren't logged.

Author: Dima Datsenko

Reviewers: Ori Yosefi, David Strausberg

Forefront TMG can now automatically block adult text, images, and videos from search results by major web search engines. The same SafeSearch feature that users can activate in their browsers can now be enforced on Forefront TMG, and applied to groups of users or to the entire organization.

When SafeSearch is enabled on Forefront TMG, the following happens:

· When a user submits a query to a major search engine, Forefront TMG modifies the query string, causing the search engine to treat the request as a SafeSearch request, and return filtered results.

· End-users cannot receive unfiltered content, even if they try to disable the feature in their browsers.

· SafeSearch is enforced over secure connections when HTTPS inspection is enabled on Forefront TMG. If a user establishes an HTTPS session with a search engine, Forefront TMG strictly enforces SafeSearch results.

This functionality is especially useful to schools and other organizations that want to block inappropriate web content.

Configuring SafeSearch

To enable SafeSearch, do the following:

1. In the Forefront TMG Management console, click the Web Access Policy node, and in the Tasks pane, click Configure SafeSearch.

2. On the General tab, click Enable SafeSearch.

3. If you want to disable SafeSearch enforcement for certain authenticated users, click on the Users tab and add the users or user groups.

Note: You must enable URL filtering to use the SafeSearch feature on Forefront TMG, because SafeSearch makes use of the Search Engines URL Category.

SafeSearch System Policy Rule

When SafeSearch is enabled for the first time, a system policy rule is created. This rule serves as a container for the user white list and handles authentication when the list is not empty. The rule has the following properties:

• Protocols: HTTP/HTTPS
• Source: Internal
• Destination: Search Engines (URL Category)
• Users: All Users with exclusion of users from the white list

After the rule is created for the first time, enabling or disabling SafeSearch will affect the rule state (enabled/disabled).

Enforcement is performed only for traffic matching this rule. The rule is identified by its internal ID and can only be created by enabling SafeSearch in the Management console, or by calling ConfigureSafeSearchRule in COM:

interface IFPCPolicyRules2 : IFPCEEPolicyRules

{

HRESULT ConfigureSafeSearchRule([out,retval] IFPCPolicyRule** ppVal);

};

This COM function returns a newly created or already existing SafeSearch rule, while resetting all its properties to SafeSearch rule defaults. The default setting for this rule is to enforce SafeSearch for all users, but it can be configured to exclude specific users or user groups.

Static Configuration

The feature has a configuration file “SafeSearchConfiguration.xml”, located in the installation directory:

<Configuration>

    <provider domainPattern=".google." safeSearchSuffix="&amp;safe=active" >

        <searchQuery pattern="/search?" />

        <searchQuery pattern="/images?" />

    </provider>

    <provider domainPattern=".yahoo.com" safeSearchSuffix="&amp;vm=r" >

        <searchQuery pattern="/search?" />

        <searchQuery pattern="/search;" />

        <searchQuery pattern="/search/images?" />

        <searchQuery pattern="/search/images;" />

        <searchQuery pattern="/search/video?" />

        <searchQuery pattern="/search/video;" />

    </provider>

    <provider domainPattern="www.bing.com" safeSearchSuffix="&amp;adlt=strict" >

        <searchQuery pattern="/search?" />

    </provider>

</Configuration>

SafeSearchConfiguration.xml can be altered to support additional search engines (by adding a new provider) or changing a level of enforcement (e.g., from strict to moderate). If altered, the file must be manually distributed over all members of the affected array and the firewall service must be restarted.

Author: Dima Datsenko

Reviewers: Dotan Elharrar, David Strausberg

Exciting times in the world of Forefront TMG literature!

In English:

Deploying Microsoft Forefront Threat Management Gateway by Yuri Diogenes and Dr. Tom Shinder, which can be preordered here.

In German:

Microsoft Forefront Threat Management Gateway – Das Handbuch von Marc Grote, Christian Gröbner, und Dieter Rauscher, which can be ordered here.

In Hungarian:

A kapun túl by Gál Tamás, which is available as a free download.

Enjoy!

Nathan Bigman, Content Publishing Manager

1. Introduction

Enhanced Malware Protection (EMP) is the mechanism that Forefront TMG uses to block malware coming through Web access. EMP provides a single point of Malware scanning that ensures all downloaded files are scanned with latest malware definitions. In summary the scanning process occurs in the following manner:

• HTTP traffic is intercepted by a web filter
• The content is accumulated in memory or on disk (depending on size)
• Actual scanning is handled by Microsoft Malware Protection Engine (MPEngine)
• Engine and signature updates are downloaded by the update center from Microsoft Update and loaded by the EMP scanner without service interruption

Note: for more information on EMP process review the article “Monitoring Malware through the Edge with Microsoft Forefront Threat Management Gateway” at Microsoft Technet.

This post is about a scenario where when the user is trying to download a file and you receive the error message below:

On the event viewer you will also notice the event 23461, which says: “The client ClientAddress exceeded the per-client accumulation limit for malware inspection. Requests that generate this event are blocked.”

2. Customizing the Storage Settings

The EMP Resource Allocation manager is responsible for calculate user’s quota while accumulating and scanning requests. The user’s quotas are:

For scenarios where such error message is appearing and users are unable to download content, you can use the counters below to find out if the limits are reached:

 
Note: for more information on Malware Inspection counters review the Forefront TMG perfmon counters here.

To change the default user’s quota, refer to Managing temporary storage settings article, there you will find the scripts to view and change the default configuration.

Authors

Bala Natarajan
Sr Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

Yuri Diogenes
Sr Security Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team

The Microsoft TMG/UAG Server support teams in Reading and Munich are looking to hire a Full Time Employee. If you’d be interested to come and work with us and learn more about TMG/UAG internals than you probably would anywhere else in the world (apart from Redmond maybe), then this might be the job for you.

Although you’ll get some training and access to innumerable amounts of internal only documentation, we’re looking for someone very experienced who can hit the ground running and could be expected to work the majority of our easier support cases without too much assistance. Therefore you’d need a minimum of 5 years’ experience (preferably more) working with ISA/TMG/UAG Server day to day in an administrative capacity (probably as a production administrator or developer ). Specialties of particular interest to us would be:

Setup / configuration / migration

Administration of ISA/TMG and ideally IAG/UAG

Good Network skills like reading network traces and such

Crashes, Hangs and Unexpected behavior

Beyond that if you have experience of other Microsoft server products (Windows, SharePoint, Exchange), windbg, c# or c++, it won’t do you any harm.

If you think you’d be interested, please send a brief summary of your experience and a CV in an email to Dahna Heineman dahnah@microsoft.com.

 Detailed Job description UK: https://careers.microsoft.com/JobDetails.aspx?ss=&pg=0&so=&rw=1&jid=26831&jlang=EN

Detailed Job Description Germany: https://careers.microsoft.com/JobDetails.aspx?ss=&pg=0&so=&rw=3&jid=26313&jlang=EN

Note:  for the position in Munich, Germany fluent German language skills are required.