Troubleshooting KMS Activation: Resolving DNS Error 0x80072338 on Windows Server

Table of Contents

Key Management Service (KMS) activation is a method used by volume licensing customers to activate Windows and Microsoft Office products within their organizational network. A KMS host server is configured to serve activation requests from KMS clients. For clients to locate the KMS host automatically, the host typically registers a Service Location (SRV) record in the Domain Name System (DNS). This integration is crucial for the seamless operation of KMS in many environments. However, the process of registering this SRV record is not always straightforward and can sometimes fail, leading to activation issues for clients.

When setting up or configuring a KMS host on a Windows Server, one might encounter specific errors logged in the system’s event logs. These errors provide vital clues about the root cause of any failure in the KMS setup or operation. A common issue relates to the KMS host’s interaction with the corporate DNS infrastructure, particularly concerning the creation or update of the necessary SRV records. Identifying the specific error code associated with the failure is the first step in diagnosing the problem.

Symptoms: Encountering Event ID 12293¶

A clear indication that there is a problem with the KMS host registering its presence in DNS is the appearance of Event ID 12293 in the Application event log. This event is sourced by Security-SPP, which relates to the Software Protection Platform service responsible for handling licensing and activation. The event message explicitly states that publishing the Key Management Service (KMS) to DNS failed for a specified domain, such as contoso.com in a typical example.

Accompanying the Event ID 12293 is an informational code, 0x80072338. This hexadecimal code is a standard Windows error code that provides more specific details about the failure. In this context, 0x80072338 translates to DNS_ERROR RCODE_BADSIG, indicating that a DNS signature failed to verify. While the error message mentions a signature failure, in the context of KMS SRV record updates via Dynamic DNS (DDNS), this error frequently points towards a permission issue rather than a cryptographic signature validation failure in the traditional sense. It suggests that the attempt to update or register the record was rejected by the DNS server because the requesting entity lacked the necessary authority.

The full event details might look something like this:

Source: Security-SPP
Event ID: 12293
Publishing the Key Managment Service (KMS) to DNS in the contoso.com domain failed.
Info: 0x80072338

0x80072338: DNS_ERROR RCODE_BADSIG
DNS signature failed to verify.

This event essentially means the KMS host tried to tell the DNS server “I am a KMS host, here is my address,” but the DNS server refused the registration attempt because the KMS host didn’t have the proper authorization to make changes to the specific record name it was targeting. Without this successful registration, KMS clients relying on DNS lookup will be unable to automatically discover the KMS host, potentially preventing them from activating.

Cause: Insufficient Permissions for DNS Updates¶

The root cause of Event ID 12293 with error code 0x80072338 when setting up a KMS host is almost always related to insufficient permissions for the KMS host computer account to modify or update the relevant DNS record. The specific record in question is the Service Location (SRV) record used by KMS clients to find KMS hosts. This record is standardized and uses the name _VLMCS._TCP.

In a standard Dynamic DNS (DDNS) configuration within an Active Directory environment, the first entity (typically the first KMS host) to successfully register an SRV record with a particular name, such as _VLMCS._TCP.<domain>, becomes the “Creator/Owner” of that specific record name. Subsequent attempts by other computers to register or update an SRV record using the same name (_VLMCS._TCP) will be denied by default unless those computers have been explicitly granted permissions to do so. This security model prevents unauthorized machines from hijacking or modifying service advertisement records for critical services like KMS, LDAP, Kerberos, etc., which also use SRV records.

When a new KMS host is brought online and attempts to register its _VLMCS._TCP SRV record via DDNS, it finds that a record with that name already exists (created by a previous KMS host or perhaps a placeholder). Because the new KMS host is not the “Creator/Owner” of the _VLMCS._TCP name entry in DNS and hasn’t been explicitly granted write permissions, its DDNS update request for that name is rejected by the DNS server. The DNS server interprets this unauthorized update attempt as a signature mismatch or validation failure in the context of the DDNS security handshake, resulting in the 0x80072338 error code reported by the KMS host’s Software Protection Platform service. Essentially, the security mechanism designed to protect critical DNS records prevents the new KMS host from advertising itself using the standard, pre-existing record name.

Resolution: Granting DNS Record Permissions¶

To resolve the 0x80072338 error and allow the new KMS host to successfully register its _VLMCS._TCP SRV record in DNS, you must explicitly grant the computer account of the new KMS host the necessary permissions on the existing _VLMCS._TCP record within your DNS zone. This tells the DNS server that this specific computer is authorized to make updates to that particular record name. The process involves modifying the Access Control List (ACL) for the DNS record object.

Follow these steps carefully within your DNS management console:

1. Open DNS Manager: Access the DNS management tool on a server that hosts the relevant DNS zone or from a management workstation with appropriate administrative privileges.
2. Navigate to the Forward Lookup Zone: In the DNS Manager console tree, expand the server name, then expand Forward Lookup Zones. Locate and expand the forward lookup zone corresponding to your domain (e.g., contoso.com).
3. Locate the _tcp Folder: Within your domain’s forward lookup zone, find and expand the _tcp folder. This folder typically contains SRV records for various services.
4. Find the _VLMCS Record: Scroll through the list of records in the _tcp folder to locate the _VLMCS record. There might be multiple entries under this name if you have several KMS hosts configured, but they are all associated with the single _VLMCS._TCP name. Right-click on the _VLMCS entry (it might just show _VLMCS under the _tcp folder) and select Properties.
5. Modify Security Permissions: In the properties window for the _VLMCS record, navigate to the Security tab. This tab shows the permissions currently assigned to the DNS object representing the _VLMCS._TCP record name.
6. Add the New KMS Host Computer Account: Click the Add… button. In the “Select Users, Computers, Service Accounts, or Groups” dialog box, click Object Types… and ensure that Computers is selected. Click OK. Then, in the “Enter the object names to select” field, type the computer name of your new KMS host (e.g., KMSHOST2$). Remember to include the dollar sign$` at the end of the computer name as this is how computer accounts are represented. Click Check Names to verify the name resolves correctly. Click OK.
7. Grant Full Control: With the new KMS host computer account selected in the “Group or user names” list, check the Allow box for Full Control under the “Permissions for [KMS Host Computer Name]” section. Granting “Full Control” ensures the KMS host has sufficient rights to create, modify, and delete its specific entry under the _VLMCS._TCP name group. Click Apply, then OK.
8. Restart Relevant Service on KMS Host: Return to the new KMS host server. Open the Services console (services.msc). Locate either the Software Protection service (sppsvc) or the Server License Manager service (slsvc). The exact service name might vary slightly depending on the Windows Server version and specific configuration, but sppsvc is the most common for KMS in modern Windows Server releases. Restart the identified service. Restarting the service will prompt the KMS host to re-attempt the DNS registration process with its newly acquired permissions.

After completing these steps, check the Application event log on the KMS host again. If the permissions were applied correctly and there are no other underlying issues, you should see Event ID 12294 logged by Security-SPP, indicating that publishing the Key Management Service to DNS was successful.

More Information: Understanding KMS, DNS, and SRV Records¶

Understanding the interaction between Key Management Service (KMS) and DNS is crucial for diagnosing and resolving activation issues like the 0x80072338 error. KMS is a distributed activation service that allows organizations to activate systems within their own network, eliminating the need for individual computers to connect to Microsoft for activation. A central KMS host activates KMS clients (Windows or Office installations configured to use KMS).

For KMS clients to find a KMS host on the network, they rely on DNS. Specifically, KMS clients query DNS for Service Location (SRV) records. SRV records are a type of DNS resource record that specifies the location, i.e., the hostname and port number, of servers for specified services. They are commonly used by various protocols to locate domain controllers, messaging servers, and other services. For KMS, the standardized SRV record name is _VLMCS._TCP.<DomainName>, where <DomainName> is the DNS domain suffix of the client computer.

For example, a client in the contoso.com domain will look for an SRV record named _VLMCS._TCP.contoso.com. This record lists the hostnames (Target) and TCP port numbers (default 1688) of available KMS hosts. The client then attempts to contact a KMS host from the list returned by the DNS query.

When a KMS host is configured, it attempts to register its own information as an entry under the _VLMCS._TCP.<DomainName> SRV record in the specified DNS zone using Dynamic DNS (DDNS). DDNS allows authorized machines to update DNS records automatically. However, for security reasons, the first machine that registers a record name (like _VLMCS._TCP) typically becomes the “Creator/Owner” of that name entry. This owner has default rights to modify records associated with that name, while other machines do not.

Think of the _VLMCS._TCP.<DomainName> entry in DNS not as a single record for one server, but as a container or group name under which multiple SRV records (each pointing to a different KMS host) can be listed. When a new KMS host comes online, it wants to add its own server details to this list associated with the _VLMCS._TCP name. If a previous KMS host created the _VLMCS._TCP name entry, the new host’s attempt to update this existing entry will be denied by default due to lacking permissions, leading to the 0x80072338 error. Granting the new KMS host’s computer account “Full Control” on the _VLMCS record object in DNS explicitly authorizes it to add or modify entries under that name, resolving the permission issue.

Successful DNS publishing by the KMS host is indicated by Event ID 12294 from the Security-SPP source in the Application event log. This event confirms that the KMS host has successfully registered its SRV record, allowing KMS clients to potentially discover it via DNS lookups. While resolving the 0x80072338 error fixes the DNS registration permission problem, it’s important to ensure other factors like network connectivity between clients and the KMS host, firewall rules (TCP port 1688), and time synchronization (clock skew) are also correctly configured for successful KMS activation.

Implementing robust DNS practices, such as ensuring scavenging is properly configured to remove stale SRV records from decommissioned KMS hosts and reviewing the security permissions on critical service records like _VLMCS._TCP, is vital for maintaining a healthy KMS activation environment. The 0x80072338 error serves as a reminder of the importance of understanding the underlying DNS security model and how it interacts with services like KMS.

Summary of Key Concepts¶

• KMS (Key Management Service): Volume activation method using an internal host.
• DNS (Domain Name System): Resolves hostnames to IP addresses; used by KMS clients to find KMS hosts.
• SRV Records: Special DNS records (_VLMCS._TCP) listing KMS host locations.
• DDNS (Dynamic DNS): Allows hosts to automatically update DNS records.
• Creator/Owner: The first entity registering a record name in DDNS gains ownership and default exclusive update rights.
• Error 0x80072338 (DNS_ERROR RCODE_BADSIG): In this context, often indicates a permission failure during a DDNS update attempt for the _VLMCS._TCP record name.

By granting explicit permissions to the new KMS host’s computer account on the existing _VLMCS DNS record object, you bypass the default ownership restriction, allowing the KMS host to successfully register itself and enable automatic discovery by KMS clients. This is a specific and common solution for this particular error code during KMS setup.

Have you encountered this specific DNS error when setting up your KMS host? How did you troubleshoot and resolve it? Share your experiences or any questions you might have in the comments below!