A look at SMB Compression

SMB compression – a rocky start

Many people got their first exposure to SMB compression through CVE-2020-0796, which is about a Windows SMBv3 Client/Server Remote Code Execution Vulnerability, which addresses this vulnerability. That is about a remote code execution vulnerability in how the Microsoft Server Message Block 3.1.1 (SMBv3) protocol handles specific requests. Microsoft fixed this in an update on March 12th, 2020—KB4551762 (OS Builds 18362.720 and 18363.720) (microsoft.com).

At that time, SMB compression, while already in the code of specific Windows releases (Windows 10, version 1903/1909 & Windows Server version 1903/1909), was not yet widely available, let alone known. So, its use was minimal, if used at all, bar by some technical people in the know for testing purposes.

But despite this somewhat “exceptional” entry on the stage, it is a welcome capability to have in the toolbox for various use cases. While network bandwidth keeps growing, so does the amount of data and the size of files. Using compression to save on bandwidth and storage is nothing new in this industry. Think about WinZip, 7-Zip, or WinRAR or backup software like Veeam Backup & Replication leveraging compression to get the job done faster and with less bandwidth or storage space needed. Considering these benefits, it is no surprise that this capability has come to SMB 3.

SMB compression

SMB compression allows a user or an application to request the compression of files before sending them over the wire. When the server is capable of SMB compression, it accepts the request. Examples are when mapping a share or when using tools like Xcopy, Robocopy, or PowerShell. Note that this does not mean the connection will fail if the server does not support SMB compression. No, in that case, the server ignores the request. Either because it has no clue about it or because an administrator configured it to reject compression requests.

Yes, an administrator can configure both client and server to reject SMB compression requests. Similarly, an administrator can configure a client to request SMB compression. That comes in handy for end-users that don’t have to worry about asking for it and with 3^rd party tools that have no parameter to request it themselves.

Compressing files saves on the amount of data you have to transport over the wire. The higher the compression ratio is, the higher those savings are. Compressed files consume less bandwidth and, as such, take less time to transfer. The cost for this is extra CPU cycles. But SMB compression is intelligent about that. It will evaluate its compression success factors and not use compression when the return on investment does not justify the cost.

Typical end-user network connectivity is 1Gbps LAN or Wireless which can be anything in between 20 Mbps (802.11d) and 200 Mbps (802.11ac) in most environments, using practically achievable bandwidth, not theoretical ones. With 802.11ax on the rise, this goes up to 2 Gbps. That is nice, but that is not widespread, and the Wi-Fi NICs need to catch up. In addition, client networks typically are limited in bandwidth and throughput and can be congested at times. On high-speed networks (10/25/40/50/100Gbps), this is less of a concern but might very well still come into play, and it pays to preserve bandwidth and optimize transfer times there as well.

SMB compression requirements and characteristics

Both Windows Server 2022 and Windows 11 support SMB compression

SMB compression supports:

• Algorithms XPRESS (LZ77), XPRESS Huffman (LZ77+Huffman), LZNT1 (see [MS-XCA]: Xpress Compression Algorithm | Microsoft Docs) or PATTERN_V1* (see [MS-SMB2]: Algorithm for Scanning Data Patterns V1 | Microsoft Docs). XPRESS is the default.
• SMB signing and SMB encryption
• Supports SMB over QUIC
• Supports SMB Multichannel

SMB Compression does not support SMB Direct over RDMA (for now).

No SMB Direct?!

As my readers know, I am an early adopter of RDMA with SMB Direct, and I have always had an affinity for this technology. So naturally, anytime new features or capabilities appear that cannot leverage SMB Direct, I am a bit upset. Now, that is not always rational, but hey, I am human and far from perfect.

For now, SMB compression doesn’t support SMB Direct. So, when SMB Direct is possible between the client and the server, and the client requests compression to a server that supports it, SMB compression will not kick in. So you could say that SMB Direct trumps SMB compression. But I did indeed say “for now” because support for SMB compression with SMB Direct will arrive later. When? I don’t know. Microsoft does not share its release plans with me.

Benefits

SMB compression allows a user or an application to compress files before transferring them over the network. The savings in bandwidth and transfer times are apparent to most of us.

Applications can benefit from this, and a prime example is when using SMB for live migration. Until now, it was only TCP that supported compression for Hyper-V live migration. That’s nice and very effective, but you lose SMB Multichannel, which allows you to aggregate bandwidth and provides redundancy. Well, no longer. You now can leverage SMB with Multichannel and compression.

As a user, you also benefit. You no longer have to manually select and compress files and folders to transfer them significantly faster on slower or congested networks. That makes sense. It relieves the users from work through automation, either actively or automatically. That is especially true if you do not want to store the data compressed. That means you would then have to decompress it again, which is even more manual work.

The various ways of configuring SMB compression

You configure SMB compression from both the client and the server. The client and server have to be Windows Server 2022 or Windows 11, but either can have the client or server role while connecting to a file share and transferring data.

SMB compression on file shares

You can configure file shares to request SMB compression when clients connect to them. That means you have to opt-in on the server-side. It is not enabled by default on a file share unless you configure it explicitly. To do so, you use either PowerShell or Windows Admin Center. That setting works with UNC paths or mapped shares. Note that you can configure SMB compression from both a client and server perspective to control whether they will use it.

PowerShell

In an elevated PowerShell console, run New-SMBShare with -CompressData set to $True to create a new file share. Doing so forces a client that supports SMB encryption to request it while connecting to the file share.

You can also change this setting for an existing file share with Set-SMBShare, as shown below.

Using Windows Admin Center

• With Windows Admin Center, you can set the data compression option on or off on new or existing file shares.
• Open Windows Admin Center and connect to your server.
• Navigate to “Files and File Sharing” in the left menu pane.
• Click on “File Sharing,” where you can create a new or edit an existing file share
• You can check or uncheck the “Compress data” option and then click “Add” or “Edit” to confirm the setting

SMB compression on mapped file shares

To request that all data copied over a mapped drive should attempt compression, you have to configure it to do so. Such mapped drives will try SMB compressions via applications, users, and scripts that read and write to it. You can use PowerShell or NET USE to configure this.

Via PowerShell

Open a PowerShell console and run New-SMBMapping with the -CompressNetworkTraffic set to $True to create a new drive mapping to a file share.

Via NET USE

Open a CMD prompt and map a drive using NET USE with the /REQUESTCOMPRESSION parameter set to YES to create a new drive mapping to a file share.

Configuring always require or always reject SMB compression requests

You can configure an SMB client always to request compression and always to reject requests for compression. An SMB server you can only configure always to reject requests for compression.

Configure the SMB client always to request SMB compression

On the SMB client, you can configure the option always to request to compress SMB traffic. You cannot do this for the SMB server.

On the SMB client, start regedit.exe

1. Navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkstation
2. Create a new REG_DWORD value named “EnableCompressedTraffic”
3. Set the value to 1
4. Close regedit.exe

This setting takes effect immediately. There is no need to reboot.

Configure the SMB client never to use SMB compression

To ignore all requests for compression, whether it comes from a mapped drive, a Microsoft tool, application, or service, do the following. In an elevated PowerShell console, run Set-SMbClientConfiguration with –DisableCompression set.

You can also configure this via the registry, but the PowerShell configuration you saw above does not reflect this setting in the registry.

1. On the SMB client, start Regedit.exe.
2. Navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkstation\Parameters
3. Create a new REG_DWORD value named “DisableCompression”
4. Set the value to 1
5. Close regedit.exe

This setting takes effect immediately. There is no need to reboot the server

Configure the SMB server never to use SMB compression

For some reason (I think it is a work in progress), there is no PowerShell option to set this.

1. On the SMB server, start Regedit.exe.
2. Navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManServer\Parameters
3. Create a new REG_DWORD value named “DisableCompression”
4. Set the value to 1
5. Close regedit.exe

This setting takes effect immediately. There is no need to reboot.

Tooling that supports requesting SMB compression

Some file copy tools support requesting SMB compression via their parameters. The tools I know of for the moment are Robocopy and Xcopy.

I can envision some applications or services like Hyper-V with live migration to also do this. But for 3^rd party tools and file explorer or applications to use SMB compression, you should do what we showed you above. Map the file shares to drives with SMB compression (client), enable SMB compression on the file shares (server), and set SMB clients always to try SMB compression.

For your understanding, if you want 3^rd part file explorer, third-party copy tools, or applications to use compression, map the target shares with compression and enable compression on shares. You can also set SMB clients always to try SMB compression.

Robocopy

In a command prompt or PowerShell console, run Robocopy with the /COMPRESS flag.

Xcopy

In a command prompt or PowerShell console, run Xcopy with the /COMPRESS flag.

Controlling SMB compression behavior

SMB compression in the preview edition of Windows Server 2022 and Windows 11 clients and now in GA use a default algorithm that attempts to compress the first 524,288,000 bytes (500MiB) of a file during transfer. It then tracks the process to see that it compresses at least 104,857,600 bytes (100MiB) of the data within that 500-MB range. If it does not succeed, meaning it could only compress less than 100 MiB, SMB compression stops compressing the rest of the file. On the other hand, if at least 100 MiB gets compressed, SMB compression will compress the rest of the file.

The result is that huge files with many compressible data, VHD or VHDX virtual disk files come to mind, will very likely be compressed. Relatively small files, on the other hand, even with very compression-friendly data, probably will not compress.

The default behavior will change in the future GA editions of Windows Server 2022 and Windows 11. It will always compress when you have enabled compression. I think Microsoft determined this was the better default as the CPU overhead is not debilitating high. However, you will still be able to use the sampling configuration to control how SMB compression behaves. They changed this first in Windows 11 preview 22449 (September 2^nd, 2021).

If you want SMB compression to only compress under certain conditions, you can override the default settings. Here is an example.

• On the SMB client computer, start Regedit.exe.
• Navigate to KEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanManWorkstation
• Create a REG_DWORD value and name it CompressibilitySamplingSize
• Set this value to 4294967296 (decimal)
• Create a second REG_DWORD value and name it CompressibleThreshold
• Set the value to 1048576 (decimal)

This setting takes effect immediately. It does not require a reboot. The result of these two settings is as follows. SMB compression will attempt to compress the first 4 GiB of a file (CompressibilitySamplingSize). If it succeeds in compressing at least 100 MiB (CompressibleThreshold), it will compress the rest of the file as much as possible.

Does SMB Compression work?

Hell yeah! It sure does. Look at the example from my lab below. I have a server Node-B with a 1gbps NIC on which I host virtual machine images. I created a file share called VMImages, where I can copy VHDX files.

When I copy a VHDX from that share without compression enabled, I get the maximum speed you can expect to achieve over a 1Gbps link. 111MB/s is pretty much line speed.

In Performance Monitor, you can see that there is no SMB compression in play whatsoever.

But when I enable SMB compression, you can see the magic of compression in the image below. I can pull a 33GB VHDX file down at +/- 220 MB/s. Over a 1Gbps link. That’s not magic. That is SMB compression.

You can see the transfer speed is not such a beautiful flat liner compared to a copy job without compression. The CPUs have to get the work done compressing the data, so this is not unexpected. That brings us to the effect of compression on the CPU load.

At rest, my SMB file share server is doing next to nothing, as is evident from the screenshot below.

When I copy the 33GB VHDX file without compression, I get up to 5% CPU load. Nothing significant.

The load on the CPUs with SMB compression enabled while copying a VHDX file increases up to 20%. The CPU fluctuates between 14 -20 % during the copy job. Mind you; my lab server has rather old CPUs.

Now, sure compression consumes CPU cycles. But that is something for you to monitor and decide if that cost is worth the benefits of SMB compression. For many use cases, that will be the case. If not, well, don’t use SMB compression.

If you are wondering, yes, SMB compression can and does work when sending data to the file server share. As long as the client and server OS both support SMB compression and allow it, you can benefit from it.

At home, I have it enabled on my file shares. Yes, that’s one reason to have a small server instead of a NAS appliance; you get the latest SMB goodies as soon as they are available. Consider this a call to action for anyone involved in producing NAS appliances. Make sure your SMB 3 support is up to date and supports compression! That is an excellent capability to offer to your customers.

Next to that, I already have it enabled on some file shares at one organization. That way, as they roll out Windows 11, they will start enjoying the benefits. Meanwhile, we gain experience with SMB compression in a real-world environment.

Conclusion

SMB compression will prove itself to be a valuable capability for a variety of data copy scenarios. So start playing with this in the lab and try it out where and when you have the opportunity. I think you will find it beneficial.

An IT Pro always has data to copy, and this is a valuable tool to help us save time and bandwidth. For power users who need to copy files regularly, this is likely to be a time saver. They are often more bandwidth-constrained than CPU constrained.

I also like that it is configurable to make it only kick in based on sampling results. So, you have control to determine when it is beneficial, avoiding wasted CPU cycles.

One thing I noted is that SMB compression works when connecting to a UNC path. Thus, in contrast to SMB write-through, which is only available via a mapped drive configuration setting, SMB Compression is more versatile and automation-friendly in that matter.