Extreme fragmentation when writing large bkf files to comp..

Hi Folks,

I'm experimenting with using NTFS compression when backing up across the network
to a USB2.0 drive on a workstation. Backup software is the native ntbackup, the
uncompressed bkf files are around 38GB, the USB drive is 250GB. Ntbackup is
running on a Win2k SP4 server, and the USB drive is on a WinXPP SP2 machine.

Just to verify that compression was working, I copied an existing 16.8GB bkf
file to the empty compressed USB drive, where its size on disk was 14.1GB. Disk
Defragmenter sees the copied bkf file as having 2,587 fragments.

Next, I ran a backup across the network to the same compressed USB drive. The
result is a 37.9GB bkf file whose size on disk is 29.2GB. Disk Defragmenter
sees this file as having 151,238 fragments.

Merely deleting a file like this takes 10-20 minutes of maxed-out cpu.

So I'm wondering whether this is expected behavior with large files and NTFS
compression?

Thanks

Frank
Frank Denman
Denman Systems
news.DeleteThis@denmansystemsx.com
[Please delete the "x" from my email address]

> sees this file as having 151,238 fragments.

In the graphical representation does it still show one contiguous block
occupied by the file?

When NTFS compresses a file it divides the uncompressed data into units of
16 clusters. If its able to compress it by more than one cluster it
compresses it, otherwise it is left uncompressed. Due to this method of
division NTFS may record each 16 cluster units as individual "runs"*.
Defrag utilties frequently queries for number of runs in a given file to
determine the number of fragments in the file. In case of normal files if
NTFS finds a contiguous free space big enough for the file it will allocate
it in a single run. With the compressed files, however, even if it finds
contiguous space it will record these chunks as runs. These runs may be
right next to each other but defrag utilties may consider them to be
fragments. In my opinion future versions of defrag utility should consider
not reporting file as fragmented if runs themselves are contiguous.

The delete time you mention below looks extreme to me and I dont have a good
explaination as to why it may take that long. If I come across it I will
let you know.

Also note: Depending upon yor cluster size you may be approaching the
limitation on compressed file size. It is between 30GB to 69 GB.

*run: Extent. Read more on NTFS disk structure.
--
This posting is provided "AS IS" with no warranties, and confers no rights.
OR if you wish to include a script sample in your post please add "Use of
included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm"



"Frank B Denman" <news.TakeThisOut@denmansystemsx.com> wrote in message
news:a16sh25j8chpu0252h4h41uvfrv878uvru@4ax.com...
> Hi Folks,
>
> I'm experimenting with using NTFS compression when backing up across the
> network
> to a USB2.0 drive on a workstation. Backup software is the native
> ntbackup, the
> uncompressed bkf files are around 38GB, the USB drive is 250GB. Ntbackup
> is
> running on a Win2k SP4 server, and the USB drive is on a WinXPP SP2
> machine.
>
> Just to verify that compression was working, I copied an existing 16.8GB
> bkf
> file to the empty compressed USB drive, where its size on disk was 14.1GB.
> Disk
> Defragmenter sees the copied bkf file as having 2,587 fragments.
>
> Next, I ran a backup across the network to the same compressed USB drive.
> The
> result is a 37.9GB bkf file whose size on disk is 29.2GB. Disk
> Defragmenter
> sees this file as having 151,238 fragments.
>
> Merely deleting a file like this takes 10-20 minutes of maxed-out cpu.
>
> So I'm wondering whether this is expected behavior with large files and
> NTFS
> compression?
>
> Thanks
>
> Frank
> Frank Denman
> Denman Systems
> news.TakeThisOut@denmansystemsx.com
> [Please delete the "x" from my email address]