Problems with USB drives
	There are many possible problems with USB drives. Here are some problems and solutions: Troubleshooting for USB drives

	General
	An USB pen drive (also called 'flash drive', 'memory stick' or in german 'USB-Stick') is drive and media in one. Therefore the drive letter comes and goes with it. In contrast a card reader stays attached and only the card is plugged and unplugged, so the drive letters stay. Because it's drive and media in one, each time you plug a pen drive for the first time the Windows plug'n play starts up a assigns the first available drive letter. When the USB has no USB serial then this happens again if it's attached to a different USB port. To get rid of the 'first free letter assignment' I wrote the USB Drive Letter Manager which works under Windows 2000, XP and higher.

	How XP assigns drive letters
	For a drive which XP has no drive letter assignment stored it assigns the first available drive letter. The search goes upwards in the alphabet and starts at 'A' for floppy drives, at 'D' for CD-ROM drives and at 'C' for all other types. Windows can save exactly one assignment per drive letter. So when a drive is attached again it gets the same letter as before. But the drive letter is not reserved while a drive is not present and therefore reassigned to another drive if required. When this happens the former assignment is overwritten and the first drive gets the first available drive letter again even its former drive letter is available when it's being attached. The former assignment just doesn't exist anymore. Network and substs drives are completely unconsidered by XP here. XP has no problem assigning a drive to letter which is currently used as network share! Microsoft knows that this is 'a problem' and since December 2007 a hotfix is available (WindowsXP-KB297694-x86-ENU.exe). The Service Pack 3 fixed it too (more or less). Not fixed this this scenario: Local drives C: and D:, USB drive at E:. USB drive is removed, a network share at E: created, the USB drive reattached... XP with SP3 assigns E: again to the USB drive, even E: is used as a network share. This is fixed under Vista. To get persistent drive letters for external drives assign one exclusive letter per drive. My USB Drive Letter Manager solves all this.

	Removable or what?
	USB drives can appear as 'Removable Drive' or as 'Local Drive'. The actual meaning of 'removable' is that the drive has a removable media, like floppy and ZIP drives or flash card readers. Even USB pen drives have no removable media they pretend to have by having the removable media bit (RMB) set in their hardware device descriptor. Strictly spoken this is just wrong but in fact nearly all USB flash drives have the RMB set while USB hard disks have not. Appearing as removable or local drive makes a difference under Windows. Here the differences for XP up to Windows 8: Removable Drive Local Drive GetDriveType returns DRIVE_REMOVABLE DRIVE_FIXED more than one partition/volume possible no yes Recycle Bin no yes autorun.inf - open= works no yes (until XP-SP1) 'Eject' in XP's Explorer context menu yes no Windows XP activates system restore for the drive no yes Windows creates folder 'System Volume Information' no yes Formatting with NTFS with policy 'Optimize for quick removal' no yes Formatting with NTFS with policy 'Optimize for performance' yes yes write cache active on FAT and policy 'Optimize for quick removal' no no write cache active on FAT and policy 'Optimize for performance' no yes write cache active on NTFS and policy 'Optimize for quick removal' yes yes write cache active on NTFS and policy 'Optimize for performance' yes yes Because of the ability to create multiple partitions on a non removable drive and getting a write cache under XP, some users are searching for pen drives which do not pretend to have a removable media. They are extremely rare, no major brands are available. But it's possible to turn a 'removable' drive into a 'fixed' one by a filter driver, see here.

	Cache or not
	Under Windows XP writing of lots of small files to a 'Removable' pen drive is extremely slow, while it's fast under Windows 2000. Writing large files there is no appreciable difference. Obviously Windows XP doesn't enable a write cache for USB drives that appear as 'Removable'. The option "Enable write caching on the disk" is grayed out for USB drive (this mean the hardware cache of the drive) and the removal policy setting ('Optimize for quick removal' or 'Optimize for performance') doesn't seem to make any difference, except that the latter enables the user to format 'Removable' USB drives with NTFS. But with NTFS Windows XP enables a write cache, writing small files becomes lightning fast. Update Feb 2011 I was pointed to the fact that by means of the registry value WriteCacheEnableOverride a write cache can be forced for any USB drive, even FAT formatted removeable drives. And it indeed works! I have made a litte tool: USB-WriteCache So, NTFS is not essential for having a write-cache. Use NTFS anyway for USB flash drives? Advantages: Write-cache on all computers NTFS stores small files together with the directory entry, this safes write accesses NTFS supports file compression, this safes write accesses too (for compressible files only) FAT/FAT32 becomes linear slower with the number of files in folder, NTFS does not NTFS supports files >4GB But actually NTFS isn't suitable for flash medias because as journalling file system it has some overhead that wears out flash memory. But Windows XP optimizes and bundles write accesses to pen drives only when they are NTFS formatted, so I see NTFS as the right choice. Furthermore NTFS stores small files together with the file information so they are written into the same flash block which is the best that can happen. A drawback with NTFS and flash medias is that NTFS saves the access time when you read a file, so a simple read access causes flash wear out. In fact it is not as dramatically because Windows writes the data not immediately onto the media. It does it when it has to update something else there, when one hour is over on when a media is unmounted. Actually I've never seen XP writing a last access time to a USB drive. The last access time is also updated on 'directory listing' (whatever this means). This can be disabled, unfortunately for all drives only: Create a new DWord value under [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem] called NtfsDisableLastAccessUpdate and set it to 1. Or download the following REG file and doubleclick it: NtfsDisableLastAccessUpdate1.reg Information: NtfsDisableLastAccessUpdate at Microsoft Technet To format a pen drive with NTFS there is no need to switch to 'optimize for speed'. Using the commandline tool CONVERT you can convert a FAT formatted drive to NTFS. Sample: convert u: /fs:ntfs But XP's CONVERT creates a NTFS with a cluster size of 512 Bytes when the FAT clusters are not 4K aligned. If the drive is FAT formatted under XP, then the clusers are 4K aligned and XP's CONVERT works fine. W2K's CONVERT seems to creates 512 Bytes clusters in any case. Using the Windows format dialog you can choose: 2048 or 4096 Bytes are ok. With a cluster size above 4096 Bytes file compression and encryption becomes unavailable. Of course a NTFS formatted drive doesn't work under Windows 95/98/ME. Furthermore devices with a standalone functionality as MP3 players or image tanks will not work with NTFS. If both, format and convert fail while formatting with FAT/FAT32 works, then the drive is a fake with a fraction of real memory only. The NTFS file system is written into the middle of the drive while the FAT is written to at the begin of the drive. So, if the drive is fake then the NTFS file system is written into non existent memory... The effect of having a write cache or not depends on the USB drive: Hard drives have an 2 or 8 MB of cache RAM which prevents to much performance impact. Today's USB flash have no such cache, so their access times are very important for their real live speed. By means of HD Tune you can check the read access times. Here a screenshot of the bad example 'Corsair Flash Voyager' (2GB) - have a look at the yellow dots above the diagram... It's from 2005, the current exemplars are better. To test the file cache behaviour I've made FileCacheTest It can read and write a test file using different flags for the API function CreateFile and using different block sizes. Remember the difference between writing a file for the first time and to an existing file: Writing the first time, the file size has to be adjusted after each block. The most remarkable effects under XP are: no buffering of accesses to the file system on FAT formatted removable USB drives writing to FAT formatted removable USB drives using a block size below 4096 Bytes is cached, others are not (for NTFS drives the magic block size is about 512 KB, under Windows 7 32 MB!) using flag FILE_FLAG_RANDOM_ACCESS causes a blows up of the file cache's working set using flag FILE_FLAG_NO_BUFFERING is the most efficient way to read and write large files for 'removable' USB drives the removal policies 'Optimize for speed' and 'Optimize for quick removal' have absolutely no effect on the cache's behaviour Deactivate write-cache There is no known way to deactivate a write cache if Windows decided to activate it, e.g. on NTFS formatted drives there is always a write cache. On a FAT formatted drive a solution could be to turn a drive into a "removable" drive by setting the "removable media bit" (RMB) by means of a filter driver like the Hitachi filter drive but for the opposite purpose. Just a theory...

	Safe removal
	An USB drive should never be removed without logging it off, especially when the drive has a write cache. The official way is thru a symbol in the systray and some mouse clicks. For 'removable' drives as internal card readers the media should be 'ejected': Right click the drive in the Windows Explorer, select 'Eject' here. Under XP this is not allowed for restricted users but this can be enabled by a policy. There are some useful commandline tools for this purpose but they all have some disadvantages. So I wrote my own: RemoveDrive prepares drives for safe removal EjectMedia ejects a media for a drive Reactivate a USB drive When a drive has been prepared for safe removal then it can be reactivated only by replugging it. Meanwhile I have learned that this is true only for devices which have the problem code 47 after the safe removal. Devices which have "Code 21" can be reactivated, I have made a tool for this: RestartSrDev. This works when RemoveDrive was started with admin previleges. Other solution: Put a USB hub before the device in question. If the hub is deactivated and reactivated by means of DEVCON then all there attached devices wakes up even if they had been prepared for safe removal. The hub's device ID is shown too by my ListUsbDrives when stared with -A (as all info), it's the 'Ctrl2 DevID'. Sample: devcon disable USB\VID_067B*PID_2517* devcon enable USB\VID_067B*PID_2517* The & has a special meaning on the Windows command line, so replacing it by * (placeholder for any characters) is an easy way to prevent problems.

	My Tools for USB drives
	Developing USBDLM I've made some other tools. They got their own page: Drive Tools for Windows

	Data Alignment
	SSD and pen drives work with flashmemory which is organized in blocks of a unknown size, probably up to 512 KB, the manufactures don't tell us. For efficient writing the data clusters of the file system should be aligned as good as possible as these flash blocks to avoid unnecessary flash erases. Therefore new pen drives usually come with a nicely aligned primary partition and a FAT32 file system with aligned clusters, provided the manufacturer made things right... If a flash drive is partitioned or formatted by the user it is essential to use the right tool. Windows Vista and Windows 7 do a good job here. They align new partitions and data clusters very well. In contrast Windows XP creates partitions in the classic way at track or cylinder borders which nearly always results in unaligned partitions. Data clusters: The NTFS file system has its tables somewhere in the middle of the partition, so the data clusters just start at the beginning, no alignment issues here. FAT/FAT32 have their allocation tables at the beginning, the data cluster follow after some 'reserved' sectors. Modifying the reserved sector count is the key to align the data clusters and this is what Vista and Win7 do very well while XP does a 4K alignment only. But all this is relative to the partitions!. With non aligned partitions the data clusters are created non aligned too! I've implemented the display of partition and cluster alignment in ListUsbDrives. As explained above it shows these information only if they are relevant. For SuperFloppy formatted drives there is no partition alignment and for NTFS drives the first cluster alignment is identical to the partition alignment. Even with a perfectly aligned first cluster, with a cluster size smaller than the flash block size files are perfectly aligned only with a certain probility: If the cluster have half the size of the flash blocks the every second file is perfectly aligned, with a quarter every fourth and so on. Only with clusters of the size of the flash blocks and at least this alignment full write speed is guaranteed. But larger clusters lead to more non-usable memory (slack space). And with clusters larger than 4KB there is no more NTFS compression. By the way: TrueCrypt since V6.0 align the data as the container is aligned. From the history: "the start of the data area will always be aligned with the start of a host-filesystem/physical sector" How to align a partition Aligning a partition without data loss is reported being possible by means of GParted (based upon a bootable Linux) by moving the partition back and forth, but for a flash drive I think it is not worth the effort. Just delete and recreate it under Vista/Win7. On "removable" drives the Windows disk management does no like to delete or create partitions. But the Vista/Win7 commandline tool DISKPART can. Another alternative is the SD Memory Card Formatting Software which creates an partition with aligned clusters on USB "removable" drives. The partition is aligned only if the drive is 4GB or larger, so don't format smaller drives with NTFS after formatting them by means of this software! A nice and free partition software for Windows which supports alignment is the "Active@ Partition Manager". By default it uses the classic alignment but is offers an option for the exact partition offset. Increase this number to the next value which can be evenly divided by 1024 to get a 512K alignment and decrease the size by the value you have increased the start offset with. Then it shows a warning plus question "Partition geometry is not aligned with disk track borders. Align on track borders?", we answer "No" because we know better... Letting format the drive with FAT resulted here in a 1K cluster alignment. Formatting again with XP (FORMAT or dialog) resulted in the expected 4K alignment. So, better let Windows do the formatting... What will we get? Depends on the device... With an old 128MB SD card I had no improvement of the write speed at all while a Corsair Voyager 16GB writes large files now with 15 MB/s instead of 6.5 thanks to aligned 64K clusters.

	Automatic actions on plug in and out
	My USB Drive Letter Manager can execute AutoRun events on arrival of a drive, on removal request and after removal. Automatic actions on arrival of a USB drive can be easily abused to copy interesting data from it in the background. So, never, never attach a USB drive with sensible data to a foreign computer even if you stand beside all the time! It's very easy to copy all DOC, XLS and PDF files in the background!

	USB drive as trojan horse
	Have a look at a found USB pen drive - who can resist... With a normal pen drive there is a danger only under Windows 2000. Since XP the line open= in the autorun.inf file is no more executed by windows. But XP does it on CD-ROM drives! Therefore these U3 flash drives use a fake CD-ROM drive on their pen drives which launches the 'U3 launchpad' automatically. But the contents of the CD-ROM can be easily changed into malware!. So, whenever you attach a foreign USB drive, hold down the Shift key to skip AutoRun. Or completely deactivate AutoRun by means of my tool AutoRunSettings Microsoft's TweakUI is buggy here. It completely ignores the default values, so once used, AutoRun for unknown and network drives gets activated! But there are more dangerous mechanisms, see here: How to: Quick intro to hacking autorun for USB flash drives In August 2008 Microsoft finally recognized that there is a security problem and offers updates which apply the autorun settings to the manual actions too, see here: http://support.microsoft.com/kb/953252

	Pen drive under Windows 98 and NT4
	For MS-DOS there is a lot of USB stuff. This page gives a good summary: USB DOS Ressources This one too: USB Bootdisk For Windows 98 you get drivers with the drive. If not you can try to use a driver of a foreign drive. Here is a german howto: http://www.techwriter.de/beispiel/usb-mem2.htm From COS Memory there are drivers which can be adopted in the same way: http://www.cos-memory.de/downloads/treiberdownload.htm Meanwhile there is an universal driver package for USB drives: http://www.technical-assistance.co.uk/kb/usbmsd98.php It uses Windows 2000 files, so you should have an additional valid Windows 2000 licence when you use this package... Windows NT4 got no USB support by Microsoft - we shall buy a new system from time to time... Other vendors help: USB driver by Inside Out Networks for Dell Latitude Laptop (doesn't work with USB hubs. Using an additional package by Inside Out Networks you can add hub support (link doesn't work anymore): ip4n4260.exe There seems to be no read or write cache with NT4, so pen drives behaves very slow here. From Digi International there is a driver package for a USB camera which shall include a driver for USB mass storage. I havn't tested it. http://ftp1.digi.com/support/driver/i4usb403.exe From China comes another USB driver for NT4 which supports USB 2.0 highspeed. I've made a short test: On a NForce2 board it hangs sometimes but I think that's a NForce2 problem. On several other machines it worked fine and fast. Read and write cache is active. The only drawback is that it assigns always drive letter Z: even according to the homepage this should be fixed. You can change this with the disk manager but on next reboot it's again Z:. Woodhead's Home One more page: Windows NT 4.0 and USB

	Encrypt a USB drive
	All encryption solutions which work thru an encrypted drive with drive letter have on thing in common: Administrator privileges are required, at least once to install the ecryption software. This is a problem on foreign computers. When a guest wants to install his encryption software, he must been trusted unconditional because he is granted to execute his software with administrator privileges. But no one can be sure what this software does despite the encrytion job. The computer is just compromised then! Even an antivirus software is useless because the foreign software is executed with admin privileges an the user will grant all requests to make it work. NTFS encryption is no help too because dealing with certificates is no fun and they are not compatible between Windows 2000 and XP. Solution: The approved open source software TrueCrypt: http://www.truecrypt.org It requires admin previlegs too but only once for installation. And the admin can download the software itself and validate its integrity by checking its PGP signature. Using TrueCrypt is documented on countless pages in the internet, just enter TrueCrypt at your preferred search engine.

	Links
	HP USB Disk Storage Format Tool SD Memory Card Formatting Software USB flash drive FAQ Booting from USB Fixing Windows 2000/XP Drive Letters Letter Assigner for Windows 95/98/ME

Counter since Feb. 1998

last change: 14 Nov 2012

Uwe Sieber

Back