Page 1 of 1
USB transfer speeds slow
Posted: Sat Feb 10, 2024 12:27 pm
by theprofa325
Code: Select all
System: Kernel: 5.10.0-27-amd64 [5.10.205-2] x86_64 bits: 64 compiler: gcc v: 10.2.1
parameters: BOOT_IMAGE=/boot/vmlinuz-5.10.0-27-amd64 root=UUID=<filter> ro quiet splash
Desktop: Xfce 4.18.1 tk: Gtk 3.24.24 info: xfce4-panel wm: xfwm 4.18.0 vt: 7
dm: LightDM 1.26.0 Distro: MX-21.3_x64 Wildflower October 20 2021
base: Debian GNU/Linux 11 (bullseye)
Machine: Type: Laptop System: SAMSUNG product: 940X3G/930X3G v: P05ACJ serial: <filter> Chassis:
type: 9 serial: <filter>
Mobo: SAMSUNG model: NP940X3G-K04US v: SAMSUNG_SW_REVISION_1234567890ABCD
serial: <filter> UEFI: American Megatrends v: P05ACJ.128.140819.dg date: 08/19/2014
Battery: ID-1: BAT1 charge: 38.2 Wh (79.7%) condition: 47.9/55.5 Wh (86.3%) volts: 7.8 min: 7.6
model: SAMSUNG Electronics SR Real Battery type: Li-ion serial: <filter>
status: Discharging cycles: 1522
CPU: Info: Dual Core model: Intel Core i7-4500U bits: 64 type: MT MCP arch: Haswell
family: 6 model-id: 45 (69) stepping: 1 microcode: 26 cache: L2: 4 MiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 19155
Speed: 898 MHz min/max: 800/3000 MHz Core speeds (MHz): 1: 898 2: 898 3: 898 4: 898
Vulnerabilities: Type: gather_data_sampling status: Not affected
Type: itlb_multihit status: KVM: VMX disabled
Type: l1tf mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
Type: mds mitigation: Clear CPU buffers; SMT vulnerable
Type: meltdown mitigation: PTI
Type: mmio_stale_data status: Unknown: No mitigations
Type: retbleed status: Not affected
Type: spec_rstack_overflow status: Not affected
Type: spec_store_bypass
mitigation: Speculative Store Bypass disabled via prctl and seccomp
Type: spectre_v1 mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2 mitigation: Retpolines, IBPB: conditional, IBRS_FW, STIBP:
conditional, RSB filling, PBRSB-eIBRS: Not affected
Type: srbds mitigation: Microcode
Type: tsx_async_abort status: Not affected
Graphics: Device-1: Intel Haswell-ULT Integrated Graphics vendor: Samsung Co driver: i915
v: kernel bus-ID: 00:02.0 chip-ID: 8086:0a16 class-ID: 0300
Device-2: Silicon Motion WebCam SC-10HDP12B24N type: USB driver: uvcvideo bus-ID: 1-5:3
chip-ID: 2232:1049 class-ID: 0e02 serial: <filter>
Display: x11 server: X.Org 1.20.11 compositor: xfwm4 v: 4.18.0 driver:
loaded: modesetting unloaded: fbdev,vesa display-ID: :0.0 screens: 1
Screen-1: 0 s-res: 3200x1800 s-dpi: 96 s-size: 847x476mm (33.3x18.7")
s-diag: 972mm (38.3")
Monitor-1: eDP-1 res: 3200x1800 hz: 60 dpi: 277 size: 293x165mm (11.5x6.5")
diag: 336mm (13.2")
OpenGL: renderer: Mesa DRI Intel HD Graphics 4400 (HSW GT2) v: 4.5 Mesa 20.3.5
compat-v: 3.0 direct render: Yes
Audio: Device-1: Intel Haswell-ULT HD Audio vendor: Samsung Co driver: snd_hda_intel v: kernel
bus-ID: 00:03.0 chip-ID: 8086:0a0c class-ID: 0403
Device-2: Intel 8 Series HD Audio vendor: Samsung Co driver: snd_hda_intel v: kernel
bus-ID: 00:1b.0 chip-ID: 8086:9c20 class-ID: 0403
Sound Server-1: ALSA v: k5.10.0-27-amd64 running: yes
Sound Server-2: JACK v: 1.9.21 running: no
Sound Server-3: PulseAudio v: 14.2 running: yes
Network: Device-1: Intel Wireless 7260 driver: iwlwifi v: kernel modules: wl port: f040
bus-ID: 02:00.0 chip-ID: 8086:08b1 class-ID: 0280
IF: wlan0 state: up mac: <filter>
Device-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet vendor: Samsung Co
driver: r8169 v: kernel port: e000 bus-ID: 03:00.0 chip-ID: 10ec:8168 class-ID: 0200
IF: eth0 state: down mac: <filter>
IF-ID-1: tun0 state: unknown speed: 10 Mbps duplex: full mac: N/A
Bluetooth: Device-1: Intel Bluetooth wireless interface type: USB driver: btusb v: 0.8
bus-ID: 1-4:2 chip-ID: 8087:07dc class-ID: e001
Report: hciconfig ID: hci0 rfk-id: 21 state: up address: <filter> bt-v: 2.1 lmp-v: 4.0
sub-v: 500 hci-v: 4.0 rev: 500
Info: acl-mtu: 1021:5 sco-mtu: 96:5 link-policy: rswitch hold sniff
link-mode: slave accept service-classes: rendering, capturing, object transfer, audio
Drives: Local Storage: total: 469.47 GiB used: 237.38 GiB (50.6%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/sda maj-min: 8:0 vendor: Samsung model: MZNTE256HMHP-000 size: 238.47 GiB
block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s type: SSD serial: <filter>
rev: 1K0Q scheme: GPT
ID-2: /dev/sdb maj-min: 8:16 type: USB vendor: Kingston model: DT microDuo 3C
size: 231 GiB block-size: physical: 512 B logical: 512 B type: N/A serial: <filter>
rev: PMAP scheme: MBR
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition: ID-1: / raw-size: 70.31 GiB size: 68.66 GiB (97.65%) used: 49.37 GiB (71.9%) fs: ext4
dev: /dev/sda3 maj-min: 8:3
ID-2: /boot/efi raw-size: 1024 MiB size: 1022 MiB (99.80%) used: 420 KiB (0.0%)
fs: vfat dev: /dev/sda1 maj-min: 8:1
ID-3: /home raw-size: 151.16 GiB size: 147.73 GiB (97.73%) used: 126.12 GiB (85.4%)
fs: ext4 dev: /dev/sda4 maj-min: 8:4
Swap: Kernel: swappiness: 15 (default 60) cache-pressure: 100 (default)
ID-1: swap-1 type: partition size: 16 GiB used: 185.2 MiB (1.1%) priority: -2
dev: /dev/sda2 maj-min: 8:2
Sensors: System Temperatures: cpu: 48.0 C mobo: 48.0 C
Fan Speeds (RPM): N/A
Repos: Packages: note: see --pkg apt: 2609 lib: 1316 flatpak: 0
No active apt repos in: /etc/apt/sources.list
Active apt repos in: /etc/apt/sources.list.d/brave-browser-release.list
1: deb [signed-by=/usr/share/keyrings/brave-browser-archive-keyring.gpg] https://brave-browser-apt-release.s3.brave.com/ stable main
Active apt repos in: /etc/apt/sources.list.d/debian-stable-updates.list
1: deb http://deb.debian.org/debian/ bullseye-updates main contrib non-free
Active apt repos in: /etc/apt/sources.list.d/debian.list
1: deb http://deb.debian.org/debian/ bullseye main contrib non-free
2: deb http://security.debian.org/debian-security/ bullseye-security main contrib non-free
Active apt repos in: /etc/apt/sources.list.d/mx.list
1: deb http://mxrepo.com/mx/repo/ bullseye non-free main
Info: Processes: 259 Uptime: 11d 17h 29m wakeups: 9 Memory: 7.68 GiB used: 3 GiB (39.1%)
Init: SysVinit v: 2.96 runlevel: 5 default: 5 tool: systemctl Compilers: gcc: 10.2.1
alt: 10 Client: shell wrapper v: 5.1.4-release inxi: 3.3.06
Boot Mode: UEFI
I just took ONE WEEK 24/7 to rsync 65 GB to the Kingston flash drive given in above system info, plugged into a blue "S/S" usb port (avg tfr rate ~ 0.5 MB/s).
Then I tried the echoes to /proc/sys/vm/dirty... suggested above. It increased the speed to 13.5 MB/s for rsync on a single .deb file. Unfortunately when I did the same rsync transfer with the default values (dirty_bytes=20 000 000 dirty_background_bytes = 0) it went at 7 MBPS. The multiple file transfer is apparently 10x slower than a single file.
13.5 MBPS still sucks. Claimed transfer rate for this USB3.2 Gen 1 drive is 5 GbPS ~ 600 MBPS. It is a factor 40 slow. Later in the week I will try the same drive on a Windoze machine at work and report the result here.
Does anyone know whether Kingston flash drives are slow? I don't do this often enough to spend a lot of time and money trying different brand name drives and some earlier posts suggest that's a waste of time anyhow.
ANy other suggestions welcome.
Re: USB transfer speeds slow/ data needs to be written message
Posted: Sat Feb 10, 2024 12:39 pm
by j2mcgreg
@theprofa325
Don't tack on to an old topic. Instead start a new one.
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 1:52 pm
by Eadwine Rose
Topic split off, so fixed now, but don't dig up old topics next time.
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 2:24 pm
by TTwrs
You're not the only one to such results...
https://www.amazon.com/Kingston-DTDUO3CG3/dp/B0B1W53H1W
https://www.amazon.com/gp/customer-revi ... D2HFSTIYHO
As a possible explanation, see my comment here...
viewtopic.php?p=600275#p600275
You might will be better off with one of these instead (fast, fast, fast)...
https://www.amazon.com/AXE-MEMORY-Super ... B09TTXS151
Detail: LABEL=AXE-DRIVE TYPE=exfat
Vendor: AXE | Model: USB_DISK_3.2 gen 2 -- Portable SSD
Drive: 465.8GB | Partition: 466GB | Free: 466GB | Blocksize: 256KB
Test filesize: 1GB | Loops: 1 | Rlimit: 120s (4k:60s) | Q1T1
R/W tests are 1.0MB-Sequential and 512kB, 128kB, 4.0kB Random
1.0MB Read: 369 MiB/s (387 MB/s) Write: 579 MiB/s (607 MB/s)
512kB Read: 328 MiB/s (344 MB/s) Write: 540 MiB/s (566 MB/s)
128kB Read: 214 MiB/s (225 MB/s) Write: 472 MiB/s (495 MB/s)
4.0kB Read: 24.0 MiB/s (25.2 MB/s) Write: 90.7 MiB/s (95.1 MB/s)
Test date: 2023-06-22 06:30:59 | fio version: 3.25 | Total test-time: 76sec.
You could try CrystalDiskMark (Windows) or KDiskMark (Linux) for testing.
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 2:49 pm
by Mauser
You are not alone on this one because I and many others are experiencing this when it comes to Linux. As for if this happening on Windows or not I can't comment on it since I haven't used Windows since October 2015 with a very few, very rare occasion. I gave up on finding a solution to this issue and have accepted very slow USB transfer speeds as "it is what it is."
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 3:00 pm
by figueroa
It's probably the USB flash drive itself. All makers stumble from time to time, some more than others. I've had satisfactory results with the Samsung Bar Plus flash drives, all sizes. They are currently still available and decently priced. You never get the advertised speed on a sustained basis.
I always test new flash memory for counterfeit with f3 (f3write, f3read, f3probe). See
https://fight-flash-fraud.readthedocs.i ... index.html
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 8:55 pm
by theprofa325
Thanks, folks for all the help. I have tried 4 USB devices now, with results that really surprised me but seems to put the topic to rest.
Using rsync -v to transfer a single 1.4 GB file. Using the dirty_... values suggested in old thread.
Kingston DT microDuo 3C 13.5 MBPS
Generic freeby from oscilloscope company: 12.3 MBPS
Generic "NXT" branded stick: 33.4 GBPS
Seagate 3eeap6-500 external spinning hard drive with USB-A to micro-B cable: 102 MBPS
What I get from this-
* apparently usb sticks do differ significantly in speed ON LINUX. I'll try this again on Windoze this week.
* even the fastest device does not approach 5 BbPS ~ 600 MBPS.
Re: USB transfer speeds slow
Posted: Sat Feb 10, 2024 10:23 pm
by m_pav
I disagree with that, a slow USB device is slow, period, irrespective of what OS is running the computer. Linux just makes it VERY clear that writing to the drive is very slow and most people running Windows or Apple simply do not know the difference so whatever it takes is whatever it takes and they just drink it up without any critique. FAT32 is a terrible file system, and it's the slowest of all the major types in use today.
I dealt with this so many times when I operated an IT shop that it was not funny. Small businesses using Sandisk Cruzer Flash drives and other el-cheapo chinese junk drives, esp those promo swivel drives with supplier logos on the steel band, were consistently getting into trouble with failed writes to these horrid devices.
When you fail to do your research and trust the advertised major spec, you'll swallow anything that's dished up to you. So many flash drives are labelled as USB3.2 but in reality, they barely make USB2.0 speeds which have a theoretical maximum of 480mbps (60MBps), of which you're incredibly lucky if you get 28MBps write speeds and 40MBps read. The better USB Flash drives might make 65MBps at a burst, but they tail of very quickly to 42 or less, that is in all known OS's.
The key thing to respect here is the drive storage format type has a massive part to play in it's performance. With the hardware being identical and capable, as I will show soon, you simply can not expect a Windows formatted drive to perform at its best when connected to a Linux Host. Apple knows this, so they refused to allow NTFS support in their OS, smart move. Linux Supports NTFS because we're all inclusive, but it's not native to Linux and it will never perform as well as it would when connected to its native host OS. To compare like for like, it is necessary to use format types that are native to the OS when doing speed comparisons, keeping in mind the ability of the drives internal firmware to accept the format type in use, else the test is a complete failure before it's even started. That is why it is imperative that only quality flash drives and external media are used as these will have a more complete firmware load and superior data handling capabilities. Anything else provides a false economy.
I have seen "expert" Linux reviewers publish articles claiming 20 minutes to make a Live USB using MX Live USB Maker. I no longer have the acutal data, but from memory, I get between 2.5 to 5 minutes on most half decent USB Flash drives, but I get a consistent 11 seconds to make a MX23.2 Xfce Live USB on my Netac 128Gb USB-SSD Flash drive. Keep in mind this includes erasing the drive no less than 2x, assigning and creating 2 partitions that combined consume only 15% of the drive space, unpacking the ISO and writing it to the drive, marking the drive as bootable, then writing the logfile to my host machine. This is using my daily driver which only has USB3.0 ports, I'd expect greater speeds from a machine running USB3.2 ports.
I proved the 11 seconds by ripping the drive out the USB port as soon as the process completed, sticking it into another machine and booting it with the option to do the checksum tests, all passed and the boot completed without error. On these drives, of which I have 3, I create a data partition using the ExFat File System. It's not the fastest, but it performs relatively equally well wherever I use it, and that's everywhere. There is only a minor difference between all 3 major OS's in its overall speeds, almost indecipherable.
Having said all that, I suggest you try for a week or two one of the Liquorix Kernels. Your laptop, like mine is a little exotic, in that not that many were made as compared to consumer level machines. The more exotic machines tend to have a slightly higher percentage of issues with Linux. With the stock Debian kernel, mine has sleep/wake issues tied to the TPM chip that strangely enough affect the USB ports, they suddenly slow to a crawl. The Liquorix kernels don't put a stop to it, but I get 3-4 x longer power-on times and sleep/wake events before the USB degradation happens and my graphics capabilities are significantly improved.
Re: USB transfer speeds NOT SO slow
Posted: Wed Feb 21, 2024 11:04 pm
by theprofa325
New data- it is a combination of the drive and the file system.
Bought a new 256 GB Kingston Data Traveler Max Type A based on an internet speed test page that I can no longer find. They claimed over 900 MB/s.
Copying a single 2.2 GB file onto the as-delivered vfat formatted with rsync -v gave 318 MB/sec.
Reformatting to ext4 using MX formatting tool gave 683 MB/s.
Re: USB transfer speeds NOT SO slow
Posted: Wed Feb 21, 2024 11:43 pm
by DukeComposed
theprofa325 wrote: ↑Wed Feb 21, 2024 11:04 pm
New data- it is a combination of the drive and the file system.
Bought a new 256 GB Kingston Data Traveler Max Type A based on an internet speed test page that I can no longer find. They claimed over 900 MB/s.
Copying a single 2.2 GB file onto the as-delivered vfat formatted with rsync -v gave 318 MB/sec.
Reformatting to ext4 using MX formatting tool gave 683 MB/s.
It's been a long time since I've worried about file system performance comparisons, but apparently it's a known issue that
FAT32 is weak at copying many small files. This hearsay detail doesn't match your rsync test, but I think it's indicative that the choice of file system can have an impact on your drive's throughput.
Re: USB transfer speeds slow
Posted: Sun Aug 18, 2024 10:43 pm
by crazysquirrel
I am lucky to get 13.5MB/s or less for flash or external drive.
Can't afford a new computer - sigh...
Most of my flash drives are either ntfs or burned ISO image.
Fat32 too small. Ext anything doesn't work in Windows XP.
Yes, I dual boot with XP to play many games but not use for internet.
My hardware will not support win10 or newer. Not even sure it will support win7,
Something about the CPU is the issue.
Re: USB transfer speeds slow
Posted: Mon Aug 19, 2024 2:38 am
by Eadwine Rose
With all help requests, please share your full Quick System Info, found in the menu under Favorites. Press the Copy for forum button, then click paste in a reply. If need be, you can do this with the LiveUSB, thanks.