Search This Blog

Showing posts with label script. Show all posts
Showing posts with label script. Show all posts

Saturday, December 13, 2025

Improve Steam Game Patching Speed

Game was patching slow (Helldivers 2)

Low mbps (200) taking forever

Game disk was SSD but Steam using old Game Location (HDD) and Random IO was slow

Made Script to Improve:

  • MAKE SURE TARGET SSD HAS ENOUGH SPACE
  • MAKE SURE BOTH FILE SYSTEM TYPES ARE 'NTFS'

Desktop Production View



# Move-Steam-Downloading-To-SSD.ps1
# Run PowerShell "As Administrator"

$D = "D:\SteamLibrary\steamapps\downloading"
$E = "E:\SteamLibrary\steamapps\downloading"

function Assert-Admin {
    $isAdmin = ([Security.Principal.WindowsPrincipal] `
        [Security.Principal.WindowsIdentity]::GetCurrent() `
    ).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

    if (-not $isAdmin) {
        throw "Run this script as Administrator (needed for mklink)."
    }
}

function Ensure-Dir($path) {
    if (-not (Test-Path $path)) {
        New-Item -ItemType Directory -Path $path | Out-Null
    }
}

Assert-Admin

Write-Host "Attempting to stop Steam (ignore errors if it's already closed)..."
Get-Process steam -ErrorAction SilentlyContinue | Stop-Process -Force -ErrorAction SilentlyContinue

# Ensure parent directories exist
Ensure-Dir (Split-Path $D -Parent)
Ensure-Dir (Split-Path $E -Parent)

# If E:\...\downloading exists, rename it to downloading.old (with timestamp if needed)
if (Test-Path $E) {
    $oldBase = Join-Path (Split-Path $E -Parent) "downloading.old"
    $old = $oldBase
    if (Test-Path $old) {
        $stamp = Get-Date -Format "yyyyMMdd-HHmmss"
        $old = "$oldBase.$stamp"
    }
    Write-Host "Renaming SSD folder:`n  $E`n-> $old"
    Rename-Item -Path $E -NewName (Split-Path $old -Leaf)
}

# If D path is a junction already, remove it (so we can move/create cleanly)
if (Test-Path $D) {
    $item = Get-Item $D -Force
    if ($item.Attributes.ToString().Contains("ReparsePoint")) {
        Write-Host "D folder is a reparse point (junction/symlink). Removing it:`n  $D"
        Remove-Item -Path $D -Force
    }
}

# Move D:\...\downloading to E:\...\downloading (if it exists)
if (Test-Path $D) {
    Write-Host "Moving HDD folder:`n  $D`n-> $E"
    Move-Item -Path $D -Destination $E
} else {
    Write-Host "No HDD downloading folder found at $D (creating empty target on SSD)."
    Ensure-Dir $E
}

# Create junction from D to E
Write-Host "Creating junction:`n  $D`n=> $E"
cmd /c "mklink /J `"$D`" `"$E`""

Write-Host "`nVerification:"
cmd /c "dir `"$([System.IO.Path]::GetDirectoryName($D))`""
cmd /c "fsutil reparsepoint query `"$D`""

Write-Host "`nDone."
Write-Host "Steam should now see D:\...\downloading, but it will actually use the SSD at E:\...\downloading."
You may need to enabled Windows PowerShell Script Execution Policy for the Current User. When you successfully Run the script, it will take time to move the original Steam downloads data to the destination device you want to use (to NVMe SSD from HDD, 114 GB "reserved" for Hell Divers 2 in this case).

Desktop Production Running Script to Improve Steam Patching Game


"Why did I have to create this script and move the data? Why was steam using the D: for the Download Cache instead of the E: drive even though the game is installed on the E: drive? This seems like the wrong drive."

Steam’s “download cache” isn’t “the drive the game is on.” It’s “the Steam library folder (or client install folder) Steam has decided to use for staging,” and that choice can easily end up being your HDD even when the game files live on the SSD.

The key idea: Steam updates aren’t written straight into the game folder. Steam typically uses a staging area (steamapps\downloading, sometimes plus other cache folders) to download chunks, unpack/decrypt, and apply deltas. Steam picks where that staging lives based on library association and configuration, not on where you think the game “is.”

Common “why did it pick D?” reasons:

  • The app is logically associated with the D: library (even if you manually moved the game files at some point, or the manifest stayed on D:). Steam uses the library containing the app’s appmanifest_*.acf to decide staging. If the manifest is under D:\SteamLibrary\steamapps, Steam will stage under D.

  • D: was your default / primary library at the time of install/update. Steam will keep using that library for that app unless you explicitly “Move install folder” inside Steam.

  • Steam client caches can live where Steam itself is installed. Even if the game is on E:, certain caches (appcache, depotcache) may still hammer the drive that contains steam.exe.

  • Steam doesn’t automatically optimize for SSD vs HDD. It follows its rules: “use the library/client path I’m configured for,” not “use the fastest disk.”

So it feels like “the wrong drive,” but it’s Steam behaving consistently with its internal mapping: game location and staging/cache location are related, but not guaranteed to match.

Why your script/junction helps:

  • If Steam insists on using D:\SteamLibrary\steamapps\downloading, you can “lie” to it safely with a junction so it still uses the path it expects, while the I/O lands on E:. This is especially helpful because downloading is often random-I/O heavy, where an SSD can be dramatically faster than an HDD.

If you want a “Steam-native” fix (no junction), the usual solution is:

  • Steam → Settings → Storage → select the game → Move to the E: library

  • Also set E: as the default library

  • Ensure the relevant appmanifest_*.acf ends up under E:\SteamLibrary\steamapps\

But if Steam keeps choosing D: for staging anyway, your junction approach is a perfectly logical workaround.

Thursday, July 28, 2016

Parsing tcpdump/pcap files in bash, without Wireshark.

Sometimes I work in an environment in which software access is restricted.  Things like Wireshark are DEFINITELY not allowed.

However, I'm still expected to troubleshoot network connectivity issues.  Luckily, tcpdump exists in the EPEL Repo, but leaves much to be desired when trying to read the PCAP files.  I decided to read up on the PCAP spec, and create my own parser in bash.  Crazy? Maybe.  But, it does work.

Another nice capability here is that using something like vim, you can easily tweak this script to dump application data for further debugging.  For example, switch your HTTPd to run in Non-SSL, run a capture, and dump the application data.  Of course, you may have to work out the correct byte offset for your packets. As this is a bare minimal implementation of a pcap parser and does not nearly support the full scope of capability provided by libpcap.

The real issue here is that I have not taken the time to parse some flags which consist of single bits.  There is also much work to be done in regard to writing the conditionals for various Ethernet, IP Routing and TCP frames.  But this is a good baseline to start.

To use this script just run 'tcpdump -w file.pcap -X' to run a capture.  Once done, pass the file as an argument to the pcap-analyzer bash script, and you will be able to read your frames on the CLI!  This should work out of the box with Centos/RHEL 6 Minimal, with the addition of the tcpdump RPM.

Here is a screenshot.



As you can see frame 2 starts to break because of the parsing.  Some flags actually mean, the header is going to be 28 bytes instead of 26.  As a result of the byte variations, the byte iterator ends up on some obscure part of the stream I don't handle, and things get start to break.  Git forks are welcome if you like this little utility and are interested in making it more robust.

https://github.com/charlescva/common/blob/master/pcap-analyzer.sh

One final important characteristic about Ethernet wire data is that it is ALWAYS Big Endian.  This is confusing because most CPUs are little endian.  To work around this issue, I use both 'hexdump' and 'od' because one enforces big endian while the other is CPU dependent.

Other resources Used:
http://www.tcpdump.org/manpages/pcap-savefile.5.html
http://www.tcpdump.org/linktypes.html
https://en.wikipedia.org/wiki/Ethernet_frame#Ethernet_II
https://en.wikipedia.org/wiki/IPv4#Packet_structure
https://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure

Friday, September 19, 2014

Just Install Docker on Ubuntu 14.04 64-bit...

It has been a while since I have posted, so I'm making this one short.  Ever want a clean and concise script written in Bash that will install the latest version of Docker (And stay up to date)?  Well, here. Note that Docker wants you to pick the DNS.  So I've used 8.8.8.8 (Google-DNS) for this example.

Note: This will also install the kernal extras to enable AUFS support.
#!/bin/bash
#Update our local package index
sudo apt-get update
#Make sure Apt supports HTTPS
[ -e /usr/lib/apt/methods/https ] || {
  apt-get install -y apt-transport-https
}
# Get server key for repo
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
# Update Repo Sources...
[ -e /etc/apt/sources.list.d/docker.list ] || {
  rm /etc/apt/sources.list.d/docker.list
}
sudo echo "deb https://get.docker.io/ubuntu docker main" > /etc/apt/sources.list.d/docker.list
sudo apt-get update
# ensure  Linux kernel extra modules are installed which support cpu cgroups, etc...
sudo apt-get install -y linux-image-extra-$(uname -r)
# install docker
sudo apt-get install -y lxc-docker
# update docker DNS and listen on localhost tcp
sudo echo 'DOCKER_OPTS="-dns 8.8.8.8 -H unix:///var/run/docker.sock -H tcp://127.0.0.1:5555"' > /etc/default/docker