Edmondo's Vault

Deleted-But-Open Files

May 18, 20263 min read

Deleted-But-Open Files

This is the single most common cause of large invisible disk usage on macOS (and any Unix system). It is invisible to ls, du, find, Finder, and DaisyDisk — and none of those tools can ever detect it.

How it works

  1. A process opens a file (gets a file descriptor via open(2))
  2. Something deletes the file — rm, newsyslog (macOS’s log rotator), logrotate, or any cleanup script
  3. The directory entry is gone — the file has no name in the filesystem
  4. But the process still holds the file descriptor open
  5. POSIX rule: disk blocks are not freed until the last file descriptor is closed

The space is genuinely consumed. diskutil info reports it as used. But no directory-tree tool can see it because there is no directory entry to find.

The diagnostic command

lsof (list open files) can find these because it queries the kernel’s file descriptor tables, not the directory tree. The +L1 flag filters for files with a link count less than 1 — i.e., deleted files.

# List all deleted-open files
sudo lsof +L1
 
# Total size (upper bound — double-counts shared file descriptors)
sudo lsof +L1 2>/dev/null | awk 'NR>1{sum+=$7} END{printf "%.1f GB in deleted-open files\n", sum/1073741824}'
 
# Show the biggest offenders with process name and PID
sudo lsof +L1 2>/dev/null | awk 'NR>1 && $7+0 > 100000000 {printf "%10.1f GB  PID=%-8s %s  %s\n", $7/1073741824, $2, $1, $9}' | sort -rn | head -30

lsof double-counts shared descriptors

If multiple processes (or forked children) share the same open file descriptor, lsof reports the file size once per process. The actual disk consumption can be much less than the raw sum. In one real case, lsof reported 1,513 GB but the real consumption was 708 GB.

The fix

Kill or restart the process holding the files. Blocks are freed instantly — no garbage collection, no fsck, no reboot needed.

# If launchd-managed (PPID = 1), kill it and launchd respawns it clean
sudo kill <PID>
 
# Or restart by service label
sudo launchctl kickstart -k system/<service.label>

Common culprits

  • Log shipping agents (Gravwell FileFollower, Splunk forwarder, Filebeat, Fluentd) that tail log files and don’t handle rotation — when the log file is rotated, the agent keeps its file descriptor to the old (now deleted) file and never closes it
  • Docker holding deleted disk images or container logs
  • Corporate endpoint agents (CrowdStrike, SentinelOne) with large database files that get replaced during updates
  • Any long-running daemon whose log files get rotated by newsyslog or logrotate while the daemon holds them open

Why this is so dangerous on macOS

On Linux, this problem exists but is easier to diagnose because du and df work reliably — the gap between du / and df / immediately points to deleted-open files or snapshots. On macOS, du is already broken by firmlinks, df is broken by purgeable space and shared container pools, and DaisyDisk labels everything it can’t see as “hidden.” Deleted-open files get lost in the noise of all the other APFS-related accounting confusion.

lsof +L1 is the only tool that can see them. It should be the second command you run after diskutil apfs list in any macOS disk space investigation.

See also

Graph View

Backlinks