qlyoung's wiki
Page Tools
User Tools
    Trace:

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
degoogling [2025/10/06 04:04] – write about working set, sync strategy qlyoungdegoogling [2025/10/06 04:06] (current) – Move cloud storage writing to tiered storage page qlyoung
Line 49: Line 49:
 There's other reasons, mostly to do with reliability engineering, but those are the main two. There's other reasons, mostly to do with reliability engineering, but those are the main two.
  
-I think a lot of people - myself included - get fixated on the "single source of truth" paradigm. Files live in one big beautiful cluster that you can access from any device. I see the appeal. You can focus all your efforts making that cluster as big and beautiful as you want, as resilient as you want. But after spending lots of time building out a big beautiful cluster, I kept having the experience that I wanted some file and I just couldn't get to it. And when I could get to it the experience of browsing around, searching, transferring files all over a network just...sucks. After you buy into the paradigm, after a while you get used to it and forget that it doesn't have to be like that. It's totally possible to have all that data on a superfast SSD drive right next to your CPU, easily searchable and accessible at all times. +TL;DR I now use [[tiered storage]]; what little need I have for internet-attached storage is now met by my home server, which has a lot of storage and is always accessible via tailscale. More on that in [[personal infrastructure]].
- +
-I finally realized that I fundamentally hate network storage as a primary storage location. It requires a network and even when you have one it's just too slow. Shortly after that I looked at storage prices, and suddenly, it turns out that you can get huge amounts of storage for not a lot of money. Granted, it depends on the size of your frequently accessed data, which I term your "working set". For me that's photos, music, documents, and projects. For me, at today's storage unit prices, I can afford around 6-8tb of working set. That's a lot! My actual working set, the stuff I access on a regular basis, fits in that with a couple tb to spare. +
- +
-Yes, some things are just too big to store on device. At today's storage unit prices, storing 30tb of movies that you access once a year on your laptop doesn't make sense yet for most people. And for that, networked storage is fine - you rarely access those items. +
- +
-After assessing my current data distribution and doing some consolidation I established that 8tb could store all of my  +
-frequently accessed" files. This covers my music, photos, documents, and projects. It doesn't include stuff that falls in the bucket of "huge binary files" - movies, 300gb DNxHD Resolve projects, that sort of thing. +
- +
-Since it was about time for a new laptop anyway, I converted every computer I own to have at least 8tb of storage, consolidated my working set into a single directory, and synced it to all devices in one epic multi-day multi-terabyte sync session. +
- +
-I feel like I hit storage nirvana. I get on an airplane, spend a few hours editing a spreadsheet or something, close my laptop. By the time I get home all of that work has opportunistically synced to all my other computers and I can sit down at my desktop and open that file. It's amazing. I barely need to touch my big chungus pool.+
 ==== Google Photos ==== ==== Google Photos ====
  
Panorama theme by desbest
degoogling.txt · Last modified: by qlyoung
CC Attribution-Noncommercial-Share Alike 4.0 International Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International