Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| personal_infrastructure [2023/09/08 23:09] – [Typical Deployment] qlyoung | personal_infrastructure [2024/06/16 01:02] (current) – [Cost] qlyoung | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| + | ====== personal infrastructure ====== | ||
| + | |||
| + | |||
| //tldr - docker compose, isolated servers// | //tldr - docker compose, isolated servers// | ||
| Line 29: | Line 32: | ||
| pub((public)) | pub((public)) | ||
| priv((private)) | priv((private)) | ||
| - | | + | |
| - | data(stores sensitive data) --> | + | onlyme{used only by me?} |
| - | onlyme(used only by me) --> priv | + | data --> |
| + | data --> | ||
| + | onlyme --> | ||
| + | onlyme --> |no| pub | ||
| </ | </ | ||
| Line 40: | Line 46: | ||
| ===== Public ===== | ===== Public ===== | ||
| - | For the public sphere, I use cloud-based Linux VMs from one of the affordable providers. I run most of my stuff on a single shared CPU VM with 4 CPU cores, | + | For the public sphere, I use cloud-based Linux VMs from one of the affordable providers. I run most of my stuff on a single shared CPU VM with 4 CPU cores, |
| For things that need to be exposed in the internet I think cloud is the best choice. From a network isolation perspective serving things from your home means untrusted traffic will be flowing within your home network. Apart from security concerns there are other problems: | For things that need to be exposed in the internet I think cloud is the best choice. From a network isolation perspective serving things from your home means untrusted traffic will be flowing within your home network. Apart from security concerns there are other problems: | ||
| Line 49: | Line 55: | ||
| * Dynamic IP addresses makes access annoying | * Dynamic IP addresses makes access annoying | ||
| * Advertising your home IP address is a mild security concern | * Advertising your home IP address is a mild security concern | ||
| + | * can be mitigated if you rent an ultra cheap VPS and use it as a gateway; I did this initially but realized I can run everything on a cheap vps to begin with | ||
| * Weather / residential power outages impact uptime | * Weather / residential power outages impact uptime | ||
| Line 60: | Line 67: | ||
| * Case: Fractal Node 804 | * Case: Fractal Node 804 | ||
| - | * Motherboard: | + | * Motherboard: |
| - | * RAM: 128gb ECC memory (4x [KSM32ED8/ | + | * RAM: 128gb ECC memory (4x [[https:// |
| * Storage: | * Storage: | ||
| - | * 4x WD Blue 3.5" 4tb HDD | + | |
| - | * 4x Hitachi 3.5" 4tb HDD | + | * 4x Hitachi 3.5" 4tb HDD |
| - | * 1tb WD Black NVME | + | * 1tb WD Black NVME |
| * CPU: AMD Ryzen 7 3800X | * CPU: AMD Ryzen 7 3800X | ||
| * GPU: RTX 3070 | * GPU: RTX 3070 | ||
| Line 84: | Line 91: | ||
| * Much better control and monitoring of hardware resources | * Much better control and monitoring of hardware resources | ||
| - | * Full suite of hardware monitoring tools built into a clean web interface; disk diagnostics, | + | |
| - | * Ability to easily partition storage resources among VMs | + | * Ability to easily partition storage resources among VMs |
| - | * Ability to set fine grained memory and CPU resource limits | + | * Ability to set fine grained memory and CPU resource limits |
| * Full snapshots for tenant VMs | * Full snapshots for tenant VMs | ||
| - | * Ability to snapshot the entire machine (VM) at the " | + | |
| * Ability to create new machines whenever I want to try something | * Ability to create new machines whenever I want to try something | ||
| - | * E.g. I have a Windows VM that runs some Windows-only server software | + | |
| - | * I can spin up a VM with resources of my choosing to try out an operating system or run some experiment without worrying about damaging any infrastructure I rely on | + | * I can spin up a VM with resources of my choosing to try out an operating system or run some experiment without worrying about damaging any infrastructure I rely on |
| Of course, you don't really need a dedicated hypervisor to do any of this; you can do it all with KVM on a traditional bare metal server OS. But it's much easier and more convenient in a hypervisor and in practice, there' | Of course, you don't really need a dedicated hypervisor to do any of this; you can do it all with KVM on a traditional bare metal server OS. But it's much easier and more convenient in a hypervisor and in practice, there' | ||
| The main downside is that hardware passthrough can be tricky. I pass through the RTX 3070 to the VM, and then into docker containers, in order to get accelerated encoding for media related services. KVM GPU passthrough is annoying. | The main downside is that hardware passthrough can be tricky. I pass through the RTX 3070 to the VM, and then into docker containers, in order to get accelerated encoding for media related services. KVM GPU passthrough is annoying. | ||
| - | |||
| - | --- | ||
| The OS that applications run on (ignoring Docker) is always a recent Ubuntu Server LTS with automatic security upgrades enabled. I find the Debian-based platform familiar, comfortable and stable. Ubuntu Server also has a very wide install base which means most problems can be resolved with a web search. This helps keep the maintenance overhead as low as possible. I enjoy building my infrastructure, | The OS that applications run on (ignoring Docker) is always a recent Ubuntu Server LTS with automatic security upgrades enabled. I find the Debian-based platform familiar, comfortable and stable. Ubuntu Server also has a very wide install base which means most problems can be resolved with a web search. This helps keep the maintenance overhead as low as possible. I enjoy building my infrastructure, | ||
| Line 243: | Line 248: | ||
| </ | </ | ||
| - | If possible, I also configure applications so that users are served files directly from B2 rather than having them proxied through my VM, which improves performance both in terms of load time and bandwidth usage. When you watch a video on my Peertube server, your browser is downloading the video directly from B2, instead of `B2 → VM → You`. | + | If possible, I also configure applications so that users are served files directly from B2 rather than having them proxied through my VM, which improves performance both in terms of load time and bandwidth usage. When you watch a video on my Peertube server, your browser is downloading the video directly from B2, instead of '' |
| ====== DNS & HTTP & TLS ====== | ====== DNS & HTTP & TLS ====== | ||
| Line 270: | Line 275: | ||
| ===== DNS ===== | ===== DNS ===== | ||
| - | Each service is on its own subdomain. All subdomains | + | Each service is on its own subdomain. All subdomains |
| ===== HTTP ===== | ===== HTTP ===== | ||
| - | nginx runs on the host and binds host ports 80 and 443. All docker containers bind to `(127.0.0.1, P)` where `P` is a host port number of my choosing. Each service has its own subdomain and a corresponding nginx configuration: | + | nginx runs on the host and binds host ports 80 and 443. All docker containers bind to '' |
| < | < | ||
| Line 385: | Line 390: | ||
| </ | </ | ||
| - | 4. Set up new `CNAME` record; | + | 4. Set up new '' |
| 5. Configure nginx, request and install TLS certificate | 5. Configure nginx, request and install TLS certificate | ||
| - | <code bash> | + | <code bash> |
| # create nginx configuration, | # create nginx configuration, | ||
| $ vim / | $ vim / | ||
| Line 396: | Line 401: | ||
| $ certbot --nginx -d recipes.qlyoung.net | $ certbot --nginx -d recipes.qlyoung.net | ||
| </ | </ | ||
| + | |||
| 6. ??? | 6. ??? | ||
| 7. Profit | 7. Profit | ||
| Line 401: | Line 407: | ||
| The deployment process is identical for both internal and external services. | The deployment process is identical for both internal and external services. | ||
| - | ===== Backups ===== | + | ====== Backups |
| Everything, private and public, is backed up with [[https:// | Everything, private and public, is backed up with [[https:// | ||
| - | ===== Cost ===== | + | ====== Cost ====== |
| Line 412: | Line 418: | ||
| Public bill: | Public bill: | ||
| - | * Compute: $40/mo | + | * Compute: $24/mo |
| * B2 Storage (~200gb): ~$2/mo | * B2 Storage (~200gb): ~$2/mo | ||
| - | * Total: ~$45/mo | + | |
| + | |||
| + | {{tag> | ||
personal_infrastructure.1694214543.txt.gz · Last modified: 2023/09/08 23:09 by qlyoung
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International