qlyoung's wiki

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
personal_infrastructure [2023/09/08 23:10] – [Private] qlyoungpersonal_infrastructure [2024/06/16 01:02] (current) – [Cost] qlyoung
Line 1: Line 1:
 +====== personal infrastructure ======
 +
 +
 //tldr - docker compose, isolated servers// //tldr - docker compose, isolated servers//
  
Line 29: Line 32:
   pub((public))   pub((public))
   priv((private))   priv((private))
-  access(needs to be publicly accessible) --> pub +  data{stores sensitive data?} 
-  data(stores sensitive data) --> priv +  onlyme{used only by me?} 
-  onlyme(used only by me) --> priv+  data --> |yes| priv 
 +  data --> |no| onlyme 
 +  onlyme --> |yes| priv 
 +  onlyme --> |no| pub
 </mermaid> </mermaid>
  
Line 40: Line 46:
 ===== Public ===== ===== Public =====
  
-For the public sphere, I use cloud-based Linux VMs from one of the affordable providers. I run most of my stuff on a single shared CPU VM with 4 CPU cores, 8gb of RAM and 50gb of disk space (storage is a later section).+For the public sphere, I use cloud-based Linux VMs from one of the affordable providers. I run most of my stuff on a single shared CPU VM with 4 CPU cores, 4gb of RAM and 50gb of disk space (storage is a later section).
  
 For things that need to be exposed in the internet I think cloud is the best choice. From a network isolation perspective serving things from your home means untrusted traffic will be flowing within your home network. Apart from security concerns there are other problems: For things that need to be exposed in the internet I think cloud is the best choice. From a network isolation perspective serving things from your home means untrusted traffic will be flowing within your home network. Apart from security concerns there are other problems:
Line 49: Line 55:
   * Dynamic IP addresses makes access annoying   * Dynamic IP addresses makes access annoying
   * Advertising your home IP address is a mild security concern   * Advertising your home IP address is a mild security concern
 +    * can be mitigated if you rent an ultra cheap VPS and use it as a gateway; I did this initially but realized I can run everything on a cheap vps to begin with
   * Weather / residential power outages impact uptime   * Weather / residential power outages impact uptime
  
Line 60: Line 67:
  
   * Case: Fractal Node 804    * Case: Fractal Node 804 
-  * Motherboard: [[https://www.asrockrack.com/general/productdetail.asp?Model=X570D4U#Specifications|[Asrock RACK X570D4U]]+  * Motherboard: [[https://www.asrockrack.com/general/productdetail.asp?Model=X570D4U#Specifications|Asrock RACK X570D4U]]
   * RAM: 128gb ECC memory (4x [[https://www.kingston.com/dataSheets/KSM32ED8_32ME.pdf|KSM32ED8/32ME]])   * RAM: 128gb ECC memory (4x [[https://www.kingston.com/dataSheets/KSM32ED8_32ME.pdf|KSM32ED8/32ME]])
   * Storage:   * Storage:
Line 84: Line 91:
  
   * Much better control and monitoring of hardware resources   * Much better control and monitoring of hardware resources
- * Full suite of hardware monitoring tools built into a clean web interface; disk diagnostics, memory / CPU / network dashboards, etc. +    * Full suite of hardware monitoring tools built into a clean web interface; disk diagnostics, memory / CPU / network dashboards, etc. 
- * Ability to easily partition storage resources among VMs +    * Ability to easily partition storage resources among VMs 
- * Ability to set fine grained memory and CPU resource limits+    * Ability to set fine grained memory and CPU resource limits
   * Full snapshots for tenant VMs   * Full snapshots for tenant VMs
- * Ability to snapshot the entire machine (VM) at the "hardware" level before making major changes is very valuable; makes risky operations like OS upgrades risk free+    * Ability to snapshot the entire machine (VM) at the "hardware" level before making major changes is very valuable; makes risky operations like OS upgrades risk free
   * Ability to create new machines whenever I want to try something   * Ability to create new machines whenever I want to try something
- * E.g. I have a Windows VM that runs some Windows-only server software +    * E.g. I have a Windows VM that runs some Windows-only server software 
- * I can spin up a VM with resources of my choosing to try out an operating system or run some experiment without worrying about damaging any infrastructure I rely on+    * I can spin up a VM with resources of my choosing to try out an operating system or run some experiment without worrying about damaging any infrastructure I rely on
  
 Of course, you don't really need a dedicated hypervisor to do any of this; you can do it all with KVM on a traditional bare metal server OS. But it's much easier and more convenient in a hypervisor and in practice, there's very few downsides to the hypervisor. Of course, you don't really need a dedicated hypervisor to do any of this; you can do it all with KVM on a traditional bare metal server OS. But it's much easier and more convenient in a hypervisor and in practice, there's very few downsides to the hypervisor.
  
 The main downside is that hardware passthrough can be tricky. I pass through the RTX 3070 to the VM, and then into docker containers, in order to get accelerated encoding for media related services. KVM GPU passthrough is annoying. The main downside is that hardware passthrough can be tricky. I pass through the RTX 3070 to the VM, and then into docker containers, in order to get accelerated encoding for media related services. KVM GPU passthrough is annoying.
- 
---- 
  
 The OS that applications run on (ignoring Docker) is always a recent Ubuntu Server LTS with automatic security upgrades enabled. I find the Debian-based platform familiar, comfortable and stable. Ubuntu Server also has a very wide install base which means most problems can be resolved with a web search. This helps keep the maintenance overhead as low as possible. I enjoy building my infrastructure, but I do rely on it and dislike being forced to fix it, so choices in system software tend to be on the less exotic side. The OS that applications run on (ignoring Docker) is always a recent Ubuntu Server LTS with automatic security upgrades enabled. I find the Debian-based platform familiar, comfortable and stable. Ubuntu Server also has a very wide install base which means most problems can be resolved with a web search. This helps keep the maintenance overhead as low as possible. I enjoy building my infrastructure, but I do rely on it and dislike being forced to fix it, so choices in system software tend to be on the less exotic side.
Line 243: Line 248:
 </mermaid> </mermaid>
  
-If possible, I also configure applications so that users are served files directly from B2 rather than having them proxied through my VM, which improves performance both in terms of load time and bandwidth usage. When you watch a video on my Peertube server, your browser is downloading the video directly from B2, instead of `B2 → VM → You`.+If possible, I also configure applications so that users are served files directly from B2 rather than having them proxied through my VM, which improves performance both in terms of load time and bandwidth usage. When you watch a video on my Peertube server, your browser is downloading the video directly from B2, instead of ''B2 → VM → You''.
  
 ====== DNS & HTTP & TLS ====== ====== DNS & HTTP & TLS ======
Line 270: Line 275:
 ===== DNS ===== ===== DNS =====
  
-Each service is on its own subdomain. All subdomains `CNAME` to the main domain name (`qlyoung.net`) which has an `A` pointing to the public IP of the VM. In my private infrastructure, the `A` record instead points to a Tailscale IP - but more on that in the networking section.+Each service is on its own subdomain. All subdomains ''CNAME'' to the main domain name (''qlyoung.net'') which has an ''A'' pointing to the public IP of the VM. In my private infrastructure, the ''A'' record instead points to a Tailscale IP - but more on that in the networking section.
  
 ===== HTTP ===== ===== HTTP =====
  
-nginx runs on the host and binds host ports 80 and 443. All docker containers bind to `(127.0.0.1, P)` where `P` is a host port number of my choosing. Each service has its own subdomain and a corresponding nginx configuration:+nginx runs on the host and binds host ports 80 and 443. All docker containers bind to ''(127.0.0.1, P)'' where ''P'' is a host port number of my choosing. Each service has its own subdomain and a corresponding nginx configuration:
  
 <code> <code>
Line 385: Line 390:
 </code> </code>
  
-4. Set up new `CNAME` record; `CNAME recipes.qlyoung.net -> qlyoung.net`+4. Set up new ''CNAME'' record; ''CNAME recipes.qlyoung.net -> qlyoung.net''
 5. Configure nginx, request and install TLS certificate 5. Configure nginx, request and install TLS certificate
 <code bash> <code bash>
Line 396: Line 401:
    $ certbot --nginx -d recipes.qlyoung.net    $ certbot --nginx -d recipes.qlyoung.net
 </code> </code>
 +
 6. ??? 6. ???
 7. Profit 7. Profit
Line 401: Line 407:
 The deployment process is identical for both internal and external services. The deployment process is identical for both internal and external services.
  
-===== Backups =====+====== Backups ======
  
 Everything, private and public, is backed up with [[https://restic.net/|restic]] to offsite locations. It runs daily on a ''cron'' job. Everything, private and public, is backed up with [[https://restic.net/|restic]] to offsite locations. It runs daily on a ''cron'' job.
  
-===== Cost =====+====== Cost ======
  
  
Line 412: Line 418:
 Public bill: Public bill:
  
-  * Compute: $40/mo+  * Compute: $24/mo
   * B2 Storage (~200gb): ~$2/mo   * B2 Storage (~200gb): ~$2/mo
-* Total: ~$45/mo+  * Total: ~$26/mo 
 + 
 +{{tag>from_blog technology}}
Panorama theme by desbest
personal_infrastructure.1694214649.txt.gz · Last modified: 2023/09/08 23:10 by qlyoung
CC Attribution-Noncommercial-Share Alike 4.0 International Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International