Dane.Kouttron

[02.02.22]  History of Web Hosting transistor-man

It's story time!

I have a peculiar website: I choose to host (almost) everything locally and use static HTML.
This makes things incredibly portable but somewhat painstaking to maintain, especially as view-ports changed from 4:3 -> 16:9 -> 9:16 (mobile phones).
With a decade of projects and literally 200+ GB of related photos and data-sets, I moved my hosting around a few times.
This page documents those experiences.

The beginning of transistor-man.com

I started transistor-man.com while I was working on my undergrad at RPI. Prior to buying the domain, I hosted using a free domain redirect service, under dklabs.co.nr. This wasn't great but it was free, I was using RPI webservers with an external free domain service. The quotas at RPI were ~150mb for your public file share and they also used a non-standard version of Apache which looked for CAPITAL I index files, which, is a legacy issue that I still need to fix on my personal website.

I picked the domain transistor-man while I was completing my summer internship @ DEKA research, in Manchester NH. Having just launched a 'science' weather balloon [link], I needed somewhere to put photos and document projects. I was inspired by Charles [etotheipiplusone.net] to setup some kind of documentation / personal website. I got to work on some very slow New Hampshire DSL and this website was born. Webhostingpad worked fairly well for nearly 3 years, but like anything, I ran into space constraints.

Here's an archive.org version of this website, from 2009. Its a bit more rectangular, with a heavy emphasis on html tables, but it works!
You can check it out here: [archive.org link]

Generally speaking, so far, I've moved from on-campus hosting @RPI -> Webhostingpad -> Bitaccel -> Budgetvm -> Self-hosting MIT

BitAccel


After running out of space in Webhostingpad, I did some searching and lowendbox had recommended BitAccel [link]. It was a reasonable deal, especially with a fairly good coupon code. The plan I purchased was a OpenVZ vps, with 2 CPU threads, 1GB ram 60gb storage, 1GBIT Uplink, hosted in Texas for around 4-5 $/ month. It was a fairly OK experience, but I did run into issues where there would be downtime, sometimes for over a day, and not really a detailed explanation as to why. Trace-routes were sometimes fairly odd, and transfer was maybe 90mbit on average, but again, this worked fairly well. I transferred to Budget VM after an extended BitAccel outage.

Budget VM


The service plan I was moving from was a dual core Xeon VPS with a 150gb spinning disk drive, 100mbit up-link, 4TB transfer, and 2GB of ram.
This is completely reasonable and actually a fairly good deal, I really had no issues with BudgetVm. 100mbit up is not terrifically fast but for all purposes it was never really a constraint. I chose Chicago as the VPS location nominally as it was mid-united states so the latency should be reasonable to either coast. Unfortunately I somehow had 150+ GB of photographs, video and project files, and the next step up was a bit pricey. A 200gb spinning disk bumped things up significantly. Realistically, I could have easily batch-compressed larger / raw photos. Not doing so was not a choice of being lazy, I dislike loosing information and detail for the sake of convenience.
Going on a trip to Detroit, taking 10GB of photos, and wanting to be able to upload them without space constraints is a really free-ing feeling.
I really dislike using 3rd party services for images, over the last 10 years this website existed Googles' image sharing has changed three times. Each change broke previous versions. Picassa evaporated, image resolution hosting requirements changed and the carpet keeps getting pulled from under users. 

Hosting @ MIT

Budgetvm was great, but academia has its perks (They are increasingly fewer and fewer, but some still exist). In the before times, MIT actually had the whole '18 block' of IPV4 IP's, so like literally 1/256'th of all addressable IP space. I was paying [~2019] ~80$+tax /yr to host @ Budgetvm, which was not terrible, it worked remarkably well for the price. I was however running into a bit of a boundary condition: space. I was bumping up against my 150gb of storage quota. For about 2 months I played games trimming some things down to continue using the service, but the signs were clear, I wanted an open field to roam about in, not a cramped container.


Time to build a self-hosting web-server (with no moving parts)




Freebie 1TB ssd's
A quick look at the 'new' hardware:

For self-hosting, we need a small single board computer. Initially I was looking out for an Intel NUC as they are mechanically small. Some support 2.5" drives, some support 12v inputs and some are fan-less. After some hunting I couldn't find one for a reasonable price that had all three features. I took a look at the NUC knockoff's from Zotac, but I wasn't very impressed. I really wanted all three, and I found a really curious contender: The fitpc4 pro. When i say i was shopping around, i actually just came across this in a lab clean out. Thanks CSAIL. This appears to be a digital signage computer, its one of those things that sits behind a screen and runs 24/7 showing some kind of advertisement for an event or a reminder, or the like. Fan-less was really important, as it removes a potential major mechanical thing that's prone to failing, especially in drafty dusty MIT Albany street buildings. 
Battery Backup

Given that my lil single board computer could run off of a variable power source, ranging from 10-20v, I was able to get away with a DC-UPS. These things are intended for consumer router / modem backup power. While they aren't industrial grade, they are pretty reasonable for this application. The way they work is incredibly simple: A mains power supply charges a '12v' Pb Acid battery and presents the battery terminals out to the target device. As long as your target load is less than the battery charger (less than 36w), that's it you're system is up and running. One benefit of this approach is simplicity, if i went the mains-style UPS approach, I would be going 110v -> ups -> 24v battery -> 110v inverter, each step shaving off a bunch of efficiency and adding complexity.

This UPS is a DTC36U12V, 36W max output. It does have a big brother, the DTC50U12V, which has a 50W max output. The datasheet [link] [local copy] is available and here's the product page [link]. The build quality is reasonable, and the front facing LED's are fine.

An adorable package

This is it, the single board fitpc4 fits remarkably well on top of the tiny 12v ups. In this photo the server is literally still running. A single oversized ziptie between the heat sink and the ups is really all that is holding it together. On the back there is a yoctovolt USB connected voltage monitoring board [link] so I can read the SBC input voltage. As this is also the same thing as the battery voltage, we can log it to represent UPS status. Otherwise non-used ports have gaffers tape over them to limit dust intrusion, and the only real input is the UPS mains connection and the gig-e network connection.

Lets speed test.

To get a best-case speed test, I opted for testing on a Saturday afternoon. Relatively noone is in the building, so this should give me a rough estimate of maximum upload and download is. So best case we see 924 Mbps Down and 932 Mbps up. Admittedly I did this test earlier on but didn't save a screenshot, so here's a run in early 2022 for reference. For speed testing i used fast, which is Netflix's speedtest cli. You can install from npn or snap fairly easily. I chose to install thru snap with
sudo snap install fast
Pretty snappy for a single board computer. This confirms the hardware is working, and its provisioned on the MIT network with correct permissions.

Some mechanical bits

Inspecting and prepping the DC UPS

The cyber-power UPS is small, and internally just consists of a ~50w open board power supply PCB, and a cavity for a 12V7 style battery module. Shown (right) is some conformal coating spray. I opted to coat portions of the board (not heat-sinks, just IC's). Conformal coating helps prevent humidity from effecting ic's or high impedance circuits. I was somewhat surprised the board was not coated from the factory but then again it is more of a consumer-grade device. The cabling to the 12v7 battery was fairly mediocre, roughly 18 or 20 gauge, and while the maximum current the battery sees is only 3A, it would have been a bit more pleasant to see 16 gauge wire.
Removing the buzzer and some PCB analysis

The buzzer used to indicate when power is not available is not 'mute-able' in a permanent fashion. You can silence the unit on power loss, but, that needs to happen any time a power loss event occurs. I wasn't interested in this, so I de-soldered the buzzer.  This has no effect on the performance of the little UPS.

It is remarkable how little there is on this UPS. It appears that the 36 max wattage, is actually closer to 50w total input power, the supply is configured as a CC-CV supply, such that the battery charge rate maximum is 50watts. This is visible as, during a deeper discharge, with no load connected, the UPS pulls ~50w from mains. If the external load pulls 25w, the remaining difference will go into battery charging.

One side effect of this is the output voltage is not regulated it is more or less raw battery voltage. More curiously I do not see a DC side Fuse, which is somewhat mediocre, they may be relying on a trace acting as a fuse.
Lets verify the power supply and UPS

The UPS has a 36w max limit for its output, and given that this is ~consumer grade hardware, I'd prefer to keep it far away from that threshold. To test the power consumption. A really easy way to do this is to use the program 'stress'. Stress is intended to load cores for stress testing. In this case I want to see how much power I pull from the wall at various states of load testing.

For the sake of curiosity, we're going to stress test 1-4 cores and verify that our inlet power is less than that threshold. Initially i had thought about just measuring from a mains watt-meter, however they are terribly inaccurate at low power. I opted instead for a recently calibrated BK precision bench supply, feeding the board directly from a barrel jack. The multiplication of the input volts and amps gives us a great indication of input power required. Bizarrely, I do not get CPU temperature from this particular single board computer, I tried all the usual tricks but alas its not available from sensors or any of the normal Ubuntu sensor packages.

Observing how CPU load effects UPS volts

With stress running on all 4 cores for 10 minutes, I did notice that the external voltage monitoring on the 12v dc-input did show a 20mV drop, what was 13.65v in became 13.48v. It did level off, so i know that the battery itself was not discharging, but it was interesting that CPU load is visible on the UPS voltage plots. Note that this was running 4-core stress, starting at roughly 7:04 on this plot. After discontinuing the stress test it bounced right back up to 13.6V.
What about power outages @MIT

Interestingly, MIT network gear is locally battery backed up. I'm fairly certain this is to ensure that phones (almost every phone is POE powered) work in case of an emergency / power outage. When the building the web-server is in is powered down for an extended outage, what happens to the web-server?

It keeps running! During this particular power outage (6hrs and 15 minutes), shown right, I had no issues accessing this website, and speeds remained the same.

 For this example we're observing a plot of UPS volts vs time, with a building power outage starting at roughly 8:15AM and ending at 2:30pm, we learned that a 6 hour power outage has no effect. Note that 6 hour long outages on campus are not terribly common and mostly tied to building specific repairs, etc. Its completely opaque as to what the power backup is for the network gear, but its likely a combo of backup generator + local batteries. Note this particular outage the UPS was running a K2 systems 6.4ah 12v7 style pack. At the time of this writing K2 systems now have an 11 amp-hour battery in the same form factor, which is just remarkable. I'd guess that approaching 11.8v should be the minimum state of charge voltage on this graph, so there was still some reserve remaining. If I were to estimate it'd be around 8-9 hrs maximum on a 6.4ah battery, so likely closer to 13-14 hrs of backup on a 11ah battery. The concept of having that much run time on something so small is wonderful.


But Wait! You need a content distribution network!

Its somewhat silly how much preaching there is on hosting on the big clouds, be it AWS or Azure. One of the motivators for moving from a VPS provider to, a single board computer was this posting from ycombinator news [link]. While this posting was in 2018, its still accurate in 2022.

Ycombinator news runs on what is effectively something that shows up on the media-lab loading dock for recycling. Given that I get 20-40k request monthly, not 4 million daily, the single board "10w" computer seems reasonable.

After a year of hosting @MIT, I really should have done this sooner. Performance has been fairly stellar, and I save ~90$ / year on hosting or approximately 8 burritos. I removed any advertisements that I had, to comply with the IS&T hosting rules. Adsense was fairly useless anyway, I think over 3 years it had acquired 72 USD, so its not like it was paying its own fare for hosting.
In terms of issues, one of the only real holdups is the somewhat-outsourced MIT IS&T network team, they are unfortunately incredibly vague and opaque.




Analytics and AWStats

I spent a significant amount of effort cleaning out google analytics from this website, which takes a bit due to so many static HTML pages. I opted to self-host, which has its limits, but is nominally free. AWStats [link] is very basic, but does a reasonable job in displaying things like hits, bandwidth and countries visitors come from. In 2021 I didn't really release any big projects mildly because the world was on fire. For 3/4 of a year we see roughly 2.2TB of transfer. A majority of this is just JPEG's. 90% of traffic is loading images. Interestingly i started using embedded mp4's instead of animated gif's towards 2022, so it will be curious to see what the ratio of JPEG's to mp4's become.

What did we learn from 3/4 of a year of traffic?
I was honestly surprised: 1/4 million hits / month & 1/4 TB of transfer per month. That's incredible, for a single board computer the size of a shoe and a single Ethernet port in a normal office. Folks all around the world tuned in to look at 2TB of my content, be it photos of snowy robots or silly contraptions.
AWStats pros and cons.

AWStats is locally hosted and fairly basic, it gets around the GPDR because well, it doesn't collect anything it just parses normal Apache log files. I don't know

MIT network
One mistake I've made was port-scanning. If you are trying to diagnose something and you port-scan from an external address, congrats you're likely booted from accessing MIT resources for a few hours.





Trying to buy transistorman.com





(There's other photos in the photo gallery)
Concluding Remarks:

If you have questions or comments, ask below or send over an email.
Comments:
HTML Comment Box is loading comments...


Dane.Kouttron
Rensselaer Polytechnic Institute 
Electrical & Electrical Power
631.978.1650