Blogpost: Monoliths can do Multicloud. Using Kamal to partially get out of big tech

[yoelcabo]

At Happy Scribe we've used Kamal to do a partial migration out of AWS.

We moved the expensive long-running jobs to Hetzner while keeping the web nodes on good-old Heroku.

A remarkable thing is that we took the opportunity to also bring those jobs back to our monolith from very ugly Fargate microservices. All thanks to Kamal making the deployment straightforward. Thanks a lot!

We are a very small tech team, so it will take us some more time to do a full migration. But I think we'll get there 🦾

Our full post below ⬇️

https://www.happyscribe.com/blog/en/monoliths-can-do-multicloud

[nickchomey]

Very cool, congrats! Which servers do you use at Hetzner?

And did you consider backblaze or wasabi instead of R2? They're far cheaper and are part of cloudflare's bandwidth alliance.

[yoelcabo]

Again really good questions 🎯

I'm curious why you chose 8 EX44 rather than some combination of their other servers - 4x EX 101, 4x AX52 2x AX102, 1x RX170 or even one of the DX servers?

Honestly, I didn't put a lot of thought on the specific hardware. I benchmarked our transcoding workloads with s an AX41 and the EX44 and also with some Hetzner Cloud servers. I saw AX41 was 3x faster than our Fargate setup (4 fargate vCPUs) and that EX44 was 4x. EX44 had alsoless CPU usage (of course: it has more cores), which meant I could make things cheaper

Maybe in the future we'll do another iteration of benchmarks to choose an even better hardware for our use case, but at the current speed and cost we are happy with this 😄

Yeah, I'm not a fan of Wasabi's 90 day policy either. But Backblaze doesnt have such a policy, has similar pricing (much cheaper than R2) and free bandwidth due to the bandwidth alliance. Did you consider them?

I did! But as far as I know, Hetzner is on the bandwidth alliance, which would mean we would need to pay transfer rates for processing the files our users upload. Another big cost factor was the download of videos with hard-coded subtitles by our customers. As you can see, we have lots of transfer and temporary files!

One other interesting datapoint: We also run some simple file download tests with AWS, Wasabi and Cloudflare (Backblaze didn't make it here either) and it seemed Cloudflare was way faster than Wasabi.

[nickchomey]

Yeah that all makes sense. It would be worth testing an ax52 for a month - the processor is considerably faster than the ax41, and whatever transcoding etc you're doing might also benefit from new low level stuff that's included in the zen 4 processors (eg avx512).

I don't think Hetzner is in the bandwidth alliance. But as far as I know they generally provide unlimited bandwidth. But backblaze IS part of the alliance, so it's just as good as using r2 in that regard - but cheaper.

Anyway, thanks for the detailed responses. As well as for sharing your company's experience about moving off-cloud!

[yoelcabo]

Thanks a lot for your suggestions ✨

BTW, I just checked Backblaze pricing again, and it seems it changed since last time I checked a few months ago:
https://www.backblaze.com/blog/2023-product-announcement/

Now they don't charge for data transfer unless it exceeds 3x of your storage, which is totally fine for our use-case of customers exporting their videos with subtitles. And of course unlimited inside the bandwidth alliance. Indeed as you were saying with no minimum storage policy 🙌 It seems they would be our ideal storage partner. I'll definitely benchmark their performance and go for them if they are as good as the others! Thanks a lot!

BTW maybe I am misunderstanding what the bandwidth alliance means, but I understand downloads from Backblaze to Hetzner would actually count as outside the alliance. So in the previous pricing they had ($10/TB egress), we would be paying for that.

[nickchomey]

Glad to be helpful! Please let me/us know when you've benchmarked their performance!

Yeah, I think you're right about the bandwidth alliance - it's only free within the alliance. If you transfer data to/from a provider outside of the alliance, you would get charged. I wasn't aware of backblaze's previous pricing, but now that it is free (for up to 3x the storage volume) it shouldn't be a problem. It all depends on your application though.

For my project, I intend to use Hetzner for the application, backblaze for off-site object storage and cloudflare for caching, workers, api gateway etc.

Though may also use CF R2 and/or Streams for serving videos - they are fairly explicit in their TOS that you're not really allowed to be serving video via their cdn unless you pay for such products. https://blog.cloudflare.com/updated-tos/

[nickchomey]

I actually just bought an EX44 for something else, and after buying it I realized it only has DDR4 RAM whereas all of the others have DDR5. Just one more reason to consider other servers.