It took a while but I am really happy with the custom made trancievers from fs.com that will fit to the brands of my network cards at a fraction of the cost if I bought it directly from the network supplier. This means we will at some point soon have a downtime but I want to have a good epoch – so far we have done 18 out of 19 and we could make up to 10 more this epoch. Given we had some poor luck in draws last two epochs that means I want to wait a bit before doing the rest of the networking equipment. I will update on twitter when the server will have some downtime.
We had a few rough periods and needed to rethink our approach. In that regard we made a new server structure as can be seen below in the image.
This has improved our stability and now our main pool server has had an uptime over a day without any problems. We are converting now around 82% of blocks an improvement of 75% from previous settings.
Finally an update on network equipment: Router has arrived and network cards as well. But we are waiting for the transceivers / cable equipment. Probably by next week we have everything and ready to install. The installation is likely to take half a day or with maximum preparations I could probably manage to get the pool up and running again in 1-2 hours. I will try to select a point in between blocks that is optimal – probably during next weekend.
ADA North Pool invests in the Mikrotik RB4011 for more powerful router hardware, this gives us capability to utilize some of the amazing performance of the Mellanox Connect-X 4 cards we have purchased. The router can handle roughly 5-5.5 million packets per second worth of routing small packages of 64 bytes (typical for staking pools who only need to transmit small info to each IP address)
As a first step to a potential data center class capability ADA North Pool as purchased both a Mellanox ConnectX-4 MCX4111A-XCAT 10Gigabit Ethernet Card and a Mellanox ConnectX-4 MCX4121A-ACAT 25Gigabit Ethernet Card. In theory these cards can in optimal situations handle 75 million TCP packets of information per second.
With a very high input/output for our files I have decided that its more important with NVME SSD performance for input output file operations than 4Terrabyte of RAID and will likely switch over to full NVME hardware if we are successful in the Testnet period. It also has the side benefit of a 90% less watt usage per disk.
Had a fun weekend with my daughter Hannah helping me build the server and throwing in a ton of gaming stickers on every surface of that server for good luck charms 🙂
Hannah with a helping handI am quite happy with the cable management here. Clean server. Maybe in 1 year or so I will do an update on how the server looks (and obviously maintenance. I am always happy when I have a screw loose that I can put back in its place 🙂2 UPS and server where they will hopefully reside for a while going forward.One of the screens of the server during testing. (0.8.0 RC3 not doing that great.)
I had some qualms regarding showing hardware or not because it is an issue of security vs building trust in community. I landed on thinking I have security measures as a “fish” in the big ocean and that if I scale I would probably buy more equipment and security measures. Besides – its fun learning this as I go along and nice to enjoy comments and feedback from readers in that journey. I am pretty happy the powerdraw will be around 65 watt max even for a server that has a lot of horsepower. So without further ado here are some current hardware that I picked up today:
One UPS for server. One for network. Overkill? Perhaps 🙂 32 GB ECC ram. (32 more on its way for 64GB total). Sadly no 3200 MHZ ram only 2667 MHZ available in unbuffered ECC in market (as far as I am aware)Ryzen 3700x and Arorus motherboard with latest Intel LAN chips and room for a lot of disks.2x 4TB enterprise disks and 2x 500gb NVME
To prepare for the coming testnet period with incentive rewards ADA North Pool has invested in hardware that should scale well for the next coming years. Here are some of the highlights (most brands are left out for security reasons):
For redundancy we have gone with two 1200 watt UPS from PowerWalker that will be our backup source both for the routers to keep internet up and running but also for the computers to keep them running during a power outage.
Two 4 Terrabyte enterprise disks in raid 1 that are tested and meant to run 24/7 and with a high reliability and long age before mean time of failure. Draws roughly 10 Watt in operational mode and some 5 watts in idle mode. The reason for such large capacity was a study I did where I saw on main net the largest epoch could consume up to 1.7 Gigabytes of disk space. So to scale well for the future we have room for 4000 gigabyte 🙂
Two NVME 500 Gigabyte disks for the server operation system. Draws around 2 watts in operational mode and less than a single watt in idle mode.
64 Gigabyte of 2666 MHZ error correcting ram. This will make sure the server is stable in a 24/7 environment where you could risk with regular ram having at least an 8% chance of a failure on regular types of RAM per year according to some studies, but these RAM are error correcting so will avoid this risk.
Ryzen 3700X for 8 cores (16 threads) at only 65 watts. There are some tweaks out there and I will be looking into lowering the wattage to around 45 watts hopefully as I want the staking pool to be as energy efficient as possible.
Finally while I will not detail the motherboard and what type of lan chip it uses, what I can say it is is a very highly regarded and has received many awards regarding stability and energy efficiency and LAN performance. The same goes for a new router that is ordered that has a very high input/output and high hardware specs and enterprise level security solutions.
