ADA North PoolI notice degrading performance in our block production and decided this was a good opportunity to install the Mellanox Connect X 4 card for the server. We have not yet installed the new router as we are waiting on some transceivers but having now installed the network card and it is operational should cut downtime when we do change over new router as well.
We had a few rough periods and needed to rethink our approach. In that regard we made a new server structure as can be seen below in the image.
This has improved our stability and now our main pool server has had an uptime over a day without any problems. We are converting now around 82% of blocks an improvement of 75% from previous settings.
Finally an update on network equipment: Router has arrived and network cards as well. But we are waiting for the transceivers / cable equipment. Probably by next week we have everything and ready to install. The installation is likely to take half a day or with maximum preparations I could probably manage to get the pool up and running again in 1-2 hours. I will try to select a point in between blocks that is optimal – probably during next weekend.
On the cardano forums there has long been an active kiva lending team (see https://forum.cardano.org/t/cardano-community-kiva-lending-team/17613/26 ) and given that we are now almost half way to paying down the hardware of the server I figured it was time to also try and reach out for a good cause. You could argue its selfish in branding the pool but at the same time I hope it is something that also other pools would consider and that it could become a positive thing for the Cardano community that not only has there been an active lending team but also now pool operators help as well.
In any case I have picked two projects from Philippines as that is where my wife came from and it has a special place in my heart. Today is her birth day and we decided we wanted to do something special so we wanted to support Kiva projects in Philippines. I know also well the social system there and I know that a sari-sari store is very much ingrained in the culture there and supporting someone making one is a good cause. The same with jeepneys and motorcycle / tricyle type of transportation projects that is very much part of everyday life.
Finaly perhaps this post is also a good time to direct some attention to an idea I have regarding how to cut cost for oversea family members to cut costs for transfers to family members. https://forum.cardano.org/t/possible-idea-for-family-economic-connection/25247
ADA North Pool invests in the Mikrotik RB4011 for more powerful router hardware, this gives us capability to utilize some of the amazing performance of the Mellanox Connect-X 4 cards we have purchased. The router can handle roughly 5-5.5 million packets per second worth of routing small packages of 64 bytes (typical for staking pools who only need to transmit small info to each IP address)
As a first step to a potential data center class capability ADA North Pool as purchased both a Mellanox ConnectX-4 MCX4111A-XCAT 10Gigabit Ethernet Card and a Mellanox ConnectX-4 MCX4121A-ACAT 25Gigabit Ethernet Card. In theory these cards can in optimal situations handle 75 million TCP packets of information per second.
I tought I would share my current screen setup on the server. I am using terminator bash terminals and hardware monitoring is your standard stuff like Glances, Htop and Iotop. But also I use Chrony for making sure my system clock is up to date and tuned to make sure my network is up to date, both running as services on the computer. I had my own custom made scripts for forking / bootstrap checking but replaced them with the great script Redoracle made. I also run Prometheus, Grafana and Nginx for monitoring and websites and a few security measurements, one of them that I think its fine to mention is I run Fail2ban. I have removed some information that could affect security and replaced with a red bar.
Celebrating a milestone for ADA North pool we have come a long way since epoch 1 and 2 where we produce one block in each epoch! We have tweaked the numbers even more when it comes to connection settings and believe we have found a sweet spot that will give us even more optimal operations.
To keep everything running smoothly we upgrade to latest packages for our webserver technology and we also added chronyd to make sure our server keeps running at correct time (in millisecond measurement ranges) while also adding maintenance software that automates upgrades and keeps the system tuned and performing over time.
I compiled 0.8.5 because the pre-compiled options do not offer Journald log support and you can get this if you compile the binaries yourself.