Categories
General Media Pool status updates

ADA North Pool adds jormungandr monitoring

Had another long night where I tweaked the node cluster (less nodes but more quality to each nodes with more connections and file limits for each) and also optimized the server painstakingly reducing one by one some metrics I know affects the quality the serer has to read and process incoming connections fast until I found an optimal point. After this the server seems again more stable (hoping this time it is going to last for a while!)

Now with the server more stable I have had some time to add more “bells and whistles” type of features so I have now not only a hardware monitoring but also a Jormungandr server monitoring that user can look how the server is doing right here on the web page, I have added these as menu links. As always feel free to contact me for encouragement / suggestions.

Categories
General Hardware Pool status updates

ADA North Pool moves main server to SSD

With a very high input/output for our files I have decided that its more important with NVME SSD performance for input output file operations than 4Terrabyte of RAID and will likely switch over to full NVME hardware if we are successful in the Testnet period. It also has the side benefit of a 90% less watt usage per disk.

Categories
General Pool status updates

ADA North Pool ads Google BBR for protection against TCP network congestion

To improve system performance we have added several tweaks and also some software that will help us better handle large connection loads that is needed to keep having an “overview” of the network that will help our server not following forks. One such measure is the google BBR algorithm that has been tried and tested by a major corporation in high stress situations. https://cloud.google.com/blog/products/gcp/tcp-bbr-congestion-control-comes-to-gcp-your-internet-just-got-faster

Hopefully epoch 7 will be a good one! To be sure we are still following with dedication and keep improving the network performance looking under any stone we can find.

Categories
General Pool status updates

ADA North Pool performs well during a “forking” epoch 6

I had to pull an all nighter as even while the new cluster of nodes is helping out a lot in stability it did not help when there where suddenly a strong propagation in the network for several forks. To help with this issue we also increased the network capacity of the main server so now it sees most of the active nodes in the network. This allows the server to look at what is the longest chain and correctly identify what path to take regarding forks.

Also when you look at the numbers we had an amazing performance. We lost some blocks during the early time period of forks but around 4 AM (as I told I did an all nighter for this) after also bootrapping to the nearest IOHK servers (3 servers are located in EU) and implementing these changes the pool performed admirable. Lets look at the math:

We know that 4320 blocks are on offer each epoch (10% of the total 43200 slots) We also know that currently only around half of this is actually produced by pools (around 2600). In our case we have a stake of around 66 million delegated to us and 0.95% of the total stake. With 4320 blocks we should with 0.95% recieve 41 blocks. Guess how many blocks ADA North Pool produced during the epoch? 41! So overall given all that happened epoch 6 I am actually very pleased with the performance and keep in mind that performance matric is a short term performance metric, you should also ask yourself the question how many blocks is this pool producing compared with its stake?

Categories
General Pool status updates

ADA North Pool adds cluster of monitoring nodes for better performance and updates html server

Epoch 5 was rough on ADA North Pool who only managed to produce around 60% of blocks. Granted it was rough for many other pool operators but ANP prides itself in trying to the best of its ability to produce blocks. To that extent we have utilized our strong server hardware and added a cluster of nodes that will help keep the network stable and performing well. We have also done several more tweaks to the nginx html server to improve its performance so the site is available.

Categories
General Media Pool status updates

Fixed website and added more features to error log parsing script for automation of server

First of all I would like to apologize that the webserver has been down for 24 hours. The site was simply not ready for the kind of load that came with the Test Net 0.8.2 ITN when all users of the Deadelus wallet wanted to access it. I have done a lot of optimization these last hours and hope to come back much stronger because of this. Due to DNS propagation it will take 24-48 hours before users see these changes. As I said earlier I am learning this as we go.

In that regard we also had some trouble with bootstrapping simply getting stuck due to the massive amount of bootstrapping calls to IOHK servers. To make sure this is not a problem in the feature we have added both our own backup nodes, private nodes in a pool alliance and added more features to error script logging for automation of handling any incidents.

In short I am still learning but the pool and the website is getting stronger and stronger day by day.

Categories
General Pool status updates

ADA North Pool adds log error parser for automation

As the 0.8.2 ITN had its first problem with block not moving up ADA North Pool has added a temporary fail safe in an automated script to make sure server resets if the block is not moving up again. More longer term we are working into more advance automation of logging tools (probably trough Greylog) but for the time being this is an easy workaround.

Categories
General Pool status updates

ADA North Pool 0.8.2 ITN server reset – service offline for less than 1 second

ADA North Pool had its first error that caused the service to have to reset. This took somewhat less than 1 second so did not in any way affect the pools ability to produce blocks but in line with transparency and since we are in TestNet phase this is reported here on server status updates.

Sensitive IP data edited away. A connection error caused an error (code 24 – reported to IOHK) and this caused our service to terminate.
As can be seen by the service active stat since 22:03:44 the service rebooted and where back up again roughly 1 second later. Given that you have a small time frame window to produce blocks this would not have affected the pools ability to produce a block only giving it a slightly higher time from wake up to finished if unlucky.
Categories
General Pool status updates

ADA North Pool produces it second block

Another historic moment ADA North Pool has now been lucky to be selected for a block to be produced in two epochs in a row. Also this epoch our block was produced successfully and took 0.0033 seconds from wake up time until finished time.

Another historic moment 0.8.2 ITN Epoch 2 block scheduled and produced in slot 3116.

Categories
General Media

#historyofcardano and Daedelus reward wallet very close

Given we are at a quite historic moment I tought I should give some praise to some of the people (I have definitively not an overview of all the amazing work done so feel free to add in comment other persons who made an impression on you) that has made an impression on me during the early days of testnet.

#historyofcardano https://twitter.com/adanorthpool/status/1205999519881682945 https://twitter.com/adanorthpool/status/1206000325058027520 https://twitter.com/adanorthpool/status/1206000921261625344 https://twitter.com/adanorthpool/status/1206002426530213888 https://twitter.com/adanorthpool/status/1206002803170250753 https://twitter.com/adanorthpool/status/1206003318872559619

Finally a tiny sneak peak: Daedelus reward wallet is coming very soon!