Week 11 and 12

During these weeks, I wrote a docker script for running these tests as part of Continuous Integration. This allowed the tests to be run as a Github Action after every push. This can be found here.

Project Report

Lnprototest

Lnprototest is a set of test helpers written in Python3, designed to make it easy to write new tests when you propose changes to the lightning network protocol, as well as test existing implementations.

Tests are simply a list of Events, like Msg to send a message and ExpectMsg to receive a response. But the tool goes much further: TryAll indicates multiple different variants should all be attempted, such as running the test with different features or trying reconnection in the middle of the test. It also has Events to create commitment transactions, add and remove HTLCs, and other things you’d expect.

Testers can also write their own custom Events for more specialized testing and Events can be conditional on what features were negotiated or other runtime behavior.

LDK

Lightning Development Kit is a library that allows you to build a lightning node without worrying about implementing low-level lightning logic correctly.

LDK Logo



LDK is based on Rust-Lightning, a full-featured but also incredibly flexible lightning implementation designed to be modular and customizable. It allows you to build a lightning node without having to worry about all of the low-level lightning logic like the state machine or routing.

The project

In order to integrate LDK with Lnprototest, few things should be kept in mind.

  • LDK is not a node, its a lightning implementation that can be used to create a node.
  • In order to integrate with Lnprototest, we need a node implemented using LDK.
  • For these reasons, I will be using LDK-Sample.

LDK-Sample is a sample node implementation which uses LDK.

In order to integrate it with Lnprototest, some changes are required in LDK-Sample and a ldk.py script is needed in Lnprototest. This script will contain a ldk runner class which will act as a middleman between Lnprototest and LDK-Sample.

This class consists of various functions which need to be implemented for ensuring proper functionality.

Progress Made

  • Implementing myKeysManager struct in LDK-Sample. This struct manages the node secrets and generates custom keys. The advantage of using this instead of the default KeysManager is that it allows me to use my own set of channel secrets(funding_key, revocation_base_key, payment_key, delayed_payment_base_key, htlc_base_key) instead of the randomly generated ones. This was useful for setting up the configuration required by Lnprototest.
  • Implementing KeysInterface trait for the myKeysManager struct. This was essential for myKeysManager as it has a lot of useful methods.
  • Implementing openchannel_without_peer_addr method in LDK-Sample. This method allows opening a fundchannel without providing the host or port.
  • Adding the ldk.py script inside Lnprototest. This script does all the heavy lifting in the integration process.
  • Implementing the LDKConn class inside the script. This class is used to connect to the node using pyln.proto.wire module and intercept all the messages that the node initiates and receives.
  • Implementing the LDK Runner class. This class contains all the functions used to start/stop bitcoind, start/stop LDK-Sample node, adding a connection, connecting to a peer, carrying out transactions, receiving output messages etc. The tests are run against this class and it basically tells the node to do a particular task(like connecting to a peer) and the LDKConn class intercepts the messages sent by the node.
  • Testing the BOLT tests present inside Lnprototest. This included a total of 45 tests.
  • Updating the tests so that they are compatible with other implementations apart from C-Lightning.
  • Introducing new methods like runner_features which helps with cross compatibility.
  • Ensuring that all the tests pass.
  • Writing a docker script to include LDK runner as part of Github Continuous Integration.

My work can be found in my forked repos.
Lnprototest
LDK-Sample

Conclusion

The project is completed and all tests are running successfully. Only merging to the main repos is pending. Through this project, I am happy to conclude that LDK is incredibly robust and spec compliant.
Throughout this project, I did not find any spec related bug in LDK which simply showcases the phenomenal work LDK Developers have done over the years.

The LDK community has been really helpful to me during this time. All the developers are friendly and have supported me extensively through my journey. I would like to thank my mentor Conor Okus for always being there for me and guiding me. I would also like to thank Matt Corallo, Jeffrey Czyz for entertaining my silly questions.
A special thanks to Vincenzo Palazzo for solving my complex queries and steering me towards the right path.