Introduction to the Bitcoin Network Protocol using Python and TCP Sockets

, October 22nd 2019

Blockchain technology is built around consensus algorithms that allow distributed nodes to share a common ledger. A fundamental dependency of these algorithms is a common network protocol to enable communication between participating nodes. Today, let's write a Python program from scratch to interact with a real Bitcoin node.

This post will assume you're familiar with the fundamentals of blockchain technology. If you aren't, I would recommend checking out the Bitcoin White Paper by Satoshi Nakamoto.

Bitcoin nodes speak to each other over the Internet

Bitcoin nodes communicate with each other using the TCP protocol. Nodes will typically listen on port number 8333.

For a detailed description of the bitcoin network protocol, check out this resource.

What does a typical TCP flow between two nodes look like?

Today, we are going to write a Python program to connect to a bitcoin node and fetch the details of a specific transaction. Here is a diagram of the message flow that will be developed.

Figure 1: Bitcoin Message Flow Chart

Let's talk to a bitcoin node using python

Before we start coding our program, we must make one point clear. Interacting with a node using raw TCP sockets is reinventing the wheel. This has already been done by python packages such as python-bitcoinlib.

If you want to write sophisticated applications, you should use the correct tool for the job. With that said, however, programming with TCP sockets is a great way to improve your low-level understanding of a network protocol.

Import the required dependencies

To begin, let's import the dependencies our program will require.

#!/usr/bin/env python

# Filename:                     bitcoin-network-tutorial.py
# Command to run the program:   python bitcoin-network-tutorial.py

# Import dependencies
import socket
import time
import random
import struct
import hashlib
import binascii

How to construct the "version" message

Let's now define the methods required for constructing the "version" request message.

# Binary encode the sub-version
def create_sub_version():
    sub_version = "/Satoshi:0.7.2/"
    return b'\x0F' + sub_version.encode()

# Binary encode the network addresses
def create_network_address(ip_address, port):
    network_address = struct.pack('>8s16sH', b'\x01', 
        bytearray.fromhex("00000000000000000000ffff") + socket.inet_aton(ip_address), port)
    return(network_address)

# Create the TCP request object
def create_message(magic, command, payload):
    checksum = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[0:4]
    return(struct.pack('L12sL4s', magic, command.encode(), len(payload), checksum) + payload)

# Create the "version" request payload
def create_payload_version(peer_ip_address):
    version = 60002
    services = 1
    timestamp = int(time.time())
    addr_local = create_network_address("127.0.0.1", 8333)
    addr_peer = create_network_address(peer_ip_address, 8333)
    nonce = random.getrandbits(64)
    start_height = 0
    payload = struct.pack('<LQQ26s26sQ16sL', version, services, timestamp, addr_peer,
                          addr_local, nonce, create_sub_version(), start_height)
    return(payload)

The struct module is used for packing binary data. The hashlib module is used for generating message checksums. For a full understanding of the code, you'll need to cross-reference the data encoding with the protocol documentation.

How to construct the "verack" message

Next, let's add a method for creating the "verack" message. The verack command name is derived from "version acknowledge".

# Create the "verack" request message
def create_message_verack():
    return bytearray.fromhex("f9beb4d976657261636b000000000000000000005df6e0e2")

How to fetch the details of a transaction

With the mandatory messages out of the way, we may now create our "getdata" method for retrieving the details of a specific transaction.

# Create the "getdata" request payload
def create_payload_getdata(tx_id):
    count = 1
    type = 1
    hash = bytearray.fromhex(tx_id)
    payload = struct.pack('<bb32s', count, type, hash)
    return(payload)

Please note that not all nodes will be able to return arbitrary transaction data; some will prune their history to save disk space.

We'll print our binary data as hex

We'll also create a method for printing TCP data to the terminal.

# Print request/response data
def print_response(command, request_data, response_data):
    print("")
    print("Command: " + command)
    print("Request:")
    print(binascii.hexlify(request_data))
    print("Response:")
    print(binascii.hexlify(response_data))

Let's get ready to execute out program

We may now add our main method that will connect to a bitcoin node and execute the desired message flow.

if __name__ == '__main__':
    # Set constants
    magic_value = 0xd9b4bef9
    tx_id = "fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20"
    peer_ip_address = '104.199.184.15'
    peer_tcp_port = 8333
    buffer_size = 1024

    # Create Request Objects
    version_payload = create_payload_version(peer_ip_address)
    version_message = create_message(magic_value, 'version', version_payload)
    verack_message = create_message_verack()
    getdata_payload = create_payload_getdata(tx_id)
    getdata_message = create_message(magic_value, 'getdata', getdata_payload)

    # Establish TCP Connection
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((peer_ip_address, peer_tcp_port))

    # Send message "version"
    s.send(version_message)
    response_data = s.recv(buffer_size)
    print_response("version", version_message, response_data)

    # Send message "verack"
    s.send(verack_message)
    response_data = s.recv(buffer_size)
    print_response("verack", verack_message, response_data)

    # Send message "getdata"
    s.send(getdata_message)
    response_data = s.recv(buffer_size)
    print_response("getdata", getdata_message, response_data)

    # Close the TCP connection
    s.close()

We found the IP address of the node using Bitnodes. Details of the transaction we elected to query can be found on a block explorer.

Execute the program!

Execute the program on a terminal with the command python bitcoin-network-tutorial.py. Sample output is provided below.

Command: version
Request:
b'f9beb4d976657273696f6e000000000064000000f4de76b762ea00000100000000000000c8c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d010000000000000000000000000000000000ffff7f000001208d0f2f736a397699b60f2f5361746f7368693a302e372e322f00000000'
Response:
b'f9beb4d976657273696f6e000000000066000000fe4aee167f1101000d04000000000000c2c6ae5d00000000010000000000000000000000000000000000ffff68c7b80f208d0d040000000000000000000000000000000000000000000000000b63185e17ebcdb3102f5361746f7368693a302e31382e302fbc29090001'

Command: verack
Request:
b'f9beb4d976657261636b000000000000000000005df6e0e2'
Response:
b'f9beb4d976657261636b000000000000000000005df6e0e2'

Command: getdata
Request:
b'f9beb4d9676574646174610000000000220000007b00a9b50101fc57704eff327aecfadb2cf3774edc919ba69aba624b836461ce2be9c00a0c20'
Response:
b'f9beb4d9616c65727400000000000000c0000000d2f50d9ef9beb4d9616c65727400000000000000a80000001bf9aaea60010000000000000000000000ffffff7f00000000ffffff7ffeffff7f01ffffff7f00000000ffffff7f00ffffff7f002f555247454e543a20416c657274206b657920636f6d70726f6d697365642c2075706772616465207265717569726564004630440220653febd6410f470f6bae11cad19c48413becb1ac2c17f908fd0fd53bdc3abd5202206d0e9c96fe88d4a0f01ed9dedae2b6f9e00da94cad0fecaae66ecf689bf71b50'

That concludes the tutorial! Stay tuned for more.

ULTRA CONFIG GENERATOR

Have you heard of Ultra Config Generator? If you haven't, I highly recommend you check it out.

We designed the product to allow network engineers to generate and automate network configuration in a highly flexible, efficient and elegant manner. Our users love the application and I hope that you will too.

Take care until next time!

Ultra Config