We Launched a Course on Using Blockchain Indexers!

This post briefly covers the basics concerning blockchain indexers and explains why they are needed. If you want to use them, well, we’ve got a course just about that!

Blockchain as a Database and Its Problems

Blockchain is a decentralized database of on-chain transactions. Any user can request the contents of a block or details of an individual transaction from a public RPC node, which stores its copy of the blockchain.

That said, the blockchain and the node software are not designed for fast and efficient searching. It is obvious from an example: try to retrieve the details of the transaction tez with hash op…SL from the RPC node.

The command to request the details will not look like “GET BY HASH” or something like that:

tezos-client rpc get /chains/main/blocks/2283698/operations/3/2

The Tezos RPC API has no command to search for a transaction. To retrieve its details from the blockchain, you need to know exactly where they are stored: in block 2283698, among operations initiated by users (index 3) under id 2. And similar difficulties arise with searching a large amount of data.

What Do Blockchain Indexers Do

Indexers are programs that read each new block, recognize its contents and write everything into their own database. They also create indexes, which are special data structures for quickly finding records by key.

For example, searching for the same transaction by hash in a TzKT indexer would look like querying the operations table:

https://reserve-2.tzkt.io/v1/operations/opRjkzJxJ1xZaUnBDykGUjrRV8qgHFvchcYnbkkcotS1Y7idCSL

And in TzStats, as a request to the table explorer/op:

https://api.tzstats.com/explorer/op/opRjkzJxJ1xZaUnBDykGUjrRV8qgHFvchcYnbkkcotS1Y7idCSL

Thanks to indexers, their structured databases, and indexes, we have blockchain browsers TzKT, TzStats, and BetterCallDev with the ability to find any account and smart contract, see Tezos network statistics, contract content, and more.

All decentralized applications, including wallets, use indexers. For example, Temple Wallet gets its token and NFT balances from the TzKT indexer.

We covered indexers in detail in the first lesson.

What Kid of Indexers There Are on Tezos

There are two types of blockchain indexers: full and selective.

Full ones write all blockchain content into their database and create indexes for it. They can be used to find any on-chain data, from the list of bakers who voted to activate Lima to the gas that Kolibri Finance spent when publishing the kUSD smart contract. Full indexers include TzKT and TzStats.

Selective indexers read new blocks but write only those transactions to the user’s specified database. Tezos has three popular ones:

• Que Pasa: a simple indexer that records the contents of the specified contract storage, entry point calls, and their parameters;
• DipDup: a framework for creating selective indexers in Python. The user has to write the rules for indexing the selected contract: which entry points to the index, which parameters to write to the database, and according to which rules;
• Dappetizer is a JavaScript/TypeScript framework for creating selective indexers. To work with it you also need to describe the indexing logic yourself, but apart from operations with smart contracts, it can write any other operations in blocks.

The different types of indexers and their capabilities are explained in detail in Lesson 2.

How to Use Blockchain Indexers

To make your first dApp, you don’t have to learn smart contract development (although it’s interesting). You just need to know what storage and entry points are and know a bit about backend development.

In the third lesson, we show how to use HTML, AJAX queries to TzKT, and basic knowledge of JavaScript to make a dashboard to display statistics on liquidity baking.

But it’s better not to use TzKT API in production to avoid overloading the public indexer with queries. So in the fourth lesson, we already host Que Pasa on the local machine and do a more complex project.

We index the NFT-collaterized lending service Signum.loans smart contract, display the loan data and generate links to the corresponding NFTs on objkt.com. In addition to Que Pasa, we explain the basics of PostgreSQL, set up a PHP server and ignore the existence of CSS.

And in the last lesson, we show how to set up DipDup and Dappetizer to index tzBTC contracts. Writing the data processing logic is hard, but the result is worth it. In the end, we have calculated the volume of all tzBTC transactions with one SQL query.

Final Thoughts

You need an indexer if you want to display or use on-chain data in an application. Our course will help you choose the right solution for your task and give you an understanding of how to use it.

Also, refer to the first two lessons if you’re just interested in learning more about databases, effective ways to search for data, and how Tezos works in general.