Configuration

The initial design of db-sync was a one size fits all approach. It served as a general-purpose backend for Cardano applications, including light wallets, explorers, etc. Over time, many new features have been added, including historic rewards with Shelley, scripts with Allegra, multiassets with Mary, Plutus scripts and redeemers with Alonzo, stake pool metadata with the integration of SMASH, etc.

Most application use only a small fraction of these features. Therefore, db-sync offers options that turn off some of these features, especially the most expensive ones in terms of performance.

Configuration File

Starting with db-sync 13.3.0.0, customizing features is done in the standard db-sync configuration file (db-sync-config.json or db-sync-config.yaml).

Simple Example

Below is a sample insert_options section that shows the recommended defaults:

{
  // <-- Rest of configuration -->
  // ...
  "insert_options": {
    "tx_cbor": "disable",
    "tx_out": {
      "value": "enable"
    },
    "ledger": "enable",
    "shelley": {
      "enable": true
    },
    "multi_asset": {
      "enable": true
    },
    "metadata": {
      "enable": true
    },
    "plutus": {
      "enable": true
    },
    "governance": "enable",
    "offchain_pool_data": "disable",
    "offchain_vote_data": "disable",
    "json_type": "text"
  }
}

Note: Starting from version 13.7.0.1, offchain_pool_data and offchain_vote_data default to "disable". Enable them if you need offchain metadata fetching.

Properties

insert_options may contain the following elements:

Preset

Preset is an aggregate setting that overrides all other properties. For example, setting preset to "full" will enable all insert options except "tx_cbor", "offchain_pool_data", and "offchain_vote_data" (the last two default to "disable" since 13.7.0.1).

preset

• Type: string

enum: The value of this property must be equal to one of the following values:

Full

This is equivalent to setting:

"tx_cbor": "disable",
"tx_out": {
  "value": "enable",
  "use_address_table": false
},
"ledger": "enable",
"shelley": {
  "enable": true
},
"multi_asset": {
  "enable": true
},
"metadata": {
  "enable": true
},
"plutus": {
  "enable": true
},
"governance": "enable",
"offchain_pool_data": "disable",
"offchain_vote_data": "disable",
"pool_stat": "enable",
"json_type": "text"

offchain_pool_data and offchain_vote_data default to "disable" since 13.7.0.1 - set them to "enable" explicitly if you need offchain metadata fetching.

Only UTxO

This is equivalent to setting:

"tx_cbor": "disable",
"tx_out": {
  "value": "bootstrap"
},
"ledger": "ignore",
"shelley": {
  "enable": false
},
"metadata": {
  "enable": "false"
},
"multi_asset": {
  "enable": "true"
},
"plutus": {
  "enable": false
},
"governance": "disable",
"offchain_pool_data": "disable",
"offchain_vote_data": "disable",
"pool_stat": "disable"

Initially populates only a few tables, like block and tx. It maintains a ledger state but doesn’t use any of its data. When syncing is completed, it loads the whole UTxO set from the ledger to the tx_out and ma_tx_out tables. After that db-sync can be restarted with ledger set to "disable" to continue syncing without maintaining the ledger

Only Governance

This is equivalent to setting:

"tx_cbor": "disable",
"tx_out": {
  "value": "disable"
},
"ledger": "enable",
"shelley": {
  "enable": false
},
"multi_asset": {
  "enable": false
},
"plutus": {
  "enable": false
},
"governance": "enable",
"offchain_pool_data": "disable",
"offchain_vote_data": "disable",
"pool_stat": "enable"

Disables most data except block, tx, and governance data. View governance for more indepth information into what is effected.

Disable All

This is equivalent to setting:

"tx_cbor": "disable",
"tx_out": {
  "value": "disable"
},
"ledger": "disable",
"shelley": {
  "enable": false
},
"multi_asset": {
  "enable": false
},
"plutus": {
  "enable": false
},
"governance": "disable",
"offchain_pool_data": "disable",
"offchain_vote_data": "disable",
"pool_stat": "disable"

Disables almost all data except block and tx tables.

Tx CBOR

tx_cbor

• Type: string

enum: The value of this property must be equal to one of the following values:

Tx Out

tx_out

• Type: object

Example

"tx_out": {
  "value": "consumed",
  "force_tx_in": false,
  "use_address_table": true,
},

Tx Out Properties:

Value

tx_out.value

• Type: string

enum: the value of this property must be equal to one of the following values:

Enable

Enable all inputs and outputs.

Disable

Disable inputs and outputs. With this flag:

• tx_in table is left empty
• tx_out table is left empty
• ma_tx_out table is left empty
• tx.fee has a wrong value 0
• redeemer.script_hash is left Null

It’s similar to "bootstrap" except the UTxO is never populated. However, after using this flag db-sync can be stopped and restarted with "bootstrap" to load the UTxO from the ledger.

Consumed

Adds a new field tx_out (consumed_by_tx_id) and populates it accordingly. This allows users to query the tx_out table for unspent outputs directly, without the need to join with the tx_in table. If this is set once, then it must be always be set on following executions of db-sync, unless prune-tx-out is used instead.

Prune

Periodically prunes the consumed tx_out table. This allows users to query for utxo without having to maintain the whole tx_out table. Deletes to tx_out are propagated to ma_tx_out through foreign keys. If this is set once, then it must be always set on subsequent executions of db-sync. Failure to do this can result in crashes and db-sync currently has no way to detect it.

Bootstrap

Results in a similar db schema as using "prune", except it syncs faster. The difference is that instead of inserting/updating/deleting outputs, it delays the insertion of UTxO until the tip of the chain. By doing so, it avoid costly db operations for the majority of outputs, that are eventually consumed and as a result deleted. UTxO are eventually inserted in bulk from the ledger state. The initial implementation of the feautures assumes ledger is set to "enable" , since the ledger state is used. The networks needs to be in Babbage or Conway era for this to work. The following fields are left empty:

• tx.fee has a wrong value 0
• redeemer.script_hash is left Null

Until the ledger state migration happens, any restart requires this setting. After completion, this can be changed.

Force Tx In

tx_out.force_tx_in

• Type: boolean

When using consumed configuration tx_in will not be populated. That behaviour can be overridden by setting this value to true.

Address Table

tx_out.use_address_table

• Type: boolean

This new variant representation introduces an additional Address table to normalize the address-related data. This change allows for more efficient storage and querying of address information, especially in cases where multiple transaction outputs (TxOuts) reference the same address.

Key changes in the variant representation:

1. New address table:

   • Contains fields: address, raw, has_script, payment_cred, and stake_address_id
   • Centralizes address information that was previously duplicated across multiple TxOuts

2. Modified tx_out table:

   • Remove address, address_has_script, and payment_cred.
   • Add address_id references the new Address table

3. Modified collateral_tx_out table

   • Remove address, address_has_script, and payment_cred.
   • Add address_id references the new Address table

The address table can only be used on an empty database due to the schema restructuring which would cause data loss.

The following indexes are added to the new address table:

1. idx_address_payment_cred ON address(payment_cred)
2. idx_address_raw ON address(raw)

Then address.id having a unique constraint.

Ledger

One of the db-sync features that uses the most resources is that it maintains a ledger state and replays all the ledger rules. This is the only way to get historic reward details and other data that is not included in the blocks (ie. historic stake distribution, ada pots, epoch parameters, etc).

ledger

• Type: string

enum: The value of this property must be equal to one of the following values:

Enable

Maintain ledger state and replay all ledger rules.

Disable

Turn off ledger state and significantly reduce memory usage (by up to 10GB on mainnet) and sync time. Another benefit of this option is that there are no rollbacks on startup, which tend to take quite some time, since there are no ledger snapshots maintained on disk.

When this flag is enabled:

• redeemer.fee is left null
• reward table is left empty
• epoch_stake table is left empty
• ada_pots table is left empty
• epoch_param table is left empty
• tx.deposit is left null (too expensive to calculate without the ledger)
• drep_distr is left empty
• governance_action.x_epoch is left null
• governance_action.expiration is left null
• stake_registration.deposit is left null
• pool_update.deposit is left null

Warning: Running db-sync with this setting and restarting it with a different one will cause crashes and should be avoided.

Warning: If this setting is used with the --state-dir command line option, an error will occur.

Released snapshots are compatible with this option. Since the snapshots are created with ledger enabled, there still can be some minor inconsistencies. The above data may exist up to the slot/epoch of the snapshot creation and can be missing afterward. To fix this, when db-sync is initiated with ledger disabled, it will automatically remove this data.

Warning: This will irreversibly delete data from existing snapshots.

Here are the exact queries db-sync with this flag will run after restoring a snapshot:

update redeemer set fee = null;
delete from reward;
delete from epoch_stake;
delete from ada_pots;
delete from epoch_param;

Ignore

Maintains the ledger state, but doesn’t use any of its data, except to load UTxO. To be used with tx_out set to "bootstrap"

Shelley

shelley

• Type: object

Shelley Properties:

Enable

Enables or disables data related to shelley: all certificates, withdrawals, and param proposals. Does not control epoch_stake and rewards, For this check ledger.

shelley.enable

• Type: boolean

Multi Asset

multi_asset

• Type: object

Multi Asset Properties:

Enable

Enables or disables multi assets tables and entries.

multi_asset.enable

• Type: boolean

Metadata

metadata

• Type: object

Metadata Properties:

Enable

Enables or disables the tx_metadata table.

metadata.enable

• Type: boolean

Keys

If set, only keep metadata with the specified keys.

metadata.keys

• Type: integer[]

Plutus

plutus

• Type: object

Plutus Properties:

Enable

Enables or disables most tables and entries related to plutus and scripts.

plutus.enable

• Type: boolean

Governance

governance

This will effect all governance related data/functionality.

• Type: string

enum: The value of this property must be equal to one of the following values:

Example

"governance": "enable"

Tables associated with governance

Offchain Pool Data

offchain_pool_data

• Type: string

enum: The value of this property must be equal to one of the following values:

Controls whether db-sync fetches offchain metadata for stake pools. This metadata includes pool descriptions, ticker symbols, homepages, and other information referenced in pool registrations.

Important: Starting from version 13.7.0.1, this defaults to "disable". Explicitly enable it if you need pool metadata.

When enabled, db-sync fetches metadata from URLs stored on-chain with the following restrictions:

• HTTPS URLs required (HTTP allowed only for localhost testing)
• Private/internal IP addresses blocked (RFC1918, loopback, link-local, IPv6 ULA)
• DNS resolution performed immediately before requests

Offchain Vote Data

offchain_vote_data

• Type: string

enum: The value of this property must be equal to one of the following values:

Controls whether db-sync fetches offchain metadata for governance voting anchors, including:

• DRep registrations (CIP-119)
• Governance action proposals (CIP-108)
• Vote anchors and references (CIP-100)

This setting is independent of the governance flag, allowing granular control over general governance data versus offchain metadata fetching.

Important: Starting from version 13.7.0.1, this defaults to "disable". Explicitly enable it if you need governance vote metadata.

When enabled, the same URL restrictions apply as offchain_pool_data (HTTPS required, private IPs blocked, immediate DNS resolution).

Example

{
  "insert_options": {
    "governance": "enable",
    "offchain_vote_data": "enable"
  }
}

Migration Note: Prior to version 13.7.0.1, governance vote metadata fetching was controlled by the governance flag.

Pool Stat

pool_stat

• Type: string

enum: The value of this property must be equal to one of the following values:

Remove Jsonb From Schema

remove_jsonb_from_schema

To improve inserting performance you can remove Jsonb data types in the schema. They can be reintroduced by using disable or by simply not using all together. A warning will logw if remove_jsonb_from_schema was previously set to enable and then either removed from the configuration file or set to disabled.

• Type: string

enum: The value of this property must be equal to one of the following values:

Example

"remove_jsonb_from_schema": "enable"

Data Types Effected

When enabling this config, the following columns will no longer have the jsonb data type:

Stop At Block

stop_at_block

• Type: integer
• Optional: When not specified, db-sync runs indefinitely

Stops db-sync after processing the specified block number. Useful for testing and debugging.

Example

{
  "insert_options": {
    "stop_at_block": 12345
  }
}

Snapshot Interval

snapshot_interval

• Type: object
• Optional: When not specified, uses default values

Controls how frequently ledger state snapshots are taken during sync. Taking snapshots less frequently during initial sync can significantly improve sync performance by reducing IOPS and disk throughput consumption.

Snapshot Interval Properties:

Near Tip Epoch

snapshot_interval.near_tip_epoch

• Type: integer
• Default: 580

Epoch threshold used to determine snapshot behavior. When syncing reaches this epoch or later, db-sync is considered to be approaching or at the current tip of the chain. Combined with time-based detection (within 10 days of current time), this ensures snapshots are taken every epoch when near the tip for fast rollback recovery. During earlier epochs or when syncing behind, snapshots are taken every 10 epochs.

Example

{
  "snapshot_interval": {
    "near_tip_epoch": 580
  }
}

Performance Considerations

• Lower near_tip_epoch value: Start taking frequent epoch-based snapshots earlier in the chain history
• Higher near_tip_epoch value: Delay frequent snapshots until later in the chain, improving sync speed for longer. During initial sync (before reaching near_tip_epoch), snapshots are taken every 10 epochs
• Near tip detection: Automatically switches to epoch-based snapshots when within 10 days of current time, regardless of epoch number

Ledger Backend

The ledger_backend top-level config option selects how the UTxO set is stored.

Example

{
  "ledger_backend": "lsm"
}

InMemory (default)

Keeps the full UTxO set in memory. Faster but uses significantly more RAM (~16GB on mainnet at tip).

LSM

Stores the UTxO set on disk using LSM trees, reducing RAM usage to ~2-3GB. The LSM database is stored in <state-dir>/lsm/. Requires more disk I/O but is suitable for memory-constrained environments.

Snapshots are compatible between backends and can be converted using the snapshot-converter tool shipped with cardano-node. See State Snapshot for details.